2024-09-10T07:14:06.310 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T07:14:06.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T07:14:06.421 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} email: null first_in_suite: false job_id: '7898854' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGwMLq9D1nlQLLWFYOKZKsrpaT5qHKIKro/js0YhEV+KWdBPrgodevP+ZsiWEsGZsDY+11KRmtoQWsxCZAIc3bU= smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLF6zwZ58H44vDsXxixsKFE7zUjbHSulOJJ/4Vt0KNEX5Vyw3eO61lpoHp/BAwz81CUdWjwQEc1zrflt62A6t40= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBTqNM0nuwqUawfLST/Geya/waj4eSB/HYFolt8l4U9n5sRkustuDPZ7EKmnD8z9RbTTb7pmVXt+5Bfm87AZs6g= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T07:14:06.421 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T07:14:06.422 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T07:14:06.422 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T07:14:06.424 INFO:teuthology.task.internal:Checking packages... 2024-09-10T07:14:06.445 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T07:14:06.445 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T07:14:06.445 INFO:teuthology.packaging:ref: None 2024-09-10T07:14:06.445 INFO:teuthology.packaging:tag: None 2024-09-10T07:14:06.445 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T07:14:06.446 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:14:06.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T07:14:06.587 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T07:14:06.589 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T07:14:06.596 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T07:14:06.596 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T07:14:06.603 INFO:teuthology.task.internal:Saving configuration 2024-09-10T07:14:06.621 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T07:14:06.628 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T07:14:06.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 07:08:57.409257', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGwMLq9D1nlQLLWFYOKZKsrpaT5qHKIKro/js0YhEV+KWdBPrgodevP+ZsiWEsGZsDY+11KRmtoQWsxCZAIc3bU='} 2024-09-10T07:14:06.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 07:08:57.408073', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLF6zwZ58H44vDsXxixsKFE7zUjbHSulOJJ/4Vt0KNEX5Vyw3eO61lpoHp/BAwz81CUdWjwQEc1zrflt62A6t40='} 2024-09-10T07:14:06.686 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 07:08:57.410285', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBTqNM0nuwqUawfLST/Geya/waj4eSB/HYFolt8l4U9n5sRkustuDPZ7EKmnD8z9RbTTb7pmVXt+5Bfm87AZs6g='} 2024-09-10T07:14:06.686 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T07:14:06.692 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T07:14:06.693 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T07:14:06.693 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T07:14:06.693 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T07:14:06.766 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff9927c3be0>, signals=[15]) 2024-09-10T07:14:06.766 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T07:14:06.772 INFO:teuthology.task.internal:Opening connections... 2024-09-10T07:14:06.772 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:14:06.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:14:06.849 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-09-10T07:14:06.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:14:06.925 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T07:14:06.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:14:06.998 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T07:14:07.004 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-09-10T07:14:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-09-10T07:14:07.022 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-09-10T07:14:07.079 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T07:14:07.080 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T07:14:07.081 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T07:14:07.081 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-09-10T07:14:07.096 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-09-10T07:14:07.114 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-09-10T07:14:07.114 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-09-10T07:14:07.172 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-09-10T07:14:07.172 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-09-10T07:14:07.172 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-09-10T07:14:07.172 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-09-10T07:14:07.172 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-09-10T07:14:07.172 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-09-10T07:14:07.172 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T07:14:07.173 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-09-10T07:14:07.173 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-09-10T07:14:07.173 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T07:14:07.173 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-09-10T07:14:07.173 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T07:14:07.173 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T07:14:07.173 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T07:14:07.173 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-09-10T07:14:07.196 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-09-10T07:14:07.214 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-09-10T07:14:07.214 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-09-10T07:14:07.272 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T07:14:07.273 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T07:14:07.274 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T07:14:07.274 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-09-10T07:14:07.297 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T07:14:07.307 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T07:14:07.312 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T07:14:07.312 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-09-10T07:14:07.314 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-09-10T07:14:07.317 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-09-10T07:14:07.333 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T07:14:07.340 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T07:14:07.340 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-09-10T07:14:07.371 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-09-10T07:14:07.374 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-09-10T07:14:07.492 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T07:14:07.550 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T07:14:07.564 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T07:14:07.564 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-10T07:14:07.564 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-10T07:14:07.564 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T07:14:07.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:14:07.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T07:14:07.790 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:14:07.790 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T07:14:07.791 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-10T07:14:07.808 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-503.el9.x86_64 2024-09-10T07:14:07.809 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:14:07.809 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T07:14:07.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum upgrade -y linux-firmware 2024-09-10T07:14:12.119 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.2 MB 00:02 2024-09-10T07:14:15.254 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-10T07:14:19.570 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-10T07:14:21.121 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-10T07:14:21.875 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-10T07:14:27.539 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-10T07:14:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T07:14:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T07:14:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T07:14:29.940 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T07:14:29.941 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:14:30.093 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T07:14:30.093 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T07:14:30.096 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-09-10T07:14:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10 07:14:41 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T07:14:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:FINISHED --2024-09-10 07:14:41-- 2024-09-10T07:14:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:Total wall clock time: 11s 2024-09-10T07:14:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:Downloaded: 1 files, 343M in 11s (31.2 MB/s) 2024-09-10T07:14:41.402 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T07:14:41.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T07:14:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying... ######################################## 2024-09-10T07:14:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing... ######################################## 2024-09-10T07:14:43.879 INFO:teuthology.orchestra.run.smithi046.stdout:Updating / installing... 2024-09-10T07:14:52.192 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T07:15:32.130 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T07:15:32.130 DEBUG:teuthology.orchestra.run.smithi046:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T07:15:32.209 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:32.209 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:32.209 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:32.209 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:32.209 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T07:15:32.210 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T07:15:32.211 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T07:15:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T07:15:32.213 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T07:15:32.214 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T07:15:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T07:15:32.216 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T07:15:32.217 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T07:15:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T07:15:32.219 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T07:15:32.220 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T07:15:32.221 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T07:15:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T07:15:32.223 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T07:15:32.224 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T07:15:32.225 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T07:15:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T07:15:32.227 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T07:15:32.228 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T07:15:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T07:15:32.230 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T07:15:32.231 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T07:15:32.232 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T07:15:32.233 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T07:15:32.234 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T07:15:32.235 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T07:15:32.236 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T07:15:32.237 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T07:15:32.238 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T07:15:32.239 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T07:15:32.240 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T07:15:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T07:15:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T07:15:32.243 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T07:15:32.244 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T07:15:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T07:15:32.246 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T07:15:32.247 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T07:15:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T07:15:32.249 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T07:15:32.250 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T07:15:32.251 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T07:15:32.252 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T07:15:32.253 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T07:15:32.254 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T07:15:32.255 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T07:15:32.256 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T07:15:32.257 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T07:15:32.258 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T07:15:32.259 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T07:15:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T07:15:32.261 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T07:15:32.262 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T07:15:32.263 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T07:15:32.264 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T07:15:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T07:15:32.266 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T07:15:32.267 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T07:15:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T07:15:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T07:15:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T07:15:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T07:15:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T07:15:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T07:15:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T07:15:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T07:15:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T07:15:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T07:15:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T07:15:32.279 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T07:15:32.280 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T07:15:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T07:15:32.282 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T07:15:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T07:15:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T07:15:32.285 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T07:15:32.286 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T07:15:32.287 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T07:15:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T07:15:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T07:15:32.290 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T07:15:32.291 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T07:15:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T07:15:32.293 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T07:15:32.294 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T07:15:32.295 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T07:15:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T07:15:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T07:15:32.298 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T07:15:32.299 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T07:15:32.300 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T07:15:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T07:15:32.302 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T07:15:32.303 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T07:15:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T07:15:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T07:15:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T07:15:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T07:15:32.308 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T07:15:32.309 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T07:15:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T07:15:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T07:15:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T07:15:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T07:15:32.314 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T07:15:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T07:15:32.316 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T07:15:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T07:15:32.318 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T07:15:32.319 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T07:15:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T07:15:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T07:15:32.322 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T07:15:32.323 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T07:15:32.324 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T07:15:32.325 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T07:15:32.326 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T07:15:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T07:15:32.328 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T07:15:32.329 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T07:15:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T07:15:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T07:15:32.332 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T07:15:32.333 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T07:15:32.334 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T07:15:32.335 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T07:15:32.336 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T07:15:32.337 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T07:15:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T07:15:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T07:15:32.340 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T07:15:32.341 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T07:15:32.342 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T07:15:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T07:15:32.344 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T07:15:32.345 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T07:15:32.346 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T07:15:32.347 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T07:15:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T07:15:32.349 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T07:15:32.350 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T07:15:32.351 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T07:15:32.351 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T07:15:32.354 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:32.355 DEBUG:teuthology.orchestra.run.smithi046:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T07:15:32.378 INFO:teuthology.orchestra.run.smithi046.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout:for file in vmlinuz System.map config; do 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout: fi 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout:done 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:15:32.379 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-09-10T07:15:32.380 INFO:teuthology.orchestra.run.smithi046.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T07:15:32.380 INFO:teuthology.orchestra.run.smithi046.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:32.380 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-09-10T07:15:32.380 INFO:teuthology.orchestra.run.smithi046.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T07:15:32.380 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T07:15:32.380 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:32.380 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-09-10T07:15:32.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-09-10T07:15:32.464 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-09-10T07:15:32.464 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-09-10T07:15:32.464 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-09-10T07:15:32.464 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 82.el9 2024-09-10T07:15:32.464 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8274670 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-09-10T07:15:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-09-10T07:15:32.466 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:15:32.466 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T07:15:32.466 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T07:15:32.466 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T07:15:32.466 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-09-10T07:15:32.466 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:15:32.466 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T07:15:32.467 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T07:15:32.467 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:32.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T07:15:33.000 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-09-10T07:15:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T07:15:34.528 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-09-10T07:15:34.530 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-10T07:15:34.545 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.wUDhQ5EWSb 2024-09-10T07:15:34.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wUDhQ5EWSb 2024-09-10T07:15:34.611 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.wUDhQ5EWSb 2024-09-10T07:15:34.749 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.wUDhQ5EWSb is 6KB 2024-09-10T07:15:34.797 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.wUDhQ5EWSb 2024-09-10T07:15:34.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T07:15:34.874 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T07:15:34.874 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T07:15:34.874 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T07:15:34.874 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T07:15:34.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:15:35.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-09-10T07:16:05.048 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T07:16:05.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T07:16:05.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:16:23.485 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-10T07:16:32.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T07:16:32.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:16:35.549 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-10T07:16:47.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T07:16:47.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:17:05.981 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-10T07:17:20.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T07:17:20.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:17:24.065 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-10T07:17:42.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T07:17:42.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:17:42.393 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-10T07:17:42.854 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T07:17:42.854 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T07:17:42.854 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-10T07:17:42.888 INFO:teuthology.orchestra.run.smithi046.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:17:42.889 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:17:42.889 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T07:17:42.889 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T07:17:43.890 DEBUG:teuthology.parallel:result is None 2024-09-10T07:17:43.890 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T07:17:43.900 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T07:17:43.901 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T07:17:43.905 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T07:17:43.908 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T07:17:43.939 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T07:17:43.946 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T07:17:43.956 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T07:17:43.956 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T07:17:43.984 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T07:17:43.986 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T07:17:44.097 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T07:17:44.103 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T07:17:44.103 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T07:17:44.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:17:44.121 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T07:17:44.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:17:44.138 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T07:17:44.142 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T07:17:44.164 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T07:17:44.208 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:17:44.219 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:17:44.222 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:17:44.237 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:17:44.286 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:17:44.320 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:17:44.323 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T07:17:44.330 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T07:17:44.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T07:17:44.334 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T07:17:44.336 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T07:17:44.388 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T07:17:44.397 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T07:17:44.398 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T07:17:44.401 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T07:17:44.404 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T07:17:44.432 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:17:44.560 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:17:44.631 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:17:44.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:17:44.768 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:17:44.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T07:17:44.823 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:17:44.860 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:17:44.915 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:17:44.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:17:45.041 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:17:45.042 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T07:17:45.110 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:17:45.147 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:17:45.213 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:17:45.296 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:17:45.339 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:17:45.340 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T07:17:45.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-09-10T07:17:45.410 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-09-10T07:17:45.412 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-09-10T07:17:45.453 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:17:45.483 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:17:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:17:45.951 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T07:17:45.960 INFO:teuthology.task.internal:Starting timer... 2024-09-10T07:17:45.960 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T07:17:45.971 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T07:17:45.979 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T07:17:45.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-09-10T07:17:46.058 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-09-10T07:17:46.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-09-10T07:17:46.122 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-09-10T07:17:46.124 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-09-10T07:17:46.170 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-09-10T07:17:46.172 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T07:17:46.172 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-09-10T07:17:46.215 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-09-10T07:17:46.215 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-09-10T07:17:46.236 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-09-10T07:17:46.237 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-09-10T07:17:46.257 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-09-10T07:17:46.258 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi063.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2024-09-10T07:17:46.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:17:46.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:17:46.315 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:17:46.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:17:46.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:17:46.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:17:46.372 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T07:17:46.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-09-10T07:17:46.433 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-09-10T07:17:46.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-09-10T07:17:46.516 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T07:17:46.527 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T07:17:46.530 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T07:17:46.651 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T07:17:46.662 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T07:17:46.663 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi063.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T07:22:24.805 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi063.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2024-09-10T07:22:24.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T07:22:24.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:22:24.903 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-10T07:22:25.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T07:22:25.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-10T07:22:25.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:22:25.096 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-09-10T07:22:25.178 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-10T07:22:25.179 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-10T07:22:25.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:22:25.259 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-09-10T07:22:25.336 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-10T07:22:25.337 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T07:22:25.351 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T07:22:25.351 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T07:22:25.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:22:25.356 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T07:22:25.356 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:22:25.359 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T07:22:25.359 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:22:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T07:22:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T07:22:25.407 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T07:22:25.416 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T07:22:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T07:22:25.428 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-09-10T07:22:25.439 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-09-10T07:22:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T07:22:25.447 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-09-10T07:22:25.454 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T07:22:25.458 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-09-10T07:22:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T07:22:25.473 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T07:22:25.487 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T07:22:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-09-10T07:22:25.521 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-09-10T07:22:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:22:25.529 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:22:25.529 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-09-10T07:22:25.529 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.529 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.529 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.529 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.540 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:22:25.544 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:22:25.544 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-09-10T07:22:25.544 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.544 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.544 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.544 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.551 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T07:22:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T07:22:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:22:25.678 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:22:25.678 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-09-10T07:22:25.678 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.678 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.678 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.678 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:22:25.679 INFO:teuthology.run_tasks:Running task install... 2024-09-10T07:22:25.688 DEBUG:teuthology.task.install:project ceph 2024-09-10T07:22:25.689 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T07:22:25.689 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T07:22:25.689 INFO:teuthology.task.install:Using flavor: default 2024-09-10T07:22:25.695 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T07:22:25.695 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T07:22:25.696 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T07:22:25.697 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:22:25.699 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T07:22:25.699 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:22:25.700 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T07:22:25.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:22:25.852 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T07:22:25.852 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T07:22:25.862 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T07:22:25.862 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T07:22:25.865 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T07:22:25.865 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T07:22:25.965 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T07:22:25.965 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:22:25.965 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T07:22:25.972 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T07:22:25.972 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:22:25.972 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T07:22:25.977 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T07:22:25.978 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:22:25.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T07:22:25.991 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T07:22:25.991 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T07:22:25.999 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T07:22:26.000 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T07:22:26.034 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T07:22:26.035 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T07:22:26.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T07:22:26.066 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T07:22:26.131 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T07:22:26.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T07:22:26.140 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T07:22:26.196 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-09-10T07:22:26.197 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-09-10T07:22:26.204 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-09-10T07:22:26.205 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-09-10T07:22:26.211 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T07:22:26.266 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-09-10T07:22:26.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-09-10T07:22:26.420 INFO:teuthology.orchestra.run.smithi181.stdout:45 files removed 2024-09-10T07:22:26.435 INFO:teuthology.orchestra.run.smithi063.stdout:45 files removed 2024-09-10T07:22:26.448 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T07:22:26.463 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T07:22:26.541 INFO:teuthology.orchestra.run.smithi046.stdout:45 files removed 2024-09-10T07:22:26.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T07:22:27.010 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 292 kB/s | 84 kB 00:00 2024-09-10T07:22:27.126 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 219 kB/s | 84 kB 00:00 2024-09-10T07:22:27.281 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-10T07:22:27.294 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T07:22:27.410 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T07:22:27.565 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-10T07:22:27.569 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T07:22:27.669 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-10T07:22:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 7.4 kB/s | 1.9 kB 00:00 2024-09-10T07:22:28.341 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-10T07:22:29.253 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-10T07:22:29.510 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-10T07:22:30.459 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-10T07:22:31.071 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-10T07:22:31.337 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-10T07:22:35.430 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-10T07:22:36.651 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-10T07:22:37.346 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-10T07:22:37.856 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-10T07:22:38.597 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 19 kB 00:00 2024-09-10T07:22:38.635 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T07:22:38.913 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-10T07:22:39.366 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T07:22:39.769 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-10T07:22:43.974 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T07:22:45.431 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T07:22:45.995 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-10T07:22:46.268 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:22:46.268 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:22:46.280 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T07:22:46.281 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T07:22:46.282 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T07:22:46.282 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T07:22:46.284 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T07:22:46.287 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T07:22:46.287 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T07:22:46.350 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-09-10T07:22:46.369 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-09-10T07:22:46.369 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-09-10T07:22:46.369 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-09-10T07:22:46.369 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-09-10T07:22:46.369 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T07:22:46.370 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T07:22:46.371 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T07:22:46.371 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T07:22:46.371 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T07:22:46.371 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T07:22:46.371 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T07:22:46.371 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T07:22:46.372 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T07:22:46.372 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T07:22:46.373 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T07:22:46.374 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T07:22:46.375 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T07:22:46.376 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T07:22:46.377 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T07:22:46.378 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T07:22:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T07:22:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T07:22:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T07:22:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T07:22:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout:Install 148 Packages 2024-09-10T07:22:46.384 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 8 Packages 2024-09-10T07:22:46.385 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:22:46.385 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 226 M 2024-09-10T07:22:46.385 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-09-10T07:22:47.000 INFO:teuthology.orchestra.run.smithi181.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-10T07:22:47.192 INFO:teuthology.orchestra.run.smithi181.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 4.6 MB/s | 897 kB 00:00 2024-09-10T07:22:47.284 INFO:teuthology.orchestra.run.smithi181.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.5 MB/s | 145 kB 00:00 2024-09-10T07:22:47.352 INFO:teuthology.orchestra.run.smithi181.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-10T07:22:47.460 INFO:teuthology.orchestra.run.smithi181.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-09-10T07:22:47.519 INFO:teuthology.orchestra.run.smithi181.stdout:(6/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 10 MB/s | 2.4 MB 00:00 2024-09-10T07:22:47.761 INFO:teuthology.orchestra.run.smithi181.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-10T07:22:47.809 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:22:47.809 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:22:47.822 INFO:teuthology.orchestra.run.smithi063.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T07:22:47.823 INFO:teuthology.orchestra.run.smithi063.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T07:22:47.824 INFO:teuthology.orchestra.run.smithi063.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T07:22:47.825 INFO:teuthology.orchestra.run.smithi063.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T07:22:47.829 INFO:teuthology.orchestra.run.smithi063.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T07:22:47.829 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T07:22:47.830 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T07:22:47.898 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout:====================================================================================== 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout:====================================================================================== 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T07:22:47.918 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T07:22:47.919 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T07:22:47.920 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T07:22:47.921 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-09-10T07:22:47.922 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T07:22:47.923 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T07:22:47.924 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T07:22:47.925 INFO:teuthology.orchestra.run.smithi063.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T07:22:47.926 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T07:22:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T07:22:47.928 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T07:22:47.929 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T07:22:47.929 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T07:22:47.929 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T07:22:47.929 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T07:22:47.929 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T07:22:47.929 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T07:22:47.929 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T07:22:47.930 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T07:22:47.931 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T07:22:47.932 INFO:teuthology.orchestra.run.smithi063.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout:Installing weak dependencies: 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:22:47.933 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-09-10T07:22:47.934 INFO:teuthology.orchestra.run.smithi063.stdout:====================================================================================== 2024-09-10T07:22:47.934 INFO:teuthology.orchestra.run.smithi063.stdout:Install 148 Packages 2024-09-10T07:22:47.934 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 8 Packages 2024-09-10T07:22:47.934 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:22:47.934 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 226 M 2024-09-10T07:22:47.934 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-09-10T07:22:48.053 INFO:teuthology.orchestra.run.smithi181.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 18 MB/s | 21 MB 00:01 2024-09-10T07:22:48.094 INFO:teuthology.orchestra.run.smithi181.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 608 kB/s | 25 kB 00:00 2024-09-10T07:22:48.403 INFO:teuthology.orchestra.run.smithi181.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 17 MB/s | 11 MB 00:00 2024-09-10T07:22:48.587 INFO:teuthology.orchestra.run.smithi181.stdout:(11/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 16 MB/s | 17 MB 00:01 2024-09-10T07:22:48.612 INFO:teuthology.orchestra.run.smithi181.stdout:(12/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.1 MB/s | 228 kB 00:00 2024-09-10T07:22:48.637 INFO:teuthology.orchestra.run.smithi181.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 539 kB/s | 27 kB 00:00 2024-09-10T07:22:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:22:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:22:48.662 INFO:teuthology.orchestra.run.smithi181.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 632 kB/s | 32 kB 00:00 2024-09-10T07:22:48.663 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T07:22:48.663 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T07:22:48.664 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T07:22:48.665 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T07:22:48.667 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T07:22:48.670 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T07:22:48.670 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T07:22:48.696 INFO:teuthology.orchestra.run.smithi181.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 751 kB 00:00 2024-09-10T07:22:48.721 INFO:teuthology.orchestra.run.smithi181.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.7 MB/s | 164 kB 00:00 2024-09-10T07:22:48.740 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T07:22:48.747 INFO:teuthology.orchestra.run.smithi181.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T07:22:48.760 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T07:22:48.761 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T07:22:48.762 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T07:22:48.762 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T07:22:48.762 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T07:22:48.762 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T07:22:48.762 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T07:22:48.762 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T07:22:48.762 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T07:22:48.762 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T07:22:48.763 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T07:22:48.763 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T07:22:48.763 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T07:22:48.763 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T07:22:48.763 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T07:22:48.763 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T07:22:48.764 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T07:22:48.764 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T07:22:48.764 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T07:22:48.764 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T07:22:48.764 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T07:22:48.764 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T07:22:48.765 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T07:22:48.765 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T07:22:48.765 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T07:22:48.765 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T07:22:48.765 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T07:22:48.766 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T07:22:48.766 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T07:22:48.766 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T07:22:48.766 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T07:22:48.767 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-09-10T07:22:48.767 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T07:22:48.767 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T07:22:48.767 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T07:22:48.767 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T07:22:48.767 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T07:22:48.768 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T07:22:48.768 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T07:22:48.768 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T07:22:48.768 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T07:22:48.768 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T07:22:48.769 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T07:22:48.769 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T07:22:48.769 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T07:22:48.769 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T07:22:48.769 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T07:22:48.769 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T07:22:48.770 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T07:22:48.770 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T07:22:48.770 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T07:22:48.770 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T07:22:48.770 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T07:22:48.771 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T07:22:48.771 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T07:22:48.771 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T07:22:48.771 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T07:22:48.771 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T07:22:48.771 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T07:22:48.772 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T07:22:48.772 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T07:22:48.772 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T07:22:48.772 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T07:22:48.772 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T07:22:48.772 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T07:22:48.773 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T07:22:48.773 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T07:22:48.773 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T07:22:48.773 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T07:22:48.773 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T07:22:48.773 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T07:22:48.774 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T07:22:48.774 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T07:22:48.774 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T07:22:48.774 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T07:22:48.774 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T07:22:48.774 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T07:22:48.775 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T07:22:48.775 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T07:22:48.775 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T07:22:48.775 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T07:22:48.775 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T07:22:48.776 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T07:22:48.776 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T07:22:48.776 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T07:22:48.776 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T07:22:48.776 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T07:22:48.777 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T07:22:48.777 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T07:22:48.777 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T07:22:48.777 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T07:22:48.777 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T07:22:48.777 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T07:22:48.778 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T07:22:48.778 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T07:22:48.778 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T07:22:48.778 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T07:22:48.778 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T07:22:48.779 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T07:22:48.779 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T07:22:48.779 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T07:22:48.779 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T07:22:48.779 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T07:22:48.779 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T07:22:48.780 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T07:22:48.780 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T07:22:48.780 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T07:22:48.780 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T07:22:48.780 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T07:22:48.780 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T07:22:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T07:22:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T07:22:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T07:22:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T07:22:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T07:22:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T07:22:48.782 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T07:22:48.782 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T07:22:48.782 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T07:22:48.782 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T07:22:48.782 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T07:22:48.782 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T07:22:48.783 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T07:22:48.783 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T07:22:48.783 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T07:22:48.783 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T07:22:48.784 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T07:22:48.784 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T07:22:48.784 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T07:22:48.784 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T07:22:48.784 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T07:22:48.784 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T07:22:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T07:22:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T07:22:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T07:22:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T07:22:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T07:22:48.785 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T07:22:48.786 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T07:22:48.786 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T07:22:48.786 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T07:22:48.786 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T07:22:48.786 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T07:22:48.786 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T07:22:48.787 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T07:22:48.787 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T07:22:48.787 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T07:22:48.787 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T07:22:48.788 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T07:22:48.788 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T07:22:48.788 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T07:22:48.788 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-09-10T07:22:48.788 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T07:22:48.789 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:22:48.789 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T07:22:48.789 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-10T07:22:48.789 INFO:teuthology.orchestra.run.smithi046.stdout:Install 148 Packages 2024-09-10T07:22:48.789 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 8 Packages 2024-09-10T07:22:48.789 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:22:48.791 INFO:teuthology.orchestra.run.smithi181.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 9.8 MB/s | 506 kB 00:00 2024-09-10T07:22:48.791 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 226 M 2024-09-10T07:22:48.791 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T07:22:48.822 INFO:teuthology.orchestra.run.smithi181.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 905 kB/s | 45 kB 00:00 2024-09-10T07:22:48.898 INFO:teuthology.orchestra.run.smithi181.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.8 MB/s | 136 kB 00:00 2024-09-10T07:22:49.015 INFO:teuthology.orchestra.run.smithi181.stdout:(21/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 20 MB/s | 5.3 MB 00:00 2024-09-10T07:22:49.040 INFO:teuthology.orchestra.run.smithi181.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.1 MB/s | 161 kB 00:00 2024-09-10T07:22:49.066 INFO:teuthology.orchestra.run.smithi063.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 48 kB/s | 6.5 kB 00:00 2024-09-10T07:22:49.073 INFO:teuthology.orchestra.run.smithi181.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 5.4 MB/s | 320 kB 00:00 2024-09-10T07:22:49.099 INFO:teuthology.orchestra.run.smithi181.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.0 MB/s | 301 kB 00:00 2024-09-10T07:22:49.207 INFO:teuthology.orchestra.run.smithi181.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 744 kB/s | 99 kB 00:00 2024-09-10T07:22:49.233 INFO:teuthology.orchestra.run.smithi181.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 637 kB/s | 85 kB 00:00 2024-09-10T07:22:49.300 INFO:teuthology.orchestra.run.smithi181.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-10T07:22:49.392 INFO:teuthology.orchestra.run.smithi181.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-10T07:22:49.399 INFO:teuthology.orchestra.run.smithi063.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.6 MB/s | 897 kB 00:00 2024-09-10T07:22:49.417 INFO:teuthology.orchestra.run.smithi181.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 272 kB/s | 32 kB 00:00 2024-09-10T07:22:49.451 INFO:teuthology.orchestra.run.smithi181.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.4 MB/s | 146 kB 00:00 2024-09-10T07:22:49.500 INFO:teuthology.orchestra.run.smithi063.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.4 MB/s | 145 kB 00:00 2024-09-10T07:22:49.634 INFO:teuthology.orchestra.run.smithi181.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 16 MB/s | 3.6 MB 00:00 2024-09-10T07:22:49.727 INFO:teuthology.orchestra.run.smithi181.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.7 MB/s | 249 kB 00:00 2024-09-10T07:22:49.810 INFO:teuthology.orchestra.run.smithi181.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 592 kB/s | 49 kB 00:00 2024-09-10T07:22:49.855 INFO:teuthology.orchestra.run.smithi046.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 26 kB/s | 6.5 kB 00:00 2024-09-10T07:22:49.935 INFO:teuthology.orchestra.run.smithi181.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-10T07:22:49.952 INFO:teuthology.orchestra.run.smithi181.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 116 kB/s | 16 kB 00:00 2024-09-10T07:22:50.003 INFO:teuthology.orchestra.run.smithi181.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.2 MB/s | 285 kB 00:00 2024-09-10T07:22:50.045 INFO:teuthology.orchestra.run.smithi181.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 8.1 MB/s | 766 kB 00:00 2024-09-10T07:22:50.061 INFO:teuthology.orchestra.run.smithi181.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 313 kB/s | 18 kB 00:00 2024-09-10T07:22:50.108 INFO:teuthology.orchestra.run.smithi063.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 3.9 MB/s | 2.4 MB 00:00 2024-09-10T07:22:50.206 INFO:teuthology.orchestra.run.smithi046.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.5 MB/s | 897 kB 00:00 2024-09-10T07:22:50.256 INFO:teuthology.orchestra.run.smithi046.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 2.9 MB/s | 145 kB 00:00 2024-09-10T07:22:50.333 INFO:teuthology.orchestra.run.smithi063.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 6.8 MB/s | 1.5 MB 00:00 2024-09-10T07:22:50.459 INFO:teuthology.orchestra.run.smithi063.stdout:(6/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 3.5 MB/s | 5.4 MB 00:01 2024-09-10T07:22:50.615 INFO:teuthology.orchestra.run.smithi046.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 5.4 MB/s | 5.4 MB 00:01 2024-09-10T07:22:50.629 INFO:teuthology.orchestra.run.smithi181.stdout:(39/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 19 MB/s | 49 MB 00:02 2024-09-10T07:22:50.718 INFO:teuthology.orchestra.run.smithi046.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 5.2 MB/s | 2.4 MB 00:00 2024-09-10T07:22:50.834 INFO:teuthology.orchestra.run.smithi063.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 9.4 MB/s | 4.7 MB 00:00 2024-09-10T07:22:50.848 INFO:teuthology.orchestra.run.smithi046.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-10T07:22:51.095 INFO:teuthology.orchestra.run.smithi181.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 76 kB/s | 80 kB 00:01 2024-09-10T07:22:51.362 INFO:teuthology.orchestra.run.smithi181.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 616 kB/s | 801 kB 00:01 2024-09-10T07:22:51.387 INFO:teuthology.orchestra.run.smithi181.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 257 kB/s | 191 kB 00:00 2024-09-10T07:22:51.551 INFO:teuthology.orchestra.run.smithi063.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 8.1 MB/s | 21 MB 00:02 2024-09-10T07:22:51.634 INFO:teuthology.orchestra.run.smithi046.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 5.1 MB/s | 4.7 MB 00:00 2024-09-10T07:22:51.642 INFO:teuthology.orchestra.run.smithi063.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 275 kB/s | 25 kB 00:00 2024-09-10T07:22:51.871 INFO:teuthology.orchestra.run.smithi181.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 327 kB/s | 253 kB 00:00 2024-09-10T07:22:51.893 INFO:teuthology.orchestra.run.smithi046.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 9.3 MB/s | 21 MB 00:02 2024-09-10T07:22:51.935 INFO:teuthology.orchestra.run.smithi063.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 9.6 MB/s | 11 MB 00:01 2024-09-10T07:22:51.951 INFO:teuthology.orchestra.run.smithi046.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 435 kB/s | 25 kB 00:00 2024-09-10T07:22:52.063 INFO:teuthology.orchestra.run.smithi181.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-10T07:22:52.102 INFO:teuthology.orchestra.run.smithi063.stdout:(11/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 10 MB/s | 17 MB 00:01 2024-09-10T07:22:52.127 INFO:teuthology.orchestra.run.smithi063.stdout:(12/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.2 MB/s | 228 kB 00:00 2024-09-10T07:22:52.177 INFO:teuthology.orchestra.run.smithi063.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 360 kB/s | 27 kB 00:00 2024-09-10T07:22:52.202 INFO:teuthology.orchestra.run.smithi063.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 421 kB/s | 32 kB 00:00 2024-09-10T07:22:52.286 INFO:teuthology.orchestra.run.smithi063.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 6.8 MB/s | 751 kB 00:00 2024-09-10T07:22:52.319 INFO:teuthology.orchestra.run.smithi063.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 1.4 MB/s | 164 kB 00:00 2024-09-10T07:22:52.370 INFO:teuthology.orchestra.run.smithi063.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 1.5 MB/s | 127 kB 00:00 2024-09-10T07:22:52.380 INFO:teuthology.orchestra.run.smithi181.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-09-10T07:22:52.395 INFO:teuthology.orchestra.run.smithi063.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 6.6 MB/s | 506 kB 00:00 2024-09-10T07:22:52.470 INFO:teuthology.orchestra.run.smithi063.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 602 kB/s | 45 kB 00:00 2024-09-10T07:22:52.519 INFO:teuthology.orchestra.run.smithi046.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 10 MB/s | 17 MB 00:01 2024-09-10T07:22:52.570 INFO:teuthology.orchestra.run.smithi063.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.3 MB/s | 136 kB 00:00 2024-09-10T07:22:52.572 INFO:teuthology.orchestra.run.smithi181.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 193 kB/s | 135 kB 00:00 2024-09-10T07:22:52.610 INFO:teuthology.orchestra.run.smithi046.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 2.4 MB/s | 228 kB 00:00 2024-09-10T07:22:52.679 INFO:teuthology.orchestra.run.smithi063.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.5 MB/s | 161 kB 00:00 2024-09-10T07:22:52.694 INFO:teuthology.orchestra.run.smithi046.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 324 kB/s | 27 kB 00:00 2024-09-10T07:22:52.761 INFO:teuthology.orchestra.run.smithi046.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 478 kB/s | 32 kB 00:00 2024-09-10T07:22:52.762 INFO:teuthology.orchestra.run.smithi063.stdout:(22/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-10T07:22:52.764 INFO:teuthology.orchestra.run.smithi181.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 178 kB/s | 125 kB 00:00 2024-09-10T07:22:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 2.9 MB/s | 320 kB 00:00 2024-09-10T07:22:52.821 INFO:teuthology.orchestra.run.smithi063.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.0 MB/s | 301 kB 00:00 2024-09-10T07:22:52.831 INFO:teuthology.orchestra.run.smithi181.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-09-10T07:22:52.863 INFO:teuthology.orchestra.run.smithi063.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.3 MB/s | 99 kB 00:00 2024-09-10T07:22:52.880 INFO:teuthology.orchestra.run.smithi063.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-10T07:22:52.881 INFO:teuthology.orchestra.run.smithi181.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-10T07:22:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-10T07:22:52.931 INFO:teuthology.orchestra.run.smithi181.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 436 kB/s | 22 kB 00:00 2024-09-10T07:22:52.947 INFO:teuthology.orchestra.run.smithi063.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-10T07:22:52.990 INFO:teuthology.orchestra.run.smithi181.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-10T07:22:53.014 INFO:teuthology.orchestra.run.smithi063.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 476 kB/s | 32 kB 00:00 2024-09-10T07:22:53.036 INFO:teuthology.orchestra.run.smithi046.stdout:(15/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 7.6 MB/s | 11 MB 00:01 2024-09-10T07:22:53.049 INFO:teuthology.orchestra.run.smithi181.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-09-10T07:22:53.061 INFO:teuthology.orchestra.run.smithi046.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 1.0 MB/s | 164 kB 00:00 2024-09-10T07:22:53.073 INFO:teuthology.orchestra.run.smithi063.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.5 MB/s | 146 kB 00:00 2024-09-10T07:22:53.083 INFO:teuthology.orchestra.run.smithi181.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-10T07:22:53.103 INFO:teuthology.orchestra.run.smithi046.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 1.9 MB/s | 127 kB 00:00 2024-09-10T07:22:53.132 INFO:teuthology.orchestra.run.smithi063.stdout:(30/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-10T07:22:53.137 INFO:teuthology.orchestra.run.smithi046.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 6.5 MB/s | 506 kB 00:00 2024-09-10T07:22:53.140 INFO:teuthology.orchestra.run.smithi181.stdout:(54/156): python3-urllib3-1.26.5-6.el9.noarch.r 283 kB/s | 215 kB 00:00 2024-09-10T07:22:53.166 INFO:teuthology.orchestra.run.smithi181.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-09-10T07:22:53.225 INFO:teuthology.orchestra.run.smithi181.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-10T07:22:53.246 INFO:teuthology.orchestra.run.smithi046.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 417 kB/s | 45 kB 00:00 2024-09-10T07:22:53.250 INFO:teuthology.orchestra.run.smithi181.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-10T07:22:53.283 INFO:teuthology.orchestra.run.smithi181.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-09-10T07:22:53.352 INFO:teuthology.orchestra.run.smithi181.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 8.4 MB/s | 581 kB 00:00 2024-09-10T07:22:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.2 MB/s | 136 kB 00:00 2024-09-10T07:22:53.390 INFO:teuthology.orchestra.run.smithi063.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 11 MB/s | 3.6 MB 00:00 2024-09-10T07:22:53.402 INFO:teuthology.orchestra.run.smithi181.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-09-10T07:22:53.430 INFO:teuthology.orchestra.run.smithi046.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T07:22:53.461 INFO:teuthology.orchestra.run.smithi181.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-10T07:22:53.491 INFO:teuthology.orchestra.run.smithi063.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.4 MB/s | 249 kB 00:00 2024-09-10T07:22:53.511 INFO:teuthology.orchestra.run.smithi181.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-09-10T07:22:53.530 INFO:teuthology.orchestra.run.smithi046.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 3.1 MB/s | 320 kB 00:00 2024-09-10T07:22:53.570 INFO:teuthology.orchestra.run.smithi181.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-10T07:22:53.582 INFO:teuthology.orchestra.run.smithi063.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 539 kB/s | 49 kB 00:00 2024-09-10T07:22:53.620 INFO:teuthology.orchestra.run.smithi181.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 773 kB/s | 38 kB 00:00 2024-09-10T07:22:53.622 INFO:teuthology.orchestra.run.smithi046.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.2 MB/s | 301 kB 00:00 2024-09-10T07:22:53.645 INFO:teuthology.orchestra.run.smithi181.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-10T07:22:53.683 INFO:teuthology.orchestra.run.smithi063.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 164 kB/s | 16 kB 00:00 2024-09-10T07:22:53.714 INFO:teuthology.orchestra.run.smithi046.stdout:(24/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 8.8 MB/s | 5.3 MB 00:00 2024-09-10T07:22:53.757 INFO:teuthology.orchestra.run.smithi046.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 743 kB/s | 99 kB 00:00 2024-09-10T07:22:53.758 INFO:teuthology.orchestra.run.smithi063.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 3.7 MB/s | 285 kB 00:00 2024-09-10T07:22:53.798 INFO:teuthology.orchestra.run.smithi046.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-10T07:22:53.867 INFO:teuthology.orchestra.run.smithi063.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 10 MB/s | 7.4 MB 00:00 2024-09-10T07:22:53.890 INFO:teuthology.orchestra.run.smithi046.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-10T07:22:53.900 INFO:teuthology.orchestra.run.smithi063.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 5.3 MB/s | 766 kB 00:00 2024-09-10T07:22:53.942 INFO:teuthology.orchestra.run.smithi063.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 243 kB/s | 18 kB 00:00 2024-09-10T07:22:53.957 INFO:teuthology.orchestra.run.smithi046.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 478 kB/s | 32 kB 00:00 2024-09-10T07:22:54.032 INFO:teuthology.orchestra.run.smithi046.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.9 MB/s | 146 kB 00:00 2024-09-10T07:22:54.091 INFO:teuthology.orchestra.run.smithi046.stdout:(30/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 9.3 MB/s | 3.1 MB 00:00 2024-09-10T07:22:54.092 INFO:teuthology.orchestra.run.smithi063.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 416 kB/s | 80 kB 00:00 2024-09-10T07:22:54.209 INFO:teuthology.orchestra.run.smithi063.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-10T07:22:54.251 INFO:teuthology.orchestra.run.smithi063.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-10T07:22:54.301 INFO:teuthology.orchestra.run.smithi063.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-10T07:22:54.337 INFO:teuthology.orchestra.run.smithi181.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.3 MB/s | 3.0 MB 00:01 2024-09-10T07:22:54.360 INFO:teuthology.orchestra.run.smithi063.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T07:22:54.383 INFO:teuthology.orchestra.run.smithi046.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 10 MB/s | 3.6 MB 00:00 2024-09-10T07:22:54.402 INFO:teuthology.orchestra.run.smithi063.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-10T07:22:54.427 INFO:teuthology.orchestra.run.smithi063.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-10T07:22:54.450 INFO:teuthology.orchestra.run.smithi046.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.7 MB/s | 249 kB 00:00 2024-09-10T07:22:54.461 INFO:teuthology.orchestra.run.smithi063.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-10T07:22:54.494 INFO:teuthology.orchestra.run.smithi063.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-10T07:22:54.517 INFO:teuthology.orchestra.run.smithi046.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 739 kB/s | 49 kB 00:00 2024-09-10T07:22:54.976 INFO:teuthology.orchestra.run.smithi046.stdout:(34/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 16 MB/s | 49 MB 00:03 2024-09-10T07:22:54.993 INFO:teuthology.orchestra.run.smithi046.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 35 kB/s | 16 kB 00:00 2024-09-10T07:22:55.045 INFO:teuthology.orchestra.run.smithi063.stdout:(48/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 14 MB/s | 49 MB 00:03 2024-09-10T07:22:55.051 INFO:teuthology.orchestra.run.smithi046.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 3.7 MB/s | 285 kB 00:00 2024-09-10T07:22:55.078 INFO:teuthology.orchestra.run.smithi063.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-09-10T07:22:55.120 INFO:teuthology.orchestra.run.smithi063.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-10T07:22:55.129 INFO:teuthology.orchestra.run.smithi181.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-10T07:22:55.160 INFO:teuthology.orchestra.run.smithi046.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 6.9 MB/s | 7.4 MB 00:01 2024-09-10T07:22:55.177 INFO:teuthology.orchestra.run.smithi046.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 146 kB/s | 18 kB 00:00 2024-09-10T07:22:55.187 INFO:teuthology.orchestra.run.smithi063.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-10T07:22:55.202 INFO:teuthology.orchestra.run.smithi046.stdout:(39/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 3.6 MB/s | 766 kB 00:00 2024-09-10T07:22:55.212 INFO:teuthology.orchestra.run.smithi063.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 640 kB/s | 107 kB 00:00 2024-09-10T07:22:55.237 INFO:teuthology.orchestra.run.smithi063.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-10T07:22:55.254 INFO:teuthology.orchestra.run.smithi063.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-10T07:22:55.277 INFO:teuthology.orchestra.run.smithi046.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 684 kB/s | 80 kB 00:00 2024-09-10T07:22:55.279 INFO:teuthology.orchestra.run.smithi063.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-10T07:22:55.296 INFO:teuthology.orchestra.run.smithi063.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-09-10T07:22:55.336 INFO:teuthology.orchestra.run.smithi046.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-10T07:22:55.347 INFO:teuthology.orchestra.run.smithi063.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 909 kB/s | 45 kB 00:00 2024-09-10T07:22:55.362 INFO:teuthology.orchestra.run.smithi181.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-09-10T07:22:55.369 INFO:teuthology.orchestra.run.smithi046.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-10T07:22:55.372 INFO:teuthology.orchestra.run.smithi063.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-10T07:22:55.412 INFO:teuthology.orchestra.run.smithi046.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-10T07:22:55.462 INFO:teuthology.orchestra.run.smithi046.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-10T07:22:55.479 INFO:teuthology.orchestra.run.smithi181.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-10T07:22:55.504 INFO:teuthology.orchestra.run.smithi046.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-10T07:22:55.529 INFO:teuthology.orchestra.run.smithi046.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-10T07:22:55.539 INFO:teuthology.orchestra.run.smithi063.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-10T07:22:55.572 INFO:teuthology.orchestra.run.smithi181.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.5 MB/s | 4.9 MB 00:01 2024-09-10T07:22:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-10T07:22:55.605 INFO:teuthology.orchestra.run.smithi046.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-10T07:22:55.614 INFO:teuthology.orchestra.run.smithi063.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-10T07:22:55.638 INFO:teuthology.orchestra.run.smithi046.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-09-10T07:22:55.640 INFO:teuthology.orchestra.run.smithi063.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T07:22:55.680 INFO:teuthology.orchestra.run.smithi181.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 1.6 MB/s | 910 kB 00:00 2024-09-10T07:22:55.724 INFO:teuthology.orchestra.run.smithi063.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T07:22:55.749 INFO:teuthology.orchestra.run.smithi063.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-10T07:22:55.772 INFO:teuthology.orchestra.run.smithi181.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 954 kB/s | 87 kB 00:00 2024-09-10T07:22:55.774 INFO:teuthology.orchestra.run.smithi063.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-10T07:22:56.016 INFO:teuthology.orchestra.run.smithi063.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-09-10T07:22:56.063 INFO:teuthology.orchestra.run.smithi046.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 252 kB/s | 107 kB 00:00 2024-09-10T07:22:56.122 INFO:teuthology.orchestra.run.smithi181.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-09-10T07:22:56.164 INFO:teuthology.orchestra.run.smithi046.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 217 kB/s | 22 kB 00:00 2024-09-10T07:22:56.172 INFO:teuthology.orchestra.run.smithi181.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-10T07:22:56.230 INFO:teuthology.orchestra.run.smithi046.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 983 kB/s | 689 kB 00:00 2024-09-10T07:22:56.289 INFO:teuthology.orchestra.run.smithi046.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-10T07:22:56.314 INFO:teuthology.orchestra.run.smithi181.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-10T07:22:56.441 INFO:teuthology.orchestra.run.smithi063.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-10T07:22:56.465 INFO:teuthology.orchestra.run.smithi046.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 140 kB/s | 33 kB 00:00 2024-09-10T07:22:56.473 INFO:teuthology.orchestra.run.smithi181.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-10T07:22:56.490 INFO:teuthology.orchestra.run.smithi046.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 963 kB/s | 313 kB 00:00 2024-09-10T07:22:56.523 INFO:teuthology.orchestra.run.smithi181.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-09-10T07:22:56.574 INFO:teuthology.orchestra.run.smithi046.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-10T07:22:56.574 INFO:teuthology.orchestra.run.smithi181.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-10T07:22:56.598 INFO:teuthology.orchestra.run.smithi046.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-10T07:22:56.666 INFO:teuthology.orchestra.run.smithi063.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.0 MB/s | 910 kB 00:00 2024-09-10T07:22:56.690 INFO:teuthology.orchestra.run.smithi181.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-10T07:22:56.740 INFO:teuthology.orchestra.run.smithi181.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-10T07:22:56.750 INFO:teuthology.orchestra.run.smithi063.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-10T07:22:56.837 INFO:teuthology.orchestra.run.smithi046.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 608 kB/s | 160 kB 00:00 2024-09-10T07:22:56.899 INFO:teuthology.orchestra.run.smithi046.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.9 MB/s | 3.0 MB 00:00 2024-09-10T07:22:56.925 INFO:teuthology.orchestra.run.smithi046.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 139 kB/s | 45 kB 00:00 2024-09-10T07:22:56.942 INFO:teuthology.orchestra.run.smithi063.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-10T07:22:57.033 INFO:teuthology.orchestra.run.smithi046.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 227 kB/s | 24 kB 00:00 2024-09-10T07:22:57.042 INFO:teuthology.orchestra.run.smithi063.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.8 MB/s | 4.9 MB 00:01 2024-09-10T07:22:57.075 INFO:teuthology.orchestra.run.smithi046.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-10T07:22:57.093 INFO:teuthology.orchestra.run.smithi063.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-10T07:22:57.109 INFO:teuthology.orchestra.run.smithi046.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-09-10T07:22:57.134 INFO:teuthology.orchestra.run.smithi046.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-10T07:22:57.184 INFO:teuthology.orchestra.run.smithi046.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T07:22:57.218 INFO:teuthology.orchestra.run.smithi046.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-10T07:22:57.226 INFO:teuthology.orchestra.run.smithi063.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-09-10T07:22:57.265 INFO:teuthology.orchestra.run.smithi046.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 293 kB/s | 38 kB 00:00 2024-09-10T07:22:57.310 INFO:teuthology.orchestra.run.smithi063.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 737 kB/s | 61 kB 00:00 2024-09-10T07:22:57.418 INFO:teuthology.orchestra.run.smithi063.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T07:22:57.527 INFO:teuthology.orchestra.run.smithi063.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-10T07:22:57.544 INFO:teuthology.orchestra.run.smithi046.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-10T07:22:57.577 INFO:teuthology.orchestra.run.smithi063.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-09-10T07:22:57.627 INFO:teuthology.orchestra.run.smithi063.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-10T07:22:57.669 INFO:teuthology.orchestra.run.smithi046.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-10T07:22:57.711 INFO:teuthology.orchestra.run.smithi063.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-10T07:22:57.777 INFO:teuthology.orchestra.run.smithi046.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-10T07:22:57.808 INFO:teuthology.orchestra.run.smithi181.stdout:(81/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.8 MB/s | 5.1 MB 00:01 2024-09-10T07:22:57.828 INFO:teuthology.orchestra.run.smithi046.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-09-10T07:22:57.845 INFO:teuthology.orchestra.run.smithi063.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 260 kB/s | 35 kB 00:00 2024-09-10T07:22:57.866 INFO:teuthology.orchestra.run.smithi181.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-10T07:22:57.933 INFO:teuthology.orchestra.run.smithi181.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-09-10T07:22:57.953 INFO:teuthology.orchestra.run.smithi046.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-10T07:22:58.001 INFO:teuthology.orchestra.run.smithi181.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-10T07:22:58.050 INFO:teuthology.orchestra.run.smithi181.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-10T07:22:58.061 INFO:teuthology.orchestra.run.smithi063.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 4.4 MB/s | 5.8 MB 00:01 2024-09-10T07:22:58.062 INFO:teuthology.orchestra.run.smithi046.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-09-10T07:22:58.109 INFO:teuthology.orchestra.run.smithi181.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-10T07:22:58.145 INFO:teuthology.orchestra.run.smithi063.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T07:22:58.162 INFO:teuthology.orchestra.run.smithi046.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-09-10T07:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-10T07:22:58.227 INFO:teuthology.orchestra.run.smithi181.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-10T07:22:58.262 INFO:teuthology.orchestra.run.smithi046.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 614 kB/s | 61 kB 00:00 2024-09-10T07:22:58.287 INFO:teuthology.orchestra.run.smithi063.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-09-10T07:22:58.302 INFO:teuthology.orchestra.run.smithi181.stdout:(89/156): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-09-10T07:22:58.327 INFO:teuthology.orchestra.run.smithi181.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-09-10T07:22:58.337 INFO:teuthology.orchestra.run.smithi063.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T07:22:58.352 INFO:teuthology.orchestra.run.smithi181.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T07:22:58.371 INFO:teuthology.orchestra.run.smithi046.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T07:22:58.412 INFO:teuthology.orchestra.run.smithi063.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 831 kB/s | 222 kB 00:00 2024-09-10T07:22:58.428 INFO:teuthology.orchestra.run.smithi181.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-09-10T07:22:58.437 INFO:teuthology.orchestra.run.smithi063.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-09-10T07:22:58.479 INFO:teuthology.orchestra.run.smithi046.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-10T07:22:58.495 INFO:teuthology.orchestra.run.smithi181.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 664 kB/s | 44 kB 00:00 2024-09-10T07:22:58.496 INFO:teuthology.orchestra.run.smithi063.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-10T07:22:58.522 INFO:teuthology.orchestra.run.smithi063.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-10T07:22:58.580 INFO:teuthology.orchestra.run.smithi063.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 877 kB/s | 51 kB 00:00 2024-09-10T07:22:58.596 INFO:teuthology.orchestra.run.smithi046.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-09-10T07:22:58.605 INFO:teuthology.orchestra.run.smithi063.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-10T07:22:58.622 INFO:teuthology.orchestra.run.smithi063.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-10T07:22:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T07:22:58.697 INFO:teuthology.orchestra.run.smithi046.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-09-10T07:22:58.706 INFO:teuthology.orchestra.run.smithi063.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-10T07:22:58.720 INFO:teuthology.orchestra.run.smithi181.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-09-10T07:22:58.790 INFO:teuthology.orchestra.run.smithi063.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-10T07:22:58.795 INFO:teuthology.orchestra.run.smithi181.stdout:(95/156): postgresql-server-13.16-1.el9.x86_64. 1.7 MB/s | 5.8 MB 00:03 2024-09-10T07:22:58.805 INFO:teuthology.orchestra.run.smithi046.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-10T07:22:58.820 INFO:teuthology.orchestra.run.smithi181.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-09-10T07:22:58.832 INFO:teuthology.orchestra.run.smithi063.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 860 kB/s | 36 kB 00:00 2024-09-10T07:22:58.846 INFO:teuthology.orchestra.run.smithi181.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 629 kB/s | 221 kB 00:00 2024-09-10T07:22:58.871 INFO:teuthology.orchestra.run.smithi181.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-10T07:22:58.874 INFO:teuthology.orchestra.run.smithi063.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T07:22:58.897 INFO:teuthology.orchestra.run.smithi181.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T07:22:58.905 INFO:teuthology.orchestra.run.smithi046.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-09-10T07:22:58.921 INFO:teuthology.orchestra.run.smithi181.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T07:22:58.957 INFO:teuthology.orchestra.run.smithi063.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-10T07:22:58.972 INFO:teuthology.orchestra.run.smithi181.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-10T07:22:58.997 INFO:teuthology.orchestra.run.smithi181.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 635 kB/s | 64 kB 00:00 2024-09-10T07:22:59.007 INFO:teuthology.orchestra.run.smithi063.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T07:22:59.039 INFO:teuthology.orchestra.run.smithi181.stdout:(103/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-10T07:22:59.058 INFO:teuthology.orchestra.run.smithi063.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-10T07:22:59.098 INFO:teuthology.orchestra.run.smithi181.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T07:22:59.115 INFO:teuthology.orchestra.run.smithi063.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-10T07:22:59.123 INFO:teuthology.orchestra.run.smithi181.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-09-10T07:22:59.157 INFO:teuthology.orchestra.run.smithi181.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.1 MB/s | 309 kB 00:00 2024-09-10T07:22:59.165 INFO:teuthology.orchestra.run.smithi063.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-09-10T07:22:59.181 INFO:teuthology.orchestra.run.smithi046.stdout:(81/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-10T07:22:59.182 INFO:teuthology.orchestra.run.smithi181.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-09-10T07:22:59.207 INFO:teuthology.orchestra.run.smithi181.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T07:22:59.215 INFO:teuthology.orchestra.run.smithi063.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-09-10T07:22:59.233 INFO:teuthology.orchestra.run.smithi181.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T07:22:59.258 INFO:teuthology.orchestra.run.smithi181.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T07:22:59.283 INFO:teuthology.orchestra.run.smithi181.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T07:22:59.289 INFO:teuthology.orchestra.run.smithi046.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-10T07:22:59.325 INFO:teuthology.orchestra.run.smithi181.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-10T07:22:59.358 INFO:teuthology.orchestra.run.smithi181.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-10T07:22:59.398 INFO:teuthology.orchestra.run.smithi046.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-10T07:22:59.400 INFO:teuthology.orchestra.run.smithi181.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-10T07:22:59.416 INFO:teuthology.orchestra.run.smithi063.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-09-10T07:22:59.434 INFO:teuthology.orchestra.run.smithi181.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.0 MB/s | 32 kB 00:00 2024-09-10T07:22:59.460 INFO:teuthology.orchestra.run.smithi181.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 431 kB/s | 43 kB 00:00 2024-09-10T07:22:59.466 INFO:teuthology.orchestra.run.smithi063.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T07:22:59.485 INFO:teuthology.orchestra.run.smithi181.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-10T07:22:59.491 INFO:teuthology.orchestra.run.smithi063.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T07:22:59.506 INFO:teuthology.orchestra.run.smithi046.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 713 kB/s | 77 kB 00:00 2024-09-10T07:22:59.510 INFO:teuthology.orchestra.run.smithi181.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T07:22:59.517 INFO:teuthology.orchestra.run.smithi063.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-10T07:22:59.544 INFO:teuthology.orchestra.run.smithi181.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-09-10T07:22:59.569 INFO:teuthology.orchestra.run.smithi181.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-10T07:22:59.594 INFO:teuthology.orchestra.run.smithi181.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T07:22:59.607 INFO:teuthology.orchestra.run.smithi046.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-09-10T07:22:59.617 INFO:teuthology.orchestra.run.smithi063.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-10T07:22:59.634 INFO:teuthology.orchestra.run.smithi063.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-10T07:22:59.651 INFO:teuthology.orchestra.run.smithi063.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-10T07:22:59.661 INFO:teuthology.orchestra.run.smithi181.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-09-10T07:22:59.668 INFO:teuthology.orchestra.run.smithi063.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 4.0 MB/s | 67 kB 00:00 2024-09-10T07:22:59.687 INFO:teuthology.orchestra.run.smithi181.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-10T07:22:59.693 INFO:teuthology.orchestra.run.smithi063.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T07:22:59.712 INFO:teuthology.orchestra.run.smithi181.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-10T07:22:59.715 INFO:teuthology.orchestra.run.smithi046.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-10T07:22:59.737 INFO:teuthology.orchestra.run.smithi181.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-09-10T07:22:59.752 INFO:teuthology.orchestra.run.smithi063.stdout:(112/156): python3-numpy-1.20.1-5.el9.x86_64.rp 2.7 MB/s | 5.1 MB 00:01 2024-09-10T07:22:59.762 INFO:teuthology.orchestra.run.smithi181.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-10T07:22:59.788 INFO:teuthology.orchestra.run.smithi181.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-10T07:22:59.802 INFO:teuthology.orchestra.run.smithi063.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-10T07:22:59.813 INFO:teuthology.orchestra.run.smithi181.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T07:22:59.824 INFO:teuthology.orchestra.run.smithi046.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-10T07:22:59.836 INFO:teuthology.orchestra.run.smithi063.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-10T07:22:59.838 INFO:teuthology.orchestra.run.smithi181.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T07:22:59.853 INFO:teuthology.orchestra.run.smithi063.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-10T07:22:59.878 INFO:teuthology.orchestra.run.smithi063.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-10T07:22:59.903 INFO:teuthology.orchestra.run.smithi063.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-10T07:22:59.909 INFO:teuthology.orchestra.run.smithi181.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-09-10T07:22:59.939 INFO:teuthology.orchestra.run.smithi181.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.9 MB/s | 58 kB 00:00 2024-09-10T07:22:59.941 INFO:teuthology.orchestra.run.smithi046.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-10T07:22:59.945 INFO:teuthology.orchestra.run.smithi063.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-09-10T07:22:59.972 INFO:teuthology.orchestra.run.smithi181.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-10T07:22:59.979 INFO:teuthology.orchestra.run.smithi063.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-10T07:22:59.998 INFO:teuthology.orchestra.run.smithi181.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 653 kB/s | 16 kB 00:00 2024-09-10T07:23:00.005 INFO:teuthology.orchestra.run.smithi063.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T07:23:00.029 INFO:teuthology.orchestra.run.smithi063.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T07:23:00.032 INFO:teuthology.orchestra.run.smithi181.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-09-10T07:23:00.050 INFO:teuthology.orchestra.run.smithi046.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 472 kB/s | 51 kB 00:00 2024-09-10T07:23:00.054 INFO:teuthology.orchestra.run.smithi063.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T07:23:00.057 INFO:teuthology.orchestra.run.smithi181.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-10T07:23:00.080 INFO:teuthology.orchestra.run.smithi063.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-10T07:23:00.082 INFO:teuthology.orchestra.run.smithi181.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T07:23:00.105 INFO:teuthology.orchestra.run.smithi063.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-10T07:23:00.107 INFO:teuthology.orchestra.run.smithi181.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-10T07:23:00.130 INFO:teuthology.orchestra.run.smithi063.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-09-10T07:23:00.133 INFO:teuthology.orchestra.run.smithi181.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T07:23:00.156 INFO:teuthology.orchestra.run.smithi063.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-10T07:23:00.158 INFO:teuthology.orchestra.run.smithi181.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T07:23:00.158 INFO:teuthology.orchestra.run.smithi046.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 496 kB/s | 54 kB 00:00 2024-09-10T07:23:00.181 INFO:teuthology.orchestra.run.smithi063.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-10T07:23:00.183 INFO:teuthology.orchestra.run.smithi181.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-10T07:23:00.209 INFO:teuthology.orchestra.run.smithi181.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T07:23:00.248 INFO:teuthology.orchestra.run.smithi063.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-10T07:23:00.250 INFO:teuthology.orchestra.run.smithi181.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-09-10T07:23:00.276 INFO:teuthology.orchestra.run.smithi181.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-10T07:23:00.282 INFO:teuthology.orchestra.run.smithi063.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-09-10T07:23:00.301 INFO:teuthology.orchestra.run.smithi181.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-10T07:23:00.307 INFO:teuthology.orchestra.run.smithi063.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 792 kB/s | 46 kB 00:00 2024-09-10T07:23:00.324 INFO:teuthology.orchestra.run.smithi063.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-10T07:23:00.327 INFO:teuthology.orchestra.run.smithi181.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-10T07:23:00.349 INFO:teuthology.orchestra.run.smithi063.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-09-10T07:23:00.360 INFO:teuthology.orchestra.run.smithi181.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T07:23:00.374 INFO:teuthology.orchestra.run.smithi063.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-10T07:23:00.394 INFO:teuthology.orchestra.run.smithi181.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-10T07:23:00.400 INFO:teuthology.orchestra.run.smithi063.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T07:23:00.425 INFO:teuthology.orchestra.run.smithi063.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T07:23:00.436 INFO:teuthology.orchestra.run.smithi181.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-10T07:23:00.450 INFO:teuthology.orchestra.run.smithi063.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T07:23:00.476 INFO:teuthology.orchestra.run.smithi063.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-10T07:23:00.501 INFO:teuthology.orchestra.run.smithi063.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T07:23:00.526 INFO:teuthology.orchestra.run.smithi063.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T07:23:00.543 INFO:teuthology.orchestra.run.smithi063.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-10T07:23:00.568 INFO:teuthology.orchestra.run.smithi063.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-10T07:23:00.594 INFO:teuthology.orchestra.run.smithi063.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T07:23:00.619 INFO:teuthology.orchestra.run.smithi063.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T07:23:00.644 INFO:teuthology.orchestra.run.smithi063.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-10T07:23:00.678 INFO:teuthology.orchestra.run.smithi063.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-10T07:23:00.678 INFO:teuthology.orchestra.run.smithi181.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-10T07:23:00.711 INFO:teuthology.orchestra.run.smithi063.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-10T07:23:00.745 INFO:teuthology.orchestra.run.smithi181.stdout:(150/156): librados2-19.1.1-305.g1cf819b7.el9.x 8.9 MB/s | 3.4 MB 00:00 2024-09-10T07:23:00.870 INFO:teuthology.orchestra.run.smithi063.stdout:(147/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.2 MB/s | 16 MB 00:02 2024-09-10T07:23:00.921 INFO:teuthology.orchestra.run.smithi063.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-10T07:23:00.934 INFO:teuthology.orchestra.run.smithi046.stdout:(91/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-10T07:23:01.034 INFO:teuthology.orchestra.run.smithi046.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-09-10T07:23:01.054 INFO:teuthology.orchestra.run.smithi063.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-10T07:23:01.096 INFO:teuthology.orchestra.run.smithi063.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-10T07:23:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-09-10T07:23:01.171 INFO:teuthology.orchestra.run.smithi063.stdout:(151/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-10T07:23:01.188 INFO:teuthology.orchestra.run.smithi063.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 444 kB/s | 41 kB 00:00 2024-09-10T07:23:01.230 INFO:teuthology.orchestra.run.smithi063.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 10 MB/s | 1.8 MB 00:00 2024-09-10T07:23:01.251 INFO:teuthology.orchestra.run.smithi046.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-10T07:23:01.272 INFO:teuthology.orchestra.run.smithi063.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-10T07:23:01.312 INFO:teuthology.orchestra.run.smithi181.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 72 kB/s | 41 kB 00:00 2024-09-10T07:23:01.347 INFO:teuthology.orchestra.run.smithi063.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-10T07:23:01.368 INFO:teuthology.orchestra.run.smithi046.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-10T07:23:01.455 INFO:teuthology.orchestra.run.smithi063.stdout:(156/156): NetworkManager-tui-1.49.90-1.el9.x86 876 kB/s | 249 kB 00:00 2024-09-10T07:23:01.460 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:23:01.460 INFO:teuthology.orchestra.run.smithi063.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-10T07:23:01.477 INFO:teuthology.orchestra.run.smithi046.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-09-10T07:23:01.546 INFO:teuthology.orchestra.run.smithi181.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-10T07:23:01.610 INFO:teuthology.orchestra.run.smithi046.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-10T07:23:01.722 INFO:teuthology.orchestra.run.smithi046.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-09-10T07:23:01.754 INFO:teuthology.orchestra.run.smithi181.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-09-10T07:23:01.828 INFO:teuthology.orchestra.run.smithi046.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 885 kB/s | 96 kB 00:00 2024-09-10T07:23:01.936 INFO:teuthology.orchestra.run.smithi046.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-09-10T07:23:02.096 INFO:teuthology.orchestra.run.smithi181.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 317 kB/s | 249 kB 00:00 2024-09-10T07:23:02.137 INFO:teuthology.orchestra.run.smithi046.stdout:(101/156): openblas-openmp-0.3.26-2.el9.x86_64. 1.0 MB/s | 4.9 MB 00:04 2024-09-10T07:23:02.162 INFO:teuthology.orchestra.run.smithi046.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 199 kB/s | 45 kB 00:00 2024-09-10T07:23:02.221 INFO:teuthology.orchestra.run.smithi046.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-09-10T07:23:02.279 INFO:teuthology.orchestra.run.smithi046.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T07:23:02.313 INFO:teuthology.orchestra.run.smithi046.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-09-10T07:23:02.347 INFO:teuthology.orchestra.run.smithi046.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 588 kB/s | 19 kB 00:00 2024-09-10T07:23:02.380 INFO:teuthology.orchestra.run.smithi046.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-09-10T07:23:02.431 INFO:teuthology.orchestra.run.smithi046.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-10T07:23:02.487 INFO:teuthology.orchestra.run.smithi046.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 870 kB/s | 49 kB 00:00 2024-09-10T07:23:02.515 INFO:teuthology.orchestra.run.smithi046.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-10T07:23:02.540 INFO:teuthology.orchestra.run.smithi046.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T07:23:02.554 INFO:teuthology.orchestra.run.smithi181.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:01 2024-09-10T07:23:02.599 INFO:teuthology.orchestra.run.smithi046.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-10T07:23:02.641 INFO:teuthology.orchestra.run.smithi046.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-10T07:23:02.666 INFO:teuthology.orchestra.run.smithi046.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T07:23:02.682 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-09-10T07:23:02.691 INFO:teuthology.orchestra.run.smithi046.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T07:23:02.716 INFO:teuthology.orchestra.run.smithi046.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-10T07:23:02.742 INFO:teuthology.orchestra.run.smithi046.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-10T07:23:02.776 INFO:teuthology.orchestra.run.smithi046.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-10T07:23:02.809 INFO:teuthology.orchestra.run.smithi046.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-10T07:23:02.813 INFO:teuthology.orchestra.run.smithi181.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-10T07:23:02.820 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:23:02.820 INFO:teuthology.orchestra.run.smithi181.stdout:Total 14 MB/s | 226 MB 00:16 2024-09-10T07:23:02.846 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-09-10T07:23:02.847 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-09-10T07:23:02.876 INFO:teuthology.orchestra.run.smithi046.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-09-10T07:23:02.952 INFO:teuthology.orchestra.run.smithi046.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 1.5 MB/s | 220 kB 00:00 2024-09-10T07:23:02.986 INFO:teuthology.orchestra.run.smithi046.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T07:23:03.011 INFO:teuthology.orchestra.run.smithi046.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-10T07:23:03.036 INFO:teuthology.orchestra.run.smithi046.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-10T07:23:03.061 INFO:teuthology.orchestra.run.smithi046.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 936 kB/s | 23 kB 00:00 2024-09-10T07:23:03.087 INFO:teuthology.orchestra.run.smithi046.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-09-10T07:23:03.111 INFO:teuthology.orchestra.run.smithi046.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 800 kB/s | 20 kB 00:00 2024-09-10T07:23:03.170 INFO:teuthology.orchestra.run.smithi046.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-10T07:23:03.204 INFO:teuthology.orchestra.run.smithi046.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-10T07:23:03.263 INFO:teuthology.orchestra.run.smithi046.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-09-10T07:23:03.288 INFO:teuthology.orchestra.run.smithi046.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 947 kB/s | 79 kB 00:00 2024-09-10T07:23:03.343 INFO:teuthology.orchestra.run.smithi046.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 722 kB/s | 58 kB 00:00 2024-09-10T07:23:03.372 INFO:teuthology.orchestra.run.smithi046.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 579 kB/s | 16 kB 00:00 2024-09-10T07:23:03.397 INFO:teuthology.orchestra.run.smithi046.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T07:23:03.422 INFO:teuthology.orchestra.run.smithi046.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-10T07:23:03.448 INFO:teuthology.orchestra.run.smithi046.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-09-10T07:23:03.473 INFO:teuthology.orchestra.run.smithi046.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-10T07:23:03.498 INFO:teuthology.orchestra.run.smithi046.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T07:23:03.523 INFO:teuthology.orchestra.run.smithi046.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T07:23:03.549 INFO:teuthology.orchestra.run.smithi046.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-10T07:23:03.574 INFO:teuthology.orchestra.run.smithi046.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T07:23:03.600 INFO:teuthology.orchestra.run.smithi046.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T07:23:03.633 INFO:teuthology.orchestra.run.smithi046.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-10T07:23:03.658 INFO:teuthology.orchestra.run.smithi046.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 814 kB/s | 20 kB 00:00 2024-09-10T07:23:03.701 INFO:teuthology.orchestra.run.smithi046.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-10T07:23:03.734 INFO:teuthology.orchestra.run.smithi046.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T07:23:03.801 INFO:teuthology.orchestra.run.smithi046.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T07:23:04.002 INFO:teuthology.orchestra.run.smithi046.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-10T07:23:04.059 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-09-10T07:23:04.110 INFO:teuthology.orchestra.run.smithi046.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-10T07:23:04.210 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-09-10T07:23:04.210 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-09-10T07:23:04.219 INFO:teuthology.orchestra.run.smithi046.stdout:(150/156): python3-babel-2.9.1-2.el9.noarch.rpm 992 kB/s | 6.0 MB 00:06 2024-09-10T07:23:04.295 INFO:teuthology.orchestra.run.smithi046.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 543 kB/s | 41 kB 00:00 2024-09-10T07:23:04.378 INFO:teuthology.orchestra.run.smithi046.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-10T07:23:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 3.7 MB/s | 2.3 MB 00:00 2024-09-10T07:23:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 4.6 MB/s | 1.4 MB 00:00 2024-09-10T07:23:04.712 INFO:teuthology.orchestra.run.smithi046.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 3.0 MB/s | 1.8 MB 00:00 2024-09-10T07:23:04.748 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-09-10T07:23:04.749 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-09-10T07:23:05.295 INFO:teuthology.orchestra.run.smithi046.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 2.4 MB 00:00 2024-09-10T07:23:05.300 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:23:05.301 INFO:teuthology.orchestra.run.smithi046.stdout:Total 14 MB/s | 226 MB 00:16 2024-09-10T07:23:06.020 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-09-10T07:23:06.020 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-09-10T07:23:07.441 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T07:23:07.635 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T07:23:07.635 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T07:23:10.079 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T07:23:10.080 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T07:23:11.463 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-09-10T07:23:11.609 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T07:23:11.648 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T07:23:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T07:23:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T07:23:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T07:23:11.829 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T07:23:12.176 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T07:23:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:23:12.410 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:23:12.437 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:23:12.510 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-09-10T07:23:12.529 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:23:12.558 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:23:12.633 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T07:23:12.671 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T07:23:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T07:23:12.741 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T07:23:12.775 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T07:23:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T07:23:12.801 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T07:23:12.808 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:23:12.852 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T07:23:12.913 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:23:12.942 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:23:13.005 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:23:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:23:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:23:13.173 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T07:23:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:23:13.286 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:23:13.338 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:23:13.416 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:23:13.445 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:23:13.477 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T07:23:13.537 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:23:13.566 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:23:13.573 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:13.651 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:13.714 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T07:23:13.763 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T07:23:13.799 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:23:13.868 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:13.903 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:23:13.934 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:23:13.978 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T07:23:13.997 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:23:14.027 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:23:14.046 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:23:14.073 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T07:23:14.105 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T07:23:14.141 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T07:23:14.216 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T07:23:14.245 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:23:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T07:23:14.307 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T07:23:14.350 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T07:23:14.383 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T07:23:14.392 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:23:14.410 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T07:23:14.482 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T07:23:14.508 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T07:23:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T07:23:14.563 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T07:23:14.599 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T07:23:14.602 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:14.678 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T07:23:14.685 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:14.709 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T07:23:14.745 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T07:23:14.840 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T07:23:14.909 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:15.029 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T07:23:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T07:23:15.113 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T07:23:15.130 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T07:23:15.165 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T07:23:15.190 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T07:23:15.209 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T07:23:15.299 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T07:23:15.338 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T07:23:15.342 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T07:23:15.374 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T07:23:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T07:23:15.418 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T07:23:15.450 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T07:23:15.487 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T07:23:15.556 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T07:23:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T07:23:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T07:23:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T07:23:15.773 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T07:23:15.814 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T07:23:15.866 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T07:23:15.895 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T07:23:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T07:23:15.965 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T07:23:16.010 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T07:23:16.040 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T07:23:16.073 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T07:23:16.085 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T07:23:16.116 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T07:23:16.154 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T07:23:16.195 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T07:23:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T07:23:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T07:23:16.357 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T07:23:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T07:23:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T07:23:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T07:23:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T07:23:16.613 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T07:23:16.661 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T07:23:16.695 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T07:23:16.739 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T07:23:16.781 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T07:23:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T07:23:16.866 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T07:23:16.895 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T07:23:16.938 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T07:23:16.976 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T07:23:17.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T07:23:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T07:23:17.158 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T07:23:17.185 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T07:23:17.200 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T07:23:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T07:23:17.245 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T07:23:17.301 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T07:23:17.307 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T07:23:17.341 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T07:23:17.373 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T07:23:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T07:23:17.380 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T07:23:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T07:23:17.406 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T07:23:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T07:23:17.430 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T07:23:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T07:23:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T07:23:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T07:23:17.509 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T07:23:17.524 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T07:23:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T07:23:17.568 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T07:23:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T07:23:17.610 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T07:23:17.636 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T07:23:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T07:23:17.680 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T07:23:17.707 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T07:23:17.742 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T07:23:17.756 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T07:23:17.788 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T07:23:17.845 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T07:23:17.860 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T07:23:17.874 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T07:23:17.909 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T07:23:17.942 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T07:23:17.984 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T07:23:18.018 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T07:23:18.040 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T07:23:18.108 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T07:23:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T07:23:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:23:18.165 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T07:23:18.196 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T07:23:18.220 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T07:23:18.254 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T07:23:18.276 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:23:18.294 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T07:23:18.304 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T07:23:18.304 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:23:18.328 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T07:23:18.353 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T07:23:18.355 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T07:23:18.402 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:23:18.428 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T07:23:18.440 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:23:18.562 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T07:23:18.610 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T07:23:18.638 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T07:23:18.681 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T07:23:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T07:23:18.728 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T07:23:18.748 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T07:23:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T07:23:18.775 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:23:18.789 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T07:23:18.832 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T07:23:18.865 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T07:23:18.907 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T07:23:18.912 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:23:18.950 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T07:23:19.037 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T07:23:19.043 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:23:19.081 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T07:23:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:23:19.128 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T07:23:19.129 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T07:23:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:23:19.179 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T07:23:19.196 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:23:19.213 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T07:23:19.232 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T07:23:19.279 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T07:23:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T07:23:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T07:23:19.422 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T07:23:19.446 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:23:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T07:23:19.548 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T07:23:19.639 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:23:19.740 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T07:23:19.770 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T07:23:19.850 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T07:23:19.869 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T07:23:19.990 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:19.999 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T07:23:20.053 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T07:23:20.086 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:20.087 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T07:23:20.186 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T07:23:20.242 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T07:23:20.287 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T07:23:20.375 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T07:23:20.393 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:23:20.406 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T07:23:20.534 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T07:23:20.573 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T07:23:20.591 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T07:23:20.621 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T07:23:20.648 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T07:23:20.698 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T07:23:20.699 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T07:23:20.744 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T07:23:20.779 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T07:23:20.863 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T07:23:20.916 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T07:23:20.953 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T07:23:21.009 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T07:23:21.046 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T07:23:21.080 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T07:23:21.209 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T07:23:21.284 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T07:23:21.317 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T07:23:21.341 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T07:23:21.364 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T07:23:21.448 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T07:23:21.452 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T07:23:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T07:23:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T07:23:21.593 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T07:23:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T07:23:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T07:23:21.850 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T07:23:21.950 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T07:23:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T07:23:22.032 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T07:23:22.068 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T07:23:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T07:23:22.101 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T07:23:22.428 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T07:23:22.481 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:23:22.572 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:23:22.602 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T07:23:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T07:23:22.683 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T07:23:22.775 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T07:23:22.778 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T07:23:22.802 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T07:23:22.836 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T07:23:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T07:23:22.927 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T07:23:23.001 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T07:23:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T07:23:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T07:23:23.148 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T07:23:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:23:23.187 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T07:23:23.199 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:23:23.254 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:23:23.277 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T07:23:23.290 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:23:23.320 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T07:23:23.365 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T07:23:23.443 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T07:23:23.448 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T07:23:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T07:23:23.602 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T07:23:23.655 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T07:23:23.691 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T07:23:23.767 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T07:23:23.806 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T07:23:23.846 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T07:23:23.853 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T07:23:23.885 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:23:23.903 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T07:23:23.956 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:23:23.987 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T07:23:24.083 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T07:23:24.164 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T07:23:24.213 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T07:23:24.344 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T07:23:24.394 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T07:23:24.436 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T07:23:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T07:23:24.499 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T07:23:24.634 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T07:23:24.802 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T07:23:24.844 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T07:23:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T07:23:24.916 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T07:23:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T07:23:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T07:23:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T07:23:25.117 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T07:23:25.150 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T07:23:25.194 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T07:23:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T07:23:25.329 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T07:23:25.386 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T07:23:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T07:23:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T07:23:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:25.653 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T07:23:25.673 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:26.247 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:26.249 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:26.293 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:23:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:26.387 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:23:26.387 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T07:23:26.387 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T07:23:26.387 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:26.410 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:23:26.593 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T07:23:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T07:23:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T07:23:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T07:23:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T07:23:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:23:26.967 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T07:23:27.020 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:23:27.020 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T07:23:27.020 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T07:23:27.020 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:27.047 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:23:27.076 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T07:23:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T07:23:27.506 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T07:23:27.537 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T07:23:27.667 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T07:23:28.541 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T07:23:28.705 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T07:23:30.346 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T07:23:30.452 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T07:23:30.479 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T07:23:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T07:23:30.873 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T07:23:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:23:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:23:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T07:23:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T07:23:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T07:23:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:23:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:23:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T07:23:34.672 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:34.708 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:35.374 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:23:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:23:35.559 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:23:35.559 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T07:23:35.559 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T07:23:35.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:35.613 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:23:38.899 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:23:38.899 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-09-10T07:23:38.899 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-09-10T07:23:38.899 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-09-10T07:23:38.900 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-09-10T07:23:38.900 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-09-10T07:23:38.900 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-09-10T07:23:38.900 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-09-10T07:23:38.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:38.903 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:23:38.904 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-09-10T07:23:38.904 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-09-10T07:23:38.904 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-09-10T07:23:38.904 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-09-10T07:23:38.904 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-09-10T07:23:38.904 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /root 2024-09-10T07:23:38.904 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-09-10T07:23:38.904 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:23:39.083 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:23:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:23:39.099 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:39.099 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T07:23:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:39.100 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:23:39.100 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:39.100 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T07:23:39.100 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:40.586 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:23:40.613 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:23:40.613 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:40.613 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T07:23:40.613 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:23:40.614 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:23:40.614 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:40.632 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:23:40.658 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:23:40.658 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:40.658 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T07:23:40.658 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:23:40.658 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:23:40.659 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:40.845 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:23:40.869 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:23:40.869 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:40.869 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T07:23:40.869 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:23:40.869 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:23:40.869 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:40.899 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:23:40.922 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:23:40.922 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:40.922 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T07:23:40.923 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:23:40.923 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:23:40.923 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:41.313 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:23:41.336 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:23:41.336 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:41.336 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T07:23:41.336 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:23:41.337 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:23:41.337 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:41.391 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:23:41.414 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:23:41.414 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:41.414 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T07:23:41.414 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:23:41.414 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:23:41.414 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:41.443 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T07:23:41.491 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T07:23:41.522 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T07:23:41.546 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T07:23:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T07:23:41.644 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T07:23:41.737 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T07:23:41.788 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T07:23:41.849 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T07:23:41.857 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T07:23:41.901 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T07:23:41.944 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T07:23:41.981 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T07:23:42.071 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T07:23:42.082 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T07:23:42.147 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T07:23:42.179 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T07:23:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T07:23:42.222 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T07:23:42.275 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T07:23:42.307 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T07:23:42.334 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T07:23:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:23:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:23:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T07:23:42.780 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:23:42.790 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T07:23:42.808 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:23:42.820 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:23:42.840 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T07:23:42.904 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T07:23:42.931 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:23:43.856 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:23:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:23:44.037 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:23:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:23:44.859 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:23:44.889 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:23:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:23:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:23:45.152 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T07:23:45.158 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:23:45.200 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:23:45.224 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:23:45.224 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:45.225 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T07:23:45.225 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:23:45.225 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:23:45.225 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:45.267 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:23:45.275 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:23:45.287 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:23:45.384 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T07:23:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T07:23:45.444 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T07:23:45.468 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:23:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:23:45.492 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:45.492 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T07:23:45.492 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:23:45.492 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:23:45.492 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:45.543 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:23:45.555 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:23:45.657 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T07:23:45.710 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T07:23:46.352 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T07:23:46.386 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:23:46.410 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:23:46.410 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:46.410 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T07:23:46.410 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:23:46.410 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:23:46.410 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:46.453 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:23:46.476 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:23:46.477 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:46.477 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T07:23:46.477 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:46.521 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:23:46.545 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:23:46.545 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:46.545 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T07:23:46.545 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:23:46.545 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:23:46.545 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:46.689 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T07:23:46.721 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:23:46.745 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:23:46.745 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:46.745 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T07:23:46.745 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:23:46.745 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:23:46.745 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:46.788 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:23:46.811 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:23:46.811 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:46.811 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T07:23:46.811 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:46.846 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:23:46.864 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:23:46.871 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:23:46.871 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:46.871 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T07:23:46.871 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:23:46.871 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:23:46.871 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:23:46.887 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:23:46.887 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:46.888 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T07:23:46.888 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:23:46.888 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:23:46.888 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:47.207 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:23:47.230 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:23:47.230 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:47.230 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T07:23:47.230 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:23:47.230 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:23:47.231 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:23:49.173 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:23:49.173 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-09-10T07:23:49.173 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-09-10T07:23:49.173 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-09-10T07:23:49.173 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-09-10T07:23:49.173 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-09-10T07:23:49.174 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-09-10T07:23:49.174 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-09-10T07:23:49.174 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:49.393 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:23:49.449 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:23:49.450 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:49.450 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T07:23:49.450 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:51.114 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:23:51.174 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:23:51.174 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:51.174 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T07:23:51.174 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:23:51.174 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:23:51.174 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:51.448 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:23:51.499 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:23:51.499 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:51.499 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T07:23:51.499 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:23:51.499 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:23:51.499 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:51.526 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T07:23:51.575 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T07:23:51.615 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T07:23:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T07:23:51.675 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T07:23:51.854 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T07:23:51.905 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T07:23:51.936 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T07:23:51.965 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T07:23:52.012 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T07:23:52.015 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:23:52.061 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T07:23:52.067 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:23:52.067 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:52.067 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T07:23:52.067 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:23:52.068 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:23:52.068 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:52.103 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T07:23:52.144 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T07:23:52.175 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T07:23:52.199 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T07:23:52.211 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T07:23:52.261 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T07:23:52.350 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T07:23:52.411 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T07:23:52.456 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T07:23:52.487 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T07:23:52.525 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T07:23:52.579 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T07:23:52.596 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T07:23:52.639 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:23:52.653 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T07:23:52.676 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:23:52.697 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:23:52.743 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T07:23:52.870 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T07:23:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T07:23:53.121 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T07:23:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:23:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T07:23:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:23:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T07:23:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:23:53.528 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T07:23:53.631 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T07:23:53.671 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T07:23:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:23:53.751 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T07:23:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:23:53.784 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T07:23:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T07:23:53.809 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T07:23:53.843 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T07:23:53.844 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:23:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T07:23:53.906 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:23:53.936 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:23:53.936 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:23:54.020 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:23:54.045 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T07:23:54.051 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:23:54.115 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:23:54.132 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T07:23:54.230 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:23:54.230 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:23:54.231 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T07:23:54.273 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T07:23:54.323 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:23:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T07:23:54.388 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:23:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T07:23:54.426 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T07:23:54.451 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T07:23:54.452 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:23:54.535 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:23:54.536 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:23:54.628 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:23:54.661 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T07:23:54.666 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:23:54.666 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:23:54.855 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:23:54.855 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:23:54.939 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:23:55.005 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:23:55.208 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:23:55.283 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:23:56.435 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:23:56.466 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:23:56.612 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:23:56.797 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T07:23:56.827 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:23:56.880 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:23:56.880 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:56.880 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T07:23:56.880 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:23:56.881 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:23:56.881 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:56.936 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:23:56.964 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:23:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T07:23:57.162 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T07:23:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T07:23:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:23:58.342 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:23:58.342 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:58.342 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T07:23:58.342 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:23:58.342 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:23:58.342 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:58.389 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:23:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:23:58.441 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:58.441 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T07:23:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:23:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:23:58.551 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:58.551 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T07:23:58.551 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:23:58.551 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:23:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:23:58.899 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:23:58.952 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:23:58.952 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:23:58.952 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T07:23:58.952 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:23:58.952 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:23:58.952 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:24:00.073 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:24:00.073 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T07:24:00.073 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T07:24:00.074 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T07:24:00.075 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T07:24:00.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T07:24:00.091 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T07:24:00.092 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T07:24:00.093 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T07:24:00.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T07:24:00.095 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T07:24:00.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T07:24:00.097 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T07:24:00.098 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T07:24:00.099 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T07:24:00.099 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T07:24:00.099 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T07:24:00.099 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T07:24:00.099 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T07:24:00.099 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T07:24:00.513 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:24:00.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T07:24:00.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T07:24:00.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T07:24:00.513 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T07:24:00.514 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T07:24:00.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T07:24:00.526 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T07:24:00.526 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T07:24:00.526 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T07:24:00.526 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T07:24:00.527 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T07:24:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T07:24:00.529 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T07:24:00.530 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T07:24:00.531 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T07:24:00.532 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T07:24:00.533 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T07:24:00.534 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T07:24:00.535 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T07:24:00.536 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T07:24:02.732 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:02.733 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.735 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T07:24:02.736 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.737 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:02.738 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.739 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T07:24:02.740 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T07:24:02.741 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T07:24:02.742 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T07:24:02.743 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T07:24:02.744 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T07:24:02.745 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T07:24:02.745 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T07:24:02.745 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T07:24:02.745 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T07:24:02.745 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:24:02.745 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T07:24:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T07:24:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T07:24:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T07:24:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T07:24:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T07:24:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T07:24:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T07:24:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T07:24:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T07:24:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T07:24:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T07:24:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T07:24:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:24:02.996 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-09-10T07:24:03.170 DEBUG:teuthology.parallel:result is None 2024-09-10T07:24:03.399 DEBUG:teuthology.parallel:result is None 2024-09-10T07:24:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T07:24:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T07:24:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T07:24:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T07:24:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T07:24:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T07:24:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T07:24:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T07:24:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T07:24:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T07:24:05.624 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:24:05.671 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:24:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:24:07.079 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T07:24:07.186 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T07:24:07.240 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T07:24:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T07:24:07.404 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T07:24:07.430 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T07:24:07.467 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T07:24:07.467 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:24:07.566 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:24:07.566 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:24:07.658 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:24:07.712 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T07:24:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:24:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:24:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:24:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:24:08.078 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:24:08.154 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:24:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:24:08.367 INFO:teuthology.orchestra.run.smithi046.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T07:24:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:24:13.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T07:24:13.857 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T07:24:13.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T07:24:13.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T07:24:13.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T07:24:13.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T07:24:13.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T07:24:13.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T07:24:13.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T07:24:13.871 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T07:24:13.871 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T07:24:13.871 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T07:24:13.871 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T07:24:13.871 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T07:24:13.871 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T07:24:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T07:24:13.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T07:24:13.874 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T07:24:13.875 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T07:24:13.876 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T07:24:13.877 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T07:24:13.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T07:24:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T07:24:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T07:24:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T07:24:13.882 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T07:24:13.882 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T07:24:13.882 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T07:24:13.882 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T07:24:13.882 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T07:24:13.882 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T07:24:13.882 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T07:24:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:24:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:24:16.241 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-09-10T07:24:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T07:24:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T07:24:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T07:24:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T07:24:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T07:24:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T07:24:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T07:24:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T07:24:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T07:24:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:24:16.253 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T07:24:16.887 DEBUG:teuthology.parallel:result is None 2024-09-10T07:24:16.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:24:17.038 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T07:24:17.084 INFO:teuthology.orchestra.run.smithi046.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T07:24:17.084 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T07:24:17.084 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T07:24:17.086 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:24:17.244 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T07:24:17.270 INFO:teuthology.orchestra.run.smithi063.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T07:24:17.270 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T07:24:17.270 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T07:24:17.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:24:17.414 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T07:24:17.441 INFO:teuthology.orchestra.run.smithi181.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T07:24:17.441 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T07:24:17.441 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T07:24:17.443 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T07:24:17.444 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:24:17.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T07:24:17.504 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:24:17.504 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T07:24:17.535 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:24:17.535 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T07:24:17.568 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T07:24:17.568 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:24:17.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T07:24:17.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T07:24:17.722 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:24:17.722 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T07:24:17.753 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T07:24:17.822 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:24:17.822 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T07:24:17.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T07:24:17.921 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T07:24:17.922 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:24:17.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T07:24:17.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T07:24:18.075 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:24:18.075 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T07:24:18.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T07:24:18.166 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:24:18.166 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T07:24:18.190 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T07:24:18.256 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T07:24:18.256 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:24:18.257 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T07:24:18.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T07:24:18.411 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:24:18.411 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T07:24:18.436 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T07:24:18.501 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:24:18.501 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T07:24:18.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T07:24:18.590 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T07:24:18.690 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T07:24:18.690 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:24:18.691 INFO:tasks.cephadm:Cluster fsid is b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:24:18.691 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T07:24:18.691 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.46', 'mon.b': '172.21.15.63', 'mon.c': '172.21.15.181'} 2024-09-10T07:24:18.691 INFO:tasks.cephadm:First mon is mon.a on smithi046 2024-09-10T07:24:18.692 INFO:tasks.cephadm:First mgr is x 2024-09-10T07:24:18.692 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T07:24:18.692 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-09-10T07:24:18.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname $(hostname -s) 2024-09-10T07:24:18.775 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2024-09-10T07:24:18.800 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T07:24:18.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:24:18.945 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T07:24:19.081 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:24:19.082 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T07:24:19.083 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T07:24:19.083 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:19.515 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 07:24 /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:19.515 DEBUG:teuthology.orchestra.run.smithi063:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:19.895 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 07:24 /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:19.895 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:20.536 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 07:24 /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:20.537 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:20.575 DEBUG:teuthology.orchestra.run.smithi063:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:20.594 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T07:24:20.628 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T07:24:20.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T07:24:20.632 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T07:24:20.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T07:24:22.254 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:24:22.283 INFO:teuthology.orchestra.run.smithi063.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:24:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:24:56.043 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:24:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T07:24:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T07:24:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-09-10T07:24:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T07:24:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:24:56.044 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:24:56.781 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T07:24:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T07:24:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T07:24:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2024-09-10T07:24:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T07:24:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-10T07:24:56.782 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T07:25:07.843 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-10T07:25:07.844 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T07:25:07.844 INFO:teuthology.orchestra.run.smithi063.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T07:25:07.844 INFO:teuthology.orchestra.run.smithi063.stdout: "repo_digests": [ 2024-09-10T07:25:07.844 INFO:teuthology.orchestra.run.smithi063.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T07:25:07.844 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-10T07:25:07.844 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-10T07:25:07.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-09-10T07:25:07.935 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph 2024-09-10T07:25:07.962 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2024-09-10T07:25:07.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-09-10T07:25:08.053 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /etc/ceph 2024-09-10T07:25:08.077 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2024-09-10T07:25:08.106 INFO:tasks.cephadm:Writing seed config... 2024-09-10T07:25:08.108 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T07:25:08.108 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T07:25:08.108 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T07:25:08.108 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T07:25:08.108 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T07:25:08.108 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T07:25:08.109 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T07:25:08.110 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T07:25:08.111 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T07:25:08.112 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:25:08.113 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T07:25:08.148 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b120bdf2-6f45-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T07:25:08.148 DEBUG:teuthology.orchestra.run.smithi046:mon.a> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T07:25:08.192 DEBUG:teuthology.orchestra.run.smithi046:mgr.x> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T07:25:08.238 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T07:25:08.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:25:08.504 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', 'b120bdf2-6f45-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-09-10T07:25:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T07:25:08.505 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-09-10T07:25:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.2.2 2024-09-10T07:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-09-10T07:25:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-09-10T07:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T07:25:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T07:25:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T07:25:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T07:25:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-09-10T07:25:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-09-10T07:25:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-09-10T07:25:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-09-10T07:25:08.648 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.2.2 2024-09-10T07:25:08.648 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T07:25:08.648 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-09-10T07:25:08.648 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-09-10T07:25:08.661 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T07:25:08.661 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T07:25:08.673 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T07:25:08.676 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T07:25:08.690 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-09-10T07:25:08.702 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-09-10T07:25:08.703 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-09-10T07:25:08.703 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-09-10T07:25:08.703 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:08.703 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 140429755356016 on /run/cephadm/b120bdf2-6f45-11ef-bcea-c7b262605968.lock 2024-09-10T07:25:08.703 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 140429755356016 acquired on /run/cephadm/b120bdf2-6f45-11ef-bcea-c7b262605968.lock 2024-09-10T07:25:08.703 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-09-10T07:25:08.704 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-09-10T07:25:08.704 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-10T07:25:08.711 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.46 metric 100 2024-09-10T07:25:08.711 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-09-10T07:25:08.717 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T07:25:08.717 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T07:25:08.717 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-09-10T07:25:08.723 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T07:25:08.723 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T07:25:08.723 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T07:25:08.723 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T07:25:08.723 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-10T07:25:08.723 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T07:25:08.724 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-09-10T07:25:08.724 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-09-10T07:25:08.724 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T07:25:08.724 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T07:25:08.725 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:25:09.377 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T07:25:09.377 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:25:09.377 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T07:25:09.377 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T07:25:09.377 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T07:25:09.377 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T07:25:09.377 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T07:25:10.527 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T07:25:10.527 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T07:25:10.527 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T07:25:11.603 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-09-10T07:25:11.603 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-09-10T07:25:12.676 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQBY9N9mB3knAhAAjasJ8Ycno+8vWVvlfztI+w== 2024-09-10T07:25:13.758 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQBZ9N9mAKtqBBAArJN/+ZsWuCh+kVb9UNnVAg== 2024-09-10T07:25:15.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQBa9N9mHcFCCxAAHFelzZ//WIyB9kQP494ofg== 2024-09-10T07:25:15.374 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-09-10T07:25:16.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for a [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:25:16.503 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-09-10T07:25:17.558 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.a on 2024-09-10T07:25:17.858 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T07:25:18.080 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T07:25:18.338 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968.target. 2024-09-10T07:25:18.339 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968.target. 2024-09-10T07:25:18.679 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a 2024-09-10T07:25:18.679 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T07:25:18.953 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968.target.wants/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@.service. 2024-09-10T07:25:19.599 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:19 smithi046 podman[30327]: 2024-09-10 07:25:19.537768501 +0000 UTC m=+0.357627388 container start 7f7c830cac0134b722cafa91158c39f3209600eddd9fa71f345668d4f14735e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T07:25:19.632 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T07:25:19.632 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-10T07:25:19.645 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T07:25:19.645 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T07:25:19.645 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-10T07:25:19.645 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T07:25:19.646 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-09-10T07:25:19.646 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-09-10T07:25:19.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:19 smithi046 ceph-mon[30346]: mkfs b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:19.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:19 smithi046 ceph-mon[30346]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T07:25:19.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:19 smithi046 bash[30327]: 7f7c830cac0134b722cafa91158c39f3209600eddd9fa71f345668d4f14735e1 2024-09-10T07:25:19.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:19 smithi046 systemd[1]: Started Ceph mon.a for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:25:20.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T07:25:20.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: monmap epoch 1 2024-09-10T07:25:20.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:20.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: last_changed 2024-09-10T07:25:15.786098+0000 2024-09-10T07:25:20.854 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:25:20.854 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: min_mon_release 19 (squid) 2024-09-10T07:25:20.854 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: election_strategy: 1 2024-09-10T07:25:20.854 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:25:20.854 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: fsmap 2024-09-10T07:25:20.854 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T07:25:20.854 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: mgrmap e1: no daemons active 2024-09-10T07:25:20.854 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:20 smithi046 ceph-mon[30346]: from='client.? 172.21.15.46:0/2425262670' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T07:25:20.864 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T07:25:20.864 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:20.864 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T07:25:20.864 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.600201s) 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:20.865 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.117+0000 7fc2cb400640 1 Processor -- start 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.117+0000 7fc2cb400640 1 -- start start 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.118+0000 7fc2cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c4107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.118+0000 7fc2cb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4107f50 con 0x7fc2c4107580 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.118+0000 7fc2c8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c4107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.119+0000 7fc2c8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c4107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42388/0 (socket says 172.21.15.46:42388) 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.119+0000 7fc2c8a00640 1 -- 172.21.15.46:0/2587744587 learned_addr learned my addr 172.21.15.46:0/2587744587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.119+0000 7fc2c8a00640 1 -- 172.21.15.46:0/2587744587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c4108780 con 0x7fc2c4107580 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.122+0000 7fc2c8a00640 1 --2- 172.21.15.46:0/2587744587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c4107980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009820 tx=0x7fc2b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496ff4b68497722e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.122+0000 7fc2c3400640 1 -- 172.21.15.46:0/2587744587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2b8037070 con 0x7fc2c4107580 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.122+0000 7fc2c3400640 1 -- 172.21.15.46:0/2587744587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b8032440 con 0x7fc2c4107580 2024-09-10T07:25:20.866 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.122+0000 7fc2c3400640 1 -- 172.21.15.46:0/2587744587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2b8032740 con 0x7fc2c4107580 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.122+0000 7fc2cb400640 1 -- 172.21.15.46:0/2587744587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 msgr2=0x7fc2c4107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.122+0000 7fc2cb400640 1 --2- 172.21.15.46:0/2587744587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c4107980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009820 tx=0x7fc2b802a270 comp rx=0 tx=0).stop 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.123+0000 7fc2cb400640 1 -- 172.21.15.46:0/2587744587 shutdown_connections 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.123+0000 7fc2cb400640 1 --2- 172.21.15.46:0/2587744587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c4107980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.123+0000 7fc2cb400640 1 -- 172.21.15.46:0/2587744587 >> 172.21.15.46:0/2587744587 conn(0x7fc2c4102d90 msgr2=0x7fc2c41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.123+0000 7fc2cb400640 1 -- 172.21.15.46:0/2587744587 shutdown_connections 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.123+0000 7fc2cb400640 1 -- 172.21.15.46:0/2587744587 wait complete. 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.124+0000 7fc2cb400640 1 Processor -- start 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.124+0000 7fc2cb400640 1 -- start start 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.124+0000 7fc2cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c41a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.124+0000 7fc2cb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4108d20 con 0x7fc2c4107580 2024-09-10T07:25:20.867 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.124+0000 7fc2c8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c41a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.124+0000 7fc2c8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c41a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42400/0 (socket says 172.21.15.46:42400) 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.124+0000 7fc2c8a00640 1 -- 172.21.15.46:0/2425262670 learned_addr learned my addr 172.21.15.46:0/2425262670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.125+0000 7fc2c8a00640 1 -- 172.21.15.46:0/2425262670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c41a1bc0 con 0x7fc2c4107580 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.125+0000 7fc2c8a00640 1 --2- 172.21.15.46:0/2425262670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c41a1680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80090b0 tx=0x7fc2b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.125+0000 7fc2c1600640 1 -- 172.21.15.46:0/2425262670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2b8041070 con 0x7fc2c4107580 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.125+0000 7fc2c1600640 1 -- 172.21.15.46:0/2425262670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b803c7a0 con 0x7fc2c4107580 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.125+0000 7fc2c1600640 1 -- 172.21.15.46:0/2425262670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2b8037040 con 0x7fc2c4107580 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.125+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c41a1e50 con 0x7fc2c4107580 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.125+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c41a4b40 con 0x7fc2c4107580 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.126+0000 7fc2c1600640 1 -- 172.21.15.46:0/2425262670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc2b8044070 con 0x7fc2c4107580 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.126+0000 7fc2c1600640 1 -- 172.21.15.46:0/2425262670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2b803d3f0 con 0x7fc2c4107580 2024-09-10T07:25:20.868 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.127+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c4109370 con 0x7fc2c4107580 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.130+0000 7fc2c1600640 1 -- 172.21.15.46:0/2425262670 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc2b8046450 con 0x7fc2c4107580 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.194+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc2c4075f30 con 0x7fc2c4107580 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.195+0000 7fc2c1600640 1 -- 172.21.15.46:0/2425262670 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc2c4075f30 con 0x7fc2c4107580 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.196+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 msgr2=0x7fc2c41a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.196+0000 7fc2cb400640 1 --2- 172.21.15.46:0/2425262670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c41a1680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80090b0 tx=0x7fc2b80046c0 comp rx=0 tx=0).stop 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.196+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 shutdown_connections 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.196+0000 7fc2cb400640 1 --2- 172.21.15.46:0/2425262670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c4107580 0x7fc2c41a1680 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.196+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 >> 172.21.15.46:0/2425262670 conn(0x7fc2c4102d90 msgr2=0x7fc2c418fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.196+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 shutdown_connections 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:20.196+0000 7fc2cb400640 1 -- 172.21.15.46:0/2425262670 wait complete. 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-09-10T07:25:20.869 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T07:25:21.757 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:21 smithi046 ceph-mon[30346]: from='client.? 172.21.15.46:0/900125825' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T07:25:21.757 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:21 smithi046 ceph-mon[30346]: from='client.? 172.21.15.46:0/900125825' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T07:25:22.012 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:22.012 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T07:25:22.012 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T07:25:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.348+0000 7f42a5e00640 1 Processor -- start 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.348+0000 7f42a5e00640 1 -- start start 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.348+0000 7f42a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a0107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.348+0000 7f42a5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42a0107f30 con 0x7f42a0107560 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.349+0000 7f429ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a0107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.349+0000 7f429ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a0107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42412/0 (socket says 172.21.15.46:42412) 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.349+0000 7f429ea00640 1 -- 172.21.15.46:0/323829989 learned_addr learned my addr 172.21.15.46:0/323829989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.349+0000 7f429ea00640 1 -- 172.21.15.46:0/323829989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a0108760 con 0x7f42a0107560 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.350+0000 7f429ea00640 1 --2- 172.21.15.46:0/323829989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a0107960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4290009820 tx=0x7f429002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311eb3a968d0612e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.351+0000 7f429d600640 1 -- 172.21.15.46:0/323829989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4290037070 con 0x7f42a0107560 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.351+0000 7f429d600640 1 -- 172.21.15.46:0/323829989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4290032440 con 0x7f42a0107560 2024-09-10T07:25:22.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.351+0000 7f429d600640 1 -- 172.21.15.46:0/323829989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4290032740 con 0x7f42a0107560 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.351+0000 7f42a5e00640 1 -- 172.21.15.46:0/323829989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 msgr2=0x7f42a0107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.351+0000 7f42a5e00640 1 --2- 172.21.15.46:0/323829989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a0107960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4290009820 tx=0x7f429002a270 comp rx=0 tx=0).stop 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.351+0000 7f42a5e00640 1 -- 172.21.15.46:0/323829989 shutdown_connections 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.351+0000 7f42a5e00640 1 --2- 172.21.15.46:0/323829989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a0107960 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.351+0000 7f42a5e00640 1 -- 172.21.15.46:0/323829989 >> 172.21.15.46:0/323829989 conn(0x7f42a0102cf0 msgr2=0x7f42a0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.352+0000 7f42a5e00640 1 -- 172.21.15.46:0/323829989 shutdown_connections 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.352+0000 7f42a5e00640 1 -- 172.21.15.46:0/323829989 wait complete. 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.352+0000 7f42a5e00640 1 Processor -- start 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.353+0000 7f42a5e00640 1 -- start start 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.353+0000 7f42a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a007db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.353+0000 7f42a5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42a0108d00 con 0x7f42a0107560 2024-09-10T07:25:22.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.353+0000 7f429ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a007db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.353+0000 7f429ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a007db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42420/0 (socket says 172.21.15.46:42420) 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.353+0000 7f429ea00640 1 -- 172.21.15.46:0/900125825 learned_addr learned my addr 172.21.15.46:0/900125825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.353+0000 7f429ea00640 1 -- 172.21.15.46:0/900125825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a007e060 con 0x7f42a0107560 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.354+0000 7f429ea00640 1 --2- 172.21.15.46:0/900125825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a007db20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f42900090b0 tx=0x7f42900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.354+0000 7f42a4a00640 1 -- 172.21.15.46:0/900125825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4290041070 con 0x7f42a0107560 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.354+0000 7f42a4a00640 1 -- 172.21.15.46:0/900125825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429003c7a0 con 0x7f42a0107560 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.354+0000 7f42a4a00640 1 -- 172.21.15.46:0/900125825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4290037040 con 0x7f42a0107560 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.354+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a007c250 con 0x7f42a0107560 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.354+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a007c790 con 0x7f42a0107560 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.355+0000 7f42a4a00640 1 -- 172.21.15.46:0/900125825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4290044070 con 0x7f42a0107560 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.355+0000 7f42a4a00640 1 -- 172.21.15.46:0/900125825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f429003d3f0 con 0x7f42a0107560 2024-09-10T07:25:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.355+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a0109350 con 0x7f42a0107560 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.358+0000 7f42a4a00640 1 -- 172.21.15.46:0/900125825 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4290046450 con 0x7f42a0107560 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.420+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f42a01943b0 con 0x7f42a0107560 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.424+0000 7f42a4a00640 1 -- 172.21.15.46:0/900125825 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f4290046630 con 0x7f42a0107560 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.425+0000 7f42a4a00640 1 -- 172.21.15.46:0/900125825 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42900584b0 con 0x7f42a0107560 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.426+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 msgr2=0x7f42a007db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.426+0000 7f42a5e00640 1 --2- 172.21.15.46:0/900125825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a007db20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f42900090b0 tx=0x7f42900046c0 comp rx=0 tx=0).stop 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.426+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 shutdown_connections 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.426+0000 7f42a5e00640 1 --2- 172.21.15.46:0/900125825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a0107560 0x7f42a007db20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.426+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 >> 172.21.15.46:0/900125825 conn(0x7f42a0102cf0 msgr2=0x7f42a0191a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.426+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 shutdown_connections 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:21.426+0000 7f42a5e00640 1 -- 172.21.15.46:0/900125825 wait complete. 2024-09-10T07:25:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-09-10T07:25:22.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:22 smithi046 ceph-mon[30346]: from='client.? 172.21.15.46:0/2918053502' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:25:23.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.503+0000 7f5c0f200640 1 Processor -- start 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.504+0000 7f5c0f200640 1 -- start start 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.504+0000 7f5c0f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c10109c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.504+0000 7f5c0f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010a1f0 con 0x7f5c10109820 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.504+0000 7f5c0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c10109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.504+0000 7f5c0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c10109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42428/0 (socket says 172.21.15.46:42428) 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.504+0000 7f5c0de00640 1 -- 172.21.15.46:0/118130964 learned_addr learned my addr 172.21.15.46:0/118130964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.504+0000 7f5c0de00640 1 -- 172.21.15.46:0/118130964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1010a370 con 0x7f5c10109820 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.505+0000 7f5c0de00640 1 --2- 172.21.15.46:0/118130964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c10109c20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009820 tx=0x7f5c0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728bd487aa2e8368 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0d400640 1 -- 172.21.15.46:0/118130964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c00037070 con 0x7f5c10109820 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0d400640 1 -- 172.21.15.46:0/118130964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5c00032900 con 0x7f5c10109820 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0d400640 1 -- 172.21.15.46:0/118130964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c00030850 con 0x7f5c10109820 2024-09-10T07:25:23.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0f200640 1 -- 172.21.15.46:0/118130964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 msgr2=0x7f5c10109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0f200640 1 --2- 172.21.15.46:0/118130964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c10109c20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009820 tx=0x7f5c0002a270 comp rx=0 tx=0).stop 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0f200640 1 -- 172.21.15.46:0/118130964 shutdown_connections 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0f200640 1 --2- 172.21.15.46:0/118130964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c10109c20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0f200640 1 -- 172.21.15.46:0/118130964 >> 172.21.15.46:0/118130964 conn(0x7f5c1007cba0 msgr2=0x7f5c1007cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0f200640 1 -- 172.21.15.46:0/118130964 shutdown_connections 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.506+0000 7f5c0f200640 1 -- 172.21.15.46:0/118130964 wait complete. 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.507+0000 7f5c0f200640 1 Processor -- start 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.507+0000 7f5c0f200640 1 -- start start 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.508+0000 7f5c0f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c1019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:23.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.508+0000 7f5c0f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010da50 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.508+0000 7f5c0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c1019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.508+0000 7f5c0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c1019d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42444/0 (socket says 172.21.15.46:42444) 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.508+0000 7f5c0de00640 1 -- 172.21.15.46:0/2918053502 learned_addr learned my addr 172.21.15.46:0/2918053502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.508+0000 7f5c0de00640 1 -- 172.21.15.46:0/2918053502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1019dd80 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.509+0000 7f5c0de00640 1 --2- 172.21.15.46:0/2918053502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c1019d840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097f0 tx=0x7f5c00030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.509+0000 7f5c06a00640 1 -- 172.21.15.46:0/2918053502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c00037040 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.509+0000 7f5c06a00640 1 -- 172.21.15.46:0/2918053502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5c00040070 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.509+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1019e010 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.509+0000 7f5c06a00640 1 -- 172.21.15.46:0/2918053502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0003a390 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.509+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c101a0d00 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.509+0000 7f5c06a00640 1 -- 172.21.15.46:0/2918053502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c0003a5e0 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.509+0000 7f5c06a00640 1 -- 172.21.15.46:0/2918053502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5c00045950 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.510+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c10109ca0 con 0x7f5c10109820 2024-09-10T07:25:23.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.513+0000 7f5c06a00640 1 -- 172.21.15.46:0/2918053502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5c0003b430 con 0x7f5c10109820 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.575+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f5c101973b0 con 0x7f5c10109820 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.576+0000 7f5c06a00640 1 -- 172.21.15.46:0/2918053502 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f5c0003b7c0 con 0x7f5c10109820 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.577+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 msgr2=0x7f5c1019d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.577+0000 7f5c0f200640 1 --2- 172.21.15.46:0/2918053502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c1019d840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097f0 tx=0x7f5c00030d80 comp rx=0 tx=0).stop 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.577+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 shutdown_connections 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.577+0000 7f5c0f200640 1 --2- 172.21.15.46:0/2918053502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c10109820 0x7f5c1019d840 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.577+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 >> 172.21.15.46:0/2918053502 conn(0x7f5c1007cba0 msgr2=0x7f5c101075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.577+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 shutdown_connections 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:22.577+0000 7f5c0f200640 1 -- 172.21.15.46:0/2918053502 wait complete. 2024-09-10T07:25:23.195 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-09-10T07:25:23.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:23 smithi046 systemd[1]: Stopping Ceph mon.a for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:25:23.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:23 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a[30342]: 2024-09-10T07:25:23.538+0000 7f34b4c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:25:23.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:23 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a[30342]: 2024-09-10T07:25:23.538+0000 7f34b4c00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T07:25:23.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:23 smithi046 podman[30584]: 2024-09-10 07:25:23.80614967 +0000 UTC m=+0.386108224 container died 7f7c830cac0134b722cafa91158c39f3209600eddd9fa71f345668d4f14735e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T07:25:24.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 podman[30584]: 2024-09-10 07:25:24.168290321 +0000 UTC m=+0.748248873 container cleanup 7f7c830cac0134b722cafa91158c39f3209600eddd9fa71f345668d4f14735e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T07:25:24.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 bash[30584]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a 2024-09-10T07:25:24.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 podman[30596]: 2024-09-10 07:25:24.385857862 +0000 UTC m=+0.566608054 container remove 7f7c830cac0134b722cafa91158c39f3209600eddd9fa71f345668d4f14735e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T07:25:24.961 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T07:25:24.961 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 systemd[1]: Stopped Ceph mon.a for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:25:24.961 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.217s CPU time. 2024-09-10T07:25:24.961 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 systemd[1]: Starting Ceph mon.a for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:25:25.214 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 podman[30686]: 2024-09-10 07:25:24.961743933 +0000 UTC m=+0.109722618 container create 735e6e66d0038d4e4e9ba09118b2733a8892bf9c2fead7ab272e4351cd046b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T07:25:25.214 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:24 smithi046 podman[30686]: 2024-09-10 07:25:24.878978255 +0000 UTC m=+0.026956960 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:25:25.214 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 podman[30686]: 2024-09-10 07:25:25.214280963 +0000 UTC m=+0.362259649 container init 735e6e66d0038d4e4e9ba09118b2733a8892bf9c2fead7ab272e4351cd046b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:25:25.305 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T07:25:25.489 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 podman[30686]: 2024-09-10 07:25:25.221389567 +0000 UTC m=+0.369368253 container start 735e6e66d0038d4e4e9ba09118b2733a8892bf9c2fead7ab272e4351cd046b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T07:25:25.489 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T07:25:25.489 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: pidfile_write: ignore empty --pid-file 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: load: jerasure load: lrc 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: RocksDB version: 7.9.2 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Git sha 0 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: DB SUMMARY 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: DB Session ID: 104C23AWT211WZYG8PZ7 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: CURRENT file: CURRENT 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104824 ; 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.error_if_exists: 0 2024-09-10T07:25:25.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.create_if_missing: 0 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.paranoid_checks: 1 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.env: 0x562dd9162940 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.info_log: 0x562e008096a0 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.statistics: (nil) 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.use_fsync: 0 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_log_file_size: 0 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T07:25:25.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T07:25:25.492 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T07:25:25.492 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.allow_fallocate: 1 2024-09-10T07:25:25.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T07:25:25.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T07:25:25.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.use_direct_reads: 0 2024-09-10T07:25:25.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T07:25:25.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T07:25:25.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.db_log_dir: 2024-09-10T07:25:25.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.wal_dir: 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.write_buffer_manager: 0x562e00818dc0 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T07:25:25.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.unordered_write: 0 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.row_cache: None 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.wal_filter: None 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T07:25:25.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.two_write_queues: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.wal_compression: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.atomic_flush: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.log_readahead_size: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T07:25:25.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_background_jobs: 2 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_background_compactions: -1 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_subcompactions: 1 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_open_files: -1 2024-09-10T07:25:25.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_background_flushes: -1 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Compression algorithms supported: 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: kZSTD supported: 0 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: kXpressCompression supported: 0 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: kBZip2Compression supported: 0 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: kLZ4Compression supported: 1 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: kZlibCompression supported: 1 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: kSnappyCompression supported: 1 2024-09-10T07:25:25.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.merge_operator: 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_filter: None 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e00809300) 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T07:25:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_type: 0 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_index_type: 0 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_shortening: 1 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: checksum: 4 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: no_block_cache: 0 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache: 0x562e007fd1f0 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_options: 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: capacity : 536870912 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_shard_bits : 4 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: strict_capacity_limit : 0 2024-09-10T07:25:25.500 INFO:journalctl@ceph.mon.a.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_compressed: (nil) 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: persistent_cache: (nil) 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size: 4096 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size_deviation: 10 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_restart_interval: 16 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_block_restart_interval: 1 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: metadata_block_size: 4096 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: partition_filters: 0 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: use_delta_encoding: 1 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: filter_policy: bloomfilter 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: whole_key_filtering: 1 2024-09-10T07:25:25.501 INFO:journalctl@ceph.mon.a.smithi046.stdout: verify_compression: 0 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout: format_version: 5 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_index_compression: 1 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_align: 0 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout: max_auto_readahead_size: 262144 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout: prepopulate_block_cache: 0 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout: initial_auto_readahead_size: 8192 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression: NoCompression 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T07:25:25.502 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.num_levels: 7 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T07:25:25.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T07:25:25.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T07:25:25.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T07:25:25.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.inplace_update_support: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.bloom_locality: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.max_successive_merges: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.ttl: 2592000 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T07:25:25.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.enable_blob_files: false 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.min_blob_size: 0 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: edeb33ac-0520-488e-8b2f-c1d38c40c23d 2024-09-10T07:25:25.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725953125264125, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725953125265626, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99928, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97814, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14179, "raw_average_key_size": 49, "raw_value_size": 90150, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725953125, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "edeb33ac-0520-488e-8b2f-c1d38c40c23d", "db_session_id": "104C23AWT211WZYG8PZ7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725953125265852, "job": 1, "event": "recovery_finished"} 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e0082ce00 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: DB pointer 0x562e00928000 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** DB Stats ** 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T07:25:25.509 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 99.45 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 108.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 99.45 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 108.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 108.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T07:25:25.510 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 108.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T07:25:25.511 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 16.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 16.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache BinnedLRUCache@0x562e007fd1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: starting mon.a rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a@-1(???) e1 preinit fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a@-1(???).mds e1 new map 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a@-1(???).mds e1 print_map 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: e1 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: btime 2024-09-10T07:25:19:596582+0000 2024-09-10T07:25:25.512 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout: legacy client fscid: -1 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout: No filesystems configured 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 bash[30686]: 735e6e66d0038d4e4e9ba09118b2733a8892bf9c2fead7ab272e4351cd046b03 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 systemd[1]: Started Ceph mon.a for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: monmap epoch 1 2024-09-10T07:25:25.513 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:25.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: last_changed 2024-09-10T07:25:15.786098+0000 2024-09-10T07:25:25.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:25:25.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: min_mon_release 19 (squid) 2024-09-10T07:25:25.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: election_strategy: 1 2024-09-10T07:25:25.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:25:25.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: fsmap 2024-09-10T07:25:25.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T07:25:25.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:25 smithi046 ceph-mon[30705]: mgrmap e1: no daemons active 2024-09-10T07:25:26.696 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.932+0000 7f70ed000640 1 Processor -- start 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.932+0000 7f70ed000640 1 -- start start 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.932+0000 7f70ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e8109be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.932+0000 7f70ed000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70e810a1b0 con 0x7f70e81097e0 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.933+0000 7f70e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e8109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.933+0000 7f70e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e8109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53482/0 (socket says 172.21.15.46:53482) 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.933+0000 7f70e5e00640 1 -- 172.21.15.46:0/1055657257 learned_addr learned my addr 172.21.15.46:0/1055657257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.933+0000 7f70e5e00640 1 -- 172.21.15.46:0/1055657257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e810a990 con 0x7f70e81097e0 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.936+0000 7f70e5e00640 1 --2- 172.21.15.46:0/1055657257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e8109be0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009820 tx=0x7f70d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5c966ad1bd93ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.937+0000 7f70e4a00640 1 -- 172.21.15.46:0/1055657257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d8037070 con 0x7f70e81097e0 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.937+0000 7f70e4a00640 1 -- 172.21.15.46:0/1055657257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f70d8032900 con 0x7f70e81097e0 2024-09-10T07:25:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.937+0000 7f70e4a00640 1 -- 172.21.15.46:0/1055657257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d8030850 con 0x7f70e81097e0 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.937+0000 7f70ed000640 1 -- 172.21.15.46:0/1055657257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 msgr2=0x7f70e8109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.937+0000 7f70ed000640 1 --2- 172.21.15.46:0/1055657257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e8109be0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009820 tx=0x7f70d802a270 comp rx=0 tx=0).stop 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.938+0000 7f70ed000640 1 -- 172.21.15.46:0/1055657257 shutdown_connections 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.938+0000 7f70ed000640 1 --2- 172.21.15.46:0/1055657257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e8109be0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.938+0000 7f70ed000640 1 -- 172.21.15.46:0/1055657257 >> 172.21.15.46:0/1055657257 conn(0x7f70e807cdf0 msgr2=0x7f70e807d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.938+0000 7f70ed000640 1 -- 172.21.15.46:0/1055657257 shutdown_connections 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.938+0000 7f70ed000640 1 -- 172.21.15.46:0/1055657257 wait complete. 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.939+0000 7f70ed000640 1 Processor -- start 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.939+0000 7f70ed000640 1 -- start start 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.939+0000 7f70ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.939+0000 7f70ed000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70e810dd40 con 0x7f70e81097e0 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.939+0000 7f70e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:26.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.939+0000 7f70e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e819dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53490/0 (socket says 172.21.15.46:53490) 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.939+0000 7f70e5e00640 1 -- 172.21.15.46:0/2629669941 learned_addr learned my addr 172.21.15.46:0/2629669941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.940+0000 7f70e5e00640 1 -- 172.21.15.46:0/2629669941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e819e020 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.940+0000 7f70e5e00640 1 --2- 172.21.15.46:0/2629669941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e819dae0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009940 tx=0x7f70d8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.940+0000 7f70dea00640 1 -- 172.21.15.46:0/2629669941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d8042070 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.940+0000 7f70dea00640 1 -- 172.21.15.46:0/2629669941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f70d80308c0 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.940+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e819e2b0 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.940+0000 7f70dea00640 1 -- 172.21.15.46:0/2629669941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d8037040 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.941+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81a0fa0 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.941+0000 7f70dea00640 1 -- 172.21.15.46:0/2629669941 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f70d8045070 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.941+0000 7f70dea00640 1 -- 172.21.15.46:0/2629669941 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f70d8047b70 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.942+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e8109be0 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:25.945+0000 7f70dea00640 1 -- 172.21.15.46:0/2629669941 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f70d80327b0 con 0x7f70e81097e0 2024-09-10T07:25:26.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.008+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f70e81976f0 con 0x7f70e81097e0 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.011+0000 7f70dea00640 1 -- 172.21.15.46:0/2629669941 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f70d803c830 con 0x7f70e81097e0 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.012+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 msgr2=0x7f70e819dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.012+0000 7f70ed000640 1 --2- 172.21.15.46:0/2629669941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e819dae0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009940 tx=0x7f70d8032ce0 comp rx=0 tx=0).stop 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.012+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 shutdown_connections 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.012+0000 7f70ed000640 1 --2- 172.21.15.46:0/2629669941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e81097e0 0x7f70e819dae0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.012+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 >> 172.21.15.46:0/2629669941 conn(0x7f70e807cdf0 msgr2=0x7f70e81075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.012+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 shutdown_connections 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:26.013+0000 7f70ed000640 1 -- 172.21.15.46:0/2629669941 wait complete. 2024-09-10T07:25:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T07:25:26.720 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:25:26.720 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-09-10T07:25:26.721 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T07:25:26.721 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T07:25:27.075 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.x 2024-09-10T07:25:27.075 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T07:25:27.310 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968.target.wants/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@.service. 2024-09-10T07:25:27.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:27 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2629669941' entity='client.admin' 2024-09-10T07:25:27.656 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:27 smithi046 systemd[1]: Starting Ceph mgr.x for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:25:27.908 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:27 smithi046 podman[30924]: 2024-09-10 07:25:27.560473551 +0000 UTC m=+0.027283795 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:25:27.908 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:27 smithi046 podman[30924]: 2024-09-10 07:25:27.664834191 +0000 UTC m=+0.131644417 container create 500c990df560c6dc580deb272da0dd8787604db33b2d0600753b9bf8b5e03698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T07:25:27.909 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:27 smithi046 podman[30924]: 2024-09-10 07:25:27.909096771 +0000 UTC m=+0.375907024 container init 500c990df560c6dc580deb272da0dd8787604db33b2d0600753b9bf8b5e03698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T07:25:28.043 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T07:25:28.043 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-10T07:25:28.057 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T07:25:28.057 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T07:25:28.057 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-10T07:25:28.057 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T07:25:28.074 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T07:25:28.075 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-10T07:25:28.088 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T07:25:28.088 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T07:25:28.088 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-10T07:25:28.089 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T07:25:28.089 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-09-10T07:25:28.089 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-09-10T07:25:28.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:27 smithi046 podman[30924]: 2024-09-10 07:25:27.915381276 +0000 UTC m=+0.382191499 container start 500c990df560c6dc580deb272da0dd8787604db33b2d0600753b9bf8b5e03698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T07:25:28.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:28 smithi046 bash[30924]: 500c990df560c6dc580deb272da0dd8787604db33b2d0600753b9bf8b5e03698 2024-09-10T07:25:28.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:28 smithi046 systemd[1]: Started Ceph mgr.x for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:25:28.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:28.088+0000 7fe548e13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:25:28.448 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:28.163+0000 7fe548e13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:25:28.448 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:28.363+0000 7fe548e13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:25:28.448 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:28.448+0000 7fe548e13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:25:28.732 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:28.635+0000 7fe548e13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:25:29.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:28 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/878871809' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T07:25:29.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:28.813+0000 7fe548e13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:25:29.286 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:29 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:29.211+0000 7fe548e13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:25:29.287 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:29 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:29.286+0000 7fe548e13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:25:29.325 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-10T07:25:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T07:25:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T07:25:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T07:25:19:596582+0000", 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T07:25:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T07:25:19.597537+0000", 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.606+0000 7f184d000640 1 Processor -- start 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.606+0000 7f184d000640 1 -- start start 2024-09-10T07:25:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.606+0000 7f184d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f1848107800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.606+0000 7f184d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1848107dd0 con 0x7f1848107400 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.607+0000 7f1847400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f1848107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.607+0000 7f1847400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f1848107800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53520/0 (socket says 172.21.15.46:53520) 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.607+0000 7f1847400640 1 -- 172.21.15.46:0/2304757875 learned_addr learned my addr 172.21.15.46:0/2304757875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.607+0000 7f1847400640 1 -- 172.21.15.46:0/2304757875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18481085b0 con 0x7f1848107400 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.608+0000 7f1847400640 1 --2- 172.21.15.46:0/2304757875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f1848107800 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1838009820 tx=0x7f183802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4547095f643a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.608+0000 7f1846000640 1 -- 172.21.15.46:0/2304757875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1838037070 con 0x7f1848107400 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.608+0000 7f1846000640 1 -- 172.21.15.46:0/2304757875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1838032900 con 0x7f1848107400 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.608+0000 7f1846000640 1 -- 172.21.15.46:0/2304757875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1838030850 con 0x7f1848107400 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.609+0000 7f184d000640 1 -- 172.21.15.46:0/2304757875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 msgr2=0x7f1848107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.609+0000 7f184d000640 1 --2- 172.21.15.46:0/2304757875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f1848107800 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1838009820 tx=0x7f183802a270 comp rx=0 tx=0).stop 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.609+0000 7f184d000640 1 -- 172.21.15.46:0/2304757875 shutdown_connections 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.609+0000 7f184d000640 1 --2- 172.21.15.46:0/2304757875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f1848107800 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.609+0000 7f184d000640 1 -- 172.21.15.46:0/2304757875 >> 172.21.15.46:0/2304757875 conn(0x7f1848102bb0 msgr2=0x7f1848104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.609+0000 7f184d000640 1 -- 172.21.15.46:0/2304757875 shutdown_connections 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.609+0000 7f184d000640 1 -- 172.21.15.46:0/2304757875 wait complete. 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.610+0000 7f184d000640 1 Processor -- start 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.610+0000 7f184d000640 1 -- start start 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.610+0000 7f184d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f184807db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.610+0000 7f184d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f184810b960 con 0x7f1848107400 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.611+0000 7f1847400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f184807db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.611+0000 7f1847400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f184807db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53526/0 (socket says 172.21.15.46:53526) 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.611+0000 7f1847400640 1 -- 172.21.15.46:0/878871809 learned_addr learned my addr 172.21.15.46:0/878871809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.611+0000 7f1847400640 1 -- 172.21.15.46:0/878871809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184807e060 con 0x7f1848107400 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.611+0000 7f1847400640 1 --2- 172.21.15.46:0/878871809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f184807db20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f18380097f0 tx=0x7f1838030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.612+0000 7f183fe00640 1 -- 172.21.15.46:0/878871809 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1838037040 con 0x7f1848107400 2024-09-10T07:25:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.612+0000 7f183fe00640 1 -- 172.21.15.46:0/878871809 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1838040070 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.612+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184807c250 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.612+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184807c730 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.612+0000 7f183fe00640 1 -- 172.21.15.46:0/878871809 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f183803a390 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.612+0000 7f183fe00640 1 -- 172.21.15.46:0/878871809 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f183803a660 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.612+0000 7f183fe00640 1 -- 172.21.15.46:0/878871809 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18380459b0 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.613+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18280052c0 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.616+0000 7f183fe00640 1 -- 172.21.15.46:0/878871809 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1838045460 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.681+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1828005880 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.681+0000 7f183fe00640 1 -- 172.21.15.46:0/878871809 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f183803b430 con 0x7f1848107400 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.682+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 msgr2=0x7f184807db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.682+0000 7f184d000640 1 --2- 172.21.15.46:0/878871809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f184807db20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f18380097f0 tx=0x7f1838030d80 comp rx=0 tx=0).stop 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.682+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 shutdown_connections 2024-09-10T07:25:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.682+0000 7f184d000640 1 --2- 172.21.15.46:0/878871809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1848107400 0x7f184807db20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:29.341 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.682+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 >> 172.21.15.46:0/878871809 conn(0x7f1848102bb0 msgr2=0x7f1848103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:29.341 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.683+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 shutdown_connections 2024-09-10T07:25:29.341 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:28.683+0000 7f184d000640 1 -- 172.21.15.46:0/878871809 wait complete. 2024-09-10T07:25:29.341 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-09-10T07:25:29.659 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:29 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:29.511+0000 7fe548e13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:25:31.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:30.672+0000 7fe548e13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:25:31.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:30.756+0000 7fe548e13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:25:32.014 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:31 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:31.756+0000 7fe548e13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:25:32.014 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:31 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:31.905+0000 7fe548e13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:25:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/542273908' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T07:25:32.344 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.086+0000 7fe548e13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:25:32.344 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.163+0000 7fe548e13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:25:32.344 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.343+0000 7fe548e13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:25:32.659 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.430+0000 7fe548e13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:25:32.659 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.506+0000 7fe548e13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:25:32.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:32.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:25:32.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:25:32.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T07:25:32.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T07:25:32.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T07:25:32.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T07:25:32.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T07:25:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T07:25:32.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:32.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T07:25:32.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:32.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T07:25:32.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T07:25:32.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T07:25:32.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T07:25:32.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T07:25:32.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T07:25:32.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:32.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T07:25:32.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T07:25:32.698 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T07:25:19:596582+0000", 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T07:25:32.699 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:25:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T07:25:19.597537+0000", 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.886+0000 7fdf47e00640 1 Processor -- start 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.887+0000 7fdf47e00640 1 -- start start 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.887+0000 7fdf47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf48109d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:32.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.887+0000 7fdf47e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4810a2f0 con 0x7fdf48109920 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.887+0000 7fdf46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf48109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.887+0000 7fdf46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf48109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53528/0 (socket says 172.21.15.46:53528) 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.887+0000 7fdf46a00640 1 -- 172.21.15.46:0/2461668408 learned_addr learned my addr 172.21.15.46:0/2461668408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.888+0000 7fdf46a00640 1 -- 172.21.15.46:0/2461668408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4810a780 con 0x7fdf48109920 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.889+0000 7fdf46a00640 1 --2- 172.21.15.46:0/2461668408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf48109d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009820 tx=0x7fdf3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20de970ee7a4bab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.889+0000 7fdf45600640 1 -- 172.21.15.46:0/2461668408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf38037070 con 0x7fdf48109920 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.889+0000 7fdf45600640 1 -- 172.21.15.46:0/2461668408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf38032900 con 0x7fdf48109920 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.889+0000 7fdf45600640 1 -- 172.21.15.46:0/2461668408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf38030850 con 0x7fdf48109920 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.890+0000 7fdf47e00640 1 -- 172.21.15.46:0/2461668408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 msgr2=0x7fdf48109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:32.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.890+0000 7fdf47e00640 1 --2- 172.21.15.46:0/2461668408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf48109d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009820 tx=0x7fdf3802a270 comp rx=0 tx=0).stop 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.890+0000 7fdf47e00640 1 -- 172.21.15.46:0/2461668408 shutdown_connections 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.890+0000 7fdf47e00640 1 --2- 172.21.15.46:0/2461668408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf48109d20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.890+0000 7fdf47e00640 1 -- 172.21.15.46:0/2461668408 >> 172.21.15.46:0/2461668408 conn(0x7fdf4807cbe0 msgr2=0x7fdf4807cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.890+0000 7fdf47e00640 1 -- 172.21.15.46:0/2461668408 shutdown_connections 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.890+0000 7fdf47e00640 1 -- 172.21.15.46:0/2461668408 wait complete. 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.891+0000 7fdf47e00640 1 Processor -- start 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.891+0000 7fdf47e00640 1 -- start start 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.891+0000 7fdf47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf4819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.892+0000 7fdf47e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4810db30 con 0x7fdf48109920 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.892+0000 7fdf46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf4819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.892+0000 7fdf46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf4819d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53532/0 (socket says 172.21.15.46:53532) 2024-09-10T07:25:32.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.892+0000 7fdf46a00640 1 -- 172.21.15.46:0/542273908 learned_addr learned my addr 172.21.15.46:0/542273908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.892+0000 7fdf46a00640 1 -- 172.21.15.46:0/542273908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4819de10 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.893+0000 7fdf46a00640 1 --2- 172.21.15.46:0/542273908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf4819d8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009940 tx=0x7fdf38032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.893+0000 7fdf3f400640 1 -- 172.21.15.46:0/542273908 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf38042070 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.893+0000 7fdf3f400640 1 -- 172.21.15.46:0/542273908 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf380308c0 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.893+0000 7fdf3f400640 1 -- 172.21.15.46:0/542273908 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf38037040 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.893+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4819e0a0 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.893+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf481a0d90 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.893+0000 7fdf3f400640 1 -- 172.21.15.46:0/542273908 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf38045070 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.894+0000 7fdf3f400640 1 -- 172.21.15.46:0/542273908 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf38047b70 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.894+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf48109da0 con 0x7fdf48109920 2024-09-10T07:25:32.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.897+0000 7fdf3f400640 1 -- 172.21.15.46:0/542273908 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdf380327b0 con 0x7fdf48109920 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.963+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdf4802d7c0 con 0x7fdf48109920 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.963+0000 7fdf3f400640 1 -- 172.21.15.46:0/542273908 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fdf3803c760 con 0x7fdf48109920 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.964+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 msgr2=0x7fdf4819d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.964+0000 7fdf47e00640 1 --2- 172.21.15.46:0/542273908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf4819d8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009940 tx=0x7fdf38032ce0 comp rx=0 tx=0).stop 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.964+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 shutdown_connections 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.964+0000 7fdf47e00640 1 --2- 172.21.15.46:0/542273908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf48109920 0x7fdf4819d8d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.964+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 >> 172.21.15.46:0/542273908 conn(0x7fdf4807cbe0 msgr2=0x7fdf48107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.964+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 shutdown_connections 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:31.964+0000 7fdf47e00640 1 -- 172.21.15.46:0/542273908 wait complete. 2024-09-10T07:25:32.705 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-09-10T07:25:32.964 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.699+0000 7fe548e13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:25:32.964 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.780+0000 7fe548e13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:25:32.964 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.863+0000 7fe548e13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:25:32.964 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:32.964+0000 7fe548e13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:25:33.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:33 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:33.038+0000 7fe548e13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:25:33.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:33 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:33.111+0000 7fe548e13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:25:34.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:34.076+0000 7fe548e13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:25:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: Activating manager daemon x 2024-09-10T07:25:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: mgrmap e2: x(active, starting, since 0.00404492s) 2024-09-10T07:25:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T07:25:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: Manager daemon x is now available 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' 2024-09-10T07:25:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:34 smithi046 ceph-mon[30705]: from='mgr.14100 172.21.15.46:0/1629024746' entity='mgr.x' 2024-09-10T07:25:36.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:36 smithi046 ceph-mon[30705]: mgrmap e3: x(active, since 1.00908s) 2024-09-10T07:25:36.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:36 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/685348249' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T07:25:36.394 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T07:25:36.395 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T07:25:36.397 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:36.397 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T07:25:36.397 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T07:25:36.397 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T07:25:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T07:25:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T07:25:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T07:25:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T07:25:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T07:25:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T07:25:19:596582+0000", 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T07:25:36.407 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:36.408 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T07:25:19.597537+0000", 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.227+0000 7fd5e4a00640 1 Processor -- start 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.227+0000 7fd5e4a00640 1 -- start start 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.228+0000 7fd5e4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e0107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.228+0000 7fd5e4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0107f60 con 0x7fd5e0107590 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.228+0000 7fd5dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e0107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.228+0000 7fd5dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e0107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53608/0 (socket says 172.21.15.46:53608) 2024-09-10T07:25:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.228+0000 7fd5dde00640 1 -- 172.21.15.46:0/2589392964 learned_addr learned my addr 172.21.15.46:0/2589392964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.229+0000 7fd5dde00640 1 -- 172.21.15.46:0/2589392964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e0108740 con 0x7fd5e0107590 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.230+0000 7fd5dde00640 1 --2- 172.21.15.46:0/2589392964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e0107990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009820 tx=0x7fd5d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8192b02c849002d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.230+0000 7fd5dd400640 1 -- 172.21.15.46:0/2589392964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5d0037070 con 0x7fd5e0107590 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.230+0000 7fd5dd400640 1 -- 172.21.15.46:0/2589392964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5d0032900 con 0x7fd5e0107590 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.230+0000 7fd5dd400640 1 -- 172.21.15.46:0/2589392964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5d0030850 con 0x7fd5e0107590 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.230+0000 7fd5e4a00640 1 -- 172.21.15.46:0/2589392964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 msgr2=0x7fd5e0107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.230+0000 7fd5e4a00640 1 --2- 172.21.15.46:0/2589392964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e0107990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009820 tx=0x7fd5d002a270 comp rx=0 tx=0).stop 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.231+0000 7fd5e4a00640 1 -- 172.21.15.46:0/2589392964 shutdown_connections 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.231+0000 7fd5e4a00640 1 --2- 172.21.15.46:0/2589392964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e0107990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.231+0000 7fd5e4a00640 1 -- 172.21.15.46:0/2589392964 >> 172.21.15.46:0/2589392964 conn(0x7fd5e0102d20 msgr2=0x7fd5e0105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.231+0000 7fd5e4a00640 1 -- 172.21.15.46:0/2589392964 shutdown_connections 2024-09-10T07:25:36.410 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.231+0000 7fd5e4a00640 1 -- 172.21.15.46:0/2589392964 wait complete. 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.232+0000 7fd5e4a00640 1 Processor -- start 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.232+0000 7fd5e4a00640 1 -- start start 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.232+0000 7fd5e4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.232+0000 7fd5e4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e010baf0 con 0x7fd5e0107590 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.232+0000 7fd5dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.232+0000 7fd5dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53614/0 (socket says 172.21.15.46:53614) 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.232+0000 7fd5dde00640 1 -- 172.21.15.46:0/685348249 learned_addr learned my addr 172.21.15.46:0/685348249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.233+0000 7fd5dde00640 1 -- 172.21.15.46:0/685348249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e019bd80 con 0x7fd5e0107590 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.233+0000 7fd5dde00640 1 --2- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e019b840 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00097f0 tx=0x7fd5d0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.233+0000 7fd5d6a00640 1 -- 172.21.15.46:0/685348249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5d0037040 con 0x7fd5e0107590 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.233+0000 7fd5d6a00640 1 -- 172.21.15.46:0/685348249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5d0040070 con 0x7fd5e0107590 2024-09-10T07:25:36.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.233+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e019c010 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.233+0000 7fd5d6a00640 1 -- 172.21.15.46:0/685348249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5d003a390 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.234+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e019ed00 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.235+0000 7fd5d6a00640 1 -- 172.21.15.46:0/685348249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fd5d003b430 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.235+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e0107a10 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.235+0000 7fd5d6a00640 1 --2- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fd5c4038e10 0x7fd5c403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.235+0000 7fd5d6a00640 1 -- 172.21.15.46:0/685348249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd5d006bf00 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.236+0000 7fd5d7e00640 1 --2- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fd5c4038e10 0x7fd5c403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.236+0000 7fd5d7e00640 1 --2- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fd5c4038e10 0x7fd5c403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80098c0 tx=0x7fd5c8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.241+0000 7fd5d6a00640 1 -- 172.21.15.46:0/685348249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5d00371e0 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.497+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd5e002d7c0 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.497+0000 7fd5d6a00640 1 -- 172.21.15.46:0/685348249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fd5d0032dd0 con 0x7fd5e0107590 2024-09-10T07:25:36.412 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fd5c4038e10 msgr2=0x7fd5c403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 --2- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fd5c4038e10 0x7fd5c403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80098c0 tx=0x7fd5c8006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 msgr2=0x7fd5e019b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 --2- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e019b840 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00097f0 tx=0x7fd5d0030d80 comp rx=0 tx=0).stop 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 shutdown_connections 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 --2- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fd5c4038e10 0x7fd5c403b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 --2- 172.21.15.46:0/685348249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0107590 0x7fd5e019b840 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 >> 172.21.15.46:0/685348249 conn(0x7fd5e0102d20 msgr2=0x7fd5e01039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 shutdown_connections 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:35.500+0000 7fd5e4a00640 1 -- 172.21.15.46:0/685348249 wait complete. 2024-09-10T07:25:36.413 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-09-10T07:25:37.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:37 smithi046 ceph-mon[30705]: mgrmap e4: x(active, since 2s) 2024-09-10T07:25:37.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:37 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2811808484' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T07:25:37.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.879+0000 7f4319600640 1 Processor -- start 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.879+0000 7f4319600640 1 -- start start 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.880+0000 7f4319600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f4314107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.880+0000 7f4319600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4314107f60 con 0x7f4314107590 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.880+0000 7f4312a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f4314107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.880+0000 7f4312a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f4314107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39126/0 (socket says 172.21.15.46:39126) 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.880+0000 7f4312a00640 1 -- 172.21.15.46:0/3490208618 learned_addr learned my addr 172.21.15.46:0/3490208618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.880+0000 7f4312a00640 1 -- 172.21.15.46:0/3490208618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4314108740 con 0x7f4314107590 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.881+0000 7f4312a00640 1 --2- 172.21.15.46:0/3490208618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f4314107990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4304009820 tx=0x7f430402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaef41fdd7f1d08b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4312000640 1 -- 172.21.15.46:0/3490208618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4304037070 con 0x7f4314107590 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4312000640 1 -- 172.21.15.46:0/3490208618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4304032900 con 0x7f4314107590 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4312000640 1 -- 172.21.15.46:0/3490208618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4304030850 con 0x7f4314107590 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4319600640 1 -- 172.21.15.46:0/3490208618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 msgr2=0x7f4314107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4319600640 1 --2- 172.21.15.46:0/3490208618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f4314107990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4304009820 tx=0x7f430402a270 comp rx=0 tx=0).stop 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4319600640 1 -- 172.21.15.46:0/3490208618 shutdown_connections 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4319600640 1 --2- 172.21.15.46:0/3490208618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f4314107990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4319600640 1 -- 172.21.15.46:0/3490208618 >> 172.21.15.46:0/3490208618 conn(0x7f4314102d20 msgr2=0x7f4314105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.882+0000 7f4319600640 1 -- 172.21.15.46:0/3490208618 shutdown_connections 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.883+0000 7f4319600640 1 -- 172.21.15.46:0/3490208618 wait complete. 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.883+0000 7f4319600640 1 Processor -- start 2024-09-10T07:25:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.884+0000 7f4319600640 1 -- start start 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.884+0000 7f4319600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f431419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.884+0000 7f4319600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f431410baf0 con 0x7f4314107590 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.884+0000 7f4312a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f431419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.884+0000 7f4312a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f431419da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39136/0 (socket says 172.21.15.46:39136) 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.884+0000 7f4312a00640 1 -- 172.21.15.46:0/2811808484 learned_addr learned my addr 172.21.15.46:0/2811808484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.884+0000 7f4312a00640 1 -- 172.21.15.46:0/2811808484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431419dfc0 con 0x7f4314107590 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.885+0000 7f4312a00640 1 --2- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f431419da80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f43040097f0 tx=0x7f4304030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.885+0000 7f430b400640 1 -- 172.21.15.46:0/2811808484 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4304037040 con 0x7f4314107590 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.885+0000 7f430b400640 1 -- 172.21.15.46:0/2811808484 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4304040070 con 0x7f4314107590 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.885+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431419e250 con 0x7f4314107590 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.885+0000 7f430b400640 1 -- 172.21.15.46:0/2811808484 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430403a390 con 0x7f4314107590 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.885+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141a0f40 con 0x7f4314107590 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.886+0000 7f430b400640 1 -- 172.21.15.46:0/2811808484 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f4304032900 con 0x7f4314107590 2024-09-10T07:25:37.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.886+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4314107a10 con 0x7f4314107590 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.887+0000 7f430b400640 1 --2- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f42f8038f20 0x7f42f803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.887+0000 7f430b400640 1 -- 172.21.15.46:0/2811808484 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f430406c0c0 con 0x7f4314107590 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.887+0000 7f430be00640 1 --2- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f42f8038f20 0x7f42f803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.888+0000 7f430be00640 1 --2- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f42f8038f20 0x7f42f803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0098c0 tx=0x7f42fc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:36.892+0000 7f430b400640 1 -- 172.21.15.46:0/2811808484 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f430406be20 con 0x7f4314107590 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.081+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4314075df0 con 0x7f4314107590 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.082+0000 7f430b400640 1 -- 172.21.15.46:0/2811808484 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f4304032bf0 con 0x7f4314107590 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.084+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f42f8038f20 msgr2=0x7f42f803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.084+0000 7f4319600640 1 --2- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f42f8038f20 0x7f42f803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0098c0 tx=0x7f42fc006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.084+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 msgr2=0x7f431419da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.084+0000 7f4319600640 1 --2- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f431419da80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f43040097f0 tx=0x7f4304030d80 comp rx=0 tx=0).stop 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.085+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 shutdown_connections 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.085+0000 7f4319600640 1 --2- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f42f8038f20 0x7f42f803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:37.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.085+0000 7f4319600640 1 --2- 172.21.15.46:0/2811808484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4314107590 0x7f431419da80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:37.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.085+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 >> 172.21.15.46:0/2811808484 conn(0x7f4314102d20 msgr2=0x7f4314103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:37.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.085+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 shutdown_connections 2024-09-10T07:25:37.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:37.085+0000 7f4319600640 1 -- 172.21.15.46:0/2811808484 wait complete. 2024-09-10T07:25:37.777 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-09-10T07:25:38.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:38 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3432939379' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T07:25:38.909 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:38 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: ignoring --setuser ceph since I am not root 2024-09-10T07:25:38.909 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:38 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: ignoring --setgroup ceph since I am not root 2024-09-10T07:25:39.204 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:38 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:38.929+0000 7f5ff0209040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:25:39.205 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:39 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:39.004+0000 7f5ff0209040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:25:39.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.404+0000 7fba62000640 1 Processor -- start 2024-09-10T07:25:39.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.404+0000 7fba62000640 1 -- start start 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.404+0000 7fba62000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c081940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.404+0000 7fba62000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c081f10 con 0x7fba5c081540 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.405+0000 7fba5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c081940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.405+0000 7fba5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c081940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39140/0 (socket says 172.21.15.46:39140) 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.405+0000 7fba5b400640 1 -- 172.21.15.46:0/3218508103 learned_addr learned my addr 172.21.15.46:0/3218508103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.405+0000 7fba5b400640 1 -- 172.21.15.46:0/3218508103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c10ed80 con 0x7fba5c081540 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.406+0000 7fba5b400640 1 --2- 172.21.15.46:0/3218508103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c081940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009820 tx=0x7fba4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14c53c6eef221a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.406+0000 7fba5aa00640 1 -- 172.21.15.46:0/3218508103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba4c037070 con 0x7fba5c081540 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.406+0000 7fba5aa00640 1 -- 172.21.15.46:0/3218508103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba4c032900 con 0x7fba5c081540 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.406+0000 7fba5aa00640 1 -- 172.21.15.46:0/3218508103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba4c030850 con 0x7fba5c081540 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.407+0000 7fba62000640 1 -- 172.21.15.46:0/3218508103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 msgr2=0x7fba5c081940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:39.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.407+0000 7fba62000640 1 --2- 172.21.15.46:0/3218508103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c081940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009820 tx=0x7fba4c02a270 comp rx=0 tx=0).stop 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.407+0000 7fba62000640 1 -- 172.21.15.46:0/3218508103 shutdown_connections 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.407+0000 7fba62000640 1 --2- 172.21.15.46:0/3218508103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c081940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.407+0000 7fba62000640 1 -- 172.21.15.46:0/3218508103 >> 172.21.15.46:0/3218508103 conn(0x7fba5c07cd20 msgr2=0x7fba5c07f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.407+0000 7fba62000640 1 -- 172.21.15.46:0/3218508103 shutdown_connections 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.407+0000 7fba62000640 1 -- 172.21.15.46:0/3218508103 wait complete. 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.408+0000 7fba62000640 1 Processor -- start 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.408+0000 7fba62000640 1 -- start start 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.409+0000 7fba62000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.409+0000 7fba62000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c112130 con 0x7fba5c081540 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.409+0000 7fba5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.409+0000 7fba5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c1a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39156/0 (socket says 172.21.15.46:39156) 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.409+0000 7fba5b400640 1 -- 172.21.15.46:0/3432939379 learned_addr learned my addr 172.21.15.46:0/3432939379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:39.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.409+0000 7fba5b400640 1 -- 172.21.15.46:0/3432939379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c1a23a0 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.409+0000 7fba5b400640 1 --2- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c1a1e60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fba4c032ce0 tx=0x7fba4c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.410+0000 7fba58c00640 1 -- 172.21.15.46:0/3432939379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba4c042070 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.410+0000 7fba58c00640 1 -- 172.21.15.46:0/3432939379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba4c0308c0 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.410+0000 7fba58c00640 1 -- 172.21.15.46:0/3432939379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba4c037040 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.410+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c1a2630 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.410+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c1a5320 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.411+0000 7fba58c00640 1 -- 172.21.15.46:0/3432939379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fba4c045070 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.411+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba5c081940 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.411+0000 7fba58c00640 1 --2- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fba40038f20 0x7fba4003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.411+0000 7fba58c00640 1 -- 172.21.15.46:0/3432939379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fba4c06bf30 con 0x7fba5c081540 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.412+0000 7fba53e00640 1 --2- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fba40038f20 0x7fba4003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:39.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.412+0000 7fba53e00640 1 --2- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fba40038f20 0x7fba4003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba440098c0 tx=0x7fba44006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.417+0000 7fba58c00640 1 -- 172.21.15.46:0/3432939379 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba4c04b170 con 0x7fba5c081540 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.650+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fba5c1a5680 con 0x7fba5c081540 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.702+0000 7fba58c00640 1 -- 172.21.15.46:0/3432939379 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fba4c06b950 con 0x7fba5c081540 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.703+0000 7fba58c00640 1 -- 172.21.15.46:0/3432939379 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fba4c03d430 con 0x7fba5c081540 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fba40038f20 msgr2=0x7fba4003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 --2- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fba40038f20 0x7fba4003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba440098c0 tx=0x7fba44006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 msgr2=0x7fba5c1a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 --2- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c1a1e60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fba4c032ce0 tx=0x7fba4c032d10 comp rx=0 tx=0).stop 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 shutdown_connections 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 --2- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7fba40038f20 0x7fba4003b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 --2- 172.21.15.46:0/3432939379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c081540 0x7fba5c1a1e60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 >> 172.21.15.46:0/3432939379 conn(0x7fba5c07cd20 msgr2=0x7fba5c07d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 shutdown_connections 2024-09-10T07:25:39.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:38.705+0000 7fba62000640 1 -- 172.21.15.46:0/3432939379 wait complete. 2024-09-10T07:25:39.478 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:39 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:39.204+0000 7f5ff0209040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:25:39.478 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:39 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:39.287+0000 7f5ff0209040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:25:39.478 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:39 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:39.478+0000 7f5ff0209040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:25:39.732 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:39 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3432939379' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T07:25:39.732 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:39 smithi046 ceph-mon[30705]: mgrmap e5: x(active, since 4s) 2024-09-10T07:25:39.732 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:39 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:39.652+0000 7f5ff0209040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:25:40.313 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:40 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:40.041+0000 7f5ff0209040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:25:40.313 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:40 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:40.118+0000 7f5ff0209040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:25:40.659 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:40 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:40.338+0000 7f5ff0209040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:25:41.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:40 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2657167224' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T07:25:41.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:25:41.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T07:25:41.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T07:25:41.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T07:25:41.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T07:25:41.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:25:41.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.103+0000 7f99d0a00640 1 Processor -- start 2024-09-10T07:25:41.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.104+0000 7f99d0a00640 1 -- start start 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.104+0000 7f99d0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.104+0000 7f99d0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc07c8c0 con 0x7f99cc07de40 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.104+0000 7f99ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.104+0000 7f99ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39184/0 (socket says 172.21.15.46:39184) 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.104+0000 7f99ca800640 1 -- 172.21.15.46:0/559974043 learned_addr learned my addr 172.21.15.46:0/559974043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.105+0000 7f99ca800640 1 -- 172.21.15.46:0/559974043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc07ca40 con 0x7f99cc07de40 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.106+0000 7f99ca800640 1 --2- 172.21.15.46:0/559974043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc07c2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009820 tx=0x7f99bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcab975c584700d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.106+0000 7f99c9e00640 1 -- 172.21.15.46:0/559974043 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99bc037070 con 0x7f99cc07de40 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.106+0000 7f99c9e00640 1 -- 172.21.15.46:0/559974043 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f99bc032900 con 0x7f99cc07de40 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.106+0000 7f99c9e00640 1 -- 172.21.15.46:0/559974043 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99bc030850 con 0x7f99cc07de40 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.107+0000 7f99d0a00640 1 -- 172.21.15.46:0/559974043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 msgr2=0x7f99cc07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.107+0000 7f99d0a00640 1 --2- 172.21.15.46:0/559974043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc07c2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009820 tx=0x7f99bc02a270 comp rx=0 tx=0).stop 2024-09-10T07:25:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.107+0000 7f99d0a00640 1 -- 172.21.15.46:0/559974043 shutdown_connections 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.107+0000 7f99d0a00640 1 --2- 172.21.15.46:0/559974043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc07c2f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.107+0000 7f99d0a00640 1 -- 172.21.15.46:0/559974043 >> 172.21.15.46:0/559974043 conn(0x7f99cc102bb0 msgr2=0x7f99cc104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.107+0000 7f99d0a00640 1 -- 172.21.15.46:0/559974043 shutdown_connections 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.107+0000 7f99d0a00640 1 -- 172.21.15.46:0/559974043 wait complete. 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.108+0000 7f99d0a00640 1 Processor -- start 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.108+0000 7f99d0a00640 1 -- start start 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.108+0000 7f99d0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc1a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.108+0000 7f99d0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc1095c0 con 0x7f99cc07de40 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.109+0000 7f99ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc1a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.109+0000 7f99ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc1a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39198/0 (socket says 172.21.15.46:39198) 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.109+0000 7f99ca800640 1 -- 172.21.15.46:0/2657167224 learned_addr learned my addr 172.21.15.46:0/2657167224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.109+0000 7f99ca800640 1 -- 172.21.15.46:0/2657167224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc1a21b0 con 0x7f99cc07de40 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.109+0000 7f99ca800640 1 --2- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc1a1c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0097f0 tx=0x7f99bc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:41.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.110+0000 7f99c3400640 1 -- 172.21.15.46:0/2657167224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99bc037040 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.110+0000 7f99c3400640 1 -- 172.21.15.46:0/2657167224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f99bc040070 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.110+0000 7f99c3400640 1 -- 172.21.15.46:0/2657167224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99bc03a390 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.110+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc1a2440 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.110+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc1a5130 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.111+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc07c370 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.111+0000 7f99c3400640 1 -- 172.21.15.46:0/2657167224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f99bc032900 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.112+0000 7f99c3400640 1 --2- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f99b0038fc0 0x7f99b003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.112+0000 7f99c3400640 1 -- 172.21.15.46:0/2657167224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f99bc06c100 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.112+0000 7f99c3e00640 1 -- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f99b0038fc0 msgr2=0x7f99b003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/141501936 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.112+0000 7f99c3e00640 1 --2- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f99b0038fc0 0x7f99b003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.117+0000 7f99c3400640 1 -- 172.21.15.46:0/2657167224 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99bc06be60 con 0x7f99cc07de40 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.313+0000 7f99c3e00640 1 -- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f99b0038fc0 msgr2=0x7f99b003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/141501936 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.313+0000 7f99c3e00640 1 --2- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f99b0038fc0 0x7f99b003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T07:25:41.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.348+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f99cc02d7c0 con 0x7f99cc07de40 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.349+0000 7f99c3400640 1 -- 172.21.15.46:0/2657167224 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f99bc040ca0 con 0x7f99cc07de40 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f99b0038fc0 msgr2=0x7f99b003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 --2- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f99b0038fc0 0x7f99b003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 msgr2=0x7f99cc1a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 --2- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc1a1c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0097f0 tx=0x7f99bc030d80 comp rx=0 tx=0).stop 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 shutdown_connections 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 --2- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f99b0038fc0 0x7f99b003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 --2- 172.21.15.46:0/2657167224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99cc07de40 0x7f99cc1a1c70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 >> 172.21.15.46:0/2657167224 conn(0x7f99cc102bb0 msgr2=0x7f99cc1986f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 shutdown_connections 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:40.351+0000 7f99d0a00640 1 -- 172.21.15.46:0/2657167224 wait complete. 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-09-10T07:25:41.294 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-09-10T07:25:41.568 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:41 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:41.484+0000 7f5ff0209040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:25:41.847 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:41 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:41.568+0000 7f5ff0209040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:25:42.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:42.527+0000 7f5ff0209040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:25:42.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:42.667+0000 7f5ff0209040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:25:42.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:42.836+0000 7f5ff0209040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:25:43.086 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:42.910+0000 7f5ff0209040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:25:43.087 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.086+0000 7f5ff0209040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:25:43.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.170+0000 7f5ff0209040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:25:43.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.244+0000 7f5ff0209040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:25:43.691 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.431+0000 7f5ff0209040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:25:43.691 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.510+0000 7f5ff0209040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:25:43.691 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.591+0000 7f5ff0209040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:25:43.691 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.691+0000 7f5ff0209040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:25:44.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.764+0000 7f5ff0209040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:25:44.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:43.836+0000 7f5ff0209040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:25:45.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:44.778+0000 7f5ff0209040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:25:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: Active manager daemon x restarted 2024-09-10T07:25:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: Activating manager daemon x 2024-09-10T07:25:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T07:25:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: mgrmap e6: x(active, starting, since 0.00421261s) 2024-09-10T07:25:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:25:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T07:25:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T07:25:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T07:25:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T07:25:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: Manager daemon x is now available 2024-09-10T07:25:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: Found migration_current of "None". Setting to last migration. 2024-09-10T07:25:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:25:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:25:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:25:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:44 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:25:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:45 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T07:25:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:45 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T07:25:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:45 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:25:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:45 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:25:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:45 smithi046 ceph-mon[30705]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T07:25:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:45 smithi046 ceph-mon[30705]: mgrmap e7: x(active, since 1.00679s) 2024-09-10T07:25:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:45 smithi046 ceph-mon[30705]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T07:25:46.525 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.148+0000 7f190e400640 1 Processor -- start 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.148+0000 7f190e400640 1 -- start start 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.148+0000 7f190e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f1908109c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.148+0000 7f190e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f190810a1f0 con 0x7f1908109820 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.149+0000 7f1907400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f1908109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.149+0000 7f1907400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f1908109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39210/0 (socket says 172.21.15.46:39210) 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.149+0000 7f1907400640 1 -- 172.21.15.46:0/1741352760 learned_addr learned my addr 172.21.15.46:0/1741352760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:46.526 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.149+0000 7f1907400640 1 -- 172.21.15.46:0/1741352760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190810a9b0 con 0x7f1908109820 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.150+0000 7f1907400640 1 --2- 172.21.15.46:0/1741352760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f1908109c20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009820 tx=0x7f18f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85f8a55af6ac92c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.150+0000 7f1906000640 1 -- 172.21.15.46:0/1741352760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18f8037070 con 0x7f1908109820 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.151+0000 7f1906000640 1 -- 172.21.15.46:0/1741352760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f18f8032900 con 0x7f1908109820 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.151+0000 7f1906000640 1 -- 172.21.15.46:0/1741352760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18f8030850 con 0x7f1908109820 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.151+0000 7f190e400640 1 -- 172.21.15.46:0/1741352760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 msgr2=0x7f1908109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.151+0000 7f190e400640 1 --2- 172.21.15.46:0/1741352760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f1908109c20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009820 tx=0x7f18f802a270 comp rx=0 tx=0).stop 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.151+0000 7f190e400640 1 -- 172.21.15.46:0/1741352760 shutdown_connections 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.151+0000 7f190e400640 1 --2- 172.21.15.46:0/1741352760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f1908109c20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.151+0000 7f190e400640 1 -- 172.21.15.46:0/1741352760 >> 172.21.15.46:0/1741352760 conn(0x7f190807cd90 msgr2=0x7f190807d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.151+0000 7f190e400640 1 -- 172.21.15.46:0/1741352760 shutdown_connections 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.152+0000 7f190e400640 1 -- 172.21.15.46:0/1741352760 wait complete. 2024-09-10T07:25:46.527 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.152+0000 7f190e400640 1 Processor -- start 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.153+0000 7f190e400640 1 -- start start 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.153+0000 7f190e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f190819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.153+0000 7f190e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f190810dd60 con 0x7f1908109820 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.153+0000 7f1907400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f190819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.153+0000 7f1907400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f190819da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39226/0 (socket says 172.21.15.46:39226) 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.153+0000 7f1907400640 1 -- 172.21.15.46:0/2564411022 learned_addr learned my addr 172.21.15.46:0/2564411022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.154+0000 7f1907400640 1 -- 172.21.15.46:0/2564411022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190819dfc0 con 0x7f1908109820 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.154+0000 7f1907400640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f190819da80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18f8032ce0 tx=0x7f18f8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.154+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18f8042070 con 0x7f1908109820 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.154+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f18f80308c0 con 0x7f1908109820 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.154+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190819e250 con 0x7f1908109820 2024-09-10T07:25:46.528 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.154+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18f8037040 con 0x7f1908109820 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.155+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19081a0f40 con 0x7f1908109820 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.156+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f18f8045070 con 0x7f1908109820 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.156+0000 7f18ffe00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 0x7f18ec03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.156+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 --> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18ec03bb90 con 0x7f18ec038fc0 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.156+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18f806bf30 con 0x7f1908109820 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.156+0000 7f1906a00640 1 -- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 msgr2=0x7f18ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/141501936 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.157+0000 7f1906a00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 0x7f18ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.358+0000 7f1906a00640 1 -- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 msgr2=0x7f18ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/141501936 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.358+0000 7f1906a00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 0x7f18ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.759+0000 7f1906a00640 1 -- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 msgr2=0x7f18ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/141501936 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:42.759+0000 7f1906a00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 0x7f18ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:43.560+0000 7f1906a00640 1 -- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 msgr2=0x7f18ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/141501936 2024-09-10T07:25:46.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:43.560+0000 7f1906a00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 0x7f18ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:44.783+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f18f806a9f0 con 0x7f1908109820 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:44.783+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 msgr2=0x7f18ec03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:44.783+0000 7f18ffe00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 0x7f18ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.784+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f18f803d430 con 0x7f1908109820 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.784+0000 7f18ffe00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f18ec03c8f0 0x7f18ec03ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.785+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18f8065060 con 0x7f18ec03c8f0 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.785+0000 7f1906a00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f18ec03c8f0 0x7f18ec03ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.786+0000 7f1906a00640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f18ec03c8f0 0x7f18ec03ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18f4003980 tx=0x7f18f40092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.787+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mgr.14118 v2:172.21.15.46:6800/2972952762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f18f8065060 con 0x7f18ec03c8f0 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.800+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1908109c20 con 0x7f18ec03c8f0 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f18ffe00640 1 -- 172.21.15.46:0/2564411022 <== mgr.14118 v2:172.21.15.46:6800/2972952762 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f1908109c20 con 0x7f18ec03c8f0 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f18ec03c8f0 msgr2=0x7f18ec03ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:46.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f18ec03c8f0 0x7f18ec03ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18f4003980 tx=0x7f18f40092b0 comp rx=0 tx=0).stop 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 msgr2=0x7f190819da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f190819da80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18f8032ce0 tx=0x7f18f8032d10 comp rx=0 tx=0).stop 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 shutdown_connections 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f18ec03c8f0 0x7f18ec03ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:6800/141501936,v1:172.21.15.46:6801/141501936] conn(0x7f18ec038fc0 0x7f18ec03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 --2- 172.21.15.46:0/2564411022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908109820 0x7f190819da80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 >> 172.21.15.46:0/2564411022 conn(0x7f190807cd90 msgr2=0x7f1908107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 shutdown_connections 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:45.801+0000 7f190e400640 1 -- 172.21.15.46:0/2564411022 wait complete. 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-09-10T07:25:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-09-10T07:25:47.835 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:47.557+0000 7f5f4e600640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:07:25:47] ENGINE Error in HTTPServer.serve 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: Traceback (most recent call last): 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: self._connections.run(self.expiration_interval) 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: self._run(expiration_interval) 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T07:25:47.836 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: s = self.context.wrap_socket( 2024-09-10T07:25:47.837 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T07:25:47.837 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: return self.sslsocket_class._create( 2024-09-10T07:25:47.837 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T07:25:47.837 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: self.do_handshake() 2024-09-10T07:25:47.837 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T07:25:47.837 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: self._sslobj.do_handshake() 2024-09-10T07:25:47.837 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T07:25:47.837 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.040+0000 7fba4b200640 1 Processor -- start 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.040+0000 7fba4b200640 1 -- start start 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.040+0000 7fba4b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c07c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.040+0000 7fba4b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c07c860 con 0x7fba4c07de30 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.040+0000 7fba49e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c07c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.041+0000 7fba49e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c07c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48844/0 (socket says 172.21.15.46:48844) 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.041+0000 7fba49e00640 1 -- 172.21.15.46:0/1961022062 learned_addr learned my addr 172.21.15.46:0/1961022062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.041+0000 7fba49e00640 1 -- 172.21.15.46:0/1961022062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c07c9e0 con 0x7fba4c07de30 2024-09-10T07:25:47.852 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.042+0000 7fba49e00640 1 --2- 172.21.15.46:0/1961022062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c07c290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009820 tx=0x7fba3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2bb4f4a03d7883 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.042+0000 7fba49400640 1 -- 172.21.15.46:0/1961022062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba3c037070 con 0x7fba4c07de30 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.042+0000 7fba49400640 1 -- 172.21.15.46:0/1961022062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba3c032900 con 0x7fba4c07de30 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.042+0000 7fba49400640 1 -- 172.21.15.46:0/1961022062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba3c030850 con 0x7fba4c07de30 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.043+0000 7fba4b200640 1 -- 172.21.15.46:0/1961022062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 msgr2=0x7fba4c07c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.043+0000 7fba4b200640 1 --2- 172.21.15.46:0/1961022062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c07c290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009820 tx=0x7fba3c02a270 comp rx=0 tx=0).stop 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.043+0000 7fba4b200640 1 -- 172.21.15.46:0/1961022062 shutdown_connections 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.043+0000 7fba4b200640 1 --2- 172.21.15.46:0/1961022062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c07c290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.043+0000 7fba4b200640 1 -- 172.21.15.46:0/1961022062 >> 172.21.15.46:0/1961022062 conn(0x7fba4c102b40 msgr2=0x7fba4c104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.043+0000 7fba4b200640 1 -- 172.21.15.46:0/1961022062 shutdown_connections 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.043+0000 7fba4b200640 1 -- 172.21.15.46:0/1961022062 wait complete. 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.044+0000 7fba4b200640 1 Processor -- start 2024-09-10T07:25:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.044+0000 7fba4b200640 1 -- start start 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.045+0000 7fba4b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.045+0000 7fba4b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c10d9a0 con 0x7fba4c07de30 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.045+0000 7fba49e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.045+0000 7fba49e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c19d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48846/0 (socket says 172.21.15.46:48846) 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.045+0000 7fba49e00640 1 -- 172.21.15.46:0/978063907 learned_addr learned my addr 172.21.15.46:0/978063907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.045+0000 7fba49e00640 1 -- 172.21.15.46:0/978063907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c19dd60 con 0x7fba4c07de30 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.046+0000 7fba49e00640 1 --2- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c19d820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0097f0 tx=0x7fba3c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.046+0000 7fba42a00640 1 -- 172.21.15.46:0/978063907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba3c037040 con 0x7fba4c07de30 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.046+0000 7fba42a00640 1 -- 172.21.15.46:0/978063907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba3c040070 con 0x7fba4c07de30 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.046+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c19dff0 con 0x7fba4c07de30 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.046+0000 7fba42a00640 1 -- 172.21.15.46:0/978063907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba3c03a390 con 0x7fba4c07de30 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.046+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c1a0ce0 con 0x7fba4c07de30 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.047+0000 7fba42a00640 1 -- 172.21.15.46:0/978063907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fba3c03b430 con 0x7fba4c07de30 2024-09-10T07:25:47.854 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.047+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4c07c310 con 0x7fba4c07de30 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.048+0000 7fba42a00640 1 --2- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fba30038f70 0x7fba3003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.048+0000 7fba42a00640 1 -- 172.21.15.46:0/978063907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fba3c06c550 con 0x7fba4c07de30 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.049+0000 7fba43e00640 1 --2- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fba30038f70 0x7fba3003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.049+0000 7fba43e00640 1 --2- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fba30038f70 0x7fba3003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba340098c0 tx=0x7fba34006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.054+0000 7fba42a00640 1 -- 172.21.15.46:0/978063907 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba3c06bea0 con 0x7fba4c07de30 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.254+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fba4c02d830 con 0x7fba30038f70 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.260+0000 7fba42a00640 1 -- 172.21.15.46:0/978063907 <== mgr.14118 v2:172.21.15.46:6800/2972952762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba4c02d830 con 0x7fba30038f70 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fba30038f70 msgr2=0x7fba3003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 --2- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fba30038f70 0x7fba3003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba340098c0 tx=0x7fba34006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 msgr2=0x7fba4c19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:47.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 --2- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c19d820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0097f0 tx=0x7fba3c030d80 comp rx=0 tx=0).stop 2024-09-10T07:25:47.856 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 shutdown_connections 2024-09-10T07:25:47.856 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 --2- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fba30038f70 0x7fba3003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:47.856 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 --2- 172.21.15.46:0/978063907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba4c07de30 0x7fba4c19d820 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:47.856 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 >> 172.21.15.46:0/978063907 conn(0x7fba4c102b40 msgr2=0x7fba4c103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:47.856 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 shutdown_connections 2024-09-10T07:25:47.856 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:47.263+0000 7fba4b200640 1 -- 172.21.15.46:0/978063907 wait complete. 2024-09-10T07:25:48.125 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: mgrmap e8: x(active, since 2s) 2024-09-10T07:25:48.125 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:25:48.125 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:25:48.125 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:25:48.125 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: [10/Sep/2024:07:25:47] ENGINE Bus STARTING 2024-09-10T07:25:48.125 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: [10/Sep/2024:07:25:47] ENGINE Serving on https://172.21.15.46:7150 2024-09-10T07:25:48.126 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: [10/Sep/2024:07:25:47] ENGINE Error in HTTPServer.serve 2024-09-10T07:25:48.126 INFO:journalctl@ceph.mon.a.smithi046.stdout: Traceback (most recent call last): 2024-09-10T07:25:48.126 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T07:25:48.126 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-10T07:25:48.126 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T07:25:48.126 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._run(expiration_interval) 2024-09-10T07:25:48.126 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T07:25:48.126 INFO:journalctl@ceph.mon.a.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: s = self.context.wrap_socket( 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: return self.sslsocket_class._create( 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: self.do_handshake() 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._sslobj.do_handshake() 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: [10/Sep/2024:07:25:47] ENGINE Serving on http://172.21.15.46:8765 2024-09-10T07:25:48.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: [10/Sep/2024:07:25:47] ENGINE Bus STARTED 2024-09-10T07:25:48.128 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:47 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:25:49.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:48 smithi046 ceph-mon[30705]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:25:49.257 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T07:25:49.257 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.360+0000 7f392f200640 1 Processor -- start 2024-09-10T07:25:49.257 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.360+0000 7f392f200640 1 -- start start 2024-09-10T07:25:49.258 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.361+0000 7f392f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f3928107970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:49.258 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.361+0000 7f392f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3928107f40 con 0x7f3928107570 2024-09-10T07:25:49.258 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.361+0000 7f3927e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f3928107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:49.258 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.361+0000 7f3927e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f3928107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48862/0 (socket says 172.21.15.46:48862) 2024-09-10T07:25:49.258 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.361+0000 7f3927e00640 1 -- 172.21.15.46:0/1353994962 learned_addr learned my addr 172.21.15.46:0/1353994962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:49.258 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.361+0000 7f3927e00640 1 -- 172.21.15.46:0/1353994962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3928108770 con 0x7f3928107570 2024-09-10T07:25:49.258 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.362+0000 7f3927e00640 1 --2- 172.21.15.46:0/1353994962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f3928107970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3918009820 tx=0x7f391802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfab582f35aaf017 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f3927400640 1 -- 172.21.15.46:0/1353994962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3918037070 con 0x7f3928107570 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f3927400640 1 -- 172.21.15.46:0/1353994962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3918032900 con 0x7f3928107570 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f3927400640 1 -- 172.21.15.46:0/1353994962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3918030850 con 0x7f3928107570 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f392f200640 1 -- 172.21.15.46:0/1353994962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 msgr2=0x7f3928107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f392f200640 1 --2- 172.21.15.46:0/1353994962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f3928107970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3918009820 tx=0x7f391802a270 comp rx=0 tx=0).stop 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f392f200640 1 -- 172.21.15.46:0/1353994962 shutdown_connections 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f392f200640 1 --2- 172.21.15.46:0/1353994962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f3928107970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f392f200640 1 -- 172.21.15.46:0/1353994962 >> 172.21.15.46:0/1353994962 conn(0x7f3928102cc0 msgr2=0x7f3928105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f392f200640 1 -- 172.21.15.46:0/1353994962 shutdown_connections 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.363+0000 7f392f200640 1 -- 172.21.15.46:0/1353994962 wait complete. 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.364+0000 7f392f200640 1 Processor -- start 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.365+0000 7f392f200640 1 -- start start 2024-09-10T07:25:49.259 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.365+0000 7f392f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f392819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.365+0000 7f392f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f392810bb20 con 0x7f3928107570 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.365+0000 7f3927e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f392819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.365+0000 7f3927e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f392819b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48876/0 (socket says 172.21.15.46:48876) 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.365+0000 7f3927e00640 1 -- 172.21.15.46:0/4201355382 learned_addr learned my addr 172.21.15.46:0/4201355382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.365+0000 7f3927e00640 1 -- 172.21.15.46:0/4201355382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392819be20 con 0x7f3928107570 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.366+0000 7f3927e00640 1 --2- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f392819b8e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3918009940 tx=0x7f39180326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.366+0000 7f3925600640 1 -- 172.21.15.46:0/4201355382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3918042070 con 0x7f3928107570 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.366+0000 7f3925600640 1 -- 172.21.15.46:0/4201355382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3918032be0 con 0x7f3928107570 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.366+0000 7f3925600640 1 -- 172.21.15.46:0/4201355382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3918037040 con 0x7f3928107570 2024-09-10T07:25:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.366+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392819c0b0 con 0x7f3928107570 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.366+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392819eda0 con 0x7f3928107570 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.367+0000 7f3925600640 1 -- 172.21.15.46:0/4201355382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3918045070 con 0x7f3928107570 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.367+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f392810bca0 con 0x7f3928107570 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.368+0000 7f3925600640 1 --2- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f390c038fc0 0x7f390c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.368+0000 7f3925600640 1 -- 172.21.15.46:0/4201355382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f391806bf50 con 0x7f3928107570 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.368+0000 7f391f400640 1 --2- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f390c038fc0 0x7f390c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.369+0000 7f391f400640 1 --2- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f390c038fc0 0x7f390c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f39100098c0 tx=0x7f3910006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.373+0000 7f3925600640 1 -- 172.21.15.46:0/4201355382 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f391804b170 con 0x7f3928107570 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.558+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3928062630 con 0x7f390c038fc0 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.559+0000 7f3925600640 1 -- 172.21.15.46:0/4201355382 <== mgr.14118 v2:172.21.15.46:6800/2972952762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3928062630 con 0x7f390c038fc0 2024-09-10T07:25:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f390c038fc0 msgr2=0x7f390c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 --2- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f390c038fc0 0x7f390c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f39100098c0 tx=0x7f3910006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 msgr2=0x7f392819b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 --2- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f392819b8e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3918009940 tx=0x7f39180326e0 comp rx=0 tx=0).stop 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 shutdown_connections 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 --2- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f390c038fc0 0x7f390c03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 --2- 172.21.15.46:0/4201355382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3928107570 0x7f392819b8e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 >> 172.21.15.46:0/4201355382 conn(0x7f3928102cc0 msgr2=0x7f3928103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 shutdown_connections 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:48.561+0000 7f392f200640 1 -- 172.21.15.46:0/4201355382 wait complete. 2024-09-10T07:25:49.262 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-09-10T07:25:50.659 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: Generating public/private rsa key pair. 2024-09-10T07:25:50.660 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: Your identification has been saved in /tmp/tmpdc8tmlva/key 2024-09-10T07:25:50.660 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: Your public key has been saved in /tmp/tmpdc8tmlva/key.pub 2024-09-10T07:25:50.660 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: The key fingerprint is: 2024-09-10T07:25:50.660 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: SHA256:5HOt6xjMcUbeXyewllJ0GFiEiM26ZWC4hfJ90cNhl+k ceph-b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:50.660 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: The key's randomart image is: 2024-09-10T07:25:50.661 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: +---[RSA 3072]----+ 2024-09-10T07:25:50.661 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | o + +o**=. | 2024-09-10T07:25:50.661 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | . o = =.*o+. | 2024-09-10T07:25:50.661 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | o = o...oo | 2024-09-10T07:25:50.661 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | o oo+o +E+ | 2024-09-10T07:25:50.661 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | =S * * . o| 2024-09-10T07:25:50.662 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | .o * + . o.| 2024-09-10T07:25:50.662 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | + . . | 2024-09-10T07:25:50.662 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | o . | 2024-09-10T07:25:50.662 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: | ..o | 2024-09-10T07:25:50.662 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:25:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: +----[SHA256]-----+ 2024-09-10T07:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.787+0000 7f1735e00640 1 Processor -- start 2024-09-10T07:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.787+0000 7f1735e00640 1 -- start start 2024-09-10T07:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.787+0000 7f1735e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f1730109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.787+0000 7f1735e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f173010a240 con 0x7f1730109870 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.788+0000 7f172ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f1730109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.788+0000 7f172ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f1730109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48882/0 (socket says 172.21.15.46:48882) 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.788+0000 7f172ea00640 1 -- 172.21.15.46:0/3738735205 learned_addr learned my addr 172.21.15.46:0/3738735205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.788+0000 7f172ea00640 1 -- 172.21.15.46:0/3738735205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173010a980 con 0x7f1730109870 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.789+0000 7f172ea00640 1 --2- 172.21.15.46:0/3738735205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f1730109c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1720009820 tx=0x7f172002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfe82afb474f6f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.789+0000 7f172d600640 1 -- 172.21.15.46:0/3738735205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1720037070 con 0x7f1730109870 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.789+0000 7f172d600640 1 -- 172.21.15.46:0/3738735205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1720032900 con 0x7f1730109870 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.789+0000 7f172d600640 1 -- 172.21.15.46:0/3738735205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1720030850 con 0x7f1730109870 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.790+0000 7f1735e00640 1 -- 172.21.15.46:0/3738735205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 msgr2=0x7f1730109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.790+0000 7f1735e00640 1 --2- 172.21.15.46:0/3738735205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f1730109c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1720009820 tx=0x7f172002a270 comp rx=0 tx=0).stop 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.790+0000 7f1735e00640 1 -- 172.21.15.46:0/3738735205 shutdown_connections 2024-09-10T07:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.790+0000 7f1735e00640 1 --2- 172.21.15.46:0/3738735205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f1730109c70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.790+0000 7f1735e00640 1 -- 172.21.15.46:0/3738735205 >> 172.21.15.46:0/3738735205 conn(0x7f173007cdc0 msgr2=0x7f173007d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.790+0000 7f1735e00640 1 -- 172.21.15.46:0/3738735205 shutdown_connections 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.790+0000 7f1735e00640 1 -- 172.21.15.46:0/3738735205 wait complete. 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.791+0000 7f1735e00640 1 Processor -- start 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.791+0000 7f1735e00640 1 -- start start 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.791+0000 7f1735e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f173019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.791+0000 7f1735e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f173010dd30 con 0x7f1730109870 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.792+0000 7f172ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f173019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.792+0000 7f172ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f173019daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48888/0 (socket says 172.21.15.46:48888) 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.792+0000 7f172ea00640 1 -- 172.21.15.46:0/2115633135 learned_addr learned my addr 172.21.15.46:0/2115633135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.792+0000 7f172ea00640 1 -- 172.21.15.46:0/2115633135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173019dfe0 con 0x7f1730109870 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.792+0000 7f172ea00640 1 --2- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f173019daa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1720009940 tx=0x7f1720032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.793+0000 7f1734a00640 1 -- 172.21.15.46:0/2115633135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1720042070 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.793+0000 7f1734a00640 1 -- 172.21.15.46:0/2115633135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17200308c0 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.793+0000 7f1734a00640 1 -- 172.21.15.46:0/2115633135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1720037040 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.793+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173019e270 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.793+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301a0f60 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.794+0000 7f1734a00640 1 -- 172.21.15.46:0/2115633135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1720045070 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.794+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173010deb0 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.794+0000 7f1734a00640 1 --2- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f1714038fc0 0x7f171403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.794+0000 7f1734a00640 1 -- 172.21.15.46:0/2115633135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f172006c0a0 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.794+0000 7f172e000640 1 --2- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f1714038fc0 0x7f171403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.795+0000 7f172e000640 1 --2- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f1714038fc0 0x7f171403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f171c0098c0 tx=0x7f171c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.799+0000 7f1734a00640 1 -- 172.21.15.46:0/2115633135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f172006b9f0 con 0x7f1730109870 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:49.987+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1730062630 con 0x7f1714038fc0 2024-09-10T07:25:51.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.337+0000 7f1734a00640 1 -- 172.21.15.46:0/2115633135 <== mgr.14118 v2:172.21.15.46:6800/2972952762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1730062630 con 0x7f1714038fc0 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.339+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f1714038fc0 msgr2=0x7f171403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.339+0000 7f1735e00640 1 --2- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f1714038fc0 0x7f171403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f171c0098c0 tx=0x7f171c006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.339+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 msgr2=0x7f173019daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.339+0000 7f1735e00640 1 --2- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f173019daa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1720009940 tx=0x7f1720032ce0 comp rx=0 tx=0).stop 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.340+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 shutdown_connections 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.340+0000 7f1735e00640 1 --2- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f1714038fc0 0x7f171403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.340+0000 7f1735e00640 1 --2- 172.21.15.46:0/2115633135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f173019daa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.340+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 >> 172.21.15.46:0/2115633135 conn(0x7f173007cdc0 msgr2=0x7f17301076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.340+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 shutdown_connections 2024-09-10T07:25:51.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:50.340+0000 7f1735e00640 1 -- 172.21.15.46:0/2115633135 wait complete. 2024-09-10T07:25:51.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:51 smithi046 ceph-mon[30705]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:25:51.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:51 smithi046 ceph-mon[30705]: Generating ssh key... 2024-09-10T07:25:51.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:51 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:25:51.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:51 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:25:52.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCQRy7LsG7PifQiff+CekI70KyfAJ2HcOit9fQY7KoyAmHZWgW64isMuUqeuax9d5iybNrBUehoxWm5jh8/F9riV2jif2dJNzr/jpyK/oNS6GKTPzUIge5ftOfGJKkvk2u+NP1/yV7UZ+T2+fk82OqUM6X37P7j2L7z5oWJZs9nX+HUi9jZKVukEIm/+yrfOTGi4zzKXFf9VNdHlSfC2E30TXftm8Q/7tvRVuVLWrRVREwF+iNkqI9VRoR6NNoLfO5WrDU7i88xWOgkrRUOrVC5bCOCwZTSm8h9qiErfjtZCxoQeMhRfRA0o7z/TVgXVCGLeo3KwQPgi0LPTpZ5qTLqMjWigvXt9baw/6BJPq/jKL9R2nkeZK/BSv7XjCHmaato9XIICm21CHLBBWuX0Xizufnu+AQwbahR2mQyExS/4s3RfGT5S500Q6sYbu9S0v7FZCDQ+XqKME1AEV1fE52GKQ839anKFS05SkkGqekfouGFhULA+xyfz9wer/PXq8= ceph-b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:25:52.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.532+0000 7fac6f200640 1 Processor -- start 2024-09-10T07:25:52.327 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.532+0000 7fac6f200640 1 -- start start 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.533+0000 7fac6f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac70109d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.533+0000 7fac6f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac7010a2d0 con 0x7fac70109900 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.533+0000 7fac6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac70109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.533+0000 7fac6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac70109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48892/0 (socket says 172.21.15.46:48892) 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.533+0000 7fac6de00640 1 -- 172.21.15.46:0/1375627812 learned_addr learned my addr 172.21.15.46:0/1375627812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.533+0000 7fac6de00640 1 -- 172.21.15.46:0/1375627812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7010a7a0 con 0x7fac70109900 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.534+0000 7fac6de00640 1 --2- 172.21.15.46:0/1375627812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac70109d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac60009820 tx=0x7fac6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84630fec75caf334 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.535+0000 7fac6ca00640 1 -- 172.21.15.46:0/1375627812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac60037070 con 0x7fac70109900 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.535+0000 7fac6ca00640 1 -- 172.21.15.46:0/1375627812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac60032900 con 0x7fac70109900 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.535+0000 7fac6ca00640 1 -- 172.21.15.46:0/1375627812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac60030850 con 0x7fac70109900 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.535+0000 7fac6f200640 1 -- 172.21.15.46:0/1375627812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 msgr2=0x7fac70109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.535+0000 7fac6f200640 1 --2- 172.21.15.46:0/1375627812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac70109d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac60009820 tx=0x7fac6002a270 comp rx=0 tx=0).stop 2024-09-10T07:25:52.328 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.535+0000 7fac6f200640 1 -- 172.21.15.46:0/1375627812 shutdown_connections 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.535+0000 7fac6f200640 1 --2- 172.21.15.46:0/1375627812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac70109d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.535+0000 7fac6f200640 1 -- 172.21.15.46:0/1375627812 >> 172.21.15.46:0/1375627812 conn(0x7fac7007cb40 msgr2=0x7fac7007cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.536+0000 7fac6f200640 1 -- 172.21.15.46:0/1375627812 shutdown_connections 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.536+0000 7fac6f200640 1 -- 172.21.15.46:0/1375627812 wait complete. 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.537+0000 7fac6f200640 1 Processor -- start 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.537+0000 7fac6f200640 1 -- start start 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.537+0000 7fac6f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac7019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.537+0000 7fac6f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac7010db50 con 0x7fac70109900 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.537+0000 7fac6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac7019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.537+0000 7fac6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac7019d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48904/0 (socket says 172.21.15.46:48904) 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.537+0000 7fac6de00640 1 -- 172.21.15.46:0/609540496 learned_addr learned my addr 172.21.15.46:0/609540496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.538+0000 7fac6de00640 1 -- 172.21.15.46:0/609540496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7019ddc0 con 0x7fac70109900 2024-09-10T07:25:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.538+0000 7fac6de00640 1 --2- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac7019d880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fac600097f0 tx=0x7fac60030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.538+0000 7fac66a00640 1 -- 172.21.15.46:0/609540496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac60037040 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.538+0000 7fac66a00640 1 -- 172.21.15.46:0/609540496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac60040070 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.538+0000 7fac66a00640 1 -- 172.21.15.46:0/609540496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6003a390 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.538+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7019e050 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.538+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac701a0d40 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.539+0000 7fac66a00640 1 -- 172.21.15.46:0/609540496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fac6003b430 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.539+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac70109d80 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.540+0000 7fac66a00640 1 --2- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fac54038f70 0x7fac5403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.540+0000 7fac66a00640 1 -- 172.21.15.46:0/609540496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac6006c510 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.540+0000 7fac6d400640 1 --2- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fac54038f70 0x7fac5403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.541+0000 7fac6d400640 1 --2- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fac54038f70 0x7fac5403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0098c0 tx=0x7fac5c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.546+0000 7fac66a00640 1 -- 172.21.15.46:0/609540496 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac60032dd0 con 0x7fac70109900 2024-09-10T07:25:52.330 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.731+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fac7002d830 con 0x7fac54038f70 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.732+0000 7fac66a00640 1 -- 172.21.15.46:0/609540496 <== mgr.14118 v2:172.21.15.46:6800/2972952762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fac7002d830 con 0x7fac54038f70 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.734+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fac54038f70 msgr2=0x7fac5403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.734+0000 7fac6f200640 1 --2- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fac54038f70 0x7fac5403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0098c0 tx=0x7fac5c006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.734+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 msgr2=0x7fac7019d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.734+0000 7fac6f200640 1 --2- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac7019d880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fac600097f0 tx=0x7fac60030d80 comp rx=0 tx=0).stop 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.735+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 shutdown_connections 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.735+0000 7fac6f200640 1 --2- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7fac54038f70 0x7fac5403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.735+0000 7fac6f200640 1 --2- 172.21.15.46:0/609540496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac70109900 0x7fac7019d880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.735+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 >> 172.21.15.46:0/609540496 conn(0x7fac7007cb40 msgr2=0x7fac701075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.735+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 shutdown_connections 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:51.735+0000 7fac6f200640 1 -- 172.21.15.46:0/609540496 wait complete. 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T07:25:52.331 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T07:25:52.332 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-09-10T07:25:52.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:52 smithi046 ceph-mon[30705]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:25:54.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:53 smithi046 ceph-mon[30705]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:25:55.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:54 smithi046 ceph-mon[30705]: Deploying cephadm binary to smithi046 2024-09-10T07:25:58.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-09-10T07:25:58.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.814+0000 7f42da000640 1 Processor -- start 2024-09-10T07:25:58.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.814+0000 7f42da000640 1 -- start start 2024-09-10T07:25:58.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.815+0000 7f42da000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d407c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.815+0000 7f42da000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42d407c7f0 con 0x7f42d407de00 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.815+0000 7f42d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d407c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.815+0000 7f42d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d407c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48916/0 (socket says 172.21.15.46:48916) 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.815+0000 7f42d3400640 1 -- 172.21.15.46:0/1787281088 learned_addr learned my addr 172.21.15.46:0/1787281088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.815+0000 7f42d3400640 1 -- 172.21.15.46:0/1787281088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d407c970 con 0x7f42d407de00 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.816+0000 7f42d3400640 1 --2- 172.21.15.46:0/1787281088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d407c2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009820 tx=0x7f42c402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5565c7dfc4ef3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.817+0000 7f42d2a00640 1 -- 172.21.15.46:0/1787281088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42c4037070 con 0x7f42d407de00 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.817+0000 7f42d2a00640 1 -- 172.21.15.46:0/1787281088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42c4032900 con 0x7f42d407de00 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.817+0000 7f42d2a00640 1 -- 172.21.15.46:0/1787281088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42c40308e0 con 0x7f42d407de00 2024-09-10T07:25:58.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.817+0000 7f42da000640 1 -- 172.21.15.46:0/1787281088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 msgr2=0x7f42d407c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:58.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.817+0000 7f42da000640 1 --2- 172.21.15.46:0/1787281088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d407c2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009820 tx=0x7f42c402a300 comp rx=0 tx=0).stop 2024-09-10T07:25:58.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.817+0000 7f42da000640 1 -- 172.21.15.46:0/1787281088 shutdown_connections 2024-09-10T07:25:58.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.817+0000 7f42da000640 1 --2- 172.21.15.46:0/1787281088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d407c2b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:58.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.817+0000 7f42da000640 1 -- 172.21.15.46:0/1787281088 >> 172.21.15.46:0/1787281088 conn(0x7f42d4102d20 msgr2=0x7f42d4105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:58.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.818+0000 7f42da000640 1 -- 172.21.15.46:0/1787281088 shutdown_connections 2024-09-10T07:25:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.818+0000 7f42da000640 1 -- 172.21.15.46:0/1787281088 wait complete. 2024-09-10T07:25:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.819+0000 7f42da000640 1 Processor -- start 2024-09-10T07:25:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.819+0000 7f42da000640 1 -- start start 2024-09-10T07:25:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.819+0000 7f42da000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.819+0000 7f42da000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42d410dba0 con 0x7f42d407de00 2024-09-10T07:25:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.819+0000 7f42d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.819+0000 7f42d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48922/0 (socket says 172.21.15.46:48922) 2024-09-10T07:25:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.819+0000 7f42d3400640 1 -- 172.21.15.46:0/1659693539 learned_addr learned my addr 172.21.15.46:0/1659693539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.820+0000 7f42d3400640 1 -- 172.21.15.46:0/1659693539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d419dfa0 con 0x7f42d407de00 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.820+0000 7f42d3400640 1 --2- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d419da60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42c402a820 tx=0x7f42c4030900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.820+0000 7f42d0c00640 1 -- 172.21.15.46:0/1659693539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42c4037040 con 0x7f42d407de00 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.820+0000 7f42d0c00640 1 -- 172.21.15.46:0/1659693539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42c4040070 con 0x7f42d407de00 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.820+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d419e230 con 0x7f42d407de00 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.820+0000 7f42d0c00640 1 -- 172.21.15.46:0/1659693539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42c403a390 con 0x7f42d407de00 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.820+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d41a0f20 con 0x7f42d407de00 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.821+0000 7f42d0c00640 1 -- 172.21.15.46:0/1659693539 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f42c4030b70 con 0x7f42d407de00 2024-09-10T07:25:58.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.821+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d407c330 con 0x7f42d407de00 2024-09-10T07:25:58.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.822+0000 7f42d0c00640 1 --2- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f42b8038fc0 0x7f42b803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:58.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.822+0000 7f42d0c00640 1 -- 172.21.15.46:0/1659693539 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f42c406c1d0 con 0x7f42d407de00 2024-09-10T07:25:58.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.822+0000 7f42caa00640 1 --2- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f42b8038fc0 0x7f42b803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:58.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.823+0000 7f42caa00640 1 --2- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f42b8038fc0 0x7f42b803b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0098c0 tx=0x7f42bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:58.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:52.827+0000 7f42d0c00640 1 -- 172.21.15.46:0/1659693539 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42c406bb20 con 0x7f42d407de00 2024-09-10T07:25:58.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:53.019+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}) -- 0x7f42d4062630 con 0x7f42b8038fc0 2024-09-10T07:25:58.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.557+0000 7f42d0c00640 1 -- 172.21.15.46:0/1659693539 <== mgr.14118 v2:172.21.15.46:6800/2972952762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f42d4062630 con 0x7f42b8038fc0 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.560+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f42b8038fc0 msgr2=0x7f42b803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.560+0000 7f42da000640 1 --2- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f42b8038fc0 0x7f42b803b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0098c0 tx=0x7f42bc006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.560+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 msgr2=0x7f42d419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.560+0000 7f42da000640 1 --2- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d419da60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42c402a820 tx=0x7f42c4030900 comp rx=0 tx=0).stop 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.560+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 shutdown_connections 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.560+0000 7f42da000640 1 --2- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f42b8038fc0 0x7f42b803b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.560+0000 7f42da000640 1 --2- 172.21.15.46:0/1659693539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d407de00 0x7f42d419da60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.560+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 >> 172.21.15.46:0/1659693539 conn(0x7f42d4102d20 msgr2=0x7f42d4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:58.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.561+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 shutdown_connections 2024-09-10T07:25:58.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:57.561+0000 7f42da000640 1 -- 172.21.15.46:0/1659693539 wait complete. 2024-09-10T07:25:58.242 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mon service... 2024-09-10T07:25:58.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:58 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:25:58.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:58 smithi046 ceph-mon[30705]: Added host smithi046 2024-09-10T07:25:58.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:58 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:25:59.763 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T07:25:59.764 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.718+0000 7f5861a00640 1 Processor -- start 2024-09-10T07:25:59.764 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.718+0000 7f5861a00640 1 -- start start 2024-09-10T07:25:59.764 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.719+0000 7f5861a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:59.764 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.719+0000 7f5861a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f585c07c8c0 con 0x7f585c07de90 2024-09-10T07:25:59.764 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.719+0000 7f585aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:59.764 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.719+0000 7f585aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40062/0 (socket says 172.21.15.46:40062) 2024-09-10T07:25:59.765 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.719+0000 7f585aa00640 1 -- 172.21.15.46:0/3738035413 learned_addr learned my addr 172.21.15.46:0/3738035413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:59.765 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.720+0000 7f585aa00640 1 -- 172.21.15.46:0/3738035413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c07ca40 con 0x7f585c07de90 2024-09-10T07:25:59.765 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.721+0000 7f585aa00640 1 --2- 172.21.15.46:0/3738035413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c07c2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f584c009820 tx=0x7f584c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd922338946987 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:59.765 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.721+0000 7f5859600640 1 -- 172.21.15.46:0/3738035413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f584c037070 con 0x7f585c07de90 2024-09-10T07:25:59.765 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.721+0000 7f5859600640 1 -- 172.21.15.46:0/3738035413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f584c032900 con 0x7f585c07de90 2024-09-10T07:25:59.765 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.721+0000 7f5859600640 1 -- 172.21.15.46:0/3738035413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f584c030850 con 0x7f585c07de90 2024-09-10T07:25:59.766 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.721+0000 7f5861a00640 1 -- 172.21.15.46:0/3738035413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 msgr2=0x7f585c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:59.766 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.721+0000 7f5861a00640 1 --2- 172.21.15.46:0/3738035413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c07c2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f584c009820 tx=0x7f584c02a270 comp rx=0 tx=0).stop 2024-09-10T07:25:59.766 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.722+0000 7f5861a00640 1 -- 172.21.15.46:0/3738035413 shutdown_connections 2024-09-10T07:25:59.766 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.722+0000 7f5861a00640 1 --2- 172.21.15.46:0/3738035413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c07c2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:59.766 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.722+0000 7f5861a00640 1 -- 172.21.15.46:0/3738035413 >> 172.21.15.46:0/3738035413 conn(0x7f585c102d90 msgr2=0x7f585c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:59.766 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.722+0000 7f5861a00640 1 -- 172.21.15.46:0/3738035413 shutdown_connections 2024-09-10T07:25:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.722+0000 7f5861a00640 1 -- 172.21.15.46:0/3738035413 wait complete. 2024-09-10T07:25:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.723+0000 7f5861a00640 1 Processor -- start 2024-09-10T07:25:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.723+0000 7f5861a00640 1 -- start start 2024-09-10T07:25:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.723+0000 7f5861a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.723+0000 7f5861a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f585c1097a0 con 0x7f585c07de90 2024-09-10T07:25:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.723+0000 7f585aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.723+0000 7f585aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c1a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40068/0 (socket says 172.21.15.46:40068) 2024-09-10T07:25:59.768 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.723+0000 7f585aa00640 1 -- 172.21.15.46:0/3724950653 learned_addr learned my addr 172.21.15.46:0/3724950653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:25:59.768 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.724+0000 7f585aa00640 1 -- 172.21.15.46:0/3724950653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c1a23d0 con 0x7f585c07de90 2024-09-10T07:25:59.768 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.724+0000 7f585aa00640 1 --2- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c1a1e90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f584c0097f0 tx=0x7f584c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:59.768 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.724+0000 7f5853400640 1 -- 172.21.15.46:0/3724950653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f584c037040 con 0x7f585c07de90 2024-09-10T07:25:59.768 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.724+0000 7f5853400640 1 -- 172.21.15.46:0/3724950653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f584c040070 con 0x7f585c07de90 2024-09-10T07:25:59.768 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.724+0000 7f5853400640 1 -- 172.21.15.46:0/3724950653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f584c03a390 con 0x7f585c07de90 2024-09-10T07:25:59.769 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.724+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c1a2660 con 0x7f585c07de90 2024-09-10T07:25:59.769 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.724+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c1a5350 con 0x7f585c07de90 2024-09-10T07:25:59.769 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.725+0000 7f5853400640 1 -- 172.21.15.46:0/3724950653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f584c03b430 con 0x7f585c07de90 2024-09-10T07:25:59.769 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.725+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c07c370 con 0x7f585c07de90 2024-09-10T07:25:59.769 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.726+0000 7f5853400640 1 --2- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f5840038fc0 0x7f584003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:25:59.769 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.726+0000 7f5853400640 1 -- 172.21.15.46:0/3724950653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f584c06c420 con 0x7f585c07de90 2024-09-10T07:25:59.770 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.726+0000 7f585a000640 1 --2- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f5840038fc0 0x7f584003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:25:59.770 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.727+0000 7f585a000640 1 --2- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f5840038fc0 0x7f584003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58480098c0 tx=0x7f5848006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:25:59.770 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.731+0000 7f5853400640 1 -- 172.21.15.46:0/3724950653 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f584c032900 con 0x7f585c07de90 2024-09-10T07:25:59.770 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.931+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f585c02d830 con 0x7f5840038fc0 2024-09-10T07:25:59.770 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.937+0000 7f5853400640 1 -- 172.21.15.46:0/3724950653 <== mgr.14118 v2:172.21.15.46:6800/2972952762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f585c02d830 con 0x7f5840038fc0 2024-09-10T07:25:59.770 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f5840038fc0 msgr2=0x7f584003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:59.771 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 --2- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f5840038fc0 0x7f584003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58480098c0 tx=0x7f5848006db0 comp rx=0 tx=0).stop 2024-09-10T07:25:59.771 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 msgr2=0x7f585c1a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:25:59.771 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 --2- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c1a1e90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f584c0097f0 tx=0x7f584c030d80 comp rx=0 tx=0).stop 2024-09-10T07:25:59.771 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 shutdown_connections 2024-09-10T07:25:59.771 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 --2- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f5840038fc0 0x7f584003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:59.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 --2- 172.21.15.46:0/3724950653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f585c07de90 0x7f585c1a1e90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:25:59.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 >> 172.21.15.46:0/3724950653 conn(0x7f585c102d90 msgr2=0x7f585c198870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:25:59.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 shutdown_connections 2024-09-10T07:25:59.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:25:58.939+0000 7f5861a00640 1 -- 172.21.15.46:0/3724950653 wait complete. 2024-09-10T07:25:59.772 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mgr service... 2024-09-10T07:26:00.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:59 smithi046 ceph-mon[30705]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:00.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:59 smithi046 ceph-mon[30705]: Saving service mon spec with placement count:5 2024-09-10T07:26:00.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:25:59 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:26:01.631 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T07:26:01.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.673+0000 7faafac00640 1 Processor -- start 2024-09-10T07:26:01.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.673+0000 7faafac00640 1 -- start start 2024-09-10T07:26:01.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.673+0000 7faafac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf4109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.673+0000 7faafac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410a240 con 0x7faaf4109870 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.673+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf4109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.674+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf4109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40076/0 (socket says 172.21.15.46:40076) 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.674+0000 7faaf3e00640 1 -- 172.21.15.46:0/1930316545 learned_addr learned my addr 172.21.15.46:0/1930316545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.674+0000 7faaf3e00640 1 -- 172.21.15.46:0/1930316545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf410a980 con 0x7faaf4109870 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.675+0000 7faaf3e00640 1 --2- 172.21.15.46:0/1930316545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf4109c70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faae4009820 tx=0x7faae402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dba15d030ff14c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.675+0000 7faaf2a00640 1 -- 172.21.15.46:0/1930316545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4037070 con 0x7faaf4109870 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.675+0000 7faaf2a00640 1 -- 172.21.15.46:0/1930316545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faae4032900 con 0x7faaf4109870 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.675+0000 7faaf2a00640 1 -- 172.21.15.46:0/1930316545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4030850 con 0x7faaf4109870 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.676+0000 7faafac00640 1 -- 172.21.15.46:0/1930316545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 msgr2=0x7faaf4109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.676+0000 7faafac00640 1 --2- 172.21.15.46:0/1930316545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf4109c70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faae4009820 tx=0x7faae402a270 comp rx=0 tx=0).stop 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.676+0000 7faafac00640 1 -- 172.21.15.46:0/1930316545 shutdown_connections 2024-09-10T07:26:01.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.676+0000 7faafac00640 1 --2- 172.21.15.46:0/1930316545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf4109c70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.676+0000 7faafac00640 1 -- 172.21.15.46:0/1930316545 >> 172.21.15.46:0/1930316545 conn(0x7faaf407cdc0 msgr2=0x7faaf407d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.676+0000 7faafac00640 1 -- 172.21.15.46:0/1930316545 shutdown_connections 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.676+0000 7faafac00640 1 -- 172.21.15.46:0/1930316545 wait complete. 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.677+0000 7faafac00640 1 Processor -- start 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.677+0000 7faafac00640 1 -- start start 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.677+0000 7faafac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.677+0000 7faafac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410dd30 con 0x7faaf4109870 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.678+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.678+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf419da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40090/0 (socket says 172.21.15.46:40090) 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.678+0000 7faaf3e00640 1 -- 172.21.15.46:0/1264972083 learned_addr learned my addr 172.21.15.46:0/1264972083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.678+0000 7faaf3e00640 1 -- 172.21.15.46:0/1264972083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf419dfc0 con 0x7faaf4109870 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.678+0000 7faaf3e00640 1 --2- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf419da80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faae40097f0 tx=0x7faae4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.678+0000 7faaf0c00640 1 -- 172.21.15.46:0/1264972083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4037040 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.679+0000 7faaf0c00640 1 -- 172.21.15.46:0/1264972083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faae4040070 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.679+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf419e250 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.679+0000 7faaf0c00640 1 -- 172.21.15.46:0/1264972083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae403a390 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.679+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a0f40 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.680+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf410deb0 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.680+0000 7faaf0c00640 1 -- 172.21.15.46:0/1264972083 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7faae403b430 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.680+0000 7faaf0c00640 1 --2- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7faad8038fc0 0x7faad803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.680+0000 7faaf0c00640 1 -- 172.21.15.46:0/1264972083 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faae406c4e0 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.681+0000 7faaf3400640 1 --2- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7faad8038fc0 0x7faad803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.681+0000 7faaf3400640 1 --2- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7faad8038fc0 0x7faad803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faae00098c0 tx=0x7faae0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.686+0000 7faaf0c00640 1 -- 172.21.15.46:0/1264972083 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faae406be30 con 0x7faaf4109870 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.882+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7faaf4062630 con 0x7faad8038fc0 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.886+0000 7faaf0c00640 1 -- 172.21.15.46:0/1264972083 <== mgr.14118 v2:172.21.15.46:6800/2972952762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faaf4062630 con 0x7faad8038fc0 2024-09-10T07:26:01.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7faad8038fc0 msgr2=0x7faad803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 --2- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7faad8038fc0 0x7faad803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faae00098c0 tx=0x7faae0006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 msgr2=0x7faaf419da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 --2- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf419da80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faae40097f0 tx=0x7faae4030d80 comp rx=0 tx=0).stop 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 shutdown_connections 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 --2- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7faad8038fc0 0x7faad803b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 --2- 172.21.15.46:0/1264972083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaf4109870 0x7faaf419da80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 >> 172.21.15.46:0/1264972083 conn(0x7faaf407cdc0 msgr2=0x7faaf4107670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 shutdown_connections 2024-09-10T07:26:01.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:00.889+0000 7faafac00640 1 -- 172.21.15.46:0/1264972083 wait complete. 2024-09-10T07:26:02.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:01 smithi046 ceph-mon[30705]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:02.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:01 smithi046 ceph-mon[30705]: Saving service mgr spec with placement count:2 2024-09-10T07:26:02.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:01 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:26:03.335 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:03 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:26:03.335 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:03 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:26:03.335 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:03 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2793317523' entity='client.admin' 2024-09-10T07:26:03.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.487+0000 7f46c0c00640 1 Processor -- start 2024-09-10T07:26:03.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.487+0000 7f46c0c00640 1 -- start start 2024-09-10T07:26:03.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.488+0000 7f46c0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:03.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.488+0000 7f46c0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc07c8c0 con 0x7f46bc07de90 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.488+0000 7f46b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.488+0000 7f46b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40094/0 (socket says 172.21.15.46:40094) 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.488+0000 7f46b9e00640 1 -- 172.21.15.46:0/2984815078 learned_addr learned my addr 172.21.15.46:0/2984815078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.488+0000 7f46b9e00640 1 -- 172.21.15.46:0/2984815078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc07ca40 con 0x7f46bc07de90 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.489+0000 7f46b9e00640 1 --2- 172.21.15.46:0/2984815078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc07c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009820 tx=0x7f46ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5ea79ce12428fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.490+0000 7f46b8a00640 1 -- 172.21.15.46:0/2984815078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ac037070 con 0x7f46bc07de90 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.490+0000 7f46b8a00640 1 -- 172.21.15.46:0/2984815078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac032920 con 0x7f46bc07de90 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.490+0000 7f46b8a00640 1 -- 172.21.15.46:0/2984815078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ac030810 con 0x7f46bc07de90 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.490+0000 7f46c0c00640 1 -- 172.21.15.46:0/2984815078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 msgr2=0x7f46bc07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.490+0000 7f46c0c00640 1 --2- 172.21.15.46:0/2984815078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc07c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009820 tx=0x7f46ac02a270 comp rx=0 tx=0).stop 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.491+0000 7f46c0c00640 1 -- 172.21.15.46:0/2984815078 shutdown_connections 2024-09-10T07:26:03.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.491+0000 7f46c0c00640 1 --2- 172.21.15.46:0/2984815078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc07c2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.491+0000 7f46c0c00640 1 -- 172.21.15.46:0/2984815078 >> 172.21.15.46:0/2984815078 conn(0x7f46bc102d90 msgr2=0x7f46bc1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.491+0000 7f46c0c00640 1 -- 172.21.15.46:0/2984815078 shutdown_connections 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.491+0000 7f46c0c00640 1 -- 172.21.15.46:0/2984815078 wait complete. 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.492+0000 7f46c0c00640 1 Processor -- start 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.492+0000 7f46c0c00640 1 -- start start 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.493+0000 7f46c0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.493+0000 7f46c0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc109800 con 0x7f46bc07de90 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.493+0000 7f46b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.493+0000 7f46b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc1a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40104/0 (socket says 172.21.15.46:40104) 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.493+0000 7f46b9e00640 1 -- 172.21.15.46:0/2793317523 learned_addr learned my addr 172.21.15.46:0/2793317523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.493+0000 7f46b9e00640 1 -- 172.21.15.46:0/2793317523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc1a23c0 con 0x7f46bc07de90 2024-09-10T07:26:03.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.494+0000 7f46b9e00640 1 --2- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc1a1e80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0097f0 tx=0x7f46ac0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.494+0000 7f46b2a00640 1 -- 172.21.15.46:0/2793317523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ac037040 con 0x7f46bc07de90 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.494+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc1a2650 con 0x7f46bc07de90 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.494+0000 7f46b2a00640 1 -- 172.21.15.46:0/2793317523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac030cc0 con 0x7f46bc07de90 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.494+0000 7f46b2a00640 1 -- 172.21.15.46:0/2793317523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ac03a640 con 0x7f46bc07de90 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.494+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc1a5340 con 0x7f46bc07de90 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.495+0000 7f46b2a00640 1 -- 172.21.15.46:0/2793317523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f46ac03a8f0 con 0x7f46bc07de90 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.495+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46bc07c370 con 0x7f46bc07de90 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.496+0000 7f46b2a00640 1 --2- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f46a0038fc0 0x7f46a003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.496+0000 7f46b2a00640 1 -- 172.21.15.46:0/2793317523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f46ac06c720 con 0x7f46bc07de90 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.496+0000 7f46b9400640 1 --2- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f46a0038fc0 0x7f46a003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.497+0000 7f46b9400640 1 --2- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f46a0038fc0 0x7f46a003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f46a80098c0 tx=0x7f46a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.501+0000 7f46b2a00640 1 -- 172.21.15.46:0/2793317523 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46ac0328c0 con 0x7f46bc07de90 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.689+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f46bc02d7c0 con 0x7f46bc07de90 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.692+0000 7f46b2a00640 1 -- 172.21.15.46:0/2793317523 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f46ac031990 con 0x7f46bc07de90 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f46a0038fc0 msgr2=0x7f46a003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 --2- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f46a0038fc0 0x7f46a003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f46a80098c0 tx=0x7f46a8006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 msgr2=0x7f46bc1a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 --2- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc1a1e80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0097f0 tx=0x7f46ac0327b0 comp rx=0 tx=0).stop 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 shutdown_connections 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 --2- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f46a0038fc0 0x7f46a003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 --2- 172.21.15.46:0/2793317523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46bc07de90 0x7f46bc1a1e80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 >> 172.21.15.46:0/2793317523 conn(0x7f46bc102d90 msgr2=0x7f46bc198870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 shutdown_connections 2024-09-10T07:26:03.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:02.695+0000 7f46c0c00640 1 -- 172.21.15.46:0/2793317523 wait complete. 2024-09-10T07:26:05.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:05 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3850740739' entity='client.admin' 2024-09-10T07:26:05.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:05 smithi046 ceph-mon[30705]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:05.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:05 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.832+0000 7f39ada00640 1 Processor -- start 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.832+0000 7f39ada00640 1 -- start start 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.832+0000 7f39ada00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a8109d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.832+0000 7f39ada00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39a810a2f0 con 0x7f39a8109920 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.833+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a8109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.833+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a8109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40118/0 (socket says 172.21.15.46:40118) 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.833+0000 7f39a7e00640 1 -- 172.21.15.46:0/4257157105 learned_addr learned my addr 172.21.15.46:0/4257157105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.833+0000 7f39a7e00640 1 -- 172.21.15.46:0/4257157105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a810a780 con 0x7f39a8109920 2024-09-10T07:26:05.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.834+0000 7f39a7e00640 1 --2- 172.21.15.46:0/4257157105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a8109d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3998009820 tx=0x7f399802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937b6d5c4aa65510 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.835+0000 7f39a6a00640 1 -- 172.21.15.46:0/4257157105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3998037070 con 0x7f39a8109920 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.835+0000 7f39a6a00640 1 -- 172.21.15.46:0/4257157105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3998032920 con 0x7f39a8109920 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.835+0000 7f39a6a00640 1 -- 172.21.15.46:0/4257157105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3998030810 con 0x7f39a8109920 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.836+0000 7f39ada00640 1 -- 172.21.15.46:0/4257157105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 msgr2=0x7f39a8109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.836+0000 7f39ada00640 1 --2- 172.21.15.46:0/4257157105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a8109d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3998009820 tx=0x7f399802a270 comp rx=0 tx=0).stop 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.836+0000 7f39ada00640 1 -- 172.21.15.46:0/4257157105 shutdown_connections 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.836+0000 7f39ada00640 1 --2- 172.21.15.46:0/4257157105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a8109d20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.836+0000 7f39ada00640 1 -- 172.21.15.46:0/4257157105 >> 172.21.15.46:0/4257157105 conn(0x7f39a807cbe0 msgr2=0x7f39a807cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.836+0000 7f39ada00640 1 -- 172.21.15.46:0/4257157105 shutdown_connections 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.836+0000 7f39ada00640 1 -- 172.21.15.46:0/4257157105 wait complete. 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.837+0000 7f39ada00640 1 Processor -- start 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.837+0000 7f39ada00640 1 -- start start 2024-09-10T07:26:05.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.837+0000 7f39ada00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.837+0000 7f39ada00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39a810db30 con 0x7f39a8109920 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.838+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.838+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40128/0 (socket says 172.21.15.46:40128) 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.838+0000 7f39a7e00640 1 -- 172.21.15.46:0/3850740739 learned_addr learned my addr 172.21.15.46:0/3850740739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.838+0000 7f39a7e00640 1 -- 172.21.15.46:0/3850740739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a819de40 con 0x7f39a8109920 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.838+0000 7f39a7e00640 1 --2- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a819d900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3998030560 tx=0x7f3998032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.839+0000 7f39a4c00640 1 -- 172.21.15.46:0/3850740739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3998037040 con 0x7f39a8109920 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.839+0000 7f39a4c00640 1 -- 172.21.15.46:0/3850740739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3998040070 con 0x7f39a8109920 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.839+0000 7f39a4c00640 1 -- 172.21.15.46:0/3850740739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f399803a740 con 0x7f39a8109920 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.839+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a819e0d0 con 0x7f39a8109920 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.839+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a81a0dc0 con 0x7f39a8109920 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.840+0000 7f39a4c00640 1 -- 172.21.15.46:0/3850740739 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f399803bb90 con 0x7f39a8109920 2024-09-10T07:26:05.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.840+0000 7f39a4c00640 1 --2- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f398c038fc0 0x7f398c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.840+0000 7f39a4c00640 1 -- 172.21.15.46:0/3850740739 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f399806c600 con 0x7f39a8109920 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.840+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a8109da0 con 0x7f39a8109920 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.841+0000 7f39a7400640 1 --2- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f398c038fc0 0x7f398c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.841+0000 7f39a7400640 1 --2- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f398c038fc0 0x7f398c03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f39940098c0 tx=0x7f3994006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:03.846+0000 7f39a4c00640 1 -- 172.21.15.46:0/3850740739 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f399804a200 con 0x7f39a8109920 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.040+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f39a802d7c0 con 0x7f39a8109920 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.043+0000 7f39a4c00640 1 -- 172.21.15.46:0/3850740739 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3998031e80 con 0x7f39a8109920 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.045+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f398c038fc0 msgr2=0x7f398c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.045+0000 7f39ada00640 1 --2- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f398c038fc0 0x7f398c03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f39940098c0 tx=0x7f3994006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.045+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 msgr2=0x7f39a819d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.045+0000 7f39ada00640 1 --2- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a819d900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3998030560 tx=0x7f3998032670 comp rx=0 tx=0).stop 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.046+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 shutdown_connections 2024-09-10T07:26:05.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.046+0000 7f39ada00640 1 --2- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f398c038fc0 0x7f398c03b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:05.393 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.046+0000 7f39ada00640 1 --2- 172.21.15.46:0/3850740739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a8109920 0x7f39a819d900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:05.393 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.046+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 >> 172.21.15.46:0/3850740739 conn(0x7f39a807cbe0 msgr2=0x7f39a8107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:05.393 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.046+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 shutdown_connections 2024-09-10T07:26:05.393 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:04.046+0000 7f39ada00640 1 -- 172.21.15.46:0/3850740739 wait complete. 2024-09-10T07:26:05.393 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-09-10T07:26:06.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:06 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3315723486' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T07:26:06.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:06 smithi046 ceph-mon[30705]: from='mgr.14118 172.21.15.46:0/3403943271' entity='mgr.x' 2024-09-10T07:26:06.498 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: ignoring --setuser ceph since I am not root 2024-09-10T07:26:06.498 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: ignoring --setgroup ceph since I am not root 2024-09-10T07:26:06.498 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:06.497+0000 7f757e42d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:26:06.760 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:06.569+0000 7f757e42d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:26:07.038 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:06.776+0000 7f757e42d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:26:07.039 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:06.860+0000 7f757e42d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:26:07.039 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:07.038+0000 7f757e42d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:26:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.933+0000 7f8297200640 1 Processor -- start 2024-09-10T07:26:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.934+0000 7f8297200640 1 -- start start 2024-09-10T07:26:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.934+0000 7f8297200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f8290107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.934+0000 7f8297200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8290107f50 con 0x7f8290107580 2024-09-10T07:26:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.935+0000 7f828fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f8290107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.935+0000 7f828fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f8290107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32926/0 (socket says 172.21.15.46:32926) 2024-09-10T07:26:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.935+0000 7f828fe00640 1 -- 172.21.15.46:0/558129571 learned_addr learned my addr 172.21.15.46:0/558129571 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.935+0000 7f828fe00640 1 -- 172.21.15.46:0/558129571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8290108780 con 0x7f8290107580 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.936+0000 7f828fe00640 1 --2- 172.21.15.46:0/558129571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f8290107980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8280009820 tx=0x7f828002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927d1eba5ee6b337 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f828ea00640 1 -- 172.21.15.46:0/558129571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8280037070 con 0x7f8290107580 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f828ea00640 1 -- 172.21.15.46:0/558129571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8280032920 con 0x7f8290107580 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f828ea00640 1 -- 172.21.15.46:0/558129571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8280030810 con 0x7f8290107580 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f8297200640 1 -- 172.21.15.46:0/558129571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 msgr2=0x7f8290107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f8297200640 1 --2- 172.21.15.46:0/558129571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f8290107980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8280009820 tx=0x7f828002a270 comp rx=0 tx=0).stop 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f8297200640 1 -- 172.21.15.46:0/558129571 shutdown_connections 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f8297200640 1 --2- 172.21.15.46:0/558129571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f8290107980 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f8297200640 1 -- 172.21.15.46:0/558129571 >> 172.21.15.46:0/558129571 conn(0x7f8290102d90 msgr2=0x7f82901051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:07.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f8297200640 1 -- 172.21.15.46:0/558129571 shutdown_connections 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.937+0000 7f8297200640 1 -- 172.21.15.46:0/558129571 wait complete. 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.938+0000 7f8297200640 1 Processor -- start 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.939+0000 7f8297200640 1 -- start start 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.939+0000 7f8297200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f829007db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.939+0000 7f8297200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f829010bb30 con 0x7f8290107580 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.939+0000 7f828fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f829007db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.939+0000 7f828fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f829007db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32932/0 (socket says 172.21.15.46:32932) 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.939+0000 7f828fe00640 1 -- 172.21.15.46:0/3315723486 learned_addr learned my addr 172.21.15.46:0/3315723486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.939+0000 7f828fe00640 1 -- 172.21.15.46:0/3315723486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829007e060 con 0x7f8290107580 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.940+0000 7f828fe00640 1 --2- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f829007db20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82800097f0 tx=0x7f82800327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.940+0000 7f828cc00640 1 -- 172.21.15.46:0/3315723486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8280037040 con 0x7f8290107580 2024-09-10T07:26:07.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.940+0000 7f828cc00640 1 -- 172.21.15.46:0/3315723486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8280030cc0 con 0x7f8290107580 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.940+0000 7f828cc00640 1 -- 172.21.15.46:0/3315723486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f828003a640 con 0x7f8290107580 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.940+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829007c250 con 0x7f8290107580 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.940+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829007c790 con 0x7f8290107580 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.941+0000 7f828cc00640 1 -- 172.21.15.46:0/3315723486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f828003b3f0 con 0x7f8290107580 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.941+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829010bcb0 con 0x7f8290107580 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.942+0000 7f828cc00640 1 --2- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f8274038fc0 0x7f827403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.942+0000 7f828cc00640 1 -- 172.21.15.46:0/3315723486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f828006c570 con 0x7f8290107580 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.942+0000 7f828f400640 1 --2- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f8274038fc0 0x7f827403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:07.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.943+0000 7f828f400640 1 --2- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f8274038fc0 0x7f827403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098c0 tx=0x7f827c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:07.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:05.947+0000 7f828cc00640 1 -- 172.21.15.46:0/3315723486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8280042e10 con 0x7f8290107580 2024-09-10T07:26:07.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.195+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f8290107a00 con 0x7f8290107580 2024-09-10T07:26:07.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.247+0000 7f828cc00640 1 -- 172.21.15.46:0/3315723486 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f82800328c0 con 0x7f8290107580 2024-09-10T07:26:07.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.248+0000 7f828cc00640 1 -- 172.21.15.46:0/3315723486 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f8280049ca0 con 0x7f8290107580 2024-09-10T07:26:07.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.250+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f8274038fc0 msgr2=0x7f827403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.250+0000 7f8297200640 1 --2- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f8274038fc0 0x7f827403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098c0 tx=0x7f827c006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.250+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 msgr2=0x7f829007db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.250+0000 7f8297200640 1 --2- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f829007db20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82800097f0 tx=0x7f82800327b0 comp rx=0 tx=0).stop 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.251+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 shutdown_connections 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.251+0000 7f8297200640 1 --2- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f8274038fc0 0x7f827403b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.251+0000 7f8297200640 1 --2- 172.21.15.46:0/3315723486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8290107580 0x7f829007db20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.251+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 >> 172.21.15.46:0/3315723486 conn(0x7f8290102d90 msgr2=0x7f8290103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.252+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 shutdown_connections 2024-09-10T07:26:07.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:06.252+0000 7f8297200640 1 -- 172.21.15.46:0/3315723486 wait complete. 2024-09-10T07:26:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:07 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3315723486' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T07:26:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:07 smithi046 ceph-mon[30705]: mgrmap e9: x(active, since 21s) 2024-09-10T07:26:07.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:07.215+0000 7f757e42d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:26:07.622 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:07.601+0000 7f757e42d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:26:07.881 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:07.677+0000 7f757e42d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:26:08.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:07.899+0000 7f757e42d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:26:08.583 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:08 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1071618381' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T07:26:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:26:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T07:26:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T07:26:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.622+0000 7f0a8b000640 1 Processor -- start 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.622+0000 7f0a8b000640 1 -- start start 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.622+0000 7f0a8b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.622+0000 7f0a8b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8407c8a0 con 0x7f0a8407de70 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.623+0000 7f0a89c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.623+0000 7f0a89c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32962/0 (socket says 172.21.15.46:32962) 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.623+0000 7f0a89c00640 1 -- 172.21.15.46:0/1155523501 learned_addr learned my addr 172.21.15.46:0/1155523501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.623+0000 7f0a89c00640 1 -- 172.21.15.46:0/1155523501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8407ca20 con 0x7f0a8407de70 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.624+0000 7f0a89c00640 1 --2- 172.21.15.46:0/1155523501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8407c2d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009820 tx=0x7f0a7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3360eb8b72c559e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.624+0000 7f0a83e00640 1 -- 172.21.15.46:0/1155523501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a78037070 con 0x7f0a8407de70 2024-09-10T07:26:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.625+0000 7f0a83e00640 1 -- 172.21.15.46:0/1155523501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a78032920 con 0x7f0a8407de70 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.625+0000 7f0a83e00640 1 -- 172.21.15.46:0/1155523501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a78030810 con 0x7f0a8407de70 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.625+0000 7f0a8b000640 1 -- 172.21.15.46:0/1155523501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 msgr2=0x7f0a8407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.625+0000 7f0a8b000640 1 --2- 172.21.15.46:0/1155523501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8407c2d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009820 tx=0x7f0a7802a270 comp rx=0 tx=0).stop 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.625+0000 7f0a8b000640 1 -- 172.21.15.46:0/1155523501 shutdown_connections 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.625+0000 7f0a8b000640 1 --2- 172.21.15.46:0/1155523501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8407c2d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.625+0000 7f0a8b000640 1 -- 172.21.15.46:0/1155523501 >> 172.21.15.46:0/1155523501 conn(0x7f0a84102b40 msgr2=0x7f0a84104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.626+0000 7f0a8b000640 1 -- 172.21.15.46:0/1155523501 shutdown_connections 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.626+0000 7f0a8b000640 1 -- 172.21.15.46:0/1155523501 wait complete. 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.626+0000 7f0a8b000640 1 Processor -- start 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.627+0000 7f0a8b000640 1 -- start start 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.627+0000 7f0a8b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.627+0000 7f0a8b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8410da10 con 0x7f0a8407de70 2024-09-10T07:26:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.627+0000 7f0a89c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.627+0000 7f0a89c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8419d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32964/0 (socket says 172.21.15.46:32964) 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.627+0000 7f0a89c00640 1 -- 172.21.15.46:0/1071618381 learned_addr learned my addr 172.21.15.46:0/1071618381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.628+0000 7f0a89c00640 1 -- 172.21.15.46:0/1071618381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8419ddd0 con 0x7f0a8407de70 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.628+0000 7f0a89c00640 1 --2- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8419d890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0a78030560 tx=0x7f0a78032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.628+0000 7f0a82000640 1 -- 172.21.15.46:0/1071618381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a78037040 con 0x7f0a8407de70 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.628+0000 7f0a82000640 1 -- 172.21.15.46:0/1071618381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a78040070 con 0x7f0a8407de70 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.628+0000 7f0a82000640 1 -- 172.21.15.46:0/1071618381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a7803a740 con 0x7f0a8407de70 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.628+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8419e060 con 0x7f0a8407de70 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.628+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a841a0d50 con 0x7f0a8407de70 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.630+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8407c350 con 0x7f0a8407de70 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.630+0000 7f0a82000640 1 -- 172.21.15.46:0/1071618381 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f0a7803bb90 con 0x7f0a8407de70 2024-09-10T07:26:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.630+0000 7f0a82000640 1 --2- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f0a6c038fc0 0x7f0a6c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.630+0000 7f0a82000640 1 -- 172.21.15.46:0/1071618381 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a7806c6d0 con 0x7f0a8407de70 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.630+0000 7f0a89200640 1 -- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f0a6c038fc0 msgr2=0x7f0a6c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2972952762 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.631+0000 7f0a89200640 1 --2- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f0a6c038fc0 0x7f0a6c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.635+0000 7f0a82000640 1 -- 172.21.15.46:0/1071618381 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a7804a200 con 0x7f0a8407de70 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.832+0000 7f0a89200640 1 -- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f0a6c038fc0 msgr2=0x7f0a6c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2972952762 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.832+0000 7f0a89200640 1 --2- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f0a6c038fc0 0x7f0a6c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.866+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0a8402d7c0 con 0x7f0a8407de70 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.866+0000 7f0a82000640 1 -- 172.21.15.46:0/1071618381 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f0a78031e80 con 0x7f0a8407de70 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f0a6c038fc0 msgr2=0x7f0a6c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 --2- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f0a6c038fc0 0x7f0a6c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 msgr2=0x7f0a8419d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 --2- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8419d890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0a78030560 tx=0x7f0a78032670 comp rx=0 tx=0).stop 2024-09-10T07:26:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 shutdown_connections 2024-09-10T07:26:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 --2- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f0a6c038fc0 0x7f0a6c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 --2- 172.21.15.46:0/1071618381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a8407de70 0x7f0a8419d890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 >> 172.21.15.46:0/1071618381 conn(0x7f0a84102b40 msgr2=0x7f0a84103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 shutdown_connections 2024-09-10T07:26:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:07.869+0000 7f0a8b000640 1 -- 172.21.15.46:0/1071618381 wait complete. 2024-09-10T07:26:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-09-10T07:26:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-09-10T07:26:09.122 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:09 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:09.061+0000 7f757e42d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:26:09.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:09 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:09.150+0000 7f757e42d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:26:10.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:10.101+0000 7f757e42d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:26:10.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:10.243+0000 7f757e42d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:26:10.665 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:10.412+0000 7f757e42d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:26:10.666 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:10.487+0000 7f757e42d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:26:11.010 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:10.664+0000 7f757e42d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:26:11.011 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:10.748+0000 7f757e42d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:26:11.011 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:10.822+0000 7f757e42d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:26:11.011 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:11.009+0000 7f757e42d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:26:11.272 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:11.089+0000 7f757e42d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:26:11.272 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:11.171+0000 7f757e42d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:26:11.659 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:11.271+0000 7f757e42d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:26:11.659 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:11.344+0000 7f757e42d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:26:11.659 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:11.416+0000 7f757e42d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:26:12.409 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:12.364+0000 7f757e42d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:26:12.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: Active manager daemon x restarted 2024-09-10T07:26:12.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: Activating manager daemon x 2024-09-10T07:26:12.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T07:26:12.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: mgrmap e10: x(active, starting, since 0.00398808s) 2024-09-10T07:26:12.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:26:12.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T07:26:12.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T07:26:12.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T07:26:12.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T07:26:12.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: Manager daemon x is now available 2024-09-10T07:26:12.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:26:13.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T07:26:13.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T07:26:13.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:13 smithi046 ceph-mon[30705]: mgrmap e11: x(active, since 1.00826s) 2024-09-10T07:26:13.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:13 smithi046 ceph-mon[30705]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T07:26:14.180 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.121+0000 7f77d3400640 1 Processor -- start 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.121+0000 7f77d3400640 1 -- start start 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.121+0000 7f77d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.121+0000 7f77d3400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc07c8c0 con 0x7f77cc07de90 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.122+0000 7f77d0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:14.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.122+0000 7f77d0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32978/0 (socket says 172.21.15.46:32978) 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.122+0000 7f77d0a00640 1 -- 172.21.15.46:0/3409344263 learned_addr learned my addr 172.21.15.46:0/3409344263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.122+0000 7f77d0a00640 1 -- 172.21.15.46:0/3409344263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc07ca40 con 0x7f77cc07de90 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.123+0000 7f77d0a00640 1 --2- 172.21.15.46:0/3409344263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc07c2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009820 tx=0x7f77c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f0a2e608ee2f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.123+0000 7f77cb400640 1 -- 172.21.15.46:0/3409344263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0037070 con 0x7f77cc07de90 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.124+0000 7f77cb400640 1 -- 172.21.15.46:0/3409344263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c0032920 con 0x7f77cc07de90 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.124+0000 7f77cb400640 1 -- 172.21.15.46:0/3409344263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0030810 con 0x7f77cc07de90 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.124+0000 7f77d3400640 1 -- 172.21.15.46:0/3409344263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 msgr2=0x7f77cc07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.124+0000 7f77d3400640 1 --2- 172.21.15.46:0/3409344263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc07c2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009820 tx=0x7f77c002a270 comp rx=0 tx=0).stop 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.124+0000 7f77d3400640 1 -- 172.21.15.46:0/3409344263 shutdown_connections 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.124+0000 7f77d3400640 1 --2- 172.21.15.46:0/3409344263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc07c2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.124+0000 7f77d3400640 1 -- 172.21.15.46:0/3409344263 >> 172.21.15.46:0/3409344263 conn(0x7f77cc102d20 msgr2=0x7f77cc105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.125+0000 7f77d3400640 1 -- 172.21.15.46:0/3409344263 shutdown_connections 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.125+0000 7f77d3400640 1 -- 172.21.15.46:0/3409344263 wait complete. 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.126+0000 7f77d3400640 1 Processor -- start 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.126+0000 7f77d3400640 1 -- start start 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.127+0000 7f77d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.127+0000 7f77d3400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc109810 con 0x7f77cc07de90 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.127+0000 7f77d0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.127+0000 7f77d0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc1a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32982/0 (socket says 172.21.15.46:32982) 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.127+0000 7f77d0a00640 1 -- 172.21.15.46:0/2257192441 learned_addr learned my addr 172.21.15.46:0/2257192441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.127+0000 7f77d0a00640 1 -- 172.21.15.46:0/2257192441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc1a23f0 con 0x7f77cc07de90 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.128+0000 7f77d0a00640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc1a1eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77c00097f0 tx=0x7f77c00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.128+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0037040 con 0x7f77cc07de90 2024-09-10T07:26:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.128+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c0030cc0 con 0x7f77cc07de90 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.128+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c003a640 con 0x7f77cc07de90 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.128+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77cc1a2680 con 0x7f77cc07de90 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.128+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77cc1a5370 con 0x7f77cc07de90 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.129+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f77c003b3f0 con 0x7f77cc07de90 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.130+0000 7f77c9600640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 0x7f77b403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.130+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 --> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f77b403bbf0 con 0x7f77b4039020 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.130+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f77c006c5c0 con 0x7f77cc07de90 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.130+0000 7f77cbe00640 1 -- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 msgr2=0x7f77b403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2972952762 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.130+0000 7f77cbe00640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 0x7f77b403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.331+0000 7f77cbe00640 1 -- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 msgr2=0x7f77b403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2972952762 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.331+0000 7f77cbe00640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 0x7f77b403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.732+0000 7f77cbe00640 1 -- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 msgr2=0x7f77b403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2972952762 2024-09-10T07:26:14.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:09.732+0000 7f77cbe00640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 0x7f77b403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:10.533+0000 7f77cbe00640 1 -- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 msgr2=0x7f77b403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2972952762 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:10.533+0000 7f77cbe00640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 0x7f77b403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:12.134+0000 7f77cbe00640 1 -- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 msgr2=0x7f77b403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2972952762 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:12.135+0000 7f77cbe00640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 0x7f77b403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:12.368+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f77c003a8f0 con 0x7f77cc07de90 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:12.368+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 msgr2=0x7f77b403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:12.368+0000 7f77c9600640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 0x7f77b403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.371+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f77c006e8d0 con 0x7f77cc07de90 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.371+0000 7f77c9600640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f77b403cae0 0x7f77b403eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.371+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f77c0047e50 con 0x7f77b403cae0 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.372+0000 7f77cbe00640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f77b403cae0 0x7f77b403eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.373+0000 7f77cbe00640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f77b403cae0 0x7f77b403eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77b8003980 tx=0x7f77b80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:14.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.374+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f77c0047e50 con 0x7f77b403cae0 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.382+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f77cc07c370 con 0x7f77b403cae0 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.382+0000 7f77c9600640 1 -- 172.21.15.46:0/2257192441 <== mgr.14150 v2:172.21.15.46:6800/1492955812 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f77cc07c370 con 0x7f77b403cae0 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f77b403cae0 msgr2=0x7f77b403eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f77b403cae0 0x7f77b403eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77b8003980 tx=0x7f77b80092b0 comp rx=0 tx=0).stop 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 msgr2=0x7f77cc1a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc1a1eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77c00097f0 tx=0x7f77c00327b0 comp rx=0 tx=0).stop 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 shutdown_connections 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/2972952762,v1:172.21.15.46:6801/2972952762] conn(0x7f77b4039020 0x7f77b403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f77b403cae0 0x7f77b403eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 --2- 172.21.15.46:0/2257192441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77cc07de90 0x7f77cc1a1eb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 >> 172.21.15.46:0/2257192441 conn(0x7f77cc102d20 msgr2=0x7f77cc198880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 shutdown_connections 2024-09-10T07:26:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:13.383+0000 7f77d3400640 1 -- 172.21.15.46:0/2257192441 wait complete. 2024-09-10T07:26:14.193 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-09-10T07:26:14.193 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-09-10T07:26:14.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-mon[30705]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:14.788+0000 7f74d4a00640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:07:26:14] ENGINE Error in HTTPServer.serve 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: Traceback (most recent call last): 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: self._connections.run(self.expiration_interval) 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: self._run(expiration_interval) 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T07:26:15.071 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: s = self.context.wrap_socket( 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: return self.sslsocket_class._create( 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: self.do_handshake() 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: self._sslobj.do_handshake() 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T07:26:15.072 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:26:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:26:15.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:15 smithi046 ceph-mon[30705]: mgrmap e12: x(active, since 2s) 2024-09-10T07:26:15.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:15 smithi046 ceph-mon[30705]: [10/Sep/2024:07:26:14] ENGINE Bus STARTING 2024-09-10T07:26:15.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:15 smithi046 ceph-mon[30705]: [10/Sep/2024:07:26:14] ENGINE Serving on http://172.21.15.46:8765 2024-09-10T07:26:15.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:15 smithi046 ceph-mon[30705]: [10/Sep/2024:07:26:14] ENGINE Serving on https://172.21.15.46:7150 2024-09-10T07:26:15.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:15 smithi046 ceph-mon[30705]: [10/Sep/2024:07:26:14] ENGINE Bus STARTED 2024-09-10T07:26:15.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:15 smithi046 ceph-mon[30705]: [10/Sep/2024:07:26:14] ENGINE Error in HTTPServer.serve 2024-09-10T07:26:15.659 INFO:journalctl@ceph.mon.a.smithi046.stdout: Traceback (most recent call last): 2024-09-10T07:26:15.659 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._run(expiration_interval) 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: s = self.context.wrap_socket( 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: return self.sslsocket_class._create( 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: self.do_handshake() 2024-09-10T07:26:15.660 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T07:26:15.661 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._sslobj.do_handshake() 2024-09-10T07:26:15.661 INFO:journalctl@ceph.mon.a.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T07:26:15.661 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:16.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.181+0000 7feff5e00640 1 Processor -- start 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.181+0000 7feff5e00640 1 -- start start 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.181+0000 7feff5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff0109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.181+0000 7feff5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feff010a240 con 0x7feff0109870 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.182+0000 7fefeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff0109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.182+0000 7fefeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff0109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33052/0 (socket says 172.21.15.46:33052) 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.182+0000 7fefeea00640 1 -- 172.21.15.46:0/112493790 learned_addr learned my addr 172.21.15.46:0/112493790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.182+0000 7fefeea00640 1 -- 172.21.15.46:0/112493790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff010a980 con 0x7feff0109870 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.183+0000 7fefeea00640 1 --2- 172.21.15.46:0/112493790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff0109c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009820 tx=0x7fefe002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd74b20f3adcdaa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.184+0000 7fefed600640 1 -- 172.21.15.46:0/112493790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0037070 con 0x7feff0109870 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.184+0000 7fefed600640 1 -- 172.21.15.46:0/112493790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe0032920 con 0x7feff0109870 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.184+0000 7fefed600640 1 -- 172.21.15.46:0/112493790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0030810 con 0x7feff0109870 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.184+0000 7feff5e00640 1 -- 172.21.15.46:0/112493790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 msgr2=0x7feff0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.184+0000 7feff5e00640 1 --2- 172.21.15.46:0/112493790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff0109c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009820 tx=0x7fefe002a270 comp rx=0 tx=0).stop 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.185+0000 7feff5e00640 1 -- 172.21.15.46:0/112493790 shutdown_connections 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.185+0000 7feff5e00640 1 --2- 172.21.15.46:0/112493790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff0109c70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.185+0000 7feff5e00640 1 -- 172.21.15.46:0/112493790 >> 172.21.15.46:0/112493790 conn(0x7feff007cdc0 msgr2=0x7feff007d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.185+0000 7feff5e00640 1 -- 172.21.15.46:0/112493790 shutdown_connections 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.185+0000 7feff5e00640 1 -- 172.21.15.46:0/112493790 wait complete. 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.186+0000 7feff5e00640 1 Processor -- start 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.186+0000 7feff5e00640 1 -- start start 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.186+0000 7feff5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.186+0000 7feff5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feff010dd30 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.186+0000 7fefeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.186+0000 7fefeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff019daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33064/0 (socket says 172.21.15.46:33064) 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.186+0000 7fefeea00640 1 -- 172.21.15.46:0/1941880288 learned_addr learned my addr 172.21.15.46:0/1941880288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.187+0000 7fefeea00640 1 -- 172.21.15.46:0/1941880288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff019dfe0 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.187+0000 7fefeea00640 1 --2- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff019daa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fefe0030560 tx=0x7fefe0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.187+0000 7feff4a00640 1 -- 172.21.15.46:0/1941880288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0037040 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.187+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff019e270 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.187+0000 7feff4a00640 1 -- 172.21.15.46:0/1941880288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe0040070 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.187+0000 7feff4a00640 1 -- 172.21.15.46:0/1941880288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe003a6e0 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.188+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff01a0f60 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.189+0000 7feff4a00640 1 -- 172.21.15.46:0/1941880288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fefe003b420 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.189+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff010deb0 con 0x7feff0109870 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.189+0000 7feff4a00640 1 --2- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fefd4038fc0 0x7fefd403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.189+0000 7feff4a00640 1 -- 172.21.15.46:0/1941880288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fefe006c720 con 0x7feff0109870 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.189+0000 7fefee000640 1 --2- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fefd4038fc0 0x7fefd403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.190+0000 7fefee000640 1 --2- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fefd4038fc0 0x7fefd403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0098c0 tx=0x7fefdc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.199+0000 7feff4a00640 1 -- 172.21.15.46:0/1941880288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefe0038280 con 0x7feff0109870 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.394+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7feff0062630 con 0x7fefd4038fc0 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.461+0000 7feff4a00640 1 -- 172.21.15.46:0/1941880288 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7feff0062630 con 0x7fefd4038fc0 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.463+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fefd4038fc0 msgr2=0x7fefd403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.463+0000 7feff5e00640 1 --2- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fefd4038fc0 0x7fefd403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0098c0 tx=0x7fefdc006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.463+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 msgr2=0x7feff019daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.463+0000 7feff5e00640 1 --2- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff019daa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fefe0030560 tx=0x7fefe0032670 comp rx=0 tx=0).stop 2024-09-10T07:26:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.463+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 shutdown_connections 2024-09-10T07:26:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.463+0000 7feff5e00640 1 --2- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fefd4038fc0 0x7fefd403b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.463+0000 7feff5e00640 1 --2- 172.21.15.46:0/1941880288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff0109870 0x7feff019daa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.463+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 >> 172.21.15.46:0/1941880288 conn(0x7feff007cdc0 msgr2=0x7feff01076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.464+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 shutdown_connections 2024-09-10T07:26:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:15.464+0000 7feff5e00640 1 -- 172.21.15.46:0/1941880288 wait complete. 2024-09-10T07:26:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-09-10T07:26:16.572 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:16 smithi046 ceph-mon[30705]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:16.572 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:16.572 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:16.572 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:18 smithi046 ceph-mon[30705]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$V78RxE5jULjG6xG/hchNbeBK9dWRe/1qZZAdw9PiGi0kEmiiMS8zW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725953177, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T07:26:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.873+0000 7fdadb200640 1 Processor -- start 2024-09-10T07:26:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.873+0000 7fdadb200640 1 -- start start 2024-09-10T07:26:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.873+0000 7fdadb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc107800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.873+0000 7fdadb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc107dd0 con 0x7fdadc107400 2024-09-10T07:26:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.874+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.874+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc107800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55172/0 (socket says 172.21.15.46:55172) 2024-09-10T07:26:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.874+0000 7fdad9e00640 1 -- 172.21.15.46:0/4174063352 learned_addr learned my addr 172.21.15.46:0/4174063352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:18.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.874+0000 7fdad9e00640 1 -- 172.21.15.46:0/4174063352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc1085b0 con 0x7fdadc107400 2024-09-10T07:26:18.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.875+0000 7fdad9e00640 1 --2- 172.21.15.46:0/4174063352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc107800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009820 tx=0x7fdacc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86d7406d205e964 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:18.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.876+0000 7fdad8a00640 1 -- 172.21.15.46:0/4174063352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdacc037070 con 0x7fdadc107400 2024-09-10T07:26:18.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.876+0000 7fdad8a00640 1 -- 172.21.15.46:0/4174063352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc032920 con 0x7fdadc107400 2024-09-10T07:26:18.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.876+0000 7fdad8a00640 1 -- 172.21.15.46:0/4174063352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdacc030810 con 0x7fdadc107400 2024-09-10T07:26:18.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.876+0000 7fdadb200640 1 -- 172.21.15.46:0/4174063352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 msgr2=0x7fdadc107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:18.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.876+0000 7fdadb200640 1 --2- 172.21.15.46:0/4174063352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc107800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009820 tx=0x7fdacc02a270 comp rx=0 tx=0).stop 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.877+0000 7fdadb200640 1 -- 172.21.15.46:0/4174063352 shutdown_connections 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.877+0000 7fdadb200640 1 --2- 172.21.15.46:0/4174063352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc107800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.877+0000 7fdadb200640 1 -- 172.21.15.46:0/4174063352 >> 172.21.15.46:0/4174063352 conn(0x7fdadc102bb0 msgr2=0x7fdadc104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.877+0000 7fdadb200640 1 -- 172.21.15.46:0/4174063352 shutdown_connections 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.877+0000 7fdadb200640 1 -- 172.21.15.46:0/4174063352 wait complete. 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.878+0000 7fdadb200640 1 Processor -- start 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.878+0000 7fdadb200640 1 -- start start 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.878+0000 7fdadb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.878+0000 7fdadb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc10b960 con 0x7fdadc107400 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.878+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.879+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc19d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55178/0 (socket says 172.21.15.46:55178) 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.879+0000 7fdad9e00640 1 -- 172.21.15.46:0/1944756502 learned_addr learned my addr 172.21.15.46:0/1944756502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:18.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.879+0000 7fdad9e00640 1 -- 172.21.15.46:0/1944756502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc19ddf0 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.879+0000 7fdad9e00640 1 --2- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc19d8b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0097f0 tx=0x7fdacc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.879+0000 7fdad2a00640 1 -- 172.21.15.46:0/1944756502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdacc037040 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.879+0000 7fdad2a00640 1 -- 172.21.15.46:0/1944756502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc030cc0 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.879+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc19e080 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.879+0000 7fdad2a00640 1 -- 172.21.15.46:0/1944756502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdacc03a640 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.880+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc1a0d70 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.881+0000 7fdad2a00640 1 -- 172.21.15.46:0/1944756502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fdacc03a8f0 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.881+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdadc10bae0 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.881+0000 7fdad2a00640 1 --2- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdac0039020 0x7fdac003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.881+0000 7fdad2a00640 1 -- 172.21.15.46:0/1944756502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdacc06c5d0 con 0x7fdadc107400 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.881+0000 7fdad9400640 1 --2- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdac0039020 0x7fdac003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:18.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.882+0000 7fdad9400640 1 --2- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdac0039020 0x7fdac003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdac80098c0 tx=0x7fdac8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:16.887+0000 7fdad2a00640 1 -- 172.21.15.46:0/1944756502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdacc0328c0 con 0x7fdadc107400 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.088+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fdadc107880 con 0x7fdac0039020 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.338+0000 7fdad2a00640 1 -- 172.21.15.46:0/1944756502 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fdadc107880 con 0x7fdac0039020 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.341+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdac0039020 msgr2=0x7fdac003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.341+0000 7fdadb200640 1 --2- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdac0039020 0x7fdac003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdac80098c0 tx=0x7fdac8006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.341+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 msgr2=0x7fdadc19d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.341+0000 7fdadb200640 1 --2- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc19d8b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0097f0 tx=0x7fdacc0327b0 comp rx=0 tx=0).stop 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.342+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 shutdown_connections 2024-09-10T07:26:18.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.342+0000 7fdadb200640 1 --2- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdac0039020 0x7fdac003b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:18.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.342+0000 7fdadb200640 1 --2- 172.21.15.46:0/1944756502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc107400 0x7fdadc19d8b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:18.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.342+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 >> 172.21.15.46:0/1944756502 conn(0x7fdadc102bb0 msgr2=0x7fdadc1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:18.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.342+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 shutdown_connections 2024-09-10T07:26:18.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:17.342+0000 7fdadb200640 1 -- 172.21.15.46:0/1944756502 wait complete. 2024-09-10T07:26:18.920 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-09-10T07:26:20.343 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T07:26:20.343 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.457+0000 7f1aeb600640 1 Processor -- start 2024-09-10T07:26:20.343 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.458+0000 7f1aeb600640 1 -- start start 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.458+0000 7f1aeb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae407c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.458+0000 7f1aeb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae407c7b0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.458+0000 7f1ae8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.458+0000 7f1ae8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae407c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55186/0 (socket says 172.21.15.46:55186) 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.458+0000 7f1ae8c00640 1 -- 172.21.15.46:0/3601421763 learned_addr learned my addr 172.21.15.46:0/3601421763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.459+0000 7f1ae8c00640 1 -- 172.21.15.46:0/3601421763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae407c930 con 0x7f1ae407ddc0 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.460+0000 7f1ae8c00640 1 --2- 172.21.15.46:0/3601421763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae407c270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009820 tx=0x7f1ad802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae16b071eef2a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.460+0000 7f1ae3400640 1 -- 172.21.15.46:0/3601421763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad8037070 con 0x7f1ae407ddc0 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.460+0000 7f1ae3400640 1 -- 172.21.15.46:0/3601421763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad8032920 con 0x7f1ae407ddc0 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.460+0000 7f1ae3400640 1 -- 172.21.15.46:0/3601421763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad8030810 con 0x7f1ae407ddc0 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.461+0000 7f1aeb600640 1 -- 172.21.15.46:0/3601421763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 msgr2=0x7f1ae407c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.461+0000 7f1aeb600640 1 --2- 172.21.15.46:0/3601421763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae407c270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009820 tx=0x7f1ad802a270 comp rx=0 tx=0).stop 2024-09-10T07:26:20.344 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.461+0000 7f1aeb600640 1 -- 172.21.15.46:0/3601421763 shutdown_connections 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.461+0000 7f1aeb600640 1 --2- 172.21.15.46:0/3601421763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae407c270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.461+0000 7f1aeb600640 1 -- 172.21.15.46:0/3601421763 >> 172.21.15.46:0/3601421763 conn(0x7f1ae4102d20 msgr2=0x7f1ae4105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.461+0000 7f1aeb600640 1 -- 172.21.15.46:0/3601421763 shutdown_connections 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.461+0000 7f1aeb600640 1 -- 172.21.15.46:0/3601421763 wait complete. 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.462+0000 7f1aeb600640 1 Processor -- start 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.462+0000 7f1aeb600640 1 -- start start 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.462+0000 7f1aeb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.462+0000 7f1aeb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae410dbe0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.462+0000 7f1ae8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.462+0000 7f1ae8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae419da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55200/0 (socket says 172.21.15.46:55200) 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.462+0000 7f1ae8c00640 1 -- 172.21.15.46:0/1126474864 learned_addr learned my addr 172.21.15.46:0/1126474864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.463+0000 7f1ae8c00640 1 -- 172.21.15.46:0/1126474864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae419df70 con 0x7f1ae407ddc0 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.463+0000 7f1ae8c00640 1 --2- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae419da30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80097f0 tx=0x7f1ad80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:20.345 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.463+0000 7f1ae1600640 1 -- 172.21.15.46:0/1126474864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad8037040 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.463+0000 7f1ae1600640 1 -- 172.21.15.46:0/1126474864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad8030cc0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.463+0000 7f1ae1600640 1 -- 172.21.15.46:0/1126474864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad803a6b0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.463+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae419e200 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.464+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae41a0ef0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.465+0000 7f1ae1600640 1 -- 172.21.15.46:0/1126474864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1ad803baf0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.465+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae407c2f0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.465+0000 7f1ae1600640 1 --2- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1acc0390c0 0x7f1acc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.465+0000 7f1ae1600640 1 -- 172.21.15.46:0/1126474864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ad806c7e0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.465+0000 7f1ae3e00640 1 --2- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1acc0390c0 0x7f1acc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.466+0000 7f1ae3e00640 1 --2- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1acc0390c0 0x7f1acc03b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00098c0 tx=0x7f1ad0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.470+0000 7f1ae1600640 1 -- 172.21.15.46:0/1126474864 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ad803a850 con 0x7f1ae407ddc0 2024-09-10T07:26:20.346 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.653+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f1ae402d7c0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.654+0000 7f1ae1600640 1 -- 172.21.15.46:0/1126474864 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f1ad8049ca0 con 0x7f1ae407ddc0 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1acc0390c0 msgr2=0x7f1acc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 --2- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1acc0390c0 0x7f1acc03b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00098c0 tx=0x7f1ad0006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 msgr2=0x7f1ae419da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 --2- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae419da30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80097f0 tx=0x7f1ad80327b0 comp rx=0 tx=0).stop 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 shutdown_connections 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 --2- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1acc0390c0 0x7f1acc03b580 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 --2- 172.21.15.46:0/1126474864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae407ddc0 0x7f1ae419da30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 >> 172.21.15.46:0/1126474864 conn(0x7f1ae4102d20 msgr2=0x7f1ae4103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 shutdown_connections 2024-09-10T07:26:20.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:19.656+0000 7f1aeb600640 1 -- 172.21.15.46:0/1126474864 wait complete. 2024-09-10T07:26:20.360 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T07:26:20.360 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-10T07:26:20.374 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T07:26:20.375 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T07:26:20.375 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-10T07:26:20.375 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T07:26:20.377 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-09-10T07:26:20.377 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:20.377 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046.front.sepia.ceph.com:8443/ 2024-09-10T07:26:20.377 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-09-10T07:26:20.377 INFO:teuthology.orchestra.run.smithi046.stdout: Password: sesexot09d 2024-09-10T07:26:20.377 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:20.378 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config directory 2024-09-10T07:26:20.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:20 smithi046 ceph-mon[30705]: mgrmap e13: x(active, since 6s) 2024-09-10T07:26:20.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:20 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1126474864' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T07:26:21.835 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.895+0000 7f34c7200640 1 Processor -- start 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.895+0000 7f34c7200640 1 -- start start 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.896+0000 7f34c7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.896+0000 7f34c7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34c007c8a0 con 0x7f34c007de70 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.896+0000 7f34bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.896+0000 7f34bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55210/0 (socket says 172.21.15.46:55210) 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.896+0000 7f34bfe00640 1 -- 172.21.15.46:0/928995488 learned_addr learned my addr 172.21.15.46:0/928995488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.897+0000 7f34bfe00640 1 -- 172.21.15.46:0/928995488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c007ca20 con 0x7f34c007de70 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.898+0000 7f34bfe00640 1 --2- 172.21.15.46:0/928995488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c007c2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009820 tx=0x7f34b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa109cc2c90ed1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.898+0000 7f34bf400640 1 -- 172.21.15.46:0/928995488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34b0037070 con 0x7f34c007de70 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.898+0000 7f34bf400640 1 -- 172.21.15.46:0/928995488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b0032920 con 0x7f34c007de70 2024-09-10T07:26:21.836 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.898+0000 7f34bf400640 1 -- 172.21.15.46:0/928995488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34b0030810 con 0x7f34c007de70 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.899+0000 7f34c7200640 1 -- 172.21.15.46:0/928995488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 msgr2=0x7f34c007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.899+0000 7f34c7200640 1 --2- 172.21.15.46:0/928995488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c007c2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009820 tx=0x7f34b002a270 comp rx=0 tx=0).stop 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.899+0000 7f34c7200640 1 -- 172.21.15.46:0/928995488 shutdown_connections 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.899+0000 7f34c7200640 1 --2- 172.21.15.46:0/928995488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c007c2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.899+0000 7f34c7200640 1 -- 172.21.15.46:0/928995488 >> 172.21.15.46:0/928995488 conn(0x7f34c0102cf0 msgr2=0x7f34c0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.899+0000 7f34c7200640 1 -- 172.21.15.46:0/928995488 shutdown_connections 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.899+0000 7f34c7200640 1 -- 172.21.15.46:0/928995488 wait complete. 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.900+0000 7f34c7200640 1 Processor -- start 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.900+0000 7f34c7200640 1 -- start start 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.900+0000 7f34c7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.900+0000 7f34c7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34c010dc10 con 0x7f34c007de70 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.900+0000 7f34bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.900+0000 7f34bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c019dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55216/0 (socket says 172.21.15.46:55216) 2024-09-10T07:26:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.900+0000 7f34bfe00640 1 -- 172.21.15.46:0/3733904699 learned_addr learned my addr 172.21.15.46:0/3733904699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.901+0000 7f34bfe00640 1 -- 172.21.15.46:0/3733904699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c019e010 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.901+0000 7f34bfe00640 1 --2- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c019dad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34b0030560 tx=0x7f34b0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.901+0000 7f34bd600640 1 -- 172.21.15.46:0/3733904699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34b0037040 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.901+0000 7f34bd600640 1 -- 172.21.15.46:0/3733904699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b0040070 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.901+0000 7f34bd600640 1 -- 172.21.15.46:0/3733904699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34b003a6d0 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.902+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c019e2a0 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.902+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c01a0f90 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.903+0000 7f34bd600640 1 -- 172.21.15.46:0/3733904699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f34b003bb20 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.903+0000 7f34bd600640 1 --2- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34a40390c0 0x7f34a403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.903+0000 7f34bd600640 1 -- 172.21.15.46:0/3733904699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f34b006c7a0 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.903+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c007c350 con 0x7f34c007de70 2024-09-10T07:26:21.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.904+0000 7f34b7e00640 1 --2- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34a40390c0 0x7f34a403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.904+0000 7f34b7e00640 1 --2- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34a40390c0 0x7f34a403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f34a80098c0 tx=0x7f34a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:20.909+0000 7f34bd600640 1 -- 172.21.15.46:0/3733904699 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34b0042940 con 0x7f34c007de70 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.186+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f34c002d7c0 con 0x7f34c007de70 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.189+0000 7f34bd600640 1 -- 172.21.15.46:0/3733904699 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f34b004a200 con 0x7f34c007de70 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.191+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34a40390c0 msgr2=0x7f34a403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.191+0000 7f34c7200640 1 --2- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34a40390c0 0x7f34a403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f34a80098c0 tx=0x7f34a8006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.191+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 msgr2=0x7f34c019dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.191+0000 7f34c7200640 1 --2- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c019dad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34b0030560 tx=0x7f34b0032670 comp rx=0 tx=0).stop 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.191+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 shutdown_connections 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.191+0000 7f34c7200640 1 --2- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34a40390c0 0x7f34a403b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:21.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.191+0000 7f34c7200640 1 --2- 172.21.15.46:0/3733904699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c007de70 0x7f34c019dad0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.191+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 >> 172.21.15.46:0/3733904699 conn(0x7f34c0102cf0 msgr2=0x7f34c0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.192+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 shutdown_connections 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-10T07:26:21.192+0000 7f34c7200640 1 -- 172.21.15.46:0/3733904699 wait complete. 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-09-10T07:26:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:21.841 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-09-10T07:26:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T07:26:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:21.841 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-09-10T07:26:21.893 INFO:tasks.cephadm:Fetching config... 2024-09-10T07:26:21.893 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:26:21.893 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T07:26:21.930 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T07:26:21.930 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:26:21.930 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T07:26:22.005 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T07:26:22.005 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:26:22.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T07:26:22.101 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T07:26:22.101 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:26:22.101 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T07:26:22.175 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T07:26:22.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCQRy7LsG7PifQiff+CekI70KyfAJ2HcOit9fQY7KoyAmHZWgW64isMuUqeuax9d5iybNrBUehoxWm5jh8/F9riV2jif2dJNzr/jpyK/oNS6GKTPzUIge5ftOfGJKkvk2u+NP1/yV7UZ+T2+fk82OqUM6X37P7j2L7z5oWJZs9nX+HUi9jZKVukEIm/+yrfOTGi4zzKXFf9VNdHlSfC2E30TXftm8Q/7tvRVuVLWrRVREwF+iNkqI9VRoR6NNoLfO5WrDU7i88xWOgkrRUOrVC5bCOCwZTSm8h9qiErfjtZCxoQeMhRfRA0o7z/TVgXVCGLeo3KwQPgi0LPTpZ5qTLqMjWigvXt9baw/6BJPq/jKL9R2nkeZK/BSv7XjCHmaato9XIICm21CHLBBWuX0Xizufnu+AQwbahR2mQyExS/4s3RfGT5S500Q6sYbu9S0v7FZCDQ+XqKME1AEV1fE52GKQ839anKFS05SkkGqekfouGFhULA+xyfz9wer/PXq8= ceph-b120bdf2-6f45-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T07:26:22.318 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCQRy7LsG7PifQiff+CekI70KyfAJ2HcOit9fQY7KoyAmHZWgW64isMuUqeuax9d5iybNrBUehoxWm5jh8/F9riV2jif2dJNzr/jpyK/oNS6GKTPzUIge5ftOfGJKkvk2u+NP1/yV7UZ+T2+fk82OqUM6X37P7j2L7z5oWJZs9nX+HUi9jZKVukEIm/+yrfOTGi4zzKXFf9VNdHlSfC2E30TXftm8Q/7tvRVuVLWrRVREwF+iNkqI9VRoR6NNoLfO5WrDU7i88xWOgkrRUOrVC5bCOCwZTSm8h9qiErfjtZCxoQeMhRfRA0o7z/TVgXVCGLeo3KwQPgi0LPTpZ5qTLqMjWigvXt9baw/6BJPq/jKL9R2nkeZK/BSv7XjCHmaato9XIICm21CHLBBWuX0Xizufnu+AQwbahR2mQyExS/4s3RfGT5S500Q6sYbu9S0v7FZCDQ+XqKME1AEV1fE52GKQ839anKFS05SkkGqekfouGFhULA+xyfz9wer/PXq8= ceph-b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:26:22.343 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCQRy7LsG7PifQiff+CekI70KyfAJ2HcOit9fQY7KoyAmHZWgW64isMuUqeuax9d5iybNrBUehoxWm5jh8/F9riV2jif2dJNzr/jpyK/oNS6GKTPzUIge5ftOfGJKkvk2u+NP1/yV7UZ+T2+fk82OqUM6X37P7j2L7z5oWJZs9nX+HUi9jZKVukEIm/+yrfOTGi4zzKXFf9VNdHlSfC2E30TXftm8Q/7tvRVuVLWrRVREwF+iNkqI9VRoR6NNoLfO5WrDU7i88xWOgkrRUOrVC5bCOCwZTSm8h9qiErfjtZCxoQeMhRfRA0o7z/TVgXVCGLeo3KwQPgi0LPTpZ5qTLqMjWigvXt9baw/6BJPq/jKL9R2nkeZK/BSv7XjCHmaato9XIICm21CHLBBWuX0Xizufnu+AQwbahR2mQyExS/4s3RfGT5S500Q6sYbu9S0v7FZCDQ+XqKME1AEV1fE52GKQ839anKFS05SkkGqekfouGFhULA+xyfz9wer/PXq8= ceph-b120bdf2-6f45-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T07:26:22.384 INFO:teuthology.orchestra.run.smithi063.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCQRy7LsG7PifQiff+CekI70KyfAJ2HcOit9fQY7KoyAmHZWgW64isMuUqeuax9d5iybNrBUehoxWm5jh8/F9riV2jif2dJNzr/jpyK/oNS6GKTPzUIge5ftOfGJKkvk2u+NP1/yV7UZ+T2+fk82OqUM6X37P7j2L7z5oWJZs9nX+HUi9jZKVukEIm/+yrfOTGi4zzKXFf9VNdHlSfC2E30TXftm8Q/7tvRVuVLWrRVREwF+iNkqI9VRoR6NNoLfO5WrDU7i88xWOgkrRUOrVC5bCOCwZTSm8h9qiErfjtZCxoQeMhRfRA0o7z/TVgXVCGLeo3KwQPgi0LPTpZ5qTLqMjWigvXt9baw/6BJPq/jKL9R2nkeZK/BSv7XjCHmaato9XIICm21CHLBBWuX0Xizufnu+AQwbahR2mQyExS/4s3RfGT5S500Q6sYbu9S0v7FZCDQ+XqKME1AEV1fE52GKQ839anKFS05SkkGqekfouGFhULA+xyfz9wer/PXq8= ceph-b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:26:22.397 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCQRy7LsG7PifQiff+CekI70KyfAJ2HcOit9fQY7KoyAmHZWgW64isMuUqeuax9d5iybNrBUehoxWm5jh8/F9riV2jif2dJNzr/jpyK/oNS6GKTPzUIge5ftOfGJKkvk2u+NP1/yV7UZ+T2+fk82OqUM6X37P7j2L7z5oWJZs9nX+HUi9jZKVukEIm/+yrfOTGi4zzKXFf9VNdHlSfC2E30TXftm8Q/7tvRVuVLWrRVREwF+iNkqI9VRoR6NNoLfO5WrDU7i88xWOgkrRUOrVC5bCOCwZTSm8h9qiErfjtZCxoQeMhRfRA0o7z/TVgXVCGLeo3KwQPgi0LPTpZ5qTLqMjWigvXt9baw/6BJPq/jKL9R2nkeZK/BSv7XjCHmaato9XIICm21CHLBBWuX0Xizufnu+AQwbahR2mQyExS/4s3RfGT5S500Q6sYbu9S0v7FZCDQ+XqKME1AEV1fE52GKQ839anKFS05SkkGqekfouGFhULA+xyfz9wer/PXq8= ceph-b120bdf2-6f45-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T07:26:22.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:22 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3733904699' entity='client.admin' 2024-09-10T07:26:22.438 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCQRy7LsG7PifQiff+CekI70KyfAJ2HcOit9fQY7KoyAmHZWgW64isMuUqeuax9d5iybNrBUehoxWm5jh8/F9riV2jif2dJNzr/jpyK/oNS6GKTPzUIge5ftOfGJKkvk2u+NP1/yV7UZ+T2+fk82OqUM6X37P7j2L7z5oWJZs9nX+HUi9jZKVukEIm/+yrfOTGi4zzKXFf9VNdHlSfC2E30TXftm8Q/7tvRVuVLWrRVREwF+iNkqI9VRoR6NNoLfO5WrDU7i88xWOgkrRUOrVC5bCOCwZTSm8h9qiErfjtZCxoQeMhRfRA0o7z/TVgXVCGLeo3KwQPgi0LPTpZ5qTLqMjWigvXt9baw/6BJPq/jKL9R2nkeZK/BSv7XjCHmaato9XIICm21CHLBBWuX0Xizufnu+AQwbahR2mQyExS/4s3RfGT5S500Q6sYbu9S0v7FZCDQ+XqKME1AEV1fE52GKQ839anKFS05SkkGqekfouGFhULA+xyfz9wer/PXq8= ceph-b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:26:22.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T07:26:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:26:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.390+0000 7fb465200640 1 -- 172.21.15.46:0/1578358975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 msgr2=0x7fb460104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.390+0000 7fb465200640 1 --2- 172.21.15.46:0/1578358975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 0x7fb460104480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb450009880 tx=0x7fb45002a5d0 comp rx=0 tx=0).stop 2024-09-10T07:26:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.390+0000 7fb465200640 1 -- 172.21.15.46:0/1578358975 shutdown_connections 2024-09-10T07:26:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.390+0000 7fb465200640 1 --2- 172.21.15.46:0/1578358975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 0x7fb460104480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.390+0000 7fb465200640 1 -- 172.21.15.46:0/1578358975 >> 172.21.15.46:0/1578358975 conn(0x7fb4600fbc90 msgr2=0x7fb4600fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.390+0000 7fb465200640 1 -- 172.21.15.46:0/1578358975 shutdown_connections 2024-09-10T07:26:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.391+0000 7fb465200640 1 -- 172.21.15.46:0/1578358975 wait complete. 2024-09-10T07:26:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.391+0000 7fb465200640 1 Processor -- start 2024-09-10T07:26:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.392+0000 7fb465200640 1 -- start start 2024-09-10T07:26:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.392+0000 7fb465200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 0x7fb460101910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.392+0000 7fb465200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601076a0 con 0x7fb460102090 2024-09-10T07:26:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.392+0000 7fb45de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 0x7fb460101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.392+0000 7fb45de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 0x7fb460101910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55234/0 (socket says 172.21.15.46:55234) 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.392+0000 7fb45de00640 1 -- 172.21.15.46:0/618999656 learned_addr learned my addr 172.21.15.46:0/618999656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.393+0000 7fb45de00640 1 -- 172.21.15.46:0/618999656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb460101e50 con 0x7fb460102090 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.393+0000 7fb45de00640 1 --2- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 0x7fb460101910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb450004f60 tx=0x7fb4500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.393+0000 7fb456a00640 1 -- 172.21.15.46:0/618999656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45003f070 con 0x7fb460102090 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.393+0000 7fb456a00640 1 -- 172.21.15.46:0/618999656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb450003ad0 con 0x7fb460102090 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.393+0000 7fb456a00640 1 -- 172.21.15.46:0/618999656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb450035030 con 0x7fb460102090 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.393+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb460100040 con 0x7fb460102090 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.394+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb460100580 con 0x7fb460102090 2024-09-10T07:26:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.395+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb460107e20 con 0x7fb460102090 2024-09-10T07:26:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.395+0000 7fb456a00640 1 -- 172.21.15.46:0/618999656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb450042070 con 0x7fb460102090 2024-09-10T07:26:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.396+0000 7fb456a00640 1 --2- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4440390c0 0x7fb44403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.396+0000 7fb456a00640 1 -- 172.21.15.46:0/618999656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb45006ceb0 con 0x7fb460102090 2024-09-10T07:26:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.396+0000 7fb457e00640 1 --2- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4440390c0 0x7fb44403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.397+0000 7fb457e00640 1 --2- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4440390c0 0x7fb44403b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb4480098c0 tx=0x7fb448006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.401+0000 7fb456a00640 1 -- 172.21.15.46:0/618999656 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb45003a3c0 con 0x7fb460102090 2024-09-10T07:26:23.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.588+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb4601901e0 con 0x7fb460102090 2024-09-10T07:26:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.590+0000 7fb456a00640 1 -- 172.21.15.46:0/618999656 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb4500412c0 con 0x7fb460102090 2024-09-10T07:26:23.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4440390c0 msgr2=0x7fb44403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:23.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 --2- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4440390c0 0x7fb44403b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb4480098c0 tx=0x7fb448006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:23.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 msgr2=0x7fb460101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:23.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 --2- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 0x7fb460101910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb450004f60 tx=0x7fb4500048f0 comp rx=0 tx=0).stop 2024-09-10T07:26:23.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 shutdown_connections 2024-09-10T07:26:23.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 --2- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4440390c0 0x7fb44403b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:23.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 --2- 172.21.15.46:0/618999656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460102090 0x7fb460101910 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:23.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 >> 172.21.15.46:0/618999656 conn(0x7fb4600fbc90 msgr2=0x7fb4600fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.593+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 shutdown_connections 2024-09-10T07:26:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:23.594+0000 7fb465200640 1 -- 172.21.15.46:0/618999656 wait complete. 2024-09-10T07:26:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:24 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/618999656' entity='client.admin' 2024-09-10T07:26:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:26:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:26:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:24.798 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T07:26:24.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T07:26:25.164 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.676+0000 7fad0e600640 1 -- 172.21.15.46:0/1796536987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 msgr2=0x7fad08076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.676+0000 7fad0e600640 1 --2- 172.21.15.46:0/1796536987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 0x7fad08076fa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7facf8009880 tx=0x7facf802a5d0 comp rx=0 tx=0).stop 2024-09-10T07:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.676+0000 7fad0e600640 1 -- 172.21.15.46:0/1796536987 shutdown_connections 2024-09-10T07:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.676+0000 7fad0e600640 1 --2- 172.21.15.46:0/1796536987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 0x7fad08076fa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:25.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.676+0000 7fad0e600640 1 -- 172.21.15.46:0/1796536987 >> 172.21.15.46:0/1796536987 conn(0x7fad080fefc0 msgr2=0x7fad08101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:25.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.676+0000 7fad0e600640 1 -- 172.21.15.46:0/1796536987 shutdown_connections 2024-09-10T07:26:25.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.676+0000 7fad0e600640 1 -- 172.21.15.46:0/1796536987 wait complete. 2024-09-10T07:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.677+0000 7fad0e600640 1 Processor -- start 2024-09-10T07:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.677+0000 7fad0e600640 1 -- start start 2024-09-10T07:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.677+0000 7fad0e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 0x7fad0819d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.677+0000 7fad0e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810dcf0 con 0x7fad08076ba0 2024-09-10T07:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.678+0000 7fad07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 0x7fad0819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.678+0000 7fad07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 0x7fad0819d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55254/0 (socket says 172.21.15.46:55254) 2024-09-10T07:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.678+0000 7fad07400640 1 -- 172.21.15.46:0/684950034 learned_addr learned my addr 172.21.15.46:0/684950034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.678+0000 7fad07400640 1 -- 172.21.15.46:0/684950034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0819d860 con 0x7fad08076ba0 2024-09-10T07:26:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.678+0000 7fad07400640 1 --2- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 0x7fad0819d320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7facf8004f60 tx=0x7facf80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.679+0000 7facffe00640 1 -- 172.21.15.46:0/684950034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facf803f070 con 0x7fad08076ba0 2024-09-10T07:26:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.679+0000 7facffe00640 1 -- 172.21.15.46:0/684950034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf8003ad0 con 0x7fad08076ba0 2024-09-10T07:26:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.679+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0819daf0 con 0x7fad08076ba0 2024-09-10T07:26:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.679+0000 7facffe00640 1 -- 172.21.15.46:0/684950034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facf8035030 con 0x7fad08076ba0 2024-09-10T07:26:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.679+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081a07e0 con 0x7fad08076ba0 2024-09-10T07:26:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.680+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0810e470 con 0x7fad08076ba0 2024-09-10T07:26:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.680+0000 7facffe00640 1 -- 172.21.15.46:0/684950034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7facf8042070 con 0x7fad08076ba0 2024-09-10T07:26:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.681+0000 7facffe00640 1 --2- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facec0390c0 0x7facec03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.681+0000 7facffe00640 1 -- 172.21.15.46:0/684950034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7facf806cf00 con 0x7fad08076ba0 2024-09-10T07:26:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.681+0000 7fad06a00640 1 --2- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facec0390c0 0x7facec03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.681+0000 7fad06a00640 1 --2- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facec0390c0 0x7facec03b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7facf40098c0 tx=0x7facf4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.690+0000 7facffe00640 1 -- 172.21.15.46:0/684950034 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7facf803a3c0 con 0x7fad08076ba0 2024-09-10T07:26:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.880+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fad0802d830 con 0x7facec0390c0 2024-09-10T07:26:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.884+0000 7facffe00640 1 -- 172.21.15.46:0/684950034 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad0802d830 con 0x7facec0390c0 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facec0390c0 msgr2=0x7facec03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 --2- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facec0390c0 0x7facec03b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7facf40098c0 tx=0x7facf4006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 msgr2=0x7fad0819d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 --2- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 0x7fad0819d320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7facf8004f60 tx=0x7facf80048f0 comp rx=0 tx=0).stop 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 shutdown_connections 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 --2- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facec0390c0 0x7facec03b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 --2- 172.21.15.46:0/684950034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad08076ba0 0x7fad0819d320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 >> 172.21.15.46:0/684950034 conn(0x7fad080fefc0 msgr2=0x7fad080ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 shutdown_connections 2024-09-10T07:26:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:25.887+0000 7fad0e600640 1 -- 172.21.15.46:0/684950034 wait complete. 2024-09-10T07:26:26.697 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi063 2024-09-10T07:26:26.697 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:26:26.697 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/etc/ceph/ceph.conf 2024-09-10T07:26:26.716 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:26:26.717 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:26:26.774 INFO:tasks.cephadm:Adding host smithi063 to orchestrator... 2024-09-10T07:26:26.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch host add smithi063 2024-09-10T07:26:26.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:26 smithi046 ceph-mon[30705]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:26.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:26.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:26:26.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:26.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:26:26.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:26 smithi046 ceph-mon[30705]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T07:26:27.159 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:26:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.773+0000 7f0e35e00640 1 -- 172.21.15.46:0/2925138097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 msgr2=0x7f0e30109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.773+0000 7f0e35e00640 1 --2- 172.21.15.46:0/2925138097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 0x7f0e30109b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009880 tx=0x7f0e2002a5d0 comp rx=0 tx=0).stop 2024-09-10T07:26:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.773+0000 7f0e35e00640 1 -- 172.21.15.46:0/2925138097 shutdown_connections 2024-09-10T07:26:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.773+0000 7f0e35e00640 1 --2- 172.21.15.46:0/2925138097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 0x7f0e30109b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.773+0000 7f0e35e00640 1 -- 172.21.15.46:0/2925138097 >> 172.21.15.46:0/2925138097 conn(0x7f0e300ff450 msgr2=0x7f0e30101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.773+0000 7f0e35e00640 1 -- 172.21.15.46:0/2925138097 shutdown_connections 2024-09-10T07:26:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.773+0000 7f0e35e00640 1 -- 172.21.15.46:0/2925138097 wait complete. 2024-09-10T07:26:27.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.774+0000 7f0e35e00640 1 Processor -- start 2024-09-10T07:26:27.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.774+0000 7f0e35e00640 1 -- start start 2024-09-10T07:26:27.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.774+0000 7f0e35e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 0x7f0e301993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:27.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.774+0000 7f0e35e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3010de30 con 0x7f0e30109770 2024-09-10T07:26:27.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.775+0000 7f0e2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 0x7f0e301993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:27.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.775+0000 7f0e2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 0x7f0e301993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54896/0 (socket says 172.21.15.46:54896) 2024-09-10T07:26:27.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.775+0000 7f0e2ea00640 1 -- 172.21.15.46:0/3586677472 learned_addr learned my addr 172.21.15.46:0/3586677472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:27.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.775+0000 7f0e2ea00640 1 -- 172.21.15.46:0/3586677472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e301998e0 con 0x7f0e30109770 2024-09-10T07:26:27.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.776+0000 7f0e2ea00640 1 --2- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 0x7f0e301993a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009230 tx=0x7f0e20003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.776+0000 7f0e34a00640 1 -- 172.21.15.46:0/3586677472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e2003f070 con 0x7f0e30109770 2024-09-10T07:26:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.776+0000 7f0e34a00640 1 -- 172.21.15.46:0/3586677472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2003bd40 con 0x7f0e30109770 2024-09-10T07:26:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.776+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e30199bd0 con 0x7f0e30109770 2024-09-10T07:26:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.776+0000 7f0e34a00640 1 -- 172.21.15.46:0/3586677472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e20035030 con 0x7f0e30109770 2024-09-10T07:26:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.776+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3019dc70 con 0x7f0e30109770 2024-09-10T07:26:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.777+0000 7f0e34a00640 1 -- 172.21.15.46:0/3586677472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0e20042070 con 0x7f0e30109770 2024-09-10T07:26:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.777+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e30104eb0 con 0x7f0e30109770 2024-09-10T07:26:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.778+0000 7f0e34a00640 1 --2- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0e140390c0 0x7f0e1403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.778+0000 7f0e34a00640 1 -- 172.21.15.46:0/3586677472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0e2006c830 con 0x7f0e30109770 2024-09-10T07:26:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.778+0000 7f0e2e000640 1 --2- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0e140390c0 0x7f0e1403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.779+0000 7f0e2e000640 1 --2- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0e140390c0 0x7f0e1403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0098c0 tx=0x7f0e1c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.783+0000 7f0e34a00640 1 -- 172.21.15.46:0/3586677472 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e20044e10 con 0x7f0e30109770 2024-09-10T07:26:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:27 smithi046 ceph-mon[30705]: Updating smithi046:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:26:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:27 smithi046 ceph-mon[30705]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:26:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:27 smithi046 ceph-mon[30705]: Updating smithi046:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T07:26:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:27.988+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi063", "target": ["mon-mgr", ""]}) -- 0x7f0e30109c30 con 0x7f0e140390c0 2024-09-10T07:26:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:28 smithi046 ceph-mon[30705]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi063", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:29 smithi046 ceph-mon[30705]: Deploying cephadm binary to smithi063 2024-09-10T07:26:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.874+0000 7f0e34a00640 1 -- 172.21.15.46:0/3586677472 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f0e30109c30 con 0x7f0e140390c0 2024-09-10T07:26:31.875 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi063' with addr '172.21.15.63' 2024-09-10T07:26:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.876+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0e140390c0 msgr2=0x7f0e1403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.876+0000 7f0e35e00640 1 --2- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0e140390c0 0x7f0e1403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0098c0 tx=0x7f0e1c006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.876+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 msgr2=0x7f0e301993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.876+0000 7f0e35e00640 1 --2- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 0x7f0e301993a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009230 tx=0x7f0e20003d80 comp rx=0 tx=0).stop 2024-09-10T07:26:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.876+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 shutdown_connections 2024-09-10T07:26:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.876+0000 7f0e35e00640 1 --2- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0e140390c0 0x7f0e1403b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.876+0000 7f0e35e00640 1 --2- 172.21.15.46:0/3586677472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e30109770 0x7f0e301993a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.876+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 >> 172.21.15.46:0/3586677472 conn(0x7f0e300ff450 msgr2=0x7f0e30107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.877+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 shutdown_connections 2024-09-10T07:26:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:31.877+0000 7f0e35e00640 1 -- 172.21.15.46:0/3586677472 wait complete. 2024-09-10T07:26:32.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T07:26:32.948 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:26:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:32 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:32 smithi046 ceph-mon[30705]: Added host smithi063 2024-09-10T07:26:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:32 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:26:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:32 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:32 smithi046 ceph-mon[30705]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:32 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.460+0000 7f97ad800640 1 -- 172.21.15.46:0/3097682435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 msgr2=0x7f97a8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.461+0000 7f97ad800640 1 --2- 172.21.15.46:0/3097682435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 0x7f97a8074970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9798009880 tx=0x7f979802a670 comp rx=0 tx=0).stop 2024-09-10T07:26:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.461+0000 7f97ad800640 1 -- 172.21.15.46:0/3097682435 shutdown_connections 2024-09-10T07:26:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.461+0000 7f97ad800640 1 --2- 172.21.15.46:0/3097682435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 0x7f97a8074970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.461+0000 7f97ad800640 1 -- 172.21.15.46:0/3097682435 >> 172.21.15.46:0/3097682435 conn(0x7f97a80fd290 msgr2=0x7f97a80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.461+0000 7f97ad800640 1 -- 172.21.15.46:0/3097682435 shutdown_connections 2024-09-10T07:26:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.461+0000 7f97ad800640 1 -- 172.21.15.46:0/3097682435 wait complete. 2024-09-10T07:26:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.462+0000 7f97ad800640 1 Processor -- start 2024-09-10T07:26:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.462+0000 7f97ad800640 1 -- start start 2024-09-10T07:26:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.462+0000 7f97ad800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 0x7f97a81a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.462+0000 7f97ad800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81122b0 con 0x7f97a8074590 2024-09-10T07:26:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.463+0000 7f97a6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 0x7f97a81a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.463+0000 7f97a6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 0x7f97a81a33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54920/0 (socket says 172.21.15.46:54920) 2024-09-10T07:26:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.463+0000 7f97a6a00640 1 -- 172.21.15.46:0/1718023106 learned_addr learned my addr 172.21.15.46:0/1718023106 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.463+0000 7f97a6a00640 1 -- 172.21.15.46:0/1718023106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a81a3930 con 0x7f97a8074590 2024-09-10T07:26:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.463+0000 7f97a6a00640 1 --2- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 0x7f97a81a33f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9798009230 tx=0x7f9798003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.464+0000 7f979f400640 1 -- 172.21.15.46:0/1718023106 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f979803f070 con 0x7f97a8074590 2024-09-10T07:26:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.464+0000 7f979f400640 1 -- 172.21.15.46:0/1718023106 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979803bd40 con 0x7f97a8074590 2024-09-10T07:26:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.464+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a81a3bc0 con 0x7f97a8074590 2024-09-10T07:26:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.464+0000 7f979f400640 1 -- 172.21.15.46:0/1718023106 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9798035030 con 0x7f97a8074590 2024-09-10T07:26:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.464+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a819d5b0 con 0x7f97a8074590 2024-09-10T07:26:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.465+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a8075cd0 con 0x7f97a8074590 2024-09-10T07:26:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.465+0000 7f979f400640 1 -- 172.21.15.46:0/1718023106 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9798042070 con 0x7f97a8074590 2024-09-10T07:26:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.465+0000 7f979f400640 1 --2- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f978c039060 0x7f978c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.465+0000 7f979f400640 1 -- 172.21.15.46:0/1718023106 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f979806cd20 con 0x7f97a8074590 2024-09-10T07:26:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.466+0000 7f979fe00640 1 --2- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f978c039060 0x7f978c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:33.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.466+0000 7f979fe00640 1 --2- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f978c039060 0x7f978c03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97900098c0 tx=0x7f9790006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.470+0000 7f979f400640 1 -- 172.21.15.46:0/1718023106 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97980412c0 con 0x7f97a8074590 2024-09-10T07:26:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.663+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97a819eae0 con 0x7f978c039060 2024-09-10T07:26:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.664+0000 7f979f400640 1 -- 172.21.15.46:0/1718023106 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f97a819eae0 con 0x7f978c039060 2024-09-10T07:26:33.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:33.665 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}] 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f978c039060 msgr2=0x7f978c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 --2- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f978c039060 0x7f978c03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97900098c0 tx=0x7f9790006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 msgr2=0x7f97a81a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 --2- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 0x7f97a81a33f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9798009230 tx=0x7f9798003d80 comp rx=0 tx=0).stop 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 shutdown_connections 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 --2- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f978c039060 0x7f978c03b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 --2- 172.21.15.46:0/1718023106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a8074590 0x7f97a81a33f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 >> 172.21.15.46:0/1718023106 conn(0x7f97a80fd290 msgr2=0x7f97a80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 shutdown_connections 2024-09-10T07:26:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:33.667+0000 7f97ad800640 1 -- 172.21.15.46:0/1718023106 wait complete. 2024-09-10T07:26:34.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:34.398 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2024-09-10T07:26:34.398 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:26:34.398 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2024-09-10T07:26:34.416 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:26:34.416 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:26:34.471 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2024-09-10T07:26:34.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch host add smithi181 2024-09-10T07:26:34.834 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:26:35.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.334+0000 7fa871000640 1 -- 172.21.15.46:0/4068481226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 msgr2=0x7fa86c109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:35.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.334+0000 7fa871000640 1 --2- 172.21.15.46:0/4068481226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 0x7fa86c109b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009880 tx=0x7fa85c02a5d0 comp rx=0 tx=0).stop 2024-09-10T07:26:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.334+0000 7fa871000640 1 -- 172.21.15.46:0/4068481226 shutdown_connections 2024-09-10T07:26:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.334+0000 7fa871000640 1 --2- 172.21.15.46:0/4068481226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 0x7fa86c109b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.334+0000 7fa871000640 1 -- 172.21.15.46:0/4068481226 >> 172.21.15.46:0/4068481226 conn(0x7fa86c0ff450 msgr2=0x7fa86c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.334+0000 7fa871000640 1 -- 172.21.15.46:0/4068481226 shutdown_connections 2024-09-10T07:26:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.334+0000 7fa871000640 1 -- 172.21.15.46:0/4068481226 wait complete. 2024-09-10T07:26:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.335+0000 7fa871000640 1 Processor -- start 2024-09-10T07:26:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.335+0000 7fa871000640 1 -- start start 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.335+0000 7fa871000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 0x7fa86c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.335+0000 7fa871000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c10de30 con 0x7fa86c109770 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.336+0000 7fa869e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 0x7fa86c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.336+0000 7fa869e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 0x7fa86c1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54932/0 (socket says 172.21.15.46:54932) 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.336+0000 7fa869e00640 1 -- 172.21.15.46:0/129107954 learned_addr learned my addr 172.21.15.46:0/129107954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.336+0000 7fa869e00640 1 -- 172.21.15.46:0/129107954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c1998e0 con 0x7fa86c109770 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.336+0000 7fa869e00640 1 --2- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 0x7fa86c1993a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009230 tx=0x7fa85c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.337+0000 7fa862a00640 1 -- 172.21.15.46:0/129107954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa85c03f070 con 0x7fa86c109770 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.337+0000 7fa862a00640 1 -- 172.21.15.46:0/129107954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa85c03bd40 con 0x7fa86c109770 2024-09-10T07:26:35.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.337+0000 7fa862a00640 1 -- 172.21.15.46:0/129107954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa85c035030 con 0x7fa86c109770 2024-09-10T07:26:35.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.337+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c199bd0 con 0x7fa86c109770 2024-09-10T07:26:35.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.337+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c19dc70 con 0x7fa86c109770 2024-09-10T07:26:35.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.338+0000 7fa862a00640 1 -- 172.21.15.46:0/129107954 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa85c042070 con 0x7fa86c109770 2024-09-10T07:26:35.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.338+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa86c104eb0 con 0x7fa86c109770 2024-09-10T07:26:35.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.338+0000 7fa862a00640 1 --2- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500390c0 0x7fa85003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:35.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.338+0000 7fa862a00640 1 -- 172.21.15.46:0/129107954 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa85c06cc70 con 0x7fa86c109770 2024-09-10T07:26:35.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.339+0000 7fa869400640 1 --2- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500390c0 0x7fa85003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:35.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.339+0000 7fa869400640 1 --2- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500390c0 0x7fa85003b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa8580098c0 tx=0x7fa858006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:35.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.344+0000 7fa862a00640 1 -- 172.21.15.46:0/129107954 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa85c030d60 con 0x7fa86c109770 2024-09-10T07:26:35.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:35.532+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) -- 0x7fa86c109c30 con 0x7fa8500390c0 2024-09-10T07:26:35.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:35 smithi046 ceph-mon[30705]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:26:35.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:35 smithi046 ceph-mon[30705]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: Updating smithi063:/etc/ceph/ceph.conf 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: Updating smithi063:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:37.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:36 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:37 smithi046 ceph-mon[30705]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:26:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:37 smithi046 ceph-mon[30705]: Updating smithi063:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T07:26:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:37 smithi046 ceph-mon[30705]: Deploying cephadm binary to smithi181 2024-09-10T07:26:39.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:38 smithi046 ceph-mon[30705]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.453+0000 7fa862a00640 1 -- 172.21.15.46:0/129107954 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa86c109c30 con 0x7fa8500390c0 2024-09-10T07:26:39.454 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi181' with addr '172.21.15.181' 2024-09-10T07:26:39.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.455+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500390c0 msgr2=0x7fa85003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.455+0000 7fa871000640 1 --2- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500390c0 0x7fa85003b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa8580098c0 tx=0x7fa858006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.455+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 msgr2=0x7fa86c1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.455+0000 7fa871000640 1 --2- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 0x7fa86c1993a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009230 tx=0x7fa85c003d80 comp rx=0 tx=0).stop 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.456+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 shutdown_connections 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.456+0000 7fa871000640 1 --2- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500390c0 0x7fa85003b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.456+0000 7fa871000640 1 --2- 172.21.15.46:0/129107954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c109770 0x7fa86c1993a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.456+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 >> 172.21.15.46:0/129107954 conn(0x7fa86c0ff450 msgr2=0x7fa86c107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.456+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 shutdown_connections 2024-09-10T07:26:39.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:39.456+0000 7fa871000640 1 -- 172.21.15.46:0/129107954 wait complete. 2024-09-10T07:26:40.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T07:26:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:26:40.802 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:40.802 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:40 smithi046 ceph-mon[30705]: Added host smithi181 2024-09-10T07:26:40.802 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:26:40.802 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:40.802 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:40.803 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:40 smithi046 ceph-mon[30705]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.080+0000 7f76ca600640 1 -- 172.21.15.46:0/3381601531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 msgr2=0x7f76c4109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.080+0000 7f76ca600640 1 --2- 172.21.15.46:0/3381601531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 0x7f76c4109b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009880 tx=0x7f76b802a5d0 comp rx=0 tx=0).stop 2024-09-10T07:26:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.080+0000 7f76ca600640 1 -- 172.21.15.46:0/3381601531 shutdown_connections 2024-09-10T07:26:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.080+0000 7f76ca600640 1 --2- 172.21.15.46:0/3381601531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 0x7f76c4109b20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.080+0000 7f76ca600640 1 -- 172.21.15.46:0/3381601531 >> 172.21.15.46:0/3381601531 conn(0x7f76c40ff2c0 msgr2=0x7f76c41016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.080+0000 7f76ca600640 1 -- 172.21.15.46:0/3381601531 shutdown_connections 2024-09-10T07:26:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.080+0000 7f76ca600640 1 -- 172.21.15.46:0/3381601531 wait complete. 2024-09-10T07:26:41.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.081+0000 7f76ca600640 1 Processor -- start 2024-09-10T07:26:41.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.081+0000 7f76ca600640 1 -- start start 2024-09-10T07:26:41.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.081+0000 7f76ca600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 0x7f76c41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:41.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.081+0000 7f76ca600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76c410dd50 con 0x7f76c4109740 2024-09-10T07:26:41.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.081+0000 7f76c9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 0x7f76c41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:41.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.082+0000 7f76c9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 0x7f76c41991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49074/0 (socket says 172.21.15.46:49074) 2024-09-10T07:26:41.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.082+0000 7f76c9200640 1 -- 172.21.15.46:0/2886328316 learned_addr learned my addr 172.21.15.46:0/2886328316 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:41.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.082+0000 7f76c9200640 1 -- 172.21.15.46:0/2886328316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c4199720 con 0x7f76c4109740 2024-09-10T07:26:41.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.082+0000 7f76c9200640 1 --2- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 0x7f76c41991e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009230 tx=0x7f76b8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.082+0000 7f76c1600640 1 -- 172.21.15.46:0/2886328316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b803f070 con 0x7f76c4109740 2024-09-10T07:26:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.082+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c4199a10 con 0x7f76c4109740 2024-09-10T07:26:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.082+0000 7f76c1600640 1 -- 172.21.15.46:0/2886328316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b803bc80 con 0x7f76c4109740 2024-09-10T07:26:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.083+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c419dab0 con 0x7f76c4109740 2024-09-10T07:26:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.083+0000 7f76c1600640 1 -- 172.21.15.46:0/2886328316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b8049060 con 0x7f76c4109740 2024-09-10T07:26:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.084+0000 7f76c1600640 1 -- 172.21.15.46:0/2886328316 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f76b8042070 con 0x7f76c4109740 2024-09-10T07:26:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.084+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c4104ce0 con 0x7f76c4109740 2024-09-10T07:26:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.084+0000 7f76c1600640 1 --2- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76ac039060 0x7f76ac03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.084+0000 7f76c1600640 1 -- 172.21.15.46:0/2886328316 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f76b806c9c0 con 0x7f76c4109740 2024-09-10T07:26:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.084+0000 7f76c3e00640 1 --2- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76ac039060 0x7f76ac03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:41.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.085+0000 7f76c3e00640 1 --2- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76ac039060 0x7f76ac03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f76b00098c0 tx=0x7f76b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.089+0000 7f76c1600640 1 -- 172.21.15.46:0/2886328316 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76b8031570 con 0x7f76c4109740 2024-09-10T07:26:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.278+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76c4109bd0 con 0x7f76ac039060 2024-09-10T07:26:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.279+0000 7f76c1600640 1 -- 172.21.15.46:0/2886328316 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f76c4109bd0 con 0x7f76ac039060 2024-09-10T07:26:41.280 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:26:41.280 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2024-09-10T07:26:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.281+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76ac039060 msgr2=0x7f76ac03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.281+0000 7f76ca600640 1 --2- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76ac039060 0x7f76ac03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f76b00098c0 tx=0x7f76b0006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.281+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 msgr2=0x7f76c41991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.281+0000 7f76ca600640 1 --2- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 0x7f76c41991e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009230 tx=0x7f76b8003d40 comp rx=0 tx=0).stop 2024-09-10T07:26:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.283+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 shutdown_connections 2024-09-10T07:26:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.283+0000 7f76ca600640 1 --2- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76ac039060 0x7f76ac03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.283+0000 7f76ca600640 1 --2- 172.21.15.46:0/2886328316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c4109740 0x7f76c41991e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.283+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 >> 172.21.15.46:0/2886328316 conn(0x7f76c40ff2c0 msgr2=0x7f76c40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.283+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 shutdown_connections 2024-09-10T07:26:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:41.283+0000 7f76ca600640 1 -- 172.21.15.46:0/2886328316 wait complete. 2024-09-10T07:26:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:41 smithi046 ceph-mon[30705]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:26:41.951 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T07:26:41.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T07:26:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.897+0000 7f220f800640 1 -- 172.21.15.46:0/1856384295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 msgr2=0x7f2208109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.897+0000 7f220f800640 1 --2- 172.21.15.46:0/1856384295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 0x7f2208109b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009880 tx=0x7f21fc02a5d0 comp rx=0 tx=0).stop 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.897+0000 7f220f800640 1 -- 172.21.15.46:0/1856384295 shutdown_connections 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.897+0000 7f220f800640 1 --2- 172.21.15.46:0/1856384295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 0x7f2208109b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.897+0000 7f220f800640 1 -- 172.21.15.46:0/1856384295 >> 172.21.15.46:0/1856384295 conn(0x7f22080ff450 msgr2=0x7f2208101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.897+0000 7f220f800640 1 -- 172.21.15.46:0/1856384295 shutdown_connections 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.897+0000 7f220f800640 1 -- 172.21.15.46:0/1856384295 wait complete. 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.898+0000 7f220f800640 1 Processor -- start 2024-09-10T07:26:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.898+0000 7f220f800640 1 -- start start 2024-09-10T07:26:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.898+0000 7f220f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 0x7f22081993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.898+0000 7f220f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f220810de30 con 0x7f2208109770 2024-09-10T07:26:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.899+0000 7f220ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 0x7f22081993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.899+0000 7f220ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 0x7f22081993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49090/0 (socket says 172.21.15.46:49090) 2024-09-10T07:26:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.899+0000 7f220ce00640 1 -- 172.21.15.46:0/485082121 learned_addr learned my addr 172.21.15.46:0/485082121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:26:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.899+0000 7f220ce00640 1 -- 172.21.15.46:0/485082121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081998e0 con 0x7f2208109770 2024-09-10T07:26:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.899+0000 7f220ce00640 1 --2- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 0x7f22081993a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009230 tx=0x7f21fc003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.900+0000 7f2205600640 1 -- 172.21.15.46:0/485082121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21fc03f070 con 0x7f2208109770 2024-09-10T07:26:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.900+0000 7f2205600640 1 -- 172.21.15.46:0/485082121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc03bd40 con 0x7f2208109770 2024-09-10T07:26:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.900+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2208199bd0 con 0x7f2208109770 2024-09-10T07:26:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.900+0000 7f2205600640 1 -- 172.21.15.46:0/485082121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21fc035030 con 0x7f2208109770 2024-09-10T07:26:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.900+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220819dc70 con 0x7f2208109770 2024-09-10T07:26:42.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.901+0000 7f2205600640 1 -- 172.21.15.46:0/485082121 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f21fc042070 con 0x7f2208109770 2024-09-10T07:26:42.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.901+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2208104eb0 con 0x7f2208109770 2024-09-10T07:26:42.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.902+0000 7f2205600640 1 --2- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f21f0039060 0x7f21f003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:42.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.902+0000 7f2205600640 1 -- 172.21.15.46:0/485082121 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f21fc06cc70 con 0x7f2208109770 2024-09-10T07:26:42.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.902+0000 7f2207e00640 1 --2- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f21f0039060 0x7f21f003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:42.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.903+0000 7f2207e00640 1 --2- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f21f0039060 0x7f21f003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21f40098c0 tx=0x7f21f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:42.906+0000 7f2205600640 1 -- 172.21.15.46:0/485082121 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21fc036df0 con 0x7f2208109770 2024-09-10T07:26:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.096+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2208109bd0 con 0x7f2208109770 2024-09-10T07:26:43.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:42 smithi046 ceph-mon[30705]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.824+0000 7f2205600640 1 -- 172.21.15.46:0/485082121 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f21fc030d60 con 0x7f2208109770 2024-09-10T07:26:43.825 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-09-10T07:26:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.826+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f21f0039060 msgr2=0x7f21f003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.826+0000 7f220f800640 1 --2- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f21f0039060 0x7f21f003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21f40098c0 tx=0x7f21f4006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.827+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 msgr2=0x7f22081993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.827+0000 7f220f800640 1 --2- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 0x7f22081993a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009230 tx=0x7f21fc003d80 comp rx=0 tx=0).stop 2024-09-10T07:26:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.827+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 shutdown_connections 2024-09-10T07:26:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.827+0000 7f220f800640 1 --2- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f21f0039060 0x7f21f003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.827+0000 7f220f800640 1 --2- 172.21.15.46:0/485082121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2208109770 0x7f22081993a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.827+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 >> 172.21.15.46:0/485082121 conn(0x7f22080ff450 msgr2=0x7f2208107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.827+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 shutdown_connections 2024-09-10T07:26:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:26:43.827+0000 7f220f800640 1 -- 172.21.15.46:0/485082121 wait complete. 2024-09-10T07:26:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:43 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/485082121' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T07:26:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:26:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:26:44.590 INFO:tasks.cephadm:Adding mon.a on smithi046 2024-09-10T07:26:44.590 INFO:tasks.cephadm:Adding mon.b on smithi063 2024-09-10T07:26:44.591 INFO:tasks.cephadm:Adding mon.c on smithi181 2024-09-10T07:26:44.591 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi046:172.21.15.46=a;smithi063:172.21.15.63=b;smithi181:172.21.15.181=c' 2024-09-10T07:26:44.821 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:26:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-10T07:26:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: Updating smithi181:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:26:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/485082121' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T07:26:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:26:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:26:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: Updating smithi181:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T07:26:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:45.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:44 smithi046 ceph-mon[30705]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:45.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.288+0000 7eff13bae640 1 -- 172.21.15.181:0/145746636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 msgr2=0x7eff0c102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:45.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.288+0000 7eff13bae640 1 --2- 172.21.15.181:0/145746636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 0x7eff0c102b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff00009810 tx=0x7eff0002a540 comp rx=0 tx=0).stop 2024-09-10T07:26:45.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.288+0000 7eff13bae640 1 -- 172.21.15.181:0/145746636 shutdown_connections 2024-09-10T07:26:45.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.288+0000 7eff13bae640 1 --2- 172.21.15.181:0/145746636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 0x7eff0c102b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:45.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.288+0000 7eff13bae640 1 -- 172.21.15.181:0/145746636 >> 172.21.15.181:0/145746636 conn(0x7eff0c077310 msgr2=0x7eff0c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:45.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.289+0000 7eff13bae640 1 -- 172.21.15.181:0/145746636 shutdown_connections 2024-09-10T07:26:45.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.289+0000 7eff13bae640 1 -- 172.21.15.181:0/145746636 wait complete. 2024-09-10T07:26:45.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.289+0000 7eff13bae640 1 Processor -- start 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.289+0000 7eff13bae640 1 -- start start 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.289+0000 7eff13bae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 0x7eff0c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.289+0000 7eff13bae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10bc80 con 0x7eff0c102730 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.290+0000 7eff1194a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 0x7eff0c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.290+0000 7eff1194a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 0x7eff0c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.181:59240/0 (socket says 172.21.15.181:59240) 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.290+0000 7eff1194a640 1 -- 172.21.15.181:0/735323284 learned_addr learned my addr 172.21.15.181:0/735323284 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.290+0000 7eff1194a640 1 -- 172.21.15.181:0/735323284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c1976f0 con 0x7eff0c102730 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.290+0000 7eff1194a640 1 --2- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 0x7eff0c1971b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff000045e0 tx=0x7eff00004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.290+0000 7efefeffd640 1 -- 172.21.15.181:0/735323284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00030e70 con 0x7eff0c102730 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.290+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c197980 con 0x7eff0c102730 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.291+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c197de0 con 0x7eff0c102730 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.291+0000 7efefeffd640 1 -- 172.21.15.181:0/735323284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff000318e0 con 0x7eff0c102730 2024-09-10T07:26:45.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.291+0000 7efefeffd640 1 -- 172.21.15.181:0/735323284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00049060 con 0x7eff0c102730 2024-09-10T07:26:45.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.291+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef0005290 con 0x7eff0c102730 2024-09-10T07:26:45.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.291+0000 7efefeffd640 1 -- 172.21.15.181:0/735323284 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7eff00039040 con 0x7eff0c102730 2024-09-10T07:26:45.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.292+0000 7efefeffd640 1 --2- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efef8039060 0x7efef803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:45.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.292+0000 7efefeffd640 1 -- 172.21.15.181:0/735323284 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7eff0006cf70 con 0x7eff0c102730 2024-09-10T07:26:45.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.292+0000 7eff11149640 1 --2- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efef8039060 0x7efef803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:45.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.293+0000 7eff11149640 1 --2- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efef8039060 0x7efef803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff080098c0 tx=0x7eff08006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:45.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.297+0000 7efefeffd640 1 -- 172.21.15.181:0/735323284 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff00040030 con 0x7eff0c102730 2024-09-10T07:26:45.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.482+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi063:172.21.15.63=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}) -- 0x7efef0000c80 con 0x7efef8039060 2024-09-10T07:26:45.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.488+0000 7efefeffd640 1 -- 172.21.15.181:0/735323284 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efef0000c80 con 0x7efef8039060 2024-09-10T07:26:45.488 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mon update... 2024-09-10T07:26:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efef8039060 msgr2=0x7efef803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 --2- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efef8039060 0x7efef803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff080098c0 tx=0x7eff08006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 msgr2=0x7eff0c1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 --2- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 0x7eff0c1971b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff000045e0 tx=0x7eff00004610 comp rx=0 tx=0).stop 2024-09-10T07:26:45.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 shutdown_connections 2024-09-10T07:26:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 --2- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efef8039060 0x7efef803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 --2- 172.21.15.181:0/735323284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c102730 0x7eff0c1971b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 >> 172.21.15.181:0/735323284 conn(0x7eff0c077310 msgr2=0x7eff0c1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 shutdown_connections 2024-09-10T07:26:45.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:45.490+0000 7eff13bae640 1 -- 172.21.15.181:0/735323284 wait complete. 2024-09-10T07:26:46.072 DEBUG:teuthology.orchestra.run.smithi063:mon.b> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T07:26:46.075 DEBUG:teuthology.orchestra.run.smithi181:mon.c> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T07:26:46.078 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T07:26:46.078 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T07:26:46.792 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:26:46.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi063:172.21.15.63=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:46.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi063:172.21.15.63=b;smithi181:172.21.15.181=c;count:3 2024-09-10T07:26:46.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:46.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:26:46.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:46.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:26:46.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:46.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:26:46.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:46.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: Deploying daemon mon.c on smithi181 2024-09-10T07:26:46.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:46 smithi046 ceph-mon[30705]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:48.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.569+0000 7f103c902640 1 -- 172.21.15.181:0/596494914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 msgr2=0x7f1038106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:48.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.569+0000 7f103c902640 1 --2- 172.21.15.181:0/596494914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 0x7f1038106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1028009810 tx=0x7f102802a540 comp rx=0 tx=0).stop 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.569+0000 7f103c902640 1 -- 172.21.15.181:0/596494914 shutdown_connections 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.569+0000 7f103c902640 1 --2- 172.21.15.181:0/596494914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 0x7f1038106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.569+0000 7f103c902640 1 -- 172.21.15.181:0/596494914 >> 172.21.15.181:0/596494914 conn(0x7f1038077310 msgr2=0x7f10380fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.569+0000 7f103c902640 1 -- 172.21.15.181:0/596494914 shutdown_connections 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.569+0000 7f103c902640 1 -- 172.21.15.181:0/596494914 wait complete. 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.569+0000 7f103c902640 1 Processor -- start 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.570+0000 7f103c902640 1 -- start start 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.570+0000 7f103c902640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 0x7f1038196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.570+0000 7f103c902640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f103810ab70 con 0x7f1038106540 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.570+0000 7f103659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 0x7f1038196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.570+0000 7f103659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 0x7f1038196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.181:59264/0 (socket says 172.21.15.181:59264) 2024-09-10T07:26:48.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.570+0000 7f103659c640 1 -- 172.21.15.181:0/267609573 learned_addr learned my addr 172.21.15.181:0/267609573 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.570+0000 7f103659c640 1 -- 172.21.15.181:0/267609573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10381965a0 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.571+0000 7f103659c640 1 --2- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 0x7f1038196060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10280045e0 tx=0x7f1028004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.571+0000 7f10277fe640 1 -- 172.21.15.181:0/267609573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1028030e70 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.571+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1038196830 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.571+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1038196c90 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.571+0000 7f10277fe640 1 -- 172.21.15.181:0/267609573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10280318e0 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.571+0000 7f10277fe640 1 -- 172.21.15.181:0/267609573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1028049060 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.572+0000 7f10277fe640 1 -- 172.21.15.181:0/267609573 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1028039040 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.572+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1038101bb0 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.572+0000 7f10277fe640 1 --2- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1020039060 0x7f102003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.572+0000 7f10277fe640 1 -- 172.21.15.181:0/267609573 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f102806cf60 con 0x7f1038106540 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.572+0000 7f1035d9b640 1 --2- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1020039060 0x7f102003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.573+0000 7f1035d9b640 1 --2- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1020039060 0x7f102003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f102c0098c0 tx=0x7f102c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:48.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.577+0000 7f10277fe640 1 -- 172.21.15.181:0/267609573 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1028040030 con 0x7f1038106540 2024-09-10T07:26:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.833+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1038197420 con 0x7f1038106540 2024-09-10T07:26:48.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.834+0000 7f10277fe640 1 -- 172.21.15.181:0/267609573 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f1028040e90 con 0x7f1038106540 2024-09-10T07:26:48.834 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:26:48.835 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","modified":"2024-09-10T07:25:15.786098Z","created":"2024-09-10T07:25:15.786098Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T07:26:48.835 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1020039060 msgr2=0x7f102003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 --2- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1020039060 0x7f102003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f102c0098c0 tx=0x7f102c006db0 comp rx=0 tx=0).stop 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 msgr2=0x7f1038196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 --2- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 0x7f1038196060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10280045e0 tx=0x7f1028004610 comp rx=0 tx=0).stop 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 shutdown_connections 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 --2- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1020039060 0x7f102003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 --2- 172.21.15.181:0/267609573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1038106540 0x7f1038196060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 >> 172.21.15.181:0/267609573 conn(0x7f1038077310 msgr2=0x7f10380fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 shutdown_connections 2024-09-10T07:26:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:48.837+0000 7f103c902640 1 -- 172.21.15.181:0/267609573 wait complete. 2024-09-10T07:26:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:48 smithi046 ceph-mon[30705]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:49.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:49 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/267609573' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:26:50.060 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 podman[35378]: 2024-09-10 07:26:50.038791112 +0000 UTC m=+0.346964590 container start 1f55219d589cf01ff2f7542c4364c9847ff402cf5d002330486059b685792b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True) 2024-09-10T07:26:50.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: pidfile_write: ignore empty --pid-file 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: load: jerasure load: lrc 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: RocksDB version: 7.9.2 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Git sha 0 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: DB SUMMARY 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: DB Session ID: LWYWHFAA9R85CEBFM4LT 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: CURRENT file: CURRENT 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T07:26:50.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.error_if_exists: 0 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.create_if_missing: 0 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.paranoid_checks: 1 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.env: 0x5632af11e940 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.info_log: 0x5632b148b6e0 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.statistics: (nil) 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.use_fsync: 0 2024-09-10T07:26:50.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_log_file_size: 0 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.allow_fallocate: 1 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.use_direct_reads: 0 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.db_log_dir: 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.wal_dir: 2024-09-10T07:26:50.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T07:26:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T07:26:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T07:26:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T07:26:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T07:26:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T07:26:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T07:26:50.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T07:26:50.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.write_buffer_manager: 0x5632b149adc0 2024-09-10T07:26:50.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T07:26:50.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T07:26:50.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.unordered_write: 0 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.row_cache: None 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.wal_filter: None 2024-09-10T07:26:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.two_write_queues: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.wal_compression: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.atomic_flush: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.log_readahead_size: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T07:26:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_background_jobs: 2 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_background_compactions: -1 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_subcompactions: 1 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T07:26:50.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_open_files: -1 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_background_flushes: -1 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Compression algorithms supported: 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: kZSTD supported: 0 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: kXpressCompression supported: 0 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: kBZip2Compression supported: 0 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: kLZ4Compression supported: 1 2024-09-10T07:26:50.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: kZlibCompression supported: 1 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: kSnappyCompression supported: 1 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.merge_operator: 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_filter: None 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5632b148b320) 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout: cache_index_and_filter_blocks: 1 2024-09-10T07:26:50.411 INFO:journalctl@ceph.mon.c.smithi181.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: pin_top_level_index_and_filter: 1 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_type: 0 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: data_block_index_type: 0 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_shortening: 1 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: checksum: 4 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: no_block_cache: 0 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache: 0x5632b147f1f0 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_name: BinnedLRUCache 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_options: 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: capacity : 536870912 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: num_shard_bits : 4 2024-09-10T07:26:50.412 INFO:journalctl@ceph.mon.c.smithi181.stdout: strict_capacity_limit : 0 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: high_pri_pool_ratio: 0.000 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_compressed: (nil) 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: persistent_cache: (nil) 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_size: 4096 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_size_deviation: 10 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_restart_interval: 16 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_block_restart_interval: 1 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: metadata_block_size: 4096 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: partition_filters: 0 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: use_delta_encoding: 1 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: filter_policy: bloomfilter 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: whole_key_filtering: 1 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: verify_compression: 0 2024-09-10T07:26:50.413 INFO:journalctl@ceph.mon.c.smithi181.stdout: read_amp_bytes_per_bit: 0 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout: format_version: 5 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout: enable_index_compression: 1 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_align: 0 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout: max_auto_readahead_size: 262144 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout: prepopulate_block_cache: 0 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout: initial_auto_readahead_size: 8192 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression: NoCompression 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.num_levels: 7 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T07:26:50.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T07:26:50.415 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T07:26:50.416 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T07:26:50.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.inplace_update_support: 0 2024-09-10T07:26:50.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.bloom_locality: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.max_successive_merges: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.ttl: 2592000 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.enable_blob_files: false 2024-09-10T07:26:50.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.min_blob_size: 0 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 186c8dd9-48d6-498d-896e-5425ae9616ef 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725953210065859, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T07:26:50.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725953210066265, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725953210, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "186c8dd9-48d6-498d-896e-5425ae9616ef", "db_session_id": "LWYWHFAA9R85CEBFM4LT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725953210066356, "job": 1, "event": "recovery_finished"} 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5632b14ace00 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: DB pointer 0x5632b15a8000 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** DB Stats ** 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T07:26:50.421 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** Compaction Stats [default] ** 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** Compaction Stats [default] ** 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T07:26:50.422 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: Block cache BinnedLRUCache@0x5632b147f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T07:26:50.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: using public_addr v2:172.21.15.181:0/0 -> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: starting mon.c rank -1 at public addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] at bind addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(???) e0 preinit fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: e1 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: btime 2024-09-10T07:25:19:596582+0000 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: legacy client fscid: -1 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-09-10T07:26:50.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: No filesystems configured 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:50.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/485082121' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: Updating smithi181:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/485082121' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: Updating smithi181:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T07:26:50.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi063:172.21.15.63=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi063:172.21.15.63=b;smithi181:172.21.15.181=c;count:3 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: Deploying daemon mon.c on smithi181 2024-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:50.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:50.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/267609573' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:26:50.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 ceph-mon[35397]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T07:26:50.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 bash[35378]: 1f55219d589cf01ff2f7542c4364c9847ff402cf5d002330486059b685792b93 2024-09-10T07:26:50.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:50 smithi181 systemd[1]: Started Ceph mon.c for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:26:50.610 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T07:26:50.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T07:26:50.858 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:26:55.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: mon.a calling monitor election 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: mon.c calling monitor election 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: monmap epoch 2 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: last_changed 2024-09-10T07:26:50.093881+0000 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: min_mon_release 19 (squid) 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: election_strategy: 1 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: fsmap 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: mgrmap e13: x(active, since 42s) 2024-09-10T07:26:55.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: overall HEALTH_OK 2024-09-10T07:26:55.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:55.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:55.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:55.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:26:55.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:26:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: mon.a calling monitor election 2024-09-10T07:26:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: mon.c calling monitor election 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: monmap epoch 2 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: last_changed 2024-09-10T07:26:50.093881+0000 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: min_mon_release 19 (squid) 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: election_strategy: 1 2024-09-10T07:26:55.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: fsmap 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: mgrmap e13: x(active, since 42s) 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: overall HEALTH_OK 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:26:55.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:26:55.709 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.707+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3733717896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 msgr2=0x7f57c00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.707+0000 7f57dc5ce640 1 --2- 172.21.15.181:0/3733717896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 0x7f57c00059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f57d41016c0 tx=0x7f57d002c490 comp rx=0 tx=0).stop 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.707+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3733717896 shutdown_connections 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.707+0000 7f57dc5ce640 1 --2- 172.21.15.181:0/3733717896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 0x7f57c00059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.707+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3733717896 >> 172.21.15.181:0/3733717896 conn(0x7f57d40fc280 msgr2=0x7f57d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.707+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3733717896 shutdown_connections 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.707+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3733717896 wait complete. 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.708+0000 7f57dc5ce640 1 Processor -- start 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.708+0000 7f57dc5ce640 1 -- start start 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57dc5ce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 0x7f57d4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57dc5ce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 0x7f57d419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:55.710 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57dc5ce640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57d4108690 con 0x7f57c0003530 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57dc5ce640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f57d4108510 con 0x7f57d4197840 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57d9b69640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 0x7f57d419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57d9b69640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 0x7f57d419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56210/0 (socket says 172.21.15.181:56210) 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57d9b69640 1 -- 172.21.15.181:0/3752940253 learned_addr learned my addr 172.21.15.181:0/3752940253 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57da36a640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 0x7f57d4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57d9b69640 1 -- 172.21.15.181:0/3752940253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 msgr2=0x7f57d419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57d9b69640 1 -- 172.21.15.181:0/3752940253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 msgr2=0x7f57d419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57d9b69640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 0x7f57d419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57d9b69640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 0x7f57d419bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57da36a640 1 -- 172.21.15.181:0/3752940253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 msgr2=0x7f57d419bbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57da36a640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 0x7f57d419bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:55.711 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.709+0000 7f57da36a640 1 -- 172.21.15.181:0/3752940253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d4197e60 con 0x7f57c0003530 2024-09-10T07:26:55.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.710+0000 7f57da36a640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 0x7f57d4197300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f57d0012c10 tx=0x7f57d0032b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:55.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.710+0000 7f57c77fe640 1 -- 172.21.15.181:0/3752940253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f57d0038040 con 0x7f57c0003530 2024-09-10T07:26:55.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.710+0000 7f57c77fe640 1 -- 172.21.15.181:0/3752940253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d003c410 con 0x7f57c0003530 2024-09-10T07:26:55.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.710+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d419c110 con 0x7f57c0003530 2024-09-10T07:26:55.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.710+0000 7f57c77fe640 1 -- 172.21.15.181:0/3752940253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f57d003b370 con 0x7f57c0003530 2024-09-10T07:26:55.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.710+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d419c6d0 con 0x7f57c0003530 2024-09-10T07:26:55.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.711+0000 7f57c77fe640 1 -- 172.21.15.181:0/3752940253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f57d0043070 con 0x7f57c0003530 2024-09-10T07:26:55.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.711+0000 7f57c77fe640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57b8038e00 0x7f57b803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:55.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.711+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d4101cc0 con 0x7f57c0003530 2024-09-10T07:26:55.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.711+0000 7f57c77fe640 1 -- 172.21.15.181:0/3752940253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f57d006e4f0 con 0x7f57c0003530 2024-09-10T07:26:55.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.712+0000 7f57d9b69640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57b8038e00 0x7f57b803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:55.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.712+0000 7f57d9b69640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57b8038e00 0x7f57b803b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009100 tx=0x7f57c8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:55.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.718+0000 7f57c77fe640 1 -- 172.21.15.181:0/3752940253 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57d003c630 con 0x7f57c0003530 2024-09-10T07:26:55.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.957+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f57d419c980 con 0x7f57c0003530 2024-09-10T07:26:55.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.957+0000 7f57c77fe640 1 -- 172.21.15.181:0/3752940253 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f57d0043360 con 0x7f57c0003530 2024-09-10T07:26:55.958 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:26:55.958 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","modified":"2024-09-10T07:26:50.093881Z","created":"2024-09-10T07:25:15.786098Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T07:26:55.958 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 2 2024-09-10T07:26:55.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57b8038e00 msgr2=0x7f57b803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:55.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57b8038e00 0x7f57b803b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009100 tx=0x7f57c8008040 comp rx=0 tx=0).stop 2024-09-10T07:26:55.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 msgr2=0x7f57d4197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:55.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 0x7f57d4197300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f57d0012c10 tx=0x7f57d0032b50 comp rx=0 tx=0).stop 2024-09-10T07:26:55.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 shutdown_connections 2024-09-10T07:26:55.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57d4197840 0x7f57d419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:55.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57b8038e00 0x7f57b803b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:55.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 --2- 172.21.15.181:0/3752940253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0003530 0x7f57d4197300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:55.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 >> 172.21.15.181:0/3752940253 conn(0x7f57d40fc280 msgr2=0x7f57d4104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:55.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 shutdown_connections 2024-09-10T07:26:55.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:55.960+0000 7f57dc5ce640 1 -- 172.21.15.181:0/3752940253 wait complete. 2024-09-10T07:26:56.366 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:56 smithi181 ceph-mon[35397]: Deploying daemon mon.b on smithi063 2024-09-10T07:26:56.366 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:56 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/3752940253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:26:56.366 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:56 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:56 smithi046 ceph-mon[30705]: Deploying daemon mon.b on smithi063 2024-09-10T07:26:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:56 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/3752940253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:26:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:56 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:26:57.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:57 smithi046 ceph-mon[30705]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:57.604 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T07:26:57.604 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T07:26:57.634 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:57 smithi181 ceph-mon[35397]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:57.856 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:26:58.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.309+0000 7fba2d434640 1 -- 172.21.15.181:0/299165490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 msgr2=0x7fba28106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:58.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.309+0000 7fba2d434640 1 --2- 172.21.15.181:0/299165490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 0x7fba28106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba18009880 tx=0x7fba1802a5d0 comp rx=0 tx=0).stop 2024-09-10T07:26:58.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.309+0000 7fba2d434640 1 -- 172.21.15.181:0/299165490 shutdown_connections 2024-09-10T07:26:58.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.309+0000 7fba2d434640 1 --2- 172.21.15.181:0/299165490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 0x7fba28106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.309+0000 7fba2d434640 1 -- 172.21.15.181:0/299165490 >> 172.21.15.181:0/299165490 conn(0x7fba280fc220 msgr2=0x7fba280fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.309+0000 7fba2d434640 1 -- 172.21.15.181:0/299165490 shutdown_connections 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.309+0000 7fba2d434640 1 -- 172.21.15.181:0/299165490 wait complete. 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.310+0000 7fba2d434640 1 Processor -- start 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.310+0000 7fba2d434640 1 -- start start 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.310+0000 7fba2d434640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 0x7fba281962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.310+0000 7fba2d434640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 0x7fba2819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.310+0000 7fba2d434640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810ac60 con 0x7fba28106630 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.310+0000 7fba2d434640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810aae0 con 0x7fba28196820 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.310+0000 7fba1e7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 0x7fba2819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:58.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.310+0000 7fba1e7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 0x7fba2819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56226/0 (socket says 172.21.15.181:56226) 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba1e7fc640 1 -- 172.21.15.181:0/2039839637 learned_addr learned my addr 172.21.15.181:0/2039839637 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba1e7fc640 1 -- 172.21.15.181:0/2039839637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 msgr2=0x7fba2819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba1e7fc640 1 -- 172.21.15.181:0/2039839637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 msgr2=0x7fba2819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba1e7fc640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 0x7fba2819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba1e7fc640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 0x7fba2819abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba26ffd640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 0x7fba281962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba26ffd640 1 -- 172.21.15.181:0/2039839637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 msgr2=0x7fba2819abb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba26ffd640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 0x7fba2819abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:58.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba26ffd640 1 -- 172.21.15.181:0/2039839637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba28196e40 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.311+0000 7fba26ffd640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 0x7fba281962e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba18005ab0 tx=0x7fba18032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.312+0000 7fba24ff9640 1 -- 172.21.15.181:0/2039839637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fba18035070 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.312+0000 7fba24ff9640 1 -- 172.21.15.181:0/2039839637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba18032e40 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.312+0000 7fba24ff9640 1 -- 172.21.15.181:0/2039839637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fba1803bc10 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.312+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2819b0f0 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.312+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2819b5d0 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.313+0000 7fba24ff9640 1 -- 172.21.15.181:0/2039839637 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fba18042030 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.313+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba28101ca0 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.313+0000 7fba24ff9640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fba14038e50 0x7fba1403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.313+0000 7fba24ff9640 1 -- 172.21.15.181:0/2039839637 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fba18040070 con 0x7fba28106630 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.313+0000 7fba1e7fc640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fba14038e50 0x7fba1403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:26:58.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.314+0000 7fba1e7fc640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fba14038e50 0x7fba1403b310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fba10009100 tx=0x7fba10008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:26:58.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.318+0000 7fba24ff9640 1 -- 172.21.15.181:0/2039839637 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba1804c350 con 0x7fba28106630 2024-09-10T07:26:58.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.546+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fba2819b8b0 con 0x7fba28106630 2024-09-10T07:26:58.547 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:26:58 smithi181 ceph-mon[35397]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:58.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.547+0000 7fba24ff9640 1 -- 172.21.15.181:0/2039839637 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fba1803bdb0 con 0x7fba28106630 2024-09-10T07:26:58.548 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:26:58.548 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","modified":"2024-09-10T07:26:50.093881Z","created":"2024-09-10T07:25:15.786098Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T07:26:58.548 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 2 2024-09-10T07:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fba14038e50 msgr2=0x7fba1403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fba14038e50 0x7fba1403b310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fba10009100 tx=0x7fba10008040 comp rx=0 tx=0).stop 2024-09-10T07:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 msgr2=0x7fba281962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 0x7fba281962e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba18005ab0 tx=0x7fba18032c50 comp rx=0 tx=0).stop 2024-09-10T07:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 shutdown_connections 2024-09-10T07:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba28196820 0x7fba2819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:58.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fba14038e50 0x7fba1403b310 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 --2- 172.21.15.181:0/2039839637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba28106630 0x7fba281962e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 >> 172.21.15.181:0/2039839637 conn(0x7fba280fc220 msgr2=0x7fba280fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 shutdown_connections 2024-09-10T07:26:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:26:58.549+0000 7fba2d434640 1 -- 172.21.15.181:0/2039839637 wait complete. 2024-09-10T07:26:58.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:26:58 smithi046 ceph-mon[30705]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:26:59.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:26:58 smithi063 systemd[1]: Started Ceph mon.b for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:27:00.177 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T07:27:00.177 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T07:27:00.425 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:27:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.971+0000 7f4a79f91640 1 -- 172.21.15.181:0/2327999092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a74107660 msgr2=0x7f4a5c005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.971+0000 7f4a79f91640 1 --2- 172.21.15.181:0/2327999092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a74107660 0x7f4a5c005580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a64011c50 tx=0x7f4a6402be50 comp rx=0 tx=0).stop 2024-09-10T07:27:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.971+0000 7f4a79f91640 1 -- 172.21.15.181:0/2327999092 shutdown_connections 2024-09-10T07:27:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.971+0000 7f4a79f91640 1 --2- 172.21.15.181:0/2327999092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a74107660 0x7f4a5c005580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.971+0000 7f4a79f91640 1 -- 172.21.15.181:0/2327999092 >> 172.21.15.181:0/2327999092 conn(0x7f4a740fd320 msgr2=0x7f4a740ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.971+0000 7f4a79f91640 1 -- 172.21.15.181:0/2327999092 shutdown_connections 2024-09-10T07:27:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.971+0000 7f4a79f91640 1 -- 172.21.15.181:0/2327999092 wait complete. 2024-09-10T07:27:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.972+0000 7f4a79f91640 1 Processor -- start 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a79f91640 1 -- start start 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a79f91640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 0x7f4a741995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a79f91640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 0x7f4a7419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a79f91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a7419e3b0 0x7f4a7419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a79f91640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74109780 con 0x7f4a7419e3b0 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a79f91640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74109600 con 0x7f4a74107660 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a79f91640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74109900 con 0x7f4a74199ae0 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a72ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 0x7f4a7419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a72ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 0x7f4a7419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56242/0 (socket says 172.21.15.181:56242) 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a72ffd640 1 -- 172.21.15.181:0/1387972312 learned_addr learned my addr 172.21.15.181:0/1387972312 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:27:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a737fe640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 0x7f4a741995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a72ffd640 1 -- 172.21.15.181:0/1387972312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 msgr2=0x7f4a7419de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a72ffd640 1 -- 172.21.15.181:0/1387972312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 msgr2=0x7f4a7419de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a72ffd640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 0x7f4a7419de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a72ffd640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 0x7f4a7419de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a73fff640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a7419e3b0 0x7f4a7419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a737fe640 1 -- 172.21.15.181:0/1387972312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 msgr2=0x7f4a741995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a737fe640 1 -- 172.21.15.181:0/1387972312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 msgr2=0x7f4a741995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a737fe640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 0x7f4a741995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.973+0000 7f4a737fe640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 0x7f4a741995a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.974+0000 7f4a73fff640 1 -- 172.21.15.181:0/1387972312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 msgr2=0x7f4a7419de70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.974+0000 7f4a73fff640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 0x7f4a7419de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.974+0000 7f4a73fff640 1 -- 172.21.15.181:0/1387972312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 msgr2=0x7f4a741995a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:27:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.974+0000 7f4a73fff640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 0x7f4a741995a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.974+0000 7f4a73fff640 1 -- 172.21.15.181:0/1387972312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a741a6120 con 0x7f4a7419e3b0 2024-09-10T07:27:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.974+0000 7f4a73fff640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a7419e3b0 0x7f4a7419e860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00bd50 tx=0x7f4a6c00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.975+0000 7f4a70ff9640 1 -- 172.21.15.181:0/1387972312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c019070 con 0x7f4a7419e3b0 2024-09-10T07:27:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.975+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a741a6350 con 0x7f4a7419e3b0 2024-09-10T07:27:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.975+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a6860 con 0x7f4a7419e3b0 2024-09-10T07:27:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.975+0000 7f4a70ff9640 1 -- 172.21.15.181:0/1387972312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c014920 con 0x7f4a7419e3b0 2024-09-10T07:27:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.975+0000 7f4a70ff9640 1 -- 172.21.15.181:0/1387972312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c01da00 con 0x7f4a7419e3b0 2024-09-10T07:27:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.976+0000 7f4a70ff9640 1 -- 172.21.15.181:0/1387972312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4a6c01c070 con 0x7f4a7419e3b0 2024-09-10T07:27:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.976+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5c002af0 con 0x7f4a7419e3b0 2024-09-10T07:27:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.976+0000 7f4a70ff9640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a600394f0 0x7f4a6003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.976+0000 7f4a70ff9640 1 -- 172.21.15.181:0/1387972312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4a6c04bc10 con 0x7f4a7419e3b0 2024-09-10T07:27:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.976+0000 7f4a737fe640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a600394f0 0x7f4a6003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.977+0000 7f4a737fe640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a600394f0 0x7f4a6003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4a64000c00 tx=0x7f4a64010d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:03.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:03.983+0000 7f4a70ff9640 1 -- 172.21.15.181:0/1387972312 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a6c007de0 con 0x7f4a7419e3b0 2024-09-10T07:27:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: mon.a calling monitor election 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: mon.c calling monitor election 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: monmap epoch 3 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: last_changed 2024-09-10T07:26:58.803350+0000 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: min_mon_release 19 (squid) 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: election_strategy: 1 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-10T07:27:04.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: 2: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-09-10T07:27:04.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: fsmap 2024-09-10T07:27:04.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:27:04.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: mgrmap e13: x(active, since 51s) 2024-09-10T07:27:04.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: overall HEALTH_OK 2024-09-10T07:27:04.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:04.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:04.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:04.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:27:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: mon.a calling monitor election 2024-09-10T07:27:04.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: mon.c calling monitor election 2024-09-10T07:27:04.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:27:04.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:04.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:04.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:04.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:27:04.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: monmap epoch 3 2024-09-10T07:27:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:27:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: last_changed 2024-09-10T07:26:58.803350+0000 2024-09-10T07:27:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:27:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: min_mon_release 19 (squid) 2024-09-10T07:27:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: election_strategy: 1 2024-09-10T07:27:04.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:27:04.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-10T07:27:04.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: 2: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-09-10T07:27:04.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: fsmap 2024-09-10T07:27:04.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:27:04.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: mgrmap e13: x(active, since 51s) 2024-09-10T07:27:04.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: overall HEALTH_OK 2024-09-10T07:27:04.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:04.164 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:04.164 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:04.164 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.231+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4a5c002de0 con 0x7f4a7419e3b0 2024-09-10T07:27:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.231+0000 7f4a70ff9640 1 -- 172.21.15.181:0/1387972312 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f4a6c00e090 con 0x7f4a7419e3b0 2024-09-10T07:27:04.231 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:27:04.232 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","modified":"2024-09-10T07:26:58.803350Z","created":"2024-09-10T07:25:15.786098Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T07:27:04.232 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 3 2024-09-10T07:27:04.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.233+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a600394f0 msgr2=0x7f4a6003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a600394f0 0x7f4a6003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4a64000c00 tx=0x7f4a64010d00 comp rx=0 tx=0).stop 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a7419e3b0 msgr2=0x7f4a7419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a7419e3b0 0x7f4a7419e860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00bd50 tx=0x7f4a6c00be40 comp rx=0 tx=0).stop 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 shutdown_connections 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a600394f0 0x7f4a6003b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a7419e3b0 0x7f4a7419e860 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a74199ae0 0x7f4a7419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 --2- 172.21.15.181:0/1387972312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a74107660 0x7f4a741995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 >> 172.21.15.181:0/1387972312 conn(0x7f4a740fd320 msgr2=0x7f4a740ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 shutdown_connections 2024-09-10T07:27:04.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:04.234+0000 7f4a79f91640 1 -- 172.21.15.181:0/1387972312 wait complete. 2024-09-10T07:27:04.873 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T07:27:04.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T07:27:05.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:05.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:04 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/1387972312' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:27:05.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:04 smithi181 ceph-mon[35397]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:05.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:04 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/1387972312' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:27:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:04 smithi046 ceph-mon[30705]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:27:05.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.823+0000 7f6bc8c00640 1 -- 172.21.15.46:0/3426978347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bc4100350 msgr2=0x7f6bc4100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:05.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.823+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/3426978347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bc4100350 0x7f6bc4100730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009880 tx=0x7f6bb402a670 comp rx=0 tx=0).stop 2024-09-10T07:27:05.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.823+0000 7f6bc8c00640 1 -- 172.21.15.46:0/3426978347 shutdown_connections 2024-09-10T07:27:05.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.823+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/3426978347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bc4100350 0x7f6bc4100730 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:05.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.823+0000 7f6bc8c00640 1 -- 172.21.15.46:0/3426978347 >> 172.21.15.46:0/3426978347 conn(0x7f6bc4077170 msgr2=0x7f6bc40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:05.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.823+0000 7f6bc8c00640 1 -- 172.21.15.46:0/3426978347 shutdown_connections 2024-09-10T07:27:05.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.823+0000 7f6bc8c00640 1 -- 172.21.15.46:0/3426978347 wait complete. 2024-09-10T07:27:05.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.824+0000 7f6bc8c00640 1 Processor -- start 2024-09-10T07:27:05.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.824+0000 7f6bc8c00640 1 -- start start 2024-09-10T07:27:05.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.824+0000 7f6bc8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bc4100350 0x7f6bc41a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.824+0000 7f6bc8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bc41a3ad0 0x7f6bc419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.824+0000 7f6bc8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bc419dba0 0x7f6bc419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4112090 con 0x7f6bc419dba0 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc8c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4111f10 con 0x7f6bc4100350 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4112210 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bc41a3ad0 0x7f6bc419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bc4100350 0x7f6bc41a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bc4100350 0x7f6bc41a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48988/0 (socket says 172.21.15.46:48988) 2024-09-10T07:27:05.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc3400640 1 -- 172.21.15.46:0/1612975250 learned_addr learned my addr 172.21.15.46:0/1612975250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:27:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bc41a3ad0 0x7f6bc419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43914/0 (socket says 172.21.15.46:43914) 2024-09-10T07:27:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc3e00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bc419dba0 0x7f6bc419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc2a00640 1 -- 172.21.15.46:0/1612975250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bc4100350 msgr2=0x7f6bc41a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc2a00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bc4100350 0x7f6bc41a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc2a00640 1 -- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bc419dba0 msgr2=0x7f6bc419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc2a00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bc419dba0 0x7f6bc419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.825+0000 7f6bc2a00640 1 -- 172.21.15.46:0/1612975250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc41aa340 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.826+0000 7f6bc3400640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bc4100350 0x7f6bc41a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:27:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.826+0000 7f6bc2a00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bc41a3ad0 0x7f6bc419d660 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000b450 tx=0x7f6bb000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.826+0000 7f6bbbe00640 1 -- 172.21.15.46:0/1612975250 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0004230 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.826+0000 7f6bbbe00640 1 -- 172.21.15.46:0/1612975250 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb00043d0 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.826+0000 7f6bbbe00640 1 -- 172.21.15.46:0/1612975250 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0003cb0 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.826+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc41aa5d0 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.826+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41aaa90 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.827+0000 7f6bbbe00640 1 -- 172.21.15.46:0/1612975250 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6bb000f050 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.827+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc4069f90 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.828+0000 7f6bbbe00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6ba80394f0 0x7f6ba803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.828+0000 7f6bbbe00640 1 -- 172.21.15.46:0/1612975250 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6bb004bd60 con 0x7f6bc41a3ad0 2024-09-10T07:27:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.828+0000 7f6bc3400640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6ba80394f0 0x7f6ba803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.829+0000 7f6bc3400640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6ba80394f0 0x7f6ba803b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009230 tx=0x7f6bb402ab70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:05.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:05.833+0000 7f6bbbe00640 1 -- 172.21.15.46:0/1612975250 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bb000d040 con 0x7f6bc41a3ad0 2024-09-10T07:27:06.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.023+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6bc41aadc0 con 0x7f6bc41a3ad0 2024-09-10T07:27:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.024+0000 7f6bbbe00640 1 -- 172.21.15.46:0/1612975250 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f6bb0016ce0 con 0x7f6bc41a3ad0 2024-09-10T07:27:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:27:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-09-10T07:27:06.024 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:27:06.024 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2024-09-10T07:27:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.026+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6ba80394f0 msgr2=0x7f6ba803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.026+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6ba80394f0 0x7f6ba803b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009230 tx=0x7f6bb402ab70 comp rx=0 tx=0).stop 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.026+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bc41a3ad0 msgr2=0x7f6bc419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.026+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bc41a3ad0 0x7f6bc419d660 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000b450 tx=0x7f6bb000b910 comp rx=0 tx=0).stop 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.027+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 shutdown_connections 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.027+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6ba80394f0 0x7f6ba803b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.027+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bc419dba0 0x7f6bc419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.027+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bc41a3ad0 0x7f6bc419d660 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.027+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/1612975250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bc4100350 0x7f6bc41a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.027+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 >> 172.21.15.46:0/1612975250 conn(0x7f6bc4077170 msgr2=0x7f6bc40fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.027+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 shutdown_connections 2024-09-10T07:27:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:06.027+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1612975250 wait complete. 2024-09-10T07:27:06.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:27:06.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:06.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: mon.a calling monitor election 2024-09-10T07:27:06.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: mon.c calling monitor election 2024-09-10T07:27:06.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:27:06.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:06.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:06.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: monmap epoch 3 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: last_changed 2024-09-10T07:26:58.803350+0000 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: min_mon_release 19 (squid) 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: election_strategy: 1 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-10T07:27:06.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: 2: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: fsmap 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: mgrmap e13: x(active, since 51s) 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: overall HEALTH_OK 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/1387972312' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:06.153 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:06.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: mon.b calling monitor election 2024-09-10T07:27:06.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T07:27:06.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: Updating smithi063:/etc/ceph/ceph.conf 2024-09-10T07:27:06.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-10T07:27:06.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: mon.b calling monitor election 2024-09-10T07:27:06.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: mon.c calling monitor election 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: mon.a calling monitor election 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: monmap epoch 3 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: last_changed 2024-09-10T07:26:58.803350+0000 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: min_mon_release 19 (squid) 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: election_strategy: 1 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: 2: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: fsmap 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:27:06.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: mgrmap e13: x(active, since 53s) 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: overall HEALTH_OK 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: Updating smithi181:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: Updating smithi063:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1612975250' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: Updating smithi046:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:06 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:06.867 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T07:27:06.868 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:27:06.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T07:27:06.927 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:27:06.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:27:07.025 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:27:07.025 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T07:27:07.051 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: mon.b calling monitor election 2024-09-10T07:27:07.051 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T07:27:07.051 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: Updating smithi063:/etc/ceph/ceph.conf 2024-09-10T07:27:07.051 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-10T07:27:07.051 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: mon.b calling monitor election 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: mon.c calling monitor election 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: mon.a calling monitor election 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: monmap epoch 3 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: last_changed 2024-09-10T07:26:58.803350+0000 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: min_mon_release 19 (squid) 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: election_strategy: 1 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-10T07:27:07.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: 2: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: fsmap 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: mgrmap e13: x(active, since 53s) 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: overall HEALTH_OK 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: Updating smithi181:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: Updating smithi063:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1612975250' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: Updating smithi046:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:07.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.054 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:06 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.057 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:27:07.057 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:27:07.124 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:27:07.124 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T07:27:07.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: mon.b calling monitor election 2024-09-10T07:27:07.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T07:27:07.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: Updating smithi063:/etc/ceph/ceph.conf 2024-09-10T07:27:07.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-10T07:27:07.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: mon.b calling monitor election 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: mon.c calling monitor election 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: mon.a calling monitor election 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: monmap epoch 3 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: last_changed 2024-09-10T07:26:58.803350+0000 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: created 2024-09-10T07:25:15.786098+0000 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: min_mon_release 19 (squid) 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: election_strategy: 1 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: 1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: 2: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-09-10T07:27:07.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: fsmap 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: mgrmap e13: x(active, since 53s) 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: overall HEALTH_OK 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: Updating smithi181:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: Updating smithi063:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1612975250' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: Updating smithi046:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:06 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.152 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:27:07.152 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:27:07.218 INFO:tasks.cephadm:Adding mgr.x on smithi046 2024-09-10T07:27:07.218 INFO:tasks.cephadm:Adding mgr.y on smithi063 2024-09-10T07:27:07.218 INFO:tasks.cephadm:Adding mgr.z on smithi181 2024-09-10T07:27:07.218 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi046=x;smithi063=y;smithi181=z' 2024-09-10T07:27:07.492 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:27:07.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: Reconfiguring daemon mon.a on smithi046 2024-09-10T07:27:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:07.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T07:27:07.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:07.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:07.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:07.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: Reconfiguring daemon mon.a on smithi046 2024-09-10T07:27:07.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:07.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:07.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:07.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: Reconfiguring daemon mon.a on smithi046 2024-09-10T07:27:07.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- 172.21.15.181:0/78930733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4102180 msgr2=0x7f11e4104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 --2- 172.21.15.181:0/78930733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4102180 0x7f11e4104570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009880 tx=0x7f11e002a5d0 comp rx=0 tx=0).stop 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- 172.21.15.181:0/78930733 shutdown_connections 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 --2- 172.21.15.181:0/78930733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4102180 0x7f11e4104570 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- 172.21.15.181:0/78930733 >> 172.21.15.181:0/78930733 conn(0x7f11e40fbda0 msgr2=0x7f11e40fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- 172.21.15.181:0/78930733 shutdown_connections 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- 172.21.15.181:0/78930733 wait complete. 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 Processor -- start 2024-09-10T07:27:07.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- start start 2024-09-10T07:27:07.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4102180 0x7f11e4196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:07.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e41974e0 0x7f11e419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:07.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e4197960 0x7f11e419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:07.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4107790 con 0x7f11e4102180 2024-09-10T07:27:07.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4107610 con 0x7f11e4197960 2024-09-10T07:27:07.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.920+0000 7f11ebcff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4107910 con 0x7f11e41974e0 2024-09-10T07:27:07.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e41974e0 0x7f11e419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:07.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e41974e0 0x7f11e419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41398/0 (socket says 172.21.15.181:41398) 2024-09-10T07:27:07.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 -- 172.21.15.181:0/395029232 learned_addr learned my addr 172.21.15.181:0/395029232 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:27:07.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e9a9b640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4102180 0x7f11e4196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:07.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11ea29c640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e4197960 0x7f11e419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:07.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 -- 172.21.15.181:0/395029232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e4197960 msgr2=0x7f11e419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:07.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e4197960 0x7f11e419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:07.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 -- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4102180 msgr2=0x7f11e4196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:07.925 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4102180 0x7f11e4196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:07.925 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 -- 172.21.15.181:0/395029232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e419f3a0 con 0x7f11e41974e0 2024-09-10T07:27:07.925 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11e929a640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e41974e0 0x7f11e419c550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11d400b520 tx=0x7f11d400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:07.925 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11daffd640 1 -- 172.21.15.181:0/395029232 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4004230 con 0x7f11e41974e0 2024-09-10T07:27:07.925 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11daffd640 1 -- 172.21.15.181:0/395029232 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d40043d0 con 0x7f11e41974e0 2024-09-10T07:27:07.925 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e419f690 con 0x7f11e41974e0 2024-09-10T07:27:07.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11daffd640 1 -- 172.21.15.181:0/395029232 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4003d60 con 0x7f11e41974e0 2024-09-10T07:27:07.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.921+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e419fc50 con 0x7f11e41974e0 2024-09-10T07:27:07.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.922+0000 7f11daffd640 1 -- 172.21.15.181:0/395029232 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f11d400e070 con 0x7f11e41974e0 2024-09-10T07:27:07.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.922+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e4190550 con 0x7f11e41974e0 2024-09-10T07:27:07.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.922+0000 7f11daffd640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f11d00394f0 0x7f11d003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:07.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.923+0000 7f11daffd640 1 -- 172.21.15.181:0/395029232 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f11d404ba60 con 0x7f11e41974e0 2024-09-10T07:27:07.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.923+0000 7f11e9a9b640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f11d00394f0 0x7f11d003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:07.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.924+0000 7f11e9a9b640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f11d00394f0 0x7f11d003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f11e0000c00 tx=0x7f11e00315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:07.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:07.927+0000 7f11daffd640 1 -- 172.21.15.181:0/395029232 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11d4014020 con 0x7f11e41974e0 2024-09-10T07:27:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.109+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi063=y;smithi181=z", "target": ["mon-mgr", ""]}) -- 0x7f11e40626c0 con 0x7f11d00394f0 2024-09-10T07:27:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.115+0000 7f11daffd640 1 -- 172.21.15.181:0/395029232 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f11e40626c0 con 0x7f11d00394f0 2024-09-10T07:27:08.114 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mgr update... 2024-09-10T07:27:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f11d00394f0 msgr2=0x7f11d003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f11d00394f0 0x7f11d003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f11e0000c00 tx=0x7f11e00315d0 comp rx=0 tx=0).stop 2024-09-10T07:27:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e41974e0 msgr2=0x7f11e419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e41974e0 0x7f11e419c550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11d400b520 tx=0x7f11d400b9e0 comp rx=0 tx=0).stop 2024-09-10T07:27:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 shutdown_connections 2024-09-10T07:27:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e4197960 0x7f11e419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e41974e0 0x7f11e419c550 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f11d00394f0 0x7f11d003b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 --2- 172.21.15.181:0/395029232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4102180 0x7f11e4196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 >> 172.21.15.181:0/395029232 conn(0x7f11e40fbda0 msgr2=0x7f11e40fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 shutdown_connections 2024-09-10T07:27:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:27:08.117+0000 7f11ebcff640 1 -- 172.21.15.181:0/395029232 wait complete. 2024-09-10T07:27:08.701 DEBUG:teuthology.orchestra.run.smithi063:mgr.y> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T07:27:08.704 DEBUG:teuthology.orchestra.run.smithi181:mgr.z> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T07:27:08.706 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T07:27:08.707 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:27:08.707 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T07:27:08.743 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T07:27:08.744 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-09-10T07:27:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T07:27:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:08.822 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-10T07:27:08.822 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:08.822 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:08.822 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-10 07:26:23.134848208 +0000 2024-09-10T07:27:08.822 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 07:22:01.537543831 +0000 2024-09-10T07:27:08.822 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 07:22:01.537543831 +0000 2024-09-10T07:27:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-10 07:22:01.537543831 +0000 2024-09-10T07:27:08.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T07:27:08.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-10T07:27:08.920 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-10T07:27:08.920 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.00020141 s, 2.5 MB/s 2024-09-10T07:27:08.923 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T07:27:08.966 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-10 07:26:23.182847700 +0000 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 07:22:02.016539188 +0000 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 07:22:02.016539188 +0000 2024-09-10T07:27:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-10 07:22:02.016539188 +0000 2024-09-10T07:27:09.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T07:27:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-10T07:27:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-10T07:27:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000185037 s, 2.8 MB/s 2024-09-10T07:27:09.144 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T07:27:09.186 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-09-10T07:27:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T07:27:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:09.269 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-09-10T07:27:09.269 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:09.269 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:09.269 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-10 07:26:23.220847298 +0000 2024-09-10T07:27:09.269 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 07:22:02.474534748 +0000 2024-09-10T07:27:09.269 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 07:22:02.474534748 +0000 2024-09-10T07:27:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-10 07:22:02.474534748 +0000 2024-09-10T07:27:09.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T07:27:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-10T07:27:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-10T07:27:09.369 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000277837 s, 1.8 MB/s 2024-09-10T07:27:09.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi063=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:09.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: Saving service mgr spec with placement smithi046=x;smithi063=y;smithi181=z;count:3 2024-09-10T07:27:09.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:09.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:09.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:09.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:09.371 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T07:27:09.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi063=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:09.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: Saving service mgr spec with placement smithi046=x;smithi063=y;smithi181=z;count:3 2024-09-10T07:27:09.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:09.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi063=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: Saving service mgr spec with placement smithi046=x;smithi063=y;smithi181=z;count:3 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:09.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:09.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:09.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:09.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:09.409 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-09-10T07:27:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T07:27:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:09.489 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-09-10T07:27:09.489 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:09.489 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:09.489 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-10 07:26:23.259846886 +0000 2024-09-10T07:27:09.489 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 07:22:02.968529959 +0000 2024-09-10T07:27:09.489 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 07:22:02.968529959 +0000 2024-09-10T07:27:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-10 07:22:02.968529959 +0000 2024-09-10T07:27:09.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T07:27:09.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-10T07:27:09.583 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-10T07:27:09.583 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000174984 s, 2.9 MB/s 2024-09-10T07:27:09.586 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T07:27:09.626 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-10T07:27:09.626 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T07:27:09.645 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T07:27:09.645 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-09-10T07:27:09.703 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T07:27:09.704 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:09.704 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T07:27:09.704 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:09.704 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:09.704 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-09-10 07:26:58.219742541 +0000 2024-09-10T07:27:09.704 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-09-10 07:21:30.170537568 +0000 2024-09-10T07:27:09.704 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-09-10 07:21:30.170537568 +0000 2024-09-10T07:27:09.704 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-09-10 07:21:30.170537568 +0000 2024-09-10T07:27:09.704 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T07:27:09.770 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-09-10T07:27:09.770 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-09-10T07:27:09.771 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.00027968 s, 1.8 MB/s 2024-09-10T07:27:09.772 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T07:27:09.830 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-09-10 07:26:58.219742541 +0000 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-09-10 07:21:30.497533995 +0000 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-09-10 07:21:30.497533995 +0000 2024-09-10T07:27:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-09-10 07:21:30.497533995 +0000 2024-09-10T07:27:09.889 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T07:27:09.956 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-09-10T07:27:09.956 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-09-10T07:27:09.956 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000273282 s, 1.9 MB/s 2024-09-10T07:27:09.957 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T07:27:10.016 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-09-10T07:27:10.074 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T07:27:10.074 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:10.075 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T07:27:10.075 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:10.075 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:10.075 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-09-10 07:26:58.220742530 +0000 2024-09-10T07:27:10.075 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-09-10 07:21:30.824530423 +0000 2024-09-10T07:27:10.075 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-09-10 07:21:30.824530423 +0000 2024-09-10T07:27:10.075 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-09-10 07:21:30.824530423 +0000 2024-09-10T07:27:10.075 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T07:27:10.142 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-09-10T07:27:10.142 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-09-10T07:27:10.142 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000225917 s, 2.3 MB/s 2024-09-10T07:27:10.143 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T07:27:10.201 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-09-10T07:27:10.257 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T07:27:10.257 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:10.257 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T07:27:10.258 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:10.258 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:10.258 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-09-10 07:26:58.220742530 +0000 2024-09-10T07:27:10.258 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-09-10 07:21:31.151526851 +0000 2024-09-10T07:27:10.258 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-09-10 07:21:31.151526851 +0000 2024-09-10T07:27:10.258 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-09-10 07:21:31.151526851 +0000 2024-09-10T07:27:10.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T07:27:10.321 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-09-10T07:27:10.321 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-09-10T07:27:10.321 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000187367 s, 2.7 MB/s 2024-09-10T07:27:10.323 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T07:27:10.380 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T07:27:10.380 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T07:27:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:10 smithi181 ceph-mon[35397]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T07:27:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:10 smithi181 ceph-mon[35397]: Reconfiguring daemon mgr.x on smithi046 2024-09-10T07:27:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:10 smithi063 ceph-mon[35020]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T07:27:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:10 smithi063 ceph-mon[35020]: Reconfiguring daemon mgr.x on smithi046 2024-09-10T07:27:10.401 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T07:27:10.401 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-09-10T07:27:10.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:10 smithi046 ceph-mon[30705]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T07:27:10.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:10 smithi046 ceph-mon[30705]: Reconfiguring daemon mgr.x on smithi046 2024-09-10T07:27:10.460 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T07:27:10.460 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:10.460 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-10T07:27:10.460 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:10.461 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:10.461 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-10 07:26:49.342534524 +0000 2024-09-10T07:27:10.461 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-10 07:21:24.491831678 +0000 2024-09-10T07:27:10.461 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-10 07:21:24.491831678 +0000 2024-09-10T07:27:10.461 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-10 07:21:24.491831678 +0000 2024-09-10T07:27:10.461 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T07:27:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-10T07:27:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-10T07:27:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000236939 s, 2.2 MB/s 2024-09-10T07:27:10.530 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T07:27:10.589 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-09-10T07:27:10.647 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T07:27:10.648 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:10.648 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T07:27:10.648 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:10.648 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:10.648 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-10 07:26:49.343534505 +0000 2024-09-10T07:27:10.648 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-10 07:21:24.817825222 +0000 2024-09-10T07:27:10.648 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-10 07:21:24.817825222 +0000 2024-09-10T07:27:10.648 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-10 07:21:24.817825222 +0000 2024-09-10T07:27:10.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T07:27:10.715 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-10T07:27:10.715 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-10T07:27:10.715 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000234749 s, 2.2 MB/s 2024-09-10T07:27:10.717 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T07:27:10.775 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-09-10T07:27:10.833 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T07:27:10.833 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:10.833 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-10T07:27:10.833 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:10.833 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:10.833 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-10 07:26:49.343534505 +0000 2024-09-10T07:27:10.833 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-10 07:21:25.144818747 +0000 2024-09-10T07:27:10.834 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-10 07:21:25.144818747 +0000 2024-09-10T07:27:10.834 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-10 07:21:25.144818747 +0000 2024-09-10T07:27:10.834 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T07:27:10.898 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-10T07:27:10.898 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-10T07:27:10.899 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000224976 s, 2.3 MB/s 2024-09-10T07:27:10.900 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T07:27:10.957 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-09-10T07:27:11.014 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T07:27:11.014 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:27:11.014 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T07:27:11.015 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:27:11.015 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:27:11.015 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-10 07:26:49.343534505 +0000 2024-09-10T07:27:11.015 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-10 07:21:25.471812272 +0000 2024-09-10T07:27:11.015 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-10 07:21:25.471812272 +0000 2024-09-10T07:27:11.015 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-10 07:21:25.471812272 +0000 2024-09-10T07:27:11.015 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T07:27:11.080 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-10T07:27:11.080 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-10T07:27:11.080 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000224369 s, 2.3 MB/s 2024-09-10T07:27:11.081 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T07:27:11.139 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-09-10T07:27:11.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T07:27:11.520 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:27:11.578 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:11.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:11.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:11 smithi063 ceph-mon[35020]: Reconfiguring mon.b (monmap changed)... 2024-09-10T07:27:11.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:11.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:11.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:11.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:11 smithi063 ceph-mon[35020]: Reconfiguring daemon mon.b on smithi063 2024-09-10T07:27:11.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:11 smithi063 ceph-mon[35020]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:11 smithi181 ceph-mon[35397]: Reconfiguring mon.b (monmap changed)... 2024-09-10T07:27:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:11.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:11 smithi181 ceph-mon[35397]: Reconfiguring daemon mon.b on smithi063 2024-09-10T07:27:11.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:11 smithi181 ceph-mon[35397]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:11.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:11.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:11.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:11 smithi046 ceph-mon[30705]: Reconfiguring mon.b (monmap changed)... 2024-09-10T07:27:11.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:11.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:11.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:11.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:11 smithi046 ceph-mon[30705]: Reconfiguring daemon mon.b on smithi063 2024-09-10T07:27:11.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:11 smithi046 ceph-mon[30705]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:12.702 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:12.702 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:12.702 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:12 smithi046 ceph-mon[30705]: Reconfiguring mon.c (monmap changed)... 2024-09-10T07:27:12.702 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:12.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:12.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:12.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:12 smithi046 ceph-mon[30705]: Reconfiguring daemon mon.c on smithi181 2024-09-10T07:27:12.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:12 smithi046 ceph-mon[30705]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:12.888 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:12.888 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:12.889 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:12 smithi181 ceph-mon[35397]: Reconfiguring mon.c (monmap changed)... 2024-09-10T07:27:12.889 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:12.889 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:12.889 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:12.889 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:12 smithi181 ceph-mon[35397]: Reconfiguring daemon mon.c on smithi181 2024-09-10T07:27:12.889 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:12 smithi181 ceph-mon[35397]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:12.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:12.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:12.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:12 smithi063 ceph-mon[35020]: Reconfiguring mon.c (monmap changed)... 2024-09-10T07:27:12.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:27:12.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:27:12.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:12.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:12 smithi063 ceph-mon[35020]: Reconfiguring daemon mon.c on smithi181 2024-09-10T07:27:12.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:12 smithi063 ceph-mon[35020]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:14.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:14.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:14.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:14.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:14.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:27:14.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:14.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:14.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:13 smithi046 ceph-mon[30705]: Deploying daemon mgr.z on smithi181 2024-09-10T07:27:14.203 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.203 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.203 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:14.203 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:14.203 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:14.203 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:14.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:27:14.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:14.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:14.204 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:13 smithi181 ceph-mon[35397]: Deploying daemon mgr.z on smithi181 2024-09-10T07:27:14.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:27:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:13 smithi063 ceph-mon[35020]: Deploying daemon mgr.z on smithi181 2024-09-10T07:27:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:27:14.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-09-10T07:27:14.854 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:14 smithi181 systemd[1]: Starting Ceph mgr.z for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:27:14.926 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:27:15.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:14 smithi181 ceph-mon[35397]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:15.150 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:14 smithi181 podman[37188]: 2024-09-10 07:27:14.758085342 +0000 UTC m=+0.015735778 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:27:15.150 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:14 smithi181 podman[37188]: 2024-09-10 07:27:14.864569883 +0000 UTC m=+0.122220308 container create 8a1ad40a12991e5c5a1a0f755144278a059271e72525bcec86cbc3e9ad49706d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T07:27:15.150 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 podman[37188]: 2024-09-10 07:27:15.066836496 +0000 UTC m=+0.324486924 container init 8a1ad40a12991e5c5a1a0f755144278a059271e72525bcec86cbc3e9ad49706d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:27:15.150 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 podman[37188]: 2024-09-10 07:27:15.069650696 +0000 UTC m=+0.327301132 container start 8a1ad40a12991e5c5a1a0f755144278a059271e72525bcec86cbc3e9ad49706d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:27:15.150 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 ceph-mgr[37207]: -- 172.21.15.181:0/2513509913 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee60a7d0e0 con 0x55ee60a5b400 2024-09-10T07:27:15.235 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:14 smithi046 ceph-mon[30705]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:15.400 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 bash[37188]: 8a1ad40a12991e5c5a1a0f755144278a059271e72525bcec86cbc3e9ad49706d 2024-09-10T07:27:15.400 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 systemd[1]: Started Ceph mgr.z for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:27:15.400 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:15.194+0000 7f029bbd4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:27:15.400 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:15.254+0000 7f029bbd4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:27:15.400 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:15.400+0000 7f029bbd4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:27:15.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:14 smithi063 ceph-mon[35020]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.488+0000 7f1d8e200640 1 -- 172.21.15.46:0/816440133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88074bf0 msgr2=0x7f1d88075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.488+0000 7f1d8e200640 1 --2- 172.21.15.46:0/816440133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88074bf0 0x7f1d88075050 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009900 tx=0x7f1d7402a480 comp rx=0 tx=0).stop 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.488+0000 7f1d8e200640 1 -- 172.21.15.46:0/816440133 shutdown_connections 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.488+0000 7f1d8e200640 1 --2- 172.21.15.46:0/816440133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d88109aa0 0x7f1d8810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.488+0000 7f1d8e200640 1 --2- 172.21.15.46:0/816440133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88074bf0 0x7f1d88075050 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.488+0000 7f1d8e200640 1 --2- 172.21.15.46:0/816440133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880761c0 0x7f1d88074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.488+0000 7f1d8e200640 1 -- 172.21.15.46:0/816440133 >> 172.21.15.46:0/816440133 conn(0x7f1d880fd390 msgr2=0x7f1d880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.489+0000 7f1d8e200640 1 -- 172.21.15.46:0/816440133 shutdown_connections 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.489+0000 7f1d8e200640 1 -- 172.21.15.46:0/816440133 wait complete. 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.489+0000 7f1d8e200640 1 Processor -- start 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d8e200640 1 -- start start 2024-09-10T07:27:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d8e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d88074bf0 0x7f1d8819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d8e200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880761c0 0x7f1d8819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d8e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88109aa0 0x7f1d881a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d8e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8810fad0 con 0x7f1d88074bf0 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d8e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8810f950 con 0x7f1d88109aa0 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d8e200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8810fc50 con 0x7f1d880761c0 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d88074bf0 0x7f1d8819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88109aa0 0x7f1d881a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d86a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880761c0 0x7f1d8819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88109aa0 0x7f1d881a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49012/0 (socket says 172.21.15.46:49012) 2024-09-10T07:27:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.490+0000 7f1d87e00640 1 -- 172.21.15.46:0/2132054970 learned_addr learned my addr 172.21.15.46:0/2132054970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:27:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d87e00640 1 -- 172.21.15.46:0/2132054970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880761c0 msgr2=0x7f1d8819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d87e00640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880761c0 0x7f1d8819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d87e00640 1 -- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d88074bf0 msgr2=0x7f1d8819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d87e00640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d88074bf0 0x7f1d8819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d87e00640 1 -- 172.21.15.46:0/2132054970 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d881a1d30 con 0x7f1d88109aa0 2024-09-10T07:27:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d87e00640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88109aa0 0x7f1d881a16c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000eb10 tx=0x7f1d8000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80004b20 con 0x7f1d88109aa0 2024-09-10T07:27:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d80004cc0 con 0x7f1d88109aa0 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010760 con 0x7f1d88109aa0 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d881a1fc0 con 0x7f1d88109aa0 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.491+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d881a2420 con 0x7f1d88109aa0 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.492+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1d80005020 con 0x7f1d88109aa0 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.493+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d88108860 con 0x7f1d88109aa0 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.493+0000 7f1d7fe00640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d6c0392c0 0x7f1d6c03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.493+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1d8004c9d0 con 0x7f1d88109aa0 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.493+0000 7f1d87400640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d6c0392c0 0x7f1d6c03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.494+0000 7f1d87400640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d6c0392c0 0x7f1d6c03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009bd0 tx=0x7f1d78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.498+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d8000a250 con 0x7f1d88109aa0 2024-09-10T07:27:15.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:15.696+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1d88062630 con 0x7f1d6c0392c0 2024-09-10T07:27:15.748 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:15.465+0000 7f029bbd4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:27:15.748 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:15.611+0000 7f029bbd4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:27:15.748 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:15.748+0000 7f029bbd4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:27:16.399 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:16.057+0000 7f029bbd4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:27:16.399 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:16.118+0000 7f029bbd4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:16.304+0000 7f029bbd4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: Deploying daemon mgr.y on smithi063 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:27:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:27:16.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:27:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: Deploying daemon mgr.y on smithi063 2024-09-10T07:27:16.567 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:27:16.567 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:27:16.567 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:16.578 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: Deploying daemon mgr.y on smithi063 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:27:16.579 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:17.150 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 systemd[1]: Starting Ceph mgr.y for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:27:17.485 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:17 smithi063 ceph-mon[35020]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:17.485 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:17 smithi063 ceph-mon[35020]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:17.485 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 podman[36367]: 2024-09-10 07:27:17.147789898 +0000 UTC m=+0.015766789 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:27:17.485 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 podman[36367]: 2024-09-10 07:27:17.256236448 +0000 UTC m=+0.124213332 container create 1d95314dabcad8e607d547ae00b3c459e086312f1282168b5d39a089ba5260a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T07:27:17.485 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 podman[36367]: 2024-09-10 07:27:17.458456272 +0000 UTC m=+0.326433155 container init 1d95314dabcad8e607d547ae00b3c459e086312f1282168b5d39a089ba5260a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T07:27:17.485 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 podman[36367]: 2024-09-10 07:27:17.46129231 +0000 UTC m=+0.329269195 container start 1d95314dabcad8e607d547ae00b3c459e086312f1282168b5d39a089ba5260a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T07:27:17.575 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:17 smithi046 ceph-mon[30705]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:17.575 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:17 smithi046 ceph-mon[30705]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:17.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:17 smithi181 ceph-mon[35397]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:17.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:17 smithi181 ceph-mon[35397]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:17.650 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:17.213+0000 7f029bbd4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:27:17.650 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:17.283+0000 7f029bbd4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:27:17.805 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 ceph-mgr[36386]: -- 172.21.15.63:0/4128003181 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557280ff10e0 con 0x557280fce800 2024-09-10T07:27:17.805 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 bash[36367]: 1d95314dabcad8e607d547ae00b3c459e086312f1282168b5d39a089ba5260a0 2024-09-10T07:27:17.805 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 systemd[1]: Started Ceph mgr.y for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:27:17.805 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:17.589+0000 7fa8f3234040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:27:17.805 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:17.648+0000 7fa8f3234040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:27:17.805 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:17.804+0000 7fa8f3234040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:27:18.150 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:17.875+0000 7fa8f3234040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:27:18.150 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:18.035+0000 7fa8f3234040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:27:18.295 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.023+0000 7f029bbd4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:27:18.295 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.152+0000 7f029bbd4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:27:18.295 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.294+0000 7f029bbd4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:27:18.510 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:18.204+0000 7fa8f3234040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:27:18.583 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.357+0000 7f029bbd4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:27:18.583 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.513+0000 7f029bbd4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:27:18.583 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.582+0000 7f029bbd4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:27:18.803 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:18.538+0000 7fa8f3234040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:27:18.803 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:18.602+0000 7fa8f3234040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:27:18.803 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.803 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.803 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.804 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.804 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:18.804 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-mon[35020]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:18.876 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.877 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.877 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.877 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.877 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:18.877 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-mon[35397]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:18.877 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.644+0000 7f029bbd4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:27:18.877 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.806+0000 7f029bbd4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:27:18.877 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.875+0000 7f029bbd4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:27:18.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:18.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:18.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:18 smithi046 ceph-mon[30705]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:19.149 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:18.941+0000 7f029bbd4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:27:19.149 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:19.027+0000 7f029bbd4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:27:19.149 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:19.086+0000 7f029bbd4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:27:19.149 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:19.148+0000 7f029bbd4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:27:19.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1052577116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]: dispatch 2024-09-10T07:27:19.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]: dispatch 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]': finished 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:19.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:19.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:19.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3638257110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:27:19.901 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:18.802+0000 7fa8f3234040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:27:19.901 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:19.730+0000 7fa8f3234040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:27:19.901 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:19.800+0000 7fa8f3234040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:27:19.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1052577116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]: dispatch 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]: dispatch 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]': finished 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:19.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:19.903 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:19.903 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.903 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:19 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3638257110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:27:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1052577116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]: dispatch 2024-09-10T07:27:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]: dispatch 2024-09-10T07:27:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0358dc9f-4491-4ec2-9e41-9b621d65a460"}]': finished 2024-09-10T07:27:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T07:27:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:19.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:19 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3638257110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:27:20.399 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:27:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z[37203]: 2024-09-10T07:27:19.927+0000 7f029bbd4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:27:20.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:20.611+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f1d80021960 con 0x7f1d88109aa0 2024-09-10T07:27:20.818 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:20.548+0000 7fa8f3234040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:27:20.818 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:20.669+0000 7fa8f3234040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:27:20.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-mon[35020]: Standby manager daemon z started 2024-09-10T07:27:20.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-mon[35020]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T07:27:20.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-mon[35020]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:27:20.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-mon[35020]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T07:27:20.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-mon[35020]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:27:20.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-mon[35020]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:20.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:20 smithi181 ceph-mon[35397]: Standby manager daemon z started 2024-09-10T07:27:20.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:20 smithi181 ceph-mon[35397]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T07:27:20.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:20 smithi181 ceph-mon[35397]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:27:20.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:20 smithi181 ceph-mon[35397]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T07:27:20.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:20 smithi181 ceph-mon[35397]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:27:20.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:20 smithi181 ceph-mon[35397]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:20 smithi046 ceph-mon[30705]: Standby manager daemon z started 2024-09-10T07:27:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:20 smithi046 ceph-mon[30705]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T07:27:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:20 smithi046 ceph-mon[30705]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:27:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:20 smithi046 ceph-mon[30705]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T07:27:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:20 smithi046 ceph-mon[30705]: from='mgr.? 172.21.15.181:0/492717423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:27:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:20 smithi046 ceph-mon[30705]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:21.106 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:20.817+0000 7fa8f3234040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:27:21.107 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:20.880+0000 7fa8f3234040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:27:21.107 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.035+0000 7fa8f3234040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:27:21.107 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.106+0000 7fa8f3234040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:27:21.400 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.168+0000 7fa8f3234040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:27:21.400 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.332+0000 7fa8f3234040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:27:21.400 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.399+0000 7fa8f3234040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:27:21.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-mon[35020]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-10T07:27:21.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T07:27:21.651 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.468+0000 7fa8f3234040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:27:21.651 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.554+0000 7fa8f3234040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:27:21.651 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.615+0000 7fa8f3234040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:27:21.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:21 smithi181 ceph-mon[35397]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-10T07:27:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T07:27:21.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:21 smithi046 ceph-mon[30705]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-10T07:27:21.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T07:27:22.150 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:21.677+0000 7fa8f3234040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:27:22.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:22 smithi181 ceph-mon[35397]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:22.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:22 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:22.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:22 smithi181 ceph-mon[35397]: Standby manager daemon y started 2024-09-10T07:27:22.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:22 smithi181 ceph-mon[35397]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T07:27:22.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:22 smithi181 ceph-mon[35397]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:27:22.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:22 smithi181 ceph-mon[35397]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T07:27:22.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:22 smithi181 ceph-mon[35397]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:27:22.900 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:27:22 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y[36382]: 2024-09-10T07:27:22.501+0000 7fa8f3234040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:27:22.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:22 smithi063 ceph-mon[35020]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:22.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:22 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:22.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:22 smithi063 ceph-mon[35020]: Standby manager daemon y started 2024-09-10T07:27:22.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:22 smithi063 ceph-mon[35020]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T07:27:22.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:22 smithi063 ceph-mon[35020]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:27:22.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:22 smithi063 ceph-mon[35020]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T07:27:22.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:22 smithi063 ceph-mon[35020]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:27:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:22 smithi046 ceph-mon[30705]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:22 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:22 smithi046 ceph-mon[30705]: Standby manager daemon y started 2024-09-10T07:27:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:22 smithi046 ceph-mon[30705]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T07:27:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:22 smithi046 ceph-mon[30705]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:27:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:22 smithi046 ceph-mon[30705]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T07:27:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:22 smithi046 ceph-mon[30705]: from='mgr.? 172.21.15.63:0/2770186313' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:27:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:23.428+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mon.2 v2:172.21.15.63:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f1d8001f340 con 0x7f1d88109aa0 2024-09-10T07:27:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:24 smithi046 ceph-mon[30705]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T07:27:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T07:27:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:24 smithi046 ceph-mon[30705]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:24 smithi181 ceph-mon[35397]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T07:27:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T07:27:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:24 smithi181 ceph-mon[35397]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:24.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:24 smithi063 ceph-mon[35020]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T07:27:24.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T07:27:24.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:24 smithi063 ceph-mon[35020]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:26.594 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:26 smithi046 ceph-mon[30705]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:26.702 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:26 smithi181 ceph-mon[35397]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:26.702 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:26 smithi063 ceph-mon[35020]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T07:27:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:27.961 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T07:27:27.961 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T07:27:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:28.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:28 smithi046 ceph-mon[30705]: Deploying daemon osd.0 on smithi046 2024-09-10T07:27:28.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:28 smithi046 ceph-mon[30705]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:29.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:28 smithi181 ceph-mon[35397]: Deploying daemon osd.0 on smithi046 2024-09-10T07:27:29.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:28 smithi181 ceph-mon[35397]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:29.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:28 smithi063 ceph-mon[35020]: Deploying daemon osd.0 on smithi046 2024-09-10T07:27:29.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:28 smithi063 ceph-mon[35020]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:30.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:30 smithi046 ceph-mon[30705]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:30.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:30 smithi181 ceph-mon[35397]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:30.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:30 smithi063 ceph-mon[35020]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:32.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:32 smithi181 ceph-mon[35397]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:32.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:32 smithi063 ceph-mon[35020]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:32.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:32 smithi046 ceph-mon[30705]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:34.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:34 smithi046 ceph-mon[30705]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:34.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:34 smithi181 ceph-mon[35397]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:34.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:34 smithi063 ceph-mon[35020]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:35.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:35.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:35.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:35.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:35.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:35.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:36.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:36 smithi046 ceph-mon[30705]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:36.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:36 smithi181 ceph-mon[35397]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:36 smithi063 ceph-mon[35020]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:39.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:39 smithi181 ceph-mon[35397]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:39.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:39.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:39 smithi063 ceph-mon[35020]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:39.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:39.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:39.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:39.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:39 smithi046 ceph-mon[30705]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:40.457 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-09-10T07:27:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.457+0000 7f1d7fe00640 1 -- 172.21.15.46:0/2132054970 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d88062630 con 0x7f1d6c0392c0 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d6c0392c0 msgr2=0x7f1d6c03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d6c0392c0 0x7f1d6c03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009bd0 tx=0x7f1d78005f90 comp rx=0 tx=0).stop 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88109aa0 msgr2=0x7f1d881a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88109aa0 0x7f1d881a16c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000eb10 tx=0x7f1d8000c2d0 comp rx=0 tx=0).stop 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 shutdown_connections 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d88109aa0 0x7f1d881a16c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880761c0 0x7f1d8819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d6c0392c0 0x7f1d6c03b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 --2- 172.21.15.46:0/2132054970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d88074bf0 0x7f1d8819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 >> 172.21.15.46:0/2132054970 conn(0x7f1d880fd390 msgr2=0x7f1d88105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 shutdown_connections 2024-09-10T07:27:40.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:27:40.460+0000 7f1d8e200640 1 -- 172.21.15.46:0/2132054970 wait complete. 2024-09-10T07:27:40.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:40 smithi046 ceph-mon[30705]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:40.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:40.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:40 smithi181 ceph-mon[35397]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:40.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:40.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:40 smithi063 ceph-mon[35020]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:40.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:27:41.235 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T07:27:41.239 INFO:tasks.cephadm:Deploying osd.1 on smithi063 with /dev/vg_nvme/lv_4... 2024-09-10T07:27:41.239 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T07:27:41.496 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:27:41.595 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:41.595 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:41.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:41.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:41.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:41.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:42.159 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:27:41 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0[41979]: 2024-09-10T07:27:41.702+0000 7f41ee011640 -1 osd.0 0 log_to_monitors true 2024-09-10T07:27:42.654 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:42 smithi046 ceph-mon[30705]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:27:42.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:42 smithi046 ceph-mon[30705]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:42.740 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:42 smithi063 ceph-mon[35020]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:27:42.741 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:42 smithi063 ceph-mon[35020]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:42 smithi181 ceph-mon[35397]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:27:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:42 smithi181 ceph-mon[35397]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:43.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:43 smithi063 ceph-mon[35020]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T07:27:43.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:43 smithi063 ceph-mon[35020]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T07:27:43.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:43 smithi063 ceph-mon[35020]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:27:43.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:43 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:27:43.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_4 2024-09-10T07:27:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:43 smithi181 ceph-mon[35397]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T07:27:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:43 smithi181 ceph-mon[35397]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T07:27:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:43 smithi181 ceph-mon[35397]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:27:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:43 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:43 smithi046 ceph-mon[30705]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T07:27:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:43 smithi046 ceph-mon[30705]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T07:27:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:43 smithi046 ceph-mon[30705]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:27:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:43.909 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:27:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0[41979]: 2024-09-10T07:27:43.477+0000 7f41e8c00640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T07:27:43.909 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:27:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0[41979]: 2024-09-10T07:27:43.480+0000 7f41e3c00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:27:44.112 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:27:44.546 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:44 smithi063 ceph-mon[35020]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:27:44.547 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:44 smithi063 ceph-mon[35020]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T07:27:44.547 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:44.547 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:44.547 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:44 smithi063 ceph-mon[35020]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 -- 172.21.15.63:0/196390846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac107900 msgr2=0x7f48ac109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 --2- 172.21.15.63:0/196390846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac107900 0x7f48ac109d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009ca0 tx=0x7f48a402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 -- 172.21.15.63:0/196390846 shutdown_connections 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 --2- 172.21.15.63:0/196390846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac107900 0x7f48ac109d50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 --2- 172.21.15.63:0/196390846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ac104fd0 0x7f48ac1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 --2- 172.21.15.63:0/196390846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48ac1026a0 0x7f48ac104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 -- 172.21.15.63:0/196390846 >> 172.21.15.63:0/196390846 conn(0x7f48ac0fc2a0 msgr2=0x7f48ac0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 -- 172.21.15.63:0/196390846 shutdown_connections 2024-09-10T07:27:44.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.546+0000 7f48b21ce640 1 -- 172.21.15.63:0/196390846 wait complete. 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.547+0000 7f48b21ce640 1 Processor -- start 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.547+0000 7f48b21ce640 1 -- start start 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.547+0000 7f48b21ce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac1026a0 0x7f48ac197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.547+0000 7f48b21ce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48ac104fd0 0x7f48ac198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.547+0000 7f48b21ce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ac107900 0x7f48ac19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.547+0000 7f48b21ce640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac10d8f0 con 0x7f48ac107900 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.547+0000 7f48b21ce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac10d770 con 0x7f48ac1026a0 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.547+0000 7f48b21ce640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac10da70 con 0x7f48ac104fd0 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac1026a0 0x7f48ac197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac1026a0 0x7f48ac197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45416/0 (socket says 172.21.15.63:45416) 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 -- 172.21.15.63:0/3532859637 learned_addr learned my addr 172.21.15.63:0/3532859637 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-09-10T07:27:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48abfff640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ac107900 0x7f48ac19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48aaffd640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48ac104fd0 0x7f48ac198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 -- 172.21.15.63:0/3532859637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48ac104fd0 msgr2=0x7f48ac198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48ac104fd0 0x7f48ac198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 -- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ac107900 msgr2=0x7f48ac19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ac107900 0x7f48ac19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 -- 172.21.15.63:0/3532859637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac19fa70 con 0x7f48ac1026a0 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48aaffd640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48ac104fd0 0x7f48ac198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48ab7fe640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac1026a0 0x7f48ac197e40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f489400c920 tx=0x7f489400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48a8ff9640 1 -- 172.21.15.63:0/3532859637 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894007c70 con 0x7f48ac1026a0 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48a8ff9640 1 -- 172.21.15.63:0/3532859637 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4894007e10 con 0x7f48ac1026a0 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac19fd60 con 0x7f48ac1026a0 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48a8ff9640 1 -- 172.21.15.63:0/3532859637 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894018890 con 0x7f48ac1026a0 2024-09-10T07:27:44.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.548+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac1a01e0 con 0x7f48ac1026a0 2024-09-10T07:27:44.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.549+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ac0fdd90 con 0x7f48ac1026a0 2024-09-10T07:27:44.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.550+0000 7f48a8ff9640 1 -- 172.21.15.63:0/3532859637 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f4894018a30 con 0x7f48ac1026a0 2024-09-10T07:27:44.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.552+0000 7f48a8ff9640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f48980a2b20 0x7f48980a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:27:44.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.552+0000 7f48a8ff9640 1 -- 172.21.15.63:0/3532859637 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f48940cd740 con 0x7f48ac1026a0 2024-09-10T07:27:44.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.552+0000 7f48aaffd640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f48980a2b20 0x7f48980a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:27:44.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.552+0000 7f48aaffd640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f48980a2b20 0x7f48980a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48ac199360 tx=0x7f489c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:27:44.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.556+0000 7f48a8ff9640 1 -- 172.21.15.63:0/3532859637 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4894014030 con 0x7f48ac1026a0 2024-09-10T07:27:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:27:44.740+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f48ac062630 con 0x7f48980a2b20 2024-09-10T07:27:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:44 smithi181 ceph-mon[35397]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:27:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:44 smithi181 ceph-mon[35397]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T07:27:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:44 smithi181 ceph-mon[35397]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:44.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:44 smithi046 ceph-mon[30705]: from='osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:27:44.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:44 smithi046 ceph-mon[30705]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T07:27:44.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:44.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:44.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:44 smithi046 ceph-mon[30705]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:27:45.659 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:27:45.659 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:27:45.659 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:45.659 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] boot 2024-09-10T07:27:45.659 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi046 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi046 to 17094M 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:45.661 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:45.661 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] boot 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi046 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.902 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:27:45.902 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi046 to 17094M 2024-09-10T07:27:45.902 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.902 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:45.902 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:45.902 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:27:45.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:27:45.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:45.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: osd.0 [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] boot 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi046 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi046 to 17094M 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:27:45.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:27:46.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:46 smithi063 ceph-mon[35020]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T07:27:46.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:46 smithi063 ceph-mon[35020]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:46 smithi181 ceph-mon[35397]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T07:27:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:46 smithi181 ceph-mon[35397]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:46.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:46 smithi046 ceph-mon[30705]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T07:27:46.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:46 smithi046 ceph-mon[30705]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:47.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:47 smithi063 ceph-mon[35020]: from='client.? 172.21.15.63:0/3110656914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]: dispatch 2024-09-10T07:27:47.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:47 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]: dispatch 2024-09-10T07:27:47.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:47 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]': finished 2024-09-10T07:27:47.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:47 smithi063 ceph-mon[35020]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T07:27:47.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:47 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:27:47.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:47 smithi063 ceph-mon[35020]: from='client.? 172.21.15.63:0/985945884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:27:47.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:47 smithi181 ceph-mon[35397]: from='client.? 172.21.15.63:0/3110656914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]: dispatch 2024-09-10T07:27:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:47 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]: dispatch 2024-09-10T07:27:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:47 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]': finished 2024-09-10T07:27:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:47 smithi181 ceph-mon[35397]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T07:27:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:47 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:27:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:47 smithi181 ceph-mon[35397]: from='client.? 172.21.15.63:0/985945884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:27:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:47 smithi046 ceph-mon[30705]: from='client.? 172.21.15.63:0/3110656914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]: dispatch 2024-09-10T07:27:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:47 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]: dispatch 2024-09-10T07:27:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:47 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0f0a426-666d-41c0-a0e2-73abbd2f08ef"}]': finished 2024-09-10T07:27:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:47 smithi046 ceph-mon[30705]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T07:27:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:47 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:27:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:47 smithi046 ceph-mon[30705]: from='client.? 172.21.15.63:0/985945884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:27:48.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:48 smithi181 ceph-mon[35397]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:48.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:48 smithi063 ceph-mon[35020]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:48 smithi046 ceph-mon[30705]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:50.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:50 smithi181 ceph-mon[35397]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:50.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:50 smithi063 ceph-mon[35020]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:50.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:50 smithi046 ceph-mon[30705]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:52.712 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:52 smithi063 ceph-mon[35020]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:52.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:52 smithi181 ceph-mon[35397]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:52.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:52 smithi046 ceph-mon[30705]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:54.489 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:54 smithi063 ceph-mon[35020]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:54.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:54 smithi181 ceph-mon[35397]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:54.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:54 smithi046 ceph-mon[30705]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:55.544 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:55 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T07:27:55.544 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:55 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:55.544 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:55 smithi063 ceph-mon[35020]: Deploying daemon osd.1 on smithi063 2024-09-10T07:27:55.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T07:27:55.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:55.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:55 smithi046 ceph-mon[30705]: Deploying daemon osd.1 on smithi063 2024-09-10T07:27:55.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T07:27:55.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:27:55.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:55 smithi181 ceph-mon[35397]: Deploying daemon osd.1 on smithi063 2024-09-10T07:27:56.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:56 smithi063 ceph-mon[35020]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:56.800 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:56 smithi181 ceph-mon[35397]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:56.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:56 smithi046 ceph-mon[30705]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:27:58 smithi046 ceph-mon[30705]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:58.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:27:58 smithi181 ceph-mon[35397]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:27:58.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:27:58 smithi063 ceph-mon[35020]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:00.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:00 smithi181 ceph-mon[35397]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:00.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:00 smithi063 ceph-mon[35020]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:00.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:00 smithi046 ceph-mon[30705]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:01.828 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:01 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:01.829 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:01 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:01.829 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:01 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:01.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:01 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:01.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:01 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:01.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:01 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:01.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:01 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:01.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:01 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:01.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:01 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:02.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:02 smithi063 ceph-mon[35020]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:02.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:02 smithi181 ceph-mon[35397]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:02.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:02 smithi046 ceph-mon[30705]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:04.146 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.146 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.146 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:04.146 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:04.146 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.731+0000 7f48a8ff9640 1 -- 172.21.15.63:0/3532859637 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f48ac062630 con 0x7f48980a2b20 2024-09-10T07:28:04.731 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 1 on host 'smithi063' 2024-09-10T07:28:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f48980a2b20 msgr2=0x7f48980a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f48980a2b20 0x7f48980a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48ac199360 tx=0x7f489c005e50 comp rx=0 tx=0).stop 2024-09-10T07:28:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac1026a0 msgr2=0x7f48ac197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac1026a0 0x7f48ac197e40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f489400c920 tx=0x7f489400cde0 comp rx=0 tx=0).stop 2024-09-10T07:28:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 shutdown_connections 2024-09-10T07:28:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f48980a2b20 0x7f48980a4fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ac107900 0x7f48ac19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48ac104fd0 0x7f48ac198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 --2- 172.21.15.63:0/3532859637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48ac1026a0 0x7f48ac197e40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 >> 172.21.15.63:0/3532859637 conn(0x7f48ac0fc2a0 msgr2=0x7f48ac102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:28:04.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 shutdown_connections 2024-09-10T07:28:04.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:28:04.734+0000 7f48b21ce640 1 -- 172.21.15.63:0/3532859637 wait complete. 2024-09-10T07:28:05.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:04 smithi181 ceph-mon[35397]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:05.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:05.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:05.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:05.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:04 smithi063 ceph-mon[35020]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:05.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:05.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:05.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:04 smithi046 ceph-mon[30705]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:05.470 DEBUG:teuthology.orchestra.run.smithi063:osd.1> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T07:28:05.472 INFO:tasks.cephadm:Deploying osd.2 on smithi181 with /dev/vg_nvme/lv_4... 2024-09-10T07:28:05.473 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T07:28:05.722 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:28:06.731 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:06 smithi181 ceph-mon[35397]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:06.753 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:06 smithi063 ceph-mon[35020]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:06.754 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1[39498]: 2024-09-10T07:28:06.517+0000 7fd9ab768640 -1 osd.1 0 log_to_monitors true 2024-09-10T07:28:06.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:06 smithi046 ceph-mon[30705]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:07.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:07 smithi181 ceph-mon[35397]: from='osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:28:07.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:07 smithi181 ceph-mon[35397]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:28:07.813 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:07 smithi046 ceph-mon[30705]: from='osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:28:07.813 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:07 smithi046 ceph-mon[30705]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:28:07.854 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:07 smithi063 ceph-mon[35020]: from='osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:28:07.854 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:07 smithi063 ceph-mon[35020]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:28:08.065 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:28:08.091 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2024-09-10T07:28:08.342 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi063 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi063 to 17088M 2024-09-10T07:28:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:08.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:08.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:08.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:28:08.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T07:28:08.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 -- 172.21.15.181:0/3783179455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0308101c00 msgr2=0x7f0308102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 --2- 172.21.15.181:0/3783179455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0308101c00 0x7f0308102000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009900 tx=0x7f02fc02a410 comp rx=0 tx=0).stop 2024-09-10T07:28:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 -- 172.21.15.181:0/3783179455 shutdown_connections 2024-09-10T07:28:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 --2- 172.21.15.181:0/3783179455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03081037c0 0x7f030810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 --2- 172.21.15.181:0/3783179455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0308102e00 0x7f0308103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 --2- 172.21.15.181:0/3783179455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0308101c00 0x7f0308102000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 -- 172.21.15.181:0/3783179455 >> 172.21.15.181:0/3783179455 conn(0x7f03080fd3b0 msgr2=0x7f03080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 -- 172.21.15.181:0/3783179455 shutdown_connections 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.784+0000 7f030d7c6640 1 -- 172.21.15.181:0/3783179455 wait complete. 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f030d7c6640 1 Processor -- start 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f030d7c6640 1 -- start start 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f030d7c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0308101c00 0x7f0308075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f030d7c6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308102e00 0x7f03080745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f030d7c6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03081037c0 0x7f0308074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f030d7c6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f030810d890 con 0x7f0308101c00 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f030d7c6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f030810d710 con 0x7f0308102e00 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f030d7c6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f030810da10 con 0x7f03081037c0 2024-09-10T07:28:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03077fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03081037c0 0x7f0308074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:28:08.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03077fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03081037c0 0x7f0308074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49028/0 (socket says 172.21.15.181:49028) 2024-09-10T07:28:08.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03077fe640 1 -- 172.21.15.181:0/2087488609 learned_addr learned my addr 172.21.15.181:0/2087488609 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:28:08.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03067fc640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308102e00 0x7f03080745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:28:08.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f0306ffd640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0308101c00 0x7f0308075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:28:08.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03077fe640 1 -- 172.21.15.181:0/2087488609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308102e00 msgr2=0x7f03080745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:08.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03077fe640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308102e00 0x7f03080745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:08.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03077fe640 1 -- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0308101c00 msgr2=0x7f0308075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03077fe640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0308101c00 0x7f0308075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03077fe640 1 -- 172.21.15.181:0/2087488609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03080753e0 con 0x7f03081037c0 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f0306ffd640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0308101c00 0x7f0308075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.785+0000 7f03067fc640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308102e00 0x7f03080745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.786+0000 7f03077fe640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03081037c0 0x7f0308074b20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f030000eb10 tx=0x7f030000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.786+0000 7f02effff640 1 -- 172.21.15.181:0/2087488609 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030000ce20 con 0x7f03081037c0 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.786+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03081a1fc0 con 0x7f03081037c0 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.786+0000 7f02effff640 1 -- 172.21.15.181:0/2087488609 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0300005020 con 0x7f03081037c0 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.786+0000 7f02effff640 1 -- 172.21.15.181:0/2087488609 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300010690 con 0x7f03081037c0 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.786+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03081a24c0 con 0x7f03081037c0 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.787+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0308108f30 con 0x7f03081037c0 2024-09-10T07:28:08.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.788+0000 7f02effff640 1 -- 172.21.15.181:0/2087488609 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f0300005540 con 0x7f03081037c0 2024-09-10T07:28:08.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.789+0000 7f02effff640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02f00a2af0 0x7f02f00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:28:08.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.789+0000 7f02effff640 1 -- 172.21.15.181:0/2087488609 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f03000ce130 con 0x7f03081037c0 2024-09-10T07:28:08.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.789+0000 7f0306ffd640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02f00a2af0 0x7f02f00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:28:08.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.790+0000 7f0306ffd640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02f00a2af0 0x7f02f00a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f02fc000c00 tx=0x7f02fc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:28:08.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.792+0000 7f02effff640 1 -- 172.21.15.181:0/2087488609 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f030001f340 con 0x7f03081037c0 2024-09-10T07:28:08.900 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1[39498]: 2024-09-10T07:28:08.470+0000 7fd9a76eb640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T07:28:08.901 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1[39498]: 2024-09-10T07:28:08.473+0000 7fd9a3513640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:28:08.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T07:28:08.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T07:28:08.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:28:08.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:08.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:28:08.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi063 2024-09-10T07:28:08.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi063 to 17088M 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T07:28:08.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T07:28:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T07:28:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:28:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:28:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi063 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi063 to 17088M 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:08.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:08.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:28:08.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T07:28:08.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:08.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:08.972+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0308062630 con 0x7f02f00a2af0 2024-09-10T07:28:09.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:09.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:09 smithi181 ceph-mon[35397]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:28:09.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:28:09.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:28:09.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:09.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:09.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:09 smithi063 ceph-mon[35020]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:28:09.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:28:09.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:28:09.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:09.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:09.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:09 smithi046 ceph-mon[30705]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:28:09.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:28:09.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:28:09.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:10.598 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:10 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:28:10.598 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:10 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:28:10.598 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:10 smithi181 ceph-mon[35397]: osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] boot 2024-09-10T07:28:10.599 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:10 smithi181 ceph-mon[35397]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T07:28:10.599 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:10 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:10.599 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:10 smithi181 ceph-mon[35397]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:10.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:10 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:28:10.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:10 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:28:10.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:10 smithi063 ceph-mon[35020]: osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] boot 2024-09-10T07:28:10.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:10 smithi063 ceph-mon[35020]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T07:28:10.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:10 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:10.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:10 smithi063 ceph-mon[35020]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:10.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:10 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:28:10.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:10 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:28:10.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:10 smithi046 ceph-mon[30705]: osd.1 [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] boot 2024-09-10T07:28:10.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:10 smithi046 ceph-mon[30705]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T07:28:10.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:10 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:28:10.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:10 smithi046 ceph-mon[30705]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:28:11.812 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:11 smithi181 ceph-mon[35397]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T07:28:11.812 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:11 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/1420331774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]: dispatch 2024-09-10T07:28:11.813 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:11 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]: dispatch 2024-09-10T07:28:11.813 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:11 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]': finished 2024-09-10T07:28:11.813 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:11 smithi181 ceph-mon[35397]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T07:28:11.813 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:11.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:11 smithi063 ceph-mon[35020]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T07:28:11.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:11 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/1420331774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]: dispatch 2024-09-10T07:28:11.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:11 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]: dispatch 2024-09-10T07:28:11.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:11 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]': finished 2024-09-10T07:28:11.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:11 smithi063 ceph-mon[35020]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T07:28:11.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:11.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:11 smithi046 ceph-mon[30705]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T07:28:11.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:11 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/1420331774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]: dispatch 2024-09-10T07:28:11.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:11 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]: dispatch 2024-09-10T07:28:11.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:11 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16642ece-80e2-4852-a809-8a4f57c5c16d"}]': finished 2024-09-10T07:28:11.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:11 smithi046 ceph-mon[30705]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T07:28:11.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:12.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:12 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/2251808776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:28:12.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:12 smithi181 ceph-mon[35397]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:12.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:12 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/2251808776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:28:12.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:12 smithi063 ceph-mon[35020]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:12.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:12 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/2251808776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:28:12.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:12 smithi046 ceph-mon[30705]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:14.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:14 smithi181 ceph-mon[35397]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:14.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:14 smithi063 ceph-mon[35020]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:14.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:14 smithi046 ceph-mon[30705]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:16.765 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:16 smithi181 ceph-mon[35397]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:16.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:16 smithi063 ceph-mon[35020]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:16.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:16 smithi046 ceph-mon[30705]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:18.623 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:18 smithi181 ceph-mon[35397]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:18.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:18 smithi063 ceph-mon[35020]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:18.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:18 smithi046 ceph-mon[30705]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:19.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T07:28:19.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:19 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:19.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:19 smithi181 ceph-mon[35397]: Deploying daemon osd.2 on smithi181 2024-09-10T07:28:19.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T07:28:19.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:19 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:19.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:19 smithi063 ceph-mon[35020]: Deploying daemon osd.2 on smithi181 2024-09-10T07:28:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T07:28:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:19 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:19.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:19 smithi046 ceph-mon[30705]: Deploying daemon osd.2 on smithi181 2024-09-10T07:28:20.739 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:20 smithi181 ceph-mon[35397]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:20.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:20 smithi063 ceph-mon[35020]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:20 smithi046 ceph-mon[30705]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:22.710 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:22 smithi181 ceph-mon[35397]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:22.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:22 smithi063 ceph-mon[35020]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:22 smithi046 ceph-mon[30705]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:24 smithi181 ceph-mon[35397]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:24.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:24 smithi063 ceph-mon[35020]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:24.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:24 smithi046 ceph-mon[30705]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:25.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:25.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:25.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:25.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:25.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:25.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:25.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:25.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:25.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:26.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:26 smithi181 ceph-mon[35397]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:26.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:26 smithi063 ceph-mon[35020]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:26.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:26 smithi046 ceph-mon[30705]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:29.165 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:29.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:29.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:29 smithi181 ceph-mon[35397]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:29.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.206+0000 7f02effff640 1 -- 172.21.15.181:0/2087488609 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0308062630 con 0x7f02f00a2af0 2024-09-10T07:28:29.207 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 2 on host 'smithi181' 2024-09-10T07:28:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02f00a2af0 msgr2=0x7f02f00a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02f00a2af0 0x7f02f00a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f02fc000c00 tx=0x7f02fc031400 comp rx=0 tx=0).stop 2024-09-10T07:28:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03081037c0 msgr2=0x7f0308074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:29.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03081037c0 0x7f0308074b20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f030000eb10 tx=0x7f030000ef10 comp rx=0 tx=0).stop 2024-09-10T07:28:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 shutdown_connections 2024-09-10T07:28:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03081037c0 0x7f0308074b20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308102e00 0x7f03080745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02f00a2af0 0x7f02f00a4fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 --2- 172.21.15.181:0/2087488609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0308101c00 0x7f0308075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 >> 172.21.15.181:0/2087488609 conn(0x7f03080fd3b0 msgr2=0x7f0308106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:28:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 shutdown_connections 2024-09-10T07:28:29.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:28:29.210+0000 7f030d7c6640 1 -- 172.21.15.181:0/2087488609 wait complete. 2024-09-10T07:28:29.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:29.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:29.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:29 smithi063 ceph-mon[35020]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:29.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:29.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:29.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:29.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:29 smithi046 ceph-mon[30705]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:29.863 DEBUG:teuthology.orchestra.run.smithi181:osd.2> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T07:28:29.865 INFO:tasks.cephadm:Deploying osd.3 on smithi046 with /dev/vg_nvme/lv_3... 2024-09-10T07:28:29.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T07:28:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:28:30.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:30 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:30.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:30 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:30.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:30 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:30 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:30 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:30 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:30 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:30 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:30 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:31.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:31 smithi181 ceph-mon[35397]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:31.278 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2[40365]: 2024-09-10T07:28:31.150+0000 7f23c6619640 -1 osd.2 0 log_to_monitors true 2024-09-10T07:28:31.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:31 smithi063 ceph-mon[35020]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:31.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:31 smithi046 ceph-mon[30705]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:32.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:32 smithi181 ceph-mon[35397]: from='osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T07:28:32.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:32 smithi181 ceph-mon[35397]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T07:28:32.393 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:32 smithi046 ceph-mon[30705]: from='osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T07:28:32.393 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:32 smithi046 ceph-mon[30705]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T07:28:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:32 smithi063 ceph-mon[35020]: from='osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T07:28:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:32 smithi063 ceph-mon[35020]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T07:28:33.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:33.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:28:33.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:33.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:33.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T07:28:33.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T07:28:33.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:33.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:28:33.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:28:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:28:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:28:33.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-09-10T07:28:33.899 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2[40365]: 2024-09-10T07:28:33.489+0000 7f23c2db0640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T07:28:33.899 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2[40365]: 2024-09-10T07:28:33.493+0000 7f23ba3bc640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:28:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:28:34.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:34 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi181 2024-09-10T07:28:34.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:34 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi181 to 17088M 2024-09-10T07:28:34.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:34 smithi046 ceph-mon[30705]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:28:34.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:34 smithi046 ceph-mon[30705]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T07:28:34.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:34.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:34.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:34 smithi046 ceph-mon[30705]: from='osd.2 ' entity='osd.2' 2024-09-10T07:28:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:34 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi181 2024-09-10T07:28:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:34 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi181 to 17088M 2024-09-10T07:28:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:34 smithi181 ceph-mon[35397]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:28:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:34 smithi181 ceph-mon[35397]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T07:28:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:34.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:34 smithi181 ceph-mon[35397]: from='osd.2 ' entity='osd.2' 2024-09-10T07:28:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:34 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi181 2024-09-10T07:28:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:34 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi181 to 17088M 2024-09-10T07:28:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:34 smithi063 ceph-mon[35020]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:28:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:34 smithi063 ceph-mon[35020]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T07:28:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:34.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:34.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:34 smithi063 ceph-mon[35020]: from='osd.2 ' entity='osd.2' 2024-09-10T07:28:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 -- 172.21.15.46:0/897618651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 msgr2=0x7fd4f0109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/897618651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 0x7fd4f0109e90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009900 tx=0x7fd4e802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:28:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 -- 172.21.15.46:0/897618651 shutdown_connections 2024-09-10T07:28:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/897618651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 0x7fd4f0109e90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/897618651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f0102c40 0x7fd4f01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/897618651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4f0101a40 0x7fd4f0101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 -- 172.21.15.46:0/897618651 >> 172.21.15.46:0/897618651 conn(0x7fd4f00fd1f0 msgr2=0x7fd4f00ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 -- 172.21.15.46:0/897618651 shutdown_connections 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.629+0000 7fd4f6e00640 1 -- 172.21.15.46:0/897618651 wait complete. 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.630+0000 7fd4f6e00640 1 Processor -- start 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.630+0000 7fd4f6e00640 1 -- start start 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.630+0000 7fd4f6e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f0101a40 0x7fd4f0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.630+0000 7fd4f6e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4f0102c40 0x7fd4f00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.630+0000 7fd4f6e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 0x7fd4f0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.630+0000 7fd4f6e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f010d710 con 0x7fd4f0103600 2024-09-10T07:28:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.630+0000 7fd4f6e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f010d590 con 0x7fd4f0102c40 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.630+0000 7fd4f6e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f010d890 con 0x7fd4f0101a40 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f0101a40 0x7fd4f0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 0x7fd4f0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f0101a40 0x7fd4f0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50282/0 (socket says 172.21.15.46:50282) 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 -- 172.21.15.46:0/4031629100 learned_addr learned my addr 172.21.15.46:0/4031629100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5000640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4f0102c40 0x7fd4f00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 -- 172.21.15.46:0/4031629100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4f0102c40 msgr2=0x7fd4f00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4f0102c40 0x7fd4f00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 -- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 msgr2=0x7fd4f0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 0x7fd4f0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 -- 172.21.15.46:0/4031629100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f00753e0 con 0x7fd4f0101a40 2024-09-10T07:28:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f6400640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 0x7fd4f0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.631+0000 7fd4f5a00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f0101a40 0x7fd4f0075f90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009d20 tx=0x7fd4e400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.632+0000 7fd4ee000640 1 -- 172.21.15.46:0/4031629100 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4010630 con 0x7fd4f0101a40 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.632+0000 7fd4ee000640 1 -- 172.21.15.46:0/4031629100 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e4010cb0 con 0x7fd4f0101a40 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.632+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f01a1e70 con 0x7fd4f0101a40 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.632+0000 7fd4ee000640 1 -- 172.21.15.46:0/4031629100 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4018de0 con 0x7fd4f0101a40 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.632+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01a2260 con 0x7fd4f0101a40 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.633+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f0108d70 con 0x7fd4f0101a40 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.635+0000 7fd4ee000640 1 -- 172.21.15.46:0/4031629100 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fd4e401f020 con 0x7fd4f0101a40 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.636+0000 7fd4ee000640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4d80a2af0 0x7fd4d80a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:28:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.636+0000 7fd4ee000640 1 -- 172.21.15.46:0/4031629100 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fd4e40cdeb0 con 0x7fd4f0101a40 2024-09-10T07:28:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.636+0000 7fd4f5000640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4d80a2af0 0x7fd4d80a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:28:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.637+0000 7fd4f5000640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4d80a2af0 0x7fd4d80a4fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0075d10 tx=0x7fd4dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:28:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.638+0000 7fd4ee000640 1 -- 172.21.15.46:0/4031629100 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4e409c3a0 con 0x7fd4f0101a40 2024-09-10T07:28:34.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:34.836+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd4f0062630 con 0x7fd4d80a2af0 2024-09-10T07:28:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:28:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] boot 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:35.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] boot 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:28:35.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: osd.2 [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] boot 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:28:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:28:35.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:36 smithi181 ceph-mon[35397]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:28:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:36 smithi063 ceph-mon[35020]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:28:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:36 smithi046 ceph-mon[30705]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:28:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:37 smithi181 ceph-mon[35397]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T07:28:37.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:37 smithi181 ceph-mon[35397]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:37.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:37 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:28:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:37 smithi063 ceph-mon[35020]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T07:28:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:37 smithi063 ceph-mon[35020]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:37 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:28:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:37 smithi046 ceph-mon[30705]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T07:28:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:37 smithi046 ceph-mon[30705]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:37 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:28:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T07:28:38.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:38 smithi063 ceph-mon[35020]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T07:28:38.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:28:38.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:38 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3596186521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8019d3f-952b-42df-9c70-c0574211eaca"}]: dispatch 2024-09-10T07:28:38.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T07:28:38.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:38 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3596186521' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8019d3f-952b-42df-9c70-c0574211eaca"}]': finished 2024-09-10T07:28:38.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:38 smithi063 ceph-mon[35020]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T07:28:38.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:28:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T07:28:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:38 smithi046 ceph-mon[30705]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T07:28:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:28:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:38 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3596186521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8019d3f-952b-42df-9c70-c0574211eaca"}]: dispatch 2024-09-10T07:28:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T07:28:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:38 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3596186521' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8019d3f-952b-42df-9c70-c0574211eaca"}]': finished 2024-09-10T07:28:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:38 smithi046 ceph-mon[30705]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T07:28:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:28:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T07:28:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:38 smithi181 ceph-mon[35397]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T07:28:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:28:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:38 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3596186521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8019d3f-952b-42df-9c70-c0574211eaca"}]: dispatch 2024-09-10T07:28:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T07:28:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:38 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3596186521' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8019d3f-952b-42df-9c70-c0574211eaca"}]': finished 2024-09-10T07:28:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:38 smithi181 ceph-mon[35397]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T07:28:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:28:38.731 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:28:38 smithi046 sudo[46416]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T07:28:38.732 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:28:38 smithi046 sudo[46416]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:38.732 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:28:38 smithi046 sudo[46416]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:38.732 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:28:38 smithi046 sudo[46416]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:39.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:28:39.169+0000 7fd4ee000640 1 -- 172.21.15.46:0/4031629100 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd4e4098310 con 0x7fd4f0101a40 2024-09-10T07:28:39.593 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:39 smithi063 ceph-mon[35020]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:39.593 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:39 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3182397211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:28:39.593 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:39 smithi063 ceph-mon[35020]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T07:28:39.593 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:28:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:39 smithi181 ceph-mon[35397]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:39 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3182397211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:28:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:39 smithi181 ceph-mon[35397]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T07:28:39.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:28:39.659 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:28:39 smithi046 sudo[46466]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T07:28:39.659 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:28:39 smithi046 sudo[46466]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:39.659 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:28:39 smithi046 sudo[46466]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:39.659 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:28:39 smithi046 sudo[46466]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:39 smithi046 ceph-mon[30705]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:39 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3182397211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:28:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:39 smithi046 ceph-mon[30705]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T07:28:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:28:39.863 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:39 smithi063 sudo[43611]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T07:28:39.863 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:39 smithi063 sudo[43611]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:39.863 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:39 smithi063 sudo[43611]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:39.863 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:39 smithi063 sudo[43611]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:40.150 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:39 smithi063 sudo[43614]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T07:28:40.150 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:39 smithi063 sudo[43614]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:40.150 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:39 smithi063 sudo[43614]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:40.150 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:28:39 smithi063 sudo[43614]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:40.389 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:40 smithi181 sudo[44440]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T07:28:40.389 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:40 smithi181 sudo[44440]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:40.389 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:40 smithi181 sudo[44440]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:40.389 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:40 smithi181 sudo[44440]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:40.389 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:40 smithi181 ceph-mon[35397]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T07:28:40.389 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:40.389 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:40.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:40 smithi181 sudo[44443]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T07:28:40.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:40 smithi181 sudo[44443]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:40.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:40 smithi181 sudo[44443]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:40.650 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:28:40 smithi181 sudo[44443]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:40 smithi063 ceph-mon[35020]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T07:28:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:40.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:40 smithi046 ceph-mon[30705]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T07:28:40.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:40.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:41.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:40 smithi046 sudo[46853]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T07:28:41.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:40 smithi046 sudo[46853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:41.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:40 smithi046 sudo[46853]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:41.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:40 smithi046 sudo[46853]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:41.819 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:41 smithi181 ceph-mon[35397]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:41.819 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:41.819 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:41 smithi181 ceph-mon[35397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:41.819 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:41 smithi181 sudo[44446]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T07:28:41.820 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:41 smithi181 sudo[44446]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:41.820 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:41 smithi181 sudo[44446]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:41.820 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:41 smithi181 sudo[44446]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:41.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:41 smithi063 ceph-mon[35020]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:41.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:41.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:41 smithi063 ceph-mon[35020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:41 smithi046 sudo[47225]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T07:28:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:41 smithi046 sudo[47225]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:41 smithi046 sudo[47225]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:41 smithi046 sudo[47225]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:41 smithi046 ceph-mon[30705]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:41 smithi046 ceph-mon[30705]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:42.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 sudo[44449]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T07:28:42.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 sudo[44449]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:42.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 sudo[44449]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:42.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 sudo[44449]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:42.588 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 sudo[43617]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T07:28:42.588 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 sudo[43617]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:42.589 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 sudo[43617]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:42.589 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 sudo[43617]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:42.885 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:42.885 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.885 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.885 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:42 smithi046 ceph-mon[30705]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:42 smithi181 ceph-mon[35397]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 sudo[43620]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T07:28:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 sudo[43620]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T07:28:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 sudo[43620]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T07:28:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 sudo[43620]: pam_unix(sudo:session): session closed for user root 2024-09-10T07:28:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:28:42.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:42 smithi063 ceph-mon[35020]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:43 smithi181 ceph-mon[35397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:43.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:43 smithi181 ceph-mon[35397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:43.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:43 smithi063 ceph-mon[35020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:43.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:43 smithi063 ceph-mon[35020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:43 smithi046 ceph-mon[30705]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T07:28:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:43 smithi046 ceph-mon[30705]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T07:28:44.855 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:44 smithi046 ceph-mon[30705]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:44.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:44 smithi181 ceph-mon[35397]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:44.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:44 smithi063 ceph-mon[35020]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:46.639 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:46 smithi046 ceph-mon[30705]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T07:28:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:46 smithi181 ceph-mon[35397]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T07:28:46.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:46 smithi063 ceph-mon[35020]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T07:28:47.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:47 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T07:28:47.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:47 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:47.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:47 smithi046 ceph-mon[30705]: Deploying daemon osd.3 on smithi046 2024-09-10T07:28:47.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:47 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T07:28:47.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:47 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:47.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:47 smithi181 ceph-mon[35397]: Deploying daemon osd.3 on smithi046 2024-09-10T07:28:47.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:47 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T07:28:47.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:47 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:47.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:47 smithi063 ceph-mon[35020]: Deploying daemon osd.3 on smithi046 2024-09-10T07:28:48.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:48 smithi046 ceph-mon[30705]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T07:28:48.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:48 smithi181 ceph-mon[35397]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T07:28:48.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:48 smithi063 ceph-mon[35020]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T07:28:50.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:50 smithi046 ceph-mon[30705]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T07:28:50.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:50 smithi181 ceph-mon[35397]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T07:28:50.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:50 smithi063 ceph-mon[35020]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T07:28:52.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:52 smithi181 ceph-mon[35397]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T07:28:52.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:52 smithi063 ceph-mon[35020]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T07:28:52.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:52 smithi046 ceph-mon[30705]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T07:28:54.763 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:54 smithi046 ceph-mon[30705]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T07:28:54.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:54 smithi181 ceph-mon[35397]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T07:28:54.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:54 smithi063 ceph-mon[35020]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T07:28:55.774 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:55.775 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:55.775 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:55.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:55.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:55.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:55.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:55 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:28:55.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:55 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:55.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:55 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:56 smithi046 ceph-mon[30705]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T07:28:56.801 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:56 smithi181 ceph-mon[35397]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T07:28:56.801 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:56 smithi063 ceph-mon[35020]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T07:28:59.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:59 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:59 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:59 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:59.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:59 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:59.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:59 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:28:59 smithi181 ceph-mon[35397]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:59 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:59 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:59 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:59.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:59 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:59.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:59 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:28:59 smithi063 ceph-mon[35020]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:28:59.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:59 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:59 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:59 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:28:59.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:59 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:28:59.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:59 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:28:59.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:28:59 smithi046 ceph-mon[30705]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:29:00.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.102+0000 7fd4ee000640 1 -- 172.21.15.46:0/4031629100 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4f0062630 con 0x7fd4d80a2af0 2024-09-10T07:29:00.103 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 3 on host 'smithi046' 2024-09-10T07:29:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4d80a2af0 msgr2=0x7fd4d80a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4d80a2af0 0x7fd4d80a4fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0075d10 tx=0x7fd4dc005f90 comp rx=0 tx=0).stop 2024-09-10T07:29:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f0101a40 msgr2=0x7fd4f0075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f0101a40 0x7fd4f0075f90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009d20 tx=0x7fd4e400c6a0 comp rx=0 tx=0).stop 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 shutdown_connections 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4d80a2af0 0x7fd4d80a4fb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f0103600 0x7fd4f0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4f0102c40 0x7fd4f00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 --2- 172.21.15.46:0/4031629100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f0101a40 0x7fd4f0075f90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 >> 172.21.15.46:0/4031629100 conn(0x7fd4f00fd1f0 msgr2=0x7fd4f0105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.105+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 shutdown_connections 2024-09-10T07:29:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:00.106+0000 7fd4f6e00640 1 -- 172.21.15.46:0/4031629100 wait complete. 2024-09-10T07:29:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:00.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:00.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:00.847 DEBUG:teuthology.orchestra.run.smithi046:osd.3> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T07:29:00.851 INFO:tasks.cephadm:Deploying osd.4 on smithi063 with /dev/vg_nvme/lv_3... 2024-09-10T07:29:00.851 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T07:29:01.110 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:29:01.497 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:01 smithi063 ceph-mon[35020]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:29:01.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:01 smithi046 ceph-mon[30705]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:29:01.532 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:29:01 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:29:01.215+0000 7f3de1023640 -1 osd.3 0 log_to_monitors true 2024-09-10T07:29:01.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:01 smithi181 ceph-mon[35397]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:29:02.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:02 smithi046 ceph-mon[30705]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T07:29:02.587 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:02 smithi063 ceph-mon[35020]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T07:29:02.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:02 smithi181 ceph-mon[35397]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T07:29:03.571 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:03 smithi063 ceph-mon[35020]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T07:29:03.572 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:03 smithi063 ceph-mon[35020]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T07:29:03.572 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:03 smithi063 ceph-mon[35020]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:29:03.572 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:03.572 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:03 smithi063 ceph-mon[35020]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:29:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:03 smithi181 ceph-mon[35397]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T07:29:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:03 smithi181 ceph-mon[35397]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T07:29:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:03 smithi181 ceph-mon[35397]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:29:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:03.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:03 smithi181 ceph-mon[35397]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:29:03.652 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:29:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:03 smithi046 ceph-mon[30705]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T07:29:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:03 smithi046 ceph-mon[30705]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T07:29:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:03 smithi046 ceph-mon[30705]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:29:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:03 smithi046 ceph-mon[30705]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T07:29:03.659 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:29:03 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:29:03.212+0000 7f3ddbc00640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T07:29:03.659 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:29:03 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:29:03.218+0000 7f3dd6c00640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:29:03.678 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_3 2024-09-10T07:29:03.931 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:29:04.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:04 smithi063 ceph-mon[35020]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:29:04.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:04 smithi063 ceph-mon[35020]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T07:29:04.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:04.303 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:04.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2124196583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac101e40 msgr2=0x7f05ac1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2124196583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac101e40 0x7f05ac1086d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009ca0 tx=0x7f05a402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2124196583 shutdown_connections 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2124196583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac101e40 0x7f05ac1086d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2124196583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac101480 0x7f05ac101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2124196583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac100280 0x7f05ac100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2124196583 >> 172.21.15.63:0/2124196583 conn(0x7f05ac0fba30 msgr2=0x7f05ac0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2124196583 shutdown_connections 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.393+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2124196583 wait complete. 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.394+0000 7f05b1fbc640 1 Processor -- start 2024-09-10T07:29:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.394+0000 7f05b1fbc640 1 -- start start 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.394+0000 7f05b1fbc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac100280 0x7f05ac198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.394+0000 7f05b1fbc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac101480 0x7f05ac198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.394+0000 7f05b1fbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac101e40 0x7f05ac19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.394+0000 7f05b1fbc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac10c000 con 0x7f05ac101e40 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.394+0000 7f05b1fbc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac10be80 con 0x7f05ac100280 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.394+0000 7f05b1fbc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac10c180 con 0x7f05ac101480 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac100280 0x7f05ac198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac100280 0x7f05ac198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33780/0 (socket says 172.21.15.63:33780) 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 -- 172.21.15.63:0/2082354137 learned_addr learned my addr 172.21.15.63:0/2082354137 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05a3fff640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac101480 0x7f05ac198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 -- 172.21.15.63:0/2082354137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac101480 msgr2=0x7f05ac198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac101480 0x7f05ac198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 -- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac101e40 msgr2=0x7f05ac19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac101e40 0x7f05ac19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 -- 172.21.15.63:0/2082354137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac19fc70 con 0x7f05ac100280 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05ab7fe640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac100280 0x7f05ac198040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f059c00b450 tx=0x7f059c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05a97fa640 1 -- 172.21.15.63:0/2082354137 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c015020 con 0x7f05ac100280 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05a97fa640 1 -- 172.21.15.63:0/2082354137 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059c011d70 con 0x7f05ac100280 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ac19ff60 con 0x7f05ac100280 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05a97fa640 1 -- 172.21.15.63:0/2082354137 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c00f340 con 0x7f05ac100280 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.395+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ac1a0430 con 0x7f05ac100280 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.396+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ac1a06f0 con 0x7f05ac100280 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.398+0000 7f05a97fa640 1 -- 172.21.15.63:0/2082354137 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f059c018020 con 0x7f05ac100280 2024-09-10T07:29:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.398+0000 7f05a97fa640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05980a2bb0 0x7f05980a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:04.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.398+0000 7f05a97fa640 1 -- 172.21.15.63:0/2082354137 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f059c0cded0 con 0x7f05ac100280 2024-09-10T07:29:04.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.399+0000 7f05a3fff640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05980a2bb0 0x7f05980a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:04.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.399+0000 7f05a3fff640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05980a2bb0 0x7f05980a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05ac199560 tx=0x7f0594008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:29:04.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.402+0000 7f05a97fa640 1 -- 172.21.15.63:0/2082354137 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f059c01f020 con 0x7f05ac100280 2024-09-10T07:29:04.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:04.588+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f05ac1a08c0 con 0x7f05980a2bb0 2024-09-10T07:29:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:04 smithi181 ceph-mon[35397]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:29:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:04 smithi181 ceph-mon[35397]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T07:29:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:04.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:04.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:04 smithi046 ceph-mon[30705]: from='osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:29:04.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:04 smithi046 ceph-mon[30705]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T07:29:04.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:04.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:05.322 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:29:05.322 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:29:05.322 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] boot 2024-09-10T07:29:05.322 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T07:29:05.322 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:05.322 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:05.322 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.323 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:05 smithi063 ceph-mon[35020]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] boot 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:05.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:05.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:05.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:05.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:05 smithi181 ceph-mon[35397]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T07:29:05.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:29:05.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:29:05.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: osd.3 [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] boot 2024-09-10T07:29:05.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T07:29:05.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T07:29:05.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:05.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:05.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:05 smithi046 ceph-mon[30705]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T07:29:06.332 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:06 smithi063 ceph-mon[35020]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:06.332 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:06 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi046 2024-09-10T07:29:06.332 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:06 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi046 to 8547M 2024-09-10T07:29:06.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:06 smithi181 ceph-mon[35397]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:06.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:06 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi046 2024-09-10T07:29:06.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:06 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi046 to 8547M 2024-09-10T07:29:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:06 smithi046 ceph-mon[30705]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:06 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi046 2024-09-10T07:29:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:06 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi046 to 8547M 2024-09-10T07:29:07.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:07 smithi063 ceph-mon[35020]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T07:29:07.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:07 smithi063 ceph-mon[35020]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:07.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:07 smithi063 ceph-mon[35020]: from='client.? 172.21.15.63:0/2226620458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]: dispatch 2024-09-10T07:29:07.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:07 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]: dispatch 2024-09-10T07:29:07.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:07 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]': finished 2024-09-10T07:29:07.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:07 smithi063 ceph-mon[35020]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T07:29:07.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:07 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:07.575 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:07 smithi181 ceph-mon[35397]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T07:29:07.575 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:07 smithi181 ceph-mon[35397]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:07.576 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:07 smithi181 ceph-mon[35397]: from='client.? 172.21.15.63:0/2226620458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]: dispatch 2024-09-10T07:29:07.576 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:07 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]: dispatch 2024-09-10T07:29:07.576 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:07 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]': finished 2024-09-10T07:29:07.576 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:07 smithi181 ceph-mon[35397]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T07:29:07.576 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:07 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:07.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:07 smithi046 ceph-mon[30705]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T07:29:07.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:07 smithi046 ceph-mon[30705]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:07.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:07 smithi046 ceph-mon[30705]: from='client.? 172.21.15.63:0/2226620458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]: dispatch 2024-09-10T07:29:07.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:07 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]: dispatch 2024-09-10T07:29:07.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:07 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe349fc9-2de4-439f-a982-e9282f26d488"}]': finished 2024-09-10T07:29:07.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:07 smithi046 ceph-mon[30705]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T07:29:07.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:07 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:08.533 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:08 smithi181 ceph-mon[35397]: from='client.? 172.21.15.63:0/133570433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:29:08.533 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:08 smithi181 ceph-mon[35397]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T07:29:08.533 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:08.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:08 smithi063 ceph-mon[35020]: from='client.? 172.21.15.63:0/133570433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:29:08.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:08 smithi063 ceph-mon[35020]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T07:29:08.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:08.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:08 smithi046 ceph-mon[30705]: from='client.? 172.21.15.63:0/133570433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:29:08.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:08 smithi046 ceph-mon[30705]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T07:29:08.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:09.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:09 smithi181 ceph-mon[35397]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:09.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:09 smithi063 ceph-mon[35020]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:09.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:09 smithi046 ceph-mon[30705]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:10.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:10 smithi181 ceph-mon[35397]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:10.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:10 smithi063 ceph-mon[35020]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:10.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:10 smithi046 ceph-mon[30705]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:12.862 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:12 smithi063 ceph-mon[35020]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T07:29:12.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:12 smithi181 ceph-mon[35397]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T07:29:12.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:12 smithi046 ceph-mon[30705]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T07:29:14.547 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:14 smithi063 ceph-mon[35020]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:29:14.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:14 smithi181 ceph-mon[35397]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:29:14.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:14 smithi046 ceph-mon[30705]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:29:15.782 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T07:29:15.782 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:15.782 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:15 smithi063 ceph-mon[35020]: Deploying daemon osd.4 on smithi063 2024-09-10T07:29:15.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T07:29:15.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:15.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:15 smithi181 ceph-mon[35397]: Deploying daemon osd.4 on smithi063 2024-09-10T07:29:15.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T07:29:15.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:15.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:15 smithi046 ceph-mon[30705]: Deploying daemon osd.4 on smithi063 2024-09-10T07:29:16.756 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:16 smithi063 ceph-mon[35020]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T07:29:16.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:16 smithi181 ceph-mon[35397]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T07:29:16.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:16 smithi046 ceph-mon[30705]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T07:29:18.596 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:18 smithi063 ceph-mon[35020]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T07:29:18.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:18 smithi181 ceph-mon[35397]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T07:29:18.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:18 smithi046 ceph-mon[30705]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T07:29:20.633 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:20 smithi063 ceph-mon[35020]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:29:20.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:20 smithi046 ceph-mon[30705]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:29:20.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:20 smithi181 ceph-mon[35397]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:29:21.753 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:21.753 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:21.753 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:21.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:21.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:21.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:21.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:21.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:21.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:22.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:22 smithi063 ceph-mon[35020]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:29:22.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:22 smithi181 ceph-mon[35397]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:29:22.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:22 smithi046 ceph-mon[30705]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:29:24.819 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.819 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.819 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:24.819 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:24.819 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.819 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:24 smithi063 ceph-mon[35020]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:24.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.861+0000 7f05a97fa640 1 -- 172.21.15.63:0/2082354137 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f05ac1a08c0 con 0x7f05980a2bb0 2024-09-10T07:29:24.860 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 4 on host 'smithi063' 2024-09-10T07:29:24.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05980a2bb0 msgr2=0x7f05980a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05980a2bb0 0x7f05980a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05ac199560 tx=0x7f0594008040 comp rx=0 tx=0).stop 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac100280 msgr2=0x7f05ac198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac100280 0x7f05ac198040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f059c00b450 tx=0x7f059c00b910 comp rx=0 tx=0).stop 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 shutdown_connections 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05980a2bb0 0x7f05980a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac101e40 0x7f05ac19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac101480 0x7f05ac198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 --2- 172.21.15.63:0/2082354137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05ac100280 0x7f05ac198040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 >> 172.21.15.63:0/2082354137 conn(0x7f05ac0fba30 msgr2=0x7f05ac0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 shutdown_connections 2024-09-10T07:29:24.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:29:24.863+0000 7f05b1fbc640 1 -- 172.21.15.63:0/2082354137 wait complete. 2024-09-10T07:29:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:24.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:24 smithi181 ceph-mon[35397]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:24.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:24.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:24.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:24.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:24 smithi046 ceph-mon[30705]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:25.458 DEBUG:teuthology.orchestra.run.smithi063:osd.4> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T07:29:25.460 INFO:tasks.cephadm:Deploying osd.5 on smithi181 with /dev/vg_nvme/lv_3... 2024-09-10T07:29:25.460 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T07:29:25.672 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:25.673 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:25.673 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:25.714 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:29:25.834 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:25.834 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:25.834 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:25.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:25.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:25.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:26.834 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:26 smithi063 ceph-mon[35020]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:26.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:26 smithi181 ceph-mon[35397]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:26.915 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:26 smithi046 ceph-mon[30705]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:27.150 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:29:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:29:27.018+0000 7f9e3099c640 -1 osd.4 0 log_to_monitors true 2024-09-10T07:29:27.872 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:27 smithi063 ceph-mon[35020]: from='osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T07:29:27.873 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:27 smithi063 ceph-mon[35020]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T07:29:27.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:27 smithi181 ceph-mon[35397]: from='osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T07:29:27.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:27 smithi181 ceph-mon[35397]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T07:29:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:27 smithi046 ceph-mon[30705]: from='osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T07:29:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:27 smithi046 ceph-mon[30705]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T07:29:28.603 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:29:28.628 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2024-09-10T07:29:28.869 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:29:28.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T07:29:28.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi063 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi063 to 8544M 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:28.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:28.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:28 smithi181 ceph-mon[35397]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:29.150 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:29:28.673+0000 7f9e2c91f640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T07:29:29.150 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:29:28.680+0000 7f9e28747640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi063 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi063 to 8544M 2024-09-10T07:29:29.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:29.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:29.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:29.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:29.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:28 smithi063 ceph-mon[35020]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi063 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:29.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:29.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:29.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi063 to 8544M 2024-09-10T07:29:29.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:29.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:29.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:29.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:29.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:28 smithi046 ceph-mon[30705]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 -- 172.21.15.181:0/229785253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff01031f0 msgr2=0x7f7ff01035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/229785253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff01031f0 0x7f7ff01035f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009900 tx=0x7f7fe002a410 comp rx=0 tx=0).stop 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 -- 172.21.15.181:0/229785253 shutdown_connections 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/229785253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0100330 0x7f7ff010a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/229785253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff0103bc0 0x7f7ff00ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/229785253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff01031f0 0x7f7ff01035f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 -- 172.21.15.181:0/229785253 >> 172.21.15.181:0/229785253 conn(0x7f7ff00fba40 msgr2=0x7f7ff00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 -- 172.21.15.181:0/229785253 shutdown_connections 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 -- 172.21.15.181:0/229785253 wait complete. 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.396+0000 7f7ff57f0640 1 Processor -- start 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7ff57f0640 1 -- start start 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7ff57f0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff0100330 0x7f7ff019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7ff57f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff01031f0 0x7f7ff019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7ff57f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff0103bc0 0x7f7ff01a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7ff57f0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff010d930 con 0x7f7ff01031f0 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7ff57f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff010d7b0 con 0x7f7ff0103bc0 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7ff57f0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff010dab0 con 0x7f7ff0100330 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff0100330 0x7f7ff019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff0100330 0x7f7ff019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43760/0 (socket says 172.21.15.181:43760) 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 -- 172.21.15.181:0/3853986677 learned_addr learned my addr 172.21.15.181:0/3853986677 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7fef7fe640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff0103bc0 0x7f7ff01a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7fee7fc640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff01031f0 0x7f7ff019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 -- 172.21.15.181:0/3853986677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff0103bc0 msgr2=0x7f7ff01a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff0103bc0 0x7f7ff01a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 -- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff01031f0 msgr2=0x7f7ff019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff01031f0 0x7f7ff019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:29.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 -- 172.21.15.181:0/3853986677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff01a1dd0 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7fee7fc640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff01031f0 0x7f7ff019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7feeffd640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff0100330 0x7f7ff019a1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0031610 tx=0x7f7fe00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.397+0000 7f7fd7fff640 1 -- 172.21.15.181:0/3853986677 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe003f030 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.398+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff01a2060 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.398+0000 7f7fd7fff640 1 -- 172.21.15.181:0/3853986677 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe0003ce0 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.398+0000 7f7fd7fff640 1 -- 172.21.15.181:0/3853986677 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0035070 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.398+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff01a25c0 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.399+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff01a2820 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.401+0000 7f7fd7fff640 1 -- 172.21.15.181:0/3853986677 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7fe0042030 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.401+0000 7f7fd7fff640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7fd80a2bb0 0x7f7fd80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.401+0000 7f7fd7fff640 1 -- 172.21.15.181:0/3853986677 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f7fe00ee6c0 con 0x7f7ff0100330 2024-09-10T07:29:29.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.401+0000 7f7fee7fc640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7fd80a2bb0 0x7f7fd80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:29.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.402+0000 7f7fee7fc640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7fd80a2bb0 0x7f7fd80a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ff019b6c0 tx=0x7f7fe4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:29:29.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.405+0000 7f7fd7fff640 1 -- 172.21.15.181:0/3853986677 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7fe00bc7b0 con 0x7f7ff0100330 2024-09-10T07:29:29.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:29.578+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7ff0109e80 con 0x7f7fd80a2bb0 2024-09-10T07:29:29.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:29 smithi181 ceph-mon[35397]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:29:29.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:29 smithi181 ceph-mon[35397]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T07:29:29.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:29.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:29.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:29.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:29.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:29 smithi063 ceph-mon[35020]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:29:30.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:29 smithi063 ceph-mon[35020]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T07:29:30.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:30.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:30.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:30.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:30.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:29 smithi046 ceph-mon[30705]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:29:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:29 smithi046 ceph-mon[30705]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T07:29:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:31.079 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:30 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:29:31.079 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:30 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:29:31.079 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:30 smithi181 ceph-mon[35397]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:31.079 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:30 smithi181 ceph-mon[35397]: osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] boot 2024-09-10T07:29:31.079 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:30 smithi181 ceph-mon[35397]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T07:29:31.080 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:30 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:31.080 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:30 smithi181 ceph-mon[35397]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:31.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:30 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:29:31.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:30 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:29:31.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:30 smithi063 ceph-mon[35020]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:31.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:30 smithi063 ceph-mon[35020]: osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] boot 2024-09-10T07:29:31.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:30 smithi063 ceph-mon[35020]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T07:29:31.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:30 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:31.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:30 smithi063 ceph-mon[35020]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:31.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:30 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:29:31.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:30 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:29:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:30 smithi046 ceph-mon[30705]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:30 smithi046 ceph-mon[30705]: osd.4 [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] boot 2024-09-10T07:29:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:30 smithi046 ceph-mon[30705]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T07:29:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:30 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T07:29:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:30 smithi046 ceph-mon[30705]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T07:29:31.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:31 smithi181 ceph-mon[35397]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T07:29:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:31 smithi063 ceph-mon[35020]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T07:29:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:31 smithi046 ceph-mon[30705]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T07:29:32.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:32 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/3936513623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]: dispatch 2024-09-10T07:29:32.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:32 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]: dispatch 2024-09-10T07:29:32.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:32 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]': finished 2024-09-10T07:29:32.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:32 smithi181 ceph-mon[35397]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T07:29:32.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:32 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:32.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:32 smithi181 ceph-mon[35397]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:32.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:32 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/1221313983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:29:33.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:32 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/3936513623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]: dispatch 2024-09-10T07:29:33.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:32 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]: dispatch 2024-09-10T07:29:33.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:32 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]': finished 2024-09-10T07:29:33.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:32 smithi063 ceph-mon[35020]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T07:29:33.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:32 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:33.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:32 smithi063 ceph-mon[35020]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:33.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:32 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/1221313983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:29:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:32 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/3936513623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]: dispatch 2024-09-10T07:29:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:32 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]: dispatch 2024-09-10T07:29:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:32 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "619c621c-ab3a-40fc-bdd4-498347c3a5fc"}]': finished 2024-09-10T07:29:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:32 smithi046 ceph-mon[30705]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T07:29:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:32 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:32 smithi046 ceph-mon[30705]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:33.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:32 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/1221313983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:29:34.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:34 smithi181 ceph-mon[35397]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:34.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:34 smithi063 ceph-mon[35020]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:34.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:34 smithi046 ceph-mon[30705]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:36.819 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:36 smithi181 ceph-mon[35397]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:36 smithi063 ceph-mon[35020]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:36.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:36 smithi046 ceph-mon[30705]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:38.575 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:38 smithi181 ceph-mon[35397]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:38 smithi063 ceph-mon[35020]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:38.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:38 smithi046 ceph-mon[30705]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T07:29:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T07:29:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:40.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T07:29:40.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:41.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:41 smithi181 ceph-mon[35397]: Deploying daemon osd.5 on smithi181 2024-09-10T07:29:41.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:41 smithi181 ceph-mon[35397]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:41.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:41 smithi063 ceph-mon[35020]: Deploying daemon osd.5 on smithi181 2024-09-10T07:29:41.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:41 smithi063 ceph-mon[35020]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:41.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:41 smithi046 ceph-mon[30705]: Deploying daemon osd.5 on smithi181 2024-09-10T07:29:41.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:41 smithi046 ceph-mon[30705]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:42.787 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:42 smithi181 ceph-mon[35397]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:42 smithi063 ceph-mon[35020]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:42.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:42 smithi046 ceph-mon[30705]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:44 smithi181 ceph-mon[35397]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:44.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:44 smithi063 ceph-mon[35020]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:44.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:44 smithi046 ceph-mon[30705]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:46 smithi181 ceph-mon[35397]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:46.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:46 smithi063 ceph-mon[35020]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:46.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:46 smithi046 ceph-mon[30705]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:47.638 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:47 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:47.639 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:47 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:47.639 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:47 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:47.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:47 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:47.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:47 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:47.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:47 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:47 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:47 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:47.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:47 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:48.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:48 smithi181 ceph-mon[35397]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:48.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:48 smithi063 ceph-mon[35020]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:48 smithi046 ceph-mon[30705]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:50.499 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:50.499 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:50.500 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:50.500 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:50.500 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:50.541 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 5 on host 'smithi181' 2024-09-10T07:29:50.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.539+0000 7f7fd7fff640 1 -- 172.21.15.181:0/3853986677 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7ff0109e80 con 0x7f7fd80a2bb0 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7fd80a2bb0 msgr2=0x7f7fd80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7fd80a2bb0 0x7f7fd80a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ff019b6c0 tx=0x7f7fe4005e50 comp rx=0 tx=0).stop 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff0100330 msgr2=0x7f7ff019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff0100330 0x7f7ff019a1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0031610 tx=0x7f7fe00043b0 comp rx=0 tx=0).stop 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 shutdown_connections 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff0103bc0 0x7f7ff01a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7fd80a2bb0 0x7f7fd80a5070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff01031f0 0x7f7ff019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:50.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 --2- 172.21.15.181:0/3853986677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff0100330 0x7f7ff019a1a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:50.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 >> 172.21.15.181:0/3853986677 conn(0x7f7ff00fba40 msgr2=0x7f7ff0108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:29:50.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 shutdown_connections 2024-09-10T07:29:50.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:29:50.542+0000 7f7ff57f0640 1 -- 172.21.15.181:0/3853986677 wait complete. 2024-09-10T07:29:50.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:50.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:50.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:50.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:50.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:50.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:50.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:51.230 DEBUG:teuthology.orchestra.run.smithi181:osd.5> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T07:29:51.232 INFO:tasks.cephadm:Deploying osd.6 on smithi046 with /dev/vg_nvme/lv_2... 2024-09-10T07:29:51.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T07:29:51.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:51 smithi181 ceph-mon[35397]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:51.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:51.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:51.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:51.612 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:29:51.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:51 smithi063 ceph-mon[35020]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:51.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:51.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:51.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:51.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:51 smithi046 ceph-mon[30705]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:51.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:29:51.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:51.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:52.351 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:52 smithi181 ceph-mon[35397]: from='osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T07:29:52.351 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:52 smithi181 ceph-mon[35397]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T07:29:52.352 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:29:52 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:29:52.070+0000 7f5ab6350640 -1 osd.5 0 log_to_monitors true 2024-09-10T07:29:52.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:52 smithi046 ceph-mon[30705]: from='osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T07:29:52.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:52 smithi046 ceph-mon[30705]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T07:29:52.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:52 smithi063 ceph-mon[35020]: from='osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T07:29:52.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:52 smithi063 ceph-mon[35020]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T07:29:53.458 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:53 smithi046 ceph-mon[30705]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T07:29:53.458 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:53 smithi046 ceph-mon[30705]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T07:29:53.459 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:53 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:53.459 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:53 smithi046 ceph-mon[30705]: from='osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:29:53.459 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:53 smithi046 ceph-mon[30705]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:29:53.649 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:29:53 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:29:53.204+0000 7f5ab22d3640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T07:29:53.650 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:29:53 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:29:53.210+0000 7f5aae0fb640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:29:53.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:53 smithi181 ceph-mon[35397]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T07:29:53.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:53 smithi181 ceph-mon[35397]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T07:29:53.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:53 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:53.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:53 smithi181 ceph-mon[35397]: from='osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:29:53.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:53 smithi181 ceph-mon[35397]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:29:53.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:53 smithi063 ceph-mon[35020]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T07:29:53.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:53 smithi063 ceph-mon[35020]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T07:29:53.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:53 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:53.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:53 smithi063 ceph-mon[35020]: from='osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:29:53.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:53 smithi063 ceph-mon[35020]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:29:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:29:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T07:29:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:54.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:54.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:54.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:54.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:54.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:54.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:29:54.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:29:55.039 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:29:55.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-09-10T07:29:55.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:55 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:29:55.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:55 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:29:55.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:55 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi181 2024-09-10T07:29:55.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:55 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi181 to 8544M 2024-09-10T07:29:55.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:55.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:55 smithi046 ceph-mon[30705]: osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] boot 2024-09-10T07:29:55.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:55 smithi046 ceph-mon[30705]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T07:29:55.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:29:55.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:55 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:29:55.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:55 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:29:55.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:55 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi181 2024-09-10T07:29:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:55 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi181 to 8544M 2024-09-10T07:29:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:55 smithi181 ceph-mon[35397]: osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] boot 2024-09-10T07:29:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:55 smithi181 ceph-mon[35397]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T07:29:55.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:55.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:55 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:29:55.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:55 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:29:55.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:55 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi181 2024-09-10T07:29:55.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:55 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi181 to 8544M 2024-09-10T07:29:55.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:55 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:55.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:55 smithi063 ceph-mon[35020]: osd.5 [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] boot 2024-09-10T07:29:55.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:55 smithi063 ceph-mon[35020]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T07:29:55.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:55 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T07:29:56.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 -- 172.21.15.46:0/3968776253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2224074c50 msgr2=0x7f22240750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 --2- 172.21.15.46:0/3968776253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2224074c50 0x7f22240750b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2210009860 tx=0x7f221002a340 comp rx=0 tx=0).stop 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 -- 172.21.15.46:0/3968776253 shutdown_connections 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 --2- 172.21.15.46:0/3968776253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222410e1e0 0x7f22241105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 --2- 172.21.15.46:0/3968776253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2224074c50 0x7f22240750b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 --2- 172.21.15.46:0/3968776253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2224076220 0x7f2224074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 -- 172.21.15.46:0/3968776253 >> 172.21.15.46:0/3968776253 conn(0x7f22240fd390 msgr2=0x7f22240ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 -- 172.21.15.46:0/3968776253 shutdown_connections 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.028+0000 7f222a400640 1 -- 172.21.15.46:0/3968776253 wait complete. 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.029+0000 7f222a400640 1 Processor -- start 2024-09-10T07:29:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.029+0000 7f222a400640 1 -- start start 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.029+0000 7f222a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2224074c50 0x7f222419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f222a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2224076220 0x7f222419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f222a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222410e1e0 0x7f22241a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f222a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2224113ea0 con 0x7f2224074c50 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f222a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2224113d20 con 0x7f2224076220 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f222a400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2224114020 con 0x7f222410e1e0 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2224074c50 0x7f222419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2224074c50 0x7f222419e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38586/0 (socket says 172.21.15.46:38586) 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223400640 1 -- 172.21.15.46:0/2052333980 learned_addr learned my addr 172.21.15.46:0/2052333980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223e00640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222410e1e0 0x7f22241a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2222a00640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2224076220 0x7f222419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223400640 1 -- 172.21.15.46:0/2052333980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222410e1e0 msgr2=0x7f22241a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222410e1e0 0x7f22241a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223400640 1 -- 172.21.15.46:0/2052333980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2224076220 msgr2=0x7f222419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2224076220 0x7f222419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.030+0000 7f2223400640 1 -- 172.21.15.46:0/2052333980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22241a6140 con 0x7f2224074c50 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.031+0000 7f2223400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2224074c50 0x7f222419e510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9f0 tx=0x7f221400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.031+0000 7f221be00640 1 -- 172.21.15.46:0/2052333980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400ccc0 con 0x7f2224074c50 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.031+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22241a6430 con 0x7f2224074c50 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.031+0000 7f221be00640 1 -- 172.21.15.46:0/2052333980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2214010910 con 0x7f2224074c50 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.031+0000 7f221be00640 1 -- 172.21.15.46:0/2052333980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214018900 con 0x7f2224074c50 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.031+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241a6990 con 0x7f2224074c50 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.032+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22241043b0 con 0x7f2224074c50 2024-09-10T07:29:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.034+0000 7f221be00640 1 -- 172.21.15.46:0/2052333980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2214010ac0 con 0x7f2224074c50 2024-09-10T07:29:56.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.035+0000 7f221be00640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22080a2bb0 0x7f22080a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:29:56.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.035+0000 7f221be00640 1 -- 172.21.15.46:0/2052333980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4562+0+0 (secure 0 0 0) 0x7f22140ce490 con 0x7f2224074c50 2024-09-10T07:29:56.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.036+0000 7f2222a00640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22080a2bb0 0x7f22080a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:29:56.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.036+0000 7f2222a00640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22080a2bb0 0x7f22080a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f221002a930 tx=0x7f2210030570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:29:56.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.040+0000 7f221be00640 1 -- 172.21.15.46:0/2052333980 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f221409bba0 con 0x7f2224074c50 2024-09-10T07:29:56.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:29:56.235+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2224062630 con 0x7f22080a2bb0 2024-09-10T07:29:56.550 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:56 smithi181 ceph-mon[35397]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:56.551 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:56 smithi181 ceph-mon[35397]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T07:29:56.551 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:56 smithi063 ceph-mon[35020]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:56.551 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:56 smithi063 ceph-mon[35020]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T07:29:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:56 smithi046 ceph-mon[30705]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T07:29:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:56 smithi046 ceph-mon[30705]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T07:29:57.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:57 smithi046 ceph-mon[30705]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T07:29:57.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:57 smithi046 ceph-mon[30705]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:57.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:57 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:57.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:57 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:57.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:57 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:57.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:57 smithi181 ceph-mon[35397]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T07:29:57.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:57 smithi181 ceph-mon[35397]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:57.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:57 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:57.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:57 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:57.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:57 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:57.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:57 smithi063 ceph-mon[35020]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T07:29:57.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:57 smithi063 ceph-mon[35020]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:29:57.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:57 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:29:57.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:57 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:29:57.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:57 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:29:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:58 smithi181 ceph-mon[35397]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:29:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:58 smithi181 ceph-mon[35397]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T07:29:58.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:58 smithi063 ceph-mon[35020]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:29:58.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:58 smithi063 ceph-mon[35020]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T07:29:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:58 smithi046 ceph-mon[30705]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:29:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:58 smithi046 ceph-mon[30705]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T07:29:59.501 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:29:59 smithi046 ceph-mon[30705]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T07:29:59.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:29:59 smithi181 ceph-mon[35397]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T07:29:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:29:59 smithi063 ceph-mon[35020]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T07:30:00.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:00 smithi046 ceph-mon[30705]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:00.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:00 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/974501545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]: dispatch 2024-09-10T07:30:00.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:00 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]: dispatch 2024-09-10T07:30:00.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:00 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]': finished 2024-09-10T07:30:00.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:00 smithi046 ceph-mon[30705]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T07:30:00.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:00.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:00 smithi046 ceph-mon[30705]: overall HEALTH_OK 2024-09-10T07:30:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:00 smithi181 ceph-mon[35397]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:00 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/974501545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]: dispatch 2024-09-10T07:30:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:00 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]: dispatch 2024-09-10T07:30:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:00 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]': finished 2024-09-10T07:30:00.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:00 smithi181 ceph-mon[35397]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T07:30:00.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:00.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:00 smithi181 ceph-mon[35397]: overall HEALTH_OK 2024-09-10T07:30:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:00 smithi063 ceph-mon[35020]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:00 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/974501545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]: dispatch 2024-09-10T07:30:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:00 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]: dispatch 2024-09-10T07:30:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:00 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bd4a464-96a7-4436-bc64-809acbdf2041"}]': finished 2024-09-10T07:30:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:00 smithi063 ceph-mon[35020]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T07:30:00.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:00.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:00 smithi063 ceph-mon[35020]: overall HEALTH_OK 2024-09-10T07:30:01.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:01 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1721406471' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:01.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:01 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1721406471' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:01.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:01 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1721406471' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:02.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:02 smithi181 ceph-mon[35397]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:02.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:02 smithi063 ceph-mon[35020]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:02.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:02 smithi046 ceph-mon[30705]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:04.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:04 smithi046 ceph-mon[30705]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T07:30:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:04 smithi181 ceph-mon[35397]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T07:30:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:04 smithi063 ceph-mon[35020]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T07:30:06.567 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:06 smithi046 ceph-mon[30705]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T07:30:06.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:06 smithi181 ceph-mon[35397]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T07:30:06.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:06 smithi063 ceph-mon[35020]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T07:30:08.452 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:08 smithi046 ceph-mon[30705]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T07:30:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:08 smithi181 ceph-mon[35397]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T07:30:08.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:08 smithi063 ceph-mon[35020]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T07:30:09.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T07:30:09.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:09 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:09.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T07:30:09.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:09 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:09.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T07:30:09.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:09 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:10.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:10 smithi046 ceph-mon[30705]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T07:30:10.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:10 smithi046 ceph-mon[30705]: Deploying daemon osd.6 on smithi046 2024-09-10T07:30:10.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:10 smithi181 ceph-mon[35397]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T07:30:10.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:10 smithi181 ceph-mon[35397]: Deploying daemon osd.6 on smithi046 2024-09-10T07:30:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:10 smithi063 ceph-mon[35020]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T07:30:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:10 smithi063 ceph-mon[35020]: Deploying daemon osd.6 on smithi046 2024-09-10T07:30:12.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:12 smithi181 ceph-mon[35397]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T07:30:12.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:12 smithi063 ceph-mon[35020]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T07:30:12.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:12 smithi046 ceph-mon[30705]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T07:30:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:14 smithi181 ceph-mon[35397]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:30:14.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:14 smithi063 ceph-mon[35020]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:30:14.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:14 smithi046 ceph-mon[30705]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T07:30:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:16 smithi046 ceph-mon[30705]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:16 smithi181 ceph-mon[35397]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:16 smithi063 ceph-mon[35020]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:18 smithi181 ceph-mon[35397]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:18.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:18.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:18 smithi063 ceph-mon[35020]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:18 smithi046 ceph-mon[30705]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:18.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:20.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:20 smithi181 ceph-mon[35397]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:20.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:20 smithi063 ceph-mon[35020]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:20.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:20 smithi046 ceph-mon[30705]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:21.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.752+0000 7f221be00640 1 -- 172.21.15.46:0/2052333980 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2224062630 con 0x7f22080a2bb0 2024-09-10T07:30:21.753 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 6 on host 'smithi046' 2024-09-10T07:30:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.754+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22080a2bb0 msgr2=0x7f22080a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22080a2bb0 0x7f22080a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f221002a930 tx=0x7f2210030570 comp rx=0 tx=0).stop 2024-09-10T07:30:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2224074c50 msgr2=0x7f222419e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2224074c50 0x7f222419e510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9f0 tx=0x7f221400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:30:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 shutdown_connections 2024-09-10T07:30:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222410e1e0 0x7f22241a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2224076220 0x7f222419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22080a2bb0 0x7f22080a5070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 --2- 172.21.15.46:0/2052333980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2224074c50 0x7f222419e510 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 >> 172.21.15.46:0/2052333980 conn(0x7f22240fd390 msgr2=0x7f2224101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:30:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 shutdown_connections 2024-09-10T07:30:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:30:21.755+0000 7f222a400640 1 -- 172.21.15.46:0/2052333980 wait complete. 2024-09-10T07:30:22.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:22.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:22.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:22.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:22.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:22.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:22.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:22.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:22.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:22.629 DEBUG:teuthology.orchestra.run.smithi046:osd.6> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T07:30:22.633 INFO:tasks.cephadm:Deploying osd.7 on smithi063 with /dev/vg_nvme/lv_2... 2024-09-10T07:30:22.633 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T07:30:22.896 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:30:22.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:22 smithi046 ceph-mon[30705]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T07:30:22.935 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:30:22 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:30:22.780+0000 7faf7761b640 -1 osd.6 0 log_to_monitors true 2024-09-10T07:30:23.001 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:22 smithi063 ceph-mon[35020]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T07:30:23.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:22 smithi181 ceph-mon[35397]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T07:30:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:23 smithi181 ceph-mon[35397]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:23 smithi181 ceph-mon[35397]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T07:30:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:23 smithi181 ceph-mon[35397]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T07:30:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:23 smithi181 ceph-mon[35397]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:30:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:23 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:23 smithi063 ceph-mon[35020]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:23 smithi063 ceph-mon[35020]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T07:30:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:23 smithi063 ceph-mon[35020]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T07:30:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:23 smithi063 ceph-mon[35020]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:30:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:23 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:23 smithi046 ceph-mon[30705]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:23 smithi046 ceph-mon[30705]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T07:30:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:23 smithi046 ceph-mon[30705]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T07:30:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:23 smithi046 ceph-mon[30705]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:30:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:23 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:24.462 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:30:24 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:30:24.443+0000 7faf72200640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T07:30:24.462 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:30:24 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:30:24.462+0000 7faf68200640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:30:24.997 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:24 smithi046 ceph-mon[30705]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:30:24.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:24 smithi046 ceph-mon[30705]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T07:30:24.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:24.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:24.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:24 smithi046 ceph-mon[30705]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' 2024-09-10T07:30:25.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:24 smithi181 ceph-mon[35397]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:30:25.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:24 smithi181 ceph-mon[35397]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T07:30:25.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:25.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:25.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:24 smithi181 ceph-mon[35397]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' 2024-09-10T07:30:25.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:24 smithi063 ceph-mon[35020]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:30:25.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:24 smithi063 ceph-mon[35020]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T07:30:25.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:25.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:25.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:24 smithi063 ceph-mon[35020]: from='osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775]' entity='osd.6' 2024-09-10T07:30:25.610 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:30:25.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_2 2024-09-10T07:30:25.892 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:30:25.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:25 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:30:25.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:25 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:30:25.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:25 smithi063 ceph-mon[35020]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:25.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:25.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:25 smithi063 ceph-mon[35020]: osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] boot 2024-09-10T07:30:25.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:25 smithi063 ceph-mon[35020]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T07:30:25.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:26.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:25 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:30:26.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:25 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:30:26.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:25 smithi181 ceph-mon[35397]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:26.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:26.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:25 smithi181 ceph-mon[35397]: osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] boot 2024-09-10T07:30:26.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:25 smithi181 ceph-mon[35397]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T07:30:26.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:25 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:30:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:25 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:30:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:25 smithi046 ceph-mon[30705]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T07:30:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:25 smithi046 ceph-mon[30705]: osd.6 [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] boot 2024-09-10T07:30:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:25 smithi046 ceph-mon[30705]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T07:30:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 -- 172.21.15.63:0/4266657969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 msgr2=0x7fda28108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/4266657969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 0x7fda28108f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fda20009ca0 tx=0x7fda2002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 -- 172.21.15.63:0/4266657969 shutdown_connections 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/4266657969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 0x7fda28108f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/4266657969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda28101cd0 0x7fda28102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/4266657969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda28100ad0 0x7fda28100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 -- 172.21.15.63:0/4266657969 >> 172.21.15.63:0/4266657969 conn(0x7fda280fc280 msgr2=0x7fda280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 -- 172.21.15.63:0/4266657969 shutdown_connections 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.338+0000 7fda2eeaa640 1 -- 172.21.15.63:0/4266657969 wait complete. 2024-09-10T07:30:26.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2eeaa640 1 Processor -- start 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2eeaa640 1 -- start start 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2eeaa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda28100ad0 0x7fda2819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2eeaa640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda28101cd0 0x7fda2819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2eeaa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 0x7fda281a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2eeaa640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda2810c7d0 con 0x7fda28100ad0 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2eeaa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda2810c650 con 0x7fda28102690 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2eeaa640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda2810c950 con 0x7fda28101cd0 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2d447640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 0x7fda281a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2d447640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 0x7fda281a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56368/0 (socket says 172.21.15.63:56368) 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda2d447640 1 -- 172.21.15.63:0/1293796514 learned_addr learned my addr 172.21.15.63:0/1293796514 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.339+0000 7fda27fff640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda28101cd0 0x7fda2819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2cc46640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda28100ad0 0x7fda2819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:30:26.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2d447640 1 -- 172.21.15.63:0/1293796514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda28101cd0 msgr2=0x7fda2819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2d447640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda28101cd0 0x7fda2819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2d447640 1 -- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda28100ad0 msgr2=0x7fda2819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2d447640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda28100ad0 0x7fda2819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2d447640 1 -- 172.21.15.63:0/1293796514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda281a1d20 con 0x7fda28102690 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2d447640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 0x7fda281a16b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fda20005e30 tx=0x7fda20031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda25ffb640 1 -- 172.21.15.63:0/1293796514 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20003b70 con 0x7fda28102690 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda25ffb640 1 -- 172.21.15.63:0/1293796514 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda20003d10 con 0x7fda28102690 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda25ffb640 1 -- 172.21.15.63:0/1293796514 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2002fd30 con 0x7fda28102690 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda281a1fb0 con 0x7fda28102690 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.340+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda281a2490 con 0x7fda28102690 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.341+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda281a27b0 con 0x7fda28102690 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.342+0000 7fda25ffb640 1 -- 172.21.15.63:0/1293796514 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fda20004050 con 0x7fda28102690 2024-09-10T07:30:26.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.343+0000 7fda25ffb640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda180a2c30 0x7fda180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:30:26.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.343+0000 7fda25ffb640 1 -- 172.21.15.63:0/1293796514 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7fda200ef510 con 0x7fda28102690 2024-09-10T07:30:26.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.343+0000 7fda2cc46640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda180a2c30 0x7fda180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:30:26.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.344+0000 7fda2cc46640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda180a2c30 0x7fda180a50f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005fd0 tx=0x7fda1c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:30:26.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.347+0000 7fda25ffb640 1 -- 172.21.15.63:0/1293796514 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda200bd180 con 0x7fda28102690 2024-09-10T07:30:26.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:26.535+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fda281a2980 con 0x7fda180a2c30 2024-09-10T07:30:27.817 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:27.817 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T07:30:27.817 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi046 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi046 to 5698M 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:27.818 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:27.819 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi046 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi046 to 5698M 2024-09-10T07:30:27.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:27.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:27.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T07:30:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:30:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:30:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:30:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi046 2024-09-10T07:30:27.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi046 to 5698M 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:27.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:29.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:29 smithi181 ceph-mon[35397]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:29.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:29 smithi181 ceph-mon[35397]: from='client.? 172.21.15.63:0/2747233824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]: dispatch 2024-09-10T07:30:29.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:29 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]: dispatch 2024-09-10T07:30:29.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:29 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]': finished 2024-09-10T07:30:29.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:29 smithi181 ceph-mon[35397]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T07:30:29.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:29 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:29.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:29 smithi181 ceph-mon[35397]: from='client.? 172.21.15.63:0/3792224409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:29.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:29 smithi063 ceph-mon[35020]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:29.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:29 smithi063 ceph-mon[35020]: from='client.? 172.21.15.63:0/2747233824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]: dispatch 2024-09-10T07:30:29.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:29 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]: dispatch 2024-09-10T07:30:29.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:29 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]': finished 2024-09-10T07:30:29.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:29 smithi063 ceph-mon[35020]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T07:30:29.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:29 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:29.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:29 smithi063 ceph-mon[35020]: from='client.? 172.21.15.63:0/3792224409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:29 smithi046 ceph-mon[30705]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:29 smithi046 ceph-mon[30705]: from='client.? 172.21.15.63:0/2747233824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]: dispatch 2024-09-10T07:30:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:29 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]: dispatch 2024-09-10T07:30:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:29 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32078aa6-503d-49ed-b587-2a840b513145"}]': finished 2024-09-10T07:30:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:29 smithi046 ceph-mon[30705]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T07:30:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:29 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:29 smithi046 ceph-mon[30705]: from='client.? 172.21.15.63:0/3792224409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:31.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:31 smithi181 ceph-mon[35397]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:31.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:31 smithi063 ceph-mon[35020]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:31.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:31 smithi046 ceph-mon[30705]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:33.750 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:33 smithi063 ceph-mon[35020]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:33.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:33 smithi181 ceph-mon[35397]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:33.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:33 smithi046 ceph-mon[30705]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:35 smithi046 ceph-mon[30705]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:35.799 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:35 smithi063 ceph-mon[35020]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:35.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:35 smithi181 ceph-mon[35397]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:37.760 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:37 smithi063 ceph-mon[35020]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:37.760 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:37 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T07:30:37.760 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:37 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:37.760 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:37 smithi063 ceph-mon[35020]: Deploying daemon osd.7 on smithi063 2024-09-10T07:30:37.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:37 smithi181 ceph-mon[35397]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:37.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:37 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T07:30:37.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:37 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:37.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:37 smithi181 ceph-mon[35397]: Deploying daemon osd.7 on smithi063 2024-09-10T07:30:37.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:37 smithi046 ceph-mon[30705]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:37.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:37 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T07:30:37.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:37 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:37.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:37 smithi046 ceph-mon[30705]: Deploying daemon osd.7 on smithi063 2024-09-10T07:30:39.845 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:39 smithi063 ceph-mon[35020]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:39.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:39 smithi181 ceph-mon[35397]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:39.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:39 smithi046 ceph-mon[30705]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:41.805 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:41 smithi063 ceph-mon[35020]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:41.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:41 smithi181 ceph-mon[35397]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:41 smithi046 ceph-mon[30705]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:43.761 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:43 smithi063 ceph-mon[35020]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:43.761 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:43 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:43.761 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:43 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:43.761 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:43 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:43 smithi181 ceph-mon[35397]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:43 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:43 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:43.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:43 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:43 smithi046 ceph-mon[30705]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:43.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:43 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:45.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:45 smithi181 ceph-mon[35397]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:45.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:45 smithi063 ceph-mon[35020]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:45.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:45 smithi046 ceph-mon[30705]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:46.727 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 7 on host 'smithi063' 2024-09-10T07:30:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.727+0000 7fda25ffb640 1 -- 172.21.15.63:0/1293796514 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda281a2980 con 0x7fda180a2c30 2024-09-10T07:30:46.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda180a2c30 msgr2=0x7fda180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:46.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda180a2c30 0x7fda180a50f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005fd0 tx=0x7fda1c005850 comp rx=0 tx=0).stop 2024-09-10T07:30:46.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 msgr2=0x7fda281a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 0x7fda281a16b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fda20005e30 tx=0x7fda20031ee0 comp rx=0 tx=0).stop 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 shutdown_connections 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda28102690 0x7fda281a16b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda28101cd0 0x7fda2819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda180a2c30 0x7fda180a50f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 --2- 172.21.15.63:0/1293796514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda28100ad0 0x7fda2819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 >> 172.21.15.63:0/1293796514 conn(0x7fda280fc280 msgr2=0x7fda28069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 shutdown_connections 2024-09-10T07:30:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:30:46.730+0000 7fda2eeaa640 1 -- 172.21.15.63:0/1293796514 wait complete. 2024-09-10T07:30:47.029 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.030 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.030 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:47.030 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:47.030 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.030 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:47.030 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.030 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:47.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:47.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:47.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:30:47.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:47.339 DEBUG:teuthology.orchestra.run.smithi063:osd.7> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T07:30:47.342 INFO:tasks.cephadm:Deploying osd.8 on smithi181 with /dev/vg_nvme/lv_2... 2024-09-10T07:30:47.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T07:30:47.597 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:30:47.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:47 smithi063 ceph-mon[35020]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:48.051 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:47 smithi181 ceph-mon[35397]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:48.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:47 smithi046 ceph-mon[30705]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:49.401 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:30:49 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:30:49.069+0000 7ff2a47e4640 -1 osd.7 0 log_to_monitors true 2024-09-10T07:30:49.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:49 smithi181 ceph-mon[35397]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:49.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:49 smithi181 ceph-mon[35397]: from='osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T07:30:49.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:49 smithi181 ceph-mon[35397]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T07:30:50.010 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:49 smithi063 ceph-mon[35020]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:50.011 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:49 smithi063 ceph-mon[35020]: from='osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T07:30:50.011 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:49 smithi063 ceph-mon[35020]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T07:30:50.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:49 smithi046 ceph-mon[30705]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:50.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:49 smithi046 ceph-mon[30705]: from='osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T07:30:50.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:49 smithi046 ceph-mon[30705]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T07:30:50.416 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:30:50.442 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2024-09-10T07:30:50.692 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi063 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.075 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi063 to 5696M 2024-09-10T07:30:51.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:51.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:51.076 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:50 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.150 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:30:50.764+0000 7ff2a0767640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T07:30:51.150 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:30:50.774+0000 7ff29c58f640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi063 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi063 to 5696M 2024-09-10T07:30:51.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:51.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:51.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:50 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T07:30:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T07:30:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:30:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:30:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi063 2024-09-10T07:30:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:30:51.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi063 to 5696M 2024-09-10T07:30:51.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:51.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:30:51.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:50 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.156+0000 7fe6290c5640 1 -- 172.21.15.181:0/1565790375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624102690 msgr2=0x7fe624108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.156+0000 7fe6290c5640 1 --2- 172.21.15.181:0/1565790375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624102690 0x7fe624108f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009ca0 tx=0x7fe61c02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.156+0000 7fe6290c5640 1 -- 172.21.15.181:0/1565790375 shutdown_connections 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.156+0000 7fe6290c5640 1 --2- 172.21.15.181:0/1565790375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624102690 0x7fe624108f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.156+0000 7fe6290c5640 1 --2- 172.21.15.181:0/1565790375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe624101cd0 0x7fe624102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.156+0000 7fe6290c5640 1 --2- 172.21.15.181:0/1565790375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624100ad0 0x7fe624100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.156+0000 7fe6290c5640 1 -- 172.21.15.181:0/1565790375 >> 172.21.15.181:0/1565790375 conn(0x7fe6240fc280 msgr2=0x7fe6240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.156+0000 7fe6290c5640 1 -- 172.21.15.181:0/1565790375 shutdown_connections 2024-09-10T07:30:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.157+0000 7fe6290c5640 1 -- 172.21.15.181:0/1565790375 wait complete. 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.157+0000 7fe6290c5640 1 Processor -- start 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.157+0000 7fe6290c5640 1 -- start start 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe6290c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624100ad0 0x7fe624197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe6290c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624101cd0 0x7fe624198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe6290c5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe624102690 0x7fe62419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe6290c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe62410c800 con 0x7fe624101cd0 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe6290c5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe62410c680 con 0x7fe624102690 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe6290c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe62410c980 con 0x7fe624100ad0 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624100ad0 0x7fe624197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624100ad0 0x7fe624197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53764/0 (socket says 172.21.15.181:53764) 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 -- 172.21.15.181:0/2653653456 learned_addr learned my addr 172.21.15.181:0/2653653456 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:30:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 -- 172.21.15.181:0/2653653456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe624102690 msgr2=0x7fe62419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe62359e640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe624102690 0x7fe62419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe624102690 0x7fe62419f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 -- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624101cd0 msgr2=0x7fe624198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624101cd0 0x7fe624198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 -- 172.21.15.181:0/2653653456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62419fb80 con 0x7fe624100ad0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe622d9d640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624100ad0 0x7fe624197f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b450 tx=0x7fe60c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe61bfff640 1 -- 172.21.15.181:0/2653653456 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c015020 con 0x7fe624100ad0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe61bfff640 1 -- 172.21.15.181:0/2653653456 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60c011d70 con 0x7fe624100ad0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62419fe70 con 0x7fe624100ad0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe61bfff640 1 -- 172.21.15.181:0/2653653456 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00f340 con 0x7fe624100ad0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.158+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6241a03a0 con 0x7fe624100ad0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.159+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6241a0630 con 0x7fe624100ad0 2024-09-10T07:30:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.160+0000 7fe61bfff640 1 -- 172.21.15.181:0/2653653456 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe60c018020 con 0x7fe624100ad0 2024-09-10T07:30:51.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.161+0000 7fe61bfff640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2bb0 0x7fe6100a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:30:51.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.161+0000 7fe61bfff640 1 -- 172.21.15.181:0/2653653456 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5117+0+0 (secure 0 0 0) 0x7fe60c0ce3a0 con 0x7fe624100ad0 2024-09-10T07:30:51.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.162+0000 7fe62259c640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2bb0 0x7fe6100a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:30:51.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.162+0000 7fe62259c640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2bb0 0x7fe6100a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe624199470 tx=0x7fe614005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:30:51.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.164+0000 7fe61bfff640 1 -- 172.21.15.181:0/2653653456 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe60c01f020 con 0x7fe624100ad0 2024-09-10T07:30:51.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:30:51.340+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe6241a0800 con 0x7fe6100a2bb0 2024-09-10T07:30:52.019 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:52.020 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:30:52.020 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T07:30:52.020 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:52.020 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:52.020 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:30:52.020 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:30:52.020 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:30:52.020 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:52.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:52.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:30:52.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T07:30:52.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:52.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:52.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:30:52.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:30:52.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:30:52.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:30:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:30:53.096 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:52 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:30:53.096 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:52 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:30:53.096 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:52 smithi181 ceph-mon[35397]: osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] boot 2024-09-10T07:30:53.096 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:52 smithi181 ceph-mon[35397]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T07:30:53.096 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:52 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:53.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:52 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:30:53.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:52 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:30:53.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:52 smithi063 ceph-mon[35020]: osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] boot 2024-09-10T07:30:53.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:52 smithi063 ceph-mon[35020]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T07:30:53.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:52 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:53.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:52 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:30:53.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:52 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:30:53.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:52 smithi046 ceph-mon[30705]: osd.7 [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] boot 2024-09-10T07:30:53.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:52 smithi046 ceph-mon[30705]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T07:30:53.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:52 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T07:30:54.085 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:53 smithi181 ceph-mon[35397]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:30:54.085 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:53 smithi181 ceph-mon[35397]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T07:30:54.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:53 smithi063 ceph-mon[35020]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:30:54.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:53 smithi063 ceph-mon[35020]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T07:30:54.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:53 smithi046 ceph-mon[30705]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:30:54.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:53 smithi046 ceph-mon[30705]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T07:30:55.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:54 smithi046 ceph-mon[30705]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T07:30:55.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:54 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/261991180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]: dispatch 2024-09-10T07:30:55.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:54 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]: dispatch 2024-09-10T07:30:55.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:54 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]': finished 2024-09-10T07:30:55.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:54 smithi046 ceph-mon[30705]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T07:30:55.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:54 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:30:55.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:54 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/341458747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:55.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:54 smithi181 ceph-mon[35397]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T07:30:55.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:54 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/261991180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]: dispatch 2024-09-10T07:30:55.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:54 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]: dispatch 2024-09-10T07:30:55.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:54 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]': finished 2024-09-10T07:30:55.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:54 smithi181 ceph-mon[35397]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:54 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:54 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/341458747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:54 smithi063 ceph-mon[35020]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:54 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/261991180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]: dispatch 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:54 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]: dispatch 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:54 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36b80e03-cf0c-41b5-bad6-2021b07c92ca"}]': finished 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:54 smithi063 ceph-mon[35020]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:54 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:30:55.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:54 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/341458747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:30:56.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:55 smithi181 ceph-mon[35397]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:30:56.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:55 smithi181 ceph-mon[35397]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T07:30:56.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:55 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:30:56.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:55 smithi063 ceph-mon[35020]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:30:56.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:55 smithi063 ceph-mon[35020]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T07:30:56.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:55 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:30:56.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:55 smithi046 ceph-mon[30705]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:30:56.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:55 smithi046 ceph-mon[30705]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T07:30:56.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:55 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:30:58.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:57 smithi181 ceph-mon[35397]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:30:58.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:57 smithi063 ceph-mon[35020]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:30:58.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:57 smithi046 ceph-mon[30705]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:00.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:30:59 smithi181 ceph-mon[35397]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:00.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:30:59 smithi063 ceph-mon[35020]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:00.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:30:59 smithi046 ceph-mon[30705]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:02.045 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:01 smithi181 ceph-mon[35397]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:02.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:01 smithi063 ceph-mon[35020]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:02.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:01 smithi046 ceph-mon[30705]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T07:31:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:02 smithi181 ceph-mon[35397]: Deploying daemon osd.8 on smithi181 2024-09-10T07:31:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T07:31:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:02 smithi063 ceph-mon[35020]: Deploying daemon osd.8 on smithi181 2024-09-10T07:31:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T07:31:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:02 smithi046 ceph-mon[30705]: Deploying daemon osd.8 on smithi181 2024-09-10T07:31:04.088 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:03 smithi181 ceph-mon[35397]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:04.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:03 smithi063 ceph-mon[35020]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:03 smithi046 ceph-mon[30705]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:06.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:05 smithi181 ceph-mon[35397]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:05 smithi063 ceph-mon[35020]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:06.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:05 smithi046 ceph-mon[30705]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:08.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:07 smithi181 ceph-mon[35397]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:08.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:07 smithi063 ceph-mon[35020]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:08.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:07 smithi046 ceph-mon[30705]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:09.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:09.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:09.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:09.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:09.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:09.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:09.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:09.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:09.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:10.309 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:10 smithi181 ceph-mon[35397]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:10 smithi063 ceph-mon[35020]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:10.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:10 smithi046 ceph-mon[30705]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:11.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.882+0000 7fe61bfff640 1 -- 172.21.15.181:0/2653653456 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe6241a0800 con 0x7fe6100a2bb0 2024-09-10T07:31:11.882 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 8 on host 'smithi181' 2024-09-10T07:31:11.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2bb0 msgr2=0x7fe6100a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:11.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2bb0 0x7fe6100a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe624199470 tx=0x7fe614005bf0 comp rx=0 tx=0).stop 2024-09-10T07:31:11.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624100ad0 msgr2=0x7fe624197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:11.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624100ad0 0x7fe624197f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b450 tx=0x7fe60c00b910 comp rx=0 tx=0).stop 2024-09-10T07:31:11.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 shutdown_connections 2024-09-10T07:31:11.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe624102690 0x7fe62419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:11.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2bb0 0x7fe6100a5070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:11.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624101cd0 0x7fe624198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:11.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 --2- 172.21.15.181:0/2653653456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624100ad0 0x7fe624197f50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:11.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 >> 172.21.15.181:0/2653653456 conn(0x7fe6240fc280 msgr2=0x7fe62406a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:31:11.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.884+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 shutdown_connections 2024-09-10T07:31:11.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:31:11.885+0000 7fe6290c5640 1 -- 172.21.15.181:0/2653653456 wait complete. 2024-09-10T07:31:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:12.471 DEBUG:teuthology.orchestra.run.smithi181:osd.8> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T07:31:12.473 INFO:tasks.cephadm:Deploying osd.9 on smithi046 with /dev/vg_nvme/lv_1... 2024-09-10T07:31:12.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T07:31:12.853 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:31:14.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:13 smithi181 ceph-mon[35397]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:14.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:13 smithi181 ceph-mon[35397]: from='osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T07:31:14.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:13 smithi181 ceph-mon[35397]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T07:31:14.151 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:31:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:31:13.907+0000 7f926b64c640 -1 osd.8 0 log_to_monitors true 2024-09-10T07:31:14.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:13 smithi063 ceph-mon[35020]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:14.450 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:13 smithi063 ceph-mon[35020]: from='osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T07:31:14.450 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:13 smithi063 ceph-mon[35020]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T07:31:14.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:13 smithi046 ceph-mon[30705]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:14.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:13 smithi046 ceph-mon[30705]: from='osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T07:31:14.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:13 smithi046 ceph-mon[30705]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T07:31:15.335 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:14 smithi181 ceph-mon[35397]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T07:31:15.335 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:14 smithi181 ceph-mon[35397]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T07:31:15.335 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:15.335 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:14 smithi181 ceph-mon[35397]: from='osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:31:15.335 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:14 smithi181 ceph-mon[35397]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:31:15.335 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:31:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:31:14.961+0000 7f92675cf640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T07:31:15.335 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:31:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:31:14.970+0000 7f92633f7640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:31:15.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:14 smithi063 ceph-mon[35020]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T07:31:15.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:14 smithi063 ceph-mon[35020]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T07:31:15.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:15.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:14 smithi063 ceph-mon[35020]: from='osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:31:15.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:14 smithi063 ceph-mon[35020]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:31:15.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:14 smithi046 ceph-mon[30705]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T07:31:15.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:14 smithi046 ceph-mon[30705]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T07:31:15.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:15.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:14 smithi046 ceph-mon[30705]: from='osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:31:15.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:14 smithi046 ceph-mon[30705]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:31:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] boot 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi181 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi181 to 5696M 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:16.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] boot 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi181 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi181 to 5696M 2024-09-10T07:31:16.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:16.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:16.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: osd.8 [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] boot 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T07:31:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi181 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi181 to 5696M 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:16.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:31:16.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-09-10T07:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:31:17.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:17 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:31:17.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:17 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:31:17.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:17 smithi046 ceph-mon[30705]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.571+0000 7f55cf800640 1 -- 172.21.15.46:0/2675105359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8078790 msgr2=0x7f55c8079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.571+0000 7f55cf800640 1 --2- 172.21.15.46:0/2675105359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8078790 0x7f55c8079030 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009ca0 tx=0x7f55c002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.572+0000 7f55cf800640 1 -- 172.21.15.46:0/2675105359 shutdown_connections 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.572+0000 7f55cf800640 1 --2- 172.21.15.46:0/2675105359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8078790 0x7f55c8079030 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.572+0000 7f55cf800640 1 --2- 172.21.15.46:0/2675105359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c8077df0 0x7f55c8078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.572+0000 7f55cf800640 1 --2- 172.21.15.46:0/2675105359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55c8076ba0 0x7f55c8076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.572+0000 7f55cf800640 1 -- 172.21.15.46:0/2675105359 >> 172.21.15.46:0/2675105359 conn(0x7f55c80ff5b0 msgr2=0x7f55c81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.572+0000 7f55cf800640 1 -- 172.21.15.46:0/2675105359 shutdown_connections 2024-09-10T07:31:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.572+0000 7f55cf800640 1 -- 172.21.15.46:0/2675105359 wait complete. 2024-09-10T07:31:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.572+0000 7f55cf800640 1 Processor -- start 2024-09-10T07:31:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cf800640 1 -- start start 2024-09-10T07:31:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cf800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c8076ba0 0x7f55c819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:31:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cf800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8077df0 0x7f55c819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:31:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cf800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55c8078790 0x7f55c81a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:31:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cf800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f55c8113ea0 con 0x7f55c8077df0 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cf800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55c8113d20 con 0x7f55c8078790 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cf800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55c8114020 con 0x7f55c8076ba0 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8077df0 0x7f55c819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c8076ba0 0x7f55c819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c8076ba0 0x7f55c819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37628/0 (socket says 172.21.15.46:37628) 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cce00640 1 -- 172.21.15.46:0/390078348 learned_addr learned my addr 172.21.15.46:0/390078348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8077df0 0x7f55c819eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39476/0 (socket says 172.21.15.46:39476) 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.573+0000 7f55cd800640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55c8078790 0x7f55c81a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55cce00640 1 -- 172.21.15.46:0/390078348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55c8078790 msgr2=0x7f55c81a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55cce00640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55c8078790 0x7f55c81a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55cce00640 1 -- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8077df0 msgr2=0x7f55c819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55cce00640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8077df0 0x7f55c819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55cce00640 1 -- 172.21.15.46:0/390078348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c81a6160 con 0x7f55c8076ba0 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55c7e00640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8077df0 0x7f55c819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55cce00640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c8076ba0 0x7f55c819e5a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00d950 tx=0x7f55bc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55c5600640 1 -- 172.21.15.46:0/390078348 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc017030 con 0x7f55c8076ba0 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55c5600640 1 -- 172.21.15.46:0/390078348 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc004020 con 0x7f55c8076ba0 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c81a6450 con 0x7f55c8076ba0 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.574+0000 7f55c5600640 1 -- 172.21.15.46:0/390078348 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0047a0 con 0x7f55c8076ba0 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.575+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c81a6990 con 0x7f55c8076ba0 2024-09-10T07:31:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.576+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c8110780 con 0x7f55c8076ba0 2024-09-10T07:31:17.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.578+0000 7f55c5600640 1 -- 172.21.15.46:0/390078348 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f55bc0041d0 con 0x7f55c8076ba0 2024-09-10T07:31:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.579+0000 7f55c5600640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55b00a2bb0 0x7f55b00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:31:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.579+0000 7f55c5600640 1 -- 172.21.15.46:0/390078348 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5858+0+0 (secure 0 0 0) 0x7f55bc0ce120 con 0x7f55c8076ba0 2024-09-10T07:31:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.579+0000 7f55c7e00640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55b00a2bb0 0x7f55b00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:31:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.580+0000 7f55c7e00640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55b00a2bb0 0x7f55b00a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55c819fa50 tx=0x7f55b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:31:17.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.583+0000 7f55c5600640 1 -- 172.21.15.46:0/390078348 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55bc09b970 con 0x7f55c8076ba0 2024-09-10T07:31:17.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:17 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:31:17.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:17 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:31:17.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:17 smithi181 ceph-mon[35397]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T07:31:17.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:17 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:31:17.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:17 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:31:17.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:17 smithi063 ceph-mon[35020]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T07:31:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:17.777+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f55c8062630 con 0x7f55b00a2bb0 2024-09-10T07:31:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:18 smithi181 ceph-mon[35397]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:18.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:18 smithi181 ceph-mon[35397]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T07:31:18.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:31:18.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:31:18.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:18 smithi063 ceph-mon[35020]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:18 smithi063 ceph-mon[35020]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T07:31:18.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:31:18.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:31:18.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:18 smithi046 ceph-mon[30705]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:18 smithi046 ceph-mon[30705]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T07:31:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:31:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:31:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:19.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:19 smithi181 ceph-mon[35397]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:31:19.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:19 smithi181 ceph-mon[35397]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T07:31:19.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:19 smithi063 ceph-mon[35020]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:31:19.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:19 smithi063 ceph-mon[35020]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T07:31:19.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:19 smithi046 ceph-mon[30705]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:31:19.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:19 smithi046 ceph-mon[30705]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T07:31:20.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:20 smithi181 ceph-mon[35397]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:20.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:20 smithi181 ceph-mon[35397]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T07:31:20.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:20 smithi063 ceph-mon[35020]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:20.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:20 smithi063 ceph-mon[35020]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T07:31:20.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:20 smithi046 ceph-mon[30705]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:20.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:20 smithi046 ceph-mon[30705]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T07:31:21.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:21 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3821951598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f9edf7d-f94e-4913-9928-5b6c4d3c724e"}]: dispatch 2024-09-10T07:31:21.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:21 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3821951598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f9edf7d-f94e-4913-9928-5b6c4d3c724e"}]': finished 2024-09-10T07:31:21.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:21 smithi181 ceph-mon[35397]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T07:31:21.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:21.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:21 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3821951598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f9edf7d-f94e-4913-9928-5b6c4d3c724e"}]: dispatch 2024-09-10T07:31:21.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:21 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3821951598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f9edf7d-f94e-4913-9928-5b6c4d3c724e"}]': finished 2024-09-10T07:31:21.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:21 smithi063 ceph-mon[35020]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T07:31:21.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:21.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:21 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3821951598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f9edf7d-f94e-4913-9928-5b6c4d3c724e"}]: dispatch 2024-09-10T07:31:21.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:21 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3821951598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f9edf7d-f94e-4913-9928-5b6c4d3c724e"}]': finished 2024-09-10T07:31:21.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:21 smithi046 ceph-mon[30705]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T07:31:21.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:22.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:22 smithi181 ceph-mon[35397]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:22.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:22 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3649559455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:31:22.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:22 smithi063 ceph-mon[35020]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:22.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:22 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3649559455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:31:22.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:22 smithi046 ceph-mon[30705]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:22.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:22 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3649559455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:31:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:24 smithi181 ceph-mon[35397]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-10T07:31:24.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:24 smithi063 ceph-mon[35020]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-10T07:31:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:24 smithi046 ceph-mon[30705]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-10T07:31:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:26 smithi181 ceph-mon[35397]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T07:31:26.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:26 smithi063 ceph-mon[35020]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T07:31:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:26 smithi046 ceph-mon[30705]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T07:31:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:28 smithi181 ceph-mon[35397]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:31:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:28 smithi063 ceph-mon[35020]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:31:28.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:28 smithi046 ceph-mon[30705]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:31:30.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:30 smithi181 ceph-mon[35397]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T07:31:30.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:30 smithi063 ceph-mon[35020]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T07:31:30.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:30 smithi046 ceph-mon[30705]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T07:31:31.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:31 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T07:31:31.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:31 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:31.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:31 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T07:31:31.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:31 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:31.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:31 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T07:31:31.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:31 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:32.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:32 smithi181 ceph-mon[35397]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T07:31:32.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:32 smithi181 ceph-mon[35397]: Deploying daemon osd.9 on smithi046 2024-09-10T07:31:32.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:32 smithi063 ceph-mon[35020]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T07:31:32.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:32 smithi063 ceph-mon[35020]: Deploying daemon osd.9 on smithi046 2024-09-10T07:31:32.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:32 smithi046 ceph-mon[30705]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T07:31:32.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:32 smithi046 ceph-mon[30705]: Deploying daemon osd.9 on smithi046 2024-09-10T07:31:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:34 smithi181 ceph-mon[35397]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T07:31:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:34 smithi063 ceph-mon[35020]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T07:31:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:34 smithi046 ceph-mon[30705]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T07:31:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:36 smithi181 ceph-mon[35397]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:36 smithi063 ceph-mon[35020]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:36.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:36 smithi046 ceph-mon[30705]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:38 smithi181 ceph-mon[35397]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:38 smithi063 ceph-mon[35020]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:38.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:38 smithi046 ceph-mon[30705]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:39.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:39.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:39.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:39.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:39.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:39.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:40.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:40 smithi046 ceph-mon[30705]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:40.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:40 smithi181 ceph-mon[35397]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:40.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:40 smithi063 ceph-mon[35020]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:42 smithi181 ceph-mon[35397]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:42.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:42 smithi063 ceph-mon[35020]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:42.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:42.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:42 smithi046 ceph-mon[30705]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:42.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:42.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:42.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:42.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:42.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.487+0000 7f55c5600640 1 -- 172.21.15.46:0/390078348 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55c8062630 con 0x7f55b00a2bb0 2024-09-10T07:31:43.487 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 9 on host 'smithi046' 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55b00a2bb0 msgr2=0x7f55b00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55b00a2bb0 0x7f55b00a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55c819fa50 tx=0x7f55b4005f90 comp rx=0 tx=0).stop 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c8076ba0 msgr2=0x7f55c819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c8076ba0 0x7f55c819e5a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00d950 tx=0x7f55bc00de10 comp rx=0 tx=0).stop 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 shutdown_connections 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55c8078790 0x7f55c81a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55b00a2bb0 0x7f55b00a5070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c8077df0 0x7f55c819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 --2- 172.21.15.46:0/390078348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c8076ba0 0x7f55c819e5a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 >> 172.21.15.46:0/390078348 conn(0x7f55c80ff5b0 msgr2=0x7f55c8101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:31:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 shutdown_connections 2024-09-10T07:31:43.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:31:43.490+0000 7f55cf800640 1 -- 172.21.15.46:0/390078348 wait complete. 2024-09-10T07:31:44.312 DEBUG:teuthology.orchestra.run.smithi046:osd.9> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T07:31:44.317 INFO:tasks.cephadm:Deploying osd.10 on smithi063 with /dev/vg_nvme/lv_1... 2024-09-10T07:31:44.317 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T07:31:44.579 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:31:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:44 smithi063 ceph-mon[35020]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:44.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:44.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:44.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:44 smithi046 ceph-mon[30705]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:44.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:44.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:44.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:44.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:44 smithi181 ceph-mon[35397]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:31:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:44.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:45.325 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:31:44 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:31:44.952+0000 7ff0a7c35640 -1 osd.9 0 log_to_monitors true 2024-09-10T07:31:45.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:45 smithi046 ceph-mon[30705]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T07:31:45.739 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:45 smithi063 ceph-mon[35020]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T07:31:45.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:45 smithi181 ceph-mon[35397]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T07:31:46.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:46 smithi046 ceph-mon[30705]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:46.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:46 smithi046 ceph-mon[30705]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T07:31:46.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:46 smithi046 ceph-mon[30705]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T07:31:46.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:46 smithi046 ceph-mon[30705]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:31:46.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:46 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:46.686 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:46 smithi063 ceph-mon[35020]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:46.687 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:46 smithi063 ceph-mon[35020]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T07:31:46.687 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:46 smithi063 ceph-mon[35020]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T07:31:46.687 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:46 smithi063 ceph-mon[35020]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:31:46.687 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:46 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:46 smithi181 ceph-mon[35397]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:46 smithi181 ceph-mon[35397]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T07:31:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:46 smithi181 ceph-mon[35397]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T07:31:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:46 smithi181 ceph-mon[35397]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T07:31:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:46 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:47.187 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-10T07:31:47.212 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_1 2024-09-10T07:31:47.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:31:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:31:47.083+0000 7ff0a2400640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T07:31:47.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:31:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:31:47.119+0000 7ff098400640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:31:47.463 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:31:47.583 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:47 smithi063 ceph-mon[35020]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:31:47.584 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:47 smithi063 ceph-mon[35020]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T07:31:47.584 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:47 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:47.584 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:47 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:47.584 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:47 smithi063 ceph-mon[35020]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' 2024-09-10T07:31:47.766 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:47 smithi046 ceph-mon[30705]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:31:47.766 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:47 smithi046 ceph-mon[30705]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T07:31:47.766 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:47 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:47.766 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:47 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:47.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:47 smithi046 ceph-mon[30705]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' 2024-09-10T07:31:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:47 smithi181 ceph-mon[35397]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T07:31:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:47 smithi181 ceph-mon[35397]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T07:31:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:47 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:47 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:47.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:47 smithi181 ceph-mon[35397]: from='osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364]' entity='osd.9' 2024-09-10T07:31:47.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 -- 172.21.15.63:0/1511065440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd734107900 msgr2=0x7fd734109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:47.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 --2- 172.21.15.63:0/1511065440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd734107900 0x7fd734109d50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009ca0 tx=0x7fd72c02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:31:47.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 -- 172.21.15.63:0/1511065440 shutdown_connections 2024-09-10T07:31:47.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 --2- 172.21.15.63:0/1511065440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd734107900 0x7fd734109d50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 --2- 172.21.15.63:0/1511065440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd734104fd0 0x7fd7341073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 --2- 172.21.15.63:0/1511065440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7341026a0 0x7fd734104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 -- 172.21.15.63:0/1511065440 >> 172.21.15.63:0/1511065440 conn(0x7fd7340fc2a0 msgr2=0x7fd7340fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 -- 172.21.15.63:0/1511065440 shutdown_connections 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.922+0000 7fd739718640 1 -- 172.21.15.63:0/1511065440 wait complete. 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd739718640 1 Processor -- start 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd739718640 1 -- start start 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd739718640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7341026a0 0x7fd734198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd739718640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd734104fd0 0x7fd734198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd739718640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd734107900 0x7fd73419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd739718640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd73410d990 con 0x7fd734104fd0 2024-09-10T07:31:47.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd739718640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd73410d810 con 0x7fd7341026a0 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd739718640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd73410db10 con 0x7fd734107900 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7341026a0 0x7fd734198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7341026a0 0x7fd734198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53258/0 (socket says 172.21.15.63:53258) 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd732ffd640 1 -- 172.21.15.63:0/2920024567 learned_addr learned my addr 172.21.15.63:0/2920024567 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.923+0000 7fd7337fe640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd734107900 0x7fd73419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd732ffd640 1 -- 172.21.15.63:0/2920024567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd734107900 msgr2=0x7fd73419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd732ffd640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd734107900 0x7fd73419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd732ffd640 1 -- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd734104fd0 msgr2=0x7fd734198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd732ffd640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd734104fd0 0x7fd734198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:31:47.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd732ffd640 1 -- 172.21.15.63:0/2920024567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73419fc70 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd732ffd640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7341026a0 0x7fd734198040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd72400b4a0 tx=0x7fd72400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd71bfff640 1 -- 172.21.15.63:0/2920024567 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724014070 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd71bfff640 1 -- 172.21.15.63:0/2920024567 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd724010af0 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73419ff60 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd71bfff640 1 -- 172.21.15.63:0/2920024567 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724017d10 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.924+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7341a0490 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.925+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7340fdd90 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.927+0000 7fd71bfff640 1 -- 172.21.15.63:0/2920024567 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd724017070 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.928+0000 7fd71bfff640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd71c0a2bb0 0x7fd71c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.928+0000 7fd71bfff640 1 -- 172.21.15.63:0/2920024567 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5981+0+0 (secure 0 0 0) 0x7fd7240cddf0 con 0x7fd7341026a0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.928+0000 7fd7327fc640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd71c0a2bb0 0x7fd71c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:31:47.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.928+0000 7fd7327fc640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd71c0a2bb0 0x7fd71c0a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd734199560 tx=0x7fd728006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:31:47.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:47.931+0000 7fd71bfff640 1 -- 172.21.15.63:0/2920024567 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd72409b670 con 0x7fd7341026a0 2024-09-10T07:31:48.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:31:48.116+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd734062630 con 0x7fd71c0a2bb0 2024-09-10T07:31:48.766 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:31:48.766 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:31:48.766 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:48.766 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:48.766 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] boot 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.767 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:48.768 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:48.768 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:48 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] boot 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:48.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:48.901 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:48 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:31:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:31:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-10T07:31:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: osd.9 [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] boot 2024-09-10T07:31:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T07:31:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T07:31:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:31:48.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:48 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:31:49.481 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:49 smithi063 ceph-mon[35020]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:31:49.481 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:49 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi046 2024-09-10T07:31:49.481 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:49 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi046 to 4273M 2024-09-10T07:31:49.481 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:49 smithi063 ceph-mon[35020]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:49.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:49 smithi181 ceph-mon[35397]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:31:49.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:49 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi046 2024-09-10T07:31:49.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:49 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi046 to 4273M 2024-09-10T07:31:49.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:49 smithi181 ceph-mon[35397]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:49.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:49 smithi046 ceph-mon[30705]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:31:49.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:49 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi046 2024-09-10T07:31:49.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:49 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi046 to 4273M 2024-09-10T07:31:49.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:49 smithi046 ceph-mon[30705]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:50.719 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:50 smithi063 ceph-mon[35020]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T07:31:50.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:50 smithi181 ceph-mon[35397]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T07:31:50.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:50 smithi046 ceph-mon[30705]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T07:31:51.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:51 smithi181 ceph-mon[35397]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:51.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:51 smithi181 ceph-mon[35397]: from='client.? 172.21.15.63:0/1528873360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]: dispatch 2024-09-10T07:31:51.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:51 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]: dispatch 2024-09-10T07:31:51.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:51 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]': finished 2024-09-10T07:31:51.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:51 smithi181 ceph-mon[35397]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T07:31:51.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:51 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:31:51.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:51 smithi181 ceph-mon[35397]: from='client.? 172.21.15.63:0/624058814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:31:51.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:51 smithi063 ceph-mon[35020]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:51.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:51 smithi063 ceph-mon[35020]: from='client.? 172.21.15.63:0/1528873360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]: dispatch 2024-09-10T07:31:51.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:51 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]: dispatch 2024-09-10T07:31:51.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:51 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]': finished 2024-09-10T07:31:51.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:51 smithi063 ceph-mon[35020]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T07:31:51.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:51 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:31:51.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:51 smithi063 ceph-mon[35020]: from='client.? 172.21.15.63:0/624058814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:31:51.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:51 smithi046 ceph-mon[30705]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:51.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:51 smithi046 ceph-mon[30705]: from='client.? 172.21.15.63:0/1528873360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]: dispatch 2024-09-10T07:31:51.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:51 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]: dispatch 2024-09-10T07:31:51.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:51 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc6d6ef6-c067-495e-a673-88cbab957e7e"}]': finished 2024-09-10T07:31:51.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:51 smithi046 ceph-mon[30705]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T07:31:51.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:51 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:31:51.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:51 smithi046 ceph-mon[30705]: from='client.? 172.21.15.63:0/624058814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:31:53.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:53 smithi181 ceph-mon[35397]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:53.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:53 smithi063 ceph-mon[35020]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:53.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:53 smithi046 ceph-mon[30705]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:55.637 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:55 smithi063 ceph-mon[35020]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:55.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:55 smithi181 ceph-mon[35397]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:55.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:55 smithi046 ceph-mon[30705]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:57.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:57 smithi063 ceph-mon[35020]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:57.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:57 smithi046 ceph-mon[30705]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:57.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:57 smithi181 ceph-mon[35397]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:59.846 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:59 smithi063 ceph-mon[35020]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:59.846 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:59 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T07:31:59.846 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:31:59 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:59.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:59 smithi181 ceph-mon[35397]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:59.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:59 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T07:31:59.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:31:59 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:31:59.912 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:59 smithi046 ceph-mon[30705]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:31:59.913 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:59 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T07:31:59.913 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:31:59 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:00.801 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:00 smithi063 ceph-mon[35020]: Deploying daemon osd.10 on smithi063 2024-09-10T07:32:00.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:00 smithi181 ceph-mon[35397]: Deploying daemon osd.10 on smithi063 2024-09-10T07:32:00.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:00 smithi046 ceph-mon[30705]: Deploying daemon osd.10 on smithi063 2024-09-10T07:32:01.759 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:01 smithi063 ceph-mon[35020]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:01.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:01 smithi181 ceph-mon[35397]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:01.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:01 smithi046 ceph-mon[30705]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:03.870 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:03 smithi063 ceph-mon[35020]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:03.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:03 smithi181 ceph-mon[35397]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:03.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:03 smithi046 ceph-mon[30705]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:05.739 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:05 smithi063 ceph-mon[35020]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:05.740 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:05.740 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:05.740 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:05.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:05 smithi181 ceph-mon[35397]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:05.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:05.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:05.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:05.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:05 smithi046 ceph-mon[30705]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:05.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:05.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:05.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:07.636 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:07 smithi063 ceph-mon[35020]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:07.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:07 smithi181 ceph-mon[35397]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:07.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:07 smithi046 ceph-mon[30705]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:08.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.525+0000 7fd71bfff640 1 -- 172.21.15.63:0/2920024567 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd734062630 con 0x7fd71c0a2bb0 2024-09-10T07:32:08.525 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 10 on host 'smithi063' 2024-09-10T07:32:08.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd71c0a2bb0 msgr2=0x7fd71c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd71c0a2bb0 0x7fd71c0a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd734199560 tx=0x7fd728006c90 comp rx=0 tx=0).stop 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7341026a0 msgr2=0x7fd734198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7341026a0 0x7fd734198040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd72400b4a0 tx=0x7fd72400b960 comp rx=0 tx=0).stop 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 shutdown_connections 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd734107900 0x7fd73419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd71c0a2bb0 0x7fd71c0a5070 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd734104fd0 0x7fd734198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 --2- 172.21.15.63:0/2920024567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7341026a0 0x7fd734198040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 >> 172.21.15.63:0/2920024567 conn(0x7fd7340fc2a0 msgr2=0x7fd734102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 shutdown_connections 2024-09-10T07:32:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-10T07:32:08.527+0000 7fd739718640 1 -- 172.21.15.63:0/2920024567 wait complete. 2024-09-10T07:32:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:08.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:09.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:09.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:09.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:09.213 DEBUG:teuthology.orchestra.run.smithi063:osd.10> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T07:32:09.215 INFO:tasks.cephadm:Deploying osd.11 on smithi181 with /dev/vg_nvme/lv_1... 2024-09-10T07:32:09.216 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T07:32:09.469 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:32:10.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:09 smithi181 ceph-mon[35397]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:10.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:09 smithi063 ceph-mon[35020]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:10.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:09 smithi046 ceph-mon[30705]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:11.059 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:10 smithi181 ceph-mon[35397]: from='osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T07:32:11.059 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:10 smithi181 ceph-mon[35397]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T07:32:11.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:10 smithi063 ceph-mon[35020]: from='osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T07:32:11.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:10 smithi063 ceph-mon[35020]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T07:32:11.151 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:32:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:32:10.849+0000 7f2c358e8640 -1 osd.10 0 log_to_monitors true 2024-09-10T07:32:11.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:10 smithi046 ceph-mon[30705]: from='osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T07:32:11.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:10 smithi046 ceph-mon[30705]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T07:32:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:11 smithi181 ceph-mon[35397]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:11 smithi181 ceph-mon[35397]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T07:32:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:11 smithi181 ceph-mon[35397]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T07:32:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:11 smithi181 ceph-mon[35397]: from='osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:32:12.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:11 smithi181 ceph-mon[35397]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:32:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:11 smithi063 ceph-mon[35020]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:11 smithi063 ceph-mon[35020]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T07:32:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:11 smithi063 ceph-mon[35020]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T07:32:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:12.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:11 smithi063 ceph-mon[35020]: from='osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:32:12.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:11 smithi063 ceph-mon[35020]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:32:12.151 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:32:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:32:11.932+0000 7f2c3186b640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T07:32:12.151 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:32:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:32:11.948+0000 7f2c2d693640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:32:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:11 smithi046 ceph-mon[30705]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:11 smithi046 ceph-mon[30705]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T07:32:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:11 smithi046 ceph-mon[30705]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T07:32:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:11 smithi046 ceph-mon[30705]: from='osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:32:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:11 smithi046 ceph-mon[30705]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-09-10T07:32:12.200 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T07:32:12.226 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2024-09-10T07:32:12.482 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:32:12.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.953+0000 7ff75b4cc640 1 -- 172.21.15.181:0/3610620053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754100ad0 msgr2=0x7ff754100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:12.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.953+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/3610620053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754100ad0 0x7ff754100ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff744009900 tx=0x7ff74402a410 comp rx=0 tx=0).stop 2024-09-10T07:32:12.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.953+0000 7ff75b4cc640 1 -- 172.21.15.181:0/3610620053 shutdown_connections 2024-09-10T07:32:12.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.953+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/3610620053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff754102690 0x7ff754108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:12.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.953+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/3610620053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff754101cd0 0x7ff754102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.953+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/3610620053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754100ad0 0x7ff754100ed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.953+0000 7ff75b4cc640 1 -- 172.21.15.181:0/3610620053 >> 172.21.15.181:0/3610620053 conn(0x7ff7540fc280 msgr2=0x7ff7540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 -- 172.21.15.181:0/3610620053 shutdown_connections 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 -- 172.21.15.181:0/3610620053 wait complete. 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 Processor -- start 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 -- start start 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff754100ad0 0x7ff75406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754101cd0 0x7ff75419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff754102690 0x7ff75419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff75410c830 con 0x7ff754102690 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff75410c6b0 con 0x7ff754100ad0 2024-09-10T07:32:12.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff75b4cc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff75410c9b0 con 0x7ff754101cd0 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff758a67640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754101cd0 0x7ff75419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff758a67640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754101cd0 0x7ff75419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52026/0 (socket says 172.21.15.181:52026) 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff758a67640 1 -- 172.21.15.181:0/1826909576 learned_addr learned my addr 172.21.15.181:0/1826909576 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.954+0000 7ff759268640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff754100ad0 0x7ff75406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff759a69640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff754102690 0x7ff75419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff758a67640 1 -- 172.21.15.181:0/1826909576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff754100ad0 msgr2=0x7ff75406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff758a67640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff754100ad0 0x7ff75406ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff758a67640 1 -- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff754102690 msgr2=0x7ff75419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff758a67640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff754102690 0x7ff75419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff758a67640 1 -- 172.21.15.181:0/1826909576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75419fc70 con 0x7ff754101cd0 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff758a67640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754101cd0 0x7ff75419cd60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff75000b4a0 tx=0x7ff75000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:12.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff74a7fc640 1 -- 172.21.15.181:0/1826909576 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750014070 con 0x7ff754101cd0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff74a7fc640 1 -- 172.21.15.181:0/1826909576 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff750010af0 con 0x7ff754101cd0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75419ff60 con 0x7ff754101cd0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff74a7fc640 1 -- 172.21.15.181:0/1826909576 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750017d10 con 0x7ff754101cd0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.955+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7541a0440 con 0x7ff754101cd0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.956+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff754107f10 con 0x7ff754101cd0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.957+0000 7ff74a7fc640 1 -- 172.21.15.181:0/1826909576 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff750017070 con 0x7ff754101cd0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.958+0000 7ff74a7fc640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7400a2b80 0x7ff7400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.958+0000 7ff74a7fc640 1 -- 172.21.15.181:0/1826909576 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6727+0+0 (secure 0 0 0) 0x7ff7500cdde0 con 0x7ff754101cd0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.958+0000 7ff759268640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7400a2b80 0x7ff7400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.959+0000 7ff759268640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7400a2b80 0x7ff7400a5040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff744005ab0 tx=0x7ff744000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:12.962+0000 7ff74a7fc640 1 -- 172.21.15.181:0/1826909576 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff75009b370 con 0x7ff754101cd0 2024-09-10T07:32:13.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:13.137+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff754062630 con 0x7ff7400a2b80 2024-09-10T07:32:13.137 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:32:13.137 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T07:32:13.137 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi063 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi063 to 4272M 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:13.138 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:13.139 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:32:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T07:32:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi063 2024-09-10T07:32:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi063 to 4272M 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:13.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi063 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:13.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi063 to 4272M 2024-09-10T07:32:13.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:13.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:13.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:13.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:32:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:32:14.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:14.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:14.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] boot 2024-09-10T07:32:14.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T07:32:14.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:14.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:32:14.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:32:14.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:32:14.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:14.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:32:14.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:32:14.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:14.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:14.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] boot 2024-09-10T07:32:14.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T07:32:14.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:14.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:32:14.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:32:14.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:32:14.152 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: osd.10 [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] boot 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:32:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:15.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:14 smithi181 ceph-mon[35397]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T07:32:15.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:14 smithi063 ceph-mon[35020]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T07:32:15.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:14 smithi046 ceph-mon[30705]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T07:32:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:15 smithi181 ceph-mon[35397]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:15 smithi181 ceph-mon[35397]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T07:32:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:15 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/708085369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]: dispatch 2024-09-10T07:32:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:15 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]: dispatch 2024-09-10T07:32:16.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:15 smithi181 ceph-mon[35397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]': finished 2024-09-10T07:32:16.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:15 smithi181 ceph-mon[35397]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T07:32:16.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:15 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:15 smithi063 ceph-mon[35020]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:15 smithi063 ceph-mon[35020]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T07:32:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:15 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/708085369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]: dispatch 2024-09-10T07:32:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:15 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]: dispatch 2024-09-10T07:32:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:15 smithi063 ceph-mon[35020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]': finished 2024-09-10T07:32:16.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:15 smithi063 ceph-mon[35020]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T07:32:16.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:15 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:15 smithi046 ceph-mon[30705]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:15 smithi046 ceph-mon[30705]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T07:32:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:15 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/708085369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]: dispatch 2024-09-10T07:32:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:15 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]: dispatch 2024-09-10T07:32:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:15 smithi046 ceph-mon[30705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a4826cd-7a7a-4a20-9af7-734e74374081"}]': finished 2024-09-10T07:32:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:15 smithi046 ceph-mon[30705]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T07:32:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:15 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:17.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:16 smithi181 ceph-mon[35397]: from='client.? 172.21.15.181:0/2958526193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:32:17.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:16 smithi181 ceph-mon[35397]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T07:32:17.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:17.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:16 smithi063 ceph-mon[35020]: from='client.? 172.21.15.181:0/2958526193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:32:17.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:16 smithi063 ceph-mon[35020]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T07:32:17.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:17.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:16 smithi046 ceph-mon[30705]: from='client.? 172.21.15.181:0/2958526193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:32:17.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:16 smithi046 ceph-mon[30705]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T07:32:17.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:18.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:17 smithi046 ceph-mon[30705]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:18.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:17 smithi046 ceph-mon[30705]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T07:32:18.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:17 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:17 smithi181 ceph-mon[35397]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:18.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:17 smithi181 ceph-mon[35397]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T07:32:18.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:17 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:17 smithi063 ceph-mon[35020]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:17 smithi063 ceph-mon[35020]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T07:32:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:17 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:20.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:19 smithi046 ceph-mon[30705]: pgmap v248: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T07:32:20.353 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:19 smithi181 ceph-mon[35397]: pgmap v248: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T07:32:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:19 smithi063 ceph-mon[35020]: pgmap v248: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T07:32:22.056 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:21 smithi181 ceph-mon[35397]: pgmap v249: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T07:32:22.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:21 smithi046 ceph-mon[30705]: pgmap v249: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T07:32:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:21 smithi063 ceph-mon[35020]: pgmap v249: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T07:32:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:23 smithi181 ceph-mon[35397]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T07:32:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:23 smithi063 ceph-mon[35020]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T07:32:24.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:23 smithi046 ceph-mon[30705]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T07:32:25.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T07:32:25.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:25.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:24 smithi181 ceph-mon[35397]: Deploying daemon osd.11 on smithi181 2024-09-10T07:32:25.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T07:32:25.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:25.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:24 smithi063 ceph-mon[35020]: Deploying daemon osd.11 on smithi181 2024-09-10T07:32:25.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T07:32:25.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:25.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:24 smithi046 ceph-mon[30705]: Deploying daemon osd.11 on smithi181 2024-09-10T07:32:26.267 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:25 smithi181 ceph-mon[35397]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:32:26.301 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:25 smithi063 ceph-mon[35020]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:32:26.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:25 smithi046 ceph-mon[30705]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T07:32:28.211 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:27 smithi046 ceph-mon[30705]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:28.231 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:27 smithi181 ceph-mon[35397]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:28.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:27 smithi063 ceph-mon[35020]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:29 smithi181 ceph-mon[35397]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:29 smithi063 ceph-mon[35020]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:30.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:29 smithi046 ceph-mon[30705]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:31.226 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:30 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:31.226 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:30 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:31.226 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:30 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:31.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:30 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:31.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:30 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:31.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:30 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:31.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:30 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:31.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:30 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:31.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:30 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:32.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:31 smithi181 ceph-mon[35397]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:31 smithi063 ceph-mon[35020]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:32.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:31 smithi046 ceph-mon[30705]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:34.077 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.075+0000 7ff74a7fc640 1 -- 172.21.15.181:0/1826909576 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff754062630 con 0x7ff7400a2b80 2024-09-10T07:32:34.077 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 11 on host 'smithi181' 2024-09-10T07:32:34.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7400a2b80 msgr2=0x7ff7400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:34.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7400a2b80 0x7ff7400a5040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff744005ab0 tx=0x7ff744000bc0 comp rx=0 tx=0).stop 2024-09-10T07:32:34.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754101cd0 msgr2=0x7ff75419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:34.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754101cd0 0x7ff75419cd60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff75000b4a0 tx=0x7ff75000b960 comp rx=0 tx=0).stop 2024-09-10T07:32:34.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 shutdown_connections 2024-09-10T07:32:34.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7400a2b80 0x7ff7400a5040 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:34.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff754102690 0x7ff75419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:34.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff754101cd0 0x7ff75419cd60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:34.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 --2- 172.21.15.181:0/1826909576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff754100ad0 0x7ff75406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:34.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 >> 172.21.15.181:0/1826909576 conn(0x7ff7540fc280 msgr2=0x7ff7541050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:34.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 shutdown_connections 2024-09-10T07:32:34.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:34.078+0000 7ff75b4cc640 1 -- 172.21.15.181:0/1826909576 wait complete. 2024-09-10T07:32:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:34 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:34 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:34 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:34.793 DEBUG:teuthology.orchestra.run.smithi181:osd.11> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T07:32:34.795 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T07:32:34.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T07:32:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.775+0000 7fb769600640 1 -- 172.21.15.46:0/306451171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb764102c40 msgr2=0x7fb7641030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.775+0000 7fb769600640 1 --2- 172.21.15.46:0/306451171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb764102c40 0x7fb7641030c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009900 tx=0x7fb74c02a410 comp rx=0 tx=0).stop 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.776+0000 7fb769600640 1 -- 172.21.15.46:0/306451171 shutdown_connections 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.776+0000 7fb769600640 1 --2- 172.21.15.46:0/306451171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764103600 0x7fb764109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.776+0000 7fb769600640 1 --2- 172.21.15.46:0/306451171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb764102c40 0x7fb7641030c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.776+0000 7fb769600640 1 --2- 172.21.15.46:0/306451171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb764101a40 0x7fb764101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.776+0000 7fb769600640 1 -- 172.21.15.46:0/306451171 >> 172.21.15.46:0/306451171 conn(0x7fb7640fd1f0 msgr2=0x7fb7640ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.776+0000 7fb769600640 1 -- 172.21.15.46:0/306451171 shutdown_connections 2024-09-10T07:32:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.776+0000 7fb769600640 1 -- 172.21.15.46:0/306451171 wait complete. 2024-09-10T07:32:35.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb769600640 1 Processor -- start 2024-09-10T07:32:35.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb769600640 1 -- start start 2024-09-10T07:32:35.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb769600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb764101a40 0x7fb764075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:35.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb769600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb764102c40 0x7fb7640745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:35.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb769600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764103600 0x7fb764074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb769600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb76410d710 con 0x7fb764102c40 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb769600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb76410d590 con 0x7fb764101a40 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb769600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb76410d890 con 0x7fb764103600 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb75be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb764102c40 0x7fb7640745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb763e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb764101a40 0x7fb764075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb763e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb764101a40 0x7fb764075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40362/0 (socket says 172.21.15.46:40362) 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.777+0000 7fb763e00640 1 -- 172.21.15.46:0/1824115954 learned_addr learned my addr 172.21.15.46:0/1824115954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb768c00640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764103600 0x7fb764074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb763e00640 1 -- 172.21.15.46:0/1824115954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764103600 msgr2=0x7fb764074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb763e00640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764103600 0x7fb764074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb763e00640 1 -- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb764102c40 msgr2=0x7fb7640745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb763e00640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb764102c40 0x7fb7640745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb763e00640 1 -- 172.21.15.46:0/1824115954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7640753e0 con 0x7fb764101a40 2024-09-10T07:32:35.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb763e00640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb764101a40 0x7fb764075f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb75400e9c0 tx=0x7fb75400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:35.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb761600640 1 -- 172.21.15.46:0/1824115954 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75400cd80 con 0x7fb764101a40 2024-09-10T07:32:35.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb761600640 1 -- 172.21.15.46:0/1824115954 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7540035f0 con 0x7fb764101a40 2024-09-10T07:32:35.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.778+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7641a1e70 con 0x7fb764101a40 2024-09-10T07:32:35.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.779+0000 7fb761600640 1 -- 172.21.15.46:0/1824115954 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754010690 con 0x7fb764101a40 2024-09-10T07:32:35.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.779+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7641a2260 con 0x7fb764101a40 2024-09-10T07:32:35.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.780+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb764108df0 con 0x7fb764101a40 2024-09-10T07:32:35.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.782+0000 7fb761600640 1 -- 172.21.15.46:0/1824115954 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb7540037a0 con 0x7fb764101a40 2024-09-10T07:32:35.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.783+0000 7fb761600640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7480a2b80 0x7fb7480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:35.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.783+0000 7fb761600640 1 -- 172.21.15.46:0/1824115954 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6809+0+0 (secure 0 0 0) 0x7fb7540ceeb0 con 0x7fb764101a40 2024-09-10T07:32:35.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.783+0000 7fb75be00640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7480a2b80 0x7fb7480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:35.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.784+0000 7fb75be00640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7480a2b80 0x7fb7480a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb74c031440 tx=0x7fb74c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:35.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.785+0000 7fb761600640 1 -- 172.21.15.46:0/1824115954 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb75409c350 con 0x7fb764101a40 2024-09-10T07:32:35.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.970+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb76402da30 con 0x7fb764101a40 2024-09-10T07:32:35.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.971+0000 7fb761600640 1 -- 172.21.15.46:0/1824115954 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fb75409bc50 con 0x7fb764101a40 2024-09-10T07:32:35.971 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:32:35.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7480a2b80 msgr2=0x7fb7480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:35.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7480a2b80 0x7fb7480a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb74c031440 tx=0x7fb74c0315b0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb764101a40 msgr2=0x7fb764075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb764101a40 0x7fb764075f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb75400e9c0 tx=0x7fb75400ee80 comp rx=0 tx=0).stop 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 shutdown_connections 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764103600 0x7fb764074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7480a2b80 0x7fb7480a5040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb764102c40 0x7fb7640745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 --2- 172.21.15.46:0/1824115954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb764101a40 0x7fb764075f90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 >> 172.21.15.46:0/1824115954 conn(0x7fb7640fd1f0 msgr2=0x7fb764105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.973+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 shutdown_connections 2024-09-10T07:32:35.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:35.974+0000 7fb769600640 1 -- 172.21.15.46:0/1824115954 wait complete. 2024-09-10T07:32:36.500 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:36 smithi181 ceph-mon[35397]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:36.500 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:36 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1824115954' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T07:32:36.500 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:36 smithi181 ceph-mon[35397]: from='osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T07:32:36.500 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:36 smithi181 ceph-mon[35397]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T07:32:36.501 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:32:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:32:36.242+0000 7f48e1728640 -1 osd.11 0 log_to_monitors true 2024-09-10T07:32:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:36 smithi063 ceph-mon[35020]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:36 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1824115954' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T07:32:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:36 smithi063 ceph-mon[35020]: from='osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T07:32:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:36 smithi063 ceph-mon[35020]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T07:32:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:36 smithi046 ceph-mon[30705]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:36 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1824115954' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T07:32:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:36 smithi046 ceph-mon[30705]: from='osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T07:32:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:36 smithi046 ceph-mon[30705]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T07:32:36.897 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1725953532,"num_in_osds":12,"osd_in_since":1725953535,"num_remapped_pgs":0} 2024-09-10T07:32:37.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:37 smithi063 ceph-mon[35020]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T07:32:37.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:37 smithi063 ceph-mon[35020]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T07:32:37.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:37 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:37.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:37 smithi063 ceph-mon[35020]: from='osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:32:37.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:37 smithi063 ceph-mon[35020]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:32:37.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:37 smithi181 ceph-mon[35397]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T07:32:37.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:37 smithi181 ceph-mon[35397]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T07:32:37.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:37 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:37.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:37 smithi181 ceph-mon[35397]: from='osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:32:37.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:37 smithi181 ceph-mon[35397]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:32:37.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:37 smithi046 ceph-mon[30705]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T07:32:37.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:37 smithi046 ceph-mon[30705]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T07:32:37.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:37 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:37.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:37 smithi046 ceph-mon[30705]: from='osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:32:37.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:37 smithi046 ceph-mon[30705]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-10T07:32:37.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T07:32:38.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:32:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:32:37.712+0000 7f48dd6ab640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T07:32:38.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:32:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:32:37.729+0000 7f48d54cb640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:32:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:32:38.602 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:38.602 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:32:38.602 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T07:32:38.602 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:38.602 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:38.602 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='osd.11 ' entity='osd.11' 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:38.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='osd.11 ' entity='osd.11' 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:38.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='osd.11 ' entity='osd.11' 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:38.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.859+0000 7fb273800640 1 -- 172.21.15.46:0/2240985846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 msgr2=0x7fb26c076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.859+0000 7fb273800640 1 --2- 172.21.15.46:0/2240985846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 0x7fb26c076fa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb260009900 tx=0x7fb26002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:32:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.860+0000 7fb273800640 1 -- 172.21.15.46:0/2240985846 shutdown_connections 2024-09-10T07:32:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.860+0000 7fb273800640 1 --2- 172.21.15.46:0/2240985846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb26c078790 0x7fb26c079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.860+0000 7fb273800640 1 --2- 172.21.15.46:0/2240985846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26c077df0 0x7fb26c078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.860+0000 7fb273800640 1 --2- 172.21.15.46:0/2240985846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 0x7fb26c076fa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.860+0000 7fb273800640 1 -- 172.21.15.46:0/2240985846 >> 172.21.15.46:0/2240985846 conn(0x7fb26c0ff5b0 msgr2=0x7fb26c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.860+0000 7fb273800640 1 -- 172.21.15.46:0/2240985846 shutdown_connections 2024-09-10T07:32:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.860+0000 7fb273800640 1 -- 172.21.15.46:0/2240985846 wait complete. 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.861+0000 7fb273800640 1 Processor -- start 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.861+0000 7fb273800640 1 -- start start 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.861+0000 7fb273800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 0x7fb26c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.861+0000 7fb273800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb26c077df0 0x7fb26c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.861+0000 7fb273800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26c078790 0x7fb26c1a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.861+0000 7fb273800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c113f10 con 0x7fb26c076ba0 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.861+0000 7fb273800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c113d90 con 0x7fb26c077df0 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.861+0000 7fb273800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c114090 con 0x7fb26c078790 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb270e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 0x7fb26c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb270e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 0x7fb26c19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53252/0 (socket says 172.21.15.46:53252) 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb270e00640 1 -- 172.21.15.46:0/1124899686 learned_addr learned my addr 172.21.15.46:0/1124899686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb26be00640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb26c077df0 0x7fb26c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb271800640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26c078790 0x7fb26c1a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb26be00640 1 -- 172.21.15.46:0/1124899686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26c078790 msgr2=0x7fb26c1a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb26be00640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26c078790 0x7fb26c1a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb26be00640 1 -- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 msgr2=0x7fb26c19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb26be00640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 0x7fb26c19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb26be00640 1 -- 172.21.15.46:0/1124899686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c1a6100 con 0x7fb26c077df0 2024-09-10T07:32:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.862+0000 7fb270e00640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 0x7fb26c19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:32:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.863+0000 7fb26be00640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb26c077df0 0x7fb26c19ea10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb25800e9f0 tx=0x7fb25800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.863+0000 7fb269600640 1 -- 172.21.15.46:0/1124899686 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25800ccc0 con 0x7fb26c077df0 2024-09-10T07:32:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.863+0000 7fb269600640 1 -- 172.21.15.46:0/1124899686 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb258005020 con 0x7fb26c077df0 2024-09-10T07:32:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.863+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c1a63f0 con 0x7fb26c077df0 2024-09-10T07:32:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.863+0000 7fb269600640 1 -- 172.21.15.46:0/1124899686 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2580049e0 con 0x7fb26c077df0 2024-09-10T07:32:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.863+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c1a6930 con 0x7fb26c077df0 2024-09-10T07:32:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.865+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c110780 con 0x7fb26c077df0 2024-09-10T07:32:38.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.866+0000 7fb269600640 1 -- 172.21.15.46:0/1124899686 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb258005560 con 0x7fb26c077df0 2024-09-10T07:32:38.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.868+0000 7fb269600640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2540a2c30 0x7fb2540a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:38.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.868+0000 7fb269600640 1 -- 172.21.15.46:0/1124899686 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fb2580ceb10 con 0x7fb26c077df0 2024-09-10T07:32:38.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.868+0000 7fb270e00640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2540a2c30 0x7fb2540a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:38.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.869+0000 7fb270e00640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2540a2c30 0x7fb2540a50f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb2600096c0 tx=0x7fb2600315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:38.871+0000 7fb269600640 1 -- 172.21.15.46:0/1124899686 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2580051d0 con 0x7fb26c077df0 2024-09-10T07:32:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.061+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb26c062df0 con 0x7fb26c077df0 2024-09-10T07:32:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.062+0000 7fb269600640 1 -- 172.21.15.46:0/1124899686 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7fb25809be70 con 0x7fb26c077df0 2024-09-10T07:32:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:32:39.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.065+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2540a2c30 msgr2=0x7fb2540a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:39.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.065+0000 7fb273800640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2540a2c30 0x7fb2540a50f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb2600096c0 tx=0x7fb2600315b0 comp rx=0 tx=0).stop 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.065+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb26c077df0 msgr2=0x7fb26c19ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.065+0000 7fb273800640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb26c077df0 0x7fb26c19ea10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb25800e9f0 tx=0x7fb25800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.066+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 shutdown_connections 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.066+0000 7fb273800640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26c078790 0x7fb26c1a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.066+0000 7fb273800640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb26c077df0 0x7fb26c19ea10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.066+0000 7fb273800640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2540a2c30 0x7fb2540a50f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.066+0000 7fb273800640 1 --2- 172.21.15.46:0/1124899686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb26c076ba0 0x7fb26c19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.066+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 >> 172.21.15.46:0/1124899686 conn(0x7fb26c0ff5b0 msgr2=0x7fb26c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.066+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 shutdown_connections 2024-09-10T07:32:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:39.066+0000 7fb273800640 1 -- 172.21.15.46:0/1124899686 wait complete. 2024-09-10T07:32:39.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: purged_snaps scrub starts 2024-09-10T07:32:39.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: purged_snaps scrub ok 2024-09-10T07:32:39.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: Detected new or changed devices on smithi181 2024-09-10T07:32:39.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi181 to 4272M 2024-09-10T07:32:39.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:39.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] boot 2024-09-10T07:32:39.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T07:32:39.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:39.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:39 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1124899686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T07:32:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: purged_snaps scrub starts 2024-09-10T07:32:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: purged_snaps scrub ok 2024-09-10T07:32:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: Detected new or changed devices on smithi181 2024-09-10T07:32:39.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi181 to 4272M 2024-09-10T07:32:39.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:39.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] boot 2024-09-10T07:32:39.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T07:32:39.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:39.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:39 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1124899686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T07:32:39.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: purged_snaps scrub starts 2024-09-10T07:32:39.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: purged_snaps scrub ok 2024-09-10T07:32:39.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: Detected new or changed devices on smithi181 2024-09-10T07:32:39.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi181 to 4272M 2024-09-10T07:32:39.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:39.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: osd.11 [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] boot 2024-09-10T07:32:39.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T07:32:39.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T07:32:39.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:39 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1124899686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T07:32:40.030 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1725953558,"num_in_osds":12,"osd_in_since":1725953535,"num_remapped_pgs":0} 2024-09-10T07:32:40.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:32:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:32:40.608 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:40 smithi046 ceph-mon[30705]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:40.608 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:40 smithi046 ceph-mon[30705]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T07:32:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:40 smithi181 ceph-mon[35397]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:40 smithi181 ceph-mon[35397]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T07:32:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:40 smithi063 ceph-mon[35020]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-10T07:32:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:40 smithi063 ceph-mon[35020]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T07:32:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 -- 172.21.15.46:0/2512225039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89501019c0 msgr2=0x7f8950101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 --2- 172.21.15.46:0/2512225039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89501019c0 0x7f8950101e20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8938009930 tx=0x7f893802a420 comp rx=0 tx=0).stop 2024-09-10T07:32:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 -- 172.21.15.46:0/2512225039 shutdown_connections 2024-09-10T07:32:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 --2- 172.21.15.46:0/2512225039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8950102360 0x7f895010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 --2- 172.21.15.46:0/2512225039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89501019c0 0x7f8950101e20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 --2- 172.21.15.46:0/2512225039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89501079c0 0x7f8950107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 -- 172.21.15.46:0/2512225039 >> 172.21.15.46:0/2512225039 conn(0x7f89500fd740 msgr2=0x7f89500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 -- 172.21.15.46:0/2512225039 shutdown_connections 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.031+0000 7f8954c00640 1 -- 172.21.15.46:0/2512225039 wait complete. 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.032+0000 7f8954c00640 1 Processor -- start 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.032+0000 7f8954c00640 1 -- start start 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8954c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89501019c0 0x7f89501982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8954c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8950102360 0x7f8950198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8954c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89501079c0 0x7f895019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8954c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8950111dd0 con 0x7f8950102360 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8954c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8950111c50 con 0x7f89501079c0 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8954c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8950111f50 con 0x7f89501019c0 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8947e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8950102360 0x7f8950198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8947e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8950102360 0x7f8950198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53276/0 (socket says 172.21.15.46:53276) 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8947e00640 1 -- 172.21.15.46:0/3716838203 learned_addr learned my addr 172.21.15.46:0/3716838203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:32:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f894f400640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89501019c0 0x7f89501982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f894fe00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89501079c0 0x7f895019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8947e00640 1 -- 172.21.15.46:0/3716838203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89501019c0 msgr2=0x7f89501982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8947e00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89501019c0 0x7f89501982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8947e00640 1 -- 172.21.15.46:0/3716838203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89501079c0 msgr2=0x7f895019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8947e00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89501079c0 0x7f895019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.033+0000 7f8947e00640 1 -- 172.21.15.46:0/3716838203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895019d280 con 0x7f8950102360 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.034+0000 7f8947e00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8950102360 0x7f8950198810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f893802a940 tx=0x7f8938003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.034+0000 7f894cc00640 1 -- 172.21.15.46:0/3716838203 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893803f070 con 0x7f8950102360 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.034+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895019d510 con 0x7f8950102360 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.034+0000 7f894cc00640 1 -- 172.21.15.46:0/3716838203 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8938031c90 con 0x7f8950102360 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.034+0000 7f894cc00640 1 -- 172.21.15.46:0/3716838203 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938035030 con 0x7f8950102360 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.034+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89501a4e50 con 0x7f8950102360 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.035+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8950103100 con 0x7f8950102360 2024-09-10T07:32:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.037+0000 7f894cc00640 1 -- 172.21.15.46:0/3716838203 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8938042070 con 0x7f8950102360 2024-09-10T07:32:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.038+0000 7f894cc00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f89340a2b80 0x7f89340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.038+0000 7f894cc00640 1 -- 172.21.15.46:0/3716838203 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f89380ef240 con 0x7f8950102360 2024-09-10T07:32:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.039+0000 7f894f400640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f89340a2b80 0x7f89340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.039+0000 7f894f400640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f89340a2b80 0x7f89340a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8940009b50 tx=0x7f8940005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.043+0000 7f894cc00640 1 -- 172.21.15.46:0/3716838203 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89380bbf50 con 0x7f8950102360 2024-09-10T07:32:41.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.229+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8950109f50 con 0x7f8950102360 2024-09-10T07:32:41.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.230+0000 7f894cc00640 1 -- 172.21.15.46:0/3716838203 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18384 (secure 0 0 0) 0x7f89380c0250 con 0x7f8950102360 2024-09-10T07:32:41.231 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:32:41.231 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:32:39.727796+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:32:41.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f89340a2b80 msgr2=0x7f89340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f89340a2b80 0x7f89340a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8940009b50 tx=0x7f8940005f90 comp rx=0 tx=0).stop 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8950102360 msgr2=0x7f8950198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8950102360 0x7f8950198810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f893802a940 tx=0x7f8938003b20 comp rx=0 tx=0).stop 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 shutdown_connections 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89501079c0 0x7f895019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f89340a2b80 0x7f89340a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8950102360 0x7f8950198810 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 --2- 172.21.15.46:0/3716838203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89501019c0 0x7f89501982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 >> 172.21.15.46:0/3716838203 conn(0x7f89500fd740 msgr2=0x7f89500fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 shutdown_connections 2024-09-10T07:32:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:41.233+0000 7f8954c00640 1 -- 172.21.15.46:0/3716838203 wait complete. 2024-09-10T07:32:41.587 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:41 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3716838203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:32:41.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:41 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3716838203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:32:41.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:41 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3716838203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:32:42.038 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:28:36.994371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T07:32:42.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T07:32:42.410 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:32:42.616 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:42 smithi046 ceph-mon[30705]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:42.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:42 smithi181 ceph-mon[35397]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:42.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:42 smithi063 ceph-mon[35020]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 -- 172.21.15.46:0/687345232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e00103ff0 msgr2=0x7f6e001043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 --2- 172.21.15.46:0/687345232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e00103ff0 0x7f6e001043d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009900 tx=0x7f6df402a410 comp rx=0 tx=0).stop 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 -- 172.21.15.46:0/687345232 shutdown_connections 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 --2- 172.21.15.46:0/687345232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e00109120 0x7f6e00110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 --2- 172.21.15.46:0/687345232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e001049a0 0x7f6e001089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 --2- 172.21.15.46:0/687345232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e00103ff0 0x7f6e001043d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 -- 172.21.15.46:0/687345232 >> 172.21.15.46:0/687345232 conn(0x7f6e000ffa90 msgr2=0x7f6e00101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 -- 172.21.15.46:0/687345232 shutdown_connections 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.009+0000 7f6e08800640 1 -- 172.21.15.46:0/687345232 wait complete. 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.010+0000 7f6e08800640 1 Processor -- start 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.010+0000 7f6e08800640 1 -- start start 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.010+0000 7f6e08800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e00103ff0 0x7f6e0019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.010+0000 7f6e08800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e001049a0 0x7f6e0019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.010+0000 7f6e08800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e00109120 0x7f6e0019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.010+0000 7f6e08800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00114190 con 0x7f6e001049a0 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.010+0000 7f6e08800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00114010 con 0x7f6e00109120 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.010+0000 7f6e08800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00114310 con 0x7f6e00103ff0 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e001049a0 0x7f6e0019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e001049a0 0x7f6e0019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53294/0 (socket says 172.21.15.46:53294) 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 -- 172.21.15.46:0/3664795213 learned_addr learned my addr 172.21.15.46:0/3664795213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e06800640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e00109120 0x7f6e0019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05e00640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e00103ff0 0x7f6e0019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 -- 172.21.15.46:0/3664795213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e00103ff0 msgr2=0x7f6e0019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e00103ff0 0x7f6e0019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 -- 172.21.15.46:0/3664795213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e00109120 msgr2=0x7f6e0019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:43.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e00109120 0x7f6e0019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 -- 172.21.15.46:0/3664795213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0019f5c0 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6e05400640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e001049a0 0x7f6e0019ab50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6df000e9f0 tx=0x7f6df000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.011+0000 7f6dfaa00640 1 -- 172.21.15.46:0/3664795213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df000cd80 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.012+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0019f8b0 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.012+0000 7f6dfaa00640 1 -- 172.21.15.46:0/3664795213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df0005500 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.012+0000 7f6dfaa00640 1 -- 172.21.15.46:0/3664795213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0010690 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.012+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e001a7190 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.013+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e001043d0 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.015+0000 7f6dfaa00640 1 -- 172.21.15.46:0/3664795213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6df00046c0 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.017+0000 7f6dfaa00640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6de80a2b80 0x7f6de80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.017+0000 7f6dfaa00640 1 -- 172.21.15.46:0/3664795213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f6df00cf0d0 con 0x7f6e001049a0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.017+0000 7f6e05e00640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6de80a2b80 0x7f6de80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:43.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.018+0000 7f6e05e00640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6de80a2b80 0x7f6de80a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6df4000c00 tx=0x7f6df4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.021+0000 7f6dfaa00640 1 -- 172.21.15.46:0/3664795213 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6df009c430 con 0x7f6e001049a0 2024-09-10T07:32:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.208+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e0002d7c0 con 0x7f6e001049a0 2024-09-10T07:32:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.209+0000 7f6dfaa00640 1 -- 172.21.15.46:0/3664795213 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f6df009bd30 con 0x7f6e001049a0 2024-09-10T07:32:43.210 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6de80a2b80 msgr2=0x7f6de80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6de80a2b80 0x7f6de80a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6df4000c00 tx=0x7f6df4005a20 comp rx=0 tx=0).stop 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e001049a0 msgr2=0x7f6e0019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e001049a0 0x7f6e0019ab50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6df000e9f0 tx=0x7f6df000edf0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 shutdown_connections 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e00109120 0x7f6e0019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6de80a2b80 0x7f6de80a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e001049a0 0x7f6e0019ab50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 --2- 172.21.15.46:0/3664795213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e00103ff0 0x7f6e0019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:43.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 >> 172.21.15.46:0/3664795213 conn(0x7f6e000ffa90 msgr2=0x7f6e00100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:43.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 shutdown_connections 2024-09-10T07:32:43.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:43.211+0000 7f6e08800640 1 -- 172.21.15.46:0/3664795213 wait complete. 2024-09-10T07:32:43.620 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:43 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3664795213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:32:43.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:43 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3664795213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:32:43.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:43 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3664795213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:32:44.028 INFO:tasks.cephadm:Adding mds.a on smithi046 2024-09-10T07:32:44.028 INFO:tasks.cephadm:Adding mds.d on smithi046 2024-09-10T07:32:44.028 INFO:tasks.cephadm:Adding mds.g on smithi046 2024-09-10T07:32:44.028 INFO:tasks.cephadm:Adding mds.j on smithi046 2024-09-10T07:32:44.029 INFO:tasks.cephadm:Adding mds.b on smithi063 2024-09-10T07:32:44.029 INFO:tasks.cephadm:Adding mds.e on smithi063 2024-09-10T07:32:44.029 INFO:tasks.cephadm:Adding mds.h on smithi063 2024-09-10T07:32:44.029 INFO:tasks.cephadm:Adding mds.k on smithi063 2024-09-10T07:32:44.029 INFO:tasks.cephadm:Adding mds.c on smithi181 2024-09-10T07:32:44.029 INFO:tasks.cephadm:Adding mds.f on smithi181 2024-09-10T07:32:44.029 INFO:tasks.cephadm:Adding mds.i on smithi181 2024-09-10T07:32:44.029 INFO:tasks.cephadm:Adding mds.l on smithi181 2024-09-10T07:32:44.029 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l' 2024-09-10T07:32:44.289 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:32:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:44 smithi181 ceph-mon[35397]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:44 smithi063 ceph-mon[35020]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:44.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:44 smithi046 ceph-mon[30705]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:44.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 -- 172.21.15.181:0/2911984621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c103bc0 msgr2=0x7ffa5c0ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:44.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 --2- 172.21.15.181:0/2911984621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c103bc0 0x7ffa5c0ffc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009900 tx=0x7ffa5002a410 comp rx=0 tx=0).stop 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 -- 172.21.15.181:0/2911984621 shutdown_connections 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 --2- 172.21.15.181:0/2911984621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c100350 0x7ffa5c10a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 --2- 172.21.15.181:0/2911984621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c103bc0 0x7ffa5c0ffc90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 --2- 172.21.15.181:0/2911984621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c1031f0 0x7ffa5c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 -- 172.21.15.181:0/2911984621 >> 172.21.15.181:0/2911984621 conn(0x7ffa5c0fba40 msgr2=0x7ffa5c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 -- 172.21.15.181:0/2911984621 shutdown_connections 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.794+0000 7ffa60f26640 1 -- 172.21.15.181:0/2911984621 wait complete. 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 Processor -- start 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 -- start start 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c100350 0x7ffa5c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c1031f0 0x7ffa5c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:44.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c103bc0 0x7ffa5c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c10d9a0 con 0x7ffa5c103bc0 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c10d820 con 0x7ffa5c1031f0 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c10db20 con 0x7ffa5c100350 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c100350 0x7ffa5c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c100350 0x7ffa5c198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57106/0 (socket says 172.21.15.181:57106) 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 -- 172.21.15.181:0/3196420888 learned_addr learned my addr 172.21.15.181:0/3196420888 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 -- 172.21.15.181:0/3196420888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c1031f0 msgr2=0x7ffa5c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c1031f0 0x7ffa5c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 -- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c103bc0 msgr2=0x7ffa5c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c103bc0 0x7ffa5c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 -- 172.21.15.181:0/3196420888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c19fda0 con 0x7ffa5c100350 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa5a59c640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c100350 0x7ffa5c198170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00b450 tx=0x7ffa4c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:44.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa4b7fe640 1 -- 172.21.15.181:0/3196420888 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c015020 con 0x7ffa5c100350 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa4b7fe640 1 -- 172.21.15.181:0/3196420888 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c011d70 con 0x7ffa5c100350 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa4b7fe640 1 -- 172.21.15.181:0/3196420888 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00f340 con 0x7ffa5c100350 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.795+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1a0090 con 0x7ffa5c100350 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.796+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1a0650 con 0x7ffa5c100350 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.797+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c109280 con 0x7ffa5c100350 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.798+0000 7ffa4b7fe640 1 -- 172.21.15.181:0/3196420888 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffa4c018020 con 0x7ffa5c100350 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.799+0000 7ffa4b7fe640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffa440a2bb0 0x7ffa440a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.799+0000 7ffa4b7fe640 1 -- 172.21.15.181:0/3196420888 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ffa4c0cebd0 con 0x7ffa5c100350 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.799+0000 7ffa59d9b640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffa440a2bb0 0x7ffa440a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.800+0000 7ffa59d9b640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffa440a2bb0 0x7ffa440a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa5002a930 tx=0x7ffa50000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:44.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.802+0000 7ffa4b7fe640 1 -- 172.21.15.181:0/3196420888 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa4c01f020 con 0x7ffa5c100350 2024-09-10T07:32:44.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.979+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}) -- 0x7ffa5c062730 con 0x7ffa440a2bb0 2024-09-10T07:32:44.987 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.987+0000 7ffa4b7fe640 1 -- 172.21.15.181:0/3196420888 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ffa5c062730 con 0x7ffa440a2bb0 2024-09-10T07:32:44.987 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mds.all update... 2024-09-10T07:32:44.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.989+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffa440a2bb0 msgr2=0x7ffa440a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:44.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.989+0000 7ffa60f26640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffa440a2bb0 0x7ffa440a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa5002a930 tx=0x7ffa50000bc0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.989+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c100350 msgr2=0x7ffa5c198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:44.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.989+0000 7ffa60f26640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c100350 0x7ffa5c198170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00b450 tx=0x7ffa4c00b910 comp rx=0 tx=0).stop 2024-09-10T07:32:44.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.990+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 shutdown_connections 2024-09-10T07:32:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.990+0000 7ffa60f26640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffa440a2bb0 0x7ffa440a5070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.990+0000 7ffa60f26640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c103bc0 0x7ffa5c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.990+0000 7ffa60f26640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c1031f0 0x7ffa5c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.990+0000 7ffa60f26640 1 --2- 172.21.15.181:0/3196420888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c100350 0x7ffa5c198170 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.990+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 >> 172.21.15.181:0/3196420888 conn(0x7ffa5c0fba40 msgr2=0x7ffa5c106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.990+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 shutdown_connections 2024-09-10T07:32:44.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T07:32:44.990+0000 7ffa60f26640 1 -- 172.21.15.181:0/3196420888 wait complete. 2024-09-10T07:32:45.715 DEBUG:teuthology.orchestra.run.smithi046:mds.a> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T07:32:45.720 DEBUG:teuthology.orchestra.run.smithi046:mds.d> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T07:32:45.766 DEBUG:teuthology.orchestra.run.smithi046:mds.g> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T07:32:45.812 DEBUG:teuthology.orchestra.run.smithi046:mds.j> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T07:32:45.858 DEBUG:teuthology.orchestra.run.smithi063:mds.b> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T07:32:45.862 DEBUG:teuthology.orchestra.run.smithi063:mds.e> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T07:32:45.906 DEBUG:teuthology.orchestra.run.smithi063:mds.h> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T07:32:45.950 DEBUG:teuthology.orchestra.run.smithi063:mds.k> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T07:32:45.995 DEBUG:teuthology.orchestra.run.smithi181:mds.c> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T07:32:45.998 DEBUG:teuthology.orchestra.run.smithi181:mds.f> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T07:32:46.041 DEBUG:teuthology.orchestra.run.smithi181:mds.i> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T07:32:46.086 DEBUG:teuthology.orchestra.run.smithi181:mds.l> sudo journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T07:32:46.129 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T07:32:46.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:32:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:32:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-09-10T07:32:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:46.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:46.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:46.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:46.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:46.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:46.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:46.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:45 smithi063 ceph-mon[35020]: Deploying daemon mds.l on smithi181 2024-09-10T07:32:46.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:46.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:46.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:45 smithi046 ceph-mon[30705]: Deploying daemon mds.l on smithi181 2024-09-10T07:32:46.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:46.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:32:46.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-09-10T07:32:46.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:46.276 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:32:46.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:46.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:32:46.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:46.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:46.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:46.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:46.277 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:45 smithi181 ceph-mon[35397]: Deploying daemon mds.l on smithi181 2024-09-10T07:32:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:32:47.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 -- 172.21.15.46:0/1246884841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0102790 msgr2=0x7ff1e01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:47.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/1246884841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0102790 0x7ff1e01067e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009900 tx=0x7ff1cc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 -- 172.21.15.46:0/1246884841 shutdown_connections 2024-09-10T07:32:47.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/1246884841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e0106f10 0x7ff1e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/1246884841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0102790 0x7ff1e01067e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/1246884841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1e0101de0 0x7ff1e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 -- 172.21.15.46:0/1246884841 >> 172.21.15.46:0/1246884841 conn(0x7ff1e00fd8c0 msgr2=0x7ff1e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:47.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 -- 172.21.15.46:0/1246884841 shutdown_connections 2024-09-10T07:32:47.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.105+0000 7ff1e7e00640 1 -- 172.21.15.46:0/1246884841 wait complete. 2024-09-10T07:32:47.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.106+0000 7ff1e7e00640 1 Processor -- start 2024-09-10T07:32:47.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.106+0000 7ff1e7e00640 1 -- start start 2024-09-10T07:32:47.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0101de0 0x7ff1e019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:47.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e0102790 0x7ff1e019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:47.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1e0106f10 0x7ff1e019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:47.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e0111fc0 con 0x7ff1e0101de0 2024-09-10T07:32:47.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e0111e40 con 0x7ff1e0106f10 2024-09-10T07:32:47.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e7e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e0112140 con 0x7ff1e0102790 2024-09-10T07:32:47.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0101de0 0x7ff1e019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:47.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0101de0 0x7ff1e019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52894/0 (socket says 172.21.15.46:52894) 2024-09-10T07:32:47.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e5400640 1 -- 172.21.15.46:0/431727089 learned_addr learned my addr 172.21.15.46:0/431727089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:32:47.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e4a00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e0102790 0x7ff1e019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:47.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.107+0000 7ff1e5e00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1e0106f10 0x7ff1e019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:47.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1e5400640 1 -- 172.21.15.46:0/431727089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e0102790 msgr2=0x7ff1e019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:47.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1e5400640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e0102790 0x7ff1e019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1e5400640 1 -- 172.21.15.46:0/431727089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1e0106f10 msgr2=0x7ff1e019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1e5400640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1e0106f10 0x7ff1e019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1e5400640 1 -- 172.21.15.46:0/431727089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e019f5f0 con 0x7ff1e0101de0 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1e5400640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0101de0 0x7ff1e019a640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400ea60 tx=0x7ff1d400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1de000640 1 -- 172.21.15.46:0/431727089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400cd20 con 0x7ff1e0101de0 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1de000640 1 -- 172.21.15.46:0/431727089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d40035f0 con 0x7ff1e0101de0 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e019f8e0 con 0x7ff1e0101de0 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1de000640 1 -- 172.21.15.46:0/431727089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4010690 con 0x7ff1e0101de0 2024-09-10T07:32:47.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.108+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e01a71c0 con 0x7ff1e0101de0 2024-09-10T07:32:47.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.110+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e01021c0 con 0x7ff1e0101de0 2024-09-10T07:32:47.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.111+0000 7ff1de000640 1 -- 172.21.15.46:0/431727089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff1d40037a0 con 0x7ff1e0101de0 2024-09-10T07:32:47.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.114+0000 7ff1de000640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff1c80a2b80 0x7ff1c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:47.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.114+0000 7ff1de000640 1 -- 172.21.15.46:0/431727089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ff1d40cef60 con 0x7ff1e0101de0 2024-09-10T07:32:47.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.114+0000 7ff1e4a00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff1c80a2b80 0x7ff1c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.115+0000 7ff1e4a00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff1c80a2b80 0x7ff1c80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000c00 tx=0x7ff1cc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:47.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.118+0000 7ff1de000640 1 -- 172.21.15.46:0/431727089 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff1d409c2c0 con 0x7ff1e0101de0 2024-09-10T07:32:47.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.341+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1e01a73f0 con 0x7ff1e0101de0 2024-09-10T07:32:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.342+0000 7ff1de000640 1 -- 172.21.15.46:0/431727089 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7ff1d409bbc0 con 0x7ff1e0101de0 2024-09-10T07:32:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:32:47.343 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":1,"btime":"2024-09-10T07:25:19:596582+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T07:32:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 1 2024-09-10T07:32:47.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.344+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff1c80a2b80 msgr2=0x7ff1c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:47.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.344+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff1c80a2b80 0x7ff1c80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000c00 tx=0x7ff1cc005b50 comp rx=0 tx=0).stop 2024-09-10T07:32:47.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.344+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0101de0 msgr2=0x7ff1e019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:47.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.344+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0101de0 0x7ff1e019a640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400ea60 tx=0x7ff1d400ee60 comp rx=0 tx=0).stop 2024-09-10T07:32:47.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.345+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 shutdown_connections 2024-09-10T07:32:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.345+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1e0106f10 0x7ff1e019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.345+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e0102790 0x7ff1e019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.345+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff1c80a2b80 0x7ff1c80a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.345+0000 7ff1e7e00640 1 --2- 172.21.15.46:0/431727089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1e0101de0 0x7ff1e019a640 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.345+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 >> 172.21.15.46:0/431727089 conn(0x7ff1e00fd8c0 msgr2=0x7ff1e00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.345+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 shutdown_connections 2024-09-10T07:32:47.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:47.345+0000 7ff1e7e00640 1 -- 172.21.15.46:0/431727089 wait complete. 2024-09-10T07:32:47.649 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:47 smithi181 systemd[1]: Starting Ceph mds.l for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:32:47.989 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:47 smithi181 podman[65428]: 2024-09-10 07:32:47.633882561 +0000 UTC m=+0.015564235 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:32:47.989 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:47 smithi181 podman[65428]: 2024-09-10 07:32:47.843066097 +0000 UTC m=+0.224747761 container create b1c792b601fa90ca8efeed2a8265cd8a0fe98b00c55fdb6938ba6fcb04241e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:32:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:47 smithi181 ceph-mon[35397]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:48.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:47 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/431727089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:48.400 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:48 smithi181 podman[65428]: 2024-09-10 07:32:48.096138103 +0000 UTC m=+0.477819768 container init b1c792b601fa90ca8efeed2a8265cd8a0fe98b00c55fdb6938ba6fcb04241e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T07:32:48.400 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:48 smithi181 podman[65428]: 2024-09-10 07:32:48.099120551 +0000 UTC m=+0.480802217 container start b1c792b601fa90ca8efeed2a8265cd8a0fe98b00c55fdb6938ba6fcb04241e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T07:32:48.400 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:48 smithi181 ceph-mds[65447]: -- 172.21.15.181:0/907066092 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef4dc0d860 con 0x55ef4dbeb000 2024-09-10T07:32:48.400 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:48 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l[65443]: starting mds.l at 2024-09-10T07:32:48.401 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:48 smithi181 bash[65428]: b1c792b601fa90ca8efeed2a8265cd8a0fe98b00c55fdb6938ba6fcb04241e95 2024-09-10T07:32:48.401 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:32:48 smithi181 systemd[1]: Started Ceph mds.l for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:32:48.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:47 smithi063 ceph-mon[35020]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:48.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:47 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/431727089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:48.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:47 smithi046 ceph-mon[30705]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:47 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/431727089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:49.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] up:boot 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: fsmap 1 up:standby 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:49.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:49 smithi181 ceph-mon[35397]: Deploying daemon mds.k on smithi063 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] up:boot 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: fsmap 1 up:standby 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:49.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:49 smithi063 ceph-mon[35020]: Deploying daemon mds.k on smithi063 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] up:boot 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: fsmap 1 up:standby 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:49 smithi046 ceph-mon[30705]: Deploying daemon mds.k on smithi063 2024-09-10T07:32:50.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:32:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:32:50.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:50 smithi181 ceph-mon[35397]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:50.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:50 smithi063 ceph-mon[35020]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:50.651 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:50 smithi063 systemd[1]: Starting Ceph mds.k for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:32:50.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:50 smithi046 ceph-mon[30705]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:50.932 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:50 smithi063 podman[64534]: 2024-09-10 07:32:50.577601367 +0000 UTC m=+0.015537230 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:32:50.933 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:50 smithi063 podman[64534]: 2024-09-10 07:32:50.680054887 +0000 UTC m=+0.117990783 container create 11c786acdcc13e4ffd0830a2c0f6959c9608edfbdbf684e8049087f50446936b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T07:32:50.933 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:50 smithi063 podman[64534]: 2024-09-10 07:32:50.907303409 +0000 UTC m=+0.345239263 container init 11c786acdcc13e4ffd0830a2c0f6959c9608edfbdbf684e8049087f50446936b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-10T07:32:50.933 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:50 smithi063 podman[64534]: 2024-09-10 07:32:50.910294079 +0000 UTC m=+0.348229936 container start 11c786acdcc13e4ffd0830a2c0f6959c9608edfbdbf684e8049087f50446936b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 -- 172.21.15.46:0/1726964186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca41079c0 msgr2=0x7f9ca4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/1726964186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca41079c0 0x7f9ca4107da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009930 tx=0x7f9c9402a420 comp rx=0 tx=0).stop 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 -- 172.21.15.46:0/1726964186 shutdown_connections 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/1726964186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca4102360 0x7f9ca410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/1726964186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ca41019c0 0x7f9ca4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/1726964186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca41079c0 0x7f9ca4107da0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 -- 172.21.15.46:0/1726964186 >> 172.21.15.46:0/1726964186 conn(0x7f9ca40fd740 msgr2=0x7f9ca40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 -- 172.21.15.46:0/1726964186 shutdown_connections 2024-09-10T07:32:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.187+0000 7f9ca3e00640 1 -- 172.21.15.46:0/1726964186 wait complete. 2024-09-10T07:32:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.188+0000 7f9ca3e00640 1 Processor -- start 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.188+0000 7f9ca3e00640 1 -- start start 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.188+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ca41019c0 0x7f9ca419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.188+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca4102360 0x7f9ca419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.188+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca41079c0 0x7f9ca419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.188+0000 7f9ca3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca4111e10 con 0x7f9ca41079c0 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.188+0000 7f9ca3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca4111c90 con 0x7f9ca41019c0 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.188+0000 7f9ca3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca4111f90 con 0x7f9ca4102360 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ca41019c0 0x7f9ca419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ca41019c0 0x7f9ca419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55036/0 (socket says 172.21.15.46:55036) 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2a00640 1 -- 172.21.15.46:0/3587591870 learned_addr learned my addr 172.21.15.46:0/3587591870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:32:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca3400640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca41079c0 0x7f9ca419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2000640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca4102360 0x7f9ca419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2a00640 1 -- 172.21.15.46:0/3587591870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca4102360 msgr2=0x7f9ca419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2a00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca4102360 0x7f9ca419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2a00640 1 -- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca41079c0 msgr2=0x7f9ca419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2a00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca41079c0 0x7f9ca419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.189+0000 7f9ca2a00640 1 -- 172.21.15.46:0/3587591870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca419f3f0 con 0x7f9ca41019c0 2024-09-10T07:32:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.190+0000 7f9ca2a00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ca41019c0 0x7f9ca419a440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c9402a940 tx=0x7f9c94003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.190+0000 7f9c9b400640 1 -- 172.21.15.46:0/3587591870 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9403f070 con 0x7f9ca41019c0 2024-09-10T07:32:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.190+0000 7f9c9b400640 1 -- 172.21.15.46:0/3587591870 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c94031c90 con 0x7f9ca41019c0 2024-09-10T07:32:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.190+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca419f680 con 0x7f9ca41019c0 2024-09-10T07:32:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.190+0000 7f9c9b400640 1 -- 172.21.15.46:0/3587591870 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94035030 con 0x7f9ca41019c0 2024-09-10T07:32:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.190+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca41a6fc0 con 0x7f9ca41019c0 2024-09-10T07:32:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.191+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca4103100 con 0x7f9ca41019c0 2024-09-10T07:32:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.193+0000 7f9c9b400640 1 -- 172.21.15.46:0/3587591870 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9c94042070 con 0x7f9ca41019c0 2024-09-10T07:32:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.194+0000 7f9c9b400640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c880a2b80 0x7f9c880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.194+0000 7f9c9b400640 1 -- 172.21.15.46:0/3587591870 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f9c940ef240 con 0x7f9ca41019c0 2024-09-10T07:32:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.194+0000 7f9ca2000640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c880a2b80 0x7f9c880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.195+0000 7f9ca2000640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c880a2b80 0x7f9c880a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9c900095f0 tx=0x7f9c90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.198+0000 7f9c9b400640 1 -- 172.21.15.46:0/3587591870 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c940bbf50 con 0x7f9ca41019c0 2024-09-10T07:32:51.400 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:50 smithi063 ceph-mds[64554]: -- 172.21.15.63:0/2537931291 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651d3747860 con 0x5651d3725000 2024-09-10T07:32:51.400 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:50 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k[64550]: starting mds.k at 2024-09-10T07:32:51.400 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:51 smithi063 bash[64534]: 11c786acdcc13e4ffd0830a2c0f6959c9608edfbdbf684e8049087f50446936b 2024-09-10T07:32:51.400 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:32:51 smithi063 systemd[1]: Started Ceph mds.k for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:32:51.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.431+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ca41a7190 con 0x7f9ca41019c0 2024-09-10T07:32:51.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.431+0000 7f9c9b400640 1 -- 172.21.15.46:0/3587591870 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7f9c940c0250 con 0x7f9ca41019c0 2024-09-10T07:32:51.432 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:32:51.432 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"btime":"2024-09-10T07:32:51:292579+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T07:32:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2024-09-10T07:32:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c880a2b80 msgr2=0x7f9c880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c880a2b80 0x7f9c880a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9c900095f0 tx=0x7f9c90005f90 comp rx=0 tx=0).stop 2024-09-10T07:32:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ca41019c0 msgr2=0x7f9ca419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ca41019c0 0x7f9ca419a440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c9402a940 tx=0x7f9c94003b20 comp rx=0 tx=0).stop 2024-09-10T07:32:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 shutdown_connections 2024-09-10T07:32:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c880a2b80 0x7f9c880a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca41079c0 0x7f9ca419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ca4102360 0x7f9ca419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/3587591870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ca41019c0 0x7f9ca419a440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:51.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 >> 172.21.15.46:0/3587591870 conn(0x7f9ca40fd740 msgr2=0x7f9ca410d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:51.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.434+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 shutdown_connections 2024-09-10T07:32:51.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:51.435+0000 7f9ca3e00640 1 -- 172.21.15.46:0/3587591870 wait complete. 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: Deploying daemon mds.j on smithi046 2024-09-10T07:32:52.340 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6832/620318721,v1:172.21.15.63:6833/620318721] up:boot 2024-09-10T07:32:52.341 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: fsmap 2 up:standby 2024-09-10T07:32:52.341 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T07:32:52.341 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:52 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3587591870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:52.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:52.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: Deploying daemon mds.j on smithi046 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6832/620318721,v1:172.21.15.63:6833/620318721] up:boot 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: fsmap 2 up:standby 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T07:32:52.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:52 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3587591870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:52.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:52.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:52.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: Deploying daemon mds.j on smithi046 2024-09-10T07:32:52.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6832/620318721,v1:172.21.15.63:6833/620318721] up:boot 2024-09-10T07:32:52.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: fsmap 2 up:standby 2024-09-10T07:32:52.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T07:32:52.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:52 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3587591870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:54.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:54 smithi046 ceph-mon[30705]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:54 smithi181 ceph-mon[35397]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:54 smithi063 ceph-mon[35020]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:54.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:32:54.993 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:32:54 smithi046 podman[66998]: 2024-09-10 07:32:54.75800921 +0000 UTC m=+0.172237463 container create 568f4903558a9fc571ccf78a784aad00c66ef06364254dd79cf45cfb8c28be6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:32:54.993 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:32:54 smithi046 podman[66998]: 2024-09-10 07:32:54.993983038 +0000 UTC m=+0.408211317 container init 568f4903558a9fc571ccf78a784aad00c66ef06364254dd79cf45cfb8c28be6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T07:32:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:32:55.318 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:32:55 smithi046 podman[66998]: 2024-09-10 07:32:55.000722034 +0000 UTC m=+0.414950313 container start 568f4903558a9fc571ccf78a784aad00c66ef06364254dd79cf45cfb8c28be6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T07:32:55.318 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:32:55 smithi046 ceph-mds[67043]: -- 172.21.15.46:0/2125548545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c7b275860 con 0x560c7b252800 2024-09-10T07:32:55.319 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:32:55 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j[67026]: starting mds.j at 2024-09-10T07:32:55.319 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:32:55 smithi046 bash[66998]: 568f4903558a9fc571ccf78a784aad00c66ef06364254dd79cf45cfb8c28be6c 2024-09-10T07:32:55.319 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:32:55 smithi046 systemd[1]: Started Ceph mds.j for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 -- 172.21.15.46:0/2082249499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e4c074590 msgr2=0x7f5e4c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 --2- 172.21.15.46:0/2082249499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e4c074590 0x7f5e4c074970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009860 tx=0x7f5e4002a3d0 comp rx=0 tx=0).stop 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 -- 172.21.15.46:0/2082249499 shutdown_connections 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 --2- 172.21.15.46:0/2082249499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c112b90 0x7f5e4c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 --2- 172.21.15.46:0/2082249499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e4c074eb0 0x7f5e4c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 --2- 172.21.15.46:0/2082249499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e4c074590 0x7f5e4c074970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 -- 172.21.15.46:0/2082249499 >> 172.21.15.46:0/2082249499 conn(0x7f5e4c0fd8c0 msgr2=0x7f5e4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 -- 172.21.15.46:0/2082249499 shutdown_connections 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.789+0000 7f5e53800640 1 -- 172.21.15.46:0/2082249499 wait complete. 2024-09-10T07:32:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.790+0000 7f5e53800640 1 Processor -- start 2024-09-10T07:32:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.790+0000 7f5e53800640 1 -- start start 2024-09-10T07:32:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e53800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c074590 0x7f5e4c1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e53800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e4c074eb0 0x7f5e4c1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e53800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e4c112b90 0x7f5e4c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e53800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c1185a0 con 0x7f5e4c074eb0 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e53800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c118420 con 0x7f5e4c112b90 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e53800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c118720 con 0x7f5e4c074590 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e50e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c074590 0x7f5e4c1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e50e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c074590 0x7f5e4c1a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:42202/0 (socket says 172.21.15.46:42202) 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e50e00640 1 -- 172.21.15.46:0/256772361 learned_addr learned my addr 172.21.15.46:0/256772361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e4be00640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e4c074eb0 0x7f5e4c1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e51800640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e4c112b90 0x7f5e4c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.791+0000 7f5e50e00640 1 -- 172.21.15.46:0/256772361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e4c112b90 msgr2=0x7f5e4c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e50e00640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e4c112b90 0x7f5e4c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e50e00640 1 -- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e4c074eb0 msgr2=0x7f5e4c1a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e50e00640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e4c074eb0 0x7f5e4c1a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e50e00640 1 -- 172.21.15.46:0/256772361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c19f080 con 0x7f5e4c074590 2024-09-10T07:32:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e50e00640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c074590 0x7f5e4c1a4790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009980 tx=0x7f5e400309c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e49600640 1 -- 172.21.15.46:0/256772361 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40040070 con 0x7f5e4c074590 2024-09-10T07:32:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c19f310 con 0x7f5e4c074590 2024-09-10T07:32:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e49600640 1 -- 172.21.15.46:0/256772361 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e40031400 con 0x7f5e4c074590 2024-09-10T07:32:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e49600640 1 -- 172.21.15.46:0/256772361 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40004a00 con 0x7f5e4c074590 2024-09-10T07:32:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.792+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c1ab660 con 0x7f5e4c074590 2024-09-10T07:32:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.793+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4c074970 con 0x7f5e4c074590 2024-09-10T07:32:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.795+0000 7f5e49600640 1 -- 172.21.15.46:0/256772361 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5e40043070 con 0x7f5e4c074590 2024-09-10T07:32:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.797+0000 7f5e49600640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e340a2b80 0x7f5e340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:32:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.798+0000 7f5e49600640 1 -- 172.21.15.46:0/256772361 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f5e400ef940 con 0x7f5e4c074590 2024-09-10T07:32:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.798+0000 7f5e4be00640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e340a2b80 0x7f5e340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:32:55.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.798+0000 7f5e4be00640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e340a2b80 0x7f5e340a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0fff90 tx=0x7f5e38009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:32:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:55.800+0000 7f5e49600640 1 -- 172.21.15.46:0/256772361 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e400bc650 con 0x7f5e4c074590 2024-09-10T07:32:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.028+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e4c104fd0 con 0x7f5e4c074590 2024-09-10T07:32:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.029+0000 7f5e49600640 1 -- 172.21.15.46:0/256772361 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f5e400c0950 con 0x7f5e4c074590 2024-09-10T07:32:56.030 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:32:56.030 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":4,"btime":"2024-09-10T07:32:55:060524+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-09-10T07:32:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 4 2024-09-10T07:32:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.031+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e340a2b80 msgr2=0x7f5e340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.031+0000 7f5e53800640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e340a2b80 0x7f5e340a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0fff90 tx=0x7f5e38009260 comp rx=0 tx=0).stop 2024-09-10T07:32:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.031+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c074590 msgr2=0x7f5e4c1a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:32:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.031+0000 7f5e53800640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c074590 0x7f5e4c1a4790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009980 tx=0x7f5e400309c0 comp rx=0 tx=0).stop 2024-09-10T07:32:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.031+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 shutdown_connections 2024-09-10T07:32:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.031+0000 7f5e53800640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e4c112b90 0x7f5e4c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.032+0000 7f5e53800640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e340a2b80 0x7f5e340a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.032+0000 7f5e53800640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e4c074eb0 0x7f5e4c1a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.032+0000 7f5e53800640 1 --2- 172.21.15.46:0/256772361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c074590 0x7f5e4c1a4790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:32:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.032+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 >> 172.21.15.46:0/256772361 conn(0x7f5e4c0fd8c0 msgr2=0x7f5e4c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:32:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.032+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 shutdown_connections 2024-09-10T07:32:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:32:56.032+0000 7f5e53800640 1 -- 172.21.15.46:0/256772361 wait complete. 2024-09-10T07:32:56.348 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.46:6834/4121221095,v1:172.21.15.46:6835/4121221095] up:boot 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: fsmap 3 up:standby 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: Deploying daemon mds.d on smithi046 2024-09-10T07:32:56.349 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:56 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/256772361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:56.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.46:6834/4121221095,v1:172.21.15.46:6835/4121221095] up:boot 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: fsmap 3 up:standby 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: Deploying daemon mds.d on smithi046 2024-09-10T07:32:56.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:56 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/256772361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.46:6834/4121221095,v1:172.21.15.46:6835/4121221095] up:boot 2024-09-10T07:32:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: fsmap 3 up:standby 2024-09-10T07:32:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T07:32:56.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:32:56.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:32:56.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:32:56.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:32:56.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: Deploying daemon mds.d on smithi046 2024-09-10T07:32:56.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:56 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/256772361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:32:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:32:58 smithi181 ceph-mon[35397]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:58.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:32:58 smithi063 ceph-mon[35020]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:58.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:32:58 smithi046 ceph-mon[30705]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:32:58.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:32:58.986 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:58 smithi046 systemd[1]: Starting Ceph mds.d for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:32:59.348 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:59 smithi046 podman[67391]: 2024-09-10 07:32:58.9324701 +0000 UTC m=+0.027197987 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:32:59.348 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:59 smithi046 podman[67391]: 2024-09-10 07:32:59.079068215 +0000 UTC m=+0.173796078 container create f302cef36e52d08242ef4f11e6bba57609b7ee7c7f469f46d4332538b9a6392b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T07:32:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:32:59.659 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:59 smithi046 podman[67391]: 2024-09-10 07:32:59.348721458 +0000 UTC m=+0.443449322 container init f302cef36e52d08242ef4f11e6bba57609b7ee7c7f469f46d4332538b9a6392b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T07:32:59.659 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:59 smithi046 podman[67391]: 2024-09-10 07:32:59.357561092 +0000 UTC m=+0.452288957 container start f302cef36e52d08242ef4f11e6bba57609b7ee7c7f469f46d4332538b9a6392b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T07:32:59.659 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:59 smithi046 ceph-mds[67452]: -- 172.21.15.46:0/2940508128 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f10927f860 con 0x55f10925d400 2024-09-10T07:32:59.659 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:59 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d[67435]: starting mds.d at 2024-09-10T07:32:59.659 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:59 smithi046 bash[67391]: f302cef36e52d08242ef4f11e6bba57609b7ee7c7f469f46d4332538b9a6392b 2024-09-10T07:32:59.659 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:32:59 smithi046 systemd[1]: Started Ceph mds.d for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 -- 172.21.15.46:0/1927724260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb208103ff0 msgr2=0x7fb2081043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 --2- 172.21.15.46:0/1927724260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb208103ff0 0x7fb2081043d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009900 tx=0x7fb1f802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 -- 172.21.15.46:0/1927724260 shutdown_connections 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 --2- 172.21.15.46:0/1927724260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb208109120 0x7fb208110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 --2- 172.21.15.46:0/1927724260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2081049a0 0x7fb2081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 --2- 172.21.15.46:0/1927724260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb208103ff0 0x7fb2081043d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 -- 172.21.15.46:0/1927724260 >> 172.21.15.46:0/1927724260 conn(0x7fb2080ffa90 msgr2=0x7fb208101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 -- 172.21.15.46:0/1927724260 shutdown_connections 2024-09-10T07:33:00.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.195+0000 7fb20ee00640 1 -- 172.21.15.46:0/1927724260 wait complete. 2024-09-10T07:33:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.197+0000 7fb20ee00640 1 Processor -- start 2024-09-10T07:33:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.197+0000 7fb20ee00640 1 -- start start 2024-09-10T07:33:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.197+0000 7fb20ee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb208103ff0 0x7fb20819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.197+0000 7fb20ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2081049a0 0x7fb20819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.197+0000 7fb20ee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb208109120 0x7fb20819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.197+0000 7fb20ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb208114150 con 0x7fb2081049a0 2024-09-10T07:33:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.197+0000 7fb20ee00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb208113fd0 con 0x7fb208103ff0 2024-09-10T07:33:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.197+0000 7fb20ee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081142d0 con 0x7fb208109120 2024-09-10T07:33:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb208103ff0 0x7fb20819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb208103ff0 0x7fb20819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36800/0 (socket says 172.21.15.46:36800) 2024-09-10T07:33:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207e00640 1 -- 172.21.15.46:0/2772926984 learned_addr learned my addr 172.21.15.46:0/2772926984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207400640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2081049a0 0x7fb20819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb20ce00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb208109120 0x7fb20819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207e00640 1 -- 172.21.15.46:0/2772926984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb208109120 msgr2=0x7fb20819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207e00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb208109120 0x7fb20819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207e00640 1 -- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2081049a0 msgr2=0x7fb20819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207e00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2081049a0 0x7fb20819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.198+0000 7fb207e00640 1 -- 172.21.15.46:0/2772926984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20819f5a0 con 0x7fb208103ff0 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.199+0000 7fb207e00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb208103ff0 0x7fb20819a5f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8006010 tx=0x7fb1f80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.199+0000 7fb204c00640 1 -- 172.21.15.46:0/2772926984 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f803f030 con 0x7fb208103ff0 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.199+0000 7fb204c00640 1 -- 172.21.15.46:0/2772926984 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f8003ce0 con 0x7fb208103ff0 2024-09-10T07:33:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.199+0000 7fb204c00640 1 -- 172.21.15.46:0/2772926984 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8035070 con 0x7fb208103ff0 2024-09-10T07:33:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.199+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20819f830 con 0x7fb208103ff0 2024-09-10T07:33:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.199+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a7170 con 0x7fb208103ff0 2024-09-10T07:33:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.200+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2081043d0 con 0x7fb208103ff0 2024-09-10T07:33:00.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.202+0000 7fb204c00640 1 -- 172.21.15.46:0/2772926984 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb1f8042030 con 0x7fb208103ff0 2024-09-10T07:33:00.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.203+0000 7fb204c00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1ec0a2c00 0x7fb1ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:00.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.203+0000 7fb204c00640 1 -- 172.21.15.46:0/2772926984 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fb1f80ef0f0 con 0x7fb208103ff0 2024-09-10T07:33:00.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.203+0000 7fb207400640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1ec0a2c00 0x7fb1ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:00.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.204+0000 7fb207400640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1ec0a2c00 0x7fb1ec0a50c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2080ffeb0 tx=0x7fb1f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.207+0000 7fb204c00640 1 -- 172.21.15.46:0/2772926984 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1f80bbec0 con 0x7fb208103ff0 2024-09-10T07:33:00.275 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:00 smithi063 ceph-mon[35020]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:00.275 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.275 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.275 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.275 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:00.275 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:00.275 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:00 smithi181 ceph-mon[35397]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:00.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:00.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:00 smithi046 ceph-mon[30705]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.431+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb20819b950 con 0x7fb208103ff0 2024-09-10T07:33:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.432+0000 7fb204c00640 1 -- 172.21.15.46:0/2772926984 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7fb1f80c01c0 con 0x7fb208103ff0 2024-09-10T07:33:00.433 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:00.433 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"btime":"2024-09-10T07:33:00:071685+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-09-10T07:33:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 5 2024-09-10T07:33:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1ec0a2c00 msgr2=0x7fb1ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1ec0a2c00 0x7fb1ec0a50c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2080ffeb0 tx=0x7fb1f400b390 comp rx=0 tx=0).stop 2024-09-10T07:33:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb208103ff0 msgr2=0x7fb20819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb208103ff0 0x7fb20819a5f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8006010 tx=0x7fb1f80042f0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 shutdown_connections 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb208109120 0x7fb20819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1ec0a2c00 0x7fb1ec0a50c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2081049a0 0x7fb20819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 --2- 172.21.15.46:0/2772926984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb208103ff0 0x7fb20819a5f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.434+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 >> 172.21.15.46:0/2772926984 conn(0x7fb2080ffa90 msgr2=0x7fb208100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.435+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 shutdown_connections 2024-09-10T07:33:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:00.435+0000 7fb20ee00640 1 -- 172.21.15.46:0/2772926984 wait complete. 2024-09-10T07:33:01.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:01 smithi181 ceph-mon[35397]: Deploying daemon mds.e on smithi063 2024-09-10T07:33:01.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:01 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.46:6836/2449149651,v1:172.21.15.46:6837/2449149651] up:boot 2024-09-10T07:33:01.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:01 smithi181 ceph-mon[35397]: fsmap 4 up:standby 2024-09-10T07:33:01.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:01 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T07:33:01.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:01 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2772926984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:01.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:01 smithi063 ceph-mon[35020]: Deploying daemon mds.e on smithi063 2024-09-10T07:33:01.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:01 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.46:6836/2449149651,v1:172.21.15.46:6837/2449149651] up:boot 2024-09-10T07:33:01.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:01 smithi063 ceph-mon[35020]: fsmap 4 up:standby 2024-09-10T07:33:01.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:01 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T07:33:01.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:01 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2772926984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:01.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:01 smithi046 ceph-mon[30705]: Deploying daemon mds.e on smithi063 2024-09-10T07:33:01.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:01 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.46:6836/2449149651,v1:172.21.15.46:6837/2449149651] up:boot 2024-09-10T07:33:01.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:01 smithi046 ceph-mon[30705]: fsmap 4 up:standby 2024-09-10T07:33:01.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:01 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T07:33:01.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:01 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2772926984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:01.829 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:01 smithi063 systemd[1]: Starting Ceph mds.e for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:33:02.085 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:01 smithi063 podman[64849]: 2024-09-10 07:33:01.732585733 +0000 UTC m=+0.015574729 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:33:02.085 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:01 smithi063 podman[64849]: 2024-09-10 07:33:01.85243041 +0000 UTC m=+0.135419433 container create 7732d917143f753b4a7b0bd1a530282d861833b9d341bc26c18bceeff0c17b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T07:33:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:02 smithi181 ceph-mon[35397]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:02.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:02 smithi063 ceph-mon[35020]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:02.400 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:02 smithi063 podman[64849]: 2024-09-10 07:33:02.09647303 +0000 UTC m=+0.379462012 container init 7732d917143f753b4a7b0bd1a530282d861833b9d341bc26c18bceeff0c17b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T07:33:02.400 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:02 smithi063 podman[64849]: 2024-09-10 07:33:02.099407767 +0000 UTC m=+0.382396770 container start 7732d917143f753b4a7b0bd1a530282d861833b9d341bc26c18bceeff0c17b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T07:33:02.400 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:02 smithi063 ceph-mds[64868]: -- 172.21.15.63:0/2448137703 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc22767860 con 0x55dc22745400 2024-09-10T07:33:02.400 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e[64864]: starting mds.e at 2024-09-10T07:33:02.401 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:02 smithi063 bash[64849]: 7732d917143f753b4a7b0bd1a530282d861833b9d341bc26c18bceeff0c17b11 2024-09-10T07:33:02.401 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:33:02 smithi063 systemd[1]: Started Ceph mds.e for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:02.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:02 smithi046 ceph-mon[30705]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:03.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: Deploying daemon mds.f on smithi181 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6834/1645722263,v1:172.21.15.63:6835/1645722263] up:boot 2024-09-10T07:33:03.239 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: fsmap 5 up:standby 2024-09-10T07:33:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T07:33:03.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: Deploying daemon mds.f on smithi181 2024-09-10T07:33:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6834/1645722263,v1:172.21.15.63:6835/1645722263] up:boot 2024-09-10T07:33:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: fsmap 5 up:standby 2024-09-10T07:33:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: Deploying daemon mds.f on smithi181 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6834/1645722263,v1:172.21.15.63:6835/1645722263] up:boot 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: fsmap 5 up:standby 2024-09-10T07:33:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T07:33:04.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 -- 172.21.15.46:0/3459704266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c112a30 msgr2=0x7f167c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:04.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 --2- 172.21.15.46:0/3459704266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c112a30 0x7f167c114e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1674009900 tx=0x7f167402a410 comp rx=0 tx=0).stop 2024-09-10T07:33:04.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 -- 172.21.15.46:0/3459704266 shutdown_connections 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 --2- 172.21.15.46:0/3459704266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c112a30 0x7f167c114e20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 --2- 172.21.15.46:0/3459704266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c077040 0x7f167c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 --2- 172.21.15.46:0/3459704266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167c076720 0x7f167c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 -- 172.21.15.46:0/3459704266 >> 172.21.15.46:0/3459704266 conn(0x7f167c0ffa90 msgr2=0x7f167c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 -- 172.21.15.46:0/3459704266 shutdown_connections 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.165+0000 7f1681000640 1 -- 172.21.15.46:0/3459704266 wait complete. 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.166+0000 7f1681000640 1 Processor -- start 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.166+0000 7f1681000640 1 -- start start 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1681000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167c076720 0x7f167c1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1681000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c077040 0x7f167c1a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1681000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c112a30 0x7f167c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1681000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1185e0 con 0x7f167c076720 2024-09-10T07:33:04.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1681000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f167c118460 con 0x7f167c112a30 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1681000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f167c118760 con 0x7f167c077040 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1679e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167c076720 0x7f167c1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1671400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c077040 0x7f167c1a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1671400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c077040 0x7f167c1a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56442/0 (socket says 172.21.15.46:56442) 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1679e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167c076720 0x7f167c1a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55478/0 (socket says 172.21.15.46:55478) 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f1671400640 1 -- 172.21.15.46:0/641673268 learned_addr learned my addr 172.21.15.46:0/641673268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.167+0000 7f167a800640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c112a30 0x7f167c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1671400640 1 -- 172.21.15.46:0/641673268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c112a30 msgr2=0x7f167c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1671400640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c112a30 0x7f167c19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1671400640 1 -- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167c076720 msgr2=0x7f167c1a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1671400640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167c076720 0x7f167c1a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1671400640 1 -- 172.21.15.46:0/641673268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c19f040 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1679e00640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167c076720 0x7f167c1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1671400640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c077040 0x7f167c1a4c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f166400e9f0 tx=0x7f166400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1673400640 1 -- 172.21.15.46:0/641673268 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166400cd80 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1673400640 1 -- 172.21.15.46:0/641673268 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1664005500 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c19f330 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1673400640 1 -- 172.21.15.46:0/641673268 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664010690 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.168+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c1ab620 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.169+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f167c076b00 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.171+0000 7f1673400640 1 -- 172.21.15.46:0/641673268 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f16640046c0 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.172+0000 7f1673400640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f16600a2b80 0x7f16600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.173+0000 7f1673400640 1 -- 172.21.15.46:0/641673268 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f16640cefe0 con 0x7f167c077040 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.173+0000 7f1679e00640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f16600a2b80 0x7f16600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.173+0000 7f1679e00640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f16600a2b80 0x7f16600a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f166c009fd0 tx=0x7f166c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:04.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.176+0000 7f1673400640 1 -- 172.21.15.46:0/641673268 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f166409c340 con 0x7f167c077040 2024-09-10T07:33:04.255 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 systemd[1]: Starting Ceph mds.f for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:33:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.402+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f167c19fe40 con 0x7f167c077040 2024-09-10T07:33:04.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:04 smithi046 ceph-mon[30705]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:04.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.403+0000 7f1673400640 1 -- 172.21.15.46:0/641673268 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f167c19fe40 con 0x7f167c077040 2024-09-10T07:33:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:04.406 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"btime":"2024-09-10T07:33:03:085461+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T07:33:04.406 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2024-09-10T07:33:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.406+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f16600a2b80 msgr2=0x7f16600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.406+0000 7f1681000640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f16600a2b80 0x7f16600a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f166c009fd0 tx=0x7f166c009500 comp rx=0 tx=0).stop 2024-09-10T07:33:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c077040 msgr2=0x7f167c1a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c077040 0x7f167c1a4c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f166400e9f0 tx=0x7f166400edf0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 shutdown_connections 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c112a30 0x7f167c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f167c077040 0x7f167c1a4c90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f16600a2b80 0x7f16600a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 --2- 172.21.15.46:0/641673268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167c076720 0x7f167c1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 >> 172.21.15.46:0/641673268 conn(0x7f167c0ffa90 msgr2=0x7f167c113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 shutdown_connections 2024-09-10T07:33:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:04.407+0000 7f1681000640 1 -- 172.21.15.46:0/641673268 wait complete. 2024-09-10T07:33:04.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:04 smithi181 ceph-mon[35397]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:04.607 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 podman[65740]: 2024-09-10 07:33:04.231067716 +0000 UTC m=+0.016022639 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:33:04.607 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 podman[65740]: 2024-09-10 07:33:04.338209329 +0000 UTC m=+0.123164247 container create 034cd2c436b0c5ca64f18480424c5b30e96c0d11345b630cedc3dfd9c66d7628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:33:04.607 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 podman[65740]: 2024-09-10 07:33:04.607587873 +0000 UTC m=+0.392542792 container init 034cd2c436b0c5ca64f18480424c5b30e96c0d11345b630cedc3dfd9c66d7628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T07:33:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:04 smithi063 ceph-mon[35020]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:04.899 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 podman[65740]: 2024-09-10 07:33:04.61032199 +0000 UTC m=+0.395276907 container start 034cd2c436b0c5ca64f18480424c5b30e96c0d11345b630cedc3dfd9c66d7628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T07:33:04.899 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 ceph-mds[65759]: -- 172.21.15.181:0/2103007644 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c5818b860 con 0x563c58169400 2024-09-10T07:33:04.899 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f[65755]: starting mds.f at 2024-09-10T07:33:04.900 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 bash[65740]: 034cd2c436b0c5ca64f18480424c5b30e96c0d11345b630cedc3dfd9c66d7628 2024-09-10T07:33:04.900 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:33:04 smithi181 systemd[1]: Started Ceph mds.f for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:05.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:05 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/641673268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:05.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:05.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:05.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:05.436 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:05 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/641673268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:05.436 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.436 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.436 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.436 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:05.437 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:05.437 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:05.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:05 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/641673268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:05.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:05.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:05.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:05.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:06.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:06 smithi181 ceph-mon[35397]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:06.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:06 smithi181 ceph-mon[35397]: Deploying daemon mds.c on smithi181 2024-09-10T07:33:06.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:06 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6834/335913193,v1:172.21.15.181:6835/335913193] up:boot 2024-09-10T07:33:06.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:06 smithi181 ceph-mon[35397]: fsmap 6 up:standby 2024-09-10T07:33:06.491 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:06 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T07:33:06.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:06 smithi063 ceph-mon[35020]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:06.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:06 smithi063 ceph-mon[35020]: Deploying daemon mds.c on smithi181 2024-09-10T07:33:06.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:06 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6834/335913193,v1:172.21.15.181:6835/335913193] up:boot 2024-09-10T07:33:06.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:06 smithi063 ceph-mon[35020]: fsmap 6 up:standby 2024-09-10T07:33:06.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:06 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T07:33:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:06 smithi046 ceph-mon[30705]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:06 smithi046 ceph-mon[30705]: Deploying daemon mds.c on smithi181 2024-09-10T07:33:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:06 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6834/335913193,v1:172.21.15.181:6835/335913193] up:boot 2024-09-10T07:33:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:06 smithi046 ceph-mon[30705]: fsmap 6 up:standby 2024-09-10T07:33:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:06 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T07:33:06.883 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:06 smithi181 systemd[1]: Starting Ceph mds.c for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:33:07.149 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:06 smithi181 podman[66037]: 2024-09-10 07:33:06.786092698 +0000 UTC m=+0.015996303 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:33:07.149 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:06 smithi181 podman[66037]: 2024-09-10 07:33:06.927237227 +0000 UTC m=+0.157140818 container create 7312fd87dd8b7ec9d2a1dffff09135f6ef1e22decdaebe4bd16e0db79ab1de60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T07:33:07.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:33:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:07.649 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:07 smithi181 podman[66037]: 2024-09-10 07:33:07.163336118 +0000 UTC m=+0.393239710 container init 7312fd87dd8b7ec9d2a1dffff09135f6ef1e22decdaebe4bd16e0db79ab1de60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T07:33:07.649 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:07 smithi181 podman[66037]: 2024-09-10 07:33:07.166220085 +0000 UTC m=+0.396123680 container start 7312fd87dd8b7ec9d2a1dffff09135f6ef1e22decdaebe4bd16e0db79ab1de60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T07:33:07.649 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:07 smithi181 ceph-mds[66056]: -- 172.21.15.181:0/2073905100 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d408b1b860 con 0x55d408af8800 2024-09-10T07:33:07.650 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c[66052]: starting mds.c at 2024-09-10T07:33:07.650 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:07 smithi181 bash[66037]: 7312fd87dd8b7ec9d2a1dffff09135f6ef1e22decdaebe4bd16e0db79ab1de60 2024-09-10T07:33:07.650 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:33:07 smithi181 systemd[1]: Started Ceph mds.c for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 -- 172.21.15.46:0/3744357000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c102790 msgr2=0x7f510c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 --2- 172.21.15.46:0/3744357000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c102790 0x7f510c1067e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009860 tx=0x7f50f802a340 comp rx=0 tx=0).stop 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 -- 172.21.15.46:0/3744357000 shutdown_connections 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 --2- 172.21.15.46:0/3744357000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c106f10 0x7f510c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 --2- 172.21.15.46:0/3744357000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c102790 0x7f510c1067e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 --2- 172.21.15.46:0/3744357000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f510c101de0 0x7f510c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 -- 172.21.15.46:0/3744357000 >> 172.21.15.46:0/3744357000 conn(0x7f510c0fd8c0 msgr2=0x7f510c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 -- 172.21.15.46:0/3744357000 shutdown_connections 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.170+0000 7f5113000640 1 -- 172.21.15.46:0/3744357000 wait complete. 2024-09-10T07:33:08.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.171+0000 7f5113000640 1 Processor -- start 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.171+0000 7f5113000640 1 -- start start 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5113000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c101de0 0x7f510c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5113000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c102790 0x7f510c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5113000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f510c106f10 0x7f510c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5113000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f510c111f30 con 0x7f510c106f10 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5113000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f510c111db0 con 0x7f510c101de0 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5113000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f510c1120b0 con 0x7f510c102790 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5111000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f510c106f10 0x7f510c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5111000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f510c106f10 0x7f510c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52134/0 (socket says 172.21.15.46:52134) 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f5111000640 1 -- 172.21.15.46:0/137757208 learned_addr learned my addr 172.21.15.46:0/137757208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:08.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f510be00640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c101de0 0x7f510c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.172+0000 7f510b400640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c102790 0x7f510c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5111000640 1 -- 172.21.15.46:0/137757208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c102790 msgr2=0x7f510c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5111000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c102790 0x7f510c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5111000640 1 -- 172.21.15.46:0/137757208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c101de0 msgr2=0x7f510c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5111000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c101de0 0x7f510c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5111000640 1 -- 172.21.15.46:0/137757208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1a68f0 con 0x7f510c106f10 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f510be00640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c101de0 0x7f510c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5111000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f510c106f10 0x7f510c074f70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f510400eb10 tx=0x7f510400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5108c00640 1 -- 172.21.15.46:0/137757208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104004b20 con 0x7f510c106f10 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5108c00640 1 -- 172.21.15.46:0/137757208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5104004cc0 con 0x7f510c106f10 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c1a6be0 con 0x7f510c106f10 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5108c00640 1 -- 172.21.15.46:0/137757208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104010760 con 0x7f510c106f10 2024-09-10T07:33:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.173+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1a70f0 con 0x7f510c106f10 2024-09-10T07:33:08.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.174+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c1021c0 con 0x7f510c106f10 2024-09-10T07:33:08.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.176+0000 7f5108c00640 1 -- 172.21.15.46:0/137757208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5104005020 con 0x7f510c106f10 2024-09-10T07:33:08.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.178+0000 7f5108c00640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50f00a2c00 0x7f50f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:08.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.179+0000 7f5108c00640 1 -- 172.21.15.46:0/137757208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f51040cf0d0 con 0x7f510c106f10 2024-09-10T07:33:08.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.179+0000 7f510be00640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50f00a2c00 0x7f50f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:08.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.179+0000 7f510be00640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50f00a2c00 0x7f50f00a50c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0096c0 tx=0x7f50fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:08.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.183+0000 7f5108c00640 1 -- 172.21.15.46:0/137757208 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f510409c430 con 0x7f510c106f10 2024-09-10T07:33:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] up:boot 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: fsmap 7 up:standby 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:08.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:08 smithi063 ceph-mon[35020]: Deploying daemon mds.h on smithi063 2024-09-10T07:33:08.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.405+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f510c10a0f0 con 0x7f510c106f10 2024-09-10T07:33:08.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:08.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] up:boot 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: fsmap 7 up:standby 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:08.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:08 smithi046 ceph-mon[30705]: Deploying daemon mds.h on smithi063 2024-09-10T07:33:08.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.405+0000 7f5108c00640 1 -- 172.21.15.46:0/137757208 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6385 (secure 0 0 0) 0x7f510409bd30 con 0x7f510c106f10 2024-09-10T07:33:08.407 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:08.407 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-10T07:33:07:262973+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T07:33:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-10T07:33:08.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50f00a2c00 msgr2=0x7f50f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50f00a2c00 0x7f50f00a50c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0096c0 tx=0x7f50fc009260 comp rx=0 tx=0).stop 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f510c106f10 msgr2=0x7f510c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f510c106f10 0x7f510c074f70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f510400eb10 tx=0x7f510400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 shutdown_connections 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50f00a2c00 0x7f50f00a50c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f510c106f10 0x7f510c074f70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c102790 0x7f510c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 --2- 172.21.15.46:0/137757208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f510c101de0 0x7f510c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 >> 172.21.15.46:0/137757208 conn(0x7f510c0fd8c0 msgr2=0x7f510c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 shutdown_connections 2024-09-10T07:33:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:08.408+0000 7f5113000640 1 -- 172.21.15.46:0/137757208 wait complete. 2024-09-10T07:33:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] up:boot 2024-09-10T07:33:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: fsmap 7 up:standby 2024-09-10T07:33:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T07:33:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:08.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:08 smithi181 ceph-mon[35397]: Deploying daemon mds.h on smithi063 2024-09-10T07:33:09.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:09 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/137757208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:09.401 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 systemd[1]: Starting Ceph mds.h for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:33:09.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:09 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/137757208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:09.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:09 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/137757208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:09.793 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 podman[65151]: 2024-09-10 07:33:09.442143192 +0000 UTC m=+0.015795300 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:33:09.793 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 podman[65151]: 2024-09-10 07:33:09.549505073 +0000 UTC m=+0.123157245 container create f9f5f8b6fbfa60690ac7ed4c3695807dd0ed42d388a9fbdd26943cfbc78f7f7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:33:09.793 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 podman[65151]: 2024-09-10 07:33:09.793827293 +0000 UTC m=+0.367479401 container init f9f5f8b6fbfa60690ac7ed4c3695807dd0ed42d388a9fbdd26943cfbc78f7f7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T07:33:10.150 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 podman[65151]: 2024-09-10 07:33:09.796767851 +0000 UTC m=+0.370419961 container start f9f5f8b6fbfa60690ac7ed4c3695807dd0ed42d388a9fbdd26943cfbc78f7f7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T07:33:10.150 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 ceph-mds[65171]: -- 172.21.15.63:0/1754066871 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b57b67d860 con 0x55b57b65b400 2024-09-10T07:33:10.150 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h[65167]: starting mds.h at 2024-09-10T07:33:10.151 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 bash[65151]: f9f5f8b6fbfa60690ac7ed4c3695807dd0ed42d388a9fbdd26943cfbc78f7f7e 2024-09-10T07:33:10.151 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:33:09 smithi063 systemd[1]: Started Ceph mds.h for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:10.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:10 smithi181 ceph-mon[35397]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:10.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:10 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:10 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:10 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:10 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:10.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:10 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:10.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:10 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:10 smithi063 ceph-mon[35020]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:10 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:10 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:10 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:10 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:10 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:10 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:10.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:10 smithi046 ceph-mon[30705]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:10.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:10 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:10 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:10 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:10.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:10 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:10.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:10 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:10.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:10 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:11.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:33:11.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:11 smithi046 ceph-mon[30705]: Deploying daemon mds.i on smithi181 2024-09-10T07:33:11.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:11 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6836/4195549127,v1:172.21.15.63:6837/4195549127] up:boot 2024-09-10T07:33:11.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:11 smithi046 ceph-mon[30705]: fsmap 8 up:standby 2024-09-10T07:33:11.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T07:33:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:11.623 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:11 smithi181 ceph-mon[35397]: Deploying daemon mds.i on smithi181 2024-09-10T07:33:11.623 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:11 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6836/4195549127,v1:172.21.15.63:6837/4195549127] up:boot 2024-09-10T07:33:11.623 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:11 smithi181 ceph-mon[35397]: fsmap 8 up:standby 2024-09-10T07:33:11.623 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T07:33:11.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:11 smithi063 ceph-mon[35020]: Deploying daemon mds.i on smithi181 2024-09-10T07:33:11.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:11 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6836/4195549127,v1:172.21.15.63:6837/4195549127] up:boot 2024-09-10T07:33:11.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:11 smithi063 ceph-mon[35020]: fsmap 8 up:standby 2024-09-10T07:33:11.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T07:33:12.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 -- 172.21.15.46:0/2683226801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8112a50 msgr2=0x7f93e8114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:12.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 --2- 172.21.15.46:0/2683226801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8112a50 0x7f93e8114e40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009ca0 tx=0x7f93e402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 -- 172.21.15.46:0/2683226801 shutdown_connections 2024-09-10T07:33:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 --2- 172.21.15.46:0/2683226801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8112a50 0x7f93e8114e40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 --2- 172.21.15.46:0/2683226801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e8077040 0x7f93e8112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 --2- 172.21.15.46:0/2683226801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93e8076720 0x7f93e8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 -- 172.21.15.46:0/2683226801 >> 172.21.15.46:0/2683226801 conn(0x7f93e80ffa50 msgr2=0x7f93e8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 -- 172.21.15.46:0/2683226801 shutdown_connections 2024-09-10T07:33:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.213+0000 7f93f0800640 1 -- 172.21.15.46:0/2683226801 wait complete. 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93f0800640 1 Processor -- start 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93f0800640 1 -- start start 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93f0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8076720 0x7f93e819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93f0800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93e8077040 0x7f93e819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93f0800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e8112a50 0x7f93e81a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93f0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81185e0 con 0x7f93e8076720 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93f0800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8118460 con 0x7f93e8077040 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93f0800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8118760 con 0x7f93e8112a50 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.214+0000 7f93ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8076720 0x7f93e819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8076720 0x7f93e819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52148/0 (socket says 172.21.15.46:52148) 2024-09-10T07:33:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ede00640 1 -- 172.21.15.46:0/1961680774 learned_addr learned my addr 172.21.15.46:0/1961680774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ed400640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93e8077040 0x7f93e819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ee800640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e8112a50 0x7f93e81a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ede00640 1 -- 172.21.15.46:0/1961680774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e8112a50 msgr2=0x7f93e81a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ede00640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e8112a50 0x7f93e81a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ede00640 1 -- 172.21.15.46:0/1961680774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93e8077040 msgr2=0x7f93e819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ede00640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93e8077040 0x7f93e819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ede00640 1 -- 172.21.15.46:0/1961680774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e81a3bb0 con 0x7f93e8076720 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.215+0000 7f93ede00640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8076720 0x7f93e819eb80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9f0 tx=0x7f93dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.216+0000 7f93e2a00640 1 -- 172.21.15.46:0/1961680774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00cd80 con 0x7f93e8076720 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.216+0000 7f93e2a00640 1 -- 172.21.15.46:0/1961680774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93dc010910 con 0x7f93e8076720 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.216+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81a3ea0 con 0x7f93e8076720 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.216+0000 7f93e2a00640 1 -- 172.21.15.46:0/1961680774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc018990 con 0x7f93e8076720 2024-09-10T07:33:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.216+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81ab730 con 0x7f93e8076720 2024-09-10T07:33:12.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.217+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e8076b00 con 0x7f93e8076720 2024-09-10T07:33:12.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.219+0000 7f93e2a00640 1 -- 172.21.15.46:0/1961680774 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f93dc010ac0 con 0x7f93e8076720 2024-09-10T07:33:12.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.220+0000 7f93e2a00640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93d00a2c00 0x7f93d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:12.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.220+0000 7f93e2a00640 1 -- 172.21.15.46:0/1961680774 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f93dc0ceeb0 con 0x7f93e8076720 2024-09-10T07:33:12.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.220+0000 7f93ed400640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93d00a2c00 0x7f93d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:12.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.221+0000 7f93ed400640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93d00a2c00 0x7f93d00a50c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f93e8079360 tx=0x7f93d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:12.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.225+0000 7f93e2a00640 1 -- 172.21.15.46:0/1961680774 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93dc09bbc0 con 0x7f93e8076720 2024-09-10T07:33:12.288 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:33:12 smithi181 podman[66339]: 2024-09-10 07:33:12.252072959 +0000 UTC m=+0.137003806 container create 16f85f0e612f44ab45b8d133fbf2a6b1e50f66c6dbadab610d040b5ac1f62fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:33:12.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:12 smithi046 ceph-mon[30705]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.452+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93e819fee0 con 0x7f93e8076720 2024-09-10T07:33:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.453+0000 7f93e2a00640 1 -- 172.21.15.46:0/1961680774 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7210 (secure 0 0 0) 0x7f93dc09fec0 con 0x7f93e8076720 2024-09-10T07:33:12.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:12.454 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"btime":"2024-09-10T07:33:10:275947+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T07:33:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-09-10T07:33:12.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93d00a2c00 msgr2=0x7f93d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:12.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93d00a2c00 0x7f93d00a50c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f93e8079360 tx=0x7f93d800b390 comp rx=0 tx=0).stop 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8076720 msgr2=0x7f93e819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8076720 0x7f93e819eb80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9f0 tx=0x7f93dc00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 shutdown_connections 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e8112a50 0x7f93e81a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93e8077040 0x7f93e819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93d00a2c00 0x7f93d00a50c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 --2- 172.21.15.46:0/1961680774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8076720 0x7f93e819eb80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 >> 172.21.15.46:0/1961680774 conn(0x7f93e80ffa50 msgr2=0x7f93e8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 shutdown_connections 2024-09-10T07:33:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:12.456+0000 7f93f0800640 1 -- 172.21.15.46:0/1961680774 wait complete. 2024-09-10T07:33:12.546 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:12 smithi181 ceph-mon[35397]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:12.546 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:33:12 smithi181 podman[66339]: 2024-09-10 07:33:12.455020689 +0000 UTC m=+0.339951494 container init 16f85f0e612f44ab45b8d133fbf2a6b1e50f66c6dbadab610d040b5ac1f62fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T07:33:12.546 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:33:12 smithi181 podman[66339]: 2024-09-10 07:33:12.457984341 +0000 UTC m=+0.342915147 container start 16f85f0e612f44ab45b8d133fbf2a6b1e50f66c6dbadab610d040b5ac1f62fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T07:33:12.546 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:33:12 smithi181 ceph-mds[66359]: -- 172.21.15.181:0/3950214145 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7f6ff1860 con 0x55e7f6fcf400 2024-09-10T07:33:12.546 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:33:12 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i[66355]: starting mds.i at 2024-09-10T07:33:12.546 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:33:12 smithi181 bash[66339]: 16f85f0e612f44ab45b8d133fbf2a6b1e50f66c6dbadab610d040b5ac1f62fb5 2024-09-10T07:33:12.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:12 smithi063 ceph-mon[35020]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:12.899 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:33:12 smithi181 systemd[1]: Started Ceph mds.i for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:13.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:13 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1961680774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:13.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.469 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.469 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:13.469 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:13.469 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:13.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:13 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1961680774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:13.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:13 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1961680774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:13.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:13.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:14 smithi181 ceph-mon[35397]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:14 smithi181 ceph-mon[35397]: Deploying daemon mds.g on smithi046 2024-09-10T07:33:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:14 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] up:boot 2024-09-10T07:33:14.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:14 smithi181 ceph-mon[35397]: fsmap 9 up:standby 2024-09-10T07:33:14.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T07:33:14.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:14 smithi063 ceph-mon[35020]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:14.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:14 smithi063 ceph-mon[35020]: Deploying daemon mds.g on smithi046 2024-09-10T07:33:14.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:14 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] up:boot 2024-09-10T07:33:14.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:14 smithi063 ceph-mon[35020]: fsmap 9 up:standby 2024-09-10T07:33:14.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T07:33:14.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:14 smithi046 ceph-mon[30705]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:14.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:14 smithi046 ceph-mon[30705]: Deploying daemon mds.g on smithi046 2024-09-10T07:33:14.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:14 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] up:boot 2024-09-10T07:33:14.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:14 smithi046 ceph-mon[30705]: fsmap 9 up:standby 2024-09-10T07:33:14.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T07:33:15.234 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:33:15.809 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:15 smithi046 systemd[1]: Starting Ceph mds.g for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:33:16.144 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:15 smithi046 podman[68178]: 2024-09-10 07:33:15.713902508 +0000 UTC m=+0.029413875 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:33:16.144 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:15 smithi046 podman[68178]: 2024-09-10 07:33:15.831081573 +0000 UTC m=+0.146592929 container create 186a8a12222d9d52284eb1a7f734ad645157b281e81e29861517b5817a83609a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T07:33:16.144 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:16 smithi046 podman[68178]: 2024-09-10 07:33:16.144784671 +0000 UTC m=+0.460296028 container init 186a8a12222d9d52284eb1a7f734ad645157b281e81e29861517b5817a83609a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T07:33:16.344 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:16 smithi046 ceph-mon[30705]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:16.409 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:16 smithi046 podman[68178]: 2024-09-10 07:33:16.151385801 +0000 UTC m=+0.466897158 container start 186a8a12222d9d52284eb1a7f734ad645157b281e81e29861517b5817a83609a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T07:33:16.409 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:16 smithi046 ceph-mds[68214]: -- 172.21.15.46:0/244936004 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564855273860 con 0x564855251400 2024-09-10T07:33:16.409 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:16 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g[68210]: starting mds.g at 2024-09-10T07:33:16.409 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:16 smithi046 bash[68178]: 186a8a12222d9d52284eb1a7f734ad645157b281e81e29861517b5817a83609a 2024-09-10T07:33:16.409 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:33:16 smithi046 systemd[1]: Started Ceph mds.g for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:16 smithi181 ceph-mon[35397]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:16 smithi063 ceph-mon[35020]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 -- 172.21.15.46:0/3889879051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac102360 msgr2=0x7f1bac10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3889879051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac102360 0x7f1bac10e890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009cd0 tx=0x7f1ba402a8f0 comp rx=0 tx=0).stop 2024-09-10T07:33:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 -- 172.21.15.46:0/3889879051 shutdown_connections 2024-09-10T07:33:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3889879051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac102360 0x7f1bac10e890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3889879051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bac1019c0 0x7f1bac101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:16.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3889879051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bac1079c0 0x7f1bac107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 -- 172.21.15.46:0/3889879051 >> 172.21.15.46:0/3889879051 conn(0x7f1bac0fd740 msgr2=0x7f1bac0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 -- 172.21.15.46:0/3889879051 shutdown_connections 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.915+0000 7f1bb2000640 1 -- 172.21.15.46:0/3889879051 wait complete. 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.916+0000 7f1bb2000640 1 Processor -- start 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.916+0000 7f1bb2000640 1 -- start start 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac1019c0 0x7f1bac19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bac102360 0x7f1bac19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bac1079c0 0x7f1bac19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac111dc0 con 0x7f1bac1019c0 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb2000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac111c40 con 0x7f1bac1079c0 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb2000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac111f40 con 0x7f1bac102360 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac1019c0 0x7f1bac19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:16.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac1019c0 0x7f1bac19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52032/0 (socket says 172.21.15.46:52032) 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3357163821 learned_addr learned my addr 172.21.15.46:0/3357163821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1babe00640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bac102360 0x7f1bac19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb1600640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bac1079c0 0x7f1bac19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3357163821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bac102360 msgr2=0x7f1bac19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bac102360 0x7f1bac19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3357163821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bac1079c0 msgr2=0x7f1bac19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.917+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bac1079c0 0x7f1bac19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.918+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3357163821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac19f480 con 0x7f1bac1019c0 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.918+0000 7f1bb1600640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bac1079c0 0x7f1bac19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.918+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac1019c0 0x7f1bac19a4d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000e9f0 tx=0x7f1ba000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.918+0000 7f1ba9600640 1 -- 172.21.15.46:0/3357163821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000cd80 con 0x7f1bac1019c0 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.918+0000 7f1ba9600640 1 -- 172.21.15.46:0/3357163821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba0010910 con 0x7f1bac1019c0 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.918+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bac19f770 con 0x7f1bac1019c0 2024-09-10T07:33:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.918+0000 7f1ba9600640 1 -- 172.21.15.46:0/3357163821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0018990 con 0x7f1bac1019c0 2024-09-10T07:33:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.918+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bac1a7050 con 0x7f1bac1019c0 2024-09-10T07:33:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.919+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bac101e20 con 0x7f1bac1019c0 2024-09-10T07:33:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.921+0000 7f1ba9600640 1 -- 172.21.15.46:0/3357163821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1ba0010ac0 con 0x7f1bac1019c0 2024-09-10T07:33:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.923+0000 7f1ba9600640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1b940a2b80 0x7f1b940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.924+0000 7f1ba9600640 1 -- 172.21.15.46:0/3357163821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f1ba00cef80 con 0x7f1bac1019c0 2024-09-10T07:33:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.924+0000 7f1babe00640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1b940a2b80 0x7f1b940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.924+0000 7f1babe00640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1b940a2b80 0x7f1b940a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1bac10e5e0 tx=0x7f1b9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:16.928+0000 7f1ba9600640 1 -- 172.21.15.46:0/3357163821 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ba009c2e0 con 0x7f1bac1019c0 2024-09-10T07:33:17.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.154+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bac1a72f0 con 0x7f1bac1019c0 2024-09-10T07:33:17.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.155+0000 7f1ba9600640 1 -- 172.21.15.46:0/3357163821 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8866 (secure 0 0 0) 0x7f1ba009bbe0 con 0x7f1bac1019c0 2024-09-10T07:33:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:17.155 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":11,"btime":"2024-09-10T07:33:16:299535+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T07:33:17.155 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 11 2024-09-10T07:33:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1b940a2b80 msgr2=0x7f1b940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1b940a2b80 0x7f1b940a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1bac10e5e0 tx=0x7f1b9800b410 comp rx=0 tx=0).stop 2024-09-10T07:33:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac1019c0 msgr2=0x7f1bac19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac1019c0 0x7f1bac19a4d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000e9f0 tx=0x7f1ba000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:33:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 shutdown_connections 2024-09-10T07:33:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bac1079c0 0x7f1bac19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bac102360 0x7f1bac19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1b940a2b80 0x7f1b940a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 --2- 172.21.15.46:0/3357163821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac1019c0 0x7f1bac19a4d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.157+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 >> 172.21.15.46:0/3357163821 conn(0x7f1bac0fd740 msgr2=0x7f1bac10d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.158+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 shutdown_connections 2024-09-10T07:33:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:17.158+0000 7f1bb2000640 1 -- 172.21.15.46:0/3357163821 wait complete. 2024-09-10T07:33:17.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.46:6838/1424921396,v1:172.21.15.46:6839/1424921396] up:boot 2024-09-10T07:33:17.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: fsmap 10 up:standby 2024-09-10T07:33:17.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T07:33:17.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: Deploying daemon mds.a on smithi046 2024-09-10T07:33:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:17 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3357163821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:17.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.46:6838/1424921396,v1:172.21.15.46:6839/1424921396] up:boot 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: fsmap 10 up:standby 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: Deploying daemon mds.a on smithi046 2024-09-10T07:33:17.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:17 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3357163821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.46:6838/1424921396,v1:172.21.15.46:6839/1424921396] up:boot 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: fsmap 10 up:standby 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: Deploying daemon mds.a on smithi046 2024-09-10T07:33:17.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:17 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3357163821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:18 smithi181 ceph-mon[35397]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:18 smithi063 ceph-mon[35020]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:18.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:18 smithi046 ceph-mon[30705]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:20.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:33:20.147 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:19 smithi046 systemd[1]: Starting Ceph mds.a for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:33:20.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:20 smithi046 ceph-mon[30705]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:20.410 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:20 smithi046 podman[68561]: 2024-09-10 07:33:20.053221272 +0000 UTC m=+0.028090871 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:33:20.410 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:20 smithi046 podman[68561]: 2024-09-10 07:33:20.179587847 +0000 UTC m=+0.154457426 container create 4f93e67e290059307036bc47b384e25f1c7d3eadf5b4379153f07b4414a8a568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:33:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:20.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:20 smithi181 ceph-mon[35397]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:20.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:20 smithi063 ceph-mon[35020]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:20.769 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:20 smithi046 podman[68561]: 2024-09-10 07:33:20.42852512 +0000 UTC m=+0.403394725 container init 4f93e67e290059307036bc47b384e25f1c7d3eadf5b4379153f07b4414a8a568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T07:33:20.769 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:20 smithi046 podman[68561]: 2024-09-10 07:33:20.435396807 +0000 UTC m=+0.410266394 container start 4f93e67e290059307036bc47b384e25f1c7d3eadf5b4379153f07b4414a8a568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T07:33:20.769 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:20 smithi046 ceph-mds[68610]: -- 172.21.15.46:0/3215976728 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c132637c20 con 0x55c132615000 2024-09-10T07:33:20.769 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:20 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: starting mds.a at 2024-09-10T07:33:20.770 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:20 smithi046 bash[68561]: 4f93e67e290059307036bc47b384e25f1c7d3eadf5b4379153f07b4414a8a568 2024-09-10T07:33:20.770 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:20 smithi046 systemd[1]: Started Ceph mds.a for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 -- 172.21.15.46:0/2521710161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c109120 msgr2=0x7f808c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 --2- 172.21.15.46:0/2521710161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c109120 0x7f808c110ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8084009ca0 tx=0x7f808402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 -- 172.21.15.46:0/2521710161 shutdown_connections 2024-09-10T07:33:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 --2- 172.21.15.46:0/2521710161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c109120 0x7f808c110ca0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 --2- 172.21.15.46:0/2521710161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c1049a0 0x7f808c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 --2- 172.21.15.46:0/2521710161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f808c103ff0 0x7f808c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 -- 172.21.15.46:0/2521710161 >> 172.21.15.46:0/2521710161 conn(0x7f808c0ffa90 msgr2=0x7f808c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 -- 172.21.15.46:0/2521710161 shutdown_connections 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.238+0000 7f8093400640 1 -- 172.21.15.46:0/2521710161 wait complete. 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8093400640 1 Processor -- start 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8093400640 1 -- start start 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8093400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f808c103ff0 0x7f808c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8093400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c1049a0 0x7f808c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8093400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c109120 0x7f808c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8093400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f808c114150 con 0x7f808c109120 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8093400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f808c113fd0 con 0x7f808c1049a0 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8093400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1142d0 con 0x7f808c103ff0 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f808be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c1049a0 0x7f808c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f8091400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c109120 0x7f808c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f808be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c1049a0 0x7f808c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42880/0 (socket says 172.21.15.46:42880) 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.239+0000 7f808be00640 1 -- 172.21.15.46:0/801517290 learned_addr learned my addr 172.21.15.46:0/801517290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f8090a00640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f808c103ff0 0x7f808c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f808be00640 1 -- 172.21.15.46:0/801517290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f808c103ff0 msgr2=0x7f808c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f808be00640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f808c103ff0 0x7f808c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f808be00640 1 -- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c109120 msgr2=0x7f808c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f808be00640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c109120 0x7f808c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f808be00640 1 -- 172.21.15.46:0/801517290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c19f5e0 con 0x7f808c1049a0 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f8090a00640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f808c103ff0 0x7f808c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f808be00640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c1049a0 0x7f808c19ab70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f807800d950 tx=0x7f807800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f8089600640 1 -- 172.21.15.46:0/801517290 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078017030 con 0x7f808c1049a0 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c19f8d0 con 0x7f808c1049a0 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f8089600640 1 -- 172.21.15.46:0/801517290 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8078004020 con 0x7f808c1049a0 2024-09-10T07:33:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.240+0000 7f8089600640 1 -- 172.21.15.46:0/801517290 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80780047a0 con 0x7f808c1049a0 2024-09-10T07:33:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.241+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c1a71b0 con 0x7f808c1049a0 2024-09-10T07:33:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.242+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c1043d0 con 0x7f808c1049a0 2024-09-10T07:33:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.243+0000 7f8089600640 1 -- 172.21.15.46:0/801517290 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f80780041d0 con 0x7f808c1049a0 2024-09-10T07:33:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.245+0000 7f8089600640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80740a2b80 0x7f80740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.245+0000 7f8089600640 1 -- 172.21.15.46:0/801517290 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f80780ce5c0 con 0x7f808c1049a0 2024-09-10T07:33:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.245+0000 7f8090a00640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80740a2b80 0x7f80740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.245+0000 7f8090a00640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80740a2b80 0x7f80740a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8080009c20 tx=0x7f8080005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.249+0000 7f8089600640 1 -- 172.21.15.46:0/801517290 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f807809b920 con 0x7f808c1049a0 2024-09-10T07:33:21.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.476+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f808c1a7410 con 0x7f808c1049a0 2024-09-10T07:33:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.477+0000 7f8089600640 1 -- 172.21.15.46:0/801517290 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9692 (secure 0 0 0) 0x7f807809b220 con 0x7f808c1049a0 2024-09-10T07:33:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:21.478 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 12 2024-09-10T07:33:21.478 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":12,"btime":"2024-09-10T07:33:21:369300+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-09-10T07:33:21.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80740a2b80 msgr2=0x7f80740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:21.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80740a2b80 0x7f80740a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8080009c20 tx=0x7f8080005f90 comp rx=0 tx=0).stop 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c1049a0 msgr2=0x7f808c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c1049a0 0x7f808c19ab70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f807800d950 tx=0x7f807800de10 comp rx=0 tx=0).stop 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 shutdown_connections 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80740a2b80 0x7f80740a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808c109120 0x7f808c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c1049a0 0x7f808c19ab70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 --2- 172.21.15.46:0/801517290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f808c103ff0 0x7f808c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 >> 172.21.15.46:0/801517290 conn(0x7f808c0ffa90 msgr2=0x7f808c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 shutdown_connections 2024-09-10T07:33:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:21.480+0000 7f8093400640 1 -- 172.21.15.46:0/801517290 wait complete. 2024-09-10T07:33:21.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:21.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:21.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:21.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:21.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: Deploying daemon mds.b on smithi063 2024-09-10T07:33:21.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] up:boot 2024-09-10T07:33:21.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: fsmap 11 up:standby 2024-09-10T07:33:21.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T07:33:21.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:21 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/801517290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:21.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:21.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: Deploying daemon mds.b on smithi063 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] up:boot 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: fsmap 11 up:standby 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T07:33:21.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:21 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/801517290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: Deploying daemon mds.b on smithi063 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] up:boot 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: fsmap 11 up:standby 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T07:33:21.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:21 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/801517290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:22.650 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:22 smithi063 systemd[1]: Starting Ceph mds.b for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:33:22.998 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:22 smithi063 podman[65453]: 2024-09-10 07:33:22.648752018 +0000 UTC m=+0.017558077 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:33:22.998 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:22 smithi063 podman[65453]: 2024-09-10 07:33:22.770870239 +0000 UTC m=+0.139676298 container create 5df5365253a993e106ba926d0b3d5daf0ffc30c6eece6318e40b3de8367b426c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:33:22.998 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:22 smithi063 podman[65453]: 2024-09-10 07:33:22.998937893 +0000 UTC m=+0.367743952 container init 5df5365253a993e106ba926d0b3d5daf0ffc30c6eece6318e40b3de8367b426c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T07:33:23.400 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:23 smithi063 podman[65453]: 2024-09-10 07:33:23.001865361 +0000 UTC m=+0.370671427 container start 5df5365253a993e106ba926d0b3d5daf0ffc30c6eece6318e40b3de8367b426c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T07:33:23.400 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:23 smithi063 ceph-mds[65472]: -- 172.21.15.63:0/1116927841 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55752235f860 con 0x55752233d000 2024-09-10T07:33:23.400 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:23 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b[65468]: starting mds.b at 2024-09-10T07:33:23.400 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:23 smithi063 bash[65453]: 5df5365253a993e106ba926d0b3d5daf0ffc30c6eece6318e40b3de8367b426c 2024-09-10T07:33:23.401 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:33:23 smithi063 systemd[1]: Started Ceph mds.b for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:33:24.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:33:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] up:boot 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: fsmap 12 up:standby 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:33:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] up:boot 2024-09-10T07:33:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: fsmap 12 up:standby 2024-09-10T07:33:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] up:boot 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: fsmap 12 up:standby 2024-09-10T07:33:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T07:33:24.916 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 -- 172.21.15.46:0/3308856735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 msgr2=0x7f9bc8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 --2- 172.21.15.46:0/3308856735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 0x7f9bc8107da0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009930 tx=0x7f9bb802a420 comp rx=0 tx=0).stop 2024-09-10T07:33:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 -- 172.21.15.46:0/3308856735 shutdown_connections 2024-09-10T07:33:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 --2- 172.21.15.46:0/3308856735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bc8102360 0x7f9bc810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 --2- 172.21.15.46:0/3308856735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc81019c0 0x7f9bc8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 --2- 172.21.15.46:0/3308856735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 0x7f9bc8107da0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 -- 172.21.15.46:0/3308856735 >> 172.21.15.46:0/3308856735 conn(0x7f9bc80fd740 msgr2=0x7f9bc80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 -- 172.21.15.46:0/3308856735 shutdown_connections 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.504+0000 7f9bcda00640 1 -- 172.21.15.46:0/3308856735 wait complete. 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.505+0000 7f9bcda00640 1 Processor -- start 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.505+0000 7f9bcda00640 1 -- start start 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bcda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bc81019c0 0x7f9bc80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bcda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8102360 0x7f9bc8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bcda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 0x7f9bc8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bcda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8111e10 con 0x7f9bc81079c0 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bcda00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8111c90 con 0x7f9bc81019c0 2024-09-10T07:33:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bcda00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8111f90 con 0x7f9bc8102360 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bc81019c0 0x7f9bc80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bc81019c0 0x7f9bc80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42896/0 (socket says 172.21.15.46:42896) 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bc7e00640 1 -- 172.21.15.46:0/600472542 learned_addr learned my addr 172.21.15.46:0/600472542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bcd000640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 0x7f9bc8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bbf400640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8102360 0x7f9bc8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bc7e00640 1 -- 172.21.15.46:0/600472542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8102360 msgr2=0x7f9bc8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8102360 0x7f9bc8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bc7e00640 1 -- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 msgr2=0x7f9bc8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 0x7f9bc8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bc7e00640 1 -- 172.21.15.46:0/600472542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc81a67d0 con 0x7f9bc81019c0 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.506+0000 7f9bcd000640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 0x7f9bc8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:33:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.507+0000 7f9bc7e00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bc81019c0 0x7f9bc80744f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bb802a940 tx=0x7f9bb8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.507+0000 7f9bc5600640 1 -- 172.21.15.46:0/600472542 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb803f070 con 0x7f9bc81019c0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.507+0000 7f9bc5600640 1 -- 172.21.15.46:0/600472542 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb8031c90 con 0x7f9bc81019c0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.507+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc81a6a60 con 0x7f9bc81019c0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.507+0000 7f9bc5600640 1 -- 172.21.15.46:0/600472542 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8035030 con 0x7f9bc81019c0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.507+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc81a6f40 con 0x7f9bc81019c0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.508+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc8101e20 con 0x7f9bc81019c0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.510+0000 7f9bc5600640 1 -- 172.21.15.46:0/600472542 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9bb8042070 con 0x7f9bc81019c0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.511+0000 7f9bc5600640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9bac0a2b80 0x7f9bac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.511+0000 7f9bc5600640 1 -- 172.21.15.46:0/600472542 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f9bb80ef5f0 con 0x7f9bc81019c0 2024-09-10T07:33:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.511+0000 7f9bbf400640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9bac0a2b80 0x7f9bac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.512+0000 7f9bbf400640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9bac0a2b80 0x7f9bac0a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8075f80 tx=0x7f9bb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.515+0000 7f9bc5600640 1 -- 172.21.15.46:0/600472542 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bb80bc950 con 0x7f9bc81019c0 2024-09-10T07:33:25.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.747+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bc8109f30 con 0x7f9bc81019c0 2024-09-10T07:33:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.748+0000 7f9bc5600640 1 -- 172.21.15.46:0/600472542 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7f9bb80bc250 con 0x7f9bc81019c0 2024-09-10T07:33:25.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:25.748 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"btime":"2024-09-10T07:33:23:633035+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-09-10T07:33:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2024-09-10T07:33:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9bac0a2b80 msgr2=0x7f9bac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9bac0a2b80 0x7f9bac0a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8075f80 tx=0x7f9bb0005f90 comp rx=0 tx=0).stop 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bc81019c0 msgr2=0x7f9bc80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bc81019c0 0x7f9bc80744f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bb802a940 tx=0x7f9bb8003b20 comp rx=0 tx=0).stop 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 shutdown_connections 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9bac0a2b80 0x7f9bac0a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc81079c0 0x7f9bc8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bc8102360 0x7f9bc8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 --2- 172.21.15.46:0/600472542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bc81019c0 0x7f9bc80744f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 >> 172.21.15.46:0/600472542 conn(0x7f9bc80fd740 msgr2=0x7f9bc80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 shutdown_connections 2024-09-10T07:33:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:25.751+0000 7f9bcda00640 1 -- 172.21.15.46:0/600472542 wait complete. 2024-09-10T07:33:26.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:25 smithi063 ceph-mon[35020]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:26.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:25 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/600472542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:26.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:25 smithi181 ceph-mon[35397]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:26.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:25 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/600472542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:26.157 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:25 smithi046 ceph-mon[30705]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:26.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:26.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:25 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/600472542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:26.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T07:33:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 -- 172.21.15.46:0/4164901068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 msgr2=0x7faa8c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 --2- 172.21.15.46:0/4164901068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 0x7faa8c1043d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009900 tx=0x7faa7c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 -- 172.21.15.46:0/4164901068 shutdown_connections 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 --2- 172.21.15.46:0/4164901068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c109120 0x7faa8c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 --2- 172.21.15.46:0/4164901068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa8c1049a0 0x7faa8c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 --2- 172.21.15.46:0/4164901068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 0x7faa8c1043d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 -- 172.21.15.46:0/4164901068 >> 172.21.15.46:0/4164901068 conn(0x7faa8c0ffa90 msgr2=0x7faa8c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 -- 172.21.15.46:0/4164901068 shutdown_connections 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.586+0000 7faa90e00640 1 -- 172.21.15.46:0/4164901068 wait complete. 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.587+0000 7faa90e00640 1 Processor -- start 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.587+0000 7faa90e00640 1 -- start start 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.587+0000 7faa90e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 0x7faa8c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa90e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa8c1049a0 0x7faa8c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa90e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c109120 0x7faa8c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa90e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c1141a0 con 0x7faa8c1049a0 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa90e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c114020 con 0x7faa8c103ff0 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa90e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c114320 con 0x7faa8c109120 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 0x7faa8c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 0x7faa8c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60612/0 (socket says 172.21.15.46:60612) 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89e00640 1 -- 172.21.15.46:0/2281697429 learned_addr learned my addr 172.21.15.46:0/2281697429 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89400640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa8c1049a0 0x7faa8c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa8a800640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c109120 0x7faa8c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89e00640 1 -- 172.21.15.46:0/2281697429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c109120 msgr2=0x7faa8c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c109120 0x7faa8c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89e00640 1 -- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa8c1049a0 msgr2=0x7faa8c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa8c1049a0 0x7faa8c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa89e00640 1 -- 172.21.15.46:0/2281697429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c19f5b0 con 0x7faa8c103ff0 2024-09-10T07:33:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.588+0000 7faa8a800640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c109120 0x7faa8c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.589+0000 7faa89e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 0x7faa8c19a600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faa7c004770 tx=0x7faa7c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.589+0000 7faa82a00640 1 -- 172.21.15.46:0/2281697429 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c03f070 con 0x7faa8c103ff0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.589+0000 7faa82a00640 1 -- 172.21.15.46:0/2281697429 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa7c031ca0 con 0x7faa8c103ff0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.589+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c19f840 con 0x7faa8c103ff0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.589+0000 7faa82a00640 1 -- 172.21.15.46:0/2281697429 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c035030 con 0x7faa8c103ff0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.589+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c1a7180 con 0x7faa8c103ff0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.590+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8c1043d0 con 0x7faa8c103ff0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.592+0000 7faa82a00640 1 -- 172.21.15.46:0/2281697429 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faa7c042070 con 0x7faa8c103ff0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.593+0000 7faa82a00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faa700a2c00 0x7faa700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.593+0000 7faa82a00640 1 -- 172.21.15.46:0/2281697429 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7faa7c0ef390 con 0x7faa8c103ff0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.593+0000 7faa89400640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faa700a2c00 0x7faa700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.594+0000 7faa89400640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faa700a2c00 0x7faa700a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faa780095f0 tx=0x7faa78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.595+0000 7faa82a00640 1 -- 172.21.15.46:0/2281697429 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa7c0bc160 con 0x7faa8c103ff0 2024-09-10T07:33:27.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:27 smithi181 ceph-mon[35397]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:27.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:27.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:33:27.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:33:27.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.788+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faa8c10c300 con 0x7faa8c103ff0 2024-09-10T07:33:27.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:27 smithi046 ceph-mon[30705]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:27.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:27.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:33:27.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:33:27.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.789+0000 7faa82a00640 1 -- 172.21.15.46:0/2281697429 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7faa7c0c0460 con 0x7faa8c103ff0 2024-09-10T07:33:27.789 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:27.789 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-10T07:33:27.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:33:27.789 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-10T07:33:27.789 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-10T07:33:27.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:33:27.789 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-10T07:33:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faa700a2c00 msgr2=0x7faa700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faa700a2c00 0x7faa700a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faa780095f0 tx=0x7faa78005f90 comp rx=0 tx=0).stop 2024-09-10T07:33:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 msgr2=0x7faa8c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 0x7faa8c19a600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faa7c004770 tx=0x7faa7c003c10 comp rx=0 tx=0).stop 2024-09-10T07:33:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 shutdown_connections 2024-09-10T07:33:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c109120 0x7faa8c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faa700a2c00 0x7faa700a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa8c1049a0 0x7faa8c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 --2- 172.21.15.46:0/2281697429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa8c103ff0 0x7faa8c19a600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:27.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.791+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 >> 172.21.15.46:0/2281697429 conn(0x7faa8c0ffa90 msgr2=0x7faa8c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:27.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.792+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 shutdown_connections 2024-09-10T07:33:27.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:27.792+0000 7faa90e00640 1 -- 172.21.15.46:0/2281697429 wait complete. 2024-09-10T07:33:27.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:27 smithi063 ceph-mon[35020]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:27.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:33:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:33:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:28.572 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T07:33:28.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T07:33:28.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:28 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2281697429' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T07:33:28.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:28.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:28.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:28.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:28 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.051 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:28 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2281697429' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T07:33:29.051 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.051 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.051 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.051 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:28 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:28 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2281697429' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T07:33:29.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:28 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:29.389 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 -- 172.21.15.46:0/1926760654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681034b0 msgr2=0x7f846810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 --2- 172.21.15.46:0/1926760654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681034b0 0x7f846810b9a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8454009900 tx=0x7f845402a480 comp rx=0 tx=0).stop 2024-09-10T07:33:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 -- 172.21.15.46:0/1926760654 shutdown_connections 2024-09-10T07:33:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 --2- 172.21.15.46:0/1926760654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f846810bfc0 0x7f846810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 --2- 172.21.15.46:0/1926760654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681034b0 0x7f846810b9a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 --2- 172.21.15.46:0/1926760654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8468102b90 0x7f8468102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 -- 172.21.15.46:0/1926760654 >> 172.21.15.46:0/1926760654 conn(0x7f84680fc610 msgr2=0x7f84680fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 -- 172.21.15.46:0/1926760654 shutdown_connections 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.979+0000 7f846cc00640 1 -- 172.21.15.46:0/1926760654 wait complete. 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.980+0000 7f846cc00640 1 Processor -- start 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.980+0000 7f846cc00640 1 -- start start 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f846cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8468102b90 0x7f846819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f846cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84681034b0 0x7f846819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f846cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f846810bfc0 0x7f846819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f846cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8468111e10 con 0x7f846810bfc0 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f846cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8468111c90 con 0x7f84681034b0 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f846cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8468111f90 con 0x7f8468102b90 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8468102b90 0x7f846819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8468102b90 0x7f846819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:42592/0 (socket says 172.21.15.46:42592) 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467400640 1 -- 172.21.15.46:0/530798470 learned_addr learned my addr 172.21.15.46:0/530798470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:29.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467e00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f846810bfc0 0x7f846819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8466a00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84681034b0 0x7f846819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467400640 1 -- 172.21.15.46:0/530798470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84681034b0 msgr2=0x7f846819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467400640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84681034b0 0x7f846819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467400640 1 -- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f846810bfc0 msgr2=0x7f846819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467400640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f846810bfc0 0x7f846819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8467400640 1 -- 172.21.15.46:0/530798470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846819f470 con 0x7f8468102b90 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.981+0000 7f8466a00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84681034b0 0x7f846819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.982+0000 7f8467400640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8468102b90 0x7f846819a4c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f845800e9f0 tx=0x7f845800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.982+0000 7f845fe00640 1 -- 172.21.15.46:0/530798470 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845800cd80 con 0x7f8468102b90 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.982+0000 7f845fe00640 1 -- 172.21.15.46:0/530798470 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8458010910 con 0x7f8468102b90 2024-09-10T07:33:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.982+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846819f760 con 0x7f8468102b90 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.982+0000 7f845fe00640 1 -- 172.21.15.46:0/530798470 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580188f0 con 0x7f8468102b90 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.982+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84681a7040 con 0x7f8468102b90 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.983+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8468102f70 con 0x7f8468102b90 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.985+0000 7f845fe00640 1 -- 172.21.15.46:0/530798470 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8458010ac0 con 0x7f8468102b90 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.986+0000 7f845fe00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f844c0a2b80 0x7f844c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.986+0000 7f845fe00640 1 -- 172.21.15.46:0/530798470 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f84580cf080 con 0x7f8468102b90 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.986+0000 7f8466a00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f844c0a2b80 0x7f844c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.987+0000 7f8466a00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f844c0a2b80 0x7f844c0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:29.990+0000 7f845fe00640 1 -- 172.21.15.46:0/530798470 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f845809c3e0 con 0x7f8468102b90 2024-09-10T07:33:30.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.212+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f84681a73a0 con 0x7f8468102b90 2024-09-10T07:33:30.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.830+0000 7f845fe00640 1 -- 172.21.15.46:0/530798470 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f845809bce0 con 0x7f8468102b90 2024-09-10T07:33:30.831 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-10T07:33:30.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.836+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f844c0a2b80 msgr2=0x7f844c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:30.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.836+0000 7f846cc00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f844c0a2b80 0x7f844c0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454005a40 comp rx=0 tx=0).stop 2024-09-10T07:33:30.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.836+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8468102b90 msgr2=0x7f846819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:30.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.836+0000 7f846cc00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8468102b90 0x7f846819a4c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f845800e9f0 tx=0x7f845800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:33:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.837+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 shutdown_connections 2024-09-10T07:33:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.837+0000 7f846cc00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f844c0a2b80 0x7f844c0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.837+0000 7f846cc00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f846810bfc0 0x7f846819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.837+0000 7f846cc00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84681034b0 0x7f846819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.837+0000 7f846cc00640 1 --2- 172.21.15.46:0/530798470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8468102b90 0x7f846819a4c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.837+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 >> 172.21.15.46:0/530798470 conn(0x7f84680fc610 msgr2=0x7f84680fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.838+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 shutdown_connections 2024-09-10T07:33:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:30.838+0000 7f846cc00640 1 -- 172.21.15.46:0/530798470 wait complete. 2024-09-10T07:33:31.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:30 smithi181 ceph-mon[35397]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:31.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:30 smithi063 ceph-mon[35020]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:31.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:30 smithi046 ceph-mon[30705]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:31.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T07:33:32.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/530798470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T07:33:32.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:31 smithi181 ceph-mon[35397]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/530798470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T07:33:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T07:33:32.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:32.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:32.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:33:32.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T07:33:32.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:31 smithi063 ceph-mon[35020]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T07:33:32.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/530798470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T07:33:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:31 smithi046 ceph-mon[30705]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T07:33:32.196 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 -- 172.21.15.46:0/2519924734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0106f10 msgr2=0x7f92d010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 --2- 172.21.15.46:0/2519924734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0106f10 0x7f92d010ea90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009be0 tx=0x7f92c802a6c0 comp rx=0 tx=0).stop 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 -- 172.21.15.46:0/2519924734 shutdown_connections 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 --2- 172.21.15.46:0/2519924734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0106f10 0x7f92d010ea90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 --2- 172.21.15.46:0/2519924734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d0102790 0x7f92d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 --2- 172.21.15.46:0/2519924734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0101de0 0x7f92d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 -- 172.21.15.46:0/2519924734 >> 172.21.15.46:0/2519924734 conn(0x7f92d00fd8c0 msgr2=0x7f92d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 -- 172.21.15.46:0/2519924734 shutdown_connections 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.826+0000 7f92d4c00640 1 -- 172.21.15.46:0/2519924734 wait complete. 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.827+0000 7f92d4c00640 1 Processor -- start 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.827+0000 7f92d4c00640 1 -- start start 2024-09-10T07:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.827+0000 7f92d4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0101de0 0x7f92d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.827+0000 7f92d4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d0102790 0x7f92d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.827+0000 7f92d4c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0106f10 0x7f92d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.827+0000 7f92d4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0111f70 con 0x7f92d0102790 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.827+0000 7f92d4c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0111df0 con 0x7f92d0101de0 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.827+0000 7f92d4c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01120f0 con 0x7f92d0106f10 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d0102790 0x7f92d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d0102790 0x7f92d0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59930/0 (socket says 172.21.15.46:59930) 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 -- 172.21.15.46:0/541307936 learned_addr learned my addr 172.21.15.46:0/541307936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92cde00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0101de0 0x7f92d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92ce800640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0106f10 0x7f92d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 -- 172.21.15.46:0/541307936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0106f10 msgr2=0x7f92d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0106f10 0x7f92d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 -- 172.21.15.46:0/541307936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0101de0 msgr2=0x7f92d00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0101de0 0x7f92d00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 -- 172.21.15.46:0/541307936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d01a6880 con 0x7f92d0102790 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92cde00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0101de0 0x7f92d00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.828+0000 7f92c5400640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d0102790 0x7f92d0074a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f92b800e9c0 tx=0x7f92b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.829+0000 7f92c7400640 1 -- 172.21.15.46:0/541307936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b800cde0 con 0x7f92d0102790 2024-09-10T07:33:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.829+0000 7f92c7400640 1 -- 172.21.15.46:0/541307936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92b80035f0 con 0x7f92d0102790 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.829+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d01a6b70 con 0x7f92d0102790 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.829+0000 7f92c7400640 1 -- 172.21.15.46:0/541307936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8010650 con 0x7f92d0102790 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.829+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d01a70b0 con 0x7f92d0102790 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.830+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d01021c0 con 0x7f92d0102790 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.833+0000 7f92c7400640 1 -- 172.21.15.46:0/541307936 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f92b80037a0 con 0x7f92d0102790 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.834+0000 7f92c7400640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92b40a2b80 0x7f92b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.834+0000 7f92c7400640 1 -- 172.21.15.46:0/541307936 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7f92b80cf100 con 0x7f92d0102790 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.834+0000 7f92cde00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92b40a2b80 0x7f92b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.835+0000 7f92cde00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92b40a2b80 0x7f92b40a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009fc0 tx=0x7f92c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:32.836+0000 7f92c7400640 1 -- 172.21.15.46:0/541307936 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92b809c380 con 0x7f92d0102790 2024-09-10T07:33:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.024+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f92d0075a20 con 0x7f92d0102790 2024-09-10T07:33:33.024 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:32 smithi046 ceph-mon[30705]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T07:33:33.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:32 smithi181 ceph-mon[35397]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T07:33:33.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:32 smithi063 ceph-mon[35020]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T07:33:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.848+0000 7f92c7400640 1 -- 172.21.15.46:0/541307936 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f92d0075a20 con 0x7f92d0102790 2024-09-10T07:33:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-10T07:33:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92b40a2b80 msgr2=0x7f92b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92b40a2b80 0x7f92b40a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009fc0 tx=0x7f92c0005f90 comp rx=0 tx=0).stop 2024-09-10T07:33:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d0102790 msgr2=0x7f92d0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d0102790 0x7f92d0074a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f92b800e9c0 tx=0x7f92b800ee80 comp rx=0 tx=0).stop 2024-09-10T07:33:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 shutdown_connections 2024-09-10T07:33:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d0106f10 0x7f92d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92b40a2b80 0x7f92b40a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d0102790 0x7f92d0074a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 --2- 172.21.15.46:0/541307936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92d0101de0 0x7f92d00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.851+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 >> 172.21.15.46:0/541307936 conn(0x7f92d00fd8c0 msgr2=0x7f92d00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.852+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 shutdown_connections 2024-09-10T07:33:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:33.852+0000 7f92d4c00640 1 -- 172.21.15.46:0/541307936 wait complete. 2024-09-10T07:33:34.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:33 smithi181 ceph-mon[35397]: pgmap v291: 33 pgs: 12 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:34.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:33 smithi181 ceph-mon[35397]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T07:33:34.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:33 smithi181 ceph-mon[35397]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T07:33:34.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:33 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/541307936' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T07:33:34.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:33 smithi063 ceph-mon[35020]: pgmap v291: 33 pgs: 12 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:34.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:33 smithi063 ceph-mon[35020]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T07:33:34.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:33 smithi063 ceph-mon[35020]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T07:33:34.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:33 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/541307936' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T07:33:34.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:33 smithi046 ceph-mon[30705]: pgmap v291: 33 pgs: 12 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:34.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:33 smithi046 ceph-mon[30705]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T07:33:34.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:33 smithi046 ceph-mon[30705]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T07:33:34.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:33 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/541307936' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T07:33:34.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T07:33:35.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:34 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/541307936' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T07:33:35.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:34 smithi181 ceph-mon[35397]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T07:33:35.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:34 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/541307936' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T07:33:35.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:34 smithi063 ceph-mon[35020]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T07:33:35.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:34 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/541307936' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T07:33:35.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:34 smithi046 ceph-mon[30705]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T07:33:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 -- 172.21.15.46:0/1686575292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c109120 msgr2=0x7f2e5c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 --2- 172.21.15.46:0/1686575292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c109120 0x7f2e5c110ca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009ca0 tx=0x7f2e5402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:33:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 -- 172.21.15.46:0/1686575292 shutdown_connections 2024-09-10T07:33:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 --2- 172.21.15.46:0/1686575292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c109120 0x7f2e5c110ca0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 --2- 172.21.15.46:0/1686575292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e5c1049a0 0x7f2e5c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 --2- 172.21.15.46:0/1686575292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e5c103ff0 0x7f2e5c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 -- 172.21.15.46:0/1686575292 >> 172.21.15.46:0/1686575292 conn(0x7f2e5c0ffa90 msgr2=0x7f2e5c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 -- 172.21.15.46:0/1686575292 shutdown_connections 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.804+0000 7f2e61600640 1 -- 172.21.15.46:0/1686575292 wait complete. 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.805+0000 7f2e61600640 1 Processor -- start 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.805+0000 7f2e61600640 1 -- start start 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e61600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c103ff0 0x7f2e5c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e61600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e5c1049a0 0x7f2e5c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e61600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e5c109120 0x7f2e5c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e61600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c114180 con 0x7f2e5c103ff0 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e61600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c114000 con 0x7f2e5c1049a0 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e61600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c114300 con 0x7f2e5c109120 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c103ff0 0x7f2e5c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e5c1049a0 0x7f2e5c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c103ff0 0x7f2e5c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44346/0 (socket says 172.21.15.46:44346) 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5aa00640 1 -- 172.21.15.46:0/2876270315 learned_addr learned my addr 172.21.15.46:0/2876270315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5b400640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e5c109120 0x7f2e5c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5aa00640 1 -- 172.21.15.46:0/2876270315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e5c109120 msgr2=0x7f2e5c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5aa00640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e5c109120 0x7f2e5c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5aa00640 1 -- 172.21.15.46:0/2876270315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e5c1049a0 msgr2=0x7f2e5c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5aa00640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e5c1049a0 0x7f2e5c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.806+0000 7f2e5aa00640 1 -- 172.21.15.46:0/2876270315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c19f5d0 con 0x7f2e5c103ff0 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.807+0000 7f2e5aa00640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c103ff0 0x7f2e5c19a620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00ea60 tx=0x7f2e4c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.807+0000 7f2e53e00640 1 -- 172.21.15.46:0/2876270315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00cde0 con 0x7f2e5c103ff0 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.807+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c19f8c0 con 0x7f2e5c103ff0 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.807+0000 7f2e53e00640 1 -- 172.21.15.46:0/2876270315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c0035f0 con 0x7f2e5c103ff0 2024-09-10T07:33:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.807+0000 7f2e53e00640 1 -- 172.21.15.46:0/2876270315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c010670 con 0x7f2e5c103ff0 2024-09-10T07:33:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.807+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c1a71a0 con 0x7f2e5c103ff0 2024-09-10T07:33:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.808+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5c1043d0 con 0x7f2e5c103ff0 2024-09-10T07:33:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.810+0000 7f2e53e00640 1 -- 172.21.15.46:0/2876270315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2e4c0037a0 con 0x7f2e5c103ff0 2024-09-10T07:33:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.811+0000 7f2e53e00640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e400a2b80 0x7f2e400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.812+0000 7f2e53e00640 1 -- 172.21.15.46:0/2876270315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f2e4c0cf220 con 0x7f2e5c103ff0 2024-09-10T07:33:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.812+0000 7f2e52000640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e400a2b80 0x7f2e400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.812+0000 7f2e52000640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e400a2b80 0x7f2e400a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c10aaf0 tx=0x7f2e44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:35.816+0000 7f2e53e00640 1 -- 172.21.15.46:0/2876270315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e4c09c290 con 0x7f2e5c103ff0 2024-09-10T07:33:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.039+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2e5c1a7440 con 0x7f2e5c103ff0 2024-09-10T07:33:36.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:35 smithi046 ceph-mon[30705]: pgmap v294: 97 pgs: 31 creating+peering, 9 active+clean, 57 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:36.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:35 smithi046 ceph-mon[30705]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T07:33:36.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:35 smithi181 ceph-mon[35397]: pgmap v294: 97 pgs: 31 creating+peering, 9 active+clean, 57 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:36.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:35 smithi181 ceph-mon[35397]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T07:33:36.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:35 smithi063 ceph-mon[35020]: pgmap v294: 97 pgs: 31 creating+peering, 9 active+clean, 57 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:36.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:35 smithi063 ceph-mon[35020]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T07:33:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.871+0000 7f2e53e00640 1 -- 172.21.15.46:0/2876270315 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f2e4c09bb90 con 0x7f2e5c103ff0 2024-09-10T07:33:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T07:33:36.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e400a2b80 msgr2=0x7f2e400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:36.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e400a2b80 0x7f2e400a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c10aaf0 tx=0x7f2e44005f90 comp rx=0 tx=0).stop 2024-09-10T07:33:36.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c103ff0 msgr2=0x7f2e5c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:36.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c103ff0 0x7f2e5c19a620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00ea60 tx=0x7f2e4c00ef20 comp rx=0 tx=0).stop 2024-09-10T07:33:36.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 shutdown_connections 2024-09-10T07:33:36.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e5c109120 0x7f2e5c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:36.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e5c1049a0 0x7f2e5c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e400a2b80 0x7f2e400a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 --2- 172.21.15.46:0/2876270315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e5c103ff0 0x7f2e5c19a620 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.874+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 >> 172.21.15.46:0/2876270315 conn(0x7f2e5c0ffa90 msgr2=0x7f2e5c100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.875+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 shutdown_connections 2024-09-10T07:33:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:36.875+0000 7f2e61600640 1 -- 172.21.15.46:0/2876270315 wait complete. 2024-09-10T07:33:37.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:36 smithi181 ceph-mon[35397]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T07:33:37.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:36 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2876270315' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T07:33:37.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:36 smithi181 ceph-mon[35397]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-09-10T07:33:37.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:36 smithi063 ceph-mon[35020]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T07:33:37.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:36 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2876270315' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T07:33:37.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:36 smithi063 ceph-mon[35020]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-09-10T07:33:37.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:36 smithi046 ceph-mon[30705]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T07:33:37.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:36 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2876270315' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T07:33:37.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:36 smithi046 ceph-mon[30705]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-09-10T07:33:37.862 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T07:33:37.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T07:33:38.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:37 smithi181 ceph-mon[35397]: pgmap v297: 97 pgs: 22 creating+peering, 65 active+clean, 10 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:38.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:37 smithi181 ceph-mon[35397]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T07:33:38.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:37 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2876270315' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T07:33:38.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:37 smithi181 ceph-mon[35397]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-09-10T07:33:38.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:37 smithi181 ceph-mon[35397]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-09-10T07:33:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:37 smithi063 ceph-mon[35020]: pgmap v297: 97 pgs: 22 creating+peering, 65 active+clean, 10 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:37 smithi063 ceph-mon[35020]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T07:33:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:37 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2876270315' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T07:33:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:37 smithi063 ceph-mon[35020]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-09-10T07:33:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:37 smithi063 ceph-mon[35020]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-09-10T07:33:38.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:37 smithi046 ceph-mon[30705]: pgmap v297: 97 pgs: 22 creating+peering, 65 active+clean, 10 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:38.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:37 smithi046 ceph-mon[30705]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T07:33:38.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:37 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2876270315' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T07:33:38.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:37 smithi046 ceph-mon[30705]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-09-10T07:33:38.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:37 smithi046 ceph-mon[30705]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-09-10T07:33:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 -- 172.21.15.46:0/1940573685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 msgr2=0x7fe6001043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 --2- 172.21.15.46:0/1940573685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 0x7fe6001043f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009900 tx=0x7fe5f002a410 comp rx=0 tx=0).stop 2024-09-10T07:33:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 -- 172.21.15.46:0/1940573685 shutdown_connections 2024-09-10T07:33:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 --2- 172.21.15.46:0/1940573685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600109140 0x7fe600110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 --2- 172.21.15.46:0/1940573685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6001049c0 0x7fe600108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 --2- 172.21.15.46:0/1940573685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 0x7fe6001043f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 -- 172.21.15.46:0/1940573685 >> 172.21.15.46:0/1940573685 conn(0x7fe6000ffa50 msgr2=0x7fe600101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 -- 172.21.15.46:0/1940573685 shutdown_connections 2024-09-10T07:33:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.830+0000 7fe606e00640 1 -- 172.21.15.46:0/1940573685 wait complete. 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.831+0000 7fe606e00640 1 Processor -- start 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.831+0000 7fe606e00640 1 -- start start 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.831+0000 7fe606e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 0x7fe60019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.831+0000 7fe606e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6001049c0 0x7fe60019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.831+0000 7fe606e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600109140 0x7fe60019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.831+0000 7fe606e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe600114140 con 0x7fe600104010 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.831+0000 7fe606e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe600113fc0 con 0x7fe6001049c0 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.831+0000 7fe606e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001142c0 con 0x7fe600109140 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 0x7fe60019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 0x7fe60019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44368/0 (socket says 172.21.15.46:44368) 2024-09-10T07:33:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5ffe00640 1 -- 172.21.15.46:0/1973866918 learned_addr learned my addr 172.21.15.46:0/1973866918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5f7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6001049c0 0x7fe60019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe604e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600109140 0x7fe60019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5f7e00640 1 -- 172.21.15.46:0/1973866918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600109140 msgr2=0x7fe60019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5f7e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600109140 0x7fe60019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5f7e00640 1 -- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 msgr2=0x7fe60019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5f7e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 0x7fe60019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.832+0000 7fe5f7e00640 1 -- 172.21.15.46:0/1973866918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60019f5d0 con 0x7fe6001049c0 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.833+0000 7fe5f7e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6001049c0 0x7fe60019ab60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800e9f0 tx=0x7fe5e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.833+0000 7fe5fd600640 1 -- 172.21.15.46:0/1973866918 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800ccc0 con 0x7fe6001049c0 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.833+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60019f8c0 con 0x7fe6001049c0 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.833+0000 7fe5fd600640 1 -- 172.21.15.46:0/1973866918 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e8010910 con 0x7fe6001049c0 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.833+0000 7fe5fd600640 1 -- 172.21.15.46:0/1973866918 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8018960 con 0x7fe6001049c0 2024-09-10T07:33:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.833+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6001a71a0 con 0x7fe6001049c0 2024-09-10T07:33:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.834+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6001043f0 con 0x7fe6001049c0 2024-09-10T07:33:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.836+0000 7fe5fd600640 1 -- 172.21.15.46:0/1973866918 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe5e8010ac0 con 0x7fe6001049c0 2024-09-10T07:33:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.837+0000 7fe5fd600640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe5e40a2b80 0x7fe5e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.837+0000 7fe5fd600640 1 -- 172.21.15.46:0/1973866918 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe5e80cf6a0 con 0x7fe6001049c0 2024-09-10T07:33:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.838+0000 7fe5ffe00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe5e40a2b80 0x7fe5e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.838+0000 7fe5ffe00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe5e40a2b80 0x7fe5e40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00096c0 tx=0x7fe5f0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:38.840+0000 7fe5fd600640 1 -- 172.21.15.46:0/1973866918 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5e809c060 con 0x7fe6001049c0 2024-09-10T07:33:39.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.033+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe60002d7c0 con 0x7fe6001049c0 2024-09-10T07:33:39.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:38 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] up:active 2024-09-10T07:33:39.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:38 smithi046 ceph-mon[30705]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:39.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:38 smithi046 ceph-mon[30705]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:39.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:38 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] up:active 2024-09-10T07:33:39.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:38 smithi181 ceph-mon[35397]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:39.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:38 smithi181 ceph-mon[35397]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:39.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:38 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] up:active 2024-09-10T07:33:39.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:38 smithi063 ceph-mon[35020]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:39.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:38 smithi063 ceph-mon[35020]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.885+0000 7fe5fd600640 1 -- 172.21.15.46:0/1973866918 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v87) ==== 157+0+0 (secure 0 0 0) 0x7fe5e80a0360 con 0x7fe6001049c0 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe5e40a2b80 msgr2=0x7fe5e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe5e40a2b80 0x7fe5e40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00096c0 tx=0x7fe5f0005a20 comp rx=0 tx=0).stop 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6001049c0 msgr2=0x7fe60019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6001049c0 0x7fe60019ab60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800e9f0 tx=0x7fe5e800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 shutdown_connections 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600109140 0x7fe60019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6001049c0 0x7fe60019ab60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe5e40a2b80 0x7fe5e40a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 --2- 172.21.15.46:0/1973866918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104010 0x7fe60019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 >> 172.21.15.46:0/1973866918 conn(0x7fe6000ffa50 msgr2=0x7fe600100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:39.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 shutdown_connections 2024-09-10T07:33:39.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:39.888+0000 7fe606e00640 1 -- 172.21.15.46:0/1973866918 wait complete. 2024-09-10T07:33:40.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:39 smithi181 ceph-mon[35397]: pgmap v299: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 1 op/s 2024-09-10T07:33:40.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:39 smithi181 ceph-mon[35397]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T07:33:40.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:39 smithi181 ceph-mon[35397]: Cluster is now healthy 2024-09-10T07:33:40.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:39 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1973866918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T07:33:40.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:39 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T07:33:40.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:39 smithi063 ceph-mon[35020]: pgmap v299: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 1 op/s 2024-09-10T07:33:40.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:39 smithi063 ceph-mon[35020]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T07:33:40.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:39 smithi063 ceph-mon[35020]: Cluster is now healthy 2024-09-10T07:33:40.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:39 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1973866918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T07:33:40.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:39 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T07:33:40.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:39 smithi046 ceph-mon[30705]: pgmap v299: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 1 op/s 2024-09-10T07:33:40.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:39 smithi046 ceph-mon[30705]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T07:33:40.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:39 smithi046 ceph-mon[30705]: Cluster is now healthy 2024-09-10T07:33:40.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:39 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1973866918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T07:33:40.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:39 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T07:33:40.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T07:33:41.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:40 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T07:33:41.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:40 smithi181 ceph-mon[35397]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T07:33:41.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:40 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T07:33:41.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:40 smithi063 ceph-mon[35020]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T07:33:41.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:40 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T07:33:41.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:40 smithi046 ceph-mon[30705]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T07:33:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:41.633 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:33:41 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: 2024-09-10T07:33:41.387+0000 7f775a600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:33:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.915+0000 7fea14e00640 1 -- 172.21.15.46:0/395915739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 msgr2=0x7fea101124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.915+0000 7fea14e00640 1 --2- 172.21.15.46:0/395915739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 0x7fea101124f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009900 tx=0x7fe9f802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:33:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.915+0000 7fea14e00640 1 -- 172.21.15.46:0/395915739 shutdown_connections 2024-09-10T07:33:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.915+0000 7fea14e00640 1 --2- 172.21.15.46:0/395915739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10112a30 0x7fea10114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.915+0000 7fea14e00640 1 --2- 172.21.15.46:0/395915739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 0x7fea101124f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.915+0000 7fea14e00640 1 --2- 172.21.15.46:0/395915739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea10076720 0x7fea10076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.915+0000 7fea14e00640 1 -- 172.21.15.46:0/395915739 >> 172.21.15.46:0/395915739 conn(0x7fea100ffa90 msgr2=0x7fea10101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.915+0000 7fea14e00640 1 -- 172.21.15.46:0/395915739 shutdown_connections 2024-09-10T07:33:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.916+0000 7fea14e00640 1 -- 172.21.15.46:0/395915739 wait complete. 2024-09-10T07:33:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.917+0000 7fea14e00640 1 Processor -- start 2024-09-10T07:33:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.917+0000 7fea14e00640 1 -- start start 2024-09-10T07:33:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea14e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea10076720 0x7fea101a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea14e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 0x7fea101a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea14e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10112a30 0x7fea1019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea14e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea101185a0 con 0x7fea10077040 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea14e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea10118420 con 0x7fea10112a30 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea14e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea10118720 con 0x7fea10076720 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 0x7fea101a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 0x7fea101a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44376/0 (socket says 172.21.15.46:44376) 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea07e00640 1 -- 172.21.15.46:0/1613088943 learned_addr learned my addr 172.21.15.46:0/1613088943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea0de00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea10076720 0x7fea101a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.918+0000 7fea0e800640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10112a30 0x7fea1019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea0de00640 1 -- 172.21.15.46:0/1613088943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10112a30 msgr2=0x7fea1019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea0de00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10112a30 0x7fea1019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea0de00640 1 -- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 msgr2=0x7fea101a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea0de00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 0x7fea101a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea0de00640 1 -- 172.21.15.46:0/1613088943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1019f0e0 con 0x7fea10076720 2024-09-10T07:33:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea0de00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea10076720 0x7fea101a47f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea00005fd0 tx=0x7fea0000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea06a00640 1 -- 172.21.15.46:0/1613088943 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00009c30 con 0x7fea10076720 2024-09-10T07:33:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea06a00640 1 -- 172.21.15.46:0/1613088943 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea00009dd0 con 0x7fea10076720 2024-09-10T07:33:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1019f3d0 con 0x7fea10076720 2024-09-10T07:33:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.919+0000 7fea06a00640 1 -- 172.21.15.46:0/1613088943 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00010460 con 0x7fea10076720 2024-09-10T07:33:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.920+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea101ab6c0 con 0x7fea10076720 2024-09-10T07:33:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.921+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea10076b00 con 0x7fea10076720 2024-09-10T07:33:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.922+0000 7fea06a00640 1 -- 172.21.15.46:0/1613088943 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea00003f50 con 0x7fea10076720 2024-09-10T07:33:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.924+0000 7fea06a00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.925+0000 7fea06a00640 1 -- 172.21.15.46:0/1613088943 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8277+0+0 (secure 0 0 0) 0x7fea000ceb70 con 0x7fea10076720 2024-09-10T07:33:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.925+0000 7fea07e00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.925+0000 7fea07e00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe9f40a2b80 0x7fe9f40a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fea10079390 tx=0x7fe9f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:41.928+0000 7fea06a00640 1 -- 172.21.15.46:0/1613088943 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea00017020 con 0x7fea10076720 2024-09-10T07:33:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:42.121+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fea1010dbc0 con 0x7fea10076720 2024-09-10T07:33:42.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:41 smithi181 ceph-mon[35397]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-10T07:33:42.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:41 smithi063 ceph-mon[35020]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-10T07:33:42.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:41 smithi046 ceph-mon[30705]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-10T07:33:43.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:42 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1613088943' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T07:33:43.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:42 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T07:33:43.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:42 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1613088943' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T07:33:43.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:42 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T07:33:43.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:42 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1613088943' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T07:33:43.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:42 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T07:33:43.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.901+0000 7fea06a00640 1 -- 172.21.15.46:0/1613088943 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v89) ==== 228+0+0 (secure 0 0 0) 0x7fea0009b3f0 con 0x7fea10076720 2024-09-10T07:33:43.903 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' created 2024-09-10T07:33:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.907+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe9f40a2b80 msgr2=0x7fe9f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.907+0000 7fea14e00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe9f40a2b80 0x7fe9f40a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fea10079390 tx=0x7fe9f8005a20 comp rx=0 tx=0).stop 2024-09-10T07:33:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.907+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea10076720 msgr2=0x7fea101a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.907+0000 7fea14e00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea10076720 0x7fea101a47f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea00005fd0 tx=0x7fea0000dde0 comp rx=0 tx=0).stop 2024-09-10T07:33:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.908+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 shutdown_connections 2024-09-10T07:33:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.908+0000 7fea14e00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10112a30 0x7fea1019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.908+0000 7fea14e00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.908+0000 7fea14e00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea10077040 0x7fea101a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.908+0000 7fea14e00640 1 --2- 172.21.15.46:0/1613088943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea10076720 0x7fea101a47f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.908+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 >> 172.21.15.46:0/1613088943 conn(0x7fea100ffa90 msgr2=0x7fea10101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.908+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 shutdown_connections 2024-09-10T07:33:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:43.909+0000 7fea14e00640 1 -- 172.21.15.46:0/1613088943 wait complete. 2024-09-10T07:33:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:43 smithi046 ceph-mon[30705]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-10T07:33:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:43 smithi046 ceph-mon[30705]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T07:33:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:43 smithi181 ceph-mon[35397]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-10T07:33:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:43 smithi181 ceph-mon[35397]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T07:33:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:43 smithi063 ceph-mon[35020]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-10T07:33:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:43 smithi063 ceph-mon[35020]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T07:33:44.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T07:33:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:44 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T07:33:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:44 smithi181 ceph-mon[35397]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T07:33:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:44 smithi181 ceph-mon[35397]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T07:33:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:44 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T07:33:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:44 smithi063 ceph-mon[35020]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T07:33:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:44 smithi063 ceph-mon[35020]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T07:33:45.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:44 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T07:33:45.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:44 smithi046 ceph-mon[30705]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T07:33:45.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:44 smithi046 ceph-mon[30705]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T07:33:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.974+0000 7f7701c00640 1 -- 172.21.15.46:0/1489779354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc109120 msgr2=0x7f76fc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.974+0000 7f7701c00640 1 --2- 172.21.15.46:0/1489779354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc109120 0x7f76fc110ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009ca0 tx=0x7f76f402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:33:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.974+0000 7f7701c00640 1 -- 172.21.15.46:0/1489779354 shutdown_connections 2024-09-10T07:33:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.974+0000 7f7701c00640 1 --2- 172.21.15.46:0/1489779354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc109120 0x7f76fc110ca0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.974+0000 7f7701c00640 1 --2- 172.21.15.46:0/1489779354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76fc1049a0 0x7f76fc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.974+0000 7f7701c00640 1 --2- 172.21.15.46:0/1489779354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76fc103ff0 0x7f76fc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.974+0000 7f7701c00640 1 -- 172.21.15.46:0/1489779354 >> 172.21.15.46:0/1489779354 conn(0x7f76fc0ffa90 msgr2=0x7f76fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.975+0000 7f7701c00640 1 -- 172.21.15.46:0/1489779354 shutdown_connections 2024-09-10T07:33:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.975+0000 7f7701c00640 1 -- 172.21.15.46:0/1489779354 wait complete. 2024-09-10T07:33:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.975+0000 7f7701c00640 1 Processor -- start 2024-09-10T07:33:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f7701c00640 1 -- start start 2024-09-10T07:33:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f7701c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc103ff0 0x7f76fc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f7701c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76fc1049a0 0x7f76fc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f7701c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76fc109120 0x7f76fc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f7701c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc114180 con 0x7f76fc103ff0 2024-09-10T07:33:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f7701c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc114000 con 0x7f76fc1049a0 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f7701c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc114300 con 0x7f76fc109120 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f76faa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc103ff0 0x7f76fc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f76faa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc103ff0 0x7f76fc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40690/0 (socket says 172.21.15.46:40690) 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f76faa00640 1 -- 172.21.15.46:0/2721866425 learned_addr learned my addr 172.21.15.46:0/2721866425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f76f2000640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76fc1049a0 0x7f76fc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.976+0000 7f76fb400640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76fc109120 0x7f76fc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76faa00640 1 -- 172.21.15.46:0/2721866425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76fc109120 msgr2=0x7f76fc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76faa00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76fc109120 0x7f76fc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76faa00640 1 -- 172.21.15.46:0/2721866425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76fc1049a0 msgr2=0x7f76fc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76faa00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76fc1049a0 0x7f76fc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76faa00640 1 -- 172.21.15.46:0/2721866425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc19f5d0 con 0x7f76fc103ff0 2024-09-10T07:33:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76faa00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc103ff0 0x7f76fc19a620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00e9f0 tx=0x7f76ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76f3e00640 1 -- 172.21.15.46:0/2721866425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00cd80 con 0x7f76fc103ff0 2024-09-10T07:33:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76f3e00640 1 -- 172.21.15.46:0/2721866425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec010910 con 0x7f76fc103ff0 2024-09-10T07:33:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f76f3e00640 1 -- 172.21.15.46:0/2721866425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec018990 con 0x7f76fc103ff0 2024-09-10T07:33:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.977+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc19f8c0 con 0x7f76fc103ff0 2024-09-10T07:33:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.978+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc1a71a0 con 0x7f76fc103ff0 2024-09-10T07:33:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.979+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76fc1043d0 con 0x7f76fc103ff0 2024-09-10T07:33:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.981+0000 7f76f3e00640 1 -- 172.21.15.46:0/2721866425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f76ec010ac0 con 0x7f76fc103ff0 2024-09-10T07:33:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.982+0000 7f76f3e00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76e00a2b80 0x7f76e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.982+0000 7f76f3e00640 1 -- 172.21.15.46:0/2721866425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8770+0+0 (secure 0 0 0) 0x7f76ec0cfa10 con 0x7f76fc103ff0 2024-09-10T07:33:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.982+0000 7f76f2000640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76e00a2b80 0x7f76e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.983+0000 7f76f2000640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76e00a2b80 0x7f76e00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f76fc10aaf0 tx=0x7f76e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:45.985+0000 7f76f3e00640 1 -- 172.21.15.46:0/2721866425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76ec09c0a0 con 0x7f76fc103ff0 2024-09-10T07:33:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:45 smithi046 ceph-mon[30705]: pgmap v305: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-10T07:33:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:45 smithi046 ceph-mon[30705]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T07:33:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:45 smithi046 ceph-mon[30705]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T07:33:46.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.172+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f76fc1a73e0 con 0x7f76fc103ff0 2024-09-10T07:33:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:45 smithi181 ceph-mon[35397]: pgmap v305: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-10T07:33:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:45 smithi181 ceph-mon[35397]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T07:33:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:45 smithi181 ceph-mon[35397]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T07:33:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:45 smithi063 ceph-mon[35020]: pgmap v305: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-10T07:33:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:45 smithi063 ceph-mon[35020]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T07:33:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:45 smithi063 ceph-mon[35020]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T07:33:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.928+0000 7f76f3e00640 1 -- 172.21.15.46:0/2721866425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v92) ==== 169+0+0 (secure 0 0 0) 0x7f76ec0a03a0 con 0x7f76fc103ff0 2024-09-10T07:33:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T07:33:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76e00a2b80 msgr2=0x7f76e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76e00a2b80 0x7f76e00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f76fc10aaf0 tx=0x7f76e400b390 comp rx=0 tx=0).stop 2024-09-10T07:33:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc103ff0 msgr2=0x7f76fc19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc103ff0 0x7f76fc19a620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00e9f0 tx=0x7f76ec00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:33:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 shutdown_connections 2024-09-10T07:33:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76fc109120 0x7f76fc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76fc1049a0 0x7f76fc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76e00a2b80 0x7f76e00a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 --2- 172.21.15.46:0/2721866425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76fc103ff0 0x7f76fc19a620 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.932+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 >> 172.21.15.46:0/2721866425 conn(0x7f76fc0ffa90 msgr2=0x7f76fc100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.933+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 shutdown_connections 2024-09-10T07:33:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:46.933+0000 7f7701c00640 1 -- 172.21.15.46:0/2721866425 wait complete. 2024-09-10T07:33:47.334 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:46 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2721866425' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T07:33:47.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:46 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2721866425' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T07:33:47.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:46 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2721866425' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T07:33:47.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T07:33:48.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:47 smithi046 ceph-mon[30705]: pgmap v308: 129 pgs: 5 creating+peering, 5 unknown, 119 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:48.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:47 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2721866425' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T07:33:48.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:47 smithi046 ceph-mon[30705]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T07:33:48.234 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:47 smithi181 ceph-mon[35397]: pgmap v308: 129 pgs: 5 creating+peering, 5 unknown, 119 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:47 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2721866425' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T07:33:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:47 smithi181 ceph-mon[35397]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T07:33:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:47 smithi063 ceph-mon[35020]: pgmap v308: 129 pgs: 5 creating+peering, 5 unknown, 119 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:47 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2721866425' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T07:33:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:47 smithi063 ceph-mon[35020]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T07:33:48.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 -- 172.21.15.46:0/3524753089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c112a30 msgr2=0x7f227c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 --2- 172.21.15.46:0/3524753089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c112a30 0x7f227c114e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2274009ca0 tx=0x7f227402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 -- 172.21.15.46:0/3524753089 shutdown_connections 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 --2- 172.21.15.46:0/3524753089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c112a30 0x7f227c114e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 --2- 172.21.15.46:0/3524753089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227c077040 0x7f227c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 --2- 172.21.15.46:0/3524753089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f227c076720 0x7f227c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 -- 172.21.15.46:0/3524753089 >> 172.21.15.46:0/3524753089 conn(0x7f227c0ffa90 msgr2=0x7f227c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 -- 172.21.15.46:0/3524753089 shutdown_connections 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.862+0000 7f2281200640 1 -- 172.21.15.46:0/3524753089 wait complete. 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.863+0000 7f2281200640 1 Processor -- start 2024-09-10T07:33:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.863+0000 7f2281200640 1 -- start start 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.863+0000 7f2281200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227c076720 0x7f227c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2281200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c077040 0x7f227c1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2281200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f227c112a30 0x7f227c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2281200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f227c1185c0 con 0x7f227c077040 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2281200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f227c118440 con 0x7f227c112a30 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2281200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f227c118740 con 0x7f227c076720 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2271400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c077040 0x7f227c1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f227a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f227c112a30 0x7f227c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2271400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c077040 0x7f227c1a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40716/0 (socket says 172.21.15.46:40716) 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f227a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f227c112a30 0x7f227c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40810/0 (socket says 172.21.15.46:40810) 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2271400640 1 -- 172.21.15.46:0/2491349649 learned_addr learned my addr 172.21.15.46:0/2491349649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2279e00640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227c076720 0x7f227c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2271400640 1 -- 172.21.15.46:0/2491349649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227c076720 msgr2=0x7f227c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2271400640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227c076720 0x7f227c1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2271400640 1 -- 172.21.15.46:0/2491349649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f227c112a30 msgr2=0x7f227c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2271400640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f227c112a30 0x7f227c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f2271400640 1 -- 172.21.15.46:0/2491349649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c19efc0 con 0x7f227c077040 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.864+0000 7f227a800640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f227c112a30 0x7f227c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.865+0000 7f2271400640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c077040 0x7f227c1a4c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f226400e9f0 tx=0x7f226400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.865+0000 7f2273400640 1 -- 172.21.15.46:0/2491349649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226400cd80 con 0x7f227c077040 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.865+0000 7f2273400640 1 -- 172.21.15.46:0/2491349649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2264010910 con 0x7f227c077040 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.865+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227c19f2b0 con 0x7f227c077040 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.865+0000 7f2273400640 1 -- 172.21.15.46:0/2491349649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264018990 con 0x7f227c077040 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.865+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f227c1ab5a0 con 0x7f227c077040 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.866+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f227c076b00 con 0x7f227c077040 2024-09-10T07:33:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.868+0000 7f2273400640 1 -- 172.21.15.46:0/2491349649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2264010ac0 con 0x7f227c077040 2024-09-10T07:33:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.869+0000 7f2273400640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22600a2b80 0x7f22600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.869+0000 7f2273400640 1 -- 172.21.15.46:0/2491349649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8770+0+0 (secure 0 0 0) 0x7f22640cf550 con 0x7f227c077040 2024-09-10T07:33:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.870+0000 7f2279e00640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22600a2b80 0x7f22600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.870+0000 7f2279e00640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22600a2b80 0x7f22600a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f227c19f9d0 tx=0x7f226c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:48.873+0000 7f2273400640 1 -- 172.21.15.46:0/2491349649 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f226409bbe0 con 0x7f227c077040 2024-09-10T07:33:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.098+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f227c19fdc0 con 0x7f227c077040 2024-09-10T07:33:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.943+0000 7f2273400640 1 -- 172.21.15.46:0/2491349649 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f227c19fdc0 con 0x7f227c077040 2024-09-10T07:33:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:added data pool 4 to fsmap 2024-09-10T07:33:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.947+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22600a2b80 msgr2=0x7f22600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.947+0000 7f2281200640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22600a2b80 0x7f22600a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f227c19f9d0 tx=0x7f226c005f90 comp rx=0 tx=0).stop 2024-09-10T07:33:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.947+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c077040 msgr2=0x7f227c1a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.947+0000 7f2281200640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c077040 0x7f227c1a4c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f226400e9f0 tx=0x7f226400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:33:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.948+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 shutdown_connections 2024-09-10T07:33:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.948+0000 7f2281200640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f227c112a30 0x7f227c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.948+0000 7f2281200640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22600a2b80 0x7f22600a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.948+0000 7f2281200640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c077040 0x7f227c1a4c10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.948+0000 7f2281200640 1 --2- 172.21.15.46:0/2491349649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227c076720 0x7f227c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.948+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 >> 172.21.15.46:0/2491349649 conn(0x7f227c0ffa90 msgr2=0x7f227c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.948+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 shutdown_connections 2024-09-10T07:33:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:49.948+0000 7f2281200640 1 -- 172.21.15.46:0/2491349649 wait complete. 2024-09-10T07:33:50.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:49 smithi046 ceph-mon[30705]: pgmap v310: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:50.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:49 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2491349649' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T07:33:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:49 smithi181 ceph-mon[35397]: pgmap v310: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:49 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2491349649' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T07:33:50.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:49 smithi063 ceph-mon[35020]: pgmap v310: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:50.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:49 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2491349649' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T07:33:50.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:33:51.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:50 smithi046 ceph-mon[30705]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T07:33:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:50 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2491349649' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T07:33:51.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:50 smithi046 ceph-mon[30705]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:51.245 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:50 smithi181 ceph-mon[35397]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T07:33:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2491349649' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T07:33:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:50 smithi181 ceph-mon[35397]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:50 smithi063 ceph-mon[35020]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T07:33:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:50 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2491349649' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T07:33:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:50 smithi063 ceph-mon[35020]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 -- 172.21.15.46:0/1956730311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 msgr2=0x7f95241124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 --2- 172.21.15.46:0/1956730311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 0x7f95241124f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9510009900 tx=0x7f951002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 -- 172.21.15.46:0/1956730311 shutdown_connections 2024-09-10T07:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 --2- 172.21.15.46:0/1956730311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9524112a30 0x7f9524114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 --2- 172.21.15.46:0/1956730311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 0x7f95241124f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 --2- 172.21.15.46:0/1956730311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9524076720 0x7f9524076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 -- 172.21.15.46:0/1956730311 >> 172.21.15.46:0/1956730311 conn(0x7f95240ffa90 msgr2=0x7f9524101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 -- 172.21.15.46:0/1956730311 shutdown_connections 2024-09-10T07:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.844+0000 7f952c600640 1 -- 172.21.15.46:0/1956730311 wait complete. 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.845+0000 7f952c600640 1 Processor -- start 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.845+0000 7f952c600640 1 -- start start 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f952c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9524076720 0x7f95241a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f952c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 0x7f95241a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f952c600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9524112a30 0x7f952419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f952c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9524118610 con 0x7f9524077040 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f952c600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9524118490 con 0x7f9524112a30 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f952c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9524118790 con 0x7f9524076720 2024-09-10T07:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f9529200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 0x7f95241a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f9529200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 0x7f95241a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40742/0 (socket says 172.21.15.46:40742) 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f9529200640 1 -- 172.21.15.46:0/3871280197 learned_addr learned my addr 172.21.15.46:0/3871280197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f952a600640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9524112a30 0x7f952419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.846+0000 7f9529c00640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9524076720 0x7f95241a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9529200640 1 -- 172.21.15.46:0/3871280197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9524076720 msgr2=0x7f95241a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9529200640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9524076720 0x7f95241a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9529200640 1 -- 172.21.15.46:0/3871280197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9524112a30 msgr2=0x7f952419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9529200640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9524112a30 0x7f952419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9529200640 1 -- 172.21.15.46:0/3871280197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952419f020 con 0x7f9524077040 2024-09-10T07:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f952a600640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9524112a30 0x7f952419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9529200640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 0x7f95241a4c70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9510004770 tx=0x7f9510003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9522000640 1 -- 172.21.15.46:0/3871280197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951003f070 con 0x7f9524077040 2024-09-10T07:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9522000640 1 -- 172.21.15.46:0/3871280197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9510031ca0 con 0x7f9524077040 2024-09-10T07:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f9522000640 1 -- 172.21.15.46:0/3871280197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510035030 con 0x7f9524077040 2024-09-10T07:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.847+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952419f2b0 con 0x7f9524077040 2024-09-10T07:33:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.848+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95241ab600 con 0x7f9524077040 2024-09-10T07:33:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.849+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9524077e60 con 0x7f9524077040 2024-09-10T07:33:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.851+0000 7f9522000640 1 -- 172.21.15.46:0/3871280197 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9510042070 con 0x7f9524077040 2024-09-10T07:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.852+0000 7f9522000640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f950c0a2c00 0x7f950c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.852+0000 7f9522000640 1 -- 172.21.15.46:0/3871280197 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7f95100efed0 con 0x7f9524077040 2024-09-10T07:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.852+0000 7f9529c00640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f950c0a2c00 0x7f950c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.853+0000 7f9529c00640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f950c0a2c00 0x7f950c0a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f952419fa30 tx=0x7f951800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:51.854+0000 7f9522000640 1 -- 172.21.15.46:0/3871280197 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95100bc540 con 0x7f9524077040 2024-09-10T07:33:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.082+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9524076b00 con 0x7f9524077040 2024-09-10T07:33:52.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:51 smithi046 ceph-mon[30705]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:52.082 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:51 smithi046 ceph-mon[30705]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T07:33:52.082 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:51 smithi046 ceph-mon[30705]: Cluster is now healthy 2024-09-10T07:33:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.082+0000 7f9522000640 1 -- 172.21.15.46:0/3871280197 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f95100c0840 con 0x7f9524077040 2024-09-10T07:33:52.083 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:52.083 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"btime":"2024-09-10T07:33:49:932625+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:33:49.099669+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24535,"qdb_cluster":[24535]},"id":1}]} 2024-09-10T07:33:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f950c0a2c00 msgr2=0x7f950c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f950c0a2c00 0x7f950c0a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f952419fa30 tx=0x7f951800b390 comp rx=0 tx=0).stop 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 msgr2=0x7f95241a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 0x7f95241a4c70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9510004770 tx=0x7f9510003c10 comp rx=0 tx=0).stop 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 shutdown_connections 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9524112a30 0x7f952419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f950c0a2c00 0x7f950c0a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9524077040 0x7f95241a4c70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 --2- 172.21.15.46:0/3871280197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9524076720 0x7f95241a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 >> 172.21.15.46:0/3871280197 conn(0x7f95240ffa90 msgr2=0x7f9524101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 shutdown_connections 2024-09-10T07:33:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:52.085+0000 7f952c600640 1 -- 172.21.15.46:0/3871280197 wait complete. 2024-09-10T07:33:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:51 smithi181 ceph-mon[35397]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:51 smithi181 ceph-mon[35397]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T07:33:52.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:51 smithi181 ceph-mon[35397]: Cluster is now healthy 2024-09-10T07:33:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:51 smithi063 ceph-mon[35020]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:51 smithi063 ceph-mon[35020]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T07:33:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:51 smithi063 ceph-mon[35020]: Cluster is now healthy 2024-09-10T07:33:52.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:33:53.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:52 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3871280197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:53.211 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:53.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:52 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3871280197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:53.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:52 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3871280197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 -- 172.21.15.46:0/2729639915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf701019c0 msgr2=0x7faf70101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 --2- 172.21.15.46:0/2729639915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf701019c0 0x7faf70101e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf58009900 tx=0x7faf5802a410 comp rx=0 tx=0).stop 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 -- 172.21.15.46:0/2729639915 shutdown_connections 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 --2- 172.21.15.46:0/2729639915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf70102360 0x7faf7010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 --2- 172.21.15.46:0/2729639915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf701019c0 0x7faf70101e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 --2- 172.21.15.46:0/2729639915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701079c0 0x7faf70107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 -- 172.21.15.46:0/2729639915 >> 172.21.15.46:0/2729639915 conn(0x7faf700fd740 msgr2=0x7faf700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 -- 172.21.15.46:0/2729639915 shutdown_connections 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.792+0000 7faf74c00640 1 -- 172.21.15.46:0/2729639915 wait complete. 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.793+0000 7faf74c00640 1 Processor -- start 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.793+0000 7faf74c00640 1 -- start start 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf74c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf701019c0 0x7faf701982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf74c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf70102360 0x7faf70198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf74c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701079c0 0x7faf7019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf74c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf70111e10 con 0x7faf701079c0 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf74c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf70111c90 con 0x7faf70102360 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf74c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf70111f90 con 0x7faf701019c0 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf6f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf701019c0 0x7faf701982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf6f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf701019c0 0x7faf701982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:41376/0 (socket says 172.21.15.46:41376) 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf6f400640 1 -- 172.21.15.46:0/2444565609 learned_addr learned my addr 172.21.15.46:0/2444565609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf6fe00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701079c0 0x7faf7019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.794+0000 7faf67e00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf70102360 0x7faf70198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6f400640 1 -- 172.21.15.46:0/2444565609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf70102360 msgr2=0x7faf70198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6f400640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf70102360 0x7faf70198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6f400640 1 -- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701079c0 msgr2=0x7faf7019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6f400640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701079c0 0x7faf7019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6f400640 1 -- 172.21.15.46:0/2444565609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7019d2a0 con 0x7faf701019c0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6f400640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf701019c0 0x7faf701982f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf6000e9f0 tx=0x7faf6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6cc00640 1 -- 172.21.15.46:0/2444565609 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6000cd80 con 0x7faf701019c0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6cc00640 1 -- 172.21.15.46:0/2444565609 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf60010910 con 0x7faf701019c0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7019d590 con 0x7faf701019c0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf6cc00640 1 -- 172.21.15.46:0/2444565609 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60018990 con 0x7faf701019c0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.795+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf701a4e70 con 0x7faf701019c0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.796+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf70101e20 con 0x7faf701019c0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.798+0000 7faf6cc00640 1 -- 172.21.15.46:0/2444565609 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faf60010ac0 con 0x7faf701019c0 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.800+0000 7faf6cc00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf540a2b80 0x7faf540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.800+0000 7faf6cc00640 1 -- 172.21.15.46:0/2444565609 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7faf600cfaf0 con 0x7faf701019c0 2024-09-10T07:33:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.800+0000 7faf67e00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf540a2b80 0x7faf540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.800+0000 7faf67e00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf540a2b80 0x7faf540a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faf58000c00 tx=0x7faf58005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.802+0000 7faf6cc00640 1 -- 172.21.15.46:0/2444565609 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf6009c160 con 0x7faf701019c0 2024-09-10T07:33:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.990+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf70109f50 con 0x7faf701019c0 2024-09-10T07:33:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.992+0000 7faf6cc00640 1 -- 172.21.15.46:0/2444565609 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24128 (secure 0 0 0) 0x7faf600a0460 con 0x7faf701019c0 2024-09-10T07:33:53.992 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:53.993 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:33:49.099458+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf540a2b80 msgr2=0x7faf540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf540a2b80 0x7faf540a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faf58000c00 tx=0x7faf58005b50 comp rx=0 tx=0).stop 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf701019c0 msgr2=0x7faf701982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf701019c0 0x7faf701982f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf6000e9f0 tx=0x7faf6000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 shutdown_connections 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf540a2b80 0x7faf540a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701079c0 0x7faf7019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf70102360 0x7faf70198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 --2- 172.21.15.46:0/2444565609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf701019c0 0x7faf701982f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 >> 172.21.15.46:0/2444565609 conn(0x7faf700fd740 msgr2=0x7faf700fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 shutdown_connections 2024-09-10T07:33:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:53.995+0000 7faf74c00640 1 -- 172.21.15.46:0/2444565609 wait complete. 2024-09-10T07:33:54.397 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:53 smithi046 ceph-mon[30705]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:53 smithi181 ceph-mon[35397]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:53 smithi063 ceph-mon[35020]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:54.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:33:55.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:54 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2444565609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:33:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:55.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:54 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2444565609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:33:55.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:54 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2444565609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:33:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.914+0000 7f558f600640 1 -- 172.21.15.46:0/1633982060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55881049a0 msgr2=0x7f55881089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.914+0000 7f558f600640 1 --2- 172.21.15.46:0/1633982060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55881049a0 0x7f55881089f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5574009900 tx=0x7f557402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:33:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.914+0000 7f558f600640 1 -- 172.21.15.46:0/1633982060 shutdown_connections 2024-09-10T07:33:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.914+0000 7f558f600640 1 --2- 172.21.15.46:0/1633982060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5588109120 0x7f5588110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.914+0000 7f558f600640 1 --2- 172.21.15.46:0/1633982060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55881049a0 0x7f55881089f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.914+0000 7f558f600640 1 --2- 172.21.15.46:0/1633982060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5588103ff0 0x7f55881043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.914+0000 7f558f600640 1 -- 172.21.15.46:0/1633982060 >> 172.21.15.46:0/1633982060 conn(0x7f55880ffa90 msgr2=0x7f5588101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.915+0000 7f558f600640 1 -- 172.21.15.46:0/1633982060 shutdown_connections 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.915+0000 7f558f600640 1 -- 172.21.15.46:0/1633982060 wait complete. 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.916+0000 7f558f600640 1 Processor -- start 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.916+0000 7f558f600640 1 -- start start 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.916+0000 7f558f600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5588103ff0 0x7f558819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.916+0000 7f558f600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55881049a0 0x7f558819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5588109120 0x7f558819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5588114190 con 0x7f5588109120 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558f600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5588114010 con 0x7f5588103ff0 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558f600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5588114310 con 0x7f55881049a0 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5588103ff0 0x7f558819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5588109120 0x7f558819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5588103ff0 0x7f558819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56018/0 (socket says 172.21.15.46:56018) 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 -- 172.21.15.46:0/2853425110 learned_addr learned my addr 172.21.15.46:0/2853425110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f5587e00640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55881049a0 0x7f558819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 -- 172.21.15.46:0/2853425110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55881049a0 msgr2=0x7f558819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55881049a0 0x7f558819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 -- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5588109120 msgr2=0x7f558819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5588109120 0x7f558819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 -- 172.21.15.46:0/2853425110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558819f5e0 con 0x7f5588103ff0 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f5587e00640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55881049a0 0x7f558819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.917+0000 7f558cc00640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5588103ff0 0x7f558819a630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e9f0 tx=0x7f557c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.918+0000 7f5585600640 1 -- 172.21.15.46:0/2853425110 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c00ccc0 con 0x7f5588103ff0 2024-09-10T07:33:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.918+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558819f8d0 con 0x7f5588103ff0 2024-09-10T07:33:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.918+0000 7f5585600640 1 -- 172.21.15.46:0/2853425110 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f557c00ce60 con 0x7f5588103ff0 2024-09-10T07:33:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.918+0000 7f5585600640 1 -- 172.21.15.46:0/2853425110 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c010690 con 0x7f5588103ff0 2024-09-10T07:33:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.918+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55881a71b0 con 0x7f5588103ff0 2024-09-10T07:33:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.919+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55881043d0 con 0x7f5588103ff0 2024-09-10T07:33:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.921+0000 7f5585600640 1 -- 172.21.15.46:0/2853425110 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f557c0046c0 con 0x7f5588103ff0 2024-09-10T07:33:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.922+0000 7f5585600640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55700a2b80 0x7f55700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.923+0000 7f5585600640 1 -- 172.21.15.46:0/2853425110 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7f557c0cf780 con 0x7f5588103ff0 2024-09-10T07:33:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.923+0000 7f5587e00640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55700a2b80 0x7f55700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.923+0000 7f5587e00640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55700a2b80 0x7f55700a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5574000c00 tx=0x7f5574005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:55.925+0000 7f5585600640 1 -- 172.21.15.46:0/2853425110 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f557c09c440 con 0x7f5588103ff0 2024-09-10T07:33:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.112+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f558810c360 con 0x7f5588103ff0 2024-09-10T07:33:56.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:55 smithi046 ceph-mon[30705]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:56.113 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.114+0000 7f5585600640 1 -- 172.21.15.46:0/2853425110 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24128 (secure 0 0 0) 0x7f557c09bd40 con 0x7f5588103ff0 2024-09-10T07:33:56.115 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:33:49.099458+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:33:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55700a2b80 msgr2=0x7f55700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55700a2b80 0x7f55700a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5574000c00 tx=0x7f5574005a40 comp rx=0 tx=0).stop 2024-09-10T07:33:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5588103ff0 msgr2=0x7f558819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5588103ff0 0x7f558819a630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e9f0 tx=0x7f557c00edf0 comp rx=0 tx=0).stop 2024-09-10T07:33:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 shutdown_connections 2024-09-10T07:33:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f55700a2b80 0x7f55700a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5588109120 0x7f558819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55881049a0 0x7f558819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 --2- 172.21.15.46:0/2853425110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5588103ff0 0x7f558819a630 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 >> 172.21.15.46:0/2853425110 conn(0x7f55880ffa90 msgr2=0x7f5588100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.116+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 shutdown_connections 2024-09-10T07:33:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:56.117+0000 7f558f600640 1 -- 172.21.15.46:0/2853425110 wait complete. 2024-09-10T07:33:56.301 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:55 smithi063 ceph-mon[35020]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:55 smithi181 ceph-mon[35397]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:57.200 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:33:57.250 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:56 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2853425110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:33:57.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:56 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2853425110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:33:57.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:56 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2853425110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:33:57.577 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:33:58.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:57 smithi046 ceph-mon[30705]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 -- 172.21.15.46:0/3561143470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c1049a0 msgr2=0x7f388c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 --2- 172.21.15.46:0/3561143470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c1049a0 0x7f388c1089f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3878009900 tx=0x7f387802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 -- 172.21.15.46:0/3561143470 shutdown_connections 2024-09-10T07:33:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 --2- 172.21.15.46:0/3561143470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388c109120 0x7f388c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 --2- 172.21.15.46:0/3561143470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c1049a0 0x7f388c1089f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 --2- 172.21.15.46:0/3561143470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c103ff0 0x7f388c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 -- 172.21.15.46:0/3561143470 >> 172.21.15.46:0/3561143470 conn(0x7f388c0ffa90 msgr2=0x7f388c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 -- 172.21.15.46:0/3561143470 shutdown_connections 2024-09-10T07:33:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.178+0000 7f3891c00640 1 -- 172.21.15.46:0/3561143470 wait complete. 2024-09-10T07:33:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.180+0000 7f3891c00640 1 Processor -- start 2024-09-10T07:33:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.180+0000 7f3891c00640 1 -- start start 2024-09-10T07:33:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.180+0000 7f3891c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388c103ff0 0x7f388c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.180+0000 7f3891c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c1049a0 0x7f388c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.180+0000 7f3891c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c109120 0x7f388c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.180+0000 7f3891c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f388c114190 con 0x7f388c103ff0 2024-09-10T07:33:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.180+0000 7f3891c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f388c114010 con 0x7f388c109120 2024-09-10T07:33:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.180+0000 7f3891c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f388c114310 con 0x7f388c1049a0 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c109120 0x7f388c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388c103ff0 0x7f388c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c109120 0x7f388c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56046/0 (socket says 172.21.15.46:56046) 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 -- 172.21.15.46:0/1595261218 learned_addr learned my addr 172.21.15.46:0/1595261218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388a000640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c1049a0 0x7f388c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 -- 172.21.15.46:0/1595261218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c1049a0 msgr2=0x7f388c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c1049a0 0x7f388c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 -- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388c103ff0 msgr2=0x7f388c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388c103ff0 0x7f388c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 -- 172.21.15.46:0/1595261218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c19f5c0 con 0x7f388c109120 2024-09-10T07:33:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388a000640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c1049a0 0x7f388c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:33:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388aa00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388c103ff0 0x7f388c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:33:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.181+0000 7f388b400640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c109120 0x7f388c19eee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f388400eb10 tx=0x7f388400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.182+0000 7f3883400640 1 -- 172.21.15.46:0/1595261218 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884004b20 con 0x7f388c109120 2024-09-10T07:33:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.182+0000 7f3883400640 1 -- 172.21.15.46:0/1595261218 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3884004cc0 con 0x7f388c109120 2024-09-10T07:33:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.182+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c19f8b0 con 0x7f388c109120 2024-09-10T07:33:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.182+0000 7f3883400640 1 -- 172.21.15.46:0/1595261218 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884010760 con 0x7f388c109120 2024-09-10T07:33:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.182+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1a7190 con 0x7f388c109120 2024-09-10T07:33:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.183+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c1043d0 con 0x7f388c109120 2024-09-10T07:33:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.184+0000 7f3883400640 1 -- 172.21.15.46:0/1595261218 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3884005020 con 0x7f388c109120 2024-09-10T07:33:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.186+0000 7f3883400640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38700a2c00 0x7f38700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:33:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.186+0000 7f3883400640 1 -- 172.21.15.46:0/1595261218 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7f38840cf820 con 0x7f388c109120 2024-09-10T07:33:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.186+0000 7f388aa00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38700a2c00 0x7f38700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:33:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.187+0000 7f388aa00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38700a2c00 0x7f38700a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f387c009fc0 tx=0x7f387c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:33:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.190+0000 7f3883400640 1 -- 172.21.15.46:0/1595261218 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f388409c4e0 con 0x7f388c109120 2024-09-10T07:33:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:57 smithi181 ceph-mon[35397]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:58.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:57 smithi063 ceph-mon[35020]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:33:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.414+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f388c10bdd0 con 0x7f388c109120 2024-09-10T07:33:58.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.415+0000 7f3883400640 1 -- 172.21.15.46:0/1595261218 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f388409bde0 con 0x7f388c109120 2024-09-10T07:33:58.415 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:33:58.415 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"btime":"2024-09-10T07:33:49:932625+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:33:49.099669+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24535,"qdb_cluster":[24535]},"id":1}]} 2024-09-10T07:33:58.416 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-09-10T07:33:58.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38700a2c00 msgr2=0x7f38700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:58.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38700a2c00 0x7f38700a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f387c009fc0 tx=0x7f387c005f90 comp rx=0 tx=0).stop 2024-09-10T07:33:58.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c109120 msgr2=0x7f388c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c109120 0x7f388c19eee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f388400eb10 tx=0x7f388400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 shutdown_connections 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c109120 0x7f388c19eee0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c1049a0 0x7f388c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38700a2c00 0x7f38700a50c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 --2- 172.21.15.46:0/1595261218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388c103ff0 0x7f388c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 >> 172.21.15.46:0/1595261218 conn(0x7f388c0ffa90 msgr2=0x7f388c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 shutdown_connections 2024-09-10T07:33:58.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:33:58.418+0000 7f3891c00640 1 -- 172.21.15.46:0/1595261218 wait complete. 2024-09-10T07:33:59.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:58 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1595261218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:59.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:33:59.300 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:58 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1595261218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:59.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:58 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1595261218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:33:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:00.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:33:59 smithi046 ceph-mon[30705]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.244+0000 7f9154a00640 1 -- 172.21.15.46:0/195073097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91501049a0 msgr2=0x7f91501089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.244+0000 7f9154a00640 1 --2- 172.21.15.46:0/195073097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91501049a0 0x7f91501089f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f913c009900 tx=0x7f913c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.244+0000 7f9154a00640 1 -- 172.21.15.46:0/195073097 shutdown_connections 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.244+0000 7f9154a00640 1 --2- 172.21.15.46:0/195073097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9150109120 0x7f9150110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.244+0000 7f9154a00640 1 --2- 172.21.15.46:0/195073097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91501049a0 0x7f91501089f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.244+0000 7f9154a00640 1 --2- 172.21.15.46:0/195073097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9150103ff0 0x7f91501043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.244+0000 7f9154a00640 1 -- 172.21.15.46:0/195073097 >> 172.21.15.46:0/195073097 conn(0x7f91500ffa90 msgr2=0x7f9150101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.244+0000 7f9154a00640 1 -- 172.21.15.46:0/195073097 shutdown_connections 2024-09-10T07:34:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.245+0000 7f9154a00640 1 -- 172.21.15.46:0/195073097 wait complete. 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.246+0000 7f9154a00640 1 Processor -- start 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.246+0000 7f9154a00640 1 -- start start 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.246+0000 7f9154a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9150103ff0 0x7f915019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.246+0000 7f9154a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91501049a0 0x7f915019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.246+0000 7f9154a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9150109120 0x7f915019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.246+0000 7f9154a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9150114140 con 0x7f9150103ff0 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.246+0000 7f9154a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9150113fc0 con 0x7f9150109120 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.246+0000 7f9154a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f91501142c0 con 0x7f91501049a0 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9150103ff0 0x7f915019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9150103ff0 0x7f915019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41414/0 (socket says 172.21.15.46:41414) 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914de00640 1 -- 172.21.15.46:0/1571514500 learned_addr learned my addr 172.21.15.46:0/1571514500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914e800640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9150109120 0x7f915019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914d400640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91501049a0 0x7f915019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914de00640 1 -- 172.21.15.46:0/1571514500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91501049a0 msgr2=0x7f915019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914de00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91501049a0 0x7f915019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914de00640 1 -- 172.21.15.46:0/1571514500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9150109120 msgr2=0x7f915019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914de00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9150109120 0x7f915019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.247+0000 7f914de00640 1 -- 172.21.15.46:0/1571514500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915019f5d0 con 0x7f9150103ff0 2024-09-10T07:34:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.248+0000 7f914de00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9150103ff0 0x7f915019a620 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f914000e9f0 tx=0x7f914000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.248+0000 7f9146a00640 1 -- 172.21.15.46:0/1571514500 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914000cd80 con 0x7f9150103ff0 2024-09-10T07:34:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.248+0000 7f9146a00640 1 -- 172.21.15.46:0/1571514500 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9140005500 con 0x7f9150103ff0 2024-09-10T07:34:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.248+0000 7f9146a00640 1 -- 172.21.15.46:0/1571514500 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140010690 con 0x7f9150103ff0 2024-09-10T07:34:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.248+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915019f8c0 con 0x7f9150103ff0 2024-09-10T07:34:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.248+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91501a71a0 con 0x7f9150103ff0 2024-09-10T07:34:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.249+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91501043d0 con 0x7f9150103ff0 2024-09-10T07:34:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.251+0000 7f9146a00640 1 -- 172.21.15.46:0/1571514500 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f91400046c0 con 0x7f9150103ff0 2024-09-10T07:34:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.252+0000 7f9146a00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f91340a2c00 0x7f91340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.253+0000 7f9146a00640 1 -- 172.21.15.46:0/1571514500 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7f91400cf660 con 0x7f9150103ff0 2024-09-10T07:34:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.253+0000 7f914d400640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f91340a2c00 0x7f91340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:00.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.253+0000 7f914d400640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f91340a2c00 0x7f91340a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f913c009230 tx=0x7f913c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:00.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.257+0000 7f9146a00640 1 -- 172.21.15.46:0/1571514500 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f914009c320 con 0x7f9150103ff0 2024-09-10T07:34:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:33:59 smithi181 ceph-mon[35397]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:00.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:33:59 smithi063 ceph-mon[35020]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:00.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.444+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f915010c390 con 0x7f9150103ff0 2024-09-10T07:34:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.446+0000 7f9146a00640 1 -- 172.21.15.46:0/1571514500 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24128 (secure 0 0 0) 0x7f9140086020 con 0x7f9150103ff0 2024-09-10T07:34:00.446 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:00.446 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:33:49.099458+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:34:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.448+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f91340a2c00 msgr2=0x7f91340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.448+0000 7f9154a00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f91340a2c00 0x7f91340a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f913c009230 tx=0x7f913c0058d0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.448+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9150103ff0 msgr2=0x7f915019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.448+0000 7f9154a00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9150103ff0 0x7f915019a620 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f914000e9f0 tx=0x7f914000edf0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.449+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 shutdown_connections 2024-09-10T07:34:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.449+0000 7f9154a00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9150109120 0x7f915019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.449+0000 7f9154a00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91501049a0 0x7f915019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.449+0000 7f9154a00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f91340a2c00 0x7f91340a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.449+0000 7f9154a00640 1 --2- 172.21.15.46:0/1571514500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9150103ff0 0x7f915019a620 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.449+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 >> 172.21.15.46:0/1571514500 conn(0x7f91500ffa90 msgr2=0x7f9150100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.449+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 shutdown_connections 2024-09-10T07:34:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:00.449+0000 7f9154a00640 1 -- 172.21.15.46:0/1571514500 wait complete. 2024-09-10T07:34:01.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:00 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1571514500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:01.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:00 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1571514500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:01.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:00 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1571514500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:01.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:33:36.040805+0000', 'modified': '2024-09-10T07:33:49.099669+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24535': {'gid': 24535, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.46:6841/3146821117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6840', 'nonce': 3146821117}, {'type': 'v1', 'addr': '172.21.15.46:6841', 'nonce': 3146821117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24535, 'qdb_cluster': [24535]} 2024-09-10T07:34:01.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T07:34:01.454 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-10T07:34:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:02.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:01 smithi046 ceph-mon[30705]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:01 smithi181 ceph-mon[35397]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:02.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:01 smithi063 ceph-mon[35020]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 -- 172.21.15.46:0/2635479686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20c103e30 msgr2=0x7fa20c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 --2- 172.21.15.46:0/2635479686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20c103e30 0x7fa20c104210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009900 tx=0x7fa1fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 -- 172.21.15.46:0/2635479686 shutdown_connections 2024-09-10T07:34:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 --2- 172.21.15.46:0/2635479686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa20c108f60 0x7fa20c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 --2- 172.21.15.46:0/2635479686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa20c1047e0 0x7fa20c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 --2- 172.21.15.46:0/2635479686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20c103e30 0x7fa20c104210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 -- 172.21.15.46:0/2635479686 >> 172.21.15.46:0/2635479686 conn(0x7fa20c0ff8d0 msgr2=0x7fa20c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 -- 172.21.15.46:0/2635479686 shutdown_connections 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.419+0000 7fa211200640 1 -- 172.21.15.46:0/2635479686 wait complete. 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.420+0000 7fa211200640 1 Processor -- start 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.420+0000 7fa211200640 1 -- start start 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.420+0000 7fa211200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa20c103e30 0x7fa20c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa211200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa20c1047e0 0x7fa20c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa211200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20c108f60 0x7fa20c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa211200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c114010 con 0x7fa20c108f60 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa211200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c113e90 con 0x7fa20c103e30 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa211200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c114190 con 0x7fa20c1047e0 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa20c103e30 0x7fa20c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa20c103e30 0x7fa20c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56088/0 (socket says 172.21.15.46:56088) 2024-09-10T07:34:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20b400640 1 -- 172.21.15.46:0/501764767 learned_addr learned my addr 172.21.15.46:0/501764767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa202a00640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa20c1047e0 0x7fa20c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20be00640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20c108f60 0x7fa20c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20b400640 1 -- 172.21.15.46:0/501764767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa20c1047e0 msgr2=0x7fa20c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20b400640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa20c1047e0 0x7fa20c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20b400640 1 -- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20c108f60 msgr2=0x7fa20c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20b400640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20c108f60 0x7fa20c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.421+0000 7fa20b400640 1 -- 172.21.15.46:0/501764767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c19f440 con 0x7fa20c103e30 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.422+0000 7fa20b400640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa20c103e30 0x7fa20c19a490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc004770 tx=0x7fa1fc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.422+0000 7fa208c00640 1 -- 172.21.15.46:0/501764767 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc03f070 con 0x7fa20c103e30 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.422+0000 7fa211200640 1 -- 172.21.15.46:0/501764767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c19f6d0 con 0x7fa20c103e30 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.422+0000 7fa208c00640 1 -- 172.21.15.46:0/501764767 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc031ca0 con 0x7fa20c103e30 2024-09-10T07:34:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.422+0000 7fa208c00640 1 -- 172.21.15.46:0/501764767 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc035030 con 0x7fa20c103e30 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.422+0000 7fa211200640 1 -- 172.21.15.46:0/501764767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c1a7010 con 0x7fa20c103e30 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa21546c740 1 Processor -- start 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa21546c740 1 -- start start 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa208c00640 1 -- 172.21.15.46:0/501764767 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa1fc042070 con 0x7fa20c103e30 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa21546c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa21546c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe0878e00 0x55afe0883180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa21546c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08836c0 0x55afe0885ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa21546c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55afe07de0b0 con 0x55afe0878e00 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa21546c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55afe0785ab0 con 0x55afe08784e0 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.425+0000 7fa21546c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55afe07da070 con 0x55afe08836c0 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa20b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe0878e00 0x55afe0883180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa20b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe0878e00 0x55afe0883180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41428/0 (socket says 172.21.15.46:41428) 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa20b400640 1 -- 172.21.15.46:0/1529611041 learned_addr learned my addr 172.21.15.46:0/1529611041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa202a00640 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08836c0 0x55afe0885ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa20be00640 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa202a00640 1 -- 172.21.15.46:0/1529611041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 msgr2=0x55afe08788c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa202a00640 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08788c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa202a00640 1 -- 172.21.15.46:0/1529611041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe0878e00 msgr2=0x55afe0883180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa202a00640 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe0878e00 0x55afe0883180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa202a00640 1 -- 172.21.15.46:0/1529611041 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe07924a0 con 0x55afe08836c0 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.426+0000 7fa20b400640 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe0878e00 0x55afe0883180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.427+0000 7fa202a00640 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08836c0 0x55afe0885ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009bf0 tx=0x7fa1f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76cec8a6a8753763 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.427+0000 7fa208c00640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1f00a2b80 0x7fa1f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.427+0000 7fa200c00640 1 -- 172.21.15.46:0/1529611041 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4004050 con 0x55afe08836c0 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.427+0000 7fa208c00640 1 -- 172.21.15.46:0/501764767 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7fa1fc0ef8c0 con 0x7fa20c103e30 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.427+0000 7fa200c00640 1 -- 172.21.15.46:0/1529611041 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f4005500 con 0x55afe08836c0 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.427+0000 7fa200c00640 1 -- 172.21.15.46:0/1529611041 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40035f0 con 0x55afe08836c0 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.427+0000 7fa20be00640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1f00a2b80 0x7fa1f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 -- 172.21.15.46:0/1529611041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08836c0 msgr2=0x55afe0885ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08836c0 0x55afe0885ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009bf0 tx=0x7fa1f4011810 comp rx=0 tx=0).stop 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa20be00640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1f00a2b80 0x7fa1f00a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa204005fd0 tx=0x7fa204009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 -- 172.21.15.46:0/1529611041 shutdown_connections 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08836c0 0x55afe0885ab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe0878e00 0x55afe0883180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 --2- 172.21.15.46:0/1529611041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08788c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 -- 172.21.15.46:0/1529611041 >> 172.21.15.46:0/1529611041 conn(0x55afe0874a70 msgr2=0x55afe0874e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 -- 172.21.15.46:0/1529611041 shutdown_connections 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 -- 172.21.15.46:0/1529611041 wait complete. 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.428+0000 7fa21546c740 1 -- start start 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa21546c740 1 Processor -- start 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa21546c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08784e0 0x55afe08804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa21546c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe0880a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa21546c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08836c0 0x55afe0880f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa21546c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55afe0886330 con 0x55afe08784e0 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa21546c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55afe0766920 con 0x55afe08836c0 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa21546c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55afe08864b0 con 0x55afe0878e00 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa21546c740 10 client.0 _ll_register_callbacks cb 0x55afe078a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.429+0000 7fa20b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08784e0 0x55afe08804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa202a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe0880a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08784e0 0x55afe08804d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41440/0 (socket says 172.21.15.46:41440) 2024-09-10T07:34:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20b400640 1 -- 172.21.15.46:0/1909994869 learned_addr learned my addr 172.21.15.46:0/1909994869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20be00640 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08836c0 0x55afe0880f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20b400640 1 -- 172.21.15.46:0/1909994869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 msgr2=0x55afe0880a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20b400640 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe0880a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20b400640 1 -- 172.21.15.46:0/1909994869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08836c0 msgr2=0x55afe0880f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20b400640 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08836c0 0x55afe0880f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20b400640 1 -- 172.21.15.46:0/1909994869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe054f2b0 con 0x55afe08784e0 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa202a00640 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe0880a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20be00640 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08836c0 0x55afe0880f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa20b400640 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08784e0 0x55afe08804d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0c4680 tx=0x7fa1fc0b8350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1782cf11bbc66af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa1f9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.430+0000 7fa1f9600640 1 -- 172.21.15.46:0/1909994869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc049030 con 0x55afe08784e0 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa1f9600640 1 -- 172.21.15.46:0/1909994869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7fa1fc0b8660 con 0x55afe08784e0 2024-09-10T07:34:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 1 -- 172.21.15.46:0/1909994869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f4005500 con 0x55afe08784e0 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa1f9600640 1 -- 172.21.15.46:0/1909994869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc0bb950 con 0x55afe08784e0 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 10 client.14589 fetch_fsmap learned FSMap version 17 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 1 -- 172.21.15.46:0/1909994869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55afe0891f30 con 0x55afe08784e0 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa1f9600640 1 -- 172.21.15.46:0/1909994869 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f4005500 con 0x55afe08784e0 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa1f9600640 1 -- 172.21.15.46:0/1909994869 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fa1fc03b4f0 con 0x55afe08784e0 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 10 client.14589 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 10 client.14589 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 20 client.14589 populate_metadata read hostname 'smithi046' 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55afe0894540 0x55afe08959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 4 client.14589 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.431+0000 7fa21546c740 1 -- 172.21.15.46:0/1909994869 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afe07eb620 con 0x55afe0894540 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.432+0000 7fa202a00640 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55afe0894540 0x55afe08959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.432+0000 7fa202a00640 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55afe0894540 0x55afe08959b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4069b41235b48bdf server_cookie=549f536dfdc0c532 in_seq=0 out_seq=0 2024-09-10T07:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.432+0000 7fa1f9600640 10 client.14589 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.433+0000 7fa1f9600640 1 -- 172.21.15.46:0/1909994869 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55afe07eb620 con 0x55afe0894540 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.433+0000 7fa1f9600640 10 client.14589 handle_command_reply: tid=0 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.433+0000 7fa21546c740 1 client.14589 shutdown 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.433+0000 7fa21546c740 1 -- 172.21.15.46:0/1909994869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08784e0 msgr2=0x55afe08804d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.433+0000 7fa21546c740 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08784e0 0x55afe08804d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0c4680 tx=0x7fa1fc0b8350 comp rx=0 tx=0).stop 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 1 -- 172.21.15.46:0/1909994869 shutdown_connections 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55afe0894540 0x55afe08959b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08836c0 0x55afe0880f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa1f9600640 0 client.14589 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe0880a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 1 --2- 172.21.15.46:0/1909994869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08784e0 0x55afe08804d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 1 -- 172.21.15.46:0/1909994869 >> 172.21.15.46:0/1909994869 conn(0x55afe0874a70 msgr2=0x55afe087a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 1 -- 172.21.15.46:0/1909994869 shutdown_connections 2024-09-10T07:34:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 1 -- 172.21.15.46:0/1909994869 wait complete. 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.434+0000 7fa21546c740 20 client.14589 trim_cache size 0 max 16384 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.452+0000 7fa21546c740 1 Processor -- start 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa21546c740 1 -- start start 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa21546c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa21546c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe087fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa21546c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe088ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa21546c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55afe0893cf0 con 0x55afe08836c0 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa21546c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55afe08943c0 con 0x55afe08784e0 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa21546c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55afe0894a90 con 0x55afe0878e00 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe088ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe088ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41454/0 (socket says 172.21.15.46:41454) 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20be00640 1 -- 172.21.15.46:0/1869158256 learned_addr learned my addr 172.21.15.46:0/1869158256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20b400640 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa202a00640 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe087fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20be00640 1 -- 172.21.15.46:0/1869158256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 msgr2=0x55afe087fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20be00640 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe087fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20be00640 1 -- 172.21.15.46:0/1869158256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 msgr2=0x55afe08952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20be00640 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20be00640 1 -- 172.21.15.46:0/1869158256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe08748a0 con 0x55afe08836c0 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa20b400640 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.453+0000 7fa202a00640 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe087fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:34:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.454+0000 7fa20be00640 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe088ee40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa204004310 tx=0x7fa20401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d094021207e0d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.454+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1869158256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204004b20 con 0x55afe08836c0 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.454+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1869158256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa204004cc0 con 0x55afe08836c0 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.454+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1869158256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2040058b0 con 0x55afe08836c0 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 -- 172.21.15.46:0/1869158256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 msgr2=0x55afe088ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe088ee40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa204004310 tx=0x7fa20401a000 comp rx=0 tx=0).stop 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 -- 172.21.15.46:0/1869158256 shutdown_connections 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe088ee40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe0878e00 0x55afe087fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 --2- 172.21.15.46:0/1869158256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe08784e0 0x55afe08952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 -- 172.21.15.46:0/1869158256 >> 172.21.15.46:0/1869158256 conn(0x55afe0874a70 msgr2=0x55afe088e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 -- 172.21.15.46:0/1869158256 shutdown_connections 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 -- 172.21.15.46:0/1869158256 wait complete. 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.455+0000 7fa21546c740 1 -- start start 2024-09-10T07:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.456+0000 7fa21546c740 1 Processor -- start 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.456+0000 7fa21546c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08784e0 0x55afe089e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.456+0000 7fa21546c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe0878e00 0x55afe087d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.456+0000 7fa21546c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe087d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.456+0000 7fa21546c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55afe0895830 con 0x55afe08836c0 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.456+0000 7fa21546c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55afe0895160 con 0x55afe0878e00 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.456+0000 7fa21546c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55afe0895f00 con 0x55afe08784e0 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa21546c740 10 client.0 _ll_register_callbacks cb 0x7fa1fc031e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa20be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe087d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa202a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe0878e00 0x55afe087d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa20be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe087d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41470/0 (socket says 172.21.15.46:41470) 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa20be00640 1 -- 172.21.15.46:0/2081231948 learned_addr learned my addr 172.21.15.46:0/2081231948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa20b400640 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08784e0 0x55afe089e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa202a00640 1 -- 172.21.15.46:0/2081231948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08784e0 msgr2=0x55afe089e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa202a00640 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08784e0 0x55afe089e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa202a00640 1 -- 172.21.15.46:0/2081231948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 msgr2=0x55afe087d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa202a00640 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe087d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.457+0000 7fa202a00640 1 -- 172.21.15.46:0/2081231948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe087e090 con 0x55afe0878e00 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.458+0000 7fa202a00640 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe0878e00 0x55afe087d370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4004a20 tx=0x7fa1f4005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc804a0b42fc6a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.458+0000 7fa1f9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.458+0000 7fa1f9600640 1 -- 172.21.15.46:0/2081231948 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40035f0 con 0x55afe0878e00 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.458+0000 7fa21546c740 1 -- 172.21.15.46:0/2081231948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa204004cc0 con 0x55afe0878e00 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.458+0000 7fa1f9600640 1 -- 172.21.15.46:0/2081231948 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7fa1f4003790 con 0x55afe0878e00 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.458+0000 7fa1f9600640 1 -- 172.21.15.46:0/2081231948 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa1f403b050 con 0x55afe0878e00 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.458+0000 7fa21546c740 10 client.24632 fetch_fsmap learned FSMap version 17 2024-09-10T07:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.458+0000 7fa21546c740 1 -- 172.21.15.46:0/2081231948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55afe08fec50 con 0x55afe0878e00 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa1f9600640 1 -- 172.21.15.46:0/2081231948 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa204004cc0 con 0x55afe0878e00 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa1f9600640 1 -- 172.21.15.46:0/2081231948 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fa1f402e4d0 con 0x55afe0878e00 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa21546c740 10 client.24632 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa21546c740 10 client.24632 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa21546c740 20 client.24632 populate_metadata read hostname 'smithi046' 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa21546c740 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55afe0903630 0x55afe0902340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa21546c740 4 client.24632 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa21546c740 1 -- 172.21.15.46:0/2081231948 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "status"}) -- 0x55afe07e5b30 con 0x55afe0903630 2024-09-10T07:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.459+0000 7fa20b400640 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55afe0903630 0x55afe0902340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa20b400640 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55afe0903630 0x55afe0902340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281c671020f3a711 server_cookie=6c1de6900da20cc5 in_seq=0 out_seq=0 2024-09-10T07:34:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa1f9600640 10 client.24632 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:34:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa1f9600640 1 -- 172.21.15.46:0/2081231948 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55afe07e5b30 con 0x55afe0903630 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa1f9600640 10 client.24632 handle_command_reply: tid=0 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 client.24632 shutdown 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 -- 172.21.15.46:0/2081231948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe0878e00 msgr2=0x55afe087d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe0878e00 0x55afe087d370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4004a20 tx=0x7fa1f4005d30 comp rx=0 tx=0).stop 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 -- 172.21.15.46:0/2081231948 shutdown_connections 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55afe0903630 0x55afe0902340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afe08836c0 0x55afe087d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55afe0878e00 0x55afe087d370 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 --2- 172.21.15.46:0/2081231948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afe08784e0 0x55afe089e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa1f9600640 0 client.24632 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.460+0000 7fa21546c740 1 -- 172.21.15.46:0/2081231948 >> 172.21.15.46:0/2081231948 conn(0x55afe0874a70 msgr2=0x55afe0873bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.461+0000 7fa21546c740 1 -- 172.21.15.46:0/2081231948 shutdown_connections 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.461+0000 7fa21546c740 1 -- 172.21.15.46:0/2081231948 wait complete. 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.461+0000 7fa21546c740 20 client.24632 trim_cache size 0 max 16384 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24535, 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:34:02.477 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 25.587388254, 2024-09-10T07:34:02.478 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 17, 2024-09-10T07:34:02.478 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2024-09-10T07:34:02.478 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 93, 2024-09-10T07:34:02.478 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 41.980761784000002 2024-09-10T07:34:02.478 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:34:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.461+0000 7fa211200640 1 -- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1f00a2b80 msgr2=0x7fa1f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.461+0000 7fa211200640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1f00a2b80 0x7fa1f00a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa204005fd0 tx=0x7fa204009500 comp rx=0 tx=0).stop 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.461+0000 7fa211200640 1 -- 172.21.15.46:0/501764767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa20c103e30 msgr2=0x7fa20c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.461+0000 7fa211200640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa20c103e30 0x7fa20c19a490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc004770 tx=0x7fa1fc003c10 comp rx=0 tx=0).stop 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.462+0000 7fa211200640 1 -- 172.21.15.46:0/501764767 shutdown_connections 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.462+0000 7fa211200640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1f00a2b80 0x7fa1f00a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.462+0000 7fa211200640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20c108f60 0x7fa20c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.462+0000 7fa211200640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa20c1047e0 0x7fa20c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.462+0000 7fa211200640 1 --2- 172.21.15.46:0/501764767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa20c103e30 0x7fa20c19a490 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.462+0000 7fa211200640 1 -- 172.21.15.46:0/501764767 >> 172.21.15.46:0/501764767 conn(0x7fa20c0ff8d0 msgr2=0x7fa20c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.462+0000 7fa211200640 1 -- 172.21.15.46:0/501764767 shutdown_connections 2024-09-10T07:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:02.462+0000 7fa211200640 1 -- 172.21.15.46:0/501764767 wait complete. 2024-09-10T07:34:03.410 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T07:34:03.410 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:03.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T07:34:03.787 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:04.186 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:03 smithi046 ceph-mon[30705]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:03 smithi181 ceph-mon[35397]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:03 smithi063 ceph-mon[35020]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:04.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.419+0000 7fbd41e00640 1 -- 172.21.15.46:0/639141551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074eb0 msgr2=0x7fbd3c112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:04.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.419+0000 7fbd41e00640 1 --2- 172.21.15.46:0/639141551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074eb0 0x7fbd3c112650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009900 tx=0x7fbd2402a480 comp rx=0 tx=0).stop 2024-09-10T07:34:04.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.420+0000 7fbd41e00640 1 -- 172.21.15.46:0/639141551 shutdown_connections 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.420+0000 7fbd41e00640 1 --2- 172.21.15.46:0/639141551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3c112b90 0x7fbd3c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.420+0000 7fbd41e00640 1 --2- 172.21.15.46:0/639141551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074eb0 0x7fbd3c112650 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.420+0000 7fbd41e00640 1 --2- 172.21.15.46:0/639141551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c074590 0x7fbd3c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.420+0000 7fbd41e00640 1 -- 172.21.15.46:0/639141551 >> 172.21.15.46:0/639141551 conn(0x7fbd3c0fd8c0 msgr2=0x7fbd3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.420+0000 7fbd41e00640 1 -- 172.21.15.46:0/639141551 shutdown_connections 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.420+0000 7fbd41e00640 1 -- 172.21.15.46:0/639141551 wait complete. 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd41e00640 1 Processor -- start 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd41e00640 1 -- start start 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd41e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074590 0x7fbd3c1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd41e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c074eb0 0x7fbd3c1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd41e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3c112b90 0x7fbd3c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd41e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1185b0 con 0x7fbd3c074590 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd41e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c118430 con 0x7fbd3c112b90 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd41e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c118730 con 0x7fbd3c074eb0 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074590 0x7fbd3c1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074590 0x7fbd3c1a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41490/0 (socket says 172.21.15.46:41490) 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.421+0000 7fbd3aa00640 1 -- 172.21.15.46:0/2132695546 learned_addr learned my addr 172.21.15.46:0/2132695546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd33e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c074eb0 0x7fbd3c1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd3b400640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3c112b90 0x7fbd3c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd3aa00640 1 -- 172.21.15.46:0/2132695546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c074eb0 msgr2=0x7fbd3c1a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd3aa00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c074eb0 0x7fbd3c1a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd3aa00640 1 -- 172.21.15.46:0/2132695546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3c112b90 msgr2=0x7fbd3c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd3aa00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3c112b90 0x7fbd3c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd3aa00640 1 -- 172.21.15.46:0/2132695546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c19f020 con 0x7fbd3c074590 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd3b400640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3c112b90 0x7fbd3c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd3aa00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074590 0x7fbd3c1a4730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00e9f0 tx=0x7fbd2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.422+0000 7fbd40a00640 1 -- 172.21.15.46:0/2132695546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00cd80 con 0x7fbd3c074590 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.423+0000 7fbd40a00640 1 -- 172.21.15.46:0/2132695546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c005500 con 0x7fbd3c074590 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.423+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c19f310 con 0x7fbd3c074590 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.423+0000 7fbd40a00640 1 -- 172.21.15.46:0/2132695546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c010690 con 0x7fbd3c074590 2024-09-10T07:34:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.423+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1ab600 con 0x7fbd3c074590 2024-09-10T07:34:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.424+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3c074970 con 0x7fbd3c074590 2024-09-10T07:34:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.426+0000 7fbd40a00640 1 -- 172.21.15.46:0/2132695546 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbd2c0046c0 con 0x7fbd3c074590 2024-09-10T07:34:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.427+0000 7fbd40a00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbd200a2b80 0x7fbd200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.427+0000 7fbd40a00640 1 -- 172.21.15.46:0/2132695546 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7fbd2c0cf770 con 0x7fbd3c074590 2024-09-10T07:34:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.427+0000 7fbd33e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbd200a2b80 0x7fbd200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.428+0000 7fbd33e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbd200a2b80 0x7fbd200a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd24000c00 tx=0x7fbd24005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.431+0000 7fbd40a00640 1 -- 172.21.15.46:0/2132695546 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd2c09c430 con 0x7fbd3c074590 2024-09-10T07:34:04.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.657+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbd3c02d7c0 con 0x7fbd3c074590 2024-09-10T07:34:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.658+0000 7fbd40a00640 1 -- 172.21.15.46:0/2132695546 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fbd2c09bd30 con 0x7fbd3c074590 2024-09-10T07:34:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T07:34:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbd200a2b80 msgr2=0x7fbd200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbd200a2b80 0x7fbd200a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd24000c00 tx=0x7fbd24005b50 comp rx=0 tx=0).stop 2024-09-10T07:34:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074590 msgr2=0x7fbd3c1a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074590 0x7fbd3c1a4730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00e9f0 tx=0x7fbd2c00edf0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 shutdown_connections 2024-09-10T07:34:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3c112b90 0x7fbd3c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c074eb0 0x7fbd3c1a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbd200a2b80 0x7fbd200a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 --2- 172.21.15.46:0/2132695546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c074590 0x7fbd3c1a4730 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.660+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 >> 172.21.15.46:0/2132695546 conn(0x7fbd3c0fd8c0 msgr2=0x7fbd3c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.661+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 shutdown_connections 2024-09-10T07:34:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:04.661+0000 7fbd41e00640 1 -- 172.21.15.46:0/2132695546 wait complete. 2024-09-10T07:34:05.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:04 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2132695546' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T07:34:05.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:04 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2132695546' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T07:34:05.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:04 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2132695546' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T07:34:05.502 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T07:34:05.502 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:34:05.502 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-10T07:34:05.502 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T07:34:05.502 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:05.502 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T07:34:05.502 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T07:34:05.503 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:34:05.503 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T07:34:05.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:05.503 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-10T07:34:05.539 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:34:05.539 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:34:05.539 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:34:05.539 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:34:05.539 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42200sec preferred_lft 42200sec 2024-09-10T07:34:05.540 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-10T07:34:05.541 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:34:05.541 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:34:05.541 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:34:05.541 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-10T07:34:05.542 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T07:34:05.542 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:34:05.542 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:34:05.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-09-10T07:34:05.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-09-10T07:34:05.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-09-10T07:34:05.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T07:34:05.542 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:34:05.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:34:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:34:05.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:05.971 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:34:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-10T07:34:06.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:06.033 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-10T07:34:06.088 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-10T07:34:06.088 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:34:06.088 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:34:06.088 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:34:06.088 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:34:06.089 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:34:06.089 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:34:06.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:34:06.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:34:06.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:34:06.089 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:34:06.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:05 smithi046 ceph-mon[30705]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:06.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:34:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:05 smithi181 ceph-mon[35397]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:05 smithi063 ceph-mon[35020]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:34:06.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:06.685 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-10T07:34:06.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:06.721 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-09-10T07:34:06.797 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:34:06.797 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:34:06.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T07:34:06.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T07:34:06.797 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:34:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:34:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:34:06.979 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T07:34:06.980 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:34:06.980 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:34:06.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T07:34:06.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T07:34:06.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T07:34:06.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T07:34:06.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T07:34:06.980 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:34:07.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:34:07.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:34:07.236 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:34:07.236 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:34:07.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-10T07:34:07.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T07:34:07.236 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:34:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:34:07.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:34:07.403 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T07:34:07.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:34:07.404 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:07.439 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T07:34:07.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:34:07.440 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:07.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-10T07:34:07.616 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-10T07:34:07.653 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-10T07:34:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/0e61913d9523172580dbe2c3a8eb69e2a2ad3a11e3ca356389faa3463b59c4bd/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/e79ffbde18f4c118f543e7577cd2a33b8a80714fb96447e56da8f46811776910/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/2404bd23c1d6a9e4e46bc4c934f04db221781714ef4e89ce3e85a4c96decdbf7/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/d236d9452ff4d620ef930693e355234e5f2841870997e70d126510464daff437/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/76dd6da98b79fef8ba7b0525333bd9e8bf092dc861557dcf04081f60f880926d/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f938bb438bb4371717247e3e560a33394ce22be2ce4a538ca01129fac1129d74/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ea3a6da8a1adcfca5826774ef16d8415568ce3780e88fcb950a5be624b31f506/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/191b63cb8c7e9fde68e2461b211589ac25ded3d9b854d1238b3cb5fc66eabaa5/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/793579a55abd78d724baf9ca9f40d7d6ddf4a422d6b4112745e6f1294c4ed098/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/cb2f5c8b9bff0456d2a419869543c9ae9f13654a2dec5299dcd8344052a502ee/merged 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-10T07:34:07.655 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T07:34:07.656 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T07:34:07.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:07.657 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-10T07:34:07.732 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T07:34:07.732 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T07:34:07.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-10T07:34:07.834 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T07:34:07.872 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-10T07:34:07.872 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-10T07:34:07.872 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-10T07:34:07.872 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-10T07:34:07.872 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-10T07:34:07.872 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-10T07:34:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-10T07:34:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-10T07:34:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-10T07:34:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-10T07:34:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-10T07:34:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-10T07:34:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T07:34:07.873 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-10T07:34:07.874 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-10T07:34:07.874 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-10T07:34:07.874 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-10T07:34:07.874 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-10T07:34:07.874 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-10T07:34:07.874 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T07:34:07.874 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T07:34:07.874 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/0e61913d9523172580dbe2c3a8eb69e2a2ad3a11e3ca356389faa3463b59c4bd/merged 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/e79ffbde18f4c118f543e7577cd2a33b8a80714fb96447e56da8f46811776910/merged 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T07:34:07.875 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/2404bd23c1d6a9e4e46bc4c934f04db221781714ef4e89ce3e85a4c96decdbf7/merged 2024-09-10T07:34:07.876 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/d236d9452ff4d620ef930693e355234e5f2841870997e70d126510464daff437/merged 2024-09-10T07:34:07.876 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/76dd6da98b79fef8ba7b0525333bd9e8bf092dc861557dcf04081f60f880926d/merged 2024-09-10T07:34:07.876 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f938bb438bb4371717247e3e560a33394ce22be2ce4a538ca01129fac1129d74/merged 2024-09-10T07:34:07.876 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ea3a6da8a1adcfca5826774ef16d8415568ce3780e88fcb950a5be624b31f506/merged 2024-09-10T07:34:07.876 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/191b63cb8c7e9fde68e2461b211589ac25ded3d9b854d1238b3cb5fc66eabaa5/merged 2024-09-10T07:34:07.876 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/793579a55abd78d724baf9ca9f40d7d6ddf4a422d6b4112745e6f1294c4ed098/merged 2024-09-10T07:34:07.876 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/cb2f5c8b9bff0456d2a419869543c9ae9f13654a2dec5299dcd8344052a502ee/merged 2024-09-10T07:34:07.876 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-10T07:34:07.877 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T07:34:07.877 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T07:34:07.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:07.878 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-10T07:34:07.975 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:2024-09-10T07:34:07.976+0000 7efe6531d480 -1 init, newargv = 0x55cf6ef84410 newargc=15 2024-09-10T07:34:07.975 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[71765]: starting ceph client 2024-09-10T07:34:07.985 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[71765]: starting fuse 2024-09-10T07:34:08.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:07 smithi046 ceph-mon[30705]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:07 smithi181 ceph-mon[35397]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:07 smithi063 ceph-mon[35020]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:09.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-10T07:34:09.121 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T07:34:09.159 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-10T07:34:09.159 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-10T07:34:09.159 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-10T07:34:09.159 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-10T07:34:09.159 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-10T07:34:09.159 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-10T07:34:09.159 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-10T07:34:09.160 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/0e61913d9523172580dbe2c3a8eb69e2a2ad3a11e3ca356389faa3463b59c4bd/merged 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/e79ffbde18f4c118f543e7577cd2a33b8a80714fb96447e56da8f46811776910/merged 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/2404bd23c1d6a9e4e46bc4c934f04db221781714ef4e89ce3e85a4c96decdbf7/merged 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/d236d9452ff4d620ef930693e355234e5f2841870997e70d126510464daff437/merged 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/76dd6da98b79fef8ba7b0525333bd9e8bf092dc861557dcf04081f60f880926d/merged 2024-09-10T07:34:09.161 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f938bb438bb4371717247e3e560a33394ce22be2ce4a538ca01129fac1129d74/merged 2024-09-10T07:34:09.162 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ea3a6da8a1adcfca5826774ef16d8415568ce3780e88fcb950a5be624b31f506/merged 2024-09-10T07:34:09.162 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/191b63cb8c7e9fde68e2461b211589ac25ded3d9b854d1238b3cb5fc66eabaa5/merged 2024-09-10T07:34:09.162 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/793579a55abd78d724baf9ca9f40d7d6ddf4a422d6b4112745e6f1294c4ed098/merged 2024-09-10T07:34:09.162 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/cb2f5c8b9bff0456d2a419869543c9ae9f13654a2dec5299dcd8344052a502ee/merged 2024-09-10T07:34:09.162 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-10T07:34:09.162 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T07:34:09.162 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T07:34:09.162 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:09.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:09.163 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-10T07:34:09.241 INFO:teuthology.orchestra.run.smithi046.stdout:120 2024-09-10T07:34:09.241 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T07:34:09.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-09-10T07:34:09.242 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.admin")) 2024-09-10T07:34:09.243 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-09-10T07:34:09.408 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.admin.71765.asok 2024-09-10T07:34:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:34:09.426 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.71765.asok 2024-09-10T07:34:09.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:09.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.71765.asok status 2024-09-10T07:34:09.613 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:34:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-09-10T07:34:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T07:34:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "admin", 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "71765", 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24644, 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-09-10T07:34:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 24644 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1203732406 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1203732406 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.24644 172.21.15.46:0/1203732406", 2024-09-10T07:34:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "172.21.15.46:0/1203732406", 2024-09-10T07:34:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-09-10T07:34:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 17, 2024-09-10T07:34:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 93, 2024-09-10T07:34:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-09-10T07:34:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-09-10T07:34:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-09-10T07:34:09.616 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:34:09.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:09.626 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-09-10T07:34:09.626 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:09.662 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-09-10T07:34:09.662 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:09.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:09.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:09.759 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T07:34:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:34:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:# file: . 2024-09-10T07:34:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T07:34:09.861 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:09.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:34:09.874 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:34:09.875 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:34:09.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T07:34:10.001 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:09 smithi046 ceph-mon[30705]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:09 smithi181 ceph-mon[35397]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:09 smithi063 ceph-mon[35020]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:34:10.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 -- 172.21.15.46:0/1601456717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0101de0 msgr2=0x7faff01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 --2- 172.21.15.46:0/1601456717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0101de0 0x7faff01021c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009900 tx=0x7fafe402a400 comp rx=0 tx=0).stop 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 -- 172.21.15.46:0/1601456717 shutdown_connections 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 --2- 172.21.15.46:0/1601456717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff0106f10 0x7faff010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 --2- 172.21.15.46:0/1601456717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff0102790 0x7faff01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 --2- 172.21.15.46:0/1601456717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0101de0 0x7faff01021c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 -- 172.21.15.46:0/1601456717 >> 172.21.15.46:0/1601456717 conn(0x7faff00fd8c0 msgr2=0x7faff00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 -- 172.21.15.46:0/1601456717 shutdown_connections 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.952+0000 7faff7c00640 1 -- 172.21.15.46:0/1601456717 wait complete. 2024-09-10T07:34:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.953+0000 7faff7c00640 1 Processor -- start 2024-09-10T07:34:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.953+0000 7faff7c00640 1 -- start start 2024-09-10T07:34:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.953+0000 7faff7c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0101de0 0x7faff019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.953+0000 7faff7c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff0102790 0x7faff019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.953+0000 7faff7c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff0106f10 0x7faff019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.953+0000 7faff7c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faff0111f70 con 0x7faff0101de0 2024-09-10T07:34:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.953+0000 7faff7c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faff0111df0 con 0x7faff0102790 2024-09-10T07:34:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.953+0000 7faff7c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faff01120f0 con 0x7faff0106f10 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7fafefe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff0102790 0x7faff019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7faff5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0101de0 0x7faff019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7fafefe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff0102790 0x7faff019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50138/0 (socket says 172.21.15.46:50138) 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7fafefe00640 1 -- 172.21.15.46:0/1829996528 learned_addr learned my addr 172.21.15.46:0/1829996528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7faff5c00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff0106f10 0x7faff019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7fafefe00640 1 -- 172.21.15.46:0/1829996528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff0106f10 msgr2=0x7faff019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7fafefe00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff0106f10 0x7faff019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7fafefe00640 1 -- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0101de0 msgr2=0x7faff019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7fafefe00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0101de0 0x7faff019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.954+0000 7fafefe00640 1 -- 172.21.15.46:0/1829996528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff019f650 con 0x7faff0102790 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.955+0000 7fafefe00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff0102790 0x7faff019abe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00d770 tx=0x7fafdc00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.955+0000 7fafed600640 1 -- 172.21.15.46:0/1829996528 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc017070 con 0x7faff0102790 2024-09-10T07:34:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.955+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff019f940 con 0x7faff0102790 2024-09-10T07:34:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.955+0000 7fafed600640 1 -- 172.21.15.46:0/1829996528 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafdc005500 con 0x7faff0102790 2024-09-10T07:34:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.955+0000 7fafed600640 1 -- 172.21.15.46:0/1829996528 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc004e00 con 0x7faff0102790 2024-09-10T07:34:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.955+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01a7220 con 0x7faff0102790 2024-09-10T07:34:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.956+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff01021c0 con 0x7faff0102790 2024-09-10T07:34:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.958+0000 7fafed600640 1 -- 172.21.15.46:0/1829996528 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fafdc004fa0 con 0x7faff0102790 2024-09-10T07:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.960+0000 7fafed600640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fafd80a2b80 0x7fafd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.960+0000 7fafed600640 1 -- 172.21.15.46:0/1829996528 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7fafdc0cf090 con 0x7faff0102790 2024-09-10T07:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.960+0000 7faff5200640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fafd80a2b80 0x7fafd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.961+0000 7faff5200640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fafd80a2b80 0x7fafd80a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009230 tx=0x7fafe40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:10.962+0000 7fafed600640 1 -- 172.21.15.46:0/1829996528 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafdc09b700 con 0x7faff0102790 2024-09-10T07:34:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.163+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faff01a7550 con 0x7faff0102790 2024-09-10T07:34:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.165+0000 7fafed600640 1 -- 172.21.15.46:0/1829996528 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+590 (secure 0 0 0) 0x7fafdc09fa00 con 0x7faff0102790 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2972952762 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2972952762 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1860671836 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3724700252 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3653813867 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/141501936 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3003000819 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/141501936 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1462945451 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3598352774 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-09-10T07:34:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.167+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fafd80a2b80 msgr2=0x7fafd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.167+0000 7faff7c00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fafd80a2b80 0x7fafd80a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009230 tx=0x7fafe40058d0 comp rx=0 tx=0).stop 2024-09-10T07:34:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.167+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff0102790 msgr2=0x7faff019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.167+0000 7faff7c00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff0102790 0x7faff019abe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00d770 tx=0x7fafdc00dc30 comp rx=0 tx=0).stop 2024-09-10T07:34:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.168+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 shutdown_connections 2024-09-10T07:34:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.168+0000 7faff7c00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff0106f10 0x7faff019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.168+0000 7faff7c00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff0102790 0x7faff019abe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.168+0000 7faff7c00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fafd80a2b80 0x7fafd80a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.168+0000 7faff7c00640 1 --2- 172.21.15.46:0/1829996528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0101de0 0x7faff019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.168+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 >> 172.21.15.46:0/1829996528 conn(0x7faff00fd8c0 msgr2=0x7faff00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.168+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 shutdown_connections 2024-09-10T07:34:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:11.168+0000 7faff7c00640 1 -- 172.21.15.46:0/1829996528 wait complete. 2024-09-10T07:34:11.958 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:34:11.958 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:34:11.996 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T07:34:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:11 smithi181 ceph-mon[35397]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T07:34:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:11 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1829996528' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:34:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:11 smithi063 ceph-mon[35020]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T07:34:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:11 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1829996528' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:34:12.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:11 smithi046 ceph-mon[30705]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T07:34:12.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:11 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1829996528' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:34:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.992+0000 7f0cee000640 1 -- 172.21.15.46:0/912286438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce8103ff0 msgr2=0x7f0ce81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.992+0000 7f0cee000640 1 --2- 172.21.15.46:0/912286438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce8103ff0 0x7f0ce81043d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009900 tx=0x7f0cd802a410 comp rx=0 tx=0).stop 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.993+0000 7f0cee000640 1 -- 172.21.15.46:0/912286438 shutdown_connections 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.993+0000 7f0cee000640 1 --2- 172.21.15.46:0/912286438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ce8109120 0x7f0ce8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.993+0000 7f0cee000640 1 --2- 172.21.15.46:0/912286438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ce81049a0 0x7f0ce81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.993+0000 7f0cee000640 1 --2- 172.21.15.46:0/912286438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce8103ff0 0x7f0ce81043d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.993+0000 7f0cee000640 1 -- 172.21.15.46:0/912286438 >> 172.21.15.46:0/912286438 conn(0x7f0ce80ffa90 msgr2=0x7f0ce8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.993+0000 7f0cee000640 1 -- 172.21.15.46:0/912286438 shutdown_connections 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.993+0000 7f0cee000640 1 -- 172.21.15.46:0/912286438 wait complete. 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0cee000640 1 Processor -- start 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0cee000640 1 -- start start 2024-09-10T07:34:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0cee000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ce8103ff0 0x7f0ce819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0cee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ce81049a0 0x7f0ce819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0cee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce8109120 0x7f0ce819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0cee000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8114150 con 0x7f0ce8109120 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0cee000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8113fd0 con 0x7f0ce81049a0 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0cee000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81142d0 con 0x7f0ce8103ff0 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0ce7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ce8103ff0 0x7f0ce819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce8109120 0x7f0ce819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0ce7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ce8103ff0 0x7f0ce819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33530/0 (socket says 172.21.15.46:33530) 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.994+0000 7f0ce7400640 1 -- 172.21.15.46:0/2954667161 learned_addr learned my addr 172.21.15.46:0/2954667161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.995+0000 7f0ce6a00640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ce81049a0 0x7f0ce819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.995+0000 7f0ce7400640 1 -- 172.21.15.46:0/2954667161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ce81049a0 msgr2=0x7f0ce819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.995+0000 7f0ce7400640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ce81049a0 0x7f0ce819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.995+0000 7f0ce7400640 1 -- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce8109120 msgr2=0x7f0ce819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.995+0000 7f0ce7400640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce8109120 0x7f0ce819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.995+0000 7f0ce7400640 1 -- 172.21.15.46:0/2954667161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce819f5e0 con 0x7f0ce8103ff0 2024-09-10T07:34:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.995+0000 7f0ce7400640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ce8103ff0 0x7f0ce819a630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8006010 tx=0x7f0cd8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.996+0000 7f0cdfe00640 1 -- 172.21.15.46:0/2954667161 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd803f030 con 0x7f0ce8103ff0 2024-09-10T07:34:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.996+0000 7f0cdfe00640 1 -- 172.21.15.46:0/2954667161 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8004440 con 0x7f0ce8103ff0 2024-09-10T07:34:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.996+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce819f870 con 0x7f0ce8103ff0 2024-09-10T07:34:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.996+0000 7f0cdfe00640 1 -- 172.21.15.46:0/2954667161 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8035070 con 0x7f0ce8103ff0 2024-09-10T07:34:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.996+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce81a71b0 con 0x7f0ce8103ff0 2024-09-10T07:34:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.997+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce81054d0 con 0x7f0ce8103ff0 2024-09-10T07:34:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:12.999+0000 7f0cdfe00640 1 -- 172.21.15.46:0/2954667161 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0cd8042030 con 0x7f0ce8103ff0 2024-09-10T07:34:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.001+0000 7f0cdfe00640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ccc0a2b80 0x7f0ccc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.001+0000 7f0cdfe00640 1 -- 172.21.15.46:0/2954667161 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8802+0+0 (secure 0 0 0) 0x7f0cd80ef890 con 0x7f0ce8103ff0 2024-09-10T07:34:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.001+0000 7f0ce6a00640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ccc0a2b80 0x7f0ccc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.002+0000 7f0ce6a00640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ccc0a2b80 0x7f0ccc0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ffeb0 tx=0x7f0cd4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.002+0000 7f0cdfe00640 1 -- 172.21.15.46:0/2954667161 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cd80bbf00 con 0x7f0ce8103ff0 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-10T07:34:13.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-10T07:34:13.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.207+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ce81043d0 con 0x7f0ce8103ff0 2024-09-10T07:34:13.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.207+0000 7f0cdfe00640 1 -- 172.21.15.46:0/2954667161 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+590 (secure 0 0 0) 0x7f0cd80c0200 con 0x7f0ce8103ff0 2024-09-10T07:34:13.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2972952762 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:13.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2972952762 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:13.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1860671836 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:13.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3724700252 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:13.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3653813867 2024-09-11T07:26:12.367178+0000 2024-09-10T07:34:13.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/141501936 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:13.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3003000819 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:13.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/141501936 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:13.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1462945451 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:13.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3598352774 2024-09-11T07:25:44.780814+0000 2024-09-10T07:34:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-09-10T07:34:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ccc0a2b80 msgr2=0x7f0ccc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ccc0a2b80 0x7f0ccc0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ffeb0 tx=0x7f0cd4005f90 comp rx=0 tx=0).stop 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ce8103ff0 msgr2=0x7f0ce819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ce8103ff0 0x7f0ce819a630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8006010 tx=0x7f0cd8004040 comp rx=0 tx=0).stop 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 shutdown_connections 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ccc0a2b80 0x7f0ccc0a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce8109120 0x7f0ce819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ce81049a0 0x7f0ce819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 --2- 172.21.15.46:0/2954667161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ce8103ff0 0x7f0ce819a630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 >> 172.21.15.46:0/2954667161 conn(0x7f0ce80ffa90 msgr2=0x7f0ce8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 shutdown_connections 2024-09-10T07:34:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:13.210+0000 7f0cee000640 1 -- 172.21.15.46:0/2954667161 wait complete. 2024-09-10T07:34:13.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-10T07:34:13.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-10T07:34:13.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-09-10T07:34:13.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-10T07:34:13.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-10T07:34:13.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T07:34:13.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-10T07:34:13.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-10T07:34:13.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-10T07:34:13.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-10T07:34:13.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-10T07:34:14.044 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-09-10T07:34:14.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:14.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T07:34:14.102 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[71765]: fuse finished with error 0 and tester_r 0 2024-09-10T07:34:14.106 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:34:14.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:34:14.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:34:14.160 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T07:34:14.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:34:14.259 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T07:34:14.259 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-10T07:34:14.259 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-10T07:34:14.259 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-10T07:34:14.259 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T07:34:14.259 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T07:34:14.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T07:34:14.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-10T07:34:14.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-10T07:34:14.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-10T07:34:14.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-10T07:34:14.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T07:34:14.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:14 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2954667161' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:34:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T07:34:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-10T07:34:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T07:34:14.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:14 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2954667161' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-10T07:34:14.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-10T07:34:14.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-10T07:34:14.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T07:34:14.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:14 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2954667161' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:34:14.611 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 -- 172.21.15.46:0/2968272315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20541049a0 msgr2=0x7f20541089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 --2- 172.21.15.46:0/2968272315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20541049a0 0x7f20541089f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2040009900 tx=0x7f204002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 -- 172.21.15.46:0/2968272315 shutdown_connections 2024-09-10T07:34:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 --2- 172.21.15.46:0/2968272315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2054109120 0x7f2054110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 --2- 172.21.15.46:0/2968272315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20541049a0 0x7f20541089f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 --2- 172.21.15.46:0/2968272315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2054103ff0 0x7f20541043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 -- 172.21.15.46:0/2968272315 >> 172.21.15.46:0/2968272315 conn(0x7f20540ffa90 msgr2=0x7f2054101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 -- 172.21.15.46:0/2968272315 shutdown_connections 2024-09-10T07:34:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.259+0000 7f205c000640 1 -- 172.21.15.46:0/2968272315 wait complete. 2024-09-10T07:34:15.261 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:15 smithi046 ceph-mon[30705]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T07:34:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.260+0000 7f205c000640 1 Processor -- start 2024-09-10T07:34:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f205c000640 1 -- start start 2024-09-10T07:34:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f205c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2054103ff0 0x7f205419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f205c000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20541049a0 0x7f205419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f205c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2054109120 0x7f205419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f205c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2054114140 con 0x7f2054103ff0 2024-09-10T07:34:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f205c000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2054113fc0 con 0x7f20541049a0 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f205c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20541142c0 con 0x7f2054109120 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f2059600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2054103ff0 0x7f205419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f2058c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20541049a0 0x7f205419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f2059600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2054103ff0 0x7f205419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58144/0 (socket says 172.21.15.46:58144) 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f2058c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20541049a0 0x7f205419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50170/0 (socket says 172.21.15.46:50170) 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.261+0000 7f2059600640 1 -- 172.21.15.46:0/3064376529 learned_addr learned my addr 172.21.15.46:0/3064376529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f205a000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2054109120 0x7f205419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f2058c00640 1 -- 172.21.15.46:0/3064376529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2054109120 msgr2=0x7f205419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f2058c00640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2054109120 0x7f205419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f2058c00640 1 -- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2054103ff0 msgr2=0x7f205419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f2058c00640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2054103ff0 0x7f205419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f2058c00640 1 -- 172.21.15.46:0/3064376529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205419f650 con 0x7f20541049a0 2024-09-10T07:34:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f2059600640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2054103ff0 0x7f205419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f205a000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2054109120 0x7f205419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.262+0000 7f2058c00640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20541049a0 0x7f205419abe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2040005940 tx=0x7f20400040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.263+0000 7f2052000640 1 -- 172.21.15.46:0/3064376529 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204003f030 con 0x7f20541049a0 2024-09-10T07:34:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.263+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205419f8e0 con 0x7f20541049a0 2024-09-10T07:34:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.263+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20541a7220 con 0x7f20541049a0 2024-09-10T07:34:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.263+0000 7f2052000640 1 -- 172.21.15.46:0/3064376529 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20400044c0 con 0x7f20541049a0 2024-09-10T07:34:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.263+0000 7f2052000640 1 -- 172.21.15.46:0/3064376529 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040049060 con 0x7f20541049a0 2024-09-10T07:34:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.264+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2034005290 con 0x7f20541049a0 2024-09-10T07:34:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.266+0000 7f2052000640 1 -- 172.21.15.46:0/3064376529 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2040042030 con 0x7f20541049a0 2024-09-10T07:34:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.268+0000 7f2052000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f203c0a2b80 0x7f203c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.268+0000 7f2052000640 1 -- 172.21.15.46:0/3064376529 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f20400ef650 con 0x7f20541049a0 2024-09-10T07:34:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.268+0000 7f2059600640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f203c0a2b80 0x7f203c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.269+0000 7f2059600640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f203c0a2b80 0x7f203c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2048011550 tx=0x7f2048005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.271+0000 7f2052000640 1 -- 172.21.15.46:0/3064376529 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20400bbba0 con 0x7f20541049a0 2024-09-10T07:34:15.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:15 smithi181 ceph-mon[35397]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T07:34:15.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:15 smithi063 ceph-mon[35020]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T07:34:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.458+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2034005580 con 0x7f20541049a0 2024-09-10T07:34:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.460+0000 7f2052000640 1 -- 172.21.15.46:0/3064376529 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24597 (secure 0 0 0) 0x7f20400bfea0 con 0x7f20541049a0 2024-09-10T07:34:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:15.461 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:34:14.007613+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:34:15.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.462+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f203c0a2b80 msgr2=0x7f203c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:15.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.462+0000 7f205c000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f203c0a2b80 0x7f203c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2048011550 tx=0x7f2048005f90 comp rx=0 tx=0).stop 2024-09-10T07:34:15.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.462+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20541049a0 msgr2=0x7f205419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:15.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.462+0000 7f205c000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20541049a0 0x7f205419abe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2040005940 tx=0x7f20400040c0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.463+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 shutdown_connections 2024-09-10T07:34:15.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.463+0000 7f205c000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2054109120 0x7f205419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.463+0000 7f205c000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20541049a0 0x7f205419abe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.463+0000 7f205c000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f203c0a2b80 0x7f203c0a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.463+0000 7f205c000640 1 --2- 172.21.15.46:0/3064376529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2054103ff0 0x7f205419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.463+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 >> 172.21.15.46:0/3064376529 conn(0x7f20540ffa90 msgr2=0x7f2054100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.463+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 shutdown_connections 2024-09-10T07:34:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:15.463+0000 7f205c000640 1 -- 172.21.15.46:0/3064376529 wait complete. 2024-09-10T07:34:16.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:16 smithi046 ceph-mon[30705]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:16.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:16 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3064376529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:16.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:34:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:16 smithi181 ceph-mon[35397]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:16 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3064376529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:16.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:16 smithi063 ceph-mon[35020]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:16.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:16 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3064376529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 -- 172.21.15.46:0/649234416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb381079c0 msgr2=0x7feb38107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 --2- 172.21.15.46:0/649234416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb381079c0 0x7feb38107da0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feb28009930 tx=0x7feb2802a420 comp rx=0 tx=0).stop 2024-09-10T07:34:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 -- 172.21.15.46:0/649234416 shutdown_connections 2024-09-10T07:34:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 --2- 172.21.15.46:0/649234416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb38102360 0x7feb3810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 --2- 172.21.15.46:0/649234416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb381019c0 0x7feb38101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 --2- 172.21.15.46:0/649234416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb381079c0 0x7feb38107da0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 -- 172.21.15.46:0/649234416 >> 172.21.15.46:0/649234416 conn(0x7feb380fd740 msgr2=0x7feb380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 -- 172.21.15.46:0/649234416 shutdown_connections 2024-09-10T07:34:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.299+0000 7feb3ca00640 1 -- 172.21.15.46:0/649234416 wait complete. 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.300+0000 7feb3ca00640 1 Processor -- start 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.300+0000 7feb3ca00640 1 -- start start 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb3ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb381019c0 0x7feb380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb3ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb38102360 0x7feb38074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb3ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb381079c0 0x7feb38074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb3ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb38111e10 con 0x7feb381079c0 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb3ca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb38111c90 con 0x7feb38102360 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb3ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb38111f90 con 0x7feb381019c0 2024-09-10T07:34:17.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb37400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb381019c0 0x7feb380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb37400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb381019c0 0x7feb380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37494/0 (socket says 172.21.15.46:37494) 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb37400640 1 -- 172.21.15.46:0/995379873 learned_addr learned my addr 172.21.15.46:0/995379873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb37e00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb381079c0 0x7feb38074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb36a00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb38102360 0x7feb38074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb37400640 1 -- 172.21.15.46:0/995379873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb38102360 msgr2=0x7feb38074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb37400640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb38102360 0x7feb38074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb37400640 1 -- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb381079c0 msgr2=0x7feb38074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.301+0000 7feb37400640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb381079c0 0x7feb38074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.302+0000 7feb37400640 1 -- 172.21.15.46:0/995379873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb381aacc0 con 0x7feb381019c0 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.302+0000 7feb36a00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb38102360 0x7feb38074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:34:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.302+0000 7feb37400640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb381019c0 0x7feb380744f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb2802a940 tx=0x7feb28003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.302+0000 7feb2fe00640 1 -- 172.21.15.46:0/995379873 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2803f070 con 0x7feb381019c0 2024-09-10T07:34:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.302+0000 7feb2fe00640 1 -- 172.21.15.46:0/995379873 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb28031c90 con 0x7feb381019c0 2024-09-10T07:34:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.302+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb381aae90 con 0x7feb381019c0 2024-09-10T07:34:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.302+0000 7feb2fe00640 1 -- 172.21.15.46:0/995379873 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28035030 con 0x7feb381019c0 2024-09-10T07:34:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.302+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb381ab2f0 con 0x7feb381019c0 2024-09-10T07:34:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.303+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb38103100 con 0x7feb381019c0 2024-09-10T07:34:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.305+0000 7feb2fe00640 1 -- 172.21.15.46:0/995379873 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feb28042070 con 0x7feb381019c0 2024-09-10T07:34:17.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.307+0000 7feb2fe00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb1c0a2b80 0x7feb1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:17.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.307+0000 7feb2fe00640 1 -- 172.21.15.46:0/995379873 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7feb280efc50 con 0x7feb381019c0 2024-09-10T07:34:17.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.307+0000 7feb36a00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb1c0a2b80 0x7feb1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:17.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.308+0000 7feb36a00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb1c0a2b80 0x7feb1c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb24005fd0 tx=0x7feb24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.309+0000 7feb2fe00640 1 -- 172.21.15.46:0/995379873 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb280bc7f0 con 0x7feb381019c0 2024-09-10T07:34:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.496+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb38101e20 con 0x7feb381019c0 2024-09-10T07:34:17.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.499+0000 7feb2fe00640 1 -- 172.21.15.46:0/995379873 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24597 (secure 0 0 0) 0x7feb280bc0f0 con 0x7feb381019c0 2024-09-10T07:34:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:17.499 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:34:14.007613+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb1c0a2b80 msgr2=0x7feb1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb1c0a2b80 0x7feb1c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb24005fd0 tx=0x7feb24005850 comp rx=0 tx=0).stop 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb381019c0 msgr2=0x7feb380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb381019c0 0x7feb380744f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb2802a940 tx=0x7feb28003b90 comp rx=0 tx=0).stop 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 shutdown_connections 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb1c0a2b80 0x7feb1c0a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb381079c0 0x7feb38074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb38102360 0x7feb38074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 --2- 172.21.15.46:0/995379873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb381019c0 0x7feb380744f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 >> 172.21.15.46:0/995379873 conn(0x7feb380fd740 msgr2=0x7feb380fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 shutdown_connections 2024-09-10T07:34:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:17.502+0000 7feb3ca00640 1 -- 172.21.15.46:0/995379873 wait complete. 2024-09-10T07:34:18.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:18 smithi046 ceph-mon[30705]: pgmap v327: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:18.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:18 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/995379873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:18.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T07:34:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:18 smithi181 ceph-mon[35397]: pgmap v327: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:18 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/995379873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:18 smithi063 ceph-mon[35020]: pgmap v327: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:18 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/995379873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:18.678 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 -- 172.21.15.46:0/648835546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2848109090 msgr2=0x7f2848110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 --2- 172.21.15.46:0/648835546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2848109090 0x7f2848110c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2840009be0 tx=0x7f284002a6c0 comp rx=0 tx=0).stop 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 -- 172.21.15.46:0/648835546 shutdown_connections 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 --2- 172.21.15.46:0/648835546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2848109090 0x7f2848110c10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 --2- 172.21.15.46:0/648835546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2848104800 0x7f2848108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 --2- 172.21.15.46:0/648835546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2848103e50 0x7f2848104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 -- 172.21.15.46:0/648835546 >> 172.21.15.46:0/648835546 conn(0x7f28480ff8b0 msgr2=0x7f2848101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 -- 172.21.15.46:0/648835546 shutdown_connections 2024-09-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.281+0000 7f2847e00640 1 -- 172.21.15.46:0/648835546 wait complete. 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.282+0000 7f2847e00640 1 Processor -- start 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.282+0000 7f2847e00640 1 -- start start 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.282+0000 7f2847e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2848103e50 0x7f284819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.282+0000 7f2847e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2848104800 0x7f284819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.282+0000 7f2847e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2848109090 0x7f284819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.282+0000 7f2847e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2848114090 con 0x7f2848103e50 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.282+0000 7f2847e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2848113f10 con 0x7f2848109090 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2847e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2848114210 con 0x7f2848104800 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2846a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2848103e50 0x7f284819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2846a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2848103e50 0x7f284819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59152/0 (socket says 172.21.15.46:59152) 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2846a00640 1 -- 172.21.15.46:0/2070284008 learned_addr learned my addr 172.21.15.46:0/2070284008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f283fe00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2848104800 0x7f284819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2847400640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2848109090 0x7f284819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2846a00640 1 -- 172.21.15.46:0/2070284008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2848104800 msgr2=0x7f284819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2846a00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2848104800 0x7f284819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2846a00640 1 -- 172.21.15.46:0/2070284008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2848109090 msgr2=0x7f284819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2846a00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2848109090 0x7f284819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.283+0000 7f2846a00640 1 -- 172.21.15.46:0/2070284008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284819f4e0 con 0x7f2848103e50 2024-09-10T07:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.284+0000 7f2846a00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2848103e50 0x7f284819a530 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f283800e9c0 tx=0x7f283800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.284+0000 7f283f400640 1 -- 172.21.15.46:0/2070284008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283800cde0 con 0x7f2848103e50 2024-09-10T07:34:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.284+0000 7f283f400640 1 -- 172.21.15.46:0/2070284008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28380035f0 con 0x7f2848103e50 2024-09-10T07:34:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.284+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284819f7d0 con 0x7f2848103e50 2024-09-10T07:34:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.284+0000 7f283f400640 1 -- 172.21.15.46:0/2070284008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838010690 con 0x7f2848103e50 2024-09-10T07:34:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.284+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28481a70b0 con 0x7f2848103e50 2024-09-10T07:34:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.285+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2848105330 con 0x7f2848103e50 2024-09-10T07:34:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.287+0000 7f283f400640 1 -- 172.21.15.46:0/2070284008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f28380037a0 con 0x7f2848103e50 2024-09-10T07:34:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.288+0000 7f283f400640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f282c0a2b80 0x7f282c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.288+0000 7f283f400640 1 -- 172.21.15.46:0/2070284008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f28380cf730 con 0x7f2848103e50 2024-09-10T07:34:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.289+0000 7f283fe00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f282c0a2b80 0x7f282c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.289+0000 7f283fe00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f282c0a2b80 0x7f282c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28480ffcd0 tx=0x7f283000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.293+0000 7f283f400640 1 -- 172.21.15.46:0/2070284008 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f283809c2d0 con 0x7f2848103e50 2024-09-10T07:34:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:19.515+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f284810c2d0 con 0x7f2848103e50 2024-09-10T07:34:20.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.037+0000 7f283f400640 1 -- 172.21.15.46:0/2070284008 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f283809bbd0 con 0x7f2848103e50 2024-09-10T07:34:20.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f282c0a2b80 msgr2=0x7f282c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:20.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f282c0a2b80 0x7f282c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28480ffcd0 tx=0x7f283000b390 comp rx=0 tx=0).stop 2024-09-10T07:34:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2848103e50 msgr2=0x7f284819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2848103e50 0x7f284819a530 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f283800e9c0 tx=0x7f283800ee80 comp rx=0 tx=0).stop 2024-09-10T07:34:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 shutdown_connections 2024-09-10T07:34:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2848109090 0x7f284819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2848104800 0x7f284819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f282c0a2b80 0x7f282c0a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 --2- 172.21.15.46:0/2070284008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2848103e50 0x7f284819a530 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.040+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 >> 172.21.15.46:0/2070284008 conn(0x7f28480ff8b0 msgr2=0x7f2848100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.041+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 shutdown_connections 2024-09-10T07:34:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:20.041+0000 7f2847e00640 1 -- 172.21.15.46:0/2070284008 wait complete. 2024-09-10T07:34:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:20 smithi181 ceph-mon[35397]: pgmap v328: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:20 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2070284008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T07:34:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:20 smithi063 ceph-mon[35020]: pgmap v328: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:20 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2070284008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T07:34:20.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:20 smithi046 ceph-mon[30705]: pgmap v328: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:20.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:20 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2070284008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T07:34:20.870 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T07:34:20.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-10T07:34:21.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:21 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2070284008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T07:34:21.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:21 smithi046 ceph-mon[30705]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:34:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:21.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:21 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2070284008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T07:34:21.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:21 smithi181 ceph-mon[35397]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:34:21.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:21 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2070284008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T07:34:21.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:21 smithi063 ceph-mon[35020]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:34:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 -- 172.21.15.46:0/1211974353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c077040 msgr2=0x7ff42c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 --2- 172.21.15.46:0/1211974353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c077040 0x7ff42c112330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff418009900 tx=0x7ff41802a410 comp rx=0 tx=0).stop 2024-09-10T07:34:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 -- 172.21.15.46:0/1211974353 shutdown_connections 2024-09-10T07:34:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 --2- 172.21.15.46:0/1211974353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff42c112870 0x7ff42c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 --2- 172.21.15.46:0/1211974353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c077040 0x7ff42c112330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 --2- 172.21.15.46:0/1211974353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c076720 0x7ff42c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 -- 172.21.15.46:0/1211974353 >> 172.21.15.46:0/1211974353 conn(0x7ff42c0ff8d0 msgr2=0x7ff42c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 -- 172.21.15.46:0/1211974353 shutdown_connections 2024-09-10T07:34:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.878+0000 7ff432200640 1 -- 172.21.15.46:0/1211974353 wait complete. 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.879+0000 7ff432200640 1 Processor -- start 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.879+0000 7ff432200640 1 -- start start 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff432200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c076720 0x7ff42c1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff432200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff42c077040 0x7ff42c1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff432200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c112870 0x7ff42c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff432200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c118440 con 0x7ff42c076720 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff432200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1182c0 con 0x7ff42c112870 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff432200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1185c0 con 0x7ff42c077040 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff430e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c076720 0x7ff42c1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff430e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c076720 0x7ff42c1a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59168/0 (socket says 172.21.15.46:59168) 2024-09-10T07:34:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff430e00640 1 -- 172.21.15.46:0/2714224928 learned_addr learned my addr 172.21.15.46:0/2714224928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff42be00640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff42c077040 0x7ff42c1a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.880+0000 7ff431800640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c112870 0x7ff42c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff431800640 1 -- 172.21.15.46:0/2714224928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff42c077040 msgr2=0x7ff42c1a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff431800640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff42c077040 0x7ff42c1a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff431800640 1 -- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c076720 msgr2=0x7ff42c1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff431800640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c076720 0x7ff42c1a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff431800640 1 -- 172.21.15.46:0/2714224928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c19ef60 con 0x7ff42c112870 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff431800640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c112870 0x7ff42c19e6f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff42c07aa10 tx=0x7ff42400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff429600640 1 -- 172.21.15.46:0/2714224928 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42400cdf0 con 0x7ff42c112870 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c19f250 con 0x7ff42c112870 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff429600640 1 -- 172.21.15.46:0/2714224928 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff424005020 con 0x7ff42c112870 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.881+0000 7ff429600640 1 -- 172.21.15.46:0/2714224928 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424010690 con 0x7ff42c112870 2024-09-10T07:34:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.882+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c1ab5a0 con 0x7ff42c112870 2024-09-10T07:34:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.883+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c077e60 con 0x7ff42c112870 2024-09-10T07:34:21.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.885+0000 7ff429600640 1 -- 172.21.15.46:0/2714224928 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4240054d0 con 0x7ff42c112870 2024-09-10T07:34:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.886+0000 7ff429600640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff4140a2c00 0x7ff4140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.886+0000 7ff429600640 1 -- 172.21.15.46:0/2714224928 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7ff4240cfaa0 con 0x7ff42c112870 2024-09-10T07:34:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.886+0000 7ff430e00640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff4140a2c00 0x7ff4140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.887+0000 7ff430e00640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff4140a2c00 0x7ff4140a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4200095f0 tx=0x7ff420005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:21.888+0000 7ff429600640 1 -- 172.21.15.46:0/2714224928 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff42409c640 con 0x7ff42c112870 2024-09-10T07:34:22.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:22.113+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ff42c10da00 con 0x7ff42c112870 2024-09-10T07:34:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:22 smithi181 ceph-mon[35397]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:22 smithi063 ceph-mon[35020]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:22.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:22 smithi046 ceph-mon[30705]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:23.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.050+0000 7ff429600640 1 -- 172.21.15.46:0/2714224928 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7ff42409bf40 con 0x7ff42c112870 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff4140a2c00 msgr2=0x7ff4140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff4140a2c00 0x7ff4140a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4200095f0 tx=0x7ff420005f90 comp rx=0 tx=0).stop 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c112870 msgr2=0x7ff42c19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c112870 0x7ff42c19e6f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff42c07aa10 tx=0x7ff42400ef10 comp rx=0 tx=0).stop 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 shutdown_connections 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c112870 0x7ff42c19e6f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff42c077040 0x7ff42c1a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff4140a2c00 0x7ff4140a50c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 --2- 172.21.15.46:0/2714224928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff42c076720 0x7ff42c1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 >> 172.21.15.46:0/2714224928 conn(0x7ff42c0ff8d0 msgr2=0x7ff42c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 shutdown_connections 2024-09-10T07:34:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:23.054+0000 7ff432200640 1 -- 172.21.15.46:0/2714224928 wait complete. 2024-09-10T07:34:23.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:23 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2714224928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T07:34:23.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:23 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T07:34:23.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:23 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2714224928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T07:34:23.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:23 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T07:34:23.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:23 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2714224928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T07:34:23.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:23 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T07:34:23.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T07:34:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:24 smithi046 ceph-mon[30705]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 324 B/s wr, 0 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:24 smithi046 ceph-mon[30705]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T07:34:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:24 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T07:34:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:24 smithi046 ceph-mon[30705]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T07:34:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:24 smithi046 ceph-mon[30705]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:34:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:24 smithi046 ceph-mon[30705]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-09-10T07:34:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:24 smithi046 ceph-mon[30705]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T07:34:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:24 smithi181 ceph-mon[35397]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 324 B/s wr, 0 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:24 smithi181 ceph-mon[35397]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T07:34:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:24 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:24 smithi181 ceph-mon[35397]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:24 smithi181 ceph-mon[35397]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:24 smithi181 ceph-mon[35397]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:24 smithi181 ceph-mon[35397]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:24 smithi063 ceph-mon[35020]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 324 B/s wr, 0 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:24 smithi063 ceph-mon[35020]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:24 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:24 smithi063 ceph-mon[35020]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T07:34:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:24 smithi063 ceph-mon[35020]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-09-10T07:34:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:24 smithi063 ceph-mon[35020]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-09-10T07:34:24.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:24 smithi063 ceph-mon[35020]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T07:34:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.861+0000 7f61b5e00640 1 -- 172.21.15.46:0/250210675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074f40 msgr2=0x7f61b010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.861+0000 7f61b5e00640 1 --2- 172.21.15.46:0/250210675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074f40 0x7f61b010de70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6198009900 tx=0x7f619802a480 comp rx=0 tx=0).stop 2024-09-10T07:34:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.861+0000 7f61b5e00640 1 -- 172.21.15.46:0/250210675 shutdown_connections 2024-09-10T07:34:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.861+0000 7f61b5e00640 1 --2- 172.21.15.46:0/250210675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b010e3b0 0x7f61b0110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.861+0000 7f61b5e00640 1 --2- 172.21.15.46:0/250210675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074f40 0x7f61b010de70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.861+0000 7f61b5e00640 1 --2- 172.21.15.46:0/250210675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0074620 0x7f61b0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.861+0000 7f61b5e00640 1 -- 172.21.15.46:0/250210675 >> 172.21.15.46:0/250210675 conn(0x7f61b00fd890 msgr2=0x7f61b00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.862+0000 7f61b5e00640 1 -- 172.21.15.46:0/250210675 shutdown_connections 2024-09-10T07:34:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.862+0000 7f61b5e00640 1 -- 172.21.15.46:0/250210675 wait complete. 2024-09-10T07:34:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.863+0000 7f61b5e00640 1 Processor -- start 2024-09-10T07:34:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.863+0000 7f61b5e00640 1 -- start start 2024-09-10T07:34:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074620 0x7f61b019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0074f40 0x7f61b019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b010e3b0 0x7f61b019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61b5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01141a0 con 0x7f61b0074620 2024-09-10T07:34:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61b5e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0114020 con 0x7f61b010e3b0 2024-09-10T07:34:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61b5e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0114320 con 0x7f61b0074f40 2024-09-10T07:34:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074620 0x7f61b019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074620 0x7f61b019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59186/0 (socket says 172.21.15.46:59186) 2024-09-10T07:34:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61aea00640 1 -- 172.21.15.46:0/2408482877 learned_addr learned my addr 172.21.15.46:0/2408482877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61a6000640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0074f40 0x7f61b019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.864+0000 7f61af400640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b010e3b0 0x7f61b019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.865+0000 7f61a6000640 1 -- 172.21.15.46:0/2408482877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b010e3b0 msgr2=0x7f61b019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.865+0000 7f61a6000640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b010e3b0 0x7f61b019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.865+0000 7f61a6000640 1 -- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074620 msgr2=0x7f61b019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.865+0000 7f61a6000640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074620 0x7f61b019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.865+0000 7f61a6000640 1 -- 172.21.15.46:0/2408482877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b019f690 con 0x7f61b0074f40 2024-09-10T07:34:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.865+0000 7f61a6000640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0074f40 0x7f61b019ac20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f619802a9a0 tx=0x7f6198003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.865+0000 7f61b4a00640 1 -- 172.21.15.46:0/2408482877 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619803f070 con 0x7f61b0074f40 2024-09-10T07:34:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.865+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b019f920 con 0x7f61b0074f40 2024-09-10T07:34:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.866+0000 7f61b4a00640 1 -- 172.21.15.46:0/2408482877 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6198031c90 con 0x7f61b0074f40 2024-09-10T07:34:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.866+0000 7f61b4a00640 1 -- 172.21.15.46:0/2408482877 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198035030 con 0x7f61b0074f40 2024-09-10T07:34:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.866+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b01a7260 con 0x7f61b0074f40 2024-09-10T07:34:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.867+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b0074a00 con 0x7f61b0074f40 2024-09-10T07:34:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.869+0000 7f61b4a00640 1 -- 172.21.15.46:0/2408482877 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6198042070 con 0x7f61b0074f40 2024-09-10T07:34:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.870+0000 7f61b4a00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f61940a2c00 0x7f61940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.871+0000 7f61b4a00640 1 -- 172.21.15.46:0/2408482877 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f61980f0270 con 0x7f61b0074f40 2024-09-10T07:34:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.871+0000 7f61aea00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f61940a2c00 0x7f61940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:24.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.871+0000 7f61aea00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f61940a2c00 0x7f61940a50c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009cd0 tx=0x7f61a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:24.872+0000 7f61b4a00640 1 -- 172.21.15.46:0/2408482877 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61980bc7c0 con 0x7f61b0074f40 2024-09-10T07:34:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.107+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f61b019ba40 con 0x7f61b0074f40 2024-09-10T07:34:25.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:25 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] up:active 2024-09-10T07:34:25.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:25 smithi046 ceph-mon[30705]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T07:34:25.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:25 smithi046 ceph-mon[30705]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-09-10T07:34:25.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:25 smithi046 ceph-mon[30705]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-09-10T07:34:25.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:25 smithi046 ceph-mon[30705]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T07:34:25.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:25 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] up:active 2024-09-10T07:34:25.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:25 smithi181 ceph-mon[35397]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T07:34:25.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:25 smithi181 ceph-mon[35397]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-09-10T07:34:25.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:25 smithi181 ceph-mon[35397]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-09-10T07:34:25.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:25 smithi181 ceph-mon[35397]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T07:34:25.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:25 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] up:active 2024-09-10T07:34:25.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:25 smithi063 ceph-mon[35020]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T07:34:25.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:25 smithi063 ceph-mon[35020]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-09-10T07:34:25.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:25 smithi063 ceph-mon[35020]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-09-10T07:34:25.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:25 smithi063 ceph-mon[35020]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T07:34:25.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.797+0000 7f61b4a00640 1 -- 172.21.15.46:0/2408482877 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f61980c0ac0 con 0x7f61b0074f40 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f61940a2c00 msgr2=0x7f61940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f61940a2c00 0x7f61940a50c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009cd0 tx=0x7f61a0005f90 comp rx=0 tx=0).stop 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0074f40 msgr2=0x7f61b019ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0074f40 0x7f61b019ac20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f619802a9a0 tx=0x7f6198003b20 comp rx=0 tx=0).stop 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 shutdown_connections 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b010e3b0 0x7f61b019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0074f40 0x7f61b019ac20 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f619802a9a0 tx=0x7f6198003b20 comp rx=0 tx=0).stop 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f61940a2c00 0x7f61940a50c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 --2- 172.21.15.46:0/2408482877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b0074620 0x7f61b019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.801+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 >> 172.21.15.46:0/2408482877 conn(0x7f61b00fd890 msgr2=0x7f61b00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.802+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 shutdown_connections 2024-09-10T07:34:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:25.802+0000 7f61b5e00640 1 -- 172.21.15.46:0/2408482877 wait complete. 2024-09-10T07:34:26.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 293 B/s wr, 1 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T07:34:26.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] up:active 2024-09-10T07:34:26.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T07:34:26.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-09-10T07:34:26.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-10T07:34:26.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T07:34:26.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2408482877' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T07:34:26.391 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T07:34:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T07:34:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] up:active 2024-09-10T07:34:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T07:34:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T07:34:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: Cluster is now healthy 2024-09-10T07:34:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-10T07:34:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-10T07:34:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-mon[30705]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-10T07:34:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 293 B/s wr, 1 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T07:34:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] up:active 2024-09-10T07:34:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T07:34:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-09-10T07:34:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2408482877' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] up:active 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: Cluster is now healthy 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-10T07:34:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:26 smithi181 ceph-mon[35397]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 293 B/s wr, 1 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] up:active 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2408482877' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] up:active 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T07:34:26.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T07:34:26.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: Cluster is now healthy 2024-09-10T07:34:26.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-10T07:34:26.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-10T07:34:26.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:26 smithi063 ceph-mon[35020]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-10T07:34:26.652 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: 2024-09-10T07:34:26.389+0000 7f775a600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T07:34:26.653 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: 2024-09-10T07:34:26.389+0000 7f775a600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T07:34:26.653 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:34:26 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: 2024-09-10T07:34:26.389+0000 7f775a600640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-10T07:34:26.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T07:34:27.045 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:27.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 -- 172.21.15.46:0/1402405969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb748109140 msgr2=0x7fb748110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 --2- 172.21.15.46:0/1402405969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb748109140 0x7fb748110cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb740009ca0 tx=0x7fb74002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:34:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 -- 172.21.15.46:0/1402405969 shutdown_connections 2024-09-10T07:34:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 --2- 172.21.15.46:0/1402405969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb748109140 0x7fb748110cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 --2- 172.21.15.46:0/1402405969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7481049c0 0x7fb748108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 --2- 172.21.15.46:0/1402405969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb748104010 0x7fb7481043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 -- 172.21.15.46:0/1402405969 >> 172.21.15.46:0/1402405969 conn(0x7fb7480ffa50 msgr2=0x7fb748101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 -- 172.21.15.46:0/1402405969 shutdown_connections 2024-09-10T07:34:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.637+0000 7fb74ec00640 1 -- 172.21.15.46:0/1402405969 wait complete. 2024-09-10T07:34:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb74ec00640 1 Processor -- start 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb74ec00640 1 -- start start 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb74ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb748104010 0x7fb74819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb74ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7481049c0 0x7fb74819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb74ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb748109140 0x7fb74819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb74ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7481141a0 con 0x7fb7481049c0 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb74ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb748114020 con 0x7fb748109140 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb74ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb748114320 con 0x7fb748104010 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.638+0000 7fb73fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7481049c0 0x7fb74819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb73fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7481049c0 0x7fb74819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43872/0 (socket says 172.21.15.46:43872) 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb73fe00640 1 -- 172.21.15.46:0/497386403 learned_addr learned my addr 172.21.15.46:0/497386403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb74cc00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb748109140 0x7fb74819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb747e00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb748104010 0x7fb74819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb73fe00640 1 -- 172.21.15.46:0/497386403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb748104010 msgr2=0x7fb74819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb73fe00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb748104010 0x7fb74819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb73fe00640 1 -- 172.21.15.46:0/497386403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb748109140 msgr2=0x7fb74819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb73fe00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb748109140 0x7fb74819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb73fe00640 1 -- 172.21.15.46:0/497386403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74819f650 con 0x7fb7481049c0 2024-09-10T07:34:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.639+0000 7fb73fe00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7481049c0 0x7fb74819abe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb73000e9f0 tx=0x7fb73000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.640+0000 7fb745600640 1 -- 172.21.15.46:0/497386403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73000cd80 con 0x7fb7481049c0 2024-09-10T07:34:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.640+0000 7fb745600640 1 -- 172.21.15.46:0/497386403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb730010910 con 0x7fb7481049c0 2024-09-10T07:34:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.640+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74819f940 con 0x7fb7481049c0 2024-09-10T07:34:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.640+0000 7fb745600640 1 -- 172.21.15.46:0/497386403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730018990 con 0x7fb7481049c0 2024-09-10T07:34:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.640+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7481a7220 con 0x7fb7481049c0 2024-09-10T07:34:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.641+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7481043f0 con 0x7fb7481049c0 2024-09-10T07:34:27.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.643+0000 7fb745600640 1 -- 172.21.15.46:0/497386403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb730010ac0 con 0x7fb7481049c0 2024-09-10T07:34:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.644+0000 7fb745600640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb72c0a2b80 0x7fb72c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.645+0000 7fb745600640 1 -- 172.21.15.46:0/497386403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fb7300cf740 con 0x7fb7481049c0 2024-09-10T07:34:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.645+0000 7fb747e00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb72c0a2b80 0x7fb72c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.645+0000 7fb747e00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb72c0a2b80 0x7fb72c0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb738009b60 tx=0x7fb738005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:27.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.649+0000 7fb745600640 1 -- 172.21.15.46:0/497386403 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb73009c2e0 con 0x7fb7481049c0 2024-09-10T07:34:27.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:27.873+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb74810bdf0 con 0x7fb7481049c0 2024-09-10T07:34:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:27 smithi181 ceph-mon[35397]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-10T07:34:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:27 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] up:active 2024-09-10T07:34:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:27 smithi181 ceph-mon[35397]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-10T07:34:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:27 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:34:28.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:27 smithi063 ceph-mon[35020]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-10T07:34:28.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:27 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] up:active 2024-09-10T07:34:28.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:27 smithi063 ceph-mon[35020]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-10T07:34:28.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:27 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:34:28.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:27 smithi046 ceph-mon[30705]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-10T07:34:28.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:27 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] up:active 2024-09-10T07:34:28.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:27 smithi046 ceph-mon[30705]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-10T07:34:28.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:27 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:34:28.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.811+0000 7fb745600640 1 -- 172.21.15.46:0/497386403 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7fb73009bbe0 con 0x7fb7481049c0 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb72c0a2b80 msgr2=0x7fb72c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb72c0a2b80 0x7fb72c0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb738009b60 tx=0x7fb738005f90 comp rx=0 tx=0).stop 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7481049c0 msgr2=0x7fb74819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7481049c0 0x7fb74819abe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb73000e9f0 tx=0x7fb73000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 shutdown_connections 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb748109140 0x7fb74819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb72c0a2b80 0x7fb72c0a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7481049c0 0x7fb74819abe0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 --2- 172.21.15.46:0/497386403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb748104010 0x7fb74819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 >> 172.21.15.46:0/497386403 conn(0x7fb7480ffa50 msgr2=0x7fb748100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 shutdown_connections 2024-09-10T07:34:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:28.814+0000 7fb74ec00640 1 -- 172.21.15.46:0/497386403 wait complete. 2024-09-10T07:34:29.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:28 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/497386403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T07:34:29.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:28 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/497386403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T07:34:29.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:28 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/497386403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T07:34:29.613 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-10T07:34:29.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T07:34:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:29 smithi046 ceph-mon[30705]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.8 KiB/s wr, 11 op/s 2024-09-10T07:34:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:29 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/497386403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T07:34:29.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:29 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:34:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:29 smithi181 ceph-mon[35397]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.8 KiB/s wr, 11 op/s 2024-09-10T07:34:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:29 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/497386403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T07:34:30.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:29 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:34:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:29 smithi063 ceph-mon[35020]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.8 KiB/s wr, 11 op/s 2024-09-10T07:34:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:29 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/497386403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T07:34:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:29 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:34:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 -- 172.21.15.46:0/1848646980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 msgr2=0x7fc600074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 --2- 172.21.15.46:0/1848646980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 0x7fc600074a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009900 tx=0x7fc5f002a480 comp rx=0 tx=0).stop 2024-09-10T07:34:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 -- 172.21.15.46:0/1848646980 shutdown_connections 2024-09-10T07:34:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 --2- 172.21.15.46:0/1848646980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60010e3c0 0x7fc600110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 --2- 172.21.15.46:0/1848646980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc600074f40 0x7fc60010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 --2- 172.21.15.46:0/1848646980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 0x7fc600074a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 -- 172.21.15.46:0/1848646980 >> 172.21.15.46:0/1848646980 conn(0x7fc6000fd8c0 msgr2=0x7fc6000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:30.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 -- 172.21.15.46:0/1848646980 shutdown_connections 2024-09-10T07:34:30.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.677+0000 7fc606600640 1 -- 172.21.15.46:0/1848646980 wait complete. 2024-09-10T07:34:30.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc606600640 1 Processor -- start 2024-09-10T07:34:30.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc606600640 1 -- start start 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc606600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 0x7fc60019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc606600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc600074f40 0x7fc60019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc606600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60010e3c0 0x7fc60019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc606600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc600114150 con 0x7fc600074620 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc606600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc600113fd0 con 0x7fc60010e3c0 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc606600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001142d0 con 0x7fc600074f40 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.678+0000 7fc5ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 0x7fc60019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 0x7fc60019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43888/0 (socket says 172.21.15.46:43888) 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ff400640 1 -- 172.21.15.46:0/1317900660 learned_addr learned my addr 172.21.15.46:0/1317900660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5f7e00640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc600074f40 0x7fc60019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ffe00640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60010e3c0 0x7fc60019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ff400640 1 -- 172.21.15.46:0/1317900660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc600074f40 msgr2=0x7fc60019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ff400640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc600074f40 0x7fc60019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ff400640 1 -- 172.21.15.46:0/1317900660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60010e3c0 msgr2=0x7fc60019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ff400640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60010e3c0 0x7fc60019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ff400640 1 -- 172.21.15.46:0/1317900660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60019f6c0 con 0x7fc600074620 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.679+0000 7fc5ffe00640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60010e3c0 0x7fc60019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.680+0000 7fc5ff400640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 0x7fc60019a710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0006010 tx=0x7fc5f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.680+0000 7fc5fcc00640 1 -- 172.21.15.46:0/1317900660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f003f030 con 0x7fc600074620 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.680+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60019f950 con 0x7fc600074620 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.680+0000 7fc5fcc00640 1 -- 172.21.15.46:0/1317900660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f0003ce0 con 0x7fc600074620 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.680+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6001a7290 con 0x7fc600074620 2024-09-10T07:34:30.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.680+0000 7fc5fcc00640 1 -- 172.21.15.46:0/1317900660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0049060 con 0x7fc600074620 2024-09-10T07:34:30.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.681+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc600074a00 con 0x7fc600074620 2024-09-10T07:34:30.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.683+0000 7fc5fcc00640 1 -- 172.21.15.46:0/1317900660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc5f0042030 con 0x7fc600074620 2024-09-10T07:34:30.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.684+0000 7fc5fcc00640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5e40a2b80 0x7fc5e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:30.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.684+0000 7fc5fcc00640 1 -- 172.21.15.46:0/1317900660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fc5f00ef790 con 0x7fc600074620 2024-09-10T07:34:30.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.684+0000 7fc5f7e00640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5e40a2b80 0x7fc5e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:30.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.685+0000 7fc5f7e00640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5e40a2b80 0x7fc5e40a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc60010f8b0 tx=0x7fc5e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.688+0000 7fc5fcc00640 1 -- 172.21.15.46:0/1317900660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5f00bbce0 con 0x7fc600074620 2024-09-10T07:34:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.902+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc600062630 con 0x7fc5e40a2b80 2024-09-10T07:34:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.934+0000 7fc5fcc00640 1 -- 172.21.15.46:0/1317900660 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc600062630 con 0x7fc5e40a2b80 2024-09-10T07:34:30.935 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:34:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:34:30.908+0000 7f74dc200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T07:34:30.935 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:34:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:34:30.908+0000 7f74dc200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T07:34:30.935 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:34:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:34:30.908+0000 7f74dc200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T07:34:30.935 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:34:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:34:30.908+0000 7f74dc200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T07:34:30.935 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:34:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:34:30.908+0000 7f74dc200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T07:34:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5e40a2b80 msgr2=0x7fc5e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5e40a2b80 0x7fc5e40a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc60010f8b0 tx=0x7fc5e800b390 comp rx=0 tx=0).stop 2024-09-10T07:34:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 msgr2=0x7fc60019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 0x7fc60019a710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0006010 tx=0x7fc5f00042f0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 shutdown_connections 2024-09-10T07:34:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60010e3c0 0x7fc60019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc600074f40 0x7fc60019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5e40a2b80 0x7fc5e40a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 --2- 172.21.15.46:0/1317900660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc600074620 0x7fc60019a710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 >> 172.21.15.46:0/1317900660 conn(0x7fc6000fd8c0 msgr2=0x7fc6000fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.936+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 shutdown_connections 2024-09-10T07:34:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:30.937+0000 7fc606600640 1 -- 172.21.15.46:0/1317900660 wait complete. 2024-09-10T07:34:31.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T07:34:31.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:31 smithi181 ceph-mon[35397]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-10T07:34:31.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:31 smithi181 ceph-mon[35397]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:31.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:31 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:34:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:31 smithi063 ceph-mon[35020]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-10T07:34:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:31 smithi063 ceph-mon[35020]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:31 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:34:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:31 smithi046 ceph-mon[30705]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-10T07:34:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:31 smithi046 ceph-mon[30705]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:31 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:34:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.396+0000 7fd8bc600640 1 -- 172.21.15.46:0/3113557543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4102790 msgr2=0x7fd8b41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.396+0000 7fd8bc600640 1 --2- 172.21.15.46:0/3113557543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4102790 0x7fd8b41067e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009900 tx=0x7fd8a002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.397+0000 7fd8bc600640 1 -- 172.21.15.46:0/3113557543 shutdown_connections 2024-09-10T07:34:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.397+0000 7fd8bc600640 1 --2- 172.21.15.46:0/3113557543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8b4106f10 0x7fd8b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.397+0000 7fd8bc600640 1 --2- 172.21.15.46:0/3113557543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4102790 0x7fd8b41067e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.397+0000 7fd8bc600640 1 --2- 172.21.15.46:0/3113557543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b4101de0 0x7fd8b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.397+0000 7fd8bc600640 1 -- 172.21.15.46:0/3113557543 >> 172.21.15.46:0/3113557543 conn(0x7fd8b40fd8e0 msgr2=0x7fd8b40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.397+0000 7fd8bc600640 1 -- 172.21.15.46:0/3113557543 shutdown_connections 2024-09-10T07:34:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.397+0000 7fd8bc600640 1 -- 172.21.15.46:0/3113557543 wait complete. 2024-09-10T07:34:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.398+0000 7fd8bc600640 1 Processor -- start 2024-09-10T07:34:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.398+0000 7fd8bc600640 1 -- start start 2024-09-10T07:34:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.398+0000 7fd8bc600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4101de0 0x7fd8b419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.398+0000 7fd8bc600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8b4102790 0x7fd8b419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.398+0000 7fd8bc600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b4106f10 0x7fd8b419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.398+0000 7fd8bc600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4111fc0 con 0x7fd8b4101de0 2024-09-10T07:34:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.398+0000 7fd8bc600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4111e40 con 0x7fd8b4106f10 2024-09-10T07:34:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.398+0000 7fd8bc600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4112140 con 0x7fd8b4102790 2024-09-10T07:34:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8b9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4101de0 0x7fd8b419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8ba600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b4106f10 0x7fd8b419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8ba600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b4106f10 0x7fd8b419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44670/0 (socket says 172.21.15.46:44670) 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8b9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4101de0 0x7fd8b419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43934/0 (socket says 172.21.15.46:43934) 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8ba600640 1 -- 172.21.15.46:0/776542692 learned_addr learned my addr 172.21.15.46:0/776542692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8b9200640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8b4102790 0x7fd8b419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8b9c00640 1 -- 172.21.15.46:0/776542692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8b4102790 msgr2=0x7fd8b419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8b9c00640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8b4102790 0x7fd8b419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8b9c00640 1 -- 172.21.15.46:0/776542692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b4106f10 msgr2=0x7fd8b419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8b9c00640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b4106f10 0x7fd8b419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.399+0000 7fd8b9c00640 1 -- 172.21.15.46:0/776542692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b419f630 con 0x7fd8b4101de0 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.400+0000 7fd8b9c00640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4101de0 0x7fd8b419a680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800e9f0 tx=0x7fd8a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.400+0000 7fd8b2000640 1 -- 172.21.15.46:0/776542692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800ccc0 con 0x7fd8b4101de0 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.400+0000 7fd8b2000640 1 -- 172.21.15.46:0/776542692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a8010910 con 0x7fd8b4101de0 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.400+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b419f920 con 0x7fd8b4101de0 2024-09-10T07:34:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.400+0000 7fd8b2000640 1 -- 172.21.15.46:0/776542692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8018960 con 0x7fd8b4101de0 2024-09-10T07:34:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.400+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b41a7200 con 0x7fd8b4101de0 2024-09-10T07:34:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.401+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b41021c0 con 0x7fd8b4101de0 2024-09-10T07:34:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.404+0000 7fd8b2000640 1 -- 172.21.15.46:0/776542692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd8a8010ac0 con 0x7fd8b4101de0 2024-09-10T07:34:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.406+0000 7fd8b2000640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd89c0a2b80 0x7fd89c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.406+0000 7fd8b2000640 1 -- 172.21.15.46:0/776542692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fd8a80cfc50 con 0x7fd8b4101de0 2024-09-10T07:34:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.407+0000 7fd8b9200640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd89c0a2b80 0x7fd89c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.408+0000 7fd8b9200640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd89c0a2b80 0x7fd89c0a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0000c00 tx=0x7fd8a0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.408+0000 7fd8b2000640 1 -- 172.21.15.46:0/776542692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8a809c1a0 con 0x7fd8b4101de0 2024-09-10T07:34:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.490+0000 7fd8b2000640 1 -- 172.21.15.46:0/776542692 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd8a8098810 con 0x7fd8b4101de0 2024-09-10T07:34:33.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:33 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:33 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:33 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:33.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.668+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd8b4075430 con 0x7fd89c0a2b80 2024-09-10T07:34:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.693+0000 7fd8b2000640 1 -- 172.21.15.46:0/776542692 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8b4075430 con 0x7fd89c0a2b80 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd89c0a2b80 msgr2=0x7fd89c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd89c0a2b80 0x7fd89c0a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0000c00 tx=0x7fd8a0005b50 comp rx=0 tx=0).stop 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4101de0 msgr2=0x7fd8b419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4101de0 0x7fd8b419a680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800e9f0 tx=0x7fd8a800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 shutdown_connections 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b4106f10 0x7fd8b419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8b4102790 0x7fd8b419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd89c0a2b80 0x7fd89c0a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 --2- 172.21.15.46:0/776542692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b4101de0 0x7fd8b419a680 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 >> 172.21.15.46:0/776542692 conn(0x7fd8b40fd8e0 msgr2=0x7fd8b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 shutdown_connections 2024-09-10T07:34:33.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:33.696+0000 7fd8bc600640 1 -- 172.21.15.46:0/776542692 wait complete. 2024-09-10T07:34:34.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T07:34:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:34 smithi181 ceph-mon[35397]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 19 op/s 2024-09-10T07:34:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:34 smithi181 ceph-mon[35397]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T07:34:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:34 smithi063 ceph-mon[35020]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 19 op/s 2024-09-10T07:34:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:34 smithi063 ceph-mon[35020]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T07:34:34.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:34 smithi046 ceph-mon[30705]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 19 op/s 2024-09-10T07:34:34.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:34 smithi046 ceph-mon[30705]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T07:34:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 -- 172.21.15.46:0/2531543063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c1047e0 msgr2=0x7fe08c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 --2- 172.21.15.46:0/2531543063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c1047e0 0x7fe08c108830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe078009900 tx=0x7fe07802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 -- 172.21.15.46:0/2531543063 shutdown_connections 2024-09-10T07:34:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 --2- 172.21.15.46:0/2531543063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe08c108f60 0x7fe08c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 --2- 172.21.15.46:0/2531543063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c1047e0 0x7fe08c108830 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 --2- 172.21.15.46:0/2531543063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c103e30 0x7fe08c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 -- 172.21.15.46:0/2531543063 >> 172.21.15.46:0/2531543063 conn(0x7fe08c0ff910 msgr2=0x7fe08c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:35.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 -- 172.21.15.46:0/2531543063 shutdown_connections 2024-09-10T07:34:35.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.535+0000 7fe090a00640 1 -- 172.21.15.46:0/2531543063 wait complete. 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.536+0000 7fe090a00640 1 Processor -- start 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe090a00640 1 -- start start 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe090a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103e30 0x7fe08c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe090a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe08c1047e0 0x7fe08c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe090a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c108f60 0x7fe08c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe090a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c113fc0 con 0x7fe08c103e30 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe090a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c113e40 con 0x7fe08c1047e0 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe090a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c114140 con 0x7fe08c108f60 2024-09-10T07:34:35.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe08b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103e30 0x7fe08c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:35.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe08b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103e30 0x7fe08c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43944/0 (socket says 172.21.15.46:43944) 2024-09-10T07:34:35.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe08b400640 1 -- 172.21.15.46:0/3129007714 learned_addr learned my addr 172.21.15.46:0/3129007714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:35.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.537+0000 7fe08aa00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe08c1047e0 0x7fe08c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:35.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.538+0000 7fe08be00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c108f60 0x7fe08c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:35.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.538+0000 7fe08b400640 1 -- 172.21.15.46:0/3129007714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c108f60 msgr2=0x7fe08c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:35.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.538+0000 7fe08b400640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c108f60 0x7fe08c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.538+0000 7fe08b400640 1 -- 172.21.15.46:0/3129007714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe08c1047e0 msgr2=0x7fe08c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.538+0000 7fe08b400640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe08c1047e0 0x7fe08c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.538+0000 7fe08b400640 1 -- 172.21.15.46:0/3129007714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c19f410 con 0x7fe08c103e30 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.538+0000 7fe08b400640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103e30 0x7fe08c19a460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00d950 tx=0x7fe07c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.538+0000 7fe083e00640 1 -- 172.21.15.46:0/3129007714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c017070 con 0x7fe08c103e30 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.539+0000 7fe083e00640 1 -- 172.21.15.46:0/3129007714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe07c009c30 con 0x7fe08c103e30 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.539+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c19f700 con 0x7fe08c103e30 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.539+0000 7fe083e00640 1 -- 172.21.15.46:0/3129007714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00f6b0 con 0x7fe08c103e30 2024-09-10T07:34:35.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.539+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c1a6fe0 con 0x7fe08c103e30 2024-09-10T07:34:35.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.540+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe08c105310 con 0x7fe08c103e30 2024-09-10T07:34:35.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.542+0000 7fe083e00640 1 -- 172.21.15.46:0/3129007714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe07c0109d0 con 0x7fe08c103e30 2024-09-10T07:34:35.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.543+0000 7fe083e00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe0700a2c10 0x7fe0700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:35.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.543+0000 7fe083e00640 1 -- 172.21.15.46:0/3129007714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fe07c0cf170 con 0x7fe08c103e30 2024-09-10T07:34:35.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.543+0000 7fe08aa00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe0700a2c10 0x7fe0700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:35.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.544+0000 7fe08aa00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe0700a2c10 0x7fe0700a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe078009230 tx=0x7fe0780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.548+0000 7fe083e00640 1 -- 172.21.15.46:0/3129007714 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe07c009de0 con 0x7fe08c103e30 2024-09-10T07:34:35.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:35 smithi181 ceph-mon[35397]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:35.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:34:35.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:34:35.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:35 smithi063 ceph-mon[35020]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:35.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:34:35.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:34:35.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:35 smithi046 ceph-mon[30705]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:34:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:34:35.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:34:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.761+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe08c10c210 con 0x7fe0700a2c10 2024-09-10T07:34:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.767+0000 7fe083e00640 1 -- 172.21.15.46:0/3129007714 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe078031720 con 0x7fe0700a2c10 2024-09-10T07:34:35.767 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_0/816df620-8efd-4188-a43c-e2cbcbc08111 2024-09-10T07:34:35.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe0700a2c10 msgr2=0x7fe0700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:35.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe0700a2c10 0x7fe0700a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe078009230 tx=0x7fe0780058d0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103e30 msgr2=0x7fe08c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103e30 0x7fe08c19a460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00d950 tx=0x7fe07c00de10 comp rx=0 tx=0).stop 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 shutdown_connections 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c108f60 0x7fe08c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe08c1047e0 0x7fe08c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe0700a2c10 0x7fe0700a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 --2- 172.21.15.46:0/3129007714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103e30 0x7fe08c19a460 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 >> 172.21.15.46:0/3129007714 conn(0x7fe08c0ff910 msgr2=0x7fe08c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 shutdown_connections 2024-09-10T07:34:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:35.769+0000 7fe090a00640 1 -- 172.21.15.46:0/3129007714 wait complete. 2024-09-10T07:34:36.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:36 smithi046 ceph-mon[30705]: pgmap v336: 129 pgs: 129 active+clean; 607 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-10T07:34:36.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T07:34:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:36 smithi181 ceph-mon[35397]: pgmap v336: 129 pgs: 129 active+clean; 607 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-10T07:34:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:36 smithi063 ceph-mon[35020]: pgmap v336: 129 pgs: 129 active+clean; 607 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-10T07:34:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.620+0000 7fc466800640 1 -- 172.21.15.46:0/1570380884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460074620 msgr2=0x7fc460074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.620+0000 7fc466800640 1 --2- 172.21.15.46:0/1570380884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460074620 0x7fc460074a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc450009900 tx=0x7fc45002a480 comp rx=0 tx=0).stop 2024-09-10T07:34:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.621+0000 7fc466800640 1 -- 172.21.15.46:0/1570380884 shutdown_connections 2024-09-10T07:34:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.621+0000 7fc466800640 1 --2- 172.21.15.46:0/1570380884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc46010e3c0 0x7fc460110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.621+0000 7fc466800640 1 --2- 172.21.15.46:0/1570380884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460074f40 0x7fc46010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.621+0000 7fc466800640 1 --2- 172.21.15.46:0/1570380884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460074620 0x7fc460074a00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.621+0000 7fc466800640 1 -- 172.21.15.46:0/1570380884 >> 172.21.15.46:0/1570380884 conn(0x7fc4600fd8c0 msgr2=0x7fc4600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.621+0000 7fc466800640 1 -- 172.21.15.46:0/1570380884 shutdown_connections 2024-09-10T07:34:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.621+0000 7fc466800640 1 -- 172.21.15.46:0/1570380884 wait complete. 2024-09-10T07:34:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc466800640 1 Processor -- start 2024-09-10T07:34:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc466800640 1 -- start start 2024-09-10T07:34:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc466800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460074620 0x7fc46019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc466800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460074f40 0x7fc46019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc466800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc46010e3c0 0x7fc46019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc466800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc460114150 con 0x7fc460074f40 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc466800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc460113fd0 con 0x7fc46010e3c0 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc466800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601142d0 con 0x7fc460074620 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.622+0000 7fc45fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc46010e3c0 0x7fc46019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc46010e3c0 0x7fc46019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59434/0 (socket says 172.21.15.46:59434) 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45fe00640 1 -- 172.21.15.46:0/1872416 learned_addr learned my addr 172.21.15.46:0/1872416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc457e00640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460074f40 0x7fc46019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45f400640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460074620 0x7fc46019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45fe00640 1 -- 172.21.15.46:0/1872416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460074620 msgr2=0x7fc46019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460074620 0x7fc46019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45fe00640 1 -- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460074f40 msgr2=0x7fc46019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460074f40 0x7fc46019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45fe00640 1 -- 172.21.15.46:0/1872416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46019f600 con 0x7fc46010e3c0 2024-09-10T07:34:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc46010e3c0 0x7fc46019ef20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc45800eb10 tx=0x7fc45800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45cc00640 1 -- 172.21.15.46:0/1872416 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458004b20 con 0x7fc46010e3c0 2024-09-10T07:34:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc45cc00640 1 -- 172.21.15.46:0/1872416 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc458004cc0 con 0x7fc46010e3c0 2024-09-10T07:34:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.623+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46019f8f0 con 0x7fc46010e3c0 2024-09-10T07:34:37.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.624+0000 7fc45cc00640 1 -- 172.21.15.46:0/1872416 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458010760 con 0x7fc46010e3c0 2024-09-10T07:34:37.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.624+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601a71d0 con 0x7fc46010e3c0 2024-09-10T07:34:37.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.625+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc460074a00 con 0x7fc46010e3c0 2024-09-10T07:34:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.626+0000 7fc45cc00640 1 -- 172.21.15.46:0/1872416 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc458005020 con 0x7fc46010e3c0 2024-09-10T07:34:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.628+0000 7fc45cc00640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2c10 0x7fc4440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.628+0000 7fc45cc00640 1 -- 172.21.15.46:0/1872416 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fc4580cfb30 con 0x7fc46010e3c0 2024-09-10T07:34:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.628+0000 7fc45f400640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2c10 0x7fc4440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.629+0000 7fc45f400640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2c10 0x7fc4440a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc450005ab0 tx=0x7fc450005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:37.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.632+0000 7fc45cc00640 1 -- 172.21.15.46:0/1872416 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc45809c570 con 0x7fc46010e3c0 2024-09-10T07:34:37.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:37 smithi046 ceph-mon[30705]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.844+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc460062630 con 0x7fc4440a2c10 2024-09-10T07:34:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.868+0000 7fc45cc00640 1 -- 172.21.15.46:0/1872416 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc460062630 con 0x7fc4440a2c10 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2c10 msgr2=0x7fc4440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2c10 0x7fc4440a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc450005ab0 tx=0x7fc450005a20 comp rx=0 tx=0).stop 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc46010e3c0 msgr2=0x7fc46019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc46010e3c0 0x7fc46019ef20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc45800eb10 tx=0x7fc45800c2d0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 shutdown_connections 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc46010e3c0 0x7fc46019ef20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2c10 0x7fc4440a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460074f40 0x7fc46019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 --2- 172.21.15.46:0/1872416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460074620 0x7fc46019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:37.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 >> 172.21.15.46:0/1872416 conn(0x7fc4600fd8c0 msgr2=0x7fc4600fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:37.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 shutdown_connections 2024-09-10T07:34:37.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:37.870+0000 7fc466800640 1 -- 172.21.15.46:0/1872416 wait complete. 2024-09-10T07:34:37.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:37 smithi181 ceph-mon[35397]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:37.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:37 smithi063 ceph-mon[35020]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:38.638 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:38 smithi046 ceph-mon[30705]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.1 KiB/s wr, 23 op/s 2024-09-10T07:34:38.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T07:34:38.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:38 smithi181 ceph-mon[35397]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.1 KiB/s wr, 23 op/s 2024-09-10T07:34:38.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:38 smithi063 ceph-mon[35020]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.1 KiB/s wr, 23 op/s 2024-09-10T07:34:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:39 smithi046 ceph-mon[30705]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:39.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 -- 172.21.15.46:0/2336149942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c103ff0 msgr2=0x7fe44c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 --2- 172.21.15.46:0/2336149942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c103ff0 0x7fe44c1043d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009900 tx=0x7fe43c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 -- 172.21.15.46:0/2336149942 shutdown_connections 2024-09-10T07:34:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 --2- 172.21.15.46:0/2336149942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe44c109120 0x7fe44c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 --2- 172.21.15.46:0/2336149942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe44c1049a0 0x7fe44c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 --2- 172.21.15.46:0/2336149942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c103ff0 0x7fe44c1043d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 -- 172.21.15.46:0/2336149942 >> 172.21.15.46:0/2336149942 conn(0x7fe44c0ffa90 msgr2=0x7fe44c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 -- 172.21.15.46:0/2336149942 shutdown_connections 2024-09-10T07:34:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.660+0000 7fe451800640 1 -- 172.21.15.46:0/2336149942 wait complete. 2024-09-10T07:34:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.661+0000 7fe451800640 1 Processor -- start 2024-09-10T07:34:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.661+0000 7fe451800640 1 -- start start 2024-09-10T07:34:39.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe451800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe44c103ff0 0x7fe44c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe451800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c1049a0 0x7fe44c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe451800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe44c109120 0x7fe44c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe451800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1141a0 con 0x7fe44c1049a0 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe451800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c114020 con 0x7fe44c103ff0 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe451800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c114320 con 0x7fe44c109120 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe443e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c1049a0 0x7fe44c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe443e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c1049a0 0x7fe44c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33768/0 (socket says 172.21.15.46:33768) 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe443e00640 1 -- 172.21.15.46:0/235380410 learned_addr learned my addr 172.21.15.46:0/235380410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe44aa00640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe44c103ff0 0x7fe44c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:39.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.662+0000 7fe44b400640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe44c109120 0x7fe44c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:39.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe44aa00640 1 -- 172.21.15.46:0/235380410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe44c109120 msgr2=0x7fe44c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:39.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe44aa00640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe44c109120 0x7fe44c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe44aa00640 1 -- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c1049a0 msgr2=0x7fe44c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe44aa00640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c1049a0 0x7fe44c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe44aa00640 1 -- 172.21.15.46:0/235380410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c19f5d0 con 0x7fe44c103ff0 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe443e00640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c1049a0 0x7fe44c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe44aa00640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe44c103ff0 0x7fe44c19a620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe43c02a910 tx=0x7fe43c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe443400640 1 -- 172.21.15.46:0/235380410 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c03f070 con 0x7fe44c103ff0 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c19f860 con 0x7fe44c103ff0 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe443400640 1 -- 172.21.15.46:0/235380410 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43c031c90 con 0x7fe44c103ff0 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.663+0000 7fe443400640 1 -- 172.21.15.46:0/235380410 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c035030 con 0x7fe44c103ff0 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.664+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1a71a0 con 0x7fe44c103ff0 2024-09-10T07:34:39.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.665+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c1043d0 con 0x7fe44c103ff0 2024-09-10T07:34:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.666+0000 7fe443400640 1 -- 172.21.15.46:0/235380410 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe43c042070 con 0x7fe44c103ff0 2024-09-10T07:34:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.668+0000 7fe443400640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe4300a2c90 0x7fe4300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.668+0000 7fe443400640 1 -- 172.21.15.46:0/235380410 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fe43c0f03d0 con 0x7fe44c103ff0 2024-09-10T07:34:39.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.668+0000 7fe443e00640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe4300a2c90 0x7fe4300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:39.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.669+0000 7fe443e00640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe4300a2c90 0x7fe4300a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0fff20 tx=0x7fe434005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:39.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.671+0000 7fe443400640 1 -- 172.21.15.46:0/235380410 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe43c0bc820 con 0x7fe44c103ff0 2024-09-10T07:34:39.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.897+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe44c062630 con 0x7fe4300a2c90 2024-09-10T07:34:39.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:39 smithi181 ceph-mon[35397]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:39.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:39 smithi063 ceph-mon[35020]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:39.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.902+0000 7fe443400640 1 -- 172.21.15.46:0/235380410 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe44c062630 con 0x7fe4300a2c90 2024-09-10T07:34:39.902 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e 2024-09-10T07:34:39.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe4300a2c90 msgr2=0x7fe4300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe4300a2c90 0x7fe4300a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0fff20 tx=0x7fe434005f90 comp rx=0 tx=0).stop 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe44c103ff0 msgr2=0x7fe44c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe44c103ff0 0x7fe44c19a620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe43c02a910 tx=0x7fe43c003b20 comp rx=0 tx=0).stop 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 shutdown_connections 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe44c109120 0x7fe44c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe4300a2c90 0x7fe4300a5150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c1049a0 0x7fe44c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 --2- 172.21.15.46:0/235380410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe44c103ff0 0x7fe44c19a620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 >> 172.21.15.46:0/235380410 conn(0x7fe44c0ffa90 msgr2=0x7fe44c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 shutdown_connections 2024-09-10T07:34:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:39.904+0000 7fe451800640 1 -- 172.21.15.46:0/235380410 wait complete. 2024-09-10T07:34:40.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:40 smithi046 ceph-mon[30705]: pgmap v338: 129 pgs: 129 active+clean; 608 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-09-10T07:34:40.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:34:40.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:40 smithi181 ceph-mon[35397]: pgmap v338: 129 pgs: 129 active+clean; 608 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-09-10T07:34:40.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:40 smithi063 ceph-mon[35020]: pgmap v338: 129 pgs: 129 active+clean; 608 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-09-10T07:34:41.229 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:41.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:41 smithi046 ceph-mon[30705]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:41.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 -- 172.21.15.46:0/302946268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 msgr2=0x7f5c74074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:41.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 --2- 172.21.15.46:0/302946268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 0x7f5c74074970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009900 tx=0x7f5c6802a480 comp rx=0 tx=0).stop 2024-09-10T07:34:41.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 -- 172.21.15.46:0/302946268 shutdown_connections 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 --2- 172.21.15.46:0/302946268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74112b90 0x7f5c74114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 --2- 172.21.15.46:0/302946268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c74074eb0 0x7f5c74112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 --2- 172.21.15.46:0/302946268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 0x7f5c74074970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 -- 172.21.15.46:0/302946268 >> 172.21.15.46:0/302946268 conn(0x7f5c740fd8c0 msgr2=0x7f5c740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 -- 172.21.15.46:0/302946268 shutdown_connections 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.855+0000 7f5c7b600640 1 -- 172.21.15.46:0/302946268 wait complete. 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.856+0000 7f5c7b600640 1 Processor -- start 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.856+0000 7f5c7b600640 1 -- start start 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c7b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 0x7f5c741a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c7b600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c74074eb0 0x7f5c741a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c7b600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74112b90 0x7f5c7419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c7b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74118570 con 0x7f5c74074590 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c7b600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741183f0 con 0x7f5c74074eb0 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c7b600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741186f0 con 0x7f5c74112b90 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c78c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 0x7f5c741a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c78c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 0x7f5c741a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33782/0 (socket says 172.21.15.46:33782) 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c78c00640 1 -- 172.21.15.46:0/1421411438 learned_addr learned my addr 172.21.15.46:0/1421411438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c73e00640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c74074eb0 0x7f5c741a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c79600640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74112b90 0x7f5c7419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c78c00640 1 -- 172.21.15.46:0/1421411438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74112b90 msgr2=0x7f5c7419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c78c00640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74112b90 0x7f5c7419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c78c00640 1 -- 172.21.15.46:0/1421411438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c74074eb0 msgr2=0x7f5c741a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c78c00640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c74074eb0 0x7f5c741a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c78c00640 1 -- 172.21.15.46:0/1421411438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7419f000 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.857+0000 7f5c73e00640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c74074eb0 0x7f5c741a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.858+0000 7f5c78c00640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 0x7f5c741a4710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c68006010 tx=0x7f5c680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.858+0000 7f5c71600640 1 -- 172.21.15.46:0/1421411438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6803f030 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.858+0000 7f5c71600640 1 -- 172.21.15.46:0/1421411438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c68003ce0 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.858+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7419f290 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.858+0000 7f5c71600640 1 -- 172.21.15.46:0/1421411438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68035070 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.858+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c741ab5e0 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.860+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c74074970 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.861+0000 7f5c71600640 1 -- 172.21.15.46:0/1421411438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5c68042030 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.863+0000 7f5c71600640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c5c0a2c90 0x7f5c5c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.863+0000 7f5c71600640 1 -- 172.21.15.46:0/1421411438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f5c680ef9c0 con 0x7f5c74074590 2024-09-10T07:34:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.863+0000 7f5c73e00640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c5c0a2c90 0x7f5c5c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:41.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.864+0000 7f5c73e00640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c5c0a2c90 0x7f5c5c0a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c740ffec0 tx=0x7f5c60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:41.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:41.867+0000 7f5c71600640 1 -- 172.21.15.46:0/1421411438 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c680bbe70 con 0x7f5c74074590 2024-09-10T07:34:41.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:41 smithi181 ceph-mon[35397]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:41.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:41 smithi063 ceph-mon[35020]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:34:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.090+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c7419fe00 con 0x7f5c74074590 2024-09-10T07:34:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.091+0000 7f5c71600640 1 -- 172.21.15.46:0/1421411438 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11972 (secure 0 0 0) 0x7f5c680c0170 con 0x7f5c74074590 2024-09-10T07:34:42.092 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:42.092 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"btime":"2024-09-10T07:34:28:805113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:34:27.875261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:34:42.092 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T07:34:42.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c5c0a2c90 msgr2=0x7f5c5c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:42.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c5c0a2c90 0x7f5c5c0a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c740ffec0 tx=0x7f5c60005f90 comp rx=0 tx=0).stop 2024-09-10T07:34:42.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 msgr2=0x7f5c741a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:42.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 0x7f5c741a4710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c68006010 tx=0x7f5c680042f0 comp rx=0 tx=0).stop 2024-09-10T07:34:42.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 shutdown_connections 2024-09-10T07:34:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74112b90 0x7f5c7419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c74074eb0 0x7f5c741a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c5c0a2c90 0x7f5c5c0a5150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 --2- 172.21.15.46:0/1421411438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c74074590 0x7f5c741a4710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 >> 172.21.15.46:0/1421411438 conn(0x7f5c740fd8c0 msgr2=0x7f5c740ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 shutdown_connections 2024-09-10T07:34:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:42.093+0000 7f5c7b600640 1 -- 172.21.15.46:0/1421411438 wait complete. 2024-09-10T07:34:42.832 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:42 smithi046 ceph-mon[30705]: pgmap v339: 129 pgs: 129 active+clean; 626 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2024-09-10T07:34:42.832 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:42 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1421411438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:34:42.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:34:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:42 smithi181 ceph-mon[35397]: pgmap v339: 129 pgs: 129 active+clean; 626 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2024-09-10T07:34:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:42 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1421411438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:34:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:42 smithi063 ceph-mon[35020]: pgmap v339: 129 pgs: 129 active+clean; 626 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2024-09-10T07:34:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:42 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1421411438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:34:43.250 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:43.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 -- 172.21.15.46:0/425633582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 msgr2=0x7fd63010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 --2- 172.21.15.46:0/425633582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 0x7fd63010de80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009860 tx=0x7fd61c02a3d0 comp rx=0 tx=0).stop 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 -- 172.21.15.46:0/425633582 shutdown_connections 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 --2- 172.21.15.46:0/425633582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63010e3c0 0x7fd630110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 --2- 172.21.15.46:0/425633582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 0x7fd63010de80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 --2- 172.21.15.46:0/425633582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630074620 0x7fd630074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 -- 172.21.15.46:0/425633582 >> 172.21.15.46:0/425633582 conn(0x7fd6300fd8c0 msgr2=0x7fd6300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 -- 172.21.15.46:0/425633582 shutdown_connections 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.895+0000 7fd635200640 1 -- 172.21.15.46:0/425633582 wait complete. 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.896+0000 7fd635200640 1 Processor -- start 2024-09-10T07:34:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.896+0000 7fd635200640 1 -- start start 2024-09-10T07:34:43.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd635200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630074620 0x7fd63019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:43.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd635200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 0x7fd63019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:43.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd635200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63010e3c0 0x7fd63019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:43.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd635200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd630114140 con 0x7fd630074f40 2024-09-10T07:34:43.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd635200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd630113fc0 con 0x7fd63010e3c0 2024-09-10T07:34:43.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd635200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301142c0 con 0x7fd630074620 2024-09-10T07:34:43.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 0x7fd63019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:43.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 0x7fd63019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33800/0 (socket says 172.21.15.46:33800) 2024-09-10T07:34:43.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62d400640 1 -- 172.21.15.46:0/2332402358 learned_addr learned my addr 172.21.15.46:0/2332402358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:43.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62de00640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630074620 0x7fd63019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:43.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62e800640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63010e3c0 0x7fd63019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:43.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62d400640 1 -- 172.21.15.46:0/2332402358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630074620 msgr2=0x7fd63019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62d400640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630074620 0x7fd63019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62d400640 1 -- 172.21.15.46:0/2332402358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63010e3c0 msgr2=0x7fd63019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.897+0000 7fd62d400640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63010e3c0 0x7fd63019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.898+0000 7fd62d400640 1 -- 172.21.15.46:0/2332402358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63019f630 con 0x7fd630074f40 2024-09-10T07:34:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.898+0000 7fd62d400640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 0x7fd63019abc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd61c02a8f0 tx=0x7fd61c030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.898+0000 7fd626a00640 1 -- 172.21.15.46:0/2332402358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c040020 con 0x7fd630074f40 2024-09-10T07:34:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.898+0000 7fd626a00640 1 -- 172.21.15.46:0/2332402358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c004460 con 0x7fd630074f40 2024-09-10T07:34:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.898+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63019f8c0 con 0x7fd630074f40 2024-09-10T07:34:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.898+0000 7fd626a00640 1 -- 172.21.15.46:0/2332402358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0046c0 con 0x7fd630074f40 2024-09-10T07:34:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.898+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6301a7200 con 0x7fd630074f40 2024-09-10T07:34:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.899+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd630075cd0 con 0x7fd630074f40 2024-09-10T07:34:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.901+0000 7fd626a00640 1 -- 172.21.15.46:0/2332402358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd61c043020 con 0x7fd630074f40 2024-09-10T07:34:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.903+0000 7fd626a00640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd6140a2c10 0x7fd6140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.904+0000 7fd626a00640 1 -- 172.21.15.46:0/2332402358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fd61c0f04b0 con 0x7fd630074f40 2024-09-10T07:34:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.904+0000 7fd62de00640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd6140a2c10 0x7fd6140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.904+0000 7fd62de00640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd6140a2c10 0x7fd6140a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd620009530 tx=0x7fd620009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:43.908+0000 7fd626a00640 1 -- 172.21.15.46:0/2332402358 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd61c04a020 con 0x7fd630074f40 2024-09-10T07:34:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.096+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd630109540 con 0x7fd630074f40 2024-09-10T07:34:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.097+0000 7fd626a00640 1 -- 172.21.15.46:0/2332402358 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24597 (secure 0 0 0) 0x7fd61c0bcef0 con 0x7fd630074f40 2024-09-10T07:34:44.098 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:34:14.007613+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:34:44.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd6140a2c10 msgr2=0x7fd6140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd6140a2c10 0x7fd6140a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd620009530 tx=0x7fd620009260 comp rx=0 tx=0).stop 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 msgr2=0x7fd63019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 0x7fd63019abc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd61c02a8f0 tx=0x7fd61c030650 comp rx=0 tx=0).stop 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 shutdown_connections 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63010e3c0 0x7fd63019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd6140a2c10 0x7fd6140a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd630074f40 0x7fd63019abc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 --2- 172.21.15.46:0/2332402358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630074620 0x7fd63019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 >> 172.21.15.46:0/2332402358 conn(0x7fd6300fd8c0 msgr2=0x7fd6300fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 shutdown_connections 2024-09-10T07:34:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:44.101+0000 7fd635200640 1 -- 172.21.15.46:0/2332402358 wait complete. 2024-09-10T07:34:44.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:44 smithi046 ceph-mon[30705]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-09-10T07:34:44.790 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:44 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2332402358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:44.846 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T07:34:44.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T07:34:44.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:44 smithi181 ceph-mon[35397]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-09-10T07:34:44.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:44 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2332402358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:44.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:44 smithi063 ceph-mon[35020]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-09-10T07:34:44.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:44 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2332402358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:45.249 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:45.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 -- 172.21.15.46:0/1633377724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c109120 msgr2=0x7f172c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 --2- 172.21.15.46:0/1633377724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c109120 0x7f172c110ca0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1724009ca0 tx=0x7f172402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 -- 172.21.15.46:0/1633377724 shutdown_connections 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 --2- 172.21.15.46:0/1633377724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c109120 0x7f172c110ca0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 --2- 172.21.15.46:0/1633377724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172c1049a0 0x7f172c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 --2- 172.21.15.46:0/1633377724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f172c103ff0 0x7f172c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 -- 172.21.15.46:0/1633377724 >> 172.21.15.46:0/1633377724 conn(0x7f172c0ffa90 msgr2=0x7f172c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 -- 172.21.15.46:0/1633377724 shutdown_connections 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.919+0000 7f1731000640 1 -- 172.21.15.46:0/1633377724 wait complete. 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.920+0000 7f1731000640 1 Processor -- start 2024-09-10T07:34:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.920+0000 7f1731000640 1 -- start start 2024-09-10T07:34:45.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.920+0000 7f1731000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c103ff0 0x7f172c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:45.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.920+0000 7f1731000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172c1049a0 0x7f172c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:45.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.920+0000 7f1731000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f172c109120 0x7f172c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:45.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.920+0000 7f1731000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f172c114130 con 0x7f172c103ff0 2024-09-10T07:34:45.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.920+0000 7f1731000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f172c113fb0 con 0x7f172c109120 2024-09-10T07:34:45.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.920+0000 7f1731000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f172c1142b0 con 0x7f172c1049a0 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f172a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f172c109120 0x7f172c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f172a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f172c109120 0x7f172c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47970/0 (socket says 172.21.15.46:47970) 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f1729e00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c103ff0 0x7f172c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f1723e00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172c1049a0 0x7f172c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f172a800640 1 -- 172.21.15.46:0/3155626691 learned_addr learned my addr 172.21.15.46:0/3155626691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f1729e00640 1 -- 172.21.15.46:0/3155626691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172c1049a0 msgr2=0x7f172c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f1729e00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172c1049a0 0x7f172c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f1729e00640 1 -- 172.21.15.46:0/3155626691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f172c109120 msgr2=0x7f172c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f1729e00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f172c109120 0x7f172c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f1729e00640 1 -- 172.21.15.46:0/3155626691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c19f540 con 0x7f172c103ff0 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.921+0000 7f1723e00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172c1049a0 0x7f172c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.922+0000 7f1729e00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c103ff0 0x7f172c19a590 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f171c00e9f0 tx=0x7f171c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.922+0000 7f1722a00640 1 -- 172.21.15.46:0/3155626691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c00ccc0 con 0x7f172c103ff0 2024-09-10T07:34:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.922+0000 7f1722a00640 1 -- 172.21.15.46:0/3155626691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f171c010910 con 0x7f172c103ff0 2024-09-10T07:34:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.922+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c19f830 con 0x7f172c103ff0 2024-09-10T07:34:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.922+0000 7f1722a00640 1 -- 172.21.15.46:0/3155626691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c018960 con 0x7f172c103ff0 2024-09-10T07:34:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.922+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c1a7110 con 0x7f172c103ff0 2024-09-10T07:34:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.923+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c1043d0 con 0x7f172c103ff0 2024-09-10T07:34:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.925+0000 7f1722a00640 1 -- 172.21.15.46:0/3155626691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f171c010ac0 con 0x7f172c103ff0 2024-09-10T07:34:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.927+0000 7f1722a00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17100a2c90 0x7f17100a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.927+0000 7f1722a00640 1 -- 172.21.15.46:0/3155626691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f171c0cfd30 con 0x7f172c103ff0 2024-09-10T07:34:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.927+0000 7f1723e00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17100a2c90 0x7f17100a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.928+0000 7f1723e00640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17100a2c90 0x7f17100a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ffeb0 tx=0x7f171400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:45.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:45.929+0000 7f1722a00640 1 -- 172.21.15.46:0/3155626691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f171c09c120 con 0x7f172c103ff0 2024-09-10T07:34:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.113+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f172c100d90 con 0x7f17100a2c90 2024-09-10T07:34:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.124+0000 7f1722a00640 1 -- 172.21.15.46:0/3155626691 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340645 (secure 0 0 0) 0x7f172c100d90 con 0x7f17100a2c90 2024-09-10T07:34:46.134 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:34:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17100a2c90 msgr2=0x7f17100a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17100a2c90 0x7f17100a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ffeb0 tx=0x7f171400b390 comp rx=0 tx=0).stop 2024-09-10T07:34:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c103ff0 msgr2=0x7f172c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c103ff0 0x7f172c19a590 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f171c00e9f0 tx=0x7f171c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:34:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 shutdown_connections 2024-09-10T07:34:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f172c109120 0x7f172c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172c1049a0 0x7f172c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17100a2c90 0x7f17100a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 --2- 172.21.15.46:0/3155626691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c103ff0 0x7f172c19a590 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 >> 172.21.15.46:0/3155626691 conn(0x7f172c0ffa90 msgr2=0x7f172c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 shutdown_connections 2024-09-10T07:34:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:46.129+0000 7f1731000640 1 -- 172.21.15.46:0/3155626691 wait complete. 2024-09-10T07:34:46.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:46 smithi181 ceph-mon[35397]: pgmap v341: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-10T07:34:46.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:46 smithi063 ceph-mon[35020]: pgmap v341: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-10T07:34:46.900 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:46 smithi046 ceph-mon[30705]: pgmap v341: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-10T07:34:46.939 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-10T07:34:44.476460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649709,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":334,"num_read_kb":285,"num_write":425,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357968,"kb_used_data":12880,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583856,"statfs":{"total":1151940427776,"available":1151573868544,"internally_reserved":0,"allocated":13189120,"data_stored":8408865,"data_compressed":426912,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48682,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":121,"num_write":34,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003500"},"pg_stats":[{"pgid":"2.1f","version":"95'13","reported_seq":70,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:41.838832+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:34:41.838832+0000","last_peered":"2024-09-10T07:34:41.838832+0000","last_clean":"2024-09-10T07:34:41.838832+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:34:41.838832+0000","last_undegraded":"2024-09-10T07:34:41.838832+0000","last_fullsized":"2024-09-10T07:34:41.838832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012250+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:34:14.012250+0000","last_peered":"2024-09-10T07:34:14.012250+0000","last_clean":"2024-09-10T07:34:14.012250+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:34:14.012250+0000","last_undegraded":"2024-09-10T07:34:14.012250+0000","last_fullsized":"2024-09-10T07:34:14.012250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017508+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:34:14.017508+0000","last_peered":"2024-09-10T07:34:14.017508+0000","last_clean":"2024-09-10T07:34:14.017508+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:34:14.017508+0000","last_undegraded":"2024-09-10T07:34:14.017508+0000","last_fullsized":"2024-09-10T07:34:14.017508+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012531+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:34:14.012531+0000","last_peered":"2024-09-10T07:34:14.012531+0000","last_clean":"2024-09-10T07:34:14.012531+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:34:14.012531+0000","last_undegraded":"2024-09-10T07:34:14.012531+0000","last_fullsized":"2024-09-10T07:34:14.012531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019829+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:34:14.019829+0000","last_peered":"2024-09-10T07:34:14.019829+0000","last_clean":"2024-09-10T07:34:14.019829+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:34:14.019829+0000","last_undegraded":"2024-09-10T07:34:14.019829+0000","last_fullsized":"2024-09-10T07:34:14.019829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015733+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:34:14.015733+0000","last_peered":"2024-09-10T07:34:14.015733+0000","last_clean":"2024-09-10T07:34:14.015733+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:34:14.015733+0000","last_undegraded":"2024-09-10T07:34:14.015733+0000","last_fullsized":"2024-09-10T07:34:14.015733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.815290+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:34:25.815290+0000","last_peered":"2024-09-10T07:34:25.815290+0000","last_clean":"2024-09-10T07:34:25.815290+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:34:25.815290+0000","last_undegraded":"2024-09-10T07:34:25.815290+0000","last_fullsized":"2024-09-10T07:34:25.815290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106528+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:34:14.106528+0000","last_peered":"2024-09-10T07:34:14.106528+0000","last_clean":"2024-09-10T07:34:14.106528+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:34:14.106528+0000","last_undegraded":"2024-09-10T07:34:14.106528+0000","last_fullsized":"2024-09-10T07:34:14.106528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016008+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:34:14.016008+0000","last_peered":"2024-09-10T07:34:14.016008+0000","last_clean":"2024-09-10T07:34:14.016008+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:34:14.016008+0000","last_undegraded":"2024-09-10T07:34:14.016008+0000","last_fullsized":"2024-09-10T07:34:14.016008+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.814807+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:34:25.814807+0000","last_peered":"2024-09-10T07:34:25.814807+0000","last_clean":"2024-09-10T07:34:25.814807+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:34:25.814807+0000","last_undegraded":"2024-09-10T07:34:25.814807+0000","last_fullsized":"2024-09-10T07:34:25.814807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156901+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:34:14.156901+0000","last_peered":"2024-09-10T07:34:14.156901+0000","last_clean":"2024-09-10T07:34:14.156901+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:34:14.156901+0000","last_undegraded":"2024-09-10T07:34:14.156901+0000","last_fullsized":"2024-09-10T07:34:14.156901+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015987+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:34:14.015987+0000","last_peered":"2024-09-10T07:34:14.015987+0000","last_clean":"2024-09-10T07:34:14.015987+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:34:14.015987+0000","last_undegraded":"2024-09-10T07:34:14.015987+0000","last_fullsized":"2024-09-10T07:34:14.015987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011724+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:34:14.011724+0000","last_peered":"2024-09-10T07:34:14.011724+0000","last_clean":"2024-09-10T07:34:14.011724+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:34:14.011724+0000","last_undegraded":"2024-09-10T07:34:14.011724+0000","last_fullsized":"2024-09-10T07:34:14.011724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:23.066828+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:34:23.066828+0000","last_peered":"2024-09-10T07:34:23.066828+0000","last_clean":"2024-09-10T07:34:23.066828+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:34:23.066828+0000","last_undegraded":"2024-09-10T07:34:23.066828+0000","last_fullsized":"2024-09-10T07:34:23.066828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016148+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:34:14.016148+0000","last_peered":"2024-09-10T07:34:14.016148+0000","last_clean":"2024-09-10T07:34:14.016148+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:34:14.016148+0000","last_undegraded":"2024-09-10T07:34:14.016148+0000","last_fullsized":"2024-09-10T07:34:14.016148+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012240+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:34:14.012240+0000","last_peered":"2024-09-10T07:34:14.012240+0000","last_clean":"2024-09-10T07:34:14.012240+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:34:14.012240+0000","last_undegraded":"2024-09-10T07:34:14.012240+0000","last_fullsized":"2024-09-10T07:34:14.012240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.085370+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:34:25.085370+0000","last_peered":"2024-09-10T07:34:25.085370+0000","last_clean":"2024-09-10T07:34:25.085370+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:34:25.085370+0000","last_undegraded":"2024-09-10T07:34:25.085370+0000","last_fullsized":"2024-09-10T07:34:25.085370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016352+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:34:14.016352+0000","last_peered":"2024-09-10T07:34:14.016352+0000","last_clean":"2024-09-10T07:34:14.016352+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:34:14.016352+0000","last_undegraded":"2024-09-10T07:34:14.016352+0000","last_fullsized":"2024-09-10T07:34:14.016352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014187+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:34:14.014187+0000","last_peered":"2024-09-10T07:34:14.014187+0000","last_clean":"2024-09-10T07:34:14.014187+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:34:14.014187+0000","last_undegraded":"2024-09-10T07:34:14.014187+0000","last_fullsized":"2024-09-10T07:34:14.014187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.816443+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:34:25.816443+0000","last_peered":"2024-09-10T07:34:25.816443+0000","last_clean":"2024-09-10T07:34:25.816443+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:34:25.816443+0000","last_undegraded":"2024-09-10T07:34:25.816443+0000","last_fullsized":"2024-09-10T07:34:25.816443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018684+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:34:14.018684+0000","last_peered":"2024-09-10T07:34:14.018684+0000","last_clean":"2024-09-10T07:34:14.018684+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:34:14.018684+0000","last_undegraded":"2024-09-10T07:34:14.018684+0000","last_fullsized":"2024-09-10T07:34:14.018684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011918+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:34:14.011918+0000","last_peered":"2024-09-10T07:34:14.011918+0000","last_clean":"2024-09-10T07:34:14.011918+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:34:14.011918+0000","last_undegraded":"2024-09-10T07:34:14.011918+0000","last_fullsized":"2024-09-10T07:34:14.011918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:23.066444+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:34:23.066444+0000","last_peered":"2024-09-10T07:34:23.066444+0000","last_clean":"2024-09-10T07:34:23.066444+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:34:23.066444+0000","last_undegraded":"2024-09-10T07:34:23.066444+0000","last_fullsized":"2024-09-10T07:34:23.066444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.022602+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:34:14.022602+0000","last_peered":"2024-09-10T07:34:14.022602+0000","last_clean":"2024-09-10T07:34:14.022602+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:34:14.022602+0000","last_undegraded":"2024-09-10T07:34:14.022602+0000","last_fullsized":"2024-09-10T07:34:14.022602+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019291+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:34:14.019291+0000","last_peered":"2024-09-10T07:34:14.019291+0000","last_clean":"2024-09-10T07:34:14.019291+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:34:14.019291+0000","last_undegraded":"2024-09-10T07:34:14.019291+0000","last_fullsized":"2024-09-10T07:34:14.019291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.812957+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:34:25.812957+0000","last_peered":"2024-09-10T07:34:25.812957+0000","last_clean":"2024-09-10T07:34:25.812957+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:34:25.812957+0000","last_undegraded":"2024-09-10T07:34:25.812957+0000","last_fullsized":"2024-09-10T07:34:25.812957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017385+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:34:14.017385+0000","last_peered":"2024-09-10T07:34:14.017385+0000","last_clean":"2024-09-10T07:34:14.017385+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:34:14.017385+0000","last_undegraded":"2024-09-10T07:34:14.017385+0000","last_fullsized":"2024-09-10T07:34:14.017385+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012416+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:34:14.012416+0000","last_peered":"2024-09-10T07:34:14.012416+0000","last_clean":"2024-09-10T07:34:14.012416+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:34:14.012416+0000","last_undegraded":"2024-09-10T07:34:14.012416+0000","last_fullsized":"2024-09-10T07:34:14.012416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'14","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:40.893891+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:34:40.893891+0000","last_peered":"2024-09-10T07:34:40.893891+0000","last_clean":"2024-09-10T07:34:40.893891+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:34:40.893891+0000","last_undegraded":"2024-09-10T07:34:40.893891+0000","last_fullsized":"2024-09-10T07:34:40.893891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011477+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:34:14.011477+0000","last_peered":"2024-09-10T07:34:14.011477+0000","last_clean":"2024-09-10T07:34:14.011477+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:34:14.011477+0000","last_undegraded":"2024-09-10T07:34:14.011477+0000","last_fullsized":"2024-09-10T07:34:14.011477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156259+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:34:14.156259+0000","last_peered":"2024-09-10T07:34:14.156259+0000","last_clean":"2024-09-10T07:34:14.156259+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:34:14.156259+0000","last_undegraded":"2024-09-10T07:34:14.156259+0000","last_fullsized":"2024-09-10T07:34:14.156259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.801705+0000","last_change":"2024-09-10T07:33:31.999392+0000","last_active":"2024-09-10T07:34:25.801705+0000","last_peered":"2024-09-10T07:34:25.801705+0000","last_clean":"2024-09-10T07:34:25.801705+0000","last_became_active":"2024-09-10T07:33:31.998852+0000","last_became_peered":"2024-09-10T07:33:31.998852+0000","last_unstale":"2024-09-10T07:34:25.801705+0000","last_undegraded":"2024-09-10T07:34:25.801705+0000","last_fullsized":"2024-09-10T07:34:25.801705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:28.180972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015125+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:34:14.015125+0000","last_peered":"2024-09-10T07:34:14.015125+0000","last_clean":"2024-09-10T07:34:14.015125+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:34:14.015125+0000","last_undegraded":"2024-09-10T07:34:14.015125+0000","last_fullsized":"2024-09-10T07:34:14.015125+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.157371+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:34:14.157371+0000","last_peered":"2024-09-10T07:34:14.157371+0000","last_clean":"2024-09-10T07:34:14.157371+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:34:14.157371+0000","last_undegraded":"2024-09-10T07:34:14.157371+0000","last_fullsized":"2024-09-10T07:34:14.157371+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018224+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:34:14.018224+0000","last_peered":"2024-09-10T07:34:14.018224+0000","last_clean":"2024-09-10T07:34:14.018224+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:34:14.018224+0000","last_undegraded":"2024-09-10T07:34:14.018224+0000","last_fullsized":"2024-09-10T07:34:14.018224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011919+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:34:14.011919+0000","last_peered":"2024-09-10T07:34:14.011919+0000","last_clean":"2024-09-10T07:34:14.011919+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:34:14.011919+0000","last_undegraded":"2024-09-10T07:34:14.011919+0000","last_fullsized":"2024-09-10T07:34:14.011919+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.013923+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:34:14.013923+0000","last_peered":"2024-09-10T07:34:14.013923+0000","last_clean":"2024-09-10T07:34:14.013923+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:34:14.013923+0000","last_undegraded":"2024-09-10T07:34:14.013923+0000","last_fullsized":"2024-09-10T07:34:14.013923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:39.836464+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:34:39.836464+0000","last_peered":"2024-09-10T07:34:39.836464+0000","last_clean":"2024-09-10T07:34:39.836464+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:34:39.836464+0000","last_undegraded":"2024-09-10T07:34:39.836464+0000","last_fullsized":"2024-09-10T07:34:39.836464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156129+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:34:14.156129+0000","last_peered":"2024-09-10T07:34:14.156129+0000","last_clean":"2024-09-10T07:34:14.156129+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:34:14.156129+0000","last_undegraded":"2024-09-10T07:34:14.156129+0000","last_fullsized":"2024-09-10T07:34:14.156129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.020283+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:34:14.020283+0000","last_peered":"2024-09-10T07:34:14.020283+0000","last_clean":"2024-09-10T07:34:14.020283+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:34:14.020283+0000","last_undegraded":"2024-09-10T07:34:14.020283+0000","last_fullsized":"2024-09-10T07:34:14.020283+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:39.824828+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:34:39.824828+0000","last_peered":"2024-09-10T07:34:39.824828+0000","last_clean":"2024-09-10T07:34:39.824828+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:34:39.824828+0000","last_undegraded":"2024-09-10T07:34:39.824828+0000","last_fullsized":"2024-09-10T07:34:39.824828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156247+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:34:14.156247+0000","last_peered":"2024-09-10T07:34:14.156247+0000","last_clean":"2024-09-10T07:34:14.156247+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:34:14.156247+0000","last_undegraded":"2024-09-10T07:34:14.156247+0000","last_fullsized":"2024-09-10T07:34:14.156247+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.155684+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:34:14.155684+0000","last_peered":"2024-09-10T07:34:14.155684+0000","last_clean":"2024-09-10T07:34:14.155684+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:34:14.155684+0000","last_undegraded":"2024-09-10T07:34:14.155684+0000","last_fullsized":"2024-09-10T07:34:14.155684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:24.077410+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:34:24.077410+0000","last_peered":"2024-09-10T07:34:24.077410+0000","last_clean":"2024-09-10T07:34:24.077410+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:34:24.077410+0000","last_undegraded":"2024-09-10T07:34:24.077410+0000","last_fullsized":"2024-09-10T07:34:24.077410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.157032+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:34:14.157032+0000","last_peered":"2024-09-10T07:34:14.157032+0000","last_clean":"2024-09-10T07:34:14.157032+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:34:14.157032+0000","last_undegraded":"2024-09-10T07:34:14.157032+0000","last_fullsized":"2024-09-10T07:34:14.157032+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011946+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:34:14.011946+0000","last_peered":"2024-09-10T07:34:14.011946+0000","last_clean":"2024-09-10T07:34:14.011946+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:34:14.011946+0000","last_undegraded":"2024-09-10T07:34:14.011946+0000","last_fullsized":"2024-09-10T07:34:14.011946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.813192+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:34:25.813192+0000","last_peered":"2024-09-10T07:34:25.813192+0000","last_clean":"2024-09-10T07:34:25.813192+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:34:25.813192+0000","last_undegraded":"2024-09-10T07:34:25.813192+0000","last_fullsized":"2024-09-10T07:34:25.813192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011724+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:34:14.011724+0000","last_peered":"2024-09-10T07:34:14.011724+0000","last_clean":"2024-09-10T07:34:14.011724+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:34:14.011724+0000","last_undegraded":"2024-09-10T07:34:14.011724+0000","last_fullsized":"2024-09-10T07:34:14.011724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012041+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:34:14.012041+0000","last_peered":"2024-09-10T07:34:14.012041+0000","last_clean":"2024-09-10T07:34:14.012041+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:34:14.012041+0000","last_undegraded":"2024-09-10T07:34:14.012041+0000","last_fullsized":"2024-09-10T07:34:14.012041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012299+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:34:14.012299+0000","last_peered":"2024-09-10T07:34:14.012299+0000","last_clean":"2024-09-10T07:34:14.012299+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:34:14.012299+0000","last_undegraded":"2024-09-10T07:34:14.012299+0000","last_fullsized":"2024-09-10T07:34:14.012299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:38.825527+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:34:38.825527+0000","last_peered":"2024-09-10T07:34:38.825527+0000","last_clean":"2024-09-10T07:34:38.825527+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:34:38.825527+0000","last_undegraded":"2024-09-10T07:34:38.825527+0000","last_fullsized":"2024-09-10T07:34:38.825527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106232+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:34:14.106232+0000","last_peered":"2024-09-10T07:34:14.106232+0000","last_clean":"2024-09-10T07:34:14.106232+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:34:14.106232+0000","last_undegraded":"2024-09-10T07:34:14.106232+0000","last_fullsized":"2024-09-10T07:34:14.106232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011941+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:34:14.011941+0000","last_peered":"2024-09-10T07:34:14.011941+0000","last_clean":"2024-09-10T07:34:14.011941+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:34:14.011941+0000","last_undegraded":"2024-09-10T07:34:14.011941+0000","last_fullsized":"2024-09-10T07:34:14.011941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.801121+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:34:25.801121+0000","last_peered":"2024-09-10T07:34:25.801121+0000","last_clean":"2024-09-10T07:34:25.801121+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:34:25.801121+0000","last_undegraded":"2024-09-10T07:34:25.801121+0000","last_fullsized":"2024-09-10T07:34:25.801121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.108108+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:34:14.108108+0000","last_peered":"2024-09-10T07:34:14.108108+0000","last_clean":"2024-09-10T07:34:14.108108+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:34:14.108108+0000","last_undegraded":"2024-09-10T07:34:14.108108+0000","last_fullsized":"2024-09-10T07:34:14.108108+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012149+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:34:14.012149+0000","last_peered":"2024-09-10T07:34:14.012149+0000","last_clean":"2024-09-10T07:34:14.012149+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:34:14.012149+0000","last_undegraded":"2024-09-10T07:34:14.012149+0000","last_fullsized":"2024-09-10T07:34:14.012149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.815769+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:34:25.815769+0000","last_peered":"2024-09-10T07:34:25.815769+0000","last_clean":"2024-09-10T07:34:25.815769+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:34:25.815769+0000","last_undegraded":"2024-09-10T07:34:25.815769+0000","last_fullsized":"2024-09-10T07:34:25.815769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017144+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:34:14.017144+0000","last_peered":"2024-09-10T07:34:14.017144+0000","last_clean":"2024-09-10T07:34:14.017144+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:34:14.017144+0000","last_undegraded":"2024-09-10T07:34:14.017144+0000","last_fullsized":"2024-09-10T07:34:14.017144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014600+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:34:14.014600+0000","last_peered":"2024-09-10T07:34:14.014600+0000","last_clean":"2024-09-10T07:34:14.014600+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:34:14.014600+0000","last_undegraded":"2024-09-10T07:34:14.014600+0000","last_fullsized":"2024-09-10T07:34:14.014600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.801200+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:34:25.801200+0000","last_peered":"2024-09-10T07:34:25.801200+0000","last_clean":"2024-09-10T07:34:25.801200+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:34:25.801200+0000","last_undegraded":"2024-09-10T07:34:25.801200+0000","last_fullsized":"2024-09-10T07:34:25.801200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016342+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:34:14.016342+0000","last_peered":"2024-09-10T07:34:14.016342+0000","last_clean":"2024-09-10T07:34:14.016342+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:34:14.016342+0000","last_undegraded":"2024-09-10T07:34:14.016342+0000","last_fullsized":"2024-09-10T07:34:14.016342+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.021124+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:34:14.021124+0000","last_peered":"2024-09-10T07:34:14.021124+0000","last_clean":"2024-09-10T07:34:14.021124+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:34:14.021124+0000","last_undegraded":"2024-09-10T07:34:14.021124+0000","last_fullsized":"2024-09-10T07:34:14.021124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.801135+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:34:25.801135+0000","last_peered":"2024-09-10T07:34:25.801135+0000","last_clean":"2024-09-10T07:34:25.801135+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:34:25.801135+0000","last_undegraded":"2024-09-10T07:34:25.801135+0000","last_fullsized":"2024-09-10T07:34:25.801135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156305+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:34:14.156305+0000","last_peered":"2024-09-10T07:34:14.156305+0000","last_clean":"2024-09-10T07:34:14.156305+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:34:14.156305+0000","last_undegraded":"2024-09-10T07:34:14.156305+0000","last_fullsized":"2024-09-10T07:34:14.156305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011502+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:34:14.011502+0000","last_peered":"2024-09-10T07:34:14.011502+0000","last_clean":"2024-09-10T07:34:14.011502+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:34:14.011502+0000","last_undegraded":"2024-09-10T07:34:14.011502+0000","last_fullsized":"2024-09-10T07:34:14.011502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:38.823733+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:34:38.823733+0000","last_peered":"2024-09-10T07:34:38.823733+0000","last_clean":"2024-09-10T07:34:38.823733+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:34:38.823733+0000","last_undegraded":"2024-09-10T07:34:38.823733+0000","last_fullsized":"2024-09-10T07:34:38.823733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018813+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:34:14.018813+0000","last_peered":"2024-09-10T07:34:14.018813+0000","last_clean":"2024-09-10T07:34:14.018813+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:34:14.018813+0000","last_undegraded":"2024-09-10T07:34:14.018813+0000","last_fullsized":"2024-09-10T07:34:14.018813+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011844+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:34:14.011844+0000","last_peered":"2024-09-10T07:34:14.011844+0000","last_clean":"2024-09-10T07:34:14.011844+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:34:14.011844+0000","last_undegraded":"2024-09-10T07:34:14.011844+0000","last_fullsized":"2024-09-10T07:34:14.011844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.813467+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:34:25.813467+0000","last_peered":"2024-09-10T07:34:25.813467+0000","last_clean":"2024-09-10T07:34:25.813467+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:34:25.813467+0000","last_undegraded":"2024-09-10T07:34:25.813467+0000","last_fullsized":"2024-09-10T07:34:25.813467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156814+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:34:14.156814+0000","last_peered":"2024-09-10T07:34:14.156814+0000","last_clean":"2024-09-10T07:34:14.156814+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:34:14.156814+0000","last_undegraded":"2024-09-10T07:34:14.156814+0000","last_fullsized":"2024-09-10T07:34:14.156814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.155671+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:34:14.155671+0000","last_peered":"2024-09-10T07:34:14.155671+0000","last_clean":"2024-09-10T07:34:14.155671+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:34:14.155671+0000","last_undegraded":"2024-09-10T07:34:14.155671+0000","last_fullsized":"2024-09-10T07:34:14.155671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.816123+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:34:25.816123+0000","last_peered":"2024-09-10T07:34:25.816123+0000","last_clean":"2024-09-10T07:34:25.816123+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:34:25.816123+0000","last_undegraded":"2024-09-10T07:34:25.816123+0000","last_fullsized":"2024-09-10T07:34:25.816123+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.020853+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:34:14.020853+0000","last_peered":"2024-09-10T07:34:14.020853+0000","last_clean":"2024-09-10T07:34:14.020853+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:34:14.020853+0000","last_undegraded":"2024-09-10T07:34:14.020853+0000","last_fullsized":"2024-09-10T07:34:14.020853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012277+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:34:14.012277+0000","last_peered":"2024-09-10T07:34:14.012277+0000","last_clean":"2024-09-10T07:34:14.012277+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:34:14.012277+0000","last_undegraded":"2024-09-10T07:34:14.012277+0000","last_fullsized":"2024-09-10T07:34:14.012277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'31","reported_seq":115,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:43.831093+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:34:43.831093+0000","last_peered":"2024-09-10T07:34:43.831093+0000","last_clean":"2024-09-10T07:34:43.831093+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:34:43.831093+0000","last_undegraded":"2024-09-10T07:34:43.831093+0000","last_fullsized":"2024-09-10T07:34:43.831093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51899,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015714+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:34:14.015714+0000","last_peered":"2024-09-10T07:34:14.015714+0000","last_clean":"2024-09-10T07:34:14.015714+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:34:14.015714+0000","last_undegraded":"2024-09-10T07:34:14.015714+0000","last_fullsized":"2024-09-10T07:34:14.015714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106566+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:34:14.106566+0000","last_peered":"2024-09-10T07:34:14.106566+0000","last_clean":"2024-09-10T07:34:14.106566+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:34:14.106566+0000","last_undegraded":"2024-09-10T07:34:14.106566+0000","last_fullsized":"2024-09-10T07:34:14.106566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.085589+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:34:25.085589+0000","last_peered":"2024-09-10T07:34:25.085589+0000","last_clean":"2024-09-10T07:34:25.085589+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:34:25.085589+0000","last_undegraded":"2024-09-10T07:34:25.085589+0000","last_fullsized":"2024-09-10T07:34:25.085589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016800+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:34:14.016800+0000","last_peered":"2024-09-10T07:34:14.016800+0000","last_clean":"2024-09-10T07:34:14.016800+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:34:14.016800+0000","last_undegraded":"2024-09-10T07:34:14.016800+0000","last_fullsized":"2024-09-10T07:34:14.016800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019812+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:34:14.019812+0000","last_peered":"2024-09-10T07:34:14.019812+0000","last_clean":"2024-09-10T07:34:14.019812+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:34:14.019812+0000","last_undegraded":"2024-09-10T07:34:14.019812+0000","last_fullsized":"2024-09-10T07:34:14.019812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.813963+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:34:25.813963+0000","last_peered":"2024-09-10T07:34:25.813963+0000","last_clean":"2024-09-10T07:34:25.813963+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:34:25.813963+0000","last_undegraded":"2024-09-10T07:34:25.813963+0000","last_fullsized":"2024-09-10T07:34:25.813963+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015693+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:34:14.015693+0000","last_peered":"2024-09-10T07:34:14.015693+0000","last_clean":"2024-09-10T07:34:14.015693+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:34:14.015693+0000","last_undegraded":"2024-09-10T07:34:14.015693+0000","last_fullsized":"2024-09-10T07:34:14.015693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106737+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:34:14.106737+0000","last_peered":"2024-09-10T07:34:14.106737+0000","last_clean":"2024-09-10T07:34:14.106737+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:34:14.106737+0000","last_undegraded":"2024-09-10T07:34:14.106737+0000","last_fullsized":"2024-09-10T07:34:14.106737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:43.831124+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:34:43.831124+0000","last_peered":"2024-09-10T07:34:43.831124+0000","last_clean":"2024-09-10T07:34:43.831124+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:34:43.831124+0000","last_undegraded":"2024-09-10T07:34:43.831124+0000","last_fullsized":"2024-09-10T07:34:43.831124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011599+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:34:14.011599+0000","last_peered":"2024-09-10T07:34:14.011599+0000","last_clean":"2024-09-10T07:34:14.011599+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:34:14.011599+0000","last_undegraded":"2024-09-10T07:34:14.011599+0000","last_fullsized":"2024-09-10T07:34:14.011599+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019112+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:34:14.019112+0000","last_peered":"2024-09-10T07:34:14.019112+0000","last_clean":"2024-09-10T07:34:14.019112+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:34:14.019112+0000","last_undegraded":"2024-09-10T07:34:14.019112+0000","last_fullsized":"2024-09-10T07:34:14.019112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015976+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:34:14.015976+0000","last_peered":"2024-09-10T07:34:14.015976+0000","last_clean":"2024-09-10T07:34:14.015976+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:34:14.015976+0000","last_undegraded":"2024-09-10T07:34:14.015976+0000","last_fullsized":"2024-09-10T07:34:14.015976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.808265+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:34:25.808265+0000","last_peered":"2024-09-10T07:34:25.808265+0000","last_clean":"2024-09-10T07:34:25.808265+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:34:25.808265+0000","last_undegraded":"2024-09-10T07:34:25.808265+0000","last_fullsized":"2024-09-10T07:34:25.808265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014391+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:34:14.014391+0000","last_peered":"2024-09-10T07:34:14.014391+0000","last_clean":"2024-09-10T07:34:14.014391+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:34:14.014391+0000","last_undegraded":"2024-09-10T07:34:14.014391+0000","last_fullsized":"2024-09-10T07:34:14.014391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011618+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:34:14.011618+0000","last_peered":"2024-09-10T07:34:14.011618+0000","last_clean":"2024-09-10T07:34:14.011618+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:34:14.011618+0000","last_undegraded":"2024-09-10T07:34:14.011618+0000","last_fullsized":"2024-09-10T07:34:14.011618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:24.079967+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:34:24.079967+0000","last_peered":"2024-09-10T07:34:24.079967+0000","last_clean":"2024-09-10T07:34:24.079967+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:34:24.079967+0000","last_undegraded":"2024-09-10T07:34:24.079967+0000","last_fullsized":"2024-09-10T07:34:24.079967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.085644+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:34:25.085644+0000","last_peered":"2024-09-10T07:34:25.085644+0000","last_clean":"2024-09-10T07:34:25.085644+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:34:25.085644+0000","last_undegraded":"2024-09-10T07:34:25.085644+0000","last_fullsized":"2024-09-10T07:34:25.085644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.020936+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:34:14.020936+0000","last_peered":"2024-09-10T07:34:14.020936+0000","last_clean":"2024-09-10T07:34:14.020936+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:34:14.020936+0000","last_undegraded":"2024-09-10T07:34:14.020936+0000","last_fullsized":"2024-09-10T07:34:14.020936+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018080+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:34:14.018080+0000","last_peered":"2024-09-10T07:34:14.018080+0000","last_clean":"2024-09-10T07:34:14.018080+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:34:14.018080+0000","last_undegraded":"2024-09-10T07:34:14.018080+0000","last_fullsized":"2024-09-10T07:34:14.018080+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:38.824700+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:34:38.824700+0000","last_peered":"2024-09-10T07:34:38.824700+0000","last_clean":"2024-09-10T07:34:38.824700+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:34:38.824700+0000","last_undegraded":"2024-09-10T07:34:38.824700+0000","last_fullsized":"2024-09-10T07:34:38.824700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012030+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:34:14.012030+0000","last_peered":"2024-09-10T07:34:14.012030+0000","last_clean":"2024-09-10T07:34:14.012030+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:34:14.012030+0000","last_undegraded":"2024-09-10T07:34:14.012030+0000","last_fullsized":"2024-09-10T07:34:14.012030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018373+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:34:14.018373+0000","last_peered":"2024-09-10T07:34:14.018373+0000","last_clean":"2024-09-10T07:34:14.018373+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:34:14.018373+0000","last_undegraded":"2024-09-10T07:34:14.018373+0000","last_fullsized":"2024-09-10T07:34:14.018373+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014119+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:34:14.014119+0000","last_peered":"2024-09-10T07:34:14.014119+0000","last_clean":"2024-09-10T07:34:14.014119+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:34:14.014119+0000","last_undegraded":"2024-09-10T07:34:14.014119+0000","last_fullsized":"2024-09-10T07:34:14.014119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018035+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:34:14.018035+0000","last_peered":"2024-09-10T07:34:14.018035+0000","last_clean":"2024-09-10T07:34:14.018035+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:34:14.018035+0000","last_undegraded":"2024-09-10T07:34:14.018035+0000","last_fullsized":"2024-09-10T07:34:14.018035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017829+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:34:14.017829+0000","last_peered":"2024-09-10T07:34:14.017829+0000","last_clean":"2024-09-10T07:34:14.017829+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:34:14.017829+0000","last_undegraded":"2024-09-10T07:34:14.017829+0000","last_fullsized":"2024-09-10T07:34:14.017829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018508+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:34:14.018508+0000","last_peered":"2024-09-10T07:34:14.018508+0000","last_clean":"2024-09-10T07:34:14.018508+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:34:14.018508+0000","last_undegraded":"2024-09-10T07:34:14.018508+0000","last_fullsized":"2024-09-10T07:34:14.018508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156358+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:34:14.156358+0000","last_peered":"2024-09-10T07:34:14.156358+0000","last_clean":"2024-09-10T07:34:14.156358+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:34:14.156358+0000","last_undegraded":"2024-09-10T07:34:14.156358+0000","last_fullsized":"2024-09-10T07:34:14.156358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014595+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:34:14.014595+0000","last_peered":"2024-09-10T07:34:14.014595+0000","last_clean":"2024-09-10T07:34:14.014595+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:34:14.014595+0000","last_undegraded":"2024-09-10T07:34:14.014595+0000","last_fullsized":"2024-09-10T07:34:14.014595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018368+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:34:14.018368+0000","last_peered":"2024-09-10T07:34:14.018368+0000","last_clean":"2024-09-10T07:34:14.018368+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:34:14.018368+0000","last_undegraded":"2024-09-10T07:34:14.018368+0000","last_fullsized":"2024-09-10T07:34:14.018368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012320+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:34:14.012320+0000","last_peered":"2024-09-10T07:34:14.012320+0000","last_clean":"2024-09-10T07:34:14.012320+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:34:14.012320+0000","last_undegraded":"2024-09-10T07:34:14.012320+0000","last_fullsized":"2024-09-10T07:34:14.012320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017206+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:34:14.017206+0000","last_peered":"2024-09-10T07:34:14.017206+0000","last_clean":"2024-09-10T07:34:14.017206+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:34:14.017206+0000","last_undegraded":"2024-09-10T07:34:14.017206+0000","last_fullsized":"2024-09-10T07:34:14.017206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011891+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:34:14.011891+0000","last_peered":"2024-09-10T07:34:14.011891+0000","last_clean":"2024-09-10T07:34:14.011891+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:34:14.011891+0000","last_undegraded":"2024-09-10T07:34:14.011891+0000","last_fullsized":"2024-09-10T07:34:14.011891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106467+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:34:14.106467+0000","last_peered":"2024-09-10T07:34:14.106467+0000","last_clean":"2024-09-10T07:34:14.106467+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:34:14.106467+0000","last_undegraded":"2024-09-10T07:34:14.106467+0000","last_fullsized":"2024-09-10T07:34:14.106467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156022+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:34:14.156022+0000","last_peered":"2024-09-10T07:34:14.156022+0000","last_clean":"2024-09-10T07:34:14.156022+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:34:14.156022+0000","last_undegraded":"2024-09-10T07:34:14.156022+0000","last_fullsized":"2024-09-10T07:34:14.156022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.021473+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:34:14.021473+0000","last_peered":"2024-09-10T07:34:14.021473+0000","last_clean":"2024-09-10T07:34:14.021473+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:34:14.021473+0000","last_undegraded":"2024-09-10T07:34:14.021473+0000","last_fullsized":"2024-09-10T07:34:14.021473+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018542+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:34:14.018542+0000","last_peered":"2024-09-10T07:34:14.018542+0000","last_clean":"2024-09-10T07:34:14.018542+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:34:14.018542+0000","last_undegraded":"2024-09-10T07:34:14.018542+0000","last_fullsized":"2024-09-10T07:34:14.018542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019269+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:34:14.019269+0000","last_peered":"2024-09-10T07:34:14.019269+0000","last_clean":"2024-09-10T07:34:14.019269+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:34:14.019269+0000","last_undegraded":"2024-09-10T07:34:14.019269+0000","last_fullsized":"2024-09-10T07:34:14.019269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014760+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:34:14.014760+0000","last_peered":"2024-09-10T07:34:14.014760+0000","last_clean":"2024-09-10T07:34:14.014760+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:34:14.014760+0000","last_undegraded":"2024-09-10T07:34:14.014760+0000","last_fullsized":"2024-09-10T07:34:14.014760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106126+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:34:14.106126+0000","last_peered":"2024-09-10T07:34:14.106126+0000","last_clean":"2024-09-10T07:34:14.106126+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:34:14.106126+0000","last_undegraded":"2024-09-10T07:34:14.106126+0000","last_fullsized":"2024-09-10T07:34:14.106126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017632+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:34:14.017632+0000","last_peered":"2024-09-10T07:34:14.017632+0000","last_clean":"2024-09-10T07:34:14.017632+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:34:14.017632+0000","last_undegraded":"2024-09-10T07:34:14.017632+0000","last_fullsized":"2024-09-10T07:34:14.017632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018240+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:34:14.018240+0000","last_peered":"2024-09-10T07:34:14.018240+0000","last_clean":"2024-09-10T07:34:14.018240+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:34:14.018240+0000","last_undegraded":"2024-09-10T07:34:14.018240+0000","last_fullsized":"2024-09-10T07:34:14.018240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.155609+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:34:14.155609+0000","last_peered":"2024-09-10T07:34:14.155609+0000","last_clean":"2024-09-10T07:34:14.155609+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:34:14.155609+0000","last_undegraded":"2024-09-10T07:34:14.155609+0000","last_fullsized":"2024-09-10T07:34:14.155609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016594+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:34:14.016594+0000","last_peered":"2024-09-10T07:34:14.016594+0000","last_clean":"2024-09-10T07:34:14.016594+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:34:14.016594+0000","last_undegraded":"2024-09-10T07:34:14.016594+0000","last_fullsized":"2024-09-10T07:34:14.016594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012167+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:34:14.012167+0000","last_peered":"2024-09-10T07:34:14.012167+0000","last_clean":"2024-09-10T07:34:14.012167+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:34:14.012167+0000","last_undegraded":"2024-09-10T07:34:14.012167+0000","last_fullsized":"2024-09-10T07:34:14.012167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014446+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:34:14.014446+0000","last_peered":"2024-09-10T07:34:14.014446+0000","last_clean":"2024-09-10T07:34:14.014446+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:34:14.014446+0000","last_undegraded":"2024-09-10T07:34:14.014446+0000","last_fullsized":"2024-09-10T07:34:14.014446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106900+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:34:14.106900+0000","last_peered":"2024-09-10T07:34:14.106900+0000","last_clean":"2024-09-10T07:34:14.106900+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:34:14.106900+0000","last_undegraded":"2024-09-10T07:34:14.106900+0000","last_fullsized":"2024-09-10T07:34:14.106900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011591+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:34:14.011591+0000","last_peered":"2024-09-10T07:34:14.011591+0000","last_clean":"2024-09-10T07:34:14.011591+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:34:14.011591+0000","last_undegraded":"2024-09-10T07:34:14.011591+0000","last_fullsized":"2024-09-10T07:34:14.011591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011617+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:34:14.011617+0000","last_peered":"2024-09-10T07:34:14.011617+0000","last_clean":"2024-09-10T07:34:14.011617+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:34:14.011617+0000","last_undegraded":"2024-09-10T07:34:14.011617+0000","last_fullsized":"2024-09-10T07:34:14.011617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.013777+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:34:14.013777+0000","last_peered":"2024-09-10T07:34:14.013777+0000","last_clean":"2024-09-10T07:34:14.013777+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:34:14.013777+0000","last_undegraded":"2024-09-10T07:34:14.013777+0000","last_fullsized":"2024-09-10T07:34:14.013777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012016+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:34:14.012016+0000","last_peered":"2024-09-10T07:34:14.012016+0000","last_clean":"2024-09-10T07:34:14.012016+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:34:14.012016+0000","last_undegraded":"2024-09-10T07:34:14.012016+0000","last_fullsized":"2024-09-10T07:34:14.012016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016459+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:34:14.016459+0000","last_peered":"2024-09-10T07:34:14.016459+0000","last_clean":"2024-09-10T07:34:14.016459+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:34:14.016459+0000","last_undegraded":"2024-09-10T07:34:14.016459+0000","last_fullsized":"2024-09-10T07:34:14.016459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018534+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:34:14.018534+0000","last_peered":"2024-09-10T07:34:14.018534+0000","last_clean":"2024-09-10T07:34:14.018534+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:34:14.018534+0000","last_undegraded":"2024-09-10T07:34:14.018534+0000","last_fullsized":"2024-09-10T07:34:14.018534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019210+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:34:14.019210+0000","last_peered":"2024-09-10T07:34:14.019210+0000","last_clean":"2024-09-10T07:34:14.019210+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:34:14.019210+0000","last_undegraded":"2024-09-10T07:34:14.019210+0000","last_fullsized":"2024-09-10T07:34:14.019210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014849+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:34:14.014849+0000","last_peered":"2024-09-10T07:34:14.014849+0000","last_clean":"2024-09-10T07:34:14.014849+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:34:14.014849+0000","last_undegraded":"2024-09-10T07:34:14.014849+0000","last_fullsized":"2024-09-10T07:34:14.014849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59107,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":208,"num_read_kb":176,"num_write":242,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":168767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":542421,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710753,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1196032,"data_stored":1181068,"data_compressed":62990,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874276,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":546379,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":542661,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1139553,"data_compressed":62990,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528373,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":538180,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790012,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1134742,"data_compressed":62990,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":541861,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":537495,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":582707,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":582555,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":539243,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:34:46.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T07:34:47.328 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:47.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:47 smithi046 ceph-mon[30705]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:34:47.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:47 smithi181 ceph-mon[35397]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:34:47.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:47 smithi063 ceph-mon[35020]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:34:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 -- 172.21.15.46:0/2423931376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 msgr2=0x7fa3c01003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 --2- 172.21.15.46:0/2423931376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 0x7fa3c01003d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009900 tx=0x7fa3b002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:34:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 -- 172.21.15.46:0/2423931376 shutdown_connections 2024-09-10T07:34:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 --2- 172.21.15.46:0/2423931376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3c0105120 0x7fa3c010cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 --2- 172.21.15.46:0/2423931376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3c01009a0 0x7fa3c01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 --2- 172.21.15.46:0/2423931376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 0x7fa3c01003d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 -- 172.21.15.46:0/2423931376 >> 172.21.15.46:0/2423931376 conn(0x7fa3c00fba90 msgr2=0x7fa3c00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 -- 172.21.15.46:0/2423931376 shutdown_connections 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.926+0000 7fa3c6800640 1 -- 172.21.15.46:0/2423931376 wait complete. 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.927+0000 7fa3c6800640 1 Processor -- start 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.927+0000 7fa3c6800640 1 -- start start 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.927+0000 7fa3c6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 0x7fa3c019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3c6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3c01009a0 0x7fa3c019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3c6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3c0105120 0x7fa3c019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3c6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0111f40 con 0x7fa3c00ffff0 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3c6800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0111dc0 con 0x7fa3c01009a0 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3c6800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c01120c0 con 0x7fa3c0105120 2024-09-10T07:34:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 0x7fa3c019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3c01009a0 0x7fa3c019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 0x7fa3c019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57844/0 (socket says 172.21.15.46:57844) 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bf400640 1 -- 172.21.15.46:0/159588669 learned_addr learned my addr 172.21.15.46:0/159588669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bfe00640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3c0105120 0x7fa3c019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bea00640 1 -- 172.21.15.46:0/159588669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3c0105120 msgr2=0x7fa3c019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bea00640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3c0105120 0x7fa3c019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bea00640 1 -- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 msgr2=0x7fa3c019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bea00640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 0x7fa3c019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.928+0000 7fa3bea00640 1 -- 172.21.15.46:0/159588669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c019f620 con 0x7fa3c01009a0 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.929+0000 7fa3bea00640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3c01009a0 0x7fa3c019abb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00e9f0 tx=0x7fa3ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.929+0000 7fa3b7e00640 1 -- 172.21.15.46:0/159588669 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00ccc0 con 0x7fa3c01009a0 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.929+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c019f910 con 0x7fa3c01009a0 2024-09-10T07:34:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.929+0000 7fa3b7e00640 1 -- 172.21.15.46:0/159588669 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3ac010910 con 0x7fa3c01009a0 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.929+0000 7fa3b7e00640 1 -- 172.21.15.46:0/159588669 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac018960 con 0x7fa3c01009a0 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.929+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01a71f0 con 0x7fa3c01009a0 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.930+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c01003d0 con 0x7fa3c01009a0 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.933+0000 7fa3b7e00640 1 -- 172.21.15.46:0/159588669 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa3ac010ac0 con 0x7fa3c01009a0 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.934+0000 7fa3b7e00640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3a40a2c10 0x7fa3a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.934+0000 7fa3b7e00640 1 -- 172.21.15.46:0/159588669 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fa3ac0cfe90 con 0x7fa3c01009a0 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.934+0000 7fa3bf400640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3a40a2c10 0x7fa3a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.935+0000 7fa3bf400640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3a40a2c10 0x7fa3a40a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00096c0 tx=0x7fa3b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:47.936+0000 7fa3b7e00640 1 -- 172.21.15.46:0/159588669 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3ac09c8d0 con 0x7fa3c01009a0 2024-09-10T07:34:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.118+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3c010fb90 con 0x7fa3a40a2c10 2024-09-10T07:34:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.126+0000 7fa3b7e00640 1 -- 172.21.15.46:0/159588669 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340644 (secure 0 0 0) 0x7fa3c010fb90 con 0x7fa3a40a2c10 2024-09-10T07:34:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:34:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3a40a2c10 msgr2=0x7fa3a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3a40a2c10 0x7fa3a40a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00096c0 tx=0x7fa3b0005a20 comp rx=0 tx=0).stop 2024-09-10T07:34:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3c01009a0 msgr2=0x7fa3c019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3c01009a0 0x7fa3c019abb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00e9f0 tx=0x7fa3ac00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:34:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 shutdown_connections 2024-09-10T07:34:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3c0105120 0x7fa3c019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3c01009a0 0x7fa3c019abb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3a40a2c10 0x7fa3a40a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 --2- 172.21.15.46:0/159588669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c00ffff0 0x7fa3c019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 >> 172.21.15.46:0/159588669 conn(0x7fa3c00fba90 msgr2=0x7fa3c00fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 shutdown_connections 2024-09-10T07:34:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:48.131+0000 7fa3c6800640 1 -- 172.21.15.46:0/159588669 wait complete. 2024-09-10T07:34:48.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:48 smithi181 ceph-mon[35397]: pgmap v342: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-10T07:34:48.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:48 smithi063 ceph-mon[35020]: pgmap v342: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-10T07:34:48.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:48 smithi046 ceph-mon[30705]: pgmap v342: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-10T07:34:48.953 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-10T07:34:46.477109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649709,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":354,"num_read_kb":290,"num_write":425,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357984,"kb_used_data":12896,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583840,"statfs":{"total":1151940427776,"available":1151573852160,"internally_reserved":0,"allocated":13205504,"data_stored":8425973,"data_compressed":426912,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27946,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":85,"num_write":24,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003521"},"pg_stats":[{"pgid":"2.1f","version":"95'13","reported_seq":70,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:41.838832+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:34:41.838832+0000","last_peered":"2024-09-10T07:34:41.838832+0000","last_clean":"2024-09-10T07:34:41.838832+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:34:41.838832+0000","last_undegraded":"2024-09-10T07:34:41.838832+0000","last_fullsized":"2024-09-10T07:34:41.838832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012250+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:34:14.012250+0000","last_peered":"2024-09-10T07:34:14.012250+0000","last_clean":"2024-09-10T07:34:14.012250+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:34:14.012250+0000","last_undegraded":"2024-09-10T07:34:14.012250+0000","last_fullsized":"2024-09-10T07:34:14.012250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017508+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:34:14.017508+0000","last_peered":"2024-09-10T07:34:14.017508+0000","last_clean":"2024-09-10T07:34:14.017508+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:34:14.017508+0000","last_undegraded":"2024-09-10T07:34:14.017508+0000","last_fullsized":"2024-09-10T07:34:14.017508+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012531+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:34:14.012531+0000","last_peered":"2024-09-10T07:34:14.012531+0000","last_clean":"2024-09-10T07:34:14.012531+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:34:14.012531+0000","last_undegraded":"2024-09-10T07:34:14.012531+0000","last_fullsized":"2024-09-10T07:34:14.012531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019829+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:34:14.019829+0000","last_peered":"2024-09-10T07:34:14.019829+0000","last_clean":"2024-09-10T07:34:14.019829+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:34:14.019829+0000","last_undegraded":"2024-09-10T07:34:14.019829+0000","last_fullsized":"2024-09-10T07:34:14.019829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015733+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:34:14.015733+0000","last_peered":"2024-09-10T07:34:14.015733+0000","last_clean":"2024-09-10T07:34:14.015733+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:34:14.015733+0000","last_undegraded":"2024-09-10T07:34:14.015733+0000","last_fullsized":"2024-09-10T07:34:14.015733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.815290+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:34:25.815290+0000","last_peered":"2024-09-10T07:34:25.815290+0000","last_clean":"2024-09-10T07:34:25.815290+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:34:25.815290+0000","last_undegraded":"2024-09-10T07:34:25.815290+0000","last_fullsized":"2024-09-10T07:34:25.815290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106528+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:34:14.106528+0000","last_peered":"2024-09-10T07:34:14.106528+0000","last_clean":"2024-09-10T07:34:14.106528+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:34:14.106528+0000","last_undegraded":"2024-09-10T07:34:14.106528+0000","last_fullsized":"2024-09-10T07:34:14.106528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016008+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:34:14.016008+0000","last_peered":"2024-09-10T07:34:14.016008+0000","last_clean":"2024-09-10T07:34:14.016008+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:34:14.016008+0000","last_undegraded":"2024-09-10T07:34:14.016008+0000","last_fullsized":"2024-09-10T07:34:14.016008+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.814807+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:34:25.814807+0000","last_peered":"2024-09-10T07:34:25.814807+0000","last_clean":"2024-09-10T07:34:25.814807+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:34:25.814807+0000","last_undegraded":"2024-09-10T07:34:25.814807+0000","last_fullsized":"2024-09-10T07:34:25.814807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156901+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:34:14.156901+0000","last_peered":"2024-09-10T07:34:14.156901+0000","last_clean":"2024-09-10T07:34:14.156901+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:34:14.156901+0000","last_undegraded":"2024-09-10T07:34:14.156901+0000","last_fullsized":"2024-09-10T07:34:14.156901+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015987+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:34:14.015987+0000","last_peered":"2024-09-10T07:34:14.015987+0000","last_clean":"2024-09-10T07:34:14.015987+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:34:14.015987+0000","last_undegraded":"2024-09-10T07:34:14.015987+0000","last_fullsized":"2024-09-10T07:34:14.015987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011724+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:34:14.011724+0000","last_peered":"2024-09-10T07:34:14.011724+0000","last_clean":"2024-09-10T07:34:14.011724+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:34:14.011724+0000","last_undegraded":"2024-09-10T07:34:14.011724+0000","last_fullsized":"2024-09-10T07:34:14.011724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:23.066828+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:34:23.066828+0000","last_peered":"2024-09-10T07:34:23.066828+0000","last_clean":"2024-09-10T07:34:23.066828+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:34:23.066828+0000","last_undegraded":"2024-09-10T07:34:23.066828+0000","last_fullsized":"2024-09-10T07:34:23.066828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016148+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:34:14.016148+0000","last_peered":"2024-09-10T07:34:14.016148+0000","last_clean":"2024-09-10T07:34:14.016148+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:34:14.016148+0000","last_undegraded":"2024-09-10T07:34:14.016148+0000","last_fullsized":"2024-09-10T07:34:14.016148+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012240+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:34:14.012240+0000","last_peered":"2024-09-10T07:34:14.012240+0000","last_clean":"2024-09-10T07:34:14.012240+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:34:14.012240+0000","last_undegraded":"2024-09-10T07:34:14.012240+0000","last_fullsized":"2024-09-10T07:34:14.012240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.085370+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:34:25.085370+0000","last_peered":"2024-09-10T07:34:25.085370+0000","last_clean":"2024-09-10T07:34:25.085370+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:34:25.085370+0000","last_undegraded":"2024-09-10T07:34:25.085370+0000","last_fullsized":"2024-09-10T07:34:25.085370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016352+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:34:14.016352+0000","last_peered":"2024-09-10T07:34:14.016352+0000","last_clean":"2024-09-10T07:34:14.016352+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:34:14.016352+0000","last_undegraded":"2024-09-10T07:34:14.016352+0000","last_fullsized":"2024-09-10T07:34:14.016352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014187+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:34:14.014187+0000","last_peered":"2024-09-10T07:34:14.014187+0000","last_clean":"2024-09-10T07:34:14.014187+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:34:14.014187+0000","last_undegraded":"2024-09-10T07:34:14.014187+0000","last_fullsized":"2024-09-10T07:34:14.014187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.816443+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:34:25.816443+0000","last_peered":"2024-09-10T07:34:25.816443+0000","last_clean":"2024-09-10T07:34:25.816443+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:34:25.816443+0000","last_undegraded":"2024-09-10T07:34:25.816443+0000","last_fullsized":"2024-09-10T07:34:25.816443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018684+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:34:14.018684+0000","last_peered":"2024-09-10T07:34:14.018684+0000","last_clean":"2024-09-10T07:34:14.018684+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:34:14.018684+0000","last_undegraded":"2024-09-10T07:34:14.018684+0000","last_fullsized":"2024-09-10T07:34:14.018684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011918+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:34:14.011918+0000","last_peered":"2024-09-10T07:34:14.011918+0000","last_clean":"2024-09-10T07:34:14.011918+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:34:14.011918+0000","last_undegraded":"2024-09-10T07:34:14.011918+0000","last_fullsized":"2024-09-10T07:34:14.011918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:23.066444+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:34:23.066444+0000","last_peered":"2024-09-10T07:34:23.066444+0000","last_clean":"2024-09-10T07:34:23.066444+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:34:23.066444+0000","last_undegraded":"2024-09-10T07:34:23.066444+0000","last_fullsized":"2024-09-10T07:34:23.066444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.022602+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:34:14.022602+0000","last_peered":"2024-09-10T07:34:14.022602+0000","last_clean":"2024-09-10T07:34:14.022602+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:34:14.022602+0000","last_undegraded":"2024-09-10T07:34:14.022602+0000","last_fullsized":"2024-09-10T07:34:14.022602+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019291+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:34:14.019291+0000","last_peered":"2024-09-10T07:34:14.019291+0000","last_clean":"2024-09-10T07:34:14.019291+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:34:14.019291+0000","last_undegraded":"2024-09-10T07:34:14.019291+0000","last_fullsized":"2024-09-10T07:34:14.019291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.812957+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:34:25.812957+0000","last_peered":"2024-09-10T07:34:25.812957+0000","last_clean":"2024-09-10T07:34:25.812957+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:34:25.812957+0000","last_undegraded":"2024-09-10T07:34:25.812957+0000","last_fullsized":"2024-09-10T07:34:25.812957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017385+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:34:14.017385+0000","last_peered":"2024-09-10T07:34:14.017385+0000","last_clean":"2024-09-10T07:34:14.017385+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:34:14.017385+0000","last_undegraded":"2024-09-10T07:34:14.017385+0000","last_fullsized":"2024-09-10T07:34:14.017385+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012416+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:34:14.012416+0000","last_peered":"2024-09-10T07:34:14.012416+0000","last_clean":"2024-09-10T07:34:14.012416+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:34:14.012416+0000","last_undegraded":"2024-09-10T07:34:14.012416+0000","last_fullsized":"2024-09-10T07:34:14.012416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'14","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:40.893891+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:34:40.893891+0000","last_peered":"2024-09-10T07:34:40.893891+0000","last_clean":"2024-09-10T07:34:40.893891+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:34:40.893891+0000","last_undegraded":"2024-09-10T07:34:40.893891+0000","last_fullsized":"2024-09-10T07:34:40.893891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011477+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:34:14.011477+0000","last_peered":"2024-09-10T07:34:14.011477+0000","last_clean":"2024-09-10T07:34:14.011477+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:34:14.011477+0000","last_undegraded":"2024-09-10T07:34:14.011477+0000","last_fullsized":"2024-09-10T07:34:14.011477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156259+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:34:14.156259+0000","last_peered":"2024-09-10T07:34:14.156259+0000","last_clean":"2024-09-10T07:34:14.156259+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:34:14.156259+0000","last_undegraded":"2024-09-10T07:34:14.156259+0000","last_fullsized":"2024-09-10T07:34:14.156259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.801705+0000","last_change":"2024-09-10T07:33:31.999392+0000","last_active":"2024-09-10T07:34:25.801705+0000","last_peered":"2024-09-10T07:34:25.801705+0000","last_clean":"2024-09-10T07:34:25.801705+0000","last_became_active":"2024-09-10T07:33:31.998852+0000","last_became_peered":"2024-09-10T07:33:31.998852+0000","last_unstale":"2024-09-10T07:34:25.801705+0000","last_undegraded":"2024-09-10T07:34:25.801705+0000","last_fullsized":"2024-09-10T07:34:25.801705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:28.180972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015125+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:34:14.015125+0000","last_peered":"2024-09-10T07:34:14.015125+0000","last_clean":"2024-09-10T07:34:14.015125+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:34:14.015125+0000","last_undegraded":"2024-09-10T07:34:14.015125+0000","last_fullsized":"2024-09-10T07:34:14.015125+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.157371+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:34:14.157371+0000","last_peered":"2024-09-10T07:34:14.157371+0000","last_clean":"2024-09-10T07:34:14.157371+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:34:14.157371+0000","last_undegraded":"2024-09-10T07:34:14.157371+0000","last_fullsized":"2024-09-10T07:34:14.157371+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018224+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:34:14.018224+0000","last_peered":"2024-09-10T07:34:14.018224+0000","last_clean":"2024-09-10T07:34:14.018224+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:34:14.018224+0000","last_undegraded":"2024-09-10T07:34:14.018224+0000","last_fullsized":"2024-09-10T07:34:14.018224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011919+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:34:14.011919+0000","last_peered":"2024-09-10T07:34:14.011919+0000","last_clean":"2024-09-10T07:34:14.011919+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:34:14.011919+0000","last_undegraded":"2024-09-10T07:34:14.011919+0000","last_fullsized":"2024-09-10T07:34:14.011919+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.013923+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:34:14.013923+0000","last_peered":"2024-09-10T07:34:14.013923+0000","last_clean":"2024-09-10T07:34:14.013923+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:34:14.013923+0000","last_undegraded":"2024-09-10T07:34:14.013923+0000","last_fullsized":"2024-09-10T07:34:14.013923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:39.836464+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:34:39.836464+0000","last_peered":"2024-09-10T07:34:39.836464+0000","last_clean":"2024-09-10T07:34:39.836464+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:34:39.836464+0000","last_undegraded":"2024-09-10T07:34:39.836464+0000","last_fullsized":"2024-09-10T07:34:39.836464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156129+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:34:14.156129+0000","last_peered":"2024-09-10T07:34:14.156129+0000","last_clean":"2024-09-10T07:34:14.156129+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:34:14.156129+0000","last_undegraded":"2024-09-10T07:34:14.156129+0000","last_fullsized":"2024-09-10T07:34:14.156129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.020283+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:34:14.020283+0000","last_peered":"2024-09-10T07:34:14.020283+0000","last_clean":"2024-09-10T07:34:14.020283+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:34:14.020283+0000","last_undegraded":"2024-09-10T07:34:14.020283+0000","last_fullsized":"2024-09-10T07:34:14.020283+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:44.832253+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:34:44.832253+0000","last_peered":"2024-09-10T07:34:44.832253+0000","last_clean":"2024-09-10T07:34:44.832253+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:34:44.832253+0000","last_undegraded":"2024-09-10T07:34:44.832253+0000","last_fullsized":"2024-09-10T07:34:44.832253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156247+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:34:14.156247+0000","last_peered":"2024-09-10T07:34:14.156247+0000","last_clean":"2024-09-10T07:34:14.156247+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:34:14.156247+0000","last_undegraded":"2024-09-10T07:34:14.156247+0000","last_fullsized":"2024-09-10T07:34:14.156247+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.155684+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:34:14.155684+0000","last_peered":"2024-09-10T07:34:14.155684+0000","last_clean":"2024-09-10T07:34:14.155684+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:34:14.155684+0000","last_undegraded":"2024-09-10T07:34:14.155684+0000","last_fullsized":"2024-09-10T07:34:14.155684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:24.077410+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:34:24.077410+0000","last_peered":"2024-09-10T07:34:24.077410+0000","last_clean":"2024-09-10T07:34:24.077410+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:34:24.077410+0000","last_undegraded":"2024-09-10T07:34:24.077410+0000","last_fullsized":"2024-09-10T07:34:24.077410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.157032+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:34:14.157032+0000","last_peered":"2024-09-10T07:34:14.157032+0000","last_clean":"2024-09-10T07:34:14.157032+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:34:14.157032+0000","last_undegraded":"2024-09-10T07:34:14.157032+0000","last_fullsized":"2024-09-10T07:34:14.157032+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011946+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:34:14.011946+0000","last_peered":"2024-09-10T07:34:14.011946+0000","last_clean":"2024-09-10T07:34:14.011946+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:34:14.011946+0000","last_undegraded":"2024-09-10T07:34:14.011946+0000","last_fullsized":"2024-09-10T07:34:14.011946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.813192+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:34:25.813192+0000","last_peered":"2024-09-10T07:34:25.813192+0000","last_clean":"2024-09-10T07:34:25.813192+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:34:25.813192+0000","last_undegraded":"2024-09-10T07:34:25.813192+0000","last_fullsized":"2024-09-10T07:34:25.813192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011724+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:34:14.011724+0000","last_peered":"2024-09-10T07:34:14.011724+0000","last_clean":"2024-09-10T07:34:14.011724+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:34:14.011724+0000","last_undegraded":"2024-09-10T07:34:14.011724+0000","last_fullsized":"2024-09-10T07:34:14.011724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012041+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:34:14.012041+0000","last_peered":"2024-09-10T07:34:14.012041+0000","last_clean":"2024-09-10T07:34:14.012041+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:34:14.012041+0000","last_undegraded":"2024-09-10T07:34:14.012041+0000","last_fullsized":"2024-09-10T07:34:14.012041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012299+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:34:14.012299+0000","last_peered":"2024-09-10T07:34:14.012299+0000","last_clean":"2024-09-10T07:34:14.012299+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:34:14.012299+0000","last_undegraded":"2024-09-10T07:34:14.012299+0000","last_fullsized":"2024-09-10T07:34:14.012299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":66,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:43.831894+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:34:43.831894+0000","last_peered":"2024-09-10T07:34:43.831894+0000","last_clean":"2024-09-10T07:34:43.831894+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:34:43.831894+0000","last_undegraded":"2024-09-10T07:34:43.831894+0000","last_fullsized":"2024-09-10T07:34:43.831894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106232+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:34:14.106232+0000","last_peered":"2024-09-10T07:34:14.106232+0000","last_clean":"2024-09-10T07:34:14.106232+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:34:14.106232+0000","last_undegraded":"2024-09-10T07:34:14.106232+0000","last_fullsized":"2024-09-10T07:34:14.106232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011941+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:34:14.011941+0000","last_peered":"2024-09-10T07:34:14.011941+0000","last_clean":"2024-09-10T07:34:14.011941+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:34:14.011941+0000","last_undegraded":"2024-09-10T07:34:14.011941+0000","last_fullsized":"2024-09-10T07:34:14.011941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.801121+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:34:25.801121+0000","last_peered":"2024-09-10T07:34:25.801121+0000","last_clean":"2024-09-10T07:34:25.801121+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:34:25.801121+0000","last_undegraded":"2024-09-10T07:34:25.801121+0000","last_fullsized":"2024-09-10T07:34:25.801121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.108108+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:34:14.108108+0000","last_peered":"2024-09-10T07:34:14.108108+0000","last_clean":"2024-09-10T07:34:14.108108+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:34:14.108108+0000","last_undegraded":"2024-09-10T07:34:14.108108+0000","last_fullsized":"2024-09-10T07:34:14.108108+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012149+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:34:14.012149+0000","last_peered":"2024-09-10T07:34:14.012149+0000","last_clean":"2024-09-10T07:34:14.012149+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:34:14.012149+0000","last_undegraded":"2024-09-10T07:34:14.012149+0000","last_fullsized":"2024-09-10T07:34:14.012149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.815769+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:34:25.815769+0000","last_peered":"2024-09-10T07:34:25.815769+0000","last_clean":"2024-09-10T07:34:25.815769+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:34:25.815769+0000","last_undegraded":"2024-09-10T07:34:25.815769+0000","last_fullsized":"2024-09-10T07:34:25.815769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017144+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:34:14.017144+0000","last_peered":"2024-09-10T07:34:14.017144+0000","last_clean":"2024-09-10T07:34:14.017144+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:34:14.017144+0000","last_undegraded":"2024-09-10T07:34:14.017144+0000","last_fullsized":"2024-09-10T07:34:14.017144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014600+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:34:14.014600+0000","last_peered":"2024-09-10T07:34:14.014600+0000","last_clean":"2024-09-10T07:34:14.014600+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:34:14.014600+0000","last_undegraded":"2024-09-10T07:34:14.014600+0000","last_fullsized":"2024-09-10T07:34:14.014600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.801200+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:34:25.801200+0000","last_peered":"2024-09-10T07:34:25.801200+0000","last_clean":"2024-09-10T07:34:25.801200+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:34:25.801200+0000","last_undegraded":"2024-09-10T07:34:25.801200+0000","last_fullsized":"2024-09-10T07:34:25.801200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016342+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:34:14.016342+0000","last_peered":"2024-09-10T07:34:14.016342+0000","last_clean":"2024-09-10T07:34:14.016342+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:34:14.016342+0000","last_undegraded":"2024-09-10T07:34:14.016342+0000","last_fullsized":"2024-09-10T07:34:14.016342+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.021124+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:34:14.021124+0000","last_peered":"2024-09-10T07:34:14.021124+0000","last_clean":"2024-09-10T07:34:14.021124+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:34:14.021124+0000","last_undegraded":"2024-09-10T07:34:14.021124+0000","last_fullsized":"2024-09-10T07:34:14.021124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.801135+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:34:25.801135+0000","last_peered":"2024-09-10T07:34:25.801135+0000","last_clean":"2024-09-10T07:34:25.801135+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:34:25.801135+0000","last_undegraded":"2024-09-10T07:34:25.801135+0000","last_fullsized":"2024-09-10T07:34:25.801135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156305+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:34:14.156305+0000","last_peered":"2024-09-10T07:34:14.156305+0000","last_clean":"2024-09-10T07:34:14.156305+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:34:14.156305+0000","last_undegraded":"2024-09-10T07:34:14.156305+0000","last_fullsized":"2024-09-10T07:34:14.156305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011502+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:34:14.011502+0000","last_peered":"2024-09-10T07:34:14.011502+0000","last_clean":"2024-09-10T07:34:14.011502+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:34:14.011502+0000","last_undegraded":"2024-09-10T07:34:14.011502+0000","last_fullsized":"2024-09-10T07:34:14.011502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":65,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:43.831043+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:34:43.831043+0000","last_peered":"2024-09-10T07:34:43.831043+0000","last_clean":"2024-09-10T07:34:43.831043+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:34:43.831043+0000","last_undegraded":"2024-09-10T07:34:43.831043+0000","last_fullsized":"2024-09-10T07:34:43.831043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018813+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:34:14.018813+0000","last_peered":"2024-09-10T07:34:14.018813+0000","last_clean":"2024-09-10T07:34:14.018813+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:34:14.018813+0000","last_undegraded":"2024-09-10T07:34:14.018813+0000","last_fullsized":"2024-09-10T07:34:14.018813+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011844+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:34:14.011844+0000","last_peered":"2024-09-10T07:34:14.011844+0000","last_clean":"2024-09-10T07:34:14.011844+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:34:14.011844+0000","last_undegraded":"2024-09-10T07:34:14.011844+0000","last_fullsized":"2024-09-10T07:34:14.011844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.813467+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:34:25.813467+0000","last_peered":"2024-09-10T07:34:25.813467+0000","last_clean":"2024-09-10T07:34:25.813467+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:34:25.813467+0000","last_undegraded":"2024-09-10T07:34:25.813467+0000","last_fullsized":"2024-09-10T07:34:25.813467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156814+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:34:14.156814+0000","last_peered":"2024-09-10T07:34:14.156814+0000","last_clean":"2024-09-10T07:34:14.156814+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:34:14.156814+0000","last_undegraded":"2024-09-10T07:34:14.156814+0000","last_fullsized":"2024-09-10T07:34:14.156814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.155671+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:34:14.155671+0000","last_peered":"2024-09-10T07:34:14.155671+0000","last_clean":"2024-09-10T07:34:14.155671+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:34:14.155671+0000","last_undegraded":"2024-09-10T07:34:14.155671+0000","last_fullsized":"2024-09-10T07:34:14.155671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.816123+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:34:25.816123+0000","last_peered":"2024-09-10T07:34:25.816123+0000","last_clean":"2024-09-10T07:34:25.816123+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:34:25.816123+0000","last_undegraded":"2024-09-10T07:34:25.816123+0000","last_fullsized":"2024-09-10T07:34:25.816123+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.020853+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:34:14.020853+0000","last_peered":"2024-09-10T07:34:14.020853+0000","last_clean":"2024-09-10T07:34:14.020853+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:34:14.020853+0000","last_undegraded":"2024-09-10T07:34:14.020853+0000","last_fullsized":"2024-09-10T07:34:14.020853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012277+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:34:14.012277+0000","last_peered":"2024-09-10T07:34:14.012277+0000","last_clean":"2024-09-10T07:34:14.012277+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:34:14.012277+0000","last_undegraded":"2024-09-10T07:34:14.012277+0000","last_fullsized":"2024-09-10T07:34:14.012277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'31","reported_seq":115,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:43.831093+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:34:43.831093+0000","last_peered":"2024-09-10T07:34:43.831093+0000","last_clean":"2024-09-10T07:34:43.831093+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:34:43.831093+0000","last_undegraded":"2024-09-10T07:34:43.831093+0000","last_fullsized":"2024-09-10T07:34:43.831093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51899,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015714+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:34:14.015714+0000","last_peered":"2024-09-10T07:34:14.015714+0000","last_clean":"2024-09-10T07:34:14.015714+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:34:14.015714+0000","last_undegraded":"2024-09-10T07:34:14.015714+0000","last_fullsized":"2024-09-10T07:34:14.015714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106566+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:34:14.106566+0000","last_peered":"2024-09-10T07:34:14.106566+0000","last_clean":"2024-09-10T07:34:14.106566+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:34:14.106566+0000","last_undegraded":"2024-09-10T07:34:14.106566+0000","last_fullsized":"2024-09-10T07:34:14.106566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.085589+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:34:25.085589+0000","last_peered":"2024-09-10T07:34:25.085589+0000","last_clean":"2024-09-10T07:34:25.085589+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:34:25.085589+0000","last_undegraded":"2024-09-10T07:34:25.085589+0000","last_fullsized":"2024-09-10T07:34:25.085589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016800+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:34:14.016800+0000","last_peered":"2024-09-10T07:34:14.016800+0000","last_clean":"2024-09-10T07:34:14.016800+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:34:14.016800+0000","last_undegraded":"2024-09-10T07:34:14.016800+0000","last_fullsized":"2024-09-10T07:34:14.016800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019812+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:34:14.019812+0000","last_peered":"2024-09-10T07:34:14.019812+0000","last_clean":"2024-09-10T07:34:14.019812+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:34:14.019812+0000","last_undegraded":"2024-09-10T07:34:14.019812+0000","last_fullsized":"2024-09-10T07:34:14.019812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.813963+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:34:25.813963+0000","last_peered":"2024-09-10T07:34:25.813963+0000","last_clean":"2024-09-10T07:34:25.813963+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:34:25.813963+0000","last_undegraded":"2024-09-10T07:34:25.813963+0000","last_fullsized":"2024-09-10T07:34:25.813963+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015693+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:34:14.015693+0000","last_peered":"2024-09-10T07:34:14.015693+0000","last_clean":"2024-09-10T07:34:14.015693+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:34:14.015693+0000","last_undegraded":"2024-09-10T07:34:14.015693+0000","last_fullsized":"2024-09-10T07:34:14.015693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106737+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:34:14.106737+0000","last_peered":"2024-09-10T07:34:14.106737+0000","last_clean":"2024-09-10T07:34:14.106737+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:34:14.106737+0000","last_undegraded":"2024-09-10T07:34:14.106737+0000","last_fullsized":"2024-09-10T07:34:14.106737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:43.831124+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:34:43.831124+0000","last_peered":"2024-09-10T07:34:43.831124+0000","last_clean":"2024-09-10T07:34:43.831124+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:34:43.831124+0000","last_undegraded":"2024-09-10T07:34:43.831124+0000","last_fullsized":"2024-09-10T07:34:43.831124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011599+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:34:14.011599+0000","last_peered":"2024-09-10T07:34:14.011599+0000","last_clean":"2024-09-10T07:34:14.011599+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:34:14.011599+0000","last_undegraded":"2024-09-10T07:34:14.011599+0000","last_fullsized":"2024-09-10T07:34:14.011599+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019112+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:34:14.019112+0000","last_peered":"2024-09-10T07:34:14.019112+0000","last_clean":"2024-09-10T07:34:14.019112+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:34:14.019112+0000","last_undegraded":"2024-09-10T07:34:14.019112+0000","last_fullsized":"2024-09-10T07:34:14.019112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.015976+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:34:14.015976+0000","last_peered":"2024-09-10T07:34:14.015976+0000","last_clean":"2024-09-10T07:34:14.015976+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:34:14.015976+0000","last_undegraded":"2024-09-10T07:34:14.015976+0000","last_fullsized":"2024-09-10T07:34:14.015976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.808265+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:34:25.808265+0000","last_peered":"2024-09-10T07:34:25.808265+0000","last_clean":"2024-09-10T07:34:25.808265+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:34:25.808265+0000","last_undegraded":"2024-09-10T07:34:25.808265+0000","last_fullsized":"2024-09-10T07:34:25.808265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014391+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:34:14.014391+0000","last_peered":"2024-09-10T07:34:14.014391+0000","last_clean":"2024-09-10T07:34:14.014391+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:34:14.014391+0000","last_undegraded":"2024-09-10T07:34:14.014391+0000","last_fullsized":"2024-09-10T07:34:14.014391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011618+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:34:14.011618+0000","last_peered":"2024-09-10T07:34:14.011618+0000","last_clean":"2024-09-10T07:34:14.011618+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:34:14.011618+0000","last_undegraded":"2024-09-10T07:34:14.011618+0000","last_fullsized":"2024-09-10T07:34:14.011618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:24.079967+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:34:24.079967+0000","last_peered":"2024-09-10T07:34:24.079967+0000","last_clean":"2024-09-10T07:34:24.079967+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:34:24.079967+0000","last_undegraded":"2024-09-10T07:34:24.079967+0000","last_fullsized":"2024-09-10T07:34:24.079967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:25.085644+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:34:25.085644+0000","last_peered":"2024-09-10T07:34:25.085644+0000","last_clean":"2024-09-10T07:34:25.085644+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:34:25.085644+0000","last_undegraded":"2024-09-10T07:34:25.085644+0000","last_fullsized":"2024-09-10T07:34:25.085644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.020936+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:34:14.020936+0000","last_peered":"2024-09-10T07:34:14.020936+0000","last_clean":"2024-09-10T07:34:14.020936+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:34:14.020936+0000","last_undegraded":"2024-09-10T07:34:14.020936+0000","last_fullsized":"2024-09-10T07:34:14.020936+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018080+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:34:14.018080+0000","last_peered":"2024-09-10T07:34:14.018080+0000","last_clean":"2024-09-10T07:34:14.018080+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:34:14.018080+0000","last_undegraded":"2024-09-10T07:34:14.018080+0000","last_fullsized":"2024-09-10T07:34:14.018080+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:43.831899+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:34:43.831899+0000","last_peered":"2024-09-10T07:34:43.831899+0000","last_clean":"2024-09-10T07:34:43.831899+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:34:43.831899+0000","last_undegraded":"2024-09-10T07:34:43.831899+0000","last_fullsized":"2024-09-10T07:34:43.831899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012030+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:34:14.012030+0000","last_peered":"2024-09-10T07:34:14.012030+0000","last_clean":"2024-09-10T07:34:14.012030+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:34:14.012030+0000","last_undegraded":"2024-09-10T07:34:14.012030+0000","last_fullsized":"2024-09-10T07:34:14.012030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018373+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:34:14.018373+0000","last_peered":"2024-09-10T07:34:14.018373+0000","last_clean":"2024-09-10T07:34:14.018373+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:34:14.018373+0000","last_undegraded":"2024-09-10T07:34:14.018373+0000","last_fullsized":"2024-09-10T07:34:14.018373+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014119+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:34:14.014119+0000","last_peered":"2024-09-10T07:34:14.014119+0000","last_clean":"2024-09-10T07:34:14.014119+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:34:14.014119+0000","last_undegraded":"2024-09-10T07:34:14.014119+0000","last_fullsized":"2024-09-10T07:34:14.014119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018035+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:34:14.018035+0000","last_peered":"2024-09-10T07:34:14.018035+0000","last_clean":"2024-09-10T07:34:14.018035+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:34:14.018035+0000","last_undegraded":"2024-09-10T07:34:14.018035+0000","last_fullsized":"2024-09-10T07:34:14.018035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017829+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:34:14.017829+0000","last_peered":"2024-09-10T07:34:14.017829+0000","last_clean":"2024-09-10T07:34:14.017829+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:34:14.017829+0000","last_undegraded":"2024-09-10T07:34:14.017829+0000","last_fullsized":"2024-09-10T07:34:14.017829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018508+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:34:14.018508+0000","last_peered":"2024-09-10T07:34:14.018508+0000","last_clean":"2024-09-10T07:34:14.018508+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:34:14.018508+0000","last_undegraded":"2024-09-10T07:34:14.018508+0000","last_fullsized":"2024-09-10T07:34:14.018508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156358+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:34:14.156358+0000","last_peered":"2024-09-10T07:34:14.156358+0000","last_clean":"2024-09-10T07:34:14.156358+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:34:14.156358+0000","last_undegraded":"2024-09-10T07:34:14.156358+0000","last_fullsized":"2024-09-10T07:34:14.156358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014595+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:34:14.014595+0000","last_peered":"2024-09-10T07:34:14.014595+0000","last_clean":"2024-09-10T07:34:14.014595+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:34:14.014595+0000","last_undegraded":"2024-09-10T07:34:14.014595+0000","last_fullsized":"2024-09-10T07:34:14.014595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018368+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:34:14.018368+0000","last_peered":"2024-09-10T07:34:14.018368+0000","last_clean":"2024-09-10T07:34:14.018368+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:34:14.018368+0000","last_undegraded":"2024-09-10T07:34:14.018368+0000","last_fullsized":"2024-09-10T07:34:14.018368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012320+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:34:14.012320+0000","last_peered":"2024-09-10T07:34:14.012320+0000","last_clean":"2024-09-10T07:34:14.012320+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:34:14.012320+0000","last_undegraded":"2024-09-10T07:34:14.012320+0000","last_fullsized":"2024-09-10T07:34:14.012320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017206+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:34:14.017206+0000","last_peered":"2024-09-10T07:34:14.017206+0000","last_clean":"2024-09-10T07:34:14.017206+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:34:14.017206+0000","last_undegraded":"2024-09-10T07:34:14.017206+0000","last_fullsized":"2024-09-10T07:34:14.017206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011891+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:34:14.011891+0000","last_peered":"2024-09-10T07:34:14.011891+0000","last_clean":"2024-09-10T07:34:14.011891+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:34:14.011891+0000","last_undegraded":"2024-09-10T07:34:14.011891+0000","last_fullsized":"2024-09-10T07:34:14.011891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106467+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:34:14.106467+0000","last_peered":"2024-09-10T07:34:14.106467+0000","last_clean":"2024-09-10T07:34:14.106467+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:34:14.106467+0000","last_undegraded":"2024-09-10T07:34:14.106467+0000","last_fullsized":"2024-09-10T07:34:14.106467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.156022+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:34:14.156022+0000","last_peered":"2024-09-10T07:34:14.156022+0000","last_clean":"2024-09-10T07:34:14.156022+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:34:14.156022+0000","last_undegraded":"2024-09-10T07:34:14.156022+0000","last_fullsized":"2024-09-10T07:34:14.156022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.021473+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:34:14.021473+0000","last_peered":"2024-09-10T07:34:14.021473+0000","last_clean":"2024-09-10T07:34:14.021473+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:34:14.021473+0000","last_undegraded":"2024-09-10T07:34:14.021473+0000","last_fullsized":"2024-09-10T07:34:14.021473+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018542+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:34:14.018542+0000","last_peered":"2024-09-10T07:34:14.018542+0000","last_clean":"2024-09-10T07:34:14.018542+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:34:14.018542+0000","last_undegraded":"2024-09-10T07:34:14.018542+0000","last_fullsized":"2024-09-10T07:34:14.018542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019269+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:34:14.019269+0000","last_peered":"2024-09-10T07:34:14.019269+0000","last_clean":"2024-09-10T07:34:14.019269+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:34:14.019269+0000","last_undegraded":"2024-09-10T07:34:14.019269+0000","last_fullsized":"2024-09-10T07:34:14.019269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014760+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:34:14.014760+0000","last_peered":"2024-09-10T07:34:14.014760+0000","last_clean":"2024-09-10T07:34:14.014760+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:34:14.014760+0000","last_undegraded":"2024-09-10T07:34:14.014760+0000","last_fullsized":"2024-09-10T07:34:14.014760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106126+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:34:14.106126+0000","last_peered":"2024-09-10T07:34:14.106126+0000","last_clean":"2024-09-10T07:34:14.106126+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:34:14.106126+0000","last_undegraded":"2024-09-10T07:34:14.106126+0000","last_fullsized":"2024-09-10T07:34:14.106126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.017632+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:34:14.017632+0000","last_peered":"2024-09-10T07:34:14.017632+0000","last_clean":"2024-09-10T07:34:14.017632+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:34:14.017632+0000","last_undegraded":"2024-09-10T07:34:14.017632+0000","last_fullsized":"2024-09-10T07:34:14.017632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018240+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:34:14.018240+0000","last_peered":"2024-09-10T07:34:14.018240+0000","last_clean":"2024-09-10T07:34:14.018240+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:34:14.018240+0000","last_undegraded":"2024-09-10T07:34:14.018240+0000","last_fullsized":"2024-09-10T07:34:14.018240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.155609+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:34:14.155609+0000","last_peered":"2024-09-10T07:34:14.155609+0000","last_clean":"2024-09-10T07:34:14.155609+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:34:14.155609+0000","last_undegraded":"2024-09-10T07:34:14.155609+0000","last_fullsized":"2024-09-10T07:34:14.155609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016594+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:34:14.016594+0000","last_peered":"2024-09-10T07:34:14.016594+0000","last_clean":"2024-09-10T07:34:14.016594+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:34:14.016594+0000","last_undegraded":"2024-09-10T07:34:14.016594+0000","last_fullsized":"2024-09-10T07:34:14.016594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012167+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:34:14.012167+0000","last_peered":"2024-09-10T07:34:14.012167+0000","last_clean":"2024-09-10T07:34:14.012167+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:34:14.012167+0000","last_undegraded":"2024-09-10T07:34:14.012167+0000","last_fullsized":"2024-09-10T07:34:14.012167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014446+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:34:14.014446+0000","last_peered":"2024-09-10T07:34:14.014446+0000","last_clean":"2024-09-10T07:34:14.014446+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:34:14.014446+0000","last_undegraded":"2024-09-10T07:34:14.014446+0000","last_fullsized":"2024-09-10T07:34:14.014446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.106900+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:34:14.106900+0000","last_peered":"2024-09-10T07:34:14.106900+0000","last_clean":"2024-09-10T07:34:14.106900+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:34:14.106900+0000","last_undegraded":"2024-09-10T07:34:14.106900+0000","last_fullsized":"2024-09-10T07:34:14.106900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011591+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:34:14.011591+0000","last_peered":"2024-09-10T07:34:14.011591+0000","last_clean":"2024-09-10T07:34:14.011591+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:34:14.011591+0000","last_undegraded":"2024-09-10T07:34:14.011591+0000","last_fullsized":"2024-09-10T07:34:14.011591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.011617+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:34:14.011617+0000","last_peered":"2024-09-10T07:34:14.011617+0000","last_clean":"2024-09-10T07:34:14.011617+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:34:14.011617+0000","last_undegraded":"2024-09-10T07:34:14.011617+0000","last_fullsized":"2024-09-10T07:34:14.011617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.013777+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:34:14.013777+0000","last_peered":"2024-09-10T07:34:14.013777+0000","last_clean":"2024-09-10T07:34:14.013777+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:34:14.013777+0000","last_undegraded":"2024-09-10T07:34:14.013777+0000","last_fullsized":"2024-09-10T07:34:14.013777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.012016+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:34:14.012016+0000","last_peered":"2024-09-10T07:34:14.012016+0000","last_clean":"2024-09-10T07:34:14.012016+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:34:14.012016+0000","last_undegraded":"2024-09-10T07:34:14.012016+0000","last_fullsized":"2024-09-10T07:34:14.012016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.016459+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:34:14.016459+0000","last_peered":"2024-09-10T07:34:14.016459+0000","last_clean":"2024-09-10T07:34:14.016459+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:34:14.016459+0000","last_undegraded":"2024-09-10T07:34:14.016459+0000","last_fullsized":"2024-09-10T07:34:14.016459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.018534+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:34:14.018534+0000","last_peered":"2024-09-10T07:34:14.018534+0000","last_clean":"2024-09-10T07:34:14.018534+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:34:14.018534+0000","last_undegraded":"2024-09-10T07:34:14.018534+0000","last_fullsized":"2024-09-10T07:34:14.018534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.019210+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:34:14.019210+0000","last_peered":"2024-09-10T07:34:14.019210+0000","last_clean":"2024-09-10T07:34:14.019210+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:34:14.019210+0000","last_undegraded":"2024-09-10T07:34:14.019210+0000","last_fullsized":"2024-09-10T07:34:14.019210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T07:34:14.014849+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:34:14.014849+0000","last_peered":"2024-09-10T07:34:14.014849+0000","last_clean":"2024-09-10T07:34:14.014849+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:34:14.014849+0000","last_undegraded":"2024-09-10T07:34:14.014849+0000","last_fullsized":"2024-09-10T07:34:14.014849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59107,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":228,"num_read_kb":181,"num_write":242,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":185851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":542421,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710753,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1196032,"data_stored":1181068,"data_compressed":62990,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874276,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":546379,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":542661,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1139553,"data_compressed":62990,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":538180,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1134742,"data_compressed":62990,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":541861,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":537495,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1126400,"data_stored":591261,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":591109,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":539243,"data_compressed":26438,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:34:48.959 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T07:34:48.960 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T07:34:48.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T07:34:49.341 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:49.698 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:49 smithi046 ceph-mon[30705]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:34:49.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:49 smithi181 ceph-mon[35397]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:34:49.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:49 smithi063 ceph-mon[35020]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:34:49.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.975+0000 7f20f0400640 1 -- 172.21.15.46:0/1405358963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8104f60 msgr2=0x7f20e81053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.975+0000 7f20f0400640 1 --2- 172.21.15.46:0/1405358963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8104f60 0x7f20e81053e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009900 tx=0x7f20d402a410 comp rx=0 tx=0).stop 2024-09-10T07:34:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.976+0000 7f20f0400640 1 -- 172.21.15.46:0/1405358963 shutdown_connections 2024-09-10T07:34:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.976+0000 7f20f0400640 1 --2- 172.21.15.46:0/1405358963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20e8105920 0x7f20e810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.976+0000 7f20f0400640 1 --2- 172.21.15.46:0/1405358963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8104f60 0x7f20e81053e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.976+0000 7f20f0400640 1 --2- 172.21.15.46:0/1405358963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20e8103e00 0x7f20e8104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.976+0000 7f20f0400640 1 -- 172.21.15.46:0/1405358963 >> 172.21.15.46:0/1405358963 conn(0x7f20e80ff5b0 msgr2=0x7f20e81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.976+0000 7f20f0400640 1 -- 172.21.15.46:0/1405358963 shutdown_connections 2024-09-10T07:34:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.976+0000 7f20f0400640 1 -- 172.21.15.46:0/1405358963 wait complete. 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20f0400640 1 Processor -- start 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20f0400640 1 -- start start 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20f0400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8103e00 0x7f20e819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20f0400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20e8104f60 0x7f20e819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20f0400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20e8105920 0x7f20e81a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20f0400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20e810fb00 con 0x7f20e8103e00 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20f0400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20e810f980 con 0x7f20e8105920 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20f0400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20e810fc80 con 0x7f20e8104f60 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.977+0000 7f20eda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8103e00 0x7f20e819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20eda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8103e00 0x7f20e819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57858/0 (socket says 172.21.15.46:57858) 2024-09-10T07:34:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20eda00640 1 -- 172.21.15.46:0/3802070007 learned_addr learned my addr 172.21.15.46:0/3802070007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20ee400640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20e8105920 0x7f20e81a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20ed000640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20e8104f60 0x7f20e819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20eda00640 1 -- 172.21.15.46:0/3802070007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20e8104f60 msgr2=0x7f20e819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20eda00640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20e8104f60 0x7f20e819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20eda00640 1 -- 172.21.15.46:0/3802070007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20e8105920 msgr2=0x7f20e81a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20eda00640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20e8105920 0x7f20e81a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20eda00640 1 -- 172.21.15.46:0/3802070007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e81a1d20 con 0x7f20e8103e00 2024-09-10T07:34:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.978+0000 7f20eda00640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8103e00 0x7f20e819a0f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00d770 tx=0x7f20dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:49.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.979+0000 7f20e6000640 1 -- 172.21.15.46:0/3802070007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc017070 con 0x7f20e8103e00 2024-09-10T07:34:49.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.979+0000 7f20e6000640 1 -- 172.21.15.46:0/3802070007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20dc005500 con 0x7f20e8103e00 2024-09-10T07:34:49.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.979+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e81a1fb0 con 0x7f20e8103e00 2024-09-10T07:34:49.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.979+0000 7f20e6000640 1 -- 172.21.15.46:0/3802070007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc004e00 con 0x7f20e8103e00 2024-09-10T07:34:49.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.979+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e81a2460 con 0x7f20e8103e00 2024-09-10T07:34:49.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.980+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e810b240 con 0x7f20e8103e00 2024-09-10T07:34:49.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.982+0000 7f20e6000640 1 -- 172.21.15.46:0/3802070007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f20dc004fa0 con 0x7f20e8103e00 2024-09-10T07:34:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.983+0000 7f20e6000640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20d00a2cc0 0x7f20d00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.983+0000 7f20e6000640 1 -- 172.21.15.46:0/3802070007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f20dc0cee90 con 0x7f20e8103e00 2024-09-10T07:34:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.983+0000 7f20ed000640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20d00a2cc0 0x7f20d00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.984+0000 7f20ed000640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20d00a2cc0 0x7f20d00a5180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20d402aa00 tx=0x7f20d40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:49.987+0000 7f20e6000640 1 -- 172.21.15.46:0/3802070007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20dc09b280 con 0x7f20e8103e00 2024-09-10T07:34:50.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.240+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f20e802d7c0 con 0x7f20e8103e00 2024-09-10T07:34:50.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.242+0000 7f20e6000640 1 -- 172.21.15.46:0/3802070007 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f20e802d7c0 con 0x7f20e8103e00 2024-09-10T07:34:50.243 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-10T07:34:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCa9t9m6kpwDhAA5esv9HChWkys6gOijkOZYQ== 2024-09-10T07:34:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20d00a2cc0 msgr2=0x7f20d00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20d00a2cc0 0x7f20d00a5180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20d402aa00 tx=0x7f20d40315b0 comp rx=0 tx=0).stop 2024-09-10T07:34:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8103e00 msgr2=0x7f20e819a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8103e00 0x7f20e819a0f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00d770 tx=0x7f20dc00dc30 comp rx=0 tx=0).stop 2024-09-10T07:34:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 shutdown_connections 2024-09-10T07:34:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20e8105920 0x7f20e81a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20e8104f60 0x7f20e819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20d00a2cc0 0x7f20d00a5180 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 --2- 172.21.15.46:0/3802070007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20e8103e00 0x7f20e819a0f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.245+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 >> 172.21.15.46:0/3802070007 conn(0x7f20e80ff5b0 msgr2=0x7f20e8108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.246+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 shutdown_connections 2024-09-10T07:34:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:50.246+0000 7f20f0400640 1 -- 172.21.15.46:0/3802070007 wait complete. 2024-09-10T07:34:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:50 smithi046 ceph-mon[30705]: pgmap v343: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-10T07:34:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:50 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3802070007' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T07:34:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:50 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3802070007' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T07:34:50.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:50 smithi181 ceph-mon[35397]: pgmap v343: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-10T07:34:50.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3802070007' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T07:34:50.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3802070007' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T07:34:50.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:50 smithi063 ceph-mon[35020]: pgmap v343: 129 pgs: 129 active+clean; 634 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-10T07:34:50.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:50 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3802070007' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T07:34:50.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:50 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3802070007' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T07:34:51.108 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:34:51.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T07:34:51.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T07:34:51.188 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T07:34:51.189 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T07:34:51.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T07:34:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:52.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 -- 172.21.15.46:0/377667734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401049a0 msgr2=0x7f88401089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 --2- 172.21.15.46:0/377667734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401049a0 0x7f88401089f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8828009900 tx=0x7f882802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 -- 172.21.15.46:0/377667734 shutdown_connections 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 --2- 172.21.15.46:0/377667734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8840109120 0x7f8840110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 --2- 172.21.15.46:0/377667734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401049a0 0x7f88401089f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 --2- 172.21.15.46:0/377667734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8840103ff0 0x7f88401043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 -- 172.21.15.46:0/377667734 >> 172.21.15.46:0/377667734 conn(0x7f88400ffa90 msgr2=0x7f8840101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 -- 172.21.15.46:0/377667734 shutdown_connections 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.255+0000 7f8846c00640 1 -- 172.21.15.46:0/377667734 wait complete. 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.256+0000 7f8846c00640 1 Processor -- start 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.256+0000 7f8846c00640 1 -- start start 2024-09-10T07:34:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.256+0000 7f8846c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840103ff0 0x7f884019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.256+0000 7f8846c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88401049a0 0x7f884019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.256+0000 7f8846c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8840109120 0x7f884019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.256+0000 7f8846c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8840114190 con 0x7f8840103ff0 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.256+0000 7f8846c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8840114010 con 0x7f8840109120 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.256+0000 7f8846c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8840114310 con 0x7f88401049a0 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f883fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840103ff0 0x7f884019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f883fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840103ff0 0x7f884019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57874/0 (socket says 172.21.15.46:57874) 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f883fe00640 1 -- 172.21.15.46:0/1812804024 learned_addr learned my addr 172.21.15.46:0/1812804024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f8844c00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8840109120 0x7f884019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f8837e00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88401049a0 0x7f884019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f883fe00640 1 -- 172.21.15.46:0/1812804024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88401049a0 msgr2=0x7f884019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f883fe00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88401049a0 0x7f884019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f883fe00640 1 -- 172.21.15.46:0/1812804024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8840109120 msgr2=0x7f884019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f883fe00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8840109120 0x7f884019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f883fe00640 1 -- 172.21.15.46:0/1812804024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884019f5a0 con 0x7f8840103ff0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.257+0000 7f8844c00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8840109120 0x7f884019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.258+0000 7f883fe00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840103ff0 0x7f884019a5f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9f0 tx=0x7f883000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.258+0000 7f883d600640 1 -- 172.21.15.46:0/1812804024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883000cd80 con 0x7f8840103ff0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.258+0000 7f883d600640 1 -- 172.21.15.46:0/1812804024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8830010910 con 0x7f8840103ff0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.258+0000 7f883d600640 1 -- 172.21.15.46:0/1812804024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830018990 con 0x7f8840103ff0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.258+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884019f890 con 0x7f8840103ff0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.258+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88401a7170 con 0x7f8840103ff0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.259+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88401043d0 con 0x7f8840103ff0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.261+0000 7f883d600640 1 -- 172.21.15.46:0/1812804024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8830010ac0 con 0x7f8840103ff0 2024-09-10T07:34:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.262+0000 7f883d600640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88240a2c90 0x7f88240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:52.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.263+0000 7f883d600640 1 -- 172.21.15.46:0/1812804024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f88300cf800 con 0x7f8840103ff0 2024-09-10T07:34:52.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.263+0000 7f8837e00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88240a2c90 0x7f88240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:52.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.263+0000 7f8837e00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88240a2c90 0x7f88240a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8828000c00 tx=0x7f8828005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:52.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.266+0000 7f883d600640 1 -- 172.21.15.46:0/1812804024 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f883009bbf0 con 0x7f8840103ff0 2024-09-10T07:34:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.502+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f88401a73a0 con 0x7f8840103ff0 2024-09-10T07:34:52.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.507+0000 7f883d600640 1 -- 172.21.15.46:0/1812804024 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263199 (secure 0 0 0) 0x7f883009fef0 con 0x7f8840103ff0 2024-09-10T07:34:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88240a2c90 msgr2=0x7f88240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88240a2c90 0x7f88240a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8828000c00 tx=0x7f8828005b50 comp rx=0 tx=0).stop 2024-09-10T07:34:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840103ff0 msgr2=0x7f884019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840103ff0 0x7f884019a5f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9f0 tx=0x7f883000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 shutdown_connections 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8840109120 0x7f884019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88401049a0 0x7f884019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88240a2c90 0x7f88240a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 --2- 172.21.15.46:0/1812804024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840103ff0 0x7f884019a5f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 >> 172.21.15.46:0/1812804024 conn(0x7f88400ffa90 msgr2=0x7f8840101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.511+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 shutdown_connections 2024-09-10T07:34:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:52.512+0000 7f8846c00640 1 -- 172.21.15.46:0/1812804024 wait complete. 2024-09-10T07:34:52.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:52 smithi046 ceph-mon[30705]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-10T07:34:52.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:52 smithi181 ceph-mon[35397]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-10T07:34:52.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:52 smithi063 ceph-mon[35020]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-10T07:34:53.383 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1492955812},{"type":"v1","addr":"172.21.15.46:6801","nonce":1492955812}]},"active_addr":"172.21.15.46:6801/1492955812","active_change":"2024-09-10T07:26:12.367295+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2657753432}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3766438969}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":547561128}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3732361403}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":845763395}]}]} 2024-09-10T07:34:53.389 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T07:34:53.389 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T07:34:53.389 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:34:53.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:53 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1812804024' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T07:34:53.767 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:53.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:53 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1812804024' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T07:34:53.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:53 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1812804024' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T07:34:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.345+0000 7f8f52c00640 1 -- 172.21.15.46:0/3738702880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c102790 msgr2=0x7f8f4c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.345+0000 7f8f52c00640 1 --2- 172.21.15.46:0/3738702880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c102790 0x7f8f4c1067e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009900 tx=0x7f8f3402a410 comp rx=0 tx=0).stop 2024-09-10T07:34:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.345+0000 7f8f52c00640 1 -- 172.21.15.46:0/3738702880 shutdown_connections 2024-09-10T07:34:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.345+0000 7f8f52c00640 1 --2- 172.21.15.46:0/3738702880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f4c106f10 0x7f8f4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.345+0000 7f8f52c00640 1 --2- 172.21.15.46:0/3738702880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c102790 0x7f8f4c1067e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.345+0000 7f8f52c00640 1 --2- 172.21.15.46:0/3738702880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f4c101de0 0x7f8f4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.345+0000 7f8f52c00640 1 -- 172.21.15.46:0/3738702880 >> 172.21.15.46:0/3738702880 conn(0x7f8f4c0fd8c0 msgr2=0x7f8f4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.346+0000 7f8f52c00640 1 -- 172.21.15.46:0/3738702880 shutdown_connections 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.346+0000 7f8f52c00640 1 -- 172.21.15.46:0/3738702880 wait complete. 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.346+0000 7f8f52c00640 1 Processor -- start 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f52c00640 1 -- start start 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f52c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c101de0 0x7f8f4c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f52c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f4c102790 0x7f8f4c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f52c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f4c106f10 0x7f8f4c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f52c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c111fc0 con 0x7f8f4c101de0 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f52c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c111e40 con 0x7f8f4c106f10 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f52c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c112140 con 0x7f8f4c102790 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f4be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c101de0 0x7f8f4c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f4be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c101de0 0x7f8f4c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57882/0 (socket says 172.21.15.46:57882) 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.347+0000 7f8f4be00640 1 -- 172.21.15.46:0/1932346193 learned_addr learned my addr 172.21.15.46:0/1932346193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f50c00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f4c106f10 0x7f8f4c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f43400640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f4c102790 0x7f8f4c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f4be00640 1 -- 172.21.15.46:0/1932346193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f4c102790 msgr2=0x7f8f4c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f4be00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f4c102790 0x7f8f4c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f4be00640 1 -- 172.21.15.46:0/1932346193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f4c106f10 msgr2=0x7f8f4c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f4be00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f4c106f10 0x7f8f4c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f4be00640 1 -- 172.21.15.46:0/1932346193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c19f690 con 0x7f8f4c101de0 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f43400640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f4c102790 0x7f8f4c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f4be00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c101de0 0x7f8f4c19a6e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00d950 tx=0x7f8f3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f49600640 1 -- 172.21.15.46:0/1932346193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c017070 con 0x7f8f4c101de0 2024-09-10T07:34:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c19f980 con 0x7f8f4c101de0 2024-09-10T07:34:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.348+0000 7f8f49600640 1 -- 172.21.15.46:0/1932346193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c009c30 con 0x7f8f4c101de0 2024-09-10T07:34:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.349+0000 7f8f49600640 1 -- 172.21.15.46:0/1932346193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c020060 con 0x7f8f4c101de0 2024-09-10T07:34:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.349+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c1a7260 con 0x7f8f4c101de0 2024-09-10T07:34:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.350+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4c1021c0 con 0x7f8f4c101de0 2024-09-10T07:34:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.352+0000 7f8f49600640 1 -- 172.21.15.46:0/1932346193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8f3c0109d0 con 0x7f8f4c101de0 2024-09-10T07:34:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.353+0000 7f8f49600640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f300a2c10 0x7f8f300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.353+0000 7f8f49600640 1 -- 172.21.15.46:0/1932346193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f8f3c0cf240 con 0x7f8f4c101de0 2024-09-10T07:34:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.353+0000 7f8f43400640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f300a2c10 0x7f8f300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.354+0000 7f8f43400640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f300a2c10 0x7f8f300a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f34000c00 tx=0x7f8f34005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.357+0000 7f8f49600640 1 -- 172.21.15.46:0/1932346193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f3c009de0 con 0x7f8f4c101de0 2024-09-10T07:34:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.544+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f4c1a7430 con 0x7f8f4c101de0 2024-09-10T07:34:54.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:54 smithi046 ceph-mon[30705]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T07:34:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.546+0000 7f8f49600640 1 -- 172.21.15.46:0/1932346193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24597 (secure 0 0 0) 0x7f8f3c09bc80 con 0x7f8f4c101de0 2024-09-10T07:34:54.546 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:54.546 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:34:14.007613+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:34:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.548+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f300a2c10 msgr2=0x7f8f300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.548+0000 7f8f52c00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f300a2c10 0x7f8f300a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f34000c00 tx=0x7f8f34005a40 comp rx=0 tx=0).stop 2024-09-10T07:34:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.548+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c101de0 msgr2=0x7f8f4c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.548+0000 7f8f52c00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c101de0 0x7f8f4c19a6e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00d950 tx=0x7f8f3c00de10 comp rx=0 tx=0).stop 2024-09-10T07:34:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.549+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 shutdown_connections 2024-09-10T07:34:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.549+0000 7f8f52c00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f4c106f10 0x7f8f4c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.549+0000 7f8f52c00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f4c102790 0x7f8f4c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.549+0000 7f8f52c00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f300a2c10 0x7f8f300a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.549+0000 7f8f52c00640 1 --2- 172.21.15.46:0/1932346193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4c101de0 0x7f8f4c19a6e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.549+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 >> 172.21.15.46:0/1932346193 conn(0x7f8f4c0fd8c0 msgr2=0x7f8f4c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.549+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 shutdown_connections 2024-09-10T07:34:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:54.549+0000 7f8f52c00640 1 -- 172.21.15.46:0/1932346193 wait complete. 2024-09-10T07:34:54.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:54 smithi181 ceph-mon[35397]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T07:34:54.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:54 smithi063 ceph-mon[35020]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T07:34:55.451 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T07:34:55.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:34:55.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:55 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1932346193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:55.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:55 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1932346193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:55.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:55 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1932346193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:56.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.483+0000 7fc611600640 1 -- 172.21.15.46:0/535577357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 msgr2=0x7fc60c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:56.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.483+0000 7fc611600640 1 --2- 172.21.15.46:0/535577357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 0x7fc60c110ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc604009900 tx=0x7fc60402a400 comp rx=0 tx=0).stop 2024-09-10T07:34:56.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.483+0000 7fc611600640 1 -- 172.21.15.46:0/535577357 shutdown_connections 2024-09-10T07:34:56.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.483+0000 7fc611600640 1 --2- 172.21.15.46:0/535577357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 0x7fc60c110ca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.483+0000 7fc611600640 1 --2- 172.21.15.46:0/535577357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc60c1049a0 0x7fc60c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.483+0000 7fc611600640 1 --2- 172.21.15.46:0/535577357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60c103ff0 0x7fc60c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.483+0000 7fc611600640 1 -- 172.21.15.46:0/535577357 >> 172.21.15.46:0/535577357 conn(0x7fc60c0ffa90 msgr2=0x7fc60c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.484+0000 7fc611600640 1 -- 172.21.15.46:0/535577357 shutdown_connections 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.484+0000 7fc611600640 1 -- 172.21.15.46:0/535577357 wait complete. 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.484+0000 7fc611600640 1 Processor -- start 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc611600640 1 -- start start 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc611600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc60c103ff0 0x7fc60c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc611600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60c1049a0 0x7fc60c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc611600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 0x7fc60c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc611600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c114180 con 0x7fc60c109120 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc611600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c114000 con 0x7fc60c1049a0 2024-09-10T07:34:56.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc611600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c114300 con 0x7fc60c103ff0 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc602000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60c1049a0 0x7fc60c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc60b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 0x7fc60c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc602000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60c1049a0 0x7fc60c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38474/0 (socket says 172.21.15.46:38474) 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc602000640 1 -- 172.21.15.46:0/4000155161 learned_addr learned my addr 172.21.15.46:0/4000155161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.485+0000 7fc60aa00640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc60c103ff0 0x7fc60c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc602000640 1 -- 172.21.15.46:0/4000155161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc60c103ff0 msgr2=0x7fc60c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc602000640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc60c103ff0 0x7fc60c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc602000640 1 -- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 msgr2=0x7fc60c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc602000640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 0x7fc60c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc602000640 1 -- 172.21.15.46:0/4000155161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c19f570 con 0x7fc60c1049a0 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc60b400640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 0x7fc60c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc602000640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60c1049a0 0x7fc60c19ab00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400d950 tx=0x7fc5f400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:56.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc603e00640 1 -- 172.21.15.46:0/4000155161 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4017070 con 0x7fc60c1049a0 2024-09-10T07:34:56.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc603e00640 1 -- 172.21.15.46:0/4000155161 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f4009c30 con 0x7fc60c1049a0 2024-09-10T07:34:56.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc603e00640 1 -- 172.21.15.46:0/4000155161 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400f6b0 con 0x7fc60c1049a0 2024-09-10T07:34:56.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c19f860 con 0x7fc60c1049a0 2024-09-10T07:34:56.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.486+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c1a7140 con 0x7fc60c1049a0 2024-09-10T07:34:56.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.487+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c1043d0 con 0x7fc60c1049a0 2024-09-10T07:34:56.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.489+0000 7fc603e00640 1 -- 172.21.15.46:0/4000155161 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc5f40109d0 con 0x7fc60c1049a0 2024-09-10T07:34:56.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.491+0000 7fc603e00640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5f00a2c10 0x7fc5f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:34:56.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.491+0000 7fc603e00640 1 -- 172.21.15.46:0/4000155161 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fc5f40cf0e0 con 0x7fc60c1049a0 2024-09-10T07:34:56.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.491+0000 7fc60aa00640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5f00a2c10 0x7fc5f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:34:56.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.492+0000 7fc60aa00640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5f00a2c10 0x7fc5f00a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005fd0 tx=0x7fc5fc009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:34:56.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.495+0000 7fc603e00640 1 -- 172.21.15.46:0/4000155161 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5f4009de0 con 0x7fc60c1049a0 2024-09-10T07:34:56.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.682+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc60c1a73a0 con 0x7fc60c1049a0 2024-09-10T07:34:56.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.684+0000 7fc603e00640 1 -- 172.21.15.46:0/4000155161 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24597 (secure 0 0 0) 0x7fc5f409b4d0 con 0x7fc60c1049a0 2024-09-10T07:34:56.684 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:34:56.684 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:34:14.007613+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:34:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.686+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5f00a2c10 msgr2=0x7fc5f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.686+0000 7fc611600640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5f00a2c10 0x7fc5f00a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005fd0 tx=0x7fc5fc009680 comp rx=0 tx=0).stop 2024-09-10T07:34:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.686+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60c1049a0 msgr2=0x7fc60c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:34:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.686+0000 7fc611600640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60c1049a0 0x7fc60c19ab00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400d950 tx=0x7fc5f400de10 comp rx=0 tx=0).stop 2024-09-10T07:34:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.687+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 shutdown_connections 2024-09-10T07:34:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.687+0000 7fc611600640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc5f00a2c10 0x7fc5f00a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.687+0000 7fc611600640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc60c109120 0x7fc60c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.687+0000 7fc611600640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc60c1049a0 0x7fc60c19ab00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.687+0000 7fc611600640 1 --2- 172.21.15.46:0/4000155161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc60c103ff0 0x7fc60c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:34:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.687+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 >> 172.21.15.46:0/4000155161 conn(0x7fc60c0ffa90 msgr2=0x7fc60c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:34:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.687+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 shutdown_connections 2024-09-10T07:34:56.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:56.687+0000 7fc611600640 1 -- 172.21.15.46:0/4000155161 wait complete. 2024-09-10T07:34:56.801 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:56 smithi063 ceph-mon[35020]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-10T07:34:56.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:56 smithi181 ceph-mon[35397]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-10T07:34:56.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:56 smithi046 ceph-mon[30705]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-10T07:34:57.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T07:34:57.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T07:34:57.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T07:34:57.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T07:34:57.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T07:34:57.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T07:34:57.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T07:34:57.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T07:34:57.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T07:34:57.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T07:34:57.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T07:34:57.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T07:34:57.727 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:57 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/4000155161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:57.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:57 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/4000155161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:57.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:57 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/4000155161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:34:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.876 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.885 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:34:58.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:58 smithi181 ceph-mon[35397]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T07:34:58.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:58 smithi063 ceph-mon[35020]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T07:34:58.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:58 smithi046 ceph-mon[30705]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T07:34:59.501 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:34:59 smithi046 ceph-mon[30705]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T07:34:59.800 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:34:59 smithi181 ceph-mon[35397]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T07:34:59.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:34:59 smithi063 ceph-mon[35020]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T07:35:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.890+0000 7f8ce6a00640 1 -- 172.21.15.46:0/437459275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074620 msgr2=0x7f8ce0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.890+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/437459275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074620 0x7f8ce0074a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009900 tx=0x7f8cd402a480 comp rx=0 tx=0).stop 2024-09-10T07:35:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.890+0000 7f8ce6a00640 1 -- 172.21.15.46:0/437459275 shutdown_connections 2024-09-10T07:35:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.890+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/437459275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce010e210 0x7f8ce0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.890+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/437459275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce0074f40 0x7f8ce010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.890+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/437459275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074620 0x7f8ce0074a00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.890+0000 7f8ce6a00640 1 -- 172.21.15.46:0/437459275 >> 172.21.15.46:0/437459275 conn(0x7f8ce00fd710 msgr2=0x7f8ce00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.891+0000 7f8ce6a00640 1 -- 172.21.15.46:0/437459275 shutdown_connections 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.891+0000 7f8ce6a00640 1 -- 172.21.15.46:0/437459275 wait complete. 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.892+0000 7f8ce6a00640 1 Processor -- start 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.892+0000 7f8ce6a00640 1 -- start start 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce0074620 0x7f8ce019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074f40 0x7f8ce019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce010e210 0x7f8ce019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce6a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce0114010 con 0x7f8ce010e210 2024-09-10T07:35:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce0113e90 con 0x7f8ce0074620 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce6a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce0114190 con 0x7f8ce0074f40 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074f40 0x7f8ce019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074f40 0x7f8ce019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59778/0 (socket says 172.21.15.46:59778) 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce010e210 0x7f8ce019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce4c00640 1 -- 172.21.15.46:0/1884475799 learned_addr learned my addr 172.21.15.46:0/1884475799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.893+0000 7f8ce5600640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce0074620 0x7f8ce019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.894+0000 7f8ce4c00640 1 -- 172.21.15.46:0/1884475799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce0074620 msgr2=0x7f8ce019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.894+0000 7f8ce4c00640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce0074620 0x7f8ce019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.894+0000 7f8ce4c00640 1 -- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce010e210 msgr2=0x7f8ce019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.894+0000 7f8ce4c00640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce010e210 0x7f8ce019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.894+0000 7f8ce4c00640 1 -- 172.21.15.46:0/1884475799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce019f500 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.894+0000 7f8ce5600640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce0074620 0x7f8ce019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.894+0000 7f8ce4c00640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074f40 0x7f8ce019aa90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e9f0 tx=0x7f8ccc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.894+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00cd80 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.895+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ccc005500 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.895+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc010690 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.895+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce019f7f0 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.895+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce01a70d0 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.896+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ce0062900 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.898+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8ccc0046c0 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.899+0000 7f8cde000640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8cc80a2c90 0x7f8cc80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.899+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f8ccc0cfb20 con 0x7f8ce0074f40 2024-09-10T07:35:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.899+0000 7f8ce5600640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8cc80a2c90 0x7f8cc80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:00.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.899+0000 7f8cde000640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] conn(0x7f8cc80ade60 0x7f8cc80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:00.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.899+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 --> [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ccc004470 con 0x7f8cc80ade60 2024-09-10T07:35:00.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.899+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f8ccc0cfed0 con 0x7f8ce0074f40 2024-09-10T07:35:00.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.900+0000 7f8ce6000640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] conn(0x7f8cc80ade60 0x7f8cc80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.900+0000 7f8ce5600640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8cc80a2c90 0x7f8cc80a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4000c00 tx=0x7f8cd4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.900+0000 7f8ce6000640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] conn(0x7f8cc80ade60 0x7f8cc80b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.902+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 <== osd.0 v2:172.21.15.46:6802/12452162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ccc004470 con 0x7f8cc80ade60 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.917+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 --> [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ce0062d80 con 0x7f8cc80ade60 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.918+0000 7f8cde000640 1 -- 172.21.15.46:0/1884475799 <== osd.0 v2:172.21.15.46:6802/12452162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8ce0062d80 con 0x7f8cc80ade60 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.918+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] conn(0x7f8cc80ade60 msgr2=0x7f8cc80b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.918+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] conn(0x7f8cc80ade60 0x7f8cc80b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8cc80a2c90 msgr2=0x7f8cc80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8cc80a2c90 0x7f8cc80a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4000c00 tx=0x7f8cd4031510 comp rx=0 tx=0).stop 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074f40 msgr2=0x7f8ce019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/1884475799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ce0074f40 0x7f8ce019aa90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e9f0 tx=0x7f8ccc00edf0 comp rx=0 tx=0).stop 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6000640 1 -- 172.21.15.46:0/1884475799 reap_dead start 2024-09-10T07:35:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 shutdown_connections 2024-09-10T07:35:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 >> 172.21.15.46:0/1884475799 conn(0x7f8ce00fd710 msgr2=0x7f8ce00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 shutdown_connections 2024-09-10T07:35:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:34:59.919+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1884475799 wait complete. 2024-09-10T07:35:01.670 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:01 smithi046 ceph-mon[30705]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T07:35:01.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:01 smithi181 ceph-mon[35397]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T07:35:01.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:01 smithi063 ceph-mon[35020]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T07:35:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 -- 172.21.15.46:0/1127903136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 msgr2=0x7f4064110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 --2- 172.21.15.46:0/1127903136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 0x7f4064110ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f405c009ca0 tx=0x7f405c02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 -- 172.21.15.46:0/1127903136 shutdown_connections 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 --2- 172.21.15.46:0/1127903136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 0x7f4064110ae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 --2- 172.21.15.46:0/1127903136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40641047e0 0x7f4064108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 --2- 172.21.15.46:0/1127903136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4064103e30 0x7f4064104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 -- 172.21.15.46:0/1127903136 >> 172.21.15.46:0/1127903136 conn(0x7f40640ff8d0 msgr2=0x7f4064101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 -- 172.21.15.46:0/1127903136 shutdown_connections 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.531+0000 7f406ae00640 1 -- 172.21.15.46:0/1127903136 wait complete. 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.532+0000 7f406ae00640 1 Processor -- start 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.532+0000 7f406ae00640 1 -- start start 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.532+0000 7f406ae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4064103e30 0x7f406419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.532+0000 7f406ae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40641047e0 0x7f406419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f406ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 0x7f406419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f406ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4064113fc0 con 0x7f4064108f60 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f406ae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4064113e40 con 0x7f4064103e30 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f406ae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4064114140 con 0x7f40641047e0 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f406a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 0x7f406419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f4069a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4064103e30 0x7f406419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f406a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 0x7f406419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53600/0 (socket says 172.21.15.46:53600) 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f406a400640 1 -- 172.21.15.46:0/462251048 learned_addr learned my addr 172.21.15.46:0/462251048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f4069000640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40641047e0 0x7f406419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f4069a00640 1 -- 172.21.15.46:0/462251048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40641047e0 msgr2=0x7f406419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f4069a00640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40641047e0 0x7f406419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f4069a00640 1 -- 172.21.15.46:0/462251048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 msgr2=0x7f406419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f4069a00640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 0x7f406419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f4069a00640 1 -- 172.21.15.46:0/462251048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406419f3f0 con 0x7f4064103e30 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.533+0000 7f406a400640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064108f60 0x7f406419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.534+0000 7f4069a00640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4064103e30 0x7f406419a440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f405800e9f0 tx=0x7f405800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.538+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405800ccc0 con 0x7f4064103e30 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.538+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4058010910 con 0x7f4064103e30 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.538+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406419f6e0 con 0x7f4064103e30 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.538+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058018960 con 0x7f4064103e30 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.538+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40641a6fc0 con 0x7f4064103e30 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.539+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4064062980 con 0x7f4064103e30 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.541+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4058010ac0 con 0x7f4064103e30 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.542+0000 7f4062000640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f404c0a2c90 0x7f404c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.542+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f40580cfd90 con 0x7f4064103e30 2024-09-10T07:35:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.542+0000 7f4069000640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f404c0a2c90 0x7f404c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.542+0000 7f4062000640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] conn(0x7f404c0ade60 0x7f404c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.542+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 --> [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f404c0b09d0 con 0x7f404c0ade60 2024-09-10T07:35:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.542+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f40580d0140 con 0x7f4064103e30 2024-09-10T07:35:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.543+0000 7f406a400640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] conn(0x7f404c0ade60 0x7f404c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.543+0000 7f4069000640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f404c0a2c90 0x7f404c0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f406410a930 tx=0x7f4050005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.543+0000 7f406a400640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] conn(0x7f404c0ade60 0x7f404c0b0300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.544+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 <== osd.10 v2:172.21.15.63:6824/2494224828 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f404c0b09d0 con 0x7f404c0ade60 2024-09-10T07:35:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.561+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 --> [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4064062d80 con 0x7f404c0ade60 2024-09-10T07:35:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f4062000640 1 -- 172.21.15.46:0/462251048 <== osd.10 v2:172.21.15.63:6824/2494224828 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4064062d80 con 0x7f404c0ade60 2024-09-10T07:35:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 >> [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] conn(0x7f404c0ade60 msgr2=0x7f404c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] conn(0x7f404c0ade60 0x7f404c0b0300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f404c0a2c90 msgr2=0x7f404c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f404c0a2c90 0x7f404c0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f406410a930 tx=0x7f4050005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4064103e30 msgr2=0x7f406419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 --2- 172.21.15.46:0/462251048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4064103e30 0x7f406419a440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f405800e9f0 tx=0x7f405800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406a400640 1 -- 172.21.15.46:0/462251048 reap_dead start 2024-09-10T07:35:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 shutdown_connections 2024-09-10T07:35:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 >> 172.21.15.46:0/462251048 conn(0x7f40640ff8d0 msgr2=0x7f4064100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.562+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 shutdown_connections 2024-09-10T07:35:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.563+0000 7f406ae00640 1 -- 172.21.15.46:0/462251048 wait complete. 2024-09-10T07:35:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.846+0000 7f494a000640 1 -- 172.21.15.46:0/1009711200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944112a50 msgr2=0x7f4944114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.846+0000 7f494a000640 1 --2- 172.21.15.46:0/1009711200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944112a50 0x7f4944114e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f493c009be0 tx=0x7f493c02a6c0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.846+0000 7f494a000640 1 -- 172.21.15.46:0/1009711200 shutdown_connections 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.846+0000 7f494a000640 1 --2- 172.21.15.46:0/1009711200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944112a50 0x7f4944114e40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.846+0000 7f494a000640 1 --2- 172.21.15.46:0/1009711200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4944077040 0x7f4944112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.846+0000 7f494a000640 1 --2- 172.21.15.46:0/1009711200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4944076720 0x7f4944076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.846+0000 7f494a000640 1 -- 172.21.15.46:0/1009711200 >> 172.21.15.46:0/1009711200 conn(0x7f49440ffa50 msgr2=0x7f4944101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.847+0000 7f494a000640 1 -- 172.21.15.46:0/1009711200 shutdown_connections 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.847+0000 7f494a000640 1 -- 172.21.15.46:0/1009711200 wait complete. 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.847+0000 7f494a000640 1 Processor -- start 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f494a000640 1 -- start start 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f494a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944076720 0x7f49441a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f494a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4944077040 0x7f49441a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f494a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4944112a50 0x7f494419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f494a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49441185a0 con 0x7f4944112a50 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f494a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4944118420 con 0x7f4944077040 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f494a000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4944118720 con 0x7f4944076720 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f4943e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4944112a50 0x7f494419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f4943400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944076720 0x7f49441a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f4943e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4944112a50 0x7f494419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53574/0 (socket says 172.21.15.46:53574) 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f4942a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4944077040 0x7f49441a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f4943e00640 1 -- 172.21.15.46:0/2718400344 learned_addr learned my addr 172.21.15.46:0/2718400344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.848+0000 7f4943400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944076720 0x7f49441a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59820/0 (socket says 172.21.15.46:59820) 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f4943400640 1 -- 172.21.15.46:0/2718400344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4944077040 msgr2=0x7f49441a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f4943400640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4944077040 0x7f49441a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f4943400640 1 -- 172.21.15.46:0/2718400344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4944112a50 msgr2=0x7f494419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f4943400640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4944112a50 0x7f494419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f4943400640 1 -- 172.21.15.46:0/2718400344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494419efe0 con 0x7f4944076720 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f4943400640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944076720 0x7f49441a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4934009fd0 tx=0x7f493400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493400ccc0 con 0x7f4944076720 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494419f2d0 con 0x7f4944076720 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.849+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f493400ee70 con 0x7f4944076720 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.850+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934004a00 con 0x7f4944076720 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.850+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49441ab5c0 con 0x7f4944076720 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.850+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4944114ac0 con 0x7f4944076720 2024-09-10T07:35:02.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.854+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4934004ba0 con 0x7f4944076720 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.856+0000 7f493be00640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49280a2c90 0x7f49280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.856+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f49340cf5a0 con 0x7f4944076720 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.857+0000 7f4942a00640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49280a2c90 0x7f49280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.857+0000 7f493be00640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] conn(0x7f49280ade60 0x7f49280b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.857+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 --> [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4934043e50 con 0x7f49280ade60 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.857+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f49340cf950 con 0x7f4944076720 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.857+0000 7f4943e00640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] conn(0x7f49280ade60 0x7f49280b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.857+0000 7f4942a00640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49280a2c90 0x7f49280a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4944079290 tx=0x7f4930005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.858+0000 7f4943e00640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] conn(0x7f49280ade60 0x7f49280b0300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.859+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 <== osd.2 v2:172.21.15.181:6800/413292109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4934043e50 con 0x7f49280ade60 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.873+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 --> [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4944077e60 con 0x7f49280ade60 2024-09-10T07:35:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f493be00640 1 -- 172.21.15.46:0/2718400344 <== osd.2 v2:172.21.15.181:6800/413292109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4944077e60 con 0x7f49280ade60 2024-09-10T07:35:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 >> [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] conn(0x7f49280ade60 msgr2=0x7f49280b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] conn(0x7f49280ade60 0x7f49280b0300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49280a2c90 msgr2=0x7f49280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49280a2c90 0x7f49280a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4944079290 tx=0x7f4930005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944076720 msgr2=0x7f49441a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 --2- 172.21.15.46:0/2718400344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4944076720 0x7f49441a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4934009fd0 tx=0x7f493400ec20 comp rx=0 tx=0).stop 2024-09-10T07:35:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f4943e00640 1 -- 172.21.15.46:0/2718400344 reap_dead start 2024-09-10T07:35:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 shutdown_connections 2024-09-10T07:35:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 >> 172.21.15.46:0/2718400344 conn(0x7f49440ffa50 msgr2=0x7f4944101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.874+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 shutdown_connections 2024-09-10T07:35:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.875+0000 7f494a000640 1 -- 172.21.15.46:0/2718400344 wait complete. 2024-09-10T07:35:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 -- 172.21.15.46:0/4191582117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd490104800 msgr2=0x7fd490108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 --2- 172.21.15.46:0/4191582117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd490104800 0x7fd490108960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd478009900 tx=0x7fd47802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 -- 172.21.15.46:0/4191582117 shutdown_connections 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 --2- 172.21.15.46:0/4191582117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd490109090 0x7fd490110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 --2- 172.21.15.46:0/4191582117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd490104800 0x7fd490108960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 --2- 172.21.15.46:0/4191582117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd490103e50 0x7fd490104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 -- 172.21.15.46:0/4191582117 >> 172.21.15.46:0/4191582117 conn(0x7fd4900ff8b0 msgr2=0x7fd490101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 -- 172.21.15.46:0/4191582117 shutdown_connections 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.279+0000 7fd48f200640 1 -- 172.21.15.46:0/4191582117 wait complete. 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.280+0000 7fd48f200640 1 Processor -- start 2024-09-10T07:35:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.280+0000 7fd48f200640 1 -- start start 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd48f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd490103e50 0x7fd49019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd48f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd490104800 0x7fd49019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd48f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd490109090 0x7fd49019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd48f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd490114090 con 0x7fd490104800 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd48f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd490113f10 con 0x7fd490103e50 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd48f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd490114210 con 0x7fd490109090 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd490104800 0x7fd49019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd490104800 0x7fd49019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53628/0 (socket says 172.21.15.46:53628) 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd48de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd490103e50 0x7fd49019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd485400640 1 -- 172.21.15.46:0/4013288963 learned_addr learned my addr 172.21.15.46:0/4013288963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.281+0000 7fd48e800640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd490109090 0x7fd49019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd48de00640 1 -- 172.21.15.46:0/4013288963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd490109090 msgr2=0x7fd49019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:03.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd48de00640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd490109090 0x7fd49019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd48de00640 1 -- 172.21.15.46:0/4013288963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd490104800 msgr2=0x7fd49019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd48de00640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd490104800 0x7fd49019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd48de00640 1 -- 172.21.15.46:0/4013288963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49019f520 con 0x7fd490103e50 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd48e800640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd490109090 0x7fd49019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd48de00640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd490103e50 0x7fd49019a570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd48000eb00 tx=0x7fd48000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48000cbe0 con 0x7fd490103e50 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd48000cd80 con 0x7fd490103e50 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49019f810 con 0x7fd490103e50 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.282+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48001e7a0 con 0x7fd490103e50 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.283+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd490076630 con 0x7fd490103e50 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.283+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd490062980 con 0x7fd490103e50 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.286+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd480026020 con 0x7fd490103e50 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.287+0000 7fd487400640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4740a2c10 0x7fd4740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.288+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fd4800d8300 con 0x7fd490103e50 2024-09-10T07:35:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.288+0000 7fd485400640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4740a2c10 0x7fd4740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.288+0000 7fd487400640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] conn(0x7fd4740adde0 0x7fd4740b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.288+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 --> [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd4800020b0 con 0x7fd4740adde0 2024-09-10T07:35:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.288+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fd4800d86b0 con 0x7fd490103e50 2024-09-10T07:35:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.288+0000 7fd48e800640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] conn(0x7fd4740adde0 0x7fd4740b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.289+0000 7fd485400640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4740a2c10 0x7fd4740a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd478031550 tx=0x7fd478005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.289+0000 7fd48e800640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] conn(0x7fd4740adde0 0x7fd4740b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.290+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 <== osd.4 v2:172.21.15.63:6808/402938834 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd4800020b0 con 0x7fd4740adde0 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.304+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 --> [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd490062e00 con 0x7fd4740adde0 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.305+0000 7fd487400640 1 -- 172.21.15.46:0/4013288963 <== osd.4 v2:172.21.15.63:6808/402938834 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd490062e00 con 0x7fd4740adde0 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.305+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 >> [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] conn(0x7fd4740adde0 msgr2=0x7fd4740b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.305+0000 7fd48f200640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] conn(0x7fd4740adde0 0x7fd4740b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.305+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4740a2c10 msgr2=0x7fd4740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.305+0000 7fd48f200640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4740a2c10 0x7fd4740a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd478031550 tx=0x7fd478005a20 comp rx=0 tx=0).stop 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.305+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd490103e50 msgr2=0x7fd49019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.305+0000 7fd48f200640 1 --2- 172.21.15.46:0/4013288963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd490103e50 0x7fd49019a570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd48000eb00 tx=0x7fd48000c2d0 comp rx=0 tx=0).stop 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.305+0000 7fd48e800640 1 -- 172.21.15.46:0/4013288963 reap_dead start 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.306+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 shutdown_connections 2024-09-10T07:35:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.306+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 >> 172.21.15.46:0/4013288963 conn(0x7fd4900ff8b0 msgr2=0x7fd490100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.306+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 shutdown_connections 2024-09-10T07:35:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:03.306+0000 7fd48f200640 1 -- 172.21.15.46:0/4013288963 wait complete. 2024-09-10T07:35:03.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:03 smithi181 ceph-mon[35397]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:03.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:03 smithi063 ceph-mon[35020]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:03.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:03 smithi046 ceph-mon[30705]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:04.349 INFO:teuthology.orchestra.run.smithi046.stdout:34359738457 2024-09-10T07:35:04.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T07:35:04.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.315+0000 7f345be00640 1 -- 172.21.15.46:0/3224352035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c074620 msgr2=0x7f345c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:04.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.315+0000 7f345be00640 1 --2- 172.21.15.46:0/3224352035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c074620 0x7f345c074a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f344c009900 tx=0x7f344c02a480 comp rx=0 tx=0).stop 2024-09-10T07:35:04.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.316+0000 7f345be00640 1 -- 172.21.15.46:0/3224352035 shutdown_connections 2024-09-10T07:35:04.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.316+0000 7f345be00640 1 --2- 172.21.15.46:0/3224352035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345c10e210 0x7f345c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:04.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.316+0000 7f345be00640 1 --2- 172.21.15.46:0/3224352035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f345c074f40 0x7f345c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:04.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.316+0000 7f345be00640 1 --2- 172.21.15.46:0/3224352035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c074620 0x7f345c074a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:04.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.316+0000 7f345be00640 1 -- 172.21.15.46:0/3224352035 >> 172.21.15.46:0/3224352035 conn(0x7f345c0fd710 msgr2=0x7f345c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.317+0000 7f345be00640 1 -- 172.21.15.46:0/3224352035 shutdown_connections 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.317+0000 7f345be00640 1 -- 172.21.15.46:0/3224352035 wait complete. 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.318+0000 7f345be00640 1 Processor -- start 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.318+0000 7f345be00640 1 -- start start 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345c074620 0x7f345c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f345c074f40 0x7f345c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c10e210 0x7f345c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f345c113fc0 con 0x7f345c10e210 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f345c113e40 con 0x7f345c074620 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f345c114140 con 0x7f345c074f40 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c10e210 0x7f345c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c10e210 0x7f345c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53650/0 (socket says 172.21.15.46:53650) 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345b400640 1 -- 172.21.15.46:0/2003351217 learned_addr learned my addr 172.21.15.46:0/2003351217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345a000640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f345c074f40 0x7f345c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.319+0000 7f345aa00640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345c074620 0x7f345c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.320+0000 7f345b400640 1 -- 172.21.15.46:0/2003351217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f345c074f40 msgr2=0x7f345c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.320+0000 7f345b400640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f345c074f40 0x7f345c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.320+0000 7f345b400640 1 -- 172.21.15.46:0/2003351217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345c074620 msgr2=0x7f345c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.320+0000 7f345b400640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345c074620 0x7f345c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.320+0000 7f345b400640 1 -- 172.21.15.46:0/2003351217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c19f450 con 0x7f345c10e210 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.320+0000 7f345aa00640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345c074620 0x7f345c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.320+0000 7f345b400640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c10e210 0x7f345c19ed70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f345400eb10 tx=0x7f345400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.321+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454004b20 con 0x7f345c10e210 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.321+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3454004cc0 con 0x7f345c10e210 2024-09-10T07:35:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.321+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c19f740 con 0x7f345c10e210 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.321+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454010760 con 0x7f345c10e210 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.321+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c1a7020 con 0x7f345c10e210 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.322+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f345c062980 con 0x7f345c10e210 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.324+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3454005020 con 0x7f345c10e210 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.325+0000 7f3453400640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34400a2c90 0x7f34400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.326+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f34540cfe40 con 0x7f345c10e210 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.326+0000 7f345aa00640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34400a2c90 0x7f34400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.326+0000 7f3453400640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] conn(0x7f34400ade60 0x7f34400b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.326+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 --> [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34400b09d0 con 0x7f34400ade60 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.326+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f34540d01f0 con 0x7f345c10e210 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.326+0000 7f345a000640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] conn(0x7f34400ade60 0x7f34400b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.327+0000 7f345aa00640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34400a2c90 0x7f34400a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f344c009230 tx=0x7f344c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.327+0000 7f345a000640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] conn(0x7f34400ade60 0x7f34400b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.328+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 <== osd.9 v2:172.21.15.46:6826/805007364 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34400b09d0 con 0x7f34400ade60 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.343+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 --> [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f345c062e00 con 0x7f34400ade60 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.343+0000 7f3453400640 1 -- 172.21.15.46:0/2003351217 <== osd.9 v2:172.21.15.46:6826/805007364 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f345c062e00 con 0x7f34400ade60 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] conn(0x7f34400ade60 msgr2=0x7f34400b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] conn(0x7f34400ade60 0x7f34400b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34400a2c90 msgr2=0x7f34400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f34400a2c90 0x7f34400a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f344c009230 tx=0x7f344c0058d0 comp rx=0 tx=0).stop 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c10e210 msgr2=0x7f345c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 --2- 172.21.15.46:0/2003351217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f345c10e210 0x7f345c19ed70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f345400eb10 tx=0x7f345400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:35:04.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345b400640 1 -- 172.21.15.46:0/2003351217 reap_dead start 2024-09-10T07:35:04.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 shutdown_connections 2024-09-10T07:35:04.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 >> 172.21.15.46:0/2003351217 conn(0x7f345c0fd710 msgr2=0x7f345c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:04.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.344+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 shutdown_connections 2024-09-10T07:35:04.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:04.345+0000 7f345be00640 1 -- 172.21.15.46:0/2003351217 wait complete. 2024-09-10T07:35:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 -- 172.21.15.46:0/1999504162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530102790 msgr2=0x7ff5301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 --2- 172.21.15.46:0/1999504162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530102790 0x7ff5301067e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009860 tx=0x7ff51c02a340 comp rx=0 tx=0).stop 2024-09-10T07:35:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 -- 172.21.15.46:0/1999504162 shutdown_connections 2024-09-10T07:35:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 --2- 172.21.15.46:0/1999504162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff530106f10 0x7ff53010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 --2- 172.21.15.46:0/1999504162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530102790 0x7ff5301067e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 --2- 172.21.15.46:0/1999504162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff530101de0 0x7ff5301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 -- 172.21.15.46:0/1999504162 >> 172.21.15.46:0/1999504162 conn(0x7ff5300fd8c0 msgr2=0x7ff5300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 -- 172.21.15.46:0/1999504162 shutdown_connections 2024-09-10T07:35:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.664+0000 7ff537e00640 1 -- 172.21.15.46:0/1999504162 wait complete. 2024-09-10T07:35:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.665+0000 7ff537e00640 1 Processor -- start 2024-09-10T07:35:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.665+0000 7ff537e00640 1 -- start start 2024-09-10T07:35:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.665+0000 7ff537e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff530101de0 0x7ff53019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.665+0000 7ff537e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530102790 0x7ff53019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.665+0000 7ff537e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff530106f10 0x7ff53019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.665+0000 7ff537e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff530111f70 con 0x7ff530101de0 2024-09-10T07:35:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.665+0000 7ff537e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff530111df0 con 0x7ff530106f10 2024-09-10T07:35:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.665+0000 7ff537e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301120f0 con 0x7ff530102790 2024-09-10T07:35:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff530101de0 0x7ff53019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff530101de0 0x7ff53019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53612/0 (socket says 172.21.15.46:53612) 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 -- 172.21.15.46:0/2104201993 learned_addr learned my addr 172.21.15.46:0/2104201993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff534a00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530102790 0x7ff53019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535e00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff530106f10 0x7ff53019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 -- 172.21.15.46:0/2104201993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530102790 msgr2=0x7ff53019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530102790 0x7ff53019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 -- 172.21.15.46:0/2104201993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff530106f10 msgr2=0x7ff53019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff530106f10 0x7ff53019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 -- 172.21.15.46:0/2104201993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53019f5e0 con 0x7ff530101de0 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.666+0000 7ff535400640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff530101de0 0x7ff53019a630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff52400e9c0 tx=0x7ff52400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.667+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52400cde0 con 0x7ff530101de0 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.667+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53019f8d0 con 0x7ff530101de0 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.667+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5240035f0 con 0x7ff530101de0 2024-09-10T07:35:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.667+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524010650 con 0x7ff530101de0 2024-09-10T07:35:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.667+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5301a71b0 con 0x7ff530101de0 2024-09-10T07:35:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.668+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5300fef90 con 0x7ff530101de0 2024-09-10T07:35:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.670+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff5240037a0 con 0x7ff530101de0 2024-09-10T07:35:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.671+0000 7ff52e000640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5180a2c90 0x7ff5180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.672+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7ff5240cfae0 con 0x7ff530101de0 2024-09-10T07:35:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.672+0000 7ff534a00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5180a2c90 0x7ff5180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.672+0000 7ff52e000640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] conn(0x7ff5180ade60 0x7ff5180b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.672+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 --> [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff524024ea0 con 0x7ff5180ade60 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.672+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7ff5240cfe90 con 0x7ff530101de0 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.672+0000 7ff535e00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] conn(0x7ff5180ade60 0x7ff5180b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.672+0000 7ff534a00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5180a2c90 0x7ff5180a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff51c02a930 tx=0x7ff51c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.673+0000 7ff535e00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] conn(0x7ff5180ade60 0x7ff5180b0300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.674+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 <== osd.6 v2:172.21.15.46:6818/4145991775 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff524024ea0 con 0x7ff5180ade60 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 --> [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff5301032c0 con 0x7ff5180ade60 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff52e000640 1 -- 172.21.15.46:0/2104201993 <== osd.6 v2:172.21.15.46:6818/4145991775 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff5301032c0 con 0x7ff5180ade60 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] conn(0x7ff5180ade60 msgr2=0x7ff5180b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff537e00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] conn(0x7ff5180ade60 0x7ff5180b0300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5180a2c90 msgr2=0x7ff5180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff537e00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5180a2c90 0x7ff5180a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff51c02a930 tx=0x7ff51c0058d0 comp rx=0 tx=0).stop 2024-09-10T07:35:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff530101de0 msgr2=0x7ff53019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff537e00640 1 --2- 172.21.15.46:0/2104201993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff530101de0 0x7ff53019a630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff52400e9c0 tx=0x7ff52400ee80 comp rx=0 tx=0).stop 2024-09-10T07:35:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.689+0000 7ff535e00640 1 -- 172.21.15.46:0/2104201993 reap_dead start 2024-09-10T07:35:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.690+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 shutdown_connections 2024-09-10T07:35:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.690+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 >> 172.21.15.46:0/2104201993 conn(0x7ff5300fd8c0 msgr2=0x7ff5300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.690+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 shutdown_connections 2024-09-10T07:35:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:01.690+0000 7ff537e00640 1 -- 172.21.15.46:0/2104201993 wait complete. 2024-09-10T07:35:05.850 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:05 smithi046 ceph-mon[30705]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:05.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:05 smithi181 ceph-mon[35397]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:05.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:05 smithi063 ceph-mon[35020]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.200+0000 7fad15800640 1 -- 172.21.15.46:0/3879252633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad101079c0 msgr2=0x7fad10107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.200+0000 7fad15800640 1 --2- 172.21.15.46:0/3879252633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad101079c0 0x7fad10107da0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fad00009930 tx=0x7fad0002a420 comp rx=0 tx=0).stop 2024-09-10T07:35:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.201+0000 7fad15800640 1 -- 172.21.15.46:0/3879252633 shutdown_connections 2024-09-10T07:35:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.201+0000 7fad15800640 1 --2- 172.21.15.46:0/3879252633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad10102360 0x7fad1010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.201+0000 7fad15800640 1 --2- 172.21.15.46:0/3879252633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101019c0 0x7fad10101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.201+0000 7fad15800640 1 --2- 172.21.15.46:0/3879252633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad101079c0 0x7fad10107da0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.201+0000 7fad15800640 1 -- 172.21.15.46:0/3879252633 >> 172.21.15.46:0/3879252633 conn(0x7fad100fd740 msgr2=0x7fad100ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.201+0000 7fad15800640 1 -- 172.21.15.46:0/3879252633 shutdown_connections 2024-09-10T07:35:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.201+0000 7fad15800640 1 -- 172.21.15.46:0/3879252633 wait complete. 2024-09-10T07:35:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.202+0000 7fad15800640 1 Processor -- start 2024-09-10T07:35:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.202+0000 7fad15800640 1 -- start start 2024-09-10T07:35:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad15800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad101019c0 0x7fad1019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad15800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad10102360 0x7fad1019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad15800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101079c0 0x7fad1019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad15800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad10111dd0 con 0x7fad10102360 2024-09-10T07:35:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad15800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad10111c50 con 0x7fad101079c0 2024-09-10T07:35:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad15800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad10111f50 con 0x7fad101019c0 2024-09-10T07:35:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad10102360 0x7fad1019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad10102360 0x7fad1019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53672/0 (socket says 172.21.15.46:53672) 2024-09-10T07:35:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad07e00640 1 -- 172.21.15.46:0/2839483287 learned_addr learned my addr 172.21.15.46:0/2839483287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad0fe00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad101019c0 0x7fad1019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad14e00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101079c0 0x7fad1019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad07e00640 1 -- 172.21.15.46:0/2839483287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad101019c0 msgr2=0x7fad1019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad07e00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad101019c0 0x7fad1019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad07e00640 1 -- 172.21.15.46:0/2839483287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101079c0 msgr2=0x7fad1019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad07e00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101079c0 0x7fad1019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.203+0000 7fad07e00640 1 -- 172.21.15.46:0/2839483287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1019f4a0 con 0x7fad10102360 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.204+0000 7fad07e00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad10102360 0x7fad1019aa30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7facf800e9f0 tx=0x7facf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.204+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf800cd80 con 0x7fad10102360 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.204+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1019f790 con 0x7fad10102360 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.204+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf8010910 con 0x7fad10102360 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.204+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf80188f0 con 0x7fad10102360 2024-09-10T07:35:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.204+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a7070 con 0x7fad10102360 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.205+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad100ff040 con 0x7fad10102360 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.207+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7facf8010ac0 con 0x7fad10102360 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.208+0000 7fad0d600640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2c90 0x7facf40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.208+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7facf80cff20 con 0x7fad10102360 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.209+0000 7fad0fe00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2c90 0x7facf40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.209+0000 7fad0d600640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] conn(0x7facf40ade60 0x7facf40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.209+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 --> [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facf40b09d0 con 0x7facf40ade60 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.209+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7facf80d02d0 con 0x7fad10102360 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.209+0000 7fad14e00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] conn(0x7facf40ade60 0x7facf40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.209+0000 7fad0fe00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2c90 0x7facf40a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fad00006010 tx=0x7fad00000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.210+0000 7fad14e00640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] conn(0x7facf40ade60 0x7facf40b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.211+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 <== osd.7 v2:172.21.15.63:6816/1986919229 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf40b09d0 con 0x7facf40ade60 2024-09-10T07:35:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.225+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 --> [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad10103100 con 0x7facf40ade60 2024-09-10T07:35:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.226+0000 7fad0d600640 1 -- 172.21.15.46:0/2839483287 <== osd.7 v2:172.21.15.63:6816/1986919229 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fad10103100 con 0x7facf40ade60 2024-09-10T07:35:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.226+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 >> [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] conn(0x7facf40ade60 msgr2=0x7facf40b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.226+0000 7fad15800640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] conn(0x7facf40ade60 0x7facf40b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2c90 msgr2=0x7facf40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad15800640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2c90 0x7facf40a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fad00006010 tx=0x7fad00000bc0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad10102360 msgr2=0x7fad1019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad15800640 1 --2- 172.21.15.46:0/2839483287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad10102360 0x7fad1019aa30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7facf800e9f0 tx=0x7facf800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad14e00640 1 -- 172.21.15.46:0/2839483287 reap_dead start 2024-09-10T07:35:06.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 shutdown_connections 2024-09-10T07:35:06.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 >> 172.21.15.46:0/2839483287 conn(0x7fad100fd740 msgr2=0x7fad1010d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:06.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 shutdown_connections 2024-09-10T07:35:06.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.227+0000 7fad15800640 1 -- 172.21.15.46:0/2839483287 wait complete. 2024-09-10T07:35:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 -- 172.21.15.46:0/1067909240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feedc10e430 msgr2=0x7feedc110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 --2- 172.21.15.46:0/1067909240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feedc10e430 0x7feedc110820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feed4009ca0 tx=0x7feed402a950 comp rx=0 tx=0).stop 2024-09-10T07:35:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 -- 172.21.15.46:0/1067909240 shutdown_connections 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 --2- 172.21.15.46:0/1067909240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feedc10e430 0x7feedc110820 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 --2- 172.21.15.46:0/1067909240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc074ee0 0x7feedc10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 --2- 172.21.15.46:0/1067909240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feedc0745c0 0x7feedc0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 -- 172.21.15.46:0/1067909240 >> 172.21.15.46:0/1067909240 conn(0x7feedc0fd8b0 msgr2=0x7feedc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 -- 172.21.15.46:0/1067909240 shutdown_connections 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.716+0000 7feee3600640 1 -- 172.21.15.46:0/1067909240 wait complete. 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.717+0000 7feee3600640 1 Processor -- start 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.717+0000 7feee3600640 1 -- start start 2024-09-10T07:35:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feee3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feedc0745c0 0x7feedc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feee3600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc074ee0 0x7feedc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feee3600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feedc10e430 0x7feedc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feee3600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1141b0 con 0x7feedc0745c0 2024-09-10T07:35:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feee3600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feedc114030 con 0x7feedc10e430 2024-09-10T07:35:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feee3600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feedc114330 con 0x7feedc074ee0 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feedbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc074ee0 0x7feedc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feee0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feedc0745c0 0x7feedc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feedbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc074ee0 0x7feedc19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59800/0 (socket says 172.21.15.46:59800) 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feedbe00640 1 -- 172.21.15.46:0/1309208561 learned_addr learned my addr 172.21.15.46:0/1309208561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feee1600640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feedc10e430 0x7feedc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feedbe00640 1 -- 172.21.15.46:0/1309208561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feedc10e430 msgr2=0x7feedc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feedbe00640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feedc10e430 0x7feedc19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feedbe00640 1 -- 172.21.15.46:0/1309208561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feedc0745c0 msgr2=0x7feedc19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feedbe00640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feedc0745c0 0x7feedc19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.718+0000 7feedbe00640 1 -- 172.21.15.46:0/1309208561 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc19f600 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.719+0000 7feedbe00640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc074ee0 0x7feedc19ab90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feec800e9f0 tx=0x7feec800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.719+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec800cd80 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.719+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec8010910 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.719+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8018990 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.719+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc19f8f0 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.719+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1a71d0 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.720+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feedc062980 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.724+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feec8010ac0 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.725+0000 7feed9600640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feec40a2c90 0x7feec40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.725+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7feec80cf760 con 0x7feedc074ee0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.725+0000 7feee0c00640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feec40a2c90 0x7feec40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.725+0000 7feed9600640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] conn(0x7feec40ade60 0x7feec40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.725+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 --> [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feec8012ec0 con 0x7feec40ade60 2024-09-10T07:35:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.725+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7feec80d6050 con 0x7feedc074ee0 2024-09-10T07:35:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.726+0000 7feee1600640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] conn(0x7feec40ade60 0x7feec40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.726+0000 7feee0c00640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feec40a2c90 0x7feec40a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feed0009bd0 tx=0x7feed0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.726+0000 7feee1600640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] conn(0x7feec40ade60 0x7feec40b0300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.727+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 <== osd.5 v2:172.21.15.181:6808/120236836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feec8012ec0 con 0x7feec40ade60 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.744+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 --> [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feedc062dd0 con 0x7feec40ade60 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.744+0000 7feed9600640 1 -- 172.21.15.46:0/1309208561 <== osd.5 v2:172.21.15.181:6808/120236836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feedc062dd0 con 0x7feec40ade60 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 >> [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] conn(0x7feec40ade60 msgr2=0x7feec40b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] conn(0x7feec40ade60 0x7feec40b0300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feec40a2c90 msgr2=0x7feec40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feec40a2c90 0x7feec40a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feed0009bd0 tx=0x7feed0005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc074ee0 msgr2=0x7feedc19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 --2- 172.21.15.46:0/1309208561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc074ee0 0x7feedc19ab90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feec800e9f0 tx=0x7feec800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee1600640 1 -- 172.21.15.46:0/1309208561 reap_dead start 2024-09-10T07:35:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 shutdown_connections 2024-09-10T07:35:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 >> 172.21.15.46:0/1309208561 conn(0x7feedc0fd8b0 msgr2=0x7feedc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 shutdown_connections 2024-09-10T07:35:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:00.745+0000 7feee3600640 1 -- 172.21.15.46:0/1309208561 wait complete. 2024-09-10T07:35:07.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 -- 172.21.15.46:0/2166766699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 msgr2=0x7ff1141089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:07.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 --2- 172.21.15.46:0/2166766699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 0x7ff1141089f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff100009900 tx=0x7ff10002a410 comp rx=0 tx=0).stop 2024-09-10T07:35:07.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 -- 172.21.15.46:0/2166766699 shutdown_connections 2024-09-10T07:35:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 --2- 172.21.15.46:0/2166766699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff114109120 0x7ff114110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 --2- 172.21.15.46:0/2166766699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 0x7ff1141089f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 --2- 172.21.15.46:0/2166766699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff114103ff0 0x7ff1141043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 -- 172.21.15.46:0/2166766699 >> 172.21.15.46:0/2166766699 conn(0x7ff1140ffa90 msgr2=0x7ff114101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 -- 172.21.15.46:0/2166766699 shutdown_connections 2024-09-10T07:35:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.136+0000 7ff11a200640 1 -- 172.21.15.46:0/2166766699 wait complete. 2024-09-10T07:35:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.137+0000 7ff11a200640 1 Processor -- start 2024-09-10T07:35:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.137+0000 7ff11a200640 1 -- start start 2024-09-10T07:35:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.138+0000 7ff11a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff114103ff0 0x7ff11419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.138+0000 7ff11a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 0x7ff11419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.138+0000 7ff11a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff114109120 0x7ff11419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.138+0000 7ff11a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff114114120 con 0x7ff114103ff0 2024-09-10T07:35:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.138+0000 7ff11a200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff114113fa0 con 0x7ff114109120 2024-09-10T07:35:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.138+0000 7ff11a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141142a0 con 0x7ff1141049a0 2024-09-10T07:35:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.138+0000 7ff112a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 0x7ff11419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff112a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 0x7ff11419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49770/0 (socket says 172.21.15.46:49770) 2024-09-10T07:35:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff112a00640 1 -- 172.21.15.46:0/350706328 learned_addr learned my addr 172.21.15.46:0/350706328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff113400640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff114103ff0 0x7ff11419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff113e00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff114109120 0x7ff11419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff112a00640 1 -- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff114109120 msgr2=0x7ff11419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff112a00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff114109120 0x7ff11419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff112a00640 1 -- 172.21.15.46:0/350706328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff114103ff0 msgr2=0x7ff11419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff112a00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff114103ff0 0x7ff11419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff112a00640 1 -- 172.21.15.46:0/350706328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11419f5f0 con 0x7ff1141049a0 2024-09-10T07:35:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff113e00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff114109120 0x7ff11419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.139+0000 7ff112a00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 0x7ff11419ab80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff100006010 tx=0x7ff100004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.140+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10003f030 con 0x7ff1141049a0 2024-09-10T07:35:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.140+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff100004440 con 0x7ff1141049a0 2024-09-10T07:35:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.140+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11419f880 con 0x7ff1141049a0 2024-09-10T07:35:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.140+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100035070 con 0x7ff1141049a0 2024-09-10T07:35:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.140+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1141a71c0 con 0x7ff1141049a0 2024-09-10T07:35:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.141+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff114062980 con 0x7ff1141049a0 2024-09-10T07:35:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.143+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff100042030 con 0x7ff1141049a0 2024-09-10T07:35:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.144+0000 7ff10be00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff0f80a2c10 0x7ff0f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.144+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7ff1000f02a0 con 0x7ff1141049a0 2024-09-10T07:35:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.145+0000 7ff113400640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff0f80a2c10 0x7ff0f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.145+0000 7ff10be00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] conn(0x7ff0f80adde0 0x7ff0f80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.145+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 --> [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0f80b0950 con 0x7ff0f80adde0 2024-09-10T07:35:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.145+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7ff1000f0650 con 0x7ff1141049a0 2024-09-10T07:35:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.145+0000 7ff113e00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] conn(0x7ff0f80adde0 0x7ff0f80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.145+0000 7ff113400640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff0f80a2c10 0x7ff0f80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff104009b60 tx=0x7ff104005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.145+0000 7ff113e00640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] conn(0x7ff0f80adde0 0x7ff0f80b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.146+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 <== osd.1 v2:172.21.15.63:6800/1918625844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0f80b0950 con 0x7ff0f80adde0 2024-09-10T07:35:07.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.161+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 --> [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff114062e00 con 0x7ff0f80adde0 2024-09-10T07:35:07.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.161+0000 7ff10be00640 1 -- 172.21.15.46:0/350706328 <== osd.1 v2:172.21.15.63:6800/1918625844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff114062e00 con 0x7ff0f80adde0 2024-09-10T07:35:07.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] conn(0x7ff0f80adde0 msgr2=0x7ff0f80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] conn(0x7ff0f80adde0 0x7ff0f80b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff0f80a2c10 msgr2=0x7ff0f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff0f80a2c10 0x7ff0f80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff104009b60 tx=0x7ff104005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 msgr2=0x7ff11419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 --2- 172.21.15.46:0/350706328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1141049a0 0x7ff11419ab80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff100006010 tx=0x7ff100004040 comp rx=0 tx=0).stop 2024-09-10T07:35:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff113e00640 1 -- 172.21.15.46:0/350706328 reap_dead start 2024-09-10T07:35:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 shutdown_connections 2024-09-10T07:35:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 >> 172.21.15.46:0/350706328 conn(0x7ff1140ffa90 msgr2=0x7ff114100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 shutdown_connections 2024-09-10T07:35:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.162+0000 7ff11a200640 1 -- 172.21.15.46:0/350706328 wait complete. 2024-09-10T07:35:07.730 INFO:teuthology.orchestra.run.smithi046.stdout:158913790017 2024-09-10T07:35:07.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T07:35:08.014 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:07 smithi046 ceph-mon[30705]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:193273528378 2024-09-10T07:35:08.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T07:35:08.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:07 smithi181 ceph-mon[35397]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:08.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:07 smithi063 ceph-mon[35020]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.995+0000 7f025e200640 1 -- 172.21.15.46:0/3208209117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0258074e30 msgr2=0x7f025810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.995+0000 7f025e200640 1 --2- 172.21.15.46:0/3208209117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0258074e30 0x7f025810dd40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0240009900 tx=0x7f024002a480 comp rx=0 tx=0).stop 2024-09-10T07:35:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.995+0000 7f025e200640 1 -- 172.21.15.46:0/3208209117 shutdown_connections 2024-09-10T07:35:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.995+0000 7f025e200640 1 --2- 172.21.15.46:0/3208209117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f025810e280 0x7f0258110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.995+0000 7f025e200640 1 --2- 172.21.15.46:0/3208209117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0258074e30 0x7f025810dd40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.995+0000 7f025e200640 1 --2- 172.21.15.46:0/3208209117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0258074510 0x7f02580748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.995+0000 7f025e200640 1 -- 172.21.15.46:0/3208209117 >> 172.21.15.46:0/3208209117 conn(0x7f02580fd740 msgr2=0x7f02580ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.995+0000 7f025e200640 1 -- 172.21.15.46:0/3208209117 shutdown_connections 2024-09-10T07:35:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.996+0000 7f025e200640 1 -- 172.21.15.46:0/3208209117 wait complete. 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.996+0000 7f025e200640 1 Processor -- start 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025e200640 1 -- start start 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0258074510 0x7f025819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025e200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0258074e30 0x7f025819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025810e280 0x7f025819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0258114000 con 0x7f025810e280 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0258113e80 con 0x7f0258074510 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025e200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0258114180 con 0x7f0258074e30 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025810e280 0x7f025819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0258074510 0x7f025819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f024fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0258074e30 0x7f025819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025810e280 0x7f025819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39598/0 (socket says 172.21.15.46:39598) 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0258074510 0x7f025819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41356/0 (socket says 172.21.15.46:41356) 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.997+0000 7f025d800640 1 -- 172.21.15.46:0/2165989394 learned_addr learned my addr 172.21.15.46:0/2165989394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f025ce00640 1 -- 172.21.15.46:0/2165989394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0258074e30 msgr2=0x7f025819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f025ce00640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0258074e30 0x7f025819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f025ce00640 1 -- 172.21.15.46:0/2165989394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025810e280 msgr2=0x7f025819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f025ce00640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025810e280 0x7f025819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f025ce00640 1 -- 172.21.15.46:0/2165989394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025819f4d0 con 0x7f0258074510 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f025ce00640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0258074510 0x7f025819a520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f024800e9f0 tx=0x7f024800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800ccc0 con 0x7f0258074510 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0248010910 con 0x7f0258074510 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025819f7c0 con 0x7f0258074510 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.998+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248018960 con 0x7f0258074510 2024-09-10T07:35:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.999+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581a70a0 con 0x7f0258074510 2024-09-10T07:35:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.999+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f02580ff880 con 0x7f0258074510 2024-09-10T07:35:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.002+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0248010ac0 con 0x7f0258074510 2024-09-10T07:35:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.003+0000 7f0256000640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f023c0a2c10 0x7f023c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.003+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f02480d0140 con 0x7f0258074510 2024-09-10T07:35:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.003+0000 7f024fe00640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f023c0a2c10 0x7f023c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.003+0000 7f0256000640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] conn(0x7f023c0adde0 0x7f023c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.003+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 --> [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0248028ea0 con 0x7f023c0adde0 2024-09-10T07:35:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.003+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f02480d04f0 con 0x7f0258074510 2024-09-10T07:35:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.004+0000 7f025d800640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] conn(0x7f023c0adde0 0x7f023c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.004+0000 7f024fe00640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f023c0a2c10 0x7f023c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0240000c00 tx=0x7f0240005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.004+0000 7f025d800640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] conn(0x7f023c0adde0 0x7f023c0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.005+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 <== osd.11 v2:172.21.15.181:6824/1815305110 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0248028ea0 con 0x7f023c0adde0 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.020+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 --> [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0258075c50 con 0x7f023c0adde0 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.020+0000 7f0256000640 1 -- 172.21.15.46:0/2165989394 <== osd.11 v2:172.21.15.181:6824/1815305110 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0258075c50 con 0x7f023c0adde0 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 >> [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] conn(0x7f023c0adde0 msgr2=0x7f023c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] conn(0x7f023c0adde0 0x7f023c0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f023c0a2c10 msgr2=0x7f023c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f023c0a2c10 0x7f023c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0240000c00 tx=0x7f0240005b50 comp rx=0 tx=0).stop 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0258074510 msgr2=0x7f025819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 --2- 172.21.15.46:0/2165989394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0258074510 0x7f025819a520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f024800e9f0 tx=0x7f024800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025d800640 1 -- 172.21.15.46:0/2165989394 reap_dead start 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 shutdown_connections 2024-09-10T07:35:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 >> 172.21.15.46:0/2165989394 conn(0x7f02580fd740 msgr2=0x7f02580fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 shutdown_connections 2024-09-10T07:35:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:06.021+0000 7f025e200640 1 -- 172.21.15.46:0/2165989394 wait complete. 2024-09-10T07:35:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 -- 172.21.15.46:0/650292335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2b01047e0 msgr2=0x7fa2b0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 --2- 172.21.15.46:0/650292335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2b01047e0 0x7fa2b0108830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009900 tx=0x7fa29c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 -- 172.21.15.46:0/650292335 shutdown_connections 2024-09-10T07:35:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 --2- 172.21.15.46:0/650292335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b0108f60 0x7fa2b0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 --2- 172.21.15.46:0/650292335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2b01047e0 0x7fa2b0108830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 --2- 172.21.15.46:0/650292335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b0103e30 0x7fa2b0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 -- 172.21.15.46:0/650292335 >> 172.21.15.46:0/650292335 conn(0x7fa2b00ff8d0 msgr2=0x7fa2b0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 -- 172.21.15.46:0/650292335 shutdown_connections 2024-09-10T07:35:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.944+0000 7fa2b6400640 1 -- 172.21.15.46:0/650292335 wait complete. 2024-09-10T07:35:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.946+0000 7fa2b6400640 1 Processor -- start 2024-09-10T07:35:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.946+0000 7fa2b6400640 1 -- start start 2024-09-10T07:35:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.946+0000 7fa2b6400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b0103e30 0x7fa2b019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.946+0000 7fa2b6400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b01047e0 0x7fa2b019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.946+0000 7fa2b6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2b0108f60 0x7fa2b019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.946+0000 7fa2b6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0114010 con 0x7fa2b0108f60 2024-09-10T07:35:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.946+0000 7fa2b6400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0113e90 con 0x7fa2b01047e0 2024-09-10T07:35:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.946+0000 7fa2b6400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0114190 con 0x7fa2b0103e30 2024-09-10T07:35:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2afe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b01047e0 0x7fa2b019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2afe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b01047e0 0x7fa2b019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41388/0 (socket says 172.21.15.46:41388) 2024-09-10T07:35:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2afe00640 1 -- 172.21.15.46:0/3715492398 learned_addr learned my addr 172.21.15.46:0/3715492398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2b5a00640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2b0108f60 0x7fa2b019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2b5000640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b0103e30 0x7fa2b019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2afe00640 1 -- 172.21.15.46:0/3715492398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b0103e30 msgr2=0x7fa2b019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2afe00640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b0103e30 0x7fa2b019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2afe00640 1 -- 172.21.15.46:0/3715492398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2b0108f60 msgr2=0x7fa2b019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2afe00640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2b0108f60 0x7fa2b019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.947+0000 7fa2afe00640 1 -- 172.21.15.46:0/3715492398 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b019f460 con 0x7fa2b01047e0 2024-09-10T07:35:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.948+0000 7fa2afe00640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b01047e0 0x7fa2b019a9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa29c004770 tx=0x7fa29c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:08.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.948+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c03f070 con 0x7fa2b01047e0 2024-09-10T07:35:08.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.948+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c031ca0 con 0x7fa2b01047e0 2024-09-10T07:35:08.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.948+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b019f6f0 con 0x7fa2b01047e0 2024-09-10T07:35:08.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.948+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c035030 con 0x7fa2b01047e0 2024-09-10T07:35:08.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.949+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b01a7030 con 0x7fa2b01047e0 2024-09-10T07:35:08.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.950+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa2b0062980 con 0x7fa2b01047e0 2024-09-10T07:35:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.951+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa29c042070 con 0x7fa2b01047e0 2024-09-10T07:35:08.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.953+0000 7fa2ad600640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2980a2c90 0x7fa2980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.953+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7fa29c0efb60 con 0x7fa2b01047e0 2024-09-10T07:35:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.953+0000 7fa2b5000640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2980a2c90 0x7fa2980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.953+0000 7fa2ad600640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] conn(0x7fa2980ade60 0x7fa2980b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.953+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 --> [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa29c02fec0 con 0x7fa2980ade60 2024-09-10T07:35:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.953+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fa29c0f42a0 con 0x7fa2b01047e0 2024-09-10T07:35:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.953+0000 7fa2b5a00640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] conn(0x7fa2980ade60 0x7fa2980b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.954+0000 7fa2b5000640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2980a2c90 0x7fa2980a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400bbe0 tx=0x7fa2a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.954+0000 7fa2b5a00640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] conn(0x7fa2980ade60 0x7fa2980b0300 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.955+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 <== osd.8 v2:172.21.15.181:6816/296261223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa29c02fec0 con 0x7fa2980ade60 2024-09-10T07:35:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.970+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 --> [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa2b0062d80 con 0x7fa2980ade60 2024-09-10T07:35:08.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.970+0000 7fa2ad600640 1 -- 172.21.15.46:0/3715492398 <== osd.8 v2:172.21.15.181:6816/296261223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa2b0062d80 con 0x7fa2980ade60 2024-09-10T07:35:08.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 >> [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] conn(0x7fa2980ade60 msgr2=0x7fa2980b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] conn(0x7fa2980ade60 0x7fa2980b0300 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2980a2c90 msgr2=0x7fa2980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2980a2c90 0x7fa2980a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400bbe0 tx=0x7fa2a400b390 comp rx=0 tx=0).stop 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b01047e0 msgr2=0x7fa2b019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 --2- 172.21.15.46:0/3715492398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b01047e0 0x7fa2b019a9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa29c004770 tx=0x7fa29c003c10 comp rx=0 tx=0).stop 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b5a00640 1 -- 172.21.15.46:0/3715492398 reap_dead start 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 shutdown_connections 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 >> 172.21.15.46:0/3715492398 conn(0x7fa2b00ff8d0 msgr2=0x7fa2b0100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 shutdown_connections 2024-09-10T07:35:08.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:08.971+0000 7fa2b6400640 1 -- 172.21.15.46:0/3715492398 wait complete. 2024-09-10T07:35:09.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:09 smithi046 ceph-mon[30705]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T07:35:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:244813135921 2024-09-10T07:35:10.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T07:35:10.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:09 smithi181 ceph-mon[35397]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T07:35:10.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:09 smithi063 ceph-mon[35020]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T07:35:10.640 INFO:teuthology.orchestra.run.smithi046.stdout:137438953541 2024-09-10T07:35:10.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T07:35:11.050 INFO:teuthology.orchestra.run.smithi046.stdout:300647710757 2024-09-10T07:35:11.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 -- 172.21.15.46:0/300212342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d20109120 msgr2=0x7f3d20110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 --2- 172.21.15.46:0/300212342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d20109120 0x7f3d20110ca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009be0 tx=0x7f3d1802a6d0 comp rx=0 tx=0).stop 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 -- 172.21.15.46:0/300212342 shutdown_connections 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 --2- 172.21.15.46:0/300212342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d20109120 0x7f3d20110ca0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 --2- 172.21.15.46:0/300212342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d201049a0 0x7f3d201089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 --2- 172.21.15.46:0/300212342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d20103ff0 0x7f3d201043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 -- 172.21.15.46:0/300212342 >> 172.21.15.46:0/300212342 conn(0x7f3d200ffa90 msgr2=0x7f3d20101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 -- 172.21.15.46:0/300212342 shutdown_connections 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.850+0000 7f3d24a00640 1 -- 172.21.15.46:0/300212342 wait complete. 2024-09-10T07:35:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.851+0000 7f3d24a00640 1 Processor -- start 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d24a00640 1 -- start start 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d24a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d20103ff0 0x7f3d2019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d24a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d201049a0 0x7f3d2019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d24a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d20109120 0x7f3d2019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d24a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d20114150 con 0x7f3d20103ff0 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d24a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d20113fd0 con 0x7f3d201049a0 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d24a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d201142d0 con 0x7f3d20109120 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d1d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d201049a0 0x7f3d2019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d1de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d20103ff0 0x7f3d2019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d1d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d201049a0 0x7f3d2019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41328/0 (socket says 172.21.15.46:41328) 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d1d400640 1 -- 172.21.15.46:0/1343914371 learned_addr learned my addr 172.21.15.46:0/1343914371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.852+0000 7f3d1e800640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d20109120 0x7f3d2019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d1d400640 1 -- 172.21.15.46:0/1343914371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d20109120 msgr2=0x7f3d2019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d1d400640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d20109120 0x7f3d2019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d1d400640 1 -- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d20103ff0 msgr2=0x7f3d2019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d1d400640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d20103ff0 0x7f3d2019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d1d400640 1 -- 172.21.15.46:0/1343914371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2019f5e0 con 0x7f3d201049a0 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d1d400640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d201049a0 0x7f3d2019ab70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00e9c0 tx=0x7f3d0c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00ccc0 con 0x7f3d201049a0 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2019f8d0 con 0x7f3d201049a0 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d0c005500 con 0x7f3d201049a0 2024-09-10T07:35:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c010430 con 0x7f3d201049a0 2024-09-10T07:35:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.853+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d201a71b0 con 0x7f3d201049a0 2024-09-10T07:35:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.854+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d20062980 con 0x7f3d201049a0 2024-09-10T07:35:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.856+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3d0c0046c0 con 0x7f3d201049a0 2024-09-10T07:35:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.858+0000 7f3d16a00640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d040a2c10 0x7f3d040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.858+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9092+0+0 (secure 0 0 0) 0x7f3d0c0cfb80 con 0x7f3d201049a0 2024-09-10T07:35:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.859+0000 7f3d1de00640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d040a2c10 0x7f3d040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.859+0000 7f3d16a00640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] conn(0x7f3d040adde0 0x7f3d040b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.859+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 --> [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d0c011e50 con 0x7f3d040adde0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.859+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f3d0c0cffd0 con 0x7f3d201049a0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.859+0000 7f3d1e800640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] conn(0x7f3d040adde0 0x7f3d040b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.859+0000 7f3d1de00640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d040a2c10 0x7f3d040a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009bd0 tx=0x7f3d10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.860+0000 7f3d1e800640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] conn(0x7f3d040adde0 0x7f3d040b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.861+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 <== osd.3 v2:172.21.15.46:6810/3016450690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d0c011e50 con 0x7f3d040adde0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.876+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 --> [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d20062e00 con 0x7f3d040adde0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.876+0000 7f3d16a00640 1 -- 172.21.15.46:0/1343914371 <== osd.3 v2:172.21.15.46:6810/3016450690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3d20062e00 con 0x7f3d040adde0 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.876+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] conn(0x7f3d040adde0 msgr2=0x7f3d040b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.876+0000 7f3d24a00640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] conn(0x7f3d040adde0 0x7f3d040b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.876+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d040a2c10 msgr2=0x7f3d040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.876+0000 7f3d24a00640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d040a2c10 0x7f3d040a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009bd0 tx=0x7f3d10005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.877+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d201049a0 msgr2=0x7f3d2019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.877+0000 7f3d24a00640 1 --2- 172.21.15.46:0/1343914371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d201049a0 0x7f3d2019ab70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00e9c0 tx=0x7f3d0c00ee80 comp rx=0 tx=0).stop 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.877+0000 7f3d1e800640 1 -- 172.21.15.46:0/1343914371 reap_dead start 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.877+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 shutdown_connections 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.877+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 >> 172.21.15.46:0/1343914371 conn(0x7f3d200ffa90 msgr2=0x7f3d20100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.877+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 shutdown_connections 2024-09-10T07:35:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:05.877+0000 7f3d24a00640 1 -- 172.21.15.46:0/1343914371 wait complete. 2024-09-10T07:35:12.136 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:11 smithi046 ceph-mon[30705]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:11 smithi181 ceph-mon[35397]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:11 smithi063 ceph-mon[35020]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:12.487 INFO:teuthology.orchestra.run.smithi046.stdout:77309411408 2024-09-10T07:35:12.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T07:35:13.121 INFO:teuthology.orchestra.run.smithi046.stdout:214748364852 2024-09-10T07:35:13.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T07:35:13.744 INFO:teuthology.orchestra.run.smithi046.stdout:55834574933 2024-09-10T07:35:13.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T07:35:14.074 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:13 smithi046 ceph-mon[30705]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T07:35:14.074 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-10T07:35:14.115 INFO:teuthology.orchestra.run.smithi046.stdout:107374182474 2024-09-10T07:35:14.115 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T07:35:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:13 smithi181 ceph-mon[35397]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T07:35:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-10T07:35:14.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:13 smithi063 ceph-mon[35020]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T07:35:14.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-10T07:35:14.197 INFO:teuthology.orchestra.run.smithi046.stdout:279172874281 2024-09-10T07:35:14.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T07:35:14.654 INFO:teuthology.orchestra.run.smithi046.stdout:335007449121 2024-09-10T07:35:14.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T07:35:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:14.765 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:14.816 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:14.845 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:14 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-10T07:35:14.845 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:14 smithi046 ceph-mon[30705]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T07:35:15.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:14 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-10T07:35:15.225 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:14 smithi181 ceph-mon[35397]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T07:35:15.225 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:14 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-10T07:35:15.225 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:14 smithi063 ceph-mon[35020]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T07:35:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:16.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:15 smithi046 ceph-mon[30705]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-10T07:35:16.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:15 smithi046 ceph-mon[30705]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T07:35:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:15 smithi181 ceph-mon[35397]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-10T07:35:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:15 smithi181 ceph-mon[35397]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T07:35:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:15 smithi063 ceph-mon[35020]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-10T07:35:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:15 smithi063 ceph-mon[35020]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T07:35:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.332+0000 7fcd9cc00640 1 -- 172.21.15.46:0/3496981161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 msgr2=0x7fcd981043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.332+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/3496981161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 0x7fcd981043d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009900 tx=0x7fcd8802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.332+0000 7fcd9cc00640 1 -- 172.21.15.46:0/3496981161 shutdown_connections 2024-09-10T07:35:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.332+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/3496981161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd98109120 0x7fcd98110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.332+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/3496981161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd981049a0 0x7fcd981089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.332+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/3496981161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 0x7fcd981043d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.332+0000 7fcd9cc00640 1 -- 172.21.15.46:0/3496981161 >> 172.21.15.46:0/3496981161 conn(0x7fcd980ffa90 msgr2=0x7fcd98101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.333+0000 7fcd9cc00640 1 -- 172.21.15.46:0/3496981161 shutdown_connections 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.333+0000 7fcd9cc00640 1 -- 172.21.15.46:0/3496981161 wait complete. 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.334+0000 7fcd9cc00640 1 Processor -- start 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.334+0000 7fcd9cc00640 1 -- start start 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd9cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 0x7fcd9819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd981049a0 0x7fcd9819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd9cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd98109120 0x7fcd9819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd9cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd98114140 con 0x7fcd981049a0 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd9cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd98113fc0 con 0x7fcd98103ff0 2024-09-10T07:35:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd9cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981142c0 con 0x7fcd98109120 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 0x7fcd9819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd95e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 0x7fcd9819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38076/0 (socket says 172.21.15.46:38076) 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd95e00640 1 -- 172.21.15.46:0/1145224117 learned_addr learned my addr 172.21.15.46:0/1145224117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd8fe00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd981049a0 0x7fcd9819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd96800640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd98109120 0x7fcd9819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd95e00640 1 -- 172.21.15.46:0/1145224117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd98109120 msgr2=0x7fcd9819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd95e00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd98109120 0x7fcd9819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd95e00640 1 -- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd981049a0 msgr2=0x7fcd9819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd95e00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd981049a0 0x7fcd9819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.335+0000 7fcd95e00640 1 -- 172.21.15.46:0/1145224117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9819f6d0 con 0x7fcd98103ff0 2024-09-10T07:35:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.336+0000 7fcd95e00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 0x7fcd9819a720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd88006010 tx=0x7fcd88004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.336+0000 7fcd8ea00640 1 -- 172.21.15.46:0/1145224117 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8803f030 con 0x7fcd98103ff0 2024-09-10T07:35:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.336+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9819f960 con 0x7fcd98103ff0 2024-09-10T07:35:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.336+0000 7fcd8ea00640 1 -- 172.21.15.46:0/1145224117 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd88004440 con 0x7fcd98103ff0 2024-09-10T07:35:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.336+0000 7fcd8ea00640 1 -- 172.21.15.46:0/1145224117 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88035070 con 0x7fcd98103ff0 2024-09-10T07:35:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.336+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd981a72a0 con 0x7fcd98103ff0 2024-09-10T07:35:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.337+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd981043d0 con 0x7fcd98103ff0 2024-09-10T07:35:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.339+0000 7fcd8ea00640 1 -- 172.21.15.46:0/1145224117 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcd88042030 con 0x7fcd98103ff0 2024-09-10T07:35:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.341+0000 7fcd8ea00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcd7c0a2c90 0x7fcd7c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.341+0000 7fcd8ea00640 1 -- 172.21.15.46:0/1145224117 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcd880efce0 con 0x7fcd98103ff0 2024-09-10T07:35:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.341+0000 7fcd8fe00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcd7c0a2c90 0x7fcd7c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.342+0000 7fcd8fe00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcd7c0a2c90 0x7fcd7c0a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd980ffff0 tx=0x7fcd80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.342+0000 7fcd8ea00640 1 -- 172.21.15.46:0/1145224117 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd880bc7c0 con 0x7fcd98103ff0 2024-09-10T07:35:16.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 -- 172.21.15.46:0/1780777241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068108f60 msgr2=0x7fb068110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 --2- 172.21.15.46:0/1780777241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068108f60 0x7fb068110ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb060009ca0 tx=0x7fb06002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 -- 172.21.15.46:0/1780777241 shutdown_connections 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 --2- 172.21.15.46:0/1780777241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068108f60 0x7fb068110ae0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 --2- 172.21.15.46:0/1780777241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0681047e0 0x7fb068108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 --2- 172.21.15.46:0/1780777241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb068103e30 0x7fb068104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 -- 172.21.15.46:0/1780777241 >> 172.21.15.46:0/1780777241 conn(0x7fb0680ff8d0 msgr2=0x7fb068101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 -- 172.21.15.46:0/1780777241 shutdown_connections 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.922+0000 7fb067200640 1 -- 172.21.15.46:0/1780777241 wait complete. 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.923+0000 7fb067200640 1 Processor -- start 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.923+0000 7fb067200640 1 -- start start 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb067200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068103e30 0x7fb06819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb067200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0681047e0 0x7fb06819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb067200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb068108f60 0x7fb06819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb067200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb068114010 con 0x7fb068103e30 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb067200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb068113e90 con 0x7fb068108f60 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb067200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb068114190 con 0x7fb0681047e0 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb065e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068103e30 0x7fb06819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb05fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0681047e0 0x7fb06819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb05fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0681047e0 0x7fb06819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33596/0 (socket says 172.21.15.46:33596) 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb05fe00640 1 -- 172.21.15.46:0/1308477905 learned_addr learned my addr 172.21.15.46:0/1308477905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb066800640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb068108f60 0x7fb06819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb05fe00640 1 -- 172.21.15.46:0/1308477905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb068108f60 msgr2=0x7fb06819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb05fe00640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb068108f60 0x7fb06819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb05fe00640 1 -- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068103e30 msgr2=0x7fb06819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb05fe00640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068103e30 0x7fb06819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb05fe00640 1 -- 172.21.15.46:0/1308477905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06819f3c0 con 0x7fb0681047e0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.924+0000 7fb065e00640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068103e30 0x7fb06819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.925+0000 7fb05fe00640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0681047e0 0x7fb06819a950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb05000d950 tx=0x7fb05000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.925+0000 7fb05ea00640 1 -- 172.21.15.46:0/1308477905 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050017030 con 0x7fb0681047e0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.925+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06819f6b0 con 0x7fb0681047e0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.925+0000 7fb05ea00640 1 -- 172.21.15.46:0/1308477905 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050004020 con 0x7fb0681047e0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.925+0000 7fb05ea00640 1 -- 172.21.15.46:0/1308477905 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0500047a0 con 0x7fb0681047e0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.925+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0681a6f90 con 0x7fb0681047e0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.926+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb068104210 con 0x7fb0681047e0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.928+0000 7fb05ea00640 1 -- 172.21.15.46:0/1308477905 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb0500041d0 con 0x7fb0681047e0 2024-09-10T07:35:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.930+0000 7fb05ea00640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stdout:77309411410 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.930+0000 7fb05ea00640 1 -- 172.21.15.46:0/1308477905 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb0500cee40 con 0x7fb0681047e0 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.930+0000 7fb065e00640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.931+0000 7fb065e00640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb058009fc0 tx=0x7fb058005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:15.934+0000 7fb05ea00640 1 -- 172.21.15.46:0/1308477905 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb05009b860 con 0x7fb0681047e0 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.140+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb06802d7c0 con 0x7fb0681047e0 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.140+0000 7fb05ea00640 1 -- 172.21.15.46:0/1308477905 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb05009b160 con 0x7fb0681047e0 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2c10 msgr2=0x7fb04c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb058009fc0 tx=0x7fb058005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0681047e0 msgr2=0x7fb06819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0681047e0 0x7fb06819a950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb05000d950 tx=0x7fb05000de10 comp rx=0 tx=0).stop 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 shutdown_connections 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb068108f60 0x7fb06819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0681047e0 0x7fb06819a950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2c10 0x7fb04c0a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 --2- 172.21.15.46:0/1308477905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb068103e30 0x7fb06819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.142+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 >> 172.21.15.46:0/1308477905 conn(0x7fb0680ff8d0 msgr2=0x7fb068100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.143+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 shutdown_connections 2024-09-10T07:35:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.143+0000 7fb067200640 1 -- 172.21.15.46:0/1308477905 wait complete. 2024-09-10T07:35:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.530+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcd9810c300 con 0x7fcd98103ff0 2024-09-10T07:35:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.531+0000 7fcd8ea00640 1 -- 172.21.15.46:0/1145224117 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcd880bc0c0 con 0x7fcd98103ff0 2024-09-10T07:35:16.531 INFO:teuthology.orchestra.run.smithi046.stdout:137438953543 2024-09-10T07:35:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.533+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcd7c0a2c90 msgr2=0x7fcd7c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.533+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcd7c0a2c90 0x7fcd7c0a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd980ffff0 tx=0x7fcd80005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.533+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 msgr2=0x7fcd9819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.533+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 0x7fcd9819a720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd88006010 tx=0x7fcd88004040 comp rx=0 tx=0).stop 2024-09-10T07:35:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.533+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 shutdown_connections 2024-09-10T07:35:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.533+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd98109120 0x7fcd9819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.534+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcd7c0a2c90 0x7fcd7c0a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.534+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd981049a0 0x7fcd9819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.534+0000 7fcd9cc00640 1 --2- 172.21.15.46:0/1145224117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd98103ff0 0x7fcd9819a720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.534+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 >> 172.21.15.46:0/1145224117 conn(0x7fcd980ffa90 msgr2=0x7fcd98100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.534+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 shutdown_connections 2024-09-10T07:35:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.534+0000 7fcd9cc00640 1 -- 172.21.15.46:0/1145224117 wait complete. 2024-09-10T07:35:17.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:16 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1308477905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T07:35:17.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:16 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1145224117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T07:35:17.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:16 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1308477905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T07:35:17.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:16 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1145224117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T07:35:17.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:16 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1308477905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T07:35:17.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:16 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1145224117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T07:35:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.931+0000 7f42efe00640 1 -- 172.21.15.46:0/1250874401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8104010 msgr2=0x7f42e81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.931+0000 7f42efe00640 1 --2- 172.21.15.46:0/1250874401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8104010 0x7f42e81043f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009900 tx=0x7f42dc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.932+0000 7f42efe00640 1 -- 172.21.15.46:0/1250874401 shutdown_connections 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.932+0000 7f42efe00640 1 --2- 172.21.15.46:0/1250874401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e8109140 0x7f42e8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.932+0000 7f42efe00640 1 --2- 172.21.15.46:0/1250874401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42e81049c0 0x7f42e8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.932+0000 7f42efe00640 1 --2- 172.21.15.46:0/1250874401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8104010 0x7f42e81043f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.932+0000 7f42efe00640 1 -- 172.21.15.46:0/1250874401 >> 172.21.15.46:0/1250874401 conn(0x7f42e80ffa50 msgr2=0x7f42e8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.932+0000 7f42efe00640 1 -- 172.21.15.46:0/1250874401 shutdown_connections 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.932+0000 7f42efe00640 1 -- 172.21.15.46:0/1250874401 wait complete. 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42efe00640 1 Processor -- start 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42efe00640 1 -- start start 2024-09-10T07:35:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e8104010 0x7f42e819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e81049c0 0x7f42e819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42e8109140 0x7f42e819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42efe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8114140 con 0x7f42e81049c0 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42efe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8113fc0 con 0x7f42e8104010 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42efe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81142c0 con 0x7f42e8109140 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42eca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e81049c0 0x7f42e819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42ed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e8104010 0x7f42e819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42ed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e8104010 0x7f42e819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38086/0 (socket says 172.21.15.46:38086) 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.933+0000 7f42ed400640 1 -- 172.21.15.46:0/509106044 learned_addr learned my addr 172.21.15.46:0/509106044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42ede00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42e8109140 0x7f42e819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42eca00640 1 -- 172.21.15.46:0/509106044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42e8109140 msgr2=0x7f42e819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42eca00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42e8109140 0x7f42e819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42eca00640 1 -- 172.21.15.46:0/509106044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e8104010 msgr2=0x7f42e819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42eca00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e8104010 0x7f42e819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42eca00640 1 -- 172.21.15.46:0/509106044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e819f5b0 con 0x7f42e81049c0 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42ed400640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e8104010 0x7f42e819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42eca00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e81049c0 0x7f42e819ab40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f42d4005fd0 tx=0x7f42d400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42e6000640 1 -- 172.21.15.46:0/509106044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4009c30 con 0x7f42e81049c0 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42e6000640 1 -- 172.21.15.46:0/509106044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4009dd0 con 0x7f42e81049c0 2024-09-10T07:35:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.934+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e819f8a0 con 0x7f42e81049c0 2024-09-10T07:35:17.182 INFO:teuthology.orchestra.run.smithi046.stdout:158913790020 2024-09-10T07:35:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.935+0000 7f42e6000640 1 -- 172.21.15.46:0/509106044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4010460 con 0x7f42e81049c0 2024-09-10T07:35:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.935+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81a7180 con 0x7f42e81049c0 2024-09-10T07:35:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.936+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e81043f0 con 0x7f42e81049c0 2024-09-10T07:35:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.938+0000 7f42e6000640 1 -- 172.21.15.46:0/509106044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f42d4003ef0 con 0x7f42e81049c0 2024-09-10T07:35:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.940+0000 7f42e6000640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f42d00a2c10 0x7f42d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.940+0000 7f42e6000640 1 -- 172.21.15.46:0/509106044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f42d40cf0f0 con 0x7f42e81049c0 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.940+0000 7f42ed400640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f42d00a2c10 0x7f42d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.941+0000 7f42ed400640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f42d00a2c10 0x7f42d00a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0096c0 tx=0x7f42dc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:16.944+0000 7f42e6000640 1 -- 172.21.15.46:0/509106044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42d4017020 con 0x7f42e81049c0 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.147+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f42e802d7c0 con 0x7f42e81049c0 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.147+0000 7f42e6000640 1 -- 172.21.15.46:0/509106044 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f42d409b4c0 con 0x7f42e81049c0 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f42d00a2c10 msgr2=0x7f42d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f42d00a2c10 0x7f42d00a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0096c0 tx=0x7f42dc005a20 comp rx=0 tx=0).stop 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e81049c0 msgr2=0x7f42e819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e81049c0 0x7f42e819ab40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f42d4005fd0 tx=0x7f42d400dde0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 shutdown_connections 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42e8109140 0x7f42e819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f42d00a2c10 0x7f42d00a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e81049c0 0x7f42e819ab40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 --2- 172.21.15.46:0/509106044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e8104010 0x7f42e819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.149+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 >> 172.21.15.46:0/509106044 conn(0x7f42e80ffa50 msgr2=0x7f42e8100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.150+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 shutdown_connections 2024-09-10T07:35:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:17.150+0000 7f42efe00640 1 -- 172.21.15.46:0/509106044 wait complete. 2024-09-10T07:35:17.847 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:17 smithi046 ceph-mon[30705]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-09-10T07:35:18.131 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:17 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/509106044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T07:35:18.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:17 smithi181 ceph-mon[35397]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-09-10T07:35:18.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:17 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/509106044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T07:35:18.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:17 smithi063 ceph-mon[35020]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-09-10T07:35:18.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:17 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/509106044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T07:35:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:18.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953541 got 137438953543 for osd.4 2024-09-10T07:35:18.699 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:19.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:18 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2411609619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T07:35:19.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:18 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2411609619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T07:35:19.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:18 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2411609619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T07:35:19.224 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790017 got 158913790020 for osd.5 2024-09-10T07:35:19.224 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:19.923 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:19 smithi046 ceph-mon[30705]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T07:35:20.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:19 smithi181 ceph-mon[35397]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T07:35:20.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:19 smithi063 ceph-mon[35020]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T07:35:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.083+0000 7f0ee6200640 1 -- 172.21.15.46:0/2269300359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0076720 msgr2=0x7f0ee0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.083+0000 7f0ee6200640 1 --2- 172.21.15.46:0/2269300359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0076720 0x7f0ee0076b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009900 tx=0x7f0ed002a410 comp rx=0 tx=0).stop 2024-09-10T07:35:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.084+0000 7f0ee6200640 1 -- 172.21.15.46:0/2269300359 shutdown_connections 2024-09-10T07:35:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.084+0000 7f0ee6200640 1 --2- 172.21.15.46:0/2269300359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee0112a30 0x7f0ee0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.084+0000 7f0ee6200640 1 --2- 172.21.15.46:0/2269300359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee0077040 0x7f0ee01124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.084+0000 7f0ee6200640 1 --2- 172.21.15.46:0/2269300359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0076720 0x7f0ee0076b00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.084+0000 7f0ee6200640 1 -- 172.21.15.46:0/2269300359 >> 172.21.15.46:0/2269300359 conn(0x7f0ee00ffa90 msgr2=0x7f0ee0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.084+0000 7f0ee6200640 1 -- 172.21.15.46:0/2269300359 shutdown_connections 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.084+0000 7f0ee6200640 1 -- 172.21.15.46:0/2269300359 wait complete. 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ee6200640 1 Processor -- start 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ee6200640 1 -- start start 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ee6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee0076720 0x7f0ee01a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ee6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0077040 0x7f0ee01a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ee6200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee0112a30 0x7f0ee019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ee6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee0118610 con 0x7f0ee0077040 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ee6200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee0118490 con 0x7f0ee0112a30 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ee6200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee0118790 con 0x7f0ee0076720 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0ed7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0077040 0x7f0ee01a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.085+0000 7f0edf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee0076720 0x7f0ee01a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ed7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0077040 0x7f0ee01a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56062/0 (socket says 172.21.15.46:56062) 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ed7e00640 1 -- 172.21.15.46:0/1305328669 learned_addr learned my addr 172.21.15.46:0/1305328669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0edfe00640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee0112a30 0x7f0ee019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ed7e00640 1 -- 172.21.15.46:0/1305328669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee0076720 msgr2=0x7f0ee01a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ed7e00640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee0076720 0x7f0ee01a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ed7e00640 1 -- 172.21.15.46:0/1305328669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee0112a30 msgr2=0x7f0ee019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ed7e00640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee0112a30 0x7f0ee019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ed7e00640 1 -- 172.21.15.46:0/1305328669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee019eff0 con 0x7f0ee0077040 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ed7e00640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0077040 0x7f0ee01a4c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800e9f0 tx=0x7f0ec800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0edcc00640 1 -- 172.21.15.46:0/1305328669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec800cd80 con 0x7f0ee0077040 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0edcc00640 1 -- 172.21.15.46:0/1305328669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ec8005500 con 0x7f0ee0077040 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.086+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee019f2e0 con 0x7f0ee0077040 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.087+0000 7f0edcc00640 1 -- 172.21.15.46:0/1305328669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8010690 con 0x7f0ee0077040 2024-09-10T07:35:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.087+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee01ab5d0 con 0x7f0ee0077040 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.088+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee0076b00 con 0x7f0ee0077040 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.090+0000 7f0edcc00640 1 -- 172.21.15.46:0/1305328669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0ec80046c0 con 0x7f0ee0077040 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.091+0000 7f0edcc00640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ec40a2c90 0x7f0ec40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.091+0000 7f0edcc00640 1 -- 172.21.15.46:0/1305328669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0ec80cf8d0 con 0x7f0ee0077040 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.091+0000 7f0edf400640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ec40a2c90 0x7f0ec40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.092+0000 7f0edf400640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ec40a2c90 0x7f0ec40a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0006010 tx=0x7f0ed0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.095+0000 7f0edcc00640 1 -- 172.21.15.46:0/1305328669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ec809c350 con 0x7f0ee0077040 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.288+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0ee002d7c0 con 0x7f0ee0077040 2024-09-10T07:35:20.316 INFO:teuthology.orchestra.run.smithi046.stdout:214748364855 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.289+0000 7f0edcc00640 1 -- 172.21.15.46:0/1305328669 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0ec809bc50 con 0x7f0ee0077040 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ec40a2c90 msgr2=0x7f0ec40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ec40a2c90 0x7f0ec40a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0006010 tx=0x7f0ed0000bc0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0077040 msgr2=0x7f0ee01a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0077040 0x7f0ee01a4c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800e9f0 tx=0x7f0ec800edf0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 shutdown_connections 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee0112a30 0x7f0ee019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ec40a2c90 0x7f0ec40a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee0077040 0x7f0ee01a4c40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.291+0000 7f0ee6200640 1 --2- 172.21.15.46:0/1305328669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee0076720 0x7f0ee01a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.292+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 >> 172.21.15.46:0/1305328669 conn(0x7f0ee00ffa90 msgr2=0x7f0ee0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.292+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 shutdown_connections 2024-09-10T07:35:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.292+0000 7f0ee6200640 1 -- 172.21.15.46:0/1305328669 wait complete. 2024-09-10T07:35:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.213+0000 7fcddb200640 1 -- 172.21.15.46:0/4029785120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcddc1047e0 msgr2=0x7fcddc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.213+0000 7fcddb200640 1 --2- 172.21.15.46:0/4029785120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcddc1047e0 0x7fcddc108830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009900 tx=0x7fcdc802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.214+0000 7fcddb200640 1 -- 172.21.15.46:0/4029785120 shutdown_connections 2024-09-10T07:35:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.214+0000 7fcddb200640 1 --2- 172.21.15.46:0/4029785120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc108f60 0x7fcddc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.214+0000 7fcddb200640 1 --2- 172.21.15.46:0/4029785120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcddc1047e0 0x7fcddc108830 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.214+0000 7fcddb200640 1 --2- 172.21.15.46:0/4029785120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc103e30 0x7fcddc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.214+0000 7fcddb200640 1 -- 172.21.15.46:0/4029785120 >> 172.21.15.46:0/4029785120 conn(0x7fcddc0ff910 msgr2=0x7fcddc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.214+0000 7fcddb200640 1 -- 172.21.15.46:0/4029785120 shutdown_connections 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.214+0000 7fcddb200640 1 -- 172.21.15.46:0/4029785120 wait complete. 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.215+0000 7fcddb200640 1 Processor -- start 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.215+0000 7fcddb200640 1 -- start start 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.215+0000 7fcddb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc103e30 0x7fcddc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.215+0000 7fcddb200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc1047e0 0x7fcddc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.215+0000 7fcddb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcddc108f60 0x7fcddc19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.215+0000 7fcddb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc113fd0 con 0x7fcddc108f60 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.215+0000 7fcddb200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc113e50 con 0x7fcddc1047e0 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.215+0000 7fcddb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc114150 con 0x7fcddc103e30 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc1047e0 0x7fcddc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc103e30 0x7fcddc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc1047e0 0x7fcddc19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38146/0 (socket says 172.21.15.46:38146) 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9400640 1 -- 172.21.15.46:0/1423120446 learned_addr learned my addr 172.21.15.46:0/1423120446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdda800640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcddc108f60 0x7fcddc19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9e00640 1 -- 172.21.15.46:0/1423120446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc1047e0 msgr2=0x7fcddc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9e00640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc1047e0 0x7fcddc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9e00640 1 -- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcddc108f60 msgr2=0x7fcddc19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9e00640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcddc108f60 0x7fcddc19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.216+0000 7fcdd9e00640 1 -- 172.21.15.46:0/1423120446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc19f4d0 con 0x7fcddc103e30 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.217+0000 7fcdd9e00640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc103e30 0x7fcddc19a520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9f0 tx=0x7fcdcc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.217+0000 7fcdd2a00640 1 -- 172.21.15.46:0/1423120446 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00ccc0 con 0x7fcddc103e30 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.217+0000 7fcdd2a00640 1 -- 172.21.15.46:0/1423120446 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc010910 con 0x7fcddc103e30 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.217+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc19f7c0 con 0x7fcddc103e30 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.217+0000 7fcdd2a00640 1 -- 172.21.15.46:0/1423120446 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc018800 con 0x7fcddc103e30 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.217+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc1a70a0 con 0x7fcddc103e30 2024-09-10T07:35:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.218+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcddc105310 con 0x7fcddc103e30 2024-09-10T07:35:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.221+0000 7fcdd2a00640 1 -- 172.21.15.46:0/1423120446 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcdcc010ac0 con 0x7fcddc103e30 2024-09-10T07:35:20.626 INFO:teuthology.orchestra.run.smithi046.stdout:34359738461 2024-09-10T07:35:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.222+0000 7fcdd2a00640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcdc00a2c10 0x7fcdc00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.222+0000 7fcdd2a00640 1 -- 172.21.15.46:0/1423120446 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcdcc0cfc50 con 0x7fcddc103e30 2024-09-10T07:35:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.223+0000 7fcdd9400640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcdc00a2c10 0x7fcdc00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.223+0000 7fcdd9400640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcdc00a2c10 0x7fcdc00a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80096c0 tx=0x7fcdc80046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.224+0000 7fcdd2a00640 1 -- 172.21.15.46:0/1423120446 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcdcc09c670 con 0x7fcddc103e30 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.412+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fcddc104210 con 0x7fcddc103e30 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.413+0000 7fcdd2a00640 1 -- 172.21.15.46:0/1423120446 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcdcc09bf70 con 0x7fcddc103e30 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.415+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcdc00a2c10 msgr2=0x7fcdc00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.415+0000 7fcddb200640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcdc00a2c10 0x7fcdc00a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80096c0 tx=0x7fcdc80046c0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.415+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc103e30 msgr2=0x7fcddc19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.415+0000 7fcddb200640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc103e30 0x7fcddc19a520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9f0 tx=0x7fcdcc00edf0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.416+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 shutdown_connections 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.416+0000 7fcddb200640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcdc00a2c10 0x7fcdc00a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.416+0000 7fcddb200640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcddc108f60 0x7fcddc19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.416+0000 7fcddb200640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc1047e0 0x7fcddc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.416+0000 7fcddb200640 1 --2- 172.21.15.46:0/1423120446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcddc103e30 0x7fcddc19a520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.416+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 >> 172.21.15.46:0/1423120446 conn(0x7fcddc0ff910 msgr2=0x7fcddc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.416+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 shutdown_connections 2024-09-10T07:35:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:20.416+0000 7fcddb200640 1 -- 172.21.15.46:0/1423120446 wait complete. 2024-09-10T07:35:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:20 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1305328669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T07:35:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:20 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1423120446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T07:35:21.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:20 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1305328669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T07:35:21.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:20 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1423120446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T07:35:21.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:20 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1305328669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T07:35:21.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:20 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1423120446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 -- 172.21.15.46:0/2425116801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc10e3c0 msgr2=0x7f3dbc110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/2425116801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc10e3c0 0x7f3dbc110810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009ca0 tx=0x7f3db402a950 comp rx=0 tx=0).stop 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 -- 172.21.15.46:0/2425116801 shutdown_connections 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/2425116801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc10e3c0 0x7f3dbc110810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/2425116801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dbc074f40 0x7f3dbc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/2425116801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3dbc074620 0x7f3dbc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 -- 172.21.15.46:0/2425116801 >> 172.21.15.46:0/2425116801 conn(0x7f3dbc0fd8c0 msgr2=0x7f3dbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 -- 172.21.15.46:0/2425116801 shutdown_connections 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.367+0000 7f3dc1e00640 1 -- 172.21.15.46:0/2425116801 wait complete. 2024-09-10T07:35:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.368+0000 7f3dc1e00640 1 Processor -- start 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.368+0000 7f3dc1e00640 1 -- start start 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3dc1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dbc074620 0x7f3dbc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3dc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc074f40 0x7f3dbc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3dc1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3dbc10e3c0 0x7f3dbc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3dc1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc114170 con 0x7f3dbc074f40 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3dc1e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc113ff0 con 0x7f3dbc10e3c0 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3dc1e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc1142f0 con 0x7f3dbc074620 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3db3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc074f40 0x7f3dbc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3dbaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dbc074620 0x7f3dbc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3db3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc074f40 0x7f3dbc19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56102/0 (socket says 172.21.15.46:56102) 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3db3e00640 1 -- 172.21.15.46:0/958021337 learned_addr learned my addr 172.21.15.46:0/958021337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3dbb400640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3dbc10e3c0 0x7f3dbc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3db3e00640 1 -- 172.21.15.46:0/958021337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dbc074620 msgr2=0x7f3dbc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3db3e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dbc074620 0x7f3dbc19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3db3e00640 1 -- 172.21.15.46:0/958021337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3dbc10e3c0 msgr2=0x7f3dbc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3db3e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3dbc10e3c0 0x7f3dbc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.369+0000 7f3db3e00640 1 -- 172.21.15.46:0/958021337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dbc19f620 con 0x7f3dbc074f40 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.370+0000 7f3db3e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc074f40 0x7f3dbc19abb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e9f0 tx=0x7f3da400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.370+0000 7f3dc0a00640 1 -- 172.21.15.46:0/958021337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da400cd80 con 0x7f3dbc074f40 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.370+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dbc19f910 con 0x7f3dbc074f40 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.370+0000 7f3dc0a00640 1 -- 172.21.15.46:0/958021337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da4005500 con 0x7f3dbc074f40 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.370+0000 7f3dc0a00640 1 -- 172.21.15.46:0/958021337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4010690 con 0x7f3dbc074f40 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.370+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dbc1a71f0 con 0x7f3dbc074f40 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.372+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dbc074a00 con 0x7f3dbc074f40 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.373+0000 7f3dc0a00640 1 -- 172.21.15.46:0/958021337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3da40046c0 con 0x7f3dbc074f40 2024-09-10T07:35:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.374+0000 7f3dc0a00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3da00a2c10 0x7f3da00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.375+0000 7f3dc0a00640 1 -- 172.21.15.46:0/958021337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3da40cf950 con 0x7f3dbc074f40 2024-09-10T07:35:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.375+0000 7f3dbaa00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3da00a2c10 0x7f3da00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.376+0000 7f3dbaa00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3da00a2c10 0x7f3da00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009bd0 tx=0x7f3dac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.379+0000 7f3dc0a00640 1 -- 172.21.15.46:0/958021337 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3da409c370 con 0x7f3dbc074f40 2024-09-10T07:35:21.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.569+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f3dbc108fb0 con 0x7f3dbc074f40 2024-09-10T07:35:21.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.569+0000 7f3dc0a00640 1 -- 172.21.15.46:0/958021337 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3da409bc70 con 0x7f3dbc074f40 2024-09-10T07:35:21.569 INFO:teuthology.orchestra.run.smithi046.stdout:244813135923 2024-09-10T07:35:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3da00a2c10 msgr2=0x7f3da00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3da00a2c10 0x7f3da00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009bd0 tx=0x7f3dac005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc074f40 msgr2=0x7f3dbc19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc074f40 0x7f3dbc19abb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e9f0 tx=0x7f3da400edf0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 shutdown_connections 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3dbc10e3c0 0x7f3dbc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3da00a2c10 0x7f3da00a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dbc074f40 0x7f3dbc19abb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 --2- 172.21.15.46:0/958021337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dbc074620 0x7f3dbc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 >> 172.21.15.46:0/958021337 conn(0x7f3dbc0fd8c0 msgr2=0x7f3dbc0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 shutdown_connections 2024-09-10T07:35:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.572+0000 7f3dc1e00640 1 -- 172.21.15.46:0/958021337 wait complete. 2024-09-10T07:35:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 -- 172.21.15.46:0/1088504545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d54074eb0 msgr2=0x7f4d541124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 --2- 172.21.15.46:0/1088504545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d54074eb0 0x7f4d541124a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009900 tx=0x7f4d3c02a480 comp rx=0 tx=0).stop 2024-09-10T07:35:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 -- 172.21.15.46:0/1088504545 shutdown_connections 2024-09-10T07:35:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 --2- 172.21.15.46:0/1088504545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d541129e0 0x7f4d54114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 --2- 172.21.15.46:0/1088504545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d54074eb0 0x7f4d541124a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 --2- 172.21.15.46:0/1088504545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d54074590 0x7f4d54074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 -- 172.21.15.46:0/1088504545 >> 172.21.15.46:0/1088504545 conn(0x7f4d540fd710 msgr2=0x7f4d540ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 -- 172.21.15.46:0/1088504545 shutdown_connections 2024-09-10T07:35:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.722+0000 7f4d59600640 1 -- 172.21.15.46:0/1088504545 wait complete. 2024-09-10T07:35:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.723+0000 7f4d59600640 1 Processor -- start 2024-09-10T07:35:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d59600640 1 -- start start 2024-09-10T07:35:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d59600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d54074590 0x7f4d541a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d59600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d54074eb0 0x7f4d541a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d59600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d541129e0 0x7f4d5419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:21.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d59600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54118390 con 0x7f4d541129e0 2024-09-10T07:35:21.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d59600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54118210 con 0x7f4d54074eb0 2024-09-10T07:35:21.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d59600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54118510 con 0x7f4d54074590 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d4be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d54074eb0 0x7f4d541a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d4be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d54074eb0 0x7f4d541a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38212/0 (socket says 172.21.15.46:38212) 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.724+0000 7f4d4be00640 1 -- 172.21.15.46:0/4036044824 learned_addr learned my addr 172.21.15.46:0/4036044824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.725+0000 7f4d53e00640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d54074590 0x7f4d541a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.725+0000 7f4d58c00640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d541129e0 0x7f4d5419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.725+0000 7f4d4be00640 1 -- 172.21.15.46:0/4036044824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d54074590 msgr2=0x7f4d541a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.725+0000 7f4d4be00640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d54074590 0x7f4d541a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.725+0000 7f4d4be00640 1 -- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d541129e0 msgr2=0x7f4d5419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.725+0000 7f4d4be00640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d541129e0 0x7f4d5419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.725+0000 7f4d4be00640 1 -- 172.21.15.46:0/4036044824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5419ef30 con 0x7f4d54074eb0 2024-09-10T07:35:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.725+0000 7f4d4be00640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d54074eb0 0x7f4d541a4b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c006010 tx=0x7f4d3c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:21.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.726+0000 7f4d51600640 1 -- 172.21.15.46:0/4036044824 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c03f030 con 0x7f4d54074eb0 2024-09-10T07:35:21.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.726+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5419f1c0 con 0x7f4d54074eb0 2024-09-10T07:35:21.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.726+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541ab470 con 0x7f4d54074eb0 2024-09-10T07:35:21.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.726+0000 7f4d51600640 1 -- 172.21.15.46:0/4036044824 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c003ce0 con 0x7f4d54074eb0 2024-09-10T07:35:21.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.726+0000 7f4d51600640 1 -- 172.21.15.46:0/4036044824 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c035070 con 0x7f4d54074eb0 2024-09-10T07:35:21.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.727+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d54075cd0 con 0x7f4d54074eb0 2024-09-10T07:35:21.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.729+0000 7f4d51600640 1 -- 172.21.15.46:0/4036044824 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4d3c042030 con 0x7f4d54074eb0 2024-09-10T07:35:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.730+0000 7f4d51600640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d380a2c90 0x7f4d380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.731+0000 7f4d51600640 1 -- 172.21.15.46:0/4036044824 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d3c0efc70 con 0x7f4d54074eb0 2024-09-10T07:35:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.731+0000 7f4d53e00640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d380a2c90 0x7f4d380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.731+0000 7f4d53e00640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d380a2c90 0x7f4d380a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400bbe0 tx=0x7f4d4400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.732+0000 7f4d51600640 1 -- 172.21.15.46:0/4036044824 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d3c0bc100 con 0x7f4d54074eb0 2024-09-10T07:35:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.934+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f4d54104e20 con 0x7f4d54074eb0 2024-09-10T07:35:21.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:21 smithi046 ceph-mon[30705]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-10T07:35:21.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:21 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/958021337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T07:35:21.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:21 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1433703409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T07:35:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.934+0000 7f4d51600640 1 -- 172.21.15.46:0/4036044824 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4d3c0c0400 con 0x7f4d54074eb0 2024-09-10T07:35:21.935 INFO:teuthology.orchestra.run.smithi046.stdout:300647710761 2024-09-10T07:35:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.936+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d380a2c90 msgr2=0x7f4d380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.936+0000 7f4d59600640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d380a2c90 0x7f4d380a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400bbe0 tx=0x7f4d4400b390 comp rx=0 tx=0).stop 2024-09-10T07:35:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.936+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d54074eb0 msgr2=0x7f4d541a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.936+0000 7f4d59600640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d54074eb0 0x7f4d541a4b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c006010 tx=0x7f4d3c0042f0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.937+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 shutdown_connections 2024-09-10T07:35:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.937+0000 7f4d59600640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d380a2c90 0x7f4d380a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.937+0000 7f4d59600640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d541129e0 0x7f4d5419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.937+0000 7f4d59600640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d54074eb0 0x7f4d541a4b50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.937+0000 7f4d59600640 1 --2- 172.21.15.46:0/4036044824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d54074590 0x7f4d541a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.937+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 >> 172.21.15.46:0/4036044824 conn(0x7f4d540fd710 msgr2=0x7f4d540ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.937+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 shutdown_connections 2024-09-10T07:35:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.937+0000 7f4d59600640 1 -- 172.21.15.46:0/4036044824 wait complete. 2024-09-10T07:35:22.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:21 smithi181 ceph-mon[35397]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-10T07:35:22.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:21 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/958021337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T07:35:22.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:21 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1433703409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T07:35:22.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:21 smithi063 ceph-mon[35020]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-10T07:35:22.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:21 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/958021337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T07:35:22.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:21 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1433703409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T07:35:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 -- 172.21.15.46:0/43944803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194109120 msgr2=0x7f3194110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 --2- 172.21.15.46:0/43944803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194109120 0x7f3194110ca0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f318c009ca0 tx=0x7f318c02a8b0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 -- 172.21.15.46:0/43944803 shutdown_connections 2024-09-10T07:35:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 --2- 172.21.15.46:0/43944803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194109120 0x7f3194110ca0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 --2- 172.21.15.46:0/43944803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31941049a0 0x7f31941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 --2- 172.21.15.46:0/43944803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3194103ff0 0x7f31941043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 -- 172.21.15.46:0/43944803 >> 172.21.15.46:0/43944803 conn(0x7f31940ffa90 msgr2=0x7f3194101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 -- 172.21.15.46:0/43944803 shutdown_connections 2024-09-10T07:35:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.543+0000 7f3199800640 1 -- 172.21.15.46:0/43944803 wait complete. 2024-09-10T07:35:22.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.544+0000 7f3199800640 1 Processor -- start 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.544+0000 7f3199800640 1 -- start start 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3199800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194103ff0 0x7f319419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3199800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31941049a0 0x7f319419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3199800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3194109120 0x7f319419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3199800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3194114180 con 0x7f3194103ff0 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3199800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3194114000 con 0x7f3194109120 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3199800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3194114300 con 0x7f31941049a0 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f318be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31941049a0 0x7f319419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f318be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31941049a0 0x7f319419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33722/0 (socket says 172.21.15.46:33722) 2024-09-10T07:35:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f318be00640 1 -- 172.21.15.46:0/1016960302 learned_addr learned my addr 172.21.15.46:0/1016960302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3192a00640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194103ff0 0x7f319419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3193400640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3194109120 0x7f319419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f318be00640 1 -- 172.21.15.46:0/1016960302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3194109120 msgr2=0x7f319419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f318be00640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3194109120 0x7f319419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f318be00640 1 -- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194103ff0 msgr2=0x7f319419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f318be00640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194103ff0 0x7f319419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f318be00640 1 -- 172.21.15.46:0/1016960302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319419f5f0 con 0x7f31941049a0 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.545+0000 7f3192a00640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194103ff0 0x7f319419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.546+0000 7f318be00640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31941049a0 0x7f319419ab80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f317c00d950 tx=0x7f317c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.546+0000 7f318b400640 1 -- 172.21.15.46:0/1016960302 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c017070 con 0x7f31941049a0 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.546+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319419f8e0 con 0x7f31941049a0 2024-09-10T07:35:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.546+0000 7f318b400640 1 -- 172.21.15.46:0/1016960302 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f317c009c30 con 0x7f31941049a0 2024-09-10T07:35:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.546+0000 7f318b400640 1 -- 172.21.15.46:0/1016960302 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c00f6b0 con 0x7f31941049a0 2024-09-10T07:35:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.546+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31941a71c0 con 0x7f31941049a0 2024-09-10T07:35:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.547+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31941043d0 con 0x7f31941049a0 2024-09-10T07:35:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.549+0000 7f318b400640 1 -- 172.21.15.46:0/1016960302 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f317c0109d0 con 0x7f31941049a0 2024-09-10T07:35:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.551+0000 7f318b400640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31780a2c90 0x7f31780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.551+0000 7f318b400640 1 -- 172.21.15.46:0/1016960302 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f317c0cf0b0 con 0x7f31941049a0 2024-09-10T07:35:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.551+0000 7f3192a00640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31780a2c90 0x7f31780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.551+0000 7f3192a00640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31780a2c90 0x7f31780a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3184009c20 tx=0x7f3184005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.553+0000 7f318b400640 1 -- 172.21.15.46:0/1016960302 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f317c009de0 con 0x7f31941049a0 2024-09-10T07:35:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.740+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f31941a7420 con 0x7f31941049a0 2024-09-10T07:35:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.741+0000 7f318b400640 1 -- 172.21.15.46:0/1016960302 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f317c09bb50 con 0x7f31941049a0 2024-09-10T07:35:22.741 INFO:teuthology.orchestra.run.smithi046.stdout:193273528381 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31780a2c90 msgr2=0x7f31780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31780a2c90 0x7f31780a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3184009c20 tx=0x7f3184005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31941049a0 msgr2=0x7f319419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31941049a0 0x7f319419ab80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f317c00d950 tx=0x7f317c00de10 comp rx=0 tx=0).stop 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 shutdown_connections 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3194109120 0x7f319419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31941049a0 0x7f319419ab80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31780a2c90 0x7f31780a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 --2- 172.21.15.46:0/1016960302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3194103ff0 0x7f319419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 >> 172.21.15.46:0/1016960302 conn(0x7f31940ffa90 msgr2=0x7f3194100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 shutdown_connections 2024-09-10T07:35:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:22.743+0000 7f3199800640 1 -- 172.21.15.46:0/1016960302 wait complete. 2024-09-10T07:35:23.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:22 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/4036044824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T07:35:23.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:22 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1016960302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T07:35:23.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:22 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/4036044824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T07:35:23.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:22 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1016960302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T07:35:23.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:22 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/4036044824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T07:35:23.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:22 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1016960302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T07:35:23.200 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411410 for osd.2 2024-09-10T07:35:23.201 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 -- 172.21.15.46:0/667740059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f875c074ee0 msgr2=0x7f875c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 --2- 172.21.15.46:0/667740059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f875c074ee0 0x7f875c10dd20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f874802a480 comp rx=0 tx=0).stop 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 -- 172.21.15.46:0/667740059 shutdown_connections 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 --2- 172.21.15.46:0/667740059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875c10e260 0x7f875c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 --2- 172.21.15.46:0/667740059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f875c074ee0 0x7f875c10dd20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 --2- 172.21.15.46:0/667740059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f875c0745c0 0x7f875c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 -- 172.21.15.46:0/667740059 >> 172.21.15.46:0/667740059 conn(0x7f875c0fd740 msgr2=0x7f875c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 -- 172.21.15.46:0/667740059 shutdown_connections 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.594+0000 7f8761600640 1 -- 172.21.15.46:0/667740059 wait complete. 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.595+0000 7f8761600640 1 Processor -- start 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8761600640 1 -- start start 2024-09-10T07:35:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8761600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f875c0745c0 0x7f875c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8761600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f875c074ee0 0x7f875c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8761600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875c10e260 0x7f875c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8761600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f875c114000 con 0x7f875c10e260 2024-09-10T07:35:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8761600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f875c113e80 con 0x7f875c074ee0 2024-09-10T07:35:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8761600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f875c114180 con 0x7f875c0745c0 2024-09-10T07:35:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8760c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875c10e260 0x7f875c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8760c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875c10e260 0x7f875c19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56122/0 (socket says 172.21.15.46:56122) 2024-09-10T07:35:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.596+0000 7f8760c00640 1 -- 172.21.15.46:0/1433703409 learned_addr learned my addr 172.21.15.46:0/1433703409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f875be00640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f875c0745c0 0x7f875c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f875b400640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f875c074ee0 0x7f875c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f8760c00640 1 -- 172.21.15.46:0/1433703409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f875c0745c0 msgr2=0x7f875c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f8760c00640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f875c0745c0 0x7f875c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f8760c00640 1 -- 172.21.15.46:0/1433703409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f875c074ee0 msgr2=0x7f875c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f8760c00640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f875c074ee0 0x7f875c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f8760c00640 1 -- 172.21.15.46:0/1433703409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c19f550 con 0x7f875c10e260 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f8760c00640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875c10e260 0x7f875c19ee70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f875400eb40 tx=0x7f875400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.597+0000 7f8758c00640 1 -- 172.21.15.46:0/1433703409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754004b20 con 0x7f875c10e260 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.598+0000 7f8758c00640 1 -- 172.21.15.46:0/1433703409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8754004cc0 con 0x7f875c10e260 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.598+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875c19f840 con 0x7f875c10e260 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.598+0000 7f8758c00640 1 -- 172.21.15.46:0/1433703409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754010850 con 0x7f875c10e260 2024-09-10T07:35:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.598+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875c1a7120 con 0x7f875c10e260 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.599+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875c0749a0 con 0x7f875c10e260 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.601+0000 7f8758c00640 1 -- 172.21.15.46:0/1433703409 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8754005000 con 0x7f875c10e260 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.602+0000 7f8758c00640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87400a2c10 0x7f87400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.603+0000 7f8758c00640 1 -- 172.21.15.46:0/1433703409 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f87540cfca0 con 0x7f875c10e260 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.603+0000 7f875be00640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87400a2c10 0x7f87400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.603+0000 7f875be00640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87400a2c10 0x7f87400a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f874c009bd0 tx=0x7f874c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.609+0000 7f8758c00640 1 -- 172.21.15.46:0/1433703409 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f875409c6c0 con 0x7f875c10e260 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.808+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f875c19b900 con 0x7f875c10e260 2024-09-10T07:35:23.710 INFO:teuthology.orchestra.run.smithi046.stdout:279172874284 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.809+0000 7f8758c00640 1 -- 172.21.15.46:0/1433703409 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f875c19b900 con 0x7f875c10e260 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87400a2c10 msgr2=0x7f87400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87400a2c10 0x7f87400a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f874c009bd0 tx=0x7f874c005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875c10e260 msgr2=0x7f875c19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875c10e260 0x7f875c19ee70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f875400eb40 tx=0x7f875400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 shutdown_connections 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87400a2c10 0x7f87400a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875c10e260 0x7f875c19ee70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f875c074ee0 0x7f875c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 --2- 172.21.15.46:0/1433703409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f875c0745c0 0x7f875c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 >> 172.21.15.46:0/1433703409 conn(0x7f875c0fd740 msgr2=0x7f875c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 shutdown_connections 2024-09-10T07:35:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:21.813+0000 7f8761600640 1 -- 172.21.15.46:0/1433703409 wait complete. 2024-09-10T07:35:24.091 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:23 smithi046 ceph-mon[30705]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 12 B/s, 0 objects/s recovering 2024-09-10T07:35:24.092 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:23 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/97893213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T07:35:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:23 smithi181 ceph-mon[35397]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 12 B/s, 0 objects/s recovering 2024-09-10T07:35:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:23 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/97893213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T07:35:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:23 smithi063 ceph-mon[35020]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 12 B/s, 0 objects/s recovering 2024-09-10T07:35:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:23 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/97893213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T07:35:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.335+0000 7f39c2400640 1 -- 172.21.15.46:0/187146584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc109140 msgr2=0x7f39bc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.335+0000 7f39c2400640 1 --2- 172.21.15.46:0/187146584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc109140 0x7f39bc110cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009ca0 tx=0x7f39b402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.335+0000 7f39c2400640 1 -- 172.21.15.46:0/187146584 shutdown_connections 2024-09-10T07:35:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.335+0000 7f39c2400640 1 --2- 172.21.15.46:0/187146584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc109140 0x7f39bc110cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.335+0000 7f39c2400640 1 --2- 172.21.15.46:0/187146584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc1049c0 0x7f39bc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.335+0000 7f39c2400640 1 --2- 172.21.15.46:0/187146584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39bc104010 0x7f39bc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.335+0000 7f39c2400640 1 -- 172.21.15.46:0/187146584 >> 172.21.15.46:0/187146584 conn(0x7f39bc0ffa50 msgr2=0x7f39bc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.336+0000 7f39c2400640 1 -- 172.21.15.46:0/187146584 shutdown_connections 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.336+0000 7f39c2400640 1 -- 172.21.15.46:0/187146584 wait complete. 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.337+0000 7f39c2400640 1 Processor -- start 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.337+0000 7f39c2400640 1 -- start start 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.337+0000 7f39c2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc104010 0x7f39bc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.337+0000 7f39c2400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc1049c0 0x7f39bc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.337+0000 7f39c2400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39bc109140 0x7f39bc19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.337+0000 7f39c2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1141c0 con 0x7f39bc104010 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.337+0000 7f39c2400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc114040 con 0x7f39bc109140 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.337+0000 7f39c2400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc114340 con 0x7f39bc1049c0 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc104010 0x7f39bc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39bc109140 0x7f39bc19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc104010 0x7f39bc19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56036/0 (socket says 172.21.15.46:56036) 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 -- 172.21.15.46:0/2411609619 learned_addr learned my addr 172.21.15.46:0/2411609619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39baa00640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc1049c0 0x7f39bc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 -- 172.21.15.46:0/2411609619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc1049c0 msgr2=0x7f39bc19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc1049c0 0x7f39bc19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 -- 172.21.15.46:0/2411609619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39bc109140 msgr2=0x7f39bc19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39bc109140 0x7f39bc19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 -- 172.21.15.46:0/2411609619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc19f5b0 con 0x7f39bc104010 2024-09-10T07:35:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.338+0000 7f39bb400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc104010 0x7f39bc19a600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e9f0 tx=0x7f39ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.339+0000 7f39b3e00640 1 -- 172.21.15.46:0/2411609619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00cd80 con 0x7f39bc104010 2024-09-10T07:35:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.339+0000 7f39b3e00640 1 -- 172.21.15.46:0/2411609619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ac010910 con 0x7f39bc104010 2024-09-10T07:35:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.339+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc19f8a0 con 0x7f39bc104010 2024-09-10T07:35:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.339+0000 7f39b3e00640 1 -- 172.21.15.46:0/2411609619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac018990 con 0x7f39bc104010 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.339+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1a7180 con 0x7f39bc104010 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.340+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc1043f0 con 0x7f39bc104010 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.342+0000 7f39b3e00640 1 -- 172.21.15.46:0/2411609619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f39ac010ac0 con 0x7f39bc104010 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.343+0000 7f39b3e00640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39a00a2c10 0x7f39a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.343+0000 7f39b3e00640 1 -- 172.21.15.46:0/2411609619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f39ac0cf8c0 con 0x7f39bc104010 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.343+0000 7f39baa00640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39a00a2c10 0x7f39a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.344+0000 7f39baa00640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39a00a2c10 0x7f39a00a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39bc1002c0 tx=0x7f39a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.347+0000 7f39b3e00640 1 -- 172.21.15.46:0/2411609619 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39ac09bc90 con 0x7f39bc104010 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.539+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f39bc10bdf0 con 0x7f39bc104010 2024-09-10T07:35:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.540+0000 7f39b3e00640 1 -- 172.21.15.46:0/2411609619 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f39ac09ff90 con 0x7f39bc104010 2024-09-10T07:35:24.666 INFO:teuthology.orchestra.run.smithi046.stdout:107374182476 2024-09-10T07:35:24.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39a00a2c10 msgr2=0x7f39a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39a00a2c10 0x7f39a00a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39bc1002c0 tx=0x7f39a800b390 comp rx=0 tx=0).stop 2024-09-10T07:35:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc104010 msgr2=0x7f39bc19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc104010 0x7f39bc19a600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e9f0 tx=0x7f39ac00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 shutdown_connections 2024-09-10T07:35:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39bc109140 0x7f39bc19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc1049c0 0x7f39bc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39a00a2c10 0x7f39a00a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 --2- 172.21.15.46:0/2411609619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc104010 0x7f39bc19a600 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 >> 172.21.15.46:0/2411609619 conn(0x7f39bc0ffa50 msgr2=0x7f39bc100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 shutdown_connections 2024-09-10T07:35:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:18.542+0000 7f39c2400640 1 -- 172.21.15.46:0/2411609619 wait complete. 2024-09-10T07:35:25.599 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135921 got 244813135923 for osd.8 2024-09-10T07:35:25.599 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:25.727 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874281 got 279172874284 for osd.9 2024-09-10T07:35:25.727 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:26.048 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364852 got 214748364855 for osd.7 2024-09-10T07:35:26.048 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:26.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:25 smithi181 ceph-mon[35397]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-10T07:35:26.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:25 smithi063 ceph-mon[35020]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-10T07:35:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:25 smithi046 ceph-mon[30705]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-10T07:35:26.515 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182474 got 107374182476 for osd.3 2024-09-10T07:35:26.515 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:26.860 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710757 got 300647710761 for osd.10 2024-09-10T07:35:26.861 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:27.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528378 got 193273528381 for osd.6 2024-09-10T07:35:27.507 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:27.910 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738461 for osd.0 2024-09-10T07:35:27.910 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.630+0000 7f9ed1000640 1 -- 172.21.15.46:0/277619296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 msgr2=0x7f9ecc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.630+0000 7f9ed1000640 1 --2- 172.21.15.46:0/277619296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 0x7f9ecc110ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009ca0 tx=0x7f9ec402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.630+0000 7f9ed1000640 1 -- 172.21.15.46:0/277619296 shutdown_connections 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.630+0000 7f9ed1000640 1 --2- 172.21.15.46:0/277619296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 0x7f9ecc110ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.630+0000 7f9ed1000640 1 --2- 172.21.15.46:0/277619296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ecc1049a0 0x7f9ecc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.630+0000 7f9ed1000640 1 --2- 172.21.15.46:0/277619296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc103ff0 0x7f9ecc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.630+0000 7f9ed1000640 1 -- 172.21.15.46:0/277619296 >> 172.21.15.46:0/277619296 conn(0x7f9ecc0ffa90 msgr2=0x7f9ecc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.631+0000 7f9ed1000640 1 -- 172.21.15.46:0/277619296 shutdown_connections 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.631+0000 7f9ed1000640 1 -- 172.21.15.46:0/277619296 wait complete. 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.631+0000 7f9ed1000640 1 Processor -- start 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ed1000640 1 -- start start 2024-09-10T07:35:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ed1000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ecc103ff0 0x7f9ecc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ed1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc1049a0 0x7f9ecc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ed1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 0x7f9ecc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ed1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc114180 con 0x7f9ecc109120 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ed1000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc114000 con 0x7f9ecc1049a0 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ed1000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc114300 con 0x7f9ecc103ff0 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ec9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ecc103ff0 0x7f9ecc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9eca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 0x7f9ecc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ec9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ecc103ff0 0x7f9ecc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33746/0 (socket says 172.21.15.46:33746) 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ec9e00640 1 -- 172.21.15.46:0/97893213 learned_addr learned my addr 172.21.15.46:0/97893213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.632+0000 7f9ec3e00640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc1049a0 0x7f9ecc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec9e00640 1 -- 172.21.15.46:0/97893213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc1049a0 msgr2=0x7f9ecc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec9e00640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc1049a0 0x7f9ecc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec9e00640 1 -- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 msgr2=0x7f9ecc19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec9e00640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 0x7f9ecc19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec9e00640 1 -- 172.21.15.46:0/97893213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc19f570 con 0x7f9ecc103ff0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9eca800640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 0x7f9ecc19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec9e00640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ecc103ff0 0x7f9ecc19a5c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00e9f0 tx=0x7f9ebc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec2a00640 1 -- 172.21.15.46:0/97893213 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00cd80 con 0x7f9ecc103ff0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc19f860 con 0x7f9ecc103ff0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec2a00640 1 -- 172.21.15.46:0/97893213 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc010910 con 0x7f9ecc103ff0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ec2a00640 1 -- 172.21.15.46:0/97893213 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0188f0 con 0x7f9ecc103ff0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.633+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc1a7140 con 0x7f9ecc103ff0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.634+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ecc1043d0 con 0x7f9ecc103ff0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.636+0000 7f9ec2a00640 1 -- 172.21.15.46:0/97893213 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9ebc010ac0 con 0x7f9ecc103ff0 2024-09-10T07:35:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.638+0000 7f9ec2a00640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9eb00a2c10 0x7f9eb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.638+0000 7f9ec2a00640 1 -- 172.21.15.46:0/97893213 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9ebc0cfd40 con 0x7f9ecc103ff0 2024-09-10T07:35:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.638+0000 7f9ec3e00640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9eb00a2c10 0x7f9eb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.639+0000 7f9ec3e00640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9eb00a2c10 0x7f9eb00a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc108a30 tx=0x7f9eb400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:28.074 INFO:teuthology.orchestra.run.smithi046.stdout:55834574936 2024-09-10T07:35:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.640+0000 7f9ec2a00640 1 -- 172.21.15.46:0/97893213 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ebc09c110 con 0x7f9ecc103ff0 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.830+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9ecc19b920 con 0x7f9ecc103ff0 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.830+0000 7f9ec2a00640 1 -- 172.21.15.46:0/97893213 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ebc0a0410 con 0x7f9ecc103ff0 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9eb00a2c10 msgr2=0x7f9eb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9eb00a2c10 0x7f9eb00a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc108a30 tx=0x7f9eb400b410 comp rx=0 tx=0).stop 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ecc103ff0 msgr2=0x7f9ecc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ecc103ff0 0x7f9ecc19a5c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00e9f0 tx=0x7f9ebc00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 shutdown_connections 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9eb00a2c10 0x7f9eb00a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ecc109120 0x7f9ecc19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ecc1049a0 0x7f9ecc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 --2- 172.21.15.46:0/97893213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ecc103ff0 0x7f9ecc19a5c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.833+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 >> 172.21.15.46:0/97893213 conn(0x7f9ecc0ffa90 msgr2=0x7f9ecc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.834+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 shutdown_connections 2024-09-10T07:35:28.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:23.834+0000 7f9ed1000640 1 -- 172.21.15.46:0/97893213 wait complete. 2024-09-10T07:35:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:27 smithi181 ceph-mon[35397]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:28.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:27 smithi063 ceph-mon[35020]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:28.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:27 smithi046 ceph-mon[30705]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:28.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574936 for osd.1 2024-09-10T07:35:28.808 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 -- 172.21.15.46:0/982652561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0076720 msgr2=0x7f6fd0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 --2- 172.21.15.46:0/982652561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0076720 0x7f6fd0076b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009900 tx=0x7f6fc402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 -- 172.21.15.46:0/982652561 shutdown_connections 2024-09-10T07:35:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 --2- 172.21.15.46:0/982652561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd0112a50 0x7f6fd0114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 --2- 172.21.15.46:0/982652561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0077040 0x7f6fd0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 --2- 172.21.15.46:0/982652561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0076720 0x7f6fd0076b00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 -- 172.21.15.46:0/982652561 >> 172.21.15.46:0/982652561 conn(0x7f6fd00ffa50 msgr2=0x7f6fd0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 -- 172.21.15.46:0/982652561 shutdown_connections 2024-09-10T07:35:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.395+0000 7f6fd7600640 1 -- 172.21.15.46:0/982652561 wait complete. 2024-09-10T07:35:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.396+0000 7f6fd7600640 1 Processor -- start 2024-09-10T07:35:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd7600640 1 -- start start 2024-09-10T07:35:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd7600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd0076720 0x7f6fd01a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0077040 0x7f6fd01a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd7600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0112a50 0x7f6fd019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01185c0 con 0x7f6fd0077040 2024-09-10T07:35:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd7600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd0118440 con 0x7f6fd0076720 2024-09-10T07:35:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd7600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd0118740 con 0x7f6fd0112a50 2024-09-10T07:35:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd0076720 0x7f6fd01a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0077040 0x7f6fd01a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.397+0000 7f6fd4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd0076720 0x7f6fd01a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44250/0 (socket says 172.21.15.46:44250) 2024-09-10T07:35:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.398+0000 7f6fd4c00640 1 -- 172.21.15.46:0/2490504747 learned_addr learned my addr 172.21.15.46:0/2490504747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.398+0000 7f6fd5600640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0112a50 0x7f6fd019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.398+0000 7f6fd4c00640 1 -- 172.21.15.46:0/2490504747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0112a50 msgr2=0x7f6fd019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.398+0000 7f6fd4c00640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0112a50 0x7f6fd019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.398+0000 7f6fd4c00640 1 -- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0077040 msgr2=0x7f6fd01a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.398+0000 7f6fd4c00640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0077040 0x7f6fd01a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.398+0000 7f6fd4c00640 1 -- 172.21.15.46:0/2490504747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd019f040 con 0x7f6fd0076720 2024-09-10T07:35:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.398+0000 7f6fd4c00640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd0076720 0x7f6fd01a4750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4006010 tx=0x7f6fc4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.399+0000 7f6fcd600640 1 -- 172.21.15.46:0/2490504747 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc403f030 con 0x7f6fd0076720 2024-09-10T07:35:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.399+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd019f2d0 con 0x7f6fd0076720 2024-09-10T07:35:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.399+0000 7f6fcd600640 1 -- 172.21.15.46:0/2490504747 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc4004440 con 0x7f6fd0076720 2024-09-10T07:35:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.399+0000 7f6fcd600640 1 -- 172.21.15.46:0/2490504747 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4035070 con 0x7f6fd0076720 2024-09-10T07:35:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.399+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd01ab620 con 0x7f6fd0076720 2024-09-10T07:35:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.400+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd0076b00 con 0x7f6fd0076720 2024-09-10T07:35:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.402+0000 7f6fcd600640 1 -- 172.21.15.46:0/2490504747 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6fc4042030 con 0x7f6fd0076720 2024-09-10T07:35:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.403+0000 7f6fcd600640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fb80a2c10 0x7f6fb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.403+0000 7f6fcd600640 1 -- 172.21.15.46:0/2490504747 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6fc40efa40 con 0x7f6fd0076720 2024-09-10T07:35:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.403+0000 7f6fcfe00640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fb80a2c10 0x7f6fb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.404+0000 7f6fcfe00640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fb80a2c10 0x7f6fb80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0079220 tx=0x7f6fbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.405+0000 7f6fcd600640 1 -- 172.21.15.46:0/2490504747 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fc40bc460 con 0x7f6fd0076720 2024-09-10T07:35:29.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.595+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6fd01ab7f0 con 0x7f6fd0076720 2024-09-10T07:35:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.595+0000 7f6fcd600640 1 -- 172.21.15.46:0/2490504747 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6fc40bbd60 con 0x7f6fd0076720 2024-09-10T07:35:29.596 INFO:teuthology.orchestra.run.smithi046.stdout:335007449125 2024-09-10T07:35:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fb80a2c10 msgr2=0x7f6fb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fb80a2c10 0x7f6fb80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0079220 tx=0x7f6fbc005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd0076720 msgr2=0x7f6fd01a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd0076720 0x7f6fd01a4750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4006010 tx=0x7f6fc4004040 comp rx=0 tx=0).stop 2024-09-10T07:35:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 shutdown_connections 2024-09-10T07:35:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0112a50 0x7f6fd019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fb80a2c10 0x7f6fb80a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0077040 0x7f6fd01a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 --2- 172.21.15.46:0/2490504747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd0076720 0x7f6fd01a4750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 >> 172.21.15.46:0/2490504747 conn(0x7f6fd00ffa50 msgr2=0x7f6fd0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:29.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 shutdown_connections 2024-09-10T07:35:29.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:29.598+0000 7f6fd7600640 1 -- 172.21.15.46:0/2490504747 wait complete. 2024-09-10T07:35:29.969 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:29 smithi046 ceph-mon[30705]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:29.969 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:29 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2490504747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T07:35:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:29 smithi181 ceph-mon[35397]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:29 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2490504747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T07:35:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:29 smithi063 ceph-mon[35020]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:29 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2490504747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T07:35:30.400 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449121 got 335007449125 for osd.11 2024-09-10T07:35:30.400 DEBUG:teuthology.parallel:result is None 2024-09-10T07:35:30.400 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T07:35:30.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T07:35:30.779 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.353+0000 7fc58b800640 1 -- 172.21.15.46:0/673695102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5841049a0 msgr2=0x7fc5841089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.353+0000 7fc58b800640 1 --2- 172.21.15.46:0/673695102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5841049a0 0x7fc5841089f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc570009900 tx=0x7fc57002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.353+0000 7fc58b800640 1 -- 172.21.15.46:0/673695102 shutdown_connections 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.353+0000 7fc58b800640 1 --2- 172.21.15.46:0/673695102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc584109120 0x7fc584110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.353+0000 7fc58b800640 1 --2- 172.21.15.46:0/673695102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5841049a0 0x7fc5841089f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.353+0000 7fc58b800640 1 --2- 172.21.15.46:0/673695102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc584103ff0 0x7fc5841043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.353+0000 7fc58b800640 1 -- 172.21.15.46:0/673695102 >> 172.21.15.46:0/673695102 conn(0x7fc5840ffa90 msgr2=0x7fc584101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.354+0000 7fc58b800640 1 -- 172.21.15.46:0/673695102 shutdown_connections 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.354+0000 7fc58b800640 1 -- 172.21.15.46:0/673695102 wait complete. 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.354+0000 7fc58b800640 1 Processor -- start 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc58b800640 1 -- start start 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc58b800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc584103ff0 0x7fc58419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc58b800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5841049a0 0x7fc58419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc58b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc584109120 0x7fc58419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc58b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc584114170 con 0x7fc584109120 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc58b800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc584113ff0 con 0x7fc584103ff0 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc58b800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841142f0 con 0x7fc5841049a0 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc589800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc584109120 0x7fc58419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc589800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc584109120 0x7fc58419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41124/0 (socket says 172.21.15.46:41124) 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc589800640 1 -- 172.21.15.46:0/1685896193 learned_addr learned my addr 172.21.15.46:0/1685896193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc588e00640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc584103ff0 0x7fc58419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.355+0000 7fc583e00640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5841049a0 0x7fc58419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc589800640 1 -- 172.21.15.46:0/1685896193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5841049a0 msgr2=0x7fc58419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc589800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5841049a0 0x7fc58419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc589800640 1 -- 172.21.15.46:0/1685896193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc584103ff0 msgr2=0x7fc58419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc589800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc584103ff0 0x7fc58419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc589800640 1 -- 172.21.15.46:0/1685896193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58419f600 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc589800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc584109120 0x7fc58419ef20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00eaa0 tx=0x7fc57c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc581600640 1 -- 172.21.15.46:0/1685896193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00cdb0 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc581600640 1 -- 172.21.15.46:0/1685896193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc57c005020 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58419f8f0 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc581600640 1 -- 172.21.15.46:0/1685896193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0116b0 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.356+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841a71d0 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.357+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5841043d0 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.359+0000 7fc581600640 1 -- 172.21.15.46:0/1685896193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc57c005560 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.361+0000 7fc581600640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc56c0a2c90 0x7fc56c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.362+0000 7fc581600640 1 -- 172.21.15.46:0/1685896193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc57c0cf680 con 0x7fc584109120 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.362+0000 7fc588e00640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc56c0a2c90 0x7fc56c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.362+0000 7fc588e00640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc56c0a2c90 0x7fc56c0a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc578009bd0 tx=0x7fc578005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.366+0000 7fc581600640 1 -- 172.21.15.46:0/1685896193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc57c016030 con 0x7fc584109120 2024-09-10T07:35:31.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.549+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc584062630 con 0x7fc56c0a2c90 2024-09-10T07:35:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.558+0000 7fc581600640 1 -- 172.21.15.46:0/1685896193 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340659 (secure 0 0 0) 0x7fc584062630 con 0x7fc56c0a2c90 2024-09-10T07:35:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:35:31.563 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:35:31.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.562+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc56c0a2c90 msgr2=0x7fc56c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.562+0000 7fc58b800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc56c0a2c90 0x7fc56c0a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc578009bd0 tx=0x7fc578005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.562+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc584109120 msgr2=0x7fc58419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.562+0000 7fc58b800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc584109120 0x7fc58419ef20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00eaa0 tx=0x7fc57c00ef60 comp rx=0 tx=0).stop 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.563+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 shutdown_connections 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.563+0000 7fc58b800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc56c0a2c90 0x7fc56c0a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.563+0000 7fc58b800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc584109120 0x7fc58419ef20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.563+0000 7fc58b800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5841049a0 0x7fc58419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.563+0000 7fc58b800640 1 --2- 172.21.15.46:0/1685896193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc584103ff0 0x7fc58419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.563+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 >> 172.21.15.46:0/1685896193 conn(0x7fc5840ffa90 msgr2=0x7fc584100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.563+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 shutdown_connections 2024-09-10T07:35:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:31.563+0000 7fc58b800640 1 -- 172.21.15.46:0/1685896193 wait complete. 2024-09-10T07:35:32.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:31 smithi181 ceph-mon[35397]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:31 smithi063 ceph-mon[35020]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:32.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:31 smithi046 ceph-mon[30705]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T07:35:32.452 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-10T07:35:30.489589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662432,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":796,"num_read_kb":523,"num_write":434,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":355,"ondisk_log_size":355,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358376,"kb_used_data":13220,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583448,"statfs":{"total":1151940427776,"available":1151573450752,"internally_reserved":0,"allocated":13537280,"data_stored":8688290,"data_compressed":498372,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003367"},"pg_stats":[{"pgid":"2.1f","version":"95'13","reported_seq":119,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:26.905210+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:35:26.905210+0000","last_peered":"2024-09-10T07:35:26.905210+0000","last_clean":"2024-09-10T07:35:26.905210+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:35:26.905210+0000","last_undegraded":"2024-09-10T07:35:26.905210+0000","last_fullsized":"2024-09-10T07:35:26.905210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977492+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:35:14.977492+0000","last_peered":"2024-09-10T07:35:14.977492+0000","last_clean":"2024-09-10T07:35:14.977492+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:35:14.977492+0000","last_undegraded":"2024-09-10T07:35:14.977492+0000","last_fullsized":"2024-09-10T07:35:14.977492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842304+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:35:14.842304+0000","last_peered":"2024-09-10T07:35:14.842304+0000","last_clean":"2024-09-10T07:35:14.842304+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:35:14.842304+0000","last_undegraded":"2024-09-10T07:35:14.842304+0000","last_fullsized":"2024-09-10T07:35:14.842304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977693+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:35:14.977693+0000","last_peered":"2024-09-10T07:35:14.977693+0000","last_clean":"2024-09-10T07:35:14.977693+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:35:14.977693+0000","last_undegraded":"2024-09-10T07:35:14.977693+0000","last_fullsized":"2024-09-10T07:35:14.977693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.856805+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:35:14.856805+0000","last_peered":"2024-09-10T07:35:14.856805+0000","last_clean":"2024-09-10T07:35:14.856805+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:35:14.856805+0000","last_undegraded":"2024-09-10T07:35:14.856805+0000","last_fullsized":"2024-09-10T07:35:14.856805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881733+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:35:14.881733+0000","last_peered":"2024-09-10T07:35:14.881733+0000","last_clean":"2024-09-10T07:35:14.881733+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:35:14.881733+0000","last_undegraded":"2024-09-10T07:35:14.881733+0000","last_fullsized":"2024-09-10T07:35:14.881733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":56,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854507+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:35:14.854507+0000","last_peered":"2024-09-10T07:35:14.854507+0000","last_clean":"2024-09-10T07:35:14.854507+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:35:14.854507+0000","last_undegraded":"2024-09-10T07:35:14.854507+0000","last_fullsized":"2024-09-10T07:35:14.854507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976867+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:35:14.976867+0000","last_peered":"2024-09-10T07:35:14.976867+0000","last_clean":"2024-09-10T07:35:14.976867+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:35:14.976867+0000","last_undegraded":"2024-09-10T07:35:14.976867+0000","last_fullsized":"2024-09-10T07:35:14.976867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881927+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:35:14.881927+0000","last_peered":"2024-09-10T07:35:14.881927+0000","last_clean":"2024-09-10T07:35:14.881927+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:35:14.881927+0000","last_undegraded":"2024-09-10T07:35:14.881927+0000","last_fullsized":"2024-09-10T07:35:14.881927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854348+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:35:14.854348+0000","last_peered":"2024-09-10T07:35:14.854348+0000","last_clean":"2024-09-10T07:35:14.854348+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:35:14.854348+0000","last_undegraded":"2024-09-10T07:35:14.854348+0000","last_fullsized":"2024-09-10T07:35:14.854348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880410+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:35:14.880410+0000","last_peered":"2024-09-10T07:35:14.880410+0000","last_clean":"2024-09-10T07:35:14.880410+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:35:14.880410+0000","last_undegraded":"2024-09-10T07:35:14.880410+0000","last_fullsized":"2024-09-10T07:35:14.880410+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881828+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:35:14.881828+0000","last_peered":"2024-09-10T07:35:14.881828+0000","last_clean":"2024-09-10T07:35:14.881828+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:35:14.881828+0000","last_undegraded":"2024-09-10T07:35:14.881828+0000","last_fullsized":"2024-09-10T07:35:14.881828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842879+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:35:14.842879+0000","last_peered":"2024-09-10T07:35:14.842879+0000","last_clean":"2024-09-10T07:35:14.842879+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:35:14.842879+0000","last_undegraded":"2024-09-10T07:35:14.842879+0000","last_fullsized":"2024-09-10T07:35:14.842879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854012+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:35:14.854012+0000","last_peered":"2024-09-10T07:35:14.854012+0000","last_clean":"2024-09-10T07:35:14.854012+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:35:14.854012+0000","last_undegraded":"2024-09-10T07:35:14.854012+0000","last_fullsized":"2024-09-10T07:35:14.854012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882098+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:35:14.882098+0000","last_peered":"2024-09-10T07:35:14.882098+0000","last_clean":"2024-09-10T07:35:14.882098+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:35:14.882098+0000","last_undegraded":"2024-09-10T07:35:14.882098+0000","last_fullsized":"2024-09-10T07:35:14.882098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977445+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:35:14.977445+0000","last_peered":"2024-09-10T07:35:14.977445+0000","last_clean":"2024-09-10T07:35:14.977445+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:35:14.977445+0000","last_undegraded":"2024-09-10T07:35:14.977445+0000","last_fullsized":"2024-09-10T07:35:14.977445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842419+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:35:14.842419+0000","last_peered":"2024-09-10T07:35:14.842419+0000","last_clean":"2024-09-10T07:35:14.842419+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:35:14.842419+0000","last_undegraded":"2024-09-10T07:35:14.842419+0000","last_fullsized":"2024-09-10T07:35:14.842419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882010+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:35:14.882010+0000","last_peered":"2024-09-10T07:35:14.882010+0000","last_clean":"2024-09-10T07:35:14.882010+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:35:14.882010+0000","last_undegraded":"2024-09-10T07:35:14.882010+0000","last_fullsized":"2024-09-10T07:35:14.882010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852151+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:35:14.852151+0000","last_peered":"2024-09-10T07:35:14.852151+0000","last_clean":"2024-09-10T07:35:14.852151+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:35:14.852151+0000","last_undegraded":"2024-09-10T07:35:14.852151+0000","last_fullsized":"2024-09-10T07:35:14.852151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977217+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:35:14.977217+0000","last_peered":"2024-09-10T07:35:14.977217+0000","last_clean":"2024-09-10T07:35:14.977217+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:35:14.977217+0000","last_undegraded":"2024-09-10T07:35:14.977217+0000","last_fullsized":"2024-09-10T07:35:14.977217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851152+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:35:14.851152+0000","last_peered":"2024-09-10T07:35:14.851152+0000","last_clean":"2024-09-10T07:35:14.851152+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:35:14.851152+0000","last_undegraded":"2024-09-10T07:35:14.851152+0000","last_fullsized":"2024-09-10T07:35:14.851152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853834+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:35:14.853834+0000","last_peered":"2024-09-10T07:35:14.853834+0000","last_clean":"2024-09-10T07:35:14.853834+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:35:14.853834+0000","last_undegraded":"2024-09-10T07:35:14.853834+0000","last_fullsized":"2024-09-10T07:35:14.853834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976695+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:35:14.976695+0000","last_peered":"2024-09-10T07:35:14.976695+0000","last_clean":"2024-09-10T07:35:14.976695+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:35:14.976695+0000","last_undegraded":"2024-09-10T07:35:14.976695+0000","last_fullsized":"2024-09-10T07:35:14.976695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882157+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:35:14.882157+0000","last_peered":"2024-09-10T07:35:14.882157+0000","last_clean":"2024-09-10T07:35:14.882157+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:35:14.882157+0000","last_undegraded":"2024-09-10T07:35:14.882157+0000","last_fullsized":"2024-09-10T07:35:14.882157+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858294+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:35:14.858294+0000","last_peered":"2024-09-10T07:35:14.858294+0000","last_clean":"2024-09-10T07:35:14.858294+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:35:14.858294+0000","last_undegraded":"2024-09-10T07:35:14.858294+0000","last_fullsized":"2024-09-10T07:35:14.858294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882468+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:35:14.882468+0000","last_peered":"2024-09-10T07:35:14.882468+0000","last_clean":"2024-09-10T07:35:14.882468+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:35:14.882468+0000","last_undegraded":"2024-09-10T07:35:14.882468+0000","last_fullsized":"2024-09-10T07:35:14.882468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842318+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:35:14.842318+0000","last_peered":"2024-09-10T07:35:14.842318+0000","last_clean":"2024-09-10T07:35:14.842318+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:35:14.842318+0000","last_undegraded":"2024-09-10T07:35:14.842318+0000","last_fullsized":"2024-09-10T07:35:14.842318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977400+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:35:14.977400+0000","last_peered":"2024-09-10T07:35:14.977400+0000","last_clean":"2024-09-10T07:35:14.977400+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:35:14.977400+0000","last_undegraded":"2024-09-10T07:35:14.977400+0000","last_fullsized":"2024-09-10T07:35:14.977400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'16","reported_seq":59,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.850694+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:35:14.850694+0000","last_peered":"2024-09-10T07:35:14.850694+0000","last_clean":"2024-09-10T07:35:14.850694+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:35:14.850694+0000","last_undegraded":"2024-09-10T07:35:14.850694+0000","last_fullsized":"2024-09-10T07:35:14.850694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976854+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:35:14.976854+0000","last_peered":"2024-09-10T07:35:14.976854+0000","last_clean":"2024-09-10T07:35:14.976854+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:35:14.976854+0000","last_undegraded":"2024-09-10T07:35:14.976854+0000","last_fullsized":"2024-09-10T07:35:14.976854+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879908+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:35:14.879908+0000","last_peered":"2024-09-10T07:35:14.879908+0000","last_clean":"2024-09-10T07:35:14.879908+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:35:14.879908+0000","last_undegraded":"2024-09-10T07:35:14.879908+0000","last_fullsized":"2024-09-10T07:35:14.879908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.872520+0000","last_change":"2024-09-10T07:35:14.872520+0000","last_active":"2024-09-10T07:35:14.872520+0000","last_peered":"2024-09-10T07:35:14.872520+0000","last_clean":"2024-09-10T07:35:14.872520+0000","last_became_active":"2024-09-10T07:35:14.860686+0000","last_became_peered":"2024-09-10T07:35:14.860686+0000","last_unstale":"2024-09-10T07:35:14.872520+0000","last_undegraded":"2024-09-10T07:35:14.872520+0000","last_fullsized":"2024-09-10T07:35:14.872520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:42.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852303+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:35:14.852303+0000","last_peered":"2024-09-10T07:35:14.852303+0000","last_clean":"2024-09-10T07:35:14.852303+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:35:14.852303+0000","last_undegraded":"2024-09-10T07:35:14.852303+0000","last_fullsized":"2024-09-10T07:35:14.852303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877630+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:35:14.877630+0000","last_peered":"2024-09-10T07:35:14.877630+0000","last_clean":"2024-09-10T07:35:14.877630+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:35:14.877630+0000","last_undegraded":"2024-09-10T07:35:14.877630+0000","last_fullsized":"2024-09-10T07:35:14.877630+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848907+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:35:14.848907+0000","last_peered":"2024-09-10T07:35:14.848907+0000","last_clean":"2024-09-10T07:35:14.848907+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:35:14.848907+0000","last_undegraded":"2024-09-10T07:35:14.848907+0000","last_fullsized":"2024-09-10T07:35:14.848907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853870+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:35:14.853870+0000","last_peered":"2024-09-10T07:35:14.853870+0000","last_clean":"2024-09-10T07:35:14.853870+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:35:14.853870+0000","last_undegraded":"2024-09-10T07:35:14.853870+0000","last_fullsized":"2024-09-10T07:35:14.853870+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877678+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:35:14.877678+0000","last_peered":"2024-09-10T07:35:14.877678+0000","last_clean":"2024-09-10T07:35:14.877678+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:35:14.877678+0000","last_undegraded":"2024-09-10T07:35:14.877678+0000","last_fullsized":"2024-09-10T07:35:14.877678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":112,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:24.902398+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:35:24.902398+0000","last_peered":"2024-09-10T07:35:24.902398+0000","last_clean":"2024-09-10T07:35:24.902398+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:35:24.902398+0000","last_undegraded":"2024-09-10T07:35:24.902398+0000","last_fullsized":"2024-09-10T07:35:24.902398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879669+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:35:14.879669+0000","last_peered":"2024-09-10T07:35:14.879669+0000","last_clean":"2024-09-10T07:35:14.879669+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:35:14.879669+0000","last_undegraded":"2024-09-10T07:35:14.879669+0000","last_fullsized":"2024-09-10T07:35:14.879669+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880031+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:35:14.880031+0000","last_peered":"2024-09-10T07:35:14.880031+0000","last_clean":"2024-09-10T07:35:14.880031+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:35:14.880031+0000","last_undegraded":"2024-09-10T07:35:14.880031+0000","last_fullsized":"2024-09-10T07:35:14.880031+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":111,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:24.894634+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:35:24.894634+0000","last_peered":"2024-09-10T07:35:24.894634+0000","last_clean":"2024-09-10T07:35:24.894634+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:35:24.894634+0000","last_undegraded":"2024-09-10T07:35:24.894634+0000","last_fullsized":"2024-09-10T07:35:24.894634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880558+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:35:14.880558+0000","last_peered":"2024-09-10T07:35:14.880558+0000","last_clean":"2024-09-10T07:35:14.880558+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:35:14.880558+0000","last_undegraded":"2024-09-10T07:35:14.880558+0000","last_fullsized":"2024-09-10T07:35:14.880558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879041+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:35:14.879041+0000","last_peered":"2024-09-10T07:35:14.879041+0000","last_clean":"2024-09-10T07:35:14.879041+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:35:14.879041+0000","last_undegraded":"2024-09-10T07:35:14.879041+0000","last_fullsized":"2024-09-10T07:35:14.879041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842697+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:35:14.842697+0000","last_peered":"2024-09-10T07:35:14.842697+0000","last_clean":"2024-09-10T07:35:14.842697+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:35:14.842697+0000","last_undegraded":"2024-09-10T07:35:14.842697+0000","last_fullsized":"2024-09-10T07:35:14.842697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880370+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:35:14.880370+0000","last_peered":"2024-09-10T07:35:14.880370+0000","last_clean":"2024-09-10T07:35:14.880370+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:35:14.880370+0000","last_undegraded":"2024-09-10T07:35:14.880370+0000","last_fullsized":"2024-09-10T07:35:14.880370+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977120+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:35:14.977120+0000","last_peered":"2024-09-10T07:35:14.977120+0000","last_clean":"2024-09-10T07:35:14.977120+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:35:14.977120+0000","last_undegraded":"2024-09-10T07:35:14.977120+0000","last_fullsized":"2024-09-10T07:35:14.977120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882470+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:35:14.882470+0000","last_peered":"2024-09-10T07:35:14.882470+0000","last_clean":"2024-09-10T07:35:14.882470+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:35:14.882470+0000","last_undegraded":"2024-09-10T07:35:14.882470+0000","last_fullsized":"2024-09-10T07:35:14.882470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842861+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:35:14.842861+0000","last_peered":"2024-09-10T07:35:14.842861+0000","last_clean":"2024-09-10T07:35:14.842861+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:35:14.842861+0000","last_undegraded":"2024-09-10T07:35:14.842861+0000","last_fullsized":"2024-09-10T07:35:14.842861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843123+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:35:14.843123+0000","last_peered":"2024-09-10T07:35:14.843123+0000","last_clean":"2024-09-10T07:35:14.843123+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:35:14.843123+0000","last_undegraded":"2024-09-10T07:35:14.843123+0000","last_fullsized":"2024-09-10T07:35:14.843123+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843358+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:35:14.843358+0000","last_peered":"2024-09-10T07:35:14.843358+0000","last_clean":"2024-09-10T07:35:14.843358+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:35:14.843358+0000","last_undegraded":"2024-09-10T07:35:14.843358+0000","last_fullsized":"2024-09-10T07:35:14.843358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":115,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.900484+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:35:28.900484+0000","last_peered":"2024-09-10T07:35:28.900484+0000","last_clean":"2024-09-10T07:35:28.900484+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:35:28.900484+0000","last_undegraded":"2024-09-10T07:35:28.900484+0000","last_fullsized":"2024-09-10T07:35:28.900484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976716+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:35:14.976716+0000","last_peered":"2024-09-10T07:35:14.976716+0000","last_clean":"2024-09-10T07:35:14.976716+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:35:14.976716+0000","last_undegraded":"2024-09-10T07:35:14.976716+0000","last_fullsized":"2024-09-10T07:35:14.976716+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843096+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:35:14.843096+0000","last_peered":"2024-09-10T07:35:14.843096+0000","last_clean":"2024-09-10T07:35:14.843096+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:35:14.843096+0000","last_undegraded":"2024-09-10T07:35:14.843096+0000","last_fullsized":"2024-09-10T07:35:14.843096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880652+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:35:14.880652+0000","last_peered":"2024-09-10T07:35:14.880652+0000","last_clean":"2024-09-10T07:35:14.880652+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:35:14.880652+0000","last_undegraded":"2024-09-10T07:35:14.880652+0000","last_fullsized":"2024-09-10T07:35:14.880652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857646+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:35:14.857646+0000","last_peered":"2024-09-10T07:35:14.857646+0000","last_clean":"2024-09-10T07:35:14.857646+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:35:14.857646+0000","last_undegraded":"2024-09-10T07:35:14.857646+0000","last_fullsized":"2024-09-10T07:35:14.857646+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843150+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:35:14.843150+0000","last_peered":"2024-09-10T07:35:14.843150+0000","last_clean":"2024-09-10T07:35:14.843150+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:35:14.843150+0000","last_undegraded":"2024-09-10T07:35:14.843150+0000","last_fullsized":"2024-09-10T07:35:14.843150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880435+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:35:14.880435+0000","last_peered":"2024-09-10T07:35:14.880435+0000","last_clean":"2024-09-10T07:35:14.880435+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:35:14.880435+0000","last_undegraded":"2024-09-10T07:35:14.880435+0000","last_fullsized":"2024-09-10T07:35:14.880435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879923+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:35:14.879923+0000","last_peered":"2024-09-10T07:35:14.879923+0000","last_clean":"2024-09-10T07:35:14.879923+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:35:14.879923+0000","last_undegraded":"2024-09-10T07:35:14.879923+0000","last_fullsized":"2024-09-10T07:35:14.879923+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852535+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:35:14.852535+0000","last_peered":"2024-09-10T07:35:14.852535+0000","last_clean":"2024-09-10T07:35:14.852535+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:35:14.852535+0000","last_undegraded":"2024-09-10T07:35:14.852535+0000","last_fullsized":"2024-09-10T07:35:14.852535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854413+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:35:14.854413+0000","last_peered":"2024-09-10T07:35:14.854413+0000","last_clean":"2024-09-10T07:35:14.854413+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:35:14.854413+0000","last_undegraded":"2024-09-10T07:35:14.854413+0000","last_fullsized":"2024-09-10T07:35:14.854413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879910+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:35:14.879910+0000","last_peered":"2024-09-10T07:35:14.879910+0000","last_clean":"2024-09-10T07:35:14.879910+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:35:14.879910+0000","last_undegraded":"2024-09-10T07:35:14.879910+0000","last_fullsized":"2024-09-10T07:35:14.879910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880026+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:35:14.880026+0000","last_peered":"2024-09-10T07:35:14.880026+0000","last_clean":"2024-09-10T07:35:14.880026+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:35:14.880026+0000","last_undegraded":"2024-09-10T07:35:14.880026+0000","last_fullsized":"2024-09-10T07:35:14.880026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":61,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853872+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:35:14.853872+0000","last_peered":"2024-09-10T07:35:14.853872+0000","last_clean":"2024-09-10T07:35:14.853872+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:35:14.853872+0000","last_undegraded":"2024-09-10T07:35:14.853872+0000","last_fullsized":"2024-09-10T07:35:14.853872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880346+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:35:14.880346+0000","last_peered":"2024-09-10T07:35:14.880346+0000","last_clean":"2024-09-10T07:35:14.880346+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:35:14.880346+0000","last_undegraded":"2024-09-10T07:35:14.880346+0000","last_fullsized":"2024-09-10T07:35:14.880346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842600+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:35:14.842600+0000","last_peered":"2024-09-10T07:35:14.842600+0000","last_clean":"2024-09-10T07:35:14.842600+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:35:14.842600+0000","last_undegraded":"2024-09-10T07:35:14.842600+0000","last_fullsized":"2024-09-10T07:35:14.842600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":109,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:23.893656+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:35:23.893656+0000","last_peered":"2024-09-10T07:35:23.893656+0000","last_clean":"2024-09-10T07:35:23.893656+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:35:23.893656+0000","last_undegraded":"2024-09-10T07:35:23.893656+0000","last_fullsized":"2024-09-10T07:35:23.893656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851287+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:35:14.851287+0000","last_peered":"2024-09-10T07:35:14.851287+0000","last_clean":"2024-09-10T07:35:14.851287+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:35:14.851287+0000","last_undegraded":"2024-09-10T07:35:14.851287+0000","last_fullsized":"2024-09-10T07:35:14.851287+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842874+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:35:14.842874+0000","last_peered":"2024-09-10T07:35:14.842874+0000","last_clean":"2024-09-10T07:35:14.842874+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:35:14.842874+0000","last_undegraded":"2024-09-10T07:35:14.842874+0000","last_fullsized":"2024-09-10T07:35:14.842874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880110+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:35:14.880110+0000","last_peered":"2024-09-10T07:35:14.880110+0000","last_clean":"2024-09-10T07:35:14.880110+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:35:14.880110+0000","last_undegraded":"2024-09-10T07:35:14.880110+0000","last_fullsized":"2024-09-10T07:35:14.880110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852212+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:35:14.852212+0000","last_peered":"2024-09-10T07:35:14.852212+0000","last_clean":"2024-09-10T07:35:14.852212+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:35:14.852212+0000","last_undegraded":"2024-09-10T07:35:14.852212+0000","last_fullsized":"2024-09-10T07:35:14.852212+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879822+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:35:14.879822+0000","last_peered":"2024-09-10T07:35:14.879822+0000","last_clean":"2024-09-10T07:35:14.879822+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:35:14.879822+0000","last_undegraded":"2024-09-10T07:35:14.879822+0000","last_fullsized":"2024-09-10T07:35:14.879822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852205+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:35:14.852205+0000","last_peered":"2024-09-10T07:35:14.852205+0000","last_clean":"2024-09-10T07:35:14.852205+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:35:14.852205+0000","last_undegraded":"2024-09-10T07:35:14.852205+0000","last_fullsized":"2024-09-10T07:35:14.852205+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880377+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:35:14.880377+0000","last_peered":"2024-09-10T07:35:14.880377+0000","last_clean":"2024-09-10T07:35:14.880377+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:35:14.880377+0000","last_undegraded":"2024-09-10T07:35:14.880377+0000","last_fullsized":"2024-09-10T07:35:14.880377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854403+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:35:14.854403+0000","last_peered":"2024-09-10T07:35:14.854403+0000","last_clean":"2024-09-10T07:35:14.854403+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:35:14.854403+0000","last_undegraded":"2024-09-10T07:35:14.854403+0000","last_fullsized":"2024-09-10T07:35:14.854403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'36","reported_seq":217,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.899536+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:35:28.899536+0000","last_peered":"2024-09-10T07:35:28.899536+0000","last_clean":"2024-09-10T07:35:28.899536+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:35:28.899536+0000","last_undegraded":"2024-09-10T07:35:28.899536+0000","last_fullsized":"2024-09-10T07:35:28.899536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64622,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848049+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:35:14.848049+0000","last_peered":"2024-09-10T07:35:14.848049+0000","last_clean":"2024-09-10T07:35:14.848049+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:35:14.848049+0000","last_undegraded":"2024-09-10T07:35:14.848049+0000","last_fullsized":"2024-09-10T07:35:14.848049+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976954+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:35:14.976954+0000","last_peered":"2024-09-10T07:35:14.976954+0000","last_clean":"2024-09-10T07:35:14.976954+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:35:14.976954+0000","last_undegraded":"2024-09-10T07:35:14.976954+0000","last_fullsized":"2024-09-10T07:35:14.976954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880584+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:35:14.880584+0000","last_peered":"2024-09-10T07:35:14.880584+0000","last_clean":"2024-09-10T07:35:14.880584+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:35:14.880584+0000","last_undegraded":"2024-09-10T07:35:14.880584+0000","last_fullsized":"2024-09-10T07:35:14.880584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882281+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:35:14.882281+0000","last_peered":"2024-09-10T07:35:14.882281+0000","last_clean":"2024-09-10T07:35:14.882281+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:35:14.882281+0000","last_undegraded":"2024-09-10T07:35:14.882281+0000","last_fullsized":"2024-09-10T07:35:14.882281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.856894+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:35:14.856894+0000","last_peered":"2024-09-10T07:35:14.856894+0000","last_clean":"2024-09-10T07:35:14.856894+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:35:14.856894+0000","last_undegraded":"2024-09-10T07:35:14.856894+0000","last_fullsized":"2024-09-10T07:35:14.856894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843188+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:35:14.843188+0000","last_peered":"2024-09-10T07:35:14.843188+0000","last_clean":"2024-09-10T07:35:14.843188+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:35:14.843188+0000","last_undegraded":"2024-09-10T07:35:14.843188+0000","last_fullsized":"2024-09-10T07:35:14.843188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847334+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:35:14.847334+0000","last_peered":"2024-09-10T07:35:14.847334+0000","last_clean":"2024-09-10T07:35:14.847334+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:35:14.847334+0000","last_undegraded":"2024-09-10T07:35:14.847334+0000","last_fullsized":"2024-09-10T07:35:14.847334+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977143+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:35:14.977143+0000","last_peered":"2024-09-10T07:35:14.977143+0000","last_clean":"2024-09-10T07:35:14.977143+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:35:14.977143+0000","last_undegraded":"2024-09-10T07:35:14.977143+0000","last_fullsized":"2024-09-10T07:35:14.977143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":175,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.899686+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:35:28.899686+0000","last_peered":"2024-09-10T07:35:28.899686+0000","last_clean":"2024-09-10T07:35:28.899686+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:35:28.899686+0000","last_undegraded":"2024-09-10T07:35:28.899686+0000","last_fullsized":"2024-09-10T07:35:28.899686+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976991+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:35:14.976991+0000","last_peered":"2024-09-10T07:35:14.976991+0000","last_clean":"2024-09-10T07:35:14.976991+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:35:14.976991+0000","last_undegraded":"2024-09-10T07:35:14.976991+0000","last_fullsized":"2024-09-10T07:35:14.976991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858109+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:35:14.858109+0000","last_peered":"2024-09-10T07:35:14.858109+0000","last_clean":"2024-09-10T07:35:14.858109+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:35:14.858109+0000","last_undegraded":"2024-09-10T07:35:14.858109+0000","last_fullsized":"2024-09-10T07:35:14.858109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847742+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:35:14.847742+0000","last_peered":"2024-09-10T07:35:14.847742+0000","last_clean":"2024-09-10T07:35:14.847742+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:35:14.847742+0000","last_undegraded":"2024-09-10T07:35:14.847742+0000","last_fullsized":"2024-09-10T07:35:14.847742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880198+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:35:14.880198+0000","last_peered":"2024-09-10T07:35:14.880198+0000","last_clean":"2024-09-10T07:35:14.880198+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:35:14.880198+0000","last_undegraded":"2024-09-10T07:35:14.880198+0000","last_fullsized":"2024-09-10T07:35:14.880198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852519+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:35:14.852519+0000","last_peered":"2024-09-10T07:35:14.852519+0000","last_clean":"2024-09-10T07:35:14.852519+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:35:14.852519+0000","last_undegraded":"2024-09-10T07:35:14.852519+0000","last_fullsized":"2024-09-10T07:35:14.852519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976994+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:35:14.976994+0000","last_peered":"2024-09-10T07:35:14.976994+0000","last_clean":"2024-09-10T07:35:14.976994+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:35:14.976994+0000","last_undegraded":"2024-09-10T07:35:14.976994+0000","last_fullsized":"2024-09-10T07:35:14.976994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879143+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:35:14.879143+0000","last_peered":"2024-09-10T07:35:14.879143+0000","last_clean":"2024-09-10T07:35:14.879143+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:35:14.879143+0000","last_undegraded":"2024-09-10T07:35:14.879143+0000","last_fullsized":"2024-09-10T07:35:14.879143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877732+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:35:14.877732+0000","last_peered":"2024-09-10T07:35:14.877732+0000","last_clean":"2024-09-10T07:35:14.877732+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:35:14.877732+0000","last_undegraded":"2024-09-10T07:35:14.877732+0000","last_fullsized":"2024-09-10T07:35:14.877732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848270+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:35:14.848270+0000","last_peered":"2024-09-10T07:35:14.848270+0000","last_clean":"2024-09-10T07:35:14.848270+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:35:14.848270+0000","last_undegraded":"2024-09-10T07:35:14.848270+0000","last_fullsized":"2024-09-10T07:35:14.848270+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842456+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:35:14.842456+0000","last_peered":"2024-09-10T07:35:14.842456+0000","last_clean":"2024-09-10T07:35:14.842456+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:35:14.842456+0000","last_undegraded":"2024-09-10T07:35:14.842456+0000","last_fullsized":"2024-09-10T07:35:14.842456+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":117,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.900413+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:35:28.900413+0000","last_peered":"2024-09-10T07:35:28.900413+0000","last_clean":"2024-09-10T07:35:28.900413+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:35:28.900413+0000","last_undegraded":"2024-09-10T07:35:28.900413+0000","last_fullsized":"2024-09-10T07:35:28.900413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854042+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:35:14.854042+0000","last_peered":"2024-09-10T07:35:14.854042+0000","last_clean":"2024-09-10T07:35:14.854042+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:35:14.854042+0000","last_undegraded":"2024-09-10T07:35:14.854042+0000","last_fullsized":"2024-09-10T07:35:14.854042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842576+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:35:14.842576+0000","last_peered":"2024-09-10T07:35:14.842576+0000","last_clean":"2024-09-10T07:35:14.842576+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:35:14.842576+0000","last_undegraded":"2024-09-10T07:35:14.842576+0000","last_fullsized":"2024-09-10T07:35:14.842576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852091+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:35:14.852091+0000","last_peered":"2024-09-10T07:35:14.852091+0000","last_clean":"2024-09-10T07:35:14.852091+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:35:14.852091+0000","last_undegraded":"2024-09-10T07:35:14.852091+0000","last_fullsized":"2024-09-10T07:35:14.852091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842673+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:35:14.842673+0000","last_peered":"2024-09-10T07:35:14.842673+0000","last_clean":"2024-09-10T07:35:14.842673+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:35:14.842673+0000","last_undegraded":"2024-09-10T07:35:14.842673+0000","last_fullsized":"2024-09-10T07:35:14.842673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842346+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:35:14.842346+0000","last_peered":"2024-09-10T07:35:14.842346+0000","last_clean":"2024-09-10T07:35:14.842346+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:35:14.842346+0000","last_undegraded":"2024-09-10T07:35:14.842346+0000","last_fullsized":"2024-09-10T07:35:14.842346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857747+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:35:14.857747+0000","last_peered":"2024-09-10T07:35:14.857747+0000","last_clean":"2024-09-10T07:35:14.857747+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:35:14.857747+0000","last_undegraded":"2024-09-10T07:35:14.857747+0000","last_fullsized":"2024-09-10T07:35:14.857747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879821+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:35:14.879821+0000","last_peered":"2024-09-10T07:35:14.879821+0000","last_clean":"2024-09-10T07:35:14.879821+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:35:14.879821+0000","last_undegraded":"2024-09-10T07:35:14.879821+0000","last_fullsized":"2024-09-10T07:35:14.879821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852916+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:35:14.852916+0000","last_peered":"2024-09-10T07:35:14.852916+0000","last_clean":"2024-09-10T07:35:14.852916+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:35:14.852916+0000","last_undegraded":"2024-09-10T07:35:14.852916+0000","last_fullsized":"2024-09-10T07:35:14.852916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842926+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:35:14.842926+0000","last_peered":"2024-09-10T07:35:14.842926+0000","last_clean":"2024-09-10T07:35:14.842926+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:35:14.842926+0000","last_undegraded":"2024-09-10T07:35:14.842926+0000","last_fullsized":"2024-09-10T07:35:14.842926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977392+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:35:14.977392+0000","last_peered":"2024-09-10T07:35:14.977392+0000","last_clean":"2024-09-10T07:35:14.977392+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:35:14.977392+0000","last_undegraded":"2024-09-10T07:35:14.977392+0000","last_fullsized":"2024-09-10T07:35:14.977392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879986+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:35:14.879986+0000","last_peered":"2024-09-10T07:35:14.879986+0000","last_clean":"2024-09-10T07:35:14.879986+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:35:14.879986+0000","last_undegraded":"2024-09-10T07:35:14.879986+0000","last_fullsized":"2024-09-10T07:35:14.879986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977101+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:35:14.977101+0000","last_peered":"2024-09-10T07:35:14.977101+0000","last_clean":"2024-09-10T07:35:14.977101+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:35:14.977101+0000","last_undegraded":"2024-09-10T07:35:14.977101+0000","last_fullsized":"2024-09-10T07:35:14.977101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976876+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:35:14.976876+0000","last_peered":"2024-09-10T07:35:14.976876+0000","last_clean":"2024-09-10T07:35:14.976876+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:35:14.976876+0000","last_undegraded":"2024-09-10T07:35:14.976876+0000","last_fullsized":"2024-09-10T07:35:14.976876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879430+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:35:14.879430+0000","last_peered":"2024-09-10T07:35:14.879430+0000","last_clean":"2024-09-10T07:35:14.879430+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:35:14.879430+0000","last_undegraded":"2024-09-10T07:35:14.879430+0000","last_fullsized":"2024-09-10T07:35:14.879430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852322+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:35:14.852322+0000","last_peered":"2024-09-10T07:35:14.852322+0000","last_clean":"2024-09-10T07:35:14.852322+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:35:14.852322+0000","last_undegraded":"2024-09-10T07:35:14.852322+0000","last_fullsized":"2024-09-10T07:35:14.852322+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842741+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:35:14.842741+0000","last_peered":"2024-09-10T07:35:14.842741+0000","last_clean":"2024-09-10T07:35:14.842741+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:35:14.842741+0000","last_undegraded":"2024-09-10T07:35:14.842741+0000","last_fullsized":"2024-09-10T07:35:14.842741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.850585+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:35:14.850585+0000","last_peered":"2024-09-10T07:35:14.850585+0000","last_clean":"2024-09-10T07:35:14.850585+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:35:14.850585+0000","last_undegraded":"2024-09-10T07:35:14.850585+0000","last_fullsized":"2024-09-10T07:35:14.850585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852768+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:35:14.852768+0000","last_peered":"2024-09-10T07:35:14.852768+0000","last_clean":"2024-09-10T07:35:14.852768+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:35:14.852768+0000","last_undegraded":"2024-09-10T07:35:14.852768+0000","last_fullsized":"2024-09-10T07:35:14.852768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976577+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:35:14.976577+0000","last_peered":"2024-09-10T07:35:14.976577+0000","last_clean":"2024-09-10T07:35:14.976577+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:35:14.976577+0000","last_undegraded":"2024-09-10T07:35:14.976577+0000","last_fullsized":"2024-09-10T07:35:14.976577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857364+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:35:14.857364+0000","last_peered":"2024-09-10T07:35:14.857364+0000","last_clean":"2024-09-10T07:35:14.857364+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:35:14.857364+0000","last_undegraded":"2024-09-10T07:35:14.857364+0000","last_fullsized":"2024-09-10T07:35:14.857364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857503+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:35:14.857503+0000","last_peered":"2024-09-10T07:35:14.857503+0000","last_clean":"2024-09-10T07:35:14.857503+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:35:14.857503+0000","last_undegraded":"2024-09-10T07:35:14.857503+0000","last_fullsized":"2024-09-10T07:35:14.857503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879336+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:35:14.879336+0000","last_peered":"2024-09-10T07:35:14.879336+0000","last_clean":"2024-09-10T07:35:14.879336+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:35:14.879336+0000","last_undegraded":"2024-09-10T07:35:14.879336+0000","last_fullsized":"2024-09-10T07:35:14.879336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882260+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:35:14.882260+0000","last_peered":"2024-09-10T07:35:14.882260+0000","last_clean":"2024-09-10T07:35:14.882260+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:35:14.882260+0000","last_undegraded":"2024-09-10T07:35:14.882260+0000","last_fullsized":"2024-09-10T07:35:14.882260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843320+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:35:14.843320+0000","last_peered":"2024-09-10T07:35:14.843320+0000","last_clean":"2024-09-10T07:35:14.843320+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:35:14.843320+0000","last_undegraded":"2024-09-10T07:35:14.843320+0000","last_fullsized":"2024-09-10T07:35:14.843320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877986+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:35:14.877986+0000","last_peered":"2024-09-10T07:35:14.877986+0000","last_clean":"2024-09-10T07:35:14.877986+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:35:14.877986+0000","last_undegraded":"2024-09-10T07:35:14.877986+0000","last_fullsized":"2024-09-10T07:35:14.877986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977201+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:35:14.977201+0000","last_peered":"2024-09-10T07:35:14.977201+0000","last_clean":"2024-09-10T07:35:14.977201+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:35:14.977201+0000","last_undegraded":"2024-09-10T07:35:14.977201+0000","last_fullsized":"2024-09-10T07:35:14.977201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976968+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:35:14.976968+0000","last_peered":"2024-09-10T07:35:14.976968+0000","last_clean":"2024-09-10T07:35:14.976968+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:35:14.976968+0000","last_undegraded":"2024-09-10T07:35:14.976968+0000","last_fullsized":"2024-09-10T07:35:14.976968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842700+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:35:14.842700+0000","last_peered":"2024-09-10T07:35:14.842700+0000","last_clean":"2024-09-10T07:35:14.842700+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:35:14.842700+0000","last_undegraded":"2024-09-10T07:35:14.842700+0000","last_fullsized":"2024-09-10T07:35:14.842700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877599+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:35:14.877599+0000","last_peered":"2024-09-10T07:35:14.877599+0000","last_clean":"2024-09-10T07:35:14.877599+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:35:14.877599+0000","last_undegraded":"2024-09-10T07:35:14.877599+0000","last_fullsized":"2024-09-10T07:35:14.877599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853987+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:35:14.853987+0000","last_peered":"2024-09-10T07:35:14.853987+0000","last_clean":"2024-09-10T07:35:14.853987+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:35:14.853987+0000","last_undegraded":"2024-09-10T07:35:14.853987+0000","last_fullsized":"2024-09-10T07:35:14.853987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882152+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:35:14.882152+0000","last_peered":"2024-09-10T07:35:14.882152+0000","last_clean":"2024-09-10T07:35:14.882152+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:35:14.882152+0000","last_undegraded":"2024-09-10T07:35:14.882152+0000","last_fullsized":"2024-09-10T07:35:14.882152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.856312+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:35:14.856312+0000","last_peered":"2024-09-10T07:35:14.856312+0000","last_clean":"2024-09-10T07:35:14.856312+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:35:14.856312+0000","last_undegraded":"2024-09-10T07:35:14.856312+0000","last_fullsized":"2024-09-10T07:35:14.856312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858214+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:35:14.858214+0000","last_peered":"2024-09-10T07:35:14.858214+0000","last_clean":"2024-09-10T07:35:14.858214+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:35:14.858214+0000","last_undegraded":"2024-09-10T07:35:14.858214+0000","last_fullsized":"2024-09-10T07:35:14.858214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852694+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:35:14.852694+0000","last_peered":"2024-09-10T07:35:14.852694+0000","last_clean":"2024-09-10T07:35:14.852694+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:35:14.852694+0000","last_undegraded":"2024-09-10T07:35:14.852694+0000","last_fullsized":"2024-09-10T07:35:14.852694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71830,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":670,"num_read_kb":414,"num_write":251,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1097728,"data_stored":561212,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710763,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1232896,"data_stored":1212470,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874286,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":565058,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135925,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1093632,"data_stored":561228,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1155072,"data_stored":1158232,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1081344,"data_stored":556859,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1163264,"data_stored":1153421,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1089536,"data_stored":560540,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1093632,"data_stored":556174,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1163264,"data_stored":622663,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1150976,"data_stored":622511,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":557922,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:35:32.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T07:35:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:33.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:32 smithi181 ceph-mon[35397]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:35:33.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:32 smithi063 ceph-mon[35020]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:35:33.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:32 smithi046 ceph-mon[30705]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:35:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 -- 172.21.15.46:0/3471983842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b81019c0 msgr2=0x7f17b8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 --2- 172.21.15.46:0/3471983842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b81019c0 0x7f17b8101e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009900 tx=0x7f17a002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 -- 172.21.15.46:0/3471983842 shutdown_connections 2024-09-10T07:35:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 --2- 172.21.15.46:0/3471983842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17b8102360 0x7f17b810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 --2- 172.21.15.46:0/3471983842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b81019c0 0x7f17b8101e20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 --2- 172.21.15.46:0/3471983842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b81079c0 0x7f17b8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 -- 172.21.15.46:0/3471983842 >> 172.21.15.46:0/3471983842 conn(0x7f17b80fd740 msgr2=0x7f17b80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:33.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 -- 172.21.15.46:0/3471983842 shutdown_connections 2024-09-10T07:35:33.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.386+0000 7f17bd400640 1 -- 172.21.15.46:0/3471983842 wait complete. 2024-09-10T07:35:33.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.387+0000 7f17bd400640 1 Processor -- start 2024-09-10T07:35:33.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.387+0000 7f17bd400640 1 -- start start 2024-09-10T07:35:33.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.387+0000 7f17bd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b81019c0 0x7f17b819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.387+0000 7f17bd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b8102360 0x7f17b819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17bd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17b81079c0 0x7f17b819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17bd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8111dc0 con 0x7f17b81019c0 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17bd400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8111c40 con 0x7f17b8102360 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17bd400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8111f40 con 0x7f17b81079c0 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b81019c0 0x7f17b819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17afe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b8102360 0x7f17b819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b81019c0 0x7f17b819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41138/0 (socket says 172.21.15.46:41138) 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17b7e00640 1 -- 172.21.15.46:0/3222673280 learned_addr learned my addr 172.21.15.46:0/3222673280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17bca00640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17b81079c0 0x7f17b819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17b7e00640 1 -- 172.21.15.46:0/3222673280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17b81079c0 msgr2=0x7f17b819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17b7e00640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17b81079c0 0x7f17b819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17b7e00640 1 -- 172.21.15.46:0/3222673280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b8102360 msgr2=0x7f17b819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17b7e00640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b8102360 0x7f17b819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17b7e00640 1 -- 172.21.15.46:0/3222673280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b819f540 con 0x7f17b81019c0 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17bca00640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17b81079c0 0x7f17b819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.388+0000 7f17afe00640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b8102360 0x7f17b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.389+0000 7f17b7e00640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b81019c0 0x7f17b819a590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17a800e9f0 tx=0x7f17a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.389+0000 7f17b5600640 1 -- 172.21.15.46:0/3222673280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a800cd80 con 0x7f17b81019c0 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.389+0000 7f17b5600640 1 -- 172.21.15.46:0/3222673280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a8010910 con 0x7f17b81019c0 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.389+0000 7f17b5600640 1 -- 172.21.15.46:0/3222673280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8018990 con 0x7f17b81019c0 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.389+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b819f830 con 0x7f17b81019c0 2024-09-10T07:35:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.389+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b81a7110 con 0x7f17b81019c0 2024-09-10T07:35:33.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.390+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b8101e20 con 0x7f17b81019c0 2024-09-10T07:35:33.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.392+0000 7f17b5600640 1 -- 172.21.15.46:0/3222673280 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f17a8010ac0 con 0x7f17b81019c0 2024-09-10T07:35:33.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.393+0000 7f17b5600640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f179c0a2c10 0x7f179c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:33.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.393+0000 7f17b5600640 1 -- 172.21.15.46:0/3222673280 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f17a80cf8f0 con 0x7f17b81019c0 2024-09-10T07:35:33.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.394+0000 7f17afe00640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f179c0a2c10 0x7f179c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:33.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.394+0000 7f17afe00640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f179c0a2c10 0x7f179c0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009230 tx=0x7f17a00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.397+0000 7f17b5600640 1 -- 172.21.15.46:0/3222673280 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17a809c310 con 0x7f17b81019c0 2024-09-10T07:35:33.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.579+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17b810d410 con 0x7f179c0a2c10 2024-09-10T07:35:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.589+0000 7f17b5600640 1 -- 172.21.15.46:0/3222673280 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340656 (secure 0 0 0) 0x7f17b810d410 con 0x7f179c0a2c10 2024-09-10T07:35:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:35:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:35:33.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f179c0a2c10 msgr2=0x7f179c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:33.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f179c0a2c10 0x7f179c0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009230 tx=0x7f17a00058d0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b81019c0 msgr2=0x7f17b819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:33.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b81019c0 0x7f17b819a590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17a800e9f0 tx=0x7f17a800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 shutdown_connections 2024-09-10T07:35:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17b81079c0 0x7f17b819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17b8102360 0x7f17b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f179c0a2c10 0x7f179c0a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 --2- 172.21.15.46:0/3222673280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b81019c0 0x7f17b819a590 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.593+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 >> 172.21.15.46:0/3222673280 conn(0x7f17b80fd740 msgr2=0x7f17b810cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.594+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 shutdown_connections 2024-09-10T07:35:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:33.594+0000 7f17bd400640 1 -- 172.21.15.46:0/3222673280 wait complete. 2024-09-10T07:35:34.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:33 smithi181 ceph-mon[35397]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:35:34.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:33 smithi063 ceph-mon[35020]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:35:34.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:33 smithi046 ceph-mon[30705]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:35:34.399 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-10T07:35:32.490234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662432,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":811,"num_read_kb":528,"num_write":434,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":355,"ondisk_log_size":355,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358376,"kb_used_data":13220,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583448,"statfs":{"total":1151940427776,"available":1151573450752,"internally_reserved":0,"allocated":13537280,"data_stored":8688290,"data_compressed":498372,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003476"},"pg_stats":[{"pgid":"2.1f","version":"95'13","reported_seq":119,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:26.905210+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:35:26.905210+0000","last_peered":"2024-09-10T07:35:26.905210+0000","last_clean":"2024-09-10T07:35:26.905210+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:35:26.905210+0000","last_undegraded":"2024-09-10T07:35:26.905210+0000","last_fullsized":"2024-09-10T07:35:26.905210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977492+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:35:14.977492+0000","last_peered":"2024-09-10T07:35:14.977492+0000","last_clean":"2024-09-10T07:35:14.977492+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:35:14.977492+0000","last_undegraded":"2024-09-10T07:35:14.977492+0000","last_fullsized":"2024-09-10T07:35:14.977492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842304+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:35:14.842304+0000","last_peered":"2024-09-10T07:35:14.842304+0000","last_clean":"2024-09-10T07:35:14.842304+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:35:14.842304+0000","last_undegraded":"2024-09-10T07:35:14.842304+0000","last_fullsized":"2024-09-10T07:35:14.842304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977693+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:35:14.977693+0000","last_peered":"2024-09-10T07:35:14.977693+0000","last_clean":"2024-09-10T07:35:14.977693+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:35:14.977693+0000","last_undegraded":"2024-09-10T07:35:14.977693+0000","last_fullsized":"2024-09-10T07:35:14.977693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.856805+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:35:14.856805+0000","last_peered":"2024-09-10T07:35:14.856805+0000","last_clean":"2024-09-10T07:35:14.856805+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:35:14.856805+0000","last_undegraded":"2024-09-10T07:35:14.856805+0000","last_fullsized":"2024-09-10T07:35:14.856805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881733+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:35:14.881733+0000","last_peered":"2024-09-10T07:35:14.881733+0000","last_clean":"2024-09-10T07:35:14.881733+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:35:14.881733+0000","last_undegraded":"2024-09-10T07:35:14.881733+0000","last_fullsized":"2024-09-10T07:35:14.881733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":56,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854507+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:35:14.854507+0000","last_peered":"2024-09-10T07:35:14.854507+0000","last_clean":"2024-09-10T07:35:14.854507+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:35:14.854507+0000","last_undegraded":"2024-09-10T07:35:14.854507+0000","last_fullsized":"2024-09-10T07:35:14.854507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976867+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:35:14.976867+0000","last_peered":"2024-09-10T07:35:14.976867+0000","last_clean":"2024-09-10T07:35:14.976867+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:35:14.976867+0000","last_undegraded":"2024-09-10T07:35:14.976867+0000","last_fullsized":"2024-09-10T07:35:14.976867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881927+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:35:14.881927+0000","last_peered":"2024-09-10T07:35:14.881927+0000","last_clean":"2024-09-10T07:35:14.881927+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:35:14.881927+0000","last_undegraded":"2024-09-10T07:35:14.881927+0000","last_fullsized":"2024-09-10T07:35:14.881927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854348+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:35:14.854348+0000","last_peered":"2024-09-10T07:35:14.854348+0000","last_clean":"2024-09-10T07:35:14.854348+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:35:14.854348+0000","last_undegraded":"2024-09-10T07:35:14.854348+0000","last_fullsized":"2024-09-10T07:35:14.854348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880410+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:35:14.880410+0000","last_peered":"2024-09-10T07:35:14.880410+0000","last_clean":"2024-09-10T07:35:14.880410+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:35:14.880410+0000","last_undegraded":"2024-09-10T07:35:14.880410+0000","last_fullsized":"2024-09-10T07:35:14.880410+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881828+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:35:14.881828+0000","last_peered":"2024-09-10T07:35:14.881828+0000","last_clean":"2024-09-10T07:35:14.881828+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:35:14.881828+0000","last_undegraded":"2024-09-10T07:35:14.881828+0000","last_fullsized":"2024-09-10T07:35:14.881828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842879+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:35:14.842879+0000","last_peered":"2024-09-10T07:35:14.842879+0000","last_clean":"2024-09-10T07:35:14.842879+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:35:14.842879+0000","last_undegraded":"2024-09-10T07:35:14.842879+0000","last_fullsized":"2024-09-10T07:35:14.842879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854012+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:35:14.854012+0000","last_peered":"2024-09-10T07:35:14.854012+0000","last_clean":"2024-09-10T07:35:14.854012+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:35:14.854012+0000","last_undegraded":"2024-09-10T07:35:14.854012+0000","last_fullsized":"2024-09-10T07:35:14.854012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882098+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:35:14.882098+0000","last_peered":"2024-09-10T07:35:14.882098+0000","last_clean":"2024-09-10T07:35:14.882098+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:35:14.882098+0000","last_undegraded":"2024-09-10T07:35:14.882098+0000","last_fullsized":"2024-09-10T07:35:14.882098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977445+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:35:14.977445+0000","last_peered":"2024-09-10T07:35:14.977445+0000","last_clean":"2024-09-10T07:35:14.977445+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:35:14.977445+0000","last_undegraded":"2024-09-10T07:35:14.977445+0000","last_fullsized":"2024-09-10T07:35:14.977445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842419+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:35:14.842419+0000","last_peered":"2024-09-10T07:35:14.842419+0000","last_clean":"2024-09-10T07:35:14.842419+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:35:14.842419+0000","last_undegraded":"2024-09-10T07:35:14.842419+0000","last_fullsized":"2024-09-10T07:35:14.842419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882010+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:35:14.882010+0000","last_peered":"2024-09-10T07:35:14.882010+0000","last_clean":"2024-09-10T07:35:14.882010+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:35:14.882010+0000","last_undegraded":"2024-09-10T07:35:14.882010+0000","last_fullsized":"2024-09-10T07:35:14.882010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852151+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:35:14.852151+0000","last_peered":"2024-09-10T07:35:14.852151+0000","last_clean":"2024-09-10T07:35:14.852151+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:35:14.852151+0000","last_undegraded":"2024-09-10T07:35:14.852151+0000","last_fullsized":"2024-09-10T07:35:14.852151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977217+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:35:14.977217+0000","last_peered":"2024-09-10T07:35:14.977217+0000","last_clean":"2024-09-10T07:35:14.977217+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:35:14.977217+0000","last_undegraded":"2024-09-10T07:35:14.977217+0000","last_fullsized":"2024-09-10T07:35:14.977217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851152+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:35:14.851152+0000","last_peered":"2024-09-10T07:35:14.851152+0000","last_clean":"2024-09-10T07:35:14.851152+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:35:14.851152+0000","last_undegraded":"2024-09-10T07:35:14.851152+0000","last_fullsized":"2024-09-10T07:35:14.851152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853834+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:35:14.853834+0000","last_peered":"2024-09-10T07:35:14.853834+0000","last_clean":"2024-09-10T07:35:14.853834+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:35:14.853834+0000","last_undegraded":"2024-09-10T07:35:14.853834+0000","last_fullsized":"2024-09-10T07:35:14.853834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976695+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:35:14.976695+0000","last_peered":"2024-09-10T07:35:14.976695+0000","last_clean":"2024-09-10T07:35:14.976695+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:35:14.976695+0000","last_undegraded":"2024-09-10T07:35:14.976695+0000","last_fullsized":"2024-09-10T07:35:14.976695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882157+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:35:14.882157+0000","last_peered":"2024-09-10T07:35:14.882157+0000","last_clean":"2024-09-10T07:35:14.882157+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:35:14.882157+0000","last_undegraded":"2024-09-10T07:35:14.882157+0000","last_fullsized":"2024-09-10T07:35:14.882157+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858294+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:35:14.858294+0000","last_peered":"2024-09-10T07:35:14.858294+0000","last_clean":"2024-09-10T07:35:14.858294+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:35:14.858294+0000","last_undegraded":"2024-09-10T07:35:14.858294+0000","last_fullsized":"2024-09-10T07:35:14.858294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882468+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:35:14.882468+0000","last_peered":"2024-09-10T07:35:14.882468+0000","last_clean":"2024-09-10T07:35:14.882468+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:35:14.882468+0000","last_undegraded":"2024-09-10T07:35:14.882468+0000","last_fullsized":"2024-09-10T07:35:14.882468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842318+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:35:14.842318+0000","last_peered":"2024-09-10T07:35:14.842318+0000","last_clean":"2024-09-10T07:35:14.842318+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:35:14.842318+0000","last_undegraded":"2024-09-10T07:35:14.842318+0000","last_fullsized":"2024-09-10T07:35:14.842318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977400+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:35:14.977400+0000","last_peered":"2024-09-10T07:35:14.977400+0000","last_clean":"2024-09-10T07:35:14.977400+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:35:14.977400+0000","last_undegraded":"2024-09-10T07:35:14.977400+0000","last_fullsized":"2024-09-10T07:35:14.977400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'16","reported_seq":59,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.850694+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:35:14.850694+0000","last_peered":"2024-09-10T07:35:14.850694+0000","last_clean":"2024-09-10T07:35:14.850694+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:35:14.850694+0000","last_undegraded":"2024-09-10T07:35:14.850694+0000","last_fullsized":"2024-09-10T07:35:14.850694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976854+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:35:14.976854+0000","last_peered":"2024-09-10T07:35:14.976854+0000","last_clean":"2024-09-10T07:35:14.976854+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:35:14.976854+0000","last_undegraded":"2024-09-10T07:35:14.976854+0000","last_fullsized":"2024-09-10T07:35:14.976854+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879908+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:35:14.879908+0000","last_peered":"2024-09-10T07:35:14.879908+0000","last_clean":"2024-09-10T07:35:14.879908+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:35:14.879908+0000","last_undegraded":"2024-09-10T07:35:14.879908+0000","last_fullsized":"2024-09-10T07:35:14.879908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.872520+0000","last_change":"2024-09-10T07:35:14.872520+0000","last_active":"2024-09-10T07:35:14.872520+0000","last_peered":"2024-09-10T07:35:14.872520+0000","last_clean":"2024-09-10T07:35:14.872520+0000","last_became_active":"2024-09-10T07:35:14.860686+0000","last_became_peered":"2024-09-10T07:35:14.860686+0000","last_unstale":"2024-09-10T07:35:14.872520+0000","last_undegraded":"2024-09-10T07:35:14.872520+0000","last_fullsized":"2024-09-10T07:35:14.872520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:42.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852303+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:35:14.852303+0000","last_peered":"2024-09-10T07:35:14.852303+0000","last_clean":"2024-09-10T07:35:14.852303+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:35:14.852303+0000","last_undegraded":"2024-09-10T07:35:14.852303+0000","last_fullsized":"2024-09-10T07:35:14.852303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877630+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:35:14.877630+0000","last_peered":"2024-09-10T07:35:14.877630+0000","last_clean":"2024-09-10T07:35:14.877630+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:35:14.877630+0000","last_undegraded":"2024-09-10T07:35:14.877630+0000","last_fullsized":"2024-09-10T07:35:14.877630+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848907+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:35:14.848907+0000","last_peered":"2024-09-10T07:35:14.848907+0000","last_clean":"2024-09-10T07:35:14.848907+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:35:14.848907+0000","last_undegraded":"2024-09-10T07:35:14.848907+0000","last_fullsized":"2024-09-10T07:35:14.848907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853870+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:35:14.853870+0000","last_peered":"2024-09-10T07:35:14.853870+0000","last_clean":"2024-09-10T07:35:14.853870+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:35:14.853870+0000","last_undegraded":"2024-09-10T07:35:14.853870+0000","last_fullsized":"2024-09-10T07:35:14.853870+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877678+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:35:14.877678+0000","last_peered":"2024-09-10T07:35:14.877678+0000","last_clean":"2024-09-10T07:35:14.877678+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:35:14.877678+0000","last_undegraded":"2024-09-10T07:35:14.877678+0000","last_fullsized":"2024-09-10T07:35:14.877678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":117,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:29.909866+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:35:29.909866+0000","last_peered":"2024-09-10T07:35:29.909866+0000","last_clean":"2024-09-10T07:35:29.909866+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:35:29.909866+0000","last_undegraded":"2024-09-10T07:35:29.909866+0000","last_fullsized":"2024-09-10T07:35:29.909866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879669+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:35:14.879669+0000","last_peered":"2024-09-10T07:35:14.879669+0000","last_clean":"2024-09-10T07:35:14.879669+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:35:14.879669+0000","last_undegraded":"2024-09-10T07:35:14.879669+0000","last_fullsized":"2024-09-10T07:35:14.879669+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880031+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:35:14.880031+0000","last_peered":"2024-09-10T07:35:14.880031+0000","last_clean":"2024-09-10T07:35:14.880031+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:35:14.880031+0000","last_undegraded":"2024-09-10T07:35:14.880031+0000","last_fullsized":"2024-09-10T07:35:14.880031+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":116,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:29.901183+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:35:29.901183+0000","last_peered":"2024-09-10T07:35:29.901183+0000","last_clean":"2024-09-10T07:35:29.901183+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:35:29.901183+0000","last_undegraded":"2024-09-10T07:35:29.901183+0000","last_fullsized":"2024-09-10T07:35:29.901183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880558+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:35:14.880558+0000","last_peered":"2024-09-10T07:35:14.880558+0000","last_clean":"2024-09-10T07:35:14.880558+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:35:14.880558+0000","last_undegraded":"2024-09-10T07:35:14.880558+0000","last_fullsized":"2024-09-10T07:35:14.880558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879041+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:35:14.879041+0000","last_peered":"2024-09-10T07:35:14.879041+0000","last_clean":"2024-09-10T07:35:14.879041+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:35:14.879041+0000","last_undegraded":"2024-09-10T07:35:14.879041+0000","last_fullsized":"2024-09-10T07:35:14.879041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842761+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:35:14.842761+0000","last_peered":"2024-09-10T07:35:14.842761+0000","last_clean":"2024-09-10T07:35:14.842761+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:35:14.842761+0000","last_undegraded":"2024-09-10T07:35:14.842761+0000","last_fullsized":"2024-09-10T07:35:14.842761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880370+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:35:14.880370+0000","last_peered":"2024-09-10T07:35:14.880370+0000","last_clean":"2024-09-10T07:35:14.880370+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:35:14.880370+0000","last_undegraded":"2024-09-10T07:35:14.880370+0000","last_fullsized":"2024-09-10T07:35:14.880370+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977120+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:35:14.977120+0000","last_peered":"2024-09-10T07:35:14.977120+0000","last_clean":"2024-09-10T07:35:14.977120+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:35:14.977120+0000","last_undegraded":"2024-09-10T07:35:14.977120+0000","last_fullsized":"2024-09-10T07:35:14.977120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882470+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:35:14.882470+0000","last_peered":"2024-09-10T07:35:14.882470+0000","last_clean":"2024-09-10T07:35:14.882470+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:35:14.882470+0000","last_undegraded":"2024-09-10T07:35:14.882470+0000","last_fullsized":"2024-09-10T07:35:14.882470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842861+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:35:14.842861+0000","last_peered":"2024-09-10T07:35:14.842861+0000","last_clean":"2024-09-10T07:35:14.842861+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:35:14.842861+0000","last_undegraded":"2024-09-10T07:35:14.842861+0000","last_fullsized":"2024-09-10T07:35:14.842861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843123+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:35:14.843123+0000","last_peered":"2024-09-10T07:35:14.843123+0000","last_clean":"2024-09-10T07:35:14.843123+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:35:14.843123+0000","last_undegraded":"2024-09-10T07:35:14.843123+0000","last_fullsized":"2024-09-10T07:35:14.843123+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843358+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:35:14.843358+0000","last_peered":"2024-09-10T07:35:14.843358+0000","last_clean":"2024-09-10T07:35:14.843358+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:35:14.843358+0000","last_undegraded":"2024-09-10T07:35:14.843358+0000","last_fullsized":"2024-09-10T07:35:14.843358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":115,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.900484+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:35:28.900484+0000","last_peered":"2024-09-10T07:35:28.900484+0000","last_clean":"2024-09-10T07:35:28.900484+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:35:28.900484+0000","last_undegraded":"2024-09-10T07:35:28.900484+0000","last_fullsized":"2024-09-10T07:35:28.900484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976716+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:35:14.976716+0000","last_peered":"2024-09-10T07:35:14.976716+0000","last_clean":"2024-09-10T07:35:14.976716+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:35:14.976716+0000","last_undegraded":"2024-09-10T07:35:14.976716+0000","last_fullsized":"2024-09-10T07:35:14.976716+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843096+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:35:14.843096+0000","last_peered":"2024-09-10T07:35:14.843096+0000","last_clean":"2024-09-10T07:35:14.843096+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:35:14.843096+0000","last_undegraded":"2024-09-10T07:35:14.843096+0000","last_fullsized":"2024-09-10T07:35:14.843096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880652+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:35:14.880652+0000","last_peered":"2024-09-10T07:35:14.880652+0000","last_clean":"2024-09-10T07:35:14.880652+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:35:14.880652+0000","last_undegraded":"2024-09-10T07:35:14.880652+0000","last_fullsized":"2024-09-10T07:35:14.880652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857646+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:35:14.857646+0000","last_peered":"2024-09-10T07:35:14.857646+0000","last_clean":"2024-09-10T07:35:14.857646+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:35:14.857646+0000","last_undegraded":"2024-09-10T07:35:14.857646+0000","last_fullsized":"2024-09-10T07:35:14.857646+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843150+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:35:14.843150+0000","last_peered":"2024-09-10T07:35:14.843150+0000","last_clean":"2024-09-10T07:35:14.843150+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:35:14.843150+0000","last_undegraded":"2024-09-10T07:35:14.843150+0000","last_fullsized":"2024-09-10T07:35:14.843150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880435+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:35:14.880435+0000","last_peered":"2024-09-10T07:35:14.880435+0000","last_clean":"2024-09-10T07:35:14.880435+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:35:14.880435+0000","last_undegraded":"2024-09-10T07:35:14.880435+0000","last_fullsized":"2024-09-10T07:35:14.880435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879923+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:35:14.879923+0000","last_peered":"2024-09-10T07:35:14.879923+0000","last_clean":"2024-09-10T07:35:14.879923+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:35:14.879923+0000","last_undegraded":"2024-09-10T07:35:14.879923+0000","last_fullsized":"2024-09-10T07:35:14.879923+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852535+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:35:14.852535+0000","last_peered":"2024-09-10T07:35:14.852535+0000","last_clean":"2024-09-10T07:35:14.852535+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:35:14.852535+0000","last_undegraded":"2024-09-10T07:35:14.852535+0000","last_fullsized":"2024-09-10T07:35:14.852535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854413+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:35:14.854413+0000","last_peered":"2024-09-10T07:35:14.854413+0000","last_clean":"2024-09-10T07:35:14.854413+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:35:14.854413+0000","last_undegraded":"2024-09-10T07:35:14.854413+0000","last_fullsized":"2024-09-10T07:35:14.854413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879910+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:35:14.879910+0000","last_peered":"2024-09-10T07:35:14.879910+0000","last_clean":"2024-09-10T07:35:14.879910+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:35:14.879910+0000","last_undegraded":"2024-09-10T07:35:14.879910+0000","last_fullsized":"2024-09-10T07:35:14.879910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880026+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:35:14.880026+0000","last_peered":"2024-09-10T07:35:14.880026+0000","last_clean":"2024-09-10T07:35:14.880026+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:35:14.880026+0000","last_undegraded":"2024-09-10T07:35:14.880026+0000","last_fullsized":"2024-09-10T07:35:14.880026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":61,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853872+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:35:14.853872+0000","last_peered":"2024-09-10T07:35:14.853872+0000","last_clean":"2024-09-10T07:35:14.853872+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:35:14.853872+0000","last_undegraded":"2024-09-10T07:35:14.853872+0000","last_fullsized":"2024-09-10T07:35:14.853872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880346+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:35:14.880346+0000","last_peered":"2024-09-10T07:35:14.880346+0000","last_clean":"2024-09-10T07:35:14.880346+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:35:14.880346+0000","last_undegraded":"2024-09-10T07:35:14.880346+0000","last_fullsized":"2024-09-10T07:35:14.880346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842600+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:35:14.842600+0000","last_peered":"2024-09-10T07:35:14.842600+0000","last_clean":"2024-09-10T07:35:14.842600+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:35:14.842600+0000","last_undegraded":"2024-09-10T07:35:14.842600+0000","last_fullsized":"2024-09-10T07:35:14.842600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":114,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.900361+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:35:28.900361+0000","last_peered":"2024-09-10T07:35:28.900361+0000","last_clean":"2024-09-10T07:35:28.900361+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:35:28.900361+0000","last_undegraded":"2024-09-10T07:35:28.900361+0000","last_fullsized":"2024-09-10T07:35:28.900361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851287+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:35:14.851287+0000","last_peered":"2024-09-10T07:35:14.851287+0000","last_clean":"2024-09-10T07:35:14.851287+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:35:14.851287+0000","last_undegraded":"2024-09-10T07:35:14.851287+0000","last_fullsized":"2024-09-10T07:35:14.851287+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842874+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:35:14.842874+0000","last_peered":"2024-09-10T07:35:14.842874+0000","last_clean":"2024-09-10T07:35:14.842874+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:35:14.842874+0000","last_undegraded":"2024-09-10T07:35:14.842874+0000","last_fullsized":"2024-09-10T07:35:14.842874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880110+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:35:14.880110+0000","last_peered":"2024-09-10T07:35:14.880110+0000","last_clean":"2024-09-10T07:35:14.880110+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:35:14.880110+0000","last_undegraded":"2024-09-10T07:35:14.880110+0000","last_fullsized":"2024-09-10T07:35:14.880110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852212+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:35:14.852212+0000","last_peered":"2024-09-10T07:35:14.852212+0000","last_clean":"2024-09-10T07:35:14.852212+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:35:14.852212+0000","last_undegraded":"2024-09-10T07:35:14.852212+0000","last_fullsized":"2024-09-10T07:35:14.852212+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879822+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:35:14.879822+0000","last_peered":"2024-09-10T07:35:14.879822+0000","last_clean":"2024-09-10T07:35:14.879822+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:35:14.879822+0000","last_undegraded":"2024-09-10T07:35:14.879822+0000","last_fullsized":"2024-09-10T07:35:14.879822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852205+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:35:14.852205+0000","last_peered":"2024-09-10T07:35:14.852205+0000","last_clean":"2024-09-10T07:35:14.852205+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:35:14.852205+0000","last_undegraded":"2024-09-10T07:35:14.852205+0000","last_fullsized":"2024-09-10T07:35:14.852205+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880377+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:35:14.880377+0000","last_peered":"2024-09-10T07:35:14.880377+0000","last_clean":"2024-09-10T07:35:14.880377+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:35:14.880377+0000","last_undegraded":"2024-09-10T07:35:14.880377+0000","last_fullsized":"2024-09-10T07:35:14.880377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854403+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:35:14.854403+0000","last_peered":"2024-09-10T07:35:14.854403+0000","last_clean":"2024-09-10T07:35:14.854403+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:35:14.854403+0000","last_undegraded":"2024-09-10T07:35:14.854403+0000","last_fullsized":"2024-09-10T07:35:14.854403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'36","reported_seq":217,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.899536+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:35:28.899536+0000","last_peered":"2024-09-10T07:35:28.899536+0000","last_clean":"2024-09-10T07:35:28.899536+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:35:28.899536+0000","last_undegraded":"2024-09-10T07:35:28.899536+0000","last_fullsized":"2024-09-10T07:35:28.899536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64622,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848049+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:35:14.848049+0000","last_peered":"2024-09-10T07:35:14.848049+0000","last_clean":"2024-09-10T07:35:14.848049+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:35:14.848049+0000","last_undegraded":"2024-09-10T07:35:14.848049+0000","last_fullsized":"2024-09-10T07:35:14.848049+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976954+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:35:14.976954+0000","last_peered":"2024-09-10T07:35:14.976954+0000","last_clean":"2024-09-10T07:35:14.976954+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:35:14.976954+0000","last_undegraded":"2024-09-10T07:35:14.976954+0000","last_fullsized":"2024-09-10T07:35:14.976954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880584+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:35:14.880584+0000","last_peered":"2024-09-10T07:35:14.880584+0000","last_clean":"2024-09-10T07:35:14.880584+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:35:14.880584+0000","last_undegraded":"2024-09-10T07:35:14.880584+0000","last_fullsized":"2024-09-10T07:35:14.880584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882281+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:35:14.882281+0000","last_peered":"2024-09-10T07:35:14.882281+0000","last_clean":"2024-09-10T07:35:14.882281+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:35:14.882281+0000","last_undegraded":"2024-09-10T07:35:14.882281+0000","last_fullsized":"2024-09-10T07:35:14.882281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.856894+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:35:14.856894+0000","last_peered":"2024-09-10T07:35:14.856894+0000","last_clean":"2024-09-10T07:35:14.856894+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:35:14.856894+0000","last_undegraded":"2024-09-10T07:35:14.856894+0000","last_fullsized":"2024-09-10T07:35:14.856894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843188+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:35:14.843188+0000","last_peered":"2024-09-10T07:35:14.843188+0000","last_clean":"2024-09-10T07:35:14.843188+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:35:14.843188+0000","last_undegraded":"2024-09-10T07:35:14.843188+0000","last_fullsized":"2024-09-10T07:35:14.843188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847334+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:35:14.847334+0000","last_peered":"2024-09-10T07:35:14.847334+0000","last_clean":"2024-09-10T07:35:14.847334+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:35:14.847334+0000","last_undegraded":"2024-09-10T07:35:14.847334+0000","last_fullsized":"2024-09-10T07:35:14.847334+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977143+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:35:14.977143+0000","last_peered":"2024-09-10T07:35:14.977143+0000","last_clean":"2024-09-10T07:35:14.977143+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:35:14.977143+0000","last_undegraded":"2024-09-10T07:35:14.977143+0000","last_fullsized":"2024-09-10T07:35:14.977143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":175,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.899686+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:35:28.899686+0000","last_peered":"2024-09-10T07:35:28.899686+0000","last_clean":"2024-09-10T07:35:28.899686+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:35:28.899686+0000","last_undegraded":"2024-09-10T07:35:28.899686+0000","last_fullsized":"2024-09-10T07:35:28.899686+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976991+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:35:14.976991+0000","last_peered":"2024-09-10T07:35:14.976991+0000","last_clean":"2024-09-10T07:35:14.976991+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:35:14.976991+0000","last_undegraded":"2024-09-10T07:35:14.976991+0000","last_fullsized":"2024-09-10T07:35:14.976991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858109+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:35:14.858109+0000","last_peered":"2024-09-10T07:35:14.858109+0000","last_clean":"2024-09-10T07:35:14.858109+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:35:14.858109+0000","last_undegraded":"2024-09-10T07:35:14.858109+0000","last_fullsized":"2024-09-10T07:35:14.858109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847742+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:35:14.847742+0000","last_peered":"2024-09-10T07:35:14.847742+0000","last_clean":"2024-09-10T07:35:14.847742+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:35:14.847742+0000","last_undegraded":"2024-09-10T07:35:14.847742+0000","last_fullsized":"2024-09-10T07:35:14.847742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880198+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:35:14.880198+0000","last_peered":"2024-09-10T07:35:14.880198+0000","last_clean":"2024-09-10T07:35:14.880198+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:35:14.880198+0000","last_undegraded":"2024-09-10T07:35:14.880198+0000","last_fullsized":"2024-09-10T07:35:14.880198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852519+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:35:14.852519+0000","last_peered":"2024-09-10T07:35:14.852519+0000","last_clean":"2024-09-10T07:35:14.852519+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:35:14.852519+0000","last_undegraded":"2024-09-10T07:35:14.852519+0000","last_fullsized":"2024-09-10T07:35:14.852519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976994+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:35:14.976994+0000","last_peered":"2024-09-10T07:35:14.976994+0000","last_clean":"2024-09-10T07:35:14.976994+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:35:14.976994+0000","last_undegraded":"2024-09-10T07:35:14.976994+0000","last_fullsized":"2024-09-10T07:35:14.976994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879143+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:35:14.879143+0000","last_peered":"2024-09-10T07:35:14.879143+0000","last_clean":"2024-09-10T07:35:14.879143+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:35:14.879143+0000","last_undegraded":"2024-09-10T07:35:14.879143+0000","last_fullsized":"2024-09-10T07:35:14.879143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877732+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:35:14.877732+0000","last_peered":"2024-09-10T07:35:14.877732+0000","last_clean":"2024-09-10T07:35:14.877732+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:35:14.877732+0000","last_undegraded":"2024-09-10T07:35:14.877732+0000","last_fullsized":"2024-09-10T07:35:14.877732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848270+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:35:14.848270+0000","last_peered":"2024-09-10T07:35:14.848270+0000","last_clean":"2024-09-10T07:35:14.848270+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:35:14.848270+0000","last_undegraded":"2024-09-10T07:35:14.848270+0000","last_fullsized":"2024-09-10T07:35:14.848270+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842456+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:35:14.842456+0000","last_peered":"2024-09-10T07:35:14.842456+0000","last_clean":"2024-09-10T07:35:14.842456+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:35:14.842456+0000","last_undegraded":"2024-09-10T07:35:14.842456+0000","last_fullsized":"2024-09-10T07:35:14.842456+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":117,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:28.900413+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:35:28.900413+0000","last_peered":"2024-09-10T07:35:28.900413+0000","last_clean":"2024-09-10T07:35:28.900413+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:35:28.900413+0000","last_undegraded":"2024-09-10T07:35:28.900413+0000","last_fullsized":"2024-09-10T07:35:28.900413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854042+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:35:14.854042+0000","last_peered":"2024-09-10T07:35:14.854042+0000","last_clean":"2024-09-10T07:35:14.854042+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:35:14.854042+0000","last_undegraded":"2024-09-10T07:35:14.854042+0000","last_fullsized":"2024-09-10T07:35:14.854042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842576+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:35:14.842576+0000","last_peered":"2024-09-10T07:35:14.842576+0000","last_clean":"2024-09-10T07:35:14.842576+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:35:14.842576+0000","last_undegraded":"2024-09-10T07:35:14.842576+0000","last_fullsized":"2024-09-10T07:35:14.842576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852091+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:35:14.852091+0000","last_peered":"2024-09-10T07:35:14.852091+0000","last_clean":"2024-09-10T07:35:14.852091+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:35:14.852091+0000","last_undegraded":"2024-09-10T07:35:14.852091+0000","last_fullsized":"2024-09-10T07:35:14.852091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842673+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:35:14.842673+0000","last_peered":"2024-09-10T07:35:14.842673+0000","last_clean":"2024-09-10T07:35:14.842673+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:35:14.842673+0000","last_undegraded":"2024-09-10T07:35:14.842673+0000","last_fullsized":"2024-09-10T07:35:14.842673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842346+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:35:14.842346+0000","last_peered":"2024-09-10T07:35:14.842346+0000","last_clean":"2024-09-10T07:35:14.842346+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:35:14.842346+0000","last_undegraded":"2024-09-10T07:35:14.842346+0000","last_fullsized":"2024-09-10T07:35:14.842346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857747+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:35:14.857747+0000","last_peered":"2024-09-10T07:35:14.857747+0000","last_clean":"2024-09-10T07:35:14.857747+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:35:14.857747+0000","last_undegraded":"2024-09-10T07:35:14.857747+0000","last_fullsized":"2024-09-10T07:35:14.857747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879821+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:35:14.879821+0000","last_peered":"2024-09-10T07:35:14.879821+0000","last_clean":"2024-09-10T07:35:14.879821+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:35:14.879821+0000","last_undegraded":"2024-09-10T07:35:14.879821+0000","last_fullsized":"2024-09-10T07:35:14.879821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852916+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:35:14.852916+0000","last_peered":"2024-09-10T07:35:14.852916+0000","last_clean":"2024-09-10T07:35:14.852916+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:35:14.852916+0000","last_undegraded":"2024-09-10T07:35:14.852916+0000","last_fullsized":"2024-09-10T07:35:14.852916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842926+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:35:14.842926+0000","last_peered":"2024-09-10T07:35:14.842926+0000","last_clean":"2024-09-10T07:35:14.842926+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:35:14.842926+0000","last_undegraded":"2024-09-10T07:35:14.842926+0000","last_fullsized":"2024-09-10T07:35:14.842926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977392+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:35:14.977392+0000","last_peered":"2024-09-10T07:35:14.977392+0000","last_clean":"2024-09-10T07:35:14.977392+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:35:14.977392+0000","last_undegraded":"2024-09-10T07:35:14.977392+0000","last_fullsized":"2024-09-10T07:35:14.977392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879986+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:35:14.879986+0000","last_peered":"2024-09-10T07:35:14.879986+0000","last_clean":"2024-09-10T07:35:14.879986+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:35:14.879986+0000","last_undegraded":"2024-09-10T07:35:14.879986+0000","last_fullsized":"2024-09-10T07:35:14.879986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977101+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:35:14.977101+0000","last_peered":"2024-09-10T07:35:14.977101+0000","last_clean":"2024-09-10T07:35:14.977101+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:35:14.977101+0000","last_undegraded":"2024-09-10T07:35:14.977101+0000","last_fullsized":"2024-09-10T07:35:14.977101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976876+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:35:14.976876+0000","last_peered":"2024-09-10T07:35:14.976876+0000","last_clean":"2024-09-10T07:35:14.976876+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:35:14.976876+0000","last_undegraded":"2024-09-10T07:35:14.976876+0000","last_fullsized":"2024-09-10T07:35:14.976876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879430+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:35:14.879430+0000","last_peered":"2024-09-10T07:35:14.879430+0000","last_clean":"2024-09-10T07:35:14.879430+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:35:14.879430+0000","last_undegraded":"2024-09-10T07:35:14.879430+0000","last_fullsized":"2024-09-10T07:35:14.879430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852322+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:35:14.852322+0000","last_peered":"2024-09-10T07:35:14.852322+0000","last_clean":"2024-09-10T07:35:14.852322+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:35:14.852322+0000","last_undegraded":"2024-09-10T07:35:14.852322+0000","last_fullsized":"2024-09-10T07:35:14.852322+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842741+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:35:14.842741+0000","last_peered":"2024-09-10T07:35:14.842741+0000","last_clean":"2024-09-10T07:35:14.842741+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:35:14.842741+0000","last_undegraded":"2024-09-10T07:35:14.842741+0000","last_fullsized":"2024-09-10T07:35:14.842741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.850585+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:35:14.850585+0000","last_peered":"2024-09-10T07:35:14.850585+0000","last_clean":"2024-09-10T07:35:14.850585+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:35:14.850585+0000","last_undegraded":"2024-09-10T07:35:14.850585+0000","last_fullsized":"2024-09-10T07:35:14.850585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852768+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:35:14.852768+0000","last_peered":"2024-09-10T07:35:14.852768+0000","last_clean":"2024-09-10T07:35:14.852768+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:35:14.852768+0000","last_undegraded":"2024-09-10T07:35:14.852768+0000","last_fullsized":"2024-09-10T07:35:14.852768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976577+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:35:14.976577+0000","last_peered":"2024-09-10T07:35:14.976577+0000","last_clean":"2024-09-10T07:35:14.976577+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:35:14.976577+0000","last_undegraded":"2024-09-10T07:35:14.976577+0000","last_fullsized":"2024-09-10T07:35:14.976577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857364+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:35:14.857364+0000","last_peered":"2024-09-10T07:35:14.857364+0000","last_clean":"2024-09-10T07:35:14.857364+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:35:14.857364+0000","last_undegraded":"2024-09-10T07:35:14.857364+0000","last_fullsized":"2024-09-10T07:35:14.857364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857503+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:35:14.857503+0000","last_peered":"2024-09-10T07:35:14.857503+0000","last_clean":"2024-09-10T07:35:14.857503+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:35:14.857503+0000","last_undegraded":"2024-09-10T07:35:14.857503+0000","last_fullsized":"2024-09-10T07:35:14.857503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879336+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:35:14.879336+0000","last_peered":"2024-09-10T07:35:14.879336+0000","last_clean":"2024-09-10T07:35:14.879336+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:35:14.879336+0000","last_undegraded":"2024-09-10T07:35:14.879336+0000","last_fullsized":"2024-09-10T07:35:14.879336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882260+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:35:14.882260+0000","last_peered":"2024-09-10T07:35:14.882260+0000","last_clean":"2024-09-10T07:35:14.882260+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:35:14.882260+0000","last_undegraded":"2024-09-10T07:35:14.882260+0000","last_fullsized":"2024-09-10T07:35:14.882260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843320+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:35:14.843320+0000","last_peered":"2024-09-10T07:35:14.843320+0000","last_clean":"2024-09-10T07:35:14.843320+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:35:14.843320+0000","last_undegraded":"2024-09-10T07:35:14.843320+0000","last_fullsized":"2024-09-10T07:35:14.843320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877986+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:35:14.877986+0000","last_peered":"2024-09-10T07:35:14.877986+0000","last_clean":"2024-09-10T07:35:14.877986+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:35:14.877986+0000","last_undegraded":"2024-09-10T07:35:14.877986+0000","last_fullsized":"2024-09-10T07:35:14.877986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977201+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:35:14.977201+0000","last_peered":"2024-09-10T07:35:14.977201+0000","last_clean":"2024-09-10T07:35:14.977201+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:35:14.977201+0000","last_undegraded":"2024-09-10T07:35:14.977201+0000","last_fullsized":"2024-09-10T07:35:14.977201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976968+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:35:14.976968+0000","last_peered":"2024-09-10T07:35:14.976968+0000","last_clean":"2024-09-10T07:35:14.976968+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:35:14.976968+0000","last_undegraded":"2024-09-10T07:35:14.976968+0000","last_fullsized":"2024-09-10T07:35:14.976968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842700+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:35:14.842700+0000","last_peered":"2024-09-10T07:35:14.842700+0000","last_clean":"2024-09-10T07:35:14.842700+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:35:14.842700+0000","last_undegraded":"2024-09-10T07:35:14.842700+0000","last_fullsized":"2024-09-10T07:35:14.842700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877599+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:35:14.877599+0000","last_peered":"2024-09-10T07:35:14.877599+0000","last_clean":"2024-09-10T07:35:14.877599+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:35:14.877599+0000","last_undegraded":"2024-09-10T07:35:14.877599+0000","last_fullsized":"2024-09-10T07:35:14.877599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853987+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:35:14.853987+0000","last_peered":"2024-09-10T07:35:14.853987+0000","last_clean":"2024-09-10T07:35:14.853987+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:35:14.853987+0000","last_undegraded":"2024-09-10T07:35:14.853987+0000","last_fullsized":"2024-09-10T07:35:14.853987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882152+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:35:14.882152+0000","last_peered":"2024-09-10T07:35:14.882152+0000","last_clean":"2024-09-10T07:35:14.882152+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:35:14.882152+0000","last_undegraded":"2024-09-10T07:35:14.882152+0000","last_fullsized":"2024-09-10T07:35:14.882152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.856312+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:35:14.856312+0000","last_peered":"2024-09-10T07:35:14.856312+0000","last_clean":"2024-09-10T07:35:14.856312+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:35:14.856312+0000","last_undegraded":"2024-09-10T07:35:14.856312+0000","last_fullsized":"2024-09-10T07:35:14.856312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858214+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:35:14.858214+0000","last_peered":"2024-09-10T07:35:14.858214+0000","last_clean":"2024-09-10T07:35:14.858214+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:35:14.858214+0000","last_undegraded":"2024-09-10T07:35:14.858214+0000","last_fullsized":"2024-09-10T07:35:14.858214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852694+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:35:14.852694+0000","last_peered":"2024-09-10T07:35:14.852694+0000","last_clean":"2024-09-10T07:35:14.852694+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:35:14.852694+0000","last_undegraded":"2024-09-10T07:35:14.852694+0000","last_fullsized":"2024-09-10T07:35:14.852694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71830,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":685,"num_read_kb":419,"num_write":251,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1097728,"data_stored":561212,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710763,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1232896,"data_stored":1212470,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874286,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":565058,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135926,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1093632,"data_stored":561228,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1155072,"data_stored":1158232,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1081344,"data_stored":556859,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1163264,"data_stored":1153421,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1089536,"data_stored":560540,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1093632,"data_stored":556174,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1163264,"data_stored":622663,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1150976,"data_stored":622511,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":557922,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:35:34.404 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T07:35:34.404 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T07:35:34.404 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T07:35:34.405 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T07:35:34.782 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:35.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:34 smithi181 ceph-mon[35397]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:35:35.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:34 smithi063 ceph-mon[35020]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:35:35.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:34 smithi046 ceph-mon[30705]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 -- 172.21.15.46:0/1360297619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 msgr2=0x7f8d48104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 --2- 172.21.15.46:0/1360297619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 0x7f8d48104210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009900 tx=0x7f8d3802a410 comp rx=0 tx=0).stop 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 -- 172.21.15.46:0/1360297619 shutdown_connections 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 --2- 172.21.15.46:0/1360297619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d48108f60 0x7f8d48110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 --2- 172.21.15.46:0/1360297619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d481047e0 0x7f8d48108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 --2- 172.21.15.46:0/1360297619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 0x7f8d48104210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 -- 172.21.15.46:0/1360297619 >> 172.21.15.46:0/1360297619 conn(0x7f8d480ff8d0 msgr2=0x7f8d48101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 -- 172.21.15.46:0/1360297619 shutdown_connections 2024-09-10T07:35:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.421+0000 7f8d47e00640 1 -- 172.21.15.46:0/1360297619 wait complete. 2024-09-10T07:35:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.422+0000 7f8d47e00640 1 Processor -- start 2024-09-10T07:35:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.422+0000 7f8d47e00640 1 -- start start 2024-09-10T07:35:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.422+0000 7f8d47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 0x7f8d4819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.422+0000 7f8d47e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d481047e0 0x7f8d4819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.422+0000 7f8d47e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d48108f60 0x7f8d4819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.422+0000 7f8d47e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8d48114010 con 0x7f8d48103e30 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.422+0000 7f8d47e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8d48113e90 con 0x7f8d48108f60 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.422+0000 7f8d47e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d48114190 con 0x7f8d481047e0 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 0x7f8d4819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d48108f60 0x7f8d4819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d48108f60 0x7f8d4819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44302/0 (socket says 172.21.15.46:44302) 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d47400640 1 -- 172.21.15.46:0/2998733928 learned_addr learned my addr 172.21.15.46:0/2998733928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 0x7f8d4819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41158/0 (socket says 172.21.15.46:41158) 2024-09-10T07:35:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d3fe00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d481047e0 0x7f8d4819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d47400640 1 -- 172.21.15.46:0/2998733928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d481047e0 msgr2=0x7f8d4819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d47400640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d481047e0 0x7f8d4819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d47400640 1 -- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 msgr2=0x7f8d4819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d47400640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 0x7f8d4819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.423+0000 7f8d47400640 1 -- 172.21.15.46:0/2998733928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4819f450 con 0x7f8d48108f60 2024-09-10T07:35:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.424+0000 7f8d47400640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d48108f60 0x7f8d4819ed70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009fd0 tx=0x7f8d4000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.424+0000 7f8d3f400640 1 -- 172.21.15.46:0/2998733928 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40017070 con 0x7f8d48108f60 2024-09-10T07:35:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.424+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4819f740 con 0x7f8d48108f60 2024-09-10T07:35:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.424+0000 7f8d3f400640 1 -- 172.21.15.46:0/2998733928 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d40003e50 con 0x7f8d48108f60 2024-09-10T07:35:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.424+0000 7f8d3f400640 1 -- 172.21.15.46:0/2998733928 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d400106c0 con 0x7f8d48108f60 2024-09-10T07:35:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.424+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d481a7020 con 0x7f8d48108f60 2024-09-10T07:35:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.425+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d48104210 con 0x7f8d48108f60 2024-09-10T07:35:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.427+0000 7f8d3f400640 1 -- 172.21.15.46:0/2998733928 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8d40003970 con 0x7f8d48108f60 2024-09-10T07:35:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.429+0000 7f8d3f400640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8d2c0a2c10 0x7f8d2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.429+0000 7f8d3f400640 1 -- 172.21.15.46:0/2998733928 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8d400cf210 con 0x7f8d48108f60 2024-09-10T07:35:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.429+0000 7f8d46a00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8d2c0a2c10 0x7f8d2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.430+0000 7f8d46a00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8d2c0a2c10 0x7f8d2c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000c00 tx=0x7f8d38005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.431+0000 7f8d3f400640 1 -- 172.21.15.46:0/2998733928 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d4009b5e0 con 0x7f8d48108f60 2024-09-10T07:35:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.658+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8d4802d7c0 con 0x7f8d48108f60 2024-09-10T07:35:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.658+0000 7f8d3f400640 1 -- 172.21.15.46:0/2998733928 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8d4009f8e0 con 0x7f8d48108f60 2024-09-10T07:35:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:35:35.658 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T07:35:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8d2c0a2c10 msgr2=0x7f8d2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8d2c0a2c10 0x7f8d2c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000c00 tx=0x7f8d38005b50 comp rx=0 tx=0).stop 2024-09-10T07:35:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d48108f60 msgr2=0x7f8d4819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d48108f60 0x7f8d4819ed70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009fd0 tx=0x7f8d4000b260 comp rx=0 tx=0).stop 2024-09-10T07:35:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 shutdown_connections 2024-09-10T07:35:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d48108f60 0x7f8d4819ed70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d481047e0 0x7f8d4819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8d2c0a2c10 0x7f8d2c0a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 --2- 172.21.15.46:0/2998733928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48103e30 0x7f8d4819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 >> 172.21.15.46:0/2998733928 conn(0x7f8d480ff8d0 msgr2=0x7f8d48100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 shutdown_connections 2024-09-10T07:35:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:35.661+0000 7f8d47e00640 1 -- 172.21.15.46:0/2998733928 wait complete. 2024-09-10T07:35:35.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:35 smithi046 ceph-mon[30705]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T07:35:35.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:35 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:35:35.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:35 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2998733928' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T07:35:36.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:35 smithi181 ceph-mon[35397]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T07:35:36.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:35 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:35:36.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:35 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2998733928' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T07:35:36.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:35 smithi063 ceph-mon[35020]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T07:35:36.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:35 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:35:36.151 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:35 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2998733928' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T07:35:36.462 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T07:35:36.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:35:37.535 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 -- 172.21.15.46:0/2850804923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 msgr2=0x7f29181043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 --2- 172.21.15.46:0/2850804923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 0x7f29181043d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f290c009be0 tx=0x7f290c02a6c0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 -- 172.21.15.46:0/2850804923 shutdown_connections 2024-09-10T07:35:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 --2- 172.21.15.46:0/2850804923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2918109120 0x7f2918110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 --2- 172.21.15.46:0/2850804923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29181049a0 0x7f29181089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 --2- 172.21.15.46:0/2850804923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 0x7f29181043d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 -- 172.21.15.46:0/2850804923 >> 172.21.15.46:0/2850804923 conn(0x7f29180ffa90 msgr2=0x7f2918101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 -- 172.21.15.46:0/2850804923 shutdown_connections 2024-09-10T07:35:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.143+0000 7f2920400640 1 -- 172.21.15.46:0/2850804923 wait complete. 2024-09-10T07:35:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.144+0000 7f2920400640 1 Processor -- start 2024-09-10T07:35:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.144+0000 7f2920400640 1 -- start start 2024-09-10T07:35:38.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f2920400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 0x7f291819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:38.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f2920400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29181049a0 0x7f291819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:38.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f2920400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2918109120 0x7f291819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:38.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f2920400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2918114150 con 0x7f2918103ff0 2024-09-10T07:35:38.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f2920400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2918113fd0 con 0x7f29181049a0 2024-09-10T07:35:38.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f2920400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f29181142d0 con 0x7f2918109120 2024-09-10T07:35:38.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 0x7f291819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:38.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 0x7f291819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50774/0 (socket says 172.21.15.46:50774) 2024-09-10T07:35:38.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291da00640 1 -- 172.21.15.46:0/2073575639 learned_addr learned my addr 172.21.15.46:0/2073575639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:38.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291d000640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29181049a0 0x7f291819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:38.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291e400640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2918109120 0x7f291819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:38.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291d000640 1 -- 172.21.15.46:0/2073575639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2918109120 msgr2=0x7f291819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:38.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291d000640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2918109120 0x7f291819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291d000640 1 -- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 msgr2=0x7f291819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.145+0000 7f291d000640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 0x7f291819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.146+0000 7f291d000640 1 -- 172.21.15.46:0/2073575639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291819f5a0 con 0x7f29181049a0 2024-09-10T07:35:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.146+0000 7f291e400640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2918109120 0x7f291819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.146+0000 7f291d000640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29181049a0 0x7f291819ab30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2904005fd0 tx=0x7f290400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.146+0000 7f2916000640 1 -- 172.21.15.46:0/2073575639 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290400cd00 con 0x7f29181049a0 2024-09-10T07:35:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.146+0000 7f2916000640 1 -- 172.21.15.46:0/2073575639 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29040035f0 con 0x7f29181049a0 2024-09-10T07:35:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.146+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291819f890 con 0x7f29181049a0 2024-09-10T07:35:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.146+0000 7f2916000640 1 -- 172.21.15.46:0/2073575639 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904010670 con 0x7f29181049a0 2024-09-10T07:35:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.146+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29181a7170 con 0x7f29181049a0 2024-09-10T07:35:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.147+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29181043d0 con 0x7f29181049a0 2024-09-10T07:35:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.149+0000 7f2916000640 1 -- 172.21.15.46:0/2073575639 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f29040037a0 con 0x7f29181049a0 2024-09-10T07:35:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.151+0000 7f2916000640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29000a2c10 0x7f29000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.151+0000 7f2916000640 1 -- 172.21.15.46:0/2073575639 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f29040cf830 con 0x7f29181049a0 2024-09-10T07:35:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.151+0000 7f291da00640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29000a2c10 0x7f29000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.152+0000 7f291da00640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29000a2c10 0x7f29000a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f290c000f80 tx=0x7f290c009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.154+0000 7f2916000640 1 -- 172.21.15.46:0/2073575639 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2904009510 con 0x7f29181049a0 2024-09-10T07:35:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.389+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f291819b950 con 0x7f29181049a0 2024-09-10T07:35:38.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:37 smithi046 ceph-mon[30705]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T07:35:38.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:37 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:37 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:37 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:37 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.391+0000 7f2916000640 1 -- 172.21.15.46:0/2073575639 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11972 (secure 0 0 0) 0x7f290409c250 con 0x7f29181049a0 2024-09-10T07:35:38.391 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:35:38.391 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"btime":"2024-09-10T07:34:28:805113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:34:27.875261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:35:38.391 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T07:35:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29000a2c10 msgr2=0x7f29000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29000a2c10 0x7f29000a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f290c000f80 tx=0x7f290c009840 comp rx=0 tx=0).stop 2024-09-10T07:35:38.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29181049a0 msgr2=0x7f291819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:38.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29181049a0 0x7f291819ab30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2904005fd0 tx=0x7f290400ee50 comp rx=0 tx=0).stop 2024-09-10T07:35:38.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 shutdown_connections 2024-09-10T07:35:38.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2918109120 0x7f291819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29181049a0 0x7f291819ab30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29000a2c10 0x7f29000a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 --2- 172.21.15.46:0/2073575639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918103ff0 0x7f291819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.393+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 >> 172.21.15.46:0/2073575639 conn(0x7f29180ffa90 msgr2=0x7f2918100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.394+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 shutdown_connections 2024-09-10T07:35:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:38.394+0000 7f2920400640 1 -- 172.21.15.46:0/2073575639 wait complete. 2024-09-10T07:35:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:37 smithi181 ceph-mon[35397]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T07:35:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:37 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:37 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:37 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:37 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:37 smithi063 ceph-mon[35020]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T07:35:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:37 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:37 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:37 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:37 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:39.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:38 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2073575639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:39.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:39.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:38 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:39.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:35:39.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:38 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2073575639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:39.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:39.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:38 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:39.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:38 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2073575639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:39.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:39.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:38 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:39.633 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:40.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:39 smithi046 ceph-mon[30705]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T07:35:40.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:35:40.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:35:40.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:40.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.219+0000 7fae89600640 1 -- 172.21.15.46:0/1035407130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 msgr2=0x7fae841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:40.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.219+0000 7fae89600640 1 --2- 172.21.15.46:0/1035407130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 0x7fae841021c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fae74009900 tx=0x7fae7402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.220+0000 7fae89600640 1 -- 172.21.15.46:0/1035407130 shutdown_connections 2024-09-10T07:35:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.220+0000 7fae89600640 1 --2- 172.21.15.46:0/1035407130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84106f10 0x7fae8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.220+0000 7fae89600640 1 --2- 172.21.15.46:0/1035407130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae84102790 0x7fae841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.220+0000 7fae89600640 1 --2- 172.21.15.46:0/1035407130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 0x7fae841021c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.220+0000 7fae89600640 1 -- 172.21.15.46:0/1035407130 >> 172.21.15.46:0/1035407130 conn(0x7fae840fd8c0 msgr2=0x7fae840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.220+0000 7fae89600640 1 -- 172.21.15.46:0/1035407130 shutdown_connections 2024-09-10T07:35:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.220+0000 7fae89600640 1 -- 172.21.15.46:0/1035407130 wait complete. 2024-09-10T07:35:40.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.221+0000 7fae89600640 1 Processor -- start 2024-09-10T07:35:40.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.221+0000 7fae89600640 1 -- start start 2024-09-10T07:35:40.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.221+0000 7fae89600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 0x7fae84198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.221+0000 7fae89600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae84102790 0x7fae84198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae89600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84106f10 0x7fae8419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae89600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae84111fc0 con 0x7fae84101de0 2024-09-10T07:35:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae89600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae84111e40 con 0x7fae84106f10 2024-09-10T07:35:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae89600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112140 con 0x7fae84102790 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae82a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 0x7fae84198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae83400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84106f10 0x7fae8419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae82a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 0x7fae84198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50792/0 (socket says 172.21.15.46:50792) 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae82a00640 1 -- 172.21.15.46:0/1096572652 learned_addr learned my addr 172.21.15.46:0/1096572652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae7be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae84102790 0x7fae84198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae83400640 1 -- 172.21.15.46:0/1096572652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae84102790 msgr2=0x7fae84198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae83400640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae84102790 0x7fae84198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae83400640 1 -- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 msgr2=0x7fae84198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae83400640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 0x7fae84198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae83400640 1 -- 172.21.15.46:0/1096572652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8419d3f0 con 0x7fae84106f10 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.222+0000 7fae82a00640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 0x7fae84198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.223+0000 7fae83400640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84106f10 0x7fae8419cd10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009fd0 tx=0x7fae7c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.223+0000 7fae7b400640 1 -- 172.21.15.46:0/1096572652 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00ce20 con 0x7fae84106f10 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.223+0000 7fae7b400640 1 -- 172.21.15.46:0/1096572652 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae7c005020 con 0x7fae84106f10 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.223+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8419d6e0 con 0x7fae84106f10 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.223+0000 7fae7b400640 1 -- 172.21.15.46:0/1096572652 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c010720 con 0x7fae84106f10 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.223+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841a4fc0 con 0x7fae84106f10 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.224+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae841021c0 con 0x7fae84106f10 2024-09-10T07:35:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.226+0000 7fae7b400640 1 -- 172.21.15.46:0/1096572652 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae7c005540 con 0x7fae84106f10 2024-09-10T07:35:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.227+0000 7fae7b400640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae680a2c10 0x7fae680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.227+0000 7fae7b400640 1 -- 172.21.15.46:0/1096572652 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fae7c0cfbc0 con 0x7fae84106f10 2024-09-10T07:35:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.228+0000 7fae82a00640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae680a2c10 0x7fae680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.228+0000 7fae82a00640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae680a2c10 0x7fae680a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae740096c0 tx=0x7fae740046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.231+0000 7fae7b400640 1 -- 172.21.15.46:0/1096572652 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae7c09c640 con 0x7fae84106f10 2024-09-10T07:35:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:39 smithi181 ceph-mon[35397]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T07:35:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:35:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:35:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:39 smithi063 ceph-mon[35020]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T07:35:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:35:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:35:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:35:40.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.456+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae8402d7c0 con 0x7fae84106f10 2024-09-10T07:35:40.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.457+0000 7fae7b400640 1 -- 172.21.15.46:0/1096572652 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11972 (secure 0 0 0) 0x7fae7c09bf40 con 0x7fae84106f10 2024-09-10T07:35:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:35:40.457 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"btime":"2024-09-10T07:34:28:805113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:34:27.875261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:35:40.457 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T07:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae680a2c10 msgr2=0x7fae680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae680a2c10 0x7fae680a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae740096c0 tx=0x7fae740046c0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84106f10 msgr2=0x7fae8419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84106f10 0x7fae8419cd10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009fd0 tx=0x7fae7c00ef10 comp rx=0 tx=0).stop 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 shutdown_connections 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84106f10 0x7fae8419cd10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae84102790 0x7fae84198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae680a2c10 0x7fae680a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 --2- 172.21.15.46:0/1096572652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae84101de0 0x7fae84198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 >> 172.21.15.46:0/1096572652 conn(0x7fae840fd8c0 msgr2=0x7fae840febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 shutdown_connections 2024-09-10T07:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:40.460+0000 7fae89600640 1 -- 172.21.15.46:0/1096572652 wait complete. 2024-09-10T07:35:41.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:40 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1096572652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:41.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:35:41.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:40 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1096572652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:41.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:40 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1096572652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:42.113 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:41 smithi046 ceph-mon[30705]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-10T07:35:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.226+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2113054432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ea4109120 msgr2=0x7f9ea4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.226+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2113054432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ea4109120 0x7f9ea4110ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009ca0 tx=0x7f9e9c02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.227+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2113054432 shutdown_connections 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.227+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2113054432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ea4109120 0x7f9ea4110ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.227+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2113054432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea41049a0 0x7f9ea41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.227+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2113054432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ea4103ff0 0x7f9ea41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.227+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2113054432 >> 172.21.15.46:0/2113054432 conn(0x7f9ea40ffa90 msgr2=0x7f9ea4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.227+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2113054432 shutdown_connections 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.227+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2113054432 wait complete. 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9eaaa00640 1 Processor -- start 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9eaaa00640 1 -- start start 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ea4103ff0 0x7f9ea419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ea41049a0 0x7f9ea419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea4109120 0x7f9ea419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9eaaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea4114150 con 0x7f9ea4109120 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9eaaa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea4113fd0 con 0x7f9ea4103ff0 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9eaaa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41142d0 con 0x7f9ea41049a0 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9ea8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea4109120 0x7f9ea419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9ea8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea4109120 0x7f9ea419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50820/0 (socket says 172.21.15.46:50820) 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.228+0000 7f9ea8a00640 1 -- 172.21.15.46:0/2901142414 learned_addr learned my addr 172.21.15.46:0/2901142414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea3e00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ea4103ff0 0x7f9ea419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea3400640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ea41049a0 0x7f9ea419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea8a00640 1 -- 172.21.15.46:0/2901142414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ea41049a0 msgr2=0x7f9ea419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea8a00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ea41049a0 0x7f9ea419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea8a00640 1 -- 172.21.15.46:0/2901142414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ea4103ff0 msgr2=0x7f9ea419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea8a00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ea4103ff0 0x7f9ea419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea8a00640 1 -- 172.21.15.46:0/2901142414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea419f560 con 0x7f9ea4109120 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea3e00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ea4103ff0 0x7f9ea419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.229+0000 7f9ea8a00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea4109120 0x7f9ea419ee80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c031a20 tx=0x7f9e9c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.230+0000 7f9ea0c00640 1 -- 172.21.15.46:0/2901142414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c003b70 con 0x7f9ea4109120 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.230+0000 7f9ea0c00640 1 -- 172.21.15.46:0/2901142414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c003d10 con 0x7f9ea4109120 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.230+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea419f7f0 con 0x7f9ea4109120 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.230+0000 7f9ea0c00640 1 -- 172.21.15.46:0/2901142414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c02fc40 con 0x7f9ea4109120 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.230+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea41a7130 con 0x7f9ea4109120 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.231+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea41043d0 con 0x7f9ea4109120 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.233+0000 7f9ea0c00640 1 -- 172.21.15.46:0/2901142414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9e9c004050 con 0x7f9ea4109120 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.235+0000 7f9ea0c00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9e880a2c90 0x7f9e880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.235+0000 7f9ea0c00640 1 -- 172.21.15.46:0/2901142414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9e9c0f06b0 con 0x7f9ea4109120 2024-09-10T07:35:42.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.235+0000 7f9ea3e00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9e880a2c90 0x7f9e880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:42.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.236+0000 7f9ea3e00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9e880a2c90 0x7f9e880a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009fc0 tx=0x7f9e94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:42.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.239+0000 7f9ea0c00640 1 -- 172.21.15.46:0/2901142414 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e9c0bd270 con 0x7f9ea4109120 2024-09-10T07:35:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:41 smithi181 ceph-mon[35397]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-10T07:35:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:41 smithi063 ceph-mon[35020]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-10T07:35:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.425+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ea41a7360 con 0x7f9ea4109120 2024-09-10T07:35:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.427+0000 7f9ea0c00640 1 -- 172.21.15.46:0/2901142414 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24700 (secure 0 0 0) 0x7f9e9c0bcb70 con 0x7f9ea4109120 2024-09-10T07:35:42.427 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:35:42.427 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:35:14.826944+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:35:42.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9e880a2c90 msgr2=0x7f9e880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:42.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9e880a2c90 0x7f9e880a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009fc0 tx=0x7f9e94005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:42.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea4109120 msgr2=0x7f9ea419ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:42.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea4109120 0x7f9ea419ee80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c031a20 tx=0x7f9e9c031a50 comp rx=0 tx=0).stop 2024-09-10T07:35:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 shutdown_connections 2024-09-10T07:35:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9e880a2c90 0x7f9e880a5150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea4109120 0x7f9ea419ee80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ea41049a0 0x7f9ea419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 --2- 172.21.15.46:0/2901142414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ea4103ff0 0x7f9ea419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 >> 172.21.15.46:0/2901142414 conn(0x7f9ea40ffa90 msgr2=0x7f9ea4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 shutdown_connections 2024-09-10T07:35:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:42.430+0000 7f9eaaa00640 1 -- 172.21.15.46:0/2901142414 wait complete. 2024-09-10T07:35:43.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:42 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2901142414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:35:43.300 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:35:43.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:42 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2901142414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:35:43.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:42 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2901142414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:35:43.681 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:43 smithi046 ceph-mon[30705]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T07:35:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:43 smithi046 ceph-mon[30705]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 -- 172.21.15.46:0/2946800777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f841049c0 msgr2=0x7f0f84108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 --2- 172.21.15.46:0/2946800777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f841049c0 0x7f0f84108a10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009900 tx=0x7f0f7002a400 comp rx=0 tx=0).stop 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 -- 172.21.15.46:0/2946800777 shutdown_connections 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 --2- 172.21.15.46:0/2946800777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f84109140 0x7f0f84110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 --2- 172.21.15.46:0/2946800777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f841049c0 0x7f0f84108a10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 --2- 172.21.15.46:0/2946800777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f84104010 0x7f0f841043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 -- 172.21.15.46:0/2946800777 >> 172.21.15.46:0/2946800777 conn(0x7f0f840ffa50 msgr2=0x7f0f84101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 -- 172.21.15.46:0/2946800777 shutdown_connections 2024-09-10T07:35:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.295+0000 7f0f8c600640 1 -- 172.21.15.46:0/2946800777 wait complete. 2024-09-10T07:35:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.296+0000 7f0f8c600640 1 Processor -- start 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.296+0000 7f0f8c600640 1 -- start start 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f8c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f84104010 0x7f0f8419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f8c600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f841049c0 0x7f0f8419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f8c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f84109140 0x7f0f8419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f8c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84114140 con 0x7f0f84109140 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f8c600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84113fc0 con 0x7f0f841049c0 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f8c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841142c0 con 0x7f0f84104010 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f89c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f84104010 0x7f0f8419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f89c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f84104010 0x7f0f8419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38734/0 (socket says 172.21.15.46:38734) 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f89c00640 1 -- 172.21.15.46:0/34418426 learned_addr learned my addr 172.21.15.46:0/34418426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f8a600640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f84109140 0x7f0f8419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.297+0000 7f0f89200640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f841049c0 0x7f0f8419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f89c00640 1 -- 172.21.15.46:0/34418426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f841049c0 msgr2=0x7f0f8419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f89c00640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f841049c0 0x7f0f8419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f89c00640 1 -- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f84109140 msgr2=0x7f0f8419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f89c00640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f84109140 0x7f0f8419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f89c00640 1 -- 172.21.15.46:0/34418426 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8419f660 con 0x7f0f84104010 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f89c00640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f84104010 0x7f0f8419a6b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800e9f0 tx=0x7f0f7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f82000640 1 -- 172.21.15.46:0/34418426 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7800cd80 con 0x7f0f84104010 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8419f950 con 0x7f0f84104010 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f82000640 1 -- 172.21.15.46:0/34418426 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f78010910 con 0x7f0f84104010 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f82000640 1 -- 172.21.15.46:0/34418426 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78018990 con 0x7f0f84104010 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.298+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841a7230 con 0x7f0f84104010 2024-09-10T07:35:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.300+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f841043f0 con 0x7f0f84104010 2024-09-10T07:35:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.303+0000 7f0f82000640 1 -- 172.21.15.46:0/34418426 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f78010ac0 con 0x7f0f84104010 2024-09-10T07:35:44.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.304+0000 7f0f82000640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0f6c0a2b80 0x7f0f6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:44.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.304+0000 7f0f82000640 1 -- 172.21.15.46:0/34418426 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0f780cf7e0 con 0x7f0f84104010 2024-09-10T07:35:44.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.305+0000 7f0f89200640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0f6c0a2b80 0x7f0f6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:44.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.305+0000 7f0f89200640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0f6c0a2b80 0x7f0f6c0a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f84101f80 tx=0x7f0f70005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:44.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.308+0000 7f0f82000640 1 -- 172.21.15.46:0/34418426 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f7809c360 con 0x7f0f84104010 2024-09-10T07:35:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:43 smithi181 ceph-mon[35397]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T07:35:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:43 smithi181 ceph-mon[35397]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T07:35:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:43 smithi063 ceph-mon[35020]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T07:35:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:43 smithi063 ceph-mon[35020]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T07:35:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.533+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f8410c3e0 con 0x7f0f84104010 2024-09-10T07:35:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.534+0000 7f0f82000640 1 -- 172.21.15.46:0/34418426 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11972 (secure 0 0 0) 0x7f0f7809bc60 con 0x7f0f84104010 2024-09-10T07:35:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:35:44.534 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"btime":"2024-09-10T07:34:28:805113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:34:27.875261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:35:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T07:35:44.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.536+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0f6c0a2b80 msgr2=0x7f0f6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:44.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.536+0000 7f0f8c600640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0f6c0a2b80 0x7f0f6c0a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f84101f80 tx=0x7f0f70005a20 comp rx=0 tx=0).stop 2024-09-10T07:35:44.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.536+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f84104010 msgr2=0x7f0f8419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:44.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.536+0000 7f0f8c600640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f84104010 0x7f0f8419a6b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800e9f0 tx=0x7f0f7800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.537+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 shutdown_connections 2024-09-10T07:35:44.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.537+0000 7f0f8c600640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0f6c0a2b80 0x7f0f6c0a5040 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.537+0000 7f0f8c600640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f84109140 0x7f0f8419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.537+0000 7f0f8c600640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f841049c0 0x7f0f8419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.537+0000 7f0f8c600640 1 --2- 172.21.15.46:0/34418426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f84104010 0x7f0f8419a6b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:44.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.537+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 >> 172.21.15.46:0/34418426 conn(0x7f0f840ffa50 msgr2=0x7f0f84100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:44.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.537+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 shutdown_connections 2024-09-10T07:35:44.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:44.537+0000 7f0f8c600640 1 -- 172.21.15.46:0/34418426 wait complete. 2024-09-10T07:35:45.302 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:45 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/34418426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:45.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T07:33:36.040805+0000', 'modified': '2024-09-10T07:34:27.875261+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24535, 'mds_1': 24530, 'mds_2': 24517, 'mds_3': 24499, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6833/620318721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6832', 'nonce': 620318721}, {'type': 'v1', 'addr': '172.21.15.63:6833', 'nonce': 620318721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.181:6833/2451989162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 2451989162}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 2451989162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/4121221095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 4121221095}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 4121221095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6835/1645722263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6834', 'nonce': 1645722263}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 1645722263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/335913193', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 335913193}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 335913193}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6837/4195549127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6836', 'nonce': 4195549127}, {'type': 'v1', 'addr': '172.21.15.63:6837', 'nonce': 4195549127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.181:6837/975679519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 975679519}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 975679519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.181:6839/1085095014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1085095014}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1085095014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.63:6839/1361023409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6838', 'nonce': 1361023409}, {'type': 'v1', 'addr': '172.21.15.63:6839', 'nonce': 1361023409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.46:6841/3146821117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6840', 'nonce': 3146821117}, {'type': 'v1', 'addr': '172.21.15.46:6841', 'nonce': 3146821117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24535, 'qdb_cluster': [24448, 24499, 24517, 24530, 24535]} 2024-09-10T07:35:45.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T07:35:45.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-10T07:35:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:45 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/34418426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:45 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/34418426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:35:45.722 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:46 smithi046 ceph-mon[30705]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T07:35:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 -- 172.21.15.46:0/2562453602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc81079c0 msgr2=0x7f5dc8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 --2- 172.21.15.46:0/2562453602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc81079c0 0x7f5dc8107da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009cd0 tx=0x7f5db802a8e0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 -- 172.21.15.46:0/2562453602 shutdown_connections 2024-09-10T07:35:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 --2- 172.21.15.46:0/2562453602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc8102360 0x7f5dc810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 --2- 172.21.15.46:0/2562453602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc81019c0 0x7f5dc8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 --2- 172.21.15.46:0/2562453602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc81079c0 0x7f5dc8107da0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 -- 172.21.15.46:0/2562453602 >> 172.21.15.46:0/2562453602 conn(0x7f5dc80fd740 msgr2=0x7f5dc80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 -- 172.21.15.46:0/2562453602 shutdown_connections 2024-09-10T07:35:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.322+0000 7f5dc7200640 1 -- 172.21.15.46:0/2562453602 wait complete. 2024-09-10T07:35:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.323+0000 7f5dc7200640 1 Processor -- start 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.323+0000 7f5dc7200640 1 -- start start 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.323+0000 7f5dc7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc81019c0 0x7f5dc810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.323+0000 7f5dc7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc8102360 0x7f5dc810e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dc7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc81079c0 0x7f5dc810b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dc7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc8075870 con 0x7f5dc8102360 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dc7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80756f0 con 0x7f5dc81079c0 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dc7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80759f0 con 0x7f5dc81019c0 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dbd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc8102360 0x7f5dc810e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dbd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc8102360 0x7f5dc810e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38630/0 (socket says 172.21.15.46:38630) 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dbd400640 1 -- 172.21.15.46:0/606057421 learned_addr learned my addr 172.21.15.46:0/606057421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc81019c0 0x7f5dc810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dc6800640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc81079c0 0x7f5dc810b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dbd400640 1 -- 172.21.15.46:0/606057421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc81019c0 msgr2=0x7f5dc810df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dbd400640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc81019c0 0x7f5dc810df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dbd400640 1 -- 172.21.15.46:0/606057421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc81079c0 msgr2=0x7f5dc810b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dbd400640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc81079c0 0x7f5dc810b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dbd400640 1 -- 172.21.15.46:0/606057421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc810b970 con 0x7f5dc8102360 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.324+0000 7f5dc6800640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc81079c0 0x7f5dc810b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.325+0000 7f5dbd400640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc8102360 0x7f5dc810e4b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d770 tx=0x7f5db000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.325+0000 7f5dbf400640 1 -- 172.21.15.46:0/606057421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0017070 con 0x7f5dc8102360 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.325+0000 7f5dbf400640 1 -- 172.21.15.46:0/606057421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0005500 con 0x7f5dc8102360 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.325+0000 7f5dbf400640 1 -- 172.21.15.46:0/606057421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0004e00 con 0x7f5dc8102360 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.325+0000 7f5dc7200640 1 -- 172.21.15.46:0/606057421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc810bc60 con 0x7f5dc8102360 2024-09-10T07:35:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.325+0000 7f5dc7200640 1 -- 172.21.15.46:0/606057421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc81ab420 con 0x7f5dc8102360 2024-09-10T07:35:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.328+0000 7f5dcf488740 1 Processor -- start 2024-09-10T07:35:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.328+0000 7f5dbf400640 1 -- 172.21.15.46:0/606057421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5db0004fa0 con 0x7f5dc8102360 2024-09-10T07:35:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.328+0000 7f5dcf488740 1 -- start start 2024-09-10T07:35:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f891e00 0x560b9f89c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f89c6c0 0x560b9f89eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dcf488740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b9f7f70b0 con 0x560b9f8914e0 2024-09-10T07:35:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dcf488740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560b9f79eab0 con 0x560b9f89c6c0 2024-09-10T07:35:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dcf488740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b9f7f3070 con 0x560b9f891e00 2024-09-10T07:35:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38644/0 (socket says 172.21.15.46:38644) 2024-09-10T07:35:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc6800640 1 -- 172.21.15.46:0/3734129969 learned_addr learned my addr 172.21.15.46:0/3734129969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f891e00 0x560b9f89c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dbd400640 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f89c6c0 0x560b9f89eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc6800640 1 -- 172.21.15.46:0/3734129969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f891e00 msgr2=0x560b9f89c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc6800640 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f891e00 0x560b9f89c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc6800640 1 -- 172.21.15.46:0/3734129969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f89c6c0 msgr2=0x560b9f89eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc6800640 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f89c6c0 0x560b9f89eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dc6800640 1 -- 172.21.15.46:0/3734129969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9f7ab4a0 con 0x560b9f8914e0 2024-09-10T07:35:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.329+0000 7f5dbd400640 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f89c6c0 0x560b9f89eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.330+0000 7f5dc6800640 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8918c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0011440 tx=0x7f5dc0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354aeaf1bde279eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.330+0000 7f5dbf400640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5dac0a2c00 0x7f5dac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.330+0000 7f5dbf400640 1 -- 172.21.15.46:0/606057421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5db00ced40 con 0x7f5dc8102360 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.330+0000 7f5db7000640 1 -- 172.21.15.46:0/3734129969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0004020 con 0x560b9f8914e0 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.330+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5dac0a2c00 0x7f5dac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.330+0000 7f5db7000640 1 -- 172.21.15.46:0/3734129969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc00041c0 con 0x560b9f8914e0 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.330+0000 7f5db7000640 1 -- 172.21.15.46:0/3734129969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000f950 con 0x560b9f8914e0 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dcf488740 1 -- 172.21.15.46:0/3734129969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 msgr2=0x560b9f8918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5dac0a2c00 0x7f5dac0a50c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5db8006010 tx=0x7f5db8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dcf488740 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8918c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0011440 tx=0x7f5dc0011900 comp rx=0 tx=0).stop 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dcf488740 1 -- 172.21.15.46:0/3734129969 shutdown_connections 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dcf488740 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f89c6c0 0x560b9f89eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dcf488740 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f891e00 0x560b9f89c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dcf488740 1 --2- 172.21.15.46:0/3734129969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8918c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dcf488740 1 -- 172.21.15.46:0/3734129969 >> 172.21.15.46:0/3734129969 conn(0x560b9f88da70 msgr2=0x560b9f88de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.331+0000 7f5dcf488740 1 -- 172.21.15.46:0/3734129969 shutdown_connections 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.332+0000 7f5dcf488740 1 -- 172.21.15.46:0/3734129969 wait complete. 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.332+0000 7f5dcf488740 1 -- start start 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.332+0000 7f5dcf488740 1 Processor -- start 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f8914e0 0x560b9f899600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f891e00 0x560b9f899b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 0x560b9f89a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dcf488740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b9f89f330 con 0x560b9f891e00 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dcf488740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560b9f77f920 con 0x560b9f8914e0 2024-09-10T07:35:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dcf488740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b9f89f4b0 con 0x560b9f89c6c0 2024-09-10T07:35:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dcf488740 10 client.0 _ll_register_callbacks cb 0x560b9f7a38a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dc5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f8914e0 0x560b9f899600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dbd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f891e00 0x560b9f899b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dc5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f8914e0 0x560b9f899600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40468/0 (socket says 172.21.15.46:40468) 2024-09-10T07:35:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dc5e00640 1 -- 172.21.15.46:0/118406201 learned_addr learned my addr 172.21.15.46:0/118406201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dc6800640 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 0x560b9f89a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dbd400640 1 -- 172.21.15.46:0/118406201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 msgr2=0x560b9f89a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.333+0000 7f5dbd400640 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 0x560b9f89a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dbd400640 1 -- 172.21.15.46:0/118406201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f8914e0 msgr2=0x560b9f899600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dbd400640 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f8914e0 0x560b9f899600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dbd400640 1 -- 172.21.15.46:0/118406201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9f5682b0 con 0x560b9f891e00 2024-09-10T07:35:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f8914e0 0x560b9f899600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dbd400640 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f891e00 0x560b9f899b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5db00cc430 tx=0x7f5db009adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d8d15681cd17d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dab400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dab400640 1 -- 172.21.15.46:0/118406201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0020070 con 0x560b9f891e00 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dcf488740 1 -- 172.21.15.46:0/118406201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc00041c0 con 0x560b9f891e00 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dab400640 1 -- 172.21.15.46:0/118406201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5db00cc570 con 0x560b9f891e00 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dab400640 1 -- 172.21.15.46:0/118406201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5db009b270 con 0x560b9f891e00 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dcf488740 10 client.14937 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.334+0000 7f5dcf488740 1 -- 172.21.15.46:0/118406201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b9f8ab080 con 0x560b9f891e00 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.335+0000 7f5dab400640 1 -- 172.21.15.46:0/118406201 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc00041c0 con 0x560b9f891e00 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.335+0000 7f5dab400640 1 -- 172.21.15.46:0/118406201 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5db00978e0 con 0x560b9f891e00 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.335+0000 7f5dcf488740 10 client.14937 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.335+0000 7f5dcf488740 10 client.14937 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.335+0000 7f5dcf488740 20 client.14937 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.335+0000 7f5dcf488740 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b9f8ad570 0x560b9f8ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.335+0000 7f5dcf488740 4 client.14937 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.335+0000 7f5dcf488740 1 -- 172.21.15.46:0/118406201 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b9f804620 con 0x560b9f8ad570 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.336+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b9f8ad570 0x560b9f8ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.336+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b9f8ad570 0x560b9f8ae9e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1db8853d03851ac5 server_cookie=521718bb8c567380 in_seq=0 out_seq=0 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.336+0000 7f5dab400640 10 client.14937 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dab400640 1 -- 172.21.15.46:0/118406201 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b9f804620 con 0x560b9f8ad570 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dab400640 10 client.14937 handle_command_reply: tid=0 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 client.14937 shutdown 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 -- 172.21.15.46:0/118406201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f891e00 msgr2=0x560b9f899b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f891e00 0x560b9f899b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5db00cc430 tx=0x7f5db009adb0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 -- 172.21.15.46:0/118406201 shutdown_connections 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b9f8ad570 0x560b9f8ae9e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 0x560b9f89a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f891e00 0x560b9f899b40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 --2- 172.21.15.46:0/118406201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f8914e0 0x560b9f899600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.337+0000 7f5dcf488740 1 -- 172.21.15.46:0/118406201 >> 172.21.15.46:0/118406201 conn(0x560b9f88da70 msgr2=0x560b9f88cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.338+0000 7f5dab400640 0 client.14937 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:35:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.338+0000 7f5dcf488740 1 -- 172.21.15.46:0/118406201 shutdown_connections 2024-09-10T07:35:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.338+0000 7f5dcf488740 1 -- 172.21.15.46:0/118406201 wait complete. 2024-09-10T07:35:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.338+0000 7f5dcf488740 20 client.14937 trim_cache size 0 max 16384 2024-09-10T07:35:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.356+0000 7f5dcf488740 1 Processor -- start 2024-09-10T07:35:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dcf488740 1 -- start start 2024-09-10T07:35:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 0x560b9f88de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 0x560b9f88e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dcf488740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b9f8acd20 con 0x560b9f8914e0 2024-09-10T07:35:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dcf488740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560b9f8ad3f0 con 0x560b9f891e00 2024-09-10T07:35:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dcf488740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b9f8adac0 con 0x560b9f89c6c0 2024-09-10T07:35:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dc5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dc5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38672/0 (socket says 172.21.15.46:38672) 2024-09-10T07:35:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dc5e00640 1 -- 172.21.15.46:0/263846144 learned_addr learned my addr 172.21.15.46:0/263846144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dbd400640 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 0x560b9f88de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.357+0000 7f5dc6800640 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 0x560b9f88e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5dc5e00640 1 -- 172.21.15.46:0/263846144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 msgr2=0x560b9f88e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 0x560b9f88e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5dc5e00640 1 -- 172.21.15.46:0/263846144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 msgr2=0x560b9f88de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 0x560b9f88de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5dc5e00640 1 -- 172.21.15.46:0/263846144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9f897b50 con 0x560b9f8914e0 2024-09-10T07:35:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8ae310 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009cd0 tx=0x7f5db8044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9916b9f0e1e447f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5db6600640 1 -- 172.21.15.46:0/263846144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8004040 con 0x560b9f8914e0 2024-09-10T07:35:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5db6600640 1 -- 172.21.15.46:0/263846144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db8036040 con 0x560b9f8914e0 2024-09-10T07:35:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.358+0000 7f5db6600640 1 -- 172.21.15.46:0/263846144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8053b40 con 0x560b9f8914e0 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 -- 172.21.15.46:0/263846144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 msgr2=0x560b9f8ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8ae310 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009cd0 tx=0x7f5db8044000 comp rx=0 tx=0).stop 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 -- 172.21.15.46:0/263846144 shutdown_connections 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f89c6c0 0x560b9f88e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 0x560b9f88de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 --2- 172.21.15.46:0/263846144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f8914e0 0x560b9f8ae310 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 -- 172.21.15.46:0/263846144 >> 172.21.15.46:0/263846144 conn(0x560b9f88da70 msgr2=0x560b9f8887a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 -- 172.21.15.46:0/263846144 shutdown_connections 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.359+0000 7f5dcf488740 1 -- 172.21.15.46:0/263846144 wait complete. 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 1 -- start start 2024-09-10T07:35:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 1 Processor -- start 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f8914e0 0x560b9f8b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 0x560b9f8b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f89c6c0 0x560b9f915c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b9f8ae860 con 0x560b9f89c6c0 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560b9f8ae190 con 0x560b9f891e00 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b9f8aef30 con 0x560b9f8914e0 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dcf488740 10 client.0 _ll_register_callbacks cb 0x7f5db0097510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.360+0000 7f5dc5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f8914e0 0x560b9f8b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f8914e0 0x560b9f8b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57650/0 (socket says 172.21.15.46:57650) 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc5e00640 1 -- 172.21.15.46:0/2391019475 learned_addr learned my addr 172.21.15.46:0/2391019475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc6800640 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f89c6c0 0x560b9f915c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dbd400640 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 0x560b9f8b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc5e00640 1 -- 172.21.15.46:0/2391019475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 msgr2=0x560b9f8b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 0x560b9f8b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc5e00640 1 -- 172.21.15.46:0/2391019475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f89c6c0 msgr2=0x560b9f915c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f89c6c0 0x560b9f915c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc5e00640 1 -- 172.21.15.46:0/2391019475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9f9161b0 con 0x560b9f8914e0 2024-09-10T07:35:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f8914e0 0x560b9f8b0190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5db802ae00 tx=0x7f5db8031e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3da47ad4fc99864 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.361+0000 7f5dab400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:35:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.362+0000 7f5dab400640 1 -- 172.21.15.46:0/2391019475 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8062070 con 0x560b9f8914e0 2024-09-10T07:35:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.362+0000 7f5dab400640 1 -- 172.21.15.46:0/2391019475 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5db8004d40 con 0x560b9f8914e0 2024-09-10T07:35:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.362+0000 7f5dcf488740 1 -- 172.21.15.46:0/2391019475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db8036040 con 0x560b9f8914e0 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.362+0000 7f5dab400640 1 -- 172.21.15.46:0/2391019475 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5db806e050 con 0x560b9f8914e0 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.362+0000 7f5dcf488740 10 client.24991 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.362+0000 7f5dcf488740 1 -- 172.21.15.46:0/2391019475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b9f917430 con 0x560b9f8914e0 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.362+0000 7f5dab400640 1 -- 172.21.15.46:0/2391019475 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db8036040 con 0x560b9f8914e0 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.362+0000 7f5dab400640 1 -- 172.21.15.46:0/2391019475 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5db805a030 con 0x560b9f8914e0 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dcf488740 10 client.24991 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dcf488740 10 client.24991 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dcf488740 20 client.24991 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dcf488740 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b9f91ba60 0x560b9f91a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dcf488740 4 client.24991 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dcf488740 1 -- 172.21.15.46:0/2391019475 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "status"}) -- 0x560b9f7feb30 con 0x560b9f91ba60 2024-09-10T07:35:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dbd400640 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b9f91ba60 0x560b9f91a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dbd400640 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b9f91ba60 0x560b9f91a770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4713b4c9a136ae40 server_cookie=57b0e90ce066654e in_seq=0 out_seq=0 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.363+0000 7f5dab400640 10 client.24991 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dab400640 1 -- 172.21.15.46:0/2391019475 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560b9f7feb30 con 0x560b9f91ba60 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dab400640 10 client.24991 handle_command_reply: tid=0 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 client.24991 shutdown 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 -- 172.21.15.46:0/2391019475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f8914e0 msgr2=0x560b9f8b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f8914e0 0x560b9f8b0190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5db802ae00 tx=0x7f5db8031e00 comp rx=0 tx=0).stop 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 -- 172.21.15.46:0/2391019475 shutdown_connections 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b9f91ba60 0x560b9f91a770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b9f89c6c0 0x560b9f915c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b9f891e00 0x560b9f8b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 --2- 172.21.15.46:0/2391019475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b9f8914e0 0x560b9f8b0190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dab400640 0 client.24991 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:35:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 -- 172.21.15.46:0/2391019475 >> 172.21.15.46:0/2391019475 conn(0x560b9f88da70 msgr2=0x560b9f88de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 -- 172.21.15.46:0/2391019475 shutdown_connections 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 1 -- 172.21.15.46:0/2391019475 wait complete. 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.364+0000 7f5dcf488740 20 client.24991 trim_cache size 0 max 16384 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 4, 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24448, 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:35:46.368 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:35:46.369 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 80.561414905000007, 2024-09-10T07:35:46.369 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:35:46.369 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:35:46.369 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:35:46.369 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 178.239206115 2024-09-10T07:35:46.369 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:35:46.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 -- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5dac0a2c00 msgr2=0x7f5dac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5dac0a2c00 0x7f5dac0a50c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5db8006010 tx=0x7f5db8009360 comp rx=0 tx=0).stop 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 -- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc8102360 msgr2=0x7f5dc810e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc8102360 0x7f5dc810e4b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d770 tx=0x7f5db000dc30 comp rx=0 tx=0).stop 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 -- 172.21.15.46:0/606057421 shutdown_connections 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5dc81079c0 0x7f5dc810b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5dac0a2c00 0x7f5dac0a50c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc8102360 0x7f5dc810e4b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 --2- 172.21.15.46:0/606057421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc81019c0 0x7f5dc810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 -- 172.21.15.46:0/606057421 >> 172.21.15.46:0/606057421 conn(0x7f5dc80fd740 msgr2=0x7f5dc80fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 -- 172.21.15.46:0/606057421 shutdown_connections 2024-09-10T07:35:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:46.365+0000 7f5dc7200640 1 -- 172.21.15.46:0/606057421 wait complete. 2024-09-10T07:35:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:46 smithi181 ceph-mon[35397]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T07:35:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:46 smithi063 ceph-mon[35020]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T07:35:47.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T07:35:47.654 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:48.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:48 smithi046 ceph-mon[30705]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T07:35:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.288+0000 7f20c0c00640 1 -- 172.21.15.46:0/3789200908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc101df0 msgr2=0x7f20bc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.288+0000 7f20c0c00640 1 --2- 172.21.15.46:0/3789200908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc101df0 0x7f20bc1021d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009900 tx=0x7f20ac02a410 comp rx=0 tx=0).stop 2024-09-10T07:35:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.288+0000 7f20c0c00640 1 -- 172.21.15.46:0/3789200908 shutdown_connections 2024-09-10T07:35:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.288+0000 7f20c0c00640 1 --2- 172.21.15.46:0/3789200908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20bc106f20 0x7f20bc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.288+0000 7f20c0c00640 1 --2- 172.21.15.46:0/3789200908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20bc1027a0 0x7f20bc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.288+0000 7f20c0c00640 1 --2- 172.21.15.46:0/3789200908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc101df0 0x7f20bc1021d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.288+0000 7f20c0c00640 1 -- 172.21.15.46:0/3789200908 >> 172.21.15.46:0/3789200908 conn(0x7f20bc0fd8b0 msgr2=0x7f20bc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.289+0000 7f20c0c00640 1 -- 172.21.15.46:0/3789200908 shutdown_connections 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.289+0000 7f20c0c00640 1 -- 172.21.15.46:0/3789200908 wait complete. 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.290+0000 7f20c0c00640 1 Processor -- start 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.290+0000 7f20c0c00640 1 -- start start 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20c0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20bc101df0 0x7f20bc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20c0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc1027a0 0x7f20bc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20c0c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20bc106f20 0x7f20bc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20c0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc111fd0 con 0x7f20bc1027a0 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20c0c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc111e50 con 0x7f20bc106f20 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20c0c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc112150 con 0x7f20bc101df0 2024-09-10T07:35:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc1027a0 0x7f20bc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc1027a0 0x7f20bc1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38704/0 (socket says 172.21.15.46:38704) 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9400640 1 -- 172.21.15.46:0/495139697 learned_addr learned my addr 172.21.15.46:0/495139697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9e00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20bc101df0 0x7f20bc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20ba800640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20bc106f20 0x7f20bc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9400640 1 -- 172.21.15.46:0/495139697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20bc101df0 msgr2=0x7f20bc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9400640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20bc101df0 0x7f20bc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9400640 1 -- 172.21.15.46:0/495139697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20bc106f20 msgr2=0x7f20bc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9400640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20bc106f20 0x7f20bc19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.291+0000 7f20b9400640 1 -- 172.21.15.46:0/495139697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20bc19d420 con 0x7f20bc1027a0 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.292+0000 7f20b9400640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc1027a0 0x7f20bc1989b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e9f0 tx=0x7f20a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.292+0000 7f20b2a00640 1 -- 172.21.15.46:0/495139697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a800cd80 con 0x7f20bc1027a0 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.292+0000 7f20b2a00640 1 -- 172.21.15.46:0/495139697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a8010910 con 0x7f20bc1027a0 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.292+0000 7f20c0c00640 1 -- 172.21.15.46:0/495139697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20bc19d710 con 0x7f20bc1027a0 2024-09-10T07:35:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.292+0000 7f20b2a00640 1 -- 172.21.15.46:0/495139697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8018990 con 0x7f20bc1027a0 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.292+0000 7f20c0c00640 1 -- 172.21.15.46:0/495139697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20bc1a4ff0 con 0x7f20bc1027a0 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.295+0000 7f20c3681740 1 Processor -- start 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.295+0000 7f20b2a00640 1 -- 172.21.15.46:0/495139697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f20a8010ac0 con 0x7f20bc1027a0 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.295+0000 7f20c3681740 1 -- start start 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20c3681740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c099601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20c3681740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c09960b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20c3681740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09975fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20c3681740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c098c60b0 con 0x558c099618b0 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20c3681740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558c0986dab0 con 0x558c09960710 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20c3681740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558c098c2070 con 0x558c09973bb0 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20ba800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c099601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20ba800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c099601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38720/0 (socket says 172.21.15.46:38720) 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20ba800640 1 -- 172.21.15.46:0/2536067846 learned_addr learned my addr 172.21.15.46:0/2536067846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20b9400640 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09975fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20b9e00640 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c09960b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20ba800640 1 -- 172.21.15.46:0/2536067846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 msgr2=0x558c09975fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20ba800640 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09975fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20ba800640 1 -- 172.21.15.46:0/2536067846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 msgr2=0x558c09960b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.296+0000 7f20ba800640 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c09960b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20ba800640 1 -- 172.21.15.46:0/2536067846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0987a4a0 con 0x558c099618b0 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20b9e00640 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c09960b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20b2a00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20a00a2c00 0x7f20a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20ba800640 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c099601d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009f50 tx=0x7f20b40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d34ef6727f06d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20b2a00640 1 -- 172.21.15.46:0/495139697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f20a80cf690 con 0x7f20bc1027a0 2024-09-10T07:35:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20b9e00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20a00a2c00 0x7f20a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20a7000640 1 -- 172.21.15.46:0/2536067846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4004d20 con 0x558c099618b0 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20a7000640 1 -- 172.21.15.46:0/2536067846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b4004050 con 0x558c099618b0 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.297+0000 7f20a7000640 1 -- 172.21.15.46:0/2536067846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4005260 con 0x558c099618b0 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 -- 172.21.15.46:0/2536067846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 msgr2=0x558c099601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c099601d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009f50 tx=0x7f20b40118d0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20b9e00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20a00a2c00 0x7f20a00a50c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20ac000c00 tx=0x7f20ac005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 -- 172.21.15.46:0/2536067846 shutdown_connections 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09975fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c09960b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 --2- 172.21.15.46:0/2536067846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c099601d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 -- 172.21.15.46:0/2536067846 >> 172.21.15.46:0/2536067846 conn(0x558c0995c8c0 msgr2=0x558c0995cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 -- 172.21.15.46:0/2536067846 shutdown_connections 2024-09-10T07:35:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.298+0000 7f20c3681740 1 -- 172.21.15.46:0/2536067846 wait complete. 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 1 -- start start 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 1 Processor -- start 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c09963fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c099618b0 0x558c09964500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09964a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c09976980 con 0x558c09960710 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558c0984e920 con 0x558c099618b0 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558c09976b00 con 0x558c09973bb0 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20c3681740 10 client.0 _ll_register_callbacks cb 0x558c098728a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c09963fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c09963fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38734/0 (socket says 172.21.15.46:38734) 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.299+0000 7f20b9e00640 1 -- 172.21.15.46:0/1183723222 learned_addr learned my addr 172.21.15.46:0/1183723222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20b9400640 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c099618b0 0x558c09964500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20ba800640 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09964a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20b9e00640 1 -- 172.21.15.46:0/1183723222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 msgr2=0x558c09964a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20b9e00640 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09964a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20b9e00640 1 -- 172.21.15.46:0/1183723222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c099618b0 msgr2=0x558c09964500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20b9e00640 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c099618b0 0x558c09964500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20b9e00640 1 -- 172.21.15.46:0/1183723222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c096372b0 con 0x558c09960710 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20b9e00640 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c09963fc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009900 tx=0x7f20ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181cafa3262bca26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f209b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f209b400640 1 -- 172.21.15.46:0/1183723222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac04e070 con 0x558c09960710 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f209b400640 1 -- 172.21.15.46:0/1183723222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f20ac0048c0 con 0x558c09960710 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.300+0000 7f20c3681740 1 -- 172.21.15.46:0/1183723222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b4004050 con 0x558c09960710 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f209b400640 1 -- 172.21.15.46:0/1183723222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f20ac05a050 con 0x558c09960710 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f20c3681740 10 client.14955 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f20c3681740 1 -- 172.21.15.46:0/1183723222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c09982720 con 0x558c09960710 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f209b400640 1 -- 172.21.15.46:0/1183723222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b4004050 con 0x558c09960710 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f209b400640 1 -- 172.21.15.46:0/1183723222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f20ac0035f0 con 0x558c09960710 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f20c3681740 10 client.14955 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f20c3681740 10 client.14955 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f20c3681740 20 client.14955 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f20c3681740 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558c09984d10 0x558c09986180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f20c3681740 4 client.14955 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.301+0000 7f20c3681740 1 -- 172.21.15.46:0/1183723222 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c098d3620 con 0x558c09984d10 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.302+0000 7f20b9400640 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558c09984d10 0x558c09986180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.302+0000 7f20b9400640 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558c09984d10 0x558c09986180 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45e4b520bccd4b2f server_cookie=8df745681200a53a in_seq=0 out_seq=0 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.302+0000 7f209b400640 10 client.14955 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.303+0000 7f209b400640 1 -- 172.21.15.46:0/1183723222 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c098d3620 con 0x558c09984d10 2024-09-10T07:35:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.303+0000 7f209b400640 10 client.14955 handle_command_reply: tid=0 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.303+0000 7f20c3681740 1 client.14955 shutdown 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 -- 172.21.15.46:0/1183723222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 msgr2=0x558c09963fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c09963fc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009900 tx=0x7f20ac009260 comp rx=0 tx=0).stop 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 -- 172.21.15.46:0/1183723222 shutdown_connections 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558c09984d10 0x558c09986180 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09964a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c099618b0 0x558c09964500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 --2- 172.21.15.46:0/1183723222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c09963fc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 -- 172.21.15.46:0/1183723222 >> 172.21.15.46:0/1183723222 conn(0x558c0995c8c0 msgr2=0x558c099625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f209b400640 0 client.14955 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 -- 172.21.15.46:0/1183723222 shutdown_connections 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 1 -- 172.21.15.46:0/1183723222 wait complete. 2024-09-10T07:35:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.304+0000 7f20c3681740 20 client.14955 trim_cache size 0 max 16384 2024-09-10T07:35:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20c3681740 1 Processor -- start 2024-09-10T07:35:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20c3681740 1 -- start start 2024-09-10T07:35:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20c3681740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c099600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20c3681740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c099618b0 0x558c09964530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20c3681740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09973bb0 0x558c09964a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20c3681740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c099844c0 con 0x558c09960710 2024-09-10T07:35:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20c3681740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558c09984b90 con 0x558c09973bb0 2024-09-10T07:35:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20c3681740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558c09985260 con 0x558c099618b0 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c099600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c099600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38748/0 (socket says 172.21.15.46:38748) 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20b9e00640 1 -- 172.21.15.46:0/3755100534 learned_addr learned my addr 172.21.15.46:0/3755100534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.323+0000 7f20ba800640 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09973bb0 0x558c09964a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20b9400640 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c099618b0 0x558c09964530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20ba800640 1 -- 172.21.15.46:0/3755100534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c099618b0 msgr2=0x558c09964530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20ba800640 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c099618b0 0x558c09964530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20ba800640 1 -- 172.21.15.46:0/3755100534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 msgr2=0x558c099600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20ba800640 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c099600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20ba800640 1 -- 172.21.15.46:0/3755100534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c09973150 con 0x558c09973bb0 2024-09-10T07:35:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20b9400640 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c099618b0 0x558c09964530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20ba800640 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09973bb0 0x558c09964a70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f20b401b040 tx=0x7f20b4003be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537659532feca733 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.324+0000 7f20a6600640 1 -- 172.21.15.46:0/3755100534 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4009bf0 con 0x558c09973bb0 2024-09-10T07:35:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20a6600640 1 -- 172.21.15.46:0/3755100534 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b4004df0 con 0x558c09973bb0 2024-09-10T07:35:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20a6600640 1 -- 172.21.15.46:0/3755100534 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b40052d0 con 0x558c09973bb0 2024-09-10T07:35:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 -- 172.21.15.46:0/3755100534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09973bb0 msgr2=0x558c09964a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09973bb0 0x558c09964a70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f20b401b040 tx=0x7f20b4003be0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 -- 172.21.15.46:0/3755100534 shutdown_connections 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09973bb0 0x558c09964a70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c099618b0 0x558c09964530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 --2- 172.21.15.46:0/3755100534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c09960710 0x558c099600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 -- 172.21.15.46:0/3755100534 >> 172.21.15.46:0/3755100534 conn(0x558c0995c8c0 msgr2=0x558c09957640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 -- 172.21.15.46:0/3755100534 shutdown_connections 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.325+0000 7f20c3681740 1 -- 172.21.15.46:0/3755100534 wait complete. 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.326+0000 7f20c3681740 1 -- start start 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.326+0000 7f20c3681740 1 Processor -- start 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.326+0000 7f20c3681740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c099880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.326+0000 7f20c3681740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c09988610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.326+0000 7f20c3681740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09988b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.326+0000 7f20c3681740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c09986000 con 0x558c099618b0 2024-09-10T07:35:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.326+0000 7f20c3681740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558c09985930 con 0x558c09960710 2024-09-10T07:35:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.326+0000 7f20c3681740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558c099866d0 con 0x558c09973bb0 2024-09-10T07:35:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20c3681740 10 client.0 _ll_register_callbacks cb 0x7f20ac0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c09988610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c09988610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38758/0 (socket says 172.21.15.46:38758) 2024-09-10T07:35:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9400640 1 -- 172.21.15.46:0/1194329008 learned_addr learned my addr 172.21.15.46:0/1194329008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9e00640 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c099880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20ba800640 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09988b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9e00640 1 -- 172.21.15.46:0/1194329008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 msgr2=0x558c09988b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9e00640 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09988b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9e00640 1 -- 172.21.15.46:0/1194329008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 msgr2=0x558c09988610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9e00640 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c09988610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9e00640 1 -- 172.21.15.46:0/1194329008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0998c8c0 con 0x558c09960710 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.327+0000 7f20b9400640 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c09988610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.328+0000 7f20b9e00640 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c099880d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20ac02a930 tx=0x7f20ac0058c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea8e648caaa4d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.328+0000 7f209b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:35:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.328+0000 7f209b400640 1 -- 172.21.15.46:0/1194329008 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac04e040 con 0x558c09960710 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.328+0000 7f20c3681740 1 -- 172.21.15.46:0/1194329008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b4004df0 con 0x558c09960710 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.328+0000 7f209b400640 1 -- 172.21.15.46:0/1194329008 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f20ac003e30 con 0x558c09960710 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.328+0000 7f209b400640 1 -- 172.21.15.46:0/1194329008 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f20ac06c050 con 0x558c09960710 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.328+0000 7f20c3681740 10 client.24998 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.328+0000 7f20c3681740 1 -- 172.21.15.46:0/1194329008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c099ef630 con 0x558c09960710 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f209b400640 1 -- 172.21.15.46:0/1194329008 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b4004df0 con 0x558c09960710 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f209b400640 1 -- 172.21.15.46:0/1194329008 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f20ac032890 con 0x558c09960710 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f20c3681740 10 client.24998 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f20c3681740 10 client.24998 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f20c3681740 20 client.24998 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f20c3681740 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558c099f4210 0x558c099f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f20c3681740 4 client.24998 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:35:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f20c3681740 1 -- 172.21.15.46:0/1194329008 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "status"}) -- 0x558c098cdb30 con 0x558c099f4210 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.329+0000 7f20b9400640 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558c099f4210 0x558c099f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20b9400640 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558c099f4210 0x558c099f2f20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f19929083648be44 server_cookie=7d671a2582308837 in_seq=0 out_seq=0 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f209b400640 10 client.24998 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f209b400640 1 -- 172.21.15.46:0/1194329008 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558c098cdb30 con 0x558c099f4210 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f209b400640 10 client.24998 handle_command_reply: tid=0 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 client.24998 shutdown 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 -- 172.21.15.46:0/1194329008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 msgr2=0x558c099880d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c099880d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20ac02a930 tx=0x7f20ac0058c0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 -- 172.21.15.46:0/1194329008 shutdown_connections 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558c099f4210 0x558c099f2f20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558c09973bb0 0x558c09988b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c099618b0 0x558c09988610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 --2- 172.21.15.46:0/1194329008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558c09960710 0x558c099880d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f20c3681740 1 -- 172.21.15.46:0/1194329008 >> 172.21.15.46:0/1194329008 conn(0x558c0995c8c0 msgr2=0x558c0995b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:48.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.330+0000 7f209b400640 0 client.24998 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:35:48.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.331+0000 7f20c3681740 1 -- 172.21.15.46:0/1194329008 shutdown_connections 2024-09-10T07:35:48.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.331+0000 7f20c3681740 1 -- 172.21.15.46:0/1194329008 wait complete. 2024-09-10T07:35:48.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.331+0000 7f20c3681740 20 client.24998 trim_cache size 0 max 16384 2024-09-10T07:35:48.334 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:35:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:35:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 3, 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24499, 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 83.254332302999998, 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 161.13854127299999 2024-09-10T07:35:48.335 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.331+0000 7f20c0c00640 1 -- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20a00a2c00 msgr2=0x7f20a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.331+0000 7f20c0c00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20a00a2c00 0x7f20a00a50c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20ac000c00 tx=0x7f20ac005b50 comp rx=0 tx=0).stop 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.331+0000 7f20c0c00640 1 -- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc1027a0 msgr2=0x7f20bc1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.331+0000 7f20c0c00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc1027a0 0x7f20bc1989b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e9f0 tx=0x7f20a800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.332+0000 7f20c0c00640 1 -- 172.21.15.46:0/495139697 shutdown_connections 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.332+0000 7f20c0c00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20bc106f20 0x7f20bc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.332+0000 7f20c0c00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f20a00a2c00 0x7f20a00a50c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.332+0000 7f20c0c00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20bc1027a0 0x7f20bc1989b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.332+0000 7f20c0c00640 1 --2- 172.21.15.46:0/495139697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20bc101df0 0x7f20bc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:48.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.332+0000 7f20c0c00640 1 -- 172.21.15.46:0/495139697 >> 172.21.15.46:0/495139697 conn(0x7f20bc0fd8b0 msgr2=0x7f20bc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:48.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.332+0000 7f20c0c00640 1 -- 172.21.15.46:0/495139697 shutdown_connections 2024-09-10T07:35:48.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:48.332+0000 7f20c0c00640 1 -- 172.21.15.46:0/495139697 wait complete. 2024-09-10T07:35:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:48 smithi181 ceph-mon[35397]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T07:35:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:48 smithi063 ceph-mon[35020]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T07:35:49.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T07:35:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.053+0000 7f4f8a000640 1 -- 172.21.15.46:0/3566015167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f84103ff0 msgr2=0x7f4f841043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.053+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3566015167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f84103ff0 0x7f4f841043d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f7402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.054+0000 7f4f8a000640 1 -- 172.21.15.46:0/3566015167 shutdown_connections 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.054+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3566015167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f84109120 0x7f4f84110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.054+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3566015167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f841049a0 0x7f4f841089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.054+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3566015167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f84103ff0 0x7f4f841043d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.054+0000 7f4f8a000640 1 -- 172.21.15.46:0/3566015167 >> 172.21.15.46:0/3566015167 conn(0x7f4f840ffa90 msgr2=0x7f4f84101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.054+0000 7f4f8a000640 1 -- 172.21.15.46:0/3566015167 shutdown_connections 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.054+0000 7f4f8a000640 1 -- 172.21.15.46:0/3566015167 wait complete. 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.055+0000 7f4f8a000640 1 Processor -- start 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.055+0000 7f4f8a000640 1 -- start start 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.055+0000 7f4f8a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f84103ff0 0x7f4f8419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.055+0000 7f4f8a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f841049a0 0x7f4f8419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.055+0000 7f4f8a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f84109120 0x7f4f8419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.055+0000 7f4f8a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84114170 con 0x7f4f841049a0 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.055+0000 7f4f8a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84113ff0 con 0x7f4f84103ff0 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.055+0000 7f4f8a000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841142f0 con 0x7f4f84109120 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f84103ff0 0x7f4f8419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f841049a0 0x7f4f8419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f84103ff0 0x7f4f8419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40568/0 (socket says 172.21.15.46:40568) 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 -- 172.21.15.46:0/3510997711 learned_addr learned my addr 172.21.15.46:0/3510997711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83e00640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f84109120 0x7f4f8419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 -- 172.21.15.46:0/3510997711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f84109120 msgr2=0x7f4f8419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f84109120 0x7f4f8419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 -- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f841049a0 msgr2=0x7f4f8419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f841049a0 0x7f4f8419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 -- 172.21.15.46:0/3510997711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8419f5b0 con 0x7f4f84103ff0 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.056+0000 7f4f83400640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f84103ff0 0x7f4f8419a600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f74004770 tx=0x7f4f74003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.057+0000 7f4f80c00640 1 -- 172.21.15.46:0/3510997711 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7403f070 con 0x7f4f84103ff0 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.057+0000 7f4f80c00640 1 -- 172.21.15.46:0/3510997711 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f74031ca0 con 0x7f4f84103ff0 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.057+0000 7f4f80c00640 1 -- 172.21.15.46:0/3510997711 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74035030 con 0x7f4f84103ff0 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.057+0000 7f4f8a000640 1 -- 172.21.15.46:0/3510997711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8419f840 con 0x7f4f84103ff0 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.057+0000 7f4f8a000640 1 -- 172.21.15.46:0/3510997711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8419fcd0 con 0x7f4f84103ff0 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.060+0000 7f4f8cbd2740 1 Processor -- start 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.060+0000 7f4f80c00640 1 -- 172.21.15.46:0/3510997711 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4f74042070 con 0x7f4f84103ff0 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.060+0000 7f4f8cbd2740 1 -- start start 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.060+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf369710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.060+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf369c50 0x5643bf373fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.060+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf374510 0x5643bf376900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5643bf2cf0b0 con 0x5643bf369330 2024-09-10T07:35:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5643bf276ab0 con 0x5643bf374510 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5643bf2cb070 con 0x5643bf369c50 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf369710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf369710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38788/0 (socket says 172.21.15.46:38788) 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83e00640 1 -- 172.21.15.46:0/2822103873 learned_addr learned my addr 172.21.15.46:0/2822103873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f7be00640 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf374510 0x5643bf376900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83400640 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf369c50 0x5643bf373fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83e00640 1 -- 172.21.15.46:0/2822103873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf369c50 msgr2=0x5643bf373fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83e00640 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf369c50 0x5643bf373fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83e00640 1 -- 172.21.15.46:0/2822103873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf374510 msgr2=0x5643bf376900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83e00640 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf374510 0x5643bf376900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.061+0000 7f4f83e00640 1 -- 172.21.15.46:0/2822103873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643bf2834a0 con 0x5643bf369330 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.062+0000 7f4f83e00640 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf369710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009f70 tx=0x7f4f7c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3423188b07df005 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.062+0000 7f4f80c00640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4f680a2c00 0x7f4f680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.062+0000 7f4f73e00640 1 -- 172.21.15.46:0/2822103873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c016070 con 0x5643bf369330 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.062+0000 7f4f80c00640 1 -- 172.21.15.46:0/3510997711 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4f740ef690 con 0x7f4f84103ff0 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.062+0000 7f4f73e00640 1 -- 172.21.15.46:0/2822103873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f7c004d20 con 0x5643bf369330 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.062+0000 7f4f73e00640 1 -- 172.21.15.46:0/2822103873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c012950 con 0x5643bf369330 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.062+0000 7f4f83400640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4f680a2c00 0x7f4f680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2822103873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 msgr2=0x5643bf369710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf369710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009f70 tx=0x7f4f7c010800 comp rx=0 tx=0).stop 2024-09-10T07:35:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f83400640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4f680a2c00 0x7f4f680a50c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f740b8020 tx=0x7f4f74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2822103873 shutdown_connections 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf374510 0x5643bf376900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf369c50 0x5643bf373fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2822103873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf369710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2822103873 >> 172.21.15.46:0/2822103873 conn(0x5643bf3658c0 msgr2=0x5643bf365ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2822103873 shutdown_connections 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2822103873 wait complete. 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.063+0000 7f4f8cbd2740 1 -- start start 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f8cbd2740 1 Processor -- start 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf3715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369c50 0x5643bf371ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf372020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5643bf377060 con 0x5643bf369330 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5643bf257920 con 0x5643bf369c50 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5643bf3771e0 con 0x5643bf374510 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f8cbd2740 10 client.0 _ll_register_callbacks cb 0x5643bf27b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf3715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf3715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38802/0 (socket says 172.21.15.46:38802) 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f83e00640 1 -- 172.21.15.46:0/4111051622 learned_addr learned my addr 172.21.15.46:0/4111051622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.064+0000 7f4f7be00640 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369c50 0x5643bf371ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f83e00640 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf372020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f83e00640 1 -- 172.21.15.46:0/4111051622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 msgr2=0x5643bf372020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f83e00640 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf372020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f83e00640 1 -- 172.21.15.46:0/4111051622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369c50 msgr2=0x5643bf371ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f83e00640 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369c50 0x5643bf371ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f83e00640 1 -- 172.21.15.46:0/4111051622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643bf0402b0 con 0x5643bf369330 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f83e00640 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf3715a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c006010 tx=0x7f4f7c012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18736f192338849b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f70c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f70c00640 1 -- 172.21.15.46:0/4111051622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c016070 con 0x5643bf369330 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f70c00640 1 -- 172.21.15.46:0/4111051622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4f7c012cf0 con 0x5643bf369330 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.065+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/4111051622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f7c004d20 con 0x5643bf369330 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f70c00640 1 -- 172.21.15.46:0/4111051622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f7c004ed0 con 0x5643bf369330 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f8cbd2740 10 client.14973 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/4111051622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643bf382e40 con 0x5643bf369330 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f70c00640 1 -- 172.21.15.46:0/4111051622 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f7c004d20 con 0x5643bf369330 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f70c00640 1 -- 172.21.15.46:0/4111051622 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4f7c00f040 con 0x5643bf369330 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f8cbd2740 10 client.14973 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f8cbd2740 10 client.14973 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T07:35:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f8cbd2740 20 client.14973 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5643bf3850c0 0x5643bf386530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f8cbd2740 4 client.14973 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.066+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/4111051622 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643bf2dc620 con 0x5643bf3850c0 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.067+0000 7f4f7be00640 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5643bf3850c0 0x5643bf386530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.067+0000 7f4f7be00640 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5643bf3850c0 0x5643bf386530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88e12d1a4b2daf76 server_cookie=233e913a34675037 in_seq=0 out_seq=0 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.067+0000 7f4f70c00640 10 client.14973 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.068+0000 7f4f70c00640 1 -- 172.21.15.46:0/4111051622 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643bf2dc620 con 0x5643bf3850c0 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.068+0000 7f4f70c00640 10 client.14973 handle_command_reply: tid=0 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.068+0000 7f4f8cbd2740 1 client.14973 shutdown 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/4111051622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 msgr2=0x5643bf3715a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf3715a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c006010 tx=0x7f4f7c012620 comp rx=0 tx=0).stop 2024-09-10T07:35:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/4111051622 shutdown_connections 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5643bf3850c0 0x5643bf386530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf372020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369c50 0x5643bf371ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/4111051622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369330 0x5643bf3715a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f70c00640 0 client.14973 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/4111051622 >> 172.21.15.46:0/4111051622 conn(0x5643bf3658c0 msgr2=0x5643bf36b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/4111051622 shutdown_connections 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/4111051622 wait complete. 2024-09-10T07:35:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.069+0000 7f4f8cbd2740 20 client.14973 trim_cache size 0 max 16384 2024-09-10T07:35:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.088+0000 7f4f8cbd2740 1 Processor -- start 2024-09-10T07:35:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.088+0000 7f4f8cbd2740 1 -- start start 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.088+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf3606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.088+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf371e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.088+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf36e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.088+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5643bf384870 con 0x5643bf369c50 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.088+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5643bf384f40 con 0x5643bf369330 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.088+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5643bf385610 con 0x5643bf374510 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf371e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf3606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf3606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40594/0 (socket says 172.21.15.46:40594) 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f83e00640 1 -- 172.21.15.46:0/766396915 learned_addr learned my addr 172.21.15.46:0/766396915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf371e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38810/0 (socket says 172.21.15.46:38810) 2024-09-10T07:35:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f83e00640 1 --2- 172.21.15.46:0/766396915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf36e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f7be00640 1 -- 172.21.15.46:0/766396915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 msgr2=0x5643bf36e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f7be00640 1 --2- 172.21.15.46:0/766396915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf36e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f7be00640 1 -- 172.21.15.46:0/766396915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 msgr2=0x5643bf3606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f7be00640 1 --2- 172.21.15.46:0/766396915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf3606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f7be00640 1 -- 172.21.15.46:0/766396915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643bf373f10 con 0x5643bf369c50 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.089+0000 7f4f7be00640 1 --2- 172.21.15.46:0/766396915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf371e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5643bf386d90 tx=0x7f4f6c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5c4ecaef041a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f73400640 1 -- 172.21.15.46:0/766396915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c022020 con 0x5643bf369c50 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f73400640 1 -- 172.21.15.46:0/766396915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f6c003d60 con 0x5643bf369c50 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f73400640 1 -- 172.21.15.46:0/766396915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c01e840 con 0x5643bf369c50 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/766396915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 msgr2=0x5643bf371e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/766396915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf371e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5643bf386d90 tx=0x7f4f6c005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/766396915 shutdown_connections 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/766396915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf36e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/766396915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf371e70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/766396915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf3606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.090+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/766396915 >> 172.21.15.46:0/766396915 conn(0x5643bf3658c0 msgr2=0x5643bf365ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.091+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/766396915 shutdown_connections 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.091+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/766396915 wait complete. 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.091+0000 7f4f8cbd2740 1 -- start start 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.091+0000 7f4f8cbd2740 1 Processor -- start 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf38e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf38eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f8cbd2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf389740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5643bf3863b0 con 0x5643bf369c50 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5643bf385ce0 con 0x5643bf369330 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f8cbd2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5643bf386a80 con 0x5643bf374510 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f8cbd2740 10 client.0 _ll_register_callbacks cb 0x7f4f7c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf38eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf389740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf38eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38824/0 (socket says 172.21.15.46:38824) 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f7be00640 1 -- 172.21.15.46:0/2081245912 learned_addr learned my addr 172.21.15.46:0/2081245912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.092+0000 7f4f83e00640 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf38e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f7be00640 1 -- 172.21.15.46:0/2081245912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 msgr2=0x5643bf389740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f7be00640 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf389740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f7be00640 1 -- 172.21.15.46:0/2081245912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 msgr2=0x5643bf38e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f7be00640 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf38e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f7be00640 1 -- 172.21.15.46:0/2081245912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643bf389f50 con 0x5643bf369c50 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f83e00640 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf38e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f7be00640 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf38eeb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0057c0 tx=0x7f4f6c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c546488c013390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f70c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:35:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f70c00640 1 -- 172.21.15.46:0/2081245912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c01ea90 con 0x5643bf369c50 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f70c00640 1 -- 172.21.15.46:0/2081245912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4f6c01ec30 con 0x5643bf369c50 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.093+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2081245912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f6c003d60 con 0x5643bf369c50 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f70c00640 1 -- 172.21.15.46:0/2081245912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f6c03b050 con 0x5643bf369c50 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f8cbd2740 10 client.14979 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2081245912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643bf3f01e0 con 0x5643bf369c50 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f70c00640 1 -- 172.21.15.46:0/2081245912 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f6c003d60 con 0x5643bf369c50 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f70c00640 1 -- 172.21.15.46:0/2081245912 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4f6c030020 con 0x5643bf369c50 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f8cbd2740 10 client.14979 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f8cbd2740 10 client.14979 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f8cbd2740 20 client.14979 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5643bf3f4b50 0x5643bf3f3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f8cbd2740 4 client.14979 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:35:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.094+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2081245912 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "status"}) -- 0x5643bf2d6b30 con 0x5643bf3f4b50 2024-09-10T07:35:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.095+0000 7f4f83e00640 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5643bf3f4b50 0x5643bf3f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.095+0000 7f4f83e00640 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5643bf3f4b50 0x5643bf3f3860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bda93c3fecc2fa3c server_cookie=6aa8da953cab23b8 in_seq=0 out_seq=0 2024-09-10T07:35:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.095+0000 7f4f70c00640 10 client.14979 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.095+0000 7f4f70c00640 1 -- 172.21.15.46:0/2081245912 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5643bf2d6b30 con 0x5643bf3f4b50 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.095+0000 7f4f70c00640 10 client.14979 handle_command_reply: tid=0 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.095+0000 7f4f8cbd2740 1 client.14979 shutdown 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2081245912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 msgr2=0x5643bf38eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf38eeb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0057c0 tx=0x7f4f6c009b80 comp rx=0 tx=0).stop 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2081245912 shutdown_connections 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5643bf3f4b50 0x5643bf3f3860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643bf374510 0x5643bf389740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643bf369c50 0x5643bf38eeb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 --2- 172.21.15.46:0/2081245912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643bf369330 0x5643bf38e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2081245912 >> 172.21.15.46:0/2081245912 conn(0x5643bf3658c0 msgr2=0x5643bf37f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f70c00640 0 client.14979 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:35:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2081245912 shutdown_connections 2024-09-10T07:35:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 1 -- 172.21.15.46:0/2081245912 wait complete. 2024-09-10T07:35:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.096+0000 7f4f8cbd2740 20 client.14979 trim_cache size 0 max 16384 2024-09-10T07:35:50.102 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24517, 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 86.027046647000006, 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 157.61261560700001 2024-09-10T07:35:50.103 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 -- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4f680a2c00 msgr2=0x7f4f680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4f680a2c00 0x7f4f680a50c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f740b8020 tx=0x7f4f74009260 comp rx=0 tx=0).stop 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 -- 172.21.15.46:0/3510997711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f84103ff0 msgr2=0x7f4f8419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f84103ff0 0x7f4f8419a600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f74004770 tx=0x7f4f74003c10 comp rx=0 tx=0).stop 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 -- 172.21.15.46:0/3510997711 shutdown_connections 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f84109120 0x7f4f8419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4f680a2c00 0x7f4f680a50c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f841049a0 0x7f4f8419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 --2- 172.21.15.46:0/3510997711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f84103ff0 0x7f4f8419a600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 -- 172.21.15.46:0/3510997711 >> 172.21.15.46:0/3510997711 conn(0x7f4f840ffa90 msgr2=0x7f4f84100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 -- 172.21.15.46:0/3510997711 shutdown_connections 2024-09-10T07:35:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:50.097+0000 7f4f8a000640 1 -- 172.21.15.46:0/3510997711 wait complete. 2024-09-10T07:35:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:50 smithi181 ceph-mon[35397]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:50.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:50 smithi063 ceph-mon[35020]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:50.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:50 smithi046 ceph-mon[30705]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:50.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T07:35:51.250 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 -- 172.21.15.46:0/1896686066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8109140 msgr2=0x7fdda8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 --2- 172.21.15.46:0/1896686066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8109140 0x7fdda8110cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009ca0 tx=0x7fdda002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 -- 172.21.15.46:0/1896686066 shutdown_connections 2024-09-10T07:35:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 --2- 172.21.15.46:0/1896686066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8109140 0x7fdda8110cc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 --2- 172.21.15.46:0/1896686066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda81049c0 0x7fdda8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 --2- 172.21.15.46:0/1896686066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda8104010 0x7fdda81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 -- 172.21.15.46:0/1896686066 >> 172.21.15.46:0/1896686066 conn(0x7fdda80ffa50 msgr2=0x7fdda8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 -- 172.21.15.46:0/1896686066 shutdown_connections 2024-09-10T07:35:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.849+0000 7fddb0600640 1 -- 172.21.15.46:0/1896686066 wait complete. 2024-09-10T07:35:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.850+0000 7fddb0600640 1 Processor -- start 2024-09-10T07:35:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.850+0000 7fddb0600640 1 -- start start 2024-09-10T07:35:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.850+0000 7fddb0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8104010 0x7fdda819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.850+0000 7fddb0600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda81049c0 0x7fdda819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.850+0000 7fddb0600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda8109140 0x7fdda819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.850+0000 7fddb0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81141c0 con 0x7fdda8104010 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.850+0000 7fddb0600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8114040 con 0x7fdda81049c0 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.850+0000 7fddb0600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8114340 con 0x7fdda8109140 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddadc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8104010 0x7fdda819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddadc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8104010 0x7fdda819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38828/0 (socket says 172.21.15.46:38828) 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddadc00640 1 -- 172.21.15.46:0/2581945698 learned_addr learned my addr 172.21.15.46:0/2581945698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddad200640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda81049c0 0x7fdda819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddae600640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda8109140 0x7fdda819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddad200640 1 -- 172.21.15.46:0/2581945698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda8109140 msgr2=0x7fdda819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddad200640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda8109140 0x7fdda819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddad200640 1 -- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8104010 msgr2=0x7fdda819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddad200640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8104010 0x7fdda819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.851+0000 7fddad200640 1 -- 172.21.15.46:0/2581945698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda819f5f0 con 0x7fdda81049c0 2024-09-10T07:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.852+0000 7fddad200640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda81049c0 0x7fdda819ab80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400e9f0 tx=0x7fdd9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.852+0000 7fdda6000640 1 -- 172.21.15.46:0/2581945698 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400ccc0 con 0x7fdda81049c0 2024-09-10T07:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.852+0000 7fddb0600640 1 -- 172.21.15.46:0/2581945698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda819f8e0 con 0x7fdda81049c0 2024-09-10T07:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.852+0000 7fdda6000640 1 -- 172.21.15.46:0/2581945698 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd94010910 con 0x7fdda81049c0 2024-09-10T07:35:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.852+0000 7fdda6000640 1 -- 172.21.15.46:0/2581945698 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94018960 con 0x7fdda81049c0 2024-09-10T07:35:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.852+0000 7fddb0600640 1 -- 172.21.15.46:0/2581945698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda81a71c0 con 0x7fdda81049c0 2024-09-10T07:35:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.855+0000 7fddb30b5740 1 Processor -- start 2024-09-10T07:35:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.855+0000 7fddb30b5740 1 -- start start 2024-09-10T07:35:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.855+0000 7fdda6000640 1 -- 172.21.15.46:0/2581945698 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdd94010ac0 con 0x7fdda81049c0 2024-09-10T07:35:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b949710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b949c50 0x55e47b953fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b954510 0x55e47b956900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddb30b5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e47b8af0b0 con 0x55e47b949c50 2024-09-10T07:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddb30b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e47b856ab0 con 0x55e47b954510 2024-09-10T07:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddb30b5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e47b8ab070 con 0x55e47b949330 2024-09-10T07:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddadc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b949c50 0x55e47b953fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddae600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b949710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddadc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b949c50 0x55e47b953fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38844/0 (socket says 172.21.15.46:38844) 2024-09-10T07:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddadc00640 1 -- 172.21.15.46:0/1039993325 learned_addr learned my addr 172.21.15.46:0/1039993325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddae600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b949710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57760/0 (socket says 172.21.15.46:57760) 2024-09-10T07:35:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.856+0000 7fddad200640 1 --2- 172.21.15.46:0/1039993325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b954510 0x55e47b956900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.857+0000 7fddae600640 1 -- 172.21.15.46:0/1039993325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b954510 msgr2=0x55e47b956900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.857+0000 7fddae600640 1 --2- 172.21.15.46:0/1039993325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b954510 0x55e47b956900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.857+0000 7fddae600640 1 -- 172.21.15.46:0/1039993325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b949c50 msgr2=0x55e47b953fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.857+0000 7fddae600640 1 --2- 172.21.15.46:0/1039993325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b949c50 0x55e47b953fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.857+0000 7fddae600640 1 -- 172.21.15.46:0/1039993325 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47b8634a0 con 0x55e47b949330 2024-09-10T07:35:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.857+0000 7fddae600640 1 --2- 172.21.15.46:0/1039993325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b949710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdda0005e30 tx=0x7fdda00093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9908fe73d723a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.857+0000 7fdd9aa00640 1 -- 172.21.15.46:0/1039993325 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0031db0 con 0x55e47b949330 2024-09-10T07:35:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fdd9aa00640 1 -- 172.21.15.46:0/1039993325 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0036030 con 0x55e47b949330 2024-09-10T07:35:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fdd9aa00640 1 -- 172.21.15.46:0/1039993325 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda00042a0 con 0x55e47b949330 2024-09-10T07:35:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fdda6000640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdd900a2b80 0x7fdd900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fdda6000640 1 -- 172.21.15.46:0/2581945698 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdd940cf740 con 0x7fdda81049c0 2024-09-10T07:35:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fddadc00640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdd900a2b80 0x7fdd900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fddb30b5740 1 -- 172.21.15.46:0/1039993325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 msgr2=0x55e47b949710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1039993325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b949710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdda0005e30 tx=0x7fdda00093e0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fddb30b5740 1 -- 172.21.15.46:0/1039993325 shutdown_connections 2024-09-10T07:35:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1039993325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b954510 0x55e47b956900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1039993325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b949c50 0x55e47b953fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1039993325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b949710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.858+0000 7fddb30b5740 1 -- 172.21.15.46:0/1039993325 >> 172.21.15.46:0/1039993325 conn(0x55e47b9458c0 msgr2=0x55e47b945ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.859+0000 7fddb30b5740 1 -- 172.21.15.46:0/1039993325 shutdown_connections 2024-09-10T07:35:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.859+0000 7fddadc00640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdd900a2b80 0x7fdd900a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55e47b946720 tx=0x7fdd9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.859+0000 7fddb30b5740 1 -- 172.21.15.46:0/1039993325 wait complete. 2024-09-10T07:35:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.859+0000 7fddb30b5740 1 -- start start 2024-09-10T07:35:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddb30b5740 1 Processor -- start 2024-09-10T07:35:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b951480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b9519c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b951f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddb30b5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e47b957000 con 0x55e47b954510 2024-09-10T07:35:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddb30b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e47b837920 con 0x55e47b949c50 2024-09-10T07:35:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddb30b5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e47b957180 con 0x55e47b949330 2024-09-10T07:35:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddb30b5740 10 client.0 _ll_register_callbacks cb 0x55e47b85b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddadc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b951480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddadc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b951480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57766/0 (socket says 172.21.15.46:57766) 2024-09-10T07:35:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddadc00640 1 -- 172.21.15.46:0/1849532501 learned_addr learned my addr 172.21.15.46:0/1849532501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddae600640 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b951f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.860+0000 7fddad200640 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b9519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.861+0000 7fddadc00640 1 -- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 msgr2=0x55e47b9519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.861+0000 7fddadc00640 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b9519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.861+0000 7fddadc00640 1 -- 172.21.15.46:0/1849532501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 msgr2=0x55e47b951f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.861+0000 7fddadc00640 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b951f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.861+0000 7fddadc00640 1 -- 172.21.15.46:0/1849532501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47b6202b0 con 0x55e47b949330 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.861+0000 7fddad200640 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b9519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.861+0000 7fddadc00640 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b951480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c01e9d0 tx=0x7fdd9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a6a5654aceae0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.861+0000 7fdd8f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:35:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fdd8f400640 1 -- 172.21.15.46:0/1849532501 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c02e060 con 0x55e47b949330 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fdd8f400640 1 -- 172.21.15.46:0/1849532501 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdd9c018030 con 0x55e47b949330 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fddb30b5740 1 -- 172.21.15.46:0/1849532501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda0036030 con 0x55e47b949330 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fdd8f400640 1 -- 172.21.15.46:0/1849532501 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c035050 con 0x55e47b949330 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fddb30b5740 10 client.25039 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fddb30b5740 1 -- 172.21.15.46:0/1849532501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e47b962c80 con 0x55e47b949330 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fdd8f400640 1 -- 172.21.15.46:0/1849532501 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0036030 con 0x55e47b949330 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fdd8f400640 1 -- 172.21.15.46:0/1849532501 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdd9c00f670 con 0x55e47b949330 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.862+0000 7fddb30b5740 10 client.25039 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.863+0000 7fddb30b5740 10 client.25039 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.863+0000 7fddb30b5740 20 client.25039 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.863+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e47b965290 0x55e47b966700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.863+0000 7fddb30b5740 4 client.25039 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:35:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.863+0000 7fddb30b5740 1 -- 172.21.15.46:0/1849532501 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47b8bc620 con 0x55e47b965290 2024-09-10T07:35:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.863+0000 7fddad200640 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e47b965290 0x55e47b966700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.863+0000 7fddad200640 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e47b965290 0x55e47b966700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=457345268fb5fe77 server_cookie=87838187f74e5fdc in_seq=0 out_seq=0 2024-09-10T07:35:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.863+0000 7fdd8f400640 10 client.25039 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:35:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.864+0000 7fdd8f400640 1 -- 172.21.15.46:0/1849532501 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e47b8bc620 con 0x55e47b965290 2024-09-10T07:35:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.864+0000 7fdd8f400640 10 client.25039 handle_command_reply: tid=0 2024-09-10T07:35:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.864+0000 7fddb30b5740 1 client.25039 shutdown 2024-09-10T07:35:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 -- 172.21.15.46:0/1849532501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 msgr2=0x55e47b951480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b951480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c01e9d0 tx=0x7fdd9c004730 comp rx=0 tx=0).stop 2024-09-10T07:35:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 -- 172.21.15.46:0/1849532501 shutdown_connections 2024-09-10T07:35:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e47b965290 0x55e47b966700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b951f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b9519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 --2- 172.21.15.46:0/1849532501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b951480 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fdd8f400640 0 client.25039 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:35:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 -- 172.21.15.46:0/1849532501 >> 172.21.15.46:0/1849532501 conn(0x55e47b9458c0 msgr2=0x55e47b94b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 -- 172.21.15.46:0/1849532501 shutdown_connections 2024-09-10T07:35:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 1 -- 172.21.15.46:0/1849532501 wait complete. 2024-09-10T07:35:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.865+0000 7fddb30b5740 20 client.25039 trim_cache size 0 max 16384 2024-09-10T07:35:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.884+0000 7fddb30b5740 1 Processor -- start 2024-09-10T07:35:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.884+0000 7fddb30b5740 1 -- start start 2024-09-10T07:35:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.884+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b9519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.884+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b951f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.884+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b94e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.884+0000 7fddb30b5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e47b964a40 con 0x55e47b954510 2024-09-10T07:35:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.884+0000 7fddb30b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e47b965110 con 0x55e47b949c50 2024-09-10T07:35:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.884+0000 7fddb30b5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e47b9657e0 con 0x55e47b949330 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddae600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b94e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddad200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b951f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddae600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b94e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38866/0 (socket says 172.21.15.46:38866) 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddae600640 1 -- 172.21.15.46:0/713865326 learned_addr learned my addr 172.21.15.46:0/713865326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddadc00640 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b9519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddad200640 1 -- 172.21.15.46:0/713865326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 msgr2=0x55e47b9519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddad200640 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b9519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddad200640 1 -- 172.21.15.46:0/713865326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 msgr2=0x55e47b94e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddad200640 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b94e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddad200640 1 -- 172.21.15.46:0/713865326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47b949160 con 0x55e47b949c50 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddae600640 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b94e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.885+0000 7fddad200640 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b951f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e47b946c40 tx=0x7fdd940d5530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471a73de96f1a959 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fdd9a000640 1 -- 172.21.15.46:0/713865326 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd940d5020 con 0x55e47b949c50 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fdd9a000640 1 -- 172.21.15.46:0/713865326 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd9409bc70 con 0x55e47b949c50 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fdd9a000640 1 -- 172.21.15.46:0/713865326 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd940d1b90 con 0x55e47b949c50 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fddb30b5740 1 -- 172.21.15.46:0/713865326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 msgr2=0x55e47b951f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fddb30b5740 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b951f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e47b946c40 tx=0x7fdd940d5530 comp rx=0 tx=0).stop 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fddb30b5740 1 -- 172.21.15.46:0/713865326 shutdown_connections 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fddb30b5740 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b94e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fddb30b5740 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949c50 0x55e47b951f10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fddb30b5740 1 --2- 172.21.15.46:0/713865326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949330 0x55e47b9519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fddb30b5740 1 -- 172.21.15.46:0/713865326 >> 172.21.15.46:0/713865326 conn(0x55e47b9458c0 msgr2=0x55e47b94fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.886+0000 7fddb30b5740 1 -- 172.21.15.46:0/713865326 shutdown_connections 2024-09-10T07:35:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.887+0000 7fddb30b5740 1 -- 172.21.15.46:0/713865326 wait complete. 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.887+0000 7fddb30b5740 1 -- start start 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.887+0000 7fddb30b5740 1 Processor -- start 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.887+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949330 0x55e47b969550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949c50 0x55e47b969a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddb30b5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b969fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddb30b5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e47b966580 con 0x55e47b954510 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddb30b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e47b965eb0 con 0x55e47b949330 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddb30b5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e47b966c50 con 0x55e47b949c50 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddb30b5740 10 client.0 _ll_register_callbacks cb 0x7fdd9c01e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddadc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949330 0x55e47b969550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddad200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949c50 0x55e47b969a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddae600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b969fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddad200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949c50 0x55e47b969a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57786/0 (socket says 172.21.15.46:57786) 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddad200640 1 -- 172.21.15.46:0/2047671602 learned_addr learned my addr 172.21.15.46:0/2047671602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddadc00640 1 -- 172.21.15.46:0/2047671602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949c50 msgr2=0x55e47b969a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddadc00640 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949c50 0x55e47b969a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddadc00640 1 -- 172.21.15.46:0/2047671602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 msgr2=0x55e47b969fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddadc00640 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b969fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.888+0000 7fddadc00640 1 -- 172.21.15.46:0/2047671602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47b94f4b0 con 0x55e47b949330 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.889+0000 7fddadc00640 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949330 0x55e47b969550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00fc30 tx=0x7fdd9c01e820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8a6d2661de5bdab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.889+0000 7fdd8f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.889+0000 7fdd8f400640 1 -- 172.21.15.46:0/2047671602 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00f5d0 con 0x55e47b949330 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.889+0000 7fddb30b5740 1 -- 172.21.15.46:0/2047671602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd9409bc70 con 0x55e47b949330 2024-09-10T07:35:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.889+0000 7fdd8f400640 1 -- 172.21.15.46:0/2047671602 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdd9c00f770 con 0x55e47b949330 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.889+0000 7fdd8f400640 1 -- 172.21.15.46:0/2047671602 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c047050 con 0x55e47b949330 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.889+0000 7fddb30b5740 10 client.25034 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.889+0000 7fddb30b5740 1 -- 172.21.15.46:0/2047671602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e47b9cfa60 con 0x55e47b949330 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.890+0000 7fdd8f400640 1 -- 172.21.15.46:0/2047671602 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd9409bc70 con 0x55e47b949330 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.890+0000 7fdd8f400640 1 -- 172.21.15.46:0/2047671602 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdd9c0294b0 con 0x55e47b949330 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.890+0000 7fddb30b5740 10 client.25034 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.890+0000 7fddb30b5740 10 client.25034 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.890+0000 7fddb30b5740 20 client.25034 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.890+0000 7fddb30b5740 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e47b9d4630 0x55e47b9d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.890+0000 7fddb30b5740 4 client.25034 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.890+0000 7fddb30b5740 1 -- 172.21.15.46:0/2047671602 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "status"}) -- 0x55e47b8b6b30 con 0x55e47b9d4630 2024-09-10T07:35:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fddad200640 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e47b9d4630 0x55e47b9d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:51.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fddad200640 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e47b9d4630 0x55e47b9d3340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=668c1086d0f34bcf server_cookie=caa01f64025c9f3d in_seq=0 out_seq=0 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fdd8f400640 10 client.25034 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fdd8f400640 1 -- 172.21.15.46:0/2047671602 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e47b8b6b30 con 0x55e47b9d4630 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fdd8f400640 10 client.25034 handle_command_reply: tid=0 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fddb30b5740 1 client.25034 shutdown 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fddb30b5740 1 -- 172.21.15.46:0/2047671602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949330 msgr2=0x55e47b969550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fddb30b5740 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949330 0x55e47b969550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00fc30 tx=0x7fdd9c01e820 comp rx=0 tx=0).stop 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fddb30b5740 1 -- 172.21.15.46:0/2047671602 shutdown_connections 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fddb30b5740 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e47b9d4630 0x55e47b9d3340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.891+0000 7fddb30b5740 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e47b954510 0x55e47b969fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb30b5740 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e47b949c50 0x55e47b969a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb30b5740 1 --2- 172.21.15.46:0/2047671602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47b949330 0x55e47b969550 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fdd8f400640 0 client.25034 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb30b5740 1 -- 172.21.15.46:0/2047671602 >> 172.21.15.46:0/2047671602 conn(0x55e47b9458c0 msgr2=0x55e47b950140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb30b5740 1 -- 172.21.15.46:0/2047671602 shutdown_connections 2024-09-10T07:35:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb30b5740 1 -- 172.21.15.46:0/2047671602 wait complete. 2024-09-10T07:35:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb30b5740 20 client.25034 trim_cache size 0 max 16384 2024-09-10T07:35:51.895 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:35:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:35:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-09-10T07:35:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24530, 2024-09-10T07:35:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:35:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:35:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:35:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 88.835995671000006, 2024-09-10T07:35:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:35:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:35:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:35:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 148.86437881699999 2024-09-10T07:35:51.896 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb0600640 1 -- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdd900a2b80 msgr2=0x7fdd900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb0600640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdd900a2b80 0x7fdd900a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55e47b946720 tx=0x7fdd9c005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb0600640 1 -- 172.21.15.46:0/2581945698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda81049c0 msgr2=0x7fdda819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb0600640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda81049c0 0x7fdda819ab80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400e9f0 tx=0x7fdd9400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.892+0000 7fddb0600640 1 -- 172.21.15.46:0/2581945698 shutdown_connections 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.893+0000 7fddb0600640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda8109140 0x7fdda819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.893+0000 7fddb0600640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda81049c0 0x7fdda819ab80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.893+0000 7fddb0600640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdd900a2b80 0x7fdd900a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.893+0000 7fddb0600640 1 --2- 172.21.15.46:0/2581945698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8104010 0x7fdda819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:51.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.893+0000 7fddb0600640 1 -- 172.21.15.46:0/2581945698 >> 172.21.15.46:0/2581945698 conn(0x7fdda80ffa50 msgr2=0x7fdda8100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:51.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.893+0000 7fddb0600640 1 -- 172.21.15.46:0/2581945698 shutdown_connections 2024-09-10T07:35:51.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:51.893+0000 7fddb0600640 1 -- 172.21.15.46:0/2581945698 wait complete. 2024-09-10T07:35:52.398 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:52 smithi046 ceph-mon[30705]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:52 smithi181 ceph-mon[35397]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:52 smithi063 ceph-mon[35020]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:52.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-10T07:35:53.208 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 -- 172.21.15.46:0/3908679330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270112b90 msgr2=0x7fd270114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 --2- 172.21.15.46:0/3908679330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270112b90 0x7fd270114f80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd268009ca0 tx=0x7fd26802a950 comp rx=0 tx=0).stop 2024-09-10T07:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 -- 172.21.15.46:0/3908679330 shutdown_connections 2024-09-10T07:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 --2- 172.21.15.46:0/3908679330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270112b90 0x7fd270114f80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 --2- 172.21.15.46:0/3908679330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270074eb0 0x7fd270112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 --2- 172.21.15.46:0/3908679330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270074590 0x7fd270074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 -- 172.21.15.46:0/3908679330 >> 172.21.15.46:0/3908679330 conn(0x7fd2700fd8c0 msgr2=0x7fd2700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 -- 172.21.15.46:0/3908679330 shutdown_connections 2024-09-10T07:35:53.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.815+0000 7fd276a00640 1 -- 172.21.15.46:0/3908679330 wait complete. 2024-09-10T07:35:53.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.816+0000 7fd276a00640 1 Processor -- start 2024-09-10T07:35:53.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.816+0000 7fd276a00640 1 -- start start 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd276a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270074590 0x7fd2701a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd276a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270074eb0 0x7fd2701a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd276a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270112b90 0x7fd27019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd276a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701185b0 con 0x7fd270074590 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd276a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd270118430 con 0x7fd270074eb0 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd276a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd270118730 con 0x7fd270112b90 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd26fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270074590 0x7fd2701a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd26fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270074590 0x7fd2701a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38888/0 (socket says 172.21.15.46:38888) 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd26fe00640 1 -- 172.21.15.46:0/3176762508 learned_addr learned my addr 172.21.15.46:0/3176762508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd267400640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270074eb0 0x7fd2701a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.817+0000 7fd274a00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270112b90 0x7fd27019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.818+0000 7fd26fe00640 1 -- 172.21.15.46:0/3176762508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270112b90 msgr2=0x7fd27019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.818+0000 7fd26fe00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270112b90 0x7fd27019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.818+0000 7fd26fe00640 1 -- 172.21.15.46:0/3176762508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270074eb0 msgr2=0x7fd2701a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.818+0000 7fd26fe00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270074eb0 0x7fd2701a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.818+0000 7fd26fe00640 1 -- 172.21.15.46:0/3176762508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27019efd0 con 0x7fd270074590 2024-09-10T07:35:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.818+0000 7fd26fe00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270074590 0x7fd2701a46e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e9f0 tx=0x7fd26000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:53.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.818+0000 7fd26d600640 1 -- 172.21.15.46:0/3176762508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26000cd80 con 0x7fd270074590 2024-09-10T07:35:53.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.819+0000 7fd26d600640 1 -- 172.21.15.46:0/3176762508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd260005500 con 0x7fd270074590 2024-09-10T07:35:53.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.819+0000 7fd276a00640 1 -- 172.21.15.46:0/3176762508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27019f2c0 con 0x7fd270074590 2024-09-10T07:35:53.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.819+0000 7fd26d600640 1 -- 172.21.15.46:0/3176762508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260010690 con 0x7fd270074590 2024-09-10T07:35:53.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.819+0000 7fd276a00640 1 -- 172.21.15.46:0/3176762508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2701ab5b0 con 0x7fd270074590 2024-09-10T07:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.822+0000 7fd279640740 1 Processor -- start 2024-09-10T07:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.822+0000 7fd26d600640 1 -- 172.21.15.46:0/3176762508 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd2600046c0 con 0x7fd270074590 2024-09-10T07:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.822+0000 7fd279640740 1 -- start start 2024-09-10T07:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd279640740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015330 0x562b49015710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd279640740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015c50 0x562b4901ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd279640740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b49022900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd279640740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562b48f7b0b0 con 0x562b49020510 2024-09-10T07:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd279640740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b48f22ab0 con 0x562b49015330 2024-09-10T07:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd279640740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562b48f77070 con 0x562b49015c50 2024-09-10T07:35:53.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd26fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015c50 0x562b4901ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd274a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015330 0x562b49015710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd267400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b49022900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd274a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015330 0x562b49015710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40668/0 (socket says 172.21.15.46:40668) 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd26fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015c50 0x562b4901ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57814/0 (socket says 172.21.15.46:57814) 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.823+0000 7fd274a00640 1 -- 172.21.15.46:0/2583801123 learned_addr learned my addr 172.21.15.46:0/2583801123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd274a00640 1 -- 172.21.15.46:0/2583801123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015c50 msgr2=0x562b4901ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd274a00640 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015c50 0x562b4901ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd274a00640 1 -- 172.21.15.46:0/2583801123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 msgr2=0x562b49022900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd274a00640 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b49022900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd274a00640 1 -- 172.21.15.46:0/2583801123 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b48f2f4a0 con 0x562b49015330 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd26fe00640 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015c50 0x562b4901ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd267400640 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b49022900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd26d600640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2540a2b80 0x7fd2540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd274a00640 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015330 0x562b49015710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd270076600 tx=0x7fd268005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec522c345c77ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd26d600640 1 -- 172.21.15.46:0/3176762508 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd2600cf7a0 con 0x7fd270074590 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd26fe00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2540a2b80 0x7fd2540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd265200640 1 -- 172.21.15.46:0/2583801123 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268048070 con 0x562b49015330 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd265200640 1 -- 172.21.15.46:0/2583801123 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd268003e50 con 0x562b49015330 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.824+0000 7fd265200640 1 -- 172.21.15.46:0/2583801123 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268034040 con 0x562b49015330 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.825+0000 7fd26fe00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2540a2b80 0x7fd2540a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2600cd290 tx=0x7fd2600d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.825+0000 7fd279640740 1 -- 172.21.15.46:0/2583801123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015330 msgr2=0x562b49015710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.825+0000 7fd279640740 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015330 0x562b49015710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd270076600 tx=0x7fd268005f80 comp rx=0 tx=0).stop 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.826+0000 7fd279640740 1 -- 172.21.15.46:0/2583801123 shutdown_connections 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.826+0000 7fd279640740 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b49022900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.826+0000 7fd279640740 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015c50 0x562b4901ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.826+0000 7fd279640740 1 --2- 172.21.15.46:0/2583801123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015330 0x562b49015710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.826+0000 7fd279640740 1 -- 172.21.15.46:0/2583801123 >> 172.21.15.46:0/2583801123 conn(0x562b490118c0 msgr2=0x562b49011ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.826+0000 7fd279640740 1 -- 172.21.15.46:0/2583801123 shutdown_connections 2024-09-10T07:35:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.826+0000 7fd279640740 1 -- 172.21.15.46:0/2583801123 wait complete. 2024-09-10T07:35:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.826+0000 7fd279640740 1 -- start start 2024-09-10T07:35:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.827+0000 7fd279640740 1 Processor -- start 2024-09-10T07:35:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.827+0000 7fd279640740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4901d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.827+0000 7fd279640740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b4901dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.827+0000 7fd279640740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b4901e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.827+0000 7fd279640740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562b49023060 con 0x562b49020510 2024-09-10T07:35:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.827+0000 7fd279640740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b48f03920 con 0x562b49015c50 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.827+0000 7fd279640740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562b490231e0 con 0x562b49015330 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.827+0000 7fd279640740 10 client.0 _ll_register_callbacks cb 0x562b48f278a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd267400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b4901dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd267400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b4901dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40674/0 (socket says 172.21.15.46:40674) 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd267400640 1 -- 172.21.15.46:0/712238488 learned_addr learned my addr 172.21.15.46:0/712238488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd274a00640 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4901d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd274a00640 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b4901e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd267400640 1 -- 172.21.15.46:0/712238488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 msgr2=0x562b4901d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd267400640 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4901d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd267400640 1 -- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 msgr2=0x562b4901e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:53.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd267400640 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b4901e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd267400640 1 -- 172.21.15.46:0/712238488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b48cec2b0 con 0x562b49015c50 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.828+0000 7fd274a00640 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b4901e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.829+0000 7fd267400640 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b4901dbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd258012700 tx=0x7fd258012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a8d7e3df4818afe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.829+0000 7fd25d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.829+0000 7fd25d600640 1 -- 172.21.15.46:0/712238488 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258004050 con 0x562b49015c50 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.829+0000 7fd279640740 1 -- 172.21.15.46:0/712238488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd268003e50 con 0x562b49015c50 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.829+0000 7fd25d600640 1 -- 172.21.15.46:0/712238488 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd2580041f0 con 0x562b49015c50 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.829+0000 7fd25d600640 1 -- 172.21.15.46:0/712238488 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2580043f0 con 0x562b49015c50 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.829+0000 7fd279640740 10 client.25046 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.829+0000 7fd279640740 1 -- 172.21.15.46:0/712238488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b4902ef20 con 0x562b49015c50 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.830+0000 7fd25d600640 1 -- 172.21.15.46:0/712238488 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd268003e50 con 0x562b49015c50 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.830+0000 7fd25d600640 1 -- 172.21.15.46:0/712238488 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd258003950 con 0x562b49015c50 2024-09-10T07:35:53.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.830+0000 7fd279640740 10 client.25046 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:53.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.830+0000 7fd279640740 10 client.25046 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:35:53.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.830+0000 7fd279640740 20 client.25046 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:53.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.830+0000 7fd279640740 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b490314f0 0x562b49032960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.830+0000 7fd279640740 4 client.25046 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:35:53.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.830+0000 7fd279640740 1 -- 172.21.15.46:0/712238488 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b48f88620 con 0x562b490314f0 2024-09-10T07:35:53.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.831+0000 7fd274a00640 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b490314f0 0x562b49032960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.831+0000 7fd274a00640 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b490314f0 0x562b49032960 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b41834ba7219ea5 server_cookie=c487ca38cbbeb7e9 in_seq=0 out_seq=0 2024-09-10T07:35:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.831+0000 7fd25d600640 10 client.25046 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:35:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.832+0000 7fd25d600640 1 -- 172.21.15.46:0/712238488 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b48f88620 con 0x562b490314f0 2024-09-10T07:35:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.832+0000 7fd25d600640 10 client.25046 handle_command_reply: tid=0 2024-09-10T07:35:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.832+0000 7fd279640740 1 client.25046 shutdown 2024-09-10T07:35:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.832+0000 7fd279640740 1 -- 172.21.15.46:0/712238488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 msgr2=0x562b4901dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.832+0000 7fd279640740 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b4901dbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd258012700 tx=0x7fd258012b00 comp rx=0 tx=0).stop 2024-09-10T07:35:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 1 -- 172.21.15.46:0/712238488 shutdown_connections 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b490314f0 0x562b49032960 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49020510 0x562b4901e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b4901dbb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd25d600640 0 client.25046 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 1 --2- 172.21.15.46:0/712238488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4901d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 1 -- 172.21.15.46:0/712238488 >> 172.21.15.46:0/712238488 conn(0x562b490118c0 msgr2=0x562b490175a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 1 -- 172.21.15.46:0/712238488 shutdown_connections 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 1 -- 172.21.15.46:0/712238488 wait complete. 2024-09-10T07:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.833+0000 7fd279640740 20 client.25046 trim_cache size 0 max 16384 2024-09-10T07:35:53.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd279640740 1 Processor -- start 2024-09-10T07:35:53.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd279640740 1 -- start start 2024-09-10T07:35:53.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd279640740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b490118c0 0x562b4901a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd279640740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015330 0x562b4901ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd279640740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b49016d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd279640740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562b49030ca0 con 0x562b49015330 2024-09-10T07:35:53.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd279640740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b49031370 con 0x562b49015c50 2024-09-10T07:35:53.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd279640740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562b49031a40 con 0x562b490118c0 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.852+0000 7fd267400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015330 0x562b4901ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd267400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015330 0x562b4901ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38910/0 (socket says 172.21.15.46:38910) 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd267400640 1 -- 172.21.15.46:0/3570237212 learned_addr learned my addr 172.21.15.46:0/3570237212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd274a00640 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b49016d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd274a00640 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b490118c0 0x562b4901a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd267400640 1 -- 172.21.15.46:0/3570237212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b490118c0 msgr2=0x562b4901a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd267400640 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b490118c0 0x562b4901a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd267400640 1 -- 172.21.15.46:0/3570237212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 msgr2=0x562b49016d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd267400640 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b49016d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd267400640 1 -- 172.21.15.46:0/3570237212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4901bc00 con 0x562b49015330 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd274a00640 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b49016d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.853+0000 7fd267400640 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015330 0x562b4901ae70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd258003df0 tx=0x7fd258005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e013ab34d77a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd25fe00640 1 -- 172.21.15.46:0/3570237212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258003f60 con 0x562b49015330 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd25fe00640 1 -- 172.21.15.46:0/3570237212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd258003ad0 con 0x562b49015330 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd25fe00640 1 -- 172.21.15.46:0/3570237212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258005470 con 0x562b49015330 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd279640740 1 -- 172.21.15.46:0/3570237212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015330 msgr2=0x562b4901ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd279640740 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015330 0x562b4901ae70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd258003df0 tx=0x7fd258005d30 comp rx=0 tx=0).stop 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd279640740 1 -- 172.21.15.46:0/3570237212 shutdown_connections 2024-09-10T07:35:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd279640740 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b49015c50 0x562b49016d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd279640740 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015330 0x562b4901ae70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd279640740 1 --2- 172.21.15.46:0/3570237212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b490118c0 0x562b4901a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.854+0000 7fd279640740 1 -- 172.21.15.46:0/3570237212 >> 172.21.15.46:0/3570237212 conn(0x562b4901e480 msgr2=0x562b4901c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.855+0000 7fd279640740 1 -- 172.21.15.46:0/3570237212 shutdown_connections 2024-09-10T07:35:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.855+0000 7fd279640740 1 -- 172.21.15.46:0/3570237212 wait complete. 2024-09-10T07:35:53.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.855+0000 7fd279640740 1 -- start start 2024-09-10T07:35:53.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.855+0000 7fd279640740 1 Processor -- start 2024-09-10T07:35:53.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd279640740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b490118c0 0x562b4903adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd279640740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4903b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd279640740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015c50 0x562b49035170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd279640740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562b490327e0 con 0x562b49015c50 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd279640740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b49032110 con 0x562b490118c0 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd279640740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562b49032eb0 con 0x562b49015330 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd279640740 10 client.0 _ll_register_callbacks cb 0x7fd258009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd267400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4903b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd267400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4903b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57838/0 (socket says 172.21.15.46:57838) 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd267400640 1 -- 172.21.15.46:0/2603386240 learned_addr learned my addr 172.21.15.46:0/2603386240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd274a00640 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b490118c0 0x562b4903adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd274a00640 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015c50 0x562b49035170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd267400640 1 -- 172.21.15.46:0/2603386240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b490118c0 msgr2=0x562b4903adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:53.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd267400640 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b490118c0 0x562b4903adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd267400640 1 -- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015c50 msgr2=0x562b49035170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd267400640 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015c50 0x562b49035170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd267400640 1 -- 172.21.15.46:0/2603386240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b490359a0 con 0x562b49015330 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.856+0000 7fd274a00640 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015c50 0x562b49035170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.857+0000 7fd267400640 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4903b2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd258000c00 tx=0x7fd258004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=617e07e32c791cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.857+0000 7fd25d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.857+0000 7fd25d600640 1 -- 172.21.15.46:0/2603386240 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258038020 con 0x562b49015330 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.857+0000 7fd279640740 1 -- 172.21.15.46:0/2603386240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd258003ad0 con 0x562b49015330 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.857+0000 7fd25d600640 1 -- 172.21.15.46:0/2603386240 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd258003800 con 0x562b49015330 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.857+0000 7fd25d600640 1 -- 172.21.15.46:0/2603386240 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd258043050 con 0x562b49015330 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.857+0000 7fd279640740 10 client.25063 fetch_fsmap learned FSMap version 29 2024-09-10T07:35:53.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.857+0000 7fd279640740 1 -- 172.21.15.46:0/2603386240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b4909c6c0 con 0x562b49015330 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd25d600640 1 -- 172.21.15.46:0/2603386240 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd258003ad0 con 0x562b49015330 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd25d600640 1 -- 172.21.15.46:0/2603386240 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd25800f6d0 con 0x562b49015330 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd279640740 10 client.25063 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd279640740 10 client.25063 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd279640740 20 client.25063 populate_metadata read hostname 'smithi046' 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd279640740 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b49020510 0x562b4909fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd279640740 4 client.25063 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd279640740 1 -- 172.21.15.46:0/2603386240 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "status"}) -- 0x562b48f82b30 con 0x562b49020510 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.858+0000 7fd274a00640 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b49020510 0x562b4909fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd274a00640 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b49020510 0x562b4909fbf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e92685241b5c6f4 server_cookie=9721b571ef9b77d in_seq=0 out_seq=0 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd25d600640 10 client.25063 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd25d600640 1 -- 172.21.15.46:0/2603386240 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562b48f82b30 con 0x562b49020510 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd25d600640 10 client.25063 handle_command_reply: tid=0 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 client.25063 shutdown 2024-09-10T07:35:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 -- 172.21.15.46:0/2603386240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 msgr2=0x562b4903b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4903b2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd258000c00 tx=0x7fd258004210 comp rx=0 tx=0).stop 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 -- 172.21.15.46:0/2603386240 shutdown_connections 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b49020510 0x562b4909fbf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b49015c50 0x562b49035170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b49015330 0x562b4903b2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 --2- 172.21.15.46:0/2603386240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b490118c0 0x562b4903adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd25d600640 0 client.25063 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.859+0000 7fd279640740 1 -- 172.21.15.46:0/2603386240 >> 172.21.15.46:0/2603386240 conn(0x562b4901e480 msgr2=0x562b4901eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd279640740 1 -- 172.21.15.46:0/2603386240 shutdown_connections 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd279640740 1 -- 172.21.15.46:0/2603386240 wait complete. 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd279640740 20 client.25063 trim_cache size 0 max 16384 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24535, 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:35:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 136.98733968600001, 2024-09-10T07:35:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:35:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:35:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:35:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 153.38071680600001 2024-09-10T07:35:53.866 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:35:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 -- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2540a2b80 msgr2=0x7fd2540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2540a2b80 0x7fd2540a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2600cd290 tx=0x7fd2600d5550 comp rx=0 tx=0).stop 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 -- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270074590 msgr2=0x7fd2701a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270074590 0x7fd2701a46e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e9f0 tx=0x7fd26000edf0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 -- 172.21.15.46:0/3176762508 shutdown_connections 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd270112b90 0x7fd27019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd270074eb0 0x7fd2701a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2540a2b80 0x7fd2540a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 --2- 172.21.15.46:0/3176762508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd270074590 0x7fd2701a46e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.860+0000 7fd276a00640 1 -- 172.21.15.46:0/3176762508 >> 172.21.15.46:0/3176762508 conn(0x7fd2700fd8c0 msgr2=0x7fd2700fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.861+0000 7fd276a00640 1 -- 172.21.15.46:0/3176762508 shutdown_connections 2024-09-10T07:35:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:53.861+0000 7fd276a00640 1 -- 172.21.15.46:0/3176762508 wait complete. 2024-09-10T07:35:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:54 smithi181 ceph-mon[35397]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:54 smithi063 ceph-mon[35020]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:54.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:54 smithi046 ceph-mon[30705]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T07:35:54.908 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T07:35:54.908 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T07:35:54.920 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:35:54.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T07:35:55.302 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 -- 172.21.15.46:0/1526725939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 msgr2=0x7f6e241053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 --2- 172.21.15.46:0/1526725939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 0x7f6e241053e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009860 tx=0x7f6e0c02a340 comp rx=0 tx=0).stop 2024-09-10T07:35:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 -- 172.21.15.46:0/1526725939 shutdown_connections 2024-09-10T07:35:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 --2- 172.21.15.46:0/1526725939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24105920 0x7f6e2410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 --2- 172.21.15.46:0/1526725939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 0x7f6e241053e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 --2- 172.21.15.46:0/1526725939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e24103e00 0x7f6e24104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 -- 172.21.15.46:0/1526725939 >> 172.21.15.46:0/1526725939 conn(0x7f6e240ff5b0 msgr2=0x7f6e241019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 -- 172.21.15.46:0/1526725939 shutdown_connections 2024-09-10T07:35:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.863+0000 7f6e28c00640 1 -- 172.21.15.46:0/1526725939 wait complete. 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.864+0000 7f6e28c00640 1 Processor -- start 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.864+0000 7f6e28c00640 1 -- start start 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.864+0000 7f6e28c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24103e00 0x7f6e2419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.864+0000 7f6e28c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 0x7f6e2419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.864+0000 7f6e28c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e24105920 0x7f6e241a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.864+0000 7f6e28c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2410fab0 con 0x7f6e24103e00 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.864+0000 7f6e28c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2410f930 con 0x7f6e24104f60 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.864+0000 7f6e28c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2410fc30 con 0x7f6e24105920 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 0x7f6e2419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 0x7f6e2419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40440/0 (socket says 172.21.15.46:40440) 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e19400640 1 -- 172.21.15.46:0/2713523910 learned_addr learned my addr 172.21.15.46:0/2713523910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e21e00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24103e00 0x7f6e2419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e22800640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e24105920 0x7f6e241a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e19400640 1 -- 172.21.15.46:0/2713523910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e24105920 msgr2=0x7f6e241a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e19400640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e24105920 0x7f6e241a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e19400640 1 -- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24103e00 msgr2=0x7f6e2419a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e19400640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24103e00 0x7f6e2419a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e19400640 1 -- 172.21.15.46:0/2713523910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e241a1d70 con 0x7f6e24104f60 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.865+0000 7f6e22800640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e24105920 0x7f6e241a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:35:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.866+0000 7f6e19400640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 0x7f6e2419a680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c035040 tx=0x7f6e0c033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.866+0000 7f6e1b400640 1 -- 172.21.15.46:0/2713523910 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c037040 con 0x7f6e24104f60 2024-09-10T07:35:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.866+0000 7f6e1b400640 1 -- 172.21.15.46:0/2713523910 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c030850 con 0x7f6e24104f60 2024-09-10T07:35:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.866+0000 7f6e1b400640 1 -- 172.21.15.46:0/2713523910 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c004920 con 0x7f6e24104f60 2024-09-10T07:35:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.866+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e241a2000 con 0x7f6e24104f60 2024-09-10T07:35:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.866+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e241a24e0 con 0x7f6e24104f60 2024-09-10T07:35:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.867+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2410b1e0 con 0x7f6e24104f60 2024-09-10T07:35:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.869+0000 7f6e1b400640 1 -- 172.21.15.46:0/2713523910 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e0c004050 con 0x7f6e24104f60 2024-09-10T07:35:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.871+0000 7f6e1b400640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6e080a2bb0 0x7f6e080a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.871+0000 7f6e1b400640 1 -- 172.21.15.46:0/2713523910 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6e0c0ef0d0 con 0x7f6e24104f60 2024-09-10T07:35:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.871+0000 7f6e21e00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6e080a2bb0 0x7f6e080a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.871+0000 7f6e21e00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6e080a2bb0 0x7f6e080a5070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009bd0 tx=0x7f6e14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:55.876+0000 7f6e1b400640 1 -- 172.21.15.46:0/2713523910 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e0c049020 con 0x7f6e24104f60 2024-09-10T07:35:56.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.080+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f6e24062630 con 0x7f6e080a2bb0 2024-09-10T07:35:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.081+0000 7f6e1b400640 1 -- 172.21.15.46:0/2713523910 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f6e24062630 con 0x7f6e080a2bb0 2024-09-10T07:35:56.111 INFO:teuthology.orchestra.run.smithi046.stdout:Backend: cephadm 2024-09-10T07:35:56.111 INFO:teuthology.orchestra.run.smithi046.stdout:Available: Yes 2024-09-10T07:35:56.111 INFO:teuthology.orchestra.run.smithi046.stdout:Paused: No 2024-09-10T07:35:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6e080a2bb0 msgr2=0x7f6e080a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6e080a2bb0 0x7f6e080a5070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009bd0 tx=0x7f6e14005f90 comp rx=0 tx=0).stop 2024-09-10T07:35:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 msgr2=0x7f6e2419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 0x7f6e2419a680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c035040 tx=0x7f6e0c033d90 comp rx=0 tx=0).stop 2024-09-10T07:35:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 shutdown_connections 2024-09-10T07:35:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e24105920 0x7f6e241a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24104f60 0x7f6e2419a680 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6e080a2bb0 0x7f6e080a5070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 --2- 172.21.15.46:0/2713523910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24103e00 0x7f6e2419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 >> 172.21.15.46:0/2713523910 conn(0x7f6e240ff5b0 msgr2=0x7f6e241083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 shutdown_connections 2024-09-10T07:35:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:56.083+0000 7f6e28c00640 1 -- 172.21.15.46:0/2713523910 wait complete. 2024-09-10T07:35:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:56 smithi181 ceph-mon[35397]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:56.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:56 smithi063 ceph-mon[35020]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:56 smithi046 ceph-mon[30705]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T07:35:56.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T07:35:57.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:57 smithi046 ceph-mon[30705]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:35:57.250 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:35:57.301 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:57 smithi063 ceph-mon[35020]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:35:57.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:57 smithi181 ceph-mon[35397]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 -- 172.21.15.46:0/4124699532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18103600 msgr2=0x7f3b18109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 --2- 172.21.15.46:0/4124699532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18103600 0x7f3b18109e90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009900 tx=0x7f3b1002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 -- 172.21.15.46:0/4124699532 shutdown_connections 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 --2- 172.21.15.46:0/4124699532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18103600 0x7f3b18109e90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 --2- 172.21.15.46:0/4124699532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b18102c40 0x7f3b181030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 --2- 172.21.15.46:0/4124699532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b18101a40 0x7f3b18101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 -- 172.21.15.46:0/4124699532 >> 172.21.15.46:0/4124699532 conn(0x7f3b180fd1f0 msgr2=0x7f3b180ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 -- 172.21.15.46:0/4124699532 shutdown_connections 2024-09-10T07:35:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.904+0000 7f3b1e800640 1 -- 172.21.15.46:0/4124699532 wait complete. 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.905+0000 7f3b1e800640 1 Processor -- start 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.905+0000 7f3b1e800640 1 -- start start 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.905+0000 7f3b1e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b18101a40 0x7f3b18197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.905+0000 7f3b1e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18102c40 0x7f3b181981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.905+0000 7f3b1e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b18103600 0x7f3b1819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.905+0000 7f3b1e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1810d6f0 con 0x7f3b18101a40 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.905+0000 7f3b1e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1810d570 con 0x7f3b18102c40 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.905+0000 7f3b1e800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1810d870 con 0x7f3b18103600 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b18101a40 0x7f3b18197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18102c40 0x7f3b181981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b18101a40 0x7f3b18197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33472/0 (socket says 172.21.15.46:33472) 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1d400640 1 -- 172.21.15.46:0/192278414 learned_addr learned my addr 172.21.15.46:0/192278414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1de00640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b18103600 0x7f3b1819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1ca00640 1 -- 172.21.15.46:0/192278414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b18103600 msgr2=0x7f3b1819f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1ca00640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b18103600 0x7f3b1819f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1ca00640 1 -- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b18101a40 msgr2=0x7f3b18197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1ca00640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b18101a40 0x7f3b18197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1ca00640 1 -- 172.21.15.46:0/192278414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1819f8a0 con 0x7f3b18102c40 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.906+0000 7f3b1d400640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b18101a40 0x7f3b18197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:35:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.907+0000 7f3b1ca00640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18102c40 0x7f3b181981c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400e9f0 tx=0x7f3b0400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.907+0000 7f3b16000640 1 -- 172.21.15.46:0/192278414 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400cd80 con 0x7f3b18102c40 2024-09-10T07:35:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.907+0000 7f3b16000640 1 -- 172.21.15.46:0/192278414 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b040035f0 con 0x7f3b18102c40 2024-09-10T07:35:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.907+0000 7f3b16000640 1 -- 172.21.15.46:0/192278414 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04010690 con 0x7f3b18102c40 2024-09-10T07:35:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.907+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1819fb90 con 0x7f3b18102c40 2024-09-10T07:35:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.907+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b181a0120 con 0x7f3b18102c40 2024-09-10T07:35:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.908+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b18108df0 con 0x7f3b18102c40 2024-09-10T07:35:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.910+0000 7f3b16000640 1 -- 172.21.15.46:0/192278414 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b04003b30 con 0x7f3b18102c40 2024-09-10T07:35:57.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.911+0000 7f3b16000640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b000a2c00 0x7f3b000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:35:57.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.912+0000 7f3b16000640 1 -- 172.21.15.46:0/192278414 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b040cf6e0 con 0x7f3b18102c40 2024-09-10T07:35:57.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.912+0000 7f3b1d400640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b000a2c00 0x7f3b000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:35:57.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.912+0000 7f3b1d400640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b000a2c00 0x7f3b000a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b18198b60 tx=0x7f3b0c0096d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:35:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:57.913+0000 7f3b16000640 1 -- 172.21.15.46:0/192278414 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b0409c2f0 con 0x7f3b18102c40 2024-09-10T07:35:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.109+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3b18062630 con 0x7f3b000a2c00 2024-09-10T07:35:58.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:58 smithi046 ceph-mon[30705]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.121+0000 7f3b16000640 1 -- 172.21.15.46:0/192278414 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f3b18062630 con 0x7f3b000a2c00 2024-09-10T07:35:58.122 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T07:35:58.122 INFO:teuthology.orchestra.run.smithi046.stdout:mds.a smithi046 running (2m) 19s ago 2m 25.4M - 19.1.1-305-g1cf819b7 41d671128357 4f93e67e2900 2024-09-10T07:35:58.122 INFO:teuthology.orchestra.run.smithi046.stdout:mds.b smithi063 running (2m) 21s ago 2m 18.9M - 19.1.1-305-g1cf819b7 41d671128357 5df5365253a9 2024-09-10T07:35:58.122 INFO:teuthology.orchestra.run.smithi046.stdout:mds.c smithi181 running (2m) 21s ago 2m 17.7M - 19.1.1-305-g1cf819b7 41d671128357 7312fd87dd8b 2024-09-10T07:35:58.122 INFO:teuthology.orchestra.run.smithi046.stdout:mds.d smithi046 running (2m) 19s ago 2m 13.1M - 19.1.1-305-g1cf819b7 41d671128357 f302cef36e52 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mds.e smithi063 running (2m) 21s ago 2m 18.2M - 19.1.1-305-g1cf819b7 41d671128357 7732d917143f 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mds.f smithi181 running (2m) 21s ago 2m 18.3M - 19.1.1-305-g1cf819b7 41d671128357 034cd2c436b0 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mds.g smithi046 running (2m) 19s ago 2m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 186a8a12222d 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mds.h smithi063 running (2m) 21s ago 2m 20.9M - 19.1.1-305-g1cf819b7 41d671128357 f9f5f8b6fbfa 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mds.i smithi181 running (2m) 21s ago 2m 21.0M - 19.1.1-305-g1cf819b7 41d671128357 16f85f0e612f 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mds.j smithi046 running (3m) 19s ago 3m 16.7M - 19.1.1-305-g1cf819b7 41d671128357 568f4903558a 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mds.k smithi063 running (3m) 21s ago 3m 18.4M - 19.1.1-305-g1cf819b7 41d671128357 11c786acdcc1 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mds.l smithi181 running (3m) 21s ago 3m 21.4M - 19.1.1-305-g1cf819b7 41d671128357 b1c792b601fa 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.x smithi046 *:9283,8765 running (10m) 19s ago 10m 513M - 19.1.1-305-g1cf819b7 41d671128357 500c990df560 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.y smithi063 *:8443,8765 running (8m) 21s ago 8m 444M - 19.1.1-305-g1cf819b7 41d671128357 1d95314dabca 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.z smithi181 *:8443,8765 running (8m) 21s ago 8m 444M - 19.1.1-305-g1cf819b7 41d671128357 8a1ad40a1299 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a smithi046 running (10m) 19s ago 10m 50.6M 2048M 19.1.1-305-g1cf819b7 41d671128357 735e6e66d003 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b smithi063 running (8m) 21s ago 8m 42.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 042fae3245d2 2024-09-10T07:35:58.123 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c smithi181 running (9m) 21s ago 9m 42.8M 2048M 19.1.1-305-g1cf819b7 41d671128357 1f55219d589c 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (8m) 19s ago 8m 45.7M 4273M 19.1.1-305-g1cf819b7 41d671128357 eda15779d56e 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi063 running (7m) 21s ago 7m 48.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 14ff16b1b916 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi181 running (7m) 21s ago 7m 58.0M 4272M 19.1.1-305-g1cf819b7 41d671128357 df896b16214a 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi046 running (7m) 19s ago 7m 46.0M 4273M 19.1.1-305-g1cf819b7 41d671128357 a5da179933af 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi063 running (6m) 21s ago 6m 43.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 dd3fe9485aaf 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi181 running (6m) 21s ago 6m 46.5M 4272M 19.1.1-305-g1cf819b7 41d671128357 ddce13f13cb0 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 smithi046 running (5m) 19s ago 5m 53.9M 4273M 19.1.1-305-g1cf819b7 41d671128357 2f224a950c69 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 smithi063 running (5m) 21s ago 5m 44.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 e283876c6269 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 smithi181 running (4m) 21s ago 4m 44.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 41b2fc7a6230 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 smithi046 running (4m) 19s ago 4m 53.5M 4273M 19.1.1-305-g1cf819b7 41d671128357 96d2a24d9ae0 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 smithi063 running (3m) 21s ago 3m 43.8M 4272M 19.1.1-305-g1cf819b7 41d671128357 cd64ee8dcc55 2024-09-10T07:35:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 smithi181 running (3m) 21s ago 3m 52.8M 4272M 19.1.1-305-g1cf819b7 41d671128357 3447aaa3e46f 2024-09-10T07:35:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b000a2c00 msgr2=0x7f3b000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b000a2c00 0x7f3b000a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b18198b60 tx=0x7f3b0c0096d0 comp rx=0 tx=0).stop 2024-09-10T07:35:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18102c40 msgr2=0x7f3b181981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18102c40 0x7f3b181981c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400e9f0 tx=0x7f3b0400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 shutdown_connections 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b18103600 0x7f3b1819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b18102c40 0x7f3b181981c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b000a2c00 0x7f3b000a50c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 --2- 172.21.15.46:0/192278414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b18101a40 0x7f3b18197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 >> 172.21.15.46:0/192278414 conn(0x7f3b180fd1f0 msgr2=0x7f3b18105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 shutdown_connections 2024-09-10T07:35:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:35:58.124+0000 7f3b1e800640 1 -- 172.21.15.46:0/192278414 wait complete. 2024-09-10T07:35:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:58 smithi181 ceph-mon[35397]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:58.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:58 smithi063 ceph-mon[35020]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T07:35:59.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T07:35:59.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:35:59 smithi046 ceph-mon[30705]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:35:59.300 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:35:59 smithi181 ceph-mon[35397]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:35:59.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:35:59 smithi063 ceph-mon[35020]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:35:59.446 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 -- 172.21.15.46:0/3620362600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8077df0 msgr2=0x7f02a8078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 --2- 172.21.15.46:0/3620362600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8077df0 0x7f02a8078250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0294009900 tx=0x7f029402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 -- 172.21.15.46:0/3620362600 shutdown_connections 2024-09-10T07:36:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 --2- 172.21.15.46:0/3620362600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a8078790 0x7f02a8079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 --2- 172.21.15.46:0/3620362600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8077df0 0x7f02a8078250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 --2- 172.21.15.46:0/3620362600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a8076ba0 0x7f02a8076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 -- 172.21.15.46:0/3620362600 >> 172.21.15.46:0/3620362600 conn(0x7f02a80ff5b0 msgr2=0x7f02a81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 -- 172.21.15.46:0/3620362600 shutdown_connections 2024-09-10T07:36:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.070+0000 7f02af600640 1 -- 172.21.15.46:0/3620362600 wait complete. 2024-09-10T07:36:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.071+0000 7f02af600640 1 Processor -- start 2024-09-10T07:36:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.071+0000 7f02af600640 1 -- start start 2024-09-10T07:36:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02af600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a8076ba0 0x7f02a819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02af600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a8077df0 0x7f02a819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02af600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8078790 0x7f02a81a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02af600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8113ea0 con 0x7f02a8078790 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02af600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8113d20 con 0x7f02a8076ba0 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02af600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8114020 con 0x7f02a8077df0 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02ad600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8078790 0x7f02a81a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02a7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a8077df0 0x7f02a819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02ad600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8078790 0x7f02a81a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33482/0 (socket says 172.21.15.46:33482) 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02ad600640 1 -- 172.21.15.46:0/2368604832 learned_addr learned my addr 172.21.15.46:0/2368604832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.072+0000 7f02acc00640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a8076ba0 0x7f02a819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02ad600640 1 -- 172.21.15.46:0/2368604832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a8077df0 msgr2=0x7f02a819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02ad600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a8077df0 0x7f02a819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02ad600640 1 -- 172.21.15.46:0/2368604832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a8076ba0 msgr2=0x7f02a819e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02ad600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a8076ba0 0x7f02a819e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02ad600640 1 -- 172.21.15.46:0/2368604832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a81a6110 con 0x7f02a8078790 2024-09-10T07:36:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02ad600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8078790 0x7f02a81a5aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02a000eaa0 tx=0x7f02a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02a5600640 1 -- 172.21.15.46:0/2368604832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a000cdb0 con 0x7f02a8078790 2024-09-10T07:36:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a81a6400 con 0x7f02a8078790 2024-09-10T07:36:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02a5600640 1 -- 172.21.15.46:0/2368604832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a0005020 con 0x7f02a8078790 2024-09-10T07:36:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02a5600640 1 -- 172.21.15.46:0/2368604832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a00116b0 con 0x7f02a8078790 2024-09-10T07:36:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.073+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a81a6910 con 0x7f02a8078790 2024-09-10T07:36:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.075+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a8110780 con 0x7f02a8078790 2024-09-10T07:36:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.076+0000 7f02a5600640 1 -- 172.21.15.46:0/2368604832 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02a0005560 con 0x7f02a8078790 2024-09-10T07:36:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.078+0000 7f02a5600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02900a2bb0 0x7f02900a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.078+0000 7f02a5600640 1 -- 172.21.15.46:0/2368604832 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f02a00cf5a0 con 0x7f02a8078790 2024-09-10T07:36:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.078+0000 7f02acc00640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02900a2bb0 0x7f02900a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:00.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.079+0000 7f02acc00640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02900a2bb0 0x7f02900a5070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f029c005fd0 tx=0x7f029c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.082+0000 7f02a5600640 1 -- 172.21.15.46:0/2368604832 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02a0016030 con 0x7f02a8078790 2024-09-10T07:36:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.274+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f02a8062630 con 0x7f02900a2bb0 2024-09-10T07:36:00.275 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:00 smithi046 ceph-mon[30705]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T07:36:00.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.280+0000 7f02a5600640 1 -- 172.21.15.46:0/2368604832 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f02a8062630 con 0x7f02900a2bb0 2024-09-10T07:36:00.281 INFO:teuthology.orchestra.run.smithi046.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T07:36:00.281 INFO:teuthology.orchestra.run.smithi046.stdout:mds.all 12/12 23s ago 3m smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi063=b;smithi063=e;smithi063=h;smithi063=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-09-10T07:36:00.281 INFO:teuthology.orchestra.run.smithi046.stdout:mgr 3/3 23s ago 8m smithi046=x;smithi063=y;smithi181=z;count:3 2024-09-10T07:36:00.282 INFO:teuthology.orchestra.run.smithi046.stdout:mon 3/3 23s ago 9m smithi046:172.21.15.46=a;smithi063:172.21.15.63=b;smithi181:172.21.15.181=c;count:3 2024-09-10T07:36:00.282 INFO:teuthology.orchestra.run.smithi046.stdout:osd 12 23s ago - 2024-09-10T07:36:00.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02900a2bb0 msgr2=0x7f02900a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:00.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02900a2bb0 0x7f02900a5070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f029c005fd0 tx=0x7f029c00b390 comp rx=0 tx=0).stop 2024-09-10T07:36:00.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8078790 msgr2=0x7f02a81a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:00.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8078790 0x7f02a81a5aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02a000eaa0 tx=0x7f02a000ef60 comp rx=0 tx=0).stop 2024-09-10T07:36:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 shutdown_connections 2024-09-10T07:36:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f02900a2bb0 0x7f02900a5070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8078790 0x7f02a81a5aa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a8077df0 0x7f02a819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 --2- 172.21.15.46:0/2368604832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a8076ba0 0x7f02a819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 >> 172.21.15.46:0/2368604832 conn(0x7f02a80ff5b0 msgr2=0x7f02a8101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 shutdown_connections 2024-09-10T07:36:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:00.283+0000 7f02af600640 1 -- 172.21.15.46:0/2368604832 wait complete. 2024-09-10T07:36:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:00 smithi181 ceph-mon[35397]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T07:36:00.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:00 smithi063 ceph-mon[35020]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T07:36:01.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T07:36:01.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:01 smithi181 ceph-mon[35397]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:01.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:01 smithi063 ceph-mon[35020]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:01.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:01 smithi046 ceph-mon[30705]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:01.484 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:02.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 -- 172.21.15.46:0/1718017373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 msgr2=0x7f7688074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:02.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 --2- 172.21.15.46:0/1718017373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 0x7f7688074620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7678009900 tx=0x7f767802a480 comp rx=0 tx=0).stop 2024-09-10T07:36:02.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 -- 172.21.15.46:0/1718017373 shutdown_connections 2024-09-10T07:36:02.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 --2- 172.21.15.46:0/1718017373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76881098e0 0x7f768810bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 --2- 172.21.15.46:0/1718017373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7688074bf0 0x7f7688075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 --2- 172.21.15.46:0/1718017373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 0x7f7688074620 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 -- 172.21.15.46:0/1718017373 >> 172.21.15.46:0/1718017373 conn(0x7f76880fd1f0 msgr2=0x7f76880ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 -- 172.21.15.46:0/1718017373 shutdown_connections 2024-09-10T07:36:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.106+0000 7f7687e00640 1 -- 172.21.15.46:0/1718017373 wait complete. 2024-09-10T07:36:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.108+0000 7f7687e00640 1 Processor -- start 2024-09-10T07:36:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.108+0000 7f7687e00640 1 -- start start 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.108+0000 7f7687e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7688074bf0 0x7f7688199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.108+0000 7f7687e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 0x7f768819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.108+0000 7f7687e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76881098e0 0x7f76881a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.108+0000 7f7687e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f768810f8c0 con 0x7f76880761c0 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.108+0000 7f7687e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f768810f740 con 0x7f76881098e0 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.108+0000 7f7687e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f768810fa40 con 0x7f7688074bf0 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f767fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 0x7f768819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f7686a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7688074bf0 0x7f7688199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f767fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 0x7f768819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33498/0 (socket says 172.21.15.46:33498) 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f767fe00640 1 -- 172.21.15.46:0/2423859742 learned_addr learned my addr 172.21.15.46:0/2423859742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f7687400640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76881098e0 0x7f76881a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f767fe00640 1 -- 172.21.15.46:0/2423859742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7688074bf0 msgr2=0x7f7688199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f767fe00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7688074bf0 0x7f7688199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f767fe00640 1 -- 172.21.15.46:0/2423859742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76881098e0 msgr2=0x7f76881a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f767fe00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76881098e0 0x7f76881a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f767fe00640 1 -- 172.21.15.46:0/2423859742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76881a1b70 con 0x7f76880761c0 2024-09-10T07:36:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.109+0000 7f7687400640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76881098e0 0x7f76881a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.110+0000 7f767fe00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 0x7f768819a480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f767000e9f0 tx=0x7f767000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.110+0000 7f768ca00640 1 -- 172.21.15.46:0/2423859742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767000cd80 con 0x7f76880761c0 2024-09-10T07:36:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.110+0000 7f768ca00640 1 -- 172.21.15.46:0/2423859742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7670010910 con 0x7f76880761c0 2024-09-10T07:36:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.110+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76881a1e60 con 0x7f76880761c0 2024-09-10T07:36:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.110+0000 7f768ca00640 1 -- 172.21.15.46:0/2423859742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670018990 con 0x7f76880761c0 2024-09-10T07:36:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.110+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76881a2310 con 0x7f76880761c0 2024-09-10T07:36:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.111+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76881086a0 con 0x7f76880761c0 2024-09-10T07:36:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.113+0000 7f768ca00640 1 -- 172.21.15.46:0/2423859742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7670010ac0 con 0x7f76880761c0 2024-09-10T07:36:02.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.114+0000 7f768ca00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f766c0a2bb0 0x7f766c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:02.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.115+0000 7f768ca00640 1 -- 172.21.15.46:0/2423859742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f76700cfab0 con 0x7f76880761c0 2024-09-10T07:36:02.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.115+0000 7f7686a00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f766c0a2bb0 0x7f766c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.115+0000 7f7686a00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f766c0a2bb0 0x7f766c0a5070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76780314e0 tx=0x7f7678000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.117+0000 7f768ca00640 1 -- 172.21.15.46:0/2423859742 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f767009bfe0 con 0x7f76880761c0 2024-09-10T07:36:02.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.307+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7688062630 con 0x7f766c0a2bb0 2024-09-10T07:36:02.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:02 smithi046 ceph-mon[30705]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.309+0000 7f768ca00640 1 -- 172.21.15.46:0/2423859742 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7688062630 con 0x7f766c0a2bb0 2024-09-10T07:36:02.309 INFO:teuthology.orchestra.run.smithi046.stdout:HOST ADDR LABELS STATUS 2024-09-10T07:36:02.309 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 172.21.15.46 2024-09-10T07:36:02.310 INFO:teuthology.orchestra.run.smithi046.stdout:smithi063 172.21.15.63 2024-09-10T07:36:02.310 INFO:teuthology.orchestra.run.smithi046.stdout:smithi181 172.21.15.181 2024-09-10T07:36:02.310 INFO:teuthology.orchestra.run.smithi046.stdout:3 hosts in cluster 2024-09-10T07:36:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f766c0a2bb0 msgr2=0x7f766c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f766c0a2bb0 0x7f766c0a5070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76780314e0 tx=0x7f7678000bc0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 msgr2=0x7f768819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 0x7f768819a480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f767000e9f0 tx=0x7f767000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 shutdown_connections 2024-09-10T07:36:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76881098e0 0x7f76881a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f766c0a2bb0 0x7f766c0a5070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76880761c0 0x7f768819a480 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 --2- 172.21.15.46:0/2423859742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7688074bf0 0x7f7688199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.311+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 >> 172.21.15.46:0/2423859742 conn(0x7f76880fd1f0 msgr2=0x7f7688105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.312+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 shutdown_connections 2024-09-10T07:36:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:02.312+0000 7f7687e00640 1 -- 172.21.15.46:0/2423859742 wait complete. 2024-09-10T07:36:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:02 smithi181 ceph-mon[35397]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:02.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:02 smithi063 ceph-mon[35020]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:03.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T07:36:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:03 smithi181 ceph-mon[35397]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:03.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:03 smithi063 ceph-mon[35020]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:03.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:03 smithi046 ceph-mon[30705]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.052+0000 7fa3b0e00640 1 -- 172.21.15.46:0/779269417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3ac101a40 msgr2=0x7fa3ac101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.052+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/779269417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3ac101a40 0x7fa3ac101e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009900 tx=0x7fa39c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.052+0000 7fa3b0e00640 1 -- 172.21.15.46:0/779269417 shutdown_connections 2024-09-10T07:36:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.052+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/779269417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ac103600 0x7fa3ac109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.052+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/779269417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac102c40 0x7fa3ac1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.052+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/779269417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3ac101a40 0x7fa3ac101e40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.052+0000 7fa3b0e00640 1 -- 172.21.15.46:0/779269417 >> 172.21.15.46:0/779269417 conn(0x7fa3ac0fd1f0 msgr2=0x7fa3ac0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.052+0000 7fa3b0e00640 1 -- 172.21.15.46:0/779269417 shutdown_connections 2024-09-10T07:36:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.053+0000 7fa3b0e00640 1 -- 172.21.15.46:0/779269417 wait complete. 2024-09-10T07:36:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.053+0000 7fa3b0e00640 1 Processor -- start 2024-09-10T07:36:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3b0e00640 1 -- start start 2024-09-10T07:36:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3b0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3ac101a40 0x7fa3ac197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3b0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac102c40 0x7fa3ac198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3b0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ac103600 0x7fa3ac19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3b0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac10d6a0 con 0x7fa3ac102c40 2024-09-10T07:36:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3b0e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac10d520 con 0x7fa3ac101a40 2024-09-10T07:36:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3b0e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac10d820 con 0x7fa3ac103600 2024-09-10T07:36:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac102c40 0x7fa3ac198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac102c40 0x7fa3ac198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33510/0 (socket says 172.21.15.46:33510) 2024-09-10T07:36:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3aaa00640 1 -- 172.21.15.46:0/185008982 learned_addr learned my addr 172.21.15.46:0/185008982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3abe00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ac103600 0x7fa3ac19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.054+0000 7fa3ab400640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3ac101a40 0x7fa3ac197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3aaa00640 1 -- 172.21.15.46:0/185008982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ac103600 msgr2=0x7fa3ac19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3aaa00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ac103600 0x7fa3ac19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3aaa00640 1 -- 172.21.15.46:0/185008982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3ac101a40 msgr2=0x7fa3ac197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3aaa00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3ac101a40 0x7fa3ac197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3aaa00640 1 -- 172.21.15.46:0/185008982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac19f980 con 0x7fa3ac102c40 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3aaa00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac102c40 0x7fa3ac198290 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa39800e9f0 tx=0x7fa39800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3a3e00640 1 -- 172.21.15.46:0/185008982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39800cd80 con 0x7fa3ac102c40 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3a3e00640 1 -- 172.21.15.46:0/185008982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa398010910 con 0x7fa3ac102c40 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3a3e00640 1 -- 172.21.15.46:0/185008982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398018990 con 0x7fa3ac102c40 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.055+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac19fc70 con 0x7fa3ac102c40 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.056+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac1a01b0 con 0x7fa3ac102c40 2024-09-10T07:36:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.057+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ac108eb0 con 0x7fa3ac102c40 2024-09-10T07:36:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.059+0000 7fa3a3e00640 1 -- 172.21.15.46:0/185008982 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa398010ac0 con 0x7fa3ac102c40 2024-09-10T07:36:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.060+0000 7fa3a3e00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3900a2c30 0x7fa3900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.060+0000 7fa3a3e00640 1 -- 172.21.15.46:0/185008982 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa3980cf630 con 0x7fa3ac102c40 2024-09-10T07:36:04.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.060+0000 7fa3ab400640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3900a2c30 0x7fa3900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:04.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.061+0000 7fa3ab400640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3900a2c30 0x7fa3900a50f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005ab0 tx=0x7fa39c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.064+0000 7fa3a3e00640 1 -- 172.21.15.46:0/185008982 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa39809bb60 con 0x7fa3ac102c40 2024-09-10T07:36:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.257+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa3ac062630 con 0x7fa3900a2c30 2024-09-10T07:36:04.258 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:04 smithi046 ceph-mon[30705]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.258+0000 7fa3a3e00640 1 -- 172.21.15.46:0/185008982 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa3ac062630 con 0x7fa3900a2c30 2024-09-10T07:36:04.259 INFO:teuthology.orchestra.run.smithi046.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T07:36:04.259 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-09-10T07:36:04.259 INFO:teuthology.orchestra.run.smithi046.stdout:smithi063 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5344005B400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T07:36:04.259 INFO:teuthology.orchestra.run.smithi046.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T07:36:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3900a2c30 msgr2=0x7fa3900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3900a2c30 0x7fa3900a50f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005ab0 tx=0x7fa39c000bc0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac102c40 msgr2=0x7fa3ac198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac102c40 0x7fa3ac198290 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa39800e9f0 tx=0x7fa39800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 shutdown_connections 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3ac103600 0x7fa3ac19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3900a2c30 0x7fa3900a50f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac102c40 0x7fa3ac198290 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 --2- 172.21.15.46:0/185008982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3ac101a40 0x7fa3ac197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 >> 172.21.15.46:0/185008982 conn(0x7fa3ac0fd1f0 msgr2=0x7fa3ac106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 shutdown_connections 2024-09-10T07:36:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:04.261+0000 7fa3b0e00640 1 -- 172.21.15.46:0/185008982 wait complete. 2024-09-10T07:36:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:04 smithi181 ceph-mon[35397]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:04 smithi063 ceph-mon[35020]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:05.037 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T07:36:05.046 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:36:05.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T07:36:05.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:05 smithi181 ceph-mon[35397]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:05.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:05 smithi063 ceph-mon[35020]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:05.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:05 smithi046 ceph-mon[30705]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:36:05.424 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 -- 172.21.15.46:0/647580669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4078790 msgr2=0x7fb7b4079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/647580669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4078790 0x7fb7b4079030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009ca0 tx=0x7fb7ac02a8b0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 -- 172.21.15.46:0/647580669 shutdown_connections 2024-09-10T07:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/647580669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4078790 0x7fb7b4079030 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/647580669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4077df0 0x7fb7b4078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/647580669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4076ba0 0x7fb7b4076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 -- 172.21.15.46:0/647580669 >> 172.21.15.46:0/647580669 conn(0x7fb7b40ff3e0 msgr2=0x7fb7b4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 -- 172.21.15.46:0/647580669 shutdown_connections 2024-09-10T07:36:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.025+0000 7fb7b8a00640 1 -- 172.21.15.46:0/647580669 wait complete. 2024-09-10T07:36:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.026+0000 7fb7b8a00640 1 Processor -- start 2024-09-10T07:36:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.026+0000 7fb7b8a00640 1 -- start start 2024-09-10T07:36:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.026+0000 7fb7b8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4076ba0 0x7fb7b419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.026+0000 7fb7b8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4077df0 0x7fb7b419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.026+0000 7fb7b8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4078790 0x7fb7b41a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4113cf0 con 0x7fb7b4077df0 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b8a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4113b70 con 0x7fb7b4076ba0 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b8a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4113e70 con 0x7fb7b4078790 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4076ba0 0x7fb7b419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4076ba0 0x7fb7b419e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38434/0 (socket says 172.21.15.46:38434) 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b2800640 1 -- 172.21.15.46:0/2773292262 learned_addr learned my addr 172.21.15.46:0/2773292262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b1e00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4077df0 0x7fb7b419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b3200640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4078790 0x7fb7b41a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b2800640 1 -- 172.21.15.46:0/2773292262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4078790 msgr2=0x7fb7b41a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b2800640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4078790 0x7fb7b41a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b2800640 1 -- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4077df0 msgr2=0x7fb7b419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b2800640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4077df0 0x7fb7b419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.027+0000 7fb7b2800640 1 -- 172.21.15.46:0/2773292262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b41a5ee0 con 0x7fb7b4076ba0 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.028+0000 7fb7b2800640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4076ba0 0x7fb7b419e2b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e9f0 tx=0x7fb7a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.028+0000 7fb7ab400640 1 -- 172.21.15.46:0/2773292262 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400cd80 con 0x7fb7b4076ba0 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.028+0000 7fb7ab400640 1 -- 172.21.15.46:0/2773292262 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4010910 con 0x7fb7b4076ba0 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.028+0000 7fb7ab400640 1 -- 172.21.15.46:0/2773292262 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4018990 con 0x7fb7b4076ba0 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.028+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b41a61d0 con 0x7fb7b4076ba0 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.028+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b41a66c0 con 0x7fb7b4076ba0 2024-09-10T07:36:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.029+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b41105d0 con 0x7fb7b4076ba0 2024-09-10T07:36:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.031+0000 7fb7ab400640 1 -- 172.21.15.46:0/2773292262 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb7a4010ac0 con 0x7fb7b4076ba0 2024-09-10T07:36:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.032+0000 7fb7ab400640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7980a2bb0 0x7fb7980a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.033+0000 7fb7ab400640 1 -- 172.21.15.46:0/2773292262 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb7a40cfaf0 con 0x7fb7b4076ba0 2024-09-10T07:36:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.033+0000 7fb7b1e00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7980a2bb0 0x7fb7980a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.033+0000 7fb7b1e00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7980a2bb0 0x7fb7980a5070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00095f0 tx=0x7fb7a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.035+0000 7fb7ab400640 1 -- 172.21.15.46:0/2773292262 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7a409c020 con 0x7fb7b4076ba0 2024-09-10T07:36:06.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.255+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb7b402da70 con 0x7fb7b4076ba0 2024-09-10T07:36:06.256 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:06 smithi046 ceph-mon[30705]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:06.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.255+0000 7fb7ab400640 1 -- 172.21.15.46:0/2773292262 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3128 (secure 0 0 0) 0x7fb7a40a0320 con 0x7fb7b4076ba0 2024-09-10T07:36:06.256 INFO:teuthology.orchestra.run.smithi046.stdout:e29 2024-09-10T07:36:06.256 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T07:34:28:805113+0000 2024-09-10T07:36:06.256 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 29 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T07:33:36.040805+0000 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T07:34:27.875261+0000 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 300 2024-09-10T07:36:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 5 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1,2,3,4 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=24535,1=24530,2=24517,3=24499,4=24448} 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T07:36:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3,4] 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data disabled 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 0 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 24535 members: 24448,24499,24517,24530,24535 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a{0:24535} state up:active seq 6 addr [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.h{0:24494} state up:standby-replay seq 1 addr [v2:172.21.15.63:6836/4195549127,v1:172.21.15.63:6837/4195549127] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b{1:24530} state up:active seq 17 addr [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f{1:24493} state up:standby-replay seq 1 addr [v2:172.21.15.181:6834/335913193,v1:172.21.15.181:6835/335913193] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.i{2:24517} state up:active seq 19 addr [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e{2:24473} state up:standby-replay seq 1 addr [v2:172.21.15.63:6834/1645722263,v1:172.21.15.63:6835/1645722263] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c{3:24499} state up:active seq 21 addr [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.259 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.j{3:24449} state up:standby-replay seq 1 addr [v2:172.21.15.46:6834/4121221095,v1:172.21.15.46:6835/4121221095] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.260 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.l{4:24448} state up:active seq 26 addr [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.260 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.k{4:24437} state up:standby-replay seq 1 addr [v2:172.21.15.63:6832/620318721,v1:172.21.15.63:6833/620318721] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.260 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:06.260 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:06.260 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T07:36:06.260 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:06.260 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d{-1:14439} state up:standby seq 1 addr [v2:172.21.15.46:6836/2449149651,v1:172.21.15.46:6837/2449149651] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.260 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.g{-1:14472} state up:standby seq 1 addr [v2:172.21.15.46:6838/1424921396,v1:172.21.15.46:6839/1424921396] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7980a2bb0 msgr2=0x7fb7980a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7980a2bb0 0x7fb7980a5070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00095f0 tx=0x7fb7a0005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4076ba0 msgr2=0x7fb7b419e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4076ba0 0x7fb7b419e2b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e9f0 tx=0x7fb7a400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 shutdown_connections 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4078790 0x7fb7b41a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb7980a2bb0 0x7fb7980a5070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7b4077df0 0x7fb7b419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 --2- 172.21.15.46:0/2773292262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b4076ba0 0x7fb7b419e2b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 >> 172.21.15.46:0/2773292262 conn(0x7fb7b40ff3e0 msgr2=0x7fb7b4100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 shutdown_connections 2024-09-10T07:36:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:06.258+0000 7fb7b8a00640 1 -- 172.21.15.46:0/2773292262 wait complete. 2024-09-10T07:36:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:06 smithi181 ceph-mon[35397]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:06 smithi063 ceph-mon[35020]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:07.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T07:36:07.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:07 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2773292262' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T07:36:07.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:07 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2773292262' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T07:36:07.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:07 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2773292262' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T07:36:07.456 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.091+0000 7fcf7a800640 1 -- 172.21.15.46:0/573095085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74102e00 msgr2=0x7fcf74103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.091+0000 7fcf7a800640 1 --2- 172.21.15.46:0/573095085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74102e00 0x7fcf74103280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009900 tx=0x7fcf5c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.091+0000 7fcf7a800640 1 -- 172.21.15.46:0/573095085 shutdown_connections 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.091+0000 7fcf7a800640 1 --2- 172.21.15.46:0/573095085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf741037c0 0x7fcf7410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.091+0000 7fcf7a800640 1 --2- 172.21.15.46:0/573095085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74102e00 0x7fcf74103280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.091+0000 7fcf7a800640 1 --2- 172.21.15.46:0/573095085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf74101c00 0x7fcf74102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.091+0000 7fcf7a800640 1 -- 172.21.15.46:0/573095085 >> 172.21.15.46:0/573095085 conn(0x7fcf740fd390 msgr2=0x7fcf740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.091+0000 7fcf7a800640 1 -- 172.21.15.46:0/573095085 shutdown_connections 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.092+0000 7fcf7a800640 1 -- 172.21.15.46:0/573095085 wait complete. 2024-09-10T07:36:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.093+0000 7fcf7a800640 1 Processor -- start 2024-09-10T07:36:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.093+0000 7fcf7a800640 1 -- start start 2024-09-10T07:36:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.093+0000 7fcf7a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf74101c00 0x7fcf7419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.093+0000 7fcf7a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf74102e00 0x7fcf7419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.093+0000 7fcf7a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf741037c0 0x7fcf741a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.093+0000 7fcf7a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410d880 con 0x7fcf741037c0 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.093+0000 7fcf7a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410d700 con 0x7fcf74102e00 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.093+0000 7fcf7a800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410da00 con 0x7fcf74101c00 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf741037c0 0x7fcf741a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf6be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf74102e00 0x7fcf7419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf6be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf74102e00 0x7fcf7419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38462/0 (socket says 172.21.15.46:38462) 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf6be00640 1 -- 172.21.15.46:0/1656441002 learned_addr learned my addr 172.21.15.46:0/1656441002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf73400640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf74101c00 0x7fcf7419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf6be00640 1 -- 172.21.15.46:0/1656441002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf74101c00 msgr2=0x7fcf7419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf6be00640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf74101c00 0x7fcf7419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf6be00640 1 -- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf741037c0 msgr2=0x7fcf741a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf6be00640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf741037c0 0x7fcf741a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf6be00640 1 -- 172.21.15.46:0/1656441002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf741a1d10 con 0x7fcf74102e00 2024-09-10T07:36:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.094+0000 7fcf73e00640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf741037c0 0x7fcf741a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:08.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.095+0000 7fcf6be00640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf74102e00 0x7fcf7419a620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009230 tx=0x7fcf5c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.095+0000 7fcf70c00640 1 -- 172.21.15.46:0/1656441002 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c03f030 con 0x7fcf74102e00 2024-09-10T07:36:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.095+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf741a1fa0 con 0x7fcf74102e00 2024-09-10T07:36:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.095+0000 7fcf70c00640 1 -- 172.21.15.46:0/1656441002 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c003ce0 con 0x7fcf74102e00 2024-09-10T07:36:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.095+0000 7fcf70c00640 1 -- 172.21.15.46:0/1656441002 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c035070 con 0x7fcf74102e00 2024-09-10T07:36:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.095+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf741a2400 con 0x7fcf74102e00 2024-09-10T07:36:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.096+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf741a2630 con 0x7fcf74102e00 2024-09-10T07:36:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.098+0000 7fcf70c00640 1 -- 172.21.15.46:0/1656441002 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf5c042030 con 0x7fcf74102e00 2024-09-10T07:36:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.099+0000 7fcf70c00640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf580a2c30 0x7fcf580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.099+0000 7fcf70c00640 1 -- 172.21.15.46:0/1656441002 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcf5c0ef7e0 con 0x7fcf74102e00 2024-09-10T07:36:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.099+0000 7fcf73400640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf580a2c30 0x7fcf580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.100+0000 7fcf73400640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf580a2c30 0x7fcf580a50f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcf74102c60 tx=0x7fcf64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:08.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.103+0000 7fcf70c00640 1 -- 172.21.15.46:0/1656441002 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf5c0bbd10 con 0x7fcf74102e00 2024-09-10T07:36:08.104 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:08 smithi046 ceph-mon[30705]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.289+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fcf741a29c0 con 0x7fcf74102e00 2024-09-10T07:36:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.290+0000 7fcf70c00640 1 -- 172.21.15.46:0/1656441002 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v97) ==== 56+0+5594 (secure 0 0 0) 0x7fcf5c0c0010 con 0x7fcf74102e00 2024-09-10T07:36:08.291 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 97 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:fsid b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T07:25:19.596849+0000 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T07:35:14.826944+0000 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:crush_version 27 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:full_ratio 0.95 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:backfillfull_ratio 0.9 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:nearfull_ratio 0.85 2024-09-10T07:36:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:require_min_compat_client luminous 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:min_compat_client luminous 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:require_osd_release squid 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:stretch_mode_enabled false 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 93 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:max_osd 12 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 up in weight 1 up_from 8 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6802/12452162,v1:172.21.15.46:6803/12452162] [v2:172.21.15.46:6804/12452162,v1:172.21.15.46:6805/12452162] exists,up 0358dc9f-4491-4ec2-9e41-9b621d65a460 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6800/1918625844,v1:172.21.15.63:6801/1918625844] [v2:172.21.15.63:6802/1918625844,v1:172.21.15.63:6803/1918625844] exists,up e0f0a426-666d-41c0-a0e2-73abbd2f08ef 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/413292109,v1:172.21.15.181:6801/413292109] [v2:172.21.15.181:6802/413292109,v1:172.21.15.181:6803/413292109] exists,up 16642ece-80e2-4852-a809-8a4f57c5c16d 2024-09-10T07:36:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 up in weight 1 up_from 25 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/3016450690,v1:172.21.15.46:6811/3016450690] [v2:172.21.15.46:6812/3016450690,v1:172.21.15.46:6813/3016450690] exists,up d8019d3f-952b-42df-9c70-c0574211eaca 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 up in weight 1 up_from 32 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6808/402938834,v1:172.21.15.63:6809/402938834] [v2:172.21.15.63:6810/402938834,v1:172.21.15.63:6811/402938834] exists,up fe349fc9-2de4-439f-a982-e9282f26d488 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 up in weight 1 up_from 37 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/120236836,v1:172.21.15.181:6809/120236836] [v2:172.21.15.181:6810/120236836,v1:172.21.15.181:6811/120236836] exists,up 619c621c-ab3a-40fc-bdd4-498347c3a5fc 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 up in weight 1 up_from 45 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6818/4145991775,v1:172.21.15.46:6819/4145991775] [v2:172.21.15.46:6820/4145991775,v1:172.21.15.46:6821/4145991775] exists,up 9bd4a464-96a7-4436-bc64-809acbdf2041 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 up in weight 1 up_from 50 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6816/1986919229,v1:172.21.15.63:6817/1986919229] [v2:172.21.15.63:6818/1986919229,v1:172.21.15.63:6819/1986919229] exists,up 32078aa6-503d-49ed-b587-2a840b513145 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 up in weight 1 up_from 57 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/296261223,v1:172.21.15.181:6817/296261223] [v2:172.21.15.181:6818/296261223,v1:172.21.15.181:6819/296261223] exists,up 36b80e03-cf0c-41b5-bad6-2021b07c92ca 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 up in weight 1 up_from 65 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6826/805007364,v1:172.21.15.46:6827/805007364] [v2:172.21.15.46:6828/805007364,v1:172.21.15.46:6829/805007364] exists,up 5f9edf7d-f94e-4913-9928-5b6c4d3c724e 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 up in weight 1 up_from 70 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6824/2494224828,v1:172.21.15.63:6825/2494224828] [v2:172.21.15.63:6826/2494224828,v1:172.21.15.63:6827/2494224828] exists,up fc6d6ef6-c067-495e-a673-88cbab957e7e 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 up in weight 1 up_from 78 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/1815305110,v1:172.21.15.181:6825/1815305110] [v2:172.21.15.181:6826/1815305110,v1:172.21.15.181:6827/1815305110] exists,up 7a4826cd-7a7a-4a20-9af7-734e74374081 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.5 [8,11] 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.13 [2,11] 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.0 [4,8] 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.5 [3,9] 2024-09-10T07:36:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1b [5,9] 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1d [4,6] 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.2c [4,9] 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.6 [9,11] 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.b [9,8] 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.e [5,8] 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.14 [9,8] 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/2972952762 expires 2024-09-11T07:26:12.367178+0000 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/2972952762 expires 2024-09-11T07:26:12.367178+0000 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1860671836 expires 2024-09-11T07:26:12.367178+0000 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/3724700252 expires 2024-09-11T07:26:12.367178+0000 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/3653813867 expires 2024-09-11T07:26:12.367178+0000 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/141501936 expires 2024-09-11T07:25:44.780814+0000 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/3003000819 expires 2024-09-11T07:25:44.780814+0000 2024-09-10T07:36:08.295 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/141501936 expires 2024-09-11T07:25:44.780814+0000 2024-09-10T07:36:08.296 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1462945451 expires 2024-09-11T07:25:44.780814+0000 2024-09-10T07:36:08.296 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/3598352774 expires 2024-09-11T07:25:44.780814+0000 2024-09-10T07:36:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.292+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf580a2c30 msgr2=0x7fcf580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.292+0000 7fcf7a800640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf580a2c30 0x7fcf580a50f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcf74102c60 tx=0x7fcf64005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.292+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf74102e00 msgr2=0x7fcf7419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.292+0000 7fcf7a800640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf74102e00 0x7fcf7419a620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009230 tx=0x7fcf5c0042f0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.293+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 shutdown_connections 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.293+0000 7fcf7a800640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf580a2c30 0x7fcf580a50f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.293+0000 7fcf7a800640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf741037c0 0x7fcf741a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.293+0000 7fcf7a800640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf74102e00 0x7fcf7419a620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.293+0000 7fcf7a800640 1 --2- 172.21.15.46:0/1656441002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf74101c00 0x7fcf7419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.293+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 >> 172.21.15.46:0/1656441002 conn(0x7fcf740fd390 msgr2=0x7fcf741085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.293+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 shutdown_connections 2024-09-10T07:36:08.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:08.293+0000 7fcf7a800640 1 -- 172.21.15.46:0/1656441002 wait complete. 2024-09-10T07:36:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:08 smithi181 ceph-mon[35397]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:08 smithi063 ceph-mon[35020]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:09.162 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T07:36:09.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:36:09.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:09 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1656441002' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T07:36:09.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:09 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1656441002' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T07:36:09.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:09 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1656441002' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T07:36:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.138+0000 7f6f74e00640 1 -- 172.21.15.46:0/1598177737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70103ff0 msgr2=0x7f6f701043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.138+0000 7f6f74e00640 1 --2- 172.21.15.46:0/1598177737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70103ff0 0x7f6f701043d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009ca0 tx=0x7f6f6002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.138+0000 7f6f74e00640 1 -- 172.21.15.46:0/1598177737 shutdown_connections 2024-09-10T07:36:10.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.138+0000 7f6f74e00640 1 --2- 172.21.15.46:0/1598177737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70109120 0x7f6f70110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.138+0000 7f6f74e00640 1 --2- 172.21.15.46:0/1598177737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701049a0 0x7f6f701089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.138+0000 7f6f74e00640 1 --2- 172.21.15.46:0/1598177737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70103ff0 0x7f6f701043d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.138+0000 7f6f74e00640 1 -- 172.21.15.46:0/1598177737 >> 172.21.15.46:0/1598177737 conn(0x7f6f700ffa90 msgr2=0x7f6f70101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:10.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.138+0000 7f6f74e00640 1 -- 172.21.15.46:0/1598177737 shutdown_connections 2024-09-10T07:36:10.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.139+0000 7f6f74e00640 1 -- 172.21.15.46:0/1598177737 wait complete. 2024-09-10T07:36:10.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f74e00640 1 Processor -- start 2024-09-10T07:36:10.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f74e00640 1 -- start start 2024-09-10T07:36:10.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f74e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70103ff0 0x7f6f7019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:10.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f74e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701049a0 0x7f6f7019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:10.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f74e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70109120 0x7f6f7019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:10.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f74e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701141a0 con 0x7f6f70109120 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f74e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70114020 con 0x7f6f701049a0 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f74e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70114320 con 0x7f6f70103ff0 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.140+0000 7f6f6e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70109120 0x7f6f7019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f6e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70109120 0x7f6f7019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52908/0 (socket says 172.21.15.46:52908) 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f65400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701049a0 0x7f6f7019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f6e800640 1 -- 172.21.15.46:0/2233849442 learned_addr learned my addr 172.21.15.46:0/2233849442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f6de00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70103ff0 0x7f6f7019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f65400640 1 -- 172.21.15.46:0/2233849442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70103ff0 msgr2=0x7f6f7019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f65400640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70103ff0 0x7f6f7019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f65400640 1 -- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70109120 msgr2=0x7f6f7019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f65400640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70109120 0x7f6f7019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f65400640 1 -- 172.21.15.46:0/2233849442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7019f5d0 con 0x7f6f701049a0 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.141+0000 7f6f65400640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701049a0 0x7f6f7019ab60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800e9f0 tx=0x7f6f5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.142+0000 7f6f67400640 1 -- 172.21.15.46:0/2233849442 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800cd80 con 0x7f6f701049a0 2024-09-10T07:36:10.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.142+0000 7f6f67400640 1 -- 172.21.15.46:0/2233849442 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f580035f0 con 0x7f6f701049a0 2024-09-10T07:36:10.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.142+0000 7f6f67400640 1 -- 172.21.15.46:0/2233849442 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58010690 con 0x7f6f701049a0 2024-09-10T07:36:10.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.142+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7019f8c0 con 0x7f6f701049a0 2024-09-10T07:36:10.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.142+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f701a71a0 con 0x7f6f701049a0 2024-09-10T07:36:10.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.143+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f701043d0 con 0x7f6f701049a0 2024-09-10T07:36:10.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.145+0000 7f6f67400640 1 -- 172.21.15.46:0/2233849442 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f58003b30 con 0x7f6f701049a0 2024-09-10T07:36:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.146+0000 7f6f67400640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f540a2c00 0x7f6f540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:10.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.146+0000 7f6f67400640 1 -- 172.21.15.46:0/2233849442 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6f580cf6b0 con 0x7f6f701049a0 2024-09-10T07:36:10.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.146+0000 7f6f6de00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f540a2c00 0x7f6f540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:10.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.147+0000 7f6f6de00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f540a2c00 0x7f6f540a50c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006010 tx=0x7f6f60009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:10.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.150+0000 7f6f67400640 1 -- 172.21.15.46:0/2233849442 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f5809c230 con 0x7f6f701049a0 2024-09-10T07:36:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.369+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f7019b980 con 0x7f6f701049a0 2024-09-10T07:36:10.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:10 smithi046 ceph-mon[30705]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.370+0000 7f6f67400640 1 -- 172.21.15.46:0/2233849442 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11972 (secure 0 0 0) 0x7f6f5809bb30 con 0x7f6f701049a0 2024-09-10T07:36:10.371 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:10.371 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"btime":"2024-09-10T07:34:28:805113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:34:27.875261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:36:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T07:36:10.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f540a2c00 msgr2=0x7f6f540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f540a2c00 0x7f6f540a50c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006010 tx=0x7f6f60009820 comp rx=0 tx=0).stop 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701049a0 msgr2=0x7f6f7019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701049a0 0x7f6f7019ab60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800e9f0 tx=0x7f6f5800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 shutdown_connections 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f540a2c00 0x7f6f540a50c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70109120 0x7f6f7019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f701049a0 0x7f6f7019ab60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 --2- 172.21.15.46:0/2233849442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70103ff0 0x7f6f7019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 >> 172.21.15.46:0/2233849442 conn(0x7f6f700ffa90 msgr2=0x7f6f70100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 shutdown_connections 2024-09-10T07:36:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:10.373+0000 7f6f74e00640 1 -- 172.21.15.46:0/2233849442 wait complete. 2024-09-10T07:36:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:10 smithi181 ceph-mon[35397]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:10 smithi063 ceph-mon[35020]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:11.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:36:11.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:11 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2233849442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:11.428 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:11 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2233849442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:11.428 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:11 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2233849442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:11.551 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 -- 172.21.15.46:0/1647775015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 msgr2=0x7fada0106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 --2- 172.21.15.46:0/1647775015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 0x7fada0106840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad90009900 tx=0x7fad9002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 -- 172.21.15.46:0/1647775015 shutdown_connections 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 --2- 172.21.15.46:0/1647775015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0106f70 0x7fada010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 --2- 172.21.15.46:0/1647775015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 0x7fada0106840 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 --2- 172.21.15.46:0/1647775015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0100500 0x7fada01008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 -- 172.21.15.46:0/1647775015 >> 172.21.15.46:0/1647775015 conn(0x7fada00fc5c0 msgr2=0x7fada00fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 -- 172.21.15.46:0/1647775015 shutdown_connections 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.205+0000 7fada8800640 1 -- 172.21.15.46:0/1647775015 wait complete. 2024-09-10T07:36:12.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada8800640 1 Processor -- start 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada8800640 1 -- start start 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada8800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0100500 0x7fada01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 0x7fada01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada8800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0106f70 0x7fada019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada8800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fada0111f90 con 0x7fada0100eb0 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada8800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fada0111e10 con 0x7fada0106f70 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada8800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fada0112110 con 0x7fada0100500 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 0x7fada01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.206+0000 7fada5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 0x7fada01989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada5400640 1 -- 172.21.15.46:0/3966799031 learned_addr learned my addr 172.21.15.46:0/3966799031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada6800640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0106f70 0x7fada019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:12.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada5e00640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0100500 0x7fada01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada5400640 1 -- 172.21.15.46:0/3966799031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0100500 msgr2=0x7fada01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada5400640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0100500 0x7fada01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada5400640 1 -- 172.21.15.46:0/3966799031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0106f70 msgr2=0x7fada019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada5400640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0106f70 0x7fada019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada5400640 1 -- 172.21.15.46:0/3966799031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada019d460 con 0x7fada0100eb0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.207+0000 7fada5400640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 0x7fada01989f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad90005940 tx=0x7fad90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.208+0000 7fad9aa00640 1 -- 172.21.15.46:0/3966799031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9003f030 con 0x7fada0100eb0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.208+0000 7fad9aa00640 1 -- 172.21.15.46:0/3966799031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad90003ad0 con 0x7fada0100eb0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.208+0000 7fad9aa00640 1 -- 172.21.15.46:0/3966799031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90035070 con 0x7fada0100eb0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.208+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada019d6f0 con 0x7fada0100eb0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.208+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada01a5030 con 0x7fada0100eb0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.209+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada006a4a0 con 0x7fada0100eb0 2024-09-10T07:36:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.211+0000 7fad9aa00640 1 -- 172.21.15.46:0/3966799031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad90042030 con 0x7fada0100eb0 2024-09-10T07:36:12.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.212+0000 7fad9aa00640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad880a2b80 0x7fad880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:12.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.212+0000 7fad9aa00640 1 -- 172.21.15.46:0/3966799031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fad900efa80 con 0x7fada0100eb0 2024-09-10T07:36:12.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.212+0000 7fada5e00640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad880a2b80 0x7fad880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:12.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.213+0000 7fada5e00640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad880a2b80 0x7fad880a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad94005fd0 tx=0x7fad9400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:12.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.217+0000 7fad9aa00640 1 -- 172.21.15.46:0/3966799031 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad900f1e20 con 0x7fada0100eb0 2024-09-10T07:36:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:12 smithi181 ceph-mon[35397]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:12 smithi063 ceph-mon[35020]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:12.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:12 smithi046 ceph-mon[30705]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:12.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.439+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fada010a0e0 con 0x7fada0100eb0 2024-09-10T07:36:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.440+0000 7fad9aa00640 1 -- 172.21.15.46:0/3966799031 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11972 (secure 0 0 0) 0x7fad900bbfb0 con 0x7fada0100eb0 2024-09-10T07:36:12.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:12.441 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"btime":"2024-09-10T07:34:28:805113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:34:27.875261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:36:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T07:36:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.442+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad880a2b80 msgr2=0x7fad880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.442+0000 7fada8800640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad880a2b80 0x7fad880a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad94005fd0 tx=0x7fad9400b390 comp rx=0 tx=0).stop 2024-09-10T07:36:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.442+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 msgr2=0x7fada01989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.442+0000 7fada8800640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 0x7fada01989f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad90005940 tx=0x7fad90004270 comp rx=0 tx=0).stop 2024-09-10T07:36:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.443+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 shutdown_connections 2024-09-10T07:36:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.443+0000 7fada8800640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fada0106f70 0x7fada019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.443+0000 7fada8800640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad880a2b80 0x7fad880a5040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.443+0000 7fada8800640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada0100eb0 0x7fada01989f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.443+0000 7fada8800640 1 --2- 172.21.15.46:0/3966799031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0100500 0x7fada01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.443+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 >> 172.21.15.46:0/3966799031 conn(0x7fada00fc5c0 msgr2=0x7fada00fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.443+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 shutdown_connections 2024-09-10T07:36:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:12.443+0000 7fada8800640 1 -- 172.21.15.46:0/3966799031 wait complete. 2024-09-10T07:36:13.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:13 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3966799031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:13.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:13 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3966799031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:13.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:13 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3966799031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:13.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:36:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:14.357 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:14 smithi046 ceph-mon[30705]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:14 smithi181 ceph-mon[35397]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:14.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:14 smithi063 ceph-mon[35020]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 -- 172.21.15.46:0/903125592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064112a50 msgr2=0x7fb064114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 --2- 172.21.15.46:0/903125592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064112a50 0x7fb064114e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb060009ca0 tx=0x7fb06002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 -- 172.21.15.46:0/903125592 shutdown_connections 2024-09-10T07:36:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 --2- 172.21.15.46:0/903125592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064112a50 0x7fb064114e40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 --2- 172.21.15.46:0/903125592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb064077040 0x7fb064112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 --2- 172.21.15.46:0/903125592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb064076720 0x7fb064076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 -- 172.21.15.46:0/903125592 >> 172.21.15.46:0/903125592 conn(0x7fb0640ffa50 msgr2=0x7fb064101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 -- 172.21.15.46:0/903125592 shutdown_connections 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.471+0000 7fb06c800640 1 -- 172.21.15.46:0/903125592 wait complete. 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.472+0000 7fb06c800640 1 Processor -- start 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.472+0000 7fb06c800640 1 -- start start 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06c800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb064076720 0x7fb0641a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064077040 0x7fb0641a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06c800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb064112a50 0x7fb06419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb064118590 con 0x7fb064077040 2024-09-10T07:36:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06c800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb064118410 con 0x7fb064112a50 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06c800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb064118710 con 0x7fb064076720 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb069400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064077040 0x7fb0641a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb069e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb064076720 0x7fb0641a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb069400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064077040 0x7fb0641a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52934/0 (socket says 172.21.15.46:52934) 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb069400640 1 -- 172.21.15.46:0/3562512503 learned_addr learned my addr 172.21.15.46:0/3562512503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb064112a50 0x7fb06419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06a800640 1 -- 172.21.15.46:0/3562512503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb064076720 msgr2=0x7fb0641a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06a800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb064076720 0x7fb0641a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06a800640 1 -- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064077040 msgr2=0x7fb0641a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06a800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064077040 0x7fb0641a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.473+0000 7fb06a800640 1 -- 172.21.15.46:0/3562512503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06419f030 con 0x7fb064112a50 2024-09-10T07:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.474+0000 7fb06a800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb064112a50 0x7fb06419e900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb06002ade0 tx=0x7fb0600319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.474+0000 7fb05ea00640 1 -- 172.21.15.46:0/3562512503 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060003b70 con 0x7fb064112a50 2024-09-10T07:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.474+0000 7fb05ea00640 1 -- 172.21.15.46:0/3562512503 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb060003d10 con 0x7fb064112a50 2024-09-10T07:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.474+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06419f2c0 con 0x7fb064112a50 2024-09-10T07:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.474+0000 7fb05ea00640 1 -- 172.21.15.46:0/3562512503 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06002fc10 con 0x7fb064112a50 2024-09-10T07:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.474+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0641ab610 con 0x7fb064112a50 2024-09-10T07:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.476+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb064076b00 con 0x7fb064112a50 2024-09-10T07:36:14.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.477+0000 7fb05ea00640 1 -- 172.21.15.46:0/3562512503 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb060004050 con 0x7fb064112a50 2024-09-10T07:36:14.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.479+0000 7fb05ea00640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2b80 0x7fb04c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:14.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.479+0000 7fb05ea00640 1 -- 172.21.15.46:0/3562512503 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb0600f0350 con 0x7fb064112a50 2024-09-10T07:36:14.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.479+0000 7fb069e00640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2b80 0x7fb04c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:14.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.480+0000 7fb069e00640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2b80 0x7fb04c0a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb06419fa40 tx=0x7fb058005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:14.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.483+0000 7fb05ea00640 1 -- 172.21.15.46:0/3562512503 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0600bced0 con 0x7fb064112a50 2024-09-10T07:36:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.671+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb06410dbe0 con 0x7fb064112a50 2024-09-10T07:36:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.673+0000 7fb05ea00640 1 -- 172.21.15.46:0/3562512503 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24700 (secure 0 0 0) 0x7fb0600bc7d0 con 0x7fb064112a50 2024-09-10T07:36:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:14.674 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:35:14.826944+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:36:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2b80 msgr2=0x7fb04c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2b80 0x7fb04c0a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb06419fa40 tx=0x7fb058005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb064112a50 msgr2=0x7fb06419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb064112a50 0x7fb06419e900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb06002ade0 tx=0x7fb0600319f0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 shutdown_connections 2024-09-10T07:36:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb064112a50 0x7fb06419e900 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb04c0a2b80 0x7fb04c0a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064077040 0x7fb0641a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 --2- 172.21.15.46:0/3562512503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb064076720 0x7fb0641a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:14.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.675+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 >> 172.21.15.46:0/3562512503 conn(0x7fb0640ffa50 msgr2=0x7fb064101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:14.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.676+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 shutdown_connections 2024-09-10T07:36:14.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:14.676+0000 7fb06c800640 1 -- 172.21.15.46:0/3562512503 wait complete. 2024-09-10T07:36:15.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:15 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3562512503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:15.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:15 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3562512503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:15.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:15 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3562512503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:15.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T07:33:36.040805+0000', 'modified': '2024-09-10T07:34:27.875261+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24535, 'mds_1': 24530, 'mds_2': 24517, 'mds_3': 24499, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6833/620318721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6832', 'nonce': 620318721}, {'type': 'v1', 'addr': '172.21.15.63:6833', 'nonce': 620318721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.181:6833/2451989162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 2451989162}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 2451989162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/4121221095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 4121221095}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 4121221095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6835/1645722263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6834', 'nonce': 1645722263}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 1645722263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/335913193', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 335913193}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 335913193}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6837/4195549127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6836', 'nonce': 4195549127}, {'type': 'v1', 'addr': '172.21.15.63:6837', 'nonce': 4195549127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.181:6837/975679519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 975679519}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 975679519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.181:6839/1085095014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1085095014}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1085095014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.63:6839/1361023409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6838', 'nonce': 1361023409}, {'type': 'v1', 'addr': '172.21.15.63:6839', 'nonce': 1361023409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.46:6841/3146821117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6840', 'nonce': 3146821117}, {'type': 'v1', 'addr': '172.21.15.46:6841', 'nonce': 3146821117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24535, 'qdb_cluster': [24448, 24499, 24517, 24530, 24535]} 2024-09-10T07:36:15.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T07:36:15.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-10T07:36:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:16.348 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:16 smithi046 ceph-mon[30705]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:16 smithi181 ceph-mon[35397]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:16.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:16 smithi063 ceph-mon[35020]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:16.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 -- 172.21.15.46:0/3292397790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c103ff0 msgr2=0x7f431c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 --2- 172.21.15.46:0/3292397790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c103ff0 0x7f431c1043d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f430c009900 tx=0x7f430c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 -- 172.21.15.46:0/3292397790 shutdown_connections 2024-09-10T07:36:16.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 --2- 172.21.15.46:0/3292397790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109120 0x7f431c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 --2- 172.21.15.46:0/3292397790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431c1049a0 0x7f431c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 --2- 172.21.15.46:0/3292397790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c103ff0 0x7f431c1043d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 -- 172.21.15.46:0/3292397790 >> 172.21.15.46:0/3292397790 conn(0x7f431c0ffa90 msgr2=0x7f431c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 -- 172.21.15.46:0/3292397790 shutdown_connections 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.635+0000 7f4323800640 1 -- 172.21.15.46:0/3292397790 wait complete. 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.636+0000 7f4323800640 1 Processor -- start 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.636+0000 7f4323800640 1 -- start start 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.636+0000 7f4323800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c103ff0 0x7f431c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.636+0000 7f4323800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431c1049a0 0x7f431c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4323800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c109120 0x7f431c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4323800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f431c114170 con 0x7f431c109120 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4323800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f431c113ff0 con 0x7f431c103ff0 2024-09-10T07:36:16.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4323800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1142f0 con 0x7f431c1049a0 2024-09-10T07:36:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4321800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c109120 0x7f431c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4320e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c103ff0 0x7f431c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4313e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431c1049a0 0x7f431c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4313e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431c1049a0 0x7f431c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46202/0 (socket says 172.21.15.46:46202) 2024-09-10T07:36:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4321800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c109120 0x7f431c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46630/0 (socket says 172.21.15.46:46630) 2024-09-10T07:36:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4320e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c103ff0 0x7f431c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47718/0 (socket says 172.21.15.46:47718) 2024-09-10T07:36:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4313e00640 1 -- 172.21.15.46:0/3378175430 learned_addr learned my addr 172.21.15.46:0/3378175430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:16.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4313e00640 1 -- 172.21.15.46:0/3378175430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c103ff0 msgr2=0x7f431c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4313e00640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c103ff0 0x7f431c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4313e00640 1 -- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c109120 msgr2=0x7f431c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4313e00640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c109120 0x7f431c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.637+0000 7f4313e00640 1 -- 172.21.15.46:0/3378175430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c19f5b0 con 0x7f431c1049a0 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.638+0000 7f4313e00640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431c1049a0 0x7f431c19ab40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9f0 tx=0x7f430400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.638+0000 7f431a000640 1 -- 172.21.15.46:0/3378175430 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400cd80 con 0x7f431c1049a0 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.638+0000 7f4323800640 1 -- 172.21.15.46:0/3378175430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c19f8a0 con 0x7f431c1049a0 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.638+0000 7f431a000640 1 -- 172.21.15.46:0/3378175430 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4304010910 con 0x7f431c1049a0 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.638+0000 7f431a000640 1 -- 172.21.15.46:0/3378175430 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040188f0 con 0x7f431c1049a0 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.638+0000 7f4323800640 1 -- 172.21.15.46:0/3378175430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1a7180 con 0x7f431c1049a0 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.641+0000 7f43262b9740 1 Processor -- start 2024-09-10T07:36:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.641+0000 7f43262b9740 1 -- start start 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f431a000640 1 -- 172.21.15.46:0/3378175430 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4304010ac0 con 0x7f431c1049a0 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f43262b9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f04710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f43262b9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f43262b9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f43262b9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5654f1e6a0b0 con 0x5654f1f04c50 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f43262b9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5654f1e11ab0 con 0x5654f1f0f510 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f43262b9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5654f1e66070 con 0x5654f1f04330 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f4320e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f4321800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f04710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f4320e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46640/0 (socket says 172.21.15.46:46640) 2024-09-10T07:36:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f4320e00640 1 -- 172.21.15.46:0/3634857523 learned_addr learned my addr 172.21.15.46:0/3634857523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f4321800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f04710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46204/0 (socket says 172.21.15.46:46204) 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.642+0000 7f4313e00640 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4321800640 1 -- 172.21.15.46:0/3634857523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 msgr2=0x5654f1f11900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4321800640 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f11900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4321800640 1 -- 172.21.15.46:0/3634857523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 msgr2=0x5654f1f0efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4321800640 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4321800640 1 -- 172.21.15.46:0/3634857523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f1e1e4a0 con 0x5654f1f04330 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4320e00640 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4313e00640 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4321800640 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f04710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4314009f00 tx=0x7f4314011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89e750083ad3934f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4312000640 1 -- 172.21.15.46:0/3634857523 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314004d20 con 0x5654f1f04330 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4312000640 1 -- 172.21.15.46:0/3634857523 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4314004050 con 0x5654f1f04330 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.643+0000 7f4312000640 1 -- 172.21.15.46:0/3634857523 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314005280 con 0x5654f1f04330 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f431a000640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f43000a2b80 0x7f43000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f431a000640 1 -- 172.21.15.46:0/3378175430 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f43040cf7b0 con 0x7f431c1049a0 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f4320e00640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f43000a2b80 0x7f43000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f43262b9740 1 -- 172.21.15.46:0/3634857523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 msgr2=0x5654f1f04710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f43262b9740 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f04710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4314009f00 tx=0x7f4314011810 comp rx=0 tx=0).stop 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f43262b9740 1 -- 172.21.15.46:0/3634857523 shutdown_connections 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f43262b9740 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f43262b9740 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f43262b9740 1 --2- 172.21.15.46:0/3634857523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f04710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.644+0000 7f43262b9740 1 -- 172.21.15.46:0/3634857523 >> 172.21.15.46:0/3634857523 conn(0x5654f1f008c0 msgr2=0x5654f1f00ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.645+0000 7f43262b9740 1 -- 172.21.15.46:0/3634857523 shutdown_connections 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.645+0000 7f43262b9740 1 -- 172.21.15.46:0/3634857523 wait complete. 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.645+0000 7f4320e00640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f43000a2b80 0x7f43000a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f430c005c40 tx=0x7f430c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.645+0000 7f43262b9740 1 -- start start 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.645+0000 7f43262b9740 1 Processor -- start 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f43262b9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f0c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f43262b9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f43262b9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f0cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f43262b9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5654f1f12060 con 0x5654f1f04c50 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f43262b9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5654f1df2920 con 0x5654f1f0f510 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f43262b9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5654f1f121e0 con 0x5654f1f04330 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f43262b9740 10 client.0 _ll_register_callbacks cb 0x5654f1e168a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f4320e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f0c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f4313e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f4320e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f0c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46216/0 (socket says 172.21.15.46:46216) 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f4320e00640 1 -- 172.21.15.46:0/3291318126 learned_addr learned my addr 172.21.15.46:0/3291318126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.646+0000 7f4321800640 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f0cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f4313e00640 1 -- 172.21.15.46:0/3291318126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 msgr2=0x5654f1f0c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f4313e00640 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f0c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f4313e00640 1 -- 172.21.15.46:0/3291318126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 msgr2=0x5654f1f0cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f4313e00640 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f0cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f4313e00640 1 -- 172.21.15.46:0/3291318126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f1bdb2b0 con 0x5654f1f04c50 2024-09-10T07:36:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f4320e00640 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f0c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f4313e00640 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0cab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5654f1f01c40 tx=0x7f430409b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c041e70c4c4b432a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f430aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:36:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f430aa00640 1 -- 172.21.15.46:0/3291318126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040cfda0 con 0x5654f1f04c50 2024-09-10T07:36:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f430aa00640 1 -- 172.21.15.46:0/3291318126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f430409b130 con 0x5654f1f04c50 2024-09-10T07:36:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.647+0000 7f43262b9740 1 -- 172.21.15.46:0/3291318126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4314004050 con 0x5654f1f04c50 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f430aa00640 1 -- 172.21.15.46:0/3291318126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43040a48a0 con 0x5654f1f04c50 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f43262b9740 10 client.15087 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f43262b9740 1 -- 172.21.15.46:0/3291318126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654f1f1de50 con 0x5654f1f04c50 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f430aa00640 1 -- 172.21.15.46:0/3291318126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4314004050 con 0x5654f1f04c50 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f430aa00640 1 -- 172.21.15.46:0/3291318126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f430409aba0 con 0x5654f1f04c50 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f43262b9740 10 client.15087 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f43262b9740 10 client.15087 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f43262b9740 20 client.15087 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f43262b9740 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5654f1f203e0 0x5654f1f21850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f43262b9740 4 client.15087 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.648+0000 7f43262b9740 1 -- 172.21.15.46:0/3291318126 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f1e77620 con 0x5654f1f203e0 2024-09-10T07:36:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.649+0000 7f4320e00640 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5654f1f203e0 0x5654f1f21850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.649+0000 7f4320e00640 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5654f1f203e0 0x5654f1f21850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f516be2f7bf1cdd server_cookie=ed8725d2d16fadc6 in_seq=0 out_seq=0 2024-09-10T07:36:16.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.649+0000 7f430aa00640 10 client.15087 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.650+0000 7f430aa00640 1 -- 172.21.15.46:0/3291318126 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654f1e77620 con 0x5654f1f203e0 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.650+0000 7f430aa00640 10 client.15087 handle_command_reply: tid=0 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.650+0000 7f43262b9740 1 client.15087 shutdown 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 -- 172.21.15.46:0/3291318126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 msgr2=0x5654f1f0cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0cab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5654f1f01c40 tx=0x7f430409b640 comp rx=0 tx=0).stop 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 -- 172.21.15.46:0/3291318126 shutdown_connections 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5654f1f203e0 0x5654f1f21850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f0f510 0x5654f1f0cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f0cab0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 --2- 172.21.15.46:0/3291318126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f04330 0x5654f1f0c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 -- 172.21.15.46:0/3291318126 >> 172.21.15.46:0/3291318126 conn(0x5654f1f008c0 msgr2=0x5654f1f065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f430aa00640 0 client.15087 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:36:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 -- 172.21.15.46:0/3291318126 shutdown_connections 2024-09-10T07:36:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 1 -- 172.21.15.46:0/3291318126 wait complete. 2024-09-10T07:36:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.651+0000 7f43262b9740 20 client.15087 trim_cache size 0 max 16384 2024-09-10T07:36:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f43262b9740 1 Processor -- start 2024-09-10T07:36:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f43262b9740 1 -- start start 2024-09-10T07:36:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f43262b9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04330 0x5654f1f00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f43262b9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04c50 0x5654f1f0ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f43262b9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 0x5654f1f0b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f43262b9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5654f1f1fb90 con 0x5654f1f04330 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f43262b9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5654f1f20260 con 0x5654f1f04c50 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f43262b9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5654f1f20930 con 0x5654f1f0f510 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f4320e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04330 0x5654f1f00210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.670+0000 7f4313e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04c50 0x5654f1f0ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4313e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04c50 0x5654f1f0ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47732/0 (socket says 172.21.15.46:47732) 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4313e00640 1 -- 172.21.15.46:0/2142631182 learned_addr learned my addr 172.21.15.46:0/2142631182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4321800640 1 --2- 172.21.15.46:0/2142631182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 0x5654f1f0b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4313e00640 1 -- 172.21.15.46:0/2142631182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 msgr2=0x5654f1f0b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4313e00640 1 --2- 172.21.15.46:0/2142631182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 0x5654f1f0b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4313e00640 1 -- 172.21.15.46:0/2142631182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04330 msgr2=0x5654f1f00210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4313e00640 1 --2- 172.21.15.46:0/2142631182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04330 0x5654f1f00210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4313e00640 1 -- 172.21.15.46:0/2142631182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f1f04a80 con 0x5654f1f04c50 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4313e00640 1 --2- 172.21.15.46:0/2142631182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04c50 0x5654f1f0ae60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43040097b0 tx=0x7f43040a0160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa224a3959e8b3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.671+0000 7f4311600640 1 -- 172.21.15.46:0/2142631182 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040b9070 con 0x5654f1f04c50 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f4311600640 1 -- 172.21.15.46:0/2142631182 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43040cfb50 con 0x5654f1f04c50 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f4311600640 1 -- 172.21.15.46:0/2142631182 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430409b460 con 0x5654f1f04c50 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 -- 172.21.15.46:0/2142631182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04c50 msgr2=0x5654f1f0ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 --2- 172.21.15.46:0/2142631182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04c50 0x5654f1f0ae60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43040097b0 tx=0x7f43040a0160 comp rx=0 tx=0).stop 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 -- 172.21.15.46:0/2142631182 shutdown_connections 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 --2- 172.21.15.46:0/2142631182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 0x5654f1f0b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 --2- 172.21.15.46:0/2142631182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04c50 0x5654f1f0ae60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 --2- 172.21.15.46:0/2142631182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04330 0x5654f1f00210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 -- 172.21.15.46:0/2142631182 >> 172.21.15.46:0/2142631182 conn(0x5654f1f008c0 msgr2=0x5654f1f00ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 -- 172.21.15.46:0/2142631182 shutdown_connections 2024-09-10T07:36:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.672+0000 7f43262b9740 1 -- 172.21.15.46:0/2142631182 wait complete. 2024-09-10T07:36:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.673+0000 7f43262b9740 1 -- start start 2024-09-10T07:36:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.673+0000 7f43262b9740 1 Processor -- start 2024-09-10T07:36:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.673+0000 7f43262b9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04330 0x5654f1f29f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.673+0000 7f43262b9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f08fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.673+0000 7f43262b9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 0x5654f1f09510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.673+0000 7f43262b9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5654f1f216d0 con 0x5654f1f04c50 2024-09-10T07:36:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.673+0000 7f43262b9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5654f1f21000 con 0x5654f1f04330 2024-09-10T07:36:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.673+0000 7f43262b9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5654f1f21da0 con 0x5654f1f0f510 2024-09-10T07:36:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f43262b9740 10 client.0 _ll_register_callbacks cb 0x7f430409bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4320e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04330 0x5654f1f29f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4313e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f08fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4320e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04330 0x5654f1f29f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47736/0 (socket says 172.21.15.46:47736) 2024-09-10T07:36:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4320e00640 1 -- 172.21.15.46:0/377816714 learned_addr learned my addr 172.21.15.46:0/377816714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4321800640 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 0x5654f1f09510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4313e00640 1 -- 172.21.15.46:0/377816714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 msgr2=0x5654f1f09510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4313e00640 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 0x5654f1f09510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4313e00640 1 -- 172.21.15.46:0/377816714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04330 msgr2=0x5654f1f29f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4313e00640 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04330 0x5654f1f29f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4313e00640 1 -- 172.21.15.46:0/377816714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f1f09a50 con 0x5654f1f04c50 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.674+0000 7f4313e00640 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f08fd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5654f1f1b5f0 tx=0x7f430409b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cb80441d768498 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f430aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f430aa00640 1 -- 172.21.15.46:0/377816714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040b9070 con 0x5654f1f04c50 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f430aa00640 1 -- 172.21.15.46:0/377816714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4304010630 con 0x5654f1f04c50 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f43262b9740 1 -- 172.21.15.46:0/377816714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43040cfb50 con 0x5654f1f04c50 2024-09-10T07:36:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f430aa00640 1 -- 172.21.15.46:0/377816714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f430409b130 con 0x5654f1f04c50 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f43262b9740 10 client.15093 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f43262b9740 1 -- 172.21.15.46:0/377816714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654f1f09a50 con 0x5654f1f04c50 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f430aa00640 1 -- 172.21.15.46:0/377816714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43040cfb50 con 0x5654f1f04c50 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.675+0000 7f430aa00640 1 -- 172.21.15.46:0/377816714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f43040a6bc0 con 0x5654f1f04c50 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.676+0000 7f43262b9740 10 client.15093 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.676+0000 7f43262b9740 10 client.15093 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.676+0000 7f43262b9740 20 client.15093 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.676+0000 7f43262b9740 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5654f1f8ecf0 0x5654f1f8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.676+0000 7f43262b9740 4 client.15093 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:36:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.676+0000 7f43262b9740 1 -- 172.21.15.46:0/377816714 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "status"}) -- 0x5654f1e71b30 con 0x5654f1f8ecf0 2024-09-10T07:36:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.676+0000 7f4320e00640 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5654f1f8ecf0 0x5654f1f8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f4320e00640 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5654f1f8ecf0 0x5654f1f8da00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f247c820442c458d server_cookie=163a3edfbb032534 in_seq=0 out_seq=0 2024-09-10T07:36:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f430aa00640 10 client.15093 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f430aa00640 1 -- 172.21.15.46:0/377816714 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5654f1e71b30 con 0x5654f1f8ecf0 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f430aa00640 10 client.15093 handle_command_reply: tid=0 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 client.15093 shutdown 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 -- 172.21.15.46:0/377816714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 msgr2=0x5654f1f08fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f08fd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5654f1f1b5f0 tx=0x7f430409b340 comp rx=0 tx=0).stop 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 -- 172.21.15.46:0/377816714 shutdown_connections 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5654f1f8ecf0 0x5654f1f8da00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654f1f0f510 0x5654f1f09510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654f1f04c50 0x5654f1f08fd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 --2- 172.21.15.46:0/377816714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654f1f04330 0x5654f1f29f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f430aa00640 0 client.15093 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.677+0000 7f43262b9740 1 -- 172.21.15.46:0/377816714 >> 172.21.15.46:0/377816714 conn(0x5654f1f008c0 msgr2=0x5654f1f05d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f43262b9740 1 -- 172.21.15.46:0/377816714 shutdown_connections 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f43262b9740 1 -- 172.21.15.46:0/377816714 wait complete. 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f43262b9740 20 client.15093 trim_cache size 0 max 16384 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 4, 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24448, 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:36:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:36:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:36:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 110.875656198, 2024-09-10T07:36:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:36:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:36:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:36:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 208.553447038 2024-09-10T07:36:16.688 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 -- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f43000a2b80 msgr2=0x7f43000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f43000a2b80 0x7f43000a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f430c005c40 tx=0x7f430c005b50 comp rx=0 tx=0).stop 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 -- 172.21.15.46:0/3378175430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431c1049a0 msgr2=0x7f431c19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431c1049a0 0x7f431c19ab40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9f0 tx=0x7f430400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 -- 172.21.15.46:0/3378175430 shutdown_connections 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f43000a2b80 0x7f43000a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f431c109120 0x7f431c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431c1049a0 0x7f431c19ab40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 --2- 172.21.15.46:0/3378175430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c103ff0 0x7f431c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.678+0000 7f4323800640 1 -- 172.21.15.46:0/3378175430 >> 172.21.15.46:0/3378175430 conn(0x7f431c0ffa90 msgr2=0x7f431c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.679+0000 7f4323800640 1 -- 172.21.15.46:0/3378175430 shutdown_connections 2024-09-10T07:36:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:16.679+0000 7f4323800640 1 -- 172.21.15.46:0/3378175430 wait complete. 2024-09-10T07:36:17.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T07:36:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:18.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:18 smithi046 ceph-mon[30705]: pgmap v389: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:18 smithi181 ceph-mon[35397]: pgmap v389: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:18 smithi063 ceph-mon[35020]: pgmap v389: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 -- 172.21.15.46:0/2121005417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 msgr2=0x7f4d28110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/2121005417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 0x7f4d28110ca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ca0 tx=0x7f4d2002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 -- 172.21.15.46:0/2121005417 shutdown_connections 2024-09-10T07:36:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/2121005417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 0x7f4d28110ca0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/2121005417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d281049a0 0x7f4d281089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/2121005417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d28103ff0 0x7f4d281043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 -- 172.21.15.46:0/2121005417 >> 172.21.15.46:0/2121005417 conn(0x7f4d280ffa90 msgr2=0x7f4d28101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 -- 172.21.15.46:0/2121005417 shutdown_connections 2024-09-10T07:36:18.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.498+0000 7f4d2ce00640 1 -- 172.21.15.46:0/2121005417 wait complete. 2024-09-10T07:36:18.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.499+0000 7f4d2ce00640 1 Processor -- start 2024-09-10T07:36:18.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.499+0000 7f4d2ce00640 1 -- start start 2024-09-10T07:36:18.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d2ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28103ff0 0x7f4d2819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d2ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d281049a0 0x7f4d2819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d2ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 0x7f4d2819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d2ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281141a0 con 0x7f4d28109120 2024-09-10T07:36:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d2ce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28114020 con 0x7f4d28103ff0 2024-09-10T07:36:18.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d2ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28114320 con 0x7f4d281049a0 2024-09-10T07:36:18.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d281049a0 0x7f4d2819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 0x7f4d2819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28103ff0 0x7f4d2819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d281049a0 0x7f4d2819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46258/0 (socket says 172.21.15.46:46258) 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d1fe00640 1 -- 172.21.15.46:0/1202220234 learned_addr learned my addr 172.21.15.46:0/1202220234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.500+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28103ff0 0x7f4d2819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47758/0 (socket says 172.21.15.46:47758) 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1fe00640 1 -- 172.21.15.46:0/1202220234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28103ff0 msgr2=0x7f4d2819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28103ff0 0x7f4d2819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1fe00640 1 -- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 msgr2=0x7f4d2819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 0x7f4d2819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1fe00640 1 -- 172.21.15.46:0/1202220234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2819f590 con 0x7f4d281049a0 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d25e00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28103ff0 0x7f4d2819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d26800640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 0x7f4d2819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d281049a0 0x7f4d2819ab20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000ea60 tx=0x7f4d1000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1ea00640 1 -- 172.21.15.46:0/1202220234 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000cde0 con 0x7f4d281049a0 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1ea00640 1 -- 172.21.15.46:0/1202220234 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d100035f0 con 0x7f4d281049a0 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d1ea00640 1 -- 172.21.15.46:0/1202220234 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10010670 con 0x7f4d281049a0 2024-09-10T07:36:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d2ce00640 1 -- 172.21.15.46:0/1202220234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2819f880 con 0x7f4d281049a0 2024-09-10T07:36:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.501+0000 7f4d2ce00640 1 -- 172.21.15.46:0/1202220234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d281a7160 con 0x7f4d281049a0 2024-09-10T07:36:18.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.505+0000 7f4d2f9da740 1 Processor -- start 2024-09-10T07:36:18.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d2f9da740 1 -- start start 2024-09-10T07:36:18.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a90710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794a9d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5647949f60b0 con 0x564794a90330 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56479499dab0 con 0x564794a90c50 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5647949f2070 con 0x564794a9b510 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a90710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a90710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46688/0 (socket says 172.21.15.46:46688) 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d26800640 1 -- 172.21.15.46:0/3565503683 learned_addr learned my addr 172.21.15.46:0/3565503683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.506+0000 7f4d1ea00640 1 -- 172.21.15.46:0/1202220234 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d100037a0 con 0x7f4d281049a0 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.507+0000 7f4d25e00640 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.507+0000 7f4d26800640 1 -- 172.21.15.46:0/3565503683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 msgr2=0x564794a9d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:18.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.507+0000 7f4d26800640 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794a9d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.507+0000 7f4d26800640 1 -- 172.21.15.46:0/3565503683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 msgr2=0x564794a9afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:18.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.507+0000 7f4d26800640 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a9afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.507+0000 7f4d26800640 1 -- 172.21.15.46:0/3565503683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647949aa4a0 con 0x564794a90330 2024-09-10T07:36:18.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.507+0000 7f4d25e00640 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a9afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:18.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.507+0000 7f4d26800640 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a90710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ca0 tx=0x7f4d20033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce666776108eae50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:18.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.508+0000 7f4d17400640 1 -- 172.21.15.46:0/3565503683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20003b70 con 0x564794a90330 2024-09-10T07:36:18.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.508+0000 7f4d17400640 1 -- 172.21.15.46:0/3565503683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d20036040 con 0x564794a90330 2024-09-10T07:36:18.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.508+0000 7f4d17400640 1 -- 172.21.15.46:0/3565503683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20004500 con 0x564794a90330 2024-09-10T07:36:18.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.508+0000 7f4d2f9da740 1 -- 172.21.15.46:0/3565503683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 msgr2=0x564794a90710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.508+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a90710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ca0 tx=0x7f4d20033000 comp rx=0 tx=0).stop 2024-09-10T07:36:18.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.508+0000 7f4d1ea00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2c00 0x7f4d0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d2f9da740 1 -- 172.21.15.46:0/3565503683 shutdown_connections 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794a9d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a9afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d1ea00640 1 -- 172.21.15.46:0/1202220234 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d100cf650 con 0x7f4d281049a0 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/3565503683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a90710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d2f9da740 1 -- 172.21.15.46:0/3565503683 >> 172.21.15.46:0/3565503683 conn(0x564794a8c8c0 msgr2=0x564794a8cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d26800640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2c00 0x7f4d0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d2f9da740 1 -- 172.21.15.46:0/3565503683 shutdown_connections 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d2f9da740 1 -- 172.21.15.46:0/3565503683 wait complete. 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.509+0000 7f4d2f9da740 1 -- start start 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d26800640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2c00 0x7f4d0c0a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d20006010 tx=0x7f4d2002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:18.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d2f9da740 1 Processor -- start 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a90330 0x564794a98460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a9b510 0x564794a98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564794a9e000 con 0x564794a9b510 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56479497e920 con 0x564794a90c50 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564794a9e180 con 0x564794a90330 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d2f9da740 10 client.0 _ll_register_callbacks cb 0x5647949a28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a90330 0x564794a98460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a9b510 0x564794a98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a90330 0x564794a98460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46274/0 (socket says 172.21.15.46:46274) 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d25e00640 1 -- 172.21.15.46:0/1866451220 learned_addr learned my addr 172.21.15.46:0/1866451220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.510+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d25e00640 1 -- 172.21.15.46:0/1866451220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 msgr2=0x564794a989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d25e00640 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d25e00640 1 -- 172.21.15.46:0/1866451220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a9b510 msgr2=0x564794a98ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d25e00640 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a9b510 0x564794a98ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d25e00640 1 -- 172.21.15.46:0/1866451220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647947672b0 con 0x564794a90330 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d25e00640 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a90330 0x564794a98460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d180129b0 tx=0x7f4d18012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f7fd209e3769ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d0be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d0be00640 1 -- 172.21.15.46:0/1866451220 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800fa00 con 0x564794a90330 2024-09-10T07:36:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d2f9da740 1 -- 172.21.15.46:0/1866451220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d20036040 con 0x564794a90330 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.511+0000 7f4d0be00640 1 -- 172.21.15.46:0/1866451220 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d1800fba0 con 0x564794a90330 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.512+0000 7f4d0be00640 1 -- 172.21.15.46:0/1866451220 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d18025050 con 0x564794a90330 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.512+0000 7f4d2f9da740 10 client.25150 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.512+0000 7f4d2f9da740 1 -- 172.21.15.46:0/1866451220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564794aa9d40 con 0x564794a90330 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.512+0000 7f4d0be00640 1 -- 172.21.15.46:0/1866451220 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d20036040 con 0x564794a90330 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.512+0000 7f4d0be00640 1 -- 172.21.15.46:0/1866451220 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4d18003bd0 con 0x564794a90330 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.512+0000 7f4d2f9da740 10 client.25150 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.513+0000 7f4d2f9da740 10 client.25150 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.513+0000 7f4d2f9da740 20 client.25150 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.513+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x564794aac320 0x564794aad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.513+0000 7f4d2f9da740 4 client.25150 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.513+0000 7f4d2f9da740 1 -- 172.21.15.46:0/1866451220 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564794a03620 con 0x564794aac320 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.513+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x564794aac320 0x564794aad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.513+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x564794aac320 0x564794aad790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfec5b332d012d91 server_cookie=63515242529b3b32 in_seq=0 out_seq=0 2024-09-10T07:36:18.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.513+0000 7f4d0be00640 10 client.25150 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.514+0000 7f4d0be00640 1 -- 172.21.15.46:0/1866451220 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564794a03620 con 0x564794aac320 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.514+0000 7f4d0be00640 10 client.25150 handle_command_reply: tid=0 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.514+0000 7f4d2f9da740 1 client.25150 shutdown 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 -- 172.21.15.46:0/1866451220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a90330 msgr2=0x564794a98460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a90330 0x564794a98460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d180129b0 tx=0x7f4d18012e70 comp rx=0 tx=0).stop 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 -- 172.21.15.46:0/1866451220 shutdown_connections 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a9b510 0x564794a98ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x564794aac320 0x564794aad790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/1866451220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a90330 0x564794a98460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 -- 172.21.15.46:0/1866451220 >> 172.21.15.46:0/1866451220 conn(0x564794a8c8c0 msgr2=0x564794a925f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:18.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d0be00640 0 client.25150 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:36:18.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 -- 172.21.15.46:0/1866451220 shutdown_connections 2024-09-10T07:36:18.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 1 -- 172.21.15.46:0/1866451220 wait complete. 2024-09-10T07:36:18.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.515+0000 7f4d2f9da740 20 client.25150 trim_cache size 0 max 16384 2024-09-10T07:36:18.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d2f9da740 1 Processor -- start 2024-09-10T07:36:18.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d2f9da740 1 -- start start 2024-09-10T07:36:18.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a98ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794a956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564794aabad0 con 0x564794a90330 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564794aac1a0 con 0x564794a90c50 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564794aac870 con 0x564794a9b510 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a98ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46704/0 (socket says 172.21.15.46:46704) 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d25e00640 1 -- 172.21.15.46:0/4017809493 learned_addr learned my addr 172.21.15.46:0/4017809493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.534+0000 7f4d26800640 1 --2- 172.21.15.46:0/4017809493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794a956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d1fe00640 1 -- 172.21.15.46:0/4017809493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 msgr2=0x564794a956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/4017809493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794a956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d1fe00640 1 -- 172.21.15.46:0/4017809493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 msgr2=0x564794a989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/4017809493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d1fe00640 1 -- 172.21.15.46:0/4017809493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564794a9add0 con 0x564794a90c50 2024-09-10T07:36:18.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/4017809493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a98ef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x564794a8dc40 tx=0x7f4d1000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b012e3fc92ec8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d16a00640 1 -- 172.21.15.46:0/4017809493 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1009fe80 con 0x564794a90c50 2024-09-10T07:36:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d16a00640 1 -- 172.21.15.46:0/4017809493 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d100a4740 con 0x564794a90c50 2024-09-10T07:36:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.535+0000 7f4d16a00640 1 -- 172.21.15.46:0/4017809493 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100cf310 con 0x564794a90c50 2024-09-10T07:36:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 -- 172.21.15.46:0/4017809493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 msgr2=0x564794a98ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/4017809493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a98ef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x564794a8dc40 tx=0x7f4d1000ef90 comp rx=0 tx=0).stop 2024-09-10T07:36:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 -- 172.21.15.46:0/4017809493 shutdown_connections 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/4017809493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794a956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/4017809493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90c50 0x564794a98ef0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/4017809493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90330 0x564794a989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 -- 172.21.15.46:0/4017809493 >> 172.21.15.46:0/4017809493 conn(0x564794a8c8c0 msgr2=0x564794a96d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 -- 172.21.15.46:0/4017809493 shutdown_connections 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 -- 172.21.15.46:0/4017809493 wait complete. 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.536+0000 7f4d2f9da740 1 -- start start 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d2f9da740 1 Processor -- start 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90330 0x564794ab0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90c50 0x564794ab0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d2f9da740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794ab1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564794aad610 con 0x564794a90c50 2024-09-10T07:36:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564794aacf40 con 0x564794a90330 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d2f9da740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564794aadce0 con 0x564794a9b510 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d2f9da740 10 client.0 _ll_register_callbacks cb 0x7f4d18012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90c50 0x564794ab0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90c50 0x564794ab0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46712/0 (socket says 172.21.15.46:46712) 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d1fe00640 1 -- 172.21.15.46:0/2978222547 learned_addr learned my addr 172.21.15.46:0/2978222547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d25e00640 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90330 0x564794ab0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.537+0000 7f4d26800640 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794ab1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d1fe00640 1 -- 172.21.15.46:0/2978222547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 msgr2=0x564794ab1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794ab1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d1fe00640 1 -- 172.21.15.46:0/2978222547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90330 msgr2=0x564794ab0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90330 0x564794ab0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d1fe00640 1 -- 172.21.15.46:0/2978222547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564794ab58b0 con 0x564794a90c50 2024-09-10T07:36:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d25e00640 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90330 0x564794ab0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d1fe00640 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90c50 0x564794ab0c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4d100a4210 tx=0x7f4d10003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3388bd4ede66ddff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d0be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d0be00640 1 -- 172.21.15.46:0/2978222547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100cf310 con 0x564794a90c50 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d2f9da740 1 -- 172.21.15.46:0/2978222547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d100a4740 con 0x564794a90c50 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.538+0000 7f4d0be00640 1 -- 172.21.15.46:0/2978222547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d100b4030 con 0x564794a90c50 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d0be00640 1 -- 172.21.15.46:0/2978222547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d100ccdf0 con 0x564794a90c50 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d2f9da740 10 client.15111 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d2f9da740 1 -- 172.21.15.46:0/2978222547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564794b16e60 con 0x564794a90c50 2024-09-10T07:36:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d0be00640 1 -- 172.21.15.46:0/2978222547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d100a4740 con 0x564794a90c50 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d0be00640 1 -- 172.21.15.46:0/2978222547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4d1009af20 con 0x564794a90c50 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d2f9da740 10 client.15111 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d2f9da740 10 client.15111 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d2f9da740 20 client.15111 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x564794b1b540 0x564794b1a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d2f9da740 4 client.15111 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.539+0000 7f4d2f9da740 1 -- 172.21.15.46:0/2978222547 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "status"}) -- 0x5647949fdb30 con 0x564794b1b540 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.540+0000 7f4d25e00640 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x564794b1b540 0x564794b1a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.540+0000 7f4d25e00640 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x564794b1b540 0x564794b1a250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=347df651a175a133 server_cookie=1d37c424fb7e0151 in_seq=0 out_seq=0 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.540+0000 7f4d0be00640 10 client.15111 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.540+0000 7f4d0be00640 1 -- 172.21.15.46:0/2978222547 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5647949fdb30 con 0x564794b1b540 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.540+0000 7f4d0be00640 10 client.15111 handle_command_reply: tid=0 2024-09-10T07:36:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.540+0000 7f4d2f9da740 1 client.15111 shutdown 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 -- 172.21.15.46:0/2978222547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90c50 msgr2=0x564794ab0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90c50 0x564794ab0c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4d100a4210 tx=0x7f4d10003c30 comp rx=0 tx=0).stop 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 -- 172.21.15.46:0/2978222547 shutdown_connections 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x564794b1b540 0x564794b1a250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564794a9b510 0x564794ab1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564794a90c50 0x564794ab0c40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 --2- 172.21.15.46:0/2978222547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564794a90330 0x564794ab0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 -- 172.21.15.46:0/2978222547 >> 172.21.15.46:0/2978222547 conn(0x564794a8c8c0 msgr2=0x564794aa6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d0be00640 0 client.15111 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 -- 172.21.15.46:0/2978222547 shutdown_connections 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 1 -- 172.21.15.46:0/2978222547 wait complete. 2024-09-10T07:36:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.541+0000 7f4d2f9da740 20 client.15111 trim_cache size 0 max 16384 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 3, 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24499, 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 113.465622086, 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:36:18.546 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 191.34983154 2024-09-10T07:36:18.547 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:36:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 -- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2c00 msgr2=0x7f4d0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2c00 0x7f4d0c0a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d20006010 tx=0x7f4d2002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 -- 172.21.15.46:0/1202220234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d281049a0 msgr2=0x7f4d2819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d281049a0 0x7f4d2819ab20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000ea60 tx=0x7f4d1000ef20 comp rx=0 tx=0).stop 2024-09-10T07:36:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 -- 172.21.15.46:0/1202220234 shutdown_connections 2024-09-10T07:36:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2c00 0x7f4d0c0a50c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28109120 0x7f4d2819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d281049a0 0x7f4d2819ab20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 --2- 172.21.15.46:0/1202220234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28103ff0 0x7f4d2819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:18.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 -- 172.21.15.46:0/1202220234 >> 172.21.15.46:0/1202220234 conn(0x7f4d280ffa90 msgr2=0x7f4d281005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:18.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 -- 172.21.15.46:0/1202220234 shutdown_connections 2024-09-10T07:36:18.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:18.542+0000 7f4d2ce00640 1 -- 172.21.15.46:0/1202220234 wait complete. 2024-09-10T07:36:19.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T07:36:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:20 smithi181 ceph-mon[35397]: pgmap v390: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:20 smithi063 ceph-mon[35020]: pgmap v390: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:20.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:20 smithi046 ceph-mon[30705]: pgmap v390: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:20.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 -- 172.21.15.46:0/16118485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 msgr2=0x7fde88104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 --2- 172.21.15.46:0/16118485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 0x7fde88104210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fde78009860 tx=0x7fde7802a340 comp rx=0 tx=0).stop 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 -- 172.21.15.46:0/16118485 shutdown_connections 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 --2- 172.21.15.46:0/16118485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde88108f60 0x7fde88110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 --2- 172.21.15.46:0/16118485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde881047e0 0x7fde88108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 --2- 172.21.15.46:0/16118485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 0x7fde88104210 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 -- 172.21.15.46:0/16118485 >> 172.21.15.46:0/16118485 conn(0x7fde880ff8d0 msgr2=0x7fde88101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 -- 172.21.15.46:0/16118485 shutdown_connections 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.532+0000 7fde8ce00640 1 -- 172.21.15.46:0/16118485 wait complete. 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.533+0000 7fde8ce00640 1 Processor -- start 2024-09-10T07:36:20.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.533+0000 7fde8ce00640 1 -- start start 2024-09-10T07:36:20.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde8ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 0x7fde8819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde8ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde881047e0 0x7fde8819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde8ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde88108f60 0x7fde8819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde8ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde88113fc0 con 0x7fde881047e0 2024-09-10T07:36:20.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde8ce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde88113e40 con 0x7fde88103e30 2024-09-10T07:36:20.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde8ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fde88114140 con 0x7fde88108f60 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde881047e0 0x7fde8819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde88108f60 0x7fde8819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 0x7fde8819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde88108f60 0x7fde8819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46318/0 (socket says 172.21.15.46:46318) 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87e00640 1 -- 172.21.15.46:0/3563072692 learned_addr learned my addr 172.21.15.46:0/3563072692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87400640 1 -- 172.21.15.46:0/3563072692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde88108f60 msgr2=0x7fde8819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87400640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde88108f60 0x7fde8819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87400640 1 -- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde881047e0 msgr2=0x7fde8819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87400640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde881047e0 0x7fde8819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.534+0000 7fde87400640 1 -- 172.21.15.46:0/3563072692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8819f420 con 0x7fde88103e30 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.535+0000 7fde87400640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 0x7fde8819a470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde7802a860 tx=0x7fde78033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.535+0000 7fde84c00640 1 -- 172.21.15.46:0/3563072692 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78037040 con 0x7fde88103e30 2024-09-10T07:36:20.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.535+0000 7fde84c00640 1 -- 172.21.15.46:0/3563072692 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde78030550 con 0x7fde88103e30 2024-09-10T07:36:20.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.535+0000 7fde8ce00640 1 -- 172.21.15.46:0/3563072692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8819f6b0 con 0x7fde88103e30 2024-09-10T07:36:20.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.535+0000 7fde84c00640 1 -- 172.21.15.46:0/3563072692 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78003a30 con 0x7fde88103e30 2024-09-10T07:36:20.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.535+0000 7fde8ce00640 1 -- 172.21.15.46:0/3563072692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde881a6ff0 con 0x7fde88103e30 2024-09-10T07:36:20.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.538+0000 7fde90e8a740 1 Processor -- start 2024-09-10T07:36:20.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.538+0000 7fde84c00640 1 -- 172.21.15.46:0/3563072692 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde78042020 con 0x7fde88103e30 2024-09-10T07:36:20.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.538+0000 7fde90e8a740 1 -- start start 2024-09-10T07:36:20.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d54e0 0x556aa75d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75e0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde90e8a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556aa753b0b0 con 0x556aa75e06c0 2024-09-10T07:36:20.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde90e8a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556aa74e2ab0 con 0x556aa75d5e00 2024-09-10T07:36:20.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde90e8a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556aa7537070 con 0x556aa75d54e0 2024-09-10T07:36:20.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde87400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde87e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d54e0 0x556aa75d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde87400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75e0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47826/0 (socket says 172.21.15.46:47826) 2024-09-10T07:36:20.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.539+0000 7fde87400640 1 -- 172.21.15.46:0/4263488699 learned_addr learned my addr 172.21.15.46:0/4263488699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde87e00640 1 -- 172.21.15.46:0/4263488699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 msgr2=0x556aa75e0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde87e00640 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75e0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde87e00640 1 -- 172.21.15.46:0/4263488699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 msgr2=0x556aa75e2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde87e00640 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75e2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde87e00640 1 -- 172.21.15.46:0/4263488699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa74ef4a0 con 0x556aa75d54e0 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde87400640 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75e0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde7fe00640 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75e2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde84c00640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde6c0a2c00 0x7fde6c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde87e00640 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d54e0 0x556aa75d58c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fde800099f0 tx=0x7fde800095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1780d001b5ad8057 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde84c00640 1 -- 172.21.15.46:0/3563072692 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde780eea00 con 0x7fde88103e30 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde7cc00640 1 -- 172.21.15.46:0/4263488699 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80003970 con 0x556aa75d54e0 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde87400640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde6c0a2c00 0x7fde6c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde7cc00640 1 -- 172.21.15.46:0/4263488699 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8000d6b0 con 0x556aa75d54e0 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.540+0000 7fde7cc00640 1 -- 172.21.15.46:0/4263488699 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80004360 con 0x556aa75d54e0 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 -- 172.21.15.46:0/4263488699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d54e0 msgr2=0x556aa75d58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde87400640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde6c0a2c00 0x7fde6c0a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde780eeca0 tx=0x7fde780f4530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:20.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d54e0 0x556aa75d58c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fde800099f0 tx=0x7fde800095c0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 -- 172.21.15.46:0/4263488699 shutdown_connections 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75e2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75e0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 --2- 172.21.15.46:0/4263488699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d54e0 0x556aa75d58c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 -- 172.21.15.46:0/4263488699 >> 172.21.15.46:0/4263488699 conn(0x556aa75d1a70 msgr2=0x556aa75d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 -- 172.21.15.46:0/4263488699 shutdown_connections 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.541+0000 7fde90e8a740 1 -- 172.21.15.46:0/4263488699 wait complete. 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.542+0000 7fde90e8a740 1 -- start start 2024-09-10T07:36:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.542+0000 7fde90e8a740 1 Processor -- start 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75dd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d5e00 0x556aa75ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 0x556aa75ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde90e8a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556aa75e3330 con 0x556aa75d5e00 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde90e8a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556aa74c3920 con 0x556aa75d54e0 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde90e8a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556aa75e34b0 con 0x556aa75e06c0 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde90e8a740 10 client.0 _ll_register_callbacks cb 0x556aa74e78a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d5e00 0x556aa75ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d5e00 0x556aa75ddaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46748/0 (socket says 172.21.15.46:46748) 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde7fe00640 1 -- 172.21.15.46:0/2241530271 learned_addr learned my addr 172.21.15.46:0/2241530271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde87e00640 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75dd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.543+0000 7fde87e00640 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 0x556aa75ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde7fe00640 1 -- 172.21.15.46:0/2241530271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 msgr2=0x556aa75ddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde7fe00640 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 0x556aa75ddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde7fe00640 1 -- 172.21.15.46:0/2241530271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 msgr2=0x556aa75dd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde7fe00640 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75dd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde7fe00640 1 -- 172.21.15.46:0/2241530271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa72ac2b0 con 0x556aa75d5e00 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde87e00640 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75dd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde7fe00640 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d5e00 0x556aa75ddaa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fde7000bf30 tx=0x7fde700118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3c7f0cbe829b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde75600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde75600640 1 -- 172.21.15.46:0/2241530271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70004020 con 0x556aa75d5e00 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde90e8a740 1 -- 172.21.15.46:0/2241530271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde8000d6b0 con 0x556aa75d5e00 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.544+0000 7fde75600640 1 -- 172.21.15.46:0/2241530271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde700041c0 con 0x556aa75d5e00 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde75600640 1 -- 172.21.15.46:0/2241530271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde70024050 con 0x556aa75d5e00 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde90e8a740 10 client.15123 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde90e8a740 1 -- 172.21.15.46:0/2241530271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556aa75ef010 con 0x556aa75d5e00 2024-09-10T07:36:20.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde75600640 1 -- 172.21.15.46:0/2241530271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8000d6b0 con 0x556aa75d5e00 2024-09-10T07:36:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde75600640 1 -- 172.21.15.46:0/2241530271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fde70003980 con 0x556aa75d5e00 2024-09-10T07:36:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde90e8a740 10 client.15123 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde90e8a740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T07:36:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde90e8a740 20 client.15123 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde90e8a740 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556aa75f1640 0x556aa75f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde90e8a740 4 client.15123 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:36:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.545+0000 7fde90e8a740 1 -- 172.21.15.46:0/2241530271 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aa7548620 con 0x556aa75f1640 2024-09-10T07:36:20.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.546+0000 7fde87e00640 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556aa75f1640 0x556aa75f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.546+0000 7fde87e00640 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556aa75f1640 0x556aa75f2ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab7dd60176e55e08 server_cookie=fee345c67a6a9677 in_seq=0 out_seq=0 2024-09-10T07:36:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.546+0000 7fde75600640 10 client.15123 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:36:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.547+0000 7fde75600640 1 -- 172.21.15.46:0/2241530271 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556aa7548620 con 0x556aa75f1640 2024-09-10T07:36:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.547+0000 7fde75600640 10 client.15123 handle_command_reply: tid=0 2024-09-10T07:36:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.547+0000 7fde90e8a740 1 client.15123 shutdown 2024-09-10T07:36:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 -- 172.21.15.46:0/2241530271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d5e00 msgr2=0x556aa75ddaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d5e00 0x556aa75ddaa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fde7000bf30 tx=0x7fde700118d0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 -- 172.21.15.46:0/2241530271 shutdown_connections 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556aa75f1640 0x556aa75f2ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 0x556aa75ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde75600640 0 client.15123 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d5e00 0x556aa75ddaa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 --2- 172.21.15.46:0/2241530271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75dd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 -- 172.21.15.46:0/2241530271 >> 172.21.15.46:0/2241530271 conn(0x556aa75d1a70 msgr2=0x556aa75d7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 -- 172.21.15.46:0/2241530271 shutdown_connections 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 1 -- 172.21.15.46:0/2241530271 wait complete. 2024-09-10T07:36:20.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.548+0000 7fde90e8a740 20 client.15123 trim_cache size 0 max 16384 2024-09-10T07:36:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde90e8a740 1 Processor -- start 2024-09-10T07:36:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde90e8a740 1 -- start start 2024-09-10T07:36:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d5e00 0x556aa75d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75dde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde90e8a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556aa75f0df0 con 0x556aa75e06c0 2024-09-10T07:36:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde90e8a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556aa75f14c0 con 0x556aa75d54e0 2024-09-10T07:36:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde90e8a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556aa75f1b90 con 0x556aa75d5e00 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde87e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.567+0000 7fde7fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d5e00 0x556aa75d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde7fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d5e00 0x556aa75d7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46354/0 (socket says 172.21.15.46:46354) 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde87e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47848/0 (socket says 172.21.15.46:47848) 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde7fe00640 1 -- 172.21.15.46:0/3430991015 learned_addr learned my addr 172.21.15.46:0/3430991015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde7fe00640 1 -- 172.21.15.46:0/3430991015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 msgr2=0x556aa75f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde7fe00640 1 --2- 172.21.15.46:0/3430991015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde7fe00640 1 -- 172.21.15.46:0/3430991015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 msgr2=0x556aa75dde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde7fe00640 1 --2- 172.21.15.46:0/3430991015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75dde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde7fe00640 1 -- 172.21.15.46:0/3430991015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa75d5310 con 0x556aa75d5e00 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.568+0000 7fde7fe00640 1 --2- 172.21.15.46:0/3430991015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d5e00 0x556aa75d7f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde70001eb0 tx=0x7fde700058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3376b6929fbe75ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde77e00640 1 -- 172.21.15.46:0/3430991015 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7002c030 con 0x556aa75d5e00 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde77e00640 1 -- 172.21.15.46:0/3430991015 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde70004470 con 0x556aa75d5e00 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde77e00640 1 -- 172.21.15.46:0/3430991015 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70000e60 con 0x556aa75d5e00 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 -- 172.21.15.46:0/3430991015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d5e00 msgr2=0x556aa75d7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 --2- 172.21.15.46:0/3430991015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d5e00 0x556aa75d7f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde70001eb0 tx=0x7fde700058d0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 -- 172.21.15.46:0/3430991015 shutdown_connections 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 --2- 172.21.15.46:0/3430991015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75e06c0 0x556aa75dde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 --2- 172.21.15.46:0/3430991015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75d5e00 0x556aa75d7f40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 --2- 172.21.15.46:0/3430991015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d54e0 0x556aa75f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 -- 172.21.15.46:0/3430991015 >> 172.21.15.46:0/3430991015 conn(0x556aa75d1a70 msgr2=0x556aa75dbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 -- 172.21.15.46:0/3430991015 shutdown_connections 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.569+0000 7fde90e8a740 1 -- 172.21.15.46:0/3430991015 wait complete. 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.570+0000 7fde90e8a740 1 -- start start 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.570+0000 7fde90e8a740 1 Processor -- start 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.570+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d54e0 0x556aa75f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.570+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.570+0000 7fde90e8a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 0x556aa75f5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.570+0000 7fde90e8a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556aa75f2930 con 0x556aa75d54e0 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.570+0000 7fde90e8a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556aa75f2260 con 0x556aa75d5e00 2024-09-10T07:36:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde90e8a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556aa75f3000 con 0x556aa75e06c0 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde90e8a740 10 client.0 _ll_register_callbacks cb 0x7fde7000b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d54e0 0x556aa75f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d54e0 0x556aa75f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46770/0 (socket says 172.21.15.46:46770) 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde87e00640 1 -- 172.21.15.46:0/579982107 learned_addr learned my addr 172.21.15.46:0/579982107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde7fe00640 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde87e00640 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 0x556aa75f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde7fe00640 1 -- 172.21.15.46:0/579982107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 msgr2=0x556aa75f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde7fe00640 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 0x556aa75f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde7fe00640 1 -- 172.21.15.46:0/579982107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d54e0 msgr2=0x556aa75f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde7fe00640 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d54e0 0x556aa75f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde7fe00640 1 -- 172.21.15.46:0/579982107 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa75d9160 con 0x556aa75d5e00 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.571+0000 7fde87e00640 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d54e0 0x556aa75f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.572+0000 7fde7fe00640 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75f5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde7000bf30 tx=0x7fde70004150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e15aee78ac710a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.572+0000 7fde75600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.572+0000 7fde75600640 1 -- 172.21.15.46:0/579982107 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70034020 con 0x556aa75d5e00 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.572+0000 7fde75600640 1 -- 172.21.15.46:0/579982107 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde700035f0 con 0x556aa75d5e00 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.572+0000 7fde90e8a740 1 -- 172.21.15.46:0/579982107 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde70004470 con 0x556aa75d5e00 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.572+0000 7fde75600640 1 -- 172.21.15.46:0/579982107 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde7002c040 con 0x556aa75d5e00 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.572+0000 7fde90e8a740 10 client.25166 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.572+0000 7fde90e8a740 1 -- 172.21.15.46:0/579982107 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556aa765bdb0 con 0x556aa75d5e00 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde75600640 1 -- 172.21.15.46:0/579982107 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde70004470 con 0x556aa75d5e00 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde75600640 1 -- 172.21.15.46:0/579982107 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fde70010950 con 0x556aa75d5e00 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde90e8a740 10 client.25166 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde90e8a740 10 client.25166 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T07:36:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde90e8a740 20 client.25166 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde90e8a740 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556aa75f1640 0x556aa765f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde90e8a740 4 client.25166 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde90e8a740 1 -- 172.21.15.46:0/579982107 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "status"}) -- 0x556aa7542b30 con 0x556aa75f1640 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.573+0000 7fde87e00640 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556aa75f1640 0x556aa765f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde87e00640 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556aa75f1640 0x556aa765f1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69fd9a5b694e2c9 server_cookie=e6f76f5ee1874bdd in_seq=0 out_seq=0 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde75600640 10 client.25166 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde75600640 1 -- 172.21.15.46:0/579982107 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556aa7542b30 con 0x556aa75f1640 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde75600640 10 client.25166 handle_command_reply: tid=0 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 client.25166 shutdown 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 -- 172.21.15.46:0/579982107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 msgr2=0x556aa75f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75f5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde7000bf30 tx=0x7fde70004150 comp rx=0 tx=0).stop 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 -- 172.21.15.46:0/579982107 shutdown_connections 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556aa75f1640 0x556aa765f1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa75e06c0 0x556aa75f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aa75d5e00 0x556aa75f5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 --2- 172.21.15.46:0/579982107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa75d54e0 0x556aa75f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde75600640 0 client.25166 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.574+0000 7fde90e8a740 1 -- 172.21.15.46:0/579982107 >> 172.21.15.46:0/579982107 conn(0x556aa75d1a70 msgr2=0x556aa75eb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.575+0000 7fde90e8a740 1 -- 172.21.15.46:0/579982107 shutdown_connections 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.575+0000 7fde90e8a740 1 -- 172.21.15.46:0/579982107 wait complete. 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.575+0000 7fde90e8a740 20 client.25166 trim_cache size 0 max 16384 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-09-10T07:36:20.580 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24517, 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 116.506798186, 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 188.09236651099999 2024-09-10T07:36:20.581 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:36:20.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.575+0000 7fde8ce00640 1 -- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde6c0a2c00 msgr2=0x7fde6c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.575+0000 7fde8ce00640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde6c0a2c00 0x7fde6c0a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde780eeca0 tx=0x7fde780f4530 comp rx=0 tx=0).stop 2024-09-10T07:36:20.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.575+0000 7fde8ce00640 1 -- 172.21.15.46:0/3563072692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 msgr2=0x7fde8819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:20.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.575+0000 7fde8ce00640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 0x7fde8819a470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde7802a860 tx=0x7fde78033d90 comp rx=0 tx=0).stop 2024-09-10T07:36:20.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.576+0000 7fde8ce00640 1 -- 172.21.15.46:0/3563072692 shutdown_connections 2024-09-10T07:36:20.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.576+0000 7fde8ce00640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde88108f60 0x7fde8819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.576+0000 7fde8ce00640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde6c0a2c00 0x7fde6c0a50c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.576+0000 7fde8ce00640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde881047e0 0x7fde8819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.576+0000 7fde8ce00640 1 --2- 172.21.15.46:0/3563072692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde88103e30 0x7fde8819a470 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:20.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.576+0000 7fde8ce00640 1 -- 172.21.15.46:0/3563072692 >> 172.21.15.46:0/3563072692 conn(0x7fde880ff8d0 msgr2=0x7fde88100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:20.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.576+0000 7fde8ce00640 1 -- 172.21.15.46:0/3563072692 shutdown_connections 2024-09-10T07:36:20.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:20.576+0000 7fde8ce00640 1 -- 172.21.15.46:0/3563072692 wait complete. 2024-09-10T07:36:21.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T07:36:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:22.250 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:22 smithi046 ceph-mon[30705]: pgmap v391: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 -- 172.21.15.46:0/2504347524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 msgr2=0x7f9f801043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 --2- 172.21.15.46:0/2504347524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 0x7f9f801043d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009900 tx=0x7f9f7002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 -- 172.21.15.46:0/2504347524 shutdown_connections 2024-09-10T07:36:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 --2- 172.21.15.46:0/2504347524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f80109120 0x7f9f80110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 --2- 172.21.15.46:0/2504347524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f801049a0 0x7f9f801089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 --2- 172.21.15.46:0/2504347524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 0x7f9f801043d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 -- 172.21.15.46:0/2504347524 >> 172.21.15.46:0/2504347524 conn(0x7f9f800ffa90 msgr2=0x7f9f80101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 -- 172.21.15.46:0/2504347524 shutdown_connections 2024-09-10T07:36:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.364+0000 7f9f86800640 1 -- 172.21.15.46:0/2504347524 wait complete. 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.365+0000 7f9f86800640 1 Processor -- start 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.365+0000 7f9f86800640 1 -- start start 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.365+0000 7f9f86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 0x7f9f8019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.365+0000 7f9f86800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f801049a0 0x7f9f8019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.365+0000 7f9f86800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80109120 0x7f9f8019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.365+0000 7f9f86800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80114170 con 0x7f9f80103ff0 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.365+0000 7f9f86800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80113ff0 con 0x7f9f801049a0 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.365+0000 7f9f86800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801142f0 con 0x7f9f80109120 2024-09-10T07:36:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 0x7f9f8019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 0x7f9f8019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46786/0 (socket says 172.21.15.46:46786) 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 -- 172.21.15.46:0/535030667 learned_addr learned my addr 172.21.15.46:0/535030667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f801049a0 0x7f9f8019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80109120 0x7f9f8019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 -- 172.21.15.46:0/535030667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80109120 msgr2=0x7f9f8019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80109120 0x7f9f8019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 -- 172.21.15.46:0/535030667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f801049a0 msgr2=0x7f9f8019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f801049a0 0x7f9f8019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 -- 172.21.15.46:0/535030667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8019f5b0 con 0x7f9f80103ff0 2024-09-10T07:36:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.366+0000 7f9f7f400640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 0x7f9f8019a600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9f70004770 tx=0x7f9f70003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.367+0000 7f9f77e00640 1 -- 172.21.15.46:0/535030667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7003f070 con 0x7f9f80103ff0 2024-09-10T07:36:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.367+0000 7f9f77e00640 1 -- 172.21.15.46:0/535030667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f70031ca0 con 0x7f9f80103ff0 2024-09-10T07:36:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.367+0000 7f9f77e00640 1 -- 172.21.15.46:0/535030667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70035030 con 0x7f9f80103ff0 2024-09-10T07:36:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.367+0000 7f9f86800640 1 -- 172.21.15.46:0/535030667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8019f840 con 0x7f9f80103ff0 2024-09-10T07:36:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.367+0000 7f9f86800640 1 -- 172.21.15.46:0/535030667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8019fcd0 con 0x7f9f80103ff0 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.370+0000 7f9f89471740 1 Processor -- start 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.370+0000 7f9f77e00640 1 -- 172.21.15.46:0/535030667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f70042070 con 0x7f9f80103ff0 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.370+0000 7f9f89471740 1 -- start start 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f89471740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9330 0x5609c79c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f89471740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f89471740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 0x5609c79d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f89471740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609c792f0b0 con 0x5609c79c9c50 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f89471740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5609c78d6ab0 con 0x5609c79c9330 2024-09-10T07:36:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f89471740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5609c792b070 con 0x5609c79d4510 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9330 0x5609c79c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 0x5609c79d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9330 0x5609c79c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47872/0 (socket says 172.21.15.46:47872) 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 0x5609c79d6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46396/0 (socket says 172.21.15.46:46396) 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7fe00640 1 -- 172.21.15.46:0/1910893907 learned_addr learned my addr 172.21.15.46:0/1910893907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46796/0 (socket says 172.21.15.46:46796) 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7fe00640 1 -- 172.21.15.46:0/1910893907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 msgr2=0x5609c79d6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/1910893907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 0x5609c79d6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7fe00640 1 -- 172.21.15.46:0/1910893907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 msgr2=0x5609c79d3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/1910893907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.371+0000 7f9f7fe00640 1 -- 172.21.15.46:0/1910893907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c78e34a0 con 0x5609c79c9330 2024-09-10T07:36:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.372+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/1910893907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9330 0x5609c79c9710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009f50 tx=0x7f9f780118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c08d3804ec7fd4cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.372+0000 7f9f77e00640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f640a2c00 0x7f9f640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.372+0000 7f9f74c00640 1 -- 172.21.15.46:0/1910893907 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78004d20 con 0x5609c79c9330 2024-09-10T07:36:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.372+0000 7f9f77e00640 1 -- 172.21.15.46:0/535030667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9f700ef960 con 0x7f9f80103ff0 2024-09-10T07:36:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.372+0000 7f9f74c00640 1 -- 172.21.15.46:0/1910893907 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78004050 con 0x5609c79c9330 2024-09-10T07:36:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.372+0000 7f9f74c00640 1 -- 172.21.15.46:0/1910893907 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78005260 con 0x5609c79c9330 2024-09-10T07:36:22.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.372+0000 7f9f7f400640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f640a2c00 0x7f9f640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 -- 172.21.15.46:0/1910893907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9330 msgr2=0x5609c79c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 --2- 172.21.15.46:0/1910893907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9330 0x5609c79c9710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009f50 tx=0x7f9f780118d0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f7f400640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f640a2c00 0x7f9f640a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9f700b83b0 tx=0x7f9f700f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:22.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 -- 172.21.15.46:0/1910893907 shutdown_connections 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 --2- 172.21.15.46:0/1910893907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 0x5609c79d6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 --2- 172.21.15.46:0/1910893907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 --2- 172.21.15.46:0/1910893907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9330 0x5609c79c9710 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 -- 172.21.15.46:0/1910893907 >> 172.21.15.46:0/1910893907 conn(0x5609c79c58c0 msgr2=0x5609c79c5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 -- 172.21.15.46:0/1910893907 shutdown_connections 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.373+0000 7f9f89471740 1 -- 172.21.15.46:0/1910893907 wait complete. 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.374+0000 7f9f89471740 1 -- start start 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.374+0000 7f9f89471740 1 Processor -- start 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.374+0000 7f9f89471740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.374+0000 7f9f89471740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.374+0000 7f9f89471740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 0x5609c79d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.374+0000 7f9f89471740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609c79d7060 con 0x5609c79c9c50 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.374+0000 7f9f89471740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5609c78b7920 con 0x5609c79d4510 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f89471740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5609c79d71e0 con 0x5609c79c9330 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f89471740 10 client.0 _ll_register_callbacks cb 0x5609c78db8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46806/0 (socket says 172.21.15.46:46806) 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7ea00640 1 -- 172.21.15.46:0/3796858928 learned_addr learned my addr 172.21.15.46:0/3796858928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 0x5609c79d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7ea00640 1 -- 172.21.15.46:0/3796858928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 msgr2=0x5609c79d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7ea00640 1 -- 172.21.15.46:0/3796858928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 msgr2=0x5609c79d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 0x5609c79d2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.375+0000 7f9f7ea00640 1 -- 172.21.15.46:0/3796858928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c76a02b0 con 0x5609c79c9c50 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.376+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d1b00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c012700 tx=0x7f9f6c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8c75871ebfc8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.376+0000 7f9f69600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.376+0000 7f9f69600640 1 -- 172.21.15.46:0/3796858928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00f8c0 con 0x5609c79c9c50 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.376+0000 7f9f69600640 1 -- 172.21.15.46:0/3796858928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9f6c00fa60 con 0x5609c79c9c50 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.376+0000 7f9f89471740 1 -- 172.21.15.46:0/3796858928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78004050 con 0x5609c79c9c50 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.376+0000 7f9f69600640 1 -- 172.21.15.46:0/3796858928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f6c025050 con 0x5609c79c9c50 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.376+0000 7f9f89471740 10 client.15141 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.376+0000 7f9f89471740 1 -- 172.21.15.46:0/3796858928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609c79e2ec0 con 0x5609c79c9c50 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f69600640 1 -- 172.21.15.46:0/3796858928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78004050 con 0x5609c79c9c50 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f69600640 1 -- 172.21.15.46:0/3796858928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9f6c010430 con 0x5609c79c9c50 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f89471740 10 client.15141 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f89471740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:36:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f89471740 20 client.15141 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f89471740 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5609c79e54d0 0x5609c79e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f89471740 4 client.15141 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f89471740 1 -- 172.21.15.46:0/3796858928 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c793c620 con 0x5609c79e54d0 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.377+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5609c79e54d0 0x5609c79e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.378+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5609c79e54d0 0x5609c79e6940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b46e1264fb3dc676 server_cookie=aed0c6c7de148e52 in_seq=0 out_seq=0 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.378+0000 7f9f69600640 10 client.15141 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f69600640 1 -- 172.21.15.46:0/3796858928 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609c793c620 con 0x5609c79e54d0 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f69600640 10 client.15141 handle_command_reply: tid=0 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 client.15141 shutdown 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 -- 172.21.15.46:0/3796858928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 msgr2=0x5609c79d1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d1b00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c012700 tx=0x7f9f6c012bc0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 -- 172.21.15.46:0/3796858928 shutdown_connections 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5609c79e54d0 0x5609c79e6940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 0x5609c79d2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d1b00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f69600640 0 client.15141 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 --2- 172.21.15.46:0/3796858928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 -- 172.21.15.46:0/3796858928 >> 172.21.15.46:0/3796858928 conn(0x5609c79c58c0 msgr2=0x5609c79cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 -- 172.21.15.46:0/3796858928 shutdown_connections 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.379+0000 7f9f89471740 1 -- 172.21.15.46:0/3796858928 wait complete. 2024-09-10T07:36:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.380+0000 7f9f89471740 20 client.15141 trim_cache size 0 max 16384 2024-09-10T07:36:22.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.398+0000 7f9f89471740 1 Processor -- start 2024-09-10T07:36:22.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.398+0000 7f9f89471740 1 -- start start 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f89471740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f89471740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f89471740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 0x5609c79ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f89471740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609c79e4c80 con 0x5609c79c9c50 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f89471740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5609c79e5350 con 0x5609c79d4510 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f89471740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5609c79e5a20 con 0x5609c79c9330 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f7ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46402/0 (socket says 172.21.15.46:46402) 2024-09-10T07:36:22.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f7fe00640 1 -- 172.21.15.46:0/3829438745 learned_addr learned my addr 172.21.15.46:0/3829438745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 0x5609c79ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f7ea00640 1 -- 172.21.15.46:0/3829438745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 msgr2=0x5609c79cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.399+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.400+0000 7f9f7ea00640 1 -- 172.21.15.46:0/3829438745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 msgr2=0x5609c79ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.400+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 0x5609c79ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.400+0000 7f9f7ea00640 1 -- 172.21.15.46:0/3829438745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c79c9a80 con 0x5609c79c9c50 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.400+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.400+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d03f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0109f0 tx=0x7f9f6c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e37ece19a63e90b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.400+0000 7f9f6be00640 1 -- 172.21.15.46:0/3829438745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c010430 con 0x5609c79c9c50 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.400+0000 7f9f6be00640 1 -- 172.21.15.46:0/3829438745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c0105d0 con 0x5609c79c9c50 2024-09-10T07:36:22.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.400+0000 7f9f6be00640 1 -- 172.21.15.46:0/3829438745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00f930 con 0x5609c79c9c50 2024-09-10T07:36:22.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:22 smithi063 ceph-mon[35020]: pgmap v391: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:22.403 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:22 smithi181 ceph-mon[35397]: pgmap v391: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 -- 172.21.15.46:0/3829438745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 msgr2=0x5609c79d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d03f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0109f0 tx=0x7f9f6c009fd0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 -- 172.21.15.46:0/3829438745 shutdown_connections 2024-09-10T07:36:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79d4510 0x5609c79ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9c50 0x5609c79d03f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 --2- 172.21.15.46:0/3829438745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79c9330 0x5609c79cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 -- 172.21.15.46:0/3829438745 >> 172.21.15.46:0/3829438745 conn(0x5609c79c58c0 msgr2=0x5609c79c5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 -- 172.21.15.46:0/3829438745 shutdown_connections 2024-09-10T07:36:22.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 -- 172.21.15.46:0/3829438745 wait complete. 2024-09-10T07:36:22.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.401+0000 7f9f89471740 1 -- start start 2024-09-10T07:36:22.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f89471740 1 Processor -- start 2024-09-10T07:36:22.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f89471740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9330 0x5609c79eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f89471740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9c50 0x5609c79ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f89471740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 0x5609c79e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f89471740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609c79e67c0 con 0x5609c79c9330 2024-09-10T07:36:22.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f89471740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5609c79e60f0 con 0x5609c79c9c50 2024-09-10T07:36:22.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f89471740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5609c79e6e90 con 0x5609c79d4510 2024-09-10T07:36:22.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f89471740 10 client.0 _ll_register_callbacks cb 0x7f9f6c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:22.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f7ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9c50 0x5609c79ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9330 0x5609c79eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f7ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9c50 0x5609c79ef2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47898/0 (socket says 172.21.15.46:47898) 2024-09-10T07:36:22.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.402+0000 7f9f7ea00640 1 -- 172.21.15.46:0/2474620017 learned_addr learned my addr 172.21.15.46:0/2474620017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:22.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f7ea00640 1 -- 172.21.15.46:0/2474620017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 msgr2=0x5609c79e9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:22.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 0x5609c79e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f7ea00640 1 -- 172.21.15.46:0/2474620017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9330 msgr2=0x5609c79eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:22.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9330 0x5609c79eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f7ea00640 1 -- 172.21.15.46:0/2474620017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c79ea190 con 0x5609c79c9c50 2024-09-10T07:36:22.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9330 0x5609c79eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:22.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f7ea00640 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9c50 0x5609c79ef2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c010970 tx=0x7f9f6c010c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a269a5ac8253dd26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:22.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f69600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:36:22.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f69600640 1 -- 172.21.15.46:0/2474620017 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c02d070 con 0x5609c79c9c50 2024-09-10T07:36:22.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f69600640 1 -- 172.21.15.46:0/2474620017 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9f6c018040 con 0x5609c79c9c50 2024-09-10T07:36:22.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f89471740 1 -- 172.21.15.46:0/2474620017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f6c0105d0 con 0x5609c79c9c50 2024-09-10T07:36:22.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f69600640 1 -- 172.21.15.46:0/2474620017 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f6c039050 con 0x5609c79c9c50 2024-09-10T07:36:22.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f89471740 10 client.25184 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:22.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.403+0000 7f9f89471740 1 -- 172.21.15.46:0/2474620017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609c7a58d10 con 0x5609c79c9c50 2024-09-10T07:36:22.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f69600640 1 -- 172.21.15.46:0/2474620017 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c0105d0 con 0x5609c79c9c50 2024-09-10T07:36:22.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f69600640 1 -- 172.21.15.46:0/2474620017 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9f6c0035f0 con 0x5609c79c9c50 2024-09-10T07:36:22.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f89471740 10 client.25184 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f89471740 10 client.25184 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:36:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f89471740 20 client.25184 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f89471740 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5609c7a5d560 0x5609c7a5c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f89471740 4 client.25184 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:36:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f89471740 1 -- 172.21.15.46:0/2474620017 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "status"}) -- 0x5609c7936b30 con 0x5609c7a5d560 2024-09-10T07:36:22.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.404+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5609c7a5d560 0x5609c7a5c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:22.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5609c7a5d560 0x5609c7a5c270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a86bd86adeee09e1 server_cookie=9475028e4a240c84 in_seq=0 out_seq=0 2024-09-10T07:36:22.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f69600640 10 client.25184 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:36:22.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f69600640 1 -- 172.21.15.46:0/2474620017 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5609c7936b30 con 0x5609c7a5d560 2024-09-10T07:36:22.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f69600640 10 client.25184 handle_command_reply: tid=0 2024-09-10T07:36:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f89471740 1 client.25184 shutdown 2024-09-10T07:36:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f89471740 1 -- 172.21.15.46:0/2474620017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9c50 msgr2=0x5609c79ef2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f89471740 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9c50 0x5609c79ef2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c010970 tx=0x7f9f6c010c50 comp rx=0 tx=0).stop 2024-09-10T07:36:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f89471740 1 -- 172.21.15.46:0/2474620017 shutdown_connections 2024-09-10T07:36:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f89471740 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5609c7a5d560 0x5609c7a5c270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f89471740 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5609c79d4510 0x5609c79e9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f89471740 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5609c79c9c50 0x5609c79ef2c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f69600640 0 client.25184 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:36:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.405+0000 7f9f89471740 1 --2- 172.21.15.46:0/2474620017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c79c9330 0x5609c79eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.406+0000 7f9f89471740 1 -- 172.21.15.46:0/2474620017 >> 172.21.15.46:0/2474620017 conn(0x5609c79c58c0 msgr2=0x5609c79cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.406+0000 7f9f89471740 1 -- 172.21.15.46:0/2474620017 shutdown_connections 2024-09-10T07:36:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.406+0000 7f9f89471740 1 -- 172.21.15.46:0/2474620017 wait complete. 2024-09-10T07:36:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.406+0000 7f9f89471740 20 client.25184 trim_cache size 0 max 16384 2024-09-10T07:36:22.421 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:36:22.421 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:36:22.421 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-09-10T07:36:22.421 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24530, 2024-09-10T07:36:22.421 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:36:22.421 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:36:22.422 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:36:22.422 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 119.350955567, 2024-09-10T07:36:22.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:36:22.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:36:22.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:36:22.422 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 179.379337918 2024-09-10T07:36:22.422 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:36:22.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 -- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f640a2c00 msgr2=0x7f9f640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f640a2c00 0x7f9f640a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9f700b83b0 tx=0x7f9f700f5530 comp rx=0 tx=0).stop 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 -- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 msgr2=0x7f9f8019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 0x7f9f8019a600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9f70004770 tx=0x7f9f70003c10 comp rx=0 tx=0).stop 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 -- 172.21.15.46:0/535030667 shutdown_connections 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f80109120 0x7f9f8019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f801049a0 0x7f9f8019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f640a2c00 0x7f9f640a50c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 --2- 172.21.15.46:0/535030667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f80103ff0 0x7f9f8019a600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 -- 172.21.15.46:0/535030667 >> 172.21.15.46:0/535030667 conn(0x7f9f800ffa90 msgr2=0x7f9f80100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 -- 172.21.15.46:0/535030667 shutdown_connections 2024-09-10T07:36:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:22.407+0000 7f9f86800640 1 -- 172.21.15.46:0/535030667 wait complete. 2024-09-10T07:36:23.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-10T07:36:23.608 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 -- 172.21.15.46:0/3018378558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a8103ff0 msgr2=0x7f53a81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 --2- 172.21.15.46:0/3018378558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a8103ff0 0x7f53a81043d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f539c009900 tx=0x7f539c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 -- 172.21.15.46:0/3018378558 shutdown_connections 2024-09-10T07:36:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 --2- 172.21.15.46:0/3018378558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8109120 0x7f53a8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 --2- 172.21.15.46:0/3018378558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a81049a0 0x7f53a81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 --2- 172.21.15.46:0/3018378558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a8103ff0 0x7f53a81043d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 -- 172.21.15.46:0/3018378558 >> 172.21.15.46:0/3018378558 conn(0x7f53a80ffa90 msgr2=0x7f53a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 -- 172.21.15.46:0/3018378558 shutdown_connections 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.246+0000 7f53b0400640 1 -- 172.21.15.46:0/3018378558 wait complete. 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.247+0000 7f53b0400640 1 Processor -- start 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.247+0000 7f53b0400640 1 -- start start 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53b0400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103ff0 0x7f53a819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53b0400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a81049a0 0x7f53a819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53b0400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a8109120 0x7f53a819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53b0400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8114170 con 0x7f53a81049a0 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53b0400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8113ff0 con 0x7f53a8103ff0 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53b0400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81142f0 con 0x7f53a8109120 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ad000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a81049a0 0x7f53a819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ada00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103ff0 0x7f53a819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ad000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a81049a0 0x7f53a819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46832/0 (socket says 172.21.15.46:46832) 2024-09-10T07:36:24.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ad000640 1 -- 172.21.15.46:0/3724777386 learned_addr learned my addr 172.21.15.46:0/3724777386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ada00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103ff0 0x7f53a819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47912/0 (socket says 172.21.15.46:47912) 2024-09-10T07:36:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ae400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a8109120 0x7f53a819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ad000640 1 -- 172.21.15.46:0/3724777386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a8109120 msgr2=0x7f53a819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ad000640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a8109120 0x7f53a819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ad000640 1 -- 172.21.15.46:0/3724777386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103ff0 msgr2=0x7f53a819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ad000640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103ff0 0x7f53a819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.248+0000 7f53ad000640 1 -- 172.21.15.46:0/3724777386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a819f5e0 con 0x7f53a81049a0 2024-09-10T07:36:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.249+0000 7f53ad000640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a81049a0 0x7f53a819ab70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9f0 tx=0x7f539400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.249+0000 7f53a6000640 1 -- 172.21.15.46:0/3724777386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539400cd80 con 0x7f53a81049a0 2024-09-10T07:36:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.249+0000 7f53a6000640 1 -- 172.21.15.46:0/3724777386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5394010910 con 0x7f53a81049a0 2024-09-10T07:36:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.249+0000 7f53b0400640 1 -- 172.21.15.46:0/3724777386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a819f8d0 con 0x7f53a81049a0 2024-09-10T07:36:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.249+0000 7f53a6000640 1 -- 172.21.15.46:0/3724777386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394018990 con 0x7f53a81049a0 2024-09-10T07:36:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.249+0000 7f53b0400640 1 -- 172.21.15.46:0/3724777386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81a71b0 con 0x7f53a81049a0 2024-09-10T07:36:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.252+0000 7f53b2fd6740 1 Processor -- start 2024-09-10T07:36:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.252+0000 7f53b2fd6740 1 -- start start 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d69630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53a6000640 1 -- 172.21.15.46:0/3724777386 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5394010ac0 con 0x7f53a81049a0 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 0x564467d7f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564467ccf0b0 con 0x564467d7cd20 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564467c76ab0 con 0x564467d69250 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564467ccb070 con 0x564467d69b70 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ada00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ae400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d69630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ada00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d7c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46448/0 (socket says 172.21.15.46:46448) 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ada00640 1 -- 172.21.15.46:0/2471933588 learned_addr learned my addr 172.21.15.46:0/2471933588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ad000640 1 --2- 172.21.15.46:0/2471933588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 0x564467d7f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ada00640 1 -- 172.21.15.46:0/2471933588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 msgr2=0x564467d69630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ada00640 1 --2- 172.21.15.46:0/2471933588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d69630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ada00640 1 -- 172.21.15.46:0/2471933588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 msgr2=0x564467d7f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ada00640 1 --2- 172.21.15.46:0/2471933588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 0x564467d7f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.253+0000 7f53ada00640 1 -- 172.21.15.46:0/2471933588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564467c834a0 con 0x564467d69b70 2024-09-10T07:36:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.254+0000 7f53ada00640 1 --2- 172.21.15.46:0/2471933588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d7c7e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f539c009900 tx=0x7f539c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d821ed3dd6427ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.254+0000 7f539aa00640 1 -- 172.21.15.46:0/2471933588 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c035040 con 0x564467d69b70 2024-09-10T07:36:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.254+0000 7f539aa00640 1 -- 172.21.15.46:0/2471933588 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f539c005520 con 0x564467d69b70 2024-09-10T07:36:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.254+0000 7f539aa00640 1 -- 172.21.15.46:0/2471933588 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c003850 con 0x564467d69b70 2024-09-10T07:36:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.254+0000 7f53a6000640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53900a2b80 0x7f53900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53a6000640 1 -- 172.21.15.46:0/3724777386 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f53940cf730 con 0x7f53a81049a0 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53ae400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53900a2b80 0x7f53900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53b2fd6740 1 -- 172.21.15.46:0/2471933588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 msgr2=0x564467d7c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/2471933588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d7c7e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f539c009900 tx=0x7f539c004040 comp rx=0 tx=0).stop 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53b2fd6740 1 -- 172.21.15.46:0/2471933588 shutdown_connections 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/2471933588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 0x564467d7f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/2471933588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d7c7e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/2471933588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d69630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53b2fd6740 1 -- 172.21.15.46:0/2471933588 >> 172.21.15.46:0/2471933588 conn(0x564467d658c0 msgr2=0x564467d65ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.255+0000 7f53ae400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53900a2b80 0x7f53900a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f53a0005fd0 tx=0x7f53a00096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.256+0000 7f53b2fd6740 1 -- 172.21.15.46:0/2471933588 shutdown_connections 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.256+0000 7f53b2fd6740 1 -- 172.21.15.46:0/2471933588 wait complete. 2024-09-10T07:36:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.256+0000 7f53b2fd6740 1 -- start start 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.256+0000 7f53b2fd6740 1 Processor -- start 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69250 0x564467d79c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69b70 0x564467d7a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d7cd20 0x564467d7a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564467d7f8d0 con 0x564467d69250 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564467c57920 con 0x564467d69b70 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564467d7fa50 con 0x564467d7cd20 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53b2fd6740 10 client.0 _ll_register_callbacks cb 0x564467c7b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53ada00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69250 0x564467d79c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53ad000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69b70 0x564467d7a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53ada00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69250 0x564467d79c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46850/0 (socket says 172.21.15.46:46850) 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53ada00640 1 -- 172.21.15.46:0/1871284371 learned_addr learned my addr 172.21.15.46:0/1871284371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.257+0000 7f53ae400640 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d7cd20 0x564467d7a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53ad000640 1 -- 172.21.15.46:0/1871284371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d7cd20 msgr2=0x564467d7a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53ad000640 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d7cd20 0x564467d7a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53ad000640 1 -- 172.21.15.46:0/1871284371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69250 msgr2=0x564467d79c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53ad000640 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69250 0x564467d79c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53ad000640 1 -- 172.21.15.46:0/1871284371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564467a402b0 con 0x564467d69b70 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53ada00640 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69250 0x564467d79c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53ad000640 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69b70 0x564467d7a1d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f53940cce20 tx=0x7f539409b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daab27764aae7c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f538f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f538f400640 1 -- 172.21.15.46:0/1871284371 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539409ff60 con 0x564467d69b70 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53b2fd6740 1 -- 172.21.15.46:0/1871284371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f539c005520 con 0x564467d69b70 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f538f400640 1 -- 172.21.15.46:0/1871284371 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f539409b0b0 con 0x564467d69b70 2024-09-10T07:36:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f538f400640 1 -- 172.21.15.46:0/1871284371 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53940a4820 con 0x564467d69b70 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53b2fd6740 10 client.25196 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.258+0000 7f53b2fd6740 1 -- 172.21.15.46:0/1871284371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564467d8b6f0 con 0x564467d69b70 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f538f400640 1 -- 172.21.15.46:0/1871284371 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f539c005520 con 0x564467d69b70 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f538f400640 1 -- 172.21.15.46:0/1871284371 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5394010430 con 0x564467d69b70 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f53b2fd6740 10 client.25196 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f53b2fd6740 10 client.25196 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f53b2fd6740 20 client.25196 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564467d8dcb0 0x564467d8f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f53b2fd6740 4 client.25196 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f53b2fd6740 1 -- 172.21.15.46:0/1871284371 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564467cdc620 con 0x564467d8dcb0 2024-09-10T07:36:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.259+0000 7f53ada00640 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564467d8dcb0 0x564467d8f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.260+0000 7f538f400640 10 client.25196 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.260+0000 7f53ada00640 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564467d8dcb0 0x564467d8f120 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5f3a5ca266f44a server_cookie=4ec7998ee947e7be in_seq=0 out_seq=0 2024-09-10T07:36:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f538f400640 1 -- 172.21.15.46:0/1871284371 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564467cdc620 con 0x564467d8dcb0 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f538f400640 10 client.25196 handle_command_reply: tid=0 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 client.25196 shutdown 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 -- 172.21.15.46:0/1871284371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69b70 msgr2=0x564467d7a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69b70 0x564467d7a1d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f53940cce20 tx=0x7f539409b820 comp rx=0 tx=0).stop 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 -- 172.21.15.46:0/1871284371 shutdown_connections 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564467d8dcb0 0x564467d8f120 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d7cd20 0x564467d7a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69b70 0x564467d7a1d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/1871284371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69250 0x564467d79c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f538f400640 0 client.25196 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 -- 172.21.15.46:0/1871284371 >> 172.21.15.46:0/1871284371 conn(0x564467d658c0 msgr2=0x564467d64a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 -- 172.21.15.46:0/1871284371 shutdown_connections 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 1 -- 172.21.15.46:0/1871284371 wait complete. 2024-09-10T07:36:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.261+0000 7f53b2fd6740 20 client.25196 trim_cache size 0 max 16384 2024-09-10T07:36:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.280+0000 7f53b2fd6740 1 Processor -- start 2024-09-10T07:36:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.280+0000 7f53b2fd6740 1 -- start start 2024-09-10T07:36:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69250 0x564467d8e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69b70 0x564467d97400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d7cd20 0x564467d97940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564467d8d460 con 0x564467d69b70 2024-09-10T07:36:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564467d8db30 con 0x564467d7cd20 2024-09-10T07:36:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564467d8e200 con 0x564467d69250 2024-09-10T07:36:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ad000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69b70 0x564467d97400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ae400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d7cd20 0x564467d97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ad000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69b70 0x564467d97400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46862/0 (socket says 172.21.15.46:46862) 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ad000640 1 -- 172.21.15.46:0/3867981246 learned_addr learned my addr 172.21.15.46:0/3867981246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ada00640 1 --2- 172.21.15.46:0/3867981246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69250 0x564467d8e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ae400640 1 -- 172.21.15.46:0/3867981246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69250 msgr2=0x564467d8e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ae400640 1 --2- 172.21.15.46:0/3867981246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69250 0x564467d8e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ae400640 1 -- 172.21.15.46:0/3867981246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69b70 msgr2=0x564467d97400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ae400640 1 --2- 172.21.15.46:0/3867981246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69b70 0x564467d97400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.281+0000 7f53ae400640 1 -- 172.21.15.46:0/3867981246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564467d6a580 con 0x564467d7cd20 2024-09-10T07:36:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.282+0000 7f53ae400640 1 --2- 172.21.15.46:0/3867981246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d7cd20 0x564467d97940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53a0011db0 tx=0x7f53a0019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ef30c9db1c3617 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.282+0000 7f539a000640 1 -- 172.21.15.46:0/3867981246 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a001e9a0 con 0x564467d7cd20 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.282+0000 7f539a000640 1 -- 172.21.15.46:0/3867981246 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a0003e50 con 0x564467d7cd20 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.282+0000 7f539a000640 1 -- 172.21.15.46:0/3867981246 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0004710 con 0x564467d7cd20 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3867981246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d7cd20 msgr2=0x564467d97940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3867981246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d7cd20 0x564467d97940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53a0011db0 tx=0x7f53a0019000 comp rx=0 tx=0).stop 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3867981246 shutdown_connections 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3867981246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d7cd20 0x564467d97940 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3867981246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d69b70 0x564467d97400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3867981246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69250 0x564467d8e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3867981246 >> 172.21.15.46:0/3867981246 conn(0x564467d658c0 msgr2=0x564467d78580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3867981246 shutdown_connections 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3867981246 wait complete. 2024-09-10T07:36:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 -- start start 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.283+0000 7f53b2fd6740 1 Processor -- start 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d97380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53b2fd6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 0x564467d92330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564467d8efa0 con 0x564467d7cd20 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564467d8e8d0 con 0x564467d69250 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53b2fd6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564467d8f670 con 0x564467d69b70 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53b2fd6740 10 client.0 _ll_register_callbacks cb 0x7f539409ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53ada00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d97380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53ada00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d97380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47936/0 (socket says 172.21.15.46:47936) 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53ada00640 1 -- 172.21.15.46:0/3448332152 learned_addr learned my addr 172.21.15.46:0/3448332152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53ae400640 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 0x564467d92330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.284+0000 7f53ad000640 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f53ada00640 1 -- 172.21.15.46:0/3448332152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 msgr2=0x564467d978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f53ada00640 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f53ada00640 1 -- 172.21.15.46:0/3448332152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 msgr2=0x564467d92330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f53ada00640 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 0x564467d92330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f53ada00640 1 -- 172.21.15.46:0/3448332152 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564467d97f30 con 0x564467d69250 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f53ada00640 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d97380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f539c009900 tx=0x7f539c004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248cf3a9b16ed449 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f538f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f538f400640 1 -- 172.21.15.46:0/3448332152 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c035040 con 0x564467d69250 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f538f400640 1 -- 172.21.15.46:0/3448332152 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f539c004db0 con 0x564467d69250 2024-09-10T07:36:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.285+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3448332152 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53a0003e50 con 0x564467d69250 2024-09-10T07:36:24.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f538f400640 1 -- 172.21.15.46:0/3448332152 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f539c061050 con 0x564467d69250 2024-09-10T07:36:24.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53b2fd6740 10 client.25202 fetch_fsmap learned FSMap version 29 2024-09-10T07:36:24.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3448332152 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564467df88c0 con 0x564467d69250 2024-09-10T07:36:24.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f538f400640 1 -- 172.21.15.46:0/3448332152 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a0003e50 con 0x564467d69250 2024-09-10T07:36:24.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f538f400640 1 -- 172.21.15.46:0/3448332152 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f539c031990 con 0x564467d69250 2024-09-10T07:36:24.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53b2fd6740 10 client.25202 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T07:36:24.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53b2fd6740 10 client.25202 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:36:24.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53b2fd6740 20 client.25202 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564467dfa890 0x564467dfbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53b2fd6740 4 client.25202 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3448332152 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "status"}) -- 0x564467cd6b30 con 0x564467dfa890 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.286+0000 7f53ad000640 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564467dfa890 0x564467dfbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53ad000640 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564467dfa890 0x564467dfbd00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d109e484fccf845b server_cookie=848c56d121005eeb in_seq=0 out_seq=0 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f538f400640 10 client.25202 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f538f400640 1 -- 172.21.15.46:0/3448332152 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564467cd6b30 con 0x564467dfa890 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f538f400640 10 client.25202 handle_command_reply: tid=0 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 client.25202 shutdown 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3448332152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 msgr2=0x564467d97380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d97380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f539c009900 tx=0x7f539c004c00 comp rx=0 tx=0).stop 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3448332152 shutdown_connections 2024-09-10T07:36:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564467dfa890 0x564467dfbd00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564467d7cd20 0x564467d92330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564467d69b70 0x564467d978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f538f400640 0 client.25202 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 --2- 172.21.15.46:0/3448332152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564467d69250 0x564467d97380 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.287+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3448332152 >> 172.21.15.46:0/3448332152 conn(0x564467d658c0 msgr2=0x564467d7b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.288+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3448332152 shutdown_connections 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.288+0000 7f53b2fd6740 1 -- 172.21.15.46:0/3448332152 wait complete. 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.288+0000 7f53b2fd6740 20 client.25202 trim_cache size 0 max 16384 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:36:24.292 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24535, 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 167.41551269199999, 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 29, 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 183.80888660900001 2024-09-10T07:36:24.293 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.288+0000 7f53b0400640 1 -- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53900a2b80 msgr2=0x7f53900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.288+0000 7f53b0400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53900a2b80 0x7f53900a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f53a0005fd0 tx=0x7f53a00096a0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.288+0000 7f53b0400640 1 -- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a81049a0 msgr2=0x7f53a819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.288+0000 7f53b0400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a81049a0 0x7f53a819ab70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9f0 tx=0x7f539400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.288+0000 7f53b0400640 1 -- 172.21.15.46:0/3724777386 shutdown_connections 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.289+0000 7f53b0400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a8109120 0x7f53a819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.289+0000 7f53b0400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53900a2b80 0x7f53900a5040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.289+0000 7f53b0400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a81049a0 0x7f53a819ab70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.289+0000 7f53b0400640 1 --2- 172.21.15.46:0/3724777386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103ff0 0x7f53a819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:24.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.289+0000 7f53b0400640 1 -- 172.21.15.46:0/3724777386 >> 172.21.15.46:0/3724777386 conn(0x7f53a80ffa90 msgr2=0x7f53a8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:24.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.289+0000 7f53b0400640 1 -- 172.21.15.46:0/3724777386 shutdown_connections 2024-09-10T07:36:24.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:24.289+0000 7f53b0400640 1 -- 172.21.15.46:0/3724777386 wait complete. 2024-09-10T07:36:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:24 smithi181 ceph-mon[35397]: pgmap v392: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:24 smithi063 ceph-mon[35020]: pgmap v392: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:24.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:24 smithi046 ceph-mon[30705]: pgmap v392: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:25.260 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T07:36:25.272 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T07:36:25.273 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T07:36:25.273 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T07:36:25.273 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T07:36:25.291 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T07:36:25.292 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T07:36:25.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:25.292 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-10T07:36:25.328 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T07:36:25.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:25.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:25.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-10T07:36:25.717 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-10T07:36:25.717 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T07:36:25.718 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-10T07:36:25.718 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-10T07:36:25.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T07:36:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:26.392 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:26 smithi046 ceph-mon[30705]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:26.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:26 smithi181 ceph-mon[35397]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:26.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:26 smithi063 ceph-mon[35020]: pgmap v393: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T07:36:26.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 -- 172.21.15.46:0/679297330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a38106f10 msgr2=0x7f0a3810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:26.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 --2- 172.21.15.46:0/679297330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a38106f10 0x7f0a3810ea90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009ca0 tx=0x7f0a3002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 -- 172.21.15.46:0/679297330 shutdown_connections 2024-09-10T07:36:26.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 --2- 172.21.15.46:0/679297330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a38106f10 0x7f0a3810ea90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 --2- 172.21.15.46:0/679297330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a38102790 0x7f0a381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 --2- 172.21.15.46:0/679297330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a38101de0 0x7f0a381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 -- 172.21.15.46:0/679297330 >> 172.21.15.46:0/679297330 conn(0x7f0a380fd8c0 msgr2=0x7f0a380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 -- 172.21.15.46:0/679297330 shutdown_connections 2024-09-10T07:36:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.672+0000 7f0a40200640 1 -- 172.21.15.46:0/679297330 wait complete. 2024-09-10T07:36:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.673+0000 7f0a40200640 1 Processor -- start 2024-09-10T07:36:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.673+0000 7f0a40200640 1 -- start start 2024-09-10T07:36:26.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.673+0000 7f0a40200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a38101de0 0x7f0a381983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:26.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.673+0000 7f0a40200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a38102790 0x7f0a38198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:26.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.673+0000 7f0a40200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.673+0000 7f0a40200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38111f70 con 0x7f0a38102790 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.673+0000 7f0a40200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38111df0 con 0x7f0a38101de0 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.673+0000 7f0a40200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381120f0 con 0x7f0a38106f10 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a38101de0 0x7f0a381983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a38101de0 0x7f0a381983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50736/0 (socket says 172.21.15.46:50736) 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 -- 172.21.15.46:0/692351521 learned_addr learned my addr 172.21.15.46:0/692351521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3ce00640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a38102790 0x7f0a38198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3e200640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 -- 172.21.15.46:0/692351521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a38106f10 msgr2=0x7f0a3819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 -- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a38102790 msgr2=0x7f0a38198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a38102790 0x7f0a38198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 -- 172.21.15.46:0/692351521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3819d370 con 0x7f0a38101de0 2024-09-10T07:36:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3e200640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.674+0000 7f0a3d800640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a38101de0 0x7f0a381983c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d950 tx=0x7f0a2c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.675+0000 7f0a36000640 1 -- 172.21.15.46:0/692351521 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c017070 con 0x7f0a38101de0 2024-09-10T07:36:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.675+0000 7f0a36000640 1 -- 172.21.15.46:0/692351521 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c009c30 con 0x7f0a38101de0 2024-09-10T07:36:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.675+0000 7f0a36000640 1 -- 172.21.15.46:0/692351521 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00f6b0 con 0x7f0a38101de0 2024-09-10T07:36:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.675+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3819d660 con 0x7f0a38101de0 2024-09-10T07:36:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.675+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a381a4f40 con 0x7f0a38101de0 2024-09-10T07:36:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.676+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a381021c0 con 0x7f0a38101de0 2024-09-10T07:36:26.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.679+0000 7f0a36000640 1 -- 172.21.15.46:0/692351521 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a2c0109d0 con 0x7f0a38101de0 2024-09-10T07:36:26.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.680+0000 7f0a36000640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a200a2c00 0x7f0a200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:26.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.680+0000 7f0a36000640 1 -- 172.21.15.46:0/692351521 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0a2c0cefb0 con 0x7f0a38101de0 2024-09-10T07:36:26.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.680+0000 7f0a3ce00640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a200a2c00 0x7f0a200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:26.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.680+0000 7f0a3ce00640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a200a2c00 0x7f0a200a50c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a380fdce0 tx=0x7f0a2400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:26.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.683+0000 7f0a36000640 1 -- 172.21.15.46:0/692351521 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a2c009de0 con 0x7f0a38101de0 2024-09-10T07:36:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.903+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0a3802d7c0 con 0x7f0a38101de0 2024-09-10T07:36:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.904+0000 7f0a36000640 1 -- 172.21.15.46:0/692351521 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f0a2c09bbb0 con 0x7f0a38101de0 2024-09-10T07:36:26.904 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T07:36:26.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a200a2c00 msgr2=0x7f0a200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:26.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a200a2c00 0x7f0a200a50c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a380fdce0 tx=0x7f0a2400b390 comp rx=0 tx=0).stop 2024-09-10T07:36:26.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a38101de0 msgr2=0x7f0a381983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a38101de0 0x7f0a381983c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d950 tx=0x7f0a2c00de10 comp rx=0 tx=0).stop 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 shutdown_connections 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a38106f10 0x7f0a3819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a200a2c00 0x7f0a200a50c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a38102790 0x7f0a38198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 --2- 172.21.15.46:0/692351521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a38101de0 0x7f0a381983c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 >> 172.21.15.46:0/692351521 conn(0x7f0a380fd8c0 msgr2=0x7f0a380febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 shutdown_connections 2024-09-10T07:36:26.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:26.906+0000 7f0a40200640 1 -- 172.21.15.46:0/692351521 wait complete. 2024-09-10T07:36:27.338 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:27 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/692351521' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T07:36:27.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:27 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/692351521' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T07:36:27.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:27 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/692351521' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T07:36:27.837 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T07:36:27.838 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:36:27.838 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-10T07:36:27.838 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T07:36:27.838 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-10T07:36:27.838 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T07:36:27.838 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T07:36:27.838 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:36:27.838 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-10T07:36:27.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:27.839 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-10T07:36:27.877 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:36:27.877 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:36:27.877 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:36:27.878 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42058sec preferred_lft 42058sec 2024-09-10T07:36:27.879 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-10T07:36:27.879 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:36:27.879 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:36:27.879 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:36:27.879 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-10T07:36:27.879 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T07:36:27.879 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:36:27.879 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:36:27.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-09-10T07:36:27.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-09-10T07:36:27.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-09-10T07:36:27.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T07:36:27.880 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:36:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:36:28.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:28.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:28.110 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:36:28.170 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-10T07:36:28.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:28.173 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-10T07:36:28.212 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-10T07:36:28.212 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:36:28.212 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:36:28.212 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:36:28.212 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:36:28.213 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:36:28.213 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:36:28.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:36:28.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:36:28.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:36:28.213 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:36:28.349 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:28 smithi046 ceph-mon[30705]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:28.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:28 smithi181 ceph-mon[35397]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:28.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:28 smithi063 ceph-mon[35020]: pgmap v394: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:28.416 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:28.453 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-09-10T07:36:28.532 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:36:28.532 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:36:28.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-10T07:36:28.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-10T07:36:28.532 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:36:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:36:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:28.743 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-10T07:36:28.743 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:36:28.743 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:36:28.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-10T07:36:28.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T07:36:28.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-10T07:36:28.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-10T07:36:28.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-10T07:36:28.744 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:36:28.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:36:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:28.969 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:36:28.970 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T07:36:28.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-10T07:36:28.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T07:36:28.970 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T07:36:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:36:29.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:29.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:36:29.138 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /tmp/cephfs 2024-09-10T07:36:29.174 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-10T07:36:29.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:36:29.174 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /tmp/cephfs 2024-09-10T07:36:29.251 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T07:36:29.251 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T07:36:29.251 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e 2024-09-10T07:36:29.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:29.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-09-10T07:36:29.381 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-10T07:36:29.381 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-10T07:36:29.381 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-09-10T07:36:30.009 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-10T07:36:30.009 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-10T07:36:30.009 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-09-10T07:36:30.009 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.63:6789,172.21.15.181:6789" 2024-09-10T07:36:30.009 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.63:6789,172.21.15.181:6789:/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e 2024-09-10T07:36:30.009 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=b120bdf2-6f45-11ef-bcea-c7b262605968" will pass to kernel 2024-09-10T07:36:30.014 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:36:30.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:36:30.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /tmp/cephfs 2024-09-10T07:36:30.073 DEBUG:teuthology.orchestra.run.smithi046:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:36:30.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:36:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:30.167 INFO:teuthology.orchestra.run.smithi046.stdout:client15174 2024-09-10T07:36:30.167 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:36:30.167 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:36:30.204 DEBUG:teuthology.orchestra.run.smithi046:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:36:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:36:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:30.333 INFO:teuthology.orchestra.run.smithi046.stdout:client15174 2024-09-10T07:36:30.334 DEBUG:teuthology.orchestra.run.smithi046:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:36:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:30 smithi181 ceph-mon[35397]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:30 smithi063 ceph-mon[35020]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:30.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:30 smithi046 ceph-mon[30705]: pgmap v395: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:36:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:30.424 INFO:teuthology.orchestra.run.smithi046.stdout:client15174 2024-09-10T07:36:30.425 DEBUG:teuthology.orchestra.run.smithi046:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968.client15174/status') 2024-09-10T07:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:36:30.525 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.15174 (3)192.168.0.1:0/320870580 2024-09-10T07:36:30.525 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-09-10T07:36:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-09-10T07:36:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-09-10T07:36:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:69 bytes copied, 5.6137e-05 s, 1.2 MB/s 2024-09-10T07:36:30.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:36:30.536 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T07:36:30.550 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T07:36:30.550 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:36:30.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 -- 172.21.15.46:0/1764491058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 msgr2=0x7fac4810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 --2- 172.21.15.46:0/1764491058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 0x7fac4810c1a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fac44009ca0 tx=0x7fac4402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 -- 172.21.15.46:0/1764491058 shutdown_connections 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 --2- 172.21.15.46:0/1764491058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 0x7fac4810c1a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 --2- 172.21.15.46:0/1764491058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac48104f50 0x7fac481053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 --2- 172.21.15.46:0/1764491058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac48103d60 0x7fac48104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 -- 172.21.15.46:0/1764491058 >> 172.21.15.46:0/1764491058 conn(0x7fac480ff4e0 msgr2=0x7fac48101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 -- 172.21.15.46:0/1764491058 shutdown_connections 2024-09-10T07:36:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.708+0000 7fac50800640 1 -- 172.21.15.46:0/1764491058 wait complete. 2024-09-10T07:36:30.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.709+0000 7fac50800640 1 Processor -- start 2024-09-10T07:36:30.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.709+0000 7fac50800640 1 -- start start 2024-09-10T07:36:30.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.709+0000 7fac50800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac48103d60 0x7fac48199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:30.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.709+0000 7fac50800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac48104f50 0x7fac4819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.709+0000 7fac50800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 0x7fac481a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.709+0000 7fac50800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac4810fa30 con 0x7fac48105910 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.709+0000 7fac50800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac4810f8b0 con 0x7fac48104f50 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.709+0000 7fac50800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fac4810fbb0 con 0x7fac48103d60 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 0x7fac481a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac48104f50 0x7fac4819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac48104f50 0x7fac4819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50754/0 (socket says 172.21.15.46:50754) 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4d400640 1 -- 172.21.15.46:0/1839777448 learned_addr learned my addr 172.21.15.46:0/1839777448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4de00640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac48103d60 0x7fac48199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4e800640 1 -- 172.21.15.46:0/1839777448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac48103d60 msgr2=0x7fac48199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4e800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac48103d60 0x7fac48199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4e800640 1 -- 172.21.15.46:0/1839777448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac48104f50 msgr2=0x7fac4819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4e800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac48104f50 0x7fac4819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.710+0000 7fac4e800640 1 -- 172.21.15.46:0/1839777448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac481a1b70 con 0x7fac48105910 2024-09-10T07:36:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.711+0000 7fac4e800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 0x7fac481a1500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fac44031ae0 tx=0x7fac44031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.711+0000 7fac42a00640 1 -- 172.21.15.46:0/1839777448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44003b70 con 0x7fac48105910 2024-09-10T07:36:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.711+0000 7fac42a00640 1 -- 172.21.15.46:0/1839777448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac44003d10 con 0x7fac48105910 2024-09-10T07:36:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.711+0000 7fac42a00640 1 -- 172.21.15.46:0/1839777448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4402fb10 con 0x7fac48105910 2024-09-10T07:36:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.711+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac481a1e00 con 0x7fac48105910 2024-09-10T07:36:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.711+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac481a22c0 con 0x7fac48105910 2024-09-10T07:36:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.712+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4810b020 con 0x7fac48105910 2024-09-10T07:36:30.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.714+0000 7fac42a00640 1 -- 172.21.15.46:0/1839777448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac44004640 con 0x7fac48105910 2024-09-10T07:36:30.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.715+0000 7fac42a00640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fac300a2c30 0x7fac300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.715+0000 7fac42a00640 1 -- 172.21.15.46:0/1839777448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fac440f0700 con 0x7fac48105910 2024-09-10T07:36:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.716+0000 7fac4de00640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fac300a2c30 0x7fac300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.716+0000 7fac4de00640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fac300a2c30 0x7fac300a50f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fac48104db0 tx=0x7fac3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:30.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.720+0000 7fac42a00640 1 -- 172.21.15.46:0/1839777448 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac440bd280 con 0x7fac48105910 2024-09-10T07:36:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:30.951+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fac48062e90 con 0x7fac48105910 2024-09-10T07:36:31.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.124+0000 7fac42a00640 1 -- 172.21.15.46:0/1839777448 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7fac440bcb80 con 0x7fac48105910 2024-09-10T07:36:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fac300a2c30 msgr2=0x7fac300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fac300a2c30 0x7fac300a50f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fac48104db0 tx=0x7fac3c005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 msgr2=0x7fac481a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 0x7fac481a1500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fac44031ae0 tx=0x7fac44031750 comp rx=0 tx=0).stop 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 shutdown_connections 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fac300a2c30 0x7fac300a50f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac48105910 0x7fac481a1500 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac48104f50 0x7fac4819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 --2- 172.21.15.46:0/1839777448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac48103d60 0x7fac48199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 >> 172.21.15.46:0/1839777448 conn(0x7fac480ff4e0 msgr2=0x7fac481081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 shutdown_connections 2024-09-10T07:36:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.126+0000 7fac50800640 1 -- 172.21.15.46:0/1839777448 wait complete. 2024-09-10T07:36:31.142 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T07:36:31.155 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T07:36:31.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:36:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.348+0000 7fc54ee00640 1 -- 172.21.15.46:0/2522602825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc548074bd0 msgr2=0x7fc548075030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.348+0000 7fc54ee00640 1 --2- 172.21.15.46:0/2522602825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc548074bd0 0x7fc548075030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc530009900 tx=0x7fc53002a480 comp rx=0 tx=0).stop 2024-09-10T07:36:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.348+0000 7fc54ee00640 1 -- 172.21.15.46:0/2522602825 shutdown_connections 2024-09-10T07:36:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.348+0000 7fc54ee00640 1 --2- 172.21.15.46:0/2522602825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5481099c0 0x7fc54810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.348+0000 7fc54ee00640 1 --2- 172.21.15.46:0/2522602825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc548074bd0 0x7fc548075030 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.348+0000 7fc54ee00640 1 --2- 172.21.15.46:0/2522602825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5480761b0 0x7fc548074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.348+0000 7fc54ee00640 1 -- 172.21.15.46:0/2522602825 >> 172.21.15.46:0/2522602825 conn(0x7fc5480fd2b0 msgr2=0x7fc5480ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.349+0000 7fc54ee00640 1 -- 172.21.15.46:0/2522602825 shutdown_connections 2024-09-10T07:36:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.349+0000 7fc54ee00640 1 -- 172.21.15.46:0/2522602825 wait complete. 2024-09-10T07:36:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.349+0000 7fc54ee00640 1 Processor -- start 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc54ee00640 1 -- start start 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc54ee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548074bd0 0x7fc54819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc54ee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5480761b0 0x7fc54819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc54ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5481099c0 0x7fc5481a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc54ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc54810f970 con 0x7fc5481099c0 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc54ee00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc54810f7f0 con 0x7fc548074bd0 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc54ee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc54810faf0 con 0x7fc5480761b0 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc547e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548074bd0 0x7fc54819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc54ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5481099c0 0x7fc5481a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc547e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548074bd0 0x7fc54819a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50766/0 (socket says 172.21.15.46:50766) 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc547e00640 1 -- 172.21.15.46:0/3813397334 learned_addr learned my addr 172.21.15.46:0/3813397334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc53fe00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5480761b0 0x7fc54819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc547e00640 1 -- 172.21.15.46:0/3813397334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5480761b0 msgr2=0x7fc54819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc547e00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5480761b0 0x7fc54819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc547e00640 1 -- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5481099c0 msgr2=0x7fc5481a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc547e00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5481099c0 0x7fc5481a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.350+0000 7fc547e00640 1 -- 172.21.15.46:0/3813397334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5481a1c70 con 0x7fc548074bd0 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.351+0000 7fc547e00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548074bd0 0x7fc54819a050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc53800ea60 tx=0x7fc53800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.351+0000 7fc545600640 1 -- 172.21.15.46:0/3813397334 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53800cde0 con 0x7fc548074bd0 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.351+0000 7fc545600640 1 -- 172.21.15.46:0/3813397334 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5380035f0 con 0x7fc548074bd0 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.351+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481a1f60 con 0x7fc548074bd0 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.351+0000 7fc545600640 1 -- 172.21.15.46:0/3813397334 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538010670 con 0x7fc548074bd0 2024-09-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.351+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a23c0 con 0x7fc548074bd0 2024-09-10T07:36:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.352+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc548108780 con 0x7fc548074bd0 2024-09-10T07:36:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.354+0000 7fc545600640 1 -- 172.21.15.46:0/3813397334 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc5380037a0 con 0x7fc548074bd0 2024-09-10T07:36:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.355+0000 7fc545600640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc52c0a2b90 0x7fc52c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.356+0000 7fc545600640 1 -- 172.21.15.46:0/3813397334 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc5380cf700 con 0x7fc548074bd0 2024-09-10T07:36:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.356+0000 7fc53fe00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc52c0a2b90 0x7fc52c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:31.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.356+0000 7fc53fe00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc52c0a2b90 0x7fc52c0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc5300314e0 tx=0x7fc530000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:31.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.357+0000 7fc545600640 1 -- 172.21.15.46:0/3813397334 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc53809c280 con 0x7fc548074bd0 2024-09-10T07:36:31.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:31 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1839777448' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T07:36:31.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:31 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1839777448' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T07:36:31.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:31 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1839777448' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T07:36:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.558+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc54802da50 con 0x7fc548074bd0 2024-09-10T07:36:31.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.559+0000 7fc545600640 1 -- 172.21.15.46:0/3813397334 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24700 (secure 0 0 0) 0x7fc53809bb80 con 0x7fc548074bd0 2024-09-10T07:36:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:31.559 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:35:14.826944+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:36:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc52c0a2b90 msgr2=0x7fc52c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc52c0a2b90 0x7fc52c0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc5300314e0 tx=0x7fc530000bc0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548074bd0 msgr2=0x7fc54819a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548074bd0 0x7fc54819a050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc53800ea60 tx=0x7fc53800ef20 comp rx=0 tx=0).stop 2024-09-10T07:36:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 shutdown_connections 2024-09-10T07:36:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc52c0a2b90 0x7fc52c0a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5481099c0 0x7fc5481a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5480761b0 0x7fc54819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 --2- 172.21.15.46:0/3813397334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc548074bd0 0x7fc54819a050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 >> 172.21.15.46:0/3813397334 conn(0x7fc5480fd2b0 msgr2=0x7fc548105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 shutdown_connections 2024-09-10T07:36:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.562+0000 7fc54ee00640 1 -- 172.21.15.46:0/3813397334 wait complete. 2024-09-10T07:36:31.577 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:28:36.994371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T07:33:30.775288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T07:33:33.558621+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T07:33:43.433746+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T07:36:31.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T07:36:31.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 -- 172.21.15.46:0/992704871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf4c104f80 msgr2=0x7faf4c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 --2- 172.21.15.46:0/992704871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf4c104f80 0x7faf4c107370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf34009860 tx=0x7faf3402a340 comp rx=0 tx=0).stop 2024-09-10T07:36:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 -- 172.21.15.46:0/992704871 shutdown_connections 2024-09-10T07:36:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 --2- 172.21.15.46:0/992704871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf4c1078b0 0x7faf4c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 --2- 172.21.15.46:0/992704871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf4c104f80 0x7faf4c107370 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 --2- 172.21.15.46:0/992704871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4c102650 0x7faf4c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 -- 172.21.15.46:0/992704871 >> 172.21.15.46:0/992704871 conn(0x7faf4c0fc1b0 msgr2=0x7faf4c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 -- 172.21.15.46:0/992704871 shutdown_connections 2024-09-10T07:36:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.739+0000 7faf51c00640 1 -- 172.21.15.46:0/992704871 wait complete. 2024-09-10T07:36:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf51c00640 1 Processor -- start 2024-09-10T07:36:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf51c00640 1 -- start start 2024-09-10T07:36:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf51c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4c102650 0x7faf4c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf51c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf4c104f80 0x7faf4c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf51c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf4c1078b0 0x7faf4c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf51c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c10d7a0 con 0x7faf4c104f80 2024-09-10T07:36:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf51c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c10d620 con 0x7faf4c1078b0 2024-09-10T07:36:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf51c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c10d920 con 0x7faf4c102650 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf43e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf4c104f80 0x7faf4c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf43e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf4c104f80 0x7faf4c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48642/0 (socket says 172.21.15.46:48642) 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf43e00640 1 -- 172.21.15.46:0/731176683 learned_addr learned my addr 172.21.15.46:0/731176683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf4b400640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf4c1078b0 0x7faf4c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.740+0000 7faf4aa00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4c102650 0x7faf4c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf43e00640 1 -- 172.21.15.46:0/731176683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4c102650 msgr2=0x7faf4c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf43e00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4c102650 0x7faf4c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf43e00640 1 -- 172.21.15.46:0/731176683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf4c1078b0 msgr2=0x7faf4c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf43e00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf4c1078b0 0x7faf4c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf43e00640 1 -- 172.21.15.46:0/731176683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c1a1ce0 con 0x7faf4c104f80 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf43e00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf4c104f80 0x7faf4c19a600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faf34035040 tx=0x7faf34033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf43400640 1 -- 172.21.15.46:0/731176683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34037040 con 0x7faf4c104f80 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c1a1f70 con 0x7faf4c104f80 2024-09-10T07:36:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.741+0000 7faf43400640 1 -- 172.21.15.46:0/731176683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf34030850 con 0x7faf4c104f80 2024-09-10T07:36:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.742+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c1a23d0 con 0x7faf4c104f80 2024-09-10T07:36:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.742+0000 7faf43400640 1 -- 172.21.15.46:0/731176683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf340048c0 con 0x7faf4c104f80 2024-09-10T07:36:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.743+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c1093a0 con 0x7faf4c104f80 2024-09-10T07:36:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.745+0000 7faf43400640 1 -- 172.21.15.46:0/731176683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf34004a60 con 0x7faf4c104f80 2024-09-10T07:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.746+0000 7faf43400640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf300a2b90 0x7faf300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.746+0000 7faf43400640 1 -- 172.21.15.46:0/731176683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faf340ef330 con 0x7faf4c104f80 2024-09-10T07:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.746+0000 7faf4aa00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf300a2b90 0x7faf300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.747+0000 7faf4aa00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf300a2b90 0x7faf300a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00bbe0 tx=0x7faf3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.751+0000 7faf43400640 1 -- 172.21.15.46:0/731176683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf340bbeb0 con 0x7faf4c104f80 2024-09-10T07:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.936+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf4c062e90 con 0x7faf4c104f80 2024-09-10T07:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.936+0000 7faf43400640 1 -- 172.21.15.46:0/731176683 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7faf340bb7b0 con 0x7faf4c104f80 2024-09-10T07:36:31.936 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf300a2b90 msgr2=0x7faf300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf300a2b90 0x7faf300a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00bbe0 tx=0x7faf3c00b390 comp rx=0 tx=0).stop 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf4c104f80 msgr2=0x7faf4c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf4c104f80 0x7faf4c19a600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faf34035040 tx=0x7faf34033d90 comp rx=0 tx=0).stop 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 shutdown_connections 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf4c1078b0 0x7faf4c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf300a2b90 0x7faf300a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf4c104f80 0x7faf4c19a600 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 --2- 172.21.15.46:0/731176683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4c102650 0x7faf4c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 >> 172.21.15.46:0/731176683 conn(0x7faf4c0fc1b0 msgr2=0x7faf4c101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 shutdown_connections 2024-09-10T07:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:31.938+0000 7faf51c00640 1 -- 172.21.15.46:0/731176683 wait complete. 2024-09-10T07:36:31.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 -- 172.21.15.46:0/3437006520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c104f10 msgr2=0x7f293c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 --2- 172.21.15.46:0/3437006520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c104f10 0x7f293c105390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2928009900 tx=0x7f292802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 -- 172.21.15.46:0/3437006520 shutdown_connections 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 --2- 172.21.15.46:0/3437006520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f293c1058d0 0x7f293c10c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 --2- 172.21.15.46:0/3437006520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c104f10 0x7f293c105390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 --2- 172.21.15.46:0/3437006520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f293c103d20 0x7f293c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 -- 172.21.15.46:0/3437006520 >> 172.21.15.46:0/3437006520 conn(0x7f293c0ff4a0 msgr2=0x7f293c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 -- 172.21.15.46:0/3437006520 shutdown_connections 2024-09-10T07:36:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.116+0000 7f2943800640 1 -- 172.21.15.46:0/3437006520 wait complete. 2024-09-10T07:36:32.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.117+0000 7f2943800640 1 Processor -- start 2024-09-10T07:36:32.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.117+0000 7f2943800640 1 -- start start 2024-09-10T07:36:32.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.117+0000 7f2943800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f293c103d20 0x7f293c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.117+0000 7f2943800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f293c104f10 0x7f293c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.117+0000 7f2943800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c1058d0 0x7f293c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.117+0000 7f2943800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f293c10f980 con 0x7f293c1058d0 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.117+0000 7f2943800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f293c10f800 con 0x7f293c104f10 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.117+0000 7f2943800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f293c10fb00 con 0x7f293c103d20 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2941800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c1058d0 0x7f293c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f293be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f293c104f10 0x7f293c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f293be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f293c104f10 0x7f293c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50804/0 (socket says 172.21.15.46:50804) 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2941800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c1058d0 0x7f293c1a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48654/0 (socket says 172.21.15.46:48654) 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f293be00640 1 -- 172.21.15.46:0/1378239244 learned_addr learned my addr 172.21.15.46:0/1378239244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2940e00640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f293c103d20 0x7f293c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2941800640 1 -- 172.21.15.46:0/1378239244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f293c103d20 msgr2=0x7f293c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2941800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f293c103d20 0x7f293c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2941800640 1 -- 172.21.15.46:0/1378239244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f293c104f10 msgr2=0x7f293c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2941800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f293c104f10 0x7f293c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2941800640 1 -- 172.21.15.46:0/1378239244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c1a1c90 con 0x7f293c1058d0 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f2940e00640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f293c103d20 0x7f293c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.118+0000 7f293be00640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f293c104f10 0x7f293c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.119+0000 7f2941800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c1058d0 0x7f293c1a1620 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2934009fd0 tx=0x7f293400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.119+0000 7f2939600640 1 -- 172.21.15.46:0/1378239244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293400ce20 con 0x7f293c1058d0 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.119+0000 7f2939600640 1 -- 172.21.15.46:0/1378239244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2934005020 con 0x7f293c1058d0 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.119+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c1a1f80 con 0x7f293c1058d0 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.119+0000 7f2939600640 1 -- 172.21.15.46:0/1378239244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934010720 con 0x7f293c1058d0 2024-09-10T07:36:32.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.119+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c1a23f0 con 0x7f293c1058d0 2024-09-10T07:36:32.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.120+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293c10b180 con 0x7f293c1058d0 2024-09-10T07:36:32.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.122+0000 7f2939600640 1 -- 172.21.15.46:0/1378239244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2934005540 con 0x7f293c1058d0 2024-09-10T07:36:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.123+0000 7f2939600640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29240a2b90 0x7f29240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.123+0000 7f2939600640 1 -- 172.21.15.46:0/1378239244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f29340cfbe0 con 0x7f293c1058d0 2024-09-10T07:36:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.124+0000 7f2940e00640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29240a2b90 0x7f29240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.124+0000 7f2940e00640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29240a2b90 0x7f29240a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f293c104d70 tx=0x7f2930005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:32.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.127+0000 7f2939600640 1 -- 172.21.15.46:0/1378239244 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f293409c760 con 0x7f293c1058d0 2024-09-10T07:36:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.311+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f293c062e90 con 0x7f293c1058d0 2024-09-10T07:36:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.312+0000 7f2939600640 1 -- 172.21.15.46:0/1378239244 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f293409c060 con 0x7f293c1058d0 2024-09-10T07:36:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29240a2b90 msgr2=0x7f29240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29240a2b90 0x7f29240a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f293c104d70 tx=0x7f2930005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c1058d0 msgr2=0x7f293c1a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c1058d0 0x7f293c1a1620 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2934009fd0 tx=0x7f293400ef10 comp rx=0 tx=0).stop 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 shutdown_connections 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29240a2b90 0x7f29240a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c1058d0 0x7f293c1a1620 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f293c104f10 0x7f293c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 --2- 172.21.15.46:0/1378239244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f293c103d20 0x7f293c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 >> 172.21.15.46:0/1378239244 conn(0x7f293c0ff4a0 msgr2=0x7f293c108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 shutdown_connections 2024-09-10T07:36:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.314+0000 7f2943800640 1 -- 172.21.15.46:0/1378239244 wait complete. 2024-09-10T07:36:32.328 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:32 smithi046 ceph-mon[30705]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:32.328 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:32 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1839777448' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T07:36:32.328 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:32 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:36:32.328 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:32 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3813397334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:32.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:32 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/731176683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:36:32.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T07:36:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:32 smithi181 ceph-mon[35397]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:32 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1839777448' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T07:36:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:32 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:36:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:32 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3813397334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:32.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:32 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/731176683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:36:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:32 smithi063 ceph-mon[35020]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T07:36:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:32 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1839777448' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T07:36:32.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:32 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:36:32.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:32 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3813397334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:32.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:32 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/731176683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:36:32.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.492+0000 7f7e75000640 1 -- 172.21.15.46:0/1487133163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e700761d0 msgr2=0x7f7e70074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.492+0000 7f7e75000640 1 --2- 172.21.15.46:0/1487133163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e700761d0 0x7f7e70074620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009900 tx=0x7f7e6002a480 comp rx=0 tx=0).stop 2024-09-10T07:36:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.493+0000 7f7e75000640 1 -- 172.21.15.46:0/1487133163 shutdown_connections 2024-09-10T07:36:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.493+0000 7f7e75000640 1 --2- 172.21.15.46:0/1487133163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e70109990 0x7f7e7010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.493+0000 7f7e75000640 1 --2- 172.21.15.46:0/1487133163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70074bf0 0x7f7e70075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.493+0000 7f7e75000640 1 --2- 172.21.15.46:0/1487133163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e700761d0 0x7f7e70074620 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.493+0000 7f7e75000640 1 -- 172.21.15.46:0/1487133163 >> 172.21.15.46:0/1487133163 conn(0x7f7e700fd2a0 msgr2=0x7f7e700ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.493+0000 7f7e75000640 1 -- 172.21.15.46:0/1487133163 shutdown_connections 2024-09-10T07:36:32.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.493+0000 7f7e75000640 1 -- 172.21.15.46:0/1487133163 wait complete. 2024-09-10T07:36:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.493+0000 7f7e75000640 1 Processor -- start 2024-09-10T07:36:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e75000640 1 -- start start 2024-09-10T07:36:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e75000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e70074bf0 0x7f7e7019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e75000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e700761d0 0x7f7e7019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e75000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70109990 0x7f7e701a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e75000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010f9e0 con 0x7f7e70109990 2024-09-10T07:36:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e75000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010f860 con 0x7f7e700761d0 2024-09-10T07:36:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e75000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010fb60 con 0x7f7e70074bf0 2024-09-10T07:36:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e6e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70109990 0x7f7e701a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e6e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70109990 0x7f7e701a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48674/0 (socket says 172.21.15.46:48674) 2024-09-10T07:36:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e6e800640 1 -- 172.21.15.46:0/3191797668 learned_addr learned my addr 172.21.15.46:0/3191797668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e6de00640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e70074bf0 0x7f7e7019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.494+0000 7f7e67e00640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e700761d0 0x7f7e7019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e6e800640 1 -- 172.21.15.46:0/3191797668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e70074bf0 msgr2=0x7f7e7019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e6e800640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e70074bf0 0x7f7e7019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e6e800640 1 -- 172.21.15.46:0/3191797668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e700761d0 msgr2=0x7f7e7019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e6e800640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e700761d0 0x7f7e7019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e6e800640 1 -- 172.21.15.46:0/3191797668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e701a1c80 con 0x7f7e70109990 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e6e800640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70109990 0x7f7e701a1610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800eb10 tx=0x7f7e6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e66a00640 1 -- 172.21.15.46:0/3191797668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68004b20 con 0x7f7e70109990 2024-09-10T07:36:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e66a00640 1 -- 172.21.15.46:0/3191797668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e68004cc0 con 0x7f7e70109990 2024-09-10T07:36:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e701a1f70 con 0x7f7e70109990 2024-09-10T07:36:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e66a00640 1 -- 172.21.15.46:0/3191797668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68010760 con 0x7f7e70109990 2024-09-10T07:36:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.495+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e701a2430 con 0x7f7e70109990 2024-09-10T07:36:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.496+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e70108750 con 0x7f7e70109990 2024-09-10T07:36:32.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.498+0000 7f7e66a00640 1 -- 172.21.15.46:0/3191797668 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7e68005020 con 0x7f7e70109990 2024-09-10T07:36:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.500+0000 7f7e66a00640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7e540a2b90 0x7f7e540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.500+0000 7f7e66a00640 1 -- 172.21.15.46:0/3191797668 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7e680cfac0 con 0x7f7e70109990 2024-09-10T07:36:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.501+0000 7f7e6de00640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7e540a2b90 0x7f7e540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.501+0000 7f7e6de00640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7e540a2b90 0x7f7e540a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7e60031440 tx=0x7f7e600315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.504+0000 7f7e66a00640 1 -- 172.21.15.46:0/3191797668 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e6809c640 con 0x7f7e70109990 2024-09-10T07:36:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.689+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7e70062e90 con 0x7f7e70109990 2024-09-10T07:36:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.689+0000 7f7e66a00640 1 -- 172.21.15.46:0/3191797668 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f7e6809bf40 con 0x7f7e70109990 2024-09-10T07:36:32.689 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-09-10T07:36:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7e540a2b90 msgr2=0x7f7e540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7e540a2b90 0x7f7e540a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7e60031440 tx=0x7f7e600315b0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70109990 msgr2=0x7f7e701a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70109990 0x7f7e701a1610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800eb10 tx=0x7f7e6800c2d0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 shutdown_connections 2024-09-10T07:36:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7e540a2b90 0x7f7e540a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70109990 0x7f7e701a1610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e700761d0 0x7f7e7019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 --2- 172.21.15.46:0/3191797668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e70074bf0 0x7f7e7019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 >> 172.21.15.46:0/3191797668 conn(0x7f7e700fd2a0 msgr2=0x7f7e70105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 shutdown_connections 2024-09-10T07:36:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.692+0000 7f7e75000640 1 -- 172.21.15.46:0/3191797668 wait complete. 2024-09-10T07:36:32.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.869+0000 7f53e9200640 1 -- 172.21.15.46:0/3606025645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e41058d0 msgr2=0x7f53e410c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.869+0000 7f53e9200640 1 --2- 172.21.15.46:0/3606025645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e41058d0 0x7f53e410c160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009ca0 tx=0x7f53dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.869+0000 7f53e9200640 1 -- 172.21.15.46:0/3606025645 shutdown_connections 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.869+0000 7f53e9200640 1 --2- 172.21.15.46:0/3606025645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e41058d0 0x7f53e410c160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.869+0000 7f53e9200640 1 --2- 172.21.15.46:0/3606025645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4104f10 0x7f53e4105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.869+0000 7f53e9200640 1 --2- 172.21.15.46:0/3606025645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e4103d20 0x7f53e4104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.870+0000 7f53e9200640 1 -- 172.21.15.46:0/3606025645 >> 172.21.15.46:0/3606025645 conn(0x7f53e40ff4a0 msgr2=0x7f53e41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.870+0000 7f53e9200640 1 -- 172.21.15.46:0/3606025645 shutdown_connections 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.870+0000 7f53e9200640 1 -- 172.21.15.46:0/3606025645 wait complete. 2024-09-10T07:36:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.870+0000 7f53e9200640 1 Processor -- start 2024-09-10T07:36:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e9200640 1 -- start start 2024-09-10T07:36:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4103d20 0x7f53e4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e9200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e4104f10 0x7f53e419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e41058d0 0x7f53e41a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410f9f0 con 0x7f53e4103d20 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e9200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410f870 con 0x7f53e41058d0 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e9200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410fb70 con 0x7f53e4104f10 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4103d20 0x7f53e4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4103d20 0x7f53e4199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48694/0 (socket says 172.21.15.46:48694) 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e1e00640 1 -- 172.21.15.46:0/3817633845 learned_addr learned my addr 172.21.15.46:0/3817633845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53e2800640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e41058d0 0x7f53e41a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.871+0000 7f53d9400640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e4104f10 0x7f53e419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53e1e00640 1 -- 172.21.15.46:0/3817633845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e4104f10 msgr2=0x7f53e419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53e1e00640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e4104f10 0x7f53e419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53e1e00640 1 -- 172.21.15.46:0/3817633845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e41058d0 msgr2=0x7f53e41a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53e1e00640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e41058d0 0x7f53e41a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53e1e00640 1 -- 172.21.15.46:0/3817633845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e41a1c00 con 0x7f53e4103d20 2024-09-10T07:36:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53e1e00640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4103d20 0x7f53e4199fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53d400e9c0 tx=0x7f53d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53db400640 1 -- 172.21.15.46:0/3817633845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400cd80 con 0x7f53e4103d20 2024-09-10T07:36:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e41a1ef0 con 0x7f53e4103d20 2024-09-10T07:36:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53db400640 1 -- 172.21.15.46:0/3817633845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d40035f0 con 0x7f53e4103d20 2024-09-10T07:36:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53db400640 1 -- 172.21.15.46:0/3817633845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4010690 con 0x7f53e4103d20 2024-09-10T07:36:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.872+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e41a2350 con 0x7f53e4103d20 2024-09-10T07:36:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.873+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e410b0c0 con 0x7f53e4103d20 2024-09-10T07:36:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.875+0000 7f53db400640 1 -- 172.21.15.46:0/3817633845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f53d40037a0 con 0x7f53e4103d20 2024-09-10T07:36:32.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.876+0000 7f53db400640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53c80a2b90 0x7f53c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:32.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.877+0000 7f53db400640 1 -- 172.21.15.46:0/3817633845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f53d40cf770 con 0x7f53e4103d20 2024-09-10T07:36:32.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.877+0000 7f53d9400640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53c80a2b90 0x7f53c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:32.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.877+0000 7f53d9400640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53c80a2b90 0x7f53c80a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0095f0 tx=0x7f53cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:32.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:32.880+0000 7f53db400640 1 -- 172.21.15.46:0/3817633845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53d409c2f0 con 0x7f53e4103d20 2024-09-10T07:36:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.066+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f53e4062f30 con 0x7f53e4103d20 2024-09-10T07:36:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.066+0000 7f53db400640 1 -- 172.21.15.46:0/3817633845 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f53d409bbf0 con 0x7f53e4103d20 2024-09-10T07:36:33.066 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-09-10T07:36:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53c80a2b90 msgr2=0x7f53c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53c80a2b90 0x7f53c80a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0095f0 tx=0x7f53cc005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4103d20 msgr2=0x7f53e4199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4103d20 0x7f53e4199fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53d400e9c0 tx=0x7f53d400ee80 comp rx=0 tx=0).stop 2024-09-10T07:36:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 shutdown_connections 2024-09-10T07:36:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e41058d0 0x7f53e41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e4104f10 0x7f53e419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53c80a2b90 0x7f53c80a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 --2- 172.21.15.46:0/3817633845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4103d20 0x7f53e4199fe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.068+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 >> 172.21.15.46:0/3817633845 conn(0x7f53e40ff4a0 msgr2=0x7f53e4108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.069+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 shutdown_connections 2024-09-10T07:36:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:33.069+0000 7f53e9200640 1 -- 172.21.15.46:0/3817633845 wait complete. 2024-09-10T07:36:33.085 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T07:36:33.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:36:33.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:33 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1378239244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:33 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3191797668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:33 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3817633845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:33 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1378239244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:33 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3191797668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:33 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3817633845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:33 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1378239244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:33 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3191797668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:33 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3817633845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T07:36:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:34.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 -- 172.21.15.46:0/184920802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608112b90 msgr2=0x7f0608114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 --2- 172.21.15.46:0/184920802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608112b90 0x7f0608114f80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0604009ca0 tx=0x7f060402a950 comp rx=0 tx=0).stop 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 -- 172.21.15.46:0/184920802 shutdown_connections 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 --2- 172.21.15.46:0/184920802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608112b90 0x7f0608114f80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 --2- 172.21.15.46:0/184920802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0608074eb0 0x7f0608112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 --2- 172.21.15.46:0/184920802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0608074590 0x7f0608074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 -- 172.21.15.46:0/184920802 >> 172.21.15.46:0/184920802 conn(0x7f06080fd8c0 msgr2=0x7f06080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 -- 172.21.15.46:0/184920802 shutdown_connections 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.060+0000 7f0610800640 1 -- 172.21.15.46:0/184920802 wait complete. 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.061+0000 7f0610800640 1 Processor -- start 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.061+0000 7f0610800640 1 -- start start 2024-09-10T07:36:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f0610800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608074590 0x7f060819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f0610800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0608074eb0 0x7f060819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f0610800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0608112b90 0x7f06081a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f0610800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06081185c0 con 0x7f0608074590 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f0610800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0608118440 con 0x7f0608074eb0 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f0610800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0608118740 con 0x7f0608112b90 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608074590 0x7f060819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0608074eb0 0x7f060819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608074590 0x7f060819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48710/0 (socket says 172.21.15.46:48710) 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060de00640 1 -- 172.21.15.46:0/2905340851 learned_addr learned my addr 172.21.15.46:0/2905340851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060e800640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0608112b90 0x7f06081a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060de00640 1 -- 172.21.15.46:0/2905340851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0608112b90 msgr2=0x7f06081a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060de00640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0608112b90 0x7f06081a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060de00640 1 -- 172.21.15.46:0/2905340851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0608074eb0 msgr2=0x7f060819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060de00640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0608074eb0 0x7f060819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060de00640 1 -- 172.21.15.46:0/2905340851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06081a3b90 con 0x7f0608074590 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.062+0000 7f060d400640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0608074eb0 0x7f060819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.063+0000 7f060de00640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608074590 0x7f060819eb60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00e9f0 tx=0x7f05fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.063+0000 7f0602a00640 1 -- 172.21.15.46:0/2905340851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00cd80 con 0x7f0608074590 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.063+0000 7f0602a00640 1 -- 172.21.15.46:0/2905340851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05fc010910 con 0x7f0608074590 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.063+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06081a3e80 con 0x7f0608074590 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.063+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06081ab710 con 0x7f0608074590 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.063+0000 7f0602a00640 1 -- 172.21.15.46:0/2905340851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc018930 con 0x7f0608074590 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.064+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0608074970 con 0x7f0608074590 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.066+0000 7f0602a00640 1 -- 172.21.15.46:0/2905340851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f05fc010ac0 con 0x7f0608074590 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.068+0000 7f0602a00640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05f00a2b80 0x7f05f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:34.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.068+0000 7f0602a00640 1 -- 172.21.15.46:0/2905340851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f05fc0cf6e0 con 0x7f0608074590 2024-09-10T07:36:34.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.068+0000 7f060d400640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05f00a2b80 0x7f05f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:34.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.069+0000 7f060d400640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05f00a2b80 0x7f05f00a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06080fff20 tx=0x7f05f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:34.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.072+0000 7f0602a00640 1 -- 172.21.15.46:0/2905340851 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05fc09bc10 con 0x7f0608074590 2024-09-10T07:36:34.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.293+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0608104fd0 con 0x7f0608074590 2024-09-10T07:36:34.294 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:34 smithi046 ceph-mon[30705]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:34.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.294+0000 7f0602a00640 1 -- 172.21.15.46:0/2905340851 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f05fc09ff10 con 0x7f0608074590 2024-09-10T07:36:34.295 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:34.296 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"btime":"2024-09-10T07:36:31:116553+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:36:30.952730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:36:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-09-10T07:36:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.296+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05f00a2b80 msgr2=0x7f05f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.296+0000 7f0610800640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05f00a2b80 0x7f05f00a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06080fff20 tx=0x7f05f8005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.296+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608074590 msgr2=0x7f060819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.296+0000 7f0610800640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608074590 0x7f060819eb60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00e9f0 tx=0x7f05fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.297+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 shutdown_connections 2024-09-10T07:36:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.297+0000 7f0610800640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0608112b90 0x7f06081a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.297+0000 7f0610800640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0608074eb0 0x7f060819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.297+0000 7f0610800640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05f00a2b80 0x7f05f00a5040 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.297+0000 7f0610800640 1 --2- 172.21.15.46:0/2905340851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0608074590 0x7f060819eb60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:34.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.297+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 >> 172.21.15.46:0/2905340851 conn(0x7f06080fd8c0 msgr2=0x7f06080ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:34.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.297+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 shutdown_connections 2024-09-10T07:36:34.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:34.297+0000 7f0610800640 1 -- 172.21.15.46:0/2905340851 wait complete. 2024-09-10T07:36:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:34 smithi181 ceph-mon[35397]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:34 smithi063 ceph-mon[35020]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T07:36:35.258 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T07:36:35.258 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T07:36:35.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:36:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:35 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2905340851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:35.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:35 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2905340851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:35 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2905340851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 -- 172.21.15.46:0/556590403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8104f20 msgr2=0x7f6fe81053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 --2- 172.21.15.46:0/556590403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8104f20 0x7f6fe81053a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009900 tx=0x7f6fd402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 -- 172.21.15.46:0/556590403 shutdown_connections 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 --2- 172.21.15.46:0/556590403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fe81058e0 0x7f6fe810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 --2- 172.21.15.46:0/556590403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8104f20 0x7f6fe81053a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 --2- 172.21.15.46:0/556590403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fe8103d30 0x7f6fe8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 -- 172.21.15.46:0/556590403 >> 172.21.15.46:0/556590403 conn(0x7f6fe80ff4b0 msgr2=0x7f6fe81018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 -- 172.21.15.46:0/556590403 shutdown_connections 2024-09-10T07:36:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.420+0000 7f6fed600640 1 -- 172.21.15.46:0/556590403 wait complete. 2024-09-10T07:36:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.421+0000 7f6fed600640 1 Processor -- start 2024-09-10T07:36:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.421+0000 7f6fed600640 1 -- start start 2024-09-10T07:36:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.421+0000 7f6fed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8103d30 0x7f6fe819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.421+0000 7f6fed600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fe8104f20 0x7f6fe819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.421+0000 7f6fed600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fe81058e0 0x7f6fe81a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.421+0000 7f6fed600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe810f9e0 con 0x7f6fe8103d30 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.421+0000 7f6fed600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe810f860 con 0x7f6fe81058e0 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.421+0000 7f6fed600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe810fb60 con 0x7f6fe8104f20 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8103d30 0x7f6fe819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8103d30 0x7f6fe819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48740/0 (socket says 172.21.15.46:48740) 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 -- 172.21.15.46:0/4271410451 learned_addr learned my addr 172.21.15.46:0/4271410451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe7400640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fe81058e0 0x7f6fe81a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6000640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fe8104f20 0x7f6fe819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 -- 172.21.15.46:0/4271410451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fe8104f20 msgr2=0x7f6fe819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fe8104f20 0x7f6fe819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 -- 172.21.15.46:0/4271410451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fe81058e0 msgr2=0x7f6fe81a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fe81058e0 0x7f6fe81a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 -- 172.21.15.46:0/4271410451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe81a1c60 con 0x7f6fe8103d30 2024-09-10T07:36:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.422+0000 7f6fe6a00640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8103d30 0x7f6fe819a040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800d770 tx=0x7f6fd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.423+0000 7f6fdf400640 1 -- 172.21.15.46:0/4271410451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8017070 con 0x7f6fe8103d30 2024-09-10T07:36:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.423+0000 7f6fdf400640 1 -- 172.21.15.46:0/4271410451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd8009c30 con 0x7f6fe8103d30 2024-09-10T07:36:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.423+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe81a1f50 con 0x7f6fe8103d30 2024-09-10T07:36:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.423+0000 7f6fdf400640 1 -- 172.21.15.46:0/4271410451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8003dd0 con 0x7f6fe8103d30 2024-09-10T07:36:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.423+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe81a23c0 con 0x7f6fe8103d30 2024-09-10T07:36:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.424+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe810b140 con 0x7f6fe8103d30 2024-09-10T07:36:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.426+0000 7f6fdf400640 1 -- 172.21.15.46:0/4271410451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6fd8003fb0 con 0x7f6fe8103d30 2024-09-10T07:36:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.427+0000 7f6fdf400640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fcc0a2c10 0x7f6fcc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.427+0000 7f6fdf400640 1 -- 172.21.15.46:0/4271410451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6fd80cece0 con 0x7f6fe8103d30 2024-09-10T07:36:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.427+0000 7f6fe6000640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fcc0a2c10 0x7f6fcc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.428+0000 7f6fe6000640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fcc0a2c10 0x7f6fcc0a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4031440 tx=0x7f6fd40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.431+0000 7f6fdf400640 1 -- 172.21.15.46:0/4271410451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fd8009de0 con 0x7f6fe8103d30 2024-09-10T07:36:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.613+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fe8062630 con 0x7f6fcc0a2c10 2024-09-10T07:36:35.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.622+0000 7f6fdf400640 1 -- 172.21.15.46:0/4271410451 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340777 (secure 0 0 0) 0x7f6fe8062630 con 0x7f6fcc0a2c10 2024-09-10T07:36:35.627 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:36:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fcc0a2c10 msgr2=0x7f6fcc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fcc0a2c10 0x7f6fcc0a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4031440 tx=0x7f6fd40315b0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8103d30 msgr2=0x7f6fe819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8103d30 0x7f6fe819a040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800d770 tx=0x7f6fd800dc30 comp rx=0 tx=0).stop 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 shutdown_connections 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fe81058e0 0x7f6fe81a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fe8104f20 0x7f6fe819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6fcc0a2c10 0x7f6fcc0a50d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 --2- 172.21.15.46:0/4271410451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8103d30 0x7f6fe819a040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 >> 172.21.15.46:0/4271410451 conn(0x7f6fe80ff4b0 msgr2=0x7f6fe8108310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 shutdown_connections 2024-09-10T07:36:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.627+0000 7f6fed600640 1 -- 172.21.15.46:0/4271410451 wait complete. 2024-09-10T07:36:35.642 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-10T07:36:34.507732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668647,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1435,"num_read_kb":853,"num_write":448,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358316,"kb_used_data":13228,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583508,"statfs":{"total":1151940427776,"available":1151573512192,"internally_reserved":0,"allocated":13545472,"data_stored":8695878,"data_compressed":498372,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003122"},"pg_stats":[{"pgid":"2.1f","version":"97'14","reported_seq":186,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.007589+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:36:32.007589+0000","last_peered":"2024-09-10T07:36:32.007589+0000","last_clean":"2024-09-10T07:36:32.007589+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:36:32.007589+0000","last_undegraded":"2024-09-10T07:36:32.007589+0000","last_fullsized":"2024-09-10T07:36:32.007589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475738+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:35:42.475738+0000","last_peered":"2024-09-10T07:35:42.475738+0000","last_clean":"2024-09-10T07:35:42.475738+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:35:42.475738+0000","last_undegraded":"2024-09-10T07:35:42.475738+0000","last_fullsized":"2024-09-10T07:35:42.475738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842368+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:35:14.842368+0000","last_peered":"2024-09-10T07:35:14.842368+0000","last_clean":"2024-09-10T07:35:14.842368+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:35:14.842368+0000","last_undegraded":"2024-09-10T07:35:14.842368+0000","last_fullsized":"2024-09-10T07:35:14.842368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977747+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:35:14.977747+0000","last_peered":"2024-09-10T07:35:14.977747+0000","last_clean":"2024-09-10T07:35:14.977747+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:35:14.977747+0000","last_undegraded":"2024-09-10T07:35:14.977747+0000","last_fullsized":"2024-09-10T07:35:14.977747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502886+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:35:42.502886+0000","last_peered":"2024-09-10T07:35:42.502886+0000","last_clean":"2024-09-10T07:35:42.502886+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:35:42.502886+0000","last_undegraded":"2024-09-10T07:35:42.502886+0000","last_fullsized":"2024-09-10T07:35:42.502886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881794+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:35:14.881794+0000","last_peered":"2024-09-10T07:35:14.881794+0000","last_clean":"2024-09-10T07:35:14.881794+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:35:14.881794+0000","last_undegraded":"2024-09-10T07:35:14.881794+0000","last_fullsized":"2024-09-10T07:35:14.881794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854574+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:35:14.854574+0000","last_peered":"2024-09-10T07:35:14.854574+0000","last_clean":"2024-09-10T07:35:14.854574+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:35:14.854574+0000","last_undegraded":"2024-09-10T07:35:14.854574+0000","last_fullsized":"2024-09-10T07:35:14.854574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.495636+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:35:42.495636+0000","last_peered":"2024-09-10T07:35:42.495636+0000","last_clean":"2024-09-10T07:35:42.495636+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:35:42.495636+0000","last_undegraded":"2024-09-10T07:35:42.495636+0000","last_fullsized":"2024-09-10T07:35:42.495636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881984+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:35:14.881984+0000","last_peered":"2024-09-10T07:35:14.881984+0000","last_clean":"2024-09-10T07:35:14.881984+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:35:14.881984+0000","last_undegraded":"2024-09-10T07:35:14.881984+0000","last_fullsized":"2024-09-10T07:35:14.881984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854411+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:35:14.854411+0000","last_peered":"2024-09-10T07:35:14.854411+0000","last_clean":"2024-09-10T07:35:14.854411+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:35:14.854411+0000","last_undegraded":"2024-09-10T07:35:14.854411+0000","last_fullsized":"2024-09-10T07:35:14.854411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"97'15","reported_seq":60,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484819+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:35:42.484819+0000","last_peered":"2024-09-10T07:35:42.484819+0000","last_clean":"2024-09-10T07:35:42.484819+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:35:42.484819+0000","last_undegraded":"2024-09-10T07:35:42.484819+0000","last_fullsized":"2024-09-10T07:35:42.484819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881889+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:35:14.881889+0000","last_peered":"2024-09-10T07:35:14.881889+0000","last_clean":"2024-09-10T07:35:14.881889+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:35:14.881889+0000","last_undegraded":"2024-09-10T07:35:14.881889+0000","last_fullsized":"2024-09-10T07:35:14.881889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486825+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:35:42.486825+0000","last_peered":"2024-09-10T07:35:42.486825+0000","last_clean":"2024-09-10T07:35:42.486825+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:35:42.486825+0000","last_undegraded":"2024-09-10T07:35:42.486825+0000","last_fullsized":"2024-09-10T07:35:42.486825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854068+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:35:14.854068+0000","last_peered":"2024-09-10T07:35:14.854068+0000","last_clean":"2024-09-10T07:35:14.854068+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:35:14.854068+0000","last_undegraded":"2024-09-10T07:35:14.854068+0000","last_fullsized":"2024-09-10T07:35:14.854068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882150+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:35:14.882150+0000","last_peered":"2024-09-10T07:35:14.882150+0000","last_clean":"2024-09-10T07:35:14.882150+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:35:14.882150+0000","last_undegraded":"2024-09-10T07:35:14.882150+0000","last_fullsized":"2024-09-10T07:35:14.882150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501326+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:35:42.501326+0000","last_peered":"2024-09-10T07:35:42.501326+0000","last_clean":"2024-09-10T07:35:42.501326+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:35:42.501326+0000","last_undegraded":"2024-09-10T07:35:42.501326+0000","last_fullsized":"2024-09-10T07:35:42.501326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842483+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:35:14.842483+0000","last_peered":"2024-09-10T07:35:14.842483+0000","last_clean":"2024-09-10T07:35:14.842483+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:35:14.842483+0000","last_undegraded":"2024-09-10T07:35:14.842483+0000","last_fullsized":"2024-09-10T07:35:14.842483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882278+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:35:14.882278+0000","last_peered":"2024-09-10T07:35:14.882278+0000","last_clean":"2024-09-10T07:35:14.882278+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:35:14.882278+0000","last_undegraded":"2024-09-10T07:35:14.882278+0000","last_fullsized":"2024-09-10T07:35:14.882278+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502574+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:35:42.502574+0000","last_peered":"2024-09-10T07:35:42.502574+0000","last_clean":"2024-09-10T07:35:42.502574+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:35:42.502574+0000","last_undegraded":"2024-09-10T07:35:42.502574+0000","last_fullsized":"2024-09-10T07:35:42.502574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977257+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:35:14.977257+0000","last_peered":"2024-09-10T07:35:14.977257+0000","last_clean":"2024-09-10T07:35:14.977257+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:35:14.977257+0000","last_undegraded":"2024-09-10T07:35:14.977257+0000","last_fullsized":"2024-09-10T07:35:14.977257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851715+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:35:14.851715+0000","last_peered":"2024-09-10T07:35:14.851715+0000","last_clean":"2024-09-10T07:35:14.851715+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:35:14.851715+0000","last_undegraded":"2024-09-10T07:35:14.851715+0000","last_fullsized":"2024-09-10T07:35:14.851715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"97'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.459952+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:35:42.459952+0000","last_peered":"2024-09-10T07:35:42.459952+0000","last_clean":"2024-09-10T07:35:42.459952+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:35:42.459952+0000","last_undegraded":"2024-09-10T07:35:42.459952+0000","last_fullsized":"2024-09-10T07:35:42.459952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976752+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:35:14.976752+0000","last_peered":"2024-09-10T07:35:14.976752+0000","last_clean":"2024-09-10T07:35:14.976752+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:35:14.976752+0000","last_undegraded":"2024-09-10T07:35:14.976752+0000","last_fullsized":"2024-09-10T07:35:14.976752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882210+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:35:14.882210+0000","last_peered":"2024-09-10T07:35:14.882210+0000","last_clean":"2024-09-10T07:35:14.882210+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:35:14.882210+0000","last_undegraded":"2024-09-10T07:35:14.882210+0000","last_fullsized":"2024-09-10T07:35:14.882210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502703+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:35:42.502703+0000","last_peered":"2024-09-10T07:35:42.502703+0000","last_clean":"2024-09-10T07:35:42.502703+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:35:42.502703+0000","last_undegraded":"2024-09-10T07:35:42.502703+0000","last_fullsized":"2024-09-10T07:35:42.502703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882535+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:35:14.882535+0000","last_peered":"2024-09-10T07:35:14.882535+0000","last_clean":"2024-09-10T07:35:14.882535+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:35:14.882535+0000","last_undegraded":"2024-09-10T07:35:14.882535+0000","last_fullsized":"2024-09-10T07:35:14.882535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842382+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:35:14.842382+0000","last_peered":"2024-09-10T07:35:14.842382+0000","last_clean":"2024-09-10T07:35:14.842382+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:35:14.842382+0000","last_undegraded":"2024-09-10T07:35:14.842382+0000","last_fullsized":"2024-09-10T07:35:14.842382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499802+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:35:42.499802+0000","last_peered":"2024-09-10T07:35:42.499802+0000","last_clean":"2024-09-10T07:35:42.499802+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:35:42.499802+0000","last_undegraded":"2024-09-10T07:35:42.499802+0000","last_fullsized":"2024-09-10T07:35:42.499802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'18","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:43.451679+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:35:43.451679+0000","last_peered":"2024-09-10T07:35:43.451679+0000","last_clean":"2024-09-10T07:35:43.451679+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:35:43.451679+0000","last_undegraded":"2024-09-10T07:35:43.451679+0000","last_fullsized":"2024-09-10T07:35:43.451679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976909+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:35:14.976909+0000","last_peered":"2024-09-10T07:35:14.976909+0000","last_clean":"2024-09-10T07:35:14.976909+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:35:14.976909+0000","last_undegraded":"2024-09-10T07:35:14.976909+0000","last_fullsized":"2024-09-10T07:35:14.976909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.481577+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:35:42.481577+0000","last_peered":"2024-09-10T07:35:42.481577+0000","last_clean":"2024-09-10T07:35:42.481577+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:35:42.481577+0000","last_undegraded":"2024-09-10T07:35:42.481577+0000","last_fullsized":"2024-09-10T07:35:42.481577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.872520+0000","last_change":"2024-09-10T07:35:14.872520+0000","last_active":"2024-09-10T07:35:14.872520+0000","last_peered":"2024-09-10T07:35:14.872520+0000","last_clean":"2024-09-10T07:35:14.872520+0000","last_became_active":"2024-09-10T07:35:14.860686+0000","last_became_peered":"2024-09-10T07:35:14.860686+0000","last_unstale":"2024-09-10T07:35:14.872520+0000","last_undegraded":"2024-09-10T07:35:14.872520+0000","last_fullsized":"2024-09-10T07:35:14.872520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:42.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852355+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:35:14.852355+0000","last_peered":"2024-09-10T07:35:14.852355+0000","last_clean":"2024-09-10T07:35:14.852355+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:35:14.852355+0000","last_undegraded":"2024-09-10T07:35:14.852355+0000","last_fullsized":"2024-09-10T07:35:14.852355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"97'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489096+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:35:42.489096+0000","last_peered":"2024-09-10T07:35:42.489096+0000","last_clean":"2024-09-10T07:35:42.489096+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:35:42.489096+0000","last_undegraded":"2024-09-10T07:35:42.489096+0000","last_fullsized":"2024-09-10T07:35:42.489096+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848959+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:35:14.848959+0000","last_peered":"2024-09-10T07:35:14.848959+0000","last_clean":"2024-09-10T07:35:14.848959+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:35:14.848959+0000","last_undegraded":"2024-09-10T07:35:14.848959+0000","last_fullsized":"2024-09-10T07:35:14.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853941+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:35:14.853941+0000","last_peered":"2024-09-10T07:35:14.853941+0000","last_clean":"2024-09-10T07:35:14.853941+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:35:14.853941+0000","last_undegraded":"2024-09-10T07:35:14.853941+0000","last_fullsized":"2024-09-10T07:35:14.853941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.485460+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:35:42.485460+0000","last_peered":"2024-09-10T07:35:42.485460+0000","last_clean":"2024-09-10T07:35:42.485460+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:35:42.485460+0000","last_undegraded":"2024-09-10T07:35:42.485460+0000","last_fullsized":"2024-09-10T07:35:42.485460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"97'7","reported_seq":180,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:30.004819+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:36:30.004819+0000","last_peered":"2024-09-10T07:36:30.004819+0000","last_clean":"2024-09-10T07:36:30.004819+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:36:30.004819+0000","last_undegraded":"2024-09-10T07:36:30.004819+0000","last_fullsized":"2024-09-10T07:36:30.004819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879712+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:35:14.879712+0000","last_peered":"2024-09-10T07:35:14.879712+0000","last_clean":"2024-09-10T07:35:14.879712+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:35:14.879712+0000","last_undegraded":"2024-09-10T07:35:14.879712+0000","last_fullsized":"2024-09-10T07:35:14.879712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476516+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:35:42.476516+0000","last_peered":"2024-09-10T07:35:42.476516+0000","last_clean":"2024-09-10T07:35:42.476516+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:35:42.476516+0000","last_undegraded":"2024-09-10T07:35:42.476516+0000","last_fullsized":"2024-09-10T07:35:42.476516+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":176,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.979799+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:36:28.979799+0000","last_peered":"2024-09-10T07:36:28.979799+0000","last_clean":"2024-09-10T07:36:28.979799+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:36:28.979799+0000","last_undegraded":"2024-09-10T07:36:28.979799+0000","last_fullsized":"2024-09-10T07:36:28.979799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880600+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:35:14.880600+0000","last_peered":"2024-09-10T07:35:14.880600+0000","last_clean":"2024-09-10T07:35:14.880600+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:35:14.880600+0000","last_undegraded":"2024-09-10T07:35:14.880600+0000","last_fullsized":"2024-09-10T07:35:14.880600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.479337+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:35:42.479337+0000","last_peered":"2024-09-10T07:35:42.479337+0000","last_clean":"2024-09-10T07:35:42.479337+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:35:42.479337+0000","last_undegraded":"2024-09-10T07:35:42.479337+0000","last_fullsized":"2024-09-10T07:35:42.479337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842761+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:35:14.842761+0000","last_peered":"2024-09-10T07:35:14.842761+0000","last_clean":"2024-09-10T07:35:14.842761+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:35:14.842761+0000","last_undegraded":"2024-09-10T07:35:14.842761+0000","last_fullsized":"2024-09-10T07:35:14.842761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498940+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:35:42.498940+0000","last_peered":"2024-09-10T07:35:42.498940+0000","last_clean":"2024-09-10T07:35:42.498940+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:35:42.498940+0000","last_undegraded":"2024-09-10T07:35:42.498940+0000","last_fullsized":"2024-09-10T07:35:42.498940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882510+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:35:14.882510+0000","last_peered":"2024-09-10T07:35:14.882510+0000","last_clean":"2024-09-10T07:35:14.882510+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:35:14.882510+0000","last_undegraded":"2024-09-10T07:35:14.882510+0000","last_fullsized":"2024-09-10T07:35:14.882510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842861+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:35:14.842861+0000","last_peered":"2024-09-10T07:35:14.842861+0000","last_clean":"2024-09-10T07:35:14.842861+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:35:14.842861+0000","last_undegraded":"2024-09-10T07:35:14.842861+0000","last_fullsized":"2024-09-10T07:35:14.842861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843182+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:35:14.843182+0000","last_peered":"2024-09-10T07:35:14.843182+0000","last_clean":"2024-09-10T07:35:14.843182+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:35:14.843182+0000","last_undegraded":"2024-09-10T07:35:14.843182+0000","last_fullsized":"2024-09-10T07:35:14.843182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484404+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:35:42.484404+0000","last_peered":"2024-09-10T07:35:42.484404+0000","last_clean":"2024-09-10T07:35:42.484404+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:35:42.484404+0000","last_undegraded":"2024-09-10T07:35:42.484404+0000","last_fullsized":"2024-09-10T07:35:42.484404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":176,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980824+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:36:28.980824+0000","last_peered":"2024-09-10T07:36:28.980824+0000","last_clean":"2024-09-10T07:36:28.980824+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:36:28.980824+0000","last_undegraded":"2024-09-10T07:36:28.980824+0000","last_fullsized":"2024-09-10T07:36:28.980824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976793+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:35:14.976793+0000","last_peered":"2024-09-10T07:35:14.976793+0000","last_clean":"2024-09-10T07:35:14.976793+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:35:14.976793+0000","last_undegraded":"2024-09-10T07:35:14.976793+0000","last_fullsized":"2024-09-10T07:35:14.976793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487151+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:35:42.487151+0000","last_peered":"2024-09-10T07:35:42.487151+0000","last_clean":"2024-09-10T07:35:42.487151+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:35:42.487151+0000","last_undegraded":"2024-09-10T07:35:42.487151+0000","last_fullsized":"2024-09-10T07:35:42.487151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880698+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:35:14.880698+0000","last_peered":"2024-09-10T07:35:14.880698+0000","last_clean":"2024-09-10T07:35:14.880698+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:35:14.880698+0000","last_undegraded":"2024-09-10T07:35:14.880698+0000","last_fullsized":"2024-09-10T07:35:14.880698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857702+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:35:14.857702+0000","last_peered":"2024-09-10T07:35:14.857702+0000","last_clean":"2024-09-10T07:35:14.857702+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:35:14.857702+0000","last_undegraded":"2024-09-10T07:35:14.857702+0000","last_fullsized":"2024-09-10T07:35:14.857702+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477302+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:35:42.477302+0000","last_peered":"2024-09-10T07:35:42.477302+0000","last_clean":"2024-09-10T07:35:42.477302+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:35:42.477302+0000","last_undegraded":"2024-09-10T07:35:42.477302+0000","last_fullsized":"2024-09-10T07:35:42.477302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880480+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:35:14.880480+0000","last_peered":"2024-09-10T07:35:14.880480+0000","last_clean":"2024-09-10T07:35:14.880480+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:35:14.880480+0000","last_undegraded":"2024-09-10T07:35:14.880480+0000","last_fullsized":"2024-09-10T07:35:14.880480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879975+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:35:14.879975+0000","last_peered":"2024-09-10T07:35:14.879975+0000","last_clean":"2024-09-10T07:35:14.879975+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:35:14.879975+0000","last_undegraded":"2024-09-10T07:35:14.879975+0000","last_fullsized":"2024-09-10T07:35:14.879975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.474216+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:35:42.474216+0000","last_peered":"2024-09-10T07:35:42.474216+0000","last_clean":"2024-09-10T07:35:42.474216+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:35:42.474216+0000","last_undegraded":"2024-09-10T07:35:42.474216+0000","last_fullsized":"2024-09-10T07:35:42.474216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854462+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:35:14.854462+0000","last_peered":"2024-09-10T07:35:14.854462+0000","last_clean":"2024-09-10T07:35:14.854462+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:35:14.854462+0000","last_undegraded":"2024-09-10T07:35:14.854462+0000","last_fullsized":"2024-09-10T07:35:14.854462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879978+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:35:14.879978+0000","last_peered":"2024-09-10T07:35:14.879978+0000","last_clean":"2024-09-10T07:35:14.879978+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:35:14.879978+0000","last_undegraded":"2024-09-10T07:35:14.879978+0000","last_fullsized":"2024-09-10T07:35:14.879978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.480953+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:35:42.480953+0000","last_peered":"2024-09-10T07:35:42.480953+0000","last_clean":"2024-09-10T07:35:42.480953+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:35:42.480953+0000","last_undegraded":"2024-09-10T07:35:42.480953+0000","last_fullsized":"2024-09-10T07:35:42.480953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853997+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:35:14.853997+0000","last_peered":"2024-09-10T07:35:14.853997+0000","last_clean":"2024-09-10T07:35:14.853997+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:35:14.853997+0000","last_undegraded":"2024-09-10T07:35:14.853997+0000","last_fullsized":"2024-09-10T07:35:14.853997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880388+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:35:14.880388+0000","last_peered":"2024-09-10T07:35:14.880388+0000","last_clean":"2024-09-10T07:35:14.880388+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:35:14.880388+0000","last_undegraded":"2024-09-10T07:35:14.880388+0000","last_fullsized":"2024-09-10T07:35:14.880388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487180+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:35:42.487180+0000","last_peered":"2024-09-10T07:35:42.487180+0000","last_clean":"2024-09-10T07:35:42.487180+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:35:42.487180+0000","last_undegraded":"2024-09-10T07:35:42.487180+0000","last_fullsized":"2024-09-10T07:35:42.487180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":175,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980178+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:36:28.980178+0000","last_peered":"2024-09-10T07:36:28.980178+0000","last_clean":"2024-09-10T07:36:28.980178+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:36:28.980178+0000","last_undegraded":"2024-09-10T07:36:28.980178+0000","last_fullsized":"2024-09-10T07:36:28.980178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851402+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:35:14.851402+0000","last_peered":"2024-09-10T07:35:14.851402+0000","last_clean":"2024-09-10T07:35:14.851402+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:35:14.851402+0000","last_undegraded":"2024-09-10T07:35:14.851402+0000","last_fullsized":"2024-09-10T07:35:14.851402+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488551+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:35:42.488551+0000","last_peered":"2024-09-10T07:35:42.488551+0000","last_clean":"2024-09-10T07:35:42.488551+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:35:42.488551+0000","last_undegraded":"2024-09-10T07:35:42.488551+0000","last_fullsized":"2024-09-10T07:35:42.488551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880166+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:35:14.880166+0000","last_peered":"2024-09-10T07:35:14.880166+0000","last_clean":"2024-09-10T07:35:14.880166+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:35:14.880166+0000","last_undegraded":"2024-09-10T07:35:14.880166+0000","last_fullsized":"2024-09-10T07:35:14.880166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852262+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:35:14.852262+0000","last_peered":"2024-09-10T07:35:14.852262+0000","last_clean":"2024-09-10T07:35:14.852262+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:35:14.852262+0000","last_undegraded":"2024-09-10T07:35:14.852262+0000","last_fullsized":"2024-09-10T07:35:14.852262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500238+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:35:42.500238+0000","last_peered":"2024-09-10T07:35:42.500238+0000","last_clean":"2024-09-10T07:35:42.500238+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:35:42.500238+0000","last_undegraded":"2024-09-10T07:35:42.500238+0000","last_fullsized":"2024-09-10T07:35:42.500238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852264+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:35:14.852264+0000","last_peered":"2024-09-10T07:35:14.852264+0000","last_clean":"2024-09-10T07:35:14.852264+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:35:14.852264+0000","last_undegraded":"2024-09-10T07:35:14.852264+0000","last_fullsized":"2024-09-10T07:35:14.852264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.469244+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:35:42.469244+0000","last_peered":"2024-09-10T07:35:42.469244+0000","last_clean":"2024-09-10T07:35:42.469244+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:35:42.469244+0000","last_undegraded":"2024-09-10T07:35:42.469244+0000","last_fullsized":"2024-09-10T07:35:42.469244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"97'42","reported_seq":354,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.985620+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:36:32.985620+0000","last_peered":"2024-09-10T07:36:32.985620+0000","last_clean":"2024-09-10T07:36:32.985620+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:36:32.985620+0000","last_undegraded":"2024-09-10T07:36:32.985620+0000","last_fullsized":"2024-09-10T07:36:32.985620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70555,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":268,"num_read_kb":203,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848093+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:35:14.848093+0000","last_peered":"2024-09-10T07:35:14.848093+0000","last_clean":"2024-09-10T07:35:14.848093+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:35:14.848093+0000","last_undegraded":"2024-09-10T07:35:14.848093+0000","last_fullsized":"2024-09-10T07:35:14.848093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490736+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:35:42.490736+0000","last_peered":"2024-09-10T07:35:42.490736+0000","last_clean":"2024-09-10T07:35:42.490736+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:35:42.490736+0000","last_undegraded":"2024-09-10T07:35:42.490736+0000","last_fullsized":"2024-09-10T07:35:42.490736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880613+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:35:14.880613+0000","last_peered":"2024-09-10T07:35:14.880613+0000","last_clean":"2024-09-10T07:35:14.880613+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:35:14.880613+0000","last_undegraded":"2024-09-10T07:35:14.880613+0000","last_fullsized":"2024-09-10T07:35:14.880613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882346+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:35:14.882346+0000","last_peered":"2024-09-10T07:35:14.882346+0000","last_clean":"2024-09-10T07:35:14.882346+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:35:14.882346+0000","last_undegraded":"2024-09-10T07:35:14.882346+0000","last_fullsized":"2024-09-10T07:35:14.882346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502535+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:35:42.502535+0000","last_peered":"2024-09-10T07:35:42.502535+0000","last_clean":"2024-09-10T07:35:42.502535+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:35:42.502535+0000","last_undegraded":"2024-09-10T07:35:42.502535+0000","last_fullsized":"2024-09-10T07:35:42.502535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843243+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:35:14.843243+0000","last_peered":"2024-09-10T07:35:14.843243+0000","last_clean":"2024-09-10T07:35:14.843243+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:35:14.843243+0000","last_undegraded":"2024-09-10T07:35:14.843243+0000","last_fullsized":"2024-09-10T07:35:14.843243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847404+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:35:14.847404+0000","last_peered":"2024-09-10T07:35:14.847404+0000","last_clean":"2024-09-10T07:35:14.847404+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:35:14.847404+0000","last_undegraded":"2024-09-10T07:35:14.847404+0000","last_fullsized":"2024-09-10T07:35:14.847404+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490415+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:35:42.490415+0000","last_peered":"2024-09-10T07:35:42.490415+0000","last_clean":"2024-09-10T07:35:42.490415+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:35:42.490415+0000","last_undegraded":"2024-09-10T07:35:42.490415+0000","last_fullsized":"2024-09-10T07:35:42.490415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"97'5","reported_seq":305,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.985747+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:36:32.985747+0000","last_peered":"2024-09-10T07:36:32.985747+0000","last_clean":"2024-09-10T07:36:32.985747+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:36:32.985747+0000","last_undegraded":"2024-09-10T07:36:32.985747+0000","last_fullsized":"2024-09-10T07:36:32.985747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977046+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:35:14.977046+0000","last_peered":"2024-09-10T07:35:14.977046+0000","last_clean":"2024-09-10T07:35:14.977046+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:35:14.977046+0000","last_undegraded":"2024-09-10T07:35:14.977046+0000","last_fullsized":"2024-09-10T07:35:14.977046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858166+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:35:14.858166+0000","last_peered":"2024-09-10T07:35:14.858166+0000","last_clean":"2024-09-10T07:35:14.858166+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:35:14.858166+0000","last_undegraded":"2024-09-10T07:35:14.858166+0000","last_fullsized":"2024-09-10T07:35:14.858166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498588+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:35:42.498588+0000","last_peered":"2024-09-10T07:35:42.498588+0000","last_clean":"2024-09-10T07:35:42.498588+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:35:42.498588+0000","last_undegraded":"2024-09-10T07:35:42.498588+0000","last_fullsized":"2024-09-10T07:35:42.498588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880248+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:35:14.880248+0000","last_peered":"2024-09-10T07:35:14.880248+0000","last_clean":"2024-09-10T07:35:14.880248+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:35:14.880248+0000","last_undegraded":"2024-09-10T07:35:14.880248+0000","last_fullsized":"2024-09-10T07:35:14.880248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852575+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:35:14.852575+0000","last_peered":"2024-09-10T07:35:14.852575+0000","last_clean":"2024-09-10T07:35:14.852575+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:35:14.852575+0000","last_undegraded":"2024-09-10T07:35:14.852575+0000","last_fullsized":"2024-09-10T07:35:14.852575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499570+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:35:42.499570+0000","last_peered":"2024-09-10T07:35:42.499570+0000","last_clean":"2024-09-10T07:35:42.499570+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:35:42.499570+0000","last_undegraded":"2024-09-10T07:35:42.499570+0000","last_fullsized":"2024-09-10T07:35:42.499570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879193+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:35:14.879193+0000","last_peered":"2024-09-10T07:35:14.879193+0000","last_clean":"2024-09-10T07:35:14.879193+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:35:14.879193+0000","last_undegraded":"2024-09-10T07:35:14.879193+0000","last_fullsized":"2024-09-10T07:35:14.879193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877898+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:35:14.877898+0000","last_peered":"2024-09-10T07:35:14.877898+0000","last_clean":"2024-09-10T07:35:14.877898+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:35:14.877898+0000","last_undegraded":"2024-09-10T07:35:14.877898+0000","last_fullsized":"2024-09-10T07:35:14.877898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"97'8","reported_seq":23,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.491883+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:35:42.491883+0000","last_peered":"2024-09-10T07:35:42.491883+0000","last_clean":"2024-09-10T07:35:42.491883+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:35:42.491883+0000","last_undegraded":"2024-09-10T07:35:42.491883+0000","last_fullsized":"2024-09-10T07:35:42.491883+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842519+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:35:14.842519+0000","last_peered":"2024-09-10T07:35:14.842519+0000","last_clean":"2024-09-10T07:35:14.842519+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:35:14.842519+0000","last_undegraded":"2024-09-10T07:35:14.842519+0000","last_fullsized":"2024-09-10T07:35:14.842519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":180,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980646+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:36:28.980646+0000","last_peered":"2024-09-10T07:36:28.980646+0000","last_clean":"2024-09-10T07:36:28.980646+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:36:28.980646+0000","last_undegraded":"2024-09-10T07:36:28.980646+0000","last_fullsized":"2024-09-10T07:36:28.980646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475213+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:35:42.475213+0000","last_peered":"2024-09-10T07:35:42.475213+0000","last_clean":"2024-09-10T07:35:42.475213+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:35:42.475213+0000","last_undegraded":"2024-09-10T07:35:42.475213+0000","last_fullsized":"2024-09-10T07:35:42.475213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842633+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:35:14.842633+0000","last_peered":"2024-09-10T07:35:14.842633+0000","last_clean":"2024-09-10T07:35:14.842633+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:35:14.842633+0000","last_undegraded":"2024-09-10T07:35:14.842633+0000","last_fullsized":"2024-09-10T07:35:14.842633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477747+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:35:42.477747+0000","last_peered":"2024-09-10T07:35:42.477747+0000","last_clean":"2024-09-10T07:35:42.477747+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:35:42.477747+0000","last_undegraded":"2024-09-10T07:35:42.477747+0000","last_fullsized":"2024-09-10T07:35:42.477747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"97'12","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476532+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:35:42.476532+0000","last_peered":"2024-09-10T07:35:42.476532+0000","last_clean":"2024-09-10T07:35:42.476532+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:35:42.476532+0000","last_undegraded":"2024-09-10T07:35:42.476532+0000","last_fullsized":"2024-09-10T07:35:42.476532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486476+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:35:42.486476+0000","last_peered":"2024-09-10T07:35:42.486476+0000","last_clean":"2024-09-10T07:35:42.486476+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:35:42.486476+0000","last_undegraded":"2024-09-10T07:35:42.486476+0000","last_fullsized":"2024-09-10T07:35:42.486476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499618+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:35:42.499618+0000","last_peered":"2024-09-10T07:35:42.499618+0000","last_clean":"2024-09-10T07:35:42.499618+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:35:42.499618+0000","last_undegraded":"2024-09-10T07:35:42.499618+0000","last_fullsized":"2024-09-10T07:35:42.499618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487143+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:35:42.487143+0000","last_peered":"2024-09-10T07:35:42.487143+0000","last_clean":"2024-09-10T07:35:42.487143+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:35:42.487143+0000","last_undegraded":"2024-09-10T07:35:42.487143+0000","last_fullsized":"2024-09-10T07:35:42.487143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.470371+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:35:42.470371+0000","last_peered":"2024-09-10T07:35:42.470371+0000","last_clean":"2024-09-10T07:35:42.470371+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:35:42.470371+0000","last_undegraded":"2024-09-10T07:35:42.470371+0000","last_fullsized":"2024-09-10T07:35:42.470371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476911+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:35:42.476911+0000","last_peered":"2024-09-10T07:35:42.476911+0000","last_clean":"2024-09-10T07:35:42.476911+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:35:42.476911+0000","last_undegraded":"2024-09-10T07:35:42.476911+0000","last_fullsized":"2024-09-10T07:35:42.476911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488157+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:35:42.488157+0000","last_peered":"2024-09-10T07:35:42.488157+0000","last_clean":"2024-09-10T07:35:42.488157+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:35:42.488157+0000","last_undegraded":"2024-09-10T07:35:42.488157+0000","last_fullsized":"2024-09-10T07:35:42.488157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500450+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:35:42.500450+0000","last_peered":"2024-09-10T07:35:42.500450+0000","last_clean":"2024-09-10T07:35:42.500450+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:35:42.500450+0000","last_undegraded":"2024-09-10T07:35:42.500450+0000","last_fullsized":"2024-09-10T07:35:42.500450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499239+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:35:42.499239+0000","last_peered":"2024-09-10T07:35:42.499239+0000","last_clean":"2024-09-10T07:35:42.499239+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:35:42.499239+0000","last_undegraded":"2024-09-10T07:35:42.499239+0000","last_fullsized":"2024-09-10T07:35:42.499239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501908+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:35:42.501908+0000","last_peered":"2024-09-10T07:35:42.501908+0000","last_clean":"2024-09-10T07:35:42.501908+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:35:42.501908+0000","last_undegraded":"2024-09-10T07:35:42.501908+0000","last_fullsized":"2024-09-10T07:35:42.501908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493731+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:35:42.493731+0000","last_peered":"2024-09-10T07:35:42.493731+0000","last_clean":"2024-09-10T07:35:42.493731+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:35:42.493731+0000","last_undegraded":"2024-09-10T07:35:42.493731+0000","last_fullsized":"2024-09-10T07:35:42.493731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478046+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:35:42.478046+0000","last_peered":"2024-09-10T07:35:42.478046+0000","last_clean":"2024-09-10T07:35:42.478046+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:35:42.478046+0000","last_undegraded":"2024-09-10T07:35:42.478046+0000","last_fullsized":"2024-09-10T07:35:42.478046+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493144+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:35:42.493144+0000","last_peered":"2024-09-10T07:35:42.493144+0000","last_clean":"2024-09-10T07:35:42.493144+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:35:42.493144+0000","last_undegraded":"2024-09-10T07:35:42.493144+0000","last_fullsized":"2024-09-10T07:35:42.493144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483119+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:35:42.483119+0000","last_peered":"2024-09-10T07:35:42.483119+0000","last_clean":"2024-09-10T07:35:42.483119+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:35:42.483119+0000","last_undegraded":"2024-09-10T07:35:42.483119+0000","last_fullsized":"2024-09-10T07:35:42.483119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502676+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:35:42.502676+0000","last_peered":"2024-09-10T07:35:42.502676+0000","last_clean":"2024-09-10T07:35:42.502676+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:35:42.502676+0000","last_undegraded":"2024-09-10T07:35:42.502676+0000","last_fullsized":"2024-09-10T07:35:42.502676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502776+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:35:42.502776+0000","last_peered":"2024-09-10T07:35:42.502776+0000","last_clean":"2024-09-10T07:35:42.502776+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:35:42.502776+0000","last_undegraded":"2024-09-10T07:35:42.502776+0000","last_fullsized":"2024-09-10T07:35:42.502776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499409+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:35:42.499409+0000","last_peered":"2024-09-10T07:35:42.499409+0000","last_clean":"2024-09-10T07:35:42.499409+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:35:42.499409+0000","last_undegraded":"2024-09-10T07:35:42.499409+0000","last_fullsized":"2024-09-10T07:35:42.499409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502975+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:35:42.502975+0000","last_peered":"2024-09-10T07:35:42.502975+0000","last_clean":"2024-09-10T07:35:42.502975+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:35:42.502975+0000","last_undegraded":"2024-09-10T07:35:42.502975+0000","last_fullsized":"2024-09-10T07:35:42.502975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493842+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:35:42.493842+0000","last_peered":"2024-09-10T07:35:42.493842+0000","last_clean":"2024-09-10T07:35:42.493842+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:35:42.493842+0000","last_undegraded":"2024-09-10T07:35:42.493842+0000","last_fullsized":"2024-09-10T07:35:42.493842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484180+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:35:42.484180+0000","last_peered":"2024-09-10T07:35:42.484180+0000","last_clean":"2024-09-10T07:35:42.484180+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:35:42.484180+0000","last_undegraded":"2024-09-10T07:35:42.484180+0000","last_fullsized":"2024-09-10T07:35:42.484180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483929+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:35:42.483929+0000","last_peered":"2024-09-10T07:35:42.483929+0000","last_clean":"2024-09-10T07:35:42.483929+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:35:42.483929+0000","last_undegraded":"2024-09-10T07:35:42.483929+0000","last_fullsized":"2024-09-10T07:35:42.483929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"97'16","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483320+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:35:42.483320+0000","last_peered":"2024-09-10T07:35:42.483320+0000","last_clean":"2024-09-10T07:35:42.483320+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:35:42.483320+0000","last_undegraded":"2024-09-10T07:35:42.483320+0000","last_fullsized":"2024-09-10T07:35:42.483320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489981+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:35:42.489981+0000","last_peered":"2024-09-10T07:35:42.489981+0000","last_clean":"2024-09-10T07:35:42.489981+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:35:42.489981+0000","last_undegraded":"2024-09-10T07:35:42.489981+0000","last_fullsized":"2024-09-10T07:35:42.489981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500857+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:35:42.500857+0000","last_peered":"2024-09-10T07:35:42.500857+0000","last_clean":"2024-09-10T07:35:42.500857+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:35:42.500857+0000","last_undegraded":"2024-09-10T07:35:42.500857+0000","last_fullsized":"2024-09-10T07:35:42.500857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478497+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:35:42.478497+0000","last_peered":"2024-09-10T07:35:42.478497+0000","last_clean":"2024-09-10T07:35:42.478497+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:35:42.478497+0000","last_undegraded":"2024-09-10T07:35:42.478497+0000","last_fullsized":"2024-09-10T07:35:42.478497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487137+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:35:42.487137+0000","last_peered":"2024-09-10T07:35:42.487137+0000","last_clean":"2024-09-10T07:35:42.487137+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:35:42.487137+0000","last_undegraded":"2024-09-10T07:35:42.487137+0000","last_fullsized":"2024-09-10T07:35:42.487137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"97'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.496842+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:35:42.496842+0000","last_peered":"2024-09-10T07:35:42.496842+0000","last_clean":"2024-09-10T07:35:42.496842+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:35:42.496842+0000","last_undegraded":"2024-09-10T07:35:42.496842+0000","last_fullsized":"2024-09-10T07:35:42.496842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"97'8","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476981+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:35:42.476981+0000","last_peered":"2024-09-10T07:35:42.476981+0000","last_clean":"2024-09-10T07:35:42.476981+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:35:42.476981+0000","last_undegraded":"2024-09-10T07:35:42.476981+0000","last_fullsized":"2024-09-10T07:35:42.476981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.494172+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:35:42.494172+0000","last_peered":"2024-09-10T07:35:42.494172+0000","last_clean":"2024-09-10T07:35:42.494172+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:35:42.494172+0000","last_undegraded":"2024-09-10T07:35:42.494172+0000","last_fullsized":"2024-09-10T07:35:42.494172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502201+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:35:42.502201+0000","last_peered":"2024-09-10T07:35:42.502201+0000","last_clean":"2024-09-10T07:35:42.502201+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:35:42.502201+0000","last_undegraded":"2024-09-10T07:35:42.502201+0000","last_fullsized":"2024-09-10T07:35:42.502201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502439+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:35:42.502439+0000","last_peered":"2024-09-10T07:35:42.502439+0000","last_clean":"2024-09-10T07:35:42.502439+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:35:42.502439+0000","last_undegraded":"2024-09-10T07:35:42.502439+0000","last_fullsized":"2024-09-10T07:35:42.502439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78045,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1309,"num_read_kb":744,"num_write":265,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":228762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449139,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":561212,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710776,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1241088,"data_stored":1218403,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874299,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":565058,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":561228,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1158514,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":557000,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1153562,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":560540,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":556174,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1163264,"data_stored":623364,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":622901,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":557922,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:36:35.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:36:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 -- 172.21.15.46:0/854795992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114104f20 msgr2=0x7f81141053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 --2- 172.21.15.46:0/854795992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114104f20 0x7f81141053a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009900 tx=0x7f80fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 -- 172.21.15.46:0/854795992 shutdown_connections 2024-09-10T07:36:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 --2- 172.21.15.46:0/854795992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141058e0 0x7f811410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 --2- 172.21.15.46:0/854795992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114104f20 0x7f81141053a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 --2- 172.21.15.46:0/854795992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8114103d30 0x7f8114104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 -- 172.21.15.46:0/854795992 >> 172.21.15.46:0/854795992 conn(0x7f81140ff4b0 msgr2=0x7f81141018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 -- 172.21.15.46:0/854795992 shutdown_connections 2024-09-10T07:36:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.875+0000 7f8119600640 1 -- 172.21.15.46:0/854795992 wait complete. 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.876+0000 7f8119600640 1 Processor -- start 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.876+0000 7f8119600640 1 -- start start 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8119600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114103d30 0x7f811419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8119600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8114104f20 0x7f811419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8119600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141058e0 0x7f81141a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8119600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f811410f9e0 con 0x7f8114104f20 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8119600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f811410f860 con 0x7f8114103d30 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8119600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f811410fb60 con 0x7f81141058e0 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f810be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8114104f20 0x7f811419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8112a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114103d30 0x7f811419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f810be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8114104f20 0x7f811419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37194/0 (socket says 172.21.15.46:37194) 2024-09-10T07:36:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8112a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114103d30 0x7f811419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52260/0 (socket says 172.21.15.46:52260) 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f810be00640 1 -- 172.21.15.46:0/934096130 learned_addr learned my addr 172.21.15.46:0/934096130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8113400640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141058e0 0x7f81141a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8112a00640 1 -- 172.21.15.46:0/934096130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141058e0 msgr2=0x7f81141a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8112a00640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141058e0 0x7f81141a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8112a00640 1 -- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8114104f20 msgr2=0x7f811419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8112a00640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8114104f20 0x7f811419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8112a00640 1 -- 172.21.15.46:0/934096130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81141a1c50 con 0x7f8114103d30 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f8113400640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141058e0 0x7f81141a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.877+0000 7f810be00640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8114104f20 0x7f811419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.878+0000 7f8112a00640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114103d30 0x7f811419a030 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f810400e9c0 tx=0x7f810400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.878+0000 7f810b400640 1 -- 172.21.15.46:0/934096130 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810400ccc0 con 0x7f8114103d30 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.878+0000 7f810b400640 1 -- 172.21.15.46:0/934096130 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81040035f0 con 0x7f8114103d30 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.878+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81141a1f40 con 0x7f8114103d30 2024-09-10T07:36:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.878+0000 7f810b400640 1 -- 172.21.15.46:0/934096130 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810400ce60 con 0x7f8114103d30 2024-09-10T07:36:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.878+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a23a0 con 0x7f8114103d30 2024-09-10T07:36:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.879+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f811410b0d0 con 0x7f8114103d30 2024-09-10T07:36:35.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.881+0000 7f810b400640 1 -- 172.21.15.46:0/934096130 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f81040037a0 con 0x7f8114103d30 2024-09-10T07:36:35.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.883+0000 7f810b400640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80f80a2b90 0x7f80f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:35.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.883+0000 7f810b400640 1 -- 172.21.15.46:0/934096130 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f81040cf960 con 0x7f8114103d30 2024-09-10T07:36:35.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.883+0000 7f810be00640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80f80a2b90 0x7f80f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:35.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.884+0000 7f810be00640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80f80a2b90 0x7f80f80a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f811419b540 tx=0x7f80fc0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:35.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:35.885+0000 7f810b400640 1 -- 172.21.15.46:0/934096130 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f810409c570 con 0x7f8114103d30 2024-09-10T07:36:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.071+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8114062630 con 0x7f80f80a2b90 2024-09-10T07:36:36.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.079+0000 7f810b400640 1 -- 172.21.15.46:0/934096130 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340777 (secure 0 0 0) 0x7f8114062630 con 0x7f80f80a2b90 2024-09-10T07:36:36.083 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:36.085 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:36:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80f80a2b90 msgr2=0x7f80f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80f80a2b90 0x7f80f80a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f811419b540 tx=0x7f80fc0315b0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114103d30 msgr2=0x7f811419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114103d30 0x7f811419a030 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f810400e9c0 tx=0x7f810400ee80 comp rx=0 tx=0).stop 2024-09-10T07:36:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 shutdown_connections 2024-09-10T07:36:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81141058e0 0x7f81141a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f80f80a2b90 0x7f80f80a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8114104f20 0x7f811419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 --2- 172.21.15.46:0/934096130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8114103d30 0x7f811419a030 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 >> 172.21.15.46:0/934096130 conn(0x7f81140ff4b0 msgr2=0x7f81141082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 shutdown_connections 2024-09-10T07:36:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.084+0000 7f8119600640 1 -- 172.21.15.46:0/934096130 wait complete. 2024-09-10T07:36:36.100 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-10T07:36:34.507732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668647,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1435,"num_read_kb":853,"num_write":448,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358316,"kb_used_data":13228,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583508,"statfs":{"total":1151940427776,"available":1151573512192,"internally_reserved":0,"allocated":13545472,"data_stored":8695878,"data_compressed":498372,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003122"},"pg_stats":[{"pgid":"2.1f","version":"97'14","reported_seq":186,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.007589+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:36:32.007589+0000","last_peered":"2024-09-10T07:36:32.007589+0000","last_clean":"2024-09-10T07:36:32.007589+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:36:32.007589+0000","last_undegraded":"2024-09-10T07:36:32.007589+0000","last_fullsized":"2024-09-10T07:36:32.007589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475738+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:35:42.475738+0000","last_peered":"2024-09-10T07:35:42.475738+0000","last_clean":"2024-09-10T07:35:42.475738+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:35:42.475738+0000","last_undegraded":"2024-09-10T07:35:42.475738+0000","last_fullsized":"2024-09-10T07:35:42.475738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842368+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:35:14.842368+0000","last_peered":"2024-09-10T07:35:14.842368+0000","last_clean":"2024-09-10T07:35:14.842368+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:35:14.842368+0000","last_undegraded":"2024-09-10T07:35:14.842368+0000","last_fullsized":"2024-09-10T07:35:14.842368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977747+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:35:14.977747+0000","last_peered":"2024-09-10T07:35:14.977747+0000","last_clean":"2024-09-10T07:35:14.977747+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:35:14.977747+0000","last_undegraded":"2024-09-10T07:35:14.977747+0000","last_fullsized":"2024-09-10T07:35:14.977747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502886+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:35:42.502886+0000","last_peered":"2024-09-10T07:35:42.502886+0000","last_clean":"2024-09-10T07:35:42.502886+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:35:42.502886+0000","last_undegraded":"2024-09-10T07:35:42.502886+0000","last_fullsized":"2024-09-10T07:35:42.502886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881794+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:35:14.881794+0000","last_peered":"2024-09-10T07:35:14.881794+0000","last_clean":"2024-09-10T07:35:14.881794+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:35:14.881794+0000","last_undegraded":"2024-09-10T07:35:14.881794+0000","last_fullsized":"2024-09-10T07:35:14.881794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854574+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:35:14.854574+0000","last_peered":"2024-09-10T07:35:14.854574+0000","last_clean":"2024-09-10T07:35:14.854574+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:35:14.854574+0000","last_undegraded":"2024-09-10T07:35:14.854574+0000","last_fullsized":"2024-09-10T07:35:14.854574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.495636+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:35:42.495636+0000","last_peered":"2024-09-10T07:35:42.495636+0000","last_clean":"2024-09-10T07:35:42.495636+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:35:42.495636+0000","last_undegraded":"2024-09-10T07:35:42.495636+0000","last_fullsized":"2024-09-10T07:35:42.495636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881984+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:35:14.881984+0000","last_peered":"2024-09-10T07:35:14.881984+0000","last_clean":"2024-09-10T07:35:14.881984+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:35:14.881984+0000","last_undegraded":"2024-09-10T07:35:14.881984+0000","last_fullsized":"2024-09-10T07:35:14.881984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854411+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:35:14.854411+0000","last_peered":"2024-09-10T07:35:14.854411+0000","last_clean":"2024-09-10T07:35:14.854411+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:35:14.854411+0000","last_undegraded":"2024-09-10T07:35:14.854411+0000","last_fullsized":"2024-09-10T07:35:14.854411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"97'15","reported_seq":60,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484819+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:35:42.484819+0000","last_peered":"2024-09-10T07:35:42.484819+0000","last_clean":"2024-09-10T07:35:42.484819+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:35:42.484819+0000","last_undegraded":"2024-09-10T07:35:42.484819+0000","last_fullsized":"2024-09-10T07:35:42.484819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881889+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:35:14.881889+0000","last_peered":"2024-09-10T07:35:14.881889+0000","last_clean":"2024-09-10T07:35:14.881889+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:35:14.881889+0000","last_undegraded":"2024-09-10T07:35:14.881889+0000","last_fullsized":"2024-09-10T07:35:14.881889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486825+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:35:42.486825+0000","last_peered":"2024-09-10T07:35:42.486825+0000","last_clean":"2024-09-10T07:35:42.486825+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:35:42.486825+0000","last_undegraded":"2024-09-10T07:35:42.486825+0000","last_fullsized":"2024-09-10T07:35:42.486825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854068+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:35:14.854068+0000","last_peered":"2024-09-10T07:35:14.854068+0000","last_clean":"2024-09-10T07:35:14.854068+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:35:14.854068+0000","last_undegraded":"2024-09-10T07:35:14.854068+0000","last_fullsized":"2024-09-10T07:35:14.854068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882150+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:35:14.882150+0000","last_peered":"2024-09-10T07:35:14.882150+0000","last_clean":"2024-09-10T07:35:14.882150+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:35:14.882150+0000","last_undegraded":"2024-09-10T07:35:14.882150+0000","last_fullsized":"2024-09-10T07:35:14.882150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501326+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:35:42.501326+0000","last_peered":"2024-09-10T07:35:42.501326+0000","last_clean":"2024-09-10T07:35:42.501326+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:35:42.501326+0000","last_undegraded":"2024-09-10T07:35:42.501326+0000","last_fullsized":"2024-09-10T07:35:42.501326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842483+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:35:14.842483+0000","last_peered":"2024-09-10T07:35:14.842483+0000","last_clean":"2024-09-10T07:35:14.842483+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:35:14.842483+0000","last_undegraded":"2024-09-10T07:35:14.842483+0000","last_fullsized":"2024-09-10T07:35:14.842483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882278+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:35:14.882278+0000","last_peered":"2024-09-10T07:35:14.882278+0000","last_clean":"2024-09-10T07:35:14.882278+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:35:14.882278+0000","last_undegraded":"2024-09-10T07:35:14.882278+0000","last_fullsized":"2024-09-10T07:35:14.882278+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502574+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:35:42.502574+0000","last_peered":"2024-09-10T07:35:42.502574+0000","last_clean":"2024-09-10T07:35:42.502574+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:35:42.502574+0000","last_undegraded":"2024-09-10T07:35:42.502574+0000","last_fullsized":"2024-09-10T07:35:42.502574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977257+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:35:14.977257+0000","last_peered":"2024-09-10T07:35:14.977257+0000","last_clean":"2024-09-10T07:35:14.977257+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:35:14.977257+0000","last_undegraded":"2024-09-10T07:35:14.977257+0000","last_fullsized":"2024-09-10T07:35:14.977257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851715+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:35:14.851715+0000","last_peered":"2024-09-10T07:35:14.851715+0000","last_clean":"2024-09-10T07:35:14.851715+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:35:14.851715+0000","last_undegraded":"2024-09-10T07:35:14.851715+0000","last_fullsized":"2024-09-10T07:35:14.851715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"97'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.459952+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:35:42.459952+0000","last_peered":"2024-09-10T07:35:42.459952+0000","last_clean":"2024-09-10T07:35:42.459952+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:35:42.459952+0000","last_undegraded":"2024-09-10T07:35:42.459952+0000","last_fullsized":"2024-09-10T07:35:42.459952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976752+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:35:14.976752+0000","last_peered":"2024-09-10T07:35:14.976752+0000","last_clean":"2024-09-10T07:35:14.976752+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:35:14.976752+0000","last_undegraded":"2024-09-10T07:35:14.976752+0000","last_fullsized":"2024-09-10T07:35:14.976752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882210+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:35:14.882210+0000","last_peered":"2024-09-10T07:35:14.882210+0000","last_clean":"2024-09-10T07:35:14.882210+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:35:14.882210+0000","last_undegraded":"2024-09-10T07:35:14.882210+0000","last_fullsized":"2024-09-10T07:35:14.882210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502703+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:35:42.502703+0000","last_peered":"2024-09-10T07:35:42.502703+0000","last_clean":"2024-09-10T07:35:42.502703+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:35:42.502703+0000","last_undegraded":"2024-09-10T07:35:42.502703+0000","last_fullsized":"2024-09-10T07:35:42.502703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882535+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:35:14.882535+0000","last_peered":"2024-09-10T07:35:14.882535+0000","last_clean":"2024-09-10T07:35:14.882535+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:35:14.882535+0000","last_undegraded":"2024-09-10T07:35:14.882535+0000","last_fullsized":"2024-09-10T07:35:14.882535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842382+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:35:14.842382+0000","last_peered":"2024-09-10T07:35:14.842382+0000","last_clean":"2024-09-10T07:35:14.842382+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:35:14.842382+0000","last_undegraded":"2024-09-10T07:35:14.842382+0000","last_fullsized":"2024-09-10T07:35:14.842382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499802+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:35:42.499802+0000","last_peered":"2024-09-10T07:35:42.499802+0000","last_clean":"2024-09-10T07:35:42.499802+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:35:42.499802+0000","last_undegraded":"2024-09-10T07:35:42.499802+0000","last_fullsized":"2024-09-10T07:35:42.499802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'18","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:43.451679+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:35:43.451679+0000","last_peered":"2024-09-10T07:35:43.451679+0000","last_clean":"2024-09-10T07:35:43.451679+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:35:43.451679+0000","last_undegraded":"2024-09-10T07:35:43.451679+0000","last_fullsized":"2024-09-10T07:35:43.451679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976909+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:35:14.976909+0000","last_peered":"2024-09-10T07:35:14.976909+0000","last_clean":"2024-09-10T07:35:14.976909+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:35:14.976909+0000","last_undegraded":"2024-09-10T07:35:14.976909+0000","last_fullsized":"2024-09-10T07:35:14.976909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.481577+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:35:42.481577+0000","last_peered":"2024-09-10T07:35:42.481577+0000","last_clean":"2024-09-10T07:35:42.481577+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:35:42.481577+0000","last_undegraded":"2024-09-10T07:35:42.481577+0000","last_fullsized":"2024-09-10T07:35:42.481577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.872520+0000","last_change":"2024-09-10T07:35:14.872520+0000","last_active":"2024-09-10T07:35:14.872520+0000","last_peered":"2024-09-10T07:35:14.872520+0000","last_clean":"2024-09-10T07:35:14.872520+0000","last_became_active":"2024-09-10T07:35:14.860686+0000","last_became_peered":"2024-09-10T07:35:14.860686+0000","last_unstale":"2024-09-10T07:35:14.872520+0000","last_undegraded":"2024-09-10T07:35:14.872520+0000","last_fullsized":"2024-09-10T07:35:14.872520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:42.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852355+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:35:14.852355+0000","last_peered":"2024-09-10T07:35:14.852355+0000","last_clean":"2024-09-10T07:35:14.852355+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:35:14.852355+0000","last_undegraded":"2024-09-10T07:35:14.852355+0000","last_fullsized":"2024-09-10T07:35:14.852355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"97'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489096+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:35:42.489096+0000","last_peered":"2024-09-10T07:35:42.489096+0000","last_clean":"2024-09-10T07:35:42.489096+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:35:42.489096+0000","last_undegraded":"2024-09-10T07:35:42.489096+0000","last_fullsized":"2024-09-10T07:35:42.489096+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848959+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:35:14.848959+0000","last_peered":"2024-09-10T07:35:14.848959+0000","last_clean":"2024-09-10T07:35:14.848959+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:35:14.848959+0000","last_undegraded":"2024-09-10T07:35:14.848959+0000","last_fullsized":"2024-09-10T07:35:14.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853941+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:35:14.853941+0000","last_peered":"2024-09-10T07:35:14.853941+0000","last_clean":"2024-09-10T07:35:14.853941+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:35:14.853941+0000","last_undegraded":"2024-09-10T07:35:14.853941+0000","last_fullsized":"2024-09-10T07:35:14.853941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.485460+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:35:42.485460+0000","last_peered":"2024-09-10T07:35:42.485460+0000","last_clean":"2024-09-10T07:35:42.485460+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:35:42.485460+0000","last_undegraded":"2024-09-10T07:35:42.485460+0000","last_fullsized":"2024-09-10T07:35:42.485460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"97'7","reported_seq":180,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:30.004819+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:36:30.004819+0000","last_peered":"2024-09-10T07:36:30.004819+0000","last_clean":"2024-09-10T07:36:30.004819+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:36:30.004819+0000","last_undegraded":"2024-09-10T07:36:30.004819+0000","last_fullsized":"2024-09-10T07:36:30.004819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879712+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:35:14.879712+0000","last_peered":"2024-09-10T07:35:14.879712+0000","last_clean":"2024-09-10T07:35:14.879712+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:35:14.879712+0000","last_undegraded":"2024-09-10T07:35:14.879712+0000","last_fullsized":"2024-09-10T07:35:14.879712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476516+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:35:42.476516+0000","last_peered":"2024-09-10T07:35:42.476516+0000","last_clean":"2024-09-10T07:35:42.476516+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:35:42.476516+0000","last_undegraded":"2024-09-10T07:35:42.476516+0000","last_fullsized":"2024-09-10T07:35:42.476516+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":176,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.979799+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:36:28.979799+0000","last_peered":"2024-09-10T07:36:28.979799+0000","last_clean":"2024-09-10T07:36:28.979799+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:36:28.979799+0000","last_undegraded":"2024-09-10T07:36:28.979799+0000","last_fullsized":"2024-09-10T07:36:28.979799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880600+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:35:14.880600+0000","last_peered":"2024-09-10T07:35:14.880600+0000","last_clean":"2024-09-10T07:35:14.880600+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:35:14.880600+0000","last_undegraded":"2024-09-10T07:35:14.880600+0000","last_fullsized":"2024-09-10T07:35:14.880600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.479337+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:35:42.479337+0000","last_peered":"2024-09-10T07:35:42.479337+0000","last_clean":"2024-09-10T07:35:42.479337+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:35:42.479337+0000","last_undegraded":"2024-09-10T07:35:42.479337+0000","last_fullsized":"2024-09-10T07:35:42.479337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842761+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:35:14.842761+0000","last_peered":"2024-09-10T07:35:14.842761+0000","last_clean":"2024-09-10T07:35:14.842761+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:35:14.842761+0000","last_undegraded":"2024-09-10T07:35:14.842761+0000","last_fullsized":"2024-09-10T07:35:14.842761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498940+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:35:42.498940+0000","last_peered":"2024-09-10T07:35:42.498940+0000","last_clean":"2024-09-10T07:35:42.498940+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:35:42.498940+0000","last_undegraded":"2024-09-10T07:35:42.498940+0000","last_fullsized":"2024-09-10T07:35:42.498940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882510+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:35:14.882510+0000","last_peered":"2024-09-10T07:35:14.882510+0000","last_clean":"2024-09-10T07:35:14.882510+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:35:14.882510+0000","last_undegraded":"2024-09-10T07:35:14.882510+0000","last_fullsized":"2024-09-10T07:35:14.882510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842861+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:35:14.842861+0000","last_peered":"2024-09-10T07:35:14.842861+0000","last_clean":"2024-09-10T07:35:14.842861+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:35:14.842861+0000","last_undegraded":"2024-09-10T07:35:14.842861+0000","last_fullsized":"2024-09-10T07:35:14.842861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843182+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:35:14.843182+0000","last_peered":"2024-09-10T07:35:14.843182+0000","last_clean":"2024-09-10T07:35:14.843182+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:35:14.843182+0000","last_undegraded":"2024-09-10T07:35:14.843182+0000","last_fullsized":"2024-09-10T07:35:14.843182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484404+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:35:42.484404+0000","last_peered":"2024-09-10T07:35:42.484404+0000","last_clean":"2024-09-10T07:35:42.484404+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:35:42.484404+0000","last_undegraded":"2024-09-10T07:35:42.484404+0000","last_fullsized":"2024-09-10T07:35:42.484404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":176,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980824+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:36:28.980824+0000","last_peered":"2024-09-10T07:36:28.980824+0000","last_clean":"2024-09-10T07:36:28.980824+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:36:28.980824+0000","last_undegraded":"2024-09-10T07:36:28.980824+0000","last_fullsized":"2024-09-10T07:36:28.980824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976793+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:35:14.976793+0000","last_peered":"2024-09-10T07:35:14.976793+0000","last_clean":"2024-09-10T07:35:14.976793+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:35:14.976793+0000","last_undegraded":"2024-09-10T07:35:14.976793+0000","last_fullsized":"2024-09-10T07:35:14.976793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487151+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:35:42.487151+0000","last_peered":"2024-09-10T07:35:42.487151+0000","last_clean":"2024-09-10T07:35:42.487151+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:35:42.487151+0000","last_undegraded":"2024-09-10T07:35:42.487151+0000","last_fullsized":"2024-09-10T07:35:42.487151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880698+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:35:14.880698+0000","last_peered":"2024-09-10T07:35:14.880698+0000","last_clean":"2024-09-10T07:35:14.880698+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:35:14.880698+0000","last_undegraded":"2024-09-10T07:35:14.880698+0000","last_fullsized":"2024-09-10T07:35:14.880698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857702+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:35:14.857702+0000","last_peered":"2024-09-10T07:35:14.857702+0000","last_clean":"2024-09-10T07:35:14.857702+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:35:14.857702+0000","last_undegraded":"2024-09-10T07:35:14.857702+0000","last_fullsized":"2024-09-10T07:35:14.857702+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477302+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:35:42.477302+0000","last_peered":"2024-09-10T07:35:42.477302+0000","last_clean":"2024-09-10T07:35:42.477302+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:35:42.477302+0000","last_undegraded":"2024-09-10T07:35:42.477302+0000","last_fullsized":"2024-09-10T07:35:42.477302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880480+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:35:14.880480+0000","last_peered":"2024-09-10T07:35:14.880480+0000","last_clean":"2024-09-10T07:35:14.880480+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:35:14.880480+0000","last_undegraded":"2024-09-10T07:35:14.880480+0000","last_fullsized":"2024-09-10T07:35:14.880480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879975+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:35:14.879975+0000","last_peered":"2024-09-10T07:35:14.879975+0000","last_clean":"2024-09-10T07:35:14.879975+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:35:14.879975+0000","last_undegraded":"2024-09-10T07:35:14.879975+0000","last_fullsized":"2024-09-10T07:35:14.879975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.474216+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:35:42.474216+0000","last_peered":"2024-09-10T07:35:42.474216+0000","last_clean":"2024-09-10T07:35:42.474216+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:35:42.474216+0000","last_undegraded":"2024-09-10T07:35:42.474216+0000","last_fullsized":"2024-09-10T07:35:42.474216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854462+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:35:14.854462+0000","last_peered":"2024-09-10T07:35:14.854462+0000","last_clean":"2024-09-10T07:35:14.854462+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:35:14.854462+0000","last_undegraded":"2024-09-10T07:35:14.854462+0000","last_fullsized":"2024-09-10T07:35:14.854462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879978+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:35:14.879978+0000","last_peered":"2024-09-10T07:35:14.879978+0000","last_clean":"2024-09-10T07:35:14.879978+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:35:14.879978+0000","last_undegraded":"2024-09-10T07:35:14.879978+0000","last_fullsized":"2024-09-10T07:35:14.879978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.480953+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:35:42.480953+0000","last_peered":"2024-09-10T07:35:42.480953+0000","last_clean":"2024-09-10T07:35:42.480953+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:35:42.480953+0000","last_undegraded":"2024-09-10T07:35:42.480953+0000","last_fullsized":"2024-09-10T07:35:42.480953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853997+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:35:14.853997+0000","last_peered":"2024-09-10T07:35:14.853997+0000","last_clean":"2024-09-10T07:35:14.853997+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:35:14.853997+0000","last_undegraded":"2024-09-10T07:35:14.853997+0000","last_fullsized":"2024-09-10T07:35:14.853997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880388+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:35:14.880388+0000","last_peered":"2024-09-10T07:35:14.880388+0000","last_clean":"2024-09-10T07:35:14.880388+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:35:14.880388+0000","last_undegraded":"2024-09-10T07:35:14.880388+0000","last_fullsized":"2024-09-10T07:35:14.880388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487180+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:35:42.487180+0000","last_peered":"2024-09-10T07:35:42.487180+0000","last_clean":"2024-09-10T07:35:42.487180+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:35:42.487180+0000","last_undegraded":"2024-09-10T07:35:42.487180+0000","last_fullsized":"2024-09-10T07:35:42.487180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":175,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980178+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:36:28.980178+0000","last_peered":"2024-09-10T07:36:28.980178+0000","last_clean":"2024-09-10T07:36:28.980178+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:36:28.980178+0000","last_undegraded":"2024-09-10T07:36:28.980178+0000","last_fullsized":"2024-09-10T07:36:28.980178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851402+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:35:14.851402+0000","last_peered":"2024-09-10T07:35:14.851402+0000","last_clean":"2024-09-10T07:35:14.851402+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:35:14.851402+0000","last_undegraded":"2024-09-10T07:35:14.851402+0000","last_fullsized":"2024-09-10T07:35:14.851402+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488551+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:35:42.488551+0000","last_peered":"2024-09-10T07:35:42.488551+0000","last_clean":"2024-09-10T07:35:42.488551+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:35:42.488551+0000","last_undegraded":"2024-09-10T07:35:42.488551+0000","last_fullsized":"2024-09-10T07:35:42.488551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880166+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:35:14.880166+0000","last_peered":"2024-09-10T07:35:14.880166+0000","last_clean":"2024-09-10T07:35:14.880166+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:35:14.880166+0000","last_undegraded":"2024-09-10T07:35:14.880166+0000","last_fullsized":"2024-09-10T07:35:14.880166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852262+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:35:14.852262+0000","last_peered":"2024-09-10T07:35:14.852262+0000","last_clean":"2024-09-10T07:35:14.852262+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:35:14.852262+0000","last_undegraded":"2024-09-10T07:35:14.852262+0000","last_fullsized":"2024-09-10T07:35:14.852262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500238+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:35:42.500238+0000","last_peered":"2024-09-10T07:35:42.500238+0000","last_clean":"2024-09-10T07:35:42.500238+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:35:42.500238+0000","last_undegraded":"2024-09-10T07:35:42.500238+0000","last_fullsized":"2024-09-10T07:35:42.500238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852264+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:35:14.852264+0000","last_peered":"2024-09-10T07:35:14.852264+0000","last_clean":"2024-09-10T07:35:14.852264+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:35:14.852264+0000","last_undegraded":"2024-09-10T07:35:14.852264+0000","last_fullsized":"2024-09-10T07:35:14.852264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.469244+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:35:42.469244+0000","last_peered":"2024-09-10T07:35:42.469244+0000","last_clean":"2024-09-10T07:35:42.469244+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:35:42.469244+0000","last_undegraded":"2024-09-10T07:35:42.469244+0000","last_fullsized":"2024-09-10T07:35:42.469244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"97'42","reported_seq":354,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.985620+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:36:32.985620+0000","last_peered":"2024-09-10T07:36:32.985620+0000","last_clean":"2024-09-10T07:36:32.985620+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:36:32.985620+0000","last_undegraded":"2024-09-10T07:36:32.985620+0000","last_fullsized":"2024-09-10T07:36:32.985620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70555,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":268,"num_read_kb":203,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848093+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:35:14.848093+0000","last_peered":"2024-09-10T07:35:14.848093+0000","last_clean":"2024-09-10T07:35:14.848093+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:35:14.848093+0000","last_undegraded":"2024-09-10T07:35:14.848093+0000","last_fullsized":"2024-09-10T07:35:14.848093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490736+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:35:42.490736+0000","last_peered":"2024-09-10T07:35:42.490736+0000","last_clean":"2024-09-10T07:35:42.490736+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:35:42.490736+0000","last_undegraded":"2024-09-10T07:35:42.490736+0000","last_fullsized":"2024-09-10T07:35:42.490736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880613+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:35:14.880613+0000","last_peered":"2024-09-10T07:35:14.880613+0000","last_clean":"2024-09-10T07:35:14.880613+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:35:14.880613+0000","last_undegraded":"2024-09-10T07:35:14.880613+0000","last_fullsized":"2024-09-10T07:35:14.880613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882346+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:35:14.882346+0000","last_peered":"2024-09-10T07:35:14.882346+0000","last_clean":"2024-09-10T07:35:14.882346+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:35:14.882346+0000","last_undegraded":"2024-09-10T07:35:14.882346+0000","last_fullsized":"2024-09-10T07:35:14.882346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502535+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:35:42.502535+0000","last_peered":"2024-09-10T07:35:42.502535+0000","last_clean":"2024-09-10T07:35:42.502535+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:35:42.502535+0000","last_undegraded":"2024-09-10T07:35:42.502535+0000","last_fullsized":"2024-09-10T07:35:42.502535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843243+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:35:14.843243+0000","last_peered":"2024-09-10T07:35:14.843243+0000","last_clean":"2024-09-10T07:35:14.843243+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:35:14.843243+0000","last_undegraded":"2024-09-10T07:35:14.843243+0000","last_fullsized":"2024-09-10T07:35:14.843243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847404+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:35:14.847404+0000","last_peered":"2024-09-10T07:35:14.847404+0000","last_clean":"2024-09-10T07:35:14.847404+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:35:14.847404+0000","last_undegraded":"2024-09-10T07:35:14.847404+0000","last_fullsized":"2024-09-10T07:35:14.847404+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490415+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:35:42.490415+0000","last_peered":"2024-09-10T07:35:42.490415+0000","last_clean":"2024-09-10T07:35:42.490415+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:35:42.490415+0000","last_undegraded":"2024-09-10T07:35:42.490415+0000","last_fullsized":"2024-09-10T07:35:42.490415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"97'5","reported_seq":305,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.985747+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:36:32.985747+0000","last_peered":"2024-09-10T07:36:32.985747+0000","last_clean":"2024-09-10T07:36:32.985747+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:36:32.985747+0000","last_undegraded":"2024-09-10T07:36:32.985747+0000","last_fullsized":"2024-09-10T07:36:32.985747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977046+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:35:14.977046+0000","last_peered":"2024-09-10T07:35:14.977046+0000","last_clean":"2024-09-10T07:35:14.977046+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:35:14.977046+0000","last_undegraded":"2024-09-10T07:35:14.977046+0000","last_fullsized":"2024-09-10T07:35:14.977046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858166+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:35:14.858166+0000","last_peered":"2024-09-10T07:35:14.858166+0000","last_clean":"2024-09-10T07:35:14.858166+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:35:14.858166+0000","last_undegraded":"2024-09-10T07:35:14.858166+0000","last_fullsized":"2024-09-10T07:35:14.858166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498588+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:35:42.498588+0000","last_peered":"2024-09-10T07:35:42.498588+0000","last_clean":"2024-09-10T07:35:42.498588+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:35:42.498588+0000","last_undegraded":"2024-09-10T07:35:42.498588+0000","last_fullsized":"2024-09-10T07:35:42.498588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880248+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:35:14.880248+0000","last_peered":"2024-09-10T07:35:14.880248+0000","last_clean":"2024-09-10T07:35:14.880248+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:35:14.880248+0000","last_undegraded":"2024-09-10T07:35:14.880248+0000","last_fullsized":"2024-09-10T07:35:14.880248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852575+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:35:14.852575+0000","last_peered":"2024-09-10T07:35:14.852575+0000","last_clean":"2024-09-10T07:35:14.852575+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:35:14.852575+0000","last_undegraded":"2024-09-10T07:35:14.852575+0000","last_fullsized":"2024-09-10T07:35:14.852575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499570+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:35:42.499570+0000","last_peered":"2024-09-10T07:35:42.499570+0000","last_clean":"2024-09-10T07:35:42.499570+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:35:42.499570+0000","last_undegraded":"2024-09-10T07:35:42.499570+0000","last_fullsized":"2024-09-10T07:35:42.499570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879193+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:35:14.879193+0000","last_peered":"2024-09-10T07:35:14.879193+0000","last_clean":"2024-09-10T07:35:14.879193+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:35:14.879193+0000","last_undegraded":"2024-09-10T07:35:14.879193+0000","last_fullsized":"2024-09-10T07:35:14.879193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877898+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:35:14.877898+0000","last_peered":"2024-09-10T07:35:14.877898+0000","last_clean":"2024-09-10T07:35:14.877898+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:35:14.877898+0000","last_undegraded":"2024-09-10T07:35:14.877898+0000","last_fullsized":"2024-09-10T07:35:14.877898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"97'8","reported_seq":23,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.491883+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:35:42.491883+0000","last_peered":"2024-09-10T07:35:42.491883+0000","last_clean":"2024-09-10T07:35:42.491883+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:35:42.491883+0000","last_undegraded":"2024-09-10T07:35:42.491883+0000","last_fullsized":"2024-09-10T07:35:42.491883+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842519+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:35:14.842519+0000","last_peered":"2024-09-10T07:35:14.842519+0000","last_clean":"2024-09-10T07:35:14.842519+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:35:14.842519+0000","last_undegraded":"2024-09-10T07:35:14.842519+0000","last_fullsized":"2024-09-10T07:35:14.842519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":180,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980646+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:36:28.980646+0000","last_peered":"2024-09-10T07:36:28.980646+0000","last_clean":"2024-09-10T07:36:28.980646+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:36:28.980646+0000","last_undegraded":"2024-09-10T07:36:28.980646+0000","last_fullsized":"2024-09-10T07:36:28.980646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475213+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:35:42.475213+0000","last_peered":"2024-09-10T07:35:42.475213+0000","last_clean":"2024-09-10T07:35:42.475213+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:35:42.475213+0000","last_undegraded":"2024-09-10T07:35:42.475213+0000","last_fullsized":"2024-09-10T07:35:42.475213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842633+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:35:14.842633+0000","last_peered":"2024-09-10T07:35:14.842633+0000","last_clean":"2024-09-10T07:35:14.842633+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:35:14.842633+0000","last_undegraded":"2024-09-10T07:35:14.842633+0000","last_fullsized":"2024-09-10T07:35:14.842633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477747+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:35:42.477747+0000","last_peered":"2024-09-10T07:35:42.477747+0000","last_clean":"2024-09-10T07:35:42.477747+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:35:42.477747+0000","last_undegraded":"2024-09-10T07:35:42.477747+0000","last_fullsized":"2024-09-10T07:35:42.477747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"97'12","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476532+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:35:42.476532+0000","last_peered":"2024-09-10T07:35:42.476532+0000","last_clean":"2024-09-10T07:35:42.476532+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:35:42.476532+0000","last_undegraded":"2024-09-10T07:35:42.476532+0000","last_fullsized":"2024-09-10T07:35:42.476532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486476+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:35:42.486476+0000","last_peered":"2024-09-10T07:35:42.486476+0000","last_clean":"2024-09-10T07:35:42.486476+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:35:42.486476+0000","last_undegraded":"2024-09-10T07:35:42.486476+0000","last_fullsized":"2024-09-10T07:35:42.486476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499618+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:35:42.499618+0000","last_peered":"2024-09-10T07:35:42.499618+0000","last_clean":"2024-09-10T07:35:42.499618+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:35:42.499618+0000","last_undegraded":"2024-09-10T07:35:42.499618+0000","last_fullsized":"2024-09-10T07:35:42.499618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487143+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:35:42.487143+0000","last_peered":"2024-09-10T07:35:42.487143+0000","last_clean":"2024-09-10T07:35:42.487143+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:35:42.487143+0000","last_undegraded":"2024-09-10T07:35:42.487143+0000","last_fullsized":"2024-09-10T07:35:42.487143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.470371+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:35:42.470371+0000","last_peered":"2024-09-10T07:35:42.470371+0000","last_clean":"2024-09-10T07:35:42.470371+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:35:42.470371+0000","last_undegraded":"2024-09-10T07:35:42.470371+0000","last_fullsized":"2024-09-10T07:35:42.470371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476911+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:35:42.476911+0000","last_peered":"2024-09-10T07:35:42.476911+0000","last_clean":"2024-09-10T07:35:42.476911+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:35:42.476911+0000","last_undegraded":"2024-09-10T07:35:42.476911+0000","last_fullsized":"2024-09-10T07:35:42.476911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488157+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:35:42.488157+0000","last_peered":"2024-09-10T07:35:42.488157+0000","last_clean":"2024-09-10T07:35:42.488157+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:35:42.488157+0000","last_undegraded":"2024-09-10T07:35:42.488157+0000","last_fullsized":"2024-09-10T07:35:42.488157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500450+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:35:42.500450+0000","last_peered":"2024-09-10T07:35:42.500450+0000","last_clean":"2024-09-10T07:35:42.500450+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:35:42.500450+0000","last_undegraded":"2024-09-10T07:35:42.500450+0000","last_fullsized":"2024-09-10T07:35:42.500450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499239+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:35:42.499239+0000","last_peered":"2024-09-10T07:35:42.499239+0000","last_clean":"2024-09-10T07:35:42.499239+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:35:42.499239+0000","last_undegraded":"2024-09-10T07:35:42.499239+0000","last_fullsized":"2024-09-10T07:35:42.499239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501908+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:35:42.501908+0000","last_peered":"2024-09-10T07:35:42.501908+0000","last_clean":"2024-09-10T07:35:42.501908+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:35:42.501908+0000","last_undegraded":"2024-09-10T07:35:42.501908+0000","last_fullsized":"2024-09-10T07:35:42.501908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493731+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:35:42.493731+0000","last_peered":"2024-09-10T07:35:42.493731+0000","last_clean":"2024-09-10T07:35:42.493731+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:35:42.493731+0000","last_undegraded":"2024-09-10T07:35:42.493731+0000","last_fullsized":"2024-09-10T07:35:42.493731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478046+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:35:42.478046+0000","last_peered":"2024-09-10T07:35:42.478046+0000","last_clean":"2024-09-10T07:35:42.478046+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:35:42.478046+0000","last_undegraded":"2024-09-10T07:35:42.478046+0000","last_fullsized":"2024-09-10T07:35:42.478046+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493144+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:35:42.493144+0000","last_peered":"2024-09-10T07:35:42.493144+0000","last_clean":"2024-09-10T07:35:42.493144+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:35:42.493144+0000","last_undegraded":"2024-09-10T07:35:42.493144+0000","last_fullsized":"2024-09-10T07:35:42.493144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483119+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:35:42.483119+0000","last_peered":"2024-09-10T07:35:42.483119+0000","last_clean":"2024-09-10T07:35:42.483119+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:35:42.483119+0000","last_undegraded":"2024-09-10T07:35:42.483119+0000","last_fullsized":"2024-09-10T07:35:42.483119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502676+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:35:42.502676+0000","last_peered":"2024-09-10T07:35:42.502676+0000","last_clean":"2024-09-10T07:35:42.502676+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:35:42.502676+0000","last_undegraded":"2024-09-10T07:35:42.502676+0000","last_fullsized":"2024-09-10T07:35:42.502676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502776+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:35:42.502776+0000","last_peered":"2024-09-10T07:35:42.502776+0000","last_clean":"2024-09-10T07:35:42.502776+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:35:42.502776+0000","last_undegraded":"2024-09-10T07:35:42.502776+0000","last_fullsized":"2024-09-10T07:35:42.502776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499409+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:35:42.499409+0000","last_peered":"2024-09-10T07:35:42.499409+0000","last_clean":"2024-09-10T07:35:42.499409+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:35:42.499409+0000","last_undegraded":"2024-09-10T07:35:42.499409+0000","last_fullsized":"2024-09-10T07:35:42.499409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502975+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:35:42.502975+0000","last_peered":"2024-09-10T07:35:42.502975+0000","last_clean":"2024-09-10T07:35:42.502975+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:35:42.502975+0000","last_undegraded":"2024-09-10T07:35:42.502975+0000","last_fullsized":"2024-09-10T07:35:42.502975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493842+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:35:42.493842+0000","last_peered":"2024-09-10T07:35:42.493842+0000","last_clean":"2024-09-10T07:35:42.493842+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:35:42.493842+0000","last_undegraded":"2024-09-10T07:35:42.493842+0000","last_fullsized":"2024-09-10T07:35:42.493842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484180+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:35:42.484180+0000","last_peered":"2024-09-10T07:35:42.484180+0000","last_clean":"2024-09-10T07:35:42.484180+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:35:42.484180+0000","last_undegraded":"2024-09-10T07:35:42.484180+0000","last_fullsized":"2024-09-10T07:35:42.484180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483929+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:35:42.483929+0000","last_peered":"2024-09-10T07:35:42.483929+0000","last_clean":"2024-09-10T07:35:42.483929+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:35:42.483929+0000","last_undegraded":"2024-09-10T07:35:42.483929+0000","last_fullsized":"2024-09-10T07:35:42.483929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"97'16","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483320+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:35:42.483320+0000","last_peered":"2024-09-10T07:35:42.483320+0000","last_clean":"2024-09-10T07:35:42.483320+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:35:42.483320+0000","last_undegraded":"2024-09-10T07:35:42.483320+0000","last_fullsized":"2024-09-10T07:35:42.483320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489981+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:35:42.489981+0000","last_peered":"2024-09-10T07:35:42.489981+0000","last_clean":"2024-09-10T07:35:42.489981+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:35:42.489981+0000","last_undegraded":"2024-09-10T07:35:42.489981+0000","last_fullsized":"2024-09-10T07:35:42.489981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500857+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:35:42.500857+0000","last_peered":"2024-09-10T07:35:42.500857+0000","last_clean":"2024-09-10T07:35:42.500857+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:35:42.500857+0000","last_undegraded":"2024-09-10T07:35:42.500857+0000","last_fullsized":"2024-09-10T07:35:42.500857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478497+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:35:42.478497+0000","last_peered":"2024-09-10T07:35:42.478497+0000","last_clean":"2024-09-10T07:35:42.478497+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:35:42.478497+0000","last_undegraded":"2024-09-10T07:35:42.478497+0000","last_fullsized":"2024-09-10T07:35:42.478497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487137+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:35:42.487137+0000","last_peered":"2024-09-10T07:35:42.487137+0000","last_clean":"2024-09-10T07:35:42.487137+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:35:42.487137+0000","last_undegraded":"2024-09-10T07:35:42.487137+0000","last_fullsized":"2024-09-10T07:35:42.487137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"97'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.496842+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:35:42.496842+0000","last_peered":"2024-09-10T07:35:42.496842+0000","last_clean":"2024-09-10T07:35:42.496842+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:35:42.496842+0000","last_undegraded":"2024-09-10T07:35:42.496842+0000","last_fullsized":"2024-09-10T07:35:42.496842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"97'8","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476981+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:35:42.476981+0000","last_peered":"2024-09-10T07:35:42.476981+0000","last_clean":"2024-09-10T07:35:42.476981+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:35:42.476981+0000","last_undegraded":"2024-09-10T07:35:42.476981+0000","last_fullsized":"2024-09-10T07:35:42.476981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.494172+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:35:42.494172+0000","last_peered":"2024-09-10T07:35:42.494172+0000","last_clean":"2024-09-10T07:35:42.494172+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:35:42.494172+0000","last_undegraded":"2024-09-10T07:35:42.494172+0000","last_fullsized":"2024-09-10T07:35:42.494172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502201+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:35:42.502201+0000","last_peered":"2024-09-10T07:35:42.502201+0000","last_clean":"2024-09-10T07:35:42.502201+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:35:42.502201+0000","last_undegraded":"2024-09-10T07:35:42.502201+0000","last_fullsized":"2024-09-10T07:35:42.502201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502439+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:35:42.502439+0000","last_peered":"2024-09-10T07:35:42.502439+0000","last_clean":"2024-09-10T07:35:42.502439+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:35:42.502439+0000","last_undegraded":"2024-09-10T07:35:42.502439+0000","last_fullsized":"2024-09-10T07:35:42.502439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78045,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1309,"num_read_kb":744,"num_write":265,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":228762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449139,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":561212,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710776,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1241088,"data_stored":1218403,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874299,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":565058,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":561228,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1158514,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":557000,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1153562,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":560540,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":556174,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1163264,"data_stored":623364,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":622901,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":557922,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:36:36.105 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T07:36:36.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 -- 172.21.15.46:0/3656800051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc104d60 msgr2=0x7f92fc1051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 --2- 172.21.15.46:0/3656800051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc104d60 0x7f92fc1051e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009900 tx=0x7f92e802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 -- 172.21.15.46:0/3656800051 shutdown_connections 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 --2- 172.21.15.46:0/3656800051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92fc105720 0x7f92fc10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 --2- 172.21.15.46:0/3656800051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc104d60 0x7f92fc1051e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 --2- 172.21.15.46:0/3656800051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc103b70 0x7f92fc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 -- 172.21.15.46:0/3656800051 >> 172.21.15.46:0/3656800051 conn(0x7f92fc0ff2f0 msgr2=0x7f92fc101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 -- 172.21.15.46:0/3656800051 shutdown_connections 2024-09-10T07:36:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.300+0000 7f9300a00640 1 -- 172.21.15.46:0/3656800051 wait complete. 2024-09-10T07:36:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f9300a00640 1 Processor -- start 2024-09-10T07:36:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f9300a00640 1 -- start start 2024-09-10T07:36:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f9300a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc103b70 0x7f92fc199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f9300a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92fc104d60 0x7f92fc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f9300a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc105720 0x7f92fc1a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f9300a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10f840 con 0x7f92fc103b70 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f9300a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10f6c0 con 0x7f92fc105720 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f9300a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10f9c0 con 0x7f92fc104d60 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f92fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc103b70 0x7f92fc199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f92fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc103b70 0x7f92fc199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37224/0 (socket says 172.21.15.46:37224) 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.301+0000 7f92fb400640 1 -- 172.21.15.46:0/452214646 learned_addr learned my addr 172.21.15.46:0/452214646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92fbe00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc105720 0x7f92fc1a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92faa00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92fc104d60 0x7f92fc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92fb400640 1 -- 172.21.15.46:0/452214646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92fc104d60 msgr2=0x7f92fc19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92fb400640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92fc104d60 0x7f92fc19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92fb400640 1 -- 172.21.15.46:0/452214646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc105720 msgr2=0x7f92fc1a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92fb400640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc105720 0x7f92fc1a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92fb400640 1 -- 172.21.15.46:0/452214646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc1a1ae0 con 0x7f92fc103b70 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92faa00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92fc104d60 0x7f92fc19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92fb400640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc103b70 0x7f92fc199ec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00e9c0 tx=0x7f92ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92f3e00640 1 -- 172.21.15.46:0/452214646 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00cde0 con 0x7f92fc103b70 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92f3e00640 1 -- 172.21.15.46:0/452214646 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92ec010910 con 0x7f92fc103b70 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f92f3e00640 1 -- 172.21.15.46:0/452214646 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec018a00 con 0x7f92fc103b70 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.302+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc1a1dd0 con 0x7f92fc103b70 2024-09-10T07:36:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.303+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a2230 con 0x7f92fc103b70 2024-09-10T07:36:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.304+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc10af30 con 0x7f92fc103b70 2024-09-10T07:36:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.305+0000 7f92f3e00640 1 -- 172.21.15.46:0/452214646 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f92ec01f020 con 0x7f92fc103b70 2024-09-10T07:36:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.307+0000 7f92f3e00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92e00a2b90 0x7f92e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.307+0000 7f92f3e00640 1 -- 172.21.15.46:0/452214646 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f92ec0cf690 con 0x7f92fc103b70 2024-09-10T07:36:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.307+0000 7f92faa00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92e00a2b90 0x7f92e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.308+0000 7f92faa00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92e00a2b90 0x7f92e00a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f92e8006010 tx=0x7f92e8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.311+0000 7f92f3e00640 1 -- 172.21.15.46:0/452214646 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92ec09c210 con 0x7f92fc103b70 2024-09-10T07:36:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:36 smithi181 ceph-mon[35397]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:36 smithi063 ceph-mon[35020]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:36 smithi046 ceph-mon[30705]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.496+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92fc062630 con 0x7f92e00a2b90 2024-09-10T07:36:36.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.504+0000 7f92f3e00640 1 -- 172.21.15.46:0/452214646 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340777 (secure 0 0 0) 0x7f92fc062630 con 0x7f92e00a2b90 2024-09-10T07:36:36.509 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:36.513 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92e00a2b90 msgr2=0x7f92e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92e00a2b90 0x7f92e00a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f92e8006010 tx=0x7f92e8000bc0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc103b70 msgr2=0x7f92fc199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc103b70 0x7f92fc199ec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00e9c0 tx=0x7f92ec00ee80 comp rx=0 tx=0).stop 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 shutdown_connections 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc105720 0x7f92fc1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92fc104d60 0x7f92fc19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92e00a2b90 0x7f92e00a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 --2- 172.21.15.46:0/452214646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92fc103b70 0x7f92fc199ec0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 >> 172.21.15.46:0/452214646 conn(0x7f92fc0ff2f0 msgr2=0x7f92fc108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 shutdown_connections 2024-09-10T07:36:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:36.509+0000 7f9300a00640 1 -- 172.21.15.46:0/452214646 wait complete. 2024-09-10T07:36:36.524 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-10T07:36:34.507732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668647,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1435,"num_read_kb":853,"num_write":448,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358316,"kb_used_data":13228,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583508,"statfs":{"total":1151940427776,"available":1151573512192,"internally_reserved":0,"allocated":13545472,"data_stored":8695878,"data_compressed":498372,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003122"},"pg_stats":[{"pgid":"2.1f","version":"97'14","reported_seq":186,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.007589+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:36:32.007589+0000","last_peered":"2024-09-10T07:36:32.007589+0000","last_clean":"2024-09-10T07:36:32.007589+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:36:32.007589+0000","last_undegraded":"2024-09-10T07:36:32.007589+0000","last_fullsized":"2024-09-10T07:36:32.007589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475738+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:35:42.475738+0000","last_peered":"2024-09-10T07:35:42.475738+0000","last_clean":"2024-09-10T07:35:42.475738+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:35:42.475738+0000","last_undegraded":"2024-09-10T07:35:42.475738+0000","last_fullsized":"2024-09-10T07:35:42.475738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842368+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:35:14.842368+0000","last_peered":"2024-09-10T07:35:14.842368+0000","last_clean":"2024-09-10T07:35:14.842368+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:35:14.842368+0000","last_undegraded":"2024-09-10T07:35:14.842368+0000","last_fullsized":"2024-09-10T07:35:14.842368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977747+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:35:14.977747+0000","last_peered":"2024-09-10T07:35:14.977747+0000","last_clean":"2024-09-10T07:35:14.977747+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:35:14.977747+0000","last_undegraded":"2024-09-10T07:35:14.977747+0000","last_fullsized":"2024-09-10T07:35:14.977747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502886+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:35:42.502886+0000","last_peered":"2024-09-10T07:35:42.502886+0000","last_clean":"2024-09-10T07:35:42.502886+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:35:42.502886+0000","last_undegraded":"2024-09-10T07:35:42.502886+0000","last_fullsized":"2024-09-10T07:35:42.502886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881794+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:35:14.881794+0000","last_peered":"2024-09-10T07:35:14.881794+0000","last_clean":"2024-09-10T07:35:14.881794+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:35:14.881794+0000","last_undegraded":"2024-09-10T07:35:14.881794+0000","last_fullsized":"2024-09-10T07:35:14.881794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854574+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:35:14.854574+0000","last_peered":"2024-09-10T07:35:14.854574+0000","last_clean":"2024-09-10T07:35:14.854574+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:35:14.854574+0000","last_undegraded":"2024-09-10T07:35:14.854574+0000","last_fullsized":"2024-09-10T07:35:14.854574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.495636+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:35:42.495636+0000","last_peered":"2024-09-10T07:35:42.495636+0000","last_clean":"2024-09-10T07:35:42.495636+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:35:42.495636+0000","last_undegraded":"2024-09-10T07:35:42.495636+0000","last_fullsized":"2024-09-10T07:35:42.495636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881984+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:35:14.881984+0000","last_peered":"2024-09-10T07:35:14.881984+0000","last_clean":"2024-09-10T07:35:14.881984+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:35:14.881984+0000","last_undegraded":"2024-09-10T07:35:14.881984+0000","last_fullsized":"2024-09-10T07:35:14.881984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854411+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:35:14.854411+0000","last_peered":"2024-09-10T07:35:14.854411+0000","last_clean":"2024-09-10T07:35:14.854411+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:35:14.854411+0000","last_undegraded":"2024-09-10T07:35:14.854411+0000","last_fullsized":"2024-09-10T07:35:14.854411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"97'15","reported_seq":60,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484819+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:35:42.484819+0000","last_peered":"2024-09-10T07:35:42.484819+0000","last_clean":"2024-09-10T07:35:42.484819+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:35:42.484819+0000","last_undegraded":"2024-09-10T07:35:42.484819+0000","last_fullsized":"2024-09-10T07:35:42.484819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881889+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:35:14.881889+0000","last_peered":"2024-09-10T07:35:14.881889+0000","last_clean":"2024-09-10T07:35:14.881889+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:35:14.881889+0000","last_undegraded":"2024-09-10T07:35:14.881889+0000","last_fullsized":"2024-09-10T07:35:14.881889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486825+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:35:42.486825+0000","last_peered":"2024-09-10T07:35:42.486825+0000","last_clean":"2024-09-10T07:35:42.486825+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:35:42.486825+0000","last_undegraded":"2024-09-10T07:35:42.486825+0000","last_fullsized":"2024-09-10T07:35:42.486825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854068+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:35:14.854068+0000","last_peered":"2024-09-10T07:35:14.854068+0000","last_clean":"2024-09-10T07:35:14.854068+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:35:14.854068+0000","last_undegraded":"2024-09-10T07:35:14.854068+0000","last_fullsized":"2024-09-10T07:35:14.854068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882150+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:35:14.882150+0000","last_peered":"2024-09-10T07:35:14.882150+0000","last_clean":"2024-09-10T07:35:14.882150+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:35:14.882150+0000","last_undegraded":"2024-09-10T07:35:14.882150+0000","last_fullsized":"2024-09-10T07:35:14.882150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501326+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:35:42.501326+0000","last_peered":"2024-09-10T07:35:42.501326+0000","last_clean":"2024-09-10T07:35:42.501326+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:35:42.501326+0000","last_undegraded":"2024-09-10T07:35:42.501326+0000","last_fullsized":"2024-09-10T07:35:42.501326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842483+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:35:14.842483+0000","last_peered":"2024-09-10T07:35:14.842483+0000","last_clean":"2024-09-10T07:35:14.842483+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:35:14.842483+0000","last_undegraded":"2024-09-10T07:35:14.842483+0000","last_fullsized":"2024-09-10T07:35:14.842483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882278+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:35:14.882278+0000","last_peered":"2024-09-10T07:35:14.882278+0000","last_clean":"2024-09-10T07:35:14.882278+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:35:14.882278+0000","last_undegraded":"2024-09-10T07:35:14.882278+0000","last_fullsized":"2024-09-10T07:35:14.882278+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502574+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:35:42.502574+0000","last_peered":"2024-09-10T07:35:42.502574+0000","last_clean":"2024-09-10T07:35:42.502574+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:35:42.502574+0000","last_undegraded":"2024-09-10T07:35:42.502574+0000","last_fullsized":"2024-09-10T07:35:42.502574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977257+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:35:14.977257+0000","last_peered":"2024-09-10T07:35:14.977257+0000","last_clean":"2024-09-10T07:35:14.977257+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:35:14.977257+0000","last_undegraded":"2024-09-10T07:35:14.977257+0000","last_fullsized":"2024-09-10T07:35:14.977257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851715+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:35:14.851715+0000","last_peered":"2024-09-10T07:35:14.851715+0000","last_clean":"2024-09-10T07:35:14.851715+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:35:14.851715+0000","last_undegraded":"2024-09-10T07:35:14.851715+0000","last_fullsized":"2024-09-10T07:35:14.851715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"97'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.459952+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:35:42.459952+0000","last_peered":"2024-09-10T07:35:42.459952+0000","last_clean":"2024-09-10T07:35:42.459952+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:35:42.459952+0000","last_undegraded":"2024-09-10T07:35:42.459952+0000","last_fullsized":"2024-09-10T07:35:42.459952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976752+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:35:14.976752+0000","last_peered":"2024-09-10T07:35:14.976752+0000","last_clean":"2024-09-10T07:35:14.976752+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:35:14.976752+0000","last_undegraded":"2024-09-10T07:35:14.976752+0000","last_fullsized":"2024-09-10T07:35:14.976752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882210+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:35:14.882210+0000","last_peered":"2024-09-10T07:35:14.882210+0000","last_clean":"2024-09-10T07:35:14.882210+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:35:14.882210+0000","last_undegraded":"2024-09-10T07:35:14.882210+0000","last_fullsized":"2024-09-10T07:35:14.882210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502703+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:35:42.502703+0000","last_peered":"2024-09-10T07:35:42.502703+0000","last_clean":"2024-09-10T07:35:42.502703+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:35:42.502703+0000","last_undegraded":"2024-09-10T07:35:42.502703+0000","last_fullsized":"2024-09-10T07:35:42.502703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882535+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:35:14.882535+0000","last_peered":"2024-09-10T07:35:14.882535+0000","last_clean":"2024-09-10T07:35:14.882535+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:35:14.882535+0000","last_undegraded":"2024-09-10T07:35:14.882535+0000","last_fullsized":"2024-09-10T07:35:14.882535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842382+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:35:14.842382+0000","last_peered":"2024-09-10T07:35:14.842382+0000","last_clean":"2024-09-10T07:35:14.842382+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:35:14.842382+0000","last_undegraded":"2024-09-10T07:35:14.842382+0000","last_fullsized":"2024-09-10T07:35:14.842382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499802+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:35:42.499802+0000","last_peered":"2024-09-10T07:35:42.499802+0000","last_clean":"2024-09-10T07:35:42.499802+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:35:42.499802+0000","last_undegraded":"2024-09-10T07:35:42.499802+0000","last_fullsized":"2024-09-10T07:35:42.499802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'18","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:43.451679+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:35:43.451679+0000","last_peered":"2024-09-10T07:35:43.451679+0000","last_clean":"2024-09-10T07:35:43.451679+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:35:43.451679+0000","last_undegraded":"2024-09-10T07:35:43.451679+0000","last_fullsized":"2024-09-10T07:35:43.451679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976909+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:35:14.976909+0000","last_peered":"2024-09-10T07:35:14.976909+0000","last_clean":"2024-09-10T07:35:14.976909+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:35:14.976909+0000","last_undegraded":"2024-09-10T07:35:14.976909+0000","last_fullsized":"2024-09-10T07:35:14.976909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.481577+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:35:42.481577+0000","last_peered":"2024-09-10T07:35:42.481577+0000","last_clean":"2024-09-10T07:35:42.481577+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:35:42.481577+0000","last_undegraded":"2024-09-10T07:35:42.481577+0000","last_fullsized":"2024-09-10T07:35:42.481577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.872520+0000","last_change":"2024-09-10T07:35:14.872520+0000","last_active":"2024-09-10T07:35:14.872520+0000","last_peered":"2024-09-10T07:35:14.872520+0000","last_clean":"2024-09-10T07:35:14.872520+0000","last_became_active":"2024-09-10T07:35:14.860686+0000","last_became_peered":"2024-09-10T07:35:14.860686+0000","last_unstale":"2024-09-10T07:35:14.872520+0000","last_undegraded":"2024-09-10T07:35:14.872520+0000","last_fullsized":"2024-09-10T07:35:14.872520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:42.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852355+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:35:14.852355+0000","last_peered":"2024-09-10T07:35:14.852355+0000","last_clean":"2024-09-10T07:35:14.852355+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:35:14.852355+0000","last_undegraded":"2024-09-10T07:35:14.852355+0000","last_fullsized":"2024-09-10T07:35:14.852355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"97'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489096+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:35:42.489096+0000","last_peered":"2024-09-10T07:35:42.489096+0000","last_clean":"2024-09-10T07:35:42.489096+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:35:42.489096+0000","last_undegraded":"2024-09-10T07:35:42.489096+0000","last_fullsized":"2024-09-10T07:35:42.489096+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848959+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:35:14.848959+0000","last_peered":"2024-09-10T07:35:14.848959+0000","last_clean":"2024-09-10T07:35:14.848959+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:35:14.848959+0000","last_undegraded":"2024-09-10T07:35:14.848959+0000","last_fullsized":"2024-09-10T07:35:14.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853941+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:35:14.853941+0000","last_peered":"2024-09-10T07:35:14.853941+0000","last_clean":"2024-09-10T07:35:14.853941+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:35:14.853941+0000","last_undegraded":"2024-09-10T07:35:14.853941+0000","last_fullsized":"2024-09-10T07:35:14.853941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.485460+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:35:42.485460+0000","last_peered":"2024-09-10T07:35:42.485460+0000","last_clean":"2024-09-10T07:35:42.485460+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:35:42.485460+0000","last_undegraded":"2024-09-10T07:35:42.485460+0000","last_fullsized":"2024-09-10T07:35:42.485460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"97'7","reported_seq":180,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:30.004819+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:36:30.004819+0000","last_peered":"2024-09-10T07:36:30.004819+0000","last_clean":"2024-09-10T07:36:30.004819+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:36:30.004819+0000","last_undegraded":"2024-09-10T07:36:30.004819+0000","last_fullsized":"2024-09-10T07:36:30.004819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879712+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:35:14.879712+0000","last_peered":"2024-09-10T07:35:14.879712+0000","last_clean":"2024-09-10T07:35:14.879712+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:35:14.879712+0000","last_undegraded":"2024-09-10T07:35:14.879712+0000","last_fullsized":"2024-09-10T07:35:14.879712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476516+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:35:42.476516+0000","last_peered":"2024-09-10T07:35:42.476516+0000","last_clean":"2024-09-10T07:35:42.476516+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:35:42.476516+0000","last_undegraded":"2024-09-10T07:35:42.476516+0000","last_fullsized":"2024-09-10T07:35:42.476516+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":176,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.979799+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:36:28.979799+0000","last_peered":"2024-09-10T07:36:28.979799+0000","last_clean":"2024-09-10T07:36:28.979799+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:36:28.979799+0000","last_undegraded":"2024-09-10T07:36:28.979799+0000","last_fullsized":"2024-09-10T07:36:28.979799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880600+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:35:14.880600+0000","last_peered":"2024-09-10T07:35:14.880600+0000","last_clean":"2024-09-10T07:35:14.880600+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:35:14.880600+0000","last_undegraded":"2024-09-10T07:35:14.880600+0000","last_fullsized":"2024-09-10T07:35:14.880600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.479337+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:35:42.479337+0000","last_peered":"2024-09-10T07:35:42.479337+0000","last_clean":"2024-09-10T07:35:42.479337+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:35:42.479337+0000","last_undegraded":"2024-09-10T07:35:42.479337+0000","last_fullsized":"2024-09-10T07:35:42.479337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842761+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:35:14.842761+0000","last_peered":"2024-09-10T07:35:14.842761+0000","last_clean":"2024-09-10T07:35:14.842761+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:35:14.842761+0000","last_undegraded":"2024-09-10T07:35:14.842761+0000","last_fullsized":"2024-09-10T07:35:14.842761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498940+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:35:42.498940+0000","last_peered":"2024-09-10T07:35:42.498940+0000","last_clean":"2024-09-10T07:35:42.498940+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:35:42.498940+0000","last_undegraded":"2024-09-10T07:35:42.498940+0000","last_fullsized":"2024-09-10T07:35:42.498940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882510+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:35:14.882510+0000","last_peered":"2024-09-10T07:35:14.882510+0000","last_clean":"2024-09-10T07:35:14.882510+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:35:14.882510+0000","last_undegraded":"2024-09-10T07:35:14.882510+0000","last_fullsized":"2024-09-10T07:35:14.882510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842861+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:35:14.842861+0000","last_peered":"2024-09-10T07:35:14.842861+0000","last_clean":"2024-09-10T07:35:14.842861+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:35:14.842861+0000","last_undegraded":"2024-09-10T07:35:14.842861+0000","last_fullsized":"2024-09-10T07:35:14.842861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843182+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:35:14.843182+0000","last_peered":"2024-09-10T07:35:14.843182+0000","last_clean":"2024-09-10T07:35:14.843182+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:35:14.843182+0000","last_undegraded":"2024-09-10T07:35:14.843182+0000","last_fullsized":"2024-09-10T07:35:14.843182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484404+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:35:42.484404+0000","last_peered":"2024-09-10T07:35:42.484404+0000","last_clean":"2024-09-10T07:35:42.484404+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:35:42.484404+0000","last_undegraded":"2024-09-10T07:35:42.484404+0000","last_fullsized":"2024-09-10T07:35:42.484404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":176,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980824+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:36:28.980824+0000","last_peered":"2024-09-10T07:36:28.980824+0000","last_clean":"2024-09-10T07:36:28.980824+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:36:28.980824+0000","last_undegraded":"2024-09-10T07:36:28.980824+0000","last_fullsized":"2024-09-10T07:36:28.980824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976793+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:35:14.976793+0000","last_peered":"2024-09-10T07:35:14.976793+0000","last_clean":"2024-09-10T07:35:14.976793+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:35:14.976793+0000","last_undegraded":"2024-09-10T07:35:14.976793+0000","last_fullsized":"2024-09-10T07:35:14.976793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487151+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:35:42.487151+0000","last_peered":"2024-09-10T07:35:42.487151+0000","last_clean":"2024-09-10T07:35:42.487151+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:35:42.487151+0000","last_undegraded":"2024-09-10T07:35:42.487151+0000","last_fullsized":"2024-09-10T07:35:42.487151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880698+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:35:14.880698+0000","last_peered":"2024-09-10T07:35:14.880698+0000","last_clean":"2024-09-10T07:35:14.880698+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:35:14.880698+0000","last_undegraded":"2024-09-10T07:35:14.880698+0000","last_fullsized":"2024-09-10T07:35:14.880698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857702+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:35:14.857702+0000","last_peered":"2024-09-10T07:35:14.857702+0000","last_clean":"2024-09-10T07:35:14.857702+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:35:14.857702+0000","last_undegraded":"2024-09-10T07:35:14.857702+0000","last_fullsized":"2024-09-10T07:35:14.857702+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477302+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:35:42.477302+0000","last_peered":"2024-09-10T07:35:42.477302+0000","last_clean":"2024-09-10T07:35:42.477302+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:35:42.477302+0000","last_undegraded":"2024-09-10T07:35:42.477302+0000","last_fullsized":"2024-09-10T07:35:42.477302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880480+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:35:14.880480+0000","last_peered":"2024-09-10T07:35:14.880480+0000","last_clean":"2024-09-10T07:35:14.880480+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:35:14.880480+0000","last_undegraded":"2024-09-10T07:35:14.880480+0000","last_fullsized":"2024-09-10T07:35:14.880480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879975+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:35:14.879975+0000","last_peered":"2024-09-10T07:35:14.879975+0000","last_clean":"2024-09-10T07:35:14.879975+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:35:14.879975+0000","last_undegraded":"2024-09-10T07:35:14.879975+0000","last_fullsized":"2024-09-10T07:35:14.879975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.474216+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:35:42.474216+0000","last_peered":"2024-09-10T07:35:42.474216+0000","last_clean":"2024-09-10T07:35:42.474216+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:35:42.474216+0000","last_undegraded":"2024-09-10T07:35:42.474216+0000","last_fullsized":"2024-09-10T07:35:42.474216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854462+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:35:14.854462+0000","last_peered":"2024-09-10T07:35:14.854462+0000","last_clean":"2024-09-10T07:35:14.854462+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:35:14.854462+0000","last_undegraded":"2024-09-10T07:35:14.854462+0000","last_fullsized":"2024-09-10T07:35:14.854462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879978+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:35:14.879978+0000","last_peered":"2024-09-10T07:35:14.879978+0000","last_clean":"2024-09-10T07:35:14.879978+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:35:14.879978+0000","last_undegraded":"2024-09-10T07:35:14.879978+0000","last_fullsized":"2024-09-10T07:35:14.879978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.480953+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:35:42.480953+0000","last_peered":"2024-09-10T07:35:42.480953+0000","last_clean":"2024-09-10T07:35:42.480953+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:35:42.480953+0000","last_undegraded":"2024-09-10T07:35:42.480953+0000","last_fullsized":"2024-09-10T07:35:42.480953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853997+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:35:14.853997+0000","last_peered":"2024-09-10T07:35:14.853997+0000","last_clean":"2024-09-10T07:35:14.853997+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:35:14.853997+0000","last_undegraded":"2024-09-10T07:35:14.853997+0000","last_fullsized":"2024-09-10T07:35:14.853997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880388+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:35:14.880388+0000","last_peered":"2024-09-10T07:35:14.880388+0000","last_clean":"2024-09-10T07:35:14.880388+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:35:14.880388+0000","last_undegraded":"2024-09-10T07:35:14.880388+0000","last_fullsized":"2024-09-10T07:35:14.880388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487180+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:35:42.487180+0000","last_peered":"2024-09-10T07:35:42.487180+0000","last_clean":"2024-09-10T07:35:42.487180+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:35:42.487180+0000","last_undegraded":"2024-09-10T07:35:42.487180+0000","last_fullsized":"2024-09-10T07:35:42.487180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":175,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980178+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:36:28.980178+0000","last_peered":"2024-09-10T07:36:28.980178+0000","last_clean":"2024-09-10T07:36:28.980178+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:36:28.980178+0000","last_undegraded":"2024-09-10T07:36:28.980178+0000","last_fullsized":"2024-09-10T07:36:28.980178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851402+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:35:14.851402+0000","last_peered":"2024-09-10T07:35:14.851402+0000","last_clean":"2024-09-10T07:35:14.851402+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:35:14.851402+0000","last_undegraded":"2024-09-10T07:35:14.851402+0000","last_fullsized":"2024-09-10T07:35:14.851402+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488551+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:35:42.488551+0000","last_peered":"2024-09-10T07:35:42.488551+0000","last_clean":"2024-09-10T07:35:42.488551+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:35:42.488551+0000","last_undegraded":"2024-09-10T07:35:42.488551+0000","last_fullsized":"2024-09-10T07:35:42.488551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880166+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:35:14.880166+0000","last_peered":"2024-09-10T07:35:14.880166+0000","last_clean":"2024-09-10T07:35:14.880166+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:35:14.880166+0000","last_undegraded":"2024-09-10T07:35:14.880166+0000","last_fullsized":"2024-09-10T07:35:14.880166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852262+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:35:14.852262+0000","last_peered":"2024-09-10T07:35:14.852262+0000","last_clean":"2024-09-10T07:35:14.852262+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:35:14.852262+0000","last_undegraded":"2024-09-10T07:35:14.852262+0000","last_fullsized":"2024-09-10T07:35:14.852262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500238+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:35:42.500238+0000","last_peered":"2024-09-10T07:35:42.500238+0000","last_clean":"2024-09-10T07:35:42.500238+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:35:42.500238+0000","last_undegraded":"2024-09-10T07:35:42.500238+0000","last_fullsized":"2024-09-10T07:35:42.500238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852264+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:35:14.852264+0000","last_peered":"2024-09-10T07:35:14.852264+0000","last_clean":"2024-09-10T07:35:14.852264+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:35:14.852264+0000","last_undegraded":"2024-09-10T07:35:14.852264+0000","last_fullsized":"2024-09-10T07:35:14.852264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.469244+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:35:42.469244+0000","last_peered":"2024-09-10T07:35:42.469244+0000","last_clean":"2024-09-10T07:35:42.469244+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:35:42.469244+0000","last_undegraded":"2024-09-10T07:35:42.469244+0000","last_fullsized":"2024-09-10T07:35:42.469244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"97'42","reported_seq":354,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.985620+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:36:32.985620+0000","last_peered":"2024-09-10T07:36:32.985620+0000","last_clean":"2024-09-10T07:36:32.985620+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:36:32.985620+0000","last_undegraded":"2024-09-10T07:36:32.985620+0000","last_fullsized":"2024-09-10T07:36:32.985620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70555,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":268,"num_read_kb":203,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848093+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:35:14.848093+0000","last_peered":"2024-09-10T07:35:14.848093+0000","last_clean":"2024-09-10T07:35:14.848093+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:35:14.848093+0000","last_undegraded":"2024-09-10T07:35:14.848093+0000","last_fullsized":"2024-09-10T07:35:14.848093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490736+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:35:42.490736+0000","last_peered":"2024-09-10T07:35:42.490736+0000","last_clean":"2024-09-10T07:35:42.490736+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:35:42.490736+0000","last_undegraded":"2024-09-10T07:35:42.490736+0000","last_fullsized":"2024-09-10T07:35:42.490736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880613+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:35:14.880613+0000","last_peered":"2024-09-10T07:35:14.880613+0000","last_clean":"2024-09-10T07:35:14.880613+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:35:14.880613+0000","last_undegraded":"2024-09-10T07:35:14.880613+0000","last_fullsized":"2024-09-10T07:35:14.880613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882346+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:35:14.882346+0000","last_peered":"2024-09-10T07:35:14.882346+0000","last_clean":"2024-09-10T07:35:14.882346+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:35:14.882346+0000","last_undegraded":"2024-09-10T07:35:14.882346+0000","last_fullsized":"2024-09-10T07:35:14.882346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502535+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:35:42.502535+0000","last_peered":"2024-09-10T07:35:42.502535+0000","last_clean":"2024-09-10T07:35:42.502535+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:35:42.502535+0000","last_undegraded":"2024-09-10T07:35:42.502535+0000","last_fullsized":"2024-09-10T07:35:42.502535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843243+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:35:14.843243+0000","last_peered":"2024-09-10T07:35:14.843243+0000","last_clean":"2024-09-10T07:35:14.843243+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:35:14.843243+0000","last_undegraded":"2024-09-10T07:35:14.843243+0000","last_fullsized":"2024-09-10T07:35:14.843243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847404+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:35:14.847404+0000","last_peered":"2024-09-10T07:35:14.847404+0000","last_clean":"2024-09-10T07:35:14.847404+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:35:14.847404+0000","last_undegraded":"2024-09-10T07:35:14.847404+0000","last_fullsized":"2024-09-10T07:35:14.847404+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490415+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:35:42.490415+0000","last_peered":"2024-09-10T07:35:42.490415+0000","last_clean":"2024-09-10T07:35:42.490415+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:35:42.490415+0000","last_undegraded":"2024-09-10T07:35:42.490415+0000","last_fullsized":"2024-09-10T07:35:42.490415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"97'5","reported_seq":305,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:32.985747+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:36:32.985747+0000","last_peered":"2024-09-10T07:36:32.985747+0000","last_clean":"2024-09-10T07:36:32.985747+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:36:32.985747+0000","last_undegraded":"2024-09-10T07:36:32.985747+0000","last_fullsized":"2024-09-10T07:36:32.985747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977046+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:35:14.977046+0000","last_peered":"2024-09-10T07:35:14.977046+0000","last_clean":"2024-09-10T07:35:14.977046+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:35:14.977046+0000","last_undegraded":"2024-09-10T07:35:14.977046+0000","last_fullsized":"2024-09-10T07:35:14.977046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858166+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:35:14.858166+0000","last_peered":"2024-09-10T07:35:14.858166+0000","last_clean":"2024-09-10T07:35:14.858166+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:35:14.858166+0000","last_undegraded":"2024-09-10T07:35:14.858166+0000","last_fullsized":"2024-09-10T07:35:14.858166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498588+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:35:42.498588+0000","last_peered":"2024-09-10T07:35:42.498588+0000","last_clean":"2024-09-10T07:35:42.498588+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:35:42.498588+0000","last_undegraded":"2024-09-10T07:35:42.498588+0000","last_fullsized":"2024-09-10T07:35:42.498588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880248+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:35:14.880248+0000","last_peered":"2024-09-10T07:35:14.880248+0000","last_clean":"2024-09-10T07:35:14.880248+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:35:14.880248+0000","last_undegraded":"2024-09-10T07:35:14.880248+0000","last_fullsized":"2024-09-10T07:35:14.880248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852575+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:35:14.852575+0000","last_peered":"2024-09-10T07:35:14.852575+0000","last_clean":"2024-09-10T07:35:14.852575+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:35:14.852575+0000","last_undegraded":"2024-09-10T07:35:14.852575+0000","last_fullsized":"2024-09-10T07:35:14.852575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499570+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:35:42.499570+0000","last_peered":"2024-09-10T07:35:42.499570+0000","last_clean":"2024-09-10T07:35:42.499570+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:35:42.499570+0000","last_undegraded":"2024-09-10T07:35:42.499570+0000","last_fullsized":"2024-09-10T07:35:42.499570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879193+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:35:14.879193+0000","last_peered":"2024-09-10T07:35:14.879193+0000","last_clean":"2024-09-10T07:35:14.879193+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:35:14.879193+0000","last_undegraded":"2024-09-10T07:35:14.879193+0000","last_fullsized":"2024-09-10T07:35:14.879193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877898+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:35:14.877898+0000","last_peered":"2024-09-10T07:35:14.877898+0000","last_clean":"2024-09-10T07:35:14.877898+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:35:14.877898+0000","last_undegraded":"2024-09-10T07:35:14.877898+0000","last_fullsized":"2024-09-10T07:35:14.877898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"97'8","reported_seq":23,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.491883+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:35:42.491883+0000","last_peered":"2024-09-10T07:35:42.491883+0000","last_clean":"2024-09-10T07:35:42.491883+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:35:42.491883+0000","last_undegraded":"2024-09-10T07:35:42.491883+0000","last_fullsized":"2024-09-10T07:35:42.491883+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842519+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:35:14.842519+0000","last_peered":"2024-09-10T07:35:14.842519+0000","last_clean":"2024-09-10T07:35:14.842519+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:35:14.842519+0000","last_undegraded":"2024-09-10T07:35:14.842519+0000","last_fullsized":"2024-09-10T07:35:14.842519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":180,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:28.980646+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:36:28.980646+0000","last_peered":"2024-09-10T07:36:28.980646+0000","last_clean":"2024-09-10T07:36:28.980646+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:36:28.980646+0000","last_undegraded":"2024-09-10T07:36:28.980646+0000","last_fullsized":"2024-09-10T07:36:28.980646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475213+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:35:42.475213+0000","last_peered":"2024-09-10T07:35:42.475213+0000","last_clean":"2024-09-10T07:35:42.475213+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:35:42.475213+0000","last_undegraded":"2024-09-10T07:35:42.475213+0000","last_fullsized":"2024-09-10T07:35:42.475213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842633+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:35:14.842633+0000","last_peered":"2024-09-10T07:35:14.842633+0000","last_clean":"2024-09-10T07:35:14.842633+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:35:14.842633+0000","last_undegraded":"2024-09-10T07:35:14.842633+0000","last_fullsized":"2024-09-10T07:35:14.842633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477747+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:35:42.477747+0000","last_peered":"2024-09-10T07:35:42.477747+0000","last_clean":"2024-09-10T07:35:42.477747+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:35:42.477747+0000","last_undegraded":"2024-09-10T07:35:42.477747+0000","last_fullsized":"2024-09-10T07:35:42.477747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"97'12","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476532+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:35:42.476532+0000","last_peered":"2024-09-10T07:35:42.476532+0000","last_clean":"2024-09-10T07:35:42.476532+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:35:42.476532+0000","last_undegraded":"2024-09-10T07:35:42.476532+0000","last_fullsized":"2024-09-10T07:35:42.476532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486476+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:35:42.486476+0000","last_peered":"2024-09-10T07:35:42.486476+0000","last_clean":"2024-09-10T07:35:42.486476+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:35:42.486476+0000","last_undegraded":"2024-09-10T07:35:42.486476+0000","last_fullsized":"2024-09-10T07:35:42.486476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499618+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:35:42.499618+0000","last_peered":"2024-09-10T07:35:42.499618+0000","last_clean":"2024-09-10T07:35:42.499618+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:35:42.499618+0000","last_undegraded":"2024-09-10T07:35:42.499618+0000","last_fullsized":"2024-09-10T07:35:42.499618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487143+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:35:42.487143+0000","last_peered":"2024-09-10T07:35:42.487143+0000","last_clean":"2024-09-10T07:35:42.487143+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:35:42.487143+0000","last_undegraded":"2024-09-10T07:35:42.487143+0000","last_fullsized":"2024-09-10T07:35:42.487143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.470371+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:35:42.470371+0000","last_peered":"2024-09-10T07:35:42.470371+0000","last_clean":"2024-09-10T07:35:42.470371+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:35:42.470371+0000","last_undegraded":"2024-09-10T07:35:42.470371+0000","last_fullsized":"2024-09-10T07:35:42.470371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476911+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:35:42.476911+0000","last_peered":"2024-09-10T07:35:42.476911+0000","last_clean":"2024-09-10T07:35:42.476911+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:35:42.476911+0000","last_undegraded":"2024-09-10T07:35:42.476911+0000","last_fullsized":"2024-09-10T07:35:42.476911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488157+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:35:42.488157+0000","last_peered":"2024-09-10T07:35:42.488157+0000","last_clean":"2024-09-10T07:35:42.488157+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:35:42.488157+0000","last_undegraded":"2024-09-10T07:35:42.488157+0000","last_fullsized":"2024-09-10T07:35:42.488157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500450+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:35:42.500450+0000","last_peered":"2024-09-10T07:35:42.500450+0000","last_clean":"2024-09-10T07:35:42.500450+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:35:42.500450+0000","last_undegraded":"2024-09-10T07:35:42.500450+0000","last_fullsized":"2024-09-10T07:35:42.500450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499239+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:35:42.499239+0000","last_peered":"2024-09-10T07:35:42.499239+0000","last_clean":"2024-09-10T07:35:42.499239+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:35:42.499239+0000","last_undegraded":"2024-09-10T07:35:42.499239+0000","last_fullsized":"2024-09-10T07:35:42.499239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501908+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:35:42.501908+0000","last_peered":"2024-09-10T07:35:42.501908+0000","last_clean":"2024-09-10T07:35:42.501908+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:35:42.501908+0000","last_undegraded":"2024-09-10T07:35:42.501908+0000","last_fullsized":"2024-09-10T07:35:42.501908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493731+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:35:42.493731+0000","last_peered":"2024-09-10T07:35:42.493731+0000","last_clean":"2024-09-10T07:35:42.493731+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:35:42.493731+0000","last_undegraded":"2024-09-10T07:35:42.493731+0000","last_fullsized":"2024-09-10T07:35:42.493731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478046+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:35:42.478046+0000","last_peered":"2024-09-10T07:35:42.478046+0000","last_clean":"2024-09-10T07:35:42.478046+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:35:42.478046+0000","last_undegraded":"2024-09-10T07:35:42.478046+0000","last_fullsized":"2024-09-10T07:35:42.478046+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493144+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:35:42.493144+0000","last_peered":"2024-09-10T07:35:42.493144+0000","last_clean":"2024-09-10T07:35:42.493144+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:35:42.493144+0000","last_undegraded":"2024-09-10T07:35:42.493144+0000","last_fullsized":"2024-09-10T07:35:42.493144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483119+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:35:42.483119+0000","last_peered":"2024-09-10T07:35:42.483119+0000","last_clean":"2024-09-10T07:35:42.483119+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:35:42.483119+0000","last_undegraded":"2024-09-10T07:35:42.483119+0000","last_fullsized":"2024-09-10T07:35:42.483119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502676+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:35:42.502676+0000","last_peered":"2024-09-10T07:35:42.502676+0000","last_clean":"2024-09-10T07:35:42.502676+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:35:42.502676+0000","last_undegraded":"2024-09-10T07:35:42.502676+0000","last_fullsized":"2024-09-10T07:35:42.502676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502776+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:35:42.502776+0000","last_peered":"2024-09-10T07:35:42.502776+0000","last_clean":"2024-09-10T07:35:42.502776+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:35:42.502776+0000","last_undegraded":"2024-09-10T07:35:42.502776+0000","last_fullsized":"2024-09-10T07:35:42.502776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499409+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:35:42.499409+0000","last_peered":"2024-09-10T07:35:42.499409+0000","last_clean":"2024-09-10T07:35:42.499409+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:35:42.499409+0000","last_undegraded":"2024-09-10T07:35:42.499409+0000","last_fullsized":"2024-09-10T07:35:42.499409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502975+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:35:42.502975+0000","last_peered":"2024-09-10T07:35:42.502975+0000","last_clean":"2024-09-10T07:35:42.502975+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:35:42.502975+0000","last_undegraded":"2024-09-10T07:35:42.502975+0000","last_fullsized":"2024-09-10T07:35:42.502975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493842+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:35:42.493842+0000","last_peered":"2024-09-10T07:35:42.493842+0000","last_clean":"2024-09-10T07:35:42.493842+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:35:42.493842+0000","last_undegraded":"2024-09-10T07:35:42.493842+0000","last_fullsized":"2024-09-10T07:35:42.493842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484180+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:35:42.484180+0000","last_peered":"2024-09-10T07:35:42.484180+0000","last_clean":"2024-09-10T07:35:42.484180+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:35:42.484180+0000","last_undegraded":"2024-09-10T07:35:42.484180+0000","last_fullsized":"2024-09-10T07:35:42.484180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483929+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:35:42.483929+0000","last_peered":"2024-09-10T07:35:42.483929+0000","last_clean":"2024-09-10T07:35:42.483929+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:35:42.483929+0000","last_undegraded":"2024-09-10T07:35:42.483929+0000","last_fullsized":"2024-09-10T07:35:42.483929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"97'16","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483320+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:35:42.483320+0000","last_peered":"2024-09-10T07:35:42.483320+0000","last_clean":"2024-09-10T07:35:42.483320+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:35:42.483320+0000","last_undegraded":"2024-09-10T07:35:42.483320+0000","last_fullsized":"2024-09-10T07:35:42.483320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489981+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:35:42.489981+0000","last_peered":"2024-09-10T07:35:42.489981+0000","last_clean":"2024-09-10T07:35:42.489981+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:35:42.489981+0000","last_undegraded":"2024-09-10T07:35:42.489981+0000","last_fullsized":"2024-09-10T07:35:42.489981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500857+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:35:42.500857+0000","last_peered":"2024-09-10T07:35:42.500857+0000","last_clean":"2024-09-10T07:35:42.500857+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:35:42.500857+0000","last_undegraded":"2024-09-10T07:35:42.500857+0000","last_fullsized":"2024-09-10T07:35:42.500857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478497+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:35:42.478497+0000","last_peered":"2024-09-10T07:35:42.478497+0000","last_clean":"2024-09-10T07:35:42.478497+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:35:42.478497+0000","last_undegraded":"2024-09-10T07:35:42.478497+0000","last_fullsized":"2024-09-10T07:35:42.478497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487137+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:35:42.487137+0000","last_peered":"2024-09-10T07:35:42.487137+0000","last_clean":"2024-09-10T07:35:42.487137+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:35:42.487137+0000","last_undegraded":"2024-09-10T07:35:42.487137+0000","last_fullsized":"2024-09-10T07:35:42.487137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"97'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.496842+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:35:42.496842+0000","last_peered":"2024-09-10T07:35:42.496842+0000","last_clean":"2024-09-10T07:35:42.496842+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:35:42.496842+0000","last_undegraded":"2024-09-10T07:35:42.496842+0000","last_fullsized":"2024-09-10T07:35:42.496842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"97'8","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476981+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:35:42.476981+0000","last_peered":"2024-09-10T07:35:42.476981+0000","last_clean":"2024-09-10T07:35:42.476981+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:35:42.476981+0000","last_undegraded":"2024-09-10T07:35:42.476981+0000","last_fullsized":"2024-09-10T07:35:42.476981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.494172+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:35:42.494172+0000","last_peered":"2024-09-10T07:35:42.494172+0000","last_clean":"2024-09-10T07:35:42.494172+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:35:42.494172+0000","last_undegraded":"2024-09-10T07:35:42.494172+0000","last_fullsized":"2024-09-10T07:35:42.494172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502201+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:35:42.502201+0000","last_peered":"2024-09-10T07:35:42.502201+0000","last_clean":"2024-09-10T07:35:42.502201+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:35:42.502201+0000","last_undegraded":"2024-09-10T07:35:42.502201+0000","last_fullsized":"2024-09-10T07:35:42.502201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502439+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:35:42.502439+0000","last_peered":"2024-09-10T07:35:42.502439+0000","last_clean":"2024-09-10T07:35:42.502439+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:35:42.502439+0000","last_undegraded":"2024-09-10T07:35:42.502439+0000","last_fullsized":"2024-09-10T07:35:42.502439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78045,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1309,"num_read_kb":744,"num_write":265,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":228762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449139,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":561212,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710776,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1241088,"data_stored":1218403,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874299,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":565058,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":561228,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1158514,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":557000,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1153562,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":560540,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":556174,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1163264,"data_stored":623364,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":622901,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":557922,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:36:36.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:36:36.942 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:37.403 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:37 smithi046 ceph-mon[30705]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:37.403 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:37 smithi046 ceph-mon[30705]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:37.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.516+0000 7fd1ba600640 1 -- 172.21.15.46:0/1974645178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 msgr2=0x7fd1b4112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.516+0000 7fd1ba600640 1 --2- 172.21.15.46:0/1974645178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 0x7fd1b4112510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009860 tx=0x7fd19c02a340 comp rx=0 tx=0).stop 2024-09-10T07:36:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.517+0000 7fd1ba600640 1 -- 172.21.15.46:0/1974645178 shutdown_connections 2024-09-10T07:36:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.517+0000 7fd1ba600640 1 --2- 172.21.15.46:0/1974645178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4112a50 0x7fd1b4114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.517+0000 7fd1ba600640 1 --2- 172.21.15.46:0/1974645178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 0x7fd1b4112510 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.517+0000 7fd1ba600640 1 --2- 172.21.15.46:0/1974645178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1b4076720 0x7fd1b4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.517+0000 7fd1ba600640 1 -- 172.21.15.46:0/1974645178 >> 172.21.15.46:0/1974645178 conn(0x7fd1b40ffa50 msgr2=0x7fd1b4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.517+0000 7fd1ba600640 1 -- 172.21.15.46:0/1974645178 shutdown_connections 2024-09-10T07:36:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.517+0000 7fd1ba600640 1 -- 172.21.15.46:0/1974645178 wait complete. 2024-09-10T07:36:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1ba600640 1 Processor -- start 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1ba600640 1 -- start start 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1ba600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1b4076720 0x7fd1b419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1ba600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 0x7fd1b419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1ba600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4112a50 0x7fd1b41a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1ba600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41185c0 con 0x7fd1b4077040 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1ba600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4118440 con 0x7fd1b4076720 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1ba600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4118740 con 0x7fd1b4112a50 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 0x7fd1b419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 0x7fd1b419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37240/0 (socket says 172.21.15.46:37240) 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.518+0000 7fd1abe00640 1 -- 172.21.15.46:0/3600777693 learned_addr learned my addr 172.21.15.46:0/3600777693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1b3400640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1b4076720 0x7fd1b419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1b3e00640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4112a50 0x7fd1b41a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:37.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1b3400640 1 -- 172.21.15.46:0/3600777693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4112a50 msgr2=0x7fd1b41a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1b3400640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4112a50 0x7fd1b41a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1b3400640 1 -- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 msgr2=0x7fd1b419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1b3400640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 0x7fd1b419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1b3400640 1 -- 172.21.15.46:0/3600777693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a3b90 con 0x7fd1b4076720 2024-09-10T07:36:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1abe00640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 0x7fd1b419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.519+0000 7fd1b3400640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1b4076720 0x7fd1b419eb60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4005fd0 tx=0x7fd1a400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.520+0000 7fd1b0c00640 1 -- 172.21.15.46:0/3600777693 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400cd00 con 0x7fd1b4076720 2024-09-10T07:36:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.520+0000 7fd1b0c00640 1 -- 172.21.15.46:0/3600777693 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a40035f0 con 0x7fd1b4076720 2024-09-10T07:36:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.520+0000 7fd1b0c00640 1 -- 172.21.15.46:0/3600777693 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4010670 con 0x7fd1b4076720 2024-09-10T07:36:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.520+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a3e80 con 0x7fd1b4076720 2024-09-10T07:36:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.520+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41ab710 con 0x7fd1b4076720 2024-09-10T07:36:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.521+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b4077e60 con 0x7fd1b4076720 2024-09-10T07:36:37.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.523+0000 7fd1b0c00640 1 -- 172.21.15.46:0/3600777693 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1a40037a0 con 0x7fd1b4076720 2024-09-10T07:36:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.524+0000 7fd1b0c00640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd1980a2b80 0x7fd1980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.524+0000 7fd1b0c00640 1 -- 172.21.15.46:0/3600777693 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd1a40cf5f0 con 0x7fd1b4076720 2024-09-10T07:36:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.524+0000 7fd1abe00640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd1980a2b80 0x7fd1980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.525+0000 7fd1abe00640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd1980a2b80 0x7fd1980a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0057c0 tx=0x7fd19c0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.528+0000 7fd1b0c00640 1 -- 172.21.15.46:0/3600777693 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1a4009510 con 0x7fd1b4076720 2024-09-10T07:36:37.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:37 smithi181 ceph-mon[35397]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:37.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:37 smithi181 ceph-mon[35397]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:37.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:37 smithi063 ceph-mon[35020]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:37.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:37 smithi063 ceph-mon[35020]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:37.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.748+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1b4076b00 con 0x7fd1b4076720 2024-09-10T07:36:37.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.749+0000 7fd1b0c00640 1 -- 172.21.15.46:0/3600777693 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7fd1a409c170 con 0x7fd1b4076720 2024-09-10T07:36:37.749 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:37.749 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"btime":"2024-09-10T07:36:31:116553+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:36:30.952730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:36:37.749 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-09-10T07:36:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.751+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd1980a2b80 msgr2=0x7fd1980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.751+0000 7fd1ba600640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd1980a2b80 0x7fd1980a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0057c0 tx=0x7fd19c0056d0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.751+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1b4076720 msgr2=0x7fd1b419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.751+0000 7fd1ba600640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1b4076720 0x7fd1b419eb60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4005fd0 tx=0x7fd1a400ee50 comp rx=0 tx=0).stop 2024-09-10T07:36:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.752+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 shutdown_connections 2024-09-10T07:36:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.752+0000 7fd1ba600640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4112a50 0x7fd1b41a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.752+0000 7fd1ba600640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd1980a2b80 0x7fd1980a5040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.752+0000 7fd1ba600640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1b4077040 0x7fd1b419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.752+0000 7fd1ba600640 1 --2- 172.21.15.46:0/3600777693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1b4076720 0x7fd1b419eb60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.752+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 >> 172.21.15.46:0/3600777693 conn(0x7fd1b40ffa50 msgr2=0x7fd1b4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.752+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 shutdown_connections 2024-09-10T07:36:37.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:37.752+0000 7fd1ba600640 1 -- 172.21.15.46:0/3600777693 wait complete. 2024-09-10T07:36:38.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:38 smithi046 ceph-mon[30705]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:38 smithi046 ceph-mon[30705]: pgmap v399: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:38.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:38 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3600777693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:38.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:36:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:38 smithi181 ceph-mon[35397]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:38 smithi181 ceph-mon[35397]: pgmap v399: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:38 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3600777693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:38 smithi063 ceph-mon[35020]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:38 smithi063 ceph-mon[35020]: pgmap v399: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:38 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3600777693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:39.011 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:39.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 -- 172.21.15.46:0/341636117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac106f10 msgr2=0x7faeac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:39.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 --2- 172.21.15.46:0/341636117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac106f10 0x7faeac10ea90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faea4009ca0 tx=0x7faea402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 -- 172.21.15.46:0/341636117 shutdown_connections 2024-09-10T07:36:39.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 --2- 172.21.15.46:0/341636117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac106f10 0x7faeac10ea90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 --2- 172.21.15.46:0/341636117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeac102790 0x7faeac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 --2- 172.21.15.46:0/341636117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faeac101de0 0x7faeac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 -- 172.21.15.46:0/341636117 >> 172.21.15.46:0/341636117 conn(0x7faeac0fd8c0 msgr2=0x7faeac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:39.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 -- 172.21.15.46:0/341636117 shutdown_connections 2024-09-10T07:36:39.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.615+0000 7faeb3e00640 1 -- 172.21.15.46:0/341636117 wait complete. 2024-09-10T07:36:39.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.616+0000 7faeb3e00640 1 Processor -- start 2024-09-10T07:36:39.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb3e00640 1 -- start start 2024-09-10T07:36:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac101de0 0x7faeac19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeac102790 0x7faeac19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faeac106f10 0x7faeac19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faeac111f70 con 0x7faeac101de0 2024-09-10T07:36:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faeac111df0 con 0x7faeac106f10 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faeac1120f0 con 0x7faeac102790 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac101de0 0x7faeac19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac101de0 0x7faeac19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37244/0 (socket says 172.21.15.46:37244) 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb1400640 1 -- 172.21.15.46:0/1989940488 learned_addr learned my addr 172.21.15.46:0/1989940488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb0a00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeac102790 0x7faeac19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.617+0000 7faeb1e00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faeac106f10 0x7faeac19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeb1400640 1 -- 172.21.15.46:0/1989940488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeac102790 msgr2=0x7faeac19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeb1400640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeac102790 0x7faeac19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeb1400640 1 -- 172.21.15.46:0/1989940488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faeac106f10 msgr2=0x7faeac19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeb1400640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faeac106f10 0x7faeac19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeb1400640 1 -- 172.21.15.46:0/1989940488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac19f590 con 0x7faeac101de0 2024-09-10T07:36:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeb1400640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac101de0 0x7faeac19a5e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faea000ea60 tx=0x7faea000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:39.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeaa000640 1 -- 172.21.15.46:0/1989940488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea000cde0 con 0x7faeac101de0 2024-09-10T07:36:39.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeaa000640 1 -- 172.21.15.46:0/1989940488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea00035f0 con 0x7faeac101de0 2024-09-10T07:36:39.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.618+0000 7faeaa000640 1 -- 172.21.15.46:0/1989940488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0010670 con 0x7faeac101de0 2024-09-10T07:36:39.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.619+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac19f880 con 0x7faeac101de0 2024-09-10T07:36:39.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.619+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac1a7160 con 0x7faeac101de0 2024-09-10T07:36:39.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.620+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac1021c0 con 0x7faeac101de0 2024-09-10T07:36:39.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.622+0000 7faeaa000640 1 -- 172.21.15.46:0/1989940488 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faea00037a0 con 0x7faeac101de0 2024-09-10T07:36:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.623+0000 7faeaa000640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae940a2c00 0x7fae940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.623+0000 7faeaa000640 1 -- 172.21.15.46:0/1989940488 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faea00cf690 con 0x7faeac101de0 2024-09-10T07:36:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.623+0000 7faeb0a00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae940a2c00 0x7fae940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.624+0000 7faeb0a00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae940a2c00 0x7fae940a50c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faeac0ff6f0 tx=0x7fae98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.627+0000 7faeaa000640 1 -- 172.21.15.46:0/1989940488 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faea009c210 con 0x7faeac101de0 2024-09-10T07:36:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.816+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faeac19b940 con 0x7faeac101de0 2024-09-10T07:36:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.818+0000 7faeaa000640 1 -- 172.21.15.46:0/1989940488 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24700 (secure 0 0 0) 0x7faea009bb10 con 0x7faeac101de0 2024-09-10T07:36:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:39.818 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:35:14.826944+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae940a2c00 msgr2=0x7fae940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae940a2c00 0x7fae940a50c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faeac0ff6f0 tx=0x7fae98005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac101de0 msgr2=0x7faeac19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac101de0 0x7faeac19a5e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faea000ea60 tx=0x7faea000ef20 comp rx=0 tx=0).stop 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 shutdown_connections 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faeac106f10 0x7faeac19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeac102790 0x7faeac19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae940a2c00 0x7fae940a50c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 --2- 172.21.15.46:0/1989940488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeac101de0 0x7faeac19a5e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 >> 172.21.15.46:0/1989940488 conn(0x7faeac0fd8c0 msgr2=0x7faeac0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 shutdown_connections 2024-09-10T07:36:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:39.820+0000 7faeb3e00640 1 -- 172.21.15.46:0/1989940488 wait complete. 2024-09-10T07:36:40.233 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:40 smithi046 ceph-mon[30705]: pgmap v400: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T07:36:40.233 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:40 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1989940488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:40.233 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:36:40.233 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.233 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.233 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.233 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:40 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:40 smithi181 ceph-mon[35397]: pgmap v400: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T07:36:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:40 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1989940488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:36:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:40 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:40 smithi063 ceph-mon[35020]: pgmap v400: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T07:36:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:40 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1989940488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:36:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:40 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:40.786 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T07:36:40.786 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-10T07:36:40.800 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:36:40.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-10T07:36:40.800 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T07:36:40.800 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:36:40.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:36:40.804 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T07:36:40.818 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T07:36:40.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T07:36:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:41.252 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:41.621 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:41.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:41.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:36:41.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:36:41.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:41.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:41.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:41.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:36:41.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:36:41.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:41.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:41.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:41.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:36:41.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:36:41.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:36:42.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 -- 172.21.15.46:0/1291549618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 msgr2=0x7efc50110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 --2- 172.21.15.46:0/1291549618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 0x7efc50110810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efc48009ca0 tx=0x7efc4802a940 comp rx=0 tx=0).stop 2024-09-10T07:36:42.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 -- 172.21.15.46:0/1291549618 shutdown_connections 2024-09-10T07:36:42.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 --2- 172.21.15.46:0/1291549618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 0x7efc50110810 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 --2- 172.21.15.46:0/1291549618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc50074f40 0x7efc5010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 --2- 172.21.15.46:0/1291549618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50074620 0x7efc50074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 -- 172.21.15.46:0/1291549618 >> 172.21.15.46:0/1291549618 conn(0x7efc500fd8c0 msgr2=0x7efc500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:42.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 -- 172.21.15.46:0/1291549618 shutdown_connections 2024-09-10T07:36:42.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.012+0000 7efc58400640 1 -- 172.21.15.46:0/1291549618 wait complete. 2024-09-10T07:36:42.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.013+0000 7efc58400640 1 Processor -- start 2024-09-10T07:36:42.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc58400640 1 -- start start 2024-09-10T07:36:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc58400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50074620 0x7efc5019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc58400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc50074f40 0x7efc5019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc58400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 0x7efc5019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc58400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc50114150 con 0x7efc5010e3c0 2024-09-10T07:36:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc58400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc50113fd0 con 0x7efc50074f40 2024-09-10T07:36:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc58400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc501142d0 con 0x7efc50074620 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc56400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 0x7efc5019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc55a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50074620 0x7efc5019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc55a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50074620 0x7efc5019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39942/0 (socket says 172.21.15.46:39942) 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc56400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 0x7efc5019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37262/0 (socket says 172.21.15.46:37262) 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc55a00640 1 -- 172.21.15.46:0/895270694 learned_addr learned my addr 172.21.15.46:0/895270694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.014+0000 7efc55000640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc50074f40 0x7efc5019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.015+0000 7efc56400640 1 -- 172.21.15.46:0/895270694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50074620 msgr2=0x7efc5019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.015+0000 7efc56400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50074620 0x7efc5019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.015+0000 7efc56400640 1 -- 172.21.15.46:0/895270694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc50074f40 msgr2=0x7efc5019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.015+0000 7efc56400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc50074f40 0x7efc5019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.015+0000 7efc56400640 1 -- 172.21.15.46:0/895270694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5019f5e0 con 0x7efc5010e3c0 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.015+0000 7efc55000640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc50074f40 0x7efc5019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.015+0000 7efc56400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 0x7efc5019ef00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc48006010 tx=0x7efc4802f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.016+0000 7efc4e000640 1 -- 172.21.15.46:0/895270694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48034040 con 0x7efc5010e3c0 2024-09-10T07:36:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.016+0000 7efc58400640 1 -- 172.21.15.46:0/895270694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5019f870 con 0x7efc5010e3c0 2024-09-10T07:36:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.016+0000 7efc4e000640 1 -- 172.21.15.46:0/895270694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc48003e50 con 0x7efc5010e3c0 2024-09-10T07:36:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.016+0000 7efc4e000640 1 -- 172.21.15.46:0/895270694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480046a0 con 0x7efc5010e3c0 2024-09-10T07:36:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.016+0000 7efc58400640 1 -- 172.21.15.46:0/895270694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc501a71b0 con 0x7efc5010e3c0 2024-09-10T07:36:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc4e000640 1 -- 172.21.15.46:0/895270694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc48037020 con 0x7efc5010e3c0 2024-09-10T07:36:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc5ae7e740 1 Processor -- start 2024-09-10T07:36:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc5ae7e740 1 -- start start 2024-09-10T07:36:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x560565521ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605655223e0 0x56056552c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x56056552f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560565488290 con 0x560565521ac0 2024-09-10T07:36:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56056542fb40 con 0x56056552cca0 2024-09-10T07:36:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.019+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560565484250 con 0x5605655223e0 2024-09-10T07:36:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x560565521ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x560565521ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37272/0 (socket says 172.21.15.46:37272) 2024-09-10T07:36:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 -- 172.21.15.46:0/1773354091 learned_addr learned my addr 172.21.15.46:0/1773354091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55a00640 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605655223e0 0x56056552c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x56056552f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 -- 172.21.15.46:0/1773354091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605655223e0 msgr2=0x56056552c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605655223e0 0x56056552c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 -- 172.21.15.46:0/1773354091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 msgr2=0x56056552f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x56056552f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc55000640 1 -- 172.21.15.46:0/1773354091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605651f96a0 con 0x560565521ac0 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.020+0000 7efc4e000640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc380a2c00 0x7efc380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.021+0000 7efc55000640 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x560565521ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efc3c013970 tx=0x7efc3c009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cea2d5690e6ba97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.021+0000 7efc4e000640 1 -- 172.21.15.46:0/895270694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efc480f02d0 con 0x7efc5010e3c0 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.021+0000 7efc55a00640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc380a2c00 0x7efc380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.021+0000 7efc42a00640 1 -- 172.21.15.46:0/1773354091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c004030 con 0x560565521ac0 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.021+0000 7efc42a00640 1 -- 172.21.15.46:0/1773354091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c00fda0 con 0x560565521ac0 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.021+0000 7efc42a00640 1 -- 172.21.15.46:0/1773354091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003ab0 con 0x560565521ac0 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1773354091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 msgr2=0x560565521ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x560565521ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efc3c013970 tx=0x7efc3c009d80 comp rx=0 tx=0).stop 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc55a00640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc380a2c00 0x7efc380a50c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x56056551eeb0 tx=0x7efc4400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1773354091 shutdown_connections 2024-09-10T07:36:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x56056552f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605655223e0 0x56056552c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1773354091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x560565521ea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1773354091 >> 172.21.15.46:0/1773354091 conn(0x56056551e050 msgr2=0x56056551e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1773354091 shutdown_connections 2024-09-10T07:36:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.022+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1773354091 wait complete. 2024-09-10T07:36:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.023+0000 7efc5ae7e740 1 -- start start 2024-09-10T07:36:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.023+0000 7efc5ae7e740 1 Processor -- start 2024-09-10T07:36:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.023+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x56056552c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 0x560565529900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x560565529ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56056552f910 con 0x5605655223e0 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560565442fa0 con 0x56056552cca0 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56056552fa90 con 0x560565521ac0 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc5ae7e740 10 client.0 _ll_register_callbacks cb 0x560565196ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x56056552c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x56056552c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39954/0 (socket says 172.21.15.46:39954) 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55a00640 1 -- 172.21.15.46:0/1195927673 learned_addr learned my addr 172.21.15.46:0/1195927673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55000640 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 0x560565529900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc56400640 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x560565529ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55a00640 1 -- 172.21.15.46:0/1195927673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 msgr2=0x560565529ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55a00640 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x560565529ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55a00640 1 -- 172.21.15.46:0/1195927673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 msgr2=0x560565529900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55a00640 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 0x560565529900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.024+0000 7efc55a00640 1 -- 172.21.15.46:0/1195927673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56056552a6d0 con 0x560565521ac0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.025+0000 7efc55a00640 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x56056552c650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc44003df0 tx=0x7efc44004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def066e1762696da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.025+0000 7efc37400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.025+0000 7efc37400640 1 -- 172.21.15.46:0/1195927673 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44005020 con 0x560565521ac0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.025+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1195927673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc3c00fda0 con 0x560565521ac0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.025+0000 7efc37400640 1 -- 172.21.15.46:0/1195927673 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efc44017030 con 0x560565521ac0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.025+0000 7efc37400640 1 -- 172.21.15.46:0/1195927673 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc44032050 con 0x560565521ac0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.025+0000 7efc5ae7e740 10 client.25297 fetch_fsmap learned FSMap version 30 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.025+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1195927673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56056553b940 con 0x560565521ac0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc37400640 1 -- 172.21.15.46:0/1195927673 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c00fda0 con 0x560565521ac0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc37400640 1 -- 172.21.15.46:0/1195927673 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc44027410 con 0x560565521ac0 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc5ae7e740 10 client.25297 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:36:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc5ae7e740 10 client.25297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:36:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc5ae7e740 20 client.25297 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56056553cd50 0x56056553f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc5ae7e740 4 client.25297 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:36:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1195927673 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560565495800 con 0x56056553cd50 2024-09-10T07:36:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.026+0000 7efc55000640 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56056553cd50 0x56056553f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.027+0000 7efc55000640 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56056553cd50 0x56056553f1b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4888d80e7e1f30b1 server_cookie=1a2fc2fd8296724a in_seq=0 out_seq=0 2024-09-10T07:36:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.027+0000 7efc37400640 10 client.25297 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.027+0000 7efc37400640 1 -- 172.21.15.46:0/1195927673 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560565495800 con 0x56056553cd50 2024-09-10T07:36:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.027+0000 7efc37400640 10 client.25297 handle_command_reply: tid=0 2024-09-10T07:36:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 client.25297 shutdown 2024-09-10T07:36:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1195927673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 msgr2=0x56056552c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x56056552c650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc44003df0 tx=0x7efc44004730 comp rx=0 tx=0).stop 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1195927673 shutdown_connections 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56056553cd50 0x56056553f1b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x560565529ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 0x560565529900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/1195927673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x56056552c650 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1195927673 >> 172.21.15.46:0/1195927673 conn(0x56056551e050 msgr2=0x560565523d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc37400640 0 client.25297 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1195927673 shutdown_connections 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 1 -- 172.21.15.46:0/1195927673 wait complete. 2024-09-10T07:36:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.028+0000 7efc5ae7e740 20 client.25297 trim_cache size 0 max 16384 2024-09-10T07:36:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc5ae7e740 1 Processor -- start 2024-09-10T07:36:42.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc5ae7e740 1 -- start start 2024-09-10T07:36:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x56056552b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605655223e0 0x560565544ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56056552cca0 0x560565545010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc501b0eb0 con 0x560565521ac0 2024-09-10T07:36:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc501b29f0 con 0x5605655223e0 2024-09-10T07:36:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc501b1580 con 0x56056552cca0 2024-09-10T07:36:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc55a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x56056552b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc55a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x56056552b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37288/0 (socket says 172.21.15.46:37288) 2024-09-10T07:36:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc55a00640 1 -- 172.21.15.46:0/2984753344 learned_addr learned my addr 172.21.15.46:0/2984753344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.047+0000 7efc55000640 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605655223e0 0x560565544ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc56400640 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56056552cca0 0x560565545010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc55a00640 1 -- 172.21.15.46:0/2984753344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56056552cca0 msgr2=0x560565545010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc55a00640 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56056552cca0 0x560565545010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc55a00640 1 -- 172.21.15.46:0/2984753344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605655223e0 msgr2=0x560565544ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc55a00640 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605655223e0 0x560565544ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc55a00640 1 -- 172.21.15.46:0/2984753344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056551c760 con 0x560565521ac0 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc56400640 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56056552cca0 0x560565545010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc55a00640 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x56056552b870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efc44004150 tx=0x7efc4400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a833e2012f7165 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc42000640 1 -- 172.21.15.46:0/2984753344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc440275e0 con 0x560565521ac0 2024-09-10T07:36:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc42000640 1 -- 172.21.15.46:0/2984753344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc44003c00 con 0x560565521ac0 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.048+0000 7efc42000640 1 -- 172.21.15.46:0/2984753344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc440049c0 con 0x560565521ac0 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 -- 172.21.15.46:0/2984753344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 msgr2=0x56056552b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x56056552b870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efc44004150 tx=0x7efc4400b260 comp rx=0 tx=0).stop 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 -- 172.21.15.46:0/2984753344 shutdown_connections 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56056552cca0 0x560565545010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605655223e0 0x560565544ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/2984753344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560565521ac0 0x56056552b870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 -- 172.21.15.46:0/2984753344 >> 172.21.15.46:0/2984753344 conn(0x56056551e050 msgr2=0x56056552b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 -- 172.21.15.46:0/2984753344 shutdown_connections 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.049+0000 7efc5ae7e740 1 -- 172.21.15.46:0/2984753344 wait complete. 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.050+0000 7efc5ae7e740 1 -- start start 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.050+0000 7efc5ae7e740 1 Processor -- start 2024-09-10T07:36:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.050+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x5605655330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.050+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 0x5605655335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.050+0000 7efc5ae7e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x5605655a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.050+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc501b2320 con 0x5605655223e0 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.050+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc501b1c50 con 0x56056552cca0 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.050+0000 7efc5ae7e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc501aee90 con 0x560565521ac0 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc5ae7e740 10 client.0 _ll_register_callbacks cb 0x7efc440042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x5605655a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x5605655a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52380/0 (socket says 172.21.15.46:52380) 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 -- 172.21.15.46:0/3553252650 learned_addr learned my addr 172.21.15.46:0/3553252650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc55000640 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 0x5605655335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc55a00640 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x5605655330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 -- 172.21.15.46:0/3553252650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 msgr2=0x5605655330a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x5605655330a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 -- 172.21.15.46:0/3553252650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 msgr2=0x5605655335e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 0x5605655335e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 -- 172.21.15.46:0/3553252650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605655a6ff0 con 0x56056552cca0 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc56400640 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x5605655a6ab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc48005cc0 tx=0x7efc480b8b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78fdb6374d96d6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.051+0000 7efc37400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc37400640 1 -- 172.21.15.46:0/3553252650 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480f00c0 con 0x56056552cca0 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc37400640 1 -- 172.21.15.46:0/3553252650 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efc480a7070 con 0x56056552cca0 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 1 -- 172.21.15.46:0/3553252650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc44003c00 con 0x56056552cca0 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc37400640 1 -- 172.21.15.46:0/3553252650 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc48004840 con 0x56056552cca0 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 10 client.25298 fetch_fsmap learned FSMap version 30 2024-09-10T07:36:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 1 -- 172.21.15.46:0/3553252650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5605655a8290 con 0x56056552cca0 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc37400640 1 -- 172.21.15.46:0/3553252650 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc44003c00 con 0x56056552cca0 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc37400640 1 -- 172.21.15.46:0/3553252650 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc480a5ce0 con 0x56056552cca0 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 10 client.25298 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 10 client.25298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 20 client.25298 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5605655a8960 0x5605655aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 4 client.25298 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.052+0000 7efc5ae7e740 1 -- 172.21.15.46:0/3553252650 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56056548fd10 con 0x5605655a8960 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.053+0000 7efc55a00640 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5605655a8960 0x5605655aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.053+0000 7efc55a00640 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5605655a8960 0x5605655aad50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833523ff9eca787c server_cookie=80da572df9bbcb5d in_seq=0 out_seq=0 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.053+0000 7efc37400640 10 client.25298 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc37400640 1 -- 172.21.15.46:0/3553252650 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56056548fd10 con 0x5605655a8960 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc37400640 10 client.25298 handle_command_reply: tid=0 2024-09-10T07:36:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 client.25298 shutdown 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 -- 172.21.15.46:0/3553252650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 msgr2=0x5605655a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x5605655a6ab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc48005cc0 tx=0x7efc480b8b20 comp rx=0 tx=0).stop 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 -- 172.21.15.46:0/3553252650 shutdown_connections 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5605655a8960 0x5605655aad50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56056552cca0 0x5605655a6ab0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605655223e0 0x5605655335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 --2- 172.21.15.46:0/3553252650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560565521ac0 0x5605655330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc37400640 0 client.25298 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 -- 172.21.15.46:0/3553252650 >> 172.21.15.46:0/3553252650 conn(0x56056551e050 msgr2=0x56056551d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 -- 172.21.15.46:0/3553252650 shutdown_connections 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.055+0000 7efc5ae7e740 1 -- 172.21.15.46:0/3553252650 wait complete. 2024-09-10T07:36:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.056+0000 7efc5ae7e740 20 client.25298 trim_cache size 0 max 16384 2024-09-10T07:36:42.087 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:36:42.087 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:36:42.087 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "0860b53d-b75f-43fc-8d65-bc51a5371346", 2024-09-10T07:36:42.087 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:36:42.087 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.056+0000 7efc58400640 1 -- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc380a2c00 msgr2=0x7efc380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.056+0000 7efc58400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc380a2c00 0x7efc380a50c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x56056551eeb0 tx=0x7efc4400b480 comp rx=0 tx=0).stop 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.056+0000 7efc58400640 1 -- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 msgr2=0x7efc5019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.056+0000 7efc58400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 0x7efc5019ef00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc48006010 tx=0x7efc4802f640 comp rx=0 tx=0).stop 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.057+0000 7efc58400640 1 -- 172.21.15.46:0/895270694 shutdown_connections 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.057+0000 7efc58400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc380a2c00 0x7efc380a50c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.057+0000 7efc58400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc5010e3c0 0x7efc5019ef00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.057+0000 7efc58400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc50074f40 0x7efc5019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.057+0000 7efc58400640 1 --2- 172.21.15.46:0/895270694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50074620 0x7efc5019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.057+0000 7efc58400640 1 -- 172.21.15.46:0/895270694 >> 172.21.15.46:0/895270694 conn(0x7efc500fd8c0 msgr2=0x7efc500fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:42.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.057+0000 7efc58400640 1 -- 172.21.15.46:0/895270694 shutdown_connections 2024-09-10T07:36:42.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.057+0000 7efc58400640 1 -- 172.21.15.46:0/895270694 wait complete. 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 -- 172.21.15.46:0/2497561396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf64074510 msgr2=0x7faf640748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 --2- 172.21.15.46:0/2497561396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf64074510 0x7faf640748f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faf54009be0 tx=0x7faf5402a750 comp rx=0 tx=0).stop 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 -- 172.21.15.46:0/2497561396 shutdown_connections 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 --2- 172.21.15.46:0/2497561396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf6410e280 0x7faf64110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 --2- 172.21.15.46:0/2497561396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf64074e30 0x7faf6410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 --2- 172.21.15.46:0/2497561396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf64074510 0x7faf640748f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 -- 172.21.15.46:0/2497561396 >> 172.21.15.46:0/2497561396 conn(0x7faf640fd740 msgr2=0x7faf640ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 -- 172.21.15.46:0/2497561396 shutdown_connections 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.251+0000 7faf63e00640 1 -- 172.21.15.46:0/2497561396 wait complete. 2024-09-10T07:36:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.252+0000 7faf63e00640 1 Processor -- start 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.252+0000 7faf63e00640 1 -- start start 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf64074510 0x7faf6419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf63e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf64074e30 0x7faf6419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf63e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6410e280 0x7faf6419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf63e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf64113fb0 con 0x7faf64074510 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf63e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf64113e30 con 0x7faf6410e280 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf63e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf64114130 con 0x7faf64074e30 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf62a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf64074510 0x7faf6419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf62a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf64074510 0x7faf6419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37320/0 (socket says 172.21.15.46:37320) 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf62a00640 1 -- 172.21.15.46:0/2131113683 learned_addr learned my addr 172.21.15.46:0/2131113683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf5a000640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf64074e30 0x7faf6419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf63400640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6410e280 0x7faf6419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf62a00640 1 -- 172.21.15.46:0/2131113683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf64074e30 msgr2=0x7faf6419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.253+0000 7faf62a00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf64074e30 0x7faf6419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf62a00640 1 -- 172.21.15.46:0/2131113683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6410e280 msgr2=0x7faf6419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf62a00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6410e280 0x7faf6419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf62a00640 1 -- 172.21.15.46:0/2131113683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6419f500 con 0x7faf64074510 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf62a00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf64074510 0x7faf6419a550 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf54034040 tx=0x7faf540305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf5be00640 1 -- 172.21.15.46:0/2131113683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54041020 con 0x7faf64074510 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6419f790 con 0x7faf64074510 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf5be00640 1 -- 172.21.15.46:0/2131113683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf54003e50 con 0x7faf64074510 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a70d0 con 0x7faf64074510 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.254+0000 7faf5be00640 1 -- 172.21.15.46:0/2131113683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54036040 con 0x7faf64074510 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.255+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf640748f0 con 0x7faf64074510 2024-09-10T07:36:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.257+0000 7faf5be00640 1 -- 172.21.15.46:0/2131113683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf54044020 con 0x7faf64074510 2024-09-10T07:36:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.260+0000 7faf5be00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf480a2b80 0x7faf480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.260+0000 7faf5be00640 1 -- 172.21.15.46:0/2131113683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faf540efea0 con 0x7faf64074510 2024-09-10T07:36:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.260+0000 7faf5a000640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf480a2b80 0x7faf480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.260+0000 7faf5a000640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf480a2b80 0x7faf480a5040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf640ffd60 tx=0x7faf4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.263+0000 7faf5be00640 1 -- 172.21.15.46:0/2131113683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf5404b020 con 0x7faf64074510 2024-09-10T07:36:42.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:42 smithi046 ceph-mon[30705]: pgmap v401: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T07:36:42.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.450+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf64062630 con 0x7faf480a2b80 2024-09-10T07:36:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.459+0000 7faf5be00640 1 -- 172.21.15.46:0/2131113683 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340778 (secure 0 0 0) 0x7faf64062630 con 0x7faf480a2b80 2024-09-10T07:36:42.463 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:36:42.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.463+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf480a2b80 msgr2=0x7faf480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.463+0000 7faf63e00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf480a2b80 0x7faf480a5040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf640ffd60 tx=0x7faf4c005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf64074510 msgr2=0x7faf6419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf64074510 0x7faf6419a550 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf54034040 tx=0x7faf540305d0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 shutdown_connections 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf6410e280 0x7faf6419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf64074e30 0x7faf6419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faf480a2b80 0x7faf480a5040 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 --2- 172.21.15.46:0/2131113683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf64074510 0x7faf6419a550 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 >> 172.21.15.46:0/2131113683 conn(0x7faf640fd740 msgr2=0x7faf640feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 shutdown_connections 2024-09-10T07:36:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:42.464+0000 7faf63e00640 1 -- 172.21.15.46:0/2131113683 wait complete. 2024-09-10T07:36:42.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:42 smithi181 ceph-mon[35397]: pgmap v401: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T07:36:42.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:42 smithi063 ceph-mon[35020]: pgmap v401: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T07:36:43.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:43 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:36:43.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:43 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:36:43.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:43 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:36:43.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:43 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:36:43.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:43 smithi046 ceph-mon[30705]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:43.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:36:43.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:43 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:36:43.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:43 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:36:43.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:43 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:36:43.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:43 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:36:43.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:43 smithi181 ceph-mon[35397]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:43.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:43 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:36:43.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:43 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:36:43.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:43 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:36:43.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:43 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:36:43.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:43 smithi063 ceph-mon[35020]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:44.138 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-10T07:36:40.509161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668647,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1495,"num_read_kb":878,"num_write":448,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358332,"kb_used_data":13244,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583492,"statfs":{"total":1151940427776,"available":1151573495808,"internally_reserved":0,"allocated":13561856,"data_stored":8706935,"data_compressed":498372,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":71,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002818"},"pg_stats":[{"pgid":"2.1f","version":"97'14","reported_seq":191,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:37.015484+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:36:37.015484+0000","last_peered":"2024-09-10T07:36:37.015484+0000","last_clean":"2024-09-10T07:36:37.015484+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:36:37.015484+0000","last_undegraded":"2024-09-10T07:36:37.015484+0000","last_fullsized":"2024-09-10T07:36:37.015484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475738+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:35:42.475738+0000","last_peered":"2024-09-10T07:35:42.475738+0000","last_clean":"2024-09-10T07:35:42.475738+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:35:42.475738+0000","last_undegraded":"2024-09-10T07:35:42.475738+0000","last_fullsized":"2024-09-10T07:35:42.475738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842368+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:35:14.842368+0000","last_peered":"2024-09-10T07:35:14.842368+0000","last_clean":"2024-09-10T07:35:14.842368+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:35:14.842368+0000","last_undegraded":"2024-09-10T07:35:14.842368+0000","last_fullsized":"2024-09-10T07:35:14.842368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977747+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:35:14.977747+0000","last_peered":"2024-09-10T07:35:14.977747+0000","last_clean":"2024-09-10T07:35:14.977747+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:35:14.977747+0000","last_undegraded":"2024-09-10T07:35:14.977747+0000","last_fullsized":"2024-09-10T07:35:14.977747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502886+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:35:42.502886+0000","last_peered":"2024-09-10T07:35:42.502886+0000","last_clean":"2024-09-10T07:35:42.502886+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:35:42.502886+0000","last_undegraded":"2024-09-10T07:35:42.502886+0000","last_fullsized":"2024-09-10T07:35:42.502886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881794+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:35:14.881794+0000","last_peered":"2024-09-10T07:35:14.881794+0000","last_clean":"2024-09-10T07:35:14.881794+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:35:14.881794+0000","last_undegraded":"2024-09-10T07:35:14.881794+0000","last_fullsized":"2024-09-10T07:35:14.881794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854574+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:35:14.854574+0000","last_peered":"2024-09-10T07:35:14.854574+0000","last_clean":"2024-09-10T07:35:14.854574+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:35:14.854574+0000","last_undegraded":"2024-09-10T07:35:14.854574+0000","last_fullsized":"2024-09-10T07:35:14.854574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.495636+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:35:42.495636+0000","last_peered":"2024-09-10T07:35:42.495636+0000","last_clean":"2024-09-10T07:35:42.495636+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:35:42.495636+0000","last_undegraded":"2024-09-10T07:35:42.495636+0000","last_fullsized":"2024-09-10T07:35:42.495636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881984+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:35:14.881984+0000","last_peered":"2024-09-10T07:35:14.881984+0000","last_clean":"2024-09-10T07:35:14.881984+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:35:14.881984+0000","last_undegraded":"2024-09-10T07:35:14.881984+0000","last_fullsized":"2024-09-10T07:35:14.881984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854411+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:35:14.854411+0000","last_peered":"2024-09-10T07:35:14.854411+0000","last_clean":"2024-09-10T07:35:14.854411+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:35:14.854411+0000","last_undegraded":"2024-09-10T07:35:14.854411+0000","last_fullsized":"2024-09-10T07:35:14.854411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"97'15","reported_seq":60,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484819+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:35:42.484819+0000","last_peered":"2024-09-10T07:35:42.484819+0000","last_clean":"2024-09-10T07:35:42.484819+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:35:42.484819+0000","last_undegraded":"2024-09-10T07:35:42.484819+0000","last_fullsized":"2024-09-10T07:35:42.484819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881889+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:35:14.881889+0000","last_peered":"2024-09-10T07:35:14.881889+0000","last_clean":"2024-09-10T07:35:14.881889+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:35:14.881889+0000","last_undegraded":"2024-09-10T07:35:14.881889+0000","last_fullsized":"2024-09-10T07:35:14.881889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486825+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:35:42.486825+0000","last_peered":"2024-09-10T07:35:42.486825+0000","last_clean":"2024-09-10T07:35:42.486825+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:35:42.486825+0000","last_undegraded":"2024-09-10T07:35:42.486825+0000","last_fullsized":"2024-09-10T07:35:42.486825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854068+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:35:14.854068+0000","last_peered":"2024-09-10T07:35:14.854068+0000","last_clean":"2024-09-10T07:35:14.854068+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:35:14.854068+0000","last_undegraded":"2024-09-10T07:35:14.854068+0000","last_fullsized":"2024-09-10T07:35:14.854068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882150+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:35:14.882150+0000","last_peered":"2024-09-10T07:35:14.882150+0000","last_clean":"2024-09-10T07:35:14.882150+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:35:14.882150+0000","last_undegraded":"2024-09-10T07:35:14.882150+0000","last_fullsized":"2024-09-10T07:35:14.882150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501326+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:35:42.501326+0000","last_peered":"2024-09-10T07:35:42.501326+0000","last_clean":"2024-09-10T07:35:42.501326+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:35:42.501326+0000","last_undegraded":"2024-09-10T07:35:42.501326+0000","last_fullsized":"2024-09-10T07:35:42.501326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842483+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:35:14.842483+0000","last_peered":"2024-09-10T07:35:14.842483+0000","last_clean":"2024-09-10T07:35:14.842483+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:35:14.842483+0000","last_undegraded":"2024-09-10T07:35:14.842483+0000","last_fullsized":"2024-09-10T07:35:14.842483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882278+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:35:14.882278+0000","last_peered":"2024-09-10T07:35:14.882278+0000","last_clean":"2024-09-10T07:35:14.882278+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:35:14.882278+0000","last_undegraded":"2024-09-10T07:35:14.882278+0000","last_fullsized":"2024-09-10T07:35:14.882278+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502574+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:35:42.502574+0000","last_peered":"2024-09-10T07:35:42.502574+0000","last_clean":"2024-09-10T07:35:42.502574+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:35:42.502574+0000","last_undegraded":"2024-09-10T07:35:42.502574+0000","last_fullsized":"2024-09-10T07:35:42.502574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977257+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:35:14.977257+0000","last_peered":"2024-09-10T07:35:14.977257+0000","last_clean":"2024-09-10T07:35:14.977257+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:35:14.977257+0000","last_undegraded":"2024-09-10T07:35:14.977257+0000","last_fullsized":"2024-09-10T07:35:14.977257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851715+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:35:14.851715+0000","last_peered":"2024-09-10T07:35:14.851715+0000","last_clean":"2024-09-10T07:35:14.851715+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:35:14.851715+0000","last_undegraded":"2024-09-10T07:35:14.851715+0000","last_fullsized":"2024-09-10T07:35:14.851715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"97'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.459952+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:35:42.459952+0000","last_peered":"2024-09-10T07:35:42.459952+0000","last_clean":"2024-09-10T07:35:42.459952+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:35:42.459952+0000","last_undegraded":"2024-09-10T07:35:42.459952+0000","last_fullsized":"2024-09-10T07:35:42.459952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976752+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:35:14.976752+0000","last_peered":"2024-09-10T07:35:14.976752+0000","last_clean":"2024-09-10T07:35:14.976752+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:35:14.976752+0000","last_undegraded":"2024-09-10T07:35:14.976752+0000","last_fullsized":"2024-09-10T07:35:14.976752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882210+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:35:14.882210+0000","last_peered":"2024-09-10T07:35:14.882210+0000","last_clean":"2024-09-10T07:35:14.882210+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:35:14.882210+0000","last_undegraded":"2024-09-10T07:35:14.882210+0000","last_fullsized":"2024-09-10T07:35:14.882210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502703+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:35:42.502703+0000","last_peered":"2024-09-10T07:35:42.502703+0000","last_clean":"2024-09-10T07:35:42.502703+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:35:42.502703+0000","last_undegraded":"2024-09-10T07:35:42.502703+0000","last_fullsized":"2024-09-10T07:35:42.502703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882535+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:35:14.882535+0000","last_peered":"2024-09-10T07:35:14.882535+0000","last_clean":"2024-09-10T07:35:14.882535+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:35:14.882535+0000","last_undegraded":"2024-09-10T07:35:14.882535+0000","last_fullsized":"2024-09-10T07:35:14.882535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842382+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:35:14.842382+0000","last_peered":"2024-09-10T07:35:14.842382+0000","last_clean":"2024-09-10T07:35:14.842382+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:35:14.842382+0000","last_undegraded":"2024-09-10T07:35:14.842382+0000","last_fullsized":"2024-09-10T07:35:14.842382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499802+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:35:42.499802+0000","last_peered":"2024-09-10T07:35:42.499802+0000","last_clean":"2024-09-10T07:35:42.499802+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:35:42.499802+0000","last_undegraded":"2024-09-10T07:35:42.499802+0000","last_fullsized":"2024-09-10T07:35:42.499802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'18","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:43.451679+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:35:43.451679+0000","last_peered":"2024-09-10T07:35:43.451679+0000","last_clean":"2024-09-10T07:35:43.451679+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:35:43.451679+0000","last_undegraded":"2024-09-10T07:35:43.451679+0000","last_fullsized":"2024-09-10T07:35:43.451679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976909+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:35:14.976909+0000","last_peered":"2024-09-10T07:35:14.976909+0000","last_clean":"2024-09-10T07:35:14.976909+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:35:14.976909+0000","last_undegraded":"2024-09-10T07:35:14.976909+0000","last_fullsized":"2024-09-10T07:35:14.976909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.481577+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:35:42.481577+0000","last_peered":"2024-09-10T07:35:42.481577+0000","last_clean":"2024-09-10T07:35:42.481577+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:35:42.481577+0000","last_undegraded":"2024-09-10T07:35:42.481577+0000","last_fullsized":"2024-09-10T07:35:42.481577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.872520+0000","last_change":"2024-09-10T07:35:14.872520+0000","last_active":"2024-09-10T07:35:14.872520+0000","last_peered":"2024-09-10T07:35:14.872520+0000","last_clean":"2024-09-10T07:35:14.872520+0000","last_became_active":"2024-09-10T07:35:14.860686+0000","last_became_peered":"2024-09-10T07:35:14.860686+0000","last_unstale":"2024-09-10T07:35:14.872520+0000","last_undegraded":"2024-09-10T07:35:14.872520+0000","last_fullsized":"2024-09-10T07:35:14.872520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:42.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852355+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:35:14.852355+0000","last_peered":"2024-09-10T07:35:14.852355+0000","last_clean":"2024-09-10T07:35:14.852355+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:35:14.852355+0000","last_undegraded":"2024-09-10T07:35:14.852355+0000","last_fullsized":"2024-09-10T07:35:14.852355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"97'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489096+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:35:42.489096+0000","last_peered":"2024-09-10T07:35:42.489096+0000","last_clean":"2024-09-10T07:35:42.489096+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:35:42.489096+0000","last_undegraded":"2024-09-10T07:35:42.489096+0000","last_fullsized":"2024-09-10T07:35:42.489096+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848959+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:35:14.848959+0000","last_peered":"2024-09-10T07:35:14.848959+0000","last_clean":"2024-09-10T07:35:14.848959+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:35:14.848959+0000","last_undegraded":"2024-09-10T07:35:14.848959+0000","last_fullsized":"2024-09-10T07:35:14.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853941+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:35:14.853941+0000","last_peered":"2024-09-10T07:35:14.853941+0000","last_clean":"2024-09-10T07:35:14.853941+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:35:14.853941+0000","last_undegraded":"2024-09-10T07:35:14.853941+0000","last_fullsized":"2024-09-10T07:35:14.853941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.485460+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:35:42.485460+0000","last_peered":"2024-09-10T07:35:42.485460+0000","last_clean":"2024-09-10T07:35:42.485460+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:35:42.485460+0000","last_undegraded":"2024-09-10T07:35:42.485460+0000","last_fullsized":"2024-09-10T07:35:42.485460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"97'7","reported_seq":185,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:35.012468+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:36:35.012468+0000","last_peered":"2024-09-10T07:36:35.012468+0000","last_clean":"2024-09-10T07:36:35.012468+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:36:35.012468+0000","last_undegraded":"2024-09-10T07:36:35.012468+0000","last_fullsized":"2024-09-10T07:36:35.012468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879712+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:35:14.879712+0000","last_peered":"2024-09-10T07:35:14.879712+0000","last_clean":"2024-09-10T07:35:14.879712+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:35:14.879712+0000","last_undegraded":"2024-09-10T07:35:14.879712+0000","last_fullsized":"2024-09-10T07:35:14.879712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476516+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:35:42.476516+0000","last_peered":"2024-09-10T07:35:42.476516+0000","last_clean":"2024-09-10T07:35:42.476516+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:35:42.476516+0000","last_undegraded":"2024-09-10T07:35:42.476516+0000","last_fullsized":"2024-09-10T07:35:42.476516+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":181,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:33.986843+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:36:33.986843+0000","last_peered":"2024-09-10T07:36:33.986843+0000","last_clean":"2024-09-10T07:36:33.986843+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:36:33.986843+0000","last_undegraded":"2024-09-10T07:36:33.986843+0000","last_fullsized":"2024-09-10T07:36:33.986843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880600+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:35:14.880600+0000","last_peered":"2024-09-10T07:35:14.880600+0000","last_clean":"2024-09-10T07:35:14.880600+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:35:14.880600+0000","last_undegraded":"2024-09-10T07:35:14.880600+0000","last_fullsized":"2024-09-10T07:35:14.880600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.479337+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:35:42.479337+0000","last_peered":"2024-09-10T07:35:42.479337+0000","last_clean":"2024-09-10T07:35:42.479337+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:35:42.479337+0000","last_undegraded":"2024-09-10T07:35:42.479337+0000","last_fullsized":"2024-09-10T07:35:42.479337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842761+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:35:14.842761+0000","last_peered":"2024-09-10T07:35:14.842761+0000","last_clean":"2024-09-10T07:35:14.842761+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:35:14.842761+0000","last_undegraded":"2024-09-10T07:35:14.842761+0000","last_fullsized":"2024-09-10T07:35:14.842761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498940+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:35:42.498940+0000","last_peered":"2024-09-10T07:35:42.498940+0000","last_clean":"2024-09-10T07:35:42.498940+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:35:42.498940+0000","last_undegraded":"2024-09-10T07:35:42.498940+0000","last_fullsized":"2024-09-10T07:35:42.498940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882510+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:35:14.882510+0000","last_peered":"2024-09-10T07:35:14.882510+0000","last_clean":"2024-09-10T07:35:14.882510+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:35:14.882510+0000","last_undegraded":"2024-09-10T07:35:14.882510+0000","last_fullsized":"2024-09-10T07:35:14.882510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842861+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:35:14.842861+0000","last_peered":"2024-09-10T07:35:14.842861+0000","last_clean":"2024-09-10T07:35:14.842861+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:35:14.842861+0000","last_undegraded":"2024-09-10T07:35:14.842861+0000","last_fullsized":"2024-09-10T07:35:14.842861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843182+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:35:14.843182+0000","last_peered":"2024-09-10T07:35:14.843182+0000","last_clean":"2024-09-10T07:35:14.843182+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:35:14.843182+0000","last_undegraded":"2024-09-10T07:35:14.843182+0000","last_fullsized":"2024-09-10T07:35:14.843182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484404+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:35:42.484404+0000","last_peered":"2024-09-10T07:35:42.484404+0000","last_clean":"2024-09-10T07:35:42.484404+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:35:42.484404+0000","last_undegraded":"2024-09-10T07:35:42.484404+0000","last_fullsized":"2024-09-10T07:35:42.484404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":186,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:38.994508+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:36:38.994508+0000","last_peered":"2024-09-10T07:36:38.994508+0000","last_clean":"2024-09-10T07:36:38.994508+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:36:38.994508+0000","last_undegraded":"2024-09-10T07:36:38.994508+0000","last_fullsized":"2024-09-10T07:36:38.994508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976793+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:35:14.976793+0000","last_peered":"2024-09-10T07:35:14.976793+0000","last_clean":"2024-09-10T07:35:14.976793+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:35:14.976793+0000","last_undegraded":"2024-09-10T07:35:14.976793+0000","last_fullsized":"2024-09-10T07:35:14.976793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487151+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:35:42.487151+0000","last_peered":"2024-09-10T07:35:42.487151+0000","last_clean":"2024-09-10T07:35:42.487151+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:35:42.487151+0000","last_undegraded":"2024-09-10T07:35:42.487151+0000","last_fullsized":"2024-09-10T07:35:42.487151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880698+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:35:14.880698+0000","last_peered":"2024-09-10T07:35:14.880698+0000","last_clean":"2024-09-10T07:35:14.880698+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:35:14.880698+0000","last_undegraded":"2024-09-10T07:35:14.880698+0000","last_fullsized":"2024-09-10T07:35:14.880698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857702+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:35:14.857702+0000","last_peered":"2024-09-10T07:35:14.857702+0000","last_clean":"2024-09-10T07:35:14.857702+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:35:14.857702+0000","last_undegraded":"2024-09-10T07:35:14.857702+0000","last_fullsized":"2024-09-10T07:35:14.857702+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477302+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:35:42.477302+0000","last_peered":"2024-09-10T07:35:42.477302+0000","last_clean":"2024-09-10T07:35:42.477302+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:35:42.477302+0000","last_undegraded":"2024-09-10T07:35:42.477302+0000","last_fullsized":"2024-09-10T07:35:42.477302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880480+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:35:14.880480+0000","last_peered":"2024-09-10T07:35:14.880480+0000","last_clean":"2024-09-10T07:35:14.880480+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:35:14.880480+0000","last_undegraded":"2024-09-10T07:35:14.880480+0000","last_fullsized":"2024-09-10T07:35:14.880480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879975+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:35:14.879975+0000","last_peered":"2024-09-10T07:35:14.879975+0000","last_clean":"2024-09-10T07:35:14.879975+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:35:14.879975+0000","last_undegraded":"2024-09-10T07:35:14.879975+0000","last_fullsized":"2024-09-10T07:35:14.879975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.474216+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:35:42.474216+0000","last_peered":"2024-09-10T07:35:42.474216+0000","last_clean":"2024-09-10T07:35:42.474216+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:35:42.474216+0000","last_undegraded":"2024-09-10T07:35:42.474216+0000","last_fullsized":"2024-09-10T07:35:42.474216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854462+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:35:14.854462+0000","last_peered":"2024-09-10T07:35:14.854462+0000","last_clean":"2024-09-10T07:35:14.854462+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:35:14.854462+0000","last_undegraded":"2024-09-10T07:35:14.854462+0000","last_fullsized":"2024-09-10T07:35:14.854462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879978+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:35:14.879978+0000","last_peered":"2024-09-10T07:35:14.879978+0000","last_clean":"2024-09-10T07:35:14.879978+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:35:14.879978+0000","last_undegraded":"2024-09-10T07:35:14.879978+0000","last_fullsized":"2024-09-10T07:35:14.879978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.480953+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:35:42.480953+0000","last_peered":"2024-09-10T07:35:42.480953+0000","last_clean":"2024-09-10T07:35:42.480953+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:35:42.480953+0000","last_undegraded":"2024-09-10T07:35:42.480953+0000","last_fullsized":"2024-09-10T07:35:42.480953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853997+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:35:14.853997+0000","last_peered":"2024-09-10T07:35:14.853997+0000","last_clean":"2024-09-10T07:35:14.853997+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:35:14.853997+0000","last_undegraded":"2024-09-10T07:35:14.853997+0000","last_fullsized":"2024-09-10T07:35:14.853997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880388+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:35:14.880388+0000","last_peered":"2024-09-10T07:35:14.880388+0000","last_clean":"2024-09-10T07:35:14.880388+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:35:14.880388+0000","last_undegraded":"2024-09-10T07:35:14.880388+0000","last_fullsized":"2024-09-10T07:35:14.880388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487180+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:35:42.487180+0000","last_peered":"2024-09-10T07:35:42.487180+0000","last_clean":"2024-09-10T07:35:42.487180+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:35:42.487180+0000","last_undegraded":"2024-09-10T07:35:42.487180+0000","last_fullsized":"2024-09-10T07:35:42.487180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":180,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:33.987227+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:36:33.987227+0000","last_peered":"2024-09-10T07:36:33.987227+0000","last_clean":"2024-09-10T07:36:33.987227+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:36:33.987227+0000","last_undegraded":"2024-09-10T07:36:33.987227+0000","last_fullsized":"2024-09-10T07:36:33.987227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851402+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:35:14.851402+0000","last_peered":"2024-09-10T07:35:14.851402+0000","last_clean":"2024-09-10T07:35:14.851402+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:35:14.851402+0000","last_undegraded":"2024-09-10T07:35:14.851402+0000","last_fullsized":"2024-09-10T07:35:14.851402+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488551+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:35:42.488551+0000","last_peered":"2024-09-10T07:35:42.488551+0000","last_clean":"2024-09-10T07:35:42.488551+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:35:42.488551+0000","last_undegraded":"2024-09-10T07:35:42.488551+0000","last_fullsized":"2024-09-10T07:35:42.488551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880166+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:35:14.880166+0000","last_peered":"2024-09-10T07:35:14.880166+0000","last_clean":"2024-09-10T07:35:14.880166+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:35:14.880166+0000","last_undegraded":"2024-09-10T07:35:14.880166+0000","last_fullsized":"2024-09-10T07:35:14.880166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852262+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:35:14.852262+0000","last_peered":"2024-09-10T07:35:14.852262+0000","last_clean":"2024-09-10T07:35:14.852262+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:35:14.852262+0000","last_undegraded":"2024-09-10T07:35:14.852262+0000","last_fullsized":"2024-09-10T07:35:14.852262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500238+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:35:42.500238+0000","last_peered":"2024-09-10T07:35:42.500238+0000","last_clean":"2024-09-10T07:35:42.500238+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:35:42.500238+0000","last_undegraded":"2024-09-10T07:35:42.500238+0000","last_fullsized":"2024-09-10T07:35:42.500238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852264+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:35:14.852264+0000","last_peered":"2024-09-10T07:35:14.852264+0000","last_clean":"2024-09-10T07:35:14.852264+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:35:14.852264+0000","last_undegraded":"2024-09-10T07:35:14.852264+0000","last_fullsized":"2024-09-10T07:35:14.852264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.469244+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:35:42.469244+0000","last_peered":"2024-09-10T07:35:42.469244+0000","last_clean":"2024-09-10T07:35:42.469244+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:35:42.469244+0000","last_undegraded":"2024-09-10T07:35:42.469244+0000","last_fullsized":"2024-09-10T07:35:42.469244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"97'42","reported_seq":364,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:37.991978+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:36:37.991978+0000","last_peered":"2024-09-10T07:36:37.991978+0000","last_clean":"2024-09-10T07:36:37.991978+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:36:37.991978+0000","last_undegraded":"2024-09-10T07:36:37.991978+0000","last_fullsized":"2024-09-10T07:36:37.991978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70555,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":278,"num_read_kb":208,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848093+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:35:14.848093+0000","last_peered":"2024-09-10T07:35:14.848093+0000","last_clean":"2024-09-10T07:35:14.848093+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:35:14.848093+0000","last_undegraded":"2024-09-10T07:35:14.848093+0000","last_fullsized":"2024-09-10T07:35:14.848093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490736+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:35:42.490736+0000","last_peered":"2024-09-10T07:35:42.490736+0000","last_clean":"2024-09-10T07:35:42.490736+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:35:42.490736+0000","last_undegraded":"2024-09-10T07:35:42.490736+0000","last_fullsized":"2024-09-10T07:35:42.490736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880613+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:35:14.880613+0000","last_peered":"2024-09-10T07:35:14.880613+0000","last_clean":"2024-09-10T07:35:14.880613+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:35:14.880613+0000","last_undegraded":"2024-09-10T07:35:14.880613+0000","last_fullsized":"2024-09-10T07:35:14.880613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882346+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:35:14.882346+0000","last_peered":"2024-09-10T07:35:14.882346+0000","last_clean":"2024-09-10T07:35:14.882346+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:35:14.882346+0000","last_undegraded":"2024-09-10T07:35:14.882346+0000","last_fullsized":"2024-09-10T07:35:14.882346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502535+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:35:42.502535+0000","last_peered":"2024-09-10T07:35:42.502535+0000","last_clean":"2024-09-10T07:35:42.502535+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:35:42.502535+0000","last_undegraded":"2024-09-10T07:35:42.502535+0000","last_fullsized":"2024-09-10T07:35:42.502535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843243+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:35:14.843243+0000","last_peered":"2024-09-10T07:35:14.843243+0000","last_clean":"2024-09-10T07:35:14.843243+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:35:14.843243+0000","last_undegraded":"2024-09-10T07:35:14.843243+0000","last_fullsized":"2024-09-10T07:35:14.843243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847404+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:35:14.847404+0000","last_peered":"2024-09-10T07:35:14.847404+0000","last_clean":"2024-09-10T07:35:14.847404+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:35:14.847404+0000","last_undegraded":"2024-09-10T07:35:14.847404+0000","last_fullsized":"2024-09-10T07:35:14.847404+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490415+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:35:42.490415+0000","last_peered":"2024-09-10T07:35:42.490415+0000","last_clean":"2024-09-10T07:35:42.490415+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:35:42.490415+0000","last_undegraded":"2024-09-10T07:35:42.490415+0000","last_fullsized":"2024-09-10T07:35:42.490415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"97'5","reported_seq":315,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:37.992475+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:36:37.992475+0000","last_peered":"2024-09-10T07:36:37.992475+0000","last_clean":"2024-09-10T07:36:37.992475+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:36:37.992475+0000","last_undegraded":"2024-09-10T07:36:37.992475+0000","last_fullsized":"2024-09-10T07:36:37.992475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266,"num_read_kb":266,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977046+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:35:14.977046+0000","last_peered":"2024-09-10T07:35:14.977046+0000","last_clean":"2024-09-10T07:35:14.977046+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:35:14.977046+0000","last_undegraded":"2024-09-10T07:35:14.977046+0000","last_fullsized":"2024-09-10T07:35:14.977046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858166+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:35:14.858166+0000","last_peered":"2024-09-10T07:35:14.858166+0000","last_clean":"2024-09-10T07:35:14.858166+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:35:14.858166+0000","last_undegraded":"2024-09-10T07:35:14.858166+0000","last_fullsized":"2024-09-10T07:35:14.858166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498588+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:35:42.498588+0000","last_peered":"2024-09-10T07:35:42.498588+0000","last_clean":"2024-09-10T07:35:42.498588+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:35:42.498588+0000","last_undegraded":"2024-09-10T07:35:42.498588+0000","last_fullsized":"2024-09-10T07:35:42.498588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880248+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:35:14.880248+0000","last_peered":"2024-09-10T07:35:14.880248+0000","last_clean":"2024-09-10T07:35:14.880248+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:35:14.880248+0000","last_undegraded":"2024-09-10T07:35:14.880248+0000","last_fullsized":"2024-09-10T07:35:14.880248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852575+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:35:14.852575+0000","last_peered":"2024-09-10T07:35:14.852575+0000","last_clean":"2024-09-10T07:35:14.852575+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:35:14.852575+0000","last_undegraded":"2024-09-10T07:35:14.852575+0000","last_fullsized":"2024-09-10T07:35:14.852575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499570+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:35:42.499570+0000","last_peered":"2024-09-10T07:35:42.499570+0000","last_clean":"2024-09-10T07:35:42.499570+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:35:42.499570+0000","last_undegraded":"2024-09-10T07:35:42.499570+0000","last_fullsized":"2024-09-10T07:35:42.499570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879193+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:35:14.879193+0000","last_peered":"2024-09-10T07:35:14.879193+0000","last_clean":"2024-09-10T07:35:14.879193+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:35:14.879193+0000","last_undegraded":"2024-09-10T07:35:14.879193+0000","last_fullsized":"2024-09-10T07:35:14.879193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877898+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:35:14.877898+0000","last_peered":"2024-09-10T07:35:14.877898+0000","last_clean":"2024-09-10T07:35:14.877898+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:35:14.877898+0000","last_undegraded":"2024-09-10T07:35:14.877898+0000","last_fullsized":"2024-09-10T07:35:14.877898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"97'8","reported_seq":23,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.491883+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:35:42.491883+0000","last_peered":"2024-09-10T07:35:42.491883+0000","last_clean":"2024-09-10T07:35:42.491883+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:35:42.491883+0000","last_undegraded":"2024-09-10T07:35:42.491883+0000","last_fullsized":"2024-09-10T07:35:42.491883+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842519+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:35:14.842519+0000","last_peered":"2024-09-10T07:35:14.842519+0000","last_clean":"2024-09-10T07:35:14.842519+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:35:14.842519+0000","last_undegraded":"2024-09-10T07:35:14.842519+0000","last_fullsized":"2024-09-10T07:35:14.842519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":190,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:38.994596+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:36:38.994596+0000","last_peered":"2024-09-10T07:36:38.994596+0000","last_clean":"2024-09-10T07:36:38.994596+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:36:38.994596+0000","last_undegraded":"2024-09-10T07:36:38.994596+0000","last_fullsized":"2024-09-10T07:36:38.994596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475213+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:35:42.475213+0000","last_peered":"2024-09-10T07:35:42.475213+0000","last_clean":"2024-09-10T07:35:42.475213+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:35:42.475213+0000","last_undegraded":"2024-09-10T07:35:42.475213+0000","last_fullsized":"2024-09-10T07:35:42.475213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842633+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:35:14.842633+0000","last_peered":"2024-09-10T07:35:14.842633+0000","last_clean":"2024-09-10T07:35:14.842633+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:35:14.842633+0000","last_undegraded":"2024-09-10T07:35:14.842633+0000","last_fullsized":"2024-09-10T07:35:14.842633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477747+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:35:42.477747+0000","last_peered":"2024-09-10T07:35:42.477747+0000","last_clean":"2024-09-10T07:35:42.477747+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:35:42.477747+0000","last_undegraded":"2024-09-10T07:35:42.477747+0000","last_fullsized":"2024-09-10T07:35:42.477747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"97'12","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476532+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:35:42.476532+0000","last_peered":"2024-09-10T07:35:42.476532+0000","last_clean":"2024-09-10T07:35:42.476532+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:35:42.476532+0000","last_undegraded":"2024-09-10T07:35:42.476532+0000","last_fullsized":"2024-09-10T07:35:42.476532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486476+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:35:42.486476+0000","last_peered":"2024-09-10T07:35:42.486476+0000","last_clean":"2024-09-10T07:35:42.486476+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:35:42.486476+0000","last_undegraded":"2024-09-10T07:35:42.486476+0000","last_fullsized":"2024-09-10T07:35:42.486476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499618+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:35:42.499618+0000","last_peered":"2024-09-10T07:35:42.499618+0000","last_clean":"2024-09-10T07:35:42.499618+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:35:42.499618+0000","last_undegraded":"2024-09-10T07:35:42.499618+0000","last_fullsized":"2024-09-10T07:35:42.499618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487143+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:35:42.487143+0000","last_peered":"2024-09-10T07:35:42.487143+0000","last_clean":"2024-09-10T07:35:42.487143+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:35:42.487143+0000","last_undegraded":"2024-09-10T07:35:42.487143+0000","last_fullsized":"2024-09-10T07:35:42.487143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.470371+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:35:42.470371+0000","last_peered":"2024-09-10T07:35:42.470371+0000","last_clean":"2024-09-10T07:35:42.470371+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:35:42.470371+0000","last_undegraded":"2024-09-10T07:35:42.470371+0000","last_fullsized":"2024-09-10T07:35:42.470371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476911+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:35:42.476911+0000","last_peered":"2024-09-10T07:35:42.476911+0000","last_clean":"2024-09-10T07:35:42.476911+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:35:42.476911+0000","last_undegraded":"2024-09-10T07:35:42.476911+0000","last_fullsized":"2024-09-10T07:35:42.476911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488157+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:35:42.488157+0000","last_peered":"2024-09-10T07:35:42.488157+0000","last_clean":"2024-09-10T07:35:42.488157+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:35:42.488157+0000","last_undegraded":"2024-09-10T07:35:42.488157+0000","last_fullsized":"2024-09-10T07:35:42.488157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500450+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:35:42.500450+0000","last_peered":"2024-09-10T07:35:42.500450+0000","last_clean":"2024-09-10T07:35:42.500450+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:35:42.500450+0000","last_undegraded":"2024-09-10T07:35:42.500450+0000","last_fullsized":"2024-09-10T07:35:42.500450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499239+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:35:42.499239+0000","last_peered":"2024-09-10T07:35:42.499239+0000","last_clean":"2024-09-10T07:35:42.499239+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:35:42.499239+0000","last_undegraded":"2024-09-10T07:35:42.499239+0000","last_fullsized":"2024-09-10T07:35:42.499239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501908+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:35:42.501908+0000","last_peered":"2024-09-10T07:35:42.501908+0000","last_clean":"2024-09-10T07:35:42.501908+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:35:42.501908+0000","last_undegraded":"2024-09-10T07:35:42.501908+0000","last_fullsized":"2024-09-10T07:35:42.501908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493731+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:35:42.493731+0000","last_peered":"2024-09-10T07:35:42.493731+0000","last_clean":"2024-09-10T07:35:42.493731+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:35:42.493731+0000","last_undegraded":"2024-09-10T07:35:42.493731+0000","last_fullsized":"2024-09-10T07:35:42.493731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478046+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:35:42.478046+0000","last_peered":"2024-09-10T07:35:42.478046+0000","last_clean":"2024-09-10T07:35:42.478046+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:35:42.478046+0000","last_undegraded":"2024-09-10T07:35:42.478046+0000","last_fullsized":"2024-09-10T07:35:42.478046+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493144+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:35:42.493144+0000","last_peered":"2024-09-10T07:35:42.493144+0000","last_clean":"2024-09-10T07:35:42.493144+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:35:42.493144+0000","last_undegraded":"2024-09-10T07:35:42.493144+0000","last_fullsized":"2024-09-10T07:35:42.493144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483119+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:35:42.483119+0000","last_peered":"2024-09-10T07:35:42.483119+0000","last_clean":"2024-09-10T07:35:42.483119+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:35:42.483119+0000","last_undegraded":"2024-09-10T07:35:42.483119+0000","last_fullsized":"2024-09-10T07:35:42.483119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502676+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:35:42.502676+0000","last_peered":"2024-09-10T07:35:42.502676+0000","last_clean":"2024-09-10T07:35:42.502676+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:35:42.502676+0000","last_undegraded":"2024-09-10T07:35:42.502676+0000","last_fullsized":"2024-09-10T07:35:42.502676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502776+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:35:42.502776+0000","last_peered":"2024-09-10T07:35:42.502776+0000","last_clean":"2024-09-10T07:35:42.502776+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:35:42.502776+0000","last_undegraded":"2024-09-10T07:35:42.502776+0000","last_fullsized":"2024-09-10T07:35:42.502776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499409+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:35:42.499409+0000","last_peered":"2024-09-10T07:35:42.499409+0000","last_clean":"2024-09-10T07:35:42.499409+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:35:42.499409+0000","last_undegraded":"2024-09-10T07:35:42.499409+0000","last_fullsized":"2024-09-10T07:35:42.499409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502975+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:35:42.502975+0000","last_peered":"2024-09-10T07:35:42.502975+0000","last_clean":"2024-09-10T07:35:42.502975+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:35:42.502975+0000","last_undegraded":"2024-09-10T07:35:42.502975+0000","last_fullsized":"2024-09-10T07:35:42.502975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493842+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:35:42.493842+0000","last_peered":"2024-09-10T07:35:42.493842+0000","last_clean":"2024-09-10T07:35:42.493842+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:35:42.493842+0000","last_undegraded":"2024-09-10T07:35:42.493842+0000","last_fullsized":"2024-09-10T07:35:42.493842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484180+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:35:42.484180+0000","last_peered":"2024-09-10T07:35:42.484180+0000","last_clean":"2024-09-10T07:35:42.484180+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:35:42.484180+0000","last_undegraded":"2024-09-10T07:35:42.484180+0000","last_fullsized":"2024-09-10T07:35:42.484180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483929+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:35:42.483929+0000","last_peered":"2024-09-10T07:35:42.483929+0000","last_clean":"2024-09-10T07:35:42.483929+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:35:42.483929+0000","last_undegraded":"2024-09-10T07:35:42.483929+0000","last_fullsized":"2024-09-10T07:35:42.483929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"97'16","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483320+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:35:42.483320+0000","last_peered":"2024-09-10T07:35:42.483320+0000","last_clean":"2024-09-10T07:35:42.483320+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:35:42.483320+0000","last_undegraded":"2024-09-10T07:35:42.483320+0000","last_fullsized":"2024-09-10T07:35:42.483320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489981+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:35:42.489981+0000","last_peered":"2024-09-10T07:35:42.489981+0000","last_clean":"2024-09-10T07:35:42.489981+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:35:42.489981+0000","last_undegraded":"2024-09-10T07:35:42.489981+0000","last_fullsized":"2024-09-10T07:35:42.489981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500857+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:35:42.500857+0000","last_peered":"2024-09-10T07:35:42.500857+0000","last_clean":"2024-09-10T07:35:42.500857+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:35:42.500857+0000","last_undegraded":"2024-09-10T07:35:42.500857+0000","last_fullsized":"2024-09-10T07:35:42.500857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478497+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:35:42.478497+0000","last_peered":"2024-09-10T07:35:42.478497+0000","last_clean":"2024-09-10T07:35:42.478497+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:35:42.478497+0000","last_undegraded":"2024-09-10T07:35:42.478497+0000","last_fullsized":"2024-09-10T07:35:42.478497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487137+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:35:42.487137+0000","last_peered":"2024-09-10T07:35:42.487137+0000","last_clean":"2024-09-10T07:35:42.487137+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:35:42.487137+0000","last_undegraded":"2024-09-10T07:35:42.487137+0000","last_fullsized":"2024-09-10T07:35:42.487137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"97'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.496842+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:35:42.496842+0000","last_peered":"2024-09-10T07:35:42.496842+0000","last_clean":"2024-09-10T07:35:42.496842+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:35:42.496842+0000","last_undegraded":"2024-09-10T07:35:42.496842+0000","last_fullsized":"2024-09-10T07:35:42.496842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"97'8","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476981+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:35:42.476981+0000","last_peered":"2024-09-10T07:35:42.476981+0000","last_clean":"2024-09-10T07:35:42.476981+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:35:42.476981+0000","last_undegraded":"2024-09-10T07:35:42.476981+0000","last_fullsized":"2024-09-10T07:35:42.476981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.494172+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:35:42.494172+0000","last_peered":"2024-09-10T07:35:42.494172+0000","last_clean":"2024-09-10T07:35:42.494172+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:35:42.494172+0000","last_undegraded":"2024-09-10T07:35:42.494172+0000","last_fullsized":"2024-09-10T07:35:42.494172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502201+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:35:42.502201+0000","last_peered":"2024-09-10T07:35:42.502201+0000","last_clean":"2024-09-10T07:35:42.502201+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:35:42.502201+0000","last_undegraded":"2024-09-10T07:35:42.502201+0000","last_fullsized":"2024-09-10T07:35:42.502201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502439+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:35:42.502439+0000","last_peered":"2024-09-10T07:35:42.502439+0000","last_clean":"2024-09-10T07:35:42.502439+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:35:42.502439+0000","last_undegraded":"2024-09-10T07:35:42.502439+0000","last_fullsized":"2024-09-10T07:35:42.502439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78045,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1369,"num_read_kb":769,"num_write":265,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":234135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449140,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":561212,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710777,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1241088,"data_stored":1218403,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874300,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":565058,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135939,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":561228,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364871,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1158514,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528397,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":557000,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790037,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1153562,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953560,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":560540,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":556174,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1171456,"data_stored":628737,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":628585,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":557922,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":72993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:36:44.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T07:36:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:44 smithi181 ceph-mon[35397]: pgmap v402: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:44 smithi063 ceph-mon[35020]: pgmap v402: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:44.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:44 smithi046 ceph-mon[30705]: pgmap v402: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.874+0000 7f0504600640 1 -- 172.21.15.46:0/1300708562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc1049c0 msgr2=0x7f04fc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.874+0000 7f0504600640 1 --2- 172.21.15.46:0/1300708562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc1049c0 0x7f04fc108a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009860 tx=0x7f04e802a340 comp rx=0 tx=0).stop 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.874+0000 7f0504600640 1 -- 172.21.15.46:0/1300708562 shutdown_connections 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.874+0000 7f0504600640 1 --2- 172.21.15.46:0/1300708562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04fc109140 0x7f04fc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.874+0000 7f0504600640 1 --2- 172.21.15.46:0/1300708562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc1049c0 0x7f04fc108a10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.874+0000 7f0504600640 1 --2- 172.21.15.46:0/1300708562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04fc104010 0x7f04fc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.874+0000 7f0504600640 1 -- 172.21.15.46:0/1300708562 >> 172.21.15.46:0/1300708562 conn(0x7f04fc0ffa50 msgr2=0x7f04fc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.875+0000 7f0504600640 1 -- 172.21.15.46:0/1300708562 shutdown_connections 2024-09-10T07:36:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.875+0000 7f0504600640 1 -- 172.21.15.46:0/1300708562 wait complete. 2024-09-10T07:36:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.875+0000 7f0504600640 1 Processor -- start 2024-09-10T07:36:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0504600640 1 -- start start 2024-09-10T07:36:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0504600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc104010 0x7f04fc076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0504600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04fc1049c0 0x7f04fc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0504600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04fc109140 0x7f04fc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0504600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc114160 con 0x7f04fc109140 2024-09-10T07:36:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0504600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc113fe0 con 0x7f04fc104010 2024-09-10T07:36:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0504600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc1142e0 con 0x7f04fc1049c0 2024-09-10T07:36:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0501c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc104010 0x7f04fc076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0502600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04fc109140 0x7f04fc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0501c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc104010 0x7f04fc076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52398/0 (socket says 172.21.15.46:52398) 2024-09-10T07:36:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0501c00640 1 -- 172.21.15.46:0/3731829486 learned_addr learned my addr 172.21.15.46:0/3731829486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.876+0000 7f0501200640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04fc1049c0 0x7f04fc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.877+0000 7f0501c00640 1 -- 172.21.15.46:0/3731829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04fc1049c0 msgr2=0x7f04fc076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.877+0000 7f0501c00640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04fc1049c0 0x7f04fc076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.877+0000 7f0501c00640 1 -- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04fc109140 msgr2=0x7f04fc077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.877+0000 7f0501c00640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04fc109140 0x7f04fc077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.877+0000 7f0501c00640 1 -- 172.21.15.46:0/3731829486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc1aad00 con 0x7f04fc104010 2024-09-10T07:36:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.877+0000 7f0501c00640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc104010 0x7f04fc076700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04f000ea50 tx=0x7f04f000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.877+0000 7f04fa000640 1 -- 172.21.15.46:0/3731829486 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f000cd60 con 0x7f04fc104010 2024-09-10T07:36:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.877+0000 7f04fa000640 1 -- 172.21.15.46:0/3731829486 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0005500 con 0x7f04fc104010 2024-09-10T07:36:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.878+0000 7f0504600640 1 -- 172.21.15.46:0/3731829486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc1aaff0 con 0x7f04fc104010 2024-09-10T07:36:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.878+0000 7f04fa000640 1 -- 172.21.15.46:0/3731829486 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0010690 con 0x7f04fc104010 2024-09-10T07:36:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.878+0000 7f0504600640 1 -- 172.21.15.46:0/3731829486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc1ab450 con 0x7f04fc104010 2024-09-10T07:36:44.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.881+0000 7f04fa000640 1 -- 172.21.15.46:0/3731829486 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04f0004f30 con 0x7f04fc104010 2024-09-10T07:36:44.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.882+0000 7f0507234740 1 Processor -- start 2024-09-10T07:36:44.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.882+0000 7f0507234740 1 -- start start 2024-09-10T07:36:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.882+0000 7f0507234740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca88ffb90 0x55bca88fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.882+0000 7f0507234740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.882+0000 7f0507234740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca890d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.882+0000 7f0507234740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bca88663e0 con 0x55bca89004b0 2024-09-10T07:36:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.882+0000 7f0507234740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bca880dd80 con 0x55bca88ffb90 2024-09-10T07:36:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.882+0000 7f0507234740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bca88623a0 con 0x55bca890ad70 2024-09-10T07:36:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0502600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca88ffb90 0x55bca88fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0501200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca890d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0502600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca88ffb90 0x55bca88fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52400/0 (socket says 172.21.15.46:52400) 2024-09-10T07:36:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0502600640 1 -- 172.21.15.46:0/2906434372 learned_addr learned my addr 172.21.15.46:0/2906434372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0501c00640 1 --2- 172.21.15.46:0/2906434372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0502600640 1 -- 172.21.15.46:0/2906434372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 msgr2=0x55bca890d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0502600640 1 --2- 172.21.15.46:0/2906434372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca890d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f04fa000640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04e40a2b80 0x7f04e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0502600640 1 -- 172.21.15.46:0/2906434372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 msgr2=0x55bca890a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0502600640 1 --2- 172.21.15.46:0/2906434372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f0502600640 1 -- 172.21.15.46:0/2906434372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca882cae0 con 0x55bca88ffb90 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.883+0000 7f04fa000640 1 -- 172.21.15.46:0/3731829486 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f04f00cf770 con 0x7f04fc104010 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.884+0000 7f0502600640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04e40a2b80 0x7f04e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.884+0000 7f0502600640 1 --2- 172.21.15.46:0/2906434372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca88ffb90 0x55bca88fff70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f04f400e480 tx=0x7f04f40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a600fe507e7865f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.884+0000 7f04eea00640 1 -- 172.21.15.46:0/2906434372 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4017c70 con 0x55bca88ffb90 2024-09-10T07:36:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.885+0000 7f04eea00640 1 -- 172.21.15.46:0/2906434372 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f400da50 con 0x55bca88ffb90 2024-09-10T07:36:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.885+0000 7f04eea00640 1 -- 172.21.15.46:0/2906434372 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40235f0 con 0x55bca88ffb90 2024-09-10T07:36:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.885+0000 7f0507234740 1 -- 172.21.15.46:0/2906434372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca88ffb90 msgr2=0x55bca88fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.885+0000 7f0507234740 1 --2- 172.21.15.46:0/2906434372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca88ffb90 0x55bca88fff70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f04f400e480 tx=0x7f04f40095c0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.885+0000 7f0502600640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04e40a2b80 0x7f04e40a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04f4021000 tx=0x7f04f4024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.886+0000 7f0507234740 1 -- 172.21.15.46:0/2906434372 shutdown_connections 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.886+0000 7f0507234740 1 --2- 172.21.15.46:0/2906434372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca890d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.886+0000 7f0507234740 1 --2- 172.21.15.46:0/2906434372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.886+0000 7f0507234740 1 --2- 172.21.15.46:0/2906434372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca88ffb90 0x55bca88fff70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.886+0000 7f0507234740 1 -- 172.21.15.46:0/2906434372 >> 172.21.15.46:0/2906434372 conn(0x55bca88fc120 msgr2=0x55bca88fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.886+0000 7f0507234740 1 -- 172.21.15.46:0/2906434372 shutdown_connections 2024-09-10T07:36:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.886+0000 7f0507234740 1 -- 172.21.15.46:0/2906434372 wait complete. 2024-09-10T07:36:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.887+0000 7f0507234740 1 -- start start 2024-09-10T07:36:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.887+0000 7f0507234740 1 Processor -- start 2024-09-10T07:36:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0507234740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8907cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0507234740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 0x55bca89081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0507234740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8908730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0507234740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bca890d9e0 con 0x55bca88ffb90 2024-09-10T07:36:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0507234740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bca88210f0 con 0x55bca89004b0 2024-09-10T07:36:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0507234740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bca890db60 con 0x55bca890ad70 2024-09-10T07:36:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0507234740 10 client.0 _ll_register_callbacks cb 0x55bca88129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0501c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8907cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0501c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8907cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37354/0 (socket says 172.21.15.46:37354) 2024-09-10T07:36:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0501c00640 1 -- 172.21.15.46:0/3350390452 learned_addr learned my addr 172.21.15.46:0/3350390452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0501200640 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 0x55bca89081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.888+0000 7f0502600640 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8908730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.889+0000 7f0501c00640 1 -- 172.21.15.46:0/3350390452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 msgr2=0x55bca8908730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.889+0000 7f0501c00640 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8908730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.889+0000 7f0501c00640 1 -- 172.21.15.46:0/3350390452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 msgr2=0x55bca89081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:44.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.889+0000 7f0501c00640 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 0x55bca89081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.889+0000 7f0501c00640 1 -- 172.21.15.46:0/3350390452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca85d73c0 con 0x55bca88ffb90 2024-09-10T07:36:44.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.889+0000 7f0501c00640 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8907cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f04f009bb80 tx=0x7f04f009b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f7a356aa2e44d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:44.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.889+0000 7f04e3400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:36:44.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.889+0000 7f04e3400640 1 -- 172.21.15.46:0/3350390452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f009c2f0 con 0x55bca88ffb90 2024-09-10T07:36:44.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.890+0000 7f04e3400640 1 -- 172.21.15.46:0/3350390452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f04f0005220 con 0x55bca88ffb90 2024-09-10T07:36:44.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.890+0000 7f0507234740 1 -- 172.21.15.46:0/3350390452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f400da50 con 0x55bca88ffb90 2024-09-10T07:36:44.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.890+0000 7f04e3400640 1 -- 172.21.15.46:0/3350390452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04f009bbf0 con 0x55bca88ffb90 2024-09-10T07:36:44.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.890+0000 7f0507234740 10 client.15282 fetch_fsmap learned FSMap version 30 2024-09-10T07:36:44.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.890+0000 7f0507234740 1 -- 172.21.15.46:0/3350390452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bca89196c0 con 0x55bca88ffb90 2024-09-10T07:36:44.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.890+0000 7f04e3400640 1 -- 172.21.15.46:0/3350390452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f400da50 con 0x55bca88ffb90 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.891+0000 7f04e3400640 1 -- 172.21.15.46:0/3350390452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f04f00d2ce0 con 0x55bca88ffb90 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.891+0000 7f0507234740 10 client.15282 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.891+0000 7f0507234740 10 client.15282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.891+0000 7f0507234740 20 client.15282 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.891+0000 7f0507234740 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bca891ab80 0x55bca891d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.891+0000 7f0507234740 4 client.15282 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.891+0000 7f0507234740 1 -- 172.21.15.46:0/3350390452 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bca8873950 con 0x55bca891ab80 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.891+0000 7f0501200640 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bca891ab80 0x55bca891d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.892+0000 7f0501200640 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bca891ab80 0x55bca891d010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6343565ea6ab00e1 server_cookie=10c7c16b11ba4bb in_seq=0 out_seq=0 2024-09-10T07:36:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.892+0000 7f04e3400640 10 client.15282 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:44.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f04e3400640 1 -- 172.21.15.46:0/3350390452 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bca8873950 con 0x55bca891ab80 2024-09-10T07:36:44.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f04e3400640 10 client.15282 handle_command_reply: tid=0 2024-09-10T07:36:44.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 client.15282 shutdown 2024-09-10T07:36:44.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 -- 172.21.15.46:0/3350390452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 msgr2=0x55bca8907cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8907cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f04f009bb80 tx=0x7f04f009b9c0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 -- 172.21.15.46:0/3350390452 shutdown_connections 2024-09-10T07:36:44.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bca891ab80 0x55bca891d010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8908730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 0x55bca89081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 --2- 172.21.15.46:0/3350390452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8907cb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f0507234740 1 -- 172.21.15.46:0/3350390452 >> 172.21.15.46:0/3350390452 conn(0x55bca88fc120 msgr2=0x55bca8901e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:44.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.893+0000 7f04e3400640 0 client.15282 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:44.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.894+0000 7f0507234740 1 -- 172.21.15.46:0/3350390452 shutdown_connections 2024-09-10T07:36:44.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.894+0000 7f0507234740 1 -- 172.21.15.46:0/3350390452 wait complete. 2024-09-10T07:36:44.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.894+0000 7f0507234740 20 client.15282 trim_cache size 0 max 16384 2024-09-10T07:36:44.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.912+0000 7f0507234740 1 Processor -- start 2024-09-10T07:36:44.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0507234740 1 -- start start 2024-09-10T07:36:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0507234740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8922900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0507234740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 0x55bca8922e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0507234740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8904f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0507234740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc1b52d0 con 0x55bca88ffb90 2024-09-10T07:36:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0507234740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc1b6e10 con 0x55bca89004b0 2024-09-10T07:36:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0507234740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc1b59a0 con 0x55bca890ad70 2024-09-10T07:36:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0502600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8904f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0502600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8904f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40048/0 (socket says 172.21.15.46:40048) 2024-09-10T07:36:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.913+0000 7f0502600640 1 -- 172.21.15.46:0/4070252773 learned_addr learned my addr 172.21.15.46:0/4070252773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.914+0000 7f0501200640 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 0x55bca8922e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.914+0000 7f0501c00640 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8922900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.914+0000 7f0502600640 1 -- 172.21.15.46:0/4070252773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 msgr2=0x55bca8922e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.914+0000 7f0502600640 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 0x55bca8922e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.914+0000 7f0502600640 1 -- 172.21.15.46:0/4070252773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 msgr2=0x55bca8922900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.914+0000 7f0502600640 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8922900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.914+0000 7f0502600640 1 -- 172.21.15.46:0/4070252773 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca891d660 con 0x55bca890ad70 2024-09-10T07:36:44.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.915+0000 7f0502600640 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8904f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04f4005fd0 tx=0x7f04f4027000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed08d5c589f5a0dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:44.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.916+0000 7f04ee000640 1 -- 172.21.15.46:0/4070252773 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4023590 con 0x55bca890ad70 2024-09-10T07:36:44.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.916+0000 7f04ee000640 1 -- 172.21.15.46:0/4070252773 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f40049c0 con 0x55bca890ad70 2024-09-10T07:36:44.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.916+0000 7f04ee000640 1 -- 172.21.15.46:0/4070252773 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4023bc0 con 0x55bca890ad70 2024-09-10T07:36:44.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.916+0000 7f0507234740 1 -- 172.21.15.46:0/4070252773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 msgr2=0x55bca8904f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.916+0000 7f0507234740 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8904f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04f4005fd0 tx=0x7f04f4027000 comp rx=0 tx=0).stop 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 -- 172.21.15.46:0/4070252773 shutdown_connections 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca890ad70 0x55bca8904f10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca89004b0 0x55bca8922e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 --2- 172.21.15.46:0/4070252773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca88ffb90 0x55bca8922900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 -- 172.21.15.46:0/4070252773 >> 172.21.15.46:0/4070252773 conn(0x55bca88fc120 msgr2=0x55bca8906580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 -- 172.21.15.46:0/4070252773 shutdown_connections 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 -- 172.21.15.46:0/4070252773 wait complete. 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 -- start start 2024-09-10T07:36:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.917+0000 7f0507234740 1 Processor -- start 2024-09-10T07:36:44.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0507234740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca88ffb90 0x55bca890eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0507234740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0507234740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca890ad70 0x55bca89848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0507234740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc1b6740 con 0x55bca89004b0 2024-09-10T07:36:44.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0507234740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc1b6070 con 0x55bca890ad70 2024-09-10T07:36:44.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0507234740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc1b32b0 con 0x55bca88ffb90 2024-09-10T07:36:44.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0507234740 10 client.0 _ll_register_callbacks cb 0x7f04f0009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0501200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0501200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37380/0 (socket says 172.21.15.46:37380) 2024-09-10T07:36:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0501200640 1 -- 172.21.15.46:0/3487111026 learned_addr learned my addr 172.21.15.46:0/3487111026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.918+0000 7f0501c00640 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca88ffb90 0x55bca890eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.919+0000 7f0501200640 1 -- 172.21.15.46:0/3487111026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca88ffb90 msgr2=0x55bca890eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.919+0000 7f0501200640 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca88ffb90 0x55bca890eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.919+0000 7f0501200640 1 -- 172.21.15.46:0/3487111026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca890ad70 msgr2=0x55bca89848d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:36:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.919+0000 7f0501200640 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca890ad70 0x55bca89848d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.919+0000 7f0501200640 1 -- 172.21.15.46:0/3487111026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca890fd00 con 0x55bca89004b0 2024-09-10T07:36:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.919+0000 7f0501c00640 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca88ffb90 0x55bca890eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.919+0000 7f0501200640 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890f420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f04e80057c0 tx=0x7f04e8005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2c217e39e5c89e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.919+0000 7f04e3400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:36:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.920+0000 7f04e3400640 1 -- 172.21.15.46:0/3487111026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8037040 con 0x55bca89004b0 2024-09-10T07:36:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.920+0000 7f04e3400640 1 -- 172.21.15.46:0/3487111026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f04e8004240 con 0x55bca89004b0 2024-09-10T07:36:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.920+0000 7f0507234740 1 -- 172.21.15.46:0/3487111026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f40049c0 con 0x55bca89004b0 2024-09-10T07:36:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.920+0000 7f04e3400640 1 -- 172.21.15.46:0/3487111026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04e8004440 con 0x55bca89004b0 2024-09-10T07:36:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.920+0000 7f0507234740 10 client.15288 fetch_fsmap learned FSMap version 30 2024-09-10T07:36:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.920+0000 7f0507234740 1 -- 172.21.15.46:0/3487111026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bca8985f60 con 0x55bca89004b0 2024-09-10T07:36:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.920+0000 7f04e3400640 1 -- 172.21.15.46:0/3487111026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f40049c0 con 0x55bca89004b0 2024-09-10T07:36:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.920+0000 7f04e3400640 1 -- 172.21.15.46:0/3487111026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f04e8048070 con 0x55bca89004b0 2024-09-10T07:36:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.921+0000 7f0507234740 10 client.15288 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:36:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.921+0000 7f0507234740 10 client.15288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:36:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.921+0000 7f0507234740 20 client.15288 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.921+0000 7f0507234740 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bca8986730 0x55bca8988b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.921+0000 7f0507234740 4 client.15288 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:36:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.921+0000 7f0507234740 1 -- 172.21.15.46:0/3487111026 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bca886de60 con 0x55bca8986730 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.921+0000 7f0501c00640 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bca8986730 0x55bca8988b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.922+0000 7f0501c00640 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bca8986730 0x55bca8988b50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6bb2228149626c8 server_cookie=85dd16dd549e07bc in_seq=0 out_seq=0 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.922+0000 7f04e3400640 10 client.15288 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f04e3400640 1 -- 172.21.15.46:0/3487111026 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bca886de60 con 0x55bca8986730 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f04e3400640 10 client.15288 handle_command_reply: tid=0 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f0507234740 1 client.15288 shutdown 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f0507234740 1 -- 172.21.15.46:0/3487111026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 msgr2=0x55bca890f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f0507234740 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890f420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f04e80057c0 tx=0x7f04e8005900 comp rx=0 tx=0).stop 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f0507234740 1 -- 172.21.15.46:0/3487111026 shutdown_connections 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f0507234740 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bca8986730 0x55bca8988b50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f0507234740 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bca890ad70 0x55bca89848d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f0507234740 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bca89004b0 0x55bca890f420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f0507234740 1 --2- 172.21.15.46:0/3487111026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bca88ffb90 0x55bca890eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.923+0000 7f04e3400640 0 client.15288 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.924+0000 7f0507234740 1 -- 172.21.15.46:0/3487111026 >> 172.21.15.46:0/3487111026 conn(0x55bca88fc120 msgr2=0x55bca88fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.924+0000 7f0507234740 1 -- 172.21.15.46:0/3487111026 shutdown_connections 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.924+0000 7f0507234740 1 -- 172.21.15.46:0/3487111026 wait complete. 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.924+0000 7f0507234740 20 client.15288 trim_cache size 0 max 16384 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:36:44.925 INFO:teuthology.orchestra.run.smithi046.stdout: "0860b53d-b75f-43fc-8d65-bc51a5371346": { 2024-09-10T07:36:44.926 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:36:44.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "0860b53d-b75f-43fc-8d65-bc51a5371346", 2024-09-10T07:36:44.926 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:36:44.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:36:44.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:36:44.926 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:36:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 -- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04e40a2b80 msgr2=0x7f04e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04e40a2b80 0x7f04e40a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04f4021000 tx=0x7f04f4024000 comp rx=0 tx=0).stop 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 -- 172.21.15.46:0/3731829486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc104010 msgr2=0x7f04fc076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc104010 0x7f04fc076700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04f000ea50 tx=0x7f04f000ee50 comp rx=0 tx=0).stop 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 -- 172.21.15.46:0/3731829486 shutdown_connections 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04e40a2b80 0x7f04e40a5040 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04fc109140 0x7f04fc077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04fc1049c0 0x7f04fc076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 --2- 172.21.15.46:0/3731829486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04fc104010 0x7f04fc076700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 -- 172.21.15.46:0/3731829486 >> 172.21.15.46:0/3731829486 conn(0x7f04fc0ffa50 msgr2=0x7f04fc100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 -- 172.21.15.46:0/3731829486 shutdown_connections 2024-09-10T07:36:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:44.925+0000 7f0504600640 1 -- 172.21.15.46:0/3731829486 wait complete. 2024-09-10T07:36:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:45.897 INFO:tasks.cephfs.filesystem:scrub status for tag:0860b53d-b75f-43fc-8d65-bc51a5371346 - {'path': '/', 'tag': '0860b53d-b75f-43fc-8d65-bc51a5371346', 'options': 'recursive,force'} 2024-09-10T07:36:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.419+0000 7fbf02600640 1 -- 172.21.15.46:0/3302993675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 msgr2=0x7fbefc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.419+0000 7fbf02600640 1 --2- 172.21.15.46:0/3302993675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 0x7fbefc110cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009ca0 tx=0x7fbef402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.419+0000 7fbf02600640 1 -- 172.21.15.46:0/3302993675 shutdown_connections 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.419+0000 7fbf02600640 1 --2- 172.21.15.46:0/3302993675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 0x7fbefc110cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.419+0000 7fbf02600640 1 --2- 172.21.15.46:0/3302993675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbefc1049c0 0x7fbefc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.419+0000 7fbf02600640 1 --2- 172.21.15.46:0/3302993675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbefc104010 0x7fbefc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.419+0000 7fbf02600640 1 -- 172.21.15.46:0/3302993675 >> 172.21.15.46:0/3302993675 conn(0x7fbefc0ffa50 msgr2=0x7fbefc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.419+0000 7fbf02600640 1 -- 172.21.15.46:0/3302993675 shutdown_connections 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.420+0000 7fbf02600640 1 -- 172.21.15.46:0/3302993675 wait complete. 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbf02600640 1 Processor -- start 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbf02600640 1 -- start start 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbf02600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbefc104010 0x7fbefc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbf02600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbefc1049c0 0x7fbefc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbf02600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 0x7fbefc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbf02600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc114150 con 0x7fbefc109140 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbf02600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc113fd0 con 0x7fbefc104010 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbf02600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc1142d0 con 0x7fbefc1049c0 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbefbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 0x7fbefc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbefbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 0x7fbefc19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36922/0 (socket says 172.21.15.46:36922) 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.421+0000 7fbefbe00640 1 -- 172.21.15.46:0/1444079759 learned_addr learned my addr 172.21.15.46:0/1444079759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefb400640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbefc104010 0x7fbefc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefaa00640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbefc1049c0 0x7fbefc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefbe00640 1 -- 172.21.15.46:0/1444079759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbefc1049c0 msgr2=0x7fbefc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefbe00640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbefc1049c0 0x7fbefc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefbe00640 1 -- 172.21.15.46:0/1444079759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbefc104010 msgr2=0x7fbefc19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefbe00640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbefc104010 0x7fbefc19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefbe00640 1 -- 172.21.15.46:0/1444079759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc19f5a0 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefb400640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbefc104010 0x7fbefc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.422+0000 7fbefbe00640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 0x7fbefc19eec0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbef4031a20 tx=0x7fbef4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.423+0000 7fbef3e00640 1 -- 172.21.15.46:0/1444079759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4003c30 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.423+0000 7fbef3e00640 1 -- 172.21.15.46:0/1444079759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef4003dd0 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.423+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc19f830 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.423+0000 7fbef3e00640 1 -- 172.21.15.46:0/1444079759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef402fc40 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.423+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc1a7170 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.424+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc1043f0 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.426+0000 7fbef3e00640 1 -- 172.21.15.46:0/1444079759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbef4004050 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.428+0000 7fbef3e00640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbee00a2c00 0x7fbee00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.428+0000 7fbef3e00640 1 -- 172.21.15.46:0/1444079759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbef40f0500 con 0x7fbefc109140 2024-09-10T07:36:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.428+0000 7fbefb400640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbee00a2c00 0x7fbee00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.429+0000 7fbefb400640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbee00a2c00 0x7fbee00a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00bfd0 tx=0x7fbeec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.432+0000 7fbef3e00640 1 -- 172.21.15.46:0/1444079759 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbef40bd230 con 0x7fbefc109140 2024-09-10T07:36:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.613+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbefc062630 con 0x7fbee00a2c00 2024-09-10T07:36:46.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:46 smithi046 ceph-mon[30705]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T07:36:46.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.622+0000 7fbef3e00640 1 -- 172.21.15.46:0/1444079759 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340775 (secure 0 0 0) 0x7fbefc062630 con 0x7fbee00a2c00 2024-09-10T07:36:46.626 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:36:46.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.626+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbee00a2c00 msgr2=0x7fbee00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.626+0000 7fbf02600640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbee00a2c00 0x7fbee00a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00bfd0 tx=0x7fbeec00b390 comp rx=0 tx=0).stop 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.626+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 msgr2=0x7fbefc19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.626+0000 7fbf02600640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 0x7fbefc19eec0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbef4031a20 tx=0x7fbef4031a50 comp rx=0 tx=0).stop 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.627+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 shutdown_connections 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.627+0000 7fbf02600640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbee00a2c00 0x7fbee00a50c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.627+0000 7fbf02600640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbefc109140 0x7fbefc19eec0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.627+0000 7fbf02600640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbefc1049c0 0x7fbefc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.627+0000 7fbf02600640 1 --2- 172.21.15.46:0/1444079759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbefc104010 0x7fbefc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.627+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 >> 172.21.15.46:0/1444079759 conn(0x7fbefc0ffa50 msgr2=0x7fbefc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.627+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 shutdown_connections 2024-09-10T07:36:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:46.627+0000 7fbf02600640 1 -- 172.21.15.46:0/1444079759 wait complete. 2024-09-10T07:36:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:46 smithi181 ceph-mon[35397]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T07:36:46.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:46 smithi063 ceph-mon[35020]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T07:36:47.568 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T07:36:46.510872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668647,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1557,"num_read_kb":910,"num_write":448,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358332,"kb_used_data":13244,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583492,"statfs":{"total":1151940427776,"available":1151573495808,"internally_reserved":0,"allocated":13561856,"data_stored":8706935,"data_compressed":498372,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003140"},"pg_stats":[{"pgid":"2.1f","version":"97'14","reported_seq":197,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:42.055208+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:36:42.055208+0000","last_peered":"2024-09-10T07:36:42.055208+0000","last_clean":"2024-09-10T07:36:42.055208+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:36:42.055208+0000","last_undegraded":"2024-09-10T07:36:42.055208+0000","last_fullsized":"2024-09-10T07:36:42.055208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":139,"num_read_kb":139,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475738+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:35:42.475738+0000","last_peered":"2024-09-10T07:35:42.475738+0000","last_clean":"2024-09-10T07:35:42.475738+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:35:42.475738+0000","last_undegraded":"2024-09-10T07:35:42.475738+0000","last_fullsized":"2024-09-10T07:35:42.475738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842368+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:35:14.842368+0000","last_peered":"2024-09-10T07:35:14.842368+0000","last_clean":"2024-09-10T07:35:14.842368+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:35:14.842368+0000","last_undegraded":"2024-09-10T07:35:14.842368+0000","last_fullsized":"2024-09-10T07:35:14.842368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977747+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:35:14.977747+0000","last_peered":"2024-09-10T07:35:14.977747+0000","last_clean":"2024-09-10T07:35:14.977747+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:35:14.977747+0000","last_undegraded":"2024-09-10T07:35:14.977747+0000","last_fullsized":"2024-09-10T07:35:14.977747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502886+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:35:42.502886+0000","last_peered":"2024-09-10T07:35:42.502886+0000","last_clean":"2024-09-10T07:35:42.502886+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:35:42.502886+0000","last_undegraded":"2024-09-10T07:35:42.502886+0000","last_fullsized":"2024-09-10T07:35:42.502886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881794+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:35:14.881794+0000","last_peered":"2024-09-10T07:35:14.881794+0000","last_clean":"2024-09-10T07:35:14.881794+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:35:14.881794+0000","last_undegraded":"2024-09-10T07:35:14.881794+0000","last_fullsized":"2024-09-10T07:35:14.881794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854574+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:35:14.854574+0000","last_peered":"2024-09-10T07:35:14.854574+0000","last_clean":"2024-09-10T07:35:14.854574+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:35:14.854574+0000","last_undegraded":"2024-09-10T07:35:14.854574+0000","last_fullsized":"2024-09-10T07:35:14.854574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.495636+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:35:42.495636+0000","last_peered":"2024-09-10T07:35:42.495636+0000","last_clean":"2024-09-10T07:35:42.495636+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:35:42.495636+0000","last_undegraded":"2024-09-10T07:35:42.495636+0000","last_fullsized":"2024-09-10T07:35:42.495636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881984+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:35:14.881984+0000","last_peered":"2024-09-10T07:35:14.881984+0000","last_clean":"2024-09-10T07:35:14.881984+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:35:14.881984+0000","last_undegraded":"2024-09-10T07:35:14.881984+0000","last_fullsized":"2024-09-10T07:35:14.881984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854411+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:35:14.854411+0000","last_peered":"2024-09-10T07:35:14.854411+0000","last_clean":"2024-09-10T07:35:14.854411+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:35:14.854411+0000","last_undegraded":"2024-09-10T07:35:14.854411+0000","last_fullsized":"2024-09-10T07:35:14.854411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"97'15","reported_seq":60,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484819+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:35:42.484819+0000","last_peered":"2024-09-10T07:35:42.484819+0000","last_clean":"2024-09-10T07:35:42.484819+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:35:42.484819+0000","last_undegraded":"2024-09-10T07:35:42.484819+0000","last_fullsized":"2024-09-10T07:35:42.484819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881889+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:35:14.881889+0000","last_peered":"2024-09-10T07:35:14.881889+0000","last_clean":"2024-09-10T07:35:14.881889+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:35:14.881889+0000","last_undegraded":"2024-09-10T07:35:14.881889+0000","last_fullsized":"2024-09-10T07:35:14.881889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486825+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:35:42.486825+0000","last_peered":"2024-09-10T07:35:42.486825+0000","last_clean":"2024-09-10T07:35:42.486825+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:35:42.486825+0000","last_undegraded":"2024-09-10T07:35:42.486825+0000","last_fullsized":"2024-09-10T07:35:42.486825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854068+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:35:14.854068+0000","last_peered":"2024-09-10T07:35:14.854068+0000","last_clean":"2024-09-10T07:35:14.854068+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:35:14.854068+0000","last_undegraded":"2024-09-10T07:35:14.854068+0000","last_fullsized":"2024-09-10T07:35:14.854068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882150+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:35:14.882150+0000","last_peered":"2024-09-10T07:35:14.882150+0000","last_clean":"2024-09-10T07:35:14.882150+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:35:14.882150+0000","last_undegraded":"2024-09-10T07:35:14.882150+0000","last_fullsized":"2024-09-10T07:35:14.882150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501326+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:35:42.501326+0000","last_peered":"2024-09-10T07:35:42.501326+0000","last_clean":"2024-09-10T07:35:42.501326+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:35:42.501326+0000","last_undegraded":"2024-09-10T07:35:42.501326+0000","last_fullsized":"2024-09-10T07:35:42.501326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842483+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:35:14.842483+0000","last_peered":"2024-09-10T07:35:14.842483+0000","last_clean":"2024-09-10T07:35:14.842483+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:35:14.842483+0000","last_undegraded":"2024-09-10T07:35:14.842483+0000","last_fullsized":"2024-09-10T07:35:14.842483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882278+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:35:14.882278+0000","last_peered":"2024-09-10T07:35:14.882278+0000","last_clean":"2024-09-10T07:35:14.882278+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:35:14.882278+0000","last_undegraded":"2024-09-10T07:35:14.882278+0000","last_fullsized":"2024-09-10T07:35:14.882278+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502574+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:35:42.502574+0000","last_peered":"2024-09-10T07:35:42.502574+0000","last_clean":"2024-09-10T07:35:42.502574+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:35:42.502574+0000","last_undegraded":"2024-09-10T07:35:42.502574+0000","last_fullsized":"2024-09-10T07:35:42.502574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977257+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:35:14.977257+0000","last_peered":"2024-09-10T07:35:14.977257+0000","last_clean":"2024-09-10T07:35:14.977257+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:35:14.977257+0000","last_undegraded":"2024-09-10T07:35:14.977257+0000","last_fullsized":"2024-09-10T07:35:14.977257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851715+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:35:14.851715+0000","last_peered":"2024-09-10T07:35:14.851715+0000","last_clean":"2024-09-10T07:35:14.851715+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:35:14.851715+0000","last_undegraded":"2024-09-10T07:35:14.851715+0000","last_fullsized":"2024-09-10T07:35:14.851715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"97'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.459952+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:35:42.459952+0000","last_peered":"2024-09-10T07:35:42.459952+0000","last_clean":"2024-09-10T07:35:42.459952+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:35:42.459952+0000","last_undegraded":"2024-09-10T07:35:42.459952+0000","last_fullsized":"2024-09-10T07:35:42.459952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976752+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:35:14.976752+0000","last_peered":"2024-09-10T07:35:14.976752+0000","last_clean":"2024-09-10T07:35:14.976752+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:35:14.976752+0000","last_undegraded":"2024-09-10T07:35:14.976752+0000","last_fullsized":"2024-09-10T07:35:14.976752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882210+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:35:14.882210+0000","last_peered":"2024-09-10T07:35:14.882210+0000","last_clean":"2024-09-10T07:35:14.882210+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:35:14.882210+0000","last_undegraded":"2024-09-10T07:35:14.882210+0000","last_fullsized":"2024-09-10T07:35:14.882210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502703+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:35:42.502703+0000","last_peered":"2024-09-10T07:35:42.502703+0000","last_clean":"2024-09-10T07:35:42.502703+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:35:42.502703+0000","last_undegraded":"2024-09-10T07:35:42.502703+0000","last_fullsized":"2024-09-10T07:35:42.502703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882535+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:35:14.882535+0000","last_peered":"2024-09-10T07:35:14.882535+0000","last_clean":"2024-09-10T07:35:14.882535+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:35:14.882535+0000","last_undegraded":"2024-09-10T07:35:14.882535+0000","last_fullsized":"2024-09-10T07:35:14.882535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842382+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:35:14.842382+0000","last_peered":"2024-09-10T07:35:14.842382+0000","last_clean":"2024-09-10T07:35:14.842382+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:35:14.842382+0000","last_undegraded":"2024-09-10T07:35:14.842382+0000","last_fullsized":"2024-09-10T07:35:14.842382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499802+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:35:42.499802+0000","last_peered":"2024-09-10T07:35:42.499802+0000","last_clean":"2024-09-10T07:35:42.499802+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:35:42.499802+0000","last_undegraded":"2024-09-10T07:35:42.499802+0000","last_fullsized":"2024-09-10T07:35:42.499802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'18","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:43.451679+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:35:43.451679+0000","last_peered":"2024-09-10T07:35:43.451679+0000","last_clean":"2024-09-10T07:35:43.451679+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:35:43.451679+0000","last_undegraded":"2024-09-10T07:35:43.451679+0000","last_fullsized":"2024-09-10T07:35:43.451679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976909+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:35:14.976909+0000","last_peered":"2024-09-10T07:35:14.976909+0000","last_clean":"2024-09-10T07:35:14.976909+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:35:14.976909+0000","last_undegraded":"2024-09-10T07:35:14.976909+0000","last_fullsized":"2024-09-10T07:35:14.976909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.481577+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:35:42.481577+0000","last_peered":"2024-09-10T07:35:42.481577+0000","last_clean":"2024-09-10T07:35:42.481577+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:35:42.481577+0000","last_undegraded":"2024-09-10T07:35:42.481577+0000","last_fullsized":"2024-09-10T07:35:42.481577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.872520+0000","last_change":"2024-09-10T07:35:14.872520+0000","last_active":"2024-09-10T07:35:14.872520+0000","last_peered":"2024-09-10T07:35:14.872520+0000","last_clean":"2024-09-10T07:35:14.872520+0000","last_became_active":"2024-09-10T07:35:14.860686+0000","last_became_peered":"2024-09-10T07:35:14.860686+0000","last_unstale":"2024-09-10T07:35:14.872520+0000","last_undegraded":"2024-09-10T07:35:14.872520+0000","last_fullsized":"2024-09-10T07:35:14.872520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:42.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852355+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:35:14.852355+0000","last_peered":"2024-09-10T07:35:14.852355+0000","last_clean":"2024-09-10T07:35:14.852355+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:35:14.852355+0000","last_undegraded":"2024-09-10T07:35:14.852355+0000","last_fullsized":"2024-09-10T07:35:14.852355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"97'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489096+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:35:42.489096+0000","last_peered":"2024-09-10T07:35:42.489096+0000","last_clean":"2024-09-10T07:35:42.489096+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:35:42.489096+0000","last_undegraded":"2024-09-10T07:35:42.489096+0000","last_fullsized":"2024-09-10T07:35:42.489096+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848959+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:35:14.848959+0000","last_peered":"2024-09-10T07:35:14.848959+0000","last_clean":"2024-09-10T07:35:14.848959+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:35:14.848959+0000","last_undegraded":"2024-09-10T07:35:14.848959+0000","last_fullsized":"2024-09-10T07:35:14.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853941+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:35:14.853941+0000","last_peered":"2024-09-10T07:35:14.853941+0000","last_clean":"2024-09-10T07:35:14.853941+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:35:14.853941+0000","last_undegraded":"2024-09-10T07:35:14.853941+0000","last_fullsized":"2024-09-10T07:35:14.853941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.485460+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:35:42.485460+0000","last_peered":"2024-09-10T07:35:42.485460+0000","last_clean":"2024-09-10T07:35:42.485460+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:35:42.485460+0000","last_undegraded":"2024-09-10T07:35:42.485460+0000","last_fullsized":"2024-09-10T07:35:42.485460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"97'7","reported_seq":190,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:40.020918+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:36:40.020918+0000","last_peered":"2024-09-10T07:36:40.020918+0000","last_clean":"2024-09-10T07:36:40.020918+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:36:40.020918+0000","last_undegraded":"2024-09-10T07:36:40.020918+0000","last_fullsized":"2024-09-10T07:36:40.020918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879712+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:35:14.879712+0000","last_peered":"2024-09-10T07:35:14.879712+0000","last_clean":"2024-09-10T07:35:14.879712+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:35:14.879712+0000","last_undegraded":"2024-09-10T07:35:14.879712+0000","last_fullsized":"2024-09-10T07:35:14.879712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476516+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:35:42.476516+0000","last_peered":"2024-09-10T07:35:42.476516+0000","last_clean":"2024-09-10T07:35:42.476516+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:35:42.476516+0000","last_undegraded":"2024-09-10T07:35:42.476516+0000","last_fullsized":"2024-09-10T07:35:42.476516+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":191,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:44.000415+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:36:44.000415+0000","last_peered":"2024-09-10T07:36:44.000415+0000","last_clean":"2024-09-10T07:36:44.000415+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:36:44.000415+0000","last_undegraded":"2024-09-10T07:36:44.000415+0000","last_fullsized":"2024-09-10T07:36:44.000415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":141,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880600+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:35:14.880600+0000","last_peered":"2024-09-10T07:35:14.880600+0000","last_clean":"2024-09-10T07:35:14.880600+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:35:14.880600+0000","last_undegraded":"2024-09-10T07:35:14.880600+0000","last_fullsized":"2024-09-10T07:35:14.880600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.479337+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:35:42.479337+0000","last_peered":"2024-09-10T07:35:42.479337+0000","last_clean":"2024-09-10T07:35:42.479337+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:35:42.479337+0000","last_undegraded":"2024-09-10T07:35:42.479337+0000","last_fullsized":"2024-09-10T07:35:42.479337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842761+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:35:14.842761+0000","last_peered":"2024-09-10T07:35:14.842761+0000","last_clean":"2024-09-10T07:35:14.842761+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:35:14.842761+0000","last_undegraded":"2024-09-10T07:35:14.842761+0000","last_fullsized":"2024-09-10T07:35:14.842761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498940+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:35:42.498940+0000","last_peered":"2024-09-10T07:35:42.498940+0000","last_clean":"2024-09-10T07:35:42.498940+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:35:42.498940+0000","last_undegraded":"2024-09-10T07:35:42.498940+0000","last_fullsized":"2024-09-10T07:35:42.498940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882510+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:35:14.882510+0000","last_peered":"2024-09-10T07:35:14.882510+0000","last_clean":"2024-09-10T07:35:14.882510+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:35:14.882510+0000","last_undegraded":"2024-09-10T07:35:14.882510+0000","last_fullsized":"2024-09-10T07:35:14.882510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842861+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:35:14.842861+0000","last_peered":"2024-09-10T07:35:14.842861+0000","last_clean":"2024-09-10T07:35:14.842861+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:35:14.842861+0000","last_undegraded":"2024-09-10T07:35:14.842861+0000","last_fullsized":"2024-09-10T07:35:14.842861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843182+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:35:14.843182+0000","last_peered":"2024-09-10T07:35:14.843182+0000","last_clean":"2024-09-10T07:35:14.843182+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:35:14.843182+0000","last_undegraded":"2024-09-10T07:35:14.843182+0000","last_fullsized":"2024-09-10T07:35:14.843182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484404+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:35:42.484404+0000","last_peered":"2024-09-10T07:35:42.484404+0000","last_clean":"2024-09-10T07:35:42.484404+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:35:42.484404+0000","last_undegraded":"2024-09-10T07:35:42.484404+0000","last_fullsized":"2024-09-10T07:35:42.484404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":191,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:44.001158+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:36:44.001158+0000","last_peered":"2024-09-10T07:36:44.001158+0000","last_clean":"2024-09-10T07:36:44.001158+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:36:44.001158+0000","last_undegraded":"2024-09-10T07:36:44.001158+0000","last_fullsized":"2024-09-10T07:36:44.001158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976793+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:35:14.976793+0000","last_peered":"2024-09-10T07:35:14.976793+0000","last_clean":"2024-09-10T07:35:14.976793+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:35:14.976793+0000","last_undegraded":"2024-09-10T07:35:14.976793+0000","last_fullsized":"2024-09-10T07:35:14.976793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487151+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:35:42.487151+0000","last_peered":"2024-09-10T07:35:42.487151+0000","last_clean":"2024-09-10T07:35:42.487151+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:35:42.487151+0000","last_undegraded":"2024-09-10T07:35:42.487151+0000","last_fullsized":"2024-09-10T07:35:42.487151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880698+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:35:14.880698+0000","last_peered":"2024-09-10T07:35:14.880698+0000","last_clean":"2024-09-10T07:35:14.880698+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:35:14.880698+0000","last_undegraded":"2024-09-10T07:35:14.880698+0000","last_fullsized":"2024-09-10T07:35:14.880698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857702+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:35:14.857702+0000","last_peered":"2024-09-10T07:35:14.857702+0000","last_clean":"2024-09-10T07:35:14.857702+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:35:14.857702+0000","last_undegraded":"2024-09-10T07:35:14.857702+0000","last_fullsized":"2024-09-10T07:35:14.857702+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477302+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:35:42.477302+0000","last_peered":"2024-09-10T07:35:42.477302+0000","last_clean":"2024-09-10T07:35:42.477302+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:35:42.477302+0000","last_undegraded":"2024-09-10T07:35:42.477302+0000","last_fullsized":"2024-09-10T07:35:42.477302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880480+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:35:14.880480+0000","last_peered":"2024-09-10T07:35:14.880480+0000","last_clean":"2024-09-10T07:35:14.880480+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:35:14.880480+0000","last_undegraded":"2024-09-10T07:35:14.880480+0000","last_fullsized":"2024-09-10T07:35:14.880480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879975+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:35:14.879975+0000","last_peered":"2024-09-10T07:35:14.879975+0000","last_clean":"2024-09-10T07:35:14.879975+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:35:14.879975+0000","last_undegraded":"2024-09-10T07:35:14.879975+0000","last_fullsized":"2024-09-10T07:35:14.879975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.474216+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:35:42.474216+0000","last_peered":"2024-09-10T07:35:42.474216+0000","last_clean":"2024-09-10T07:35:42.474216+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:35:42.474216+0000","last_undegraded":"2024-09-10T07:35:42.474216+0000","last_fullsized":"2024-09-10T07:35:42.474216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854462+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:35:14.854462+0000","last_peered":"2024-09-10T07:35:14.854462+0000","last_clean":"2024-09-10T07:35:14.854462+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:35:14.854462+0000","last_undegraded":"2024-09-10T07:35:14.854462+0000","last_fullsized":"2024-09-10T07:35:14.854462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879978+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:35:14.879978+0000","last_peered":"2024-09-10T07:35:14.879978+0000","last_clean":"2024-09-10T07:35:14.879978+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:35:14.879978+0000","last_undegraded":"2024-09-10T07:35:14.879978+0000","last_fullsized":"2024-09-10T07:35:14.879978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.480953+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:35:42.480953+0000","last_peered":"2024-09-10T07:35:42.480953+0000","last_clean":"2024-09-10T07:35:42.480953+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:35:42.480953+0000","last_undegraded":"2024-09-10T07:35:42.480953+0000","last_fullsized":"2024-09-10T07:35:42.480953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853997+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:35:14.853997+0000","last_peered":"2024-09-10T07:35:14.853997+0000","last_clean":"2024-09-10T07:35:14.853997+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:35:14.853997+0000","last_undegraded":"2024-09-10T07:35:14.853997+0000","last_fullsized":"2024-09-10T07:35:14.853997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880388+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:35:14.880388+0000","last_peered":"2024-09-10T07:35:14.880388+0000","last_clean":"2024-09-10T07:35:14.880388+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:35:14.880388+0000","last_undegraded":"2024-09-10T07:35:14.880388+0000","last_fullsized":"2024-09-10T07:35:14.880388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487180+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:35:42.487180+0000","last_peered":"2024-09-10T07:35:42.487180+0000","last_clean":"2024-09-10T07:35:42.487180+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:35:42.487180+0000","last_undegraded":"2024-09-10T07:35:42.487180+0000","last_fullsized":"2024-09-10T07:35:42.487180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":190,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:44.000813+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:36:44.000813+0000","last_peered":"2024-09-10T07:36:44.000813+0000","last_clean":"2024-09-10T07:36:44.000813+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:36:44.000813+0000","last_undegraded":"2024-09-10T07:36:44.000813+0000","last_fullsized":"2024-09-10T07:36:44.000813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851402+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:35:14.851402+0000","last_peered":"2024-09-10T07:35:14.851402+0000","last_clean":"2024-09-10T07:35:14.851402+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:35:14.851402+0000","last_undegraded":"2024-09-10T07:35:14.851402+0000","last_fullsized":"2024-09-10T07:35:14.851402+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488551+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:35:42.488551+0000","last_peered":"2024-09-10T07:35:42.488551+0000","last_clean":"2024-09-10T07:35:42.488551+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:35:42.488551+0000","last_undegraded":"2024-09-10T07:35:42.488551+0000","last_fullsized":"2024-09-10T07:35:42.488551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880166+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:35:14.880166+0000","last_peered":"2024-09-10T07:35:14.880166+0000","last_clean":"2024-09-10T07:35:14.880166+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:35:14.880166+0000","last_undegraded":"2024-09-10T07:35:14.880166+0000","last_fullsized":"2024-09-10T07:35:14.880166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852262+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:35:14.852262+0000","last_peered":"2024-09-10T07:35:14.852262+0000","last_clean":"2024-09-10T07:35:14.852262+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:35:14.852262+0000","last_undegraded":"2024-09-10T07:35:14.852262+0000","last_fullsized":"2024-09-10T07:35:14.852262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500238+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:35:42.500238+0000","last_peered":"2024-09-10T07:35:42.500238+0000","last_clean":"2024-09-10T07:35:42.500238+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:35:42.500238+0000","last_undegraded":"2024-09-10T07:35:42.500238+0000","last_fullsized":"2024-09-10T07:35:42.500238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852264+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:35:14.852264+0000","last_peered":"2024-09-10T07:35:14.852264+0000","last_clean":"2024-09-10T07:35:14.852264+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:35:14.852264+0000","last_undegraded":"2024-09-10T07:35:14.852264+0000","last_fullsized":"2024-09-10T07:35:14.852264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.469244+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:35:42.469244+0000","last_peered":"2024-09-10T07:35:42.469244+0000","last_clean":"2024-09-10T07:35:42.469244+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:35:42.469244+0000","last_undegraded":"2024-09-10T07:35:42.469244+0000","last_fullsized":"2024-09-10T07:35:42.469244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"97'42","reported_seq":375,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:42.998794+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:36:42.998794+0000","last_peered":"2024-09-10T07:36:42.998794+0000","last_clean":"2024-09-10T07:36:42.998794+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:36:42.998794+0000","last_undegraded":"2024-09-10T07:36:42.998794+0000","last_fullsized":"2024-09-10T07:36:42.998794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70555,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":289,"num_read_kb":214,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848093+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:35:14.848093+0000","last_peered":"2024-09-10T07:35:14.848093+0000","last_clean":"2024-09-10T07:35:14.848093+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:35:14.848093+0000","last_undegraded":"2024-09-10T07:35:14.848093+0000","last_fullsized":"2024-09-10T07:35:14.848093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490736+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:35:42.490736+0000","last_peered":"2024-09-10T07:35:42.490736+0000","last_clean":"2024-09-10T07:35:42.490736+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:35:42.490736+0000","last_undegraded":"2024-09-10T07:35:42.490736+0000","last_fullsized":"2024-09-10T07:35:42.490736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880613+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:35:14.880613+0000","last_peered":"2024-09-10T07:35:14.880613+0000","last_clean":"2024-09-10T07:35:14.880613+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:35:14.880613+0000","last_undegraded":"2024-09-10T07:35:14.880613+0000","last_fullsized":"2024-09-10T07:35:14.880613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882346+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:35:14.882346+0000","last_peered":"2024-09-10T07:35:14.882346+0000","last_clean":"2024-09-10T07:35:14.882346+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:35:14.882346+0000","last_undegraded":"2024-09-10T07:35:14.882346+0000","last_fullsized":"2024-09-10T07:35:14.882346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502535+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:35:42.502535+0000","last_peered":"2024-09-10T07:35:42.502535+0000","last_clean":"2024-09-10T07:35:42.502535+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:35:42.502535+0000","last_undegraded":"2024-09-10T07:35:42.502535+0000","last_fullsized":"2024-09-10T07:35:42.502535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843243+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:35:14.843243+0000","last_peered":"2024-09-10T07:35:14.843243+0000","last_clean":"2024-09-10T07:35:14.843243+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:35:14.843243+0000","last_undegraded":"2024-09-10T07:35:14.843243+0000","last_fullsized":"2024-09-10T07:35:14.843243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847404+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:35:14.847404+0000","last_peered":"2024-09-10T07:35:14.847404+0000","last_clean":"2024-09-10T07:35:14.847404+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:35:14.847404+0000","last_undegraded":"2024-09-10T07:35:14.847404+0000","last_fullsized":"2024-09-10T07:35:14.847404+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490415+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:35:42.490415+0000","last_peered":"2024-09-10T07:35:42.490415+0000","last_clean":"2024-09-10T07:35:42.490415+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:35:42.490415+0000","last_undegraded":"2024-09-10T07:35:42.490415+0000","last_fullsized":"2024-09-10T07:35:42.490415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"97'5","reported_seq":325,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:42.999010+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:36:42.999010+0000","last_peered":"2024-09-10T07:36:42.999010+0000","last_clean":"2024-09-10T07:36:42.999010+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:36:42.999010+0000","last_undegraded":"2024-09-10T07:36:42.999010+0000","last_fullsized":"2024-09-10T07:36:42.999010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":276,"num_read_kb":276,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977046+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:35:14.977046+0000","last_peered":"2024-09-10T07:35:14.977046+0000","last_clean":"2024-09-10T07:35:14.977046+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:35:14.977046+0000","last_undegraded":"2024-09-10T07:35:14.977046+0000","last_fullsized":"2024-09-10T07:35:14.977046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858166+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:35:14.858166+0000","last_peered":"2024-09-10T07:35:14.858166+0000","last_clean":"2024-09-10T07:35:14.858166+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:35:14.858166+0000","last_undegraded":"2024-09-10T07:35:14.858166+0000","last_fullsized":"2024-09-10T07:35:14.858166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498588+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:35:42.498588+0000","last_peered":"2024-09-10T07:35:42.498588+0000","last_clean":"2024-09-10T07:35:42.498588+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:35:42.498588+0000","last_undegraded":"2024-09-10T07:35:42.498588+0000","last_fullsized":"2024-09-10T07:35:42.498588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880248+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:35:14.880248+0000","last_peered":"2024-09-10T07:35:14.880248+0000","last_clean":"2024-09-10T07:35:14.880248+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:35:14.880248+0000","last_undegraded":"2024-09-10T07:35:14.880248+0000","last_fullsized":"2024-09-10T07:35:14.880248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852575+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:35:14.852575+0000","last_peered":"2024-09-10T07:35:14.852575+0000","last_clean":"2024-09-10T07:35:14.852575+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:35:14.852575+0000","last_undegraded":"2024-09-10T07:35:14.852575+0000","last_fullsized":"2024-09-10T07:35:14.852575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499570+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:35:42.499570+0000","last_peered":"2024-09-10T07:35:42.499570+0000","last_clean":"2024-09-10T07:35:42.499570+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:35:42.499570+0000","last_undegraded":"2024-09-10T07:35:42.499570+0000","last_fullsized":"2024-09-10T07:35:42.499570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879193+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:35:14.879193+0000","last_peered":"2024-09-10T07:35:14.879193+0000","last_clean":"2024-09-10T07:35:14.879193+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:35:14.879193+0000","last_undegraded":"2024-09-10T07:35:14.879193+0000","last_fullsized":"2024-09-10T07:35:14.879193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877898+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:35:14.877898+0000","last_peered":"2024-09-10T07:35:14.877898+0000","last_clean":"2024-09-10T07:35:14.877898+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:35:14.877898+0000","last_undegraded":"2024-09-10T07:35:14.877898+0000","last_fullsized":"2024-09-10T07:35:14.877898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"97'8","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.492573+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:35:42.492573+0000","last_peered":"2024-09-10T07:35:42.492573+0000","last_clean":"2024-09-10T07:35:42.492573+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:35:42.492573+0000","last_undegraded":"2024-09-10T07:35:42.492573+0000","last_fullsized":"2024-09-10T07:35:42.492573+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842519+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:35:14.842519+0000","last_peered":"2024-09-10T07:35:14.842519+0000","last_clean":"2024-09-10T07:35:14.842519+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:35:14.842519+0000","last_undegraded":"2024-09-10T07:35:14.842519+0000","last_fullsized":"2024-09-10T07:35:14.842519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":195,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:44.001316+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:36:44.001316+0000","last_peered":"2024-09-10T07:36:44.001316+0000","last_clean":"2024-09-10T07:36:44.001316+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:36:44.001316+0000","last_undegraded":"2024-09-10T07:36:44.001316+0000","last_fullsized":"2024-09-10T07:36:44.001316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475213+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:35:42.475213+0000","last_peered":"2024-09-10T07:35:42.475213+0000","last_clean":"2024-09-10T07:35:42.475213+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:35:42.475213+0000","last_undegraded":"2024-09-10T07:35:42.475213+0000","last_fullsized":"2024-09-10T07:35:42.475213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842633+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:35:14.842633+0000","last_peered":"2024-09-10T07:35:14.842633+0000","last_clean":"2024-09-10T07:35:14.842633+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:35:14.842633+0000","last_undegraded":"2024-09-10T07:35:14.842633+0000","last_fullsized":"2024-09-10T07:35:14.842633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477747+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:35:42.477747+0000","last_peered":"2024-09-10T07:35:42.477747+0000","last_clean":"2024-09-10T07:35:42.477747+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:35:42.477747+0000","last_undegraded":"2024-09-10T07:35:42.477747+0000","last_fullsized":"2024-09-10T07:35:42.477747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"97'12","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476532+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:35:42.476532+0000","last_peered":"2024-09-10T07:35:42.476532+0000","last_clean":"2024-09-10T07:35:42.476532+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:35:42.476532+0000","last_undegraded":"2024-09-10T07:35:42.476532+0000","last_fullsized":"2024-09-10T07:35:42.476532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486476+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:35:42.486476+0000","last_peered":"2024-09-10T07:35:42.486476+0000","last_clean":"2024-09-10T07:35:42.486476+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:35:42.486476+0000","last_undegraded":"2024-09-10T07:35:42.486476+0000","last_fullsized":"2024-09-10T07:35:42.486476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499618+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:35:42.499618+0000","last_peered":"2024-09-10T07:35:42.499618+0000","last_clean":"2024-09-10T07:35:42.499618+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:35:42.499618+0000","last_undegraded":"2024-09-10T07:35:42.499618+0000","last_fullsized":"2024-09-10T07:35:42.499618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487143+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:35:42.487143+0000","last_peered":"2024-09-10T07:35:42.487143+0000","last_clean":"2024-09-10T07:35:42.487143+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:35:42.487143+0000","last_undegraded":"2024-09-10T07:35:42.487143+0000","last_fullsized":"2024-09-10T07:35:42.487143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.470371+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:35:42.470371+0000","last_peered":"2024-09-10T07:35:42.470371+0000","last_clean":"2024-09-10T07:35:42.470371+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:35:42.470371+0000","last_undegraded":"2024-09-10T07:35:42.470371+0000","last_fullsized":"2024-09-10T07:35:42.470371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476911+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:35:42.476911+0000","last_peered":"2024-09-10T07:35:42.476911+0000","last_clean":"2024-09-10T07:35:42.476911+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:35:42.476911+0000","last_undegraded":"2024-09-10T07:35:42.476911+0000","last_fullsized":"2024-09-10T07:35:42.476911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488157+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:35:42.488157+0000","last_peered":"2024-09-10T07:35:42.488157+0000","last_clean":"2024-09-10T07:35:42.488157+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:35:42.488157+0000","last_undegraded":"2024-09-10T07:35:42.488157+0000","last_fullsized":"2024-09-10T07:35:42.488157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500450+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:35:42.500450+0000","last_peered":"2024-09-10T07:35:42.500450+0000","last_clean":"2024-09-10T07:35:42.500450+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:35:42.500450+0000","last_undegraded":"2024-09-10T07:35:42.500450+0000","last_fullsized":"2024-09-10T07:35:42.500450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499239+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:35:42.499239+0000","last_peered":"2024-09-10T07:35:42.499239+0000","last_clean":"2024-09-10T07:35:42.499239+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:35:42.499239+0000","last_undegraded":"2024-09-10T07:35:42.499239+0000","last_fullsized":"2024-09-10T07:35:42.499239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501908+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:35:42.501908+0000","last_peered":"2024-09-10T07:35:42.501908+0000","last_clean":"2024-09-10T07:35:42.501908+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:35:42.501908+0000","last_undegraded":"2024-09-10T07:35:42.501908+0000","last_fullsized":"2024-09-10T07:35:42.501908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493731+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:35:42.493731+0000","last_peered":"2024-09-10T07:35:42.493731+0000","last_clean":"2024-09-10T07:35:42.493731+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:35:42.493731+0000","last_undegraded":"2024-09-10T07:35:42.493731+0000","last_fullsized":"2024-09-10T07:35:42.493731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478046+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:35:42.478046+0000","last_peered":"2024-09-10T07:35:42.478046+0000","last_clean":"2024-09-10T07:35:42.478046+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:35:42.478046+0000","last_undegraded":"2024-09-10T07:35:42.478046+0000","last_fullsized":"2024-09-10T07:35:42.478046+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493144+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:35:42.493144+0000","last_peered":"2024-09-10T07:35:42.493144+0000","last_clean":"2024-09-10T07:35:42.493144+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:35:42.493144+0000","last_undegraded":"2024-09-10T07:35:42.493144+0000","last_fullsized":"2024-09-10T07:35:42.493144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483119+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:35:42.483119+0000","last_peered":"2024-09-10T07:35:42.483119+0000","last_clean":"2024-09-10T07:35:42.483119+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:35:42.483119+0000","last_undegraded":"2024-09-10T07:35:42.483119+0000","last_fullsized":"2024-09-10T07:35:42.483119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502676+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:35:42.502676+0000","last_peered":"2024-09-10T07:35:42.502676+0000","last_clean":"2024-09-10T07:35:42.502676+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:35:42.502676+0000","last_undegraded":"2024-09-10T07:35:42.502676+0000","last_fullsized":"2024-09-10T07:35:42.502676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502776+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:35:42.502776+0000","last_peered":"2024-09-10T07:35:42.502776+0000","last_clean":"2024-09-10T07:35:42.502776+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:35:42.502776+0000","last_undegraded":"2024-09-10T07:35:42.502776+0000","last_fullsized":"2024-09-10T07:35:42.502776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499409+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:35:42.499409+0000","last_peered":"2024-09-10T07:35:42.499409+0000","last_clean":"2024-09-10T07:35:42.499409+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:35:42.499409+0000","last_undegraded":"2024-09-10T07:35:42.499409+0000","last_fullsized":"2024-09-10T07:35:42.499409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502975+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:35:42.502975+0000","last_peered":"2024-09-10T07:35:42.502975+0000","last_clean":"2024-09-10T07:35:42.502975+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:35:42.502975+0000","last_undegraded":"2024-09-10T07:35:42.502975+0000","last_fullsized":"2024-09-10T07:35:42.502975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493842+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:35:42.493842+0000","last_peered":"2024-09-10T07:35:42.493842+0000","last_clean":"2024-09-10T07:35:42.493842+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:35:42.493842+0000","last_undegraded":"2024-09-10T07:35:42.493842+0000","last_fullsized":"2024-09-10T07:35:42.493842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484180+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:35:42.484180+0000","last_peered":"2024-09-10T07:35:42.484180+0000","last_clean":"2024-09-10T07:35:42.484180+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:35:42.484180+0000","last_undegraded":"2024-09-10T07:35:42.484180+0000","last_fullsized":"2024-09-10T07:35:42.484180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483929+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:35:42.483929+0000","last_peered":"2024-09-10T07:35:42.483929+0000","last_clean":"2024-09-10T07:35:42.483929+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:35:42.483929+0000","last_undegraded":"2024-09-10T07:35:42.483929+0000","last_fullsized":"2024-09-10T07:35:42.483929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"97'16","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483320+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:35:42.483320+0000","last_peered":"2024-09-10T07:35:42.483320+0000","last_clean":"2024-09-10T07:35:42.483320+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:35:42.483320+0000","last_undegraded":"2024-09-10T07:35:42.483320+0000","last_fullsized":"2024-09-10T07:35:42.483320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489981+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:35:42.489981+0000","last_peered":"2024-09-10T07:35:42.489981+0000","last_clean":"2024-09-10T07:35:42.489981+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:35:42.489981+0000","last_undegraded":"2024-09-10T07:35:42.489981+0000","last_fullsized":"2024-09-10T07:35:42.489981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500857+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:35:42.500857+0000","last_peered":"2024-09-10T07:35:42.500857+0000","last_clean":"2024-09-10T07:35:42.500857+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:35:42.500857+0000","last_undegraded":"2024-09-10T07:35:42.500857+0000","last_fullsized":"2024-09-10T07:35:42.500857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478497+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:35:42.478497+0000","last_peered":"2024-09-10T07:35:42.478497+0000","last_clean":"2024-09-10T07:35:42.478497+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:35:42.478497+0000","last_undegraded":"2024-09-10T07:35:42.478497+0000","last_fullsized":"2024-09-10T07:35:42.478497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487137+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:35:42.487137+0000","last_peered":"2024-09-10T07:35:42.487137+0000","last_clean":"2024-09-10T07:35:42.487137+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:35:42.487137+0000","last_undegraded":"2024-09-10T07:35:42.487137+0000","last_fullsized":"2024-09-10T07:35:42.487137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"97'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.496842+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:35:42.496842+0000","last_peered":"2024-09-10T07:35:42.496842+0000","last_clean":"2024-09-10T07:35:42.496842+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:35:42.496842+0000","last_undegraded":"2024-09-10T07:35:42.496842+0000","last_fullsized":"2024-09-10T07:35:42.496842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"97'8","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476981+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:35:42.476981+0000","last_peered":"2024-09-10T07:35:42.476981+0000","last_clean":"2024-09-10T07:35:42.476981+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:35:42.476981+0000","last_undegraded":"2024-09-10T07:35:42.476981+0000","last_fullsized":"2024-09-10T07:35:42.476981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.494172+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:35:42.494172+0000","last_peered":"2024-09-10T07:35:42.494172+0000","last_clean":"2024-09-10T07:35:42.494172+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:35:42.494172+0000","last_undegraded":"2024-09-10T07:35:42.494172+0000","last_fullsized":"2024-09-10T07:35:42.494172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502201+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:35:42.502201+0000","last_peered":"2024-09-10T07:35:42.502201+0000","last_clean":"2024-09-10T07:35:42.502201+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:35:42.502201+0000","last_undegraded":"2024-09-10T07:35:42.502201+0000","last_fullsized":"2024-09-10T07:35:42.502201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502439+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:35:42.502439+0000","last_peered":"2024-09-10T07:35:42.502439+0000","last_clean":"2024-09-10T07:35:42.502439+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:35:42.502439+0000","last_undegraded":"2024-09-10T07:35:42.502439+0000","last_fullsized":"2024-09-10T07:35:42.502439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78045,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1431,"num_read_kb":801,"num_write":265,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":234135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449141,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":561212,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710778,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1241088,"data_stored":1218403,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874301,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":565058,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":561228,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364872,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1158514,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528399,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":557000,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790038,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1153562,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953562,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":560540,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":556174,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1171456,"data_stored":628737,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":628585,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":557922,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":72993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:36:47.573 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T07:36:47.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T07:36:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 -- 172.21.15.46:0/4189591165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 msgr2=0x7fc1401043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 --2- 172.21.15.46:0/4189591165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 0x7fc1401043d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc134009900 tx=0x7fc13402a410 comp rx=0 tx=0).stop 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 -- 172.21.15.46:0/4189591165 shutdown_connections 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 --2- 172.21.15.46:0/4189591165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc140109120 0x7fc140110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 --2- 172.21.15.46:0/4189591165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1401049a0 0x7fc1401089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 --2- 172.21.15.46:0/4189591165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 0x7fc1401043d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 -- 172.21.15.46:0/4189591165 >> 172.21.15.46:0/4189591165 conn(0x7fc1400ffa90 msgr2=0x7fc140101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 -- 172.21.15.46:0/4189591165 shutdown_connections 2024-09-10T07:36:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.544+0000 7fc147600640 1 -- 172.21.15.46:0/4189591165 wait complete. 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.545+0000 7fc147600640 1 Processor -- start 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.545+0000 7fc147600640 1 -- start start 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.545+0000 7fc147600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 0x7fc14019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.545+0000 7fc147600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1401049a0 0x7fc14019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.545+0000 7fc147600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc140109120 0x7fc14019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.545+0000 7fc147600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc140114150 con 0x7fc140103ff0 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.545+0000 7fc147600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc140113fd0 con 0x7fc1401049a0 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.545+0000 7fc147600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401142d0 con 0x7fc140109120 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc144c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 0x7fc14019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1401049a0 0x7fc14019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc144c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 0x7fc14019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36944/0 (socket says 172.21.15.46:36944) 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1401049a0 0x7fc14019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36088/0 (socket says 172.21.15.46:36088) 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc144c00640 1 -- 172.21.15.46:0/3878467924 learned_addr learned my addr 172.21.15.46:0/3878467924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc145600640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc140109120 0x7fc14019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc144c00640 1 -- 172.21.15.46:0/3878467924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc140109120 msgr2=0x7fc14019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc144c00640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc140109120 0x7fc14019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc144c00640 1 -- 172.21.15.46:0/3878467924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1401049a0 msgr2=0x7fc14019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc144c00640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1401049a0 0x7fc14019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc144c00640 1 -- 172.21.15.46:0/3878467924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14019f5a0 con 0x7fc140103ff0 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.546+0000 7fc13fe00640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1401049a0 0x7fc14019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.547+0000 7fc144c00640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 0x7fc14019a5f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc134006010 tx=0x7fc134004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.547+0000 7fc13d600640 1 -- 172.21.15.46:0/3878467924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13403f030 con 0x7fc140103ff0 2024-09-10T07:36:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.547+0000 7fc13d600640 1 -- 172.21.15.46:0/3878467924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc134004440 con 0x7fc140103ff0 2024-09-10T07:36:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.547+0000 7fc13d600640 1 -- 172.21.15.46:0/3878467924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134035070 con 0x7fc140103ff0 2024-09-10T07:36:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.547+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14019f830 con 0x7fc140103ff0 2024-09-10T07:36:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.547+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14019fcc0 con 0x7fc140103ff0 2024-09-10T07:36:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.548+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1401043d0 con 0x7fc140103ff0 2024-09-10T07:36:48.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.550+0000 7fc13d600640 1 -- 172.21.15.46:0/3878467924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc134042030 con 0x7fc140103ff0 2024-09-10T07:36:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.551+0000 7fc13d600640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc1280a2c00 0x7fc1280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.552+0000 7fc13d600640 1 -- 172.21.15.46:0/3878467924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc1340ef6e0 con 0x7fc140103ff0 2024-09-10T07:36:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.552+0000 7fc13fe00640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc1280a2c00 0x7fc1280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.552+0000 7fc13fe00640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc1280a2c00 0x7fc1280a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1400ffeb0 tx=0x7fc12c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:48.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.555+0000 7fc13d600640 1 -- 172.21.15.46:0/3878467924 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1340bbcd0 con 0x7fc140103ff0 2024-09-10T07:36:48.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:48 smithi181 ceph-mon[35397]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-10T07:36:48.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:48 smithi181 ceph-mon[35397]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:48.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:48 smithi063 ceph-mon[35020]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-10T07:36:48.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:48 smithi063 ceph-mon[35020]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:48.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:48 smithi046 ceph-mon[30705]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-10T07:36:48.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:48 smithi046 ceph-mon[30705]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.739+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 --> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc140062630 con 0x7fc1280a2c00 2024-09-10T07:36:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.747+0000 7fc13d600640 1 -- 172.21.15.46:0/3878467924 <== mgr.14150 v2:172.21.15.46:6800/1492955812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340775 (secure 0 0 0) 0x7fc140062630 con 0x7fc1280a2c00 2024-09-10T07:36:48.751 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc1280a2c00 msgr2=0x7fc1280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc1280a2c00 0x7fc1280a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1400ffeb0 tx=0x7fc12c005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 msgr2=0x7fc14019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 0x7fc14019a5f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc134006010 tx=0x7fc134004040 comp rx=0 tx=0).stop 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 shutdown_connections 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc140109120 0x7fc14019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1401049a0 0x7fc14019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc1280a2c00 0x7fc1280a50c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 --2- 172.21.15.46:0/3878467924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140103ff0 0x7fc14019a5f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 >> 172.21.15.46:0/3878467924 conn(0x7fc1400ffa90 msgr2=0x7fc140100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 shutdown_connections 2024-09-10T07:36:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:48.752+0000 7fc147600640 1 -- 172.21.15.46:0/3878467924 wait complete. 2024-09-10T07:36:49.567 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-10T07:36:48.511421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668647,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1567,"num_read_kb":915,"num_write":448,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180882,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358332,"kb_used_data":13244,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583492,"statfs":{"total":1151940427776,"available":1151573495808,"internally_reserved":0,"allocated":13561856,"data_stored":8706935,"data_compressed":498372,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":57,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003244"},"pg_stats":[{"pgid":"2.1f","version":"97'14","reported_seq":202,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:47.031066+0000","last_change":"2024-09-10T07:33:31.846056+0000","last_active":"2024-09-10T07:36:47.031066+0000","last_peered":"2024-09-10T07:36:47.031066+0000","last_clean":"2024-09-10T07:36:47.031066+0000","last_became_active":"2024-09-10T07:33:31.845675+0000","last_became_peered":"2024-09-10T07:33:31.845675+0000","last_unstale":"2024-09-10T07:36:47.031066+0000","last_undegraded":"2024-09-10T07:36:47.031066+0000","last_fullsized":"2024-09-10T07:36:47.031066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:04.988130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475738+0000","last_change":"2024-09-10T07:33:34.883206+0000","last_active":"2024-09-10T07:35:42.475738+0000","last_peered":"2024-09-10T07:35:42.475738+0000","last_clean":"2024-09-10T07:35:42.475738+0000","last_became_active":"2024-09-10T07:33:34.879712+0000","last_became_peered":"2024-09-10T07:33:34.879712+0000","last_unstale":"2024-09-10T07:35:42.475738+0000","last_undegraded":"2024-09-10T07:35:42.475738+0000","last_fullsized":"2024-09-10T07:35:42.475738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:03:54.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842368+0000","last_change":"2024-09-10T07:33:44.929277+0000","last_active":"2024-09-10T07:35:14.842368+0000","last_peered":"2024-09-10T07:35:14.842368+0000","last_clean":"2024-09-10T07:35:14.842368+0000","last_became_active":"2024-09-10T07:33:44.928809+0000","last_became_peered":"2024-09-10T07:33:44.928809+0000","last_unstale":"2024-09-10T07:35:14.842368+0000","last_undegraded":"2024-09-10T07:35:14.842368+0000","last_fullsized":"2024-09-10T07:35:14.842368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:13.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977747+0000","last_change":"2024-09-10T07:33:31.998618+0000","last_active":"2024-09-10T07:35:14.977747+0000","last_peered":"2024-09-10T07:35:14.977747+0000","last_clean":"2024-09-10T07:35:14.977747+0000","last_became_active":"2024-09-10T07:33:31.998275+0000","last_became_peered":"2024-09-10T07:33:31.998275+0000","last_unstale":"2024-09-10T07:35:14.977747+0000","last_undegraded":"2024-09-10T07:35:14.977747+0000","last_fullsized":"2024-09-10T07:35:14.977747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:24.086750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502886+0000","last_change":"2024-09-10T07:33:34.889222+0000","last_active":"2024-09-10T07:35:42.502886+0000","last_peered":"2024-09-10T07:35:42.502886+0000","last_clean":"2024-09-10T07:35:42.502886+0000","last_became_active":"2024-09-10T07:33:34.888888+0000","last_became_peered":"2024-09-10T07:33:34.888888+0000","last_unstale":"2024-09-10T07:35:42.502886+0000","last_undegraded":"2024-09-10T07:35:42.502886+0000","last_fullsized":"2024-09-10T07:35:42.502886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:38.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881794+0000","last_change":"2024-09-10T07:33:44.924207+0000","last_active":"2024-09-10T07:35:14.881794+0000","last_peered":"2024-09-10T07:35:14.881794+0000","last_clean":"2024-09-10T07:35:14.881794+0000","last_became_active":"2024-09-10T07:33:44.923827+0000","last_became_peered":"2024-09-10T07:33:44.923827+0000","last_unstale":"2024-09-10T07:35:14.881794+0000","last_undegraded":"2024-09-10T07:35:14.881794+0000","last_fullsized":"2024-09-10T07:35:14.881794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:01.543873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'12","reported_seq":57,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854574+0000","last_change":"2024-09-10T07:33:31.846414+0000","last_active":"2024-09-10T07:35:14.854574+0000","last_peered":"2024-09-10T07:35:14.854574+0000","last_clean":"2024-09-10T07:35:14.854574+0000","last_became_active":"2024-09-10T07:33:31.845969+0000","last_became_peered":"2024-09-10T07:33:31.845969+0000","last_unstale":"2024-09-10T07:35:14.854574+0000","last_undegraded":"2024-09-10T07:35:14.854574+0000","last_fullsized":"2024-09-10T07:35:14.854574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:04.308559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.495636+0000","last_change":"2024-09-10T07:33:34.875714+0000","last_active":"2024-09-10T07:35:42.495636+0000","last_peered":"2024-09-10T07:35:42.495636+0000","last_clean":"2024-09-10T07:35:42.495636+0000","last_became_active":"2024-09-10T07:33:34.875329+0000","last_became_peered":"2024-09-10T07:33:34.875329+0000","last_unstale":"2024-09-10T07:35:42.495636+0000","last_undegraded":"2024-09-10T07:35:42.495636+0000","last_fullsized":"2024-09-10T07:35:42.495636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:29:41.588407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881984+0000","last_change":"2024-09-10T07:33:44.925336+0000","last_active":"2024-09-10T07:35:14.881984+0000","last_peered":"2024-09-10T07:35:14.881984+0000","last_clean":"2024-09-10T07:35:14.881984+0000","last_became_active":"2024-09-10T07:33:44.924761+0000","last_became_peered":"2024-09-10T07:33:44.924761+0000","last_unstale":"2024-09-10T07:35:14.881984+0000","last_undegraded":"2024-09-10T07:35:14.881984+0000","last_fullsized":"2024-09-10T07:35:14.881984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:22.424724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854411+0000","last_change":"2024-09-10T07:33:31.841971+0000","last_active":"2024-09-10T07:35:14.854411+0000","last_peered":"2024-09-10T07:35:14.854411+0000","last_clean":"2024-09-10T07:35:14.854411+0000","last_became_active":"2024-09-10T07:33:31.841427+0000","last_became_peered":"2024-09-10T07:33:31.841427+0000","last_unstale":"2024-09-10T07:35:14.854411+0000","last_undegraded":"2024-09-10T07:35:14.854411+0000","last_fullsized":"2024-09-10T07:35:14.854411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:30:56.052178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"97'15","reported_seq":60,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484819+0000","last_change":"2024-09-10T07:34:14.156901+0000","last_active":"2024-09-10T07:35:42.484819+0000","last_peered":"2024-09-10T07:35:42.484819+0000","last_clean":"2024-09-10T07:35:42.484819+0000","last_became_active":"2024-09-10T07:34:14.156344+0000","last_became_peered":"2024-09-10T07:34:14.156344+0000","last_unstale":"2024-09-10T07:35:42.484819+0000","last_undegraded":"2024-09-10T07:35:42.484819+0000","last_fullsized":"2024-09-10T07:35:42.484819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.881889+0000","last_change":"2024-09-10T07:33:44.914826+0000","last_active":"2024-09-10T07:35:14.881889+0000","last_peered":"2024-09-10T07:35:14.881889+0000","last_clean":"2024-09-10T07:35:14.881889+0000","last_became_active":"2024-09-10T07:33:44.914349+0000","last_became_peered":"2024-09-10T07:33:44.914349+0000","last_unstale":"2024-09-10T07:35:14.881889+0000","last_undegraded":"2024-09-10T07:35:14.881889+0000","last_fullsized":"2024-09-10T07:35:14.881889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:14.026970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486825+0000","last_change":"2024-09-10T07:33:34.877804+0000","last_active":"2024-09-10T07:35:42.486825+0000","last_peered":"2024-09-10T07:35:42.486825+0000","last_clean":"2024-09-10T07:35:42.486825+0000","last_became_active":"2024-09-10T07:33:34.877465+0000","last_became_peered":"2024-09-10T07:33:34.877465+0000","last_unstale":"2024-09-10T07:35:42.486825+0000","last_undegraded":"2024-09-10T07:35:42.486825+0000","last_fullsized":"2024-09-10T07:35:42.486825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:26.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854068+0000","last_change":"2024-09-10T07:33:31.844942+0000","last_active":"2024-09-10T07:35:14.854068+0000","last_peered":"2024-09-10T07:35:14.854068+0000","last_clean":"2024-09-10T07:35:14.854068+0000","last_became_active":"2024-09-10T07:33:31.844615+0000","last_became_peered":"2024-09-10T07:33:31.844615+0000","last_unstale":"2024-09-10T07:35:14.854068+0000","last_undegraded":"2024-09-10T07:35:14.854068+0000","last_fullsized":"2024-09-10T07:35:14.854068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:06.668332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882150+0000","last_change":"2024-09-10T07:33:45.072044+0000","last_active":"2024-09-10T07:35:14.882150+0000","last_peered":"2024-09-10T07:35:14.882150+0000","last_clean":"2024-09-10T07:35:14.882150+0000","last_became_active":"2024-09-10T07:33:45.071605+0000","last_became_peered":"2024-09-10T07:33:45.071605+0000","last_unstale":"2024-09-10T07:35:14.882150+0000","last_undegraded":"2024-09-10T07:35:14.882150+0000","last_fullsized":"2024-09-10T07:35:14.882150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:56:01.135573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501326+0000","last_change":"2024-09-10T07:33:34.878685+0000","last_active":"2024-09-10T07:35:42.501326+0000","last_peered":"2024-09-10T07:35:42.501326+0000","last_clean":"2024-09-10T07:35:42.501326+0000","last_became_active":"2024-09-10T07:33:34.878305+0000","last_became_peered":"2024-09-10T07:33:34.878305+0000","last_unstale":"2024-09-10T07:35:42.501326+0000","last_undegraded":"2024-09-10T07:35:42.501326+0000","last_fullsized":"2024-09-10T07:35:42.501326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:41.767117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'3","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842483+0000","last_change":"2024-09-10T07:33:31.843349+0000","last_active":"2024-09-10T07:35:14.842483+0000","last_peered":"2024-09-10T07:35:14.842483+0000","last_clean":"2024-09-10T07:35:14.842483+0000","last_became_active":"2024-09-10T07:33:31.842991+0000","last_became_peered":"2024-09-10T07:33:31.842991+0000","last_unstale":"2024-09-10T07:35:14.842483+0000","last_undegraded":"2024-09-10T07:35:14.842483+0000","last_fullsized":"2024-09-10T07:35:14.842483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:48.396313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"95'1","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882278+0000","last_change":"2024-09-10T07:33:44.925710+0000","last_active":"2024-09-10T07:35:14.882278+0000","last_peered":"2024-09-10T07:35:14.882278+0000","last_clean":"2024-09-10T07:35:14.882278+0000","last_became_active":"2024-09-10T07:33:44.924669+0000","last_became_peered":"2024-09-10T07:33:44.924669+0000","last_unstale":"2024-09-10T07:35:14.882278+0000","last_undegraded":"2024-09-10T07:35:14.882278+0000","last_fullsized":"2024-09-10T07:35:14.882278+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:14.760421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502574+0000","last_change":"2024-09-10T07:33:34.873938+0000","last_active":"2024-09-10T07:35:42.502574+0000","last_peered":"2024-09-10T07:35:42.502574+0000","last_clean":"2024-09-10T07:35:42.502574+0000","last_became_active":"2024-09-10T07:33:34.873525+0000","last_became_peered":"2024-09-10T07:33:34.873525+0000","last_unstale":"2024-09-10T07:35:42.502574+0000","last_undegraded":"2024-09-10T07:35:42.502574+0000","last_fullsized":"2024-09-10T07:35:42.502574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.671224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977257+0000","last_change":"2024-09-10T07:33:31.843023+0000","last_active":"2024-09-10T07:35:14.977257+0000","last_peered":"2024-09-10T07:35:14.977257+0000","last_clean":"2024-09-10T07:35:14.977257+0000","last_became_active":"2024-09-10T07:33:31.842721+0000","last_became_peered":"2024-09-10T07:33:31.842721+0000","last_unstale":"2024-09-10T07:35:14.977257+0000","last_undegraded":"2024-09-10T07:35:14.977257+0000","last_fullsized":"2024-09-10T07:35:14.977257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:21.922043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851715+0000","last_change":"2024-09-10T07:33:44.932813+0000","last_active":"2024-09-10T07:35:14.851715+0000","last_peered":"2024-09-10T07:35:14.851715+0000","last_clean":"2024-09-10T07:35:14.851715+0000","last_became_active":"2024-09-10T07:33:44.932131+0000","last_became_peered":"2024-09-10T07:33:44.932131+0000","last_unstale":"2024-09-10T07:35:14.851715+0000","last_undegraded":"2024-09-10T07:35:14.851715+0000","last_fullsized":"2024-09-10T07:35:14.851715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:00.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"97'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.459952+0000","last_change":"2024-09-10T07:33:34.870912+0000","last_active":"2024-09-10T07:35:42.459952+0000","last_peered":"2024-09-10T07:35:42.459952+0000","last_clean":"2024-09-10T07:35:42.459952+0000","last_became_active":"2024-09-10T07:33:34.870561+0000","last_became_peered":"2024-09-10T07:33:34.870561+0000","last_unstale":"2024-09-10T07:35:42.459952+0000","last_undegraded":"2024-09-10T07:35:42.459952+0000","last_fullsized":"2024-09-10T07:35:42.459952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:47:15.886542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"95'2","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976752+0000","last_change":"2024-09-10T07:33:31.998053+0000","last_active":"2024-09-10T07:35:14.976752+0000","last_peered":"2024-09-10T07:35:14.976752+0000","last_clean":"2024-09-10T07:35:14.976752+0000","last_became_active":"2024-09-10T07:33:31.997720+0000","last_became_peered":"2024-09-10T07:33:31.997720+0000","last_unstale":"2024-09-10T07:35:14.976752+0000","last_undegraded":"2024-09-10T07:35:14.976752+0000","last_fullsized":"2024-09-10T07:35:14.976752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:43.611521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882210+0000","last_change":"2024-09-10T07:34:14.022602+0000","last_active":"2024-09-10T07:35:14.882210+0000","last_peered":"2024-09-10T07:35:14.882210+0000","last_clean":"2024-09-10T07:35:14.882210+0000","last_became_active":"2024-09-10T07:34:14.022012+0000","last_became_peered":"2024-09-10T07:34:14.022012+0000","last_unstale":"2024-09-10T07:35:14.882210+0000","last_undegraded":"2024-09-10T07:35:14.882210+0000","last_fullsized":"2024-09-10T07:35:14.882210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:19.681406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502703+0000","last_change":"2024-09-10T07:33:34.883862+0000","last_active":"2024-09-10T07:35:42.502703+0000","last_peered":"2024-09-10T07:35:42.502703+0000","last_clean":"2024-09-10T07:35:42.502703+0000","last_became_active":"2024-09-10T07:33:34.882035+0000","last_became_peered":"2024-09-10T07:33:34.882035+0000","last_unstale":"2024-09-10T07:35:42.502703+0000","last_undegraded":"2024-09-10T07:35:42.502703+0000","last_fullsized":"2024-09-10T07:35:42.502703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:19.149414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882535+0000","last_change":"2024-09-10T07:33:31.839319+0000","last_active":"2024-09-10T07:35:14.882535+0000","last_peered":"2024-09-10T07:35:14.882535+0000","last_clean":"2024-09-10T07:35:14.882535+0000","last_became_active":"2024-09-10T07:33:31.838912+0000","last_became_peered":"2024-09-10T07:33:31.838912+0000","last_unstale":"2024-09-10T07:35:14.882535+0000","last_undegraded":"2024-09-10T07:35:14.882535+0000","last_fullsized":"2024-09-10T07:35:14.882535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:40.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842382+0000","last_change":"2024-09-10T07:33:44.913392+0000","last_active":"2024-09-10T07:35:14.842382+0000","last_peered":"2024-09-10T07:35:14.842382+0000","last_clean":"2024-09-10T07:35:14.842382+0000","last_became_active":"2024-09-10T07:33:44.912170+0000","last_became_peered":"2024-09-10T07:33:44.912170+0000","last_unstale":"2024-09-10T07:35:14.842382+0000","last_undegraded":"2024-09-10T07:35:14.842382+0000","last_fullsized":"2024-09-10T07:35:14.842382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:14.641752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499802+0000","last_change":"2024-09-10T07:33:34.885990+0000","last_active":"2024-09-10T07:35:42.499802+0000","last_peered":"2024-09-10T07:35:42.499802+0000","last_clean":"2024-09-10T07:35:42.499802+0000","last_became_active":"2024-09-10T07:33:34.883433+0000","last_became_peered":"2024-09-10T07:33:34.883433+0000","last_unstale":"2024-09-10T07:35:42.499802+0000","last_undegraded":"2024-09-10T07:35:42.499802+0000","last_fullsized":"2024-09-10T07:35:42.499802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:39.292062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'18","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:43.451679+0000","last_change":"2024-09-10T07:33:31.842990+0000","last_active":"2024-09-10T07:35:43.451679+0000","last_peered":"2024-09-10T07:35:43.451679+0000","last_clean":"2024-09-10T07:35:43.451679+0000","last_became_active":"2024-09-10T07:33:31.842427+0000","last_became_peered":"2024-09-10T07:33:31.842427+0000","last_unstale":"2024-09-10T07:35:43.451679+0000","last_undegraded":"2024-09-10T07:35:43.451679+0000","last_fullsized":"2024-09-10T07:35:43.451679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:25.291403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976909+0000","last_change":"2024-09-10T07:33:45.072357+0000","last_active":"2024-09-10T07:35:14.976909+0000","last_peered":"2024-09-10T07:35:14.976909+0000","last_clean":"2024-09-10T07:35:14.976909+0000","last_became_active":"2024-09-10T07:33:45.071982+0000","last_became_peered":"2024-09-10T07:33:45.071982+0000","last_unstale":"2024-09-10T07:35:14.976909+0000","last_undegraded":"2024-09-10T07:35:14.976909+0000","last_fullsized":"2024-09-10T07:35:14.976909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:54.785001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.481577+0000","last_change":"2024-09-10T07:33:34.871890+0000","last_active":"2024-09-10T07:35:42.481577+0000","last_peered":"2024-09-10T07:35:42.481577+0000","last_clean":"2024-09-10T07:35:42.481577+0000","last_became_active":"2024-09-10T07:33:34.870538+0000","last_became_peered":"2024-09-10T07:33:34.870538+0000","last_unstale":"2024-09-10T07:35:42.481577+0000","last_undegraded":"2024-09-10T07:35:42.481577+0000","last_fullsized":"2024-09-10T07:35:42.481577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:49.552928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"95'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.872520+0000","last_change":"2024-09-10T07:35:14.872520+0000","last_active":"2024-09-10T07:35:14.872520+0000","last_peered":"2024-09-10T07:35:14.872520+0000","last_clean":"2024-09-10T07:35:14.872520+0000","last_became_active":"2024-09-10T07:35:14.860686+0000","last_became_peered":"2024-09-10T07:35:14.860686+0000","last_unstale":"2024-09-10T07:35:14.872520+0000","last_undegraded":"2024-09-10T07:35:14.872520+0000","last_fullsized":"2024-09-10T07:35:14.872520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:42.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852355+0000","last_change":"2024-09-10T07:33:44.917619+0000","last_active":"2024-09-10T07:35:14.852355+0000","last_peered":"2024-09-10T07:35:14.852355+0000","last_clean":"2024-09-10T07:35:14.852355+0000","last_became_active":"2024-09-10T07:33:44.917191+0000","last_became_peered":"2024-09-10T07:33:44.917191+0000","last_unstale":"2024-09-10T07:35:14.852355+0000","last_undegraded":"2024-09-10T07:35:14.852355+0000","last_fullsized":"2024-09-10T07:35:14.852355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:56.139200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"97'4","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489096+0000","last_change":"2024-09-10T07:34:14.157371+0000","last_active":"2024-09-10T07:35:42.489096+0000","last_peered":"2024-09-10T07:35:42.489096+0000","last_clean":"2024-09-10T07:35:42.489096+0000","last_became_active":"2024-09-10T07:34:14.156947+0000","last_became_peered":"2024-09-10T07:34:14.156947+0000","last_unstale":"2024-09-10T07:35:42.489096+0000","last_undegraded":"2024-09-10T07:35:42.489096+0000","last_fullsized":"2024-09-10T07:35:42.489096+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:42.201879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"95'1","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848959+0000","last_change":"2024-09-10T07:33:32.001383+0000","last_active":"2024-09-10T07:35:14.848959+0000","last_peered":"2024-09-10T07:35:14.848959+0000","last_clean":"2024-09-10T07:35:14.848959+0000","last_became_active":"2024-09-10T07:33:32.000774+0000","last_became_peered":"2024-09-10T07:33:32.000774+0000","last_unstale":"2024-09-10T07:35:14.848959+0000","last_undegraded":"2024-09-10T07:35:14.848959+0000","last_fullsized":"2024-09-10T07:35:14.848959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:56.780835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853941+0000","last_change":"2024-09-10T07:33:45.072245+0000","last_active":"2024-09-10T07:35:14.853941+0000","last_peered":"2024-09-10T07:35:14.853941+0000","last_clean":"2024-09-10T07:35:14.853941+0000","last_became_active":"2024-09-10T07:33:45.071704+0000","last_became_peered":"2024-09-10T07:33:45.071704+0000","last_unstale":"2024-09-10T07:35:14.853941+0000","last_undegraded":"2024-09-10T07:35:14.853941+0000","last_fullsized":"2024-09-10T07:35:14.853941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:34.213701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.485460+0000","last_change":"2024-09-10T07:33:34.873277+0000","last_active":"2024-09-10T07:35:42.485460+0000","last_peered":"2024-09-10T07:35:42.485460+0000","last_clean":"2024-09-10T07:35:42.485460+0000","last_became_active":"2024-09-10T07:33:34.872914+0000","last_became_peered":"2024-09-10T07:33:34.872914+0000","last_unstale":"2024-09-10T07:35:42.485460+0000","last_undegraded":"2024-09-10T07:35:42.485460+0000","last_fullsized":"2024-09-10T07:35:42.485460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:09.032347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"97'7","reported_seq":195,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:45.029001+0000","last_change":"2024-09-10T07:33:31.839469+0000","last_active":"2024-09-10T07:36:45.029001+0000","last_peered":"2024-09-10T07:36:45.029001+0000","last_clean":"2024-09-10T07:36:45.029001+0000","last_became_active":"2024-09-10T07:33:31.839126+0000","last_became_peered":"2024-09-10T07:33:31.839126+0000","last_unstale":"2024-09-10T07:36:45.029001+0000","last_undegraded":"2024-09-10T07:36:45.029001+0000","last_fullsized":"2024-09-10T07:36:45.029001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:55.825990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879712+0000","last_change":"2024-09-10T07:33:44.927101+0000","last_active":"2024-09-10T07:35:14.879712+0000","last_peered":"2024-09-10T07:35:14.879712+0000","last_clean":"2024-09-10T07:35:14.879712+0000","last_became_active":"2024-09-10T07:33:44.925490+0000","last_became_peered":"2024-09-10T07:33:44.925490+0000","last_unstale":"2024-09-10T07:35:14.879712+0000","last_undegraded":"2024-09-10T07:35:14.879712+0000","last_fullsized":"2024-09-10T07:35:14.879712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:02.950523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476516+0000","last_change":"2024-09-10T07:34:14.020283+0000","last_active":"2024-09-10T07:35:42.476516+0000","last_peered":"2024-09-10T07:35:42.476516+0000","last_clean":"2024-09-10T07:35:42.476516+0000","last_became_active":"2024-09-10T07:34:14.019929+0000","last_became_peered":"2024-09-10T07:34:14.019929+0000","last_unstale":"2024-09-10T07:35:42.476516+0000","last_undegraded":"2024-09-10T07:35:42.476516+0000","last_fullsized":"2024-09-10T07:35:42.476516+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:49.308768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":191,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:44.000415+0000","last_change":"2024-09-10T07:33:31.998057+0000","last_active":"2024-09-10T07:36:44.000415+0000","last_peered":"2024-09-10T07:36:44.000415+0000","last_clean":"2024-09-10T07:36:44.000415+0000","last_became_active":"2024-09-10T07:33:31.997709+0000","last_became_peered":"2024-09-10T07:33:31.997709+0000","last_unstale":"2024-09-10T07:36:44.000415+0000","last_undegraded":"2024-09-10T07:36:44.000415+0000","last_fullsized":"2024-09-10T07:36:44.000415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:53.424038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":141,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880600+0000","last_change":"2024-09-10T07:33:45.072323+0000","last_active":"2024-09-10T07:35:14.880600+0000","last_peered":"2024-09-10T07:35:14.880600+0000","last_clean":"2024-09-10T07:35:14.880600+0000","last_became_active":"2024-09-10T07:33:45.071899+0000","last_became_peered":"2024-09-10T07:33:45.071899+0000","last_unstale":"2024-09-10T07:35:14.880600+0000","last_undegraded":"2024-09-10T07:35:14.880600+0000","last_fullsized":"2024-09-10T07:35:14.880600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:20.304152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.479337+0000","last_change":"2024-09-10T07:33:34.867910+0000","last_active":"2024-09-10T07:35:42.479337+0000","last_peered":"2024-09-10T07:35:42.479337+0000","last_clean":"2024-09-10T07:35:42.479337+0000","last_became_active":"2024-09-10T07:33:34.867549+0000","last_became_peered":"2024-09-10T07:33:34.867549+0000","last_unstale":"2024-09-10T07:35:42.479337+0000","last_undegraded":"2024-09-10T07:35:42.479337+0000","last_fullsized":"2024-09-10T07:35:42.479337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:57.030422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842761+0000","last_change":"2024-09-10T07:33:31.843575+0000","last_active":"2024-09-10T07:35:14.842761+0000","last_peered":"2024-09-10T07:35:14.842761+0000","last_clean":"2024-09-10T07:35:14.842761+0000","last_became_active":"2024-09-10T07:33:31.843236+0000","last_became_peered":"2024-09-10T07:33:31.843236+0000","last_unstale":"2024-09-10T07:35:14.842761+0000","last_undegraded":"2024-09-10T07:35:14.842761+0000","last_fullsized":"2024-09-10T07:35:14.842761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:00.700409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:34:14.157032+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:34:14.156427+0000","last_became_peered":"2024-09-10T07:34:14.156427+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:23.754525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498940+0000","last_change":"2024-09-10T07:33:34.888373+0000","last_active":"2024-09-10T07:35:42.498940+0000","last_peered":"2024-09-10T07:35:42.498940+0000","last_clean":"2024-09-10T07:35:42.498940+0000","last_became_active":"2024-09-10T07:33:34.887226+0000","last_became_peered":"2024-09-10T07:33:34.887226+0000","last_unstale":"2024-09-10T07:35:42.498940+0000","last_undegraded":"2024-09-10T07:35:42.498940+0000","last_fullsized":"2024-09-10T07:35:42.498940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:59.925544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'6","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882510+0000","last_change":"2024-09-10T07:33:31.840918+0000","last_active":"2024-09-10T07:35:14.882510+0000","last_peered":"2024-09-10T07:35:14.882510+0000","last_clean":"2024-09-10T07:35:14.882510+0000","last_became_active":"2024-09-10T07:33:31.840525+0000","last_became_peered":"2024-09-10T07:33:31.840525+0000","last_unstale":"2024-09-10T07:35:14.882510+0000","last_undegraded":"2024-09-10T07:35:14.882510+0000","last_fullsized":"2024-09-10T07:35:14.882510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:58.963650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842861+0000","last_change":"2024-09-10T07:32:17.817973+0000","last_active":"2024-09-10T07:35:14.842861+0000","last_peered":"2024-09-10T07:35:14.842861+0000","last_clean":"2024-09-10T07:35:14.842861+0000","last_became_active":"2024-09-10T07:32:17.817560+0000","last_became_peered":"2024-09-10T07:32:17.817560+0000","last_unstale":"2024-09-10T07:35:14.842861+0000","last_undegraded":"2024-09-10T07:35:14.842861+0000","last_fullsized":"2024-09-10T07:35:14.842861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:28:37.148756+0000","last_clean_scrub_stamp":"2024-09-10T07:28:37.148756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:49:33.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"95'1","reported_seq":30,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843182+0000","last_change":"2024-09-10T07:33:44.924013+0000","last_active":"2024-09-10T07:35:14.843182+0000","last_peered":"2024-09-10T07:35:14.843182+0000","last_clean":"2024-09-10T07:35:14.843182+0000","last_became_active":"2024-09-10T07:33:44.923591+0000","last_became_peered":"2024-09-10T07:33:44.923591+0000","last_unstale":"2024-09-10T07:35:14.843182+0000","last_undegraded":"2024-09-10T07:35:14.843182+0000","last_fullsized":"2024-09-10T07:35:14.843182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:13.809021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484404+0000","last_change":"2024-09-10T07:33:34.877983+0000","last_active":"2024-09-10T07:35:42.484404+0000","last_peered":"2024-09-10T07:35:42.484404+0000","last_clean":"2024-09-10T07:35:42.484404+0000","last_became_active":"2024-09-10T07:33:34.877565+0000","last_became_peered":"2024-09-10T07:33:34.877565+0000","last_unstale":"2024-09-10T07:35:42.484404+0000","last_undegraded":"2024-09-10T07:35:42.484404+0000","last_fullsized":"2024-09-10T07:35:42.484404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:00.362015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'5","reported_seq":191,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:44.001158+0000","last_change":"2024-09-10T07:33:31.999121+0000","last_active":"2024-09-10T07:36:44.001158+0000","last_peered":"2024-09-10T07:36:44.001158+0000","last_clean":"2024-09-10T07:36:44.001158+0000","last_became_active":"2024-09-10T07:33:31.998642+0000","last_became_peered":"2024-09-10T07:33:31.998642+0000","last_unstale":"2024-09-10T07:36:44.001158+0000","last_undegraded":"2024-09-10T07:36:44.001158+0000","last_fullsized":"2024-09-10T07:36:44.001158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:32.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.976793+0000","last_change":"2024-09-10T07:33:44.914539+0000","last_active":"2024-09-10T07:35:14.976793+0000","last_peered":"2024-09-10T07:35:14.976793+0000","last_clean":"2024-09-10T07:35:14.976793+0000","last_became_active":"2024-09-10T07:33:44.913793+0000","last_became_peered":"2024-09-10T07:33:44.913793+0000","last_unstale":"2024-09-10T07:35:14.976793+0000","last_undegraded":"2024-09-10T07:35:14.976793+0000","last_fullsized":"2024-09-10T07:35:14.976793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:03.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487151+0000","last_change":"2024-09-10T07:33:34.886391+0000","last_active":"2024-09-10T07:35:42.487151+0000","last_peered":"2024-09-10T07:35:42.487151+0000","last_clean":"2024-09-10T07:35:42.487151+0000","last_became_active":"2024-09-10T07:33:34.886020+0000","last_became_peered":"2024-09-10T07:33:34.886020+0000","last_unstale":"2024-09-10T07:35:42.487151+0000","last_undegraded":"2024-09-10T07:35:42.487151+0000","last_fullsized":"2024-09-10T07:35:42.487151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:34.848705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880698+0000","last_change":"2024-09-10T07:33:31.844896+0000","last_active":"2024-09-10T07:35:14.880698+0000","last_peered":"2024-09-10T07:35:14.880698+0000","last_clean":"2024-09-10T07:35:14.880698+0000","last_became_active":"2024-09-10T07:33:31.844473+0000","last_became_peered":"2024-09-10T07:33:31.844473+0000","last_unstale":"2024-09-10T07:35:14.880698+0000","last_undegraded":"2024-09-10T07:35:14.880698+0000","last_fullsized":"2024-09-10T07:35:14.880698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:10.263334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.857702+0000","last_change":"2024-09-10T07:34:14.108108+0000","last_active":"2024-09-10T07:35:14.857702+0000","last_peered":"2024-09-10T07:35:14.857702+0000","last_clean":"2024-09-10T07:35:14.857702+0000","last_became_active":"2024-09-10T07:34:14.107676+0000","last_became_peered":"2024-09-10T07:34:14.107676+0000","last_unstale":"2024-09-10T07:35:14.857702+0000","last_undegraded":"2024-09-10T07:35:14.857702+0000","last_fullsized":"2024-09-10T07:35:14.857702+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:07.990906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477302+0000","last_change":"2024-09-10T07:33:34.875137+0000","last_active":"2024-09-10T07:35:42.477302+0000","last_peered":"2024-09-10T07:35:42.477302+0000","last_clean":"2024-09-10T07:35:42.477302+0000","last_became_active":"2024-09-10T07:33:34.874085+0000","last_became_peered":"2024-09-10T07:33:34.874085+0000","last_unstale":"2024-09-10T07:35:42.477302+0000","last_undegraded":"2024-09-10T07:35:42.477302+0000","last_fullsized":"2024-09-10T07:35:42.477302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:41:07.373729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880480+0000","last_change":"2024-09-10T07:33:31.841565+0000","last_active":"2024-09-10T07:35:14.880480+0000","last_peered":"2024-09-10T07:35:14.880480+0000","last_clean":"2024-09-10T07:35:14.880480+0000","last_became_active":"2024-09-10T07:33:31.841208+0000","last_became_peered":"2024-09-10T07:33:31.841208+0000","last_unstale":"2024-09-10T07:35:14.880480+0000","last_undegraded":"2024-09-10T07:35:14.880480+0000","last_fullsized":"2024-09-10T07:35:14.880480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:59.297259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879975+0000","last_change":"2024-09-10T07:33:44.913953+0000","last_active":"2024-09-10T07:35:14.879975+0000","last_peered":"2024-09-10T07:35:14.879975+0000","last_clean":"2024-09-10T07:35:14.879975+0000","last_became_active":"2024-09-10T07:33:44.913539+0000","last_became_peered":"2024-09-10T07:33:44.913539+0000","last_unstale":"2024-09-10T07:35:14.879975+0000","last_undegraded":"2024-09-10T07:35:14.879975+0000","last_fullsized":"2024-09-10T07:35:14.879975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:50.402281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.474216+0000","last_change":"2024-09-10T07:33:34.871908+0000","last_active":"2024-09-10T07:35:42.474216+0000","last_peered":"2024-09-10T07:35:42.474216+0000","last_clean":"2024-09-10T07:35:42.474216+0000","last_became_active":"2024-09-10T07:33:34.871516+0000","last_became_peered":"2024-09-10T07:33:34.871516+0000","last_unstale":"2024-09-10T07:35:42.474216+0000","last_undegraded":"2024-09-10T07:35:42.474216+0000","last_fullsized":"2024-09-10T07:35:42.474216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'2","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.854462+0000","last_change":"2024-09-10T07:33:31.842892+0000","last_active":"2024-09-10T07:35:14.854462+0000","last_peered":"2024-09-10T07:35:14.854462+0000","last_clean":"2024-09-10T07:35:14.854462+0000","last_became_active":"2024-09-10T07:33:31.841947+0000","last_became_peered":"2024-09-10T07:33:31.841947+0000","last_unstale":"2024-09-10T07:35:14.854462+0000","last_undegraded":"2024-09-10T07:35:14.854462+0000","last_fullsized":"2024-09-10T07:35:14.854462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:51.051851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879978+0000","last_change":"2024-09-10T07:33:44.923931+0000","last_active":"2024-09-10T07:35:14.879978+0000","last_peered":"2024-09-10T07:35:14.879978+0000","last_clean":"2024-09-10T07:35:14.879978+0000","last_became_active":"2024-09-10T07:33:44.923426+0000","last_became_peered":"2024-09-10T07:33:44.923426+0000","last_unstale":"2024-09-10T07:35:14.879978+0000","last_undegraded":"2024-09-10T07:35:14.879978+0000","last_fullsized":"2024-09-10T07:35:14.879978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:04.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.480953+0000","last_change":"2024-09-10T07:33:34.879017+0000","last_active":"2024-09-10T07:35:42.480953+0000","last_peered":"2024-09-10T07:35:42.480953+0000","last_clean":"2024-09-10T07:35:42.480953+0000","last_became_active":"2024-09-10T07:33:34.878263+0000","last_became_peered":"2024-09-10T07:33:34.878263+0000","last_unstale":"2024-09-10T07:35:42.480953+0000","last_undegraded":"2024-09-10T07:35:42.480953+0000","last_fullsized":"2024-09-10T07:35:42.480953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:18.461136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"95'17","reported_seq":62,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.853997+0000","last_change":"2024-09-10T07:33:31.996916+0000","last_active":"2024-09-10T07:35:14.853997+0000","last_peered":"2024-09-10T07:35:14.853997+0000","last_clean":"2024-09-10T07:35:14.853997+0000","last_became_active":"2024-09-10T07:33:31.996530+0000","last_became_peered":"2024-09-10T07:33:31.996530+0000","last_unstale":"2024-09-10T07:35:14.853997+0000","last_undegraded":"2024-09-10T07:35:14.853997+0000","last_fullsized":"2024-09-10T07:35:14.853997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:49.415964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880388+0000","last_change":"2024-09-10T07:33:44.926469+0000","last_active":"2024-09-10T07:35:14.880388+0000","last_peered":"2024-09-10T07:35:14.880388+0000","last_clean":"2024-09-10T07:35:14.880388+0000","last_became_active":"2024-09-10T07:33:44.926101+0000","last_became_peered":"2024-09-10T07:33:44.926101+0000","last_unstale":"2024-09-10T07:35:14.880388+0000","last_undegraded":"2024-09-10T07:35:14.880388+0000","last_fullsized":"2024-09-10T07:35:14.880388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:13:08.888486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487180+0000","last_change":"2024-09-10T07:33:34.879883+0000","last_active":"2024-09-10T07:35:42.487180+0000","last_peered":"2024-09-10T07:35:42.487180+0000","last_clean":"2024-09-10T07:35:42.487180+0000","last_became_active":"2024-09-10T07:33:34.879179+0000","last_became_peered":"2024-09-10T07:33:34.879179+0000","last_unstale":"2024-09-10T07:35:42.487180+0000","last_undegraded":"2024-09-10T07:35:42.487180+0000","last_fullsized":"2024-09-10T07:35:42.487180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:38.462806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'5","reported_seq":190,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:44.000813+0000","last_change":"2024-09-10T07:33:31.844761+0000","last_active":"2024-09-10T07:36:44.000813+0000","last_peered":"2024-09-10T07:36:44.000813+0000","last_clean":"2024-09-10T07:36:44.000813+0000","last_became_active":"2024-09-10T07:33:31.844356+0000","last_became_peered":"2024-09-10T07:33:31.844356+0000","last_unstale":"2024-09-10T07:36:44.000813+0000","last_undegraded":"2024-09-10T07:36:44.000813+0000","last_fullsized":"2024-09-10T07:36:44.000813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:09.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.851402+0000","last_change":"2024-09-10T07:33:44.932528+0000","last_active":"2024-09-10T07:35:14.851402+0000","last_peered":"2024-09-10T07:35:14.851402+0000","last_clean":"2024-09-10T07:35:14.851402+0000","last_became_active":"2024-09-10T07:33:44.931955+0000","last_became_peered":"2024-09-10T07:33:44.931955+0000","last_unstale":"2024-09-10T07:35:14.851402+0000","last_undegraded":"2024-09-10T07:35:14.851402+0000","last_fullsized":"2024-09-10T07:35:14.851402+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:48.522138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488551+0000","last_change":"2024-09-10T07:33:34.881097+0000","last_active":"2024-09-10T07:35:42.488551+0000","last_peered":"2024-09-10T07:35:42.488551+0000","last_clean":"2024-09-10T07:35:42.488551+0000","last_became_active":"2024-09-10T07:33:34.880764+0000","last_became_peered":"2024-09-10T07:33:34.880764+0000","last_unstale":"2024-09-10T07:35:42.488551+0000","last_undegraded":"2024-09-10T07:35:42.488551+0000","last_fullsized":"2024-09-10T07:35:42.488551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.424071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880166+0000","last_change":"2024-09-10T07:33:31.845431+0000","last_active":"2024-09-10T07:35:14.880166+0000","last_peered":"2024-09-10T07:35:14.880166+0000","last_clean":"2024-09-10T07:35:14.880166+0000","last_became_active":"2024-09-10T07:33:31.845087+0000","last_became_peered":"2024-09-10T07:33:31.845087+0000","last_unstale":"2024-09-10T07:35:14.880166+0000","last_undegraded":"2024-09-10T07:35:14.880166+0000","last_fullsized":"2024-09-10T07:35:14.880166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:21.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852262+0000","last_change":"2024-09-10T07:34:14.156814+0000","last_active":"2024-09-10T07:35:14.852262+0000","last_peered":"2024-09-10T07:35:14.852262+0000","last_clean":"2024-09-10T07:35:14.852262+0000","last_became_active":"2024-09-10T07:34:14.156302+0000","last_became_peered":"2024-09-10T07:34:14.156302+0000","last_unstale":"2024-09-10T07:35:14.852262+0000","last_undegraded":"2024-09-10T07:35:14.852262+0000","last_fullsized":"2024-09-10T07:35:14.852262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:15.838916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500238+0000","last_change":"2024-09-10T07:33:34.878374+0000","last_active":"2024-09-10T07:35:42.500238+0000","last_peered":"2024-09-10T07:35:42.500238+0000","last_clean":"2024-09-10T07:35:42.500238+0000","last_became_active":"2024-09-10T07:33:34.878032+0000","last_became_peered":"2024-09-10T07:33:34.878032+0000","last_unstale":"2024-09-10T07:35:42.500238+0000","last_undegraded":"2024-09-10T07:35:42.500238+0000","last_fullsized":"2024-09-10T07:35:42.500238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:55.254917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"95'9","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852264+0000","last_change":"2024-09-10T07:34:14.026120+0000","last_active":"2024-09-10T07:35:14.852264+0000","last_peered":"2024-09-10T07:35:14.852264+0000","last_clean":"2024-09-10T07:35:14.852264+0000","last_became_active":"2024-09-10T07:34:14.020091+0000","last_became_peered":"2024-09-10T07:34:14.020091+0000","last_unstale":"2024-09-10T07:35:14.852264+0000","last_undegraded":"2024-09-10T07:35:14.852264+0000","last_fullsized":"2024-09-10T07:35:14.852264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:36.917862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880432+0000","last_change":"2024-09-10T07:33:44.923581+0000","last_active":"2024-09-10T07:35:14.880432+0000","last_peered":"2024-09-10T07:35:14.880432+0000","last_clean":"2024-09-10T07:35:14.880432+0000","last_became_active":"2024-09-10T07:33:44.922922+0000","last_became_peered":"2024-09-10T07:33:44.922922+0000","last_unstale":"2024-09-10T07:35:14.880432+0000","last_undegraded":"2024-09-10T07:35:14.880432+0000","last_fullsized":"2024-09-10T07:35:14.880432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:28:03.774379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.469244+0000","last_change":"2024-09-10T07:33:34.873230+0000","last_active":"2024-09-10T07:35:42.469244+0000","last_peered":"2024-09-10T07:35:42.469244+0000","last_clean":"2024-09-10T07:35:42.469244+0000","last_became_active":"2024-09-10T07:33:34.872903+0000","last_became_peered":"2024-09-10T07:33:34.872903+0000","last_unstale":"2024-09-10T07:35:42.469244+0000","last_undegraded":"2024-09-10T07:35:42.469244+0000","last_fullsized":"2024-09-10T07:35:42.469244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:48.683753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"97'42","reported_seq":375,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:42.998794+0000","last_change":"2024-09-10T07:33:31.844165+0000","last_active":"2024-09-10T07:36:42.998794+0000","last_peered":"2024-09-10T07:36:42.998794+0000","last_clean":"2024-09-10T07:36:42.998794+0000","last_became_active":"2024-09-10T07:33:31.842793+0000","last_became_peered":"2024-09-10T07:33:31.842793+0000","last_unstale":"2024-09-10T07:36:42.998794+0000","last_undegraded":"2024-09-10T07:36:42.998794+0000","last_fullsized":"2024-09-10T07:36:42.998794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:14:13.462068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70555,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":289,"num_read_kb":214,"num_write":44,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.848093+0000","last_change":"2024-09-10T07:33:44.931402+0000","last_active":"2024-09-10T07:35:14.848093+0000","last_peered":"2024-09-10T07:35:14.848093+0000","last_clean":"2024-09-10T07:35:14.848093+0000","last_became_active":"2024-09-10T07:33:44.930857+0000","last_became_peered":"2024-09-10T07:33:44.930857+0000","last_unstale":"2024-09-10T07:35:14.848093+0000","last_undegraded":"2024-09-10T07:35:14.848093+0000","last_fullsized":"2024-09-10T07:35:14.848093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:29.780537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490736+0000","last_change":"2024-09-10T07:33:34.879124+0000","last_active":"2024-09-10T07:35:42.490736+0000","last_peered":"2024-09-10T07:35:42.490736+0000","last_clean":"2024-09-10T07:35:42.490736+0000","last_became_active":"2024-09-10T07:33:34.877592+0000","last_became_peered":"2024-09-10T07:33:34.877592+0000","last_unstale":"2024-09-10T07:35:42.490736+0000","last_undegraded":"2024-09-10T07:35:42.490736+0000","last_fullsized":"2024-09-10T07:35:42.490736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:27:39.465193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"95'5","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880613+0000","last_change":"2024-09-10T07:33:31.997965+0000","last_active":"2024-09-10T07:35:14.880613+0000","last_peered":"2024-09-10T07:35:14.880613+0000","last_clean":"2024-09-10T07:35:14.880613+0000","last_became_active":"2024-09-10T07:33:31.997508+0000","last_became_peered":"2024-09-10T07:33:31.997508+0000","last_unstale":"2024-09-10T07:35:14.880613+0000","last_undegraded":"2024-09-10T07:35:14.880613+0000","last_fullsized":"2024-09-10T07:35:14.880613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:38.293418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.882346+0000","last_change":"2024-09-10T07:33:44.924160+0000","last_active":"2024-09-10T07:35:14.882346+0000","last_peered":"2024-09-10T07:35:14.882346+0000","last_clean":"2024-09-10T07:35:14.882346+0000","last_became_active":"2024-09-10T07:33:44.923751+0000","last_became_peered":"2024-09-10T07:33:44.923751+0000","last_unstale":"2024-09-10T07:35:14.882346+0000","last_undegraded":"2024-09-10T07:35:14.882346+0000","last_fullsized":"2024-09-10T07:35:14.882346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:22.603162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502535+0000","last_change":"2024-09-10T07:33:34.882617+0000","last_active":"2024-09-10T07:35:42.502535+0000","last_peered":"2024-09-10T07:35:42.502535+0000","last_clean":"2024-09-10T07:35:42.502535+0000","last_became_active":"2024-09-10T07:33:34.880541+0000","last_became_peered":"2024-09-10T07:33:34.880541+0000","last_unstale":"2024-09-10T07:35:42.502535+0000","last_undegraded":"2024-09-10T07:35:42.502535+0000","last_fullsized":"2024-09-10T07:35:42.502535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:00.010225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"95'4","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.843243+0000","last_change":"2024-09-10T07:33:31.842442+0000","last_active":"2024-09-10T07:35:14.843243+0000","last_peered":"2024-09-10T07:35:14.843243+0000","last_clean":"2024-09-10T07:35:14.843243+0000","last_became_active":"2024-09-10T07:33:31.842032+0000","last_became_peered":"2024-09-10T07:33:31.842032+0000","last_unstale":"2024-09-10T07:35:14.843243+0000","last_undegraded":"2024-09-10T07:35:14.843243+0000","last_fullsized":"2024-09-10T07:35:14.843243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:49.283169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.847404+0000","last_change":"2024-09-10T07:33:44.926411+0000","last_active":"2024-09-10T07:35:14.847404+0000","last_peered":"2024-09-10T07:35:14.847404+0000","last_clean":"2024-09-10T07:35:14.847404+0000","last_became_active":"2024-09-10T07:33:44.926024+0000","last_became_peered":"2024-09-10T07:33:44.926024+0000","last_unstale":"2024-09-10T07:35:14.847404+0000","last_undegraded":"2024-09-10T07:35:14.847404+0000","last_fullsized":"2024-09-10T07:35:14.847404+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.486920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.490415+0000","last_change":"2024-09-10T07:33:34.869850+0000","last_active":"2024-09-10T07:35:42.490415+0000","last_peered":"2024-09-10T07:35:42.490415+0000","last_clean":"2024-09-10T07:35:42.490415+0000","last_became_active":"2024-09-10T07:33:34.869370+0000","last_became_peered":"2024-09-10T07:33:34.869370+0000","last_unstale":"2024-09-10T07:35:42.490415+0000","last_undegraded":"2024-09-10T07:35:42.490415+0000","last_fullsized":"2024-09-10T07:35:42.490415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:45.212954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"97'5","reported_seq":325,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:42.999010+0000","last_change":"2024-09-10T07:33:31.842336+0000","last_active":"2024-09-10T07:36:42.999010+0000","last_peered":"2024-09-10T07:36:42.999010+0000","last_clean":"2024-09-10T07:36:42.999010+0000","last_became_active":"2024-09-10T07:33:31.841068+0000","last_became_peered":"2024-09-10T07:33:31.841068+0000","last_unstale":"2024-09-10T07:36:42.999010+0000","last_undegraded":"2024-09-10T07:36:42.999010+0000","last_fullsized":"2024-09-10T07:36:42.999010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:08.209974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":276,"num_read_kb":276,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.977046+0000","last_change":"2024-09-10T07:33:44.920409+0000","last_active":"2024-09-10T07:35:14.977046+0000","last_peered":"2024-09-10T07:35:14.977046+0000","last_clean":"2024-09-10T07:35:14.977046+0000","last_became_active":"2024-09-10T07:33:44.915198+0000","last_became_peered":"2024-09-10T07:33:44.915198+0000","last_unstale":"2024-09-10T07:35:14.977046+0000","last_undegraded":"2024-09-10T07:35:14.977046+0000","last_fullsized":"2024-09-10T07:35:14.977046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:45.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.858166+0000","last_change":"2024-09-10T07:33:44.927965+0000","last_active":"2024-09-10T07:35:14.858166+0000","last_peered":"2024-09-10T07:35:14.858166+0000","last_clean":"2024-09-10T07:35:14.858166+0000","last_became_active":"2024-09-10T07:33:44.925915+0000","last_became_peered":"2024-09-10T07:33:44.925915+0000","last_unstale":"2024-09-10T07:35:14.858166+0000","last_undegraded":"2024-09-10T07:35:14.858166+0000","last_fullsized":"2024-09-10T07:35:14.858166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:55.453038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.498588+0000","last_change":"2024-09-10T07:33:34.884810+0000","last_active":"2024-09-10T07:35:42.498588+0000","last_peered":"2024-09-10T07:35:42.498588+0000","last_clean":"2024-09-10T07:35:42.498588+0000","last_became_active":"2024-09-10T07:33:34.884424+0000","last_became_peered":"2024-09-10T07:33:34.884424+0000","last_unstale":"2024-09-10T07:35:42.498588+0000","last_undegraded":"2024-09-10T07:35:42.498588+0000","last_fullsized":"2024-09-10T07:35:42.498588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:34.785394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.880248+0000","last_change":"2024-09-10T07:33:31.845749+0000","last_active":"2024-09-10T07:35:14.880248+0000","last_peered":"2024-09-10T07:35:14.880248+0000","last_clean":"2024-09-10T07:35:14.880248+0000","last_became_active":"2024-09-10T07:33:31.845356+0000","last_became_peered":"2024-09-10T07:33:31.845356+0000","last_unstale":"2024-09-10T07:35:14.880248+0000","last_undegraded":"2024-09-10T07:35:14.880248+0000","last_fullsized":"2024-09-10T07:35:14.880248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:29.771146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.852575+0000","last_change":"2024-09-10T07:33:44.925636+0000","last_active":"2024-09-10T07:35:14.852575+0000","last_peered":"2024-09-10T07:35:14.852575+0000","last_clean":"2024-09-10T07:35:14.852575+0000","last_became_active":"2024-09-10T07:33:44.925214+0000","last_became_peered":"2024-09-10T07:33:44.925214+0000","last_unstale":"2024-09-10T07:35:14.852575+0000","last_undegraded":"2024-09-10T07:35:14.852575+0000","last_fullsized":"2024-09-10T07:35:14.852575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:16.771314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499570+0000","last_change":"2024-09-10T07:33:34.878726+0000","last_active":"2024-09-10T07:35:42.499570+0000","last_peered":"2024-09-10T07:35:42.499570+0000","last_clean":"2024-09-10T07:35:42.499570+0000","last_became_active":"2024-09-10T07:33:34.875295+0000","last_became_peered":"2024-09-10T07:33:34.875295+0000","last_unstale":"2024-09-10T07:35:42.499570+0000","last_undegraded":"2024-09-10T07:35:42.499570+0000","last_fullsized":"2024-09-10T07:35:42.499570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:56.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"95'3","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.879193+0000","last_change":"2024-09-10T07:33:31.839491+0000","last_active":"2024-09-10T07:35:14.879193+0000","last_peered":"2024-09-10T07:35:14.879193+0000","last_clean":"2024-09-10T07:35:14.879193+0000","last_became_active":"2024-09-10T07:33:31.839063+0000","last_became_peered":"2024-09-10T07:33:31.839063+0000","last_unstale":"2024-09-10T07:35:14.879193+0000","last_undegraded":"2024-09-10T07:35:14.879193+0000","last_fullsized":"2024-09-10T07:35:14.879193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:47.997192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"95'7","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.877898+0000","last_change":"2024-09-10T07:33:31.839840+0000","last_active":"2024-09-10T07:35:14.877898+0000","last_peered":"2024-09-10T07:35:14.877898+0000","last_clean":"2024-09-10T07:35:14.877898+0000","last_became_active":"2024-09-10T07:33:31.839324+0000","last_became_peered":"2024-09-10T07:33:31.839324+0000","last_unstale":"2024-09-10T07:35:14.877898+0000","last_undegraded":"2024-09-10T07:35:14.877898+0000","last_fullsized":"2024-09-10T07:35:14.877898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:09:26.261837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"97'8","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.492573+0000","last_change":"2024-09-10T07:34:14.020936+0000","last_active":"2024-09-10T07:35:42.492573+0000","last_peered":"2024-09-10T07:35:42.492573+0000","last_clean":"2024-09-10T07:35:42.492573+0000","last_became_active":"2024-09-10T07:34:14.019604+0000","last_became_peered":"2024-09-10T07:34:14.019604+0000","last_unstale":"2024-09-10T07:35:42.492573+0000","last_undegraded":"2024-09-10T07:35:42.492573+0000","last_fullsized":"2024-09-10T07:35:42.492573+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:08.720750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842519+0000","last_change":"2024-09-10T07:33:44.929201+0000","last_active":"2024-09-10T07:35:14.842519+0000","last_peered":"2024-09-10T07:35:14.842519+0000","last_clean":"2024-09-10T07:35:14.842519+0000","last_became_active":"2024-09-10T07:33:44.928744+0000","last_became_peered":"2024-09-10T07:33:44.928744+0000","last_unstale":"2024-09-10T07:35:14.842519+0000","last_undegraded":"2024-09-10T07:35:14.842519+0000","last_fullsized":"2024-09-10T07:35:14.842519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:44.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'7","reported_seq":195,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:36:44.001316+0000","last_change":"2024-09-10T07:33:31.845784+0000","last_active":"2024-09-10T07:36:44.001316+0000","last_peered":"2024-09-10T07:36:44.001316+0000","last_clean":"2024-09-10T07:36:44.001316+0000","last_became_active":"2024-09-10T07:33:31.845401+0000","last_became_peered":"2024-09-10T07:33:31.845401+0000","last_unstale":"2024-09-10T07:36:44.001316+0000","last_undegraded":"2024-09-10T07:36:44.001316+0000","last_fullsized":"2024-09-10T07:36:44.001316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:30.825440+0000","last_clean_scrub_stamp":"2024-09-10T07:33:30.825440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:09.119642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.475213+0000","last_change":"2024-09-10T07:33:34.875216+0000","last_active":"2024-09-10T07:35:42.475213+0000","last_peered":"2024-09-10T07:35:42.475213+0000","last_clean":"2024-09-10T07:35:42.475213+0000","last_became_active":"2024-09-10T07:33:34.874878+0000","last_became_peered":"2024-09-10T07:33:34.874878+0000","last_unstale":"2024-09-10T07:35:42.475213+0000","last_undegraded":"2024-09-10T07:35:42.475213+0000","last_fullsized":"2024-09-10T07:35:42.475213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:53.449854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:14.842633+0000","last_change":"2024-09-10T07:33:44.914223+0000","last_active":"2024-09-10T07:35:14.842633+0000","last_peered":"2024-09-10T07:35:14.842633+0000","last_clean":"2024-09-10T07:35:14.842633+0000","last_became_active":"2024-09-10T07:33:44.913859+0000","last_became_peered":"2024-09-10T07:33:44.913859+0000","last_unstale":"2024-09-10T07:35:14.842633+0000","last_undegraded":"2024-09-10T07:35:14.842633+0000","last_fullsized":"2024-09-10T07:35:14.842633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:43.894887+0000","last_clean_scrub_stamp":"2024-09-10T07:33:43.894887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:45.317118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.477747+0000","last_change":"2024-09-10T07:33:34.871807+0000","last_active":"2024-09-10T07:35:42.477747+0000","last_peered":"2024-09-10T07:35:42.477747+0000","last_clean":"2024-09-10T07:35:42.477747+0000","last_became_active":"2024-09-10T07:33:34.871480+0000","last_became_peered":"2024-09-10T07:33:34.871480+0000","last_unstale":"2024-09-10T07:35:42.477747+0000","last_undegraded":"2024-09-10T07:35:42.477747+0000","last_fullsized":"2024-09-10T07:35:42.477747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:24.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"97'12","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476532+0000","last_change":"2024-09-10T07:33:34.880683+0000","last_active":"2024-09-10T07:35:42.476532+0000","last_peered":"2024-09-10T07:35:42.476532+0000","last_clean":"2024-09-10T07:35:42.476532+0000","last_became_active":"2024-09-10T07:33:34.880264+0000","last_became_peered":"2024-09-10T07:33:34.880264+0000","last_unstale":"2024-09-10T07:35:42.476532+0000","last_undegraded":"2024-09-10T07:35:42.476532+0000","last_fullsized":"2024-09-10T07:35:42.476532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.703923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.486476+0000","last_change":"2024-09-10T07:33:34.880503+0000","last_active":"2024-09-10T07:35:42.486476+0000","last_peered":"2024-09-10T07:35:42.486476+0000","last_clean":"2024-09-10T07:35:42.486476+0000","last_became_active":"2024-09-10T07:33:34.879898+0000","last_became_peered":"2024-09-10T07:33:34.879898+0000","last_unstale":"2024-09-10T07:35:42.486476+0000","last_undegraded":"2024-09-10T07:35:42.486476+0000","last_fullsized":"2024-09-10T07:35:42.486476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:09.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499618+0000","last_change":"2024-09-10T07:33:34.873783+0000","last_active":"2024-09-10T07:35:42.499618+0000","last_peered":"2024-09-10T07:35:42.499618+0000","last_clean":"2024-09-10T07:35:42.499618+0000","last_became_active":"2024-09-10T07:33:34.873416+0000","last_became_peered":"2024-09-10T07:33:34.873416+0000","last_unstale":"2024-09-10T07:35:42.499618+0000","last_undegraded":"2024-09-10T07:35:42.499618+0000","last_fullsized":"2024-09-10T07:35:42.499618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:06.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487143+0000","last_change":"2024-09-10T07:33:34.876318+0000","last_active":"2024-09-10T07:35:42.487143+0000","last_peered":"2024-09-10T07:35:42.487143+0000","last_clean":"2024-09-10T07:35:42.487143+0000","last_became_active":"2024-09-10T07:33:34.876028+0000","last_became_peered":"2024-09-10T07:33:34.876028+0000","last_unstale":"2024-09-10T07:35:42.487143+0000","last_undegraded":"2024-09-10T07:35:42.487143+0000","last_fullsized":"2024-09-10T07:35:42.487143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:44.607373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.470371+0000","last_change":"2024-09-10T07:33:34.868678+0000","last_active":"2024-09-10T07:35:42.470371+0000","last_peered":"2024-09-10T07:35:42.470371+0000","last_clean":"2024-09-10T07:35:42.470371+0000","last_became_active":"2024-09-10T07:33:34.868364+0000","last_became_peered":"2024-09-10T07:33:34.868364+0000","last_unstale":"2024-09-10T07:35:42.470371+0000","last_undegraded":"2024-09-10T07:35:42.470371+0000","last_fullsized":"2024-09-10T07:35:42.470371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:22.752474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"97'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476911+0000","last_change":"2024-09-10T07:33:34.871650+0000","last_active":"2024-09-10T07:35:42.476911+0000","last_peered":"2024-09-10T07:35:42.476911+0000","last_clean":"2024-09-10T07:35:42.476911+0000","last_became_active":"2024-09-10T07:33:34.871323+0000","last_became_peered":"2024-09-10T07:33:34.871323+0000","last_unstale":"2024-09-10T07:35:42.476911+0000","last_undegraded":"2024-09-10T07:35:42.476911+0000","last_fullsized":"2024-09-10T07:35:42.476911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:45.765017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.488157+0000","last_change":"2024-09-10T07:33:34.879309+0000","last_active":"2024-09-10T07:35:42.488157+0000","last_peered":"2024-09-10T07:35:42.488157+0000","last_clean":"2024-09-10T07:35:42.488157+0000","last_became_active":"2024-09-10T07:33:34.878036+0000","last_became_peered":"2024-09-10T07:33:34.878036+0000","last_unstale":"2024-09-10T07:35:42.488157+0000","last_undegraded":"2024-09-10T07:35:42.488157+0000","last_fullsized":"2024-09-10T07:35:42.488157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:48.355708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500450+0000","last_change":"2024-09-10T07:33:34.877307+0000","last_active":"2024-09-10T07:35:42.500450+0000","last_peered":"2024-09-10T07:35:42.500450+0000","last_clean":"2024-09-10T07:35:42.500450+0000","last_became_active":"2024-09-10T07:33:34.876991+0000","last_became_peered":"2024-09-10T07:33:34.876991+0000","last_unstale":"2024-09-10T07:35:42.500450+0000","last_undegraded":"2024-09-10T07:35:42.500450+0000","last_fullsized":"2024-09-10T07:35:42.500450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.947543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499239+0000","last_change":"2024-09-10T07:33:34.880923+0000","last_active":"2024-09-10T07:35:42.499239+0000","last_peered":"2024-09-10T07:35:42.499239+0000","last_clean":"2024-09-10T07:35:42.499239+0000","last_became_active":"2024-09-10T07:33:34.880227+0000","last_became_peered":"2024-09-10T07:33:34.880227+0000","last_unstale":"2024-09-10T07:35:42.499239+0000","last_undegraded":"2024-09-10T07:35:42.499239+0000","last_fullsized":"2024-09-10T07:35:42.499239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:37.691876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"97'10","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.501908+0000","last_change":"2024-09-10T07:33:34.883995+0000","last_active":"2024-09-10T07:35:42.501908+0000","last_peered":"2024-09-10T07:35:42.501908+0000","last_clean":"2024-09-10T07:35:42.501908+0000","last_became_active":"2024-09-10T07:33:34.883646+0000","last_became_peered":"2024-09-10T07:33:34.883646+0000","last_unstale":"2024-09-10T07:35:42.501908+0000","last_undegraded":"2024-09-10T07:35:42.501908+0000","last_fullsized":"2024-09-10T07:35:42.501908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:31.261610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"97'11","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493731+0000","last_change":"2024-09-10T07:33:34.875185+0000","last_active":"2024-09-10T07:35:42.493731+0000","last_peered":"2024-09-10T07:35:42.493731+0000","last_clean":"2024-09-10T07:35:42.493731+0000","last_became_active":"2024-09-10T07:33:34.874768+0000","last_became_peered":"2024-09-10T07:33:34.874768+0000","last_unstale":"2024-09-10T07:35:42.493731+0000","last_undegraded":"2024-09-10T07:35:42.493731+0000","last_fullsized":"2024-09-10T07:35:42.493731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:05.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"97'8","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478046+0000","last_change":"2024-09-10T07:34:14.021473+0000","last_active":"2024-09-10T07:35:42.478046+0000","last_peered":"2024-09-10T07:35:42.478046+0000","last_clean":"2024-09-10T07:35:42.478046+0000","last_became_active":"2024-09-10T07:34:14.021139+0000","last_became_peered":"2024-09-10T07:34:14.021139+0000","last_unstale":"2024-09-10T07:35:42.478046+0000","last_undegraded":"2024-09-10T07:35:42.478046+0000","last_fullsized":"2024-09-10T07:35:42.478046+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:20.700731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493144+0000","last_change":"2024-09-10T07:33:34.886730+0000","last_active":"2024-09-10T07:35:42.493144+0000","last_peered":"2024-09-10T07:35:42.493144+0000","last_clean":"2024-09-10T07:35:42.493144+0000","last_became_active":"2024-09-10T07:33:34.886381+0000","last_became_peered":"2024-09-10T07:33:34.886381+0000","last_unstale":"2024-09-10T07:35:42.493144+0000","last_undegraded":"2024-09-10T07:35:42.493144+0000","last_fullsized":"2024-09-10T07:35:42.493144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:20.497216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483119+0000","last_change":"2024-09-10T07:33:34.878627+0000","last_active":"2024-09-10T07:35:42.483119+0000","last_peered":"2024-09-10T07:35:42.483119+0000","last_clean":"2024-09-10T07:35:42.483119+0000","last_became_active":"2024-09-10T07:33:34.878287+0000","last_became_peered":"2024-09-10T07:33:34.878287+0000","last_unstale":"2024-09-10T07:35:42.483119+0000","last_undegraded":"2024-09-10T07:35:42.483119+0000","last_fullsized":"2024-09-10T07:35:42.483119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:48.283926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502676+0000","last_change":"2024-09-10T07:33:34.878138+0000","last_active":"2024-09-10T07:35:42.502676+0000","last_peered":"2024-09-10T07:35:42.502676+0000","last_clean":"2024-09-10T07:35:42.502676+0000","last_became_active":"2024-09-10T07:33:34.877799+0000","last_became_peered":"2024-09-10T07:33:34.877799+0000","last_unstale":"2024-09-10T07:35:42.502676+0000","last_undegraded":"2024-09-10T07:35:42.502676+0000","last_fullsized":"2024-09-10T07:35:42.502676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:37.686015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"97'9","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502776+0000","last_change":"2024-09-10T07:33:34.876513+0000","last_active":"2024-09-10T07:35:42.502776+0000","last_peered":"2024-09-10T07:35:42.502776+0000","last_clean":"2024-09-10T07:35:42.502776+0000","last_became_active":"2024-09-10T07:33:34.875464+0000","last_became_peered":"2024-09-10T07:33:34.875464+0000","last_unstale":"2024-09-10T07:35:42.502776+0000","last_undegraded":"2024-09-10T07:35:42.502776+0000","last_fullsized":"2024-09-10T07:35:42.502776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:46.956150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.499409+0000","last_change":"2024-09-10T07:33:34.871640+0000","last_active":"2024-09-10T07:35:42.499409+0000","last_peered":"2024-09-10T07:35:42.499409+0000","last_clean":"2024-09-10T07:35:42.499409+0000","last_became_active":"2024-09-10T07:33:34.871327+0000","last_became_peered":"2024-09-10T07:33:34.871327+0000","last_unstale":"2024-09-10T07:35:42.499409+0000","last_undegraded":"2024-09-10T07:35:42.499409+0000","last_fullsized":"2024-09-10T07:35:42.499409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:57.225631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502975+0000","last_change":"2024-09-10T07:33:34.885591+0000","last_active":"2024-09-10T07:35:42.502975+0000","last_peered":"2024-09-10T07:35:42.502975+0000","last_clean":"2024-09-10T07:35:42.502975+0000","last_became_active":"2024-09-10T07:33:34.885290+0000","last_became_peered":"2024-09-10T07:33:34.885290+0000","last_unstale":"2024-09-10T07:35:42.502975+0000","last_undegraded":"2024-09-10T07:35:42.502975+0000","last_fullsized":"2024-09-10T07:35:42.502975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:31.086566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"97'10","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.493842+0000","last_change":"2024-09-10T07:33:34.860150+0000","last_active":"2024-09-10T07:35:42.493842+0000","last_peered":"2024-09-10T07:35:42.493842+0000","last_clean":"2024-09-10T07:35:42.493842+0000","last_became_active":"2024-09-10T07:33:34.859750+0000","last_became_peered":"2024-09-10T07:33:34.859750+0000","last_unstale":"2024-09-10T07:35:42.493842+0000","last_undegraded":"2024-09-10T07:35:42.493842+0000","last_fullsized":"2024-09-10T07:35:42.493842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.826112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"97'7","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.484180+0000","last_change":"2024-09-10T07:33:34.871428+0000","last_active":"2024-09-10T07:35:42.484180+0000","last_peered":"2024-09-10T07:35:42.484180+0000","last_clean":"2024-09-10T07:35:42.484180+0000","last_became_active":"2024-09-10T07:33:34.871088+0000","last_became_peered":"2024-09-10T07:33:34.871088+0000","last_unstale":"2024-09-10T07:35:42.484180+0000","last_undegraded":"2024-09-10T07:35:42.484180+0000","last_fullsized":"2024-09-10T07:35:42.484180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:43.867836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"97'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483929+0000","last_change":"2024-09-10T07:33:34.880016+0000","last_active":"2024-09-10T07:35:42.483929+0000","last_peered":"2024-09-10T07:35:42.483929+0000","last_clean":"2024-09-10T07:35:42.483929+0000","last_became_active":"2024-09-10T07:33:34.878164+0000","last_became_peered":"2024-09-10T07:33:34.878164+0000","last_unstale":"2024-09-10T07:35:42.483929+0000","last_undegraded":"2024-09-10T07:35:42.483929+0000","last_fullsized":"2024-09-10T07:35:42.483929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:37.097815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"97'16","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.483320+0000","last_change":"2024-09-10T07:33:34.872812+0000","last_active":"2024-09-10T07:35:42.483320+0000","last_peered":"2024-09-10T07:35:42.483320+0000","last_clean":"2024-09-10T07:35:42.483320+0000","last_became_active":"2024-09-10T07:33:34.872404+0000","last_became_peered":"2024-09-10T07:33:34.872404+0000","last_unstale":"2024-09-10T07:35:42.483320+0000","last_undegraded":"2024-09-10T07:35:42.483320+0000","last_fullsized":"2024-09-10T07:35:42.483320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:32.683803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"97'11","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.489981+0000","last_change":"2024-09-10T07:33:34.878408+0000","last_active":"2024-09-10T07:35:42.489981+0000","last_peered":"2024-09-10T07:35:42.489981+0000","last_clean":"2024-09-10T07:35:42.489981+0000","last_became_active":"2024-09-10T07:33:34.877910+0000","last_became_peered":"2024-09-10T07:33:34.877910+0000","last_unstale":"2024-09-10T07:35:42.489981+0000","last_undegraded":"2024-09-10T07:35:42.489981+0000","last_fullsized":"2024-09-10T07:35:42.489981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:50:31.299082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"97'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.500857+0000","last_change":"2024-09-10T07:33:34.885639+0000","last_active":"2024-09-10T07:35:42.500857+0000","last_peered":"2024-09-10T07:35:42.500857+0000","last_clean":"2024-09-10T07:35:42.500857+0000","last_became_active":"2024-09-10T07:33:34.881621+0000","last_became_peered":"2024-09-10T07:33:34.881621+0000","last_unstale":"2024-09-10T07:35:42.500857+0000","last_undegraded":"2024-09-10T07:35:42.500857+0000","last_fullsized":"2024-09-10T07:35:42.500857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:15.917864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"97'5","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.478497+0000","last_change":"2024-09-10T07:33:34.881523+0000","last_active":"2024-09-10T07:35:42.478497+0000","last_peered":"2024-09-10T07:35:42.478497+0000","last_clean":"2024-09-10T07:35:42.478497+0000","last_became_active":"2024-09-10T07:33:34.880493+0000","last_became_peered":"2024-09-10T07:33:34.880493+0000","last_unstale":"2024-09-10T07:35:42.478497+0000","last_undegraded":"2024-09-10T07:35:42.478497+0000","last_fullsized":"2024-09-10T07:35:42.478497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:06.493537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"97'6","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.487137+0000","last_change":"2024-09-10T07:33:34.870711+0000","last_active":"2024-09-10T07:35:42.487137+0000","last_peered":"2024-09-10T07:35:42.487137+0000","last_clean":"2024-09-10T07:35:42.487137+0000","last_became_active":"2024-09-10T07:33:34.870295+0000","last_became_peered":"2024-09-10T07:33:34.870295+0000","last_unstale":"2024-09-10T07:35:42.487137+0000","last_undegraded":"2024-09-10T07:35:42.487137+0000","last_fullsized":"2024-09-10T07:35:42.487137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:17.998912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"97'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.496842+0000","last_change":"2024-09-10T07:33:34.877339+0000","last_active":"2024-09-10T07:35:42.496842+0000","last_peered":"2024-09-10T07:35:42.496842+0000","last_clean":"2024-09-10T07:35:42.496842+0000","last_became_active":"2024-09-10T07:33:34.876785+0000","last_became_peered":"2024-09-10T07:33:34.876785+0000","last_unstale":"2024-09-10T07:35:42.496842+0000","last_undegraded":"2024-09-10T07:35:42.496842+0000","last_fullsized":"2024-09-10T07:35:42.496842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:14:17.861718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"97'8","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.476981+0000","last_change":"2024-09-10T07:33:34.871823+0000","last_active":"2024-09-10T07:35:42.476981+0000","last_peered":"2024-09-10T07:35:42.476981+0000","last_clean":"2024-09-10T07:35:42.476981+0000","last_became_active":"2024-09-10T07:33:34.871490+0000","last_became_peered":"2024-09-10T07:33:34.871490+0000","last_unstale":"2024-09-10T07:35:42.476981+0000","last_undegraded":"2024-09-10T07:35:42.476981+0000","last_fullsized":"2024-09-10T07:35:42.476981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.736469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.494172+0000","last_change":"2024-09-10T07:33:34.886278+0000","last_active":"2024-09-10T07:35:42.494172+0000","last_peered":"2024-09-10T07:35:42.494172+0000","last_clean":"2024-09-10T07:35:42.494172+0000","last_became_active":"2024-09-10T07:33:34.885129+0000","last_became_peered":"2024-09-10T07:33:34.885129+0000","last_unstale":"2024-09-10T07:35:42.494172+0000","last_undegraded":"2024-09-10T07:35:42.494172+0000","last_fullsized":"2024-09-10T07:35:42.494172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:30.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502201+0000","last_change":"2024-09-10T07:33:34.885908+0000","last_active":"2024-09-10T07:35:42.502201+0000","last_peered":"2024-09-10T07:35:42.502201+0000","last_clean":"2024-09-10T07:35:42.502201+0000","last_became_active":"2024-09-10T07:33:34.880437+0000","last_became_peered":"2024-09-10T07:33:34.880437+0000","last_unstale":"2024-09-10T07:35:42.502201+0000","last_undegraded":"2024-09-10T07:35:42.502201+0000","last_fullsized":"2024-09-10T07:35:42.502201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:21.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"97'8","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-10T07:35:42.502439+0000","last_change":"2024-09-10T07:33:34.876627+0000","last_active":"2024-09-10T07:35:42.502439+0000","last_peered":"2024-09-10T07:35:42.502439+0000","last_clean":"2024-09-10T07:35:42.502439+0000","last_became_active":"2024-09-10T07:33:34.876336+0000","last_became_peered":"2024-09-10T07:33:34.876336+0000","last_unstale":"2024-09-10T07:35:42.502439+0000","last_undegraded":"2024-09-10T07:35:42.502439+0000","last_fullsized":"2024-09-10T07:35:42.502439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:33:33.841635+0000","last_clean_scrub_stamp":"2024-09-10T07:33:33.841635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:59.251431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78045,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1441,"num_read_kb":806,"num_write":265,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":146,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":234135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":216016,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449141,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":561212,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710778,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1241088,"data_stored":1218403,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874302,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":565058,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":561228,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364873,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1158514,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528399,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":557000,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790038,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1153562,"data_compressed":68945,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953562,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":560540,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":556174,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1171456,"data_stored":628737,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":628585,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":557922,"data_compressed":32393,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34904,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36552,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":72993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:36:49.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:36:49.950 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 -- 172.21.15.46:0/453453425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f541047e0 msgr2=0x7f7f54108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 --2- 172.21.15.46:0/453453425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f541047e0 0x7f7f54108830 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009900 tx=0x7f7f3c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 -- 172.21.15.46:0/453453425 shutdown_connections 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 --2- 172.21.15.46:0/453453425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f54108f60 0x7f7f54110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 --2- 172.21.15.46:0/453453425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f541047e0 0x7f7f54108830 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 --2- 172.21.15.46:0/453453425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f54103e30 0x7f7f54104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 -- 172.21.15.46:0/453453425 >> 172.21.15.46:0/453453425 conn(0x7f7f540ff910 msgr2=0x7f7f54101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 -- 172.21.15.46:0/453453425 shutdown_connections 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.541+0000 7f7f53e00640 1 -- 172.21.15.46:0/453453425 wait complete. 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.542+0000 7f7f53e00640 1 Processor -- start 2024-09-10T07:36:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.542+0000 7f7f53e00640 1 -- start start 2024-09-10T07:36:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.542+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f54103e30 0x7f7f5419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f541047e0 0x7f7f5419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f54108f60 0x7f7f5419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54113f80 con 0x7f7f54108f60 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54113e00 con 0x7f7f54103e30 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54114100 con 0x7f7f541047e0 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f52a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f54103e30 0x7f7f5419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f54108f60 0x7f7f5419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f4a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f541047e0 0x7f7f5419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f52a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f54103e30 0x7f7f5419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36106/0 (socket says 172.21.15.46:36106) 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f52a00640 1 -- 172.21.15.46:0/3691721582 learned_addr learned my addr 172.21.15.46:0/3691721582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53400640 1 -- 172.21.15.46:0/3691721582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f541047e0 msgr2=0x7f7f5419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53400640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f541047e0 0x7f7f5419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53400640 1 -- 172.21.15.46:0/3691721582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f54103e30 msgr2=0x7f7f5419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53400640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f54103e30 0x7f7f5419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.543+0000 7f7f53400640 1 -- 172.21.15.46:0/3691721582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5419f410 con 0x7f7f54108f60 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.544+0000 7f7f53400640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f54108f60 0x7f7f5419ed30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009fd0 tx=0x7f7f4c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.544+0000 7f7f4be00640 1 -- 172.21.15.46:0/3691721582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00ce20 con 0x7f7f54108f60 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.544+0000 7f7f4be00640 1 -- 172.21.15.46:0/3691721582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c005020 con 0x7f7f54108f60 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.544+0000 7f7f4be00640 1 -- 172.21.15.46:0/3691721582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c010720 con 0x7f7f54108f60 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.544+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5419f700 con 0x7f7f54108f60 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.544+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f541a6fe0 con 0x7f7f54108f60 2024-09-10T07:36:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.545+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f54105310 con 0x7f7f54108f60 2024-09-10T07:36:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.547+0000 7f7f4be00640 1 -- 172.21.15.46:0/3691721582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f4c005540 con 0x7f7f54108f60 2024-09-10T07:36:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.549+0000 7f7f4be00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f380a2c00 0x7f7f380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.550+0000 7f7f4be00640 1 -- 172.21.15.46:0/3691721582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7f4c0cf9e0 con 0x7f7f54108f60 2024-09-10T07:36:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.550+0000 7f7f52a00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f380a2c00 0x7f7f380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.551+0000 7f7f52a00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f380a2c00 0x7f7f380a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f440095f0 tx=0x7f7f44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.553+0000 7f7f4be00640 1 -- 172.21.15.46:0/3691721582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f4c09c560 con 0x7f7f54108f60 2024-09-10T07:36:50.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:50 smithi181 ceph-mon[35397]: pgmap v405: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T07:36:50.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:50 smithi181 ceph-mon[35397]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:50.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:50 smithi063 ceph-mon[35020]: pgmap v405: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T07:36:50.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:50 smithi063 ceph-mon[35020]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:50.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:50 smithi046 ceph-mon[30705]: pgmap v405: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T07:36:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:50 smithi046 ceph-mon[30705]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:36:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.774+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f5402d7c0 con 0x7f7f54108f60 2024-09-10T07:36:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.775+0000 7f7f4be00640 1 -- 172.21.15.46:0/3691721582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f7f4c09be60 con 0x7f7f54108f60 2024-09-10T07:36:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:50.775 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"btime":"2024-09-10T07:36:31:116553+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:36:30.952730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:36:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f380a2c00 msgr2=0x7f7f380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f380a2c00 0x7f7f380a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f440095f0 tx=0x7f7f44005f90 comp rx=0 tx=0).stop 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f54108f60 msgr2=0x7f7f5419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f54108f60 0x7f7f5419ed30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009fd0 tx=0x7f7f4c00ef10 comp rx=0 tx=0).stop 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 shutdown_connections 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f380a2c00 0x7f7f380a50c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f54108f60 0x7f7f5419ed30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f541047e0 0x7f7f5419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 --2- 172.21.15.46:0/3691721582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f54103e30 0x7f7f5419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 >> 172.21.15.46:0/3691721582 conn(0x7f7f540ff910 msgr2=0x7f7f54100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 shutdown_connections 2024-09-10T07:36:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:50.777+0000 7f7f53e00640 1 -- 172.21.15.46:0/3691721582 wait complete. 2024-09-10T07:36:51.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:51 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3691721582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:51.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:36:51.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:51 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3691721582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:51.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:51 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3691721582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 -- 172.21.15.46:0/1425289234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 msgr2=0x7f8b041043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 --2- 172.21.15.46:0/1425289234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 0x7f8b041043f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009860 tx=0x7f8af402a340 comp rx=0 tx=0).stop 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 -- 172.21.15.46:0/1425289234 shutdown_connections 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 --2- 172.21.15.46:0/1425289234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b04109140 0x7f8b04110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 --2- 172.21.15.46:0/1425289234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041049c0 0x7f8b04108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 --2- 172.21.15.46:0/1425289234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 0x7f8b041043f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 -- 172.21.15.46:0/1425289234 >> 172.21.15.46:0/1425289234 conn(0x7f8b040ffa50 msgr2=0x7f8b04101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 -- 172.21.15.46:0/1425289234 shutdown_connections 2024-09-10T07:36:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.573+0000 7f8b09c00640 1 -- 172.21.15.46:0/1425289234 wait complete. 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.574+0000 7f8b09c00640 1 Processor -- start 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.574+0000 7f8b09c00640 1 -- start start 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.574+0000 7f8b09c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 0x7f8b0419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.574+0000 7f8b09c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041049c0 0x7f8b0419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.574+0000 7f8b09c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b04109140 0x7f8b0419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.574+0000 7f8b09c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8b04114170 con 0x7f8b041049c0 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.574+0000 7f8b09c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b04113ff0 con 0x7f8b04104010 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8b09c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b041142f0 con 0x7f8b04109140 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8afa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041049c0 0x7f8b0419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8b02a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 0x7f8b0419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8b02a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 0x7f8b0419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36124/0 (socket says 172.21.15.46:36124) 2024-09-10T07:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8b02a00640 1 -- 172.21.15.46:0/2790027077 learned_addr learned my addr 172.21.15.46:0/2790027077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8b03400640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b04109140 0x7f8b0419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8afa000640 1 -- 172.21.15.46:0/2790027077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b04109140 msgr2=0x7f8b0419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8afa000640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b04109140 0x7f8b0419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8afa000640 1 -- 172.21.15.46:0/2790027077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 msgr2=0x7f8b0419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8afa000640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 0x7f8b0419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8afa000640 1 -- 172.21.15.46:0/2790027077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0419f5e0 con 0x7f8b041049c0 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.575+0000 7f8b02a00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 0x7f8b0419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.576+0000 7f8afa000640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041049c0 0x7f8b0419ab70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00e790 tx=0x7f8aec00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.576+0000 7f8afbe00640 1 -- 172.21.15.46:0/2790027077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec00eea0 con 0x7f8b041049c0 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.576+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0419f8d0 con 0x7f8b041049c0 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.576+0000 7f8afbe00640 1 -- 172.21.15.46:0/2790027077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aec00ccc0 con 0x7f8b041049c0 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.576+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b041a71b0 con 0x7f8b041049c0 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.576+0000 7f8afbe00640 1 -- 172.21.15.46:0/2790027077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec018720 con 0x7f8b041049c0 2024-09-10T07:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.577+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b041043f0 con 0x7f8b041049c0 2024-09-10T07:36:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.580+0000 7f8afbe00640 1 -- 172.21.15.46:0/2790027077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8aec010430 con 0x7f8b041049c0 2024-09-10T07:36:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.581+0000 7f8afbe00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ae80a2b80 0x7f8ae80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.581+0000 7f8afbe00640 1 -- 172.21.15.46:0/2790027077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8aec0cf560 con 0x7f8b041049c0 2024-09-10T07:36:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.581+0000 7f8b02a00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ae80a2b80 0x7f8ae80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.582+0000 7f8b02a00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ae80a2b80 0x7f8ae80a5040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8af40057c0 tx=0x7f8af40056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.585+0000 7f8afbe00640 1 -- 172.21.15.46:0/2790027077 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8aec09ba90 con 0x7f8b041049c0 2024-09-10T07:36:52.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:52 smithi181 ceph-mon[35397]: pgmap v406: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T07:36:52.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:52 smithi063 ceph-mon[35020]: pgmap v406: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T07:36:52.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:52 smithi046 ceph-mon[30705]: pgmap v406: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T07:36:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.810+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b0402d7c0 con 0x7f8b041049c0 2024-09-10T07:36:52.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.811+0000 7f8afbe00640 1 -- 172.21.15.46:0/2790027077 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f8aec09fd90 con 0x7f8b041049c0 2024-09-10T07:36:52.810 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:52.810 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"btime":"2024-09-10T07:36:31:116553+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:36:30.952730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:36:52.811 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ae80a2b80 msgr2=0x7f8ae80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ae80a2b80 0x7f8ae80a5040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8af40057c0 tx=0x7f8af40056d0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041049c0 msgr2=0x7f8b0419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041049c0 0x7f8b0419ab70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00e790 tx=0x7f8aec00ec50 comp rx=0 tx=0).stop 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 shutdown_connections 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b04109140 0x7f8b0419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ae80a2b80 0x7f8ae80a5040 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041049c0 0x7f8b0419ab70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 --2- 172.21.15.46:0/2790027077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04104010 0x7f8b0419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 >> 172.21.15.46:0/2790027077 conn(0x7f8b040ffa50 msgr2=0x7f8b04100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.813+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 shutdown_connections 2024-09-10T07:36:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:52.814+0000 7f8b09c00640 1 -- 172.21.15.46:0/2790027077 wait complete. 2024-09-10T07:36:53.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:53 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2790027077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:53.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:36:53.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:53 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2790027077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:53.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:53 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2790027077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 -- 172.21.15.46:0/1354708020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80102d50 msgr2=0x7f3b801031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 --2- 172.21.15.46:0/1354708020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80102d50 0x7f3b801031d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009900 tx=0x7f3b6c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 -- 172.21.15.46:0/1354708020 shutdown_connections 2024-09-10T07:36:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 --2- 172.21.15.46:0/1354708020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80103710 0x7f3b80109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 --2- 172.21.15.46:0/1354708020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80102d50 0x7f3b801031d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 --2- 172.21.15.46:0/1354708020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b80101b60 0x7f3b80101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 -- 172.21.15.46:0/1354708020 >> 172.21.15.46:0/1354708020 conn(0x7f3b800fd2e0 msgr2=0x7f3b800ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 -- 172.21.15.46:0/1354708020 shutdown_connections 2024-09-10T07:36:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.019+0000 7f3b87200640 1 -- 172.21.15.46:0/1354708020 wait complete. 2024-09-10T07:36:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.020+0000 7f3b87200640 1 Processor -- start 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b87200640 1 -- start start 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b87200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80101b60 0x7f3b8019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b87200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80102d50 0x7f3b8019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b87200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b80103710 0x7f3b801a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b87200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8010d830 con 0x7f3b80103710 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b87200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8010d6b0 con 0x7f3b80101b60 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b87200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8010d9b0 con 0x7f3b80102d50 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b85200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b80103710 0x7f3b801a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80101b60 0x7f3b8019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80102d50 0x7f3b8019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80101b60 0x7f3b8019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36148/0 (socket says 172.21.15.46:36148) 2024-09-10T07:36:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b7fe00640 1 -- 172.21.15.46:0/807974190 learned_addr learned my addr 172.21.15.46:0/807974190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.021+0000 7f3b85200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b80103710 0x7f3b801a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37004/0 (socket says 172.21.15.46:37004) 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7fe00640 1 -- 172.21.15.46:0/807974190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80102d50 msgr2=0x7f3b8019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80102d50 0x7f3b8019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7fe00640 1 -- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b80103710 msgr2=0x7f3b801a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b80103710 0x7f3b801a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7fe00640 1 -- 172.21.15.46:0/807974190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b801a1c60 con 0x7f3b80101b60 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80101b60 0x7f3b8019a040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000d770 tx=0x7f3b7000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7cc00640 1 -- 172.21.15.46:0/807974190 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70017070 con 0x7f3b80101b60 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7cc00640 1 -- 172.21.15.46:0/807974190 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b70009c30 con 0x7f3b80101b60 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b87200640 1 -- 172.21.15.46:0/807974190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b801a1f50 con 0x7f3b80101b60 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b7cc00640 1 -- 172.21.15.46:0/807974190 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70003dd0 con 0x7f3b80101b60 2024-09-10T07:36:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.022+0000 7f3b87200640 1 -- 172.21.15.46:0/807974190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b801a23c0 con 0x7f3b80101b60 2024-09-10T07:36:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b8a08d740 1 Processor -- start 2024-09-10T07:36:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b8a08d740 1 -- start start 2024-09-10T07:36:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b7cc00640 1 -- 172.21.15.46:0/807974190 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b70010460 con 0x7f3b80101b60 2024-09-10T07:36:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e090 0x55dfd887e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd8883490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd8883a60 0x55dfd8885f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dfd87e3100 con 0x55dfd887e090 2024-09-10T07:36:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dfd878b1b0 con 0x55dfd887e9d0 2024-09-10T07:36:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.026+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dfd87df0c0 con 0x55dfd8883a60 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd8883a60 0x55dfd8885f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd8883a60 0x55dfd8885f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34782/0 (socket says 172.21.15.46:34782) 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7f400640 1 -- 172.21.15.46:0/3346775895 learned_addr learned my addr 172.21.15.46:0/3346775895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b85200640 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e090 0x55dfd887e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd8883490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7f400640 1 -- 172.21.15.46:0/3346775895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 msgr2=0x55dfd8883490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7f400640 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd8883490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7f400640 1 -- 172.21.15.46:0/3346775895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e090 msgr2=0x55dfd887e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7f400640 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e090 0x55dfd887e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7f400640 1 -- 172.21.15.46:0/3346775895 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd84f6210 con 0x55dfd8883a60 2024-09-10T07:36:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.027+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd8883490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:36:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.028+0000 7f3b7f400640 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd8883a60 0x55dfd8885f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009900 tx=0x7f3b6c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea07abb7131a0a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.028+0000 7f3b75600640 1 -- 172.21.15.46:0/3346775895 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c035040 con 0x55dfd8883a60 2024-09-10T07:36:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.028+0000 7f3b75600640 1 -- 172.21.15.46:0/3346775895 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b6c004480 con 0x55dfd8883a60 2024-09-10T07:36:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.028+0000 7f3b75600640 1 -- 172.21.15.46:0/3346775895 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c031910 con 0x55dfd8883a60 2024-09-10T07:36:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.028+0000 7f3b7cc00640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b640a2cb0 0x7f3b640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.028+0000 7f3b7cc00640 1 -- 172.21.15.46:0/807974190 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b700cec80 con 0x7f3b80101b60 2024-09-10T07:36:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.028+0000 7f3b85200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b640a2cb0 0x7f3b640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 -- 172.21.15.46:0/3346775895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd8883a60 msgr2=0x55dfd8885f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd8883a60 0x55dfd8885f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009900 tx=0x7f3b6c005e80 comp rx=0 tx=0).stop 2024-09-10T07:36:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 -- 172.21.15.46:0/3346775895 shutdown_connections 2024-09-10T07:36:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd8883a60 0x55dfd8885f20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd8883490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/3346775895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e090 0x55dfd887e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 -- 172.21.15.46:0/3346775895 >> 172.21.15.46:0/3346775895 conn(0x55dfd8879700 msgr2=0x55dfd8879b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b85200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b640a2cb0 0x7f3b640a5170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3b78005fd0 tx=0x7f3b78009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 -- 172.21.15.46:0/3346775895 shutdown_connections 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.029+0000 7f3b8a08d740 1 -- 172.21.15.46:0/3346775895 wait complete. 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.030+0000 7f3b8a08d740 1 -- start start 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.030+0000 7f3b8a08d740 1 Processor -- start 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.030+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd88881f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.030+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e9d0 0x55dfd8888730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.030+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd8883a60 0x55dfd888f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.030+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dfd88869d0 con 0x55dfd887e9d0 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.030+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dfd876c090 con 0x55dfd8883a60 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.030+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dfd8886b50 con 0x55dfd887e090 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b8a08d740 10 client.0 _ll_register_callbacks cb 0x55dfd87c0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b85200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd8883a60 0x55dfd888f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e9d0 0x55dfd8888730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b85200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd8883a60 0x55dfd888f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36180/0 (socket says 172.21.15.46:36180) 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b85200640 1 -- 172.21.15.46:0/2505702046 learned_addr learned my addr 172.21.15.46:0/2505702046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd88881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b85200640 1 -- 172.21.15.46:0/2505702046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 msgr2=0x55dfd88881f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b85200640 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd88881f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b85200640 1 -- 172.21.15.46:0/2505702046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e9d0 msgr2=0x55dfd8888730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b85200640 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e9d0 0x55dfd8888730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b85200640 1 -- 172.21.15.46:0/2505702046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd85c2620 con 0x55dfd8883a60 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.031+0000 7f3b7f400640 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e9d0 0x55dfd8888730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b85200640 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd8883a60 0x55dfd888f770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009370 tx=0x7f3b78004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e8057a07dfa958 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b6a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:36:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b6a000640 1 -- 172.21.15.46:0/2505702046 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800f830 con 0x55dfd8883a60 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b8a08d740 1 -- 172.21.15.46:0/2505702046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b6c004480 con 0x55dfd8883a60 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b6a000640 1 -- 172.21.15.46:0/2505702046 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b7800f9d0 con 0x55dfd8883a60 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b6a000640 1 -- 172.21.15.46:0/2505702046 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b7800fbd0 con 0x55dfd8883a60 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b8a08d740 10 client.25352 fetch_fsmap learned FSMap version 30 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b8a08d740 1 -- 172.21.15.46:0/2505702046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfd88925f0 con 0x55dfd8883a60 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.032+0000 7f3b6a000640 1 -- 172.21.15.46:0/2505702046 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b6c004480 con 0x55dfd8883a60 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.033+0000 7f3b6a000640 1 -- 172.21.15.46:0/2505702046 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3b78027070 con 0x55dfd8883a60 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.033+0000 7f3b8a08d740 10 client.25352 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.033+0000 7f3b8a08d740 10 client.25352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.033+0000 7f3b8a08d740 20 client.25352 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.033+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dfd8893ce0 0x55dfd88960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.033+0000 7f3b8a08d740 4 client.25352 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:36:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.033+0000 7f3b8a08d740 1 -- 172.21.15.46:0/2505702046 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfd87f0830 con 0x55dfd8893ce0 2024-09-10T07:36:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.033+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dfd8893ce0 0x55dfd88960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.034+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dfd8893ce0 0x55dfd88960d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a8edcf0885cb4fa server_cookie=7368a24ce74e0fe2 in_seq=0 out_seq=0 2024-09-10T07:36:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.034+0000 7f3b6a000640 10 client.25352 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.034+0000 7f3b6a000640 1 -- 172.21.15.46:0/2505702046 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfd87f0830 con 0x55dfd8893ce0 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.034+0000 7f3b6a000640 10 client.25352 handle_command_reply: tid=0 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.034+0000 7f3b8a08d740 1 client.25352 shutdown 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.034+0000 7f3b8a08d740 1 -- 172.21.15.46:0/2505702046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd8883a60 msgr2=0x55dfd888f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.034+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd8883a60 0x55dfd888f770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009370 tx=0x7f3b78004a40 comp rx=0 tx=0).stop 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 1 -- 172.21.15.46:0/2505702046 shutdown_connections 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dfd8893ce0 0x55dfd88960d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd8883a60 0x55dfd888f770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd887e9d0 0x55dfd8888730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/2505702046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd88881f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b6a000640 0 client.25352 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 1 -- 172.21.15.46:0/2505702046 >> 172.21.15.46:0/2505702046 conn(0x55dfd8879700 msgr2=0x55dfd887af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 1 -- 172.21.15.46:0/2505702046 shutdown_connections 2024-09-10T07:36:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 1 -- 172.21.15.46:0/2505702046 wait complete. 2024-09-10T07:36:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.035+0000 7f3b8a08d740 20 client.25352 trim_cache size 0 max 16384 2024-09-10T07:36:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.050+0000 7f3b8a08d740 1 Processor -- start 2024-09-10T07:36:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.050+0000 7f3b8a08d740 1 -- start start 2024-09-10T07:36:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.050+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e090 0x55dfd887a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.050+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e9d0 0x55dfd888f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.050+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd887aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.050+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801ac8e0 con 0x55dfd8883a60 2024-09-10T07:36:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.050+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801ae420 con 0x55dfd887e090 2024-09-10T07:36:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.050+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801acfb0 con 0x55dfd887e9d0 2024-09-10T07:36:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b85200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd887aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e090 0x55dfd887a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e090 0x55dfd887a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36190/0 (socket says 172.21.15.46:36190) 2024-09-10T07:36:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 -- 172.21.15.46:0/556095274 learned_addr learned my addr 172.21.15.46:0/556095274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7f400640 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e9d0 0x55dfd888f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 -- 172.21.15.46:0/556095274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e9d0 msgr2=0x55dfd888f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e9d0 0x55dfd888f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 -- 172.21.15.46:0/556095274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 msgr2=0x55dfd887aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd887aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 -- 172.21.15.46:0/556095274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd887f660 con 0x55dfd887e090 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b85200640 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd887aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.051+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e090 0x55dfd887a5b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b7009b140 tx=0x7f3b70003870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa160daf1364543f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b74c00640 1 -- 172.21.15.46:0/556095274 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70020040 con 0x55dfd887e090 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b74c00640 1 -- 172.21.15.46:0/556095274 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b700a3d70 con 0x55dfd887e090 2024-09-10T07:36:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b74c00640 1 -- 172.21.15.46:0/556095274 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70097820 con 0x55dfd887e090 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b8a08d740 1 -- 172.21.15.46:0/556095274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e090 msgr2=0x55dfd887a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e090 0x55dfd887a5b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b7009b140 tx=0x7f3b70003870 comp rx=0 tx=0).stop 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b8a08d740 1 -- 172.21.15.46:0/556095274 shutdown_connections 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd887aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e9d0 0x55dfd888f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/556095274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e090 0x55dfd887a5b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b8a08d740 1 -- 172.21.15.46:0/556095274 >> 172.21.15.46:0/556095274 conn(0x55dfd8879700 msgr2=0x55dfd88881f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.052+0000 7f3b8a08d740 1 -- 172.21.15.46:0/556095274 shutdown_connections 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.053+0000 7f3b8a08d740 1 -- 172.21.15.46:0/556095274 wait complete. 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.053+0000 7f3b8a08d740 1 -- start start 2024-09-10T07:36:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.053+0000 7f3b8a08d740 1 Processor -- start 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.053+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd888c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd888ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b8a08d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd888ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801add50 con 0x55dfd8883a60 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801ad680 con 0x55dfd887e9d0 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b8a08d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b801aa8c0 con 0x55dfd887e090 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b8a08d740 10 client.0 _ll_register_callbacks cb 0x7f3b78009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b85200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd888ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b85200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd888ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37042/0 (socket says 172.21.15.46:37042) 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b85200640 1 -- 172.21.15.46:0/1844765970 learned_addr learned my addr 172.21.15.46:0/1844765970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b7f400640 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd888ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd888c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b85200640 1 -- 172.21.15.46:0/1844765970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 msgr2=0x55dfd888c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b85200640 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd888c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b85200640 1 -- 172.21.15.46:0/1844765970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 msgr2=0x55dfd888ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b85200640 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd888ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b85200640 1 -- 172.21.15.46:0/1844765970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd888e3d0 con 0x55dfd8883a60 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.054+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd888c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b85200640 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd888ddc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b78005b00 tx=0x7f3b78009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958452da6ea82792 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b6a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b6a000640 1 -- 172.21.15.46:0/1844765970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7803c060 con 0x55dfd8883a60 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b6a000640 1 -- 172.21.15.46:0/1844765970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b78037440 con 0x55dfd8883a60 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b8a08d740 1 -- 172.21.15.46:0/1844765970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b700a3d70 con 0x55dfd8883a60 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b6a000640 1 -- 172.21.15.46:0/1844765970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b78037640 con 0x55dfd8883a60 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b8a08d740 10 client.15330 fetch_fsmap learned FSMap version 30 2024-09-10T07:36:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b8a08d740 1 -- 172.21.15.46:0/1844765970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfd888f7b0 con 0x55dfd8883a60 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.055+0000 7f3b6a000640 1 -- 172.21.15.46:0/1844765970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b700a3d70 con 0x55dfd8883a60 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.056+0000 7f3b6a000640 1 -- 172.21.15.46:0/1844765970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3b78027070 con 0x55dfd8883a60 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.056+0000 7f3b8a08d740 10 client.15330 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.056+0000 7f3b8a08d740 10 client.15330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.056+0000 7f3b8a08d740 20 client.15330 populate_metadata read hostname 'smithi046' 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.056+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dfd88feee0 0x55dfd8901300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.056+0000 7f3b8a08d740 4 client.15330 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.056+0000 7f3b8a08d740 1 -- 172.21.15.46:0/1844765970 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfd87eab80 con 0x55dfd88feee0 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.056+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dfd88feee0 0x55dfd8901300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.057+0000 7f3b7fe00640 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dfd88feee0 0x55dfd8901300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe3a17b5371a1cf server_cookie=4ec7948d66b6b0cf in_seq=0 out_seq=0 2024-09-10T07:36:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.057+0000 7f3b6a000640 10 client.15330 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.069+0000 7f3b6a000640 1 -- 172.21.15.46:0/1844765970 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfd87eab80 con 0x55dfd88feee0 2024-09-10T07:36:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.069+0000 7f3b6a000640 10 client.15330 handle_command_reply: tid=0 2024-09-10T07:36:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.069+0000 7f3b8a08d740 1 client.15330 shutdown 2024-09-10T07:36:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.069+0000 7f3b8a08d740 1 -- 172.21.15.46:0/1844765970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 msgr2=0x55dfd888ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.069+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd888ddc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b78005b00 tx=0x7f3b78009ee0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.069+0000 7f3b8a08d740 1 -- 172.21.15.46:0/1844765970 shutdown_connections 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dfd88feee0 0x55dfd8901300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfd8883a60 0x55dfd888ddc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dfd887e9d0 0x55dfd888ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b8a08d740 1 --2- 172.21.15.46:0/1844765970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfd887e090 0x55dfd888c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b6a000640 0 client.15330 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b8a08d740 1 -- 172.21.15.46:0/1844765970 >> 172.21.15.46:0/1844765970 conn(0x55dfd8879700 msgr2=0x55dfd8878060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b8a08d740 1 -- 172.21.15.46:0/1844765970 shutdown_connections 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b8a08d740 1 -- 172.21.15.46:0/1844765970 wait complete. 2024-09-10T07:36:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b8a08d740 20 client.15330 trim_cache size 0 max 16384 2024-09-10T07:36:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:36:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:36:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:36:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:36:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.070+0000 7f3b87200640 1 -- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b640a2cb0 msgr2=0x7f3b640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b640a2cb0 0x7f3b640a5170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3b78005fd0 tx=0x7f3b78009500 comp rx=0 tx=0).stop 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 -- 172.21.15.46:0/807974190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80101b60 msgr2=0x7f3b8019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80101b60 0x7f3b8019a040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000d770 tx=0x7f3b7000dc30 comp rx=0 tx=0).stop 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 -- 172.21.15.46:0/807974190 shutdown_connections 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b640a2cb0 0x7f3b640a5170 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b80103710 0x7f3b801a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b80102d50 0x7f3b8019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 --2- 172.21.15.46:0/807974190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b80101b60 0x7f3b8019a040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 -- 172.21.15.46:0/807974190 >> 172.21.15.46:0/807974190 conn(0x7f3b800fd2e0 msgr2=0x7f3b80074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 -- 172.21.15.46:0/807974190 shutdown_connections 2024-09-10T07:36:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:36:54.071+0000 7f3b87200640 1 -- 172.21.15.46:0/807974190 wait complete. 2024-09-10T07:36:54.529 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:54 smithi046 ceph-mon[30705]: pgmap v407: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T07:36:54.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:54 smithi181 ceph-mon[35397]: pgmap v407: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T07:36:54.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:54 smithi063 ceph-mon[35020]: pgmap v407: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T07:36:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 -- 172.21.15.46:0/2960929092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa008103ff0 msgr2=0x7fa0081043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 --2- 172.21.15.46:0/2960929092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa008103ff0 0x7fa0081043d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009900 tx=0x7f9ff802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 -- 172.21.15.46:0/2960929092 shutdown_connections 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 --2- 172.21.15.46:0/2960929092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa008109120 0x7fa008110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 --2- 172.21.15.46:0/2960929092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0081049a0 0x7fa0081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 --2- 172.21.15.46:0/2960929092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa008103ff0 0x7fa0081043d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 -- 172.21.15.46:0/2960929092 >> 172.21.15.46:0/2960929092 conn(0x7fa0080ffa90 msgr2=0x7fa008101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 -- 172.21.15.46:0/2960929092 shutdown_connections 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.644+0000 7fa00de00640 1 -- 172.21.15.46:0/2960929092 wait complete. 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.645+0000 7fa00de00640 1 Processor -- start 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.645+0000 7fa00de00640 1 -- start start 2024-09-10T07:36:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa00de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa008103ff0 0x7fa00819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa00de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0081049a0 0x7fa00819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa00de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa008109120 0x7fa00819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa00de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa0081141a0 con 0x7fa0081049a0 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa00de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa008114020 con 0x7fa008103ff0 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa00de00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa008114320 con 0x7fa008109120 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7f9ffe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0081049a0 0x7fa00819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa006a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa008103ff0 0x7fa00819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa007400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa008109120 0x7fa00819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7f9ffe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0081049a0 0x7fa00819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37050/0 (socket says 172.21.15.46:37050) 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7f9ffe000640 1 -- 172.21.15.46:0/50842207 learned_addr learned my addr 172.21.15.46:0/50842207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7fa006a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa008103ff0 0x7fa00819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36208/0 (socket says 172.21.15.46:36208) 2024-09-10T07:36:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7f9ffe000640 1 -- 172.21.15.46:0/50842207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa008109120 msgr2=0x7fa00819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7f9ffe000640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa008109120 0x7fa00819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7f9ffe000640 1 -- 172.21.15.46:0/50842207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa008103ff0 msgr2=0x7fa00819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7f9ffe000640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa008103ff0 0x7fa00819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.646+0000 7f9ffe000640 1 -- 172.21.15.46:0/50842207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00819f5d0 con 0x7fa0081049a0 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.647+0000 7f9ffe000640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0081049a0 0x7fa00819ab60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000ea60 tx=0x7f9ff000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.647+0000 7fa00ca00640 1 -- 172.21.15.46:0/50842207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000cde0 con 0x7fa0081049a0 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.647+0000 7fa00ca00640 1 -- 172.21.15.46:0/50842207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ff00035f0 con 0x7fa0081049a0 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.647+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00819f8c0 con 0x7fa0081049a0 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.647+0000 7fa00ca00640 1 -- 172.21.15.46:0/50842207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0010670 con 0x7fa0081049a0 2024-09-10T07:36:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.647+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0081a71a0 con 0x7fa0081049a0 2024-09-10T07:36:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.648+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0081043d0 con 0x7fa0081049a0 2024-09-10T07:36:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.650+0000 7fa00ca00640 1 -- 172.21.15.46:0/50842207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ff00037a0 con 0x7fa0081049a0 2024-09-10T07:36:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.651+0000 7fa00ca00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fec0a2b80 0x7f9fec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.652+0000 7fa00ca00640 1 -- 172.21.15.46:0/50842207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9ff00cf7c0 con 0x7fa0081049a0 2024-09-10T07:36:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.652+0000 7fa006a00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fec0a2b80 0x7f9fec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.652+0000 7fa006a00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fec0a2b80 0x7f9fec0a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8000c00 tx=0x7f9ff8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.653+0000 7fa00ca00640 1 -- 172.21.15.46:0/50842207 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ff009c3d0 con 0x7fa0081049a0 2024-09-10T07:36:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.842+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa00802d7c0 con 0x7fa0081049a0 2024-09-10T07:36:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.844+0000 7fa00ca00640 1 -- 172.21.15.46:0/50842207 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24700 (secure 0 0 0) 0x7f9ff009bcd0 con 0x7fa0081049a0 2024-09-10T07:36:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:54.845 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:35:14.826944+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fec0a2b80 msgr2=0x7f9fec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fec0a2b80 0x7f9fec0a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8000c00 tx=0x7f9ff8005b50 comp rx=0 tx=0).stop 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0081049a0 msgr2=0x7fa00819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0081049a0 0x7fa00819ab60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000ea60 tx=0x7f9ff000ef20 comp rx=0 tx=0).stop 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 shutdown_connections 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa008109120 0x7fa00819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fec0a2b80 0x7f9fec0a5040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0081049a0 0x7fa00819ab60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 --2- 172.21.15.46:0/50842207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa008103ff0 0x7fa00819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 >> 172.21.15.46:0/50842207 conn(0x7fa0080ffa90 msgr2=0x7fa008100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.847+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 shutdown_connections 2024-09-10T07:36:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:54.848+0000 7fa00de00640 1 -- 172.21.15.46:0/50842207 wait complete. 2024-09-10T07:36:55.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:55 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/50842207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:55.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T07:36:55.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:55 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/50842207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:55.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:55 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/50842207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:36:56.202 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:56 smithi046 ceph-mon[30705]: pgmap v408: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T07:36:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:56 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:36:56.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:56 smithi181 ceph-mon[35397]: pgmap v408: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T07:36:56.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:56 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:36:56.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:56 smithi063 ceph-mon[35020]: pgmap v408: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T07:36:56.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:56 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:36:57.579 INFO:teuthology.orchestra.run.smithi046.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T07:36:57.580 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T07:36:57.580 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9893 2024-09-10T07:36:57.580 DEBUG:tasks.quiescer:Yielding 2024-09-10T07:36:57.580 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T07:36:57.599 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T07:36:57.599 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:36:57.599 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-10T07:36:57.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:36:57.678 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:36:57.707 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-10T07:36:57.707 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6789,172.21.15.63:6789,172.21.15.181:6789:/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e 1.1T 356M 1.1T 1% /tmp/cephfs 2024-09-10T07:36:57.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-10T07:36:57.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-10T07:36:57.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-10T07:36:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:36:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.617+0000 7f50b0c00640 1 -- 172.21.15.46:0/4217487954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac100720 msgr2=0x7f50ac100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.617+0000 7f50b0c00640 1 --2- 172.21.15.46:0/4217487954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac100720 0x7f50ac100b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f509c009900 tx=0x7f509c02a400 comp rx=0 tx=0).stop 2024-09-10T07:36:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.617+0000 7f50b0c00640 1 -- 172.21.15.46:0/4217487954 shutdown_connections 2024-09-10T07:36:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.617+0000 7f50b0c00640 1 --2- 172.21.15.46:0/4217487954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50ac10e2d0 0x7f50ac1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.617+0000 7f50b0c00640 1 --2- 172.21.15.46:0/4217487954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50ac1010d0 0x7f50ac10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.617+0000 7f50b0c00640 1 --2- 172.21.15.46:0/4217487954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac100720 0x7f50ac100b00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.617+0000 7f50b0c00640 1 -- 172.21.15.46:0/4217487954 >> 172.21.15.46:0/4217487954 conn(0x7f50ac0fc610 msgr2=0x7f50ac0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.617+0000 7f50b0c00640 1 -- 172.21.15.46:0/4217487954 shutdown_connections 2024-09-10T07:36:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.618+0000 7f50b0c00640 1 -- 172.21.15.46:0/4217487954 wait complete. 2024-09-10T07:36:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50b0c00640 1 Processor -- start 2024-09-10T07:36:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50b0c00640 1 -- start start 2024-09-10T07:36:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50b0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50ac100720 0x7f50ac19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50b0c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50ac1010d0 0x7f50ac19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50b0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac10e2d0 0x7f50ac19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50b0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac113ff0 con 0x7f50ac10e2d0 2024-09-10T07:36:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50b0c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac113e70 con 0x7f50ac1010d0 2024-09-10T07:36:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50b0c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac114170 con 0x7f50ac100720 2024-09-10T07:36:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac10e2d0 0x7f50ac19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.619+0000 7f50abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac10e2d0 0x7f50ac19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56276/0 (socket says 172.21.15.46:56276) 2024-09-10T07:36:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50abe00640 1 -- 172.21.15.46:0/528616671 learned_addr learned my addr 172.21.15.46:0/528616671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:36:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50aaa00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50ac1010d0 0x7f50ac19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50ab400640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50ac100720 0x7f50ac19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50abe00640 1 -- 172.21.15.46:0/528616671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50ac100720 msgr2=0x7f50ac19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50abe00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50ac100720 0x7f50ac19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50abe00640 1 -- 172.21.15.46:0/528616671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50ac1010d0 msgr2=0x7f50ac19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50abe00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50ac1010d0 0x7f50ac19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50abe00640 1 -- 172.21.15.46:0/528616671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac19f450 con 0x7f50ac10e2d0 2024-09-10T07:36:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50aaa00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50ac1010d0 0x7f50ac19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:36:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.620+0000 7f50abe00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac10e2d0 0x7f50ac19ed70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f50a400eb10 tx=0x7f50a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.621+0000 7f50a3e00640 1 -- 172.21.15.46:0/528616671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4004b20 con 0x7f50ac10e2d0 2024-09-10T07:36:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.621+0000 7f50a3e00640 1 -- 172.21.15.46:0/528616671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50a4004cc0 con 0x7f50ac10e2d0 2024-09-10T07:36:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.621+0000 7f50a3e00640 1 -- 172.21.15.46:0/528616671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4010790 con 0x7f50ac10e2d0 2024-09-10T07:36:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.621+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac19f740 con 0x7f50ac10e2d0 2024-09-10T07:36:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.621+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac1a7020 con 0x7f50ac10e2d0 2024-09-10T07:36:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.622+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ac06a850 con 0x7f50ac10e2d0 2024-09-10T07:36:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.624+0000 7f50a3e00640 1 -- 172.21.15.46:0/528616671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f50a4005000 con 0x7f50ac10e2d0 2024-09-10T07:36:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.626+0000 7f50a3e00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50900a2c00 0x7f50900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:36:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.626+0000 7f50a3e00640 1 -- 172.21.15.46:0/528616671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f50a40cf9e0 con 0x7f50ac10e2d0 2024-09-10T07:36:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.626+0000 7f50ab400640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50900a2c00 0x7f50900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:36:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.627+0000 7f50ab400640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50900a2c00 0x7f50900a50c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f509c009230 tx=0x7f509c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:36:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.629+0000 7f50a3e00640 1 -- 172.21.15.46:0/528616671 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50a409c5c0 con 0x7f50ac10e2d0 2024-09-10T07:36:58.702 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:58 smithi063 ceph-mon[35020]: pgmap v409: 129 pgs: 129 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T07:36:58.706 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:58 smithi046 ceph-mon[30705]: pgmap v409: 129 pgs: 129 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T07:36:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.855+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50ac100b00 con 0x7f50ac10e2d0 2024-09-10T07:36:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.856+0000 7f50a3e00640 1 -- 172.21.15.46:0/528616671 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f50ac100b00 con 0x7f50ac10e2d0 2024-09-10T07:36:58.856 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:36:58.856 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"btime":"2024-09-10T07:36:31:116553+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:36:30.952730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:36:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-09-10T07:36:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50900a2c00 msgr2=0x7f50900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50900a2c00 0x7f50900a50c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f509c009230 tx=0x7f509c0058d0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac10e2d0 msgr2=0x7f50ac19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac10e2d0 0x7f50ac19ed70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f50a400eb10 tx=0x7f50a400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 shutdown_connections 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f50900a2c00 0x7f50900a50c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50ac10e2d0 0x7f50ac19ed70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50ac1010d0 0x7f50ac19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 --2- 172.21.15.46:0/528616671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50ac100720 0x7f50ac19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 >> 172.21.15.46:0/528616671 conn(0x7f50ac0fc610 msgr2=0x7f50ac0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.859+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 shutdown_connections 2024-09-10T07:36:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:36:58.860+0000 7f50b0c00640 1 -- 172.21.15.46:0/528616671 wait complete. 2024-09-10T07:36:58.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:58 smithi181 ceph-mon[35397]: pgmap v409: 129 pgs: 129 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T07:36:59.897 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:36:59 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/528616671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:59.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:36:59 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/528616671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:36:59.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:36:59 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/528616671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:00.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T07:33:36.040805+0000', 'modified': '2024-09-10T07:36:30.952730+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24535, 'mds_1': 24530, 'mds_2': 24517, 'mds_3': 24499, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6833/620318721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6832', 'nonce': 620318721}, {'type': 'v1', 'addr': '172.21.15.63:6833', 'nonce': 620318721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.181:6833/2451989162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 2451989162}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 2451989162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/4121221095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 4121221095}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 4121221095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6835/1645722263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6834', 'nonce': 1645722263}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 1645722263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/335913193', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 335913193}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 335913193}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.63:6837/4195549127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6836', 'nonce': 4195549127}, {'type': 'v1', 'addr': '172.21.15.63:6837', 'nonce': 4195549127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.181:6837/975679519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 975679519}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 975679519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.181:6839/1085095014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1085095014}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1085095014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.63:6839/1361023409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6838', 'nonce': 1361023409}, {'type': 'v1', 'addr': '172.21.15.63:6839', 'nonce': 1361023409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.46:6841/3146821117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6840', 'nonce': 3146821117}, {'type': 'v1', 'addr': '172.21.15.46:6841', 'nonce': 3146821117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24535, 'qdb_cluster': [24535, 24530, 24517, 24499, 24448]} 2024-09-10T07:37:00.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T07:37:00.504 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-10T07:37:00.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:00 smithi046 ceph-mon[30705]: pgmap v410: 129 pgs: 129 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-10T07:37:00.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-10T07:37:00.775 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-10T07:37:00.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-10T07:37:00.800 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:00 smithi181 ceph-mon[35397]: pgmap v410: 129 pgs: 129 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-10T07:37:00.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-10T07:37:00.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:00 smithi063 ceph-mon[35020]: pgmap v410: 129 pgs: 129 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-10T07:37:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:00.958 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/: 2024-09-10T07:37:00.958 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:00.958 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 . 2024-09-10T07:37:00.958 INFO:teuthology.orchestra.run.smithi046.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 .. 2024-09-10T07:37:00.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:36 data 2024-09-10T07:37:00.959 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:00.959 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data: 2024-09-10T07:37:00.959 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:00.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:36 . 2024-09-10T07:37:00.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 .. 2024-09-10T07:37:00.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-10T07:37:01.065 INFO:teuthology.orchestra.run.smithi046.stdout:[Service] 2024-09-10T07:37:01.065 INFO:teuthology.orchestra.run.smithi046.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-10T07:37:01.065 INFO:teuthology.orchestra.run.smithi046.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-10T07:37:01.070 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-10T07:37:01.233 INFO:teuthology.orchestra.run.smithi046.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-10T07:37:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.605+0000 7f1783400640 1 -- 172.21.15.46:0/58275259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c076720 msgr2=0x7f177c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.605+0000 7f1783400640 1 --2- 172.21.15.46:0/58275259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c076720 0x7f177c076b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1770009900 tx=0x7f177002a400 comp rx=0 tx=0).stop 2024-09-10T07:37:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.606+0000 7f1783400640 1 -- 172.21.15.46:0/58275259 shutdown_connections 2024-09-10T07:37:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.606+0000 7f1783400640 1 --2- 172.21.15.46:0/58275259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177c112a30 0x7f177c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.606+0000 7f1783400640 1 --2- 172.21.15.46:0/58275259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f177c077040 0x7f177c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.606+0000 7f1783400640 1 --2- 172.21.15.46:0/58275259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c076720 0x7f177c076b00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.606+0000 7f1783400640 1 -- 172.21.15.46:0/58275259 >> 172.21.15.46:0/58275259 conn(0x7f177c0ffa90 msgr2=0x7f177c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.606+0000 7f1783400640 1 -- 172.21.15.46:0/58275259 shutdown_connections 2024-09-10T07:37:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.606+0000 7f1783400640 1 -- 172.21.15.46:0/58275259 wait complete. 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.607+0000 7f1783400640 1 Processor -- start 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.607+0000 7f1783400640 1 -- start start 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.607+0000 7f1783400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177c076720 0x7f177c1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.607+0000 7f1783400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f177c077040 0x7f177c1a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.607+0000 7f1783400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c112a30 0x7f177c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.607+0000 7f1783400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1185a0 con 0x7f177c112a30 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.607+0000 7f1783400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f177c118420 con 0x7f177c076720 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.607+0000 7f1783400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f177c118720 con 0x7f177c077040 2024-09-10T07:37:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1781400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c112a30 0x7f177c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1780a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177c076720 0x7f177c1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1781400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c112a30 0x7f177c19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56298/0 (socket says 172.21.15.46:56298) 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1781400640 1 -- 172.21.15.46:0/1554796675 learned_addr learned my addr 172.21.15.46:0/1554796675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f177be00640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f177c077040 0x7f177c1a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1781400640 1 -- 172.21.15.46:0/1554796675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f177c077040 msgr2=0x7f177c1a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1781400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f177c077040 0x7f177c1a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1781400640 1 -- 172.21.15.46:0/1554796675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177c076720 msgr2=0x7f177c1a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1781400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177c076720 0x7f177c1a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.608+0000 7f1781400640 1 -- 172.21.15.46:0/1554796675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c19f040 con 0x7f177c112a30 2024-09-10T07:37:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.609+0000 7f1781400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c112a30 0x7f177c19e910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f177400eb10 tx=0x7f177400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.609+0000 7f1779600640 1 -- 172.21.15.46:0/1554796675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774004b20 con 0x7f177c112a30 2024-09-10T07:37:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.609+0000 7f1779600640 1 -- 172.21.15.46:0/1554796675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1774004cc0 con 0x7f177c112a30 2024-09-10T07:37:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.609+0000 7f1783400640 1 -- 172.21.15.46:0/1554796675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c19f330 con 0x7f177c112a30 2024-09-10T07:37:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.609+0000 7f1779600640 1 -- 172.21.15.46:0/1554796675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774010790 con 0x7f177c112a30 2024-09-10T07:37:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.609+0000 7f1783400640 1 -- 172.21.15.46:0/1554796675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1ab620 con 0x7f177c112a30 2024-09-10T07:37:01.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.613+0000 7f178605d740 1 Processor -- start 2024-09-10T07:37:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f1779600640 1 -- 172.21.15.46:0/1554796675 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1774005020 con 0x7f177c112a30 2024-09-10T07:37:01.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f178605d740 1 -- start start 2024-09-10T07:37:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f178605d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f178605d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4bafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f178605d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4bd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f178605d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4160b0 con 0x555f7b4bb510 2024-09-10T07:37:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f178605d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555f7b3bdab0 con 0x555f7b4b0c50 2024-09-10T07:37:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f178605d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f7b412070 con 0x555f7b4b0330 2024-09-10T07:37:01.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.614+0000 7f177be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f177be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53380/0 (socket says 172.21.15.46:53380) 2024-09-10T07:37:01.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f177be00640 1 -- 172.21.15.46:0/1129094104 learned_addr learned my addr 172.21.15.46:0/1129094104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:01.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f1780a00640 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4bafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f177be00640 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4bd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f1780a00640 1 -- 172.21.15.46:0/1129094104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 msgr2=0x555f7b4b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f1780a00640 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f1780a00640 1 -- 172.21.15.46:0/1129094104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 msgr2=0x555f7b4bd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f1780a00640 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4bd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f1780a00640 1 -- 172.21.15.46:0/1129094104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7b3ca4a0 con 0x555f7b4b0c50 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f1779600640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17640a2b80 0x7f17640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.615+0000 7f1779600640 1 -- 172.21.15.46:0/1554796675 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f17740cfb20 con 0x7f177c112a30 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.616+0000 7f1780a00640 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4bafd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1770005940 tx=0x7f1770005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feed7e30b83d64cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.616+0000 7f177be00640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17640a2b80 0x7f17640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.616+0000 7f176dc00640 1 -- 172.21.15.46:0/1129094104 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770040070 con 0x555f7b4b0c50 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.616+0000 7f176dc00640 1 -- 172.21.15.46:0/1129094104 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17700318f0 con 0x555f7b4b0c50 2024-09-10T07:37:01.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.616+0000 7f176dc00640 1 -- 172.21.15.46:0/1129094104 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770035040 con 0x555f7b4b0c50 2024-09-10T07:37:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 -- 172.21.15.46:0/1129094104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 msgr2=0x555f7b4bafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f177be00640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17640a2b80 0x7f17640a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f176800bfd0 tx=0x7f1768004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4bafd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1770005940 tx=0x7f1770005e00 comp rx=0 tx=0).stop 2024-09-10T07:37:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 -- 172.21.15.46:0/1129094104 shutdown_connections 2024-09-10T07:37:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4bd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4bafd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 --2- 172.21.15.46:0/1129094104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 -- 172.21.15.46:0/1129094104 >> 172.21.15.46:0/1129094104 conn(0x555f7b4ac8c0 msgr2=0x555f7b4acca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 -- 172.21.15.46:0/1129094104 shutdown_connections 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 -- 172.21.15.46:0/1129094104 wait complete. 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.617+0000 7f178605d740 1 -- start start 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.618+0000 7f178605d740 1 Processor -- start 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.618+0000 7f178605d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.618+0000 7f178605d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0c50 0x555f7b4b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.618+0000 7f178605d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.618+0000 7f178605d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4be060 con 0x555f7b4bb510 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.618+0000 7f178605d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555f7b39e920 con 0x555f7b4b0330 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.618+0000 7f178605d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4be1e0 con 0x555f7b4b0c50 2024-09-10T07:37:01.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.618+0000 7f178605d740 10 client.0 _ll_register_callbacks cb 0x555f7b3c28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1781400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f177be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0c50 0x555f7b4b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60344/0 (socket says 172.21.15.46:60344) 2024-09-10T07:37:01.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 -- 172.21.15.46:0/4240456415 learned_addr learned my addr 172.21.15.46:0/4240456415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 -- 172.21.15.46:0/4240456415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0c50 msgr2=0x555f7b4b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0c50 0x555f7b4b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 -- 172.21.15.46:0/4240456415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 msgr2=0x555f7b4b8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4b8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 -- 172.21.15.46:0/4240456415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7b1872b0 con 0x555f7b4b0330 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1780a00640 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b84e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1770006010 tx=0x7f1770004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1d9e089470612c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.619+0000 7f1762000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.620+0000 7f1762000640 1 -- 172.21.15.46:0/4240456415 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177004c020 con 0x555f7b4b0330 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.620+0000 7f178605d740 1 -- 172.21.15.46:0/4240456415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17700318f0 con 0x555f7b4b0330 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.620+0000 7f1762000640 1 -- 172.21.15.46:0/4240456415 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1770044650 con 0x555f7b4b0330 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.620+0000 7f1762000640 1 -- 172.21.15.46:0/4240456415 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1770057050 con 0x555f7b4b0330 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.620+0000 7f178605d740 10 client.25382 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.620+0000 7f178605d740 1 -- 172.21.15.46:0/4240456415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f7b4c9da0 con 0x555f7b4b0330 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.620+0000 7f1762000640 1 -- 172.21.15.46:0/4240456415 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17700318f0 con 0x555f7b4b0330 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.620+0000 7f1762000640 1 -- 172.21.15.46:0/4240456415 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f177003c9d0 con 0x555f7b4b0330 2024-09-10T07:37:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.621+0000 7f178605d740 10 client.25382 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.621+0000 7f178605d740 10 client.25382 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T07:37:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.621+0000 7f178605d740 20 client.25382 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.621+0000 7f178605d740 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x555f7b4cbfe0 0x555f7b4cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.621+0000 7f178605d740 4 client.25382 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.621+0000 7f178605d740 1 -- 172.21.15.46:0/4240456415 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f7b423620 con 0x555f7b4cbfe0 2024-09-10T07:37:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.621+0000 7f177be00640 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x555f7b4cbfe0 0x555f7b4cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.622+0000 7f177be00640 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x555f7b4cbfe0 0x555f7b4cd450 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=558ed5d4d2b72cee server_cookie=6255a1debe9752ed in_seq=0 out_seq=0 2024-09-10T07:37:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.622+0000 7f1762000640 10 client.25382 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:37:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f1762000640 1 -- 172.21.15.46:0/4240456415 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f7b423620 con 0x555f7b4cbfe0 2024-09-10T07:37:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f1762000640 10 client.25382 handle_command_reply: tid=0 2024-09-10T07:37:01.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 client.25382 shutdown 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 -- 172.21.15.46:0/4240456415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 msgr2=0x555f7b4b84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b84e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1770006010 tx=0x7f1770004980 comp rx=0 tx=0).stop 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 -- 172.21.15.46:0/4240456415 shutdown_connections 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x555f7b4cbfe0 0x555f7b4cd450 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0c50 0x555f7b4b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f1762000640 0 client.25382 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 --2- 172.21.15.46:0/4240456415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b84e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 -- 172.21.15.46:0/4240456415 >> 172.21.15.46:0/4240456415 conn(0x555f7b4ac8c0 msgr2=0x555f7b4b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 -- 172.21.15.46:0/4240456415 shutdown_connections 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 1 -- 172.21.15.46:0/4240456415 wait complete. 2024-09-10T07:37:01.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.623+0000 7f178605d740 20 client.25382 trim_cache size 0 max 16384 2024-09-10T07:37:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.642+0000 7f178605d740 1 Processor -- start 2024-09-10T07:37:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.642+0000 7f178605d740 1 -- start start 2024-09-10T07:37:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f178605d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f178605d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4b0c50 0x555f7b4b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f178605d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4bb510 0x555f7b4c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f178605d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4cb790 con 0x555f7b4b0c50 2024-09-10T07:37:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f178605d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4cbe60 con 0x555f7b4b0330 2024-09-10T07:37:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f178605d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4cc530 con 0x555f7b4bb510 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f1780a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f1780a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60358/0 (socket says 172.21.15.46:60358) 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f1780a00640 1 -- 172.21.15.46:0/1380416811 learned_addr learned my addr 172.21.15.46:0/1380416811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f177be00640 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4b0c50 0x555f7b4b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.643+0000 7f1781400640 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4bb510 0x555f7b4c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f1780a00640 1 -- 172.21.15.46:0/1380416811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4bb510 msgr2=0x555f7b4c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f1780a00640 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4bb510 0x555f7b4c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f1780a00640 1 -- 172.21.15.46:0/1380416811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4b0c50 msgr2=0x555f7b4b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f1780a00640 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4b0c50 0x555f7b4b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f1780a00640 1 -- 172.21.15.46:0/1380416811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7b4b7490 con 0x555f7b4b0330 2024-09-10T07:37:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f1780a00640 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b8db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f177002a920 tx=0x7f177003b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f91682dc88405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f176d200640 1 -- 172.21.15.46:0/1380416811 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177004c070 con 0x555f7b4b0330 2024-09-10T07:37:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f176d200640 1 -- 172.21.15.46:0/1380416811 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17700318f0 con 0x555f7b4b0330 2024-09-10T07:37:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.644+0000 7f176d200640 1 -- 172.21.15.46:0/1380416811 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770040070 con 0x555f7b4b0330 2024-09-10T07:37:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 -- 172.21.15.46:0/1380416811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 msgr2=0x555f7b4b8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b8db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f177002a920 tx=0x7f177003b660 comp rx=0 tx=0).stop 2024-09-10T07:37:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 -- 172.21.15.46:0/1380416811 shutdown_connections 2024-09-10T07:37:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4bb510 0x555f7b4c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4b0c50 0x555f7b4b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 --2- 172.21.15.46:0/1380416811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0330 0x555f7b4b8db0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 -- 172.21.15.46:0/1380416811 >> 172.21.15.46:0/1380416811 conn(0x555f7b4ac8c0 msgr2=0x555f7b4c6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 -- 172.21.15.46:0/1380416811 shutdown_connections 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.645+0000 7f178605d740 1 -- 172.21.15.46:0/1380416811 wait complete. 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 1 -- start start 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 1 Processor -- start 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4cd2d0 con 0x555f7b4bb510 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4ccc00 con 0x555f7b4b0c50 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f7b4cd9a0 con 0x555f7b4b0330 2024-09-10T07:37:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.646+0000 7f178605d740 10 client.0 _ll_register_callbacks cb 0x7f1770005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f1781400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f1780a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f177be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f1781400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56336/0 (socket says 172.21.15.46:56336) 2024-09-10T07:37:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f1781400640 1 -- 172.21.15.46:0/891302795 learned_addr learned my addr 172.21.15.46:0/891302795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:01.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f177be00640 1 -- 172.21.15.46:0/891302795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 msgr2=0x555f7b4cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:01.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f177be00640 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f177be00640 1 -- 172.21.15.46:0/891302795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 msgr2=0x555f7b4d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:01.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f177be00640 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f177be00640 1 -- 172.21.15.46:0/891302795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7b4d4180 con 0x555f7b4b0c50 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.647+0000 7f1780a00640 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.648+0000 7f177be00640 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4cfc30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x555f7b416140 tx=0x7f17680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1375f9ad7ede4a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.648+0000 7f1762000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.648+0000 7f1762000640 1 -- 172.21.15.46:0/891302795 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768039060 con 0x555f7b4b0c50 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.648+0000 7f178605d740 1 -- 172.21.15.46:0/891302795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17700318f0 con 0x555f7b4b0c50 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.648+0000 7f1762000640 1 -- 172.21.15.46:0/891302795 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f176801e610 con 0x555f7b4b0c50 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.648+0000 7f1762000640 1 -- 172.21.15.46:0/891302795 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1768040050 con 0x555f7b4b0c50 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.648+0000 7f178605d740 10 client.25388 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.648+0000 7f178605d740 1 -- 172.21.15.46:0/891302795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f7b536ca0 con 0x555f7b4b0c50 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f1762000640 1 -- 172.21.15.46:0/891302795 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17700318f0 con 0x555f7b4b0c50 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f1762000640 1 -- 172.21.15.46:0/891302795 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f176801d070 con 0x555f7b4b0c50 2024-09-10T07:37:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f178605d740 10 client.25388 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f178605d740 10 client.25388 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f178605d740 20 client.25388 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f178605d740 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x555f7b53b850 0x555f7b53a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f178605d740 4 client.25388 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f178605d740 1 -- 172.21.15.46:0/891302795 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "status"}) -- 0x555f7b41db30 con 0x555f7b53b850 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.649+0000 7f1780a00640 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x555f7b53b850 0x555f7b53a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.650+0000 7f1780a00640 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x555f7b53b850 0x555f7b53a560 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa6d9d3dfb2a00ef server_cookie=c57db852ebad1c0f in_seq=0 out_seq=0 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.650+0000 7f1762000640 10 client.25388 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.650+0000 7f1762000640 1 -- 172.21.15.46:0/891302795 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555f7b41db30 con 0x555f7b53b850 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.650+0000 7f1762000640 10 client.25388 handle_command_reply: tid=0 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.650+0000 7f178605d740 1 client.25388 shutdown 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.650+0000 7f178605d740 1 -- 172.21.15.46:0/891302795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 msgr2=0x555f7b4cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.650+0000 7f178605d740 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4cfc30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x555f7b416140 tx=0x7f17680046c0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 1 -- 172.21.15.46:0/891302795 shutdown_connections 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x555f7b53b850 0x555f7b53a560 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f7b4bb510 0x555f7b4d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f7b4b0c50 0x555f7b4cfc30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 1 --2- 172.21.15.46:0/891302795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7b4b0330 0x555f7b4cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f1762000640 0 client.25388 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 1 -- 172.21.15.46:0/891302795 >> 172.21.15.46:0/891302795 conn(0x555f7b4ac8c0 msgr2=0x555f7b4aba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 1 -- 172.21.15.46:0/891302795 shutdown_connections 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 1 -- 172.21.15.46:0/891302795 wait complete. 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.651+0000 7f178605d740 20 client.25388 trim_cache size 0 max 16384 2024-09-10T07:37:01.653 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 4, 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24448, 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 155.84798814600001, 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 30, 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 97, 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 253.525779277 2024-09-10T07:37:01.654 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 -- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17640a2b80 msgr2=0x7f17640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17640a2b80 0x7f17640a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f176800bfd0 tx=0x7f1768004730 comp rx=0 tx=0).stop 2024-09-10T07:37:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 -- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c112a30 msgr2=0x7f177c19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c112a30 0x7f177c19e910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f177400eb10 tx=0x7f177400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 -- 172.21.15.46:0/1554796675 shutdown_connections 2024-09-10T07:37:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17640a2b80 0x7f17640a5040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177c112a30 0x7f177c19e910 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f177c077040 0x7f177c1a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 --2- 172.21.15.46:0/1554796675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177c076720 0x7f177c1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 -- 172.21.15.46:0/1554796675 >> 172.21.15.46:0/1554796675 conn(0x7f177c0ffa90 msgr2=0x7f177c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 -- 172.21.15.46:0/1554796675 shutdown_connections 2024-09-10T07:37:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:01.652+0000 7f1783400640 1 -- 172.21.15.46:0/1554796675 wait complete. 2024-09-10T07:37:02.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T07:37:02.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:02 smithi181 ceph-mon[35397]: pgmap v411: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-10T07:37:02.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:02 smithi063 ceph-mon[35020]: pgmap v411: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-10T07:37:02.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:02 smithi046 ceph-mon[30705]: pgmap v411: 129 pgs: 129 active+clean; 661 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-10T07:37:02.975 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.670+0000 7f2b3b600640 1 -- 172.21.15.46:0/2637045213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34102790 msgr2=0x7f2b341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.670+0000 7f2b3b600640 1 --2- 172.21.15.46:0/2637045213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34102790 0x7f2b341067e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009860 tx=0x7f2b2002a340 comp rx=0 tx=0).stop 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.671+0000 7f2b3b600640 1 -- 172.21.15.46:0/2637045213 shutdown_connections 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.671+0000 7f2b3b600640 1 --2- 172.21.15.46:0/2637045213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34106f10 0x7f2b3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.671+0000 7f2b3b600640 1 --2- 172.21.15.46:0/2637045213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34102790 0x7f2b341067e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.671+0000 7f2b3b600640 1 --2- 172.21.15.46:0/2637045213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b34101de0 0x7f2b341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.671+0000 7f2b3b600640 1 -- 172.21.15.46:0/2637045213 >> 172.21.15.46:0/2637045213 conn(0x7f2b340fd8c0 msgr2=0x7f2b340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.671+0000 7f2b3b600640 1 -- 172.21.15.46:0/2637045213 shutdown_connections 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.671+0000 7f2b3b600640 1 -- 172.21.15.46:0/2637045213 wait complete. 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b3b600640 1 Processor -- start 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b3b600640 1 -- start start 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b3b600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34101de0 0x7f2b3419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b3b600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34102790 0x7f2b3419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b3b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b34106f10 0x7f2b3419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b3b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34111f70 con 0x7f2b34106f10 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b3b600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34111df0 con 0x7f2b34101de0 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b3b600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341120f0 con 0x7f2b34102790 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b38c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34101de0 0x7f2b3419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b38c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34101de0 0x7f2b3419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60386/0 (socket says 172.21.15.46:60386) 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b39600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b34106f10 0x7f2b3419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.672+0000 7f2b38c00640 1 -- 172.21.15.46:0/549072498 learned_addr learned my addr 172.21.15.46:0/549072498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b33e00640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34102790 0x7f2b3419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b38c00640 1 -- 172.21.15.46:0/549072498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34102790 msgr2=0x7f2b3419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b38c00640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34102790 0x7f2b3419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b38c00640 1 -- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b34106f10 msgr2=0x7f2b3419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b38c00640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b34106f10 0x7f2b3419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b38c00640 1 -- 172.21.15.46:0/549072498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3419f5e0 con 0x7f2b34101de0 2024-09-10T07:37:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b33e00640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34102790 0x7f2b3419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b38c00640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34101de0 0x7f2b3419a630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800e9c0 tx=0x7f2b2800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.673+0000 7f2b31600640 1 -- 172.21.15.46:0/549072498 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800cde0 con 0x7f2b34101de0 2024-09-10T07:37:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.674+0000 7f2b3b600640 1 -- 172.21.15.46:0/549072498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3419f8d0 con 0x7f2b34101de0 2024-09-10T07:37:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.674+0000 7f2b31600640 1 -- 172.21.15.46:0/549072498 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b280035f0 con 0x7f2b34101de0 2024-09-10T07:37:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.674+0000 7f2b3b600640 1 -- 172.21.15.46:0/549072498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b341a71b0 con 0x7f2b34101de0 2024-09-10T07:37:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.674+0000 7f2b31600640 1 -- 172.21.15.46:0/549072498 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28010690 con 0x7f2b34101de0 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b3e1e3740 1 Processor -- start 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b31600640 1 -- 172.21.15.46:0/549072498 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b280037a0 con 0x7f2b34101de0 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b3e1e3740 1 -- start start 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d7992710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992c50 0x5556d799cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d799d510 0x5556d799f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556d78f80b0 con 0x5556d7992c50 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5556d789fab0 con 0x5556d799d510 2024-09-10T07:37:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.677+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5556d78f4070 con 0x5556d7992330 2024-09-10T07:37:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b38c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992c50 0x5556d799cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d7992710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d7992710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53412/0 (socket says 172.21.15.46:53412) 2024-09-10T07:37:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 -- 172.21.15.46:0/3543285261 learned_addr learned my addr 172.21.15.46:0/3543285261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 -- 172.21.15.46:0/3543285261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d799d510 msgr2=0x5556d799f900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:37:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 --2- 172.21.15.46:0/3543285261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d799d510 0x5556d799f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 -- 172.21.15.46:0/3543285261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992c50 msgr2=0x5556d799cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 --2- 172.21.15.46:0/3543285261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992c50 0x5556d799cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 -- 172.21.15.46:0/3543285261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556d78ac4a0 con 0x5556d7992330 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.678+0000 7f2b39600640 1 --2- 172.21.15.46:0/3543285261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d7992710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009f70 tx=0x7f2b2c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba24a2e88df9af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.679+0000 7f2b31600640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2b1c0a2b80 0x7f2b1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.679+0000 7f2b25c00640 1 -- 172.21.15.46:0/3543285261 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c016070 con 0x5556d7992330 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.679+0000 7f2b25c00640 1 -- 172.21.15.46:0/3543285261 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b2c004d20 con 0x5556d7992330 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.679+0000 7f2b25c00640 1 -- 172.21.15.46:0/3543285261 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c012950 con 0x5556d7992330 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.679+0000 7f2b31600640 1 -- 172.21.15.46:0/549072498 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2b280cf840 con 0x7f2b34101de0 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.679+0000 7f2b38c00640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2b1c0a2b80 0x7f2b1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.679+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/3543285261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 msgr2=0x5556d7992710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.679+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/3543285261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d7992710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009f70 tx=0x7f2b2c010800 comp rx=0 tx=0).stop 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/3543285261 shutdown_connections 2024-09-10T07:37:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/3543285261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d799d510 0x5556d799f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/3543285261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992c50 0x5556d799cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/3543285261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d7992710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/3543285261 >> 172.21.15.46:0/3543285261 conn(0x5556d798e8c0 msgr2=0x5556d798eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b38c00640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2b1c0a2b80 0x7f2b1c0a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2b28098690 tx=0x7f2b280d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/3543285261 shutdown_connections 2024-09-10T07:37:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/3543285261 wait complete. 2024-09-10T07:37:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 -- start start 2024-09-10T07:37:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.680+0000 7f2b3e1e3740 1 Processor -- start 2024-09-10T07:37:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992330 0x5556d799a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992c50 0x5556d799ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 0x5556d799b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556d79a0060 con 0x5556d799d510 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5556d7880920 con 0x5556d7992330 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5556d79a01e0 con 0x5556d7992c50 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b3e1e3740 10 client.0 _ll_register_callbacks cb 0x5556d78a48a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b39600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 0x5556d799b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992c50 0x5556d799ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b39600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992330 0x5556d799a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992c50 0x5556d799ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53418/0 (socket says 172.21.15.46:53418) 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.681+0000 7f2b33e00640 1 -- 172.21.15.46:0/524836657 learned_addr learned my addr 172.21.15.46:0/524836657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b33e00640 1 -- 172.21.15.46:0/524836657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992330 msgr2=0x5556d799a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b33e00640 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992330 0x5556d799a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b33e00640 1 -- 172.21.15.46:0/524836657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 msgr2=0x5556d799b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b33e00640 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 0x5556d799b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b33e00640 1 -- 172.21.15.46:0/524836657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556d76692b0 con 0x5556d7992c50 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b33e00640 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992c50 0x5556d799ab20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009250 tx=0x7f2b20004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f838510b91c42092 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b1a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b1a000640 1 -- 172.21.15.46:0/524836657 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20037040 con 0x5556d7992c50 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/524836657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b2c004d20 con 0x5556d7992c50 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.682+0000 7f2b1a000640 1 -- 172.21.15.46:0/524836657 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2b20030550 con 0x5556d7992c50 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.683+0000 7f2b1a000640 1 -- 172.21.15.46:0/524836657 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2b20049050 con 0x5556d7992c50 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.683+0000 7f2b3e1e3740 10 client.25411 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.683+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/524836657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556d79abed0 con 0x5556d7992c50 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.683+0000 7f2b1a000640 1 -- 172.21.15.46:0/524836657 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b2c004d20 con 0x5556d7992c50 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.683+0000 7f2b1a000640 1 -- 172.21.15.46:0/524836657 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2b2003a070 con 0x5556d7992c50 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.683+0000 7f2b3e1e3740 10 client.25411 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.684+0000 7f2b3e1e3740 10 client.25411 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:37:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.684+0000 7f2b3e1e3740 20 client.25411 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.684+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5556d79ae4c0 0x5556d79af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.684+0000 7f2b3e1e3740 4 client.25411 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.684+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/524836657 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556d7905620 con 0x5556d79ae4c0 2024-09-10T07:37:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.684+0000 7f2b39600640 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5556d79ae4c0 0x5556d79af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.684+0000 7f2b39600640 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5556d79ae4c0 0x5556d79af930 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e0af520ea9c77a9 server_cookie=559b2e88a7acab5d in_seq=0 out_seq=0 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.684+0000 7f2b1a000640 10 client.25411 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.685+0000 7f2b1a000640 1 -- 172.21.15.46:0/524836657 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556d7905620 con 0x5556d79ae4c0 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.685+0000 7f2b1a000640 10 client.25411 handle_command_reply: tid=0 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 client.25411 shutdown 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/524836657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992c50 msgr2=0x5556d799ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992c50 0x5556d799ab20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009250 tx=0x7f2b20004270 comp rx=0 tx=0).stop 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/524836657 shutdown_connections 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 0x5556d799b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5556d79ae4c0 0x5556d79af930 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992c50 0x5556d799ab20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/524836657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992330 0x5556d799a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b1a000640 0 client.25411 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:37:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/524836657 >> 172.21.15.46:0/524836657 conn(0x5556d798e8c0 msgr2=0x5556d79945f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/524836657 shutdown_connections 2024-09-10T07:37:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/524836657 wait complete. 2024-09-10T07:37:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.686+0000 7f2b3e1e3740 20 client.25411 trim_cache size 0 max 16384 2024-09-10T07:37:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b3e1e3740 1 Processor -- start 2024-09-10T07:37:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b3e1e3740 1 -- start start 2024-09-10T07:37:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d79addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d799aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 0x5556d7997860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556d79adc70 con 0x5556d799d510 2024-09-10T07:37:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5556d79ae340 con 0x5556d7992c50 2024-09-10T07:37:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5556d79aea10 con 0x5556d7992330 2024-09-10T07:37:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d799aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b39600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 0x5556d7997860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d799aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60422/0 (socket says 172.21.15.46:60422) 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.705+0000 7f2b33e00640 1 -- 172.21.15.46:0/2168463841 learned_addr learned my addr 172.21.15.46:0/2168463841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b39600640 1 --2- 172.21.15.46:0/2168463841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d79addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b33e00640 1 -- 172.21.15.46:0/2168463841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 msgr2=0x5556d79addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b33e00640 1 --2- 172.21.15.46:0/2168463841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d79addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b33e00640 1 -- 172.21.15.46:0/2168463841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 msgr2=0x5556d7997860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b33e00640 1 --2- 172.21.15.46:0/2168463841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 0x5556d7997860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b33e00640 1 -- 172.21.15.46:0/2168463841 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556d798f970 con 0x5556d7992c50 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b33e00640 1 --2- 172.21.15.46:0/2168463841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d799aeb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009780 tx=0x7f2b2002a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c76be6ffdecf0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b25200640 1 -- 172.21.15.46:0/2168463841 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20037040 con 0x5556d7992c50 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b25200640 1 -- 172.21.15.46:0/2168463841 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b20040070 con 0x5556d7992c50 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.706+0000 7f2b25200640 1 -- 172.21.15.46:0/2168463841 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20005150 con 0x5556d7992c50 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2168463841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 msgr2=0x5556d799aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2168463841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d799aeb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009780 tx=0x7f2b2002a340 comp rx=0 tx=0).stop 2024-09-10T07:37:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2168463841 shutdown_connections 2024-09-10T07:37:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2168463841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d799d510 0x5556d7997860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2168463841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d799aeb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2168463841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d7992330 0x5556d79addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2168463841 >> 172.21.15.46:0/2168463841 conn(0x5556d798e8c0 msgr2=0x5556d7998eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2168463841 shutdown_connections 2024-09-10T07:37:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2168463841 wait complete. 2024-09-10T07:37:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.707+0000 7f2b3e1e3740 1 -- start start 2024-09-10T07:37:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b3e1e3740 1 Processor -- start 2024-09-10T07:37:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992330 0x5556d79b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d79b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b3e1e3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d799d510 0x5556d79b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556d79af7b0 con 0x5556d7992330 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5556d79af0e0 con 0x5556d7992c50 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b3e1e3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5556d79afe80 con 0x5556d799d510 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b3e1e3740 10 client.0 _ll_register_callbacks cb 0x7f2b200096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b39600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992330 0x5556d79b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d79b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b39600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992330 0x5556d79b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56384/0 (socket says 172.21.15.46:56384) 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.708+0000 7f2b39600640 1 -- 172.21.15.46:0/2056960882 learned_addr learned my addr 172.21.15.46:0/2056960882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b39600640 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d799d510 0x5556d79b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b33e00640 1 -- 172.21.15.46:0/2056960882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d799d510 msgr2=0x5556d79b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b33e00640 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d799d510 0x5556d79b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b33e00640 1 -- 172.21.15.46:0/2056960882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992330 msgr2=0x5556d79b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b33e00640 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992330 0x5556d79b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b33e00640 1 -- 172.21.15.46:0/2056960882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556d79b3080 con 0x5556d7992c50 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b33e00640 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d79b80d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5556d79b0110 tx=0x7f2b20004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4dcdbfa6c6bdeaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b1a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b1a000640 1 -- 172.21.15.46:0/2056960882 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2005a060 con 0x5556d7992c50 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2056960882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b20040070 con 0x5556d7992c50 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.709+0000 7f2b1a000640 1 -- 172.21.15.46:0/2056960882 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2b20004ea0 con 0x5556d7992c50 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.710+0000 7f2b1a000640 1 -- 172.21.15.46:0/2056960882 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2b20061050 con 0x5556d7992c50 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.710+0000 7f2b3e1e3740 10 client.25403 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.710+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2056960882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556d7a19410 con 0x5556d7992c50 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.710+0000 7f2b1a000640 1 -- 172.21.15.46:0/2056960882 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b20040070 con 0x5556d7992c50 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.710+0000 7f2b1a000640 1 -- 172.21.15.46:0/2056960882 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2b2003a070 con 0x5556d7992c50 2024-09-10T07:37:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.710+0000 7f2b3e1e3740 10 client.25403 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.711+0000 7f2b3e1e3740 10 client.25403 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.711+0000 7f2b3e1e3740 20 client.25403 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.711+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5556d7a1db70 0x5556d7a1c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.711+0000 7f2b3e1e3740 4 client.25403 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.711+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2056960882 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "status"}) -- 0x5556d798f470 con 0x5556d7a1db70 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.711+0000 7f2b39600640 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5556d7a1db70 0x5556d7a1c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.711+0000 7f2b39600640 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5556d7a1db70 0x5556d7a1c880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=78529aea8d288af3 server_cookie=89803bc05da71fbb in_seq=0 out_seq=0 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.711+0000 7f2b1a000640 10 client.25403 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b1a000640 1 -- 172.21.15.46:0/2056960882 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5556d798f470 con 0x5556d7a1db70 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b1a000640 10 client.25403 handle_command_reply: tid=0 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 client.25403 shutdown 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2056960882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 msgr2=0x5556d79b80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d79b80d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5556d79b0110 tx=0x7f2b20004480 comp rx=0 tx=0).stop 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2056960882 shutdown_connections 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5556d7a1db70 0x5556d7a1c880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556d799d510 0x5556d79b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5556d7992c50 0x5556d79b80d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 --2- 172.21.15.46:0/2056960882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d7992330 0x5556d79b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b1a000640 0 client.25403 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2056960882 >> 172.21.15.46:0/2056960882 conn(0x5556d798e8c0 msgr2=0x5556d798d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2056960882 shutdown_connections 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 1 -- 172.21.15.46:0/2056960882 wait complete. 2024-09-10T07:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.712+0000 7f2b3e1e3740 20 client.25403 trim_cache size 0 max 16384 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 3, 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24499, 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 158.63585306600001, 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 30, 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 97, 2024-09-10T07:37:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 236.52006093 2024-09-10T07:37:03.717 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:03.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 -- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2b1c0a2b80 msgr2=0x7f2b1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2b1c0a2b80 0x7f2b1c0a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2b28098690 tx=0x7f2b280d5530 comp rx=0 tx=0).stop 2024-09-10T07:37:03.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 -- 172.21.15.46:0/549072498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34101de0 msgr2=0x7f2b3419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34101de0 0x7f2b3419a630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800e9c0 tx=0x7f2b2800ee80 comp rx=0 tx=0).stop 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 -- 172.21.15.46:0/549072498 shutdown_connections 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2b1c0a2b80 0x7f2b1c0a5040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b34106f10 0x7f2b3419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34102790 0x7f2b3419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 --2- 172.21.15.46:0/549072498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34101de0 0x7f2b3419a630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 -- 172.21.15.46:0/549072498 >> 172.21.15.46:0/549072498 conn(0x7f2b340fd8c0 msgr2=0x7f2b340fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 -- 172.21.15.46:0/549072498 shutdown_connections 2024-09-10T07:37:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:03.713+0000 7f2b3b600640 1 -- 172.21.15.46:0/549072498 wait complete. 2024-09-10T07:37:04.820 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:04 smithi046 ceph-mon[30705]: pgmap v412: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-10T07:37:04.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T07:37:04.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:04 smithi181 ceph-mon[35397]: pgmap v412: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-10T07:37:04.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:04 smithi063 ceph-mon[35020]: pgmap v412: 129 pgs: 129 active+clean; 662 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-10T07:37:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:05.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 -- 172.21.15.46:0/3797715401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d41049c0 msgr2=0x7f30d4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:05.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3797715401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d41049c0 0x7f30d4108a10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009900 tx=0x7f30c002a410 comp rx=0 tx=0).stop 2024-09-10T07:37:05.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 -- 172.21.15.46:0/3797715401 shutdown_connections 2024-09-10T07:37:05.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3797715401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d4109140 0x7f30d4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3797715401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d41049c0 0x7f30d4108a10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3797715401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30d4104010 0x7f30d41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 -- 172.21.15.46:0/3797715401 >> 172.21.15.46:0/3797715401 conn(0x7f30d40ffa50 msgr2=0x7f30d4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:05.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 -- 172.21.15.46:0/3797715401 shutdown_connections 2024-09-10T07:37:05.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.960+0000 7f30d8c00640 1 -- 172.21.15.46:0/3797715401 wait complete. 2024-09-10T07:37:05.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.961+0000 7f30d8c00640 1 Processor -- start 2024-09-10T07:37:05.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.961+0000 7f30d8c00640 1 -- start start 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4104010 0x7f30d419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30d41049c0 0x7f30d419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d4109140 0x7f30d419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4114160 con 0x7f30d4104010 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d8c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4113fe0 con 0x7f30d4109140 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41142e0 con 0x7f30d41049c0 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30d41049c0 0x7f30d419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4104010 0x7f30d419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30d41049c0 0x7f30d419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59636/0 (socket says 172.21.15.46:59636) 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d1400640 1 -- 172.21.15.46:0/3601136536 learned_addr learned my addr 172.21.15.46:0/3601136536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:05.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.962+0000 7f30d2800640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d4109140 0x7f30d419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.963+0000 7f30d1400640 1 -- 172.21.15.46:0/3601136536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d4109140 msgr2=0x7f30d419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:05.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.963+0000 7f30d1400640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d4109140 0x7f30d419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.963+0000 7f30d1400640 1 -- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4104010 msgr2=0x7f30d419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:05.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.963+0000 7f30d1400640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4104010 0x7f30d419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.963+0000 7f30d1400640 1 -- 172.21.15.46:0/3601136536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d419f6e0 con 0x7f30d41049c0 2024-09-10T07:37:05.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.963+0000 7f30d1400640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30d41049c0 0x7f30d419ac70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006010 tx=0x7f30c0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:05.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.964+0000 7f30caa00640 1 -- 172.21.15.46:0/3601136536 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c003f030 con 0x7f30d41049c0 2024-09-10T07:37:05.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.964+0000 7f30d8c00640 1 -- 172.21.15.46:0/3601136536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d419f970 con 0x7f30d41049c0 2024-09-10T07:37:05.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.964+0000 7f30caa00640 1 -- 172.21.15.46:0/3601136536 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c0004440 con 0x7f30d41049c0 2024-09-10T07:37:05.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.964+0000 7f30caa00640 1 -- 172.21.15.46:0/3601136536 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0035070 con 0x7f30d41049c0 2024-09-10T07:37:05.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.964+0000 7f30d8c00640 1 -- 172.21.15.46:0/3601136536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a72b0 con 0x7f30d41049c0 2024-09-10T07:37:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30db69b740 1 Processor -- start 2024-09-10T07:37:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30db69b740 1 -- start start 2024-09-10T07:37:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30caa00640 1 -- 172.21.15.46:0/3601136536 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f30c0042030 con 0x7f30d41049c0 2024-09-10T07:37:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30db69b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97330 0x55e633a97710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30db69b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97c50 0x55e633aa1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30db69b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633aa4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30db69b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e6339fd0b0 con 0x55e633a97c50 2024-09-10T07:37:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30db69b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e6339a4ab0 con 0x55e633a97330 2024-09-10T07:37:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.967+0000 7f30db69b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e6339f9070 con 0x55e633aa2510 2024-09-10T07:37:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97330 0x55e633a97710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97330 0x55e633a97710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58976/0 (socket says 172.21.15.46:58976) 2024-09-10T07:37:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 -- 172.21.15.46:0/3712220781 learned_addr learned my addr 172.21.15.46:0/3712220781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d1e00640 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97c50 0x55e633aa1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d1400640 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633aa4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 -- 172.21.15.46:0/3712220781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 msgr2=0x55e633aa4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633aa4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 -- 172.21.15.46:0/3712220781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97c50 msgr2=0x55e633aa1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97c50 0x55e633aa1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 -- 172.21.15.46:0/3712220781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6339b14a0 con 0x55e633a97330 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d1e00640 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97c50 0x55e633aa1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d1400640 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633aa4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.968+0000 7f30d2800640 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97330 0x55e633a97710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009f50 tx=0x7f30cc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb2c89a4657cd93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.969+0000 7f30bf000640 1 -- 172.21.15.46:0/3712220781 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc004d20 con 0x55e633a97330 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.969+0000 7f30caa00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30b80a2c00 0x7f30b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.969+0000 7f30bf000640 1 -- 172.21.15.46:0/3712220781 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30cc004050 con 0x55e633a97330 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.969+0000 7f30bf000640 1 -- 172.21.15.46:0/3712220781 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc005280 con 0x55e633a97330 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.969+0000 7f30caa00640 1 -- 172.21.15.46:0/3601136536 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f30c00ef790 con 0x7f30d41049c0 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.969+0000 7f30d1e00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30b80a2c00 0x7f30b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 -- 172.21.15.46:0/3712220781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97330 msgr2=0x55e633a97710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97330 0x55e633a97710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009f50 tx=0x7f30cc0118d0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 -- 172.21.15.46:0/3712220781 shutdown_connections 2024-09-10T07:37:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633aa4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97c50 0x55e633aa1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 --2- 172.21.15.46:0/3712220781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97330 0x55e633a97710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 -- 172.21.15.46:0/3712220781 >> 172.21.15.46:0/3712220781 conn(0x55e633a938c0 msgr2=0x55e633a93ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30d1e00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30b80a2c00 0x7f30b80a50c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55e633a94720 tx=0x7f30c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 -- 172.21.15.46:0/3712220781 shutdown_connections 2024-09-10T07:37:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.970+0000 7f30db69b740 1 -- 172.21.15.46:0/3712220781 wait complete. 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 1 -- start start 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 1 Processor -- start 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e633aa5060 con 0x55e633a97330 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e633985920 con 0x55e633a97c50 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e633aa51e0 con 0x55e633aa2510 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.971+0000 7f30db69b740 10 client.0 _ll_register_callbacks cb 0x55e6339a98a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a9f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54570/0 (socket says 172.21.15.46:54570) 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 -- 172.21.15.46:0/4241160670 learned_addr learned my addr 172.21.15.46:0/4241160670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:05.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d2800640 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 -- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 msgr2=0x55e633a9ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 -- 172.21.15.46:0/4241160670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 msgr2=0x55e633a9fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 -- 172.21.15.46:0/4241160670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e63376e2b0 con 0x55e633a97330 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d2800640 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30d1e00640 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a9f4f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005bc0 tx=0x7f30c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe982fe7789e484 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.972+0000 7f30b3400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.973+0000 7f30b3400640 1 -- 172.21.15.46:0/4241160670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c402e060 con 0x55e633a97330 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.973+0000 7f30b3400640 1 -- 172.21.15.46:0/4241160670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f30c4018030 con 0x55e633a97330 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.973+0000 7f30db69b740 1 -- 172.21.15.46:0/4241160670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30cc004050 con 0x55e633a97330 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.973+0000 7f30b3400640 1 -- 172.21.15.46:0/4241160670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30c4035050 con 0x55e633a97330 2024-09-10T07:37:05.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.973+0000 7f30db69b740 10 client.15390 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:05.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.973+0000 7f30db69b740 1 -- 172.21.15.46:0/4241160670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e633ab0dd0 con 0x55e633a97330 2024-09-10T07:37:05.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.973+0000 7f30b3400640 1 -- 172.21.15.46:0/4241160670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30cc004050 con 0x55e633a97330 2024-09-10T07:37:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.974+0000 7f30b3400640 1 -- 172.21.15.46:0/4241160670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f30c401c3a0 con 0x55e633a97330 2024-09-10T07:37:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.974+0000 7f30db69b740 10 client.15390 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.974+0000 7f30db69b740 10 client.15390 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T07:37:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.974+0000 7f30db69b740 20 client.15390 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.974+0000 7f30db69b740 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55e633ab3410 0x55e633ab4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.974+0000 7f30db69b740 4 client.15390 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.974+0000 7f30db69b740 1 -- 172.21.15.46:0/4241160670 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e633a0a620 con 0x55e633ab3410 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.974+0000 7f30d1400640 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55e633ab3410 0x55e633ab4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.975+0000 7f30d1400640 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55e633ab3410 0x55e633ab4880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ae670555218a6df server_cookie=8530a535649c347b in_seq=0 out_seq=0 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.975+0000 7f30b3400640 10 client.15390 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30b3400640 1 -- 172.21.15.46:0/4241160670 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e633a0a620 con 0x55e633ab3410 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30b3400640 10 client.15390 handle_command_reply: tid=0 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 client.15390 shutdown 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 -- 172.21.15.46:0/4241160670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 msgr2=0x55e633a9f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a9f4f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005bc0 tx=0x7f30c4004730 comp rx=0 tx=0).stop 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 -- 172.21.15.46:0/4241160670 shutdown_connections 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55e633ab3410 0x55e633ab4880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 --2- 172.21.15.46:0/4241160670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a9f4f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30b3400640 0 client.15390 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:37:05.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.976+0000 7f30db69b740 1 -- 172.21.15.46:0/4241160670 >> 172.21.15.46:0/4241160670 conn(0x55e633a938c0 msgr2=0x55e633a995f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:05.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.977+0000 7f30db69b740 1 -- 172.21.15.46:0/4241160670 shutdown_connections 2024-09-10T07:37:05.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.977+0000 7f30db69b740 1 -- 172.21.15.46:0/4241160670 wait complete. 2024-09-10T07:37:05.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.977+0000 7f30db69b740 20 client.15390 trim_cache size 0 max 16384 2024-09-10T07:37:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30db69b740 1 Processor -- start 2024-09-10T07:37:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30db69b740 1 -- start start 2024-09-10T07:37:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30db69b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a93ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30db69b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30db69b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30db69b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e633ab2bc0 con 0x55e633a97330 2024-09-10T07:37:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30db69b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e633ab3290 con 0x55e633a97c50 2024-09-10T07:37:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30db69b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e633ab3960 con 0x55e633aa2510 2024-09-10T07:37:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a93ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30d1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a93ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54574/0 (socket says 172.21.15.46:54574) 2024-09-10T07:37:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30d1e00640 1 -- 172.21.15.46:0/578611613 learned_addr learned my addr 172.21.15.46:0/578611613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.996+0000 7f30d2800640 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.997+0000 7f30d1e00640 1 -- 172.21.15.46:0/578611613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 msgr2=0x55e633a9c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.997+0000 7f30d1e00640 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.997+0000 7f30d1e00640 1 -- 172.21.15.46:0/578611613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 msgr2=0x55e633a9fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.997+0000 7f30d1e00640 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.997+0000 7f30d1e00640 1 -- 172.21.15.46:0/578611613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e633a9da80 con 0x55e633a97330 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.997+0000 7f30d1400640 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.997+0000 7f30d1e00640 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a93ca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f30c40059f0 tx=0x7f30c401e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c05c9c04feb9a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30be600640 1 -- 172.21.15.46:0/578611613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c401c3a0 con 0x55e633a97330 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30be600640 1 -- 172.21.15.46:0/578611613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c401c540 con 0x55e633a97330 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30be600640 1 -- 172.21.15.46:0/578611613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4003720 con 0x55e633a97330 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 -- 172.21.15.46:0/578611613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 msgr2=0x55e633a93ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a93ca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f30c40059f0 tx=0x7f30c401e7c0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 -- 172.21.15.46:0/578611613 shutdown_connections 2024-09-10T07:37:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633aa2510 0x55e633a9c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633a97c50 0x55e633a9fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 --2- 172.21.15.46:0/578611613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633a93ca0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 -- 172.21.15.46:0/578611613 >> 172.21.15.46:0/578611613 conn(0x55e633a938c0 msgr2=0x55e633a9de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 -- 172.21.15.46:0/578611613 shutdown_connections 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.998+0000 7f30db69b740 1 -- 172.21.15.46:0/578611613 wait complete. 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.999+0000 7f30db69b740 1 -- start start 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.999+0000 7f30db69b740 1 Processor -- start 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.999+0000 7f30db69b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633abcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.999+0000 7f30db69b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633a97c50 0x55e633abd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.999+0000 7f30db69b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633aa2510 0x55e633ab7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.999+0000 7f30db69b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e633ab4700 con 0x55e633a97330 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.999+0000 7f30db69b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e633ab4030 con 0x55e633aa2510 2024-09-10T07:37:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:05.999+0000 7f30db69b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e633ab4dd0 con 0x55e633a97c50 2024-09-10T07:37:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30db69b740 10 client.0 _ll_register_callbacks cb 0x7f30c401e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:06.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633abcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:06.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633aa2510 0x55e633ab7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633aa2510 0x55e633ab7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59000/0 (socket says 172.21.15.46:59000) 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d2800640 1 -- 172.21.15.46:0/1574150494 learned_addr learned my addr 172.21.15.46:0/1574150494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d1400640 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633a97c50 0x55e633abd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d2800640 1 -- 172.21.15.46:0/1574150494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633a97c50 msgr2=0x55e633abd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d2800640 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633a97c50 0x55e633abd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d2800640 1 -- 172.21.15.46:0/1574150494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 msgr2=0x55e633abcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d2800640 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633abcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.000+0000 7f30d2800640 1 -- 172.21.15.46:0/1574150494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e633abd7d0 con 0x55e633aa2510 2024-09-10T07:37:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30d2800640 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633aa2510 0x55e633ab7a90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30cc01b040 tx=0x7f30cc005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=336d080e28a43618 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30b3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30b3400640 1 -- 172.21.15.46:0/1574150494 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc005770 con 0x55e633aa2510 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30db69b740 1 -- 172.21.15.46:0/1574150494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c401c540 con 0x55e633aa2510 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30b3400640 1 -- 172.21.15.46:0/1574150494 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f30cc005910 con 0x55e633aa2510 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30b3400640 1 -- 172.21.15.46:0/1574150494 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30cc036050 con 0x55e633aa2510 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30db69b740 10 client.25421 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30db69b740 1 -- 172.21.15.46:0/1574150494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e633b1e160 con 0x55e633aa2510 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.001+0000 7f30b3400640 1 -- 172.21.15.46:0/1574150494 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c401c540 con 0x55e633aa2510 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.002+0000 7f30b3400640 1 -- 172.21.15.46:0/1574150494 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f30cc019550 con 0x55e633aa2510 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.002+0000 7f30db69b740 10 client.25421 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.002+0000 7f30db69b740 10 client.25421 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.002+0000 7f30db69b740 20 client.25421 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.002+0000 7f30db69b740 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55e633b22bb0 0x55e633b218c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.002+0000 7f30db69b740 4 client.25421 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.002+0000 7f30db69b740 1 -- 172.21.15.46:0/1574150494 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "status"}) -- 0x55e633a04b30 con 0x55e633b22bb0 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.002+0000 7f30d1e00640 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55e633b22bb0 0x55e633b218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30d1e00640 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55e633b22bb0 0x55e633b218c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9376da0e71f2d67f server_cookie=85451e32e470e657 in_seq=0 out_seq=0 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30b3400640 10 client.25421 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30b3400640 1 -- 172.21.15.46:0/1574150494 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e633a04b30 con 0x55e633b22bb0 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30b3400640 10 client.25421 handle_command_reply: tid=0 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 client.25421 shutdown 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 -- 172.21.15.46:0/1574150494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633aa2510 msgr2=0x55e633ab7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633aa2510 0x55e633ab7a90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30cc01b040 tx=0x7f30cc005270 comp rx=0 tx=0).stop 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 -- 172.21.15.46:0/1574150494 shutdown_connections 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55e633b22bb0 0x55e633b218c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e633aa2510 0x55e633ab7a90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e633a97c50 0x55e633abd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30b3400640 0 client.25421 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:37:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 --2- 172.21.15.46:0/1574150494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e633a97330 0x55e633abcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.003+0000 7f30db69b740 1 -- 172.21.15.46:0/1574150494 >> 172.21.15.46:0/1574150494 conn(0x55e633a938c0 msgr2=0x55e633aad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.004+0000 7f30db69b740 1 -- 172.21.15.46:0/1574150494 shutdown_connections 2024-09-10T07:37:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.004+0000 7f30db69b740 1 -- 172.21.15.46:0/1574150494 wait complete. 2024-09-10T07:37:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.004+0000 7f30db69b740 20 client.25421 trim_cache size 0 max 16384 2024-09-10T07:37:06.009 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:06.009 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:37:06.009 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-09-10T07:37:06.009 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24517, 2024-09-10T07:37:06.009 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:37:06.009 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:37:06.009 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:37:06.009 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 161.93478837200001, 2024-09-10T07:37:06.010 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 30, 2024-09-10T07:37:06.010 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:37:06.010 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 97, 2024-09-10T07:37:06.010 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 233.520357015 2024-09-10T07:37:06.010 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.004+0000 7f30d8c00640 1 -- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30b80a2c00 msgr2=0x7f30b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.004+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30b80a2c00 0x7f30b80a50c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55e633a94720 tx=0x7f30c4005f90 comp rx=0 tx=0).stop 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.004+0000 7f30d8c00640 1 -- 172.21.15.46:0/3601136536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30d41049c0 msgr2=0x7f30d419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.004+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30d41049c0 0x7f30d419ac70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006010 tx=0x7f30c0004040 comp rx=0 tx=0).stop 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.005+0000 7f30d8c00640 1 -- 172.21.15.46:0/3601136536 shutdown_connections 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.005+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d4109140 0x7f30d419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.005+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30d41049c0 0x7f30d419ac70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.005+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30b80a2c00 0x7f30b80a50c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.005+0000 7f30d8c00640 1 --2- 172.21.15.46:0/3601136536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4104010 0x7f30d419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.005+0000 7f30d8c00640 1 -- 172.21.15.46:0/3601136536 >> 172.21.15.46:0/3601136536 conn(0x7f30d40ffa50 msgr2=0x7f30d4100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.005+0000 7f30d8c00640 1 -- 172.21.15.46:0/3601136536 shutdown_connections 2024-09-10T07:37:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:06.005+0000 7f30d8c00640 1 -- 172.21.15.46:0/3601136536 wait complete. 2024-09-10T07:37:06.753 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:06 smithi046 ceph-mon[30705]: pgmap v413: 129 pgs: 129 active+clean; 2.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 194 KiB/s wr, 23 op/s 2024-09-10T07:37:06.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T07:37:06.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:06 smithi181 ceph-mon[35397]: pgmap v413: 129 pgs: 129 active+clean; 2.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 194 KiB/s wr, 23 op/s 2024-09-10T07:37:06.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:06 smithi063 ceph-mon[35020]: pgmap v413: 129 pgs: 129 active+clean; 2.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 194 KiB/s wr, 23 op/s 2024-09-10T07:37:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 -- 172.21.15.46:0/4086593920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76f0105910 msgr2=0x7f76f010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 --2- 172.21.15.46:0/4086593920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76f0105910 0x7f76f010c1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009ca0 tx=0x7f76e802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 -- 172.21.15.46:0/4086593920 shutdown_connections 2024-09-10T07:37:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 --2- 172.21.15.46:0/4086593920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76f0105910 0x7f76f010c1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 --2- 172.21.15.46:0/4086593920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76f0104f50 0x7f76f01053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 --2- 172.21.15.46:0/4086593920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76f0103d60 0x7f76f0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 -- 172.21.15.46:0/4086593920 >> 172.21.15.46:0/4086593920 conn(0x7f76f00ff4e0 msgr2=0x7f76f0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 -- 172.21.15.46:0/4086593920 shutdown_connections 2024-09-10T07:37:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.195+0000 7f76f5600640 1 -- 172.21.15.46:0/4086593920 wait complete. 2024-09-10T07:37:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.196+0000 7f76f5600640 1 Processor -- start 2024-09-10T07:37:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.196+0000 7f76f5600640 1 -- start start 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76f5600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76f0103d60 0x7f76f0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76f5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76f0104f50 0x7f76f019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76f5600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76f0105910 0x7f76f01a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76f5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76f010f9e0 con 0x7f76f0104f50 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76f5600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76f010f860 con 0x7f76f0103d60 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76f5600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76f010fb60 con 0x7f76f0105910 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76eea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76f0103d60 0x7f76f0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76eea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76f0103d60 0x7f76f0199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59008/0 (socket says 172.21.15.46:59008) 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76eea00640 1 -- 172.21.15.46:0/2866993492 learned_addr learned my addr 172.21.15.46:0/2866993492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76ee000640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76f0104f50 0x7f76f019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76ef400640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76f0105910 0x7f76f01a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76eea00640 1 -- 172.21.15.46:0/2866993492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76f0105910 msgr2=0x7f76f01a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76eea00640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76f0105910 0x7f76f01a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76eea00640 1 -- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76f0104f50 msgr2=0x7f76f019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76eea00640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76f0104f50 0x7f76f019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.197+0000 7f76eea00640 1 -- 172.21.15.46:0/2866993492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f01a1b70 con 0x7f76f0103d60 2024-09-10T07:37:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.198+0000 7f76eea00640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76f0103d60 0x7f76f0199f50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76e000e9c0 tx=0x7f76e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.198+0000 7f76e7400640 1 -- 172.21.15.46:0/2866993492 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e000cd80 con 0x7f76f0103d60 2024-09-10T07:37:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.198+0000 7f76e7400640 1 -- 172.21.15.46:0/2866993492 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e00035f0 con 0x7f76f0103d60 2024-09-10T07:37:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.199+0000 7f76f5600640 1 -- 172.21.15.46:0/2866993492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f01a1e60 con 0x7f76f0103d60 2024-09-10T07:37:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.199+0000 7f76e7400640 1 -- 172.21.15.46:0/2866993492 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0010690 con 0x7f76f0103d60 2024-09-10T07:37:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.199+0000 7f76f5600640 1 -- 172.21.15.46:0/2866993492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f01a22c0 con 0x7f76f0103d60 2024-09-10T07:37:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.202+0000 7f76f84d9740 1 Processor -- start 2024-09-10T07:37:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.202+0000 7f76f84d9740 1 -- start start 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.202+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83de7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.202+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83dec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.202+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83deef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.202+0000 7f76f84d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555e83d4c100 con 0x555e83de79d0 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.202+0000 7f76f84d9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555e83cf41b0 con 0x555e83deca60 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.202+0000 7f76f84d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555e83d480c0 con 0x555e83de7090 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83deef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83deef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59022/0 (socket says 172.21.15.46:59022) 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ee000640 1 -- 172.21.15.46:0/2629758435 learned_addr learned my addr 172.21.15.46:0/2629758435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76e7400640 1 -- 172.21.15.46:0/2866993492 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76e00037a0 con 0x7f76f0103d60 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ef400640 1 --2- 172.21.15.46:0/2629758435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83de7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ee000640 1 -- 172.21.15.46:0/2629758435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 msgr2=0x555e83de7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ee000640 1 --2- 172.21.15.46:0/2629758435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83de7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ee000640 1 -- 172.21.15.46:0/2629758435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 msgr2=0x555e83dec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ee000640 1 --2- 172.21.15.46:0/2629758435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83dec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.203+0000 7f76ee000640 1 -- 172.21.15.46:0/2629758435 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e83a5f210 con 0x555e83deca60 2024-09-10T07:37:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.204+0000 7f76ee000640 1 --2- 172.21.15.46:0/2629758435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83deef20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009ca0 tx=0x7f76dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77014bd1102d9fba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.204+0000 7f76db400640 1 -- 172.21.15.46:0/2629758435 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc01e070 con 0x555e83deca60 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.204+0000 7f76db400640 1 -- 172.21.15.46:0/2629758435 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76dc019b10 con 0x555e83deca60 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.204+0000 7f76db400640 1 -- 172.21.15.46:0/2629758435 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00eac0 con 0x555e83deca60 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.204+0000 7f76e7400640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76d40a2c30 0x7f76d40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.204+0000 7f76e7400640 1 -- 172.21.15.46:0/2866993492 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f76e00cf7f0 con 0x7f76f0103d60 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76ef400640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76d40a2c30 0x7f76d40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 -- 172.21.15.46:0/2629758435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 msgr2=0x555e83deef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2629758435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83deef20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009ca0 tx=0x7f76dc011810 comp rx=0 tx=0).stop 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 -- 172.21.15.46:0/2629758435 shutdown_connections 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2629758435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83deef20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2629758435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83dec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2629758435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83de7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 -- 172.21.15.46:0/2629758435 >> 172.21.15.46:0/2629758435 conn(0x555e83de2700 msgr2=0x555e83de2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 -- 172.21.15.46:0/2629758435 shutdown_connections 2024-09-10T07:37:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76f84d9740 1 -- 172.21.15.46:0/2629758435 wait complete. 2024-09-10T07:37:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.205+0000 7f76ef400640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76d40a2c30 0x7f76d40a50f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005e30 tx=0x7f76e803d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.206+0000 7f76f84d9740 1 -- start start 2024-09-10T07:37:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.206+0000 7f76f84d9740 1 Processor -- start 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83df1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83df17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76f84d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555e83defa30 con 0x555e83de79d0 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76f84d9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555e83cd5090 con 0x555e83deca60 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76f84d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555e83defbb0 con 0x555e83de7090 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76f84d9740 10 client.0 _ll_register_callbacks cb 0x555e83d29f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76ee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83df17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76ee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83df17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54636/0 (socket says 172.21.15.46:54636) 2024-09-10T07:37:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76ee000640 1 -- 172.21.15.46:0/2757463697 learned_addr learned my addr 172.21.15.46:0/2757463697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76eea00640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83df1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76ef400640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76ee000640 1 -- 172.21.15.46:0/2757463697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 msgr2=0x555e83df1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.207+0000 7f76ee000640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83df1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76ee000640 1 -- 172.21.15.46:0/2757463697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 msgr2=0x555e83df87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76ee000640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76ee000640 1 -- 172.21.15.46:0/2757463697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e83b2b620 con 0x555e83de79d0 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76ef400640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76eea00640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83df1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76ee000640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83df17b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f76dc011d10 tx=0x7f76dc0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eefbe52ffc58cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76cfe00640 1 -- 172.21.15.46:0/2757463697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc01e050 con 0x555e83de79d0 2024-09-10T07:37:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76cfe00640 1 -- 172.21.15.46:0/2757463697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f76dc00e7c0 con 0x555e83de79d0 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76f84d9740 1 -- 172.21.15.46:0/2757463697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76dc019b10 con 0x555e83de79d0 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76cfe00640 1 -- 172.21.15.46:0/2757463697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76dc032050 con 0x555e83de79d0 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.208+0000 7f76f84d9740 10 client.15405 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76f84d9740 1 -- 172.21.15.46:0/2757463697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e83dfb5e0 con 0x555e83de79d0 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76cfe00640 1 -- 172.21.15.46:0/2757463697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76dc019b10 con 0x555e83de79d0 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76cfe00640 1 -- 172.21.15.46:0/2757463697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f76dc004990 con 0x555e83de79d0 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76f84d9740 10 client.15405 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76f84d9740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76f84d9740 20 client.15405 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e83dfca60 0x555e83dfee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76f84d9740 4 client.15405 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.209+0000 7f76f84d9740 1 -- 172.21.15.46:0/2757463697 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e83d59830 con 0x555e83dfca60 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.210+0000 7f76eea00640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e83dfca60 0x555e83dfee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.210+0000 7f76eea00640 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e83dfca60 0x555e83dfee50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248f161d49ade42e server_cookie=5d1892c4fc36a209 in_seq=0 out_seq=0 2024-09-10T07:37:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.210+0000 7f76cfe00640 10 client.15405 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76cfe00640 1 -- 172.21.15.46:0/2757463697 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e83d59830 con 0x555e83dfca60 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76cfe00640 10 client.15405 handle_command_reply: tid=0 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 client.15405 shutdown 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 -- 172.21.15.46:0/2757463697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 msgr2=0x555e83df17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83df17b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f76dc011d10 tx=0x7f76dc0052f0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 -- 172.21.15.46:0/2757463697 shutdown_connections 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e83dfca60 0x555e83dfee50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de79d0 0x555e83df17b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 --2- 172.21.15.46:0/2757463697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de7090 0x555e83df1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76cfe00640 0 client.15405 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.211+0000 7f76f84d9740 1 -- 172.21.15.46:0/2757463697 >> 172.21.15.46:0/2757463697 conn(0x555e83de2700 msgr2=0x555e83de3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.212+0000 7f76f84d9740 1 -- 172.21.15.46:0/2757463697 shutdown_connections 2024-09-10T07:37:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.212+0000 7f76f84d9740 1 -- 172.21.15.46:0/2757463697 wait complete. 2024-09-10T07:37:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.212+0000 7f76f84d9740 20 client.15405 trim_cache size 0 max 16384 2024-09-10T07:37:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.227+0000 7f76f84d9740 1 Processor -- start 2024-09-10T07:37:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.227+0000 7f76f84d9740 1 -- start start 2024-09-10T07:37:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.227+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.227+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83de79d0 0x555e83df87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.227+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83deca60 0x555e83de3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76f84d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01ac890 con 0x555e83de7090 2024-09-10T07:37:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76f84d9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01ae3d0 con 0x555e83de79d0 2024-09-10T07:37:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76f84d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01acf60 con 0x555e83deca60 2024-09-10T07:37:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83de79d0 0x555e83df87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83de79d0 0x555e83df87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59054/0 (socket says 172.21.15.46:59054) 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ee000640 1 -- 172.21.15.46:0/3509671115 learned_addr learned my addr 172.21.15.46:0/3509671115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ef400640 1 --2- 172.21.15.46:0/3509671115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83deca60 0x555e83de3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ee000640 1 -- 172.21.15.46:0/3509671115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83deca60 msgr2=0x555e83de3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ee000640 1 --2- 172.21.15.46:0/3509671115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83deca60 0x555e83de3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ee000640 1 -- 172.21.15.46:0/3509671115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 msgr2=0x555e83df0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ee000640 1 --2- 172.21.15.46:0/3509671115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.228+0000 7f76ee000640 1 -- 172.21.15.46:0/3509671115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e83de2520 con 0x555e83de79d0 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.229+0000 7f76ee000640 1 --2- 172.21.15.46:0/3509671115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83de79d0 0x555e83df87b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76dc01c040 tx=0x7f76dc0097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5a1f147bf84c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.229+0000 7f76daa00640 1 -- 172.21.15.46:0/3509671115 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc01e040 con 0x555e83de79d0 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.229+0000 7f76daa00640 1 -- 172.21.15.46:0/3509671115 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76dc01f040 con 0x555e83de79d0 2024-09-10T07:37:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.229+0000 7f76daa00640 1 -- 172.21.15.46:0/3509671115 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc019930 con 0x555e83de79d0 2024-09-10T07:37:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 -- 172.21.15.46:0/3509671115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83de79d0 msgr2=0x555e83df87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 --2- 172.21.15.46:0/3509671115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83de79d0 0x555e83df87b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76dc01c040 tx=0x7f76dc0097c0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 -- 172.21.15.46:0/3509671115 shutdown_connections 2024-09-10T07:37:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 --2- 172.21.15.46:0/3509671115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83deca60 0x555e83de3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 --2- 172.21.15.46:0/3509671115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83de79d0 0x555e83df87b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 --2- 172.21.15.46:0/3509671115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 -- 172.21.15.46:0/3509671115 >> 172.21.15.46:0/3509671115 conn(0x555e83de2700 msgr2=0x555e83de1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 -- 172.21.15.46:0/3509671115 shutdown_connections 2024-09-10T07:37:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.230+0000 7f76f84d9740 1 -- 172.21.15.46:0/3509671115 wait complete. 2024-09-10T07:37:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.231+0000 7f76f84d9740 1 -- start start 2024-09-10T07:37:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.231+0000 7f76f84d9740 1 Processor -- start 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de79d0 0x555e83df60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76f84d9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76f84d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01add00 con 0x555e83de7090 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76f84d9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01ad630 con 0x555e83deca60 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76f84d9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01aa870 con 0x555e83de79d0 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76f84d9740 10 client.0 _ll_register_callbacks cb 0x7f76dc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54644/0 (socket says 172.21.15.46:54644) 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76eea00640 1 -- 172.21.15.46:0/770816193 learned_addr learned my addr 172.21.15.46:0/770816193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76ef400640 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76ee000640 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de79d0 0x555e83df60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76eea00640 1 -- 172.21.15.46:0/770816193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de79d0 msgr2=0x555e83df60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76eea00640 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de79d0 0x555e83df60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76eea00640 1 -- 172.21.15.46:0/770816193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 msgr2=0x555e83df70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76eea00640 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.232+0000 7f76eea00640 1 -- 172.21.15.46:0/770816193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e83df7700 con 0x555e83de7090 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76eea00640 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df5ba0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c6a0 tx=0x7f76e009bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e81030695d83870 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76cfe00640 1 -- 172.21.15.46:0/770816193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0003b30 con 0x555e83de7090 2024-09-10T07:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76cfe00640 1 -- 172.21.15.46:0/770816193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f76e00982e0 con 0x555e83de7090 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76f84d9740 1 -- 172.21.15.46:0/770816193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76dc01f040 con 0x555e83de7090 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76cfe00640 1 -- 172.21.15.46:0/770816193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76e00a4450 con 0x555e83de7090 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76f84d9740 10 client.15411 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76f84d9740 1 -- 172.21.15.46:0/770816193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e83df8ae0 con 0x555e83de7090 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.233+0000 7f76cfe00640 1 -- 172.21.15.46:0/770816193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76dc01f040 con 0x555e83de7090 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.234+0000 7f76cfe00640 1 -- 172.21.15.46:0/770816193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f76e00979c0 con 0x555e83de7090 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.234+0000 7f76f84d9740 10 client.15411 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.234+0000 7f76f84d9740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.234+0000 7f76f84d9740 20 client.15411 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.234+0000 7f76f84d9740 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e83e67d30 0x555e83e6a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.234+0000 7f76f84d9740 4 client.15411 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.234+0000 7f76f84d9740 1 -- 172.21.15.46:0/770816193 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e83d53b80 con 0x555e83e67d30 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.234+0000 7f76ee000640 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e83e67d30 0x555e83e6a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.235+0000 7f76ee000640 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e83e67d30 0x555e83e6a150 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de518f397fe147a7 server_cookie=bf0b2397ec71e317 in_seq=0 out_seq=0 2024-09-10T07:37:07.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.235+0000 7f76cfe00640 10 client.15411 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76cfe00640 1 -- 172.21.15.46:0/770816193 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e83d53b80 con 0x555e83e67d30 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76cfe00640 10 client.15411 handle_command_reply: tid=0 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 client.15411 shutdown 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 -- 172.21.15.46:0/770816193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 msgr2=0x555e83df5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df5ba0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c6a0 tx=0x7f76e009bd30 comp rx=0 tx=0).stop 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 -- 172.21.15.46:0/770816193 shutdown_connections 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e83e67d30 0x555e83e6a150 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e83deca60 0x555e83df70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e83de79d0 0x555e83df60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 --2- 172.21.15.46:0/770816193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e83de7090 0x555e83df5ba0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 -- 172.21.15.46:0/770816193 >> 172.21.15.46:0/770816193 conn(0x555e83de2700 msgr2=0x555e83de2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76cfe00640 0 client.15411 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.673+0000 7f76f84d9740 1 -- 172.21.15.46:0/770816193 shutdown_connections 2024-09-10T07:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.674+0000 7f76f84d9740 1 -- 172.21.15.46:0/770816193 wait complete. 2024-09-10T07:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.674+0000 7f76f84d9740 20 client.15411 trim_cache size 0 max 16384 2024-09-10T07:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:37:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:37:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.674+0000 7f76f5600640 1 -- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76d40a2c30 msgr2=0x7f76d40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.674+0000 7f76f5600640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76d40a2c30 0x7f76d40a50f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005e30 tx=0x7f76e803d040 comp rx=0 tx=0).stop 2024-09-10T07:37:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 -- 172.21.15.46:0/2866993492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76f0103d60 msgr2=0x7f76f0199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76f0103d60 0x7f76f0199f50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76e000e9c0 tx=0x7f76e000ee80 comp rx=0 tx=0).stop 2024-09-10T07:37:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 -- 172.21.15.46:0/2866993492 shutdown_connections 2024-09-10T07:37:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76f0105910 0x7f76f01a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f76d40a2c30 0x7f76d40a50f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76f0104f50 0x7f76f019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 --2- 172.21.15.46:0/2866993492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76f0103d60 0x7f76f0199f50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 -- 172.21.15.46:0/2866993492 >> 172.21.15.46:0/2866993492 conn(0x7f76f00ff4e0 msgr2=0x7f76f0108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 -- 172.21.15.46:0/2866993492 shutdown_connections 2024-09-10T07:37:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:07.675+0000 7f76f5600640 1 -- 172.21.15.46:0/2866993492 wait complete. 2024-09-10T07:37:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.916+0000 7f513a400640 1 -- 172.21.15.46:0/544204335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 msgr2=0x7f513411d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.916+0000 7f513a400640 1 --2- 172.21.15.46:0/544204335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 0x7f513411d940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5120009900 tx=0x7f512002a490 comp rx=0 tx=0).stop 2024-09-10T07:37:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.917+0000 7f513a400640 1 -- 172.21.15.46:0/544204335 shutdown_connections 2024-09-10T07:37:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.917+0000 7f513a400640 1 --2- 172.21.15.46:0/544204335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 0x7f513411d940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.917+0000 7f513a400640 1 --2- 172.21.15.46:0/544204335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513406c200 0x7f513411b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.917+0000 7f513a400640 1 --2- 172.21.15.46:0/544204335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f513406b8e0 0x7f513406bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.917+0000 7f513a400640 1 -- 172.21.15.46:0/544204335 >> 172.21.15.46:0/544204335 conn(0x7f5134071630 msgr2=0x7f5134073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.918+0000 7f513a400640 1 -- 172.21.15.46:0/544204335 shutdown_connections 2024-09-10T07:37:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.918+0000 7f513a400640 1 -- 172.21.15.46:0/544204335 wait complete. 2024-09-10T07:37:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.919+0000 7f513a400640 1 Processor -- start 2024-09-10T07:37:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.920+0000 7f513a400640 1 -- start start 2024-09-10T07:37:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.921+0000 7f513a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513406b8e0 0x7f51341a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.921+0000 7f513a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f513406c200 0x7f51341a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.921+0000 7f513a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 0x7f51341abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.921+0000 7f513a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5134120d70 con 0x7f513406b8e0 2024-09-10T07:37:07.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.921+0000 7f513a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5134120bf0 con 0x7f513406c200 2024-09-10T07:37:07.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.921+0000 7f513a400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5134120ef0 con 0x7f513411b550 2024-09-10T07:37:07.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.922+0000 7f5133e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 0x7f51341abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.923+0000 7f5133e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 0x7f51341abaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59736/0 (socket says 172.21.15.46:59736) 2024-09-10T07:37:07.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.923+0000 7f5133e00640 1 -- 172.21.15.46:0/2038119288 learned_addr learned my addr 172.21.15.46:0/2038119288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:07.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.923+0000 7f5132a00640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f513406c200 0x7f51341a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.923+0000 7f5132a00640 1 -- 172.21.15.46:0/2038119288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 msgr2=0x7f51341abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.923+0000 7f5132a00640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 0x7f51341abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.923+0000 7f5132a00640 1 -- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513406b8e0 msgr2=0x7f51341a71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.923+0000 7f5132a00640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513406b8e0 0x7f51341a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.923+0000 7f5132a00640 1 -- 172.21.15.46:0/2038119288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51341ac180 con 0x7f513406c200 2024-09-10T07:37:07.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.924+0000 7f5132a00640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f513406c200 0x7f51341a7710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f512400e9f0 tx=0x7f512400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.924+0000 7f512be00640 1 -- 172.21.15.46:0/2038119288 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512400ccc0 con 0x7f513406c200 2024-09-10T07:37:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.924+0000 7f512be00640 1 -- 172.21.15.46:0/2038119288 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5124010910 con 0x7f513406c200 2024-09-10T07:37:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.924+0000 7f512be00640 1 -- 172.21.15.46:0/2038119288 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124018960 con 0x7f513406c200 2024-09-10T07:37:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.925+0000 7f513a400640 1 -- 172.21.15.46:0/2038119288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51341ac470 con 0x7f513406c200 2024-09-10T07:37:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.926+0000 7f513a400640 1 -- 172.21.15.46:0/2038119288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51341b3d50 con 0x7f513406c200 2024-09-10T07:37:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.932+0000 7f513ce88740 1 Processor -- start 2024-09-10T07:37:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.932+0000 7f513ce88740 1 -- start start 2024-09-10T07:37:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.933+0000 7f513ce88740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745330 0x55f166745710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.933+0000 7f513ce88740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745c50 0x55f16674ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.933+0000 7f513ce88740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166750510 0x55f166752900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.933+0000 7f513ce88740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f1666ab0b0 con 0x55f166745c50 2024-09-10T07:37:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.933+0000 7f513ce88740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f166652ab0 con 0x55f166745330 2024-09-10T07:37:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.933+0000 7f513ce88740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f1666a7070 con 0x55f166750510 2024-09-10T07:37:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.933+0000 7f5133e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745330 0x55f166745710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.934+0000 7f5133e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745330 0x55f166745710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59094/0 (socket says 172.21.15.46:59094) 2024-09-10T07:37:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.934+0000 7f5133e00640 1 -- 172.21.15.46:0/3311425342 learned_addr learned my addr 172.21.15.46:0/3311425342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.934+0000 7f5133e00640 1 -- 172.21.15.46:0/3311425342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166750510 msgr2=0x55f166752900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:37:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.934+0000 7f5133e00640 1 --2- 172.21.15.46:0/3311425342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166750510 0x55f166752900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.934+0000 7f5133e00640 1 -- 172.21.15.46:0/3311425342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745c50 msgr2=0x55f16674ffd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:37:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.937+0000 7f5133e00640 1 --2- 172.21.15.46:0/3311425342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745c50 0x55f16674ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.937+0000 7f5133e00640 1 -- 172.21.15.46:0/3311425342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f16665f4a0 con 0x55f166745330 2024-09-10T07:37:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.937+0000 7f5133e00640 1 --2- 172.21.15.46:0/3311425342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745330 0x55f166745710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f51200314b0 tx=0x7f51200314e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa3b97e8d4114d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.938+0000 7f512be00640 1 -- 172.21.15.46:0/2038119288 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5124018b00 con 0x7f513406c200 2024-09-10T07:37:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.939+0000 7f5128c00640 1 -- 172.21.15.46:0/3311425342 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120035040 con 0x55f166745330 2024-09-10T07:37:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.939+0000 7f5128c00640 1 -- 172.21.15.46:0/3311425342 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5120004400 con 0x55f166745330 2024-09-10T07:37:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.939+0000 7f5128c00640 1 -- 172.21.15.46:0/3311425342 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120003850 con 0x55f166745330 2024-09-10T07:37:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.942+0000 7f512be00640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51180a2cc0 0x7f51180a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.942+0000 7f512be00640 1 -- 172.21.15.46:0/2038119288 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f51240cf760 con 0x7f513406c200 2024-09-10T07:37:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.942+0000 7f513ce88740 1 -- 172.21.15.46:0/3311425342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745330 msgr2=0x55f166745710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.943+0000 7f513ce88740 1 --2- 172.21.15.46:0/3311425342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745330 0x55f166745710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f51200314b0 tx=0x7f51200314e0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.943+0000 7f513ce88740 1 -- 172.21.15.46:0/3311425342 shutdown_connections 2024-09-10T07:37:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.943+0000 7f513ce88740 1 --2- 172.21.15.46:0/3311425342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166750510 0x55f166752900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.943+0000 7f513ce88740 1 --2- 172.21.15.46:0/3311425342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745c50 0x55f16674ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.943+0000 7f513ce88740 1 --2- 172.21.15.46:0/3311425342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745330 0x55f166745710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.943+0000 7f513ce88740 1 -- 172.21.15.46:0/3311425342 >> 172.21.15.46:0/3311425342 conn(0x55f1667418c0 msgr2=0x55f166741ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.946+0000 7f5133400640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51180a2cc0 0x7f51180a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.946+0000 7f513ce88740 1 -- 172.21.15.46:0/3311425342 shutdown_connections 2024-09-10T07:37:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.946+0000 7f513ce88740 1 -- 172.21.15.46:0/3311425342 wait complete. 2024-09-10T07:37:07.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.947+0000 7f513ce88740 1 -- start start 2024-09-10T07:37:07.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.947+0000 7f5133400640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51180a2cc0 0x7f51180a5180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f512c009270 tx=0x7f512c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.947+0000 7f513ce88740 1 Processor -- start 2024-09-10T07:37:07.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.951+0000 7f513ce88740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745330 0x55f16674d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.951+0000 7f513ce88740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f16674d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.952+0000 7f513ce88740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166750510 0x55f16674def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.952+0000 7f513ce88740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f1667530c0 con 0x55f166750510 2024-09-10T07:37:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.952+0000 7f513ce88740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f166633920 con 0x55f166745c50 2024-09-10T07:37:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.952+0000 7f513ce88740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f166753240 con 0x55f166745330 2024-09-10T07:37:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.953+0000 7f513ce88740 10 client.0 _ll_register_callbacks cb 0x55f1666578a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.954+0000 7f5133400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745330 0x55f16674d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.954+0000 7f5132a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f16674d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.954+0000 7f5132a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f16674d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59106/0 (socket says 172.21.15.46:59106) 2024-09-10T07:37:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.954+0000 7f5132a00640 1 -- 172.21.15.46:0/2528599380 learned_addr learned my addr 172.21.15.46:0/2528599380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.955+0000 7f5133e00640 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166750510 0x55f16674def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.955+0000 7f5132a00640 1 -- 172.21.15.46:0/2528599380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745330 msgr2=0x55f16674d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.955+0000 7f5132a00640 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745330 0x55f16674d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.955+0000 7f5132a00640 1 -- 172.21.15.46:0/2528599380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166750510 msgr2=0x55f16674def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.955+0000 7f5132a00640 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166750510 0x55f16674def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.955+0000 7f5132a00640 1 -- 172.21.15.46:0/2528599380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f16641c2b0 con 0x55f166745c50 2024-09-10T07:37:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.955+0000 7f5132a00640 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f16674d9b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f51240985b0 tx=0x7f512409b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13deb866b6be58cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.955+0000 7f511d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.956+0000 7f511d600640 1 -- 172.21.15.46:0/2528599380 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124086040 con 0x55f166745c50 2024-09-10T07:37:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.957+0000 7f511d600640 1 -- 172.21.15.46:0/2528599380 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f51240cf480 con 0x55f166745c50 2024-09-10T07:37:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.957+0000 7f513ce88740 1 -- 172.21.15.46:0/2528599380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5120004400 con 0x55f166745c50 2024-09-10T07:37:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.958+0000 7f511d600640 1 -- 172.21.15.46:0/2528599380 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51240b9050 con 0x55f166745c50 2024-09-10T07:37:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.958+0000 7f513ce88740 10 client.25451 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.958+0000 7f513ce88740 1 -- 172.21.15.46:0/2528599380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f16675eee0 con 0x55f166745c50 2024-09-10T07:37:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.958+0000 7f511d600640 1 -- 172.21.15.46:0/2528599380 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5120004400 con 0x55f166745c50 2024-09-10T07:37:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.959+0000 7f511d600640 1 -- 172.21.15.46:0/2528599380 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f512409b0c0 con 0x55f166745c50 2024-09-10T07:37:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.959+0000 7f513ce88740 10 client.25451 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.959+0000 7f513ce88740 10 client.25451 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:37:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.959+0000 7f513ce88740 20 client.25451 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.959+0000 7f513ce88740 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55f1667614e0 0x55f166762950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.959+0000 7f513ce88740 4 client.25451 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.959+0000 7f513ce88740 1 -- 172.21.15.46:0/2528599380 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1666b8620 con 0x55f1667614e0 2024-09-10T07:37:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.959+0000 7f5133400640 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55f1667614e0 0x55f166762950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.960+0000 7f511d600640 10 client.25451 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:37:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.960+0000 7f5133400640 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55f1667614e0 0x55f166762950 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a72cb58833ec684 server_cookie=f78724dcb4f18eec in_seq=0 out_seq=0 2024-09-10T07:37:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.962+0000 7f511d600640 1 -- 172.21.15.46:0/2528599380 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1666b8620 con 0x55f1667614e0 2024-09-10T07:37:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.962+0000 7f511d600640 10 client.25451 handle_command_reply: tid=0 2024-09-10T07:37:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.962+0000 7f513ce88740 1 client.25451 shutdown 2024-09-10T07:37:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.963+0000 7f513ce88740 1 -- 172.21.15.46:0/2528599380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 msgr2=0x55f16674d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.964+0000 7f513ce88740 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f16674d9b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f51240985b0 tx=0x7f512409b630 comp rx=0 tx=0).stop 2024-09-10T07:37:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.964+0000 7f513ce88740 1 -- 172.21.15.46:0/2528599380 shutdown_connections 2024-09-10T07:37:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.964+0000 7f513ce88740 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55f1667614e0 0x55f166762950 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.964+0000 7f513ce88740 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166750510 0x55f16674def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.964+0000 7f511d600640 0 client.25451 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:37:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.964+0000 7f513ce88740 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f16674d9b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.964+0000 7f513ce88740 1 --2- 172.21.15.46:0/2528599380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745330 0x55f16674d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.964+0000 7f513ce88740 1 -- 172.21.15.46:0/2528599380 >> 172.21.15.46:0/2528599380 conn(0x55f1667418c0 msgr2=0x55f166747640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.965+0000 7f513ce88740 1 -- 172.21.15.46:0/2528599380 shutdown_connections 2024-09-10T07:37:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.965+0000 7f513ce88740 1 -- 172.21.15.46:0/2528599380 wait complete. 2024-09-10T07:37:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:07.965+0000 7f513ce88740 20 client.25451 trim_cache size 0 max 16384 2024-09-10T07:37:08.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.012+0000 7f513ce88740 1 Processor -- start 2024-09-10T07:37:08.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.012+0000 7f513ce88740 1 -- start start 2024-09-10T07:37:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f513ce88740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745330 0x55f16674dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f513ce88740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f1667474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f513ce88740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166750510 0x55f16676ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f513ce88740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f166760c90 con 0x55f166745330 2024-09-10T07:37:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f513ce88740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f166761360 con 0x55f166745c50 2024-09-10T07:37:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f513ce88740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f166761a30 con 0x55f166750510 2024-09-10T07:37:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f5132a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f1667474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f5132a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f1667474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59118/0 (socket says 172.21.15.46:59118) 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.013+0000 7f5132a00640 1 -- 172.21.15.46:0/4161445370 learned_addr learned my addr 172.21.15.46:0/4161445370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.014+0000 7f5132a00640 1 -- 172.21.15.46:0/4161445370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166750510 msgr2=0x55f16676ac30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.014+0000 7f5132a00640 1 --2- 172.21.15.46:0/4161445370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166750510 0x55f16676ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.014+0000 7f5132a00640 1 -- 172.21.15.46:0/4161445370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745330 msgr2=0x55f16674dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.014+0000 7f5132a00640 1 --2- 172.21.15.46:0/4161445370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745330 0x55f16674dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.014+0000 7f5132a00640 1 -- 172.21.15.46:0/4161445370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f16674e7f0 con 0x55f166745c50 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.014+0000 7f5132a00640 1 --2- 172.21.15.46:0/4161445370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f1667474e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f512409b460 tx=0x7f5124009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97cefc94cd17a955 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.014+0000 7f511fe00640 1 -- 172.21.15.46:0/4161445370 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124086040 con 0x55f166745c50 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.015+0000 7f511fe00640 1 -- 172.21.15.46:0/4161445370 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f512409bbe0 con 0x55f166745c50 2024-09-10T07:37:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.015+0000 7f511fe00640 1 -- 172.21.15.46:0/4161445370 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512409b030 con 0x55f166745c50 2024-09-10T07:37:08.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.015+0000 7f513ce88740 1 -- 172.21.15.46:0/4161445370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 msgr2=0x55f1667474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:08.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.015+0000 7f513ce88740 1 --2- 172.21.15.46:0/4161445370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f1667474e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f512409b460 tx=0x7f5124009700 comp rx=0 tx=0).stop 2024-09-10T07:37:08.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.016+0000 7f513ce88740 1 -- 172.21.15.46:0/4161445370 shutdown_connections 2024-09-10T07:37:08.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.016+0000 7f513ce88740 1 --2- 172.21.15.46:0/4161445370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166750510 0x55f16676ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.016+0000 7f513ce88740 1 --2- 172.21.15.46:0/4161445370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166745c50 0x55f1667474e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.016+0000 7f513ce88740 1 --2- 172.21.15.46:0/4161445370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745330 0x55f16674dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.016+0000 7f513ce88740 1 -- 172.21.15.46:0/4161445370 >> 172.21.15.46:0/4161445370 conn(0x55f1667418c0 msgr2=0x55f166746dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:08.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.017+0000 7f513ce88740 1 -- 172.21.15.46:0/4161445370 shutdown_connections 2024-09-10T07:37:08.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.018+0000 7f513ce88740 1 -- 172.21.15.46:0/4161445370 wait complete. 2024-09-10T07:37:08.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.018+0000 7f513ce88740 1 -- start start 2024-09-10T07:37:08.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.018+0000 7f513ce88740 1 Processor -- start 2024-09-10T07:37:08.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.019+0000 7f513ce88740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745330 0x55f1667649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:08.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.019+0000 7f513ce88740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745c50 0x55f166764f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:08.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.019+0000 7f513ce88740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166750510 0x55f166765440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.020+0000 7f513ce88740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f1667627d0 con 0x55f166745330 2024-09-10T07:37:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.020+0000 7f513ce88740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f166762100 con 0x55f166750510 2024-09-10T07:37:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.020+0000 7f513ce88740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f166762ea0 con 0x55f166745c50 2024-09-10T07:37:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.020+0000 7f513ce88740 10 client.0 _ll_register_callbacks cb 0x7f512409b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.020+0000 7f5133e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166750510 0x55f166765440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.020+0000 7f5133e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166750510 0x55f166765440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59130/0 (socket says 172.21.15.46:59130) 2024-09-10T07:37:08.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.020+0000 7f5133e00640 1 -- 172.21.15.46:0/2709693647 learned_addr learned my addr 172.21.15.46:0/2709693647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:08.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.020+0000 7f5132a00640 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745c50 0x55f166764f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:08.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.021+0000 7f5133e00640 1 -- 172.21.15.46:0/2709693647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745c50 msgr2=0x55f166764f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.021+0000 7f5133e00640 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745c50 0x55f166764f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.021+0000 7f5133e00640 1 -- 172.21.15.46:0/2709693647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745330 msgr2=0x55f1667649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.021+0000 7f5133e00640 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745330 0x55f1667649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.021+0000 7f5133e00640 1 -- 172.21.15.46:0/2709693647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1667691e0 con 0x55f166750510 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.021+0000 7f5132a00640 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745c50 0x55f166764f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.021+0000 7f5133e00640 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166750510 0x55f166765440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5120000c00 tx=0x7f512002a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8265b3b9bc93fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.021+0000 7f511d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.022+0000 7f511d600640 1 -- 172.21.15.46:0/2709693647 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120051060 con 0x55f166750510 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.022+0000 7f511d600640 1 -- 172.21.15.46:0/2709693647 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5120005450 con 0x55f166750510 2024-09-10T07:37:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.022+0000 7f513ce88740 1 -- 172.21.15.46:0/2709693647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f512409bbe0 con 0x55f166750510 2024-09-10T07:37:08.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.022+0000 7f511d600640 1 -- 172.21.15.46:0/2709693647 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5120035050 con 0x55f166750510 2024-09-10T07:37:08.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.023+0000 7f513ce88740 10 client.25457 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:08.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.023+0000 7f513ce88740 1 -- 172.21.15.46:0/2709693647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f1667cbef0 con 0x55f166750510 2024-09-10T07:37:08.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.023+0000 7f511d600640 1 -- 172.21.15.46:0/2709693647 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f512409bbe0 con 0x55f166750510 2024-09-10T07:37:08.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.023+0000 7f511d600640 1 -- 172.21.15.46:0/2709693647 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5120031880 con 0x55f166750510 2024-09-10T07:37:08.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.024+0000 7f513ce88740 10 client.25457 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:08.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.024+0000 7f513ce88740 10 client.25457 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:37:08.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.024+0000 7f513ce88740 20 client.25457 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:08.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.024+0000 7f513ce88740 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55f1667d0610 0x55f1667cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:08.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.024+0000 7f513ce88740 4 client.25457 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:37:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.024+0000 7f513ce88740 1 -- 172.21.15.46:0/2709693647 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "status"}) -- 0x55f1666b2b30 con 0x55f1667d0610 2024-09-10T07:37:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.025+0000 7f5133400640 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55f1667d0610 0x55f1667cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.026+0000 7f5133400640 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55f1667d0610 0x55f1667cf320 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eecaa5c5b96bc2aa server_cookie=27487f2074030c7d in_seq=0 out_seq=0 2024-09-10T07:37:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.026+0000 7f511d600640 10 client.25457 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:37:08.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.027+0000 7f511d600640 1 -- 172.21.15.46:0/2709693647 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f1666b2b30 con 0x55f1667d0610 2024-09-10T07:37:08.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.027+0000 7f511d600640 10 client.25457 handle_command_reply: tid=0 2024-09-10T07:37:08.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.027+0000 7f513ce88740 1 client.25457 shutdown 2024-09-10T07:37:08.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f513ce88740 1 -- 172.21.15.46:0/2709693647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166750510 msgr2=0x55f166765440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:08.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f513ce88740 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166750510 0x55f166765440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5120000c00 tx=0x7f512002a490 comp rx=0 tx=0).stop 2024-09-10T07:37:08.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f513ce88740 1 -- 172.21.15.46:0/2709693647 shutdown_connections 2024-09-10T07:37:08.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f513ce88740 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55f1667d0610 0x55f1667cf320 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f511d600640 0 client.25457 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:37:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f513ce88740 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f166750510 0x55f166765440 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f513ce88740 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f166745c50 0x55f166764f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f513ce88740 1 --2- 172.21.15.46:0/2709693647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f166745330 0x55f1667649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.029+0000 7f513ce88740 1 -- 172.21.15.46:0/2709693647 >> 172.21.15.46:0/2709693647 conn(0x55f1667418c0 msgr2=0x55f16674b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.031+0000 7f513ce88740 1 -- 172.21.15.46:0/2709693647 shutdown_connections 2024-09-10T07:37:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.031+0000 7f513ce88740 1 -- 172.21.15.46:0/2709693647 wait complete. 2024-09-10T07:37:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.031+0000 7f513ce88740 20 client.25457 trim_cache size 0 max 16384 2024-09-10T07:37:08.034 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:08.034 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:37:08.034 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-09-10T07:37:08.034 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24530, 2024-09-10T07:37:08.034 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:37:08.034 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:37:08.034 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:37:08.035 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 164.97127048900001, 2024-09-10T07:37:08.035 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 30, 2024-09-10T07:37:08.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:37:08.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 97, 2024-09-10T07:37:08.035 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 224.99965188199999 2024-09-10T07:37:08.035 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.034+0000 7f513a400640 1 -- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51180a2cc0 msgr2=0x7f51180a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.034+0000 7f513a400640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51180a2cc0 0x7f51180a5180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f512c009270 tx=0x7f512c005bd0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.035+0000 7f513a400640 1 -- 172.21.15.46:0/2038119288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f513406c200 msgr2=0x7f51341a7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.035+0000 7f513a400640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f513406c200 0x7f51341a7710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f512400e9f0 tx=0x7f512400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.035+0000 7f513a400640 1 -- 172.21.15.46:0/2038119288 shutdown_connections 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.035+0000 7f513a400640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f513411b550 0x7f51341abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.036+0000 7f513a400640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f513406c200 0x7f51341a7710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.036+0000 7f513a400640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51180a2cc0 0x7f51180a5180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.036+0000 7f513a400640 1 --2- 172.21.15.46:0/2038119288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513406b8e0 0x7f51341a71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.036+0000 7f513a400640 1 -- 172.21.15.46:0/2038119288 >> 172.21.15.46:0/2038119288 conn(0x7f5134071630 msgr2=0x7f5134072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:08.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.037+0000 7f513a400640 1 -- 172.21.15.46:0/2038119288 shutdown_connections 2024-09-10T07:37:08.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:08.037+0000 7f513a400640 1 -- 172.21.15.46:0/2038119288 wait complete. 2024-09-10T07:37:08.427 INFO:teuthology.orchestra.run.smithi046.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-10T07:37:08.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/: 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 . 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 .. 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 data 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data: 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:total 35 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 . 2024-09-10T07:37:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 .. 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 base 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 10 07:37 global 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 log 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_commit_ts 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_dynshmem 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 10 07:37 pg_hba.conf 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 10 07:37 pg_ident.conf 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 pg_logical 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 pg_multixact 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_notify 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_replslot 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_serial 2024-09-10T07:37:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_snapshots 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_stat 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_stat_tmp 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 pg_subtrans 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_tblspc 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_twophase 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 PG_VERSION 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 pg_wal 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 pg_xact 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 10 07:37 postgresql.auto.conf 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 10 07:37 postgresql.conf 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base: 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 . 2024-09-10T07:37:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 1 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 13493 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 13494 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:total 7777 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 . 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 .. 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 112 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 113 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 1247 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1247_fsm 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1247_vm 2024-09-10T07:37:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 07:37 1249 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1249_fsm 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1249_vm 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 07:37 1255 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1255_fsm 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1255_vm 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 07:37 1259 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1259_fsm 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1259_vm 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 07:37 13344 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13344_fsm 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13344_vm 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13346 2024-09-10T07:37:08.536 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13348 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13349_fsm 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349_vm 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13351 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13353 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13354_fsm 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354_vm 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13356 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13358 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13359_fsm 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359_vm 2024-09-10T07:37:08.537 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13361 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13363 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1417 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1418 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 174 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 175 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2187 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2224 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2328 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2336 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2337 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2579 2024-09-10T07:37:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2600 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2600_fsm 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2600_vm 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2601_fsm 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601_vm 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2602 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2602_fsm 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2602_vm 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2603 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2603_fsm 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2603_vm 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2604 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2605 2024-09-10T07:37:08.539 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2605_fsm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2605_vm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2606_fsm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606_vm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2607 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2607_fsm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2607_vm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 07:37 2608 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2608_fsm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2608_vm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 07:37 2609 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2609_fsm 2024-09-10T07:37:08.540 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2609_vm 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2610 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2610_fsm 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2610_vm 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2611 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2612_fsm 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612_vm 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2613 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2615_fsm 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615_vm 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2616 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2616_fsm 2024-09-10T07:37:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2616_vm 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 07:37 2617 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2617_fsm 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2617_vm 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 07:37 2618 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2618_fsm 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2618_vm 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 07:37 2619 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2619_fsm 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2619_vm 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2620 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2650 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2651 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2652 2024-09-10T07:37:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2653 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2654 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2655 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2656 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2657 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 07:37 2658 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 2659 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2660 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2661 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2662 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2663 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2664 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2665 2024-09-10T07:37:08.543 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2666 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2667 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2668 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2669 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2670 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 07:37 2673 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 07:37 2674 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 07:37 2675 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2678 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2679 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2680 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2681 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2682 2024-09-10T07:37:08.544 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2683 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2684 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2685 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2686 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2687 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2688 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2689 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 07:37 2690 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 07:37 2691 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2692 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2693 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2696 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2699 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2701 2024-09-10T07:37:08.545 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2702 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2703 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2704 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2753 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2753_fsm 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2753_vm 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2754 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2755 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2756 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2757 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2830 2024-09-10T07:37:08.546 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2831 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2832 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2833 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2834 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2835 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2836 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2837 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 07:37 2838 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2838_fsm 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2838_vm 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2839 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840_fsm 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2840_vm 2024-09-10T07:37:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2841 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2995 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2996 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3079_fsm 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079_vm 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3080 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3081 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3085 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3118 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3119 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 07:37 3164 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3256 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3257 2024-09-10T07:37:08.548 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3258 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3350 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3351 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3379 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3380 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3381 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3394 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3394_fsm 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3394_vm 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3395 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3429 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3430 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3431 2024-09-10T07:37:08.549 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3433 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3439 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3440 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3455 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 07:37 3456 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3456_fsm 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3456_vm 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3466 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3467 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3468 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3501 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3502 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3503 2024-09-10T07:37:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3534 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3541_fsm 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541_vm 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3542 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3574 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3575 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3576 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3596 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3597 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3598 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3599 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3600_fsm 2024-09-10T07:37:08.551 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600_vm 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3601_fsm 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601_vm 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3602_fsm 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602_vm 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603_fsm 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3603_vm 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3604 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3605 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3606 2024-09-10T07:37:08.552 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3607 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3608 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 3609 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3712 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3764_fsm 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764_vm 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3766 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3767 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3997 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4143 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4144 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4145 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4146 2024-09-10T07:37:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4147 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4148 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4149 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4150 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4151 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4152 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4153 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4154 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4155 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4156 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4157 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4158 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4159 2024-09-10T07:37:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4160 2024-09-10T07:37:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4161 2024-09-10T07:37:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4162 2024-09-10T07:37:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4163 2024-09-10T07:37:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4164 2024-09-10T07:37:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4165 2024-09-10T07:37:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4166 2024-09-10T07:37:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4167 2024-09-10T07:37:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4168 2024-09-10T07:37:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4169 2024-09-10T07:37:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4170 2024-09-10T07:37:08.556 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4171 2024-09-10T07:37:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4172 2024-09-10T07:37:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4173 2024-09-10T07:37:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4174 2024-09-10T07:37:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 5002 2024-09-10T07:37:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 548 2024-09-10T07:37:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 549 2024-09-10T07:37:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6102 2024-09-10T07:37:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6104 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6106 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6110 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6111 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6112 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6113 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6117 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 826 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 827 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 828 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 07:37 pg_filenode.map 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 PG_VERSION 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:total 7777 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 . 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 .. 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 112 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 113 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 1247 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1247_fsm 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1247_vm 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 07:37 1249 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1249_fsm 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1249_vm 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 07:37 1255 2024-09-10T07:37:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1255_fsm 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1255_vm 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 07:37 1259 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1259_fsm 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1259_vm 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 07:37 13344 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13344_fsm 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13344_vm 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13346 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13348 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13349_fsm 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349_vm 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13351 2024-09-10T07:37:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13353 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13354_fsm 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354_vm 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13356 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13358 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13359_fsm 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359_vm 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13361 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13363 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1417 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1418 2024-09-10T07:37:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 174 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 175 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2187 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2224 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2328 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2336 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2337 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2579 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2600 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2600_fsm 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2600_vm 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2601_fsm 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601_vm 2024-09-10T07:37:08.562 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2602 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2602_fsm 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2602_vm 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2603 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2603_fsm 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2603_vm 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2604 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2605 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2605_fsm 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2605_vm 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2606_fsm 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606_vm 2024-09-10T07:37:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2607 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2607_fsm 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2607_vm 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 07:37 2608 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2608_fsm 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2608_vm 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 07:37 2609 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2609_fsm 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2609_vm 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2610 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2610_fsm 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2610_vm 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2611 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612 2024-09-10T07:37:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2612_fsm 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612_vm 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2613 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2615_fsm 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615_vm 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2616 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2616_fsm 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2616_vm 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 07:37 2617 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2617_fsm 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2617_vm 2024-09-10T07:37:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 07:37 2618 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2618_fsm 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2618_vm 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 07:37 2619 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2619_fsm 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2619_vm 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2620 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2650 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2651 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2652 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2653 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2654 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2655 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2656 2024-09-10T07:37:08.566 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2657 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 07:37 2658 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 2659 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2660 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2661 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2662 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2663 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2664 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2665 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2666 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2667 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2668 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2669 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2670 2024-09-10T07:37:08.567 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 07:37 2673 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 07:37 2674 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 07:37 2675 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2678 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2679 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2680 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2681 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2682 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2683 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2684 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2685 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2686 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2687 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2688 2024-09-10T07:37:08.568 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2689 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 07:37 2690 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 07:37 2691 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2692 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2693 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2696 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2699 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2701 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2702 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2703 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2704 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2753 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2753_fsm 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2753_vm 2024-09-10T07:37:08.569 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2754 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2755 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2756 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2757 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2830 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2831 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2832 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2833 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2834 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2835 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2836 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2837 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 07:37 2838 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2838_fsm 2024-09-10T07:37:08.570 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2838_vm 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2839 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840_fsm 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2840_vm 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2841 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2995 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2996 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3079_fsm 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079_vm 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3080 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3081 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3085 2024-09-10T07:37:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3118 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3119 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 07:37 3164 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3256 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3257 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3258 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3350 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3351 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3379 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3380 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3381 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3394 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3394_fsm 2024-09-10T07:37:08.572 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3394_vm 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3395 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3429 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3430 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3431 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3433 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3439 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3440 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3455 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 07:37 3456 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3456_fsm 2024-09-10T07:37:08.573 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3456_vm 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3466 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3467 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3468 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3501 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3502 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3503 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3534 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3541_fsm 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541_vm 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3542 2024-09-10T07:37:08.574 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3574 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3575 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3576 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3596 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3597 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3598 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3599 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3600_fsm 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600_vm 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3601_fsm 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601_vm 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602 2024-09-10T07:37:08.575 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3602_fsm 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602_vm 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603_fsm 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3603_vm 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3604 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3605 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3606 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3607 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3608 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 3609 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3712 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764 2024-09-10T07:37:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3764_fsm 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764_vm 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3766 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3767 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3997 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4143 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4144 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4145 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4146 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4147 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4148 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4149 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4150 2024-09-10T07:37:08.577 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4151 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4152 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4153 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4154 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4155 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4156 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4157 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4158 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4159 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4160 2024-09-10T07:37:08.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4161 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4162 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4163 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4164 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4165 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4166 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4167 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4168 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4169 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4170 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4171 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4172 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4173 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4174 2024-09-10T07:37:08.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 5002 2024-09-10T07:37:08.580 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 548 2024-09-10T07:37:08.582 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 549 2024-09-10T07:37:08.582 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6102 2024-09-10T07:37:08.582 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6104 2024-09-10T07:37:08.582 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6106 2024-09-10T07:37:08.582 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6110 2024-09-10T07:37:08.582 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6111 2024-09-10T07:37:08.582 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6112 2024-09-10T07:37:08.582 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6113 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6117 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 826 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 827 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 828 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 07:37 pg_filenode.map 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 PG_VERSION 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:total 7777 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 . 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 .. 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 112 2024-09-10T07:37:08.583 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 113 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 1247 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1247_fsm 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1247_vm 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 07:37 1249 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1249_fsm 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1249_vm 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 07:37 1255 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1255_fsm 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1255_vm 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 07:37 1259 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1259_fsm 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1259_vm 2024-09-10T07:37:08.584 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 07:37 13344 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13344_fsm 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13344_vm 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13346 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13348 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13349_fsm 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349_vm 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13351 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13353 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13354_fsm 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354_vm 2024-09-10T07:37:08.585 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13356 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13358 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13359_fsm 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359_vm 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13361 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13363 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1417 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1418 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 174 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 175 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2187 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2224 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2328 2024-09-10T07:37:08.586 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2336 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2337 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2579 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2600 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2600_fsm 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2600_vm 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2601_fsm 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601_vm 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2602 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2602_fsm 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2602_vm 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2603 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2603_fsm 2024-09-10T07:37:08.587 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2603_vm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2604 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2605 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2605_fsm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2605_vm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2606_fsm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606_vm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2607 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2607_fsm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2607_vm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 07:37 2608 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2608_fsm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2608_vm 2024-09-10T07:37:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 07:37 2609 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2609_fsm 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2609_vm 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2610 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2610_fsm 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2610_vm 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2611 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2612_fsm 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612_vm 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2613 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615 2024-09-10T07:37:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2615_fsm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615_vm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2616 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2616_fsm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2616_vm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 07:37 2617 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2617_fsm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2617_vm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 07:37 2618 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2618_fsm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2618_vm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 07:37 2619 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2619_fsm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2619_vm 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2620 2024-09-10T07:37:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2650 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2651 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2652 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2653 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2654 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2655 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2656 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2657 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 07:37 2658 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 2659 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2660 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2661 2024-09-10T07:37:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2662 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2663 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2664 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2665 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2666 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2667 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2668 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2669 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2670 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 07:37 2673 2024-09-10T07:37:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 07:37 2674 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 07:37 2675 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2678 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2679 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2680 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2681 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2682 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2683 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2684 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2685 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2686 2024-09-10T07:37:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2687 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2688 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2689 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 07:37 2690 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 07:37 2691 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2692 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2693 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2696 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2699 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2701 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2702 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2703 2024-09-10T07:37:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2704 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2753 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2753_fsm 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2753_vm 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2754 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2755 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2756 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2757 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2830 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2831 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2832 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2833 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2834 2024-09-10T07:37:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2835 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2836 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2837 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 07:37 2838 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2838_fsm 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2838_vm 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2839 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840_fsm 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2840_vm 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2841 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2995 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2996 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079 2024-09-10T07:37:08.596 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3079_fsm 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079_vm 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3080 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3081 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3085 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3118 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3119 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 07:37 3164 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3256 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3257 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3258 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3350 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3351 2024-09-10T07:37:08.597 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3379 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3380 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3381 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3394 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3394_fsm 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3394_vm 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3395 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3429 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3430 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3431 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3433 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3439 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3440 2024-09-10T07:37:08.598 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3455 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 07:37 3456 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3456_fsm 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3456_vm 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3466 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3467 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3468 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3501 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3502 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3503 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3534 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3541_fsm 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541_vm 2024-09-10T07:37:08.599 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3542 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3574 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3575 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3576 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3596 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3597 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3598 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3599 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3600_fsm 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600_vm 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3601_fsm 2024-09-10T07:37:08.600 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601_vm 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3602_fsm 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602_vm 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603_fsm 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3603_vm 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3604 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3605 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3606 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3607 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3608 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 3609 2024-09-10T07:37:08.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3712 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3764_fsm 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764_vm 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3766 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3767 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3997 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4143 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4144 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4145 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4146 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4147 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4148 2024-09-10T07:37:08.602 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4149 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4150 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4151 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4152 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4153 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4154 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4155 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4156 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4157 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4158 2024-09-10T07:37:08.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4159 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4160 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4161 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4162 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4163 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4164 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4165 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4166 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4167 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4168 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4169 2024-09-10T07:37:08.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4170 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4171 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4172 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4173 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4174 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 5002 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 548 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 549 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6102 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6104 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6106 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6110 2024-09-10T07:37:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6111 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6112 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6113 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6117 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 826 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 827 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 828 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 07:37 pg_filenode.map 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 PG_VERSION 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/global: 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:total 537 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 10 07:37 . 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1213 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1213_fsm 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1213_vm 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1214 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1214_fsm 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1214_vm 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 1232 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 1233 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1260 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1260_fsm 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1260_vm 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1261 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1261_fsm 2024-09-10T07:37:08.607 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1261_vm 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1262 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1262_fsm 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1262_vm 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2396 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2396_fsm 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2396_vm 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2397 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2671 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2672 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2676 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2677 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2694 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2695 2024-09-10T07:37:08.608 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2697 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2698 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2846 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2847 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2964 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2965 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2966 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2967 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3592 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3593 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4060 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4061 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4175 2024-09-10T07:37:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4176 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4177 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4178 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4181 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4182 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4183 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4184 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4185 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4186 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6000 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6001 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6002 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6100 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6114 2024-09-10T07:37:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6115 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 pg_control 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 07:37 pg_filenode.map 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/log: 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.611 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:total 1 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 . 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 mappings 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 07:37 replorigin_checkpoint 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 snapshots 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-10T07:37:08.612 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 .. 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 .. 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 . 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 members 2024-09-10T07:37:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 offsets 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:total 8 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 . 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 .. 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 0000 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:total 8 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 . 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 .. 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 0000 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.615 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:total 8 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 . 2024-09-10T07:37:08.616 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 0000 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-10T07:37:08.617 INFO:teuthology.orchestra.run.smithi046.stdout:total 16384 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 . 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:37 000000010000000000000001 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 archive_status 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 .. 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:total 8 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 . 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 07:37 .. 2024-09-10T07:37:08.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 0000 2024-09-10T07:37:08.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-10T07:37:08.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:08 smithi046 ceph-mon[30705]: pgmap v414: 129 pgs: 129 active+clean; 21 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2024-09-10T07:37:08.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-10T07:37:08.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:08 smithi181 ceph-mon[35397]: pgmap v414: 129 pgs: 129 active+clean; 21 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2024-09-10T07:37:08.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:08 smithi063 ceph-mon[35020]: pgmap v414: 129 pgs: 129 active+clean; 21 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2024-09-10T07:37:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:dropping old tables... 2024-09-10T07:37:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-10T07:37:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-10T07:37:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-10T07:37:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-10T07:37:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:creating tables... 2024-09-10T07:37:08.975 INFO:teuthology.orchestra.run.smithi046.stderr:generating data (client-side)... 2024-09-10T07:37:08.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-10T07:37:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.12 s, remaining 3.79 s) 2024-09-10T07:37:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:09.470 INFO:teuthology.orchestra.run.smithi046.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.46 s, remaining 6.95 s) 2024-09-10T07:37:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.76 s, remaining 7.31 s) 2024-09-10T07:37:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.84 s, remaining 5.87 s) 2024-09-10T07:37:10.145 INFO:teuthology.orchestra.run.smithi046.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.14 s, remaining 6.14 s) 2024-09-10T07:37:10.469 INFO:teuthology.orchestra.run.smithi046.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.46 s, remaining 6.34 s) 2024-09-10T07:37:10.557 INFO:teuthology.orchestra.run.smithi046.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.55 s, remaining 5.54 s) 2024-09-10T07:37:10.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:10 smithi046 ceph-mon[30705]: pgmap v415: 129 pgs: 129 active+clean; 28 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.3 MiB/s wr, 68 op/s 2024-09-10T07:37:10.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 -- 172.21.15.46:0/4059413299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141079c0 msgr2=0x7f4f14107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 --2- 172.21.15.46:0/4059413299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141079c0 0x7f4f14107da0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009930 tx=0x7f4f0402a440 comp rx=0 tx=0).stop 2024-09-10T07:37:10.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 -- 172.21.15.46:0/4059413299 shutdown_connections 2024-09-10T07:37:10.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 --2- 172.21.15.46:0/4059413299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f14102360 0x7f4f1410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 --2- 172.21.15.46:0/4059413299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f141019c0 0x7f4f14101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 --2- 172.21.15.46:0/4059413299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141079c0 0x7f4f14107da0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 -- 172.21.15.46:0/4059413299 >> 172.21.15.46:0/4059413299 conn(0x7f4f140fd740 msgr2=0x7f4f140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:10.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 -- 172.21.15.46:0/4059413299 shutdown_connections 2024-09-10T07:37:10.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.660+0000 7f4f13200640 1 -- 172.21.15.46:0/4059413299 wait complete. 2024-09-10T07:37:10.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.662+0000 7f4f13200640 1 Processor -- start 2024-09-10T07:37:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.662+0000 7f4f13200640 1 -- start start 2024-09-10T07:37:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f13200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141019c0 0x7f4f1419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f13200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f14102360 0x7f4f1419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f13200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f141079c0 0x7f4f1419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f13200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f14111e10 con 0x7f4f141019c0 2024-09-10T07:37:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f13200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f14111c90 con 0x7f4f14102360 2024-09-10T07:37:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f13200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4f14111f90 con 0x7f4f141079c0 2024-09-10T07:37:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f14102360 0x7f4f1419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f14102360 0x7f4f1419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59154/0 (socket says 172.21.15.46:59154) 2024-09-10T07:37:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11400640 1 -- 172.21.15.46:0/3107154993 learned_addr learned my addr 172.21.15.46:0/3107154993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f12800640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f141079c0 0x7f4f1419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11e00640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141019c0 0x7f4f1419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11400640 1 -- 172.21.15.46:0/3107154993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f141079c0 msgr2=0x7f4f1419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11400640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f141079c0 0x7f4f1419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11400640 1 -- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141019c0 msgr2=0x7f4f1419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11400640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141019c0 0x7f4f1419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11400640 1 -- 172.21.15.46:0/3107154993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1419f4e0 con 0x7f4f14102360 2024-09-10T07:37:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.663+0000 7f4f11e00640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141019c0 0x7f4f1419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.664+0000 7f4f11400640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f14102360 0x7f4f1419aa70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000e9f0 tx=0x7f4f0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.664+0000 7f4f0aa00640 1 -- 172.21.15.46:0/3107154993 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000cd80 con 0x7f4f14102360 2024-09-10T07:37:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.664+0000 7f4f0aa00640 1 -- 172.21.15.46:0/3107154993 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f00010910 con 0x7f4f14102360 2024-09-10T07:37:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.664+0000 7f4f0aa00640 1 -- 172.21.15.46:0/3107154993 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00018990 con 0x7f4f14102360 2024-09-10T07:37:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.664+0000 7f4f13200640 1 -- 172.21.15.46:0/3107154993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1419f7d0 con 0x7f4f14102360 2024-09-10T07:37:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.664+0000 7f4f13200640 1 -- 172.21.15.46:0/3107154993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f141a70b0 con 0x7f4f14102360 2024-09-10T07:37:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.667+0000 7f4f1bbd3740 1 Processor -- start 2024-09-10T07:37:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.668+0000 7f4f1bbd3740 1 -- start start 2024-09-10T07:37:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.668+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86412d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.668+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 0x55c864138180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.668+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 0x55c86413aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.668+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c8640930b0 con 0x55c86412de00 2024-09-10T07:37:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.668+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c86403aab0 con 0x55c8641386c0 2024-09-10T07:37:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.668+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c86408f070 con 0x55c86412d4e0 2024-09-10T07:37:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.668+0000 7f4f11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 0x55c864138180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.669+0000 7f4f12800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86412d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.669+0000 7f4f12800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86412d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59794/0 (socket says 172.21.15.46:59794) 2024-09-10T07:37:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.669+0000 7f4f12800640 1 -- 172.21.15.46:0/4204183214 learned_addr learned my addr 172.21.15.46:0/4204183214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.669+0000 7f4f12800640 1 -- 172.21.15.46:0/4204183214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 msgr2=0x55c86413aab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:37:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.669+0000 7f4f12800640 1 --2- 172.21.15.46:0/4204183214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 0x55c86413aab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.669+0000 7f4f12800640 1 -- 172.21.15.46:0/4204183214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 msgr2=0x55c864138180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.669+0000 7f4f12800640 1 --2- 172.21.15.46:0/4204183214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 0x55c864138180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.669+0000 7f4f12800640 1 -- 172.21.15.46:0/4204183214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8640474a0 con 0x55c86412d4e0 2024-09-10T07:37:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.670+0000 7f4f0aa00640 1 -- 172.21.15.46:0/3107154993 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4f00010ac0 con 0x7f4f14102360 2024-09-10T07:37:10.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.670+0000 7f4f12800640 1 --2- 172.21.15.46:0/4204183214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86412d8c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009f50 tx=0x7f4f0c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c476da979c3a104e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.670+0000 7f4eff400640 1 -- 172.21.15.46:0/4204183214 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c01e070 con 0x55c86412d4e0 2024-09-10T07:37:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.670+0000 7f4eff400640 1 -- 172.21.15.46:0/4204183214 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c00fa20 con 0x55c86412d4e0 2024-09-10T07:37:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.670+0000 7f4eff400640 1 -- 172.21.15.46:0/4204183214 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c019ba0 con 0x55c86412d4e0 2024-09-10T07:37:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/4204183214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 msgr2=0x55c86412d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/4204183214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86412d8c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009f50 tx=0x7f4f0c0118d0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/4204183214 shutdown_connections 2024-09-10T07:37:10.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/4204183214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 0x55c86413aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/4204183214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 0x55c864138180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/4204183214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86412d8c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/4204183214 >> 172.21.15.46:0/4204183214 conn(0x55c864129a70 msgr2=0x55c864129e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:10.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/4204183214 shutdown_connections 2024-09-10T07:37:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.671+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/4204183214 wait complete. 2024-09-10T07:37:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.672+0000 7f4f0aa00640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ef80a2b90 0x7f4ef80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.672+0000 7f4f1bbd3740 1 -- start start 2024-09-10T07:37:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.672+0000 7f4f0aa00640 1 -- 172.21.15.46:0/3107154993 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4f000cf740 con 0x7f4f14102360 2024-09-10T07:37:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.672+0000 7f4f1bbd3740 1 Processor -- start 2024-09-10T07:37:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412d4e0 0x55c864135670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 0x55c864135bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8641386c0 0x55c8641360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c86413b330 con 0x55c86412de00 2024-09-10T07:37:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c86401b920 con 0x55c86412d4e0 2024-09-10T07:37:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c86413b4b0 con 0x55c8641386c0 2024-09-10T07:37:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f1bbd3740 10 client.0 _ll_register_callbacks cb 0x55c86403f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f11e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412d4e0 0x55c864135670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f11e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412d4e0 0x55c864135670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59158/0 (socket says 172.21.15.46:59158) 2024-09-10T07:37:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f11e00640 1 -- 172.21.15.46:0/1390153136 learned_addr learned my addr 172.21.15.46:0/1390153136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.673+0000 7f4f12800640 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8641386c0 0x55c8641360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.674+0000 7f4f12800640 1 -- 172.21.15.46:0/1390153136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412d4e0 msgr2=0x55c864135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.674+0000 7f4f12800640 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412d4e0 0x55c864135670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.674+0000 7f4f12800640 1 -- 172.21.15.46:0/1390153136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 msgr2=0x55c864135bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.674+0000 7f4f12800640 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 0x55c864135bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.674+0000 7f4f12800640 1 -- 172.21.15.46:0/1390153136 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c863e042b0 con 0x55c8641386c0 2024-09-10T07:37:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.674+0000 7f4f12800640 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8641386c0 0x55c8641360f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c01c040 tx=0x7f4f0c00f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bb3947366e1688c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.675+0000 7f4ef3e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.675+0000 7f4f12800640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ef80a2b90 0x7f4ef80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.675+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1390153136 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c01e040 con 0x55c8641386c0 2024-09-10T07:37:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.675+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1390153136 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0c00fa20 con 0x55c8641386c0 2024-09-10T07:37:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.675+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1390153136 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4f0c004a40 con 0x55c8641386c0 2024-09-10T07:37:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1390153136 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f0c034050 con 0x55c8641386c0 2024-09-10T07:37:10.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4f1bbd3740 10 client.25471 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:10.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1390153136 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c86414f8e0 con 0x55c8641386c0 2024-09-10T07:37:10.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4f12800640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ef80a2b90 0x7f4ef80a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c023000 tx=0x7f4f0c032000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:10.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1390153136 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c00fa20 con 0x55c8641386c0 2024-09-10T07:37:10.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1390153136 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f0c021070 con 0x55c8641386c0 2024-09-10T07:37:10.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4f1bbd3740 10 client.25471 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:10.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4f1bbd3740 10 client.25471 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:37:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.676+0000 7f4f1bbd3740 20 client.25471 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.677+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c864151d40 0x55c8641531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.677+0000 7f4f1bbd3740 4 client.25471 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.677+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1390153136 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8640a0620 con 0x55c864151d40 2024-09-10T07:37:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.677+0000 7f4f11e00640 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c864151d40 0x55c8641531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.678+0000 7f4f11e00640 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c864151d40 0x55c8641531b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ebff2a45cc40f0 server_cookie=22f64ed6258b766 in_seq=0 out_seq=0 2024-09-10T07:37:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.678+0000 7f4ef3e00640 10 client.25471 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.678+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1390153136 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8640a0620 con 0x55c864151d40 2024-09-10T07:37:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.678+0000 7f4ef3e00640 10 client.25471 handle_command_reply: tid=0 2024-09-10T07:37:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.678+0000 7f4f1bbd3740 1 client.25471 shutdown 2024-09-10T07:37:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1390153136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8641386c0 msgr2=0x55c8641360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8641386c0 0x55c8641360f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c01c040 tx=0x7f4f0c00f7f0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1390153136 shutdown_connections 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c864151d40 0x55c8641531b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8641386c0 0x55c8641360f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412de00 0x55c864135bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1390153136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412d4e0 0x55c864135670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4ef3e00640 0 client.25471 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1390153136 >> 172.21.15.46:0/1390153136 conn(0x55c864129a70 msgr2=0x55c86412f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1390153136 shutdown_connections 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1390153136 wait complete. 2024-09-10T07:37:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.679+0000 7f4f1bbd3740 20 client.25471 trim_cache size 0 max 16384 2024-09-10T07:37:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.715+0000 7f4f1bbd3740 1 Processor -- start 2024-09-10T07:37:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.715+0000 7f4f1bbd3740 1 -- start start 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412d4e0 0x55c8641531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412de00 0x55c864135f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 0x55c8641328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c8641514f0 con 0x55c86412d4e0 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c864151bc0 con 0x55c8641386c0 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c864152290 con 0x55c86412de00 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412d4e0 0x55c8641531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412d4e0 0x55c8641531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54746/0 (socket says 172.21.15.46:54746) 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f11e00640 1 -- 172.21.15.46:0/3065034582 learned_addr learned my addr 172.21.15.46:0/3065034582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f11400640 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412de00 0x55c864135f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.716+0000 7f4f12800640 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 0x55c8641328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4f11400640 1 -- 172.21.15.46:0/3065034582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 msgr2=0x55c8641328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4f11400640 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 0x55c8641328d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4f11400640 1 -- 172.21.15.46:0/3065034582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412d4e0 msgr2=0x55c8641531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4f11400640 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412d4e0 0x55c8641531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4f11400640 1 -- 172.21.15.46:0/3065034582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86412d310 con 0x55c86412de00 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4f11e00640 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412d4e0 0x55c8641531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4f12800640 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 0x55c8641328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4f11400640 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412de00 0x55c864135f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55c86412adf0 tx=0x7f4f00097f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcf446f71ef0ffa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4efea00640 1 -- 172.21.15.46:0/3065034582 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0009bbc0 con 0x55c86412de00 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4efea00640 1 -- 172.21.15.46:0/3065034582 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f000b2020 con 0x55c86412de00 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.717+0000 7f4efea00640 1 -- 172.21.15.46:0/3065034582 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00018b30 con 0x55c86412de00 2024-09-10T07:37:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/3065034582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412de00 msgr2=0x55c864135f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412de00 0x55c864135f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55c86412adf0 tx=0x7f4f00097f70 comp rx=0 tx=0).stop 2024-09-10T07:37:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/3065034582 shutdown_connections 2024-09-10T07:37:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8641386c0 0x55c8641328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412de00 0x55c864135f40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/3065034582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c86412d4e0 0x55c8641531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/3065034582 >> 172.21.15.46:0/3065034582 conn(0x55c864129a70 msgr2=0x55c864133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/3065034582 shutdown_connections 2024-09-10T07:37:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.718+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/3065034582 wait complete. 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.719+0000 7f4f1bbd3740 1 -- start start 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.719+0000 7f4f1bbd3740 1 Processor -- start 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.719+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.719+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412de00 0x55c864154e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.719+0000 7f4f1bbd3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8641386c0 0x55c8641553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.719+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c864153030 con 0x55c8641386c0 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.719+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c864152960 con 0x55c86412de00 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.719+0000 7f4f1bbd3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c864153700 con 0x55c86412d4e0 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f1bbd3740 10 client.0 _ll_register_callbacks cb 0x7f4f0c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412de00 0x55c864154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412de00 0x55c864154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59174/0 (socket says 172.21.15.46:59174) 2024-09-10T07:37:10.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 -- 172.21.15.46:0/1201238899 learned_addr learned my addr 172.21.15.46:0/1201238899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11e00640 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f12800640 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8641386c0 0x55c8641553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 -- 172.21.15.46:0/1201238899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 msgr2=0x55c86415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 -- 172.21.15.46:0/1201238899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8641386c0 msgr2=0x55c8641553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8641386c0 0x55c8641553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 -- 172.21.15.46:0/1201238899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c864155c20 con 0x55c86412de00 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4f11400640 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412de00 0x55c864154e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f000cce60 tx=0x7f4f000cf440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09be2f8d237c953 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.720+0000 7f4ef3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.721+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1201238899 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f000b8030 con 0x55c86412de00 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.721+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1201238899 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4f000b3070 con 0x55c86412de00 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.721+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1201238899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f000b2020 con 0x55c86412de00 2024-09-10T07:37:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.721+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1201238899 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f000a4830 con 0x55c86412de00 2024-09-10T07:37:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.721+0000 7f4f1bbd3740 10 client.25472 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.721+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1201238899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8641bc510 con 0x55c86412de00 2024-09-10T07:37:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.721+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1201238899 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f000b2020 con 0x55c86412de00 2024-09-10T07:37:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.721+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1201238899 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f0009b010 con 0x55c86412de00 2024-09-10T07:37:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.722+0000 7f4f1bbd3740 10 client.25472 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.722+0000 7f4f1bbd3740 10 client.25472 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:37:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.722+0000 7f4f1bbd3740 20 client.25472 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.722+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c8641c0b20 0x55c8641bf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.722+0000 7f4f1bbd3740 4 client.25472 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:37:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.722+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1201238899 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "status"}) -- 0x55c86409ab30 con 0x55c8641c0b20 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.722+0000 7f4f11e00640 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c8641c0b20 0x55c8641bf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f11e00640 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c8641c0b20 0x55c8641bf830 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12d80d00016bdf server_cookie=29694db7db364cf7 in_seq=0 out_seq=0 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4ef3e00640 10 client.25472 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4ef3e00640 1 -- 172.21.15.46:0/1201238899 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c86409ab30 con 0x55c8641c0b20 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4ef3e00640 10 client.25472 handle_command_reply: tid=0 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 client.25472 shutdown 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1201238899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412de00 msgr2=0x55c864154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412de00 0x55c864154e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f000cce60 tx=0x7f4f000cf440 comp rx=0 tx=0).stop 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1201238899 shutdown_connections 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c8641c0b20 0x55c8641bf830 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8641386c0 0x55c8641553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c86412de00 0x55c864154e90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 --2- 172.21.15.46:0/1201238899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c86412d4e0 0x55c86415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4ef3e00640 0 client.25472 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1201238899 >> 172.21.15.46:0/1201238899 conn(0x55c864129a70 msgr2=0x55c864136ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.723+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1201238899 shutdown_connections 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.724+0000 7f4f1bbd3740 1 -- 172.21.15.46:0/1201238899 wait complete. 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.724+0000 7f4f1bbd3740 20 client.25472 trim_cache size 0 max 16384 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "b120bdf2-6f45-11ef-bcea-c7b262605968", 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24535, 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-10T07:37:10.726 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-10T07:37:10.727 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-10T07:37:10.727 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 213.850178132, 2024-09-10T07:37:10.727 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 30, 2024-09-10T07:37:10.727 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 97, 2024-09-10T07:37:10.727 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 97, 2024-09-10T07:37:10.727 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 230.243552185 2024-09-10T07:37:10.727 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.724+0000 7f4f13200640 1 -- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ef80a2b90 msgr2=0x7f4ef80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.724+0000 7f4f13200640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ef80a2b90 0x7f4ef80a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c023000 tx=0x7f4f0c032000 comp rx=0 tx=0).stop 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.724+0000 7f4f13200640 1 -- 172.21.15.46:0/3107154993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f14102360 msgr2=0x7f4f1419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.724+0000 7f4f13200640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f14102360 0x7f4f1419aa70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000e9f0 tx=0x7f4f0000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.725+0000 7f4f13200640 1 -- 172.21.15.46:0/3107154993 shutdown_connections 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.725+0000 7f4f13200640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f141079c0 0x7f4f1419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.725+0000 7f4f13200640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f14102360 0x7f4f1419aa70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.725+0000 7f4f13200640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ef80a2b90 0x7f4ef80a5050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.725+0000 7f4f13200640 1 --2- 172.21.15.46:0/3107154993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f141019c0 0x7f4f1419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.725+0000 7f4f13200640 1 -- 172.21.15.46:0/3107154993 >> 172.21.15.46:0/3107154993 conn(0x7f4f140fd740 msgr2=0x7f4f1410d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.725+0000 7f4f13200640 1 -- 172.21.15.46:0/3107154993 shutdown_connections 2024-09-10T07:37:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:10.725+0000 7f4f13200640 1 -- 172.21.15.46:0/3107154993 wait complete. 2024-09-10T07:37:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.87 s, remaining 5.62 s) 2024-09-10T07:37:10.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:10 smithi181 ceph-mon[35397]: pgmap v415: 129 pgs: 129 active+clean; 28 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.3 MiB/s wr, 68 op/s 2024-09-10T07:37:10.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:10 smithi063 ceph-mon[35020]: pgmap v415: 129 pgs: 129 active+clean; 28 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.3 MiB/s wr, 68 op/s 2024-09-10T07:37:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.02 s, remaining 5.17 s) 2024-09-10T07:37:11.329 INFO:teuthology.orchestra.run.smithi046.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.32 s, remaining 5.11 s) 2024-09-10T07:37:11.519 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T07:37:11.672 INFO:teuthology.orchestra.run.smithi046.stderr:1100000 of 3200000 tuples (34%) done (elapsed 2.66 s, remaining 5.09 s) 2024-09-10T07:37:11.847 INFO:teuthology.orchestra.run.smithi046.stderr:1200000 of 3200000 tuples (37%) done (elapsed 2.84 s, remaining 4.73 s) 2024-09-10T07:37:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.08 s, remaining 4.49 s) 2024-09-10T07:37:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.41 s, remaining 4.38 s) 2024-09-10T07:37:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:1500000 of 3200000 tuples (46%) done (elapsed 3.61 s, remaining 4.09 s) 2024-09-10T07:37:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:1600000 of 3200000 tuples (50%) done (elapsed 3.88 s, remaining 3.88 s) 2024-09-10T07:37:12.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:12 smithi181 ceph-mon[35397]: pgmap v416: 129 pgs: 129 active+clean; 38 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-09-10T07:37:12.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:12 smithi063 ceph-mon[35020]: pgmap v416: 129 pgs: 129 active+clean; 38 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-09-10T07:37:12.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:12 smithi046 ceph-mon[30705]: pgmap v416: 129 pgs: 129 active+clean; 38 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-09-10T07:37:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.16 s, remaining 3.67 s) 2024-09-10T07:37:13.266 INFO:teuthology.orchestra.run.smithi046.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.26 s, remaining 3.31 s) 2024-09-10T07:37:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:1900000 of 3200000 tuples (59%) done (elapsed 4.67 s, remaining 3.20 s) 2024-09-10T07:37:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2000000 of 3200000 tuples (62%) done (elapsed 4.96 s, remaining 2.97 s) 2024-09-10T07:37:14.053 INFO:teuthology.orchestra.run.smithi046.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.05 s, remaining 2.64 s) 2024-09-10T07:37:14.437 INFO:teuthology.orchestra.run.smithi046.stderr:2200000 of 3200000 tuples (68%) done (elapsed 5.43 s, remaining 2.47 s) 2024-09-10T07:37:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2300000 of 3200000 tuples (71%) done (elapsed 5.52 s, remaining 2.16 s) 2024-09-10T07:37:14.816 INFO:teuthology.orchestra.run.smithi046.stderr:2400000 of 3200000 tuples (75%) done (elapsed 5.81 s, remaining 1.94 s) 2024-09-10T07:37:14.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:14 smithi181 ceph-mon[35397]: pgmap v417: 129 pgs: 129 active+clean; 76 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 9.9 MiB/s wr, 350 op/s 2024-09-10T07:37:14.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:14 smithi063 ceph-mon[35020]: pgmap v417: 129 pgs: 129 active+clean; 76 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 9.9 MiB/s wr, 350 op/s 2024-09-10T07:37:14.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:14 smithi046 ceph-mon[30705]: pgmap v417: 129 pgs: 129 active+clean; 76 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 9.9 MiB/s wr, 350 op/s 2024-09-10T07:37:15.092 INFO:teuthology.orchestra.run.smithi046.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.09 s, remaining 1.70 s) 2024-09-10T07:37:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2600000 of 3200000 tuples (81%) done (elapsed 6.23 s, remaining 1.44 s) 2024-09-10T07:37:15.597 INFO:teuthology.orchestra.run.smithi046.stderr:2700000 of 3200000 tuples (84%) done (elapsed 6.59 s, remaining 1.22 s) 2024-09-10T07:37:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2800000 of 3200000 tuples (87%) done (elapsed 6.84 s, remaining 0.98 s) 2024-09-10T07:37:15.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:37:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2900000 of 3200000 tuples (90%) done (elapsed 6.93 s, remaining 0.72 s) 2024-09-10T07:37:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:16.390 INFO:teuthology.orchestra.run.smithi046.stderr:3000000 of 3200000 tuples (93%) done (elapsed 7.38 s, remaining 0.49 s) 2024-09-10T07:37:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:3100000 of 3200000 tuples (96%) done (elapsed 7.69 s, remaining 0.25 s) 2024-09-10T07:37:16.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:16 smithi046 ceph-mon[30705]: pgmap v418: 129 pgs: 129 active+clean; 160 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 24 MiB/s wr, 361 op/s 2024-09-10T07:37:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:3200000 of 3200000 tuples (100%) done (elapsed 7.79 s, remaining 0.00 s) 2024-09-10T07:37:16.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:16 smithi181 ceph-mon[35397]: pgmap v418: 129 pgs: 129 active+clean; 160 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 24 MiB/s wr, 361 op/s 2024-09-10T07:37:16.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:16 smithi063 ceph-mon[35020]: pgmap v418: 129 pgs: 129 active+clean; 160 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 24 MiB/s wr, 361 op/s 2024-09-10T07:37:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:vacuuming... 2024-09-10T07:37:17.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 -- 172.21.15.46:0/1108651096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda540745c0 msgr2=0x7fda540749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 --2- 172.21.15.46:0/1108651096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda540745c0 0x7fda540749a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda44009900 tx=0x7fda4402a480 comp rx=0 tx=0).stop 2024-09-10T07:37:17.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 -- 172.21.15.46:0/1108651096 shutdown_connections 2024-09-10T07:37:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 --2- 172.21.15.46:0/1108651096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda5410e420 0x7fda54110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 --2- 172.21.15.46:0/1108651096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda54074ee0 0x7fda5410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 --2- 172.21.15.46:0/1108651096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda540745c0 0x7fda540749a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 -- 172.21.15.46:0/1108651096 >> 172.21.15.46:0/1108651096 conn(0x7fda540fd8c0 msgr2=0x7fda540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 -- 172.21.15.46:0/1108651096 shutdown_connections 2024-09-10T07:37:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.048+0000 7fda59c00640 1 -- 172.21.15.46:0/1108651096 wait complete. 2024-09-10T07:37:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.049+0000 7fda59c00640 1 Processor -- start 2024-09-10T07:37:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda59c00640 1 -- start start 2024-09-10T07:37:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda59c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda540745c0 0x7fda5419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda59c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda54074ee0 0x7fda5419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda59c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5410e420 0x7fda5419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda59c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda541141c0 con 0x7fda54074ee0 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda59c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda54114040 con 0x7fda540745c0 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda59c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda54114340 con 0x7fda5410e420 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda52a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda540745c0 0x7fda5419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda52a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda540745c0 0x7fda5419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33852/0 (socket says 172.21.15.46:33852) 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda52a00640 1 -- 172.21.15.46:0/35395274 learned_addr learned my addr 172.21.15.46:0/35395274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda52000640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda54074ee0 0x7fda5419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.050+0000 7fda53400640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5410e420 0x7fda5419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda52000640 1 -- 172.21.15.46:0/35395274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5410e420 msgr2=0x7fda5419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda52000640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5410e420 0x7fda5419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda52000640 1 -- 172.21.15.46:0/35395274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda540745c0 msgr2=0x7fda5419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda52000640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda540745c0 0x7fda5419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda52000640 1 -- 172.21.15.46:0/35395274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5419f650 con 0x7fda54074ee0 2024-09-10T07:37:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda53400640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5410e420 0x7fda5419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda52000640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda54074ee0 0x7fda5419abe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9f0 tx=0x7fda4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda4b400640 1 -- 172.21.15.46:0/35395274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4000ccc0 con 0x7fda54074ee0 2024-09-10T07:37:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.051+0000 7fda59c00640 1 -- 172.21.15.46:0/35395274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5419f940 con 0x7fda54074ee0 2024-09-10T07:37:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.052+0000 7fda4b400640 1 -- 172.21.15.46:0/35395274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda40010910 con 0x7fda54074ee0 2024-09-10T07:37:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.052+0000 7fda59c00640 1 -- 172.21.15.46:0/35395274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda541a7220 con 0x7fda54074ee0 2024-09-10T07:37:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.052+0000 7fda4b400640 1 -- 172.21.15.46:0/35395274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40018900 con 0x7fda54074ee0 2024-09-10T07:37:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.056+0000 7fda4b400640 1 -- 172.21.15.46:0/35395274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda40010ac0 con 0x7fda54074ee0 2024-09-10T07:37:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.056+0000 7fda5c87b740 1 Processor -- start 2024-09-10T07:37:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda5c87b740 1 -- start start 2024-09-10T07:37:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda4b400640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda380a2b80 0x7fda380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4ed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e4f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda5c87b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56107e4493e0 con 0x56107e4e2b90 2024-09-10T07:37:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda5c87b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56107e3f0d80 con 0x56107e4e34b0 2024-09-10T07:37:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda5c87b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56107e4453a0 con 0x56107e4edd70 2024-09-10T07:37:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda4b400640 1 -- 172.21.15.46:0/35395274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fda400cf730 con 0x7fda54074ee0 2024-09-10T07:37:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.057+0000 7fda52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4ed830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33860/0 (socket says 172.21.15.46:33860) 2024-09-10T07:37:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda52000640 1 -- 172.21.15.46:0/3005842787 learned_addr learned my addr 172.21.15.46:0/3005842787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda53400640 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda52a00640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda380a2b80 0x7fda380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda53400640 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e4f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda52000640 1 -- 172.21.15.46:0/3005842787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 msgr2=0x56107e4f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda52000640 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e4f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda52000640 1 -- 172.21.15.46:0/3005842787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 msgr2=0x56107e4e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda52000640 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda52000640 1 -- 172.21.15.46:0/3005842787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107e40fae0 con 0x56107e4e34b0 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.058+0000 7fda53400640 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.059+0000 7fda52000640 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4ed830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda40098630 tx=0x7fda4009bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720fb0ea8012e087 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.059+0000 7fda3fe00640 1 -- 172.21.15.46:0/3005842787 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40099d00 con 0x56107e4e34b0 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.059+0000 7fda52a00640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda380a2b80 0x7fda380a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fda440096c0 tx=0x7fda440046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.059+0000 7fda3fe00640 1 -- 172.21.15.46:0/3005842787 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda40099f60 con 0x56107e4e34b0 2024-09-10T07:37:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.059+0000 7fda3fe00640 1 -- 172.21.15.46:0/3005842787 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400c4de0 con 0x56107e4e34b0 2024-09-10T07:37:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 -- 172.21.15.46:0/3005842787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 msgr2=0x56107e4ed830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4ed830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda40098630 tx=0x7fda4009bc60 comp rx=0 tx=0).stop 2024-09-10T07:37:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 -- 172.21.15.46:0/3005842787 shutdown_connections 2024-09-10T07:37:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e4f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4ed830 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 --2- 172.21.15.46:0/3005842787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 -- 172.21.15.46:0/3005842787 >> 172.21.15.46:0/3005842787 conn(0x56107e4df120 msgr2=0x56107e4df500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 -- 172.21.15.46:0/3005842787 shutdown_connections 2024-09-10T07:37:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.060+0000 7fda5c87b740 1 -- 172.21.15.46:0/3005842787 wait complete. 2024-09-10T07:37:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 1 -- start start 2024-09-10T07:37:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 1 Processor -- start 2024-09-10T07:37:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e2b90 0x56107e4eace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e34b0 0x56107e4eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e4eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56107e4f0980 con 0x56107e4e34b0 2024-09-10T07:37:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56107e4040f0 con 0x56107e4e2b90 2024-09-10T07:37:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56107e4f0b00 con 0x56107e4edd70 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.061+0000 7fda5c87b740 10 client.0 _ll_register_callbacks cb 0x56107e3f59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e2b90 0x56107e4eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e34b0 0x56107e4eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e34b0 0x56107e4eb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53386/0 (socket says 172.21.15.46:53386) 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e2b90 0x56107e4eace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33864/0 (socket says 172.21.15.46:33864) 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52000640 1 -- 172.21.15.46:0/487678701 learned_addr learned my addr 172.21.15.46:0/487678701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda53400640 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e4eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52000640 1 -- 172.21.15.46:0/487678701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 msgr2=0x56107e4eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52000640 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e4eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52000640 1 -- 172.21.15.46:0/487678701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e2b90 msgr2=0x56107e4eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52000640 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e2b90 0x56107e4eace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.062+0000 7fda52000640 1 -- 172.21.15.46:0/487678701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107e1ba3c0 con 0x56107e4e34b0 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.063+0000 7fda52000640 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e34b0 0x56107e4eb220 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fda4009bbf0 tx=0x7fda40098520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aeca9b223412b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.063+0000 7fda3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:37:17.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.063+0000 7fda3cc00640 1 -- 172.21.15.46:0/487678701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4009b2b0 con 0x56107e4e34b0 2024-09-10T07:37:17.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.063+0000 7fda5c87b740 1 -- 172.21.15.46:0/487678701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda40099f60 con 0x56107e4e34b0 2024-09-10T07:37:17.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.063+0000 7fda3cc00640 1 -- 172.21.15.46:0/487678701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fda40010430 con 0x56107e4e34b0 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.063+0000 7fda3cc00640 1 -- 172.21.15.46:0/487678701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda400982d0 con 0x56107e4e34b0 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.063+0000 7fda5c87b740 10 client.15438 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.063+0000 7fda5c87b740 1 -- 172.21.15.46:0/487678701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56107e4fc6b0 con 0x56107e4e34b0 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.064+0000 7fda3cc00640 1 -- 172.21.15.46:0/487678701 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda400b3070 con 0x56107e4e34b0 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.064+0000 7fda3cc00640 1 -- 172.21.15.46:0/487678701 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fda4009ab20 con 0x56107e4e34b0 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.064+0000 7fda5c87b740 10 client.15438 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.064+0000 7fda5c87b740 10 client.15438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.065+0000 7fda5c87b740 20 client.15438 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.065+0000 7fda5c87b740 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56107e4fda00 0x56107e4ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.065+0000 7fda5c87b740 4 client.15438 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:17.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.065+0000 7fda5c87b740 1 -- 172.21.15.46:0/487678701 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fda40099f60 con 0x56107e4fda00 2024-09-10T07:37:17.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.065+0000 7fda52a00640 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56107e4fda00 0x56107e4ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.066+0000 7fda52a00640 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56107e4fda00 0x56107e4ffdf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8d3d298722569f server_cookie=a904948fb35b1a8a in_seq=0 out_seq=0 2024-09-10T07:37:17.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.066+0000 7fda3cc00640 10 client.15438 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.066+0000 7fda3cc00640 1 -- 172.21.15.46:0/487678701 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fda40099f60 con 0x56107e4fda00 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.066+0000 7fda3cc00640 10 client.15438 handle_command_reply: tid=0 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.066+0000 7fda5c87b740 1 client.15438 shutdown 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 -- 172.21.15.46:0/487678701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e34b0 msgr2=0x56107e4eb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e34b0 0x56107e4eb220 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fda4009bbf0 tx=0x7fda40098520 comp rx=0 tx=0).stop 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 -- 172.21.15.46:0/487678701 shutdown_connections 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56107e4fda00 0x56107e4ffdf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e4eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e34b0 0x56107e4eb220 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda3cc00640 0 client.15438 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 --2- 172.21.15.46:0/487678701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e2b90 0x56107e4eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 -- 172.21.15.46:0/487678701 >> 172.21.15.46:0/487678701 conn(0x56107e4df120 msgr2=0x56107e4e4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:17.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 -- 172.21.15.46:0/487678701 shutdown_connections 2024-09-10T07:37:17.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.067+0000 7fda5c87b740 1 -- 172.21.15.46:0/487678701 wait complete. 2024-09-10T07:37:17.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.068+0000 7fda5c87b740 20 client.15438 trim_cache size 0 max 16384 2024-09-10T07:37:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.105+0000 7fda5c87b740 1 Processor -- start 2024-09-10T07:37:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.105+0000 7fda5c87b740 1 -- start start 2024-09-10T07:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.105+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.105+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e505390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.105+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e5058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.105+0000 7fda5c87b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda541b0f20 con 0x56107e4e2b90 2024-09-10T07:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.105+0000 7fda5c87b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda541b2a60 con 0x56107e4e34b0 2024-09-10T07:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.105+0000 7fda5c87b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda541b15f0 con 0x56107e4edd70 2024-09-10T07:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e5058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda52a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e505390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e5058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40544/0 (socket says 172.21.15.46:40544) 2024-09-10T07:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 -- 172.21.15.46:0/3142044198 learned_addr learned my addr 172.21.15.46:0/3142044198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 -- 172.21.15.46:0/3142044198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 msgr2=0x56107e505390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 --2- 172.21.15.46:0/3142044198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e505390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 -- 172.21.15.46:0/3142044198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 msgr2=0x56107e4e55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 --2- 172.21.15.46:0/3142044198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 -- 172.21.15.46:0/3142044198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107e4def50 con 0x56107e4edd70 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda52000640 1 --2- 172.21.15.46:0/3142044198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e505390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.106+0000 7fda53400640 1 --2- 172.21.15.46:0/3142044198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e5058d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fda4c012aa0 tx=0x7fda4c009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d6dba8ee86c61b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.107+0000 7fda3f400640 1 -- 172.21.15.46:0/3142044198 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c004b20 con 0x56107e4edd70 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.107+0000 7fda3f400640 1 -- 172.21.15.46:0/3142044198 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c004cc0 con 0x56107e4edd70 2024-09-10T07:37:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.107+0000 7fda3f400640 1 -- 172.21.15.46:0/3142044198 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0058f0 con 0x56107e4edd70 2024-09-10T07:37:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 -- 172.21.15.46:0/3142044198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 msgr2=0x56107e5058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 --2- 172.21.15.46:0/3142044198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e5058d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fda4c012aa0 tx=0x7fda4c009e60 comp rx=0 tx=0).stop 2024-09-10T07:37:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 -- 172.21.15.46:0/3142044198 shutdown_connections 2024-09-10T07:37:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 --2- 172.21.15.46:0/3142044198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4edd70 0x56107e5058d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 --2- 172.21.15.46:0/3142044198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e505390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 --2- 172.21.15.46:0/3142044198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4e2b90 0x56107e4e55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 -- 172.21.15.46:0/3142044198 >> 172.21.15.46:0/3142044198 conn(0x56107e4df120 msgr2=0x56107e4ecd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 -- 172.21.15.46:0/3142044198 shutdown_connections 2024-09-10T07:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.108+0000 7fda5c87b740 1 -- 172.21.15.46:0/3142044198 wait complete. 2024-09-10T07:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 1 -- start start 2024-09-10T07:37:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 1 Processor -- start 2024-09-10T07:37:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4e2b90 0x56107e4f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4edd70 0x56107e4f2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda541b2390 con 0x56107e4edd70 2024-09-10T07:37:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda541b1cc0 con 0x56107e4e34b0 2024-09-10T07:37:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda541aef00 con 0x56107e4e2b90 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.109+0000 7fda5c87b740 10 client.0 _ll_register_callbacks cb 0x7fda4009bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4e2b90 0x56107e4f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4e2b90 0x56107e4f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40558/0 (socket says 172.21.15.46:40558) 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 -- 172.21.15.46:0/529880731 learned_addr learned my addr 172.21.15.46:0/529880731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda53400640 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4edd70 0x56107e4f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 -- 172.21.15.46:0/529880731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 msgr2=0x56107e4f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 -- 172.21.15.46:0/529880731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4edd70 msgr2=0x56107e4f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4edd70 0x56107e4f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 -- 172.21.15.46:0/529880731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107e567360 con 0x56107e4e2b90 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52000640 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda52a00640 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4e2b90 0x56107e4f19d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda44005ec0 tx=0x7fda4402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8fa6a8e7cec7336 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.110+0000 7fda3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.111+0000 7fda3cc00640 1 -- 172.21.15.46:0/529880731 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44035040 con 0x56107e4e2b90 2024-09-10T07:37:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.111+0000 7fda3cc00640 1 -- 172.21.15.46:0/529880731 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fda44031cb0 con 0x56107e4e2b90 2024-09-10T07:37:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.111+0000 7fda5c87b740 1 -- 172.21.15.46:0/529880731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda4c004cc0 con 0x56107e4e2b90 2024-09-10T07:37:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.111+0000 7fda3cc00640 1 -- 172.21.15.46:0/529880731 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda440035f0 con 0x56107e4e2b90 2024-09-10T07:37:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.111+0000 7fda5c87b740 10 client.25495 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.111+0000 7fda5c87b740 1 -- 172.21.15.46:0/529880731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56107e5685d0 con 0x56107e4e2b90 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda3cc00640 1 -- 172.21.15.46:0/529880731 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4405f030 con 0x56107e4e2b90 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda3cc00640 1 -- 172.21.15.46:0/529880731 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fda4405a840 con 0x56107e4e2b90 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda5c87b740 10 client.25495 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda5c87b740 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda5c87b740 20 client.25495 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda5c87b740 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56107e568cd0 0x56107e56b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda5c87b740 4 client.25495 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda5c87b740 1 -- 172.21.15.46:0/529880731 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56107e456950 con 0x56107e568cd0 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.112+0000 7fda52000640 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56107e568cd0 0x56107e56b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda52000640 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56107e568cd0 0x56107e56b0f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29111b35a01c356 server_cookie=dddb8339759dcf2e in_seq=0 out_seq=0 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda3cc00640 10 client.25495 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda3cc00640 1 -- 172.21.15.46:0/529880731 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56107e456950 con 0x56107e568cd0 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda3cc00640 10 client.25495 handle_command_reply: tid=0 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda5c87b740 1 client.25495 shutdown 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda5c87b740 1 -- 172.21.15.46:0/529880731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4e2b90 msgr2=0x56107e4f19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda5c87b740 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4e2b90 0x56107e4f19d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda44005ec0 tx=0x7fda4402a480 comp rx=0 tx=0).stop 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda5c87b740 1 -- 172.21.15.46:0/529880731 shutdown_connections 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.113+0000 7fda5c87b740 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56107e568cd0 0x56107e56b0f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.114+0000 7fda5c87b740 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56107e4edd70 0x56107e4f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.114+0000 7fda3cc00640 0 client.25495 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.114+0000 7fda5c87b740 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56107e4e34b0 0x56107e4f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.114+0000 7fda5c87b740 1 --2- 172.21.15.46:0/529880731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56107e4e2b90 0x56107e4f19d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.114+0000 7fda5c87b740 1 -- 172.21.15.46:0/529880731 >> 172.21.15.46:0/529880731 conn(0x56107e4df120 msgr2=0x56107e4de1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.114+0000 7fda5c87b740 1 -- 172.21.15.46:0/529880731 shutdown_connections 2024-09-10T07:37:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.114+0000 7fda5c87b740 1 -- 172.21.15.46:0/529880731 wait complete. 2024-09-10T07:37:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.114+0000 7fda5c87b740 20 client.25495 trim_cache size 0 max 16384 2024-09-10T07:37:17.116 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:37:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:37:17.116 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 -- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda380a2b80 msgr2=0x7fda380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda380a2b80 0x7fda380a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fda440096c0 tx=0x7fda440046c0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 -- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda54074ee0 msgr2=0x7fda5419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda54074ee0 0x7fda5419abe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9f0 tx=0x7fda4000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 -- 172.21.15.46:0/35395274 shutdown_connections 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5410e420 0x7fda5419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda380a2b80 0x7fda380a5040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda54074ee0 0x7fda5419abe0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 --2- 172.21.15.46:0/35395274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda540745c0 0x7fda5419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 -- 172.21.15.46:0/35395274 >> 172.21.15.46:0/35395274 conn(0x7fda540fd8c0 msgr2=0x7fda540fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 -- 172.21.15.46:0/35395274 shutdown_connections 2024-09-10T07:37:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:17.115+0000 7fda59c00640 1 -- 172.21.15.46:0/35395274 wait complete. 2024-09-10T07:37:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:creating primary keys... 2024-09-10T07:37:17.998 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:37:17.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:37:18.442 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:18.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:18 smithi046 ceph-mon[30705]: pgmap v419: 129 pgs: 129 active+clean; 288 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 44 MiB/s wr, 365 op/s 2024-09-10T07:37:18.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:18 smithi181 ceph-mon[35397]: pgmap v419: 129 pgs: 129 active+clean; 288 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 44 MiB/s wr, 365 op/s 2024-09-10T07:37:18.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:18 smithi063 ceph-mon[35020]: pgmap v419: 129 pgs: 129 active+clean; 288 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 44 MiB/s wr, 365 op/s 2024-09-10T07:37:19.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.077+0000 7f84f7800640 1 -- 172.21.15.46:0/2226498625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0077040 msgr2=0x7f84f01124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:19.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.077+0000 7f84f7800640 1 --2- 172.21.15.46:0/2226498625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0077040 0x7f84f01124f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009860 tx=0x7f84dc02a340 comp rx=0 tx=0).stop 2024-09-10T07:37:19.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.078+0000 7f84f7800640 1 -- 172.21.15.46:0/2226498625 shutdown_connections 2024-09-10T07:37:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.078+0000 7f84f7800640 1 --2- 172.21.15.46:0/2226498625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84f0112a30 0x7f84f0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.078+0000 7f84f7800640 1 --2- 172.21.15.46:0/2226498625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0077040 0x7f84f01124f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.078+0000 7f84f7800640 1 --2- 172.21.15.46:0/2226498625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84f0076720 0x7f84f0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.078+0000 7f84f7800640 1 -- 172.21.15.46:0/2226498625 >> 172.21.15.46:0/2226498625 conn(0x7f84f00ffa90 msgr2=0x7f84f0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.078+0000 7f84f7800640 1 -- 172.21.15.46:0/2226498625 shutdown_connections 2024-09-10T07:37:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.078+0000 7f84f7800640 1 -- 172.21.15.46:0/2226498625 wait complete. 2024-09-10T07:37:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.079+0000 7f84f7800640 1 Processor -- start 2024-09-10T07:37:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.079+0000 7f84f7800640 1 -- start start 2024-09-10T07:37:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f7800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0076720 0x7f84f01a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f7800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84f0077040 0x7f84f01a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84f0112a30 0x7f84f019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01185a0 con 0x7f84f0112a30 2024-09-10T07:37:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f7800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0118420 con 0x7f84f0077040 2024-09-10T07:37:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f7800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0118720 con 0x7f84f0076720 2024-09-10T07:37:19.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0076720 0x7f84f01a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:19.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84f0112a30 0x7f84f019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:19.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0076720 0x7f84f01a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40582/0 (socket says 172.21.15.46:40582) 2024-09-10T07:37:19.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.080+0000 7f84f4e00640 1 -- 172.21.15.46:0/3613793465 learned_addr learned my addr 172.21.15.46:0/3613793465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:19.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.081+0000 7f84f5800640 1 -- 172.21.15.46:0/3613793465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0076720 msgr2=0x7f84f01a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:19.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.081+0000 7f84f5800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0076720 0x7f84f01a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.081+0000 7f84f5800640 1 -- 172.21.15.46:0/3613793465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84f0077040 msgr2=0x7f84f01a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.081+0000 7f84f5800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84f0077040 0x7f84f01a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.081+0000 7f84f5800640 1 -- 172.21.15.46:0/3613793465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f019f0c0 con 0x7f84f0112a30 2024-09-10T07:37:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.081+0000 7f84f5800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84f0112a30 0x7f84f019e990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84f007aa10 tx=0x7f84e800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.082+0000 7f84ed600640 1 -- 172.21.15.46:0/3613793465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8003970 con 0x7f84f0112a30 2024-09-10T07:37:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.082+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f019f3b0 con 0x7f84f0112a30 2024-09-10T07:37:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.082+0000 7f84ed600640 1 -- 172.21.15.46:0/3613793465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e8004020 con 0x7f84f0112a30 2024-09-10T07:37:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.082+0000 7f84ed600640 1 -- 172.21.15.46:0/3613793465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e800b9c0 con 0x7f84f0112a30 2024-09-10T07:37:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.082+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f01ab6a0 con 0x7f84f0112a30 2024-09-10T07:37:19.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.083+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f0076b00 con 0x7f84f0112a30 2024-09-10T07:37:19.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.086+0000 7f84ed600640 1 -- 172.21.15.46:0/3613793465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f84e801d020 con 0x7f84f0112a30 2024-09-10T07:37:19.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.088+0000 7f84ed600640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84d80a2c00 0x7f84d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:19.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.088+0000 7f84ed600640 1 -- 172.21.15.46:0/3613793465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f84e80cf850 con 0x7f84f0112a30 2024-09-10T07:37:19.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.088+0000 7f84f4e00640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84d80a2c00 0x7f84d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:19.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.089+0000 7f84f4e00640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84d80a2c00 0x7f84d80a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84f019fad0 tx=0x7f84e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:19.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.091+0000 7f84ed600640 1 -- 172.21.15.46:0/3613793465 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84e809c3d0 con 0x7f84f0112a30 2024-09-10T07:37:19.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.335+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84f019fec0 con 0x7f84f0112a30 2024-09-10T07:37:19.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.336+0000 7f84ed600640 1 -- 172.21.15.46:0/3613793465 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f84e809bcd0 con 0x7f84f0112a30 2024-09-10T07:37:19.336 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:19.336 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"btime":"2024-09-10T07:36:31:116553+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:36:30.952730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:37:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-09-10T07:37:19.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84d80a2c00 msgr2=0x7f84d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84d80a2c00 0x7f84d80a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84f019fad0 tx=0x7f84e4005f90 comp rx=0 tx=0).stop 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84f0112a30 msgr2=0x7f84f019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84f0112a30 0x7f84f019e990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84f007aa10 tx=0x7f84e800c800 comp rx=0 tx=0).stop 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 shutdown_connections 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84d80a2c00 0x7f84d80a50c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84f0112a30 0x7f84f019e990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84f0077040 0x7f84f01a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 --2- 172.21.15.46:0/3613793465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f0076720 0x7f84f01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 >> 172.21.15.46:0/3613793465 conn(0x7f84f00ffa90 msgr2=0x7f84f0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 shutdown_connections 2024-09-10T07:37:19.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:19.339+0000 7f84f7800640 1 -- 172.21.15.46:0/3613793465 wait complete. 2024-09-10T07:37:19.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:19 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3613793465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:19.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:19 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3613793465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:19.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:19 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3613793465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:20.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:37:20.409 INFO:teuthology.orchestra.run.smithi046.stderr:done in 11.46 s (drop tables 0.00 s, create tables 0.02 s, client-side generate 7.97 s, vacuum 1.01 s, primary keys 2.46 s). 2024-09-10T07:37:20.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-10T07:37:20.598 INFO:teuthology.orchestra.run.smithi046.stderr:starting vacuum...end. 2024-09-10T07:37:20.853 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.855+0000 7f180a600640 1 -- 172.21.15.46:0/2085483397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1804073680 msgr2=0x7f180410b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.856+0000 7f180a600640 1 --2- 172.21.15.46:0/2085483397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1804073680 0x7f180410b4e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009860 tx=0x7f17f402a380 comp rx=0 tx=0).stop 2024-09-10T07:37:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.856+0000 7f180a600640 1 -- 172.21.15.46:0/2085483397 shutdown_connections 2024-09-10T07:37:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.856+0000 7f180a600640 1 --2- 172.21.15.46:0/2085483397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180410bba0 0x7f1804114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.856+0000 7f180a600640 1 --2- 172.21.15.46:0/2085483397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1804073680 0x7f180410b4e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.856+0000 7f180a600640 1 --2- 172.21.15.46:0/2085483397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1804072cb0 0x7f18040730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.856+0000 7f180a600640 1 -- 172.21.15.46:0/2085483397 >> 172.21.15.46:0/2085483397 conn(0x7f180406eb50 msgr2=0x7f1804070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.857+0000 7f180a600640 1 -- 172.21.15.46:0/2085483397 shutdown_connections 2024-09-10T07:37:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.858+0000 7f180a600640 1 -- 172.21.15.46:0/2085483397 wait complete. 2024-09-10T07:37:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.859+0000 7f180a600640 1 Processor -- start 2024-09-10T07:37:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.862+0000 7f180a600640 1 -- start start 2024-09-10T07:37:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.862+0000 7f180a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1804072cb0 0x7f180410fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.862+0000 7f180a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1804073680 0x7f18041103c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.862+0000 7f180a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f180410bba0 0x7f180410c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.862+0000 7f180a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1804118090 con 0x7f1804072cb0 2024-09-10T07:37:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f180410bba0 0x7f180410c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f180410bba0 0x7f180410c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33936/0 (socket says 172.21.15.46:33936) 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809c00640 1 -- 172.21.15.46:0/3364085046 learned_addr learned my addr 172.21.15.46:0/3364085046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1804117f10 con 0x7f180410bba0 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1804118210 con 0x7f1804073680 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1803e00640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1804073680 0x7f18041103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809200640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1804072cb0 0x7f180410fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809c00640 1 -- 172.21.15.46:0/3364085046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1804073680 msgr2=0x7f18041103c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809c00640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1804073680 0x7f18041103c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809c00640 1 -- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1804072cb0 msgr2=0x7f180410fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809c00640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1804072cb0 0x7f180410fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.863+0000 7f1809c00640 1 -- 172.21.15.46:0/3364085046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180410d2b0 con 0x7f180410bba0 2024-09-10T07:37:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.865+0000 7f1809c00640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f180410bba0 0x7f180410c9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17f000d750 tx=0x7f17f000dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.866+0000 7f1801600640 1 -- 172.21.15.46:0/3364085046 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0017070 con 0x7f180410bba0 2024-09-10T07:37:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.866+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18041aeba0 con 0x7f180410bba0 2024-09-10T07:37:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.867+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18041aefa0 con 0x7f180410bba0 2024-09-10T07:37:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.867+0000 7f1801600640 1 -- 172.21.15.46:0/3364085046 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f0005500 con 0x7f180410bba0 2024-09-10T07:37:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.867+0000 7f1801600640 1 -- 172.21.15.46:0/3364085046 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0004e10 con 0x7f180410bba0 2024-09-10T07:37:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.877+0000 7f180eac7740 1 Processor -- start 2024-09-10T07:37:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.877+0000 7f1801600640 1 -- 172.21.15.46:0/3364085046 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17f0010460 con 0x7f180410bba0 2024-09-10T07:37:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.879+0000 7f1801600640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e80a2d90 0x7f17e80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.879+0000 7f1801600640 1 -- 172.21.15.46:0/3364085046 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f17f00cec70 con 0x7f180410bba0 2024-09-10T07:37:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.880+0000 7f180eac7740 1 -- start start 2024-09-10T07:37:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.880+0000 7f1803e00640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e80a2d90 0x7f17e80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.880+0000 7f180eac7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.880+0000 7f180eac7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8daf2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f180eac7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daf2c30 0x55df8daf50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f180eac7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55df8da52100 con 0x55df8daf2c30 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f180eac7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55df8d9fa1b0 con 0x55df8daed260 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f180eac7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55df8da4e0c0 con 0x55df8daedba0 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f1809200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f1809c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daf2c30 0x55df8daf50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f1803e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8daf2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f1809200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daed660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33952/0 (socket says 172.21.15.46:33952) 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.881+0000 7f1809c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daf2c30 0x55df8daf50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53446/0 (socket says 172.21.15.46:53446) 2024-09-10T07:37:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.882+0000 7f1809200640 1 -- 172.21.15.46:0/2358239311 learned_addr learned my addr 172.21.15.46:0/2358239311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.882+0000 7f1809200640 1 -- 172.21.15.46:0/2358239311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 msgr2=0x55df8daf2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.882+0000 7f1809200640 1 --2- 172.21.15.46:0/2358239311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8daf2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.882+0000 7f1809200640 1 -- 172.21.15.46:0/2358239311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daf2c30 msgr2=0x55df8daf50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.882+0000 7f1809200640 1 --2- 172.21.15.46:0/2358239311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daf2c30 0x55df8daf50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.882+0000 7f1809200640 1 -- 172.21.15.46:0/2358239311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df8d765210 con 0x55df8daed260 2024-09-10T07:37:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.883+0000 7f1809200640 1 --2- 172.21.15.46:0/2358239311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daed660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f17fc007e10 tx=0x7f17fc00c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ac6ea66279f418 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.883+0000 7f1803e00640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e80a2d90 0x7f17e80a5250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009780 tx=0x7f17f4030df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.883+0000 7f17fa000640 1 -- 172.21.15.46:0/2358239311 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc015020 con 0x55df8daed260 2024-09-10T07:37:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.884+0000 7f17fa000640 1 -- 172.21.15.46:0/2358239311 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17fc011b10 con 0x55df8daed260 2024-09-10T07:37:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.884+0000 7f17fa000640 1 -- 172.21.15.46:0/2358239311 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc010b20 con 0x55df8daed260 2024-09-10T07:37:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.884+0000 7f180eac7740 1 -- 172.21.15.46:0/2358239311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 msgr2=0x55df8daed660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.884+0000 7f180eac7740 1 --2- 172.21.15.46:0/2358239311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daed660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f17fc007e10 tx=0x7f17fc00c560 comp rx=0 tx=0).stop 2024-09-10T07:37:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.885+0000 7f180eac7740 1 -- 172.21.15.46:0/2358239311 shutdown_connections 2024-09-10T07:37:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.885+0000 7f180eac7740 1 --2- 172.21.15.46:0/2358239311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daf2c30 0x55df8daf50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.885+0000 7f180eac7740 1 --2- 172.21.15.46:0/2358239311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8daf2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.885+0000 7f180eac7740 1 --2- 172.21.15.46:0/2358239311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daed660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.885+0000 7f180eac7740 1 -- 172.21.15.46:0/2358239311 >> 172.21.15.46:0/2358239311 conn(0x55df8dae88d0 msgr2=0x55df8dae8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.886+0000 7f180eac7740 1 -- 172.21.15.46:0/2358239311 shutdown_connections 2024-09-10T07:37:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.887+0000 7f180eac7740 1 -- 172.21.15.46:0/2358239311 wait complete. 2024-09-10T07:37:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.887+0000 7f180eac7740 1 -- start start 2024-09-10T07:37:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.888+0000 7f180eac7740 1 Processor -- start 2024-09-10T07:37:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.890+0000 7f180eac7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daf7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.890+0000 7f180eac7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daedba0 0x55df8daf7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.891+0000 7f180eac7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dafe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.891+0000 7f180eac7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55df8daf5c00 con 0x55df8daedba0 2024-09-10T07:37:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.891+0000 7f180eac7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55df8d9db090 con 0x55df8daed260 2024-09-10T07:37:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.891+0000 7f180eac7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55df8daf5d80 con 0x55df8daf2c30 2024-09-10T07:37:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.891+0000 7f180eac7740 10 client.0 _ll_register_callbacks cb 0x55df8da2ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.891+0000 7f1809200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daf7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.891+0000 7f1809200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daf7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33954/0 (socket says 172.21.15.46:33954) 2024-09-10T07:37:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.891+0000 7f1809200640 1 -- 172.21.15.46:0/3914161879 learned_addr learned my addr 172.21.15.46:0/3914161879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.892+0000 7f1809200640 1 -- 172.21.15.46:0/3914161879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 msgr2=0x55df8dafe9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.892+0000 7f1809200640 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dafe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.892+0000 7f1803e00640 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daedba0 0x55df8daf7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.892+0000 7f1809200640 1 -- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daedba0 msgr2=0x55df8daf7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.892+0000 7f1809200640 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daedba0 0x55df8daf7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.893+0000 7f1809200640 1 -- 172.21.15.46:0/3914161879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df8d831620 con 0x55df8daed260 2024-09-10T07:37:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.893+0000 7f1803e00640 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daedba0 0x55df8daf7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.893+0000 7f1809200640 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daf7420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17fc000f80 tx=0x7f17fc011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5211efe0e6f0d4d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.893+0000 7f17eea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.894+0000 7f17eea00640 1 -- 172.21.15.46:0/3914161879 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc029060 con 0x55df8daed260 2024-09-10T07:37:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.894+0000 7f17eea00640 1 -- 172.21.15.46:0/3914161879 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f17fc010a60 con 0x55df8daed260 2024-09-10T07:37:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.894+0000 7f180eac7740 1 -- 172.21.15.46:0/3914161879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17fc011b10 con 0x55df8daed260 2024-09-10T07:37:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.894+0000 7f17eea00640 1 -- 172.21.15.46:0/3914161879 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17fc030050 con 0x55df8daed260 2024-09-10T07:37:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.896+0000 7f180eac7740 10 client.25502 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.896+0000 7f180eac7740 1 -- 172.21.15.46:0/3914161879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df8db01790 con 0x55df8daed260 2024-09-10T07:37:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.896+0000 7f17eea00640 1 -- 172.21.15.46:0/3914161879 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17fc011b10 con 0x55df8daed260 2024-09-10T07:37:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.897+0000 7f17eea00640 1 -- 172.21.15.46:0/3914161879 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f17fc023030 con 0x55df8daed260 2024-09-10T07:37:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.897+0000 7f180eac7740 10 client.25502 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.897+0000 7f180eac7740 10 client.25502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.897+0000 7f180eac7740 20 client.25502 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.897+0000 7f180eac7740 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55df8db02cf0 0x55df8db050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.898+0000 7f180eac7740 4 client.25502 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.898+0000 7f180eac7740 1 -- 172.21.15.46:0/3914161879 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df8da5f830 con 0x55df8db02cf0 2024-09-10T07:37:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.898+0000 7f1803e00640 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55df8db02cf0 0x55df8db050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:20 smithi181 ceph-mon[35397]: pgmap v420: 129 pgs: 129 active+clean; 300 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 45 MiB/s wr, 351 op/s 2024-09-10T07:37:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.899+0000 7f1803e00640 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55df8db02cf0 0x55df8db050e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f5719c92179d91 server_cookie=7490bbd3c1f58718 in_seq=0 out_seq=0 2024-09-10T07:37:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.900+0000 7f17eea00640 10 client.25502 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:20.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:20 smithi063 ceph-mon[35020]: pgmap v420: 129 pgs: 129 active+clean; 300 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 45 MiB/s wr, 351 op/s 2024-09-10T07:37:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.901+0000 7f17eea00640 1 -- 172.21.15.46:0/3914161879 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df8da5f830 con 0x55df8db02cf0 2024-09-10T07:37:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.901+0000 7f17eea00640 10 client.25502 handle_command_reply: tid=0 2024-09-10T07:37:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.901+0000 7f180eac7740 1 client.25502 shutdown 2024-09-10T07:37:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.902+0000 7f180eac7740 1 -- 172.21.15.46:0/3914161879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 msgr2=0x55df8daf7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.902+0000 7f180eac7740 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daf7420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17fc000f80 tx=0x7f17fc011a90 comp rx=0 tx=0).stop 2024-09-10T07:37:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.902+0000 7f180eac7740 1 -- 172.21.15.46:0/3914161879 shutdown_connections 2024-09-10T07:37:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.903+0000 7f180eac7740 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55df8db02cf0 0x55df8db050e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.903+0000 7f180eac7740 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dafe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.903+0000 7f180eac7740 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daedba0 0x55df8daf7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.903+0000 7f180eac7740 1 --2- 172.21.15.46:0/3914161879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daed260 0x55df8daf7420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.903+0000 7f180eac7740 1 -- 172.21.15.46:0/3914161879 >> 172.21.15.46:0/3914161879 conn(0x55df8dae88d0 msgr2=0x55df8daea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.903+0000 7f17eea00640 0 client.25502 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.903+0000 7f180eac7740 1 -- 172.21.15.46:0/3914161879 shutdown_connections 2024-09-10T07:37:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.904+0000 7f180eac7740 1 -- 172.21.15.46:0/3914161879 wait complete. 2024-09-10T07:37:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.904+0000 7f180eac7740 20 client.25502 trim_cache size 0 max 16384 2024-09-10T07:37:20.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:20 smithi046 ceph-mon[30705]: pgmap v420: 129 pgs: 129 active+clean; 300 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 45 MiB/s wr, 351 op/s 2024-09-10T07:37:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.946+0000 7f180eac7740 1 Processor -- start 2024-09-10T07:37:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.946+0000 7f180eac7740 1 -- start start 2024-09-10T07:37:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f180eac7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8daed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f180eac7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daedba0 0x55df8dafe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f180eac7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dae8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f180eac7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17e400f9c0 con 0x55df8daed260 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f180eac7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4011500 con 0x55df8daedba0 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f180eac7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4010090 con 0x55df8daf2c30 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f1809c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dae8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f1809c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dae8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40622/0 (socket says 172.21.15.46:40622) 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.947+0000 7f1809c00640 1 -- 172.21.15.46:0/1429469323 learned_addr learned my addr 172.21.15.46:0/1429469323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.948+0000 7f1803e00640 1 --2- 172.21.15.46:0/1429469323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daedba0 0x55df8dafe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.948+0000 7f1809c00640 1 -- 172.21.15.46:0/1429469323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daedba0 msgr2=0x55df8dafe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.948+0000 7f1809c00640 1 --2- 172.21.15.46:0/1429469323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daedba0 0x55df8dafe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.948+0000 7f1809c00640 1 -- 172.21.15.46:0/1429469323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 msgr2=0x55df8daed640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.948+0000 7f1809c00640 1 --2- 172.21.15.46:0/1429469323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8daed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.948+0000 7f1809c00640 1 -- 172.21.15.46:0/1429469323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df8db05450 con 0x55df8daf2c30 2024-09-10T07:37:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.948+0000 7f1809c00640 1 --2- 172.21.15.46:0/1429469323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dae8cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f17f00cc390 tx=0x7f17f0097410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb585f3e804be4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.949+0000 7f17f9600640 1 -- 172.21.15.46:0/1429469323 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0020040 con 0x55df8daf2c30 2024-09-10T07:37:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.949+0000 7f17f9600640 1 -- 172.21.15.46:0/1429469323 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f0015070 con 0x55df8daf2c30 2024-09-10T07:37:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.949+0000 7f17f9600640 1 -- 172.21.15.46:0/1429469323 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00bb0b0 con 0x55df8daf2c30 2024-09-10T07:37:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.950+0000 7f180eac7740 1 -- 172.21.15.46:0/1429469323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 msgr2=0x55df8dae8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.950+0000 7f180eac7740 1 --2- 172.21.15.46:0/1429469323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dae8cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f17f00cc390 tx=0x7f17f0097410 comp rx=0 tx=0).stop 2024-09-10T07:37:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.951+0000 7f180eac7740 1 -- 172.21.15.46:0/1429469323 shutdown_connections 2024-09-10T07:37:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.951+0000 7f180eac7740 1 --2- 172.21.15.46:0/1429469323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daf2c30 0x55df8dae8cb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.951+0000 7f180eac7740 1 --2- 172.21.15.46:0/1429469323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daedba0 0x55df8dafe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.951+0000 7f180eac7740 1 --2- 172.21.15.46:0/1429469323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8daed640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.951+0000 7f180eac7740 1 -- 172.21.15.46:0/1429469323 >> 172.21.15.46:0/1429469323 conn(0x55df8dae88d0 msgr2=0x55df8dae7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.951+0000 7f180eac7740 1 -- 172.21.15.46:0/1429469323 shutdown_connections 2024-09-10T07:37:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.952+0000 7f180eac7740 1 -- 172.21.15.46:0/1429469323 wait complete. 2024-09-10T07:37:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.953+0000 7f180eac7740 1 -- start start 2024-09-10T07:37:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.953+0000 7f180eac7740 1 Processor -- start 2024-09-10T07:37:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f180eac7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8dafbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f180eac7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8dafc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f180eac7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daf2c30 0x55df8dafd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f1809200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8dafbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f1809200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8dafbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53474/0 (socket says 172.21.15.46:53474) 2024-09-10T07:37:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f1809200640 1 -- 172.21.15.46:0/3474039488 learned_addr learned my addr 172.21.15.46:0/3474039488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4010e30 con 0x55df8daed260 2024-09-10T07:37:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4010760 con 0x55df8daf2c30 2024-09-10T07:37:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.955+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17e400da50 con 0x55df8daedba0 2024-09-10T07:37:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.956+0000 7f180eac7740 10 client.0 _ll_register_callbacks cb 0x7f17f0004740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.956+0000 7f1803e00640 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8dafc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.956+0000 7f1809c00640 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daf2c30 0x55df8dafd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.957+0000 7f1803e00640 1 -- 172.21.15.46:0/3474039488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daf2c30 msgr2=0x55df8dafd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.957+0000 7f1803e00640 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daf2c30 0x55df8dafd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.957+0000 7f1803e00640 1 -- 172.21.15.46:0/3474039488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 msgr2=0x55df8dafbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.957+0000 7f1803e00640 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8dafbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.957+0000 7f1803e00640 1 -- 172.21.15.46:0/3474039488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df8dafd660 con 0x55df8daedba0 2024-09-10T07:37:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.957+0000 7f1809200640 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8dafbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.957+0000 7f1803e00640 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8dafc090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17f40304b0 tx=0x7f17f402a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce8f50d30bccba5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.957+0000 7f17eea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.958+0000 7f17eea00640 1 -- 172.21.15.46:0/3474039488 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4037040 con 0x55df8daedba0 2024-09-10T07:37:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.958+0000 7f17eea00640 1 -- 172.21.15.46:0/3474039488 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f17f40048c0 con 0x55df8daedba0 2024-09-10T07:37:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.958+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f0015070 con 0x55df8daedba0 2024-09-10T07:37:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.958+0000 7f17eea00640 1 -- 172.21.15.46:0/3474039488 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17f4004ac0 con 0x55df8daedba0 2024-09-10T07:37:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.958+0000 7f180eac7740 10 client.25516 fetch_fsmap learned FSMap version 30 2024-09-10T07:37:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.958+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df8dafe9c0 con 0x55df8daedba0 2024-09-10T07:37:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.959+0000 7f17eea00640 1 -- 172.21.15.46:0/3474039488 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f0015070 con 0x55df8daedba0 2024-09-10T07:37:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.960+0000 7f17eea00640 1 -- 172.21.15.46:0/3474039488 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f17f403d070 con 0x55df8daedba0 2024-09-10T07:37:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.960+0000 7f180eac7740 10 client.25516 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T07:37:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.960+0000 7f180eac7740 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.960+0000 7f180eac7740 20 client.25516 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.960+0000 7f180eac7740 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55df8db6de50 0x55df8db70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.960+0000 7f180eac7740 4 client.25516 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:37:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.960+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df8dae8180 con 0x55df8db6de50 2024-09-10T07:37:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.961+0000 7f1809200640 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55df8db6de50 0x55df8db70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.962+0000 7f1809200640 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55df8db6de50 0x55df8db70270 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99508eac8a74e6c server_cookie=1b4ca8079e2507f8 in_seq=0 out_seq=0 2024-09-10T07:37:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:20.962+0000 7f17eea00640 10 client.25516 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.146+0000 7f17eea00640 1 -- 172.21.15.46:0/3474039488 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df8dae8180 con 0x55df8db6de50 2024-09-10T07:37:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.146+0000 7f17eea00640 10 client.25516 handle_command_reply: tid=0 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.146+0000 7f180eac7740 1 client.25516 shutdown 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.146+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 msgr2=0x55df8dafc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.146+0000 7f180eac7740 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8dafc090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17f40304b0 tx=0x7f17f402a380 comp rx=0 tx=0).stop 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 shutdown_connections 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55df8db6de50 0x55df8db70270 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df8daf2c30 0x55df8dafd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df8daedba0 0x55df8dafc090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 1 --2- 172.21.15.46:0/3474039488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df8daed260 0x55df8dafbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 >> 172.21.15.46:0/3474039488 conn(0x55df8dae88d0 msgr2=0x55df8daf6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f17eea00640 0 client.25516 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 shutdown_connections 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 1 -- 172.21.15.46:0/3474039488 wait complete. 2024-09-10T07:37:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.147+0000 7f180eac7740 20 client.25516 trim_cache size 0 max 16384 2024-09-10T07:37:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:37:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:37:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:37:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:37:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e80a2d90 msgr2=0x7f17e80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e80a2d90 0x7f17e80a5250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009780 tx=0x7f17f4030df0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f180410bba0 msgr2=0x7f180410c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f180410bba0 0x7f180410c9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17f000d750 tx=0x7f17f000dc10 comp rx=0 tx=0).stop 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 shutdown_connections 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f180410bba0 0x7f180410c9f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1804073680 0x7f18041103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e80a2d90 0x7f17e80a5250 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 --2- 172.21.15.46:0/3364085046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1804072cb0 0x7f180410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.149+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 >> 172.21.15.46:0/3364085046 conn(0x7f180406eb50 msgr2=0x7f180406f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.150+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 shutdown_connections 2024-09-10T07:37:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:21.150+0000 7f180a600640 1 -- 172.21.15.46:0/3364085046 wait complete. 2024-09-10T07:37:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 -- 172.21.15.46:0/2537803172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0109120 msgr2=0x7f06d0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 --2- 172.21.15.46:0/2537803172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0109120 0x7f06d0110ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009ca0 tx=0x7f06c802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 -- 172.21.15.46:0/2537803172 shutdown_connections 2024-09-10T07:37:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 --2- 172.21.15.46:0/2537803172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0109120 0x7f06d0110ca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 --2- 172.21.15.46:0/2537803172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06d01049a0 0x7f06d01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 --2- 172.21.15.46:0/2537803172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d0103ff0 0x7f06d01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 -- 172.21.15.46:0/2537803172 >> 172.21.15.46:0/2537803172 conn(0x7f06d00ffa90 msgr2=0x7f06d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 -- 172.21.15.46:0/2537803172 shutdown_connections 2024-09-10T07:37:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.598+0000 7f06d6600640 1 -- 172.21.15.46:0/2537803172 wait complete. 2024-09-10T07:37:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.599+0000 7f06d6600640 1 Processor -- start 2024-09-10T07:37:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.599+0000 7f06d6600640 1 -- start start 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06d6600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0103ff0 0x7f06d019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06d6600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d01049a0 0x7f06d019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06d6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06d0109120 0x7f06d019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06d6600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0114180 con 0x7f06d0109120 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06d6600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0114000 con 0x7f06d0103ff0 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06d6600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0114300 con 0x7f06d01049a0 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0103ff0 0x7f06d019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0103ff0 0x7f06d019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33998/0 (socket says 172.21.15.46:33998) 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cf400640 1 -- 172.21.15.46:0/4235649137 learned_addr learned my addr 172.21.15.46:0/4235649137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cfe00640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06d0109120 0x7f06d019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cea00640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d01049a0 0x7f06d019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cf400640 1 -- 172.21.15.46:0/4235649137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d01049a0 msgr2=0x7f06d019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cf400640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d01049a0 0x7f06d019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cf400640 1 -- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06d0109120 msgr2=0x7f06d019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cf400640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06d0109120 0x7f06d019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cf400640 1 -- 172.21.15.46:0/4235649137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d019f570 con 0x7f06d0103ff0 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.600+0000 7f06cea00640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d01049a0 0x7f06d019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.601+0000 7f06cf400640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0103ff0 0x7f06d019a5c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f06c000e9f0 tx=0x7f06c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.601+0000 7f06c7e00640 1 -- 172.21.15.46:0/4235649137 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c000cd80 con 0x7f06d0103ff0 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.601+0000 7f06c7e00640 1 -- 172.21.15.46:0/4235649137 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c0010910 con 0x7f06d0103ff0 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.601+0000 7f06c7e00640 1 -- 172.21.15.46:0/4235649137 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0018990 con 0x7f06d0103ff0 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.601+0000 7f06d6600640 1 -- 172.21.15.46:0/4235649137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d019f860 con 0x7f06d0103ff0 2024-09-10T07:37:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.601+0000 7f06d6600640 1 -- 172.21.15.46:0/4235649137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d01a7140 con 0x7f06d0103ff0 2024-09-10T07:37:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.605+0000 7f06d904e740 1 Processor -- start 2024-09-10T07:37:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06d904e740 1 -- start start 2024-09-10T07:37:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06c7e00640 1 -- 172.21.15.46:0/4235649137 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f06c0010ac0 con 0x7f06d0103ff0 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06d904e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06d904e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06d904e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd658b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06d904e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dd658123e0 con 0x55dd658abc70 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06d904e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd657b9d80 con 0x55dd658ac590 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06d904e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dd6580e3a0 con 0x55dd658b6e50 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.606+0000 7f06cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53498/0 (socket says 172.21.15.46:53498) 2024-09-10T07:37:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.607+0000 7f06cfe00640 1 -- 172.21.15.46:0/585288387 learned_addr learned my addr 172.21.15.46:0/585288387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.607+0000 7f06cf400640 1 -- 172.21.15.46:0/585288387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 msgr2=0x55dd658b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.607+0000 7f06cf400640 1 --2- 172.21.15.46:0/585288387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd658b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.607+0000 7f06cf400640 1 -- 172.21.15.46:0/585288387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 msgr2=0x55dd658ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.607+0000 7f06cf400640 1 --2- 172.21.15.46:0/585288387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.607+0000 7f06cf400640 1 -- 172.21.15.46:0/585288387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd657d8ae0 con 0x55dd658ac590 2024-09-10T07:37:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.607+0000 7f06cf400640 1 --2- 172.21.15.46:0/585288387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b6910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f06c00ccf80 tx=0x7f06c00d7690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9e5c8d63238a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.608+0000 7f06c4c00640 1 -- 172.21.15.46:0/585288387 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00de070 con 0x55dd658ac590 2024-09-10T07:37:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.608+0000 7f06c4c00640 1 -- 172.21.15.46:0/585288387 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00dab20 con 0x55dd658ac590 2024-09-10T07:37:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.608+0000 7f06c4c00640 1 -- 172.21.15.46:0/585288387 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00d9ad0 con 0x55dd658ac590 2024-09-10T07:37:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.608+0000 7f06d904e740 1 -- 172.21.15.46:0/585288387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 msgr2=0x55dd658b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.608+0000 7f06d904e740 1 --2- 172.21.15.46:0/585288387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b6910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f06c00ccf80 tx=0x7f06c00d7690 comp rx=0 tx=0).stop 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06c7e00640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f06b40a2b80 0x7f06b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06d904e740 1 -- 172.21.15.46:0/585288387 shutdown_connections 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06d904e740 1 --2- 172.21.15.46:0/585288387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd658b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06d904e740 1 --2- 172.21.15.46:0/585288387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b6910 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06d904e740 1 --2- 172.21.15.46:0/585288387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06d904e740 1 -- 172.21.15.46:0/585288387 >> 172.21.15.46:0/585288387 conn(0x55dd658a8200 msgr2=0x55dd658a85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06c7e00640 1 -- 172.21.15.46:0/4235649137 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f06c00cf830 con 0x7f06d0103ff0 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06cfe00640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f06b40a2b80 0x7f06b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06d904e740 1 -- 172.21.15.46:0/585288387 shutdown_connections 2024-09-10T07:37:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06d904e740 1 -- 172.21.15.46:0/585288387 wait complete. 2024-09-10T07:37:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.609+0000 7f06d904e740 1 -- start start 2024-09-10T07:37:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06d904e740 1 Processor -- start 2024-09-10T07:37:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06cfe00640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f06b40a2b80 0x7f06b40a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009ca0 tx=0x7f06c802f370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06d904e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06d904e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06d904e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd658b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06d904e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dd658b9a20 con 0x55dd658abc70 2024-09-10T07:37:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06d904e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd657cd0f0 con 0x55dd658ac590 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06d904e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dd658b9ba0 con 0x55dd658b6e50 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.610+0000 7f06d904e740 10 client.0 _ll_register_callbacks cb 0x55dd657b4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34026/0 (socket says 172.21.15.46:34026) 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 -- 172.21.15.46:0/3658758526 learned_addr learned my addr 172.21.15.46:0/3658758526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cf400640 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cfe00640 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd658b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 -- 172.21.15.46:0/3658758526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 msgr2=0x55dd658b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd658b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 -- 172.21.15.46:0/3658758526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 msgr2=0x55dd658b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 -- 172.21.15.46:0/3658758526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd655833c0 con 0x55dd658ac590 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06cea00640 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b4250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f06bc011480 tx=0x7f06bc011940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d4bf1d6f4dc998 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.611+0000 7f06b9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.612+0000 7f06b9600640 1 -- 172.21.15.46:0/3658758526 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc004460 con 0x55dd658ac590 2024-09-10T07:37:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.612+0000 7f06d904e740 1 -- 172.21.15.46:0/3658758526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c00dab20 con 0x55dd658ac590 2024-09-10T07:37:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.612+0000 7f06b9600640 1 -- 172.21.15.46:0/3658758526 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f06bc0035f0 con 0x55dd658ac590 2024-09-10T07:37:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.612+0000 7f06b9600640 1 -- 172.21.15.46:0/3658758526 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f06bc025050 con 0x55dd658ac590 2024-09-10T07:37:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.612+0000 7f06d904e740 10 client.25520 fetch_fsmap learned FSMap version 31 2024-09-10T07:37:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.612+0000 7f06d904e740 1 -- 172.21.15.46:0/3658758526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dd658c5a90 con 0x55dd658ac590 2024-09-10T07:37:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06b9600640 1 -- 172.21.15.46:0/3658758526 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00dab20 con 0x55dd658ac590 2024-09-10T07:37:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06b9600640 1 -- 172.21.15.46:0/3658758526 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f06bc012bb0 con 0x55dd658ac590 2024-09-10T07:37:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06d904e740 10 client.25520 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06d904e740 10 client.25520 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06d904e740 20 client.25520 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06d904e740 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55dd658c6ff0 0x55dd658c9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06d904e740 4 client.25520 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06d904e740 1 -- 172.21.15.46:0/3658758526 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd6581f950 con 0x55dd658c6ff0 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.613+0000 7f06cf400640 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55dd658c6ff0 0x55dd658c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.614+0000 7f06cf400640 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55dd658c6ff0 0x55dd658c9410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=463002c6db4daf78 server_cookie=bb8e1142e9e913f1 in_seq=0 out_seq=0 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.614+0000 7f06b9600640 10 client.25520 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06b9600640 1 -- 172.21.15.46:0/3658758526 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd6581f950 con 0x55dd658c6ff0 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06b9600640 10 client.25520 handle_command_reply: tid=0 2024-09-10T07:37:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 client.25520 shutdown 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 -- 172.21.15.46:0/3658758526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 msgr2=0x55dd658b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b4250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f06bc011480 tx=0x7f06bc011940 comp rx=0 tx=0).stop 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 -- 172.21.15.46:0/3658758526 shutdown_connections 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55dd658c6ff0 0x55dd658c9410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd658b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658b4250 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 --2- 172.21.15.46:0/3658758526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658abc70 0x55dd658b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 -- 172.21.15.46:0/3658758526 >> 172.21.15.46:0/3658758526 conn(0x55dd658a8200 msgr2=0x55dd658adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06b9600640 0 client.25520 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:37:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 -- 172.21.15.46:0/3658758526 shutdown_connections 2024-09-10T07:37:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 1 -- 172.21.15.46:0/3658758526 wait complete. 2024-09-10T07:37:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.615+0000 7f06d904e740 20 client.25520 trim_cache size 0 max 16384 2024-09-10T07:37:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.630+0000 7f06d904e740 1 Processor -- start 2024-09-10T07:37:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.630+0000 7f06d904e740 1 -- start start 2024-09-10T07:37:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.630+0000 7f06d904e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658abc70 0x55dd658b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.630+0000 7f06d904e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.630+0000 7f06d904e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658b6e50 0x55dd658cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.630+0000 7f06d904e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01b0f10 con 0x55dd658b6e50 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.630+0000 7f06d904e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01b2a80 con 0x55dd658ac590 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.630+0000 7f06d904e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01b15e0 con 0x55dd658abc70 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658b6e50 0x55dd658cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658abc70 0x55dd658b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658b6e50 0x55dd658cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53512/0 (socket says 172.21.15.46:53512) 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cfe00640 1 -- 172.21.15.46:0/726757701 learned_addr learned my addr 172.21.15.46:0/726757701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34040/0 (socket says 172.21.15.46:34040) 2024-09-10T07:37:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cf400640 1 -- 172.21.15.46:0/726757701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 msgr2=0x55dd658cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cf400640 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cf400640 1 -- 172.21.15.46:0/726757701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658b6e50 msgr2=0x55dd658cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cf400640 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658b6e50 0x55dd658cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cf400640 1 -- 172.21.15.46:0/726757701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd658ac3c0 con 0x55dd658abc70 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cea00640 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.631+0000 7f06cfe00640 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658b6e50 0x55dd658cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.632+0000 7f06cf400640 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658abc70 0x55dd658b25e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06c000e9c0 tx=0x7f06c0018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f22c7324c2072e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.632+0000 7f06bbe00640 1 -- 172.21.15.46:0/726757701 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00de040 con 0x55dd658abc70 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.632+0000 7f06bbe00640 1 -- 172.21.15.46:0/726757701 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00d7e90 con 0x55dd658abc70 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.632+0000 7f06bbe00640 1 -- 172.21.15.46:0/726757701 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00a4980 con 0x55dd658abc70 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 -- 172.21.15.46:0/726757701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658abc70 msgr2=0x55dd658b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658abc70 0x55dd658b25e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06c000e9c0 tx=0x7f06c0018420 comp rx=0 tx=0).stop 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 -- 172.21.15.46:0/726757701 shutdown_connections 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658b6e50 0x55dd658cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658ac590 0x55dd658cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 --2- 172.21.15.46:0/726757701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658abc70 0x55dd658b25e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 -- 172.21.15.46:0/726757701 >> 172.21.15.46:0/726757701 conn(0x55dd658a8200 msgr2=0x55dd658b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 -- 172.21.15.46:0/726757701 shutdown_connections 2024-09-10T07:37:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.633+0000 7f06d904e740 1 -- 172.21.15.46:0/726757701 wait complete. 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 1 -- start start 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 1 Processor -- start 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658abc70 0x55dd658baf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658ac590 0x55dd658bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd659394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01b2380 con 0x55dd658ac590 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01b1cb0 con 0x55dd658abc70 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06d01aeef0 con 0x55dd658b6e50 2024-09-10T07:37:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.634+0000 7f06d904e740 10 client.0 _ll_register_callbacks cb 0x7f06bc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658abc70 0x55dd658baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658abc70 0x55dd658baf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34056/0 (socket says 172.21.15.46:34056) 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cf400640 1 -- 172.21.15.46:0/1095577611 learned_addr learned my addr 172.21.15.46:0/1095577611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cea00640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658ac590 0x55dd658bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cfe00640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd659394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cea00640 1 -- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 msgr2=0x55dd659394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cea00640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd659394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cea00640 1 -- 172.21.15.46:0/1095577611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658abc70 msgr2=0x55dd658baf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cea00640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658abc70 0x55dd658baf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cea00640 1 -- 172.21.15.46:0/1095577611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd659399e0 con 0x55dd658ac590 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cfe00640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd659394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.635+0000 7f06cf400640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658abc70 0x55dd658baf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.636+0000 7f06cea00640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658ac590 0x55dd658bb470 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00f3b0 tx=0x7f06bc0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc15436a3fa8d0ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.636+0000 7f06b9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:37:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.636+0000 7f06b9600640 1 -- 172.21.15.46:0/1095577611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0054d0 con 0x55dd658ac590 2024-09-10T07:37:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.636+0000 7f06b9600640 1 -- 172.21.15.46:0/1095577611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f06bc01aa20 con 0x55dd658ac590 2024-09-10T07:37:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.636+0000 7f06d904e740 1 -- 172.21.15.46:0/1095577611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c00d7e90 con 0x55dd658ac590 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.636+0000 7f06b9600640 1 -- 172.21.15.46:0/1095577611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f06bc03a050 con 0x55dd658ac590 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.636+0000 7f06d904e740 10 client.15477 fetch_fsmap learned FSMap version 31 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.636+0000 7f06d904e740 1 -- 172.21.15.46:0/1095577611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dd6593ab70 con 0x55dd658ac590 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06b9600640 1 -- 172.21.15.46:0/1095577611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00d7e90 con 0x55dd658ac590 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06b9600640 1 -- 172.21.15.46:0/1095577611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f06bc012b90 con 0x55dd658ac590 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06d904e740 10 client.15477 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06d904e740 10 client.15477 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06d904e740 20 client.15477 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06d904e740 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55dd6593b2f0 0x55dd6593d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06d904e740 4 client.15477 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06d904e740 1 -- 172.21.15.46:0/1095577611 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd65819e60 con 0x55dd6593b2f0 2024-09-10T07:37:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.637+0000 7f06cf400640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55dd6593b2f0 0x55dd6593d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06cf400640 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55dd6593b2f0 0x55dd6593d710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd2acb92148ea5ac server_cookie=5b01d97fc0d6c2f3 in_seq=0 out_seq=0 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06b9600640 10 client.15477 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06b9600640 1 -- 172.21.15.46:0/1095577611 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd65819e60 con 0x55dd6593b2f0 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06b9600640 10 client.15477 handle_command_reply: tid=0 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 client.15477 shutdown 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 -- 172.21.15.46:0/1095577611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658ac590 msgr2=0x55dd658bb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658ac590 0x55dd658bb470 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00f3b0 tx=0x7f06bc0093e0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 -- 172.21.15.46:0/1095577611 shutdown_connections 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55dd6593b2f0 0x55dd6593d710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd658b6e50 0x55dd659394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd658ac590 0x55dd658bb470 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06b9600640 0 client.15477 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 --2- 172.21.15.46:0/1095577611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd658abc70 0x55dd658baf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.638+0000 7f06d904e740 1 -- 172.21.15.46:0/1095577611 >> 172.21.15.46:0/1095577611 conn(0x55dd658a8200 msgr2=0x55dd658ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.639+0000 7f06d904e740 1 -- 172.21.15.46:0/1095577611 shutdown_connections 2024-09-10T07:37:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.639+0000 7f06d904e740 1 -- 172.21.15.46:0/1095577611 wait complete. 2024-09-10T07:37:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.639+0000 7f06d904e740 20 client.15477 trim_cache size 0 max 16384 2024-09-10T07:37:21.641 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:37:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.639+0000 7f06d6600640 1 -- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f06b40a2b80 msgr2=0x7f06b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.639+0000 7f06d6600640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f06b40a2b80 0x7f06b40a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009ca0 tx=0x7f06c802f370 comp rx=0 tx=0).stop 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.639+0000 7f06d6600640 1 -- 172.21.15.46:0/4235649137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0103ff0 msgr2=0x7f06d019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.639+0000 7f06d6600640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0103ff0 0x7f06d019a5c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f06c000e9f0 tx=0x7f06c000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.640+0000 7f06d6600640 1 -- 172.21.15.46:0/4235649137 shutdown_connections 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.640+0000 7f06d6600640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f06b40a2b80 0x7f06b40a5040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.640+0000 7f06d6600640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06d0109120 0x7f06d019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.640+0000 7f06d6600640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d01049a0 0x7f06d019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.640+0000 7f06d6600640 1 --2- 172.21.15.46:0/4235649137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06d0103ff0 0x7f06d019a5c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.640+0000 7f06d6600640 1 -- 172.21.15.46:0/4235649137 >> 172.21.15.46:0/4235649137 conn(0x7f06d00ffa90 msgr2=0x7f06d01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.640+0000 7f06d6600640 1 -- 172.21.15.46:0/4235649137 shutdown_connections 2024-09-10T07:37:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:21.640+0000 7f06d6600640 1 -- 172.21.15.46:0/4235649137 wait complete. 2024-09-10T07:37:22.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:37:22.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:22 smithi181 ceph-mon[35397]: pgmap v421: 129 pgs: 129 active+clean; 393 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 59 MiB/s wr, 332 op/s 2024-09-10T07:37:22.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:22 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:22.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:22 smithi063 ceph-mon[35020]: pgmap v421: 129 pgs: 129 active+clean; 393 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 59 MiB/s wr, 332 op/s 2024-09-10T07:37:22.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:22 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:22 smithi046 ceph-mon[30705]: pgmap v421: 129 pgs: 129 active+clean; 393 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 59 MiB/s wr, 332 op/s 2024-09-10T07:37:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:22 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:23.187 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.913+0000 7f19f3200640 1 -- 172.21.15.46:0/3460330438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4108f60 msgr2=0x7f19f4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.913+0000 7f19f3200640 1 --2- 172.21.15.46:0/3460330438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4108f60 0x7f19f4110ae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ca0 tx=0x7f19ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.913+0000 7f19f3200640 1 -- 172.21.15.46:0/3460330438 shutdown_connections 2024-09-10T07:37:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.913+0000 7f19f3200640 1 --2- 172.21.15.46:0/3460330438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4108f60 0x7f19f4110ae0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.913+0000 7f19f3200640 1 --2- 172.21.15.46:0/3460330438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19f41047e0 0x7f19f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.913+0000 7f19f3200640 1 --2- 172.21.15.46:0/3460330438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f4103e30 0x7f19f4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.913+0000 7f19f3200640 1 -- 172.21.15.46:0/3460330438 >> 172.21.15.46:0/3460330438 conn(0x7f19f40ff910 msgr2=0x7f19f4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:23.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.914+0000 7f19f3200640 1 -- 172.21.15.46:0/3460330438 shutdown_connections 2024-09-10T07:37:23.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.914+0000 7f19f3200640 1 -- 172.21.15.46:0/3460330438 wait complete. 2024-09-10T07:37:23.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.915+0000 7f19f3200640 1 Processor -- start 2024-09-10T07:37:23.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.915+0000 7f19f3200640 1 -- start start 2024-09-10T07:37:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f3200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4103e30 0x7f19f419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19f41047e0 0x7f19f419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f4108f60 0x7f19f419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4114010 con 0x7f19f41047e0 2024-09-10T07:37:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4113e90 con 0x7f19f4108f60 2024-09-10T07:37:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f3200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4114190 con 0x7f19f4103e30 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f4108f60 0x7f19f419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f4108f60 0x7f19f419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34072/0 (socket says 172.21.15.46:34072) 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f2800640 1 -- 172.21.15.46:0/417925885 learned_addr learned my addr 172.21.15.46:0/417925885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.916+0000 7f19f1400640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19f41047e0 0x7f19f419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.917+0000 7f19f1e00640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4103e30 0x7f19f419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.917+0000 7f19f1e00640 1 -- 172.21.15.46:0/417925885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f4108f60 msgr2=0x7f19f419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.917+0000 7f19f1e00640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f4108f60 0x7f19f419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.917+0000 7f19f1e00640 1 -- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19f41047e0 msgr2=0x7f19f419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.917+0000 7f19f1e00640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19f41047e0 0x7f19f419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.917+0000 7f19f1e00640 1 -- 172.21.15.46:0/417925885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f419f4a0 con 0x7f19f4103e30 2024-09-10T07:37:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.918+0000 7f19f1e00640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4103e30 0x7f19f419a4f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f19e400d6d0 tx=0x7f19e400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.918+0000 7f19eaa00640 1 -- 172.21.15.46:0/417925885 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4017070 con 0x7f19f4103e30 2024-09-10T07:37:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.918+0000 7f19eaa00640 1 -- 172.21.15.46:0/417925885 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e4009820 con 0x7f19f4103e30 2024-09-10T07:37:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.918+0000 7f19f3200640 1 -- 172.21.15.46:0/417925885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f419f790 con 0x7f19f4103e30 2024-09-10T07:37:23.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.918+0000 7f19eaa00640 1 -- 172.21.15.46:0/417925885 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4004d00 con 0x7f19f4103e30 2024-09-10T07:37:23.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.918+0000 7f19f3200640 1 -- 172.21.15.46:0/417925885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f41a7070 con 0x7f19f4103e30 2024-09-10T07:37:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.923+0000 7f19fb88c740 1 Processor -- start 2024-09-10T07:37:23.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.927+0000 7f19eaa00640 1 -- 172.21.15.46:0/417925885 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19e40099d0 con 0x7f19f4103e30 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.927+0000 7f19fb88c740 1 -- start start 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.927+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.927+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x5614299ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.928+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299ef830 0x5614299f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.928+0000 7f19f2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.928+0000 7f19f2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40702/0 (socket says 172.21.15.46:40702) 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.928+0000 7f19f2800640 1 -- 172.21.15.46:0/1858960 learned_addr learned my addr 172.21.15.46:0/1858960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.928+0000 7f19f1400640 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299ef830 0x5614299f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.928+0000 7f19fb88c740 1 -- 172.21.15.46:0/1858960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614299423e0 con 0x5614299e4f70 2024-09-10T07:37:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.928+0000 7f19fb88c740 1 -- 172.21.15.46:0/1858960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5614298e9d80 con 0x5614299ef830 2024-09-10T07:37:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.928+0000 7f19fb88c740 1 -- 172.21.15.46:0/1858960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56142993e3a0 con 0x5614299e4650 2024-09-10T07:37:23.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.929+0000 7f19f1e00640 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x5614299ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.929+0000 7f19f1400640 1 -- 172.21.15.46:0/1858960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 msgr2=0x5614299e4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:23.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.929+0000 7f19f1400640 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299e4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.929+0000 7f19f1400640 1 -- 172.21.15.46:0/1858960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 msgr2=0x5614299ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:23.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.929+0000 7f19f1400640 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x5614299ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.929+0000 7f19f1400640 1 -- 172.21.15.46:0/1858960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561429908ae0 con 0x5614299ef830 2024-09-10T07:37:23.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.929+0000 7f19eaa00640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f19d80a2c00 0x7f19d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.930+0000 7f19f1400640 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299ef830 0x5614299f1c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009bf0 tx=0x7f19e0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7530473d95fef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:23.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.930+0000 7f19eaa00640 1 -- 172.21.15.46:0/417925885 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f19e40cec00 con 0x7f19f4103e30 2024-09-10T07:37:23.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.930+0000 7f19df400640 1 -- 172.21.15.46:0/1858960 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004310 con 0x5614299ef830 2024-09-10T07:37:23.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.930+0000 7f19f2800640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f19d80a2c00 0x7f19d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.930+0000 7f19df400640 1 -- 172.21.15.46:0/1858960 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e00044b0 con 0x5614299ef830 2024-09-10T07:37:23.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.931+0000 7f19fb88c740 1 -- 172.21.15.46:0/1858960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299ef830 msgr2=0x5614299f1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.931+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299ef830 0x5614299f1c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009bf0 tx=0x7f19e0011440 comp rx=0 tx=0).stop 2024-09-10T07:37:23.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.932+0000 7f19fb88c740 1 -- 172.21.15.46:0/1858960 shutdown_connections 2024-09-10T07:37:23.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.932+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299ef830 0x5614299f1c20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.932+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x5614299ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.932+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1858960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299e4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.932+0000 7f19fb88c740 1 -- 172.21.15.46:0/1858960 >> 172.21.15.46:0/1858960 conn(0x5614299e0be0 msgr2=0x5614299e0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:23.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.933+0000 7f19fb88c740 1 -- 172.21.15.46:0/1858960 shutdown_connections 2024-09-10T07:37:23.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.933+0000 7f19f2800640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f19d80a2c00 0x7f19d80a50c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f19ec02f430 tx=0x7f19ec005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:23.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.933+0000 7f19fb88c740 1 -- 172.21.15.46:0/1858960 wait complete. 2024-09-10T07:37:23.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.934+0000 7f19fb88c740 1 -- start start 2024-09-10T07:37:23.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.934+0000 7f19fb88c740 1 Processor -- start 2024-09-10T07:37:23.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.935+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ef2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.935+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4f70 0x5614299ec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.935+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.935+0000 7f19fb88c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614299f24a0 con 0x5614299ef830 2024-09-10T07:37:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.936+0000 7f19fb88c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5614298fd0f0 con 0x5614299e4650 2024-09-10T07:37:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.936+0000 7f19fb88c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5614299f2620 con 0x5614299e4f70 2024-09-10T07:37:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.936+0000 7f19fb88c740 10 client.0 _ll_register_callbacks cb 0x5614298e4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.936+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ef2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.937+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ef2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34086/0 (socket says 172.21.15.46:34086) 2024-09-10T07:37:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.937+0000 7f19f1e00640 1 -- 172.21.15.46:0/1199449162 learned_addr learned my addr 172.21.15.46:0/1199449162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.937+0000 7f19f1400640 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4f70 0x5614299ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.938+0000 7f19f2800640 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.939+0000 7f19f1e00640 1 -- 172.21.15.46:0/1199449162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4f70 msgr2=0x5614299ec560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.939+0000 7f19f1e00640 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4f70 0x5614299ec560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.939+0000 7f19f1e00640 1 -- 172.21.15.46:0/1199449162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 msgr2=0x5614299ecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.939+0000 7f19f1e00640 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299ecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.939+0000 7f19f1e00640 1 -- 172.21.15.46:0/1199449162 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614296b33c0 con 0x5614299e4650 2024-09-10T07:37:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.939+0000 7f19f2800640 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:23.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.940+0000 7f19f1e00640 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ef2b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f19e40975f0 tx=0x7f19e40974f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca8796afed7462a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:23.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.940+0000 7f19d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.941+0000 7f19d3e00640 1 -- 172.21.15.46:0/1199449162 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4020070 con 0x5614299e4650 2024-09-10T07:37:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.941+0000 7f19d3e00640 1 -- 172.21.15.46:0/1199449162 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f19e40bb0b0 con 0x5614299e4650 2024-09-10T07:37:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.941+0000 7f19fb88c740 1 -- 172.21.15.46:0/1199449162 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e00044b0 con 0x5614299e4650 2024-09-10T07:37:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.941+0000 7f19d3e00640 1 -- 172.21.15.46:0/1199449162 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19e409b190 con 0x5614299e4650 2024-09-10T07:37:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.941+0000 7f19fb88c740 10 client.25538 fetch_fsmap learned FSMap version 31 2024-09-10T07:37:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.941+0000 7f19fb88c740 1 -- 172.21.15.46:0/1199449162 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5614299fe370 con 0x5614299e4650 2024-09-10T07:37:23.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.942+0000 7f19d3e00640 1 -- 172.21.15.46:0/1199449162 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e00044b0 con 0x5614299e4650 2024-09-10T07:37:23.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.943+0000 7f19d3e00640 1 -- 172.21.15.46:0/1199449162 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f19e4097800 con 0x5614299e4650 2024-09-10T07:37:23.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.943+0000 7f19fb88c740 10 client.25538 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T07:37:23.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.943+0000 7f19fb88c740 10 client.25538 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:37:23.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.943+0000 7f19fb88c740 20 client.25538 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:23.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.944+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5614299ff830 0x561429a01ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.944+0000 7f19fb88c740 4 client.25538 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.944+0000 7f19fb88c740 1 -- 172.21.15.46:0/1199449162 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56142994f950 con 0x5614299ff830 2024-09-10T07:37:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.945+0000 7f19f1400640 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5614299ff830 0x561429a01ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.946+0000 7f19f1400640 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5614299ff830 0x561429a01ca0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79b4a66c9f63a798 server_cookie=16dbd4fa1987d75d in_seq=0 out_seq=0 2024-09-10T07:37:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.946+0000 7f19d3e00640 10 client.25538 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:37:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.946+0000 7f19d3e00640 1 -- 172.21.15.46:0/1199449162 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56142994f950 con 0x5614299ff830 2024-09-10T07:37:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.946+0000 7f19d3e00640 10 client.25538 handle_command_reply: tid=0 2024-09-10T07:37:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 client.25538 shutdown 2024-09-10T07:37:23.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 -- 172.21.15.46:0/1199449162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 msgr2=0x5614299ef2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ef2b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f19e40975f0 tx=0x7f19e40974f0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 -- 172.21.15.46:0/1199449162 shutdown_connections 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5614299ff830 0x561429a01ca0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4f70 0x5614299ec560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1199449162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ef2b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19d3e00640 0 client.25538 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.947+0000 7f19fb88c740 1 -- 172.21.15.46:0/1199449162 >> 172.21.15.46:0/1199449162 conn(0x5614299e0be0 msgr2=0x5614299e6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.948+0000 7f19fb88c740 1 -- 172.21.15.46:0/1199449162 shutdown_connections 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.948+0000 7f19fb88c740 1 -- 172.21.15.46:0/1199449162 wait complete. 2024-09-10T07:37:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.948+0000 7f19fb88c740 20 client.25538 trim_cache size 0 max 16384 2024-09-10T07:37:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.973+0000 7f19fb88c740 1 Processor -- start 2024-09-10T07:37:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.973+0000 7f19fb88c740 1 -- start start 2024-09-10T07:37:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.974+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.974+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x561429a07570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.974+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299ef830 0x561429a07ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.974+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.974+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ebe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34088/0 (socket says 172.21.15.46:34088) 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.974+0000 7f19f1e00640 1 -- 172.21.15.46:0/1999517933 learned_addr learned my addr 172.21.15.46:0/1999517933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.974+0000 7f19fb88c740 1 -- 172.21.15.46:0/1999517933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41ac950 con 0x5614299e4f70 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19fb88c740 1 -- 172.21.15.46:0/1999517933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41ae4c0 con 0x5614299e4650 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19fb88c740 1 -- 172.21.15.46:0/1999517933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41ad020 con 0x5614299ef830 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f1400640 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x561429a07570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f1e00640 1 -- 172.21.15.46:0/1999517933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299ef830 msgr2=0x561429a07ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f2800640 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299ef830 0x561429a07ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f1e00640 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299ef830 0x561429a07ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f1e00640 1 -- 172.21.15.46:0/1999517933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 msgr2=0x561429a07570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f1e00640 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x561429a07570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f1e00640 1 -- 172.21.15.46:0/1999517933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614299e0a10 con 0x5614299e4650 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f2800640 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299ef830 0x561429a07ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f1400640 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x561429a07570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.975+0000 7f19f1e00640 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ebe20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19e409ae90 tx=0x7f19e40d25e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebeb35868e651082 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.976+0000 7f19dea00640 1 -- 172.21.15.46:0/1999517933 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4020040 con 0x5614299e4650 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.976+0000 7f19dea00640 1 -- 172.21.15.46:0/1999517933 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e409f490 con 0x5614299e4650 2024-09-10T07:37:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.976+0000 7f19dea00640 1 -- 172.21.15.46:0/1999517933 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e409a6b0 con 0x5614299e4650 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.977+0000 7f19fb88c740 1 -- 172.21.15.46:0/1999517933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 msgr2=0x5614299ebe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.977+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ebe20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19e409ae90 tx=0x7f19e40d25e0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.977+0000 7f19fb88c740 1 -- 172.21.15.46:0/1999517933 shutdown_connections 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.977+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299ef830 0x561429a07ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.977+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299e4f70 0x561429a07570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.978+0000 7f19fb88c740 1 --2- 172.21.15.46:0/1999517933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4650 0x5614299ebe20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.978+0000 7f19fb88c740 1 -- 172.21.15.46:0/1999517933 >> 172.21.15.46:0/1999517933 conn(0x5614299e0be0 msgr2=0x5614299e0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.978+0000 7f19fb88c740 1 -- 172.21.15.46:0/1999517933 shutdown_connections 2024-09-10T07:37:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.978+0000 7f19fb88c740 1 -- 172.21.15.46:0/1999517933 wait complete. 2024-09-10T07:37:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.979+0000 7f19fb88c740 1 -- start start 2024-09-10T07:37:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.980+0000 7f19fb88c740 1 Processor -- start 2024-09-10T07:37:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.981+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.981+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4f70 0x5614299f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.981+0000 7f19fb88c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.981+0000 7f19fb88c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41addc0 con 0x5614299ef830 2024-09-10T07:37:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.981+0000 7f19fb88c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41ad6f0 con 0x5614299e4f70 2024-09-10T07:37:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.981+0000 7f19fb88c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41aa950 con 0x5614299e4650 2024-09-10T07:37:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.981+0000 7f19fb88c740 10 client.0 _ll_register_callbacks cb 0x7f19e40057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.982+0000 7f19f1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4f70 0x5614299f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.982+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.982+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299f0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40724/0 (socket says 172.21.15.46:40724) 2024-09-10T07:37:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.982+0000 7f19f1e00640 1 -- 172.21.15.46:0/2695019813 learned_addr learned my addr 172.21.15.46:0/2695019813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.982+0000 7f19f1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4f70 0x5614299f08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34104/0 (socket says 172.21.15.46:34104) 2024-09-10T07:37:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.983+0000 7f19f2800640 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.983+0000 7f19f1e00640 1 -- 172.21.15.46:0/2695019813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4f70 msgr2=0x5614299f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.983+0000 7f19f1e00640 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4f70 0x5614299f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.983+0000 7f19f1e00640 1 -- 172.21.15.46:0/2695019813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 msgr2=0x5614299f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.983+0000 7f19f1e00640 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.983+0000 7f19f1e00640 1 -- 172.21.15.46:0/2695019813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614299f6e00 con 0x5614299e4650 2024-09-10T07:37:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.985+0000 7f19d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.985+0000 7f19f1e00640 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299f0390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009610 tx=0x7f19e40a3e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a04b2355c8cfa8d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.986+0000 7f19d3e00640 1 -- 172.21.15.46:0/2695019813 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4020070 con 0x5614299e4650 2024-09-10T07:37:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.986+0000 7f19d3e00640 1 -- 172.21.15.46:0/2695019813 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f19e409a660 con 0x5614299e4650 2024-09-10T07:37:23.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.986+0000 7f19fb88c740 1 -- 172.21.15.46:0/2695019813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e409f490 con 0x5614299e4650 2024-09-10T07:37:23.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.987+0000 7f19d3e00640 1 -- 172.21.15.46:0/2695019813 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19e409a860 con 0x5614299e4650 2024-09-10T07:37:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.987+0000 7f19fb88c740 10 client.25546 fetch_fsmap learned FSMap version 31 2024-09-10T07:37:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.987+0000 7f19fb88c740 1 -- 172.21.15.46:0/2695019813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5614299f6e00 con 0x5614299e4650 2024-09-10T07:37:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.988+0000 7f19d3e00640 1 -- 172.21.15.46:0/2695019813 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e409f490 con 0x5614299e4650 2024-09-10T07:37:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.989+0000 7f19d3e00640 1 -- 172.21.15.46:0/2695019813 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f19e409a1e0 con 0x5614299e4650 2024-09-10T07:37:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.989+0000 7f19fb88c740 10 client.25546 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T07:37:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.989+0000 7f19fb88c740 10 client.25546 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:37:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.989+0000 7f19fb88c740 20 client.25546 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.989+0000 7f19fb88c740 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5614299f8320 0x5614299d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.989+0000 7f19fb88c740 4 client.25546 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:37:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.989+0000 7f19fb88c740 1 -- 172.21.15.46:0/2695019813 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561429949e60 con 0x5614299f8320 2024-09-10T07:37:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.990+0000 7f19f1400640 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5614299f8320 0x5614299d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.991+0000 7f19f1400640 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5614299f8320 0x5614299d9db0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef9e9fef6fb591a8 server_cookie=13c7033a8e2e3459 in_seq=0 out_seq=0 2024-09-10T07:37:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.991+0000 7f19d3e00640 10 client.25546 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:37:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.991+0000 7f19d3e00640 1 -- 172.21.15.46:0/2695019813 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561429949e60 con 0x5614299f8320 2024-09-10T07:37:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.991+0000 7f19d3e00640 10 client.25546 handle_command_reply: tid=0 2024-09-10T07:37:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.991+0000 7f19fb88c740 1 client.25546 shutdown 2024-09-10T07:37:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.992+0000 7f19fb88c740 1 -- 172.21.15.46:0/2695019813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 msgr2=0x5614299f0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.992+0000 7f19fb88c740 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299f0390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009610 tx=0x7f19e40a3e30 comp rx=0 tx=0).stop 2024-09-10T07:37:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19fb88c740 1 -- 172.21.15.46:0/2695019813 shutdown_connections 2024-09-10T07:37:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19fb88c740 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614299ef830 0x5614299f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19fb88c740 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614299e4f70 0x5614299f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19fb88c740 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5614299f8320 0x5614299d9db0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19fb88c740 1 --2- 172.21.15.46:0/2695019813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614299e4650 0x5614299f0390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19fb88c740 1 -- 172.21.15.46:0/2695019813 >> 172.21.15.46:0/2695019813 conn(0x5614299e0be0 msgr2=0x5614299ead00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19d3e00640 0 client.25546 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:37:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19fb88c740 1 -- 172.21.15.46:0/2695019813 shutdown_connections 2024-09-10T07:37:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.993+0000 7f19fb88c740 1 -- 172.21.15.46:0/2695019813 wait complete. 2024-09-10T07:37:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.994+0000 7f19fb88c740 20 client.25546 trim_cache size 0 max 16384 2024-09-10T07:37:23.994 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:37:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 -- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f19d80a2c00 msgr2=0x7f19d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f19d80a2c00 0x7f19d80a50c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f19ec02f430 tx=0x7f19ec005d80 comp rx=0 tx=0).stop 2024-09-10T07:37:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 -- 172.21.15.46:0/417925885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4103e30 msgr2=0x7f19f419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4103e30 0x7f19f419a4f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f19e400d6d0 tx=0x7f19e400db90 comp rx=0 tx=0).stop 2024-09-10T07:37:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 -- 172.21.15.46:0/417925885 shutdown_connections 2024-09-10T07:37:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f4108f60 0x7f19f419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f19d80a2c00 0x7f19d80a50c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19f41047e0 0x7f19f419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 --2- 172.21.15.46:0/417925885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f4103e30 0x7f19f419a4f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 -- 172.21.15.46:0/417925885 >> 172.21.15.46:0/417925885 conn(0x7f19f40ff910 msgr2=0x7f19f4101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 -- 172.21.15.46:0/417925885 shutdown_connections 2024-09-10T07:37:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:23.996+0000 7f19f3200640 1 -- 172.21.15.46:0/417925885 wait complete. 2024-09-10T07:37:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:24 smithi181 ceph-mon[35397]: pgmap v422: 129 pgs: 129 active+clean; 469 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 68 MiB/s wr, 150 op/s 2024-09-10T07:37:24.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:24 smithi063 ceph-mon[35020]: pgmap v422: 129 pgs: 129 active+clean; 469 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 68 MiB/s wr, 150 op/s 2024-09-10T07:37:24.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:24 smithi046 ceph-mon[30705]: pgmap v422: 129 pgs: 129 active+clean; 469 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 68 MiB/s wr, 150 op/s 2024-09-10T07:37:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 5.0 s, 2195.4 tps, lat 42.625 ms stddev 49.341 2024-09-10T07:37:25.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:37:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:26.520 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T07:37:26.520 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T07:37:26.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:37:26.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:26 smithi046 ceph-mon[30705]: pgmap v423: 129 pgs: 129 active+clean; 489 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 64 MiB/s wr, 78 op/s 2024-09-10T07:37:26.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:26 smithi181 ceph-mon[35397]: pgmap v423: 129 pgs: 129 active+clean; 489 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 64 MiB/s wr, 78 op/s 2024-09-10T07:37:26.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:26 smithi063 ceph-mon[35020]: pgmap v423: 129 pgs: 129 active+clean; 489 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 64 MiB/s wr, 78 op/s 2024-09-10T07:37:27.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.216+0000 7fbee9600640 1 -- 172.21.15.46:0/1053919392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee4113120 msgr2=0x7fbee4115510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.216+0000 7fbee9600640 1 --2- 172.21.15.46:0/1053919392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee4113120 0x7fbee4115510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009860 tx=0x7fbed002a3d0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.216+0000 7fbee9600640 1 -- 172.21.15.46:0/1053919392 shutdown_connections 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.216+0000 7fbee9600640 1 --2- 172.21.15.46:0/1053919392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee4113120 0x7fbee4115510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.216+0000 7fbee9600640 1 --2- 172.21.15.46:0/1053919392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee406c220 0x7fbee406c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.216+0000 7fbee9600640 1 --2- 172.21.15.46:0/1053919392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee4073ef0 0x7fbee406bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.216+0000 7fbee9600640 1 -- 172.21.15.46:0/1053919392 >> 172.21.15.46:0/1053919392 conn(0x7fbee4077230 msgr2=0x7fbee40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.216+0000 7fbee9600640 1 -- 172.21.15.46:0/1053919392 shutdown_connections 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.217+0000 7fbee9600640 1 -- 172.21.15.46:0/1053919392 wait complete. 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.218+0000 7fbee9600640 1 Processor -- start 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.218+0000 7fbee9600640 1 -- start start 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.218+0000 7fbee9600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee406c220 0x7fbee41a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.218+0000 7fbee9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee4073ef0 0x7fbee41a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.219+0000 7fbee9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4113120 0x7fbee40704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.219+0000 7fbee9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbee411a7b0 con 0x7fbee4113120 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.219+0000 7fbee9600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbee411a630 con 0x7fbee406c220 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.219+0000 7fbee9600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbee411a930 con 0x7fbee4073ef0 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.219+0000 7fbee2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee4073ef0 0x7fbee41a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.219+0000 7fbee2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee4073ef0 0x7fbee41a3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:52960/0 (socket says 172.21.15.46:52960) 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.219+0000 7fbee2000640 1 -- 172.21.15.46:0/2536777366 learned_addr learned my addr 172.21.15.46:0/2536777366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.219+0000 7fbee2a00640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee406c220 0x7fbee41a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.220+0000 7fbee3400640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4113120 0x7fbee40704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.220+0000 7fbee2000640 1 -- 172.21.15.46:0/2536777366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee406c220 msgr2=0x7fbee41a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.220+0000 7fbee2000640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee406c220 0x7fbee41a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.220+0000 7fbee2000640 1 -- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4113120 msgr2=0x7fbee40704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.220+0000 7fbee2000640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4113120 0x7fbee40704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.220+0000 7fbee2000640 1 -- 172.21.15.46:0/2536777366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee4070da0 con 0x7fbee4073ef0 2024-09-10T07:37:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.221+0000 7fbee2000640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee4073ef0 0x7fbee41a3b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbed400e9c0 tx=0x7fbed400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.221+0000 7fbedb400640 1 -- 172.21.15.46:0/2536777366 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed400cde0 con 0x7fbee4073ef0 2024-09-10T07:37:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.221+0000 7fbedb400640 1 -- 172.21.15.46:0/2536777366 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed40035f0 con 0x7fbee4073ef0 2024-09-10T07:37:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.221+0000 7fbee9600640 1 -- 172.21.15.46:0/2536777366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee4071090 con 0x7fbee4073ef0 2024-09-10T07:37:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.222+0000 7fbee9600640 1 -- 172.21.15.46:0/2536777366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee41b3b20 con 0x7fbee4073ef0 2024-09-10T07:37:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.222+0000 7fbedb400640 1 -- 172.21.15.46:0/2536777366 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4010650 con 0x7fbee4073ef0 2024-09-10T07:37:27.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.226+0000 7fbedb400640 1 -- 172.21.15.46:0/2536777366 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbed4004020 con 0x7fbee4073ef0 2024-09-10T07:37:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.228+0000 7fbedb400640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbec80a2c80 0x7fbec80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.229+0000 7fbedb400640 1 -- 172.21.15.46:0/2536777366 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbed40cf5b0 con 0x7fbee4073ef0 2024-09-10T07:37:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.229+0000 7fbeec000740 1 Processor -- start 2024-09-10T07:37:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.229+0000 7fbee2a00640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbec80a2c80 0x7fbec80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.230+0000 7fbeec000740 1 -- start start 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.230+0000 7fbeec000740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a90fe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.230+0000 7fbeec000740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a90fe590 0x55d5a9108910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.230+0000 7fbeec000740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a9108e50 0x55d5a910b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.231+0000 7fbee2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a90fe590 0x55d5a9108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.231+0000 7fbee3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a9108e50 0x55d5a910b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.231+0000 7fbeec000740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5a90643e0 con 0x55d5a90fdc70 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.231+0000 7fbeec000740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d5a900bd80 con 0x55d5a9108e50 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.231+0000 7fbee2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a90fe590 0x55d5a9108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:52966/0 (socket says 172.21.15.46:52966) 2024-09-10T07:37:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.231+0000 7fbee2000640 1 -- 172.21.15.46:0/2638967454 learned_addr learned my addr 172.21.15.46:0/2638967454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.231+0000 7fbeec000740 1 -- 172.21.15.46:0/2638967454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d5a90603a0 con 0x55d5a90fe590 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.231+0000 7fbee2a00640 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a90fe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee3400640 1 -- 172.21.15.46:0/2638967454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a90fe590 msgr2=0x55d5a9108910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee3400640 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a90fe590 0x55d5a9108910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee3400640 1 -- 172.21.15.46:0/2638967454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 msgr2=0x55d5a90fe050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee3400640 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a90fe050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee3400640 1 -- 172.21.15.46:0/2638967454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a902aae0 con 0x55d5a9108e50 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee2000640 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a90fe590 0x55d5a9108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee2a00640 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a90fe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee3400640 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a9108e50 0x55d5a910b240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbed002a8f0 tx=0x7fbed000b150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6aa03418cd2e21d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbee2a00640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbec80a2c80 0x7fbec80a5140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbedc013750 tx=0x7fbedc013b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbecf400640 1 -- 172.21.15.46:0/2638967454 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0004030 con 0x55d5a9108e50 2024-09-10T07:37:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbecf400640 1 -- 172.21.15.46:0/2638967454 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed0030880 con 0x55d5a9108e50 2024-09-10T07:37:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.232+0000 7fbecf400640 1 -- 172.21.15.46:0/2638967454 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0005600 con 0x55d5a9108e50 2024-09-10T07:37:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.233+0000 7fbeec000740 1 -- 172.21.15.46:0/2638967454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a9108e50 msgr2=0x55d5a910b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.233+0000 7fbeec000740 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a9108e50 0x55d5a910b240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbed002a8f0 tx=0x7fbed000b150 comp rx=0 tx=0).stop 2024-09-10T07:37:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.233+0000 7fbeec000740 1 -- 172.21.15.46:0/2638967454 shutdown_connections 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.233+0000 7fbeec000740 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a9108e50 0x55d5a910b240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.233+0000 7fbeec000740 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a90fe590 0x55d5a9108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.233+0000 7fbeec000740 1 --2- 172.21.15.46:0/2638967454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a90fe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.233+0000 7fbeec000740 1 -- 172.21.15.46:0/2638967454 >> 172.21.15.46:0/2638967454 conn(0x55d5a90fa200 msgr2=0x55d5a90fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.234+0000 7fbeec000740 1 -- 172.21.15.46:0/2638967454 shutdown_connections 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.234+0000 7fbeec000740 1 -- 172.21.15.46:0/2638967454 wait complete. 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.234+0000 7fbeec000740 1 -- start start 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.234+0000 7fbeec000740 1 Processor -- start 2024-09-10T07:37:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.235+0000 7fbeec000740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fdc70 0x55d5a9105c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.235+0000 7fbeec000740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fe590 0x55d5a91061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.235+0000 7fbeec000740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a9106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.235+0000 7fbeec000740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5a910ba20 con 0x55d5a90fe590 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.235+0000 7fbeec000740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d5a901f0f0 con 0x55d5a90fdc70 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.235+0000 7fbeec000740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d5a910bba0 con 0x55d5a9108e50 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.235+0000 7fbeec000740 10 client.0 _ll_register_callbacks cb 0x55d5a9006bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.236+0000 7fbee2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fdc70 0x55d5a9105c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.236+0000 7fbee2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fdc70 0x55d5a9105c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33656/0 (socket says 172.21.15.46:33656) 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.236+0000 7fbee2a00640 1 -- 172.21.15.46:0/2212376278 learned_addr learned my addr 172.21.15.46:0/2212376278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.236+0000 7fbee2a00640 1 -- 172.21.15.46:0/2212376278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 msgr2=0x55d5a9106710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:37:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.236+0000 7fbee2a00640 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a9106710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.236+0000 7fbee2a00640 1 -- 172.21.15.46:0/2212376278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fe590 msgr2=0x55d5a91061d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.236+0000 7fbee2a00640 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fe590 0x55d5a91061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.236+0000 7fbee2a00640 1 -- 172.21.15.46:0/2212376278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a8dd53c0 con 0x55d5a90fdc70 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.237+0000 7fbee2a00640 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fdc70 0x55d5a9105c90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00fd40 tx=0x7fbedc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91654a97dc75b89f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.237+0000 7fbec3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.237+0000 7fbec3e00640 1 -- 172.21.15.46:0/2212376278 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc01f640 con 0x55d5a90fdc70 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.237+0000 7fbec3e00640 1 -- 172.21.15.46:0/2212376278 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbedc01f7e0 con 0x55d5a90fdc70 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.237+0000 7fbeec000740 1 -- 172.21.15.46:0/2212376278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed0030880 con 0x55d5a90fdc70 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.238+0000 7fbec3e00640 1 -- 172.21.15.46:0/2212376278 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbedc02d050 con 0x55d5a90fdc70 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.238+0000 7fbeec000740 10 client.25556 fetch_fsmap learned FSMap version 32 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.238+0000 7fbeec000740 1 -- 172.21.15.46:0/2212376278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d5a91179b0 con 0x55d5a90fdc70 2024-09-10T07:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.239+0000 7fbec3e00640 1 -- 172.21.15.46:0/2212376278 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed0030880 con 0x55d5a90fdc70 2024-09-10T07:37:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.239+0000 7fbec3e00640 1 -- 172.21.15.46:0/2212376278 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fbedc028b80 con 0x55d5a90fdc70 2024-09-10T07:37:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.239+0000 7fbeec000740 10 client.25556 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T07:37:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.239+0000 7fbeec000740 10 client.25556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:37:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.239+0000 7fbeec000740 20 client.25556 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.239+0000 7fbeec000740 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55d5a9118f80 0x55d5a911b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.239+0000 7fbeec000740 4 client.25556 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.239+0000 7fbeec000740 1 -- 172.21.15.46:0/2212376278 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5a9071950 con 0x55d5a9118f80 2024-09-10T07:37:27.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.240+0000 7fbee2000640 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55d5a9118f80 0x55d5a911b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.241+0000 7fbee2000640 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55d5a9118f80 0x55d5a911b420 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=859c3c2991f4f517 server_cookie=f5aa90bdcb62df0d in_seq=0 out_seq=0 2024-09-10T07:37:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.241+0000 7fbec3e00640 10 client.25556 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:37:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.242+0000 7fbec3e00640 1 -- 172.21.15.46:0/2212376278 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5a9071950 con 0x55d5a9118f80 2024-09-10T07:37:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.242+0000 7fbec3e00640 10 client.25556 handle_command_reply: tid=0 2024-09-10T07:37:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.242+0000 7fbeec000740 1 client.25556 shutdown 2024-09-10T07:37:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 -- 172.21.15.46:0/2212376278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fdc70 msgr2=0x55d5a9105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fdc70 0x55d5a9105c90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00fd40 tx=0x7fbedc004ab0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 -- 172.21.15.46:0/2212376278 shutdown_connections 2024-09-10T07:37:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55d5a9118f80 0x55d5a911b420 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a9106710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbec3e00640 0 client.25556 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:37:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fe590 0x55d5a91061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 --2- 172.21.15.46:0/2212376278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fdc70 0x55d5a9105c90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 -- 172.21.15.46:0/2212376278 >> 172.21.15.46:0/2212376278 conn(0x55d5a90fa200 msgr2=0x55d5a90fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 -- 172.21.15.46:0/2212376278 shutdown_connections 2024-09-10T07:37:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.243+0000 7fbeec000740 1 -- 172.21.15.46:0/2212376278 wait complete. 2024-09-10T07:37:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.244+0000 7fbeec000740 20 client.25556 trim_cache size 0 max 16384 2024-09-10T07:37:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.270+0000 7fbeec000740 1 Processor -- start 2024-09-10T07:37:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.270+0000 7fbeec000740 1 -- start start 2024-09-10T07:37:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.270+0000 7fbeec000740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a9120d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.270+0000 7fbeec000740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9121270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.270+0000 7fbeec000740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a9102f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.270+0000 7fbeec000740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbec400fb20 con 0x55d5a90fdc70 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.270+0000 7fbeec000740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbec40116c0 con 0x55d5a90fe590 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.270+0000 7fbeec000740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbec40101f0 con 0x55d5a9108e50 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.271+0000 7fbee2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9121270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.271+0000 7fbee2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9121270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33660/0 (socket says 172.21.15.46:33660) 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.271+0000 7fbee2000640 1 -- 172.21.15.46:0/140877394 learned_addr learned my addr 172.21.15.46:0/140877394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.271+0000 7fbee2000640 1 -- 172.21.15.46:0/140877394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 msgr2=0x55d5a9102f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.271+0000 7fbee2000640 1 --2- 172.21.15.46:0/140877394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a9102f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.271+0000 7fbee2000640 1 -- 172.21.15.46:0/140877394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 msgr2=0x55d5a9120d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.271+0000 7fbee2000640 1 --2- 172.21.15.46:0/140877394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a9120d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.271+0000 7fbee2000640 1 -- 172.21.15.46:0/140877394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a90f5770 con 0x55d5a90fe590 2024-09-10T07:37:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.272+0000 7fbee2000640 1 --2- 172.21.15.46:0/140877394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9121270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbed4098400 tx=0x7fbed40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f9caa02ce490ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.272+0000 7fbecea00640 1 -- 172.21.15.46:0/140877394 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed40b3070 con 0x55d5a90fe590 2024-09-10T07:37:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.272+0000 7fbecea00640 1 -- 172.21.15.46:0/140877394 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed4099d30 con 0x55d5a90fe590 2024-09-10T07:37:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.272+0000 7fbecea00640 1 -- 172.21.15.46:0/140877394 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed409ae80 con 0x55d5a90fe590 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.273+0000 7fbeec000740 1 -- 172.21.15.46:0/140877394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 msgr2=0x55d5a9121270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.273+0000 7fbeec000740 1 --2- 172.21.15.46:0/140877394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9121270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbed4098400 tx=0x7fbed40046c0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.273+0000 7fbeec000740 1 -- 172.21.15.46:0/140877394 shutdown_connections 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.273+0000 7fbeec000740 1 --2- 172.21.15.46:0/140877394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a9102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.273+0000 7fbeec000740 1 --2- 172.21.15.46:0/140877394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9121270 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.273+0000 7fbeec000740 1 --2- 172.21.15.46:0/140877394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a9120d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.273+0000 7fbeec000740 1 -- 172.21.15.46:0/140877394 >> 172.21.15.46:0/140877394 conn(0x55d5a90fa200 msgr2=0x55d5a90f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.274+0000 7fbeec000740 1 -- 172.21.15.46:0/140877394 shutdown_connections 2024-09-10T07:37:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.274+0000 7fbeec000740 1 -- 172.21.15.46:0/140877394 wait complete. 2024-09-10T07:37:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.275+0000 7fbeec000740 1 -- start start 2024-09-10T07:37:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.275+0000 7fbeec000740 1 Processor -- start 2024-09-10T07:37:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.276+0000 7fbeec000740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a9110e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.276+0000 7fbeec000740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9111390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.276+0000 7fbeec000740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a910de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.276+0000 7fbeec000740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbec4010f90 con 0x55d5a90fdc70 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.276+0000 7fbeec000740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbec40108c0 con 0x55d5a90fe590 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.276+0000 7fbeec000740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbec400db00 con 0x55d5a9108e50 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.276+0000 7fbeec000740 10 client.0 _ll_register_callbacks cb 0x7fbedc007b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.277+0000 7fbee3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a910de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.277+0000 7fbee3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a910de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:52994/0 (socket says 172.21.15.46:52994) 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.277+0000 7fbee3400640 1 -- 172.21.15.46:0/321552842 learned_addr learned my addr 172.21.15.46:0/321552842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.277+0000 7fbee2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9111390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.277+0000 7fbee3400640 1 -- 172.21.15.46:0/321552842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 msgr2=0x55d5a9111390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.278+0000 7fbee3400640 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9111390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.278+0000 7fbee3400640 1 -- 172.21.15.46:0/321552842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 msgr2=0x55d5a9110e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.278+0000 7fbee3400640 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a9110e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.278+0000 7fbee3400640 1 -- 172.21.15.46:0/321552842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a90f9a90 con 0x55d5a9108e50 2024-09-10T07:37:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.278+0000 7fbee2000640 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9111390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.279+0000 7fbee3400640 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a910de20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009780 tx=0x7fbed0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744447a98c6a3300 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.279+0000 7fbec3e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.279+0000 7fbec3e00640 1 -- 172.21.15.46:0/321552842 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed00305a0 con 0x55d5a9108e50 2024-09-10T07:37:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.279+0000 7fbec3e00640 1 -- 172.21.15.46:0/321552842 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbed0030740 con 0x55d5a9108e50 2024-09-10T07:37:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.279+0000 7fbeec000740 1 -- 172.21.15.46:0/321552842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed4099d30 con 0x55d5a9108e50 2024-09-10T07:37:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.281+0000 7fbec3e00640 1 -- 172.21.15.46:0/321552842 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbed0055050 con 0x55d5a9108e50 2024-09-10T07:37:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.281+0000 7fbeec000740 10 client.25552 fetch_fsmap learned FSMap version 32 2024-09-10T07:37:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.281+0000 7fbeec000740 1 -- 172.21.15.46:0/321552842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d5a9183fb0 con 0x55d5a9108e50 2024-09-10T07:37:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.282+0000 7fbec3e00640 1 -- 172.21.15.46:0/321552842 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed0005510 con 0x55d5a9108e50 2024-09-10T07:37:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.282+0000 7fbec3e00640 1 -- 172.21.15.46:0/321552842 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fbed0004cc0 con 0x55d5a9108e50 2024-09-10T07:37:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.282+0000 7fbeec000740 10 client.25552 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T07:37:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.282+0000 7fbeec000740 10 client.25552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:37:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.282+0000 7fbeec000740 20 client.25552 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.282+0000 7fbeec000740 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55d5a91846d0 0x55d5a9186af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.282+0000 7fbeec000740 4 client.25552 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:37:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.282+0000 7fbeec000740 1 -- 172.21.15.46:0/321552842 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbed4099d30 con 0x55d5a91846d0 2024-09-10T07:37:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.285+0000 7fbee2a00640 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55d5a91846d0 0x55d5a9186af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.286+0000 7fbee2a00640 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55d5a91846d0 0x55d5a9186af0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60fed450704ffadc server_cookie=1c3901e3547254dd in_seq=0 out_seq=0 2024-09-10T07:37:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.286+0000 7fbec3e00640 10 client.25552 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:37:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.286+0000 7fbec3e00640 1 -- 172.21.15.46:0/321552842 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbed4099d30 con 0x55d5a91846d0 2024-09-10T07:37:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.286+0000 7fbec3e00640 10 client.25552 handle_command_reply: tid=0 2024-09-10T07:37:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.286+0000 7fbeec000740 1 client.25552 shutdown 2024-09-10T07:37:27.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbeec000740 1 -- 172.21.15.46:0/321552842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 msgr2=0x55d5a910de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbeec000740 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a910de20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009780 tx=0x7fbed0004270 comp rx=0 tx=0).stop 2024-09-10T07:37:27.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbeec000740 1 -- 172.21.15.46:0/321552842 shutdown_connections 2024-09-10T07:37:27.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbeec000740 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55d5a91846d0 0x55d5a9186af0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbec3e00640 0 client.25552 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:37:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbeec000740 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5a9108e50 0x55d5a910de20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbeec000740 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5a90fe590 0x55d5a9111390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbeec000740 1 --2- 172.21.15.46:0/321552842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5a90fdc70 0x55d5a9110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.287+0000 7fbeec000740 1 -- 172.21.15.46:0/321552842 >> 172.21.15.46:0/321552842 conn(0x55d5a90fa200 msgr2=0x55d5a90ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.288+0000 7fbeec000740 1 -- 172.21.15.46:0/321552842 shutdown_connections 2024-09-10T07:37:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.288+0000 7fbeec000740 1 -- 172.21.15.46:0/321552842 wait complete. 2024-09-10T07:37:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.288+0000 7fbeec000740 20 client.25552 trim_cache size 0 max 16384 2024-09-10T07:37:27.289 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:37:27.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.290+0000 7fbee9600640 1 -- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbec80a2c80 msgr2=0x7fbec80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.290+0000 7fbee9600640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbec80a2c80 0x7fbec80a5140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbedc013750 tx=0x7fbedc013b50 comp rx=0 tx=0).stop 2024-09-10T07:37:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.290+0000 7fbee9600640 1 -- 172.21.15.46:0/2536777366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee4073ef0 msgr2=0x7fbee41a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.290+0000 7fbee9600640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee4073ef0 0x7fbee41a3b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbed400e9c0 tx=0x7fbed400ee80 comp rx=0 tx=0).stop 2024-09-10T07:37:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.291+0000 7fbee9600640 1 -- 172.21.15.46:0/2536777366 shutdown_connections 2024-09-10T07:37:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.291+0000 7fbee9600640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbec80a2c80 0x7fbec80a5140 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.291+0000 7fbee9600640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4113120 0x7fbee40704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.291+0000 7fbee9600640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee4073ef0 0x7fbee41a3b00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.291+0000 7fbee9600640 1 --2- 172.21.15.46:0/2536777366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee406c220 0x7fbee41a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.291+0000 7fbee9600640 1 -- 172.21.15.46:0/2536777366 >> 172.21.15.46:0/2536777366 conn(0x7fbee4077230 msgr2=0x7fbee4071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:27.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.292+0000 7fbee9600640 1 -- 172.21.15.46:0/2536777366 shutdown_connections 2024-09-10T07:37:27.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:27.292+0000 7fbee9600640 1 -- 172.21.15.46:0/2536777366 wait complete. 2024-09-10T07:37:27.317 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:27.881 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:27 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:27.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:27 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:27.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:27 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:28.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:37:28.489 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:28 smithi046 ceph-mon[30705]: pgmap v424: 129 pgs: 129 active+clean; 606 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 53 MiB/s wr, 193 op/s 2024-09-10T07:37:28.801 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:28 smithi063 ceph-mon[35020]: pgmap v424: 129 pgs: 129 active+clean; 606 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 53 MiB/s wr, 193 op/s 2024-09-10T07:37:28.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:28 smithi181 ceph-mon[35397]: pgmap v424: 129 pgs: 129 active+clean; 606 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 53 MiB/s wr, 193 op/s 2024-09-10T07:37:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.899+0000 7f39f7600640 1 -- 172.21.15.46:0/2657007496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 msgr2=0x7f39f0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.899+0000 7f39f7600640 1 --2- 172.21.15.46:0/2657007496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 0x7f39f0100ee0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009930 tx=0x7f39e402a440 comp rx=0 tx=0).stop 2024-09-10T07:37:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.900+0000 7f39f7600640 1 -- 172.21.15.46:0/2657007496 shutdown_connections 2024-09-10T07:37:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.900+0000 7f39f7600640 1 --2- 172.21.15.46:0/2657007496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f0101420 0x7f39f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.900+0000 7f39f7600640 1 --2- 172.21.15.46:0/2657007496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 0x7f39f0100ee0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.900+0000 7f39f7600640 1 --2- 172.21.15.46:0/2657007496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39f0106a80 0x7f39f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.900+0000 7f39f7600640 1 -- 172.21.15.46:0/2657007496 >> 172.21.15.46:0/2657007496 conn(0x7f39f00fc7a0 msgr2=0x7f39f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.900+0000 7f39f7600640 1 -- 172.21.15.46:0/2657007496 shutdown_connections 2024-09-10T07:37:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.900+0000 7f39f7600640 1 -- 172.21.15.46:0/2657007496 wait complete. 2024-09-10T07:37:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.902+0000 7f39f7600640 1 Processor -- start 2024-09-10T07:37:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.902+0000 7f39f7600640 1 -- start start 2024-09-10T07:37:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39f7600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 0x7f39f01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39f7600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39f0101420 0x7f39f0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39f7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f0106a80 0x7f39f019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39f7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0110ec0 con 0x7f39f0106a80 2024-09-10T07:37:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39f7600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0110d40 con 0x7f39f0101420 2024-09-10T07:37:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39f7600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0111040 con 0x7f39f0100a80 2024-09-10T07:37:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39f0101420 0x7f39f0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39f0101420 0x7f39f0198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33692/0 (socket says 172.21.15.46:33692) 2024-09-10T07:37:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39efe00640 1 -- 172.21.15.46:0/935952191 learned_addr learned my addr 172.21.15.46:0/935952191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.903+0000 7f39f4c00640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 0x7f39f01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.905+0000 7f39f4c00640 1 -- 172.21.15.46:0/935952191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39f0101420 msgr2=0x7f39f0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.905+0000 7f39f4c00640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39f0101420 0x7f39f0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.905+0000 7f39f4c00640 1 -- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f0106a80 msgr2=0x7f39f019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.905+0000 7f39f4c00640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f0106a80 0x7f39f019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.905+0000 7f39f4c00640 1 -- 172.21.15.46:0/935952191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f019d4c0 con 0x7f39f0100a80 2024-09-10T07:37:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.906+0000 7f39f4c00640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 0x7f39f01984f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f39e000f780 tx=0x7f39e000fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.907+0000 7f39ed600640 1 -- 172.21.15.46:0/935952191 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0010780 con 0x7f39f0100a80 2024-09-10T07:37:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.907+0000 7f39ed600640 1 -- 172.21.15.46:0/935952191 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e0010e00 con 0x7f39f0100a80 2024-09-10T07:37:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.907+0000 7f39ed600640 1 -- 172.21.15.46:0/935952191 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e00183f0 con 0x7f39f0100a80 2024-09-10T07:37:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.907+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f019d750 con 0x7f39f0100a80 2024-09-10T07:37:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.908+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f0102580 con 0x7f39f0100a80 2024-09-10T07:37:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.909+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f01021c0 con 0x7f39f0100a80 2024-09-10T07:37:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.914+0000 7f39ed600640 1 -- 172.21.15.46:0/935952191 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f39e001f020 con 0x7f39f0100a80 2024-09-10T07:37:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.917+0000 7f39ed600640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39d80a2d50 0x7f39d80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.918+0000 7f39ed600640 1 -- 172.21.15.46:0/935952191 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f39e00d2940 con 0x7f39f0100a80 2024-09-10T07:37:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.919+0000 7f39efe00640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39d80a2d50 0x7f39d80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.920+0000 7f39ed600640 1 -- 172.21.15.46:0/935952191 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39e0026020 con 0x7f39f0100a80 2024-09-10T07:37:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:28.924+0000 7f39efe00640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39d80a2d50 0x7f39d80a5210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39e4005940 tx=0x7f39e40058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:29.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.323+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39f0100ee0 con 0x7f39f0100a80 2024-09-10T07:37:29.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.324+0000 7f39ed600640 1 -- 172.21.15.46:0/935952191 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11975 (secure 0 0 0) 0x7f39f0100ee0 con 0x7f39f0100a80 2024-09-10T07:37:29.326 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:29.326 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"btime":"2024-09-10T07:37:26:476895+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:37:25.489121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:37:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-09-10T07:37:29.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39d80a2d50 msgr2=0x7f39d80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:29.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39d80a2d50 0x7f39d80a5210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39e4005940 tx=0x7f39e40058b0 comp rx=0 tx=0).stop 2024-09-10T07:37:29.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 msgr2=0x7f39f01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:29.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 0x7f39f01984f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f39e000f780 tx=0x7f39e000fc40 comp rx=0 tx=0).stop 2024-09-10T07:37:29.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 shutdown_connections 2024-09-10T07:37:29.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f39d80a2d50 0x7f39d80a5210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:29.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f0106a80 0x7f39f019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:29.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39f0101420 0x7f39f0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:29.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 --2- 172.21.15.46:0/935952191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39f0100a80 0x7f39f01984f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:29.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.328+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 >> 172.21.15.46:0/935952191 conn(0x7f39f00fc7a0 msgr2=0x7f39f0104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:29.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.330+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 shutdown_connections 2024-09-10T07:37:29.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.330+0000 7f39f7600640 1 -- 172.21.15.46:0/935952191 wait complete. 2024-09-10T07:37:29.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:29 smithi046 ceph-mon[30705]: pgmap v425: 129 pgs: 129 active+clean; 676 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 33 MiB/s wr, 205 op/s 2024-09-10T07:37:29.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:29 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/935952191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:29.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:29 smithi181 ceph-mon[35397]: pgmap v425: 129 pgs: 129 active+clean; 676 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 33 MiB/s wr, 205 op/s 2024-09-10T07:37:29.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:29 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/935952191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:29.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:29 smithi063 ceph-mon[35020]: pgmap v425: 129 pgs: 129 active+clean; 676 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 33 MiB/s wr, 205 op/s 2024-09-10T07:37:29.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:29 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/935952191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 10.0 s, 1729.2 tps, lat 58.098 ms stddev 65.890 2024-09-10T07:37:30.873 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:37:30.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 -- 172.21.15.46:0/2603950847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c074f40 msgr2=0x7fa86c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 --2- 172.21.15.46:0/2603950847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c074f40 0x7fa86c10de80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa858009900 tx=0x7fa85802a480 comp rx=0 tx=0).stop 2024-09-10T07:37:30.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 -- 172.21.15.46:0/2603950847 shutdown_connections 2024-09-10T07:37:30.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 --2- 172.21.15.46:0/2603950847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa86c10e3c0 0x7fa86c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 --2- 172.21.15.46:0/2603950847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c074f40 0x7fa86c10de80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 --2- 172.21.15.46:0/2603950847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c074620 0x7fa86c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 -- 172.21.15.46:0/2603950847 >> 172.21.15.46:0/2603950847 conn(0x7fa86c0fd8c0 msgr2=0x7fa86c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:30.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 -- 172.21.15.46:0/2603950847 shutdown_connections 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.915+0000 7fa871a00640 1 -- 172.21.15.46:0/2603950847 wait complete. 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.916+0000 7fa871a00640 1 Processor -- start 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.916+0000 7fa871a00640 1 -- start start 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.916+0000 7fa871a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c074620 0x7fa86c10dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa871a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa86c074f40 0x7fa86c10add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa871a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c10e3c0 0x7fa86c10b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa871a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c114190 con 0x7fa86c074620 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa871a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c114010 con 0x7fa86c074f40 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa871a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c114310 con 0x7fa86c10e3c0 2024-09-10T07:37:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c074620 0x7fa86c10dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa86c074f40 0x7fa86c10add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c074620 0x7fa86c10dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43522/0 (socket says 172.21.15.46:43522) 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86aa00640 1 -- 172.21.15.46:0/2992271164 learned_addr learned my addr 172.21.15.46:0/2992271164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa86c074f40 0x7fa86c10add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33706/0 (socket says 172.21.15.46:33706) 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86b400640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c10e3c0 0x7fa86c10b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86aa00640 1 -- 172.21.15.46:0/2992271164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c10e3c0 msgr2=0x7fa86c10b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86aa00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c10e3c0 0x7fa86c10b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86aa00640 1 -- 172.21.15.46:0/2992271164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa86c074f40 msgr2=0x7fa86c10add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86aa00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa86c074f40 0x7fa86c10add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86aa00640 1 -- 172.21.15.46:0/2992271164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c10bbd0 con 0x7fa86c074620 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.917+0000 7fa86b400640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c10e3c0 0x7fa86c10b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.918+0000 7fa86aa00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c074620 0x7fa86c10dd20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00e9f0 tx=0x7fa85c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.918+0000 7fa863400640 1 -- 172.21.15.46:0/2992271164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c00cd80 con 0x7fa86c074620 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.918+0000 7fa863400640 1 -- 172.21.15.46:0/2992271164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa85c010910 con 0x7fa86c074620 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.918+0000 7fa871a00640 1 -- 172.21.15.46:0/2992271164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c1aaff0 con 0x7fa86c074620 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.919+0000 7fa863400640 1 -- 172.21.15.46:0/2992271164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0188f0 con 0x7fa86c074620 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.919+0000 7fa871a00640 1 -- 172.21.15.46:0/2992271164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c1ab4b0 con 0x7fa86c074620 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa863400640 1 -- 172.21.15.46:0/2992271164 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa85c010ac0 con 0x7fa86c074620 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa874624740 1 Processor -- start 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa874624740 1 -- start start 2024-09-10T07:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa874624740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c2c70 0x557d073c3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa874624740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa874624740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa874624740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557d073293e0 con 0x557d073cde50 2024-09-10T07:37:30.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa874624740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557d072d0d80 con 0x557d073c3590 2024-09-10T07:37:30.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.923+0000 7fa874624740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557d073253a0 con 0x557d073c2c70 2024-09-10T07:37:30.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c2c70 0x557d073c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c2c70 0x557d073c3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53032/0 (socket says 172.21.15.46:53032) 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 -- 172.21.15.46:0/2694753901 learned_addr learned my addr 172.21.15.46:0/2694753901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 -- 172.21.15.46:0/2694753901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 msgr2=0x557d073cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 --2- 172.21.15.46:0/2694753901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 -- 172.21.15.46:0/2694753901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 msgr2=0x557d073d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 --2- 172.21.15.46:0/2694753901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 -- 172.21.15.46:0/2694753901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d072efae0 con 0x557d073c2c70 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa863400640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500a2b80 0x7fa8500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.924+0000 7fa86b400640 1 --2- 172.21.15.46:0/2694753901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c2c70 0x557d073c3050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa864009fc0 tx=0x7fa864010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39568d3a5d4a8086 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.925+0000 7fa863400640 1 -- 172.21.15.46:0/2992271164 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa85c0cf7d0 con 0x7fa86c074620 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.925+0000 7fa857e00640 1 -- 172.21.15.46:0/2694753901 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864011700 con 0x557d073c2c70 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.925+0000 7fa857e00640 1 -- 172.21.15.46:0/2694753901 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa864011d80 con 0x557d073c2c70 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.925+0000 7fa857e00640 1 -- 172.21.15.46:0/2694753901 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86401a380 con 0x557d073c2c70 2024-09-10T07:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.925+0000 7fa86aa00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500a2b80 0x7fa8500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 -- 172.21.15.46:0/2694753901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c2c70 msgr2=0x557d073c3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 --2- 172.21.15.46:0/2694753901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c2c70 0x557d073c3050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa864009fc0 tx=0x7fa864010860 comp rx=0 tx=0).stop 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 -- 172.21.15.46:0/2694753901 shutdown_connections 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 --2- 172.21.15.46:0/2694753901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 --2- 172.21.15.46:0/2694753901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 --2- 172.21.15.46:0/2694753901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c2c70 0x557d073c3050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 -- 172.21.15.46:0/2694753901 >> 172.21.15.46:0/2694753901 conn(0x557d073bf200 msgr2=0x557d073bf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 -- 172.21.15.46:0/2694753901 shutdown_connections 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa86aa00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500a2b80 0x7fa8500a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0cd2c0 tx=0x7fa85c0d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.926+0000 7fa874624740 1 -- 172.21.15.46:0/2694753901 wait complete. 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 1 -- start start 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 1 Processor -- start 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 0x557d073cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557d073d0a20 con 0x557d073cde50 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557d072e40f0 con 0x557d073c2c70 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557d073d0ba0 con 0x557d073c3590 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.927+0000 7fa874624740 10 client.0 _ll_register_callbacks cb 0x557d072cbbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33722/0 (socket says 172.21.15.46:33722) 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 -- 172.21.15.46:0/574125122 learned_addr learned my addr 172.21.15.46:0/574125122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 0x557d073cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 -- 172.21.15.46:0/574125122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 msgr2=0x557d073cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 0x557d073cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 -- 172.21.15.46:0/574125122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 msgr2=0x557d073cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 -- 172.21.15.46:0/574125122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d0709a3c0 con 0x557d073c2c70 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa86b400640 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073cade0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa864009d50 tx=0x7fa864013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d902796bfedccc4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.928+0000 7fa854c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.929+0000 7fa854c00640 1 -- 172.21.15.46:0/574125122 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8640119d0 con 0x557d073c2c70 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.929+0000 7fa874624740 1 -- 172.21.15.46:0/574125122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa864011d80 con 0x557d073c2c70 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.929+0000 7fa854c00640 1 -- 172.21.15.46:0/574125122 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa864011b70 con 0x557d073c2c70 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.930+0000 7fa854c00640 1 -- 172.21.15.46:0/574125122 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa86402c440 con 0x557d073c2c70 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.930+0000 7fa874624740 10 client.25577 fetch_fsmap learned FSMap version 32 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.930+0000 7fa874624740 1 -- 172.21.15.46:0/574125122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557d073dcb30 con 0x557d073c2c70 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.930+0000 7fa854c00640 1 -- 172.21.15.46:0/574125122 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa864011d80 con 0x557d073c2c70 2024-09-10T07:37:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.930+0000 7fa854c00640 1 -- 172.21.15.46:0/574125122 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fa86402a020 con 0x557d073c2c70 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.930+0000 7fa874624740 10 client.25577 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.930+0000 7fa874624740 10 client.25577 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.931+0000 7fa874624740 20 client.25577 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.931+0000 7fa874624740 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557d073ddf90 0x557d073e0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.931+0000 7fa874624740 4 client.25577 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.931+0000 7fa874624740 1 -- 172.21.15.46:0/574125122 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d07336950 con 0x557d073ddf90 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.933+0000 7fa86a000640 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557d073ddf90 0x557d073e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.935+0000 7fa86a000640 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557d073ddf90 0x557d073e0380 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2447a2a6a4f0e024 server_cookie=ca99f700d9ea4999 in_seq=0 out_seq=0 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.937+0000 7fa854c00640 10 client.25577 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.937+0000 7fa854c00640 1 -- 172.21.15.46:0/574125122 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d07336950 con 0x557d073ddf90 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.937+0000 7fa854c00640 10 client.25577 handle_command_reply: tid=0 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.937+0000 7fa874624740 1 client.25577 shutdown 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa874624740 1 -- 172.21.15.46:0/574125122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 msgr2=0x557d073cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa874624740 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073cade0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa864009d50 tx=0x7fa864013040 comp rx=0 tx=0).stop 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa874624740 1 -- 172.21.15.46:0/574125122 shutdown_connections 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa874624740 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557d073ddf90 0x557d073e0380 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa874624740 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa854c00640 0 client.25577 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa874624740 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 0x557d073cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa874624740 1 --2- 172.21.15.46:0/574125122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073cade0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.940+0000 7fa874624740 1 -- 172.21.15.46:0/574125122 >> 172.21.15.46:0/574125122 conn(0x557d073bf200 msgr2=0x557d073c4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.943+0000 7fa874624740 1 -- 172.21.15.46:0/574125122 shutdown_connections 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.943+0000 7fa874624740 1 -- 172.21.15.46:0/574125122 wait complete. 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.943+0000 7fa874624740 20 client.25577 trim_cache size 0 max 16384 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.970+0000 7fa874624740 1 Processor -- start 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.971+0000 7fa874624740 1 -- start start 2024-09-10T07:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.971+0000 7fa874624740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073c2c70 0x557d073c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.971+0000 7fa874624740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.971+0000 7fa874624740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073cde50 0x557d073e59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.971+0000 7fa874624740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1b52d0 con 0x557d073c2c70 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.971+0000 7fa874624740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1b6e70 con 0x557d073c3590 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.971+0000 7fa874624740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1b59a0 con 0x557d073cde50 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.972+0000 7fa86a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.972+0000 7fa86a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33734/0 (socket says 172.21.15.46:33734) 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.972+0000 7fa86a000640 1 -- 172.21.15.46:0/2647889303 learned_addr learned my addr 172.21.15.46:0/2647889303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.973+0000 7fa86b400640 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073cde50 0x557d073e59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.973+0000 7fa86b400640 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073c2c70 0x557d073c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.973+0000 7fa86b400640 1 -- 172.21.15.46:0/2647889303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 msgr2=0x557d073cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.973+0000 7fa86b400640 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.973+0000 7fa86b400640 1 -- 172.21.15.46:0/2647889303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073c2c70 msgr2=0x557d073c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.973+0000 7fa86b400640 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073c2c70 0x557d073c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.973+0000 7fa86b400640 1 -- 172.21.15.46:0/2647889303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d073da7f0 con 0x557d073cde50 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.974+0000 7fa86b400640 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073cde50 0x557d073e59e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa86401c7d0 tx=0x7fa8640092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c98343db6bae4f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.975+0000 7fa857400640 1 -- 172.21.15.46:0/2647889303 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864010d60 con 0x557d073cde50 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.975+0000 7fa857400640 1 -- 172.21.15.46:0/2647889303 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa864009b50 con 0x557d073cde50 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.975+0000 7fa857400640 1 -- 172.21.15.46:0/2647889303 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86401ad60 con 0x557d073cde50 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.976+0000 7fa874624740 1 -- 172.21.15.46:0/2647889303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073cde50 msgr2=0x557d073e59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.976+0000 7fa874624740 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073cde50 0x557d073e59e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa86401c7d0 tx=0x7fa8640092b0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.976+0000 7fa874624740 1 -- 172.21.15.46:0/2647889303 shutdown_connections 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.976+0000 7fa874624740 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073cde50 0x557d073e59e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.976+0000 7fa874624740 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c3590 0x557d073cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.976+0000 7fa874624740 1 --2- 172.21.15.46:0/2647889303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073c2c70 0x557d073c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.976+0000 7fa874624740 1 -- 172.21.15.46:0/2647889303 >> 172.21.15.46:0/2647889303 conn(0x557d073bf200 msgr2=0x557d073c4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.977+0000 7fa874624740 1 -- 172.21.15.46:0/2647889303 shutdown_connections 2024-09-10T07:37:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.977+0000 7fa874624740 1 -- 172.21.15.46:0/2647889303 wait complete. 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.977+0000 7fa874624740 1 -- start start 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.978+0000 7fa874624740 1 Processor -- start 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.979+0000 7fa874624740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073d6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.979+0000 7fa86b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073d6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.979+0000 7fa86b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073d6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33736/0 (socket says 172.21.15.46:33736) 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.979+0000 7fa86b400640 1 -- 172.21.15.46:0/803252828 learned_addr learned my addr 172.21.15.46:0/803252828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.979+0000 7fa874624740 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 0x557d073d6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.980+0000 7fa874624740 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.980+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1b6740 con 0x557d073cde50 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.980+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1b6070 con 0x557d073c2c70 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.980+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c1b32b0 con 0x557d073c3590 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.980+0000 7fa874624740 10 client.0 _ll_register_callbacks cb 0x7fa85c0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.980+0000 7fa86a000640 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 0x557d073d6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.981+0000 7fa86b400640 1 -- 172.21.15.46:0/803252828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 msgr2=0x557d073d6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.981+0000 7fa86b400640 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 0x557d073d6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.981+0000 7fa86b400640 1 -- 172.21.15.46:0/803252828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 msgr2=0x557d073c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.981+0000 7fa86b400640 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.981+0000 7fa86b400640 1 -- 172.21.15.46:0/803252828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d073c7df0 con 0x557d073c2c70 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.981+0000 7fa86b400640 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073d6000 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa86401a670 tx=0x7fa86401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e76ca98b83926c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.982+0000 7fa854c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.982+0000 7fa854c00640 1 -- 172.21.15.46:0/803252828 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8640049c0 con 0x557d073c2c70 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.982+0000 7fa854c00640 1 -- 172.21.15.46:0/803252828 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa864027040 con 0x557d073c2c70 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.982+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa864009b50 con 0x557d073c2c70 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.983+0000 7fa854c00640 1 -- 172.21.15.46:0/803252828 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8640049c0 con 0x557d073c2c70 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.983+0000 7fa874624740 10 client.25583 fetch_fsmap learned FSMap version 32 2024-09-10T07:37:30.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.983+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557d07448430 con 0x557d073c2c70 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.984+0000 7fa854c00640 1 -- 172.21.15.46:0/803252828 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa864009b50 con 0x557d073c2c70 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.986+0000 7fa854c00640 1 -- 172.21.15.46:0/803252828 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fa86402a040 con 0x557d073c2c70 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.987+0000 7fa874624740 10 client.25583 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.987+0000 7fa874624740 10 client.25583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.987+0000 7fa874624740 20 client.25583 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.987+0000 7fa874624740 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557d07448810 0x557d0744ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.987+0000 7fa874624740 4 client.25583 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.987+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d07330e60 con 0x557d07448810 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.987+0000 7fa86a000640 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557d07448810 0x557d0744ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.988+0000 7fa86a000640 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557d07448810 0x557d0744ac00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a83fe0fe529bbd31 server_cookie=5cda9f0e3697cb89 in_seq=0 out_seq=0 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.988+0000 7fa854c00640 10 client.25583 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa854c00640 1 -- 172.21.15.46:0/803252828 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d07330e60 con 0x557d07448810 2024-09-10T07:37:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa854c00640 10 client.25583 handle_command_reply: tid=0 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 client.25583 shutdown 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 msgr2=0x557d073d6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073d6000 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa86401a670 tx=0x7fa86401c480 comp rx=0 tx=0).stop 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 shutdown_connections 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557d07448810 0x557d0744ac00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d073cde50 0x557d073c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa854c00640 0 client.25583 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557d073c3590 0x557d073d6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 --2- 172.21.15.46:0/803252828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d073c2c70 0x557d073d6000 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.989+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 >> 172.21.15.46:0/803252828 conn(0x557d073bf200 msgr2=0x557d073c4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.990+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 shutdown_connections 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.990+0000 7fa874624740 1 -- 172.21.15.46:0/803252828 wait complete. 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.990+0000 7fa874624740 20 client.25583 trim_cache size 0 max 16384 2024-09-10T07:37:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.990+0000 7fa871a00640 1 -- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500a2b80 msgr2=0x7fa8500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500a2b80 0x7fa8500a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0cd2c0 tx=0x7fa85c0d5550 comp rx=0 tx=0).stop 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 -- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c074620 msgr2=0x7fa86c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c074620 0x7fa86c10dd20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00e9f0 tx=0x7fa85c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 -- 172.21.15.46:0/2992271164 shutdown_connections 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa86c10e3c0 0x7fa86c10b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa86c074f40 0x7fa86c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8500a2b80 0x7fa8500a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 --2- 172.21.15.46:0/2992271164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86c074620 0x7fa86c10dd20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 -- 172.21.15.46:0/2992271164 >> 172.21.15.46:0/2992271164 conn(0x7fa86c0fd8c0 msgr2=0x7fa86c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 -- 172.21.15.46:0/2992271164 shutdown_connections 2024-09-10T07:37:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:29.991+0000 7fa871a00640 1 -- 172.21.15.46:0/2992271164 wait complete. 2024-09-10T07:37:31.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:31 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:31.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:31 smithi046 ceph-mon[30705]: pgmap v426: 129 pgs: 129 active+clean; 838 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 32 MiB/s wr, 328 op/s 2024-09-10T07:37:31.767 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T07:37:31.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T07:37:31.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:31 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:31.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:31 smithi181 ceph-mon[35397]: pgmap v426: 129 pgs: 129 active+clean; 838 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 32 MiB/s wr, 328 op/s 2024-09-10T07:37:31.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:31 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:31.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:31 smithi063 ceph-mon[35020]: pgmap v426: 129 pgs: 129 active+clean; 838 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 32 MiB/s wr, 328 op/s 2024-09-10T07:37:32.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:37:32.511 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:33.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.254+0000 7f68cd800640 1 -- 172.21.15.46:0/1365675935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8109100 msgr2=0x7f68c8110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.254+0000 7f68cd800640 1 --2- 172.21.15.46:0/1365675935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8109100 0x7f68c8110c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009ca0 tx=0x7f68c002a8b0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.254+0000 7f68cd800640 1 -- 172.21.15.46:0/1365675935 shutdown_connections 2024-09-10T07:37:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.254+0000 7f68cd800640 1 --2- 172.21.15.46:0/1365675935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8109100 0x7f68c8110c80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.254+0000 7f68cd800640 1 --2- 172.21.15.46:0/1365675935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c8104980 0x7f68c81089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.254+0000 7f68cd800640 1 --2- 172.21.15.46:0/1365675935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8103fd0 0x7f68c81043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.254+0000 7f68cd800640 1 -- 172.21.15.46:0/1365675935 >> 172.21.15.46:0/1365675935 conn(0x7f68c80ffa30 msgr2=0x7f68c8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.255+0000 7f68cd800640 1 -- 172.21.15.46:0/1365675935 shutdown_connections 2024-09-10T07:37:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.255+0000 7f68cd800640 1 -- 172.21.15.46:0/1365675935 wait complete. 2024-09-10T07:37:33.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.256+0000 7f68cd800640 1 Processor -- start 2024-09-10T07:37:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.257+0000 7f68cd800640 1 -- start start 2024-09-10T07:37:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.259+0000 7f68cd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8103fd0 0x7f68c810e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.259+0000 7f68cd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104980 0x7f68c810ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.259+0000 7f68cd800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c8109100 0x7f68c81101b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.259+0000 7f68cd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80769c0 con 0x7f68c8104980 2024-09-10T07:37:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.259+0000 7f68cd800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8076840 con 0x7f68c8103fd0 2024-09-10T07:37:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.259+0000 7f68cd800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8076b40 con 0x7f68c8109100 2024-09-10T07:37:33.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.260+0000 7f68c7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c8109100 0x7f68c81101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.260+0000 7f68c7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c8109100 0x7f68c81101b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53076/0 (socket says 172.21.15.46:53076) 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.260+0000 7f68c7400640 1 -- 172.21.15.46:0/3463595392 learned_addr learned my addr 172.21.15.46:0/3463595392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.260+0000 7f68c6a00640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8103fd0 0x7f68c810e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.260+0000 7f68c6000640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104980 0x7f68c810ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.261+0000 7f68c6a00640 1 -- 172.21.15.46:0/3463595392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c8109100 msgr2=0x7f68c81101b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.261+0000 7f68c6a00640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c8109100 0x7f68c81101b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.261+0000 7f68c6a00640 1 -- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104980 msgr2=0x7f68c810ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.261+0000 7f68c6a00640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104980 0x7f68c810ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.261+0000 7f68c6a00640 1 -- 172.21.15.46:0/3463595392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c8110930 con 0x7f68c8103fd0 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.262+0000 7f68c7400640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c8109100 0x7f68c81101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.262+0000 7f68c6a00640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8103fd0 0x7f68c810e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68b400b3f0 tx=0x7f68b400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.262+0000 7f68c6000640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104980 0x7f68c810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.263+0000 7f68bf400640 1 -- 172.21.15.46:0/3463595392 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4015020 con 0x7f68c8103fd0 2024-09-10T07:37:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.263+0000 7f68bf400640 1 -- 172.21.15.46:0/3463595392 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b4011d10 con 0x7f68c8103fd0 2024-09-10T07:37:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.263+0000 7f68bf400640 1 -- 172.21.15.46:0/3463595392 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b400f2e0 con 0x7f68c8103fd0 2024-09-10T07:37:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.263+0000 7f68cd800640 1 -- 172.21.15.46:0/3463595392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c8110c20 con 0x7f68c8103fd0 2024-09-10T07:37:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.263+0000 7f68cd800640 1 -- 172.21.15.46:0/3463595392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81ab5f0 con 0x7f68c8103fd0 2024-09-10T07:37:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.269+0000 7f68bf400640 1 -- 172.21.15.46:0/3463595392 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68b4018020 con 0x7f68c8103fd0 2024-09-10T07:37:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.272+0000 7f68bf400640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68ac0a2cd0 0x7f68ac0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.273+0000 7f68bf400640 1 -- 172.21.15.46:0/3463595392 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f68b40cebf0 con 0x7f68c8103fd0 2024-09-10T07:37:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.274+0000 7f68c6000640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68ac0a2cd0 0x7f68ac0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.276+0000 7f68c6000640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68ac0a2cd0 0x7f68ac0a5190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f68c810a9c0 tx=0x7f68b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.278+0000 7f68d022f740 1 Processor -- start 2024-09-10T07:37:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.279+0000 7f68d022f740 1 -- start start 2024-09-10T07:37:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.281+0000 7f68d022f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfd9e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.281+0000 7f68d022f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 0x55eadfda9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.281+0000 7f68d022f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfdab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.281+0000 7f68d022f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eadfd04530 con 0x55eadfd9ecd0 2024-09-10T07:37:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.281+0000 7f68d022f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eadfcab860 con 0x55eadfda9590 2024-09-10T07:37:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.281+0000 7f68d022f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eadfd004f0 con 0x55eadfd9e3b0 2024-09-10T07:37:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfdab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfdab980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33756/0 (socket says 172.21.15.46:33756) 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfd9e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c7400640 1 -- 172.21.15.46:0/3616669833 learned_addr learned my addr 172.21.15.46:0/3616669833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c7400640 1 -- 172.21.15.46:0/3616669833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 msgr2=0x55eadfd9e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c7400640 1 --2- 172.21.15.46:0/3616669833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfd9e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c7400640 1 -- 172.21.15.46:0/3616669833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 msgr2=0x55eadfda9050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c7400640 1 --2- 172.21.15.46:0/3616669833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 0x55eadfda9050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c7400640 1 -- 172.21.15.46:0/3616669833 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadfa75b90 con 0x55eadfda9590 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.282+0000 7f68c6a00640 1 --2- 172.21.15.46:0/3616669833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfd9e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.283+0000 7f68c7400640 1 --2- 172.21.15.46:0/3616669833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfdab980 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009430 tx=0x7f68c00304b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=416029811b0a4bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.283+0000 7f68b3e00640 1 -- 172.21.15.46:0/3616669833 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0003970 con 0x55eadfda9590 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.283+0000 7f68b3e00640 1 -- 172.21.15.46:0/3616669833 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c0036040 con 0x55eadfda9590 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.283+0000 7f68b3e00640 1 -- 172.21.15.46:0/3616669833 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00044d0 con 0x55eadfda9590 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.284+0000 7f68d022f740 1 -- 172.21.15.46:0/3616669833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 msgr2=0x55eadfdab980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.284+0000 7f68d022f740 1 --2- 172.21.15.46:0/3616669833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfdab980 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009430 tx=0x7f68c00304b0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.285+0000 7f68d022f740 1 -- 172.21.15.46:0/3616669833 shutdown_connections 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.285+0000 7f68d022f740 1 --2- 172.21.15.46:0/3616669833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfdab980 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.285+0000 7f68d022f740 1 --2- 172.21.15.46:0/3616669833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 0x55eadfda9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.285+0000 7f68d022f740 1 --2- 172.21.15.46:0/3616669833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfd9e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.285+0000 7f68d022f740 1 -- 172.21.15.46:0/3616669833 >> 172.21.15.46:0/3616669833 conn(0x55eadfd9a940 msgr2=0x55eadfd9ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.285+0000 7f68d022f740 1 -- 172.21.15.46:0/3616669833 shutdown_connections 2024-09-10T07:37:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.285+0000 7f68d022f740 1 -- 172.21.15.46:0/3616669833 wait complete. 2024-09-10T07:37:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.286+0000 7f68d022f740 1 -- start start 2024-09-10T07:37:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.287+0000 7f68d022f740 1 Processor -- start 2024-09-10T07:37:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.289+0000 7f68d022f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfda6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.289+0000 7f68d022f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 0x55eadfda6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.289+0000 7f68c6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfda6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68c6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfda6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53098/0 (socket says 172.21.15.46:53098) 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68d022f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfda6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68d022f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eadfdac010 con 0x55eadfd9ecd0 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68d022f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eadfcbee60 con 0x55eadfda9590 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68d022f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eadfdac190 con 0x55eadfd9e3b0 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68d022f740 10 client.0 _ll_register_callbacks cb 0x55eadfcc3cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfda6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfda6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33764/0 (socket says 172.21.15.46:33764) 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.290+0000 7f68c6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 0x55eadfda6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.291+0000 7f68c7400640 1 -- 172.21.15.46:0/2893383884 learned_addr learned my addr 172.21.15.46:0/2893383884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.292+0000 7f68c6a00640 1 -- 172.21.15.46:0/2893383884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 msgr2=0x55eadfda6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.292+0000 7f68c6a00640 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfda6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.292+0000 7f68c6a00640 1 -- 172.21.15.46:0/2893383884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 msgr2=0x55eadfda6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.292+0000 7f68c6a00640 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 0x55eadfda6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.292+0000 7f68c6a00640 1 -- 172.21.15.46:0/2893383884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadfcca990 con 0x55eadfd9e3b0 2024-09-10T07:37:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.293+0000 7f68c6a00640 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfda6530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f68b40cc340 tx=0x7f68b409b250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f46d74c0aa660e00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.293+0000 7f68b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.293+0000 7f68b0c00640 1 -- 172.21.15.46:0/2893383884 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b40b3070 con 0x55eadfd9e3b0 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.293+0000 7f68b0c00640 1 -- 172.21.15.46:0/2893383884 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f68b409f170 con 0x55eadfd9e3b0 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.293+0000 7f68d022f740 1 -- 172.21.15.46:0/2893383884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c0036040 con 0x55eadfd9e3b0 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.294+0000 7f68b0c00640 1 -- 172.21.15.46:0/2893383884 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f68b40bb0b0 con 0x55eadfd9e3b0 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.294+0000 7f68d022f740 10 client.25588 fetch_fsmap learned FSMap version 33 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.294+0000 7f68d022f740 1 -- 172.21.15.46:0/2893383884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eadfdb7ed0 con 0x55eadfd9e3b0 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.294+0000 7f68b0c00640 1 -- 172.21.15.46:0/2893383884 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c0036040 con 0x55eadfd9e3b0 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.295+0000 7f68b0c00640 1 -- 172.21.15.46:0/2893383884 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f68b409a640 con 0x55eadfd9e3b0 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.295+0000 7f68d022f740 10 client.25588 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T07:37:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.295+0000 7f68d022f740 10 client.25588 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:37:33.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.295+0000 7f68d022f740 20 client.25588 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:33.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.295+0000 7f68d022f740 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eadfdb9330 0x55eadfdbb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.295+0000 7f68d022f740 4 client.25588 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:33.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.295+0000 7f68d022f740 1 -- 172.21.15.46:0/2893383884 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eadfca7310 con 0x55eadfdb9330 2024-09-10T07:37:33.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.295+0000 7f68c6000640 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eadfdb9330 0x55eadfdbb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.298+0000 7f68b0c00640 10 client.25588 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.298+0000 7f68c6000640 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eadfdb9330 0x55eadfdbb750 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b8d543b08f54a70 server_cookie=47944fae15a66251 in_seq=0 out_seq=0 2024-09-10T07:37:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.300+0000 7f68b0c00640 1 -- 172.21.15.46:0/2893383884 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eadfca7310 con 0x55eadfdb9330 2024-09-10T07:37:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.300+0000 7f68b0c00640 10 client.25588 handle_command_reply: tid=0 2024-09-10T07:37:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.300+0000 7f68d022f740 1 client.25588 shutdown 2024-09-10T07:37:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.300+0000 7f68d022f740 1 -- 172.21.15.46:0/2893383884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 msgr2=0x55eadfda6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.300+0000 7f68d022f740 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfda6530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f68b40cc340 tx=0x7f68b409b250 comp rx=0 tx=0).stop 2024-09-10T07:37:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.301+0000 7f68d022f740 1 -- 172.21.15.46:0/2893383884 shutdown_connections 2024-09-10T07:37:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.301+0000 7f68d022f740 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eadfdb9330 0x55eadfdbb750 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.301+0000 7f68b0c00640 0 client.25588 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:33.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.301+0000 7f68d022f740 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfda9590 0x55eadfda6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.301+0000 7f68d022f740 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfd9ecd0 0x55eadfda6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.301+0000 7f68d022f740 1 --2- 172.21.15.46:0/2893383884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfda6530 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.301+0000 7f68d022f740 1 -- 172.21.15.46:0/2893383884 >> 172.21.15.46:0/2893383884 conn(0x55eadfd9a940 msgr2=0x55eadfd99c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:33.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.301+0000 7f68d022f740 1 -- 172.21.15.46:0/2893383884 shutdown_connections 2024-09-10T07:37:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.302+0000 7f68d022f740 1 -- 172.21.15.46:0/2893383884 wait complete. 2024-09-10T07:37:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.302+0000 7f68d022f740 20 client.25588 trim_cache size 0 max 16384 2024-09-10T07:37:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.334+0000 7f68d022f740 1 Processor -- start 2024-09-10T07:37:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.335+0000 7f68d022f740 1 -- start start 2024-09-10T07:37:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.335+0000 7f68d022f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9e3b0 0x55eadfd99a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.335+0000 7f68d022f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9ecd0 0x55eadfda4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.335+0000 7f68d022f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 0x55eadfda6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.335+0000 7f68d022f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81acc40 con 0x55eadfda9590 2024-09-10T07:37:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.335+0000 7f68d022f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81ae810 con 0x55eadfd9e3b0 2024-09-10T07:37:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.335+0000 7f68d022f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81ad310 con 0x55eadfd9ecd0 2024-09-10T07:37:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9e3b0 0x55eadfd99a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9ecd0 0x55eadfda4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9ecd0 0x55eadfda4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53100/0 (socket says 172.21.15.46:53100) 2024-09-10T07:37:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6000640 1 -- 172.21.15.46:0/2894128121 learned_addr learned my addr 172.21.15.46:0/2894128121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6000640 1 -- 172.21.15.46:0/2894128121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9e3b0 msgr2=0x55eadfd99a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6000640 1 --2- 172.21.15.46:0/2894128121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9e3b0 0x55eadfd99a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6000640 1 -- 172.21.15.46:0/2894128121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 msgr2=0x55eadfda6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6000640 1 --2- 172.21.15.46:0/2894128121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 0x55eadfda6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6000640 1 -- 172.21.15.46:0/2894128121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadfd9eb00 con 0x55eadfd9ecd0 2024-09-10T07:37:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.336+0000 7f68c6a00640 1 --2- 172.21.15.46:0/2894128121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9e3b0 0x55eadfd99a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.337+0000 7f68c6000640 1 --2- 172.21.15.46:0/2894128121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9ecd0 0x55eadfda4e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55eadfda5dc0 tx=0x7f68b8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ecf96ce8249c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.337+0000 7f68b3400640 1 -- 172.21.15.46:0/2894128121 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b802d070 con 0x55eadfd9ecd0 2024-09-10T07:37:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.337+0000 7f68b3400640 1 -- 172.21.15.46:0/2894128121 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b8012cb0 con 0x55eadfd9ecd0 2024-09-10T07:37:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.337+0000 7f68b3400640 1 -- 172.21.15.46:0/2894128121 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b80272d0 con 0x55eadfd9ecd0 2024-09-10T07:37:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.339+0000 7f68d022f740 1 -- 172.21.15.46:0/2894128121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9ecd0 msgr2=0x55eadfda4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.339+0000 7f68d022f740 1 --2- 172.21.15.46:0/2894128121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9ecd0 0x55eadfda4e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55eadfda5dc0 tx=0x7f68b8016000 comp rx=0 tx=0).stop 2024-09-10T07:37:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.340+0000 7f68d022f740 1 -- 172.21.15.46:0/2894128121 shutdown_connections 2024-09-10T07:37:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.340+0000 7f68d022f740 1 --2- 172.21.15.46:0/2894128121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 0x55eadfda6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.340+0000 7f68d022f740 1 --2- 172.21.15.46:0/2894128121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9ecd0 0x55eadfda4e00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.340+0000 7f68d022f740 1 --2- 172.21.15.46:0/2894128121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9e3b0 0x55eadfd99a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.340+0000 7f68d022f740 1 -- 172.21.15.46:0/2894128121 >> 172.21.15.46:0/2894128121 conn(0x55eadfd9a940 msgr2=0x55eadfda0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.341+0000 7f68d022f740 1 -- 172.21.15.46:0/2894128121 shutdown_connections 2024-09-10T07:37:33.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.341+0000 7f68d022f740 1 -- 172.21.15.46:0/2894128121 wait complete. 2024-09-10T07:37:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.342+0000 7f68d022f740 1 -- start start 2024-09-10T07:37:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.343+0000 7f68d022f740 1 Processor -- start 2024-09-10T07:37:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.344+0000 7f68d022f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfdb18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.344+0000 7f68d022f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9ecd0 0x55eadfdb1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.344+0000 7f68d022f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 0x55eadfdadce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.345+0000 7f68d022f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81ae0b0 con 0x55eadfda9590 2024-09-10T07:37:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.345+0000 7f68d022f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81ad9e0 con 0x55eadfd9ecd0 2024-09-10T07:37:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.345+0000 7f68d022f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68c810fc70 con 0x55eadfd9e3b0 2024-09-10T07:37:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.345+0000 7f68d022f740 10 client.0 _ll_register_callbacks cb 0x7f68b409b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.344+0000 7f68c6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfdb18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.345+0000 7f68c6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9ecd0 0x55eadfdb1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.345+0000 7f68c6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9ecd0 0x55eadfdb1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33776/0 (socket says 172.21.15.46:33776) 2024-09-10T07:37:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.345+0000 7f68c6000640 1 -- 172.21.15.46:0/3835100612 learned_addr learned my addr 172.21.15.46:0/3835100612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.345+0000 7f68c7400640 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 0x55eadfdadce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.346+0000 7f68c6000640 1 -- 172.21.15.46:0/3835100612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 msgr2=0x55eadfdb18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.346+0000 7f68c6000640 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfdb18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.346+0000 7f68c6000640 1 -- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 msgr2=0x55eadfdadce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.346+0000 7f68c6000640 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 0x55eadfdadce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.346+0000 7f68c6000640 1 -- 172.21.15.46:0/3835100612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadfdae510 con 0x55eadfd9ecd0 2024-09-10T07:37:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.346+0000 7f68c6a00640 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfdb18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.346+0000 7f68c7400640 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 0x55eadfdadce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.347+0000 7f68c6000640 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9ecd0 0x55eadfdb1e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f68b8010d40 tx=0x7f68b8003770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e2b4c9540cd0ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.347+0000 7f68b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.347+0000 7f68b0c00640 1 -- 172.21.15.46:0/3835100612 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b802d070 con 0x55eadfd9ecd0 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.347+0000 7f68d022f740 1 -- 172.21.15.46:0/3835100612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b8012cb0 con 0x55eadfd9ecd0 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.347+0000 7f68b0c00640 1 -- 172.21.15.46:0/3835100612 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f68b8005250 con 0x55eadfd9ecd0 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.348+0000 7f68b0c00640 1 -- 172.21.15.46:0/3835100612 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f68b803a050 con 0x55eadfd9ecd0 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.348+0000 7f68d022f740 10 client.25598 fetch_fsmap learned FSMap version 33 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.348+0000 7f68d022f740 1 -- 172.21.15.46:0/3835100612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eadfe24b50 con 0x55eadfd9ecd0 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.348+0000 7f68b0c00640 1 -- 172.21.15.46:0/3835100612 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b8005450 con 0x55eadfd9ecd0 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.348+0000 7f68b0c00640 1 -- 172.21.15.46:0/3835100612 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f68b800f040 con 0x55eadfd9ecd0 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.348+0000 7f68d022f740 10 client.25598 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T07:37:33.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.348+0000 7f68d022f740 10 client.25598 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:37:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.349+0000 7f68d022f740 20 client.25598 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.349+0000 7f68d022f740 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eadfe253b0 0x55eadfe277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.349+0000 7f68d022f740 4 client.25598 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T07:37:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.349+0000 7f68d022f740 1 -- 172.21.15.46:0/3835100612 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f68b8012cb0 con 0x55eadfe253b0 2024-09-10T07:37:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.350+0000 7f68c6a00640 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eadfe253b0 0x55eadfe277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.354+0000 7f68c6a00640 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eadfe253b0 0x55eadfe277d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d77d9f8f9301eea server_cookie=bf27bea1180c9074 in_seq=0 out_seq=0 2024-09-10T07:37:33.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.354+0000 7f68b0c00640 10 client.25598 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:33.747 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:33 smithi046 ceph-mon[30705]: pgmap v427: 129 pgs: 129 active+clean; 898 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 412 op/s 2024-09-10T07:37:33.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:33 smithi181 ceph-mon[35397]: pgmap v427: 129 pgs: 129 active+clean; 898 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 412 op/s 2024-09-10T07:37:33.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:33 smithi063 ceph-mon[35020]: pgmap v427: 129 pgs: 129 active+clean; 898 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 412 op/s 2024-09-10T07:37:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.986+0000 7efcc2600640 1 -- 172.21.15.46:0/2140377243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc1019c0 msgr2=0x7efcbc101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.986+0000 7efcc2600640 1 --2- 172.21.15.46:0/2140377243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc1019c0 0x7efcbc101e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efca8009900 tx=0x7efca802a400 comp rx=0 tx=0).stop 2024-09-10T07:37:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.986+0000 7efcc2600640 1 -- 172.21.15.46:0/2140377243 shutdown_connections 2024-09-10T07:37:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.986+0000 7efcc2600640 1 --2- 172.21.15.46:0/2140377243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcbc102360 0x7efcbc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.986+0000 7efcc2600640 1 --2- 172.21.15.46:0/2140377243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc1019c0 0x7efcbc101e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.986+0000 7efcc2600640 1 --2- 172.21.15.46:0/2140377243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc1079c0 0x7efcbc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.986+0000 7efcc2600640 1 -- 172.21.15.46:0/2140377243 >> 172.21.15.46:0/2140377243 conn(0x7efcbc0fd740 msgr2=0x7efcbc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.986+0000 7efcc2600640 1 -- 172.21.15.46:0/2140377243 shutdown_connections 2024-09-10T07:37:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.988+0000 7efcc2600640 1 -- 172.21.15.46:0/2140377243 wait complete. 2024-09-10T07:37:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.988+0000 7efcc2600640 1 Processor -- start 2024-09-10T07:37:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc2600640 1 -- start start 2024-09-10T07:37:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc2600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcbc1019c0 0x7efcbc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc2600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc102360 0x7efcbc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc1079c0 0x7efcbc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc2600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc111e10 con 0x7efcbc1079c0 2024-09-10T07:37:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc2600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc111c90 con 0x7efcbc102360 2024-09-10T07:37:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc2600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc111f90 con 0x7efcbc1019c0 2024-09-10T07:37:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcbc1019c0 0x7efcbc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcbc1019c0 0x7efcbc19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53122/0 (socket says 172.21.15.46:53122) 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcc1200640 1 -- 172.21.15.46:0/105475732 learned_addr learned my addr 172.21.15.46:0/105475732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.989+0000 7efcbbe00640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc102360 0x7efcbc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.990+0000 7efcc1c00640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc1079c0 0x7efcbc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.990+0000 7efcbbe00640 1 -- 172.21.15.46:0/105475732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcbc1019c0 msgr2=0x7efcbc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.990+0000 7efcbbe00640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcbc1019c0 0x7efcbc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.990+0000 7efcbbe00640 1 -- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc1079c0 msgr2=0x7efcbc19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.990+0000 7efcbbe00640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc1079c0 0x7efcbc19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.990+0000 7efcbbe00640 1 -- 172.21.15.46:0/105475732 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcbc19f470 con 0x7efcbc102360 2024-09-10T07:37:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.990+0000 7efcbbe00640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc102360 0x7efcbc19aa00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efca802a920 tx=0x7efca8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.991+0000 7efcb9600640 1 -- 172.21.15.46:0/105475732 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca803f070 con 0x7efcbc102360 2024-09-10T07:37:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.991+0000 7efcb9600640 1 -- 172.21.15.46:0/105475732 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca8031c20 con 0x7efcbc102360 2024-09-10T07:37:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.991+0000 7efcc2600640 1 -- 172.21.15.46:0/105475732 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcbc19f700 con 0x7efcbc102360 2024-09-10T07:37:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.991+0000 7efcb9600640 1 -- 172.21.15.46:0/105475732 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8035030 con 0x7efcbc102360 2024-09-10T07:37:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.991+0000 7efcc2600640 1 -- 172.21.15.46:0/105475732 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcbc1a7040 con 0x7efcbc102360 2024-09-10T07:37:33.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.994+0000 7efcb9600640 1 -- 172.21.15.46:0/105475732 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efca8042070 con 0x7efcbc102360 2024-09-10T07:37:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.995+0000 7efcc6818740 1 Processor -- start 2024-09-10T07:37:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcc6818740 1 -- start start 2024-09-10T07:37:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcb9600640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efca40a2c00 0x7efca40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcb9600640 1 -- 172.21.15.46:0/105475732 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efca80efc60 con 0x7efcbc102360 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcc6818740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e168456220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcc6818740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e168460ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcc6818740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 0x55e168463410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcc6818740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e1683bc3e0 con 0x55e168455e40 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcc6818740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e168363d80 con 0x55e168456760 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcc6818740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e1683b83a0 con 0x55e168461020 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcbbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e168460ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcbbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e168460ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33804/0 (socket says 172.21.15.46:33804) 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.996+0000 7efcbbe00640 1 -- 172.21.15.46:0/755914869 learned_addr learned my addr 172.21.15.46:0/755914869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcc1200640 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e168456220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcbbe00640 1 -- 172.21.15.46:0/755914869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 msgr2=0x55e168463410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcbbe00640 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 0x55e168463410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcbbe00640 1 -- 172.21.15.46:0/755914869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 msgr2=0x55e168456220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcbbe00640 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e168456220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcbbe00640 1 -- 172.21.15.46:0/755914869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e168382ae0 con 0x55e168456760 2024-09-10T07:37:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcc1200640 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e168456220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcc1c00640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efca40a2c00 0x7efca40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.997+0000 7efcbbe00640 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e168460ae0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efca80c4820 tx=0x7efca8003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead385771e743c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.998+0000 7efcae000640 1 -- 172.21.15.46:0/755914869 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80c0490 con 0x55e168456760 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.998+0000 7efcae000640 1 -- 172.21.15.46:0/755914869 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca80c4d50 con 0x55e168456760 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.998+0000 7efcae000640 1 -- 172.21.15.46:0/755914869 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80c0120 con 0x55e168456760 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.999+0000 7efcc6818740 1 -- 172.21.15.46:0/755914869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 msgr2=0x55e168460ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.999+0000 7efcc6818740 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e168460ae0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efca80c4820 tx=0x7efca8003f20 comp rx=0 tx=0).stop 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.999+0000 7efcc6818740 1 -- 172.21.15.46:0/755914869 shutdown_connections 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.999+0000 7efcc6818740 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 0x55e168463410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.999+0000 7efcc6818740 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e168460ae0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.999+0000 7efcc6818740 1 --2- 172.21.15.46:0/755914869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e168456220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:33.999+0000 7efcc6818740 1 -- 172.21.15.46:0/755914869 >> 172.21.15.46:0/755914869 conn(0x55e1684523d0 msgr2=0x55e1684527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.000+0000 7efcc6818740 1 -- 172.21.15.46:0/755914869 shutdown_connections 2024-09-10T07:37:34.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.000+0000 7efcc6818740 1 -- 172.21.15.46:0/755914869 wait complete. 2024-09-10T07:37:34.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.000+0000 7efcc6818740 1 -- start start 2024-09-10T07:37:34.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.001+0000 7efcc6818740 1 Processor -- start 2024-09-10T07:37:34.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.001+0000 7efcc6818740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e16845df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.001+0000 7efcc6818740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168456760 0x55e16845e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.001+0000 7efcc6818740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168461020 0x55e16845e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.001+0000 7efcc6818740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e16846c1c0 con 0x55e168455e40 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.001+0000 7efcc6818740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e1683770f0 con 0x55e168461020 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.001+0000 7efcc6818740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e16846c340 con 0x55e168456760 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.001+0000 7efcc6818740 10 client.0 _ll_register_callbacks cb 0x55e16835ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcbbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168456760 0x55e16845e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcc1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e16845df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcbbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168456760 0x55e16845e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53132/0 (socket says 172.21.15.46:53132) 2024-09-10T07:37:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcbbe00640 1 -- 172.21.15.46:0/3884452882 learned_addr learned my addr 172.21.15.46:0/3884452882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcc1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e16845df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43636/0 (socket says 172.21.15.46:43636) 2024-09-10T07:37:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcc1c00640 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168461020 0x55e16845e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcc1c00640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efca40a2c00 0x7efca40a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55e168453750 tx=0x7efcb4009bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcbbe00640 1 -- 172.21.15.46:0/3884452882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168461020 msgr2=0x55e16845e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcbbe00640 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168461020 0x55e16845e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcbbe00640 1 -- 172.21.15.46:0/3884452882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 msgr2=0x55e16845df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcbbe00640 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e16845df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.002+0000 7efcbbe00640 1 -- 172.21.15.46:0/3884452882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e16812d3c0 con 0x55e168456760 2024-09-10T07:37:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.003+0000 7efcbbe00640 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168456760 0x55e16845e470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efca80b8540 tx=0x7efca80c0580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=858a07943bc9830 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.003+0000 7efca2a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.003+0000 7efca2a00640 1 -- 172.21.15.46:0/3884452882 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80d4020 con 0x55e168456760 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.003+0000 7efca2a00640 1 -- 172.21.15.46:0/3884452882 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efca80c0120 con 0x55e168456760 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.003+0000 7efcc6818740 1 -- 172.21.15.46:0/3884452882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca80c4d50 con 0x55e168456760 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.003+0000 7efca2a00640 1 -- 172.21.15.46:0/3884452882 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efca80b8750 con 0x55e168456760 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.003+0000 7efcc6818740 10 client.25603 fetch_fsmap learned FSMap version 33 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.003+0000 7efcc6818740 1 -- 172.21.15.46:0/3884452882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e16846c1c0 con 0x55e168456760 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.004+0000 7efca2a00640 1 -- 172.21.15.46:0/3884452882 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca80c4d50 con 0x55e168456760 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.004+0000 7efca2a00640 1 -- 172.21.15.46:0/3884452882 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7efca803bb80 con 0x55e168456760 2024-09-10T07:37:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.004+0000 7efcc6818740 10 client.25603 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T07:37:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.004+0000 7efcc6818740 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.004+0000 7efcc6818740 20 client.25603 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.004+0000 7efcc6818740 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e168479360 0x55e16847b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.004+0000 7efcc6818740 4 client.25603 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.005+0000 7efcc6818740 1 -- 172.21.15.46:0/3884452882 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1683c9950 con 0x55e168479360 2024-09-10T07:37:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.005+0000 7efcc1200640 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e168479360 0x55e16847b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.006+0000 7efcc1200640 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e168479360 0x55e16847b7a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cbdb5da82e71872 server_cookie=b5b54e184d63afc in_seq=0 out_seq=0 2024-09-10T07:37:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.006+0000 7efca2a00640 10 client.25603 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.008+0000 7efca2a00640 1 -- 172.21.15.46:0/3884452882 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1683c9950 con 0x55e168479360 2024-09-10T07:37:34.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.008+0000 7efca2a00640 10 client.25603 handle_command_reply: tid=0 2024-09-10T07:37:34.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.008+0000 7efcc6818740 1 client.25603 shutdown 2024-09-10T07:37:34.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.008+0000 7efcc6818740 1 -- 172.21.15.46:0/3884452882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168456760 msgr2=0x55e16845e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.008+0000 7efcc6818740 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168456760 0x55e16845e470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efca80b8540 tx=0x7efca80c0580 comp rx=0 tx=0).stop 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 1 -- 172.21.15.46:0/3884452882 shutdown_connections 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e168479360 0x55e16847b7a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168461020 0x55e16845e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168456760 0x55e16845e470 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 1 --2- 172.21.15.46:0/3884452882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168455e40 0x55e16845df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 1 -- 172.21.15.46:0/3884452882 >> 172.21.15.46:0/3884452882 conn(0x55e1684523d0 msgr2=0x55e168458100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efca2a00640 0 client.25603 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 1 -- 172.21.15.46:0/3884452882 shutdown_connections 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 1 -- 172.21.15.46:0/3884452882 wait complete. 2024-09-10T07:37:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.009+0000 7efcc6818740 20 client.25603 trim_cache size 0 max 16384 2024-09-10T07:37:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.024+0000 7efcc6818740 1 Processor -- start 2024-09-10T07:37:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.024+0000 7efcc6818740 1 -- start start 2024-09-10T07:37:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.024+0000 7efcc6818740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168455e40 0x55e1684588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.024+0000 7efcc6818740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e16845fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.024+0000 7efcc6818740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168461020 0x55e1684810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.024+0000 7efcc6818740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc1ac970 con 0x55e168461020 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.024+0000 7efcc6818740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc1ae540 con 0x55e168456760 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.024+0000 7efcc6818740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc1ad040 con 0x55e168455e40 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e16845fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e16845fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33808/0 (socket says 172.21.15.46:33808) 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 -- 172.21.15.46:0/259750085 learned_addr learned my addr 172.21.15.46:0/259750085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcc1200640 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168455e40 0x55e1684588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcc1c00640 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168461020 0x55e1684810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 -- 172.21.15.46:0/259750085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168455e40 msgr2=0x55e1684588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168455e40 0x55e1684588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 -- 172.21.15.46:0/259750085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168461020 msgr2=0x55e1684810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168461020 0x55e1684810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 -- 172.21.15.46:0/259750085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e168455c70 con 0x55e168456760 2024-09-10T07:37:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.025+0000 7efcbbe00640 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e16845fea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efca80096c0 tx=0x7efca8003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c91ab6facd6f1de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.026+0000 7efcad600640 1 -- 172.21.15.46:0/259750085 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80d4070 con 0x55e168456760 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.026+0000 7efcad600640 1 -- 172.21.15.46:0/259750085 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca80ba4a0 con 0x55e168456760 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.026+0000 7efcad600640 1 -- 172.21.15.46:0/259750085 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca803bb80 con 0x55e168456760 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.026+0000 7efcc6818740 1 -- 172.21.15.46:0/259750085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 msgr2=0x55e16845fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.026+0000 7efcc6818740 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e16845fea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efca80096c0 tx=0x7efca8003f20 comp rx=0 tx=0).stop 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 -- 172.21.15.46:0/259750085 shutdown_connections 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168461020 0x55e1684810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168456760 0x55e16845fea0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 --2- 172.21.15.46:0/259750085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168455e40 0x55e1684588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 -- 172.21.15.46:0/259750085 >> 172.21.15.46:0/259750085 conn(0x55e1684523d0 msgr2=0x55e16845f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 -- 172.21.15.46:0/259750085 shutdown_connections 2024-09-10T07:37:34.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 -- 172.21.15.46:0/259750085 wait complete. 2024-09-10T07:37:34.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 -- start start 2024-09-10T07:37:34.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.027+0000 7efcc6818740 1 Processor -- start 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.028+0000 7efcc6818740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168455e40 0x55e16846cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.028+0000 7efcc6818740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168456760 0x55e16846d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.028+0000 7efcc6818740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 0x55e1684e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168455e40 0x55e16846cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc6818740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc1adde0 con 0x55e168456760 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc6818740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc1ad710 con 0x55e168455e40 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168455e40 0x55e16846cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33816/0 (socket says 172.21.15.46:33816) 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1200640 1 -- 172.21.15.46:0/189274396 learned_addr learned my addr 172.21.15.46:0/189274396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcbbe00640 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168456760 0x55e16846d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1c00640 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 0x55e1684e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc6818740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc1aa970 con 0x55e168461020 2024-09-10T07:37:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc6818740 10 client.0 _ll_register_callbacks cb 0x7efca80bbf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1200640 1 -- 172.21.15.46:0/189274396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 msgr2=0x55e1684e2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1200640 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 0x55e1684e2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1200640 1 -- 172.21.15.46:0/189274396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168456760 msgr2=0x55e16846d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1200640 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168456760 0x55e16846d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.029+0000 7efcc1200640 1 -- 172.21.15.46:0/189274396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e168475390 con 0x55e168455e40 2024-09-10T07:37:34.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.030+0000 7efcc1200640 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168455e40 0x55e16846cbf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009c00 tx=0x7efcb0004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1edd3ee0269521 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.030+0000 7efca2a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:34.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.030+0000 7efca2a00640 1 -- 172.21.15.46:0/189274396 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0021020 con 0x55e168455e40 2024-09-10T07:37:34.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.030+0000 7efca2a00640 1 -- 172.21.15.46:0/189274396 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efcb001d340 con 0x55e168455e40 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.030+0000 7efcc6818740 1 -- 172.21.15.46:0/189274396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1683c3e60 con 0x55e168455e40 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.030+0000 7efca2a00640 1 -- 172.21.15.46:0/189274396 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efcb002c050 con 0x55e168455e40 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.030+0000 7efcc6818740 10 client.25616 fetch_fsmap learned FSMap version 33 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efcc6818740 1 -- 172.21.15.46:0/189274396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e1684e3750 con 0x55e168455e40 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efca2a00640 1 -- 172.21.15.46:0/189274396 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e1683c3e60 con 0x55e168455e40 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efca2a00640 1 -- 172.21.15.46:0/189274396 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7efcb0012820 con 0x55e168455e40 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efcc6818740 10 client.25616 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efcc6818740 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efcc6818740 20 client.25616 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efcc6818740 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e1684e3fb0 0x55e1684e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efcc6818740 4 client.25616 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:37:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.031+0000 7efcc6818740 1 -- 172.21.15.46:0/189274396 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e168340010 con 0x55e1684e3fb0 2024-09-10T07:37:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.032+0000 7efcbbe00640 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e1684e3fb0 0x55e1684e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.032+0000 7efcbbe00640 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e1684e3fb0 0x55e1684e63d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d398edbe6c04c69 server_cookie=9b5d9715cd605945 in_seq=0 out_seq=0 2024-09-10T07:37:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.032+0000 7efca2a00640 10 client.25616 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.035+0000 7efca2a00640 1 -- 172.21.15.46:0/189274396 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e168340010 con 0x55e1684e3fb0 2024-09-10T07:37:34.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.035+0000 7efca2a00640 10 client.25616 handle_command_reply: tid=0 2024-09-10T07:37:34.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.035+0000 7efcc6818740 1 client.25616 shutdown 2024-09-10T07:37:34.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 -- 172.21.15.46:0/189274396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168455e40 msgr2=0x55e16846cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168455e40 0x55e16846cbf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009c00 tx=0x7efcb0004410 comp rx=0 tx=0).stop 2024-09-10T07:37:34.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 -- 172.21.15.46:0/189274396 shutdown_connections 2024-09-10T07:37:34.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e1684e3fb0 0x55e1684e63d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e168461020 0x55e1684e2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e168456760 0x55e16846d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 --2- 172.21.15.46:0/189274396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e168455e40 0x55e16846cbf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 -- 172.21.15.46:0/189274396 >> 172.21.15.46:0/189274396 conn(0x55e1684523d0 msgr2=0x55e168474ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efca2a00640 0 client.25616 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 -- 172.21.15.46:0/189274396 shutdown_connections 2024-09-10T07:37:34.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 1 -- 172.21.15.46:0/189274396 wait complete. 2024-09-10T07:37:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.036+0000 7efcc6818740 20 client.25616 trim_cache size 0 max 16384 2024-09-10T07:37:34.039 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:37:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 -- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efca40a2c00 msgr2=0x7efca40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efca40a2c00 0x7efca40a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55e168453750 tx=0x7efcb4009bf0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 -- 172.21.15.46:0/105475732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc102360 msgr2=0x7efcbc19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc102360 0x7efcbc19aa00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efca802a920 tx=0x7efca8003b20 comp rx=0 tx=0).stop 2024-09-10T07:37:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 -- 172.21.15.46:0/105475732 shutdown_connections 2024-09-10T07:37:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efca40a2c00 0x7efca40a50c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc1079c0 0x7efcbc19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efcbc102360 0x7efcbc19aa00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 --2- 172.21.15.46:0/105475732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcbc1019c0 0x7efcbc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.037+0000 7efcc2600640 1 -- 172.21.15.46:0/105475732 >> 172.21.15.46:0/105475732 conn(0x7efcbc0fd740 msgr2=0x7efcbc0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.038+0000 7efcc2600640 1 -- 172.21.15.46:0/105475732 shutdown_connections 2024-09-10T07:37:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.038+0000 7efcc2600640 1 -- 172.21.15.46:0/105475732 wait complete. 2024-09-10T07:37:34.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68b0c00640 1 -- 172.21.15.46:0/3835100612 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f68b8012cb0 con 0x55eadfe253b0 2024-09-10T07:37:34.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68b0c00640 10 client.25598 handle_command_reply: tid=0 2024-09-10T07:37:34.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 client.25598 shutdown 2024-09-10T07:37:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 -- 172.21.15.46:0/3835100612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9ecd0 msgr2=0x55eadfdb1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9ecd0 0x55eadfdb1e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f68b8010d40 tx=0x7f68b8003770 comp rx=0 tx=0).stop 2024-09-10T07:37:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 -- 172.21.15.46:0/3835100612 shutdown_connections 2024-09-10T07:37:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eadfe253b0 0x55eadfe277d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eadfda9590 0x55eadfdadce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eadfd9ecd0 0x55eadfdb1e10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68b0c00640 0 client.25598 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 --2- 172.21.15.46:0/3835100612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eadfd9e3b0 0x55eadfdb18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.063+0000 7f68d022f740 1 -- 172.21.15.46:0/3835100612 >> 172.21.15.46:0/3835100612 conn(0x55eadfd9a940 msgr2=0x55eadfda7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.064+0000 7f68d022f740 1 -- 172.21.15.46:0/3835100612 shutdown_connections 2024-09-10T07:37:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.064+0000 7f68d022f740 1 -- 172.21.15.46:0/3835100612 wait complete. 2024-09-10T07:37:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.064+0000 7f68d022f740 20 client.25598 trim_cache size 0 max 16384 2024-09-10T07:37:34.068 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:34.068 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 33, 2024-09-10T07:37:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:37:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 2, 2024-09-10T07:37:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:37:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "3850f863": { 2024-09-10T07:37:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 2, 2024-09-10T07:37:34.069 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:37:34.070 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:37:34.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:37:34.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:37:34.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:37:34.070 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:37:34.070 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 150.0, 2024-09-10T07:37:34.070 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:37:34.071 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:37:34.071 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:37:34.071 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:37:34.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:37:34.071 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:37:34.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:34.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:34.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:34.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:34.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:34.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 -- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68ac0a2cd0 msgr2=0x7f68ac0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68ac0a2cd0 0x7f68ac0a5190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f68c810a9c0 tx=0x7f68b8005f90 comp rx=0 tx=0).stop 2024-09-10T07:37:34.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 -- 172.21.15.46:0/3463595392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8103fd0 msgr2=0x7f68c810e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8103fd0 0x7f68c810e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68b400b3f0 tx=0x7f68b400b8b0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 -- 172.21.15.46:0/3463595392 shutdown_connections 2024-09-10T07:37:34.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c8109100 0x7f68c81101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68ac0a2cd0 0x7f68ac0a5190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104980 0x7f68c810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 --2- 172.21.15.46:0/3463595392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c8103fd0 0x7f68c810e9f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 -- 172.21.15.46:0/3463595392 >> 172.21.15.46:0/3463595392 conn(0x7f68c80ffa30 msgr2=0x7f68c8100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 -- 172.21.15.46:0/3463595392 shutdown_connections 2024-09-10T07:37:34.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:34.065+0000 7f68cd800640 1 -- 172.21.15.46:0/3463595392 wait complete. 2024-09-10T07:37:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 -- 172.21.15.46:0/3404505822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe130103d60 msgr2=0x7fe130104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 --2- 172.21.15.46:0/3404505822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe130103d60 0x7fe130104160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe120009860 tx=0x7fe12002a340 comp rx=0 tx=0).stop 2024-09-10T07:37:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 -- 172.21.15.46:0/3404505822 shutdown_connections 2024-09-10T07:37:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 --2- 172.21.15.46:0/3404505822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe130105910 0x7fe13010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 --2- 172.21.15.46:0/3404505822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe130104f50 0x7fe1301053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 --2- 172.21.15.46:0/3404505822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe130103d60 0x7fe130104160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 -- 172.21.15.46:0/3404505822 >> 172.21.15.46:0/3404505822 conn(0x7fe1300ff4e0 msgr2=0x7fe130101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 -- 172.21.15.46:0/3404505822 shutdown_connections 2024-09-10T07:37:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.276+0000 7fe136c00640 1 -- 172.21.15.46:0/3404505822 wait complete. 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.277+0000 7fe136c00640 1 Processor -- start 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.277+0000 7fe136c00640 1 -- start start 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe136c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe130103d60 0x7fe130199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe136c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe130104f50 0x7fe13019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe136c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe130105910 0x7fe1301a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe136c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe13010f9c0 con 0x7fe130103d60 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe136c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe13010f840 con 0x7fe130104f50 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe136c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe13010fb40 con 0x7fe130105910 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe130103d60 0x7fe130199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe130103d60 0x7fe130199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43692/0 (socket says 172.21.15.46:43692) 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe12fe00640 1 -- 172.21.15.46:0/3129938719 learned_addr learned my addr 172.21.15.46:0/3129938719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe127e00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe130104f50 0x7fe13019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe134c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe130105910 0x7fe1301a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe12fe00640 1 -- 172.21.15.46:0/3129938719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe130105910 msgr2=0x7fe1301a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe12fe00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe130105910 0x7fe1301a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe12fe00640 1 -- 172.21.15.46:0/3129938719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe130104f50 msgr2=0x7fe13019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe12fe00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe130104f50 0x7fe13019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe12fe00640 1 -- 172.21.15.46:0/3129938719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1301a1bd0 con 0x7fe130103d60 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.278+0000 7fe134c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe130105910 0x7fe1301a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.279+0000 7fe12fe00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe130103d60 0x7fe130199fb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe120035040 tx=0x7fe120033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.279+0000 7fe12d600640 1 -- 172.21.15.46:0/3129938719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120037040 con 0x7fe130103d60 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.279+0000 7fe12d600640 1 -- 172.21.15.46:0/3129938719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe120030850 con 0x7fe130103d60 2024-09-10T07:37:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.279+0000 7fe136c00640 1 -- 172.21.15.46:0/3129938719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1301a1e60 con 0x7fe130103d60 2024-09-10T07:37:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.279+0000 7fe12d600640 1 -- 172.21.15.46:0/3129938719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120004920 con 0x7fe130103d60 2024-09-10T07:37:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.279+0000 7fe136c00640 1 -- 172.21.15.46:0/3129938719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1301a2270 con 0x7fe130103d60 2024-09-10T07:37:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe139c11740 1 Processor -- start 2024-09-10T07:37:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe12d600640 1 -- 172.21.15.46:0/3129938719 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe120004050 con 0x7fe130103d60 2024-09-10T07:37:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe139c11740 1 -- start start 2024-09-10T07:37:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe139c11740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe139c11740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe139c11740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 0x559cdc073f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe139c11740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559cdbfd1100 con 0x559cdc06c090 2024-09-10T07:37:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe139c11740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559cdbf791b0 con 0x559cdc06c9d0 2024-09-10T07:37:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.282+0000 7fe139c11740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559cdbfcd0c0 con 0x559cdc071a60 2024-09-10T07:37:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc071490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33832/0 (socket says 172.21.15.46:33832) 2024-09-10T07:37:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 -- 172.21.15.46:0/1452016065 learned_addr learned my addr 172.21.15.46:0/1452016065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe127e00640 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 0x559cdc073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe134c00640 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 -- 172.21.15.46:0/1452016065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 msgr2=0x559cdc073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 0x559cdc073f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 -- 172.21.15.46:0/1452016065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 msgr2=0x559cdc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 -- 172.21.15.46:0/1452016065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cdbce4210 con 0x559cdc06c9d0 2024-09-10T07:37:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.283+0000 7fe12fe00640 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc071490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1200f72f0 tx=0x7fe1200f73e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac1ff9021df179 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.284+0000 7fe12d600640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe1140a2c30 0x7fe1140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.284+0000 7fe125200640 1 -- 172.21.15.46:0/1452016065 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200b7a40 con 0x559cdc06c9d0 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.284+0000 7fe12d600640 1 -- 172.21.15.46:0/3129938719 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe1200ef320 con 0x7fe130103d60 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.284+0000 7fe125200640 1 -- 172.21.15.46:0/1452016065 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1200fc030 con 0x559cdc06c9d0 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.284+0000 7fe125200640 1 -- 172.21.15.46:0/1452016065 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200babf0 con 0x559cdc06c9d0 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.284+0000 7fe134c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe1140a2c30 0x7fe1140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 -- 172.21.15.46:0/1452016065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 msgr2=0x559cdc071490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc071490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1200f72f0 tx=0x7fe1200f73e0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe134c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe1140a2c30 0x7fe1140a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe128005fd0 tx=0x7fe1280095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 -- 172.21.15.46:0/1452016065 shutdown_connections 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 0x559cdc073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc071490 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 --2- 172.21.15.46:0/1452016065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 -- 172.21.15.46:0/1452016065 >> 172.21.15.46:0/1452016065 conn(0x559cdc067700 msgr2=0x559cdc067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 -- 172.21.15.46:0/1452016065 shutdown_connections 2024-09-10T07:37:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 -- 172.21.15.46:0/1452016065 wait complete. 2024-09-10T07:37:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.285+0000 7fe139c11740 1 -- start start 2024-09-10T07:37:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe139c11740 1 Processor -- start 2024-09-10T07:37:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe139c11740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe139c11740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe139c11740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc071a60 0x559cdc07d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe139c11740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559cdc0747f0 con 0x559cdc06c090 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe139c11740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559cdbf5a090 con 0x559cdc071a60 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe139c11740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559cdc074970 con 0x559cdc06c9d0 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe139c11740 10 client.0 _ll_register_callbacks cb 0x559cdbfaef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43702/0 (socket says 172.21.15.46:43702) 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe134c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc071a60 0x559cdc07d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe12fe00640 1 -- 172.21.15.46:0/3936839188 learned_addr learned my addr 172.21.15.46:0/3936839188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.286+0000 7fe127e00640 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe127e00640 1 -- 172.21.15.46:0/3936839188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc071a60 msgr2=0x559cdc07d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe127e00640 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc071a60 0x559cdc07d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe127e00640 1 -- 172.21.15.46:0/3936839188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 msgr2=0x559cdc075eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe127e00640 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc075eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe127e00640 1 -- 172.21.15.46:0/3936839188 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cdbdb0620 con 0x559cdc06c9d0 2024-09-10T07:37:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe12fe00640 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc075eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe127e00640 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc0763f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe13019b4c0 tx=0x7fe118012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4baec5a44fe1cc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe11d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe11d600640 1 -- 172.21.15.46:0/3936839188 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11800f920 con 0x559cdc06c9d0 2024-09-10T07:37:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe139c11740 1 -- 172.21.15.46:0/3936839188 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1200fc030 con 0x559cdc06c9d0 2024-09-10T07:37:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe11d600640 1 -- 172.21.15.46:0/3936839188 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe11800fac0 con 0x559cdc06c9d0 2024-09-10T07:37:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.287+0000 7fe11d600640 1 -- 172.21.15.46:0/3936839188 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe11800fcc0 con 0x559cdc06c9d0 2024-09-10T07:37:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe139c11740 10 client.25621 fetch_fsmap learned FSMap version 33 2024-09-10T07:37:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe139c11740 1 -- 172.21.15.46:0/3936839188 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559cdc080340 con 0x559cdc06c9d0 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe11d600640 1 -- 172.21.15.46:0/3936839188 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1200fc030 con 0x559cdc06c9d0 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe11d600640 1 -- 172.21.15.46:0/3936839188 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fe11801c420 con 0x559cdc06c9d0 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe139c11740 10 client.25621 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe139c11740 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe139c11740 20 client.25621 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe139c11740 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559cdc081af0 0x559cdc083f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe139c11740 4 client.25621 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe139c11740 1 -- 172.21.15.46:0/3936839188 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cdbfde830 con 0x559cdc081af0 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.288+0000 7fe12fe00640 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559cdc081af0 0x559cdc083f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.289+0000 7fe11d600640 10 client.25621 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.289+0000 7fe12fe00640 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559cdc081af0 0x559cdc083f00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55027460d444a4d6 server_cookie=25866839a03036e9 in_seq=0 out_seq=0 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe11d600640 1 -- 172.21.15.46:0/3936839188 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cdbfde830 con 0x559cdc081af0 2024-09-10T07:37:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe11d600640 10 client.25621 handle_command_reply: tid=0 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 client.25621 shutdown 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 -- 172.21.15.46:0/3936839188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 msgr2=0x559cdc0763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc0763f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe13019b4c0 tx=0x7fe118012b60 comp rx=0 tx=0).stop 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 -- 172.21.15.46:0/3936839188 shutdown_connections 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559cdc081af0 0x559cdc083f00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc071a60 0x559cdc07d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc0763f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 --2- 172.21.15.46:0/3936839188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc075eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe11d600640 0 client.25621 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 -- 172.21.15.46:0/3936839188 >> 172.21.15.46:0/3936839188 conn(0x559cdc067700 msgr2=0x559cdc068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 -- 172.21.15.46:0/3936839188 shutdown_connections 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 1 -- 172.21.15.46:0/3936839188 wait complete. 2024-09-10T07:37:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.290+0000 7fe139c11740 20 client.25621 trim_cache size 0 max 16384 2024-09-10T07:37:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.305+0000 7fe139c11740 1 Processor -- start 2024-09-10T07:37:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.305+0000 7fe139c11740 1 -- start start 2024-09-10T07:37:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.305+0000 7fe139c11740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c090 0x559cdc083ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe139c11740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe139c11740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc071a60 0x559cdc066770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe139c11740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1301ac820 con 0x559cdc071a60 2024-09-10T07:37:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe139c11740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1301ae3f0 con 0x559cdc06c090 2024-09-10T07:37:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe139c11740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe1301acef0 con 0x559cdc06c9d0 2024-09-10T07:37:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe134c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc071a60 0x559cdc066770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c090 0x559cdc083ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe134c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc071a60 0x559cdc066770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43704/0 (socket says 172.21.15.46:43704) 2024-09-10T07:37:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c090 0x559cdc083ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33852/0 (socket says 172.21.15.46:33852) 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe134c00640 1 -- 172.21.15.46:0/3746700031 learned_addr learned my addr 172.21.15.46:0/3746700031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe127e00640 1 --2- 172.21.15.46:0/3746700031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe134c00640 1 -- 172.21.15.46:0/3746700031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 msgr2=0x559cdc07d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe134c00640 1 --2- 172.21.15.46:0/3746700031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc07d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe134c00640 1 -- 172.21.15.46:0/3746700031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c090 msgr2=0x559cdc083ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe134c00640 1 --2- 172.21.15.46:0/3746700031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c090 0x559cdc083ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.306+0000 7fe134c00640 1 -- 172.21.15.46:0/3746700031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cdc067520 con 0x559cdc071a60 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe134c00640 1 --2- 172.21.15.46:0/3746700031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc071a60 0x559cdc066770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe12800da20 tx=0x7fe128019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62b8a94f08292e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe11fe00640 1 -- 172.21.15.46:0/3746700031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128003970 con 0x559cdc071a60 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe11fe00640 1 -- 172.21.15.46:0/3746700031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe128004020 con 0x559cdc071a60 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe11fe00640 1 -- 172.21.15.46:0/3746700031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12801d790 con 0x559cdc071a60 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe139c11740 1 -- 172.21.15.46:0/3746700031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc071a60 msgr2=0x559cdc066770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe139c11740 1 --2- 172.21.15.46:0/3746700031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc071a60 0x559cdc066770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe12800da20 tx=0x7fe128019000 comp rx=0 tx=0).stop 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe139c11740 1 -- 172.21.15.46:0/3746700031 shutdown_connections 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe139c11740 1 --2- 172.21.15.46:0/3746700031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc071a60 0x559cdc066770 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe139c11740 1 --2- 172.21.15.46:0/3746700031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc06c9d0 0x559cdc07d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe139c11740 1 --2- 172.21.15.46:0/3746700031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c090 0x559cdc083ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.307+0000 7fe139c11740 1 -- 172.21.15.46:0/3746700031 >> 172.21.15.46:0/3746700031 conn(0x559cdc067700 msgr2=0x559cdc066060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 -- 172.21.15.46:0/3746700031 shutdown_connections 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 -- 172.21.15.46:0/3746700031 wait complete. 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 -- start start 2024-09-10T07:37:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 Processor -- start 2024-09-10T07:37:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc07a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc07a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 0x559cdc0ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1301adc90 con 0x559cdc06c090 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1301ad5c0 con 0x559cdc06c9d0 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe1301aa800 con 0x559cdc071a60 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.308+0000 7fe139c11740 10 client.0 _ll_register_callbacks cb 0x7fe118009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc07a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe127e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc07a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc07a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43706/0 (socket says 172.21.15.46:43706) 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe127e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc07a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33854/0 (socket says 172.21.15.46:33854) 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 -- 172.21.15.46:0/52897071 learned_addr learned my addr 172.21.15.46:0/52897071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe134c00640 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 0x559cdc0ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 -- 172.21.15.46:0/52897071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 msgr2=0x559cdc0ea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 0x559cdc0ea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 -- 172.21.15.46:0/52897071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 msgr2=0x559cdc07a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc07a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 -- 172.21.15.46:0/52897071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cdc0ead10 con 0x559cdc06c090 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe127e00640 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc07a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe12fe00640 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc07a1d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe1200b7c00 tx=0x7fe12002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bacb275bf45b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.309+0000 7fe11d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe11d600640 1 -- 172.21.15.46:0/52897071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200baf20 con 0x559cdc06c090 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe11d600640 1 -- 172.21.15.46:0/52897071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe1200b9840 con 0x559cdc06c090 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 1 -- 172.21.15.46:0/52897071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe128004020 con 0x559cdc06c090 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe11d600640 1 -- 172.21.15.46:0/52897071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe1200bb100 con 0x559cdc06c090 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 10 client.15549 fetch_fsmap learned FSMap version 33 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 1 -- 172.21.15.46:0/52897071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559cdc0ebf70 con 0x559cdc06c090 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe11d600640 1 -- 172.21.15.46:0/52897071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe128004020 con 0x559cdc06c090 2024-09-10T07:37:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe11d600640 1 -- 172.21.15.46:0/52897071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fe12003a070 con 0x559cdc06c090 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 10 client.15549 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 20 client.15549 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559cdc0ec760 0x559cdc0eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 4 client.15549 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.310+0000 7fe139c11740 1 -- 172.21.15.46:0/52897071 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cdbfd8b80 con 0x559cdc0ec760 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.311+0000 7fe127e00640 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559cdc0ec760 0x559cdc0eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.311+0000 7fe127e00640 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559cdc0ec760 0x559cdc0eeb80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd580b9b4966bcdc server_cookie=6b21ce94c47d6a8f in_seq=0 out_seq=0 2024-09-10T07:37:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.311+0000 7fe11d600640 10 client.15549 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.346+0000 7fe11d600640 1 -- 172.21.15.46:0/52897071 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559cdbfd8b80 con 0x559cdc0ec760 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.346+0000 7fe11d600640 10 client.15549 handle_command_reply: tid=0 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.346+0000 7fe139c11740 1 client.15549 shutdown 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 -- 172.21.15.46:0/52897071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 msgr2=0x559cdc07a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc07a1d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe1200b7c00 tx=0x7fe12002a340 comp rx=0 tx=0).stop 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 -- 172.21.15.46:0/52897071 shutdown_connections 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559cdc0ec760 0x559cdc0eeb80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559cdc071a60 0x559cdc0ea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc06c9d0 0x559cdc07a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe11d600640 0 client.15549 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 --2- 172.21.15.46:0/52897071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cdc06c090 0x559cdc07a1d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 -- 172.21.15.46:0/52897071 >> 172.21.15.46:0/52897071 conn(0x559cdc067700 msgr2=0x559cdc066060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 -- 172.21.15.46:0/52897071 shutdown_connections 2024-09-10T07:37:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 1 -- 172.21.15.46:0/52897071 wait complete. 2024-09-10T07:37:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.347+0000 7fe139c11740 20 client.15549 trim_cache size 0 max 16384 2024-09-10T07:37:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:37:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:37:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:37:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 -- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe1140a2c30 msgr2=0x7fe1140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe1140a2c30 0x7fe1140a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe128005fd0 tx=0x7fe1280095c0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 -- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe130103d60 msgr2=0x7fe130199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe130103d60 0x7fe130199fb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe120035040 tx=0x7fe120033d90 comp rx=0 tx=0).stop 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 -- 172.21.15.46:0/3129938719 shutdown_connections 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe130105910 0x7fe1301a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe130104f50 0x7fe13019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe1140a2c30 0x7fe1140a50f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 --2- 172.21.15.46:0/3129938719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe130103d60 0x7fe130199fb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 -- 172.21.15.46:0/3129938719 >> 172.21.15.46:0/3129938719 conn(0x7fe1300ff4e0 msgr2=0x7fe1301082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 -- 172.21.15.46:0/3129938719 shutdown_connections 2024-09-10T07:37:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:34.348+0000 7fe136c00640 1 -- 172.21.15.46:0/3129938719 wait complete. 2024-09-10T07:37:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 15.0 s, 1010.3 tps, lat 55.454 ms stddev 65.130 2024-09-10T07:37:35.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:35 smithi181 ceph-mon[35397]: pgmap v428: 129 pgs: 129 active+clean; 901 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.5 MiB/s wr, 452 op/s 2024-09-10T07:37:35.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:35 smithi063 ceph-mon[35020]: pgmap v428: 129 pgs: 129 active+clean; 901 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.5 MiB/s wr, 452 op/s 2024-09-10T07:37:35.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:35 smithi046 ceph-mon[30705]: pgmap v428: 129 pgs: 129 active+clean; 901 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.5 MiB/s wr, 452 op/s 2024-09-10T07:37:36.171 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:37:36.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:37:36.612 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3850f863', quiesce took 10.1 seconds. Will release after: 19.9 2024-09-10T07:37:36.695 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:36.834 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:36 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:36.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:36 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:36 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 -- 172.21.15.46:0/3626483271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 msgr2=0x7f88981089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 --2- 172.21.15.46:0/3626483271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 0x7f88981089f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8884009900 tx=0x7f888402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 -- 172.21.15.46:0/3626483271 shutdown_connections 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 --2- 172.21.15.46:0/3626483271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8898109120 0x7f8898110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 --2- 172.21.15.46:0/3626483271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 0x7f88981089f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 --2- 172.21.15.46:0/3626483271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8898103ff0 0x7f88981043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 -- 172.21.15.46:0/3626483271 >> 172.21.15.46:0/3626483271 conn(0x7f88980ffa90 msgr2=0x7f8898101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 -- 172.21.15.46:0/3626483271 shutdown_connections 2024-09-10T07:37:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.286+0000 7f889fa00640 1 -- 172.21.15.46:0/3626483271 wait complete. 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.287+0000 7f889fa00640 1 Processor -- start 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.287+0000 7f889fa00640 1 -- start start 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.287+0000 7f889fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8898103ff0 0x7f889819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.287+0000 7f889fa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 0x7f889819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.287+0000 7f889fa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8898109120 0x7f889819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.287+0000 7f889fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8898114140 con 0x7f8898103ff0 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.287+0000 7f889fa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8898113fc0 con 0x7f8898109120 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.287+0000 7f889fa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88981142c0 con 0x7f88981049a0 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f8897e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 0x7f889819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f889d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8898103ff0 0x7f889819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f8897e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 0x7f889819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53654/0 (socket says 172.21.15.46:53654) 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f8897e00640 1 -- 172.21.15.46:0/2425831337 learned_addr learned my addr 172.21.15.46:0/2425831337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f889da00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8898109120 0x7f889819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f889da00640 1 -- 172.21.15.46:0/2425831337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 msgr2=0x7f889819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f889da00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 0x7f889819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f889da00640 1 -- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8898103ff0 msgr2=0x7f889819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f889da00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8898103ff0 0x7f889819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f889da00640 1 -- 172.21.15.46:0/2425831337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889819f5d0 con 0x7f8898109120 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.288+0000 7f889da00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8898109120 0x7f889819eef0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8890009fd0 tx=0x7f889000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.289+0000 7f8895600640 1 -- 172.21.15.46:0/2425831337 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889000ce20 con 0x7f8898109120 2024-09-10T07:37:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.289+0000 7f889fa00640 1 -- 172.21.15.46:0/2425831337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889819f8c0 con 0x7f8898109120 2024-09-10T07:37:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.289+0000 7f8895600640 1 -- 172.21.15.46:0/2425831337 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8890005020 con 0x7f8898109120 2024-09-10T07:37:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.289+0000 7f8895600640 1 -- 172.21.15.46:0/2425831337 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890010720 con 0x7f8898109120 2024-09-10T07:37:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.289+0000 7f889fa00640 1 -- 172.21.15.46:0/2425831337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88981a71a0 con 0x7f8898109120 2024-09-10T07:37:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.292+0000 7f88a262e740 1 Processor -- start 2024-09-10T07:37:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.292+0000 7f8895600640 1 -- 172.21.15.46:0/2425831337 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8890005540 con 0x7f8898109120 2024-09-10T07:37:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.292+0000 7f88a262e740 1 -- start start 2024-09-10T07:37:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f88a262e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac6cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f88a262e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac77760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f88a262e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac7a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f88a262e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5611babd3290 con 0x5611bac77ca0 2024-09-10T07:37:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f88a262e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5611bab7ab40 con 0x5611bac6cac0 2024-09-10T07:37:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f88a262e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5611babcf250 con 0x5611bac6d3e0 2024-09-10T07:37:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f8897e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac6cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f8897e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac6cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40002/0 (socket says 172.21.15.46:40002) 2024-09-10T07:37:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f8897e00640 1 -- 172.21.15.46:0/1478942310 learned_addr learned my addr 172.21.15.46:0/1478942310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f889d000640 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac77760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f8897e00640 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac7a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f889d000640 1 -- 172.21.15.46:0/1478942310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 msgr2=0x5611bac6cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f889d000640 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac6cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f889d000640 1 -- 172.21.15.46:0/1478942310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 msgr2=0x5611bac7a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.293+0000 7f889d000640 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac7a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.294+0000 7f889d000640 1 -- 172.21.15.46:0/1478942310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ba9446a0 con 0x5611bac6d3e0 2024-09-10T07:37:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.294+0000 7f8895600640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88800a2b80 0x7f88800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.294+0000 7f889d000640 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac77760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f888c009c00 tx=0x7f888c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caaa8dc4dbdc2a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.294+0000 7f8895600640 1 -- 172.21.15.46:0/2425831337 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f88900cfaa0 con 0x7f8898109120 2024-09-10T07:37:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.294+0000 7f8897e00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88800a2b80 0x7f88800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.294+0000 7f888a000640 1 -- 172.21.15.46:0/1478942310 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c004230 con 0x5611bac6d3e0 2024-09-10T07:37:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.294+0000 7f888a000640 1 -- 172.21.15.46:0/1478942310 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888c0043d0 con 0x5611bac6d3e0 2024-09-10T07:37:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.294+0000 7f888a000640 1 -- 172.21.15.46:0/1478942310 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c004ce0 con 0x5611bac6d3e0 2024-09-10T07:37:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 -- 172.21.15.46:0/1478942310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 msgr2=0x5611bac77760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f8897e00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88800a2b80 0x7f88800a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5611bac6a3d0 tx=0x7f888402a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac77760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f888c009c00 tx=0x7f888c011810 comp rx=0 tx=0).stop 2024-09-10T07:37:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 -- 172.21.15.46:0/1478942310 shutdown_connections 2024-09-10T07:37:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac7a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac77760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 --2- 172.21.15.46:0/1478942310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac6cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 -- 172.21.15.46:0/1478942310 >> 172.21.15.46:0/1478942310 conn(0x5611bac69050 msgr2=0x5611bac69430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 -- 172.21.15.46:0/1478942310 shutdown_connections 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.295+0000 7f88a262e740 1 -- 172.21.15.46:0/1478942310 wait complete. 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.296+0000 7f88a262e740 1 -- start start 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.296+0000 7f88a262e740 1 Processor -- start 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.296+0000 7f88a262e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6cac0 0x5611bac77540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.296+0000 7f88a262e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6d3e0 0x5611bac747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.296+0000 7f88a262e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac74d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.296+0000 7f88a262e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5611bac7a910 con 0x5611bac77ca0 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.296+0000 7f88a262e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5611bab8dfa0 con 0x5611bac6d3e0 2024-09-10T07:37:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.296+0000 7f88a262e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5611bac7aa90 con 0x5611bac6cac0 2024-09-10T07:37:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f88a262e740 10 client.0 _ll_register_callbacks cb 0x5611ba8e1ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f8897e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6d3e0 0x5611bac747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f8897e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6d3e0 0x5611bac747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40004/0 (socket says 172.21.15.46:40004) 2024-09-10T07:37:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f8897e00640 1 -- 172.21.15.46:0/2837549345 learned_addr learned my addr 172.21.15.46:0/2837549345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f889d000640 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6cac0 0x5611bac77540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f889da00640 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac74d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f8897e00640 1 -- 172.21.15.46:0/2837549345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6cac0 msgr2=0x5611bac77540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f8897e00640 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6cac0 0x5611bac77540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f8897e00640 1 -- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 msgr2=0x5611bac74d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f8897e00640 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac74d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f8897e00640 1 -- 172.21.15.46:0/2837549345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611bac75530 con 0x5611bac6d3e0 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.297+0000 7f889da00640 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac74d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f8897e00640 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6d3e0 0x5611bac747f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8884009230 tx=0x7f8884009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d411d03e6ab99796 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f887ea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f887ea00640 1 -- 172.21.15.46:0/2837549345 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888404f070 con 0x5611bac6d3e0 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f88a262e740 1 -- 172.21.15.46:0/2837549345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f888c0043d0 con 0x5611bac6d3e0 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f887ea00640 1 -- 172.21.15.46:0/2837549345 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8884004940 con 0x5611bac6d3e0 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f887ea00640 1 -- 172.21.15.46:0/2837549345 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f888405b050 con 0x5611bac6d3e0 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f88a262e740 10 client.25646 fetch_fsmap learned FSMap version 34 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f88a262e740 1 -- 172.21.15.46:0/2837549345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5611bac867e0 con 0x5611bac6d3e0 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.298+0000 7f887ea00640 1 -- 172.21.15.46:0/2837549345 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888c0043d0 con 0x5611bac6d3e0 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.299+0000 7f887ea00640 1 -- 172.21.15.46:0/2837549345 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f888403db60 con 0x5611bac6d3e0 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.299+0000 7f88a262e740 10 client.25646 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.299+0000 7f88a262e740 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.299+0000 7f88a262e740 20 client.25646 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.299+0000 7f88a262e740 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5611bac87c60 0x5611bac8a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.299+0000 7f88a262e740 4 client.25646 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.299+0000 7f88a262e740 1 -- 172.21.15.46:0/2837549345 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611babe0800 con 0x5611bac87c60 2024-09-10T07:37:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.299+0000 7f889d000640 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5611bac87c60 0x5611bac8a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.300+0000 7f889d000640 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5611bac87c60 0x5611bac8a0c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6918f33725ad26f2 server_cookie=b1d36bf1975c8be7 in_seq=0 out_seq=0 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.300+0000 7f887ea00640 10 client.25646 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.300+0000 7f887ea00640 1 -- 172.21.15.46:0/2837549345 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611babe0800 con 0x5611bac87c60 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.300+0000 7f887ea00640 10 client.25646 handle_command_reply: tid=0 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.300+0000 7f88a262e740 1 client.25646 shutdown 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 -- 172.21.15.46:0/2837549345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6d3e0 msgr2=0x5611bac747f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6d3e0 0x5611bac747f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8884009230 tx=0x7f8884009260 comp rx=0 tx=0).stop 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 -- 172.21.15.46:0/2837549345 shutdown_connections 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5611bac87c60 0x5611bac8a0c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac74d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6d3e0 0x5611bac747f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 --2- 172.21.15.46:0/2837549345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6cac0 0x5611bac77540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f887ea00640 0 client.25646 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:37.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 -- 172.21.15.46:0/2837549345 >> 172.21.15.46:0/2837549345 conn(0x5611bac69050 msgr2=0x5611bac6ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:37.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 -- 172.21.15.46:0/2837549345 shutdown_connections 2024-09-10T07:37:37.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 1 -- 172.21.15.46:0/2837549345 wait complete. 2024-09-10T07:37:37.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.301+0000 7f88a262e740 20 client.25646 trim_cache size 0 max 16384 2024-09-10T07:37:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.319+0000 7f88a262e740 1 Processor -- start 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.319+0000 7f88a262e740 1 -- start start 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f88a262e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac8fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f88a262e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac6d3e0 0x5611bac8ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f88a262e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac77ca0 0x5611bac7fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f88a262e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88981b0dc0 con 0x5611bac6d3e0 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f88a262e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88981b29c0 con 0x5611bac6cac0 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f88a262e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88981b1490 con 0x5611bac77ca0 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f8897e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac6d3e0 0x5611bac8ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f8897e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac6d3e0 0x5611bac8ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39952/0 (socket says 172.21.15.46:39952) 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f8897e00640 1 -- 172.21.15.46:0/3305833251 learned_addr learned my addr 172.21.15.46:0/3305833251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f889da00640 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac77ca0 0x5611bac7fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f889d000640 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac8fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f8897e00640 1 -- 172.21.15.46:0/3305833251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac77ca0 msgr2=0x5611bac7fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f8897e00640 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac77ca0 0x5611bac7fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f8897e00640 1 -- 172.21.15.46:0/3305833251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 msgr2=0x5611bac8fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f8897e00640 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac8fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.320+0000 7f8897e00640 1 -- 172.21.15.46:0/3305833251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611bac68e80 con 0x5611bac6d3e0 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.321+0000 7f8897e00640 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac6d3e0 0x5611bac8ff60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8884005ec0 tx=0x7f888402aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb487093d48fa7ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.321+0000 7f8889600640 1 -- 172.21.15.46:0/3305833251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884063070 con 0x5611bac6d3e0 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.321+0000 7f8889600640 1 -- 172.21.15.46:0/3305833251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8884004c20 con 0x5611bac6d3e0 2024-09-10T07:37:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.321+0000 7f8889600640 1 -- 172.21.15.46:0/3305833251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888404f050 con 0x5611bac6d3e0 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 -- 172.21.15.46:0/3305833251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac6d3e0 msgr2=0x5611bac8ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac6d3e0 0x5611bac8ff60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8884005ec0 tx=0x7f888402aaf0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 -- 172.21.15.46:0/3305833251 shutdown_connections 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac77ca0 0x5611bac7fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac6d3e0 0x5611bac8ff60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 --2- 172.21.15.46:0/3305833251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac8fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 -- 172.21.15.46:0/3305833251 >> 172.21.15.46:0/3305833251 conn(0x5611bac69050 msgr2=0x5611bac738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 -- 172.21.15.46:0/3305833251 shutdown_connections 2024-09-10T07:37:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 -- 172.21.15.46:0/3305833251 wait complete. 2024-09-10T07:37:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.322+0000 7f88a262e740 1 -- start start 2024-09-10T07:37:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f88a262e740 1 Processor -- start 2024-09-10T07:37:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f88a262e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac80550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f88a262e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac7d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f88a262e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac7df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f88a262e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88981b2230 con 0x5611bac77ca0 2024-09-10T07:37:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f88a262e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88981b1b60 con 0x5611bac6cac0 2024-09-10T07:37:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f88a262e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88981aeda0 con 0x5611bac6d3e0 2024-09-10T07:37:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f88a262e740 10 client.0 _ll_register_callbacks cb 0x7f8884041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f889d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac80550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f889da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac7df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.323+0000 7f889d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac80550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40026/0 (socket says 172.21.15.46:40026) 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f889d000640 1 -- 172.21.15.46:0/3077882248 learned_addr learned my addr 172.21.15.46:0/3077882248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f8897e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac7d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f8897e00640 1 -- 172.21.15.46:0/3077882248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 msgr2=0x5611bac80550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f8897e00640 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac80550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f8897e00640 1 -- 172.21.15.46:0/3077882248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 msgr2=0x5611bac7df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f8897e00640 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac7df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f8897e00640 1 -- 172.21.15.46:0/3077882248 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611bac7e760 con 0x5611bac6d3e0 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f889d000640 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac80550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f8897e00640 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac7d940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8884009a20 tx=0x7f8884005490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c92fcff59f63c18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f887ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.324+0000 7f887ea00640 1 -- 172.21.15.46:0/3077882248 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884063070 con 0x5611bac6d3e0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 1 -- 172.21.15.46:0/3077882248 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8884004c20 con 0x5611bac6d3e0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f887ea00640 1 -- 172.21.15.46:0/3077882248 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8884004940 con 0x5611bac6d3e0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f887ea00640 1 -- 172.21.15.46:0/3077882248 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8884079050 con 0x5611bac6d3e0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 10 client.25645 fetch_fsmap learned FSMap version 34 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 1 -- 172.21.15.46:0/3077882248 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5611bacf2b50 con 0x5611bac6d3e0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f887ea00640 1 -- 172.21.15.46:0/3077882248 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8884004c20 con 0x5611bac6d3e0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f887ea00640 1 -- 172.21.15.46:0/3077882248 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f888403d940 con 0x5611bac6d3e0 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 10 client.25645 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 10 client.25645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 20 client.25645 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5611bacf3270 0x5611bacf5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 4 client.25645 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:37:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.325+0000 7f88a262e740 1 -- 172.21.15.46:0/3077882248 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5611babdad10 con 0x5611bacf3270 2024-09-10T07:37:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.326+0000 7f889d000640 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5611bacf3270 0x5611bacf5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.326+0000 7f889d000640 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5611bacf3270 0x5611bacf5690 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e282621231f3f4e server_cookie=ae443461468786fb in_seq=0 out_seq=0 2024-09-10T07:37:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.326+0000 7f887ea00640 10 client.25645 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f887ea00640 1 -- 172.21.15.46:0/3077882248 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5611babdad10 con 0x5611bacf3270 2024-09-10T07:37:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f887ea00640 10 client.25645 handle_command_reply: tid=0 2024-09-10T07:37:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 client.25645 shutdown 2024-09-10T07:37:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 -- 172.21.15.46:0/3077882248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 msgr2=0x5611bac7d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac7d940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8884009a20 tx=0x7f8884005490 comp rx=0 tx=0).stop 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 -- 172.21.15.46:0/3077882248 shutdown_connections 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5611bacf3270 0x5611bacf5690 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611bac77ca0 0x5611bac7df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5611bac6d3e0 0x5611bac7d940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 --2- 172.21.15.46:0/3077882248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5611bac6cac0 0x5611bac80550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f88a262e740 1 -- 172.21.15.46:0/3077882248 >> 172.21.15.46:0/3077882248 conn(0x5611bac69050 msgr2=0x5611bac76050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.327+0000 7f887ea00640 0 client.25645 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.328+0000 7f88a262e740 1 -- 172.21.15.46:0/3077882248 shutdown_connections 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.328+0000 7f88a262e740 1 -- 172.21.15.46:0/3077882248 wait complete. 2024-09-10T07:37:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.329+0000 7f88a262e740 20 client.25645 trim_cache size 0 max 16384 2024-09-10T07:37:37.334 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:37:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "0b5d4fbd-2f62-42f5-834c-af475c257438", 2024-09-10T07:37:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:37:37.334 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.329+0000 7f889fa00640 1 -- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88800a2b80 msgr2=0x7f88800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88800a2b80 0x7f88800a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5611bac6a3d0 tx=0x7f888402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 -- 172.21.15.46:0/2425831337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8898109120 msgr2=0x7f889819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8898109120 0x7f889819eef0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8890009fd0 tx=0x7f889000ef10 comp rx=0 tx=0).stop 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 -- 172.21.15.46:0/2425831337 shutdown_connections 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8898109120 0x7f889819eef0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88981049a0 0x7f889819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88800a2b80 0x7f88800a5040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 --2- 172.21.15.46:0/2425831337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8898103ff0 0x7f889819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 -- 172.21.15.46:0/2425831337 >> 172.21.15.46:0/2425831337 conn(0x7f88980ffa90 msgr2=0x7f8898100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 -- 172.21.15.46:0/2425831337 shutdown_connections 2024-09-10T07:37:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:37.330+0000 7f889fa00640 1 -- 172.21.15.46:0/2425831337 wait complete. 2024-09-10T07:37:37.852 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: pgmap v429: 129 pgs: 129 active+clean; 910 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-09-10T07:37:37.852 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:37:37.852 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:37:37.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:37:37.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:37:37.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.853 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: pgmap v429: 129 pgs: 129 active+clean; 910 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-09-10T07:37:37.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:37:37.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:37:37.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:37:37.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:37:37.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: pgmap v429: 129 pgs: 129 active+clean; 910 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-09-10T07:37:37.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:37:37.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:37:37.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:37:37.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:37:37.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:37.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:37:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:38.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:38 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:37:38.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:38 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:37:38.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:38 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:38 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:38 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:38 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:38 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:38 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:38 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:38 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:38 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:38 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:38 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:37:38.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:38 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:37:38.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:38 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:38 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:37:38.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:38 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:38.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:38 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:37:39.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.309+0000 7fadbe200640 1 -- 172.21.15.46:0/523216903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074f40 msgr2=0x7fadb810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.309+0000 7fadbe200640 1 --2- 172.21.15.46:0/523216903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074f40 0x7fadb810de70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fada4009900 tx=0x7fada402a480 comp rx=0 tx=0).stop 2024-09-10T07:37:39.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.309+0000 7fadbe200640 1 -- 172.21.15.46:0/523216903 shutdown_connections 2024-09-10T07:37:39.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.309+0000 7fadbe200640 1 --2- 172.21.15.46:0/523216903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadb810e3b0 0x7fadb8110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.309+0000 7fadbe200640 1 --2- 172.21.15.46:0/523216903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074f40 0x7fadb810de70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.309+0000 7fadbe200640 1 --2- 172.21.15.46:0/523216903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb8074620 0x7fadb8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.309+0000 7fadbe200640 1 -- 172.21.15.46:0/523216903 >> 172.21.15.46:0/523216903 conn(0x7fadb80fd890 msgr2=0x7fadb80ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:39.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.309+0000 7fadbe200640 1 -- 172.21.15.46:0/523216903 shutdown_connections 2024-09-10T07:37:39.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.310+0000 7fadbe200640 1 -- 172.21.15.46:0/523216903 wait complete. 2024-09-10T07:37:39.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.310+0000 7fadbe200640 1 Processor -- start 2024-09-10T07:37:39.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadbe200640 1 -- start start 2024-09-10T07:37:39.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadbe200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074620 0x7fadb819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadbe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadb8074f40 0x7fadb819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadbe200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb810e3b0 0x7fadb819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadbe200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8114180 con 0x7fadb8074f40 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadbe200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8114000 con 0x7fadb8074620 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadbe200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8114300 con 0x7fadb810e3b0 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadb7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074620 0x7fadb819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadb8074f40 0x7fadb819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadb7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074620 0x7fadb819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40046/0 (socket says 172.21.15.46:40046) 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadb7400640 1 -- 172.21.15.46:0/1699037183 learned_addr learned my addr 172.21.15.46:0/1699037183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.311+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb810e3b0 0x7fadb819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadb7400640 1 -- 172.21.15.46:0/1699037183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb810e3b0 msgr2=0x7fadb819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadb7400640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb810e3b0 0x7fadb819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadb7400640 1 -- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadb8074f40 msgr2=0x7fadb819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadb7400640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadb8074f40 0x7fadb819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadb7400640 1 -- 172.21.15.46:0/1699037183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb819f640 con 0x7fadb8074620 2024-09-10T07:37:39.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadb7400640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074620 0x7fadb819a690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fada800d950 tx=0x7fada800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:39.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadafe00640 1 -- 172.21.15.46:0/1699037183 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8017030 con 0x7fadb8074620 2024-09-10T07:37:39.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadafe00640 1 -- 172.21.15.46:0/1699037183 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada8004020 con 0x7fadb8074620 2024-09-10T07:37:39.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadafe00640 1 -- 172.21.15.46:0/1699037183 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada80047a0 con 0x7fadb8074620 2024-09-10T07:37:39.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.312+0000 7fadbe200640 1 -- 172.21.15.46:0/1699037183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb819f930 con 0x7fadb8074620 2024-09-10T07:37:39.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.313+0000 7fadbe200640 1 -- 172.21.15.46:0/1699037183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb81a7210 con 0x7fadb8074620 2024-09-10T07:37:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.315+0000 7fadc0dd8740 1 Processor -- start 2024-09-10T07:37:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadafe00640 1 -- 172.21.15.46:0/1699037183 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fada80041d0 con 0x7fadb8074620 2024-09-10T07:37:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadc0dd8740 1 -- start start 2024-09-10T07:37:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f213a0 0x555e51f21780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f2c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f2c580 0x555e51f2e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555e51e7f3e0 con 0x555e51f21cc0 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555e51e26d80 con 0x555e51f213a0 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555e51e7b3a0 con 0x555e51f2c580 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f2c580 0x555e51f2e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f2c580 0x555e51f2e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53736/0 (socket says 172.21.15.46:53736) 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadb6a00640 1 -- 172.21.15.46:0/2551841253 learned_addr learned my addr 172.21.15.46:0/2551841253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.316+0000 7fadb7400640 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f2c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fadb7e00640 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f213a0 0x555e51f21780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fadb6a00640 1 -- 172.21.15.46:0/2551841253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f213a0 msgr2=0x555e51f21780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fadb6a00640 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f213a0 0x555e51f21780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fadb6a00640 1 -- 172.21.15.46:0/2551841253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 msgr2=0x555e51f2c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fadb6a00640 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f2c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fadb6a00640 1 -- 172.21.15.46:0/2551841253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e51e45ae0 con 0x555e51f2c580 2024-09-10T07:37:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fadb6a00640 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f2c580 0x555e51f2e970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fada4009900 tx=0x7fada4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b7ad554ecb96a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fada3e00640 1 -- 172.21.15.46:0/2551841253 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4035040 con 0x555e51f2c580 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fada3e00640 1 -- 172.21.15.46:0/2551841253 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada4005520 con 0x555e51f2c580 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.317+0000 7fada3e00640 1 -- 172.21.15.46:0/2551841253 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4003850 con 0x555e51f2c580 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.318+0000 7fadafe00640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad9c0a2c00 0x7fad9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.318+0000 7fadafe00640 1 -- 172.21.15.46:0/1699037183 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fada80cece0 con 0x7fadb8074620 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.318+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad9c0a2c00 0x7fad9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.318+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2551841253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f2c580 msgr2=0x555e51f2e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.318+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f2c580 0x555e51f2e970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fada4009900 tx=0x7fada4004040 comp rx=0 tx=0).stop 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2551841253 shutdown_connections 2024-09-10T07:37:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f2c580 0x555e51f2e970 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f2c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/2551841253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f213a0 0x555e51f21780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2551841253 >> 172.21.15.46:0/2551841253 conn(0x555e51f1d930 msgr2=0x555e51f1dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad9c0a2c00 0x7fad9c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fadb0005fd0 tx=0x7fadb0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2551841253 shutdown_connections 2024-09-10T07:37:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2551841253 wait complete. 2024-09-10T07:37:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 -- start start 2024-09-10T07:37:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.319+0000 7fadc0dd8740 1 Processor -- start 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f213a0 0x555e51f29420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f21cc0 0x555e51f29960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f29ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555e51f2f1f0 con 0x555e51f213a0 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555e51e3a0f0 con 0x555e51f2c580 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555e51f2f370 con 0x555e51f21cc0 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadc0dd8740 10 client.0 _ll_register_callbacks cb 0x555e51e2b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadb7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f213a0 0x555e51f29420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f21cc0 0x555e51f29960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadb7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f213a0 0x555e51f29420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40004/0 (socket says 172.21.15.46:40004) 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadb7400640 1 -- 172.21.15.46:0/1304787614 learned_addr learned my addr 172.21.15.46:0/1304787614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.320+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f29ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadb7e00640 1 -- 172.21.15.46:0/1304787614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f21cc0 msgr2=0x555e51f29960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f21cc0 0x555e51f29960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadb7e00640 1 -- 172.21.15.46:0/1304787614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f213a0 msgr2=0x555e51f29420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f213a0 0x555e51f29420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadb7e00640 1 -- 172.21.15.46:0/1304787614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e51bf03c0 con 0x555e51f2c580 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadb6a00640 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f21cc0 0x555e51f29960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadb7400640 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f213a0 0x555e51f29420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f29ea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fadb0003f40 tx=0x7fadb0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97531f89472ce103 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fada0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fada0c00640 1 -- 172.21.15.46:0/1304787614 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0004b20 con 0x555e51f2c580 2024-09-10T07:37:39.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fada0c00640 1 -- 172.21.15.46:0/1304787614 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fadb0004cc0 con 0x555e51f2c580 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.321+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1304787614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada4005520 con 0x555e51f2c580 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fada0c00640 1 -- 172.21.15.46:0/1304787614 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fadb0030050 con 0x555e51f2c580 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fadc0dd8740 10 client.25664 fetch_fsmap learned FSMap version 34 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1304787614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e51f3aee0 con 0x555e51f2c580 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fada0c00640 1 -- 172.21.15.46:0/1304787614 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb000fcf0 con 0x555e51f2c580 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fada0c00640 1 -- 172.21.15.46:0/1304787614 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7fadb0026030 con 0x555e51f2c580 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fadc0dd8740 10 client.25664 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fadc0dd8740 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fadc0dd8740 20 client.25664 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e51f3c3f0 0x555e51f3e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fadc0dd8740 4 client.25664 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.322+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1304787614 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fada4005520 con 0x555e51f3c3f0 2024-09-10T07:37:39.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.323+0000 7fadb7400640 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e51f3c3f0 0x555e51f3e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.323+0000 7fadb7400640 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e51f3c3f0 0x555e51f3e820 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57e0906151008cd server_cookie=3843ba116bcb7a92 in_seq=0 out_seq=0 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.323+0000 7fada0c00640 10 client.25664 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fada0c00640 1 -- 172.21.15.46:0/1304787614 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fada4005520 con 0x555e51f3c3f0 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fada0c00640 10 client.25664 handle_command_reply: tid=0 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 client.25664 shutdown 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1304787614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 msgr2=0x555e51f29ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f29ea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fadb0003f40 tx=0x7fadb0004ab0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1304787614 shutdown_connections 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e51f3c3f0 0x555e51f3e820 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f29ea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f21cc0 0x555e51f29960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fada0c00640 0 client.25664 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1304787614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f213a0 0x555e51f29420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1304787614 >> 172.21.15.46:0/1304787614 conn(0x555e51f1d930 msgr2=0x555e51f23660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:39.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1304787614 shutdown_connections 2024-09-10T07:37:39.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.324+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1304787614 wait complete. 2024-09-10T07:37:39.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.325+0000 7fadc0dd8740 20 client.25664 trim_cache size 0 max 16384 2024-09-10T07:37:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.343+0000 7fadc0dd8740 1 Processor -- start 2024-09-10T07:37:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.343+0000 7fadc0dd8740 1 -- start start 2024-09-10T07:37:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.343+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.343+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f44130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.343+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f44670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.343+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81acb40 con 0x555e51f21cc0 2024-09-10T07:37:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.343+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81ae740 con 0x555e51f2c580 2024-09-10T07:37:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.343+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81ad210 con 0x555e51f213a0 2024-09-10T07:37:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f44670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f44130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f44670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40072/0 (socket says 172.21.15.46:40072) 2024-09-10T07:37:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 -- 172.21.15.46:0/2589312699 learned_addr learned my addr 172.21.15.46:0/2589312699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7400640 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 -- 172.21.15.46:0/2589312699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 msgr2=0x555e51f140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:39.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 -- 172.21.15.46:0/2589312699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 msgr2=0x555e51f44130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f44130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 -- 172.21.15.46:0/2589312699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e51f21af0 con 0x555e51f2c580 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb6a00640 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fadb7e00640 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f44670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadb0011dd0 tx=0x7fadb00049f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3f0198480892be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fada3400640 1 -- 172.21.15.46:0/2589312699 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0039070 con 0x555e51f2c580 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fada3400640 1 -- 172.21.15.46:0/2589312699 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb0005a10 con 0x555e51f2c580 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.344+0000 7fada3400640 1 -- 172.21.15.46:0/2589312699 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0004d80 con 0x555e51f2c580 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2589312699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 msgr2=0x555e51f44670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f44670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadb0011dd0 tx=0x7fadb00049f0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2589312699 shutdown_connections 2024-09-10T07:37:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f44670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/2589312699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2589312699 >> 172.21.15.46:0/2589312699 conn(0x555e51f1d930 msgr2=0x555e51f139a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2589312699 shutdown_connections 2024-09-10T07:37:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 -- 172.21.15.46:0/2589312699 wait complete. 2024-09-10T07:37:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.345+0000 7fadc0dd8740 1 -- start start 2024-09-10T07:37:39.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadc0dd8740 1 Processor -- start 2024-09-10T07:37:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f26ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadc0dd8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f27000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81adfb0 con 0x555e51f21cc0 2024-09-10T07:37:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81ad8e0 con 0x555e51f2c580 2024-09-10T07:37:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadc0dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81aab20 con 0x555e51f213a0 2024-09-10T07:37:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadc0dd8740 10 client.0 _ll_register_callbacks cb 0x7fadb0003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadb7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f26ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadb7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f26580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53762/0 (socket says 172.21.15.46:53762) 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.346+0000 7fadb7400640 1 -- 172.21.15.46:0/1364796442 learned_addr learned my addr 172.21.15.46:0/1364796442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f27000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadb7400640 1 -- 172.21.15.46:0/1364796442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 msgr2=0x555e51f27000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadb7400640 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f27000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadb7400640 1 -- 172.21.15.46:0/1364796442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 msgr2=0x555e51f26ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadb7400640 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f26ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadb7400640 1 -- 172.21.15.46:0/1364796442 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e51fa61b0 con 0x555e51f213a0 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadb7e00640 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f27000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadb7400640 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f26580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fada80975c0 tx=0x7fada80bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3aa9402b9101df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fada0c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:39.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fada0c00640 1 -- 172.21.15.46:0/1364796442 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada80b3070 con 0x555e51f213a0 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fada0c00640 1 -- 172.21.15.46:0/1364796442 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fada80977d0 con 0x555e51f213a0 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.347+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1364796442 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e51e8c950 con 0x555e51f213a0 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fada0c00640 1 -- 172.21.15.46:0/1364796442 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fada809b160 con 0x555e51f213a0 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadc0dd8740 10 client.25663 fetch_fsmap learned FSMap version 34 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1364796442 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e51fa74a0 con 0x555e51f213a0 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fada0c00640 1 -- 172.21.15.46:0/1364796442 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555e51e8c950 con 0x555e51f213a0 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fada0c00640 1 -- 172.21.15.46:0/1364796442 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7fada80bb0b0 con 0x555e51f213a0 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadc0dd8740 10 client.25663 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadc0dd8740 10 client.25663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadc0dd8740 20 client.25663 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e51fa7c90 0x555e51faa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadc0dd8740 4 client.25663 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1364796442 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e51e86e60 con 0x555e51fa7c90 2024-09-10T07:37:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.348+0000 7fadb6a00640 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e51fa7c90 0x555e51faa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.349+0000 7fadb6a00640 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e51fa7c90 0x555e51faa0b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d87453921878399 server_cookie=102a300a42c794c5 in_seq=0 out_seq=0 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.349+0000 7fada0c00640 10 client.25663 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.349+0000 7fada0c00640 1 -- 172.21.15.46:0/1364796442 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555e51e86e60 con 0x555e51fa7c90 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.349+0000 7fada0c00640 10 client.25663 handle_command_reply: tid=0 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.349+0000 7fadc0dd8740 1 client.25663 shutdown 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1364796442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 msgr2=0x555e51f26580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f26580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fada80975c0 tx=0x7fada80bb040 comp rx=0 tx=0).stop 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1364796442 shutdown_connections 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555e51fa7c90 0x555e51faa0b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e51f2c580 0x555e51f27000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e51f21cc0 0x555e51f26ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 --2- 172.21.15.46:0/1364796442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e51f213a0 0x555e51f26580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fada0c00640 0 client.25663 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1364796442 >> 172.21.15.46:0/1364796442 conn(0x555e51f1d930 msgr2=0x555e51f141c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1364796442 shutdown_connections 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 1 -- 172.21.15.46:0/1364796442 wait complete. 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.350+0000 7fadc0dd8740 20 client.25663 trim_cache size 0 max 16384 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout: "0b5d4fbd-2f62-42f5-834c-af475c257438": { 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "0b5d4fbd-2f62-42f5-834c-af475c257438", 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:39.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:39.356 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:39.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 -- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad9c0a2c00 msgr2=0x7fad9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad9c0a2c00 0x7fad9c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fadb0005fd0 tx=0x7fadb0009500 comp rx=0 tx=0).stop 2024-09-10T07:37:39.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 -- 172.21.15.46:0/1699037183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074620 msgr2=0x7fadb819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:39.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074620 0x7fadb819a690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fada800d950 tx=0x7fada800de10 comp rx=0 tx=0).stop 2024-09-10T07:37:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 -- 172.21.15.46:0/1699037183 shutdown_connections 2024-09-10T07:37:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb810e3b0 0x7fadb819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fad9c0a2c00 0x7fad9c0a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadb8074f40 0x7fadb819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 --2- 172.21.15.46:0/1699037183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadb8074620 0x7fadb819a690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 -- 172.21.15.46:0/1699037183 >> 172.21.15.46:0/1699037183 conn(0x7fadb80fd890 msgr2=0x7fadb80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 -- 172.21.15.46:0/1699037183 shutdown_connections 2024-09-10T07:37:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:39.351+0000 7fadbe200640 1 -- 172.21.15.46:0/1699037183 wait complete. 2024-09-10T07:37:39.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:39 smithi181 ceph-mon[35397]: pgmap v430: 129 pgs: 129 active+clean; 910 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.0 MiB/s wr, 512 op/s 2024-09-10T07:37:39.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:39 smithi063 ceph-mon[35020]: pgmap v430: 129 pgs: 129 active+clean; 910 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.0 MiB/s wr, 512 op/s 2024-09-10T07:37:39.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:39 smithi046 ceph-mon[30705]: pgmap v430: 129 pgs: 129 active+clean; 910 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.0 MiB/s wr, 512 op/s 2024-09-10T07:37:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 20.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:37:40.849 INFO:tasks.cephfs.filesystem:scrub status for tag:0b5d4fbd-2f62-42f5-834c-af475c257438 - {'path': '/', 'tag': '0b5d4fbd-2f62-42f5-834c-af475c257438', 'options': 'recursive,force'} 2024-09-10T07:37:41.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:41 smithi181 ceph-mon[35397]: pgmap v431: 129 pgs: 129 active+clean; 919 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 5.2 MiB/s wr, 556 op/s 2024-09-10T07:37:41.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:41 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:37:41.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:41 smithi063 ceph-mon[35020]: pgmap v431: 129 pgs: 129 active+clean; 919 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 5.2 MiB/s wr, 556 op/s 2024-09-10T07:37:41.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:41 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:37:41.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:41 smithi046 ceph-mon[30705]: pgmap v431: 129 pgs: 129 active+clean; 919 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 5.2 MiB/s wr, 556 op/s 2024-09-10T07:37:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:41 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:37:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:37:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:37:42.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:37:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:37:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:37:42.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:37:42.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:37:42.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:37:42.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:37:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:44 smithi181 ceph-mon[35397]: pgmap v432: 129 pgs: 129 active+clean; 923 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2024-09-10T07:37:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:44 smithi063 ceph-mon[35020]: pgmap v432: 129 pgs: 129 active+clean; 923 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2024-09-10T07:37:44.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:44 smithi046 ceph-mon[30705]: pgmap v432: 129 pgs: 129 active+clean; 923 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2024-09-10T07:37:45.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 25.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:37:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:46 smithi046 ceph-mon[30705]: pgmap v433: 129 pgs: 129 active+clean; 927 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-09-10T07:37:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:46 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:37:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:46 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:37:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:46 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:37:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:46 smithi181 ceph-mon[35397]: pgmap v433: 129 pgs: 129 active+clean; 927 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-09-10T07:37:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:46 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:37:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:46 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:37:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:46 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:37:46.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:46 smithi063 ceph-mon[35020]: pgmap v433: 129 pgs: 129 active+clean; 927 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-09-10T07:37:46.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:46 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:37:46.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:46 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:37:46.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:46 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:37:47.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:47 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:37:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 -- 172.21.15.46:0/302609552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec105720 msgr2=0x7f57ec10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 --2- 172.21.15.46:0/302609552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec105720 0x7f57ec10bfb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009ca0 tx=0x7f57e402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 -- 172.21.15.46:0/302609552 shutdown_connections 2024-09-10T07:37:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 --2- 172.21.15.46:0/302609552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec105720 0x7f57ec10bfb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 --2- 172.21.15.46:0/302609552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57ec104d60 0x7f57ec1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 --2- 172.21.15.46:0/302609552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57ec103b70 0x7f57ec103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 -- 172.21.15.46:0/302609552 >> 172.21.15.46:0/302609552 conn(0x7f57ec0ff2f0 msgr2=0x7f57ec101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 -- 172.21.15.46:0/302609552 shutdown_connections 2024-09-10T07:37:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.475+0000 7f57ebe00640 1 -- 172.21.15.46:0/302609552 wait complete. 2024-09-10T07:37:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.476+0000 7f57ebe00640 1 Processor -- start 2024-09-10T07:37:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.476+0000 7f57ebe00640 1 -- start start 2024-09-10T07:37:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57ebe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57ec103b70 0x7f57ec199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec104d60 0x7f57ec19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57ebe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57ec105720 0x7f57ec1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57ebe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec10f820 con 0x7f57ec104d60 2024-09-10T07:37:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57ebe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec10f6a0 con 0x7f57ec103b70 2024-09-10T07:37:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57ebe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec10f9a0 con 0x7f57ec105720 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57eaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57ec103b70 0x7f57ec199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57eaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57ec103b70 0x7f57ec199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50014/0 (socket says 172.21.15.46:50014) 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec104d60 0x7f57ec19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57eaa00640 1 -- 172.21.15.46:0/231069428 learned_addr learned my addr 172.21.15.46:0/231069428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.477+0000 7f57eb400640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57ec105720 0x7f57ec1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57eaa00640 1 -- 172.21.15.46:0/231069428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57ec105720 msgr2=0x7f57ec1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57eaa00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57ec105720 0x7f57ec1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57eaa00640 1 -- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec104d60 msgr2=0x7f57ec19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57eaa00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec104d60 0x7f57ec19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57eaa00640 1 -- 172.21.15.46:0/231069428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec1a1a30 con 0x7f57ec103b70 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57ea000640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec104d60 0x7f57ec19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57eb400640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57ec105720 0x7f57ec1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57eaa00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57ec103b70 0x7f57ec199e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00d950 tx=0x7f57dc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57e3400640 1 -- 172.21.15.46:0/231069428 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc017020 con 0x7f57ec103b70 2024-09-10T07:37:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57ebe00640 1 -- 172.21.15.46:0/231069428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ec1a1d20 con 0x7f57ec103b70 2024-09-10T07:37:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57e3400640 1 -- 172.21.15.46:0/231069428 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57dc009c30 con 0x7f57ec103b70 2024-09-10T07:37:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57e3400640 1 -- 172.21.15.46:0/231069428 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00f6b0 con 0x7f57ec103b70 2024-09-10T07:37:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.478+0000 7f57ebe00640 1 -- 172.21.15.46:0/231069428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ec1a2200 con 0x7f57ec103b70 2024-09-10T07:37:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.481+0000 7f57f4853740 1 Processor -- start 2024-09-10T07:37:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.481+0000 7f57f4853740 1 -- start start 2024-09-10T07:37:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57e3400640 1 -- 172.21.15.46:0/231069428 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f57dc01a020 con 0x7f57ec103b70 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57f4853740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d2719e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57f4853740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d2719eba0 0x562d271a3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57f4853740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d271a3c30 0x562d271a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57f4853740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d27103100 con 0x562d271a3c30 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57f4853740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562d270ab1b0 con 0x562d2719eba0 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57f4853740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562d270ff0c0 con 0x562d2719e260 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d2719e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d2719e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:36142/0 (socket says 172.21.15.46:36142) 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eb400640 1 -- 172.21.15.46:0/3291299908 learned_addr learned my addr 172.21.15.46:0/3291299908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eaa00640 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d2719eba0 0x562d271a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57ea000640 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d271a3c30 0x562d271a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eb400640 1 -- 172.21.15.46:0/3291299908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d2719eba0 msgr2=0x562d271a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eb400640 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d2719eba0 0x562d271a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eb400640 1 -- 172.21.15.46:0/3291299908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d271a3c30 msgr2=0x562d271a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eb400640 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d271a3c30 0x562d271a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.482+0000 7f57eb400640 1 -- 172.21.15.46:0/3291299908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d26e16210 con 0x562d2719e260 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.483+0000 7f57eb400640 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d2719e660 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005e30 tx=0x7f57e4005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7674697acfa2304e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.483+0000 7f57d7e00640 1 -- 172.21.15.46:0/3291299908 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4031650 con 0x562d2719e260 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.483+0000 7f57d7e00640 1 -- 172.21.15.46:0/3291299908 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e4036040 con 0x562d2719e260 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.483+0000 7f57d7e00640 1 -- 172.21.15.46:0/3291299908 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4004760 con 0x562d2719e260 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.483+0000 7f57e3400640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57d00a2c00 0x7f57d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57e3400640 1 -- 172.21.15.46:0/231069428 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f57dc0cf330 con 0x7f57ec103b70 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57eaa00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57d00a2c00 0x7f57d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 -- 172.21.15.46:0/3291299908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 msgr2=0x562d2719e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d2719e660 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005e30 tx=0x7f57e4005f70 comp rx=0 tx=0).stop 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 -- 172.21.15.46:0/3291299908 shutdown_connections 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d271a3c30 0x562d271a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d2719eba0 0x562d271a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 --2- 172.21.15.46:0/3291299908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d2719e660 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 -- 172.21.15.46:0/3291299908 >> 172.21.15.46:0/3291299908 conn(0x562d271998d0 msgr2=0x562d27199cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 -- 172.21.15.46:0/3291299908 shutdown_connections 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57f4853740 1 -- 172.21.15.46:0/3291299908 wait complete. 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.484+0000 7f57eaa00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57d00a2c00 0x7f57d00a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f57dc097c10 tx=0x7f57dc0d2d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.485+0000 7f57f4853740 1 -- start start 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.485+0000 7f57f4853740 1 Processor -- start 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.485+0000 7f57f4853740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d271a8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.485+0000 7f57f4853740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719eba0 0x562d271a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.485+0000 7f57f4853740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.485+0000 7f57f4853740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d271a6ba0 con 0x562d2719eba0 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.485+0000 7f57f4853740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562d2708c090 con 0x562d271a3c30 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.485+0000 7f57f4853740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562d271a6d20 con 0x562d2719e260 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57f4853740 10 client.0 _ll_register_callbacks cb 0x562d270e0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719eba0 0x562d271a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719eba0 0x562d271a88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53204/0 (socket says 172.21.15.46:53204) 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 -- 172.21.15.46:0/767852877 learned_addr learned my addr 172.21.15.46:0/767852877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57eb400640 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d271a8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57eb400640 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 -- 172.21.15.46:0/767852877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 msgr2=0x562d271a8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d271a8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 -- 172.21.15.46:0/767852877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 msgr2=0x562d271af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 -- 172.21.15.46:0/767852877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d26ee2620 con 0x562d2719eba0 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57eb400640 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57ea000640 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719eba0 0x562d271a88d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f57d8012d00 tx=0x7f57d800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec88746c3313339 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.486+0000 7f57d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.487+0000 7f57d4c00640 1 -- 172.21.15.46:0/767852877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8009d90 con 0x562d2719eba0 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.487+0000 7f57d4c00640 1 -- 172.21.15.46:0/767852877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f57d8012110 con 0x562d2719eba0 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.487+0000 7f57f4853740 1 -- 172.21.15.46:0/767852877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e4036040 con 0x562d2719eba0 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.487+0000 7f57d4c00640 1 -- 172.21.15.46:0/767852877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f57d8004050 con 0x562d2719eba0 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.487+0000 7f57f4853740 10 client.15597 fetch_fsmap learned FSMap version 34 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.487+0000 7f57f4853740 1 -- 172.21.15.46:0/767852877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d271b2820 con 0x562d2719eba0 2024-09-10T07:37:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.487+0000 7f57d4c00640 1 -- 172.21.15.46:0/767852877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e4036040 con 0x562d2719eba0 2024-09-10T07:37:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.488+0000 7f57d4c00640 1 -- 172.21.15.46:0/767852877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f57d8010430 con 0x562d2719eba0 2024-09-10T07:37:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.488+0000 7f57f4853740 10 client.15597 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:37:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.488+0000 7f57f4853740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.488+0000 7f57f4853740 20 client.15597 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.488+0000 7f57f4853740 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562d271b3fe0 0x562d271b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.488+0000 7f57f4853740 4 client.15597 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.488+0000 7f57f4853740 1 -- 172.21.15.46:0/767852877 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d27110830 con 0x562d271b3fe0 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.488+0000 7f57eb400640 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562d271b3fe0 0x562d271b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.489+0000 7f57eb400640 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562d271b3fe0 0x562d271b6420 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a7a2a460b1a57d server_cookie=901df37b7c42f392 in_seq=0 out_seq=0 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.489+0000 7f57d4c00640 10 client.15597 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.489+0000 7f57d4c00640 1 -- 172.21.15.46:0/767852877 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d27110830 con 0x562d271b3fe0 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.489+0000 7f57d4c00640 10 client.15597 handle_command_reply: tid=0 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.489+0000 7f57f4853740 1 client.15597 shutdown 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 -- 172.21.15.46:0/767852877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719eba0 msgr2=0x562d271a88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719eba0 0x562d271a88d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f57d8012d00 tx=0x7f57d800f790 comp rx=0 tx=0).stop 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 -- 172.21.15.46:0/767852877 shutdown_connections 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562d271b3fe0 0x562d271b6420 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719eba0 0x562d271a88d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 --2- 172.21.15.46:0/767852877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719e260 0x562d271a8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57d4c00640 0 client.15597 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 -- 172.21.15.46:0/767852877 >> 172.21.15.46:0/767852877 conn(0x562d271998d0 msgr2=0x562d2719b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 -- 172.21.15.46:0/767852877 shutdown_connections 2024-09-10T07:37:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 1 -- 172.21.15.46:0/767852877 wait complete. 2024-09-10T07:37:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.490+0000 7f57f4853740 20 client.15597 trim_cache size 0 max 16384 2024-09-10T07:37:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.505+0000 7f57f4853740 1 Processor -- start 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.505+0000 7f57f4853740 1 -- start start 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.505+0000 7f57f4853740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d27199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.505+0000 7f57f4853740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.505+0000 7f57f4853740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d2719a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.505+0000 7f57f4853740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec1ac710 con 0x562d2719e260 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.505+0000 7f57f4853740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec1ae310 con 0x562d271a3c30 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57f4853740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec1acde0 con 0x562d2719eba0 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d27199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d27199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53220/0 (socket says 172.21.15.46:53220) 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 -- 172.21.15.46:0/407782204 learned_addr learned my addr 172.21.15.46:0/407782204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57ea000640 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d2719a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 -- 172.21.15.46:0/407782204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 msgr2=0x562d271af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 -- 172.21.15.46:0/407782204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 msgr2=0x562d2719a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d2719a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 -- 172.21.15.46:0/407782204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d271a3020 con 0x562d2719e260 2024-09-10T07:37:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57ea000640 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.506+0000 7f57eb400640 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d27199cb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009620 tx=0x7f57e4005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3777a01dbf845ede server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57d7400640 1 -- 172.21.15.46:0/407782204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e404a040 con 0x562d2719e260 2024-09-10T07:37:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57d7400640 1 -- 172.21.15.46:0/407782204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e4036040 con 0x562d2719e260 2024-09-10T07:37:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57d7400640 1 -- 172.21.15.46:0/407782204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e402f610 con 0x562d2719e260 2024-09-10T07:37:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57f4853740 1 -- 172.21.15.46:0/407782204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 msgr2=0x562d27199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57f4853740 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d27199cb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009620 tx=0x7f57e4005f70 comp rx=0 tx=0).stop 2024-09-10T07:37:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57f4853740 1 -- 172.21.15.46:0/407782204 shutdown_connections 2024-09-10T07:37:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57f4853740 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d2719a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57f4853740 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57f4853740 1 --2- 172.21.15.46:0/407782204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d27199cb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.507+0000 7f57f4853740 1 -- 172.21.15.46:0/407782204 >> 172.21.15.46:0/407782204 conn(0x562d271998d0 msgr2=0x562d271a33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.508+0000 7f57f4853740 1 -- 172.21.15.46:0/407782204 shutdown_connections 2024-09-10T07:37:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.508+0000 7f57f4853740 1 -- 172.21.15.46:0/407782204 wait complete. 2024-09-10T07:37:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.508+0000 7f57f4853740 1 -- start start 2024-09-10T07:37:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.508+0000 7f57f4853740 1 Processor -- start 2024-09-10T07:37:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57f4853740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d271ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57f4853740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57f4853740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57f4853740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec1adb80 con 0x562d2719e260 2024-09-10T07:37:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57f4853740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec1ad4b0 con 0x562d271a3c30 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57f4853740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec1aa6f0 con 0x562d2719eba0 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57f4853740 10 client.0 _ll_register_callbacks cb 0x7f57d8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d271ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d271ad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53236/0 (socket says 172.21.15.46:53236) 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 -- 172.21.15.46:0/1896690740 learned_addr learned my addr 172.21.15.46:0/1896690740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57ea000640 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 -- 172.21.15.46:0/1896690740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 msgr2=0x562d271ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 -- 172.21.15.46:0/1896690740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 msgr2=0x562d271ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.509+0000 7f57eb400640 1 -- 172.21.15.46:0/1896690740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d271aeb40 con 0x562d2719e260 2024-09-10T07:37:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57eb400640 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d271ad050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f57e4004af0 tx=0x7f57e4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4aea1a1773419e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57d4c00640 1 -- 172.21.15.46:0/1896690740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4061030 con 0x562d2719e260 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57d4c00640 1 -- 172.21.15.46:0/1896690740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f57e4003ee0 con 0x562d2719e260 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57f4853740 1 -- 172.21.15.46:0/1896690740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e4036040 con 0x562d2719e260 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57d4c00640 1 -- 172.21.15.46:0/1896690740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f57e4031650 con 0x562d2719e260 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57f4853740 10 client.15603 fetch_fsmap learned FSMap version 34 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57f4853740 1 -- 172.21.15.46:0/1896690740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d2721ed60 con 0x562d2719e260 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.510+0000 7f57d4c00640 1 -- 172.21.15.46:0/1896690740 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e4036040 con 0x562d2719e260 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57d4c00640 1 -- 172.21.15.46:0/1896690740 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f57e4050040 con 0x562d2719e260 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57f4853740 10 client.15603 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57f4853740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57f4853740 20 client.15603 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57f4853740 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562d2721f260 0x562d27221680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57f4853740 4 client.15603 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:37:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57f4853740 1 -- 172.21.15.46:0/1896690740 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d2710ab80 con 0x562d2721f260 2024-09-10T07:37:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57ea000640 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562d2721f260 0x562d27221680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57ea000640 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562d2721f260 0x562d27221680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94c3a8871ff7fce server_cookie=25aafeb4e3c4a79c in_seq=0 out_seq=0 2024-09-10T07:37:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.511+0000 7f57d4c00640 10 client.15603 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.527+0000 7f57d4c00640 1 -- 172.21.15.46:0/1896690740 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d2710ab80 con 0x562d2721f260 2024-09-10T07:37:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.527+0000 7f57d4c00640 10 client.15603 handle_command_reply: tid=0 2024-09-10T07:37:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.527+0000 7f57f4853740 1 client.15603 shutdown 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 -- 172.21.15.46:0/1896690740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 msgr2=0x562d271ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d271ad050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f57e4004af0 tx=0x7f57e4000f40 comp rx=0 tx=0).stop 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 -- 172.21.15.46:0/1896690740 shutdown_connections 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562d2721f260 0x562d27221680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d271a3c30 0x562d271ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d2719eba0 0x562d271ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57d4c00640 0 client.15603 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 --2- 172.21.15.46:0/1896690740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d2719e260 0x562d271ad050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 -- 172.21.15.46:0/1896690740 >> 172.21.15.46:0/1896690740 conn(0x562d271998d0 msgr2=0x562d271981f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 -- 172.21.15.46:0/1896690740 shutdown_connections 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 1 -- 172.21.15.46:0/1896690740 wait complete. 2024-09-10T07:37:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.528+0000 7f57f4853740 20 client.15603 trim_cache size 0 max 16384 2024-09-10T07:37:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:37:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:37:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:37:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 -- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57d00a2c00 msgr2=0x7f57d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57d00a2c00 0x7f57d00a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f57dc097c10 tx=0x7f57dc0d2d70 comp rx=0 tx=0).stop 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 -- 172.21.15.46:0/231069428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57ec103b70 msgr2=0x7f57ec199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57ec103b70 0x7f57ec199e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00d950 tx=0x7f57dc00de10 comp rx=0 tx=0).stop 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 -- 172.21.15.46:0/231069428 shutdown_connections 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57ec105720 0x7f57ec1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f57d00a2c00 0x7f57d00a50c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec104d60 0x7f57ec19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 --2- 172.21.15.46:0/231069428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57ec103b70 0x7f57ec199e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 -- 172.21.15.46:0/231069428 >> 172.21.15.46:0/231069428 conn(0x7f57ec0ff2f0 msgr2=0x7f57ec108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 -- 172.21.15.46:0/231069428 shutdown_connections 2024-09-10T07:37:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:37:47.529+0000 7f57ebe00640 1 -- 172.21.15.46:0/231069428 wait complete. 2024-09-10T07:37:47.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:47 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:37:47.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:47 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:37:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:48 smithi046 ceph-mon[30705]: pgmap v434: 129 pgs: 129 active+clean; 956 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.8 MiB/s wr, 411 op/s 2024-09-10T07:37:48.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:48 smithi181 ceph-mon[35397]: pgmap v434: 129 pgs: 129 active+clean; 956 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.8 MiB/s wr, 411 op/s 2024-09-10T07:37:48.653 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:48 smithi063 ceph-mon[35020]: pgmap v434: 129 pgs: 129 active+clean; 956 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.8 MiB/s wr, 411 op/s 2024-09-10T07:37:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 30.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:37:50.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:50 smithi181 ceph-mon[35397]: pgmap v435: 129 pgs: 129 active+clean; 960 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.2 MiB/s wr, 221 op/s 2024-09-10T07:37:50.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:50 smithi063 ceph-mon[35020]: pgmap v435: 129 pgs: 129 active+clean; 960 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.2 MiB/s wr, 221 op/s 2024-09-10T07:37:50.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:50 smithi046 ceph-mon[30705]: pgmap v435: 129 pgs: 129 active+clean; 960 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.2 MiB/s wr, 221 op/s 2024-09-10T07:37:52.323 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:52 smithi046 ceph-mon[30705]: pgmap v436: 129 pgs: 129 active+clean; 968 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.9 MiB/s wr, 220 op/s 2024-09-10T07:37:52.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:52 smithi181 ceph-mon[35397]: pgmap v436: 129 pgs: 129 active+clean; 968 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.9 MiB/s wr, 220 op/s 2024-09-10T07:37:52.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:52 smithi063 ceph-mon[35020]: pgmap v436: 129 pgs: 129 active+clean; 968 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.9 MiB/s wr, 220 op/s 2024-09-10T07:37:54.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:54 smithi181 ceph-mon[35397]: pgmap v437: 129 pgs: 129 active+clean; 970 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 153 op/s 2024-09-10T07:37:54.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:54 smithi063 ceph-mon[35020]: pgmap v437: 129 pgs: 129 active+clean; 970 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 153 op/s 2024-09-10T07:37:54.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:54 smithi046 ceph-mon[30705]: pgmap v437: 129 pgs: 129 active+clean; 970 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 153 op/s 2024-09-10T07:37:55.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 35.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:37:56.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:37:56.570 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:56 smithi046 ceph-mon[30705]: pgmap v438: 129 pgs: 129 active+clean; 958 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 83 op/s 2024-09-10T07:37:56.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:56 smithi181 ceph-mon[35397]: pgmap v438: 129 pgs: 129 active+clean; 958 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 83 op/s 2024-09-10T07:37:56.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:56 smithi063 ceph-mon[35020]: pgmap v438: 129 pgs: 129 active+clean; 958 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 83 op/s 2024-09-10T07:37:56.905 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 -- 172.21.15.46:0/2571713986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64077040 msgr2=0x7fae641124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 --2- 172.21.15.46:0/2571713986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64077040 0x7fae641124f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fae50009900 tx=0x7fae5002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 -- 172.21.15.46:0/2571713986 shutdown_connections 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 --2- 172.21.15.46:0/2571713986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64112a30 0x7fae64114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 --2- 172.21.15.46:0/2571713986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64077040 0x7fae641124f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 --2- 172.21.15.46:0/2571713986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64076720 0x7fae64076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 -- 172.21.15.46:0/2571713986 >> 172.21.15.46:0/2571713986 conn(0x7fae640ffa90 msgr2=0x7fae64101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 -- 172.21.15.46:0/2571713986 shutdown_connections 2024-09-10T07:37:57.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.499+0000 7fae6b600640 1 -- 172.21.15.46:0/2571713986 wait complete. 2024-09-10T07:37:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.500+0000 7fae6b600640 1 Processor -- start 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.500+0000 7fae6b600640 1 -- start start 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae6b600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64076720 0x7fae641a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae6b600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64077040 0x7fae641a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae6b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64112a30 0x7fae6419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae6b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae641185f0 con 0x7fae64112a30 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae6b600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae64118470 con 0x7fae64076720 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae6b600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fae64118770 con 0x7fae64077040 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae68c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64076720 0x7fae641a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae68c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64076720 0x7fae641a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38678/0 (socket says 172.21.15.46:38678) 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae69600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64112a30 0x7fae6419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae68c00640 1 -- 172.21.15.46:0/1494145930 learned_addr learned my addr 172.21.15.46:0/1494145930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.501+0000 7fae63e00640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64077040 0x7fae641a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae68c00640 1 -- 172.21.15.46:0/1494145930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64077040 msgr2=0x7fae641a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae68c00640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64077040 0x7fae641a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae68c00640 1 -- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64112a30 msgr2=0x7fae6419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae68c00640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64112a30 0x7fae6419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae68c00640 1 -- 172.21.15.46:0/1494145930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6419efd0 con 0x7fae64076720 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae69600640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64112a30 0x7fae6419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae68c00640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64076720 0x7fae641a46e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae5800d950 tx=0x7fae5800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae61600640 1 -- 172.21.15.46:0/1494145930 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58017030 con 0x7fae64076720 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6419f2c0 con 0x7fae64076720 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae61600640 1 -- 172.21.15.46:0/1494145930 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae58004020 con 0x7fae64076720 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae61600640 1 -- 172.21.15.46:0/1494145930 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580047a0 con 0x7fae64076720 2024-09-10T07:37:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.502+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641ab5b0 con 0x7fae64076720 2024-09-10T07:37:57.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.503+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae64076b00 con 0x7fae64076720 2024-09-10T07:37:57.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.505+0000 7fae61600640 1 -- 172.21.15.46:0/1494145930 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae580041d0 con 0x7fae64076720 2024-09-10T07:37:57.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.507+0000 7fae61600640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae4c0a2b80 0x7fae4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:57.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.507+0000 7fae61600640 1 -- 172.21.15.46:0/1494145930 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fae580ceff0 con 0x7fae64076720 2024-09-10T07:37:57.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.507+0000 7fae63e00640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae4c0a2b80 0x7fae4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:57.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.508+0000 7fae63e00640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae4c0a2b80 0x7fae4c0a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae50000c00 tx=0x7fae50005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:57.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.509+0000 7fae61600640 1 -- 172.21.15.46:0/1494145930 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae5809bb70 con 0x7fae64076720 2024-09-10T07:37:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.732+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae6419fdd0 con 0x7fae64076720 2024-09-10T07:37:57.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.733+0000 7fae61600640 1 -- 172.21.15.46:0/1494145930 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11979 (secure 0 0 0) 0x7fae5809b470 con 0x7fae64076720 2024-09-10T07:37:57.734 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:37:57.734 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"btime":"2024-09-10T07:37:35:593020+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:37:35.488899+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:37:57.734 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-09-10T07:37:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae4c0a2b80 msgr2=0x7fae4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae4c0a2b80 0x7fae4c0a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae50000c00 tx=0x7fae50005b50 comp rx=0 tx=0).stop 2024-09-10T07:37:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64076720 msgr2=0x7fae641a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64076720 0x7fae641a46e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae5800d950 tx=0x7fae5800de10 comp rx=0 tx=0).stop 2024-09-10T07:37:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 shutdown_connections 2024-09-10T07:37:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fae4c0a2b80 0x7fae4c0a5040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64112a30 0x7fae6419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae64077040 0x7fae641a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 --2- 172.21.15.46:0/1494145930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64076720 0x7fae641a46e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.735+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 >> 172.21.15.46:0/1494145930 conn(0x7fae640ffa90 msgr2=0x7fae64101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.736+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 shutdown_connections 2024-09-10T07:37:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:57.736+0000 7fae6b600640 1 -- 172.21.15.46:0/1494145930 wait complete. 2024-09-10T07:37:58.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:58 smithi046 ceph-mon[30705]: pgmap v439: 129 pgs: 129 active+clean; 916 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 63 op/s 2024-09-10T07:37:58.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:37:58 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1494145930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:58.452 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:58 smithi063 ceph-mon[35020]: pgmap v439: 129 pgs: 129 active+clean; 916 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 63 op/s 2024-09-10T07:37:58.452 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:37:58 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1494145930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:58 smithi181 ceph-mon[35397]: pgmap v439: 129 pgs: 129 active+clean; 916 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 63 op/s 2024-09-10T07:37:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:37:58 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1494145930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:37:58.688 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id 3850f863 --release --await' 2024-09-10T07:37:58.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id 3850f863 --release --await 2024-09-10T07:37:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:37:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 -- 172.21.15.46:0/3984618272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384074590 msgr2=0x7f3384074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 --2- 172.21.15.46:0/3984618272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384074590 0x7f3384074970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3378009900 tx=0x7f337802a490 comp rx=0 tx=0).stop 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 -- 172.21.15.46:0/3984618272 shutdown_connections 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 --2- 172.21.15.46:0/3984618272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3384112b90 0x7f3384114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 --2- 172.21.15.46:0/3984618272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3384074eb0 0x7f3384112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 --2- 172.21.15.46:0/3984618272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384074590 0x7f3384074970 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 -- 172.21.15.46:0/3984618272 >> 172.21.15.46:0/3984618272 conn(0x7f33840fd8c0 msgr2=0x7f33840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 -- 172.21.15.46:0/3984618272 shutdown_connections 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.673+0000 7f338ba00640 1 -- 172.21.15.46:0/3984618272 wait complete. 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.674+0000 7f338ba00640 1 Processor -- start 2024-09-10T07:37:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.674+0000 7f338ba00640 1 -- start start 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f338ba00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3384074590 0x7f33841a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f338ba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384074eb0 0x7f33841a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f338ba00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3384112b90 0x7f338419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f338ba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3384118570 con 0x7f3384074eb0 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f338ba00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33841183f0 con 0x7f3384112b90 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f338ba00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f33841186f0 con 0x7f3384074590 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f3383e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384074eb0 0x7f33841a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f3389000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3384074590 0x7f33841a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f3389000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3384074590 0x7f33841a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60616/0 (socket says 172.21.15.46:60616) 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f3389000640 1 -- 172.21.15.46:0/320171327 learned_addr learned my addr 172.21.15.46:0/320171327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.675+0000 7f3389a00640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3384112b90 0x7f338419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.676+0000 7f3389000640 1 -- 172.21.15.46:0/320171327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3384112b90 msgr2=0x7f338419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.676+0000 7f3389000640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3384112b90 0x7f338419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.676+0000 7f3389000640 1 -- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384074eb0 msgr2=0x7f33841a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.676+0000 7f3389000640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384074eb0 0x7f33841a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.676+0000 7f3389000640 1 -- 172.21.15.46:0/320171327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338419f010 con 0x7f3384074590 2024-09-10T07:37:59.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.676+0000 7f3389000640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3384074590 0x7f33841a4720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3378006010 tx=0x7f3378004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:59.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.676+0000 7f3381600640 1 -- 172.21.15.46:0/320171327 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337803f030 con 0x7f3384074590 2024-09-10T07:37:59.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.677+0000 7f338ba00640 1 -- 172.21.15.46:0/320171327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338419f2a0 con 0x7f3384074590 2024-09-10T07:37:59.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.677+0000 7f3381600640 1 -- 172.21.15.46:0/320171327 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3378004440 con 0x7f3384074590 2024-09-10T07:37:59.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.677+0000 7f3381600640 1 -- 172.21.15.46:0/320171327 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378035070 con 0x7f3384074590 2024-09-10T07:37:59.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.677+0000 7f338ba00640 1 -- 172.21.15.46:0/320171327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33841ab5f0 con 0x7f3384074590 2024-09-10T07:37:59.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f3381600640 1 -- 172.21.15.46:0/320171327 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3378042030 con 0x7f3384074590 2024-09-10T07:37:59.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f338e417740 1 Processor -- start 2024-09-10T07:37:59.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f338e417740 1 -- start start 2024-09-10T07:37:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f338e417740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34cff26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f338e417740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2c30 0x55c34cffcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f338e417740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cffd4f0 0x55c34cfff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f338e417740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c34cf58650 con 0x55c34cff2c30 2024-09-10T07:37:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f338e417740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c34cefff10 con 0x55c34cffd4f0 2024-09-10T07:37:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.680+0000 7f338e417740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c34cf54610 con 0x55c34cff2310 2024-09-10T07:37:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2c30 0x55c34cffcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34cff26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2c30 0x55c34cffcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57908/0 (socket says 172.21.15.46:57908) 2024-09-10T07:37:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389000640 1 -- 172.21.15.46:0/2149689435 learned_addr learned my addr 172.21.15.46:0/2149689435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389a00640 1 -- 172.21.15.46:0/2149689435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cffd4f0 msgr2=0x55c34cfff8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389a00640 1 --2- 172.21.15.46:0/2149689435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cffd4f0 0x55c34cfff8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389a00640 1 -- 172.21.15.46:0/2149689435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2c30 msgr2=0x55c34cffcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389a00640 1 --2- 172.21.15.46:0/2149689435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2c30 0x55c34cffcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389a00640 1 -- 172.21.15.46:0/2149689435 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c34cc67580 con 0x55c34cff2310 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3381600640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f336c0a2b80 0x7f336c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.681+0000 7f3389a00640 1 --2- 172.21.15.46:0/2149689435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34cff26f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f337c005fd0 tx=0x7f337c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e11fb17dcb003fd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.682+0000 7f3381600640 1 -- 172.21.15.46:0/320171327 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f33780ef770 con 0x7f3384074590 2024-09-10T07:37:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.682+0000 7f3389000640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f336c0a2b80 0x7f336c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.682+0000 7f3376000640 1 -- 172.21.15.46:0/2149689435 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c004d20 con 0x55c34cff2310 2024-09-10T07:37:59.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.682+0000 7f3376000640 1 -- 172.21.15.46:0/2149689435 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337c004050 con 0x55c34cff2310 2024-09-10T07:37:59.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.682+0000 7f3376000640 1 -- 172.21.15.46:0/2149689435 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c005280 con 0x55c34cff2310 2024-09-10T07:37:59.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 -- 172.21.15.46:0/2149689435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 msgr2=0x55c34cff26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 --2- 172.21.15.46:0/2149689435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34cff26f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f337c005fd0 tx=0x7f337c011810 comp rx=0 tx=0).stop 2024-09-10T07:37:59.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f3389000640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f336c0a2b80 0x7f336c0a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f33780b8670 tx=0x7f33780f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 -- 172.21.15.46:0/2149689435 shutdown_connections 2024-09-10T07:37:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 --2- 172.21.15.46:0/2149689435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cffd4f0 0x55c34cfff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 --2- 172.21.15.46:0/2149689435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2c30 0x55c34cffcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 --2- 172.21.15.46:0/2149689435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34cff26f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 -- 172.21.15.46:0/2149689435 >> 172.21.15.46:0/2149689435 conn(0x55c34cfee8a0 msgr2=0x55c34cfeec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 -- 172.21.15.46:0/2149689435 shutdown_connections 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 -- 172.21.15.46:0/2149689435 wait complete. 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.683+0000 7f338e417740 1 -- start start 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f338e417740 1 Processor -- start 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f338e417740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2310 0x55c34cffa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f338e417740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2c30 0x55c34cffa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f338e417740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 0x55c34cffae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f338e417740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c34d0000c0 con 0x55c34cffd4f0 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f338e417740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c34cf13350 con 0x55c34cff2310 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f338e417740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c34d000240 con 0x55c34cff2c30 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f338e417740 10 client.0 _ll_register_callbacks cb 0x55c34cf36290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f3389a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 0x55c34cffae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f3383e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2c30 0x55c34cffa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.684+0000 7f3389a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2310 0x55c34cffa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3383e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2c30 0x55c34cffa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60624/0 (socket says 172.21.15.46:60624) 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3383e00640 1 -- 172.21.15.46:0/4049333761 learned_addr learned my addr 172.21.15.46:0/4049333761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3389a00640 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2310 0x55c34cffa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38704/0 (socket says 172.21.15.46:38704) 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3383e00640 1 -- 172.21.15.46:0/4049333761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2310 msgr2=0x55c34cffa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3383e00640 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2310 0x55c34cffa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3383e00640 1 -- 172.21.15.46:0/4049333761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 msgr2=0x55c34cffae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3383e00640 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 0x55c34cffae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3383e00640 1 -- 172.21.15.46:0/4049333761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c34cd31960 con 0x55c34cff2c30 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f3383e00640 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2c30 0x55c34cffa8d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f33700116f0 tx=0x7f3370011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87e653022542e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f336aa00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:37:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.685+0000 7f336aa00640 1 -- 172.21.15.46:0/4049333761 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370004460 con 0x55c34cff2c30 2024-09-10T07:37:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f338e417740 1 -- 172.21.15.46:0/4049333761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337c004050 con 0x55c34cff2c30 2024-09-10T07:37:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f336aa00640 1 -- 172.21.15.46:0/4049333761 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f337000f920 con 0x55c34cff2c30 2024-09-10T07:37:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f336aa00640 1 -- 172.21.15.46:0/4049333761 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f337000fb20 con 0x55c34cff2c30 2024-09-10T07:37:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f338e417740 10 client.25699 fetch_fsmap learned FSMap version 34 2024-09-10T07:37:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f338e417740 1 -- 172.21.15.46:0/4049333761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c34d00bd40 con 0x55c34cff2c30 2024-09-10T07:37:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f336aa00640 1 -- 172.21.15.46:0/4049333761 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337c004050 con 0x55c34cff2c30 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f336aa00640 1 -- 172.21.15.46:0/4049333761 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f3370018070 con 0x55c34cff2c30 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f338e417740 10 client.25699 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f338e417740 10 client.25699 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.686+0000 7f338e417740 20 client.25699 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.687+0000 7f338e417740 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c34d00d220 0x55c34d00f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.687+0000 7f338e417740 4 client.25699 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.687+0000 7f338e417740 1 -- 172.21.15.46:0/4049333761 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c34cf65bc0 con 0x55c34d00d220 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.687+0000 7f3389a00640 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c34d00d220 0x55c34d00f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.687+0000 7f3389a00640 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c34d00d220 0x55c34d00f640 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6cf0d8b2bcac379 server_cookie=9de0bcc6a392d403 in_seq=0 out_seq=0 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.687+0000 7f336aa00640 10 client.25699 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.688+0000 7f336aa00640 1 -- 172.21.15.46:0/4049333761 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c34cf65bc0 con 0x55c34d00d220 2024-09-10T07:37:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.688+0000 7f336aa00640 10 client.25699 handle_command_reply: tid=0 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.688+0000 7f338e417740 1 client.25699 shutdown 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.688+0000 7f338e417740 1 -- 172.21.15.46:0/4049333761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2c30 msgr2=0x55c34cffa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.688+0000 7f338e417740 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2c30 0x55c34cffa8d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f33700116f0 tx=0x7f3370011bb0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.688+0000 7f338e417740 1 -- 172.21.15.46:0/4049333761 shutdown_connections 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.688+0000 7f338e417740 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c34d00d220 0x55c34d00f640 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.689+0000 7f338e417740 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 0x55c34cffae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.689+0000 7f338e417740 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2c30 0x55c34cffa8d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.689+0000 7f338e417740 1 --2- 172.21.15.46:0/4049333761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2310 0x55c34cffa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.689+0000 7f336aa00640 0 client.25699 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.689+0000 7f338e417740 1 -- 172.21.15.46:0/4049333761 >> 172.21.15.46:0/4049333761 conn(0x55c34cfee8a0 msgr2=0x55c34cff45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.689+0000 7f338e417740 1 -- 172.21.15.46:0/4049333761 shutdown_connections 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.689+0000 7f338e417740 1 -- 172.21.15.46:0/4049333761 wait complete. 2024-09-10T07:37:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.689+0000 7f338e417740 20 client.25699 trim_cache size 0 max 16384 2024-09-10T07:37:59.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f338e417740 1 Processor -- start 2024-09-10T07:37:59.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f338e417740 1 -- start start 2024-09-10T07:37:59.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f338e417740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2310 0x55c34cffac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f338e417740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d014ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f338e417740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cffd4f0 0x55c34d015430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f338e417740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b5290 con 0x55c34cff2310 2024-09-10T07:37:59.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f338e417740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b6e90 con 0x55c34cff2c30 2024-09-10T07:37:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f338e417740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b5960 con 0x55c34cffd4f0 2024-09-10T07:37:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f3389a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2310 0x55c34cffac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f3389a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2310 0x55c34cffac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57922/0 (socket says 172.21.15.46:57922) 2024-09-10T07:37:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.707+0000 7f3389a00640 1 -- 172.21.15.46:0/1308885682 learned_addr learned my addr 172.21.15.46:0/1308885682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3383e00640 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d014ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3389a00640 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cffd4f0 0x55c34d015430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3383e00640 1 -- 172.21.15.46:0/1308885682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cffd4f0 msgr2=0x55c34d015430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:59.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3383e00640 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cffd4f0 0x55c34d015430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3383e00640 1 -- 172.21.15.46:0/1308885682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2310 msgr2=0x55c34cffac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:59.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3383e00640 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2310 0x55c34cffac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3383e00640 1 -- 172.21.15.46:0/1308885682 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c34d00bb50 con 0x55c34cff2c30 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3389a00640 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2310 0x55c34cffac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.708+0000 7f3383e00640 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d014ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f33700116c0 tx=0x7f337000f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c12962820367fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f3375600640 1 -- 172.21.15.46:0/1308885682 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337001a860 con 0x55c34cff2c30 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f3375600640 1 -- 172.21.15.46:0/1308885682 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3370005bf0 con 0x55c34cff2c30 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f3375600640 1 -- 172.21.15.46:0/1308885682 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370003d10 con 0x55c34cff2c30 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 -- 172.21.15.46:0/1308885682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 msgr2=0x55c34d014ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d014ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f33700116c0 tx=0x7f337000f810 comp rx=0 tx=0).stop 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 -- 172.21.15.46:0/1308885682 shutdown_connections 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cffd4f0 0x55c34d015430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d014ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 --2- 172.21.15.46:0/1308885682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cff2310 0x55c34cffac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 -- 172.21.15.46:0/1308885682 >> 172.21.15.46:0/1308885682 conn(0x55c34cfee8a0 msgr2=0x55c34cff9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:59.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 -- 172.21.15.46:0/1308885682 shutdown_connections 2024-09-10T07:37:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.709+0000 7f338e417740 1 -- 172.21.15.46:0/1308885682 wait complete. 2024-09-10T07:37:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.710+0000 7f338e417740 1 -- start start 2024-09-10T07:37:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.710+0000 7f338e417740 1 Processor -- start 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.710+0000 7f338e417740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34d001da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.710+0000 7f338e417740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d0022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.710+0000 7f338e417740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 0x55c34d06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.710+0000 7f338e417740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b6700 con 0x55c34cffd4f0 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f338e417740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b6030 con 0x55c34cff2c30 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f338e417740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b3270 con 0x55c34cff2310 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f338e417740 10 client.0 _ll_register_callbacks cb 0x7f33700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3383e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d0022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3383e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d0022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38716/0 (socket says 172.21.15.46:38716) 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3383e00640 1 -- 172.21.15.46:0/279164221 learned_addr learned my addr 172.21.15.46:0/279164221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:37:59.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3389a00640 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 0x55c34d06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3389a00640 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34d001da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3383e00640 1 -- 172.21.15.46:0/279164221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 msgr2=0x55c34d001da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3383e00640 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34d001da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3383e00640 1 -- 172.21.15.46:0/279164221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 msgr2=0x55c34d06fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3383e00640 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 0x55c34d06fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3383e00640 1 -- 172.21.15.46:0/279164221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c34d0706b0 con 0x55c34cff2c30 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.711+0000 7f3389a00640 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34d001da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f3383e00640 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d0022e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3370000c00 tx=0x7f337000fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97710eaca132ea3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f336aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f336aa00640 1 -- 172.21.15.46:0/279164221 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370005500 con 0x55c34cff2c30 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f338e417740 1 -- 172.21.15.46:0/279164221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c34cf600d0 con 0x55c34cff2c30 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f336aa00640 1 -- 172.21.15.46:0/279164221 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3370003950 con 0x55c34cff2c30 2024-09-10T07:37:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f336aa00640 1 -- 172.21.15.46:0/279164221 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3370041050 con 0x55c34cff2c30 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f338e417740 10 client.25706 fetch_fsmap learned FSMap version 34 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f338e417740 1 -- 172.21.15.46:0/279164221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c34d078a60 con 0x55c34cff2c30 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f336aa00640 1 -- 172.21.15.46:0/279164221 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c34cf600d0 con 0x55c34cff2c30 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.712+0000 7f336aa00640 1 -- 172.21.15.46:0/279164221 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f337003c030 con 0x55c34cff2c30 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f338e417740 10 client.25706 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f338e417740 10 client.25706 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f338e417740 20 client.25706 populate_metadata read hostname 'smithi046' 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f338e417740 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c34d0791d0 0x55c34d07b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f338e417740 4 client.25706 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3850f863", "release": true, "await": true}] 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f338e417740 1 -- 172.21.15.46:0/279164221 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3850f863", "release": true, "await": true}) -- 0x55c34cedc430 con 0x55c34d0791d0 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f3389a00640 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c34d0791d0 0x55c34d07b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f3389a00640 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c34d0791d0 0x55c34d07b5f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85bad530c31bc63c server_cookie=6e64afbb6f09782e in_seq=0 out_seq=0 2024-09-10T07:37:59.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.713+0000 7f336aa00640 10 client.25706 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:59.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.755+0000 7f336aa00640 1 -- 172.21.15.46:0/279164221 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c34cedc430 con 0x55c34d0791d0 2024-09-10T07:37:59.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.755+0000 7f336aa00640 10 client.25706 handle_command_reply: tid=0 2024-09-10T07:37:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.755+0000 7f338e417740 1 client.25706 shutdown 2024-09-10T07:37:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f338e417740 1 -- 172.21.15.46:0/279164221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 msgr2=0x55c34d0022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f338e417740 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d0022e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3370000c00 tx=0x7f337000fef0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f338e417740 1 -- 172.21.15.46:0/279164221 shutdown_connections 2024-09-10T07:37:59.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f338e417740 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c34d0791d0 0x55c34d07b5f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f338e417740 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34cffd4f0 0x55c34d06fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f338e417740 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c34cff2c30 0x55c34d0022e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f338e417740 1 --2- 172.21.15.46:0/279164221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c34cff2310 0x55c34d001da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f338e417740 1 -- 172.21.15.46:0/279164221 >> 172.21.15.46:0/279164221 conn(0x55c34cfee8a0 msgr2=0x55c34cff3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.757+0000 7f336aa00640 0 client.25706 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:37:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.758+0000 7f338e417740 1 -- 172.21.15.46:0/279164221 shutdown_connections 2024-09-10T07:37:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.758+0000 7f338e417740 1 -- 172.21.15.46:0/279164221 wait complete. 2024-09-10T07:37:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.759+0000 7f338e417740 20 client.25706 trim_cache size 0 max 16384 2024-09-10T07:37:59.760 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 34, 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 4, 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "3850f863": { 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 4, 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:37:59.761 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 150.0, 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:37:59.762 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:37:59.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.761+0000 7f338ba00640 1 -- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f336c0a2b80 msgr2=0x7f336c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.761+0000 7f338ba00640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f336c0a2b80 0x7f336c0a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f33780b8670 tx=0x7f33780f5530 comp rx=0 tx=0).stop 2024-09-10T07:37:59.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.761+0000 7f338ba00640 1 -- 172.21.15.46:0/320171327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3384074590 msgr2=0x7f33841a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:37:59.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.761+0000 7f338ba00640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3384074590 0x7f33841a4720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3378006010 tx=0x7f3378004040 comp rx=0 tx=0).stop 2024-09-10T07:37:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.761+0000 7f338ba00640 1 -- 172.21.15.46:0/320171327 shutdown_connections 2024-09-10T07:37:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.762+0000 7f338ba00640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3384112b90 0x7f338419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.762+0000 7f338ba00640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f336c0a2b80 0x7f336c0a5040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.762+0000 7f338ba00640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384074eb0 0x7f33841a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.762+0000 7f338ba00640 1 --2- 172.21.15.46:0/320171327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3384074590 0x7f33841a4720 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:37:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.762+0000 7f338ba00640 1 -- 172.21.15.46:0/320171327 >> 172.21.15.46:0/320171327 conn(0x7f33840fd8c0 msgr2=0x7f33840ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:37:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.763+0000 7f338ba00640 1 -- 172.21.15.46:0/320171327 shutdown_connections 2024-09-10T07:37:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:37:59.763+0000 7f338ba00640 1 -- 172.21.15.46:0/320171327 wait complete. 2024-09-10T07:38:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 40.0 s, 391.2 tps, lat 1393.308 ms stddev 5823.789 2024-09-10T07:38:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:00 smithi181 ceph-mon[35397]: pgmap v440: 129 pgs: 129 active+clean; 916 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.2 MiB/s wr, 32 op/s 2024-09-10T07:38:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:00 smithi063 ceph-mon[35020]: pgmap v440: 129 pgs: 129 active+clean; 916 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.2 MiB/s wr, 32 op/s 2024-09-10T07:38:00.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:00 smithi046 ceph-mon[30705]: pgmap v440: 129 pgs: 129 active+clean; 916 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.2 MiB/s wr, 32 op/s 2024-09-10T07:38:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.729+0000 7f6a14a00640 1 -- 172.21.15.46:0/3525664587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a10104db0 msgr2=0x7f6a101071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.729+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3525664587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a10104db0 0x7f6a101071a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009900 tx=0x7f6a0002a410 comp rx=0 tx=0).stop 2024-09-10T07:38:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.730+0000 7f6a14a00640 1 -- 172.21.15.46:0/3525664587 shutdown_connections 2024-09-10T07:38:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.730+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3525664587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a101076e0 0x7f6a10109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.730+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3525664587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a10104db0 0x7f6a101071a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.730+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3525664587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10102480 0x7f6a10104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.730+0000 7f6a14a00640 1 -- 172.21.15.46:0/3525664587 >> 172.21.15.46:0/3525664587 conn(0x7f6a100fbfa0 msgr2=0x7f6a100fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.730+0000 7f6a14a00640 1 -- 172.21.15.46:0/3525664587 shutdown_connections 2024-09-10T07:38:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.730+0000 7f6a14a00640 1 -- 172.21.15.46:0/3525664587 wait complete. 2024-09-10T07:38:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.732+0000 7f6a14a00640 1 Processor -- start 2024-09-10T07:38:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.732+0000 7f6a14a00640 1 -- start start 2024-09-10T07:38:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a14a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10102480 0x7f6a10199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a14a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a10104db0 0x7f6a1019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a14a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a101076e0 0x7f6a101a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a14a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101009d0 con 0x7f6a10102480 2024-09-10T07:38:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a14a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10100850 con 0x7f6a10104db0 2024-09-10T07:38:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a14a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10100b50 con 0x7f6a101076e0 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a10104db0 0x7f6a1019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a10104db0 0x7f6a1019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38738/0 (socket says 172.21.15.46:38738) 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a0ea00640 1 -- 172.21.15.46:0/3651888539 learned_addr learned my addr 172.21.15.46:0/3651888539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a101076e0 0x7f6a101a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a0f400640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10102480 0x7f6a10199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a0fe00640 1 -- 172.21.15.46:0/3651888539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a10104db0 msgr2=0x7f6a1019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a10104db0 0x7f6a1019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.733+0000 7f6a0fe00640 1 -- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10102480 msgr2=0x7f6a10199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.734+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10102480 0x7f6a10199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.734+0000 7f6a0fe00640 1 -- 172.21.15.46:0/3651888539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a101a1a90 con 0x7f6a101076e0 2024-09-10T07:38:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.734+0000 7f6a0f400640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10102480 0x7f6a10199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.734+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a101076e0 0x7f6a101a1420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009fd0 tx=0x7f6a0800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.734+0000 7f6a07e00640 1 -- 172.21.15.46:0/3651888539 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800cd60 con 0x7f6a101076e0 2024-09-10T07:38:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.734+0000 7f6a07e00640 1 -- 172.21.15.46:0/3651888539 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a08005020 con 0x7f6a101076e0 2024-09-10T07:38:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.734+0000 7f6a07e00640 1 -- 172.21.15.46:0/3651888539 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08010690 con 0x7f6a101076e0 2024-09-10T07:38:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.734+0000 7f6a14a00640 1 -- 172.21.15.46:0/3651888539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1010cef0 con 0x7f6a101076e0 2024-09-10T07:38:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.735+0000 7f6a14a00640 1 -- 172.21.15.46:0/3651888539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1010d3f0 con 0x7f6a101076e0 2024-09-10T07:38:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.742+0000 7f6a07e00640 1 -- 172.21.15.46:0/3651888539 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a08005540 con 0x7f6a101076e0 2024-09-10T07:38:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.742+0000 7f6a19068740 1 Processor -- start 2024-09-10T07:38:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.743+0000 7f6a19068740 1 -- start start 2024-09-10T07:38:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a19068740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d2260 0x5651270d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a19068740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a19068740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d7c30 0x5651270da0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a19068740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x565127037100 con 0x5651270d2260 2024-09-10T07:38:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a19068740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x565126fdf1b0 con 0x5651270d2ba0 2024-09-10T07:38:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a19068740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5651270330c0 con 0x5651270d7c30 2024-09-10T07:38:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270d7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38754/0 (socket says 172.21.15.46:38754) 2024-09-10T07:38:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.744+0000 7f6a0f400640 1 -- 172.21.15.46:0/3177145854 learned_addr learned my addr 172.21.15.46:0/3177145854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.745+0000 7f6a0f400640 1 -- 172.21.15.46:0/3177145854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d7c30 msgr2=0x5651270da0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:38:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.745+0000 7f6a0f400640 1 --2- 172.21.15.46:0/3177145854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d7c30 0x5651270da0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.745+0000 7f6a0f400640 1 -- 172.21.15.46:0/3177145854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d2260 msgr2=0x5651270d2660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:38:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.745+0000 7f6a07e00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69f40a2c80 0x7f69f40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.745+0000 7f6a07e00640 1 -- 172.21.15.46:0/3651888539 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6a080cfad0 con 0x7f6a101076e0 2024-09-10T07:38:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.746+0000 7f6a0f400640 1 --2- 172.21.15.46:0/3177145854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d2260 0x5651270d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.746+0000 7f6a0f400640 1 -- 172.21.15.46:0/3177145854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565126d4a210 con 0x5651270d2ba0 2024-09-10T07:38:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.746+0000 7f6a0f400640 1 --2- 172.21.15.46:0/3177145854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270d7660 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f69fc007a20 tx=0x7f69fc012ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc2ec69a554ade6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.746+0000 7f69fbe00640 1 -- 172.21.15.46:0/3177145854 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc010c50 con 0x5651270d2ba0 2024-09-10T07:38:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.746+0000 7f69fbe00640 1 -- 172.21.15.46:0/3177145854 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69fc010df0 con 0x5651270d2ba0 2024-09-10T07:38:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.746+0000 7f69fbe00640 1 -- 172.21.15.46:0/3177145854 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0177e0 con 0x5651270d2ba0 2024-09-10T07:38:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a0ea00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69f40a2c80 0x7f69f40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a19068740 1 -- 172.21.15.46:0/3177145854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 msgr2=0x5651270d7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a19068740 1 --2- 172.21.15.46:0/3177145854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270d7660 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f69fc007a20 tx=0x7f69fc012ae0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a19068740 1 -- 172.21.15.46:0/3177145854 shutdown_connections 2024-09-10T07:38:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a19068740 1 --2- 172.21.15.46:0/3177145854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d7c30 0x5651270da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a19068740 1 --2- 172.21.15.46:0/3177145854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270d7660 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a19068740 1 --2- 172.21.15.46:0/3177145854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d2260 0x5651270d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a19068740 1 -- 172.21.15.46:0/3177145854 >> 172.21.15.46:0/3177145854 conn(0x5651270cd8d0 msgr2=0x5651270cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.747+0000 7f6a19068740 1 -- 172.21.15.46:0/3177145854 shutdown_connections 2024-09-10T07:38:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.748+0000 7f6a19068740 1 -- 172.21.15.46:0/3177145854 wait complete. 2024-09-10T07:38:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.749+0000 7f6a19068740 1 -- start start 2024-09-10T07:38:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.749+0000 7f6a19068740 1 Processor -- start 2024-09-10T07:38:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.750+0000 7f6a0ea00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69f40a2c80 0x7f69f40a5140 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6a1019b380 tx=0x7f6a000315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.751+0000 7f6a19068740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.751+0000 7f6a19068740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a19068740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d7c30 0x5651270e3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a19068740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5651270daba0 con 0x5651270d7c30 2024-09-10T07:38:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a19068740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x565126fc0090 con 0x5651270d2ba0 2024-09-10T07:38:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a19068740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5651270dad20 con 0x5651270d2260 2024-09-10T07:38:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a19068740 10 client.0 _ll_register_callbacks cb 0x565127014f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270dc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60664/0 (socket says 172.21.15.46:60664) 2024-09-10T07:38:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0f400640 1 -- 172.21.15.46:0/1917285034 learned_addr learned my addr 172.21.15.46:0/1917285034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0ea00640 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d7c30 0x5651270e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0f400640 1 -- 172.21.15.46:0/1917285034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 msgr2=0x5651270dc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0f400640 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0f400640 1 -- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d7c30 msgr2=0x5651270e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0f400640 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d7c30 0x5651270e3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0f400640 1 -- 172.21.15.46:0/1917285034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565126e16620 con 0x5651270d2260 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.752+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d7c30 0x5651270e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.753+0000 7f6a0f400640 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270dc1f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69fc012dd0 tx=0x7f69fc017bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c5a6e37189b1b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.753+0000 7f69f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:38:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.753+0000 7f69f8c00640 1 -- 172.21.15.46:0/1917285034 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc02c070 con 0x5651270d2260 2024-09-10T07:38:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.753+0000 7f69f8c00640 1 -- 172.21.15.46:0/1917285034 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f69fc0120d0 con 0x5651270d2260 2024-09-10T07:38:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.753+0000 7f6a19068740 1 -- 172.21.15.46:0/1917285034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69fc010df0 con 0x5651270d2260 2024-09-10T07:38:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.754+0000 7f69f8c00640 1 -- 172.21.15.46:0/1917285034 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69fc038050 con 0x5651270d2260 2024-09-10T07:38:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.754+0000 7f6a19068740 10 client.25714 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.754+0000 7f6a19068740 1 -- 172.21.15.46:0/1917285034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5651270e6560 con 0x5651270d2260 2024-09-10T07:38:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.754+0000 7f69f8c00640 1 -- 172.21.15.46:0/1917285034 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69fc010df0 con 0x5651270d2260 2024-09-10T07:38:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.755+0000 7f69f8c00640 1 -- 172.21.15.46:0/1917285034 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f69fc01d9c0 con 0x5651270d2260 2024-09-10T07:38:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.755+0000 7f6a19068740 10 client.25714 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.755+0000 7f6a19068740 10 client.25714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.755+0000 7f6a19068740 20 client.25714 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.755+0000 7f6a19068740 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651270e7a70 0x5651270e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.755+0000 7f6a19068740 4 client.25714 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.755+0000 7f6a19068740 1 -- 172.21.15.46:0/1917285034 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565127044830 con 0x5651270e7a70 2024-09-10T07:38:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.756+0000 7f6a0ea00640 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651270e7a70 0x5651270e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.758+0000 7f6a0ea00640 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651270e7a70 0x5651270e9ed0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1493a899854c15 server_cookie=6da88cdf72d0cfd5 in_seq=0 out_seq=0 2024-09-10T07:38:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.758+0000 7f69f8c00640 10 client.25714 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.760+0000 7f69f8c00640 1 -- 172.21.15.46:0/1917285034 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565127044830 con 0x5651270e7a70 2024-09-10T07:38:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.760+0000 7f69f8c00640 10 client.25714 handle_command_reply: tid=0 2024-09-10T07:38:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.760+0000 7f6a19068740 1 client.25714 shutdown 2024-09-10T07:38:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 -- 172.21.15.46:0/1917285034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 msgr2=0x5651270dc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270dc1f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69fc012dd0 tx=0x7f69fc017bd0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 -- 172.21.15.46:0/1917285034 shutdown_connections 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651270e7a70 0x5651270e9ed0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d7c30 0x5651270e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 --2- 172.21.15.46:0/1917285034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270dc1f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f69f8c00640 0 client.25714 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 -- 172.21.15.46:0/1917285034 >> 172.21.15.46:0/1917285034 conn(0x5651270cd8d0 msgr2=0x5651270cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 -- 172.21.15.46:0/1917285034 shutdown_connections 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 1 -- 172.21.15.46:0/1917285034 wait complete. 2024-09-10T07:38:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.761+0000 7f6a19068740 20 client.25714 trim_cache size 0 max 16384 2024-09-10T07:38:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.797+0000 7f6a19068740 1 Processor -- start 2024-09-10T07:38:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.798+0000 7f6a19068740 1 -- start start 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.798+0000 7f6a19068740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270cd8d0 0x5651270dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.798+0000 7f6a19068740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270e3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.798+0000 7f6a19068740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.798+0000 7f6a19068740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101a8330 con 0x5651270cd8d0 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.798+0000 7f6a19068740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101a9f30 con 0x5651270d2ba0 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.798+0000 7f6a19068740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101a8a00 con 0x5651270d2260 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.799+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.799+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38762/0 (socket says 172.21.15.46:38762) 2024-09-10T07:38:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.799+0000 7f6a0fe00640 1 -- 172.21.15.46:0/1263728142 learned_addr learned my addr 172.21.15.46:0/1263728142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.799+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f6a0fe00640 1 -- 172.21.15.46:0/1263728142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 msgr2=0x5651270e3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/1263728142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270e3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f6a0fe00640 1 -- 172.21.15.46:0/1263728142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270cd8d0 msgr2=0x5651270dbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/1263728142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270cd8d0 0x5651270dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f6a0fe00640 1 -- 172.21.15.46:0/1263728142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651270daea0 con 0x5651270d2ba0 2024-09-10T07:38:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f6a0ea00640 1 --2- 172.21.15.46:0/1263728142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:38:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f6a0fe00640 1 --2- 172.21.15.46:0/1263728142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc4e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800c6a0 tx=0x7f6a08005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3991bb183713f58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f69fb400640 1 -- 172.21.15.46:0/1263728142 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a080b4070 con 0x5651270d2ba0 2024-09-10T07:38:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f69fb400640 1 -- 172.21.15.46:0/1263728142 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a080a4bc0 con 0x5651270d2ba0 2024-09-10T07:38:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.800+0000 7f69fb400640 1 -- 172.21.15.46:0/1263728142 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a080a4730 con 0x5651270d2ba0 2024-09-10T07:38:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.801+0000 7f6a19068740 1 -- 172.21.15.46:0/1263728142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 msgr2=0x5651270dc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.801+0000 7f6a19068740 1 --2- 172.21.15.46:0/1263728142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc4e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800c6a0 tx=0x7f6a08005940 comp rx=0 tx=0).stop 2024-09-10T07:38:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.802+0000 7f6a19068740 1 -- 172.21.15.46:0/1263728142 shutdown_connections 2024-09-10T07:38:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.802+0000 7f6a19068740 1 --2- 172.21.15.46:0/1263728142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2ba0 0x5651270dc4e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.802+0000 7f6a19068740 1 --2- 172.21.15.46:0/1263728142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270d2260 0x5651270e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.802+0000 7f6a19068740 1 --2- 172.21.15.46:0/1263728142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270cd8d0 0x5651270dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.802+0000 7f6a19068740 1 -- 172.21.15.46:0/1263728142 >> 172.21.15.46:0/1263728142 conn(0x5651270ce020 msgr2=0x5651270c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.802+0000 7f6a19068740 1 -- 172.21.15.46:0/1263728142 shutdown_connections 2024-09-10T07:38:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.802+0000 7f6a19068740 1 -- 172.21.15.46:0/1263728142 wait complete. 2024-09-10T07:38:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.803+0000 7f6a19068740 1 -- start start 2024-09-10T07:38:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.803+0000 7f6a19068740 1 Processor -- start 2024-09-10T07:38:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a19068740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270cd8d0 0x5651270de890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a19068740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2260 0x5651270dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a19068740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d2ba0 0x5651270e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a19068740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101a97a0 con 0x5651270d2ba0 2024-09-10T07:38:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a19068740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101a90d0 con 0x5651270d2260 2024-09-10T07:38:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a19068740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101a6330 con 0x5651270cd8d0 2024-09-10T07:38:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a19068740 10 client.0 _ll_register_callbacks cb 0x7f69fc0177e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270cd8d0 0x5651270de890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.804+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270cd8d0 0x5651270de890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60696/0 (socket says 172.21.15.46:60696) 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.805+0000 7f6a0f400640 1 -- 172.21.15.46:0/1558388360 learned_addr learned my addr 172.21.15.46:0/1558388360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.805+0000 7f6a0f400640 1 -- 172.21.15.46:0/1558388360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2260 msgr2=0x5651270dedd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.805+0000 7f6a0f400640 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2260 0x5651270dedd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.805+0000 7f6a0f400640 1 -- 172.21.15.46:0/1558388360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d2ba0 msgr2=0x5651270e1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.805+0000 7f6a0f400640 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d2ba0 0x5651270e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.805+0000 7f6a0f400640 1 -- 172.21.15.46:0/1558388360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651270e2400 con 0x5651270cd8d0 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.805+0000 7f6a0f400640 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270cd8d0 0x5651270de890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f69fc01dd90 tx=0x7f69fc012ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8634c93fd12f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.805+0000 7f69f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:38:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.806+0000 7f69f8c00640 1 -- 172.21.15.46:0/1558388360 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc02c070 con 0x5651270cd8d0 2024-09-10T07:38:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.806+0000 7f69f8c00640 1 -- 172.21.15.46:0/1558388360 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f69fc028460 con 0x5651270cd8d0 2024-09-10T07:38:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.806+0000 7f6a19068740 1 -- 172.21.15.46:0/1558388360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a080a4bc0 con 0x5651270cd8d0 2024-09-10T07:38:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.807+0000 7f69f8c00640 1 -- 172.21.15.46:0/1558388360 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69fc010e50 con 0x5651270cd8d0 2024-09-10T07:38:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.807+0000 7f6a19068740 10 client.25720 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.807+0000 7f6a19068740 1 -- 172.21.15.46:0/1558388360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5651270e3710 con 0x5651270cd8d0 2024-09-10T07:38:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.807+0000 7f69f8c00640 1 -- 172.21.15.46:0/1558388360 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a080a4bc0 con 0x5651270cd8d0 2024-09-10T07:38:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.808+0000 7f69f8c00640 1 -- 172.21.15.46:0/1558388360 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f69fc0106a0 con 0x5651270cd8d0 2024-09-10T07:38:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.809+0000 7f6a19068740 10 client.25720 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.809+0000 7f6a19068740 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.809+0000 7f6a19068740 20 client.25720 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.809+0000 7f6a19068740 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651270d7c30 0x565127165d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.809+0000 7f6a19068740 4 client.25720 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:38:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.809+0000 7f6a19068740 1 -- 172.21.15.46:0/1558388360 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56512703eb80 con 0x5651270d7c30 2024-09-10T07:38:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.810+0000 7f6a0ea00640 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651270d7c30 0x565127165d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.811+0000 7f6a0ea00640 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651270d7c30 0x565127165d60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a305a80904a1e203 server_cookie=baf1084460ea3e3f in_seq=0 out_seq=0 2024-09-10T07:38:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.811+0000 7f69f8c00640 10 client.25720 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.835+0000 7f69f8c00640 1 -- 172.21.15.46:0/1558388360 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56512703eb80 con 0x5651270d7c30 2024-09-10T07:38:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.835+0000 7f69f8c00640 10 client.25720 handle_command_reply: tid=0 2024-09-10T07:38:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.835+0000 7f6a19068740 1 client.25720 shutdown 2024-09-10T07:38:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.836+0000 7f6a19068740 1 -- 172.21.15.46:0/1558388360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270cd8d0 msgr2=0x5651270de890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.836+0000 7f6a19068740 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270cd8d0 0x5651270de890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f69fc01dd90 tx=0x7f69fc012ea0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.837+0000 7f6a19068740 1 -- 172.21.15.46:0/1558388360 shutdown_connections 2024-09-10T07:38:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.837+0000 7f6a19068740 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651270d7c30 0x565127165d60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.837+0000 7f6a19068740 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651270d2ba0 0x5651270e1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.837+0000 7f6a19068740 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651270d2260 0x5651270dedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.837+0000 7f6a19068740 1 --2- 172.21.15.46:0/1558388360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651270cd8d0 0x5651270de890 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.837+0000 7f6a19068740 1 -- 172.21.15.46:0/1558388360 >> 172.21.15.46:0/1558388360 conn(0x5651270ce020 msgr2=0x5651270dbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.837+0000 7f69f8c00640 0 client.25720 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.838+0000 7f6a19068740 1 -- 172.21.15.46:0/1558388360 shutdown_connections 2024-09-10T07:38:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.838+0000 7f6a19068740 1 -- 172.21.15.46:0/1558388360 wait complete. 2024-09-10T07:38:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.838+0000 7f6a19068740 20 client.25720 trim_cache size 0 max 16384 2024-09-10T07:38:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:38:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:38:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:38:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:38:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 -- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69f40a2c80 msgr2=0x7f69f40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69f40a2c80 0x7f69f40a5140 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6a1019b380 tx=0x7f6a000315b0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 -- 172.21.15.46:0/3651888539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a101076e0 msgr2=0x7f6a101a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a101076e0 0x7f6a101a1420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009fd0 tx=0x7f6a0800ee50 comp rx=0 tx=0).stop 2024-09-10T07:38:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 -- 172.21.15.46:0/3651888539 shutdown_connections 2024-09-10T07:38:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a101076e0 0x7f6a101a1420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a10104db0 0x7f6a1019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69f40a2c80 0x7f69f40a5140 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 --2- 172.21.15.46:0/3651888539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10102480 0x7f6a10199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.841+0000 7f6a14a00640 1 -- 172.21.15.46:0/3651888539 >> 172.21.15.46:0/3651888539 conn(0x7f6a100fbfa0 msgr2=0x7f6a10106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.842+0000 7f6a14a00640 1 -- 172.21.15.46:0/3651888539 shutdown_connections 2024-09-10T07:38:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:00.842+0000 7f6a14a00640 1 -- 172.21.15.46:0/3651888539 wait complete. 2024-09-10T07:38:00.951 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3850f863', total seconds elapsed: 34.4 2024-09-10T07:38:00.951 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T07:38:00.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:38:01.601 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 -- 172.21.15.46:0/1699341789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00d8106f00 msgr2=0x7f00d810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 --2- 172.21.15.46:0/1699341789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00d8106f00 0x7f00d810ea80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009be0 tx=0x7f00d002a6c0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 -- 172.21.15.46:0/1699341789 shutdown_connections 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 --2- 172.21.15.46:0/1699341789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00d8106f00 0x7f00d810ea80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 --2- 172.21.15.46:0/1699341789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d8102780 0x7f00d81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 --2- 172.21.15.46:0/1699341789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8101dd0 0x7f00d81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 -- 172.21.15.46:0/1699341789 >> 172.21.15.46:0/1699341789 conn(0x7f00d80fd890 msgr2=0x7f00d80ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 -- 172.21.15.46:0/1699341789 shutdown_connections 2024-09-10T07:38:02.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.357+0000 7f00dd200640 1 -- 172.21.15.46:0/1699341789 wait complete. 2024-09-10T07:38:02.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.358+0000 7f00dd200640 1 Processor -- start 2024-09-10T07:38:02.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.359+0000 7f00dd200640 1 -- start start 2024-09-10T07:38:02.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00dd200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00d8101dd0 0x7f00d819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:02.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00dd200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8102780 0x7f00d819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:02.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d8106f00 0x7f00d810dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:02.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00dd200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8111f60 con 0x7f00d8106f00 2024-09-10T07:38:02.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00dd200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8111de0 con 0x7f00d8102780 2024-09-10T07:38:02.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00dd200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81120e0 con 0x7f00d8101dd0 2024-09-10T07:38:02.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00d5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8102780 0x7f00d819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:02.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00d5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8102780 0x7f00d819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38784/0 (socket says 172.21.15.46:38784) 2024-09-10T07:38:02.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00d5400640 1 -- 172.21.15.46:0/2746736601 learned_addr learned my addr 172.21.15.46:0/2746736601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:02.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00d5e00640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00d8101dd0 0x7f00d819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:02.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.360+0000 7f00d6800640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d8106f00 0x7f00d810dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.361+0000 7f00d5400640 1 -- 172.21.15.46:0/2746736601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00d8101dd0 msgr2=0x7f00d819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.361+0000 7f00d5400640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00d8101dd0 0x7f00d819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.361+0000 7f00d5400640 1 -- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d8106f00 msgr2=0x7f00d810dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.361+0000 7f00d5400640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d8106f00 0x7f00d810dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.361+0000 7f00d5400640 1 -- 172.21.15.46:0/2746736601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d810e540 con 0x7f00d8102780 2024-09-10T07:38:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.361+0000 7f00d6800640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d8106f00 0x7f00d810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.362+0000 7f00d5400640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8102780 0x7f00d819f5d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f00c800e9c0 tx=0x7f00c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:02.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.362+0000 7f00cea00640 1 -- 172.21.15.46:0/2746736601 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c800cc40 con 0x7f00d8102780 2024-09-10T07:38:02.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.362+0000 7f00cea00640 1 -- 172.21.15.46:0/2746736601 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c800cde0 con 0x7f00d8102780 2024-09-10T07:38:02.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.362+0000 7f00cea00640 1 -- 172.21.15.46:0/2746736601 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8010430 con 0x7f00d8102780 2024-09-10T07:38:02.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.362+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d810e830 con 0x7f00d8102780 2024-09-10T07:38:02.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.363+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d819acb0 con 0x7f00d8102780 2024-09-10T07:38:02.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.364+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d81032b0 con 0x7f00d8102780 2024-09-10T07:38:02.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.369+0000 7f00cea00640 1 -- 172.21.15.46:0/2746736601 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00c8004f80 con 0x7f00d8102780 2024-09-10T07:38:02.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.371+0000 7f00cea00640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00bc0a2c40 0x7f00bc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:02.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.372+0000 7f00cea00640 1 -- 172.21.15.46:0/2746736601 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f00c80cf500 con 0x7f00d8102780 2024-09-10T07:38:02.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.374+0000 7f00d5e00640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00bc0a2c40 0x7f00bc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:02.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.375+0000 7f00cea00640 1 -- 172.21.15.46:0/2746736601 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00c809c0e0 con 0x7f00d8102780 2024-09-10T07:38:02.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.375+0000 7f00d5e00640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00bc0a2c40 0x7f00bc0a5100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00c0005d00 tx=0x7f00c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:02.450 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:02 smithi181 ceph-mon[35397]: pgmap v441: 129 pgs: 129 active+clean; 916 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 48 op/s 2024-09-10T07:38:02.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:02 smithi063 ceph-mon[35020]: pgmap v441: 129 pgs: 129 active+clean; 916 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 48 op/s 2024-09-10T07:38:02.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:02 smithi046 ceph-mon[30705]: pgmap v441: 129 pgs: 129 active+clean; 916 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 48 op/s 2024-09-10T07:38:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.725+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00d819b7c0 con 0x7f00d8102780 2024-09-10T07:38:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.725+0000 7f00cea00640 1 -- 172.21.15.46:0/2746736601 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11979 (secure 0 0 0) 0x7f00c809b9e0 con 0x7f00d8102780 2024-09-10T07:38:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:38:02.729 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"btime":"2024-09-10T07:37:35:593020+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:37:35.488899+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00bc0a2c40 msgr2=0x7f00bc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00bc0a2c40 0x7f00bc0a5100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00c0005d00 tx=0x7f00c000b410 comp rx=0 tx=0).stop 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8102780 msgr2=0x7f00d819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8102780 0x7f00d819f5d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f00c800e9c0 tx=0x7f00c800ee80 comp rx=0 tx=0).stop 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 shutdown_connections 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00bc0a2c40 0x7f00bc0a5100 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d8106f00 0x7f00d810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d8102780 0x7f00d819f5d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 --2- 172.21.15.46:0/2746736601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00d8101dd0 0x7f00d819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:02.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 >> 172.21.15.46:0/2746736601 conn(0x7f00d80fd890 msgr2=0x7f00d80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:02.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 shutdown_connections 2024-09-10T07:38:02.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:02.730+0000 7f00dd200640 1 -- 172.21.15.46:0/2746736601 wait complete. 2024-09-10T07:38:03.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:03 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2746736601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:03.615 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T07:38:03.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T07:38:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:03 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2746736601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:03.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:03 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2746736601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:04.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:04 smithi046 ceph-mon[30705]: pgmap v442: 129 pgs: 129 active+clean; 916 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-09-10T07:38:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:04 smithi181 ceph-mon[35397]: pgmap v442: 129 pgs: 129 active+clean; 916 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-09-10T07:38:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:04 smithi063 ceph-mon[35020]: pgmap v442: 129 pgs: 129 active+clean; 916 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2024-09-10T07:38:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.111+0000 7f5e29800640 1 -- 172.21.15.46:0/94165012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 msgr2=0x7f5e24076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.111+0000 7f5e29800640 1 --2- 172.21.15.46:0/94165012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 0x7f5e24076b00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009900 tx=0x7f5e1402a410 comp rx=0 tx=0).stop 2024-09-10T07:38:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.112+0000 7f5e29800640 1 -- 172.21.15.46:0/94165012 shutdown_connections 2024-09-10T07:38:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.112+0000 7f5e29800640 1 --2- 172.21.15.46:0/94165012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e24112a30 0x7f5e24114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.112+0000 7f5e29800640 1 --2- 172.21.15.46:0/94165012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e24077040 0x7f5e241124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.112+0000 7f5e29800640 1 --2- 172.21.15.46:0/94165012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 0x7f5e24076b00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.112+0000 7f5e29800640 1 -- 172.21.15.46:0/94165012 >> 172.21.15.46:0/94165012 conn(0x7f5e240ffab0 msgr2=0x7f5e24101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.112+0000 7f5e29800640 1 -- 172.21.15.46:0/94165012 shutdown_connections 2024-09-10T07:38:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.112+0000 7f5e29800640 1 -- 172.21.15.46:0/94165012 wait complete. 2024-09-10T07:38:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.113+0000 7f5e29800640 1 Processor -- start 2024-09-10T07:38:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.114+0000 7f5e29800640 1 -- start start 2024-09-10T07:38:05.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.114+0000 7f5e29800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 0x7f5e24111f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.114+0000 7f5e29800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e24077040 0x7f5e24112490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.114+0000 7f5e29800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e24112a30 0x7f5e2410b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.116+0000 7f5e22000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e24077040 0x7f5e24112490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.116+0000 7f5e23400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e24112a30 0x7f5e2410b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.116+0000 7f5e23400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e24112a30 0x7f5e2410b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60722/0 (socket says 172.21.15.46:60722) 2024-09-10T07:38:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.116+0000 7f5e23400640 1 -- 172.21.15.46:0/2072199752 learned_addr learned my addr 172.21.15.46:0/2072199752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.116+0000 7f5e22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 0x7f5e24111f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.116+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241185f0 con 0x7f5e24076720 2024-09-10T07:38:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.116+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e24118470 con 0x7f5e24077040 2024-09-10T07:38:05.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.116+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e24118770 con 0x7f5e24112a30 2024-09-10T07:38:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.117+0000 7f5e23400640 1 -- 172.21.15.46:0/2072199752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e24077040 msgr2=0x7f5e24112490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.117+0000 7f5e23400640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e24077040 0x7f5e24112490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.117+0000 7f5e23400640 1 -- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 msgr2=0x7f5e24111f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.117+0000 7f5e23400640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 0x7f5e24111f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.117+0000 7f5e23400640 1 -- 172.21.15.46:0/2072199752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2410bad0 con 0x7f5e24112a30 2024-09-10T07:38:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.117+0000 7f5e22000640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e24077040 0x7f5e24112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:38:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.118+0000 7f5e22a00640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 0x7f5e24111f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:38:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.119+0000 7f5e23400640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e24112a30 0x7f5e2410b210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00eb10 tx=0x7f5e1c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.119+0000 7f5e1b400640 1 -- 172.21.15.46:0/2072199752 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c004b20 con 0x7f5e24112a30 2024-09-10T07:38:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.119+0000 7f5e1b400640 1 -- 172.21.15.46:0/2072199752 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e1c004cc0 con 0x7f5e24112a30 2024-09-10T07:38:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.119+0000 7f5e1b400640 1 -- 172.21.15.46:0/2072199752 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c010790 con 0x7f5e24112a30 2024-09-10T07:38:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.119+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2410bd60 con 0x7f5e24112a30 2024-09-10T07:38:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.119+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2410fd00 con 0x7f5e24112a30 2024-09-10T07:38:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.124+0000 7f5e2c1fa740 1 Processor -- start 2024-09-10T07:38:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.125+0000 7f5e1b400640 1 -- 172.21.15.46:0/2072199752 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e1c010950 con 0x7f5e24112a30 2024-09-10T07:38:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.125+0000 7f5e2c1fa740 1 -- start start 2024-09-10T07:38:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.126+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.126+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.126+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 0x562de95d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.126+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562de952b530 con 0x562de95c5cd0 2024-09-10T07:38:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.126+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562de94d2860 con 0x562de95d0590 2024-09-10T07:38:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.126+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562de95274f0 con 0x562de95c53b0 2024-09-10T07:38:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.127+0000 7f5e22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.127+0000 7f5e22000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.127+0000 7f5e22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95d0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58008/0 (socket says 172.21.15.46:58008) 2024-09-10T07:38:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.127+0000 7f5e22000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60724/0 (socket says 172.21.15.46:60724) 2024-09-10T07:38:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.127+0000 7f5e22a00640 1 -- 172.21.15.46:0/3328819289 learned_addr learned my addr 172.21.15.46:0/3328819289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:05.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.128+0000 7f5e22000640 1 --2- 172.21.15.46:0/3328819289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 0x562de95d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.129+0000 7f5e1b400640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e080a2c00 0x7f5e080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.129+0000 7f5e22a00640 1 -- 172.21.15.46:0/3328819289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 msgr2=0x562de95c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.129+0000 7f5e22a00640 1 --2- 172.21.15.46:0/3328819289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.129+0000 7f5e22a00640 1 -- 172.21.15.46:0/3328819289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 msgr2=0x562de95d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.129+0000 7f5e22a00640 1 --2- 172.21.15.46:0/3328819289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 0x562de95d2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.129+0000 7f5e22a00640 1 -- 172.21.15.46:0/3328819289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de929cb90 con 0x562de95c5cd0 2024-09-10T07:38:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.129+0000 7f5e1b400640 1 -- 172.21.15.46:0/2072199752 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5e1c0cfc40 con 0x7f5e24112a30 2024-09-10T07:38:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.129+0000 7f5e23400640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e080a2c00 0x7f5e080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.130+0000 7f5e22a00640 1 --2- 172.21.15.46:0/3328819289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95d0050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009900 tx=0x7f5e14003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516f5c578672d1ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:05.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.130+0000 7f5e23400640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e080a2c00 0x7f5e080a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00eff0 tx=0x7f5e1c005f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:05.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.132+0000 7f5e0fe00640 1 -- 172.21.15.46:0/3328819289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14041070 con 0x562de95c5cd0 2024-09-10T07:38:05.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.132+0000 7f5e0fe00640 1 -- 172.21.15.46:0/3328819289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e14005000 con 0x562de95c5cd0 2024-09-10T07:38:05.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.132+0000 7f5e0fe00640 1 -- 172.21.15.46:0/3328819289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14041070 con 0x562de95c5cd0 2024-09-10T07:38:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.133+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3328819289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 msgr2=0x562de95d0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.133+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/3328819289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95d0050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009900 tx=0x7f5e14003700 comp rx=0 tx=0).stop 2024-09-10T07:38:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.134+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3328819289 shutdown_connections 2024-09-10T07:38:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.134+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/3328819289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 0x562de95d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.134+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/3328819289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95d0050 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.134+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/3328819289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.134+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3328819289 >> 172.21.15.46:0/3328819289 conn(0x562de95c1940 msgr2=0x562de95c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.135+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3328819289 shutdown_connections 2024-09-10T07:38:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.135+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3328819289 wait complete. 2024-09-10T07:38:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.136+0000 7f5e2c1fa740 1 -- start start 2024-09-10T07:38:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.136+0000 7f5e2c1fa740 1 Processor -- start 2024-09-10T07:38:05.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 0x562de95cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562de95e4170 con 0x562de95c5cd0 2024-09-10T07:38:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562de94e5e60 con 0x562de95d0590 2024-09-10T07:38:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562de95e42f0 con 0x562de95c53b0 2024-09-10T07:38:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e2c1fa740 10 client.0 _ll_register_callbacks cb 0x562de94eacc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e22000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e22000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60736/0 (socket says 172.21.15.46:60736) 2024-09-10T07:38:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e22000640 1 -- 172.21.15.46:0/2324066694 learned_addr learned my addr 172.21.15.46:0/2324066694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.137+0000 7f5e22000640 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 0x562de95cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.138+0000 7f5e22000640 1 -- 172.21.15.46:0/2324066694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 msgr2=0x562de95cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:05.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.138+0000 7f5e22a00640 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.139+0000 7f5e22000640 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 0x562de95cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.139+0000 7f5e22000640 1 -- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 msgr2=0x562de95cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:05.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.139+0000 7f5e22000640 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.139+0000 7f5e22000640 1 -- 172.21.15.46:0/2324066694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de94f1990 con 0x562de95c53b0 2024-09-10T07:38:05.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.139+0000 7f5e22a00640 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:38:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.140+0000 7f5e22000640 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95cd4d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5e10015e50 tx=0x7f5e100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d6704f6f7539d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.140+0000 7f5e0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:38:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.140+0000 7f5e0cc00640 1 -- 172.21.15.46:0/2324066694 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10013e10 con 0x562de95c53b0 2024-09-10T07:38:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.141+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/2324066694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e14005000 con 0x562de95c53b0 2024-09-10T07:38:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.141+0000 7f5e0cc00640 1 -- 172.21.15.46:0/2324066694 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5e1001e030 con 0x562de95c53b0 2024-09-10T07:38:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.142+0000 7f5e0cc00640 1 -- 172.21.15.46:0/2324066694 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5e10013e10 con 0x562de95c53b0 2024-09-10T07:38:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.142+0000 7f5e2c1fa740 10 client.25738 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.142+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/2324066694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562de95efe20 con 0x562de95c53b0 2024-09-10T07:38:05.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.142+0000 7f5e0cc00640 1 -- 172.21.15.46:0/2324066694 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e14005000 con 0x562de95c53b0 2024-09-10T07:38:05.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.144+0000 7f5e0cc00640 1 -- 172.21.15.46:0/2324066694 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f5e10004bc0 con 0x562de95c53b0 2024-09-10T07:38:05.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.144+0000 7f5e2c1fa740 10 client.25738 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.144+0000 7f5e2c1fa740 10 client.25738 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:38:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.144+0000 7f5e2c1fa740 20 client.25738 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.144+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562de95f1280 0x562de95f3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.144+0000 7f5e2c1fa740 4 client.25738 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.144+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/2324066694 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562de94ce310 con 0x562de95f1280 2024-09-10T07:38:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.145+0000 7f5e22a00640 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562de95f1280 0x562de95f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.146+0000 7f5e22a00640 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562de95f1280 0x562de95f3690 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14152a63a2cf8183 server_cookie=90f542b1e808bdb0 in_seq=0 out_seq=0 2024-09-10T07:38:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.146+0000 7f5e0cc00640 10 client.25738 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.147+0000 7f5e0cc00640 1 -- 172.21.15.46:0/2324066694 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562de94ce310 con 0x562de95f1280 2024-09-10T07:38:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.147+0000 7f5e0cc00640 10 client.25738 handle_command_reply: tid=0 2024-09-10T07:38:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.147+0000 7f5e2c1fa740 1 client.25738 shutdown 2024-09-10T07:38:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.148+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/2324066694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 msgr2=0x562de95cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.148+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95cd4d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5e10015e50 tx=0x7f5e100046c0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.148+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/2324066694 shutdown_connections 2024-09-10T07:38:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.148+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562de95f1280 0x562de95f3690 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.148+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95d0590 0x562de95cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.148+0000 7f5e0cc00640 0 client.25738 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.149+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.149+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/2324066694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95c53b0 0x562de95cd4d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.149+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/2324066694 >> 172.21.15.46:0/2324066694 conn(0x562de95c1940 msgr2=0x562de95c0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.150+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/2324066694 shutdown_connections 2024-09-10T07:38:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.150+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/2324066694 wait complete. 2024-09-10T07:38:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.150+0000 7f5e2c1fa740 20 client.25738 trim_cache size 0 max 16384 2024-09-10T07:38:05.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.182+0000 7f5e2c1fa740 1 Processor -- start 2024-09-10T07:38:05.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e2c1fa740 1 -- start start 2024-09-10T07:38:05.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c53b0 0x562de95c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 0x562de95f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241b5460 con 0x562de95c5cd0 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241b7060 con 0x562de95c53b0 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241b5b30 con 0x562de95d0590 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e22000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c53b0 0x562de95c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e22000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c53b0 0x562de95c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38828/0 (socket says 172.21.15.46:38828) 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.183+0000 7f5e22000640 1 -- 172.21.15.46:0/3316017954 learned_addr learned my addr 172.21.15.46:0/3316017954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.184+0000 7f5e22000640 1 --2- 172.21.15.46:0/3316017954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 0x562de95f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.184+0000 7f5e22000640 1 -- 172.21.15.46:0/3316017954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 msgr2=0x562de95f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.184+0000 7f5e22000640 1 --2- 172.21.15.46:0/3316017954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 0x562de95f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.184+0000 7f5e22000640 1 -- 172.21.15.46:0/3316017954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 msgr2=0x562de95cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.184+0000 7f5e22000640 1 --2- 172.21.15.46:0/3316017954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.184+0000 7f5e22000640 1 -- 172.21.15.46:0/3316017954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de95c1770 con 0x562de95c53b0 2024-09-10T07:38:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.185+0000 7f5e22000640 1 --2- 172.21.15.46:0/3316017954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c53b0 0x562de95c7e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5e100224e0 tx=0x7f5e10000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b4d07722f4e421 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.185+0000 7f5e0f400640 1 -- 172.21.15.46:0/3316017954 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10015030 con 0x562de95c53b0 2024-09-10T07:38:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.185+0000 7f5e0f400640 1 -- 172.21.15.46:0/3316017954 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e10022dc0 con 0x562de95c53b0 2024-09-10T07:38:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.186+0000 7f5e0f400640 1 -- 172.21.15.46:0/3316017954 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10015030 con 0x562de95c53b0 2024-09-10T07:38:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.187+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3316017954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c53b0 msgr2=0x562de95c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.187+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/3316017954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c53b0 0x562de95c7e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5e100224e0 tx=0x7f5e10000bc0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.187+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3316017954 shutdown_connections 2024-09-10T07:38:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.187+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/3316017954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 0x562de95f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.187+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/3316017954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c5cd0 0x562de95cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.187+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/3316017954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c53b0 0x562de95c7e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.187+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3316017954 >> 172.21.15.46:0/3316017954 conn(0x562de95c1940 msgr2=0x562de95cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.188+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3316017954 shutdown_connections 2024-09-10T07:38:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.188+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/3316017954 wait complete. 2024-09-10T07:38:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.189+0000 7f5e2c1fa740 1 -- start start 2024-09-10T07:38:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.190+0000 7f5e2c1fa740 1 Processor -- start 2024-09-10T07:38:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.192+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c53b0 0x562de95e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.192+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c5cd0 0x562de95e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.193+0000 7f5e2c1fa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 0x562de95e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.193+0000 7f5e22a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c5cd0 0x562de95e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.193+0000 7f5e22a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c5cd0 0x562de95e5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38830/0 (socket says 172.21.15.46:38830) 2024-09-10T07:38:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.193+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241b68d0 con 0x562de95c53b0 2024-09-10T07:38:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.193+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e241b6200 con 0x562de95c5cd0 2024-09-10T07:38:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.193+0000 7f5e2c1fa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e24076b40 con 0x562de95d0590 2024-09-10T07:38:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.193+0000 7f5e2c1fa740 10 client.0 _ll_register_callbacks cb 0x7f5e10015640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.194+0000 7f5e22a00640 1 -- 172.21.15.46:0/483240358 learned_addr learned my addr 172.21.15.46:0/483240358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.195+0000 7f5e22a00640 1 -- 172.21.15.46:0/483240358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 msgr2=0x562de95e60c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:38:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.195+0000 7f5e22a00640 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 0x562de95e60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.195+0000 7f5e22a00640 1 -- 172.21.15.46:0/483240358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c53b0 msgr2=0x562de95e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.195+0000 7f5e22a00640 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c53b0 0x562de95e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.195+0000 7f5e22a00640 1 -- 172.21.15.46:0/483240358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de965b260 con 0x562de95c5cd0 2024-09-10T07:38:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.196+0000 7f5e22a00640 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c5cd0 0x562de95e5b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5e1402a930 tx=0x7f5e14004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295ffab6130a4859 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.196+0000 7f5e0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:38:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.197+0000 7f5e0cc00640 1 -- 172.21.15.46:0/483240358 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14041040 con 0x562de95c5cd0 2024-09-10T07:38:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.198+0000 7f5e0cc00640 1 -- 172.21.15.46:0/483240358 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5e140051e0 con 0x562de95c5cd0 2024-09-10T07:38:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.198+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/483240358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e10022dc0 con 0x562de95c5cd0 2024-09-10T07:38:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.199+0000 7f5e0cc00640 1 -- 172.21.15.46:0/483240358 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5e140046c0 con 0x562de95c5cd0 2024-09-10T07:38:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.199+0000 7f5e2c1fa740 10 client.25742 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.199+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/483240358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562de965c540 con 0x562de95c5cd0 2024-09-10T07:38:05.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.201+0000 7f5e0cc00640 1 -- 172.21.15.46:0/483240358 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e10022dc0 con 0x562de95c5cd0 2024-09-10T07:38:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.204+0000 7f5e0cc00640 1 -- 172.21.15.46:0/483240358 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f5e140053e0 con 0x562de95c5cd0 2024-09-10T07:38:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.204+0000 7f5e2c1fa740 10 client.25742 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.204+0000 7f5e2c1fa740 10 client.25742 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:38:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.204+0000 7f5e2c1fa740 20 client.25742 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.204+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562de965c940 0x562de965ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.204+0000 7f5e2c1fa740 4 client.25742 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T07:38:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.204+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/483240358 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x562de9538c60 con 0x562de965c940 2024-09-10T07:38:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.205+0000 7f5e22000640 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562de965c940 0x562de965ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.207+0000 7f5e22000640 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562de965c940 0x562de965ed30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18884295d1765c2d server_cookie=7733b7cdec034e10 in_seq=0 out_seq=0 2024-09-10T07:38:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.207+0000 7f5e0cc00640 10 client.25742 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:05.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e0cc00640 1 -- 172.21.15.46:0/483240358 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562de9538c60 con 0x562de965c940 2024-09-10T07:38:05.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e0cc00640 10 client.25742 handle_command_reply: tid=0 2024-09-10T07:38:05.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 client.25742 shutdown 2024-09-10T07:38:05.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/483240358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c5cd0 msgr2=0x562de95e5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c5cd0 0x562de95e5b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5e1402a930 tx=0x7f5e14004ad0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/483240358 shutdown_connections 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562de965c940 0x562de965ed30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562de95d0590 0x562de95e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562de95c5cd0 0x562de95e5b40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 --2- 172.21.15.46:0/483240358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de95c53b0 0x562de95e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/483240358 >> 172.21.15.46:0/483240358 conn(0x562de95c1940 msgr2=0x562de95cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.360+0000 7f5e0cc00640 0 client.25742 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.361+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/483240358 shutdown_connections 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.361+0000 7f5e2c1fa740 1 -- 172.21.15.46:0/483240358 wait complete. 2024-09-10T07:38:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.361+0000 7f5e2c1fa740 20 client.25742 trim_cache size 0 max 16384 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 34, 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 6, 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "37a59a3d": { 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 6, 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:38:05.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 150.0, 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:05.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:05.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:05.364 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:38:05.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e080a2c00 msgr2=0x7f5e080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e080a2c00 0x7f5e080a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00eff0 tx=0x7f5e1c005f40 comp rx=0 tx=0).stop 2024-09-10T07:38:05.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e24112a30 msgr2=0x7f5e2410b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:05.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e24112a30 0x7f5e2410b210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00eb10 tx=0x7f5e1c00c2d0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 shutdown_connections 2024-09-10T07:38:05.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e24112a30 0x7f5e2410b210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e24077040 0x7f5e24112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e080a2c00 0x7f5e080a50c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 --2- 172.21.15.46:0/2072199752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e24076720 0x7f5e24111f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:05.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 >> 172.21.15.46:0/2072199752 conn(0x7f5e240ffab0 msgr2=0x7f5e24101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:05.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 shutdown_connections 2024-09-10T07:38:05.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:05.362+0000 7f5e29800640 1 -- 172.21.15.46:0/2072199752 wait complete. 2024-09-10T07:38:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 45.0 s, 2018.4 tps, lat 45.850 ms stddev 52.677 2024-09-10T07:38:06.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:06 smithi046 ceph-mon[30705]: pgmap v443: 129 pgs: 129 active+clean; 923 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-09-10T07:38:06.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:06 smithi181 ceph-mon[35397]: pgmap v443: 129 pgs: 129 active+clean; 923 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-09-10T07:38:06.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:06 smithi063 ceph-mon[35020]: pgmap v443: 129 pgs: 129 active+clean; 923 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-09-10T07:38:07.578 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '37a59a3d', quiesce took 6.6 seconds. Will release after: 23.4 2024-09-10T07:38:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:08 smithi181 ceph-mon[35397]: pgmap v444: 129 pgs: 129 active+clean; 933 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-09-10T07:38:08.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:08 smithi063 ceph-mon[35020]: pgmap v444: 129 pgs: 129 active+clean; 933 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-09-10T07:38:08.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:08 smithi046 ceph-mon[30705]: pgmap v444: 129 pgs: 129 active+clean; 933 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-09-10T07:38:10.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 50.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:38:10.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:10 smithi181 ceph-mon[35397]: pgmap v445: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.2 MiB/s wr, 306 op/s 2024-09-10T07:38:10.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:10 smithi063 ceph-mon[35020]: pgmap v445: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.2 MiB/s wr, 306 op/s 2024-09-10T07:38:10.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:10 smithi046 ceph-mon[30705]: pgmap v445: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.2 MiB/s wr, 306 op/s 2024-09-10T07:38:10.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:38:11.440 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.096+0000 7fa1d6200640 1 -- 172.21.15.46:0/1919027846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1d0077040 msgr2=0x7fa1d01124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.096+0000 7fa1d6200640 1 --2- 172.21.15.46:0/1919027846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1d0077040 0x7fa1d01124f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009900 tx=0x7fa1bc02a410 comp rx=0 tx=0).stop 2024-09-10T07:38:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.096+0000 7fa1d6200640 1 -- 172.21.15.46:0/1919027846 shutdown_connections 2024-09-10T07:38:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.096+0000 7fa1d6200640 1 --2- 172.21.15.46:0/1919027846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0112a30 0x7fa1d0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.096+0000 7fa1d6200640 1 --2- 172.21.15.46:0/1919027846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1d0077040 0x7fa1d01124f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.097+0000 7fa1d6200640 1 --2- 172.21.15.46:0/1919027846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d0076720 0x7fa1d0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.097+0000 7fa1d6200640 1 -- 172.21.15.46:0/1919027846 >> 172.21.15.46:0/1919027846 conn(0x7fa1d00ffa90 msgr2=0x7fa1d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.097+0000 7fa1d6200640 1 -- 172.21.15.46:0/1919027846 shutdown_connections 2024-09-10T07:38:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.097+0000 7fa1d6200640 1 -- 172.21.15.46:0/1919027846 wait complete. 2024-09-10T07:38:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1d6200640 1 Processor -- start 2024-09-10T07:38:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1d6200640 1 -- start start 2024-09-10T07:38:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1d6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0076720 0x7fa1d01a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1d6200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d0077040 0x7fa1d01a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1d6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1d0112a30 0x7fa1d019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1d6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0118600 con 0x7fa1d0112a30 2024-09-10T07:38:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1d6200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0118480 con 0x7fa1d0077040 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1d6200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0118780 con 0x7fa1d0076720 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1cf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0076720 0x7fa1d01a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1d0112a30 0x7fa1d019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1cf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0076720 0x7fa1d01a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46826/0 (socket says 172.21.15.46:46826) 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.098+0000 7fa1cf400640 1 -- 172.21.15.46:0/2371834772 learned_addr learned my addr 172.21.15.46:0/2371834772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.099+0000 7fa1cea00640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d0077040 0x7fa1d01a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.099+0000 7fa1cf400640 1 -- 172.21.15.46:0/2371834772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d0077040 msgr2=0x7fa1d01a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.099+0000 7fa1cf400640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d0077040 0x7fa1d01a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.099+0000 7fa1cf400640 1 -- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1d0112a30 msgr2=0x7fa1d019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.099+0000 7fa1cf400640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1d0112a30 0x7fa1d019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.099+0000 7fa1cf400640 1 -- 172.21.15.46:0/2371834772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d019eff0 con 0x7fa1d0076720 2024-09-10T07:38:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.099+0000 7fa1cf400640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0076720 0x7fa1d01a4700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9f0 tx=0x7fa1c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.100+0000 7fa1c7e00640 1 -- 172.21.15.46:0/2371834772 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000cd80 con 0x7fa1d0076720 2024-09-10T07:38:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.100+0000 7fa1c7e00640 1 -- 172.21.15.46:0/2371834772 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c0010910 con 0x7fa1d0076720 2024-09-10T07:38:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.100+0000 7fa1c7e00640 1 -- 172.21.15.46:0/2371834772 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0018990 con 0x7fa1d0076720 2024-09-10T07:38:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.100+0000 7fa1d6200640 1 -- 172.21.15.46:0/2371834772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d019f2e0 con 0x7fa1d0076720 2024-09-10T07:38:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.100+0000 7fa1d6200640 1 -- 172.21.15.46:0/2371834772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01ab5d0 con 0x7fa1d0076720 2024-09-10T07:38:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.103+0000 7fa1d8e4f740 1 Processor -- start 2024-09-10T07:38:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.104+0000 7fa1d8e4f740 1 -- start start 2024-09-10T07:38:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.104+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa683f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.104+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.104+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa691160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.104+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556baa5ea3e0 con 0x556baa68ed70 2024-09-10T07:38:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.104+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556baa591d80 con 0x556baa6844b0 2024-09-10T07:38:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.104+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556baa5e63a0 con 0x556baa683b90 2024-09-10T07:38:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1c7e00640 1 -- 172.21.15.46:0/2371834772 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa1c0010ac0 con 0x7fa1d0076720 2024-09-10T07:38:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa683f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa691160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa683f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46836/0 (socket says 172.21.15.46:46836) 2024-09-10T07:38:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 -- 172.21.15.46:0/3580440488 learned_addr learned my addr 172.21.15.46:0/3580440488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cf400640 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 -- 172.21.15.46:0/3580440488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 msgr2=0x556baa68e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 -- 172.21.15.46:0/3580440488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 msgr2=0x556baa691160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa691160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 -- 172.21.15.46:0/3580440488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556baa5b0ae0 con 0x556baa683b90 2024-09-10T07:38:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cea00640 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa691160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.105+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa683f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009f30 tx=0x7fa1c80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39227a4228881ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.106+0000 7fa1bbe00640 1 -- 172.21.15.46:0/3580440488 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c801e070 con 0x556baa683b90 2024-09-10T07:38:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.106+0000 7fa1bbe00640 1 -- 172.21.15.46:0/3580440488 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c8003c30 con 0x556baa683b90 2024-09-10T07:38:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.106+0000 7fa1bbe00640 1 -- 172.21.15.46:0/3580440488 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800f8e0 con 0x556baa683b90 2024-09-10T07:38:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.106+0000 7fa1c7e00640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1b40a2b80 0x7fa1b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.106+0000 7fa1c7e00640 1 -- 172.21.15.46:0/2371834772 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa1c00cf730 con 0x7fa1d0076720 2024-09-10T07:38:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.106+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3580440488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 msgr2=0x556baa683f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.106+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa683f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009f30 tx=0x7fa1c80118d0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1cf400640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1b40a2b80 0x7fa1b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3580440488 shutdown_connections 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa691160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/3580440488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa683f70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3580440488 >> 172.21.15.46:0/3580440488 conn(0x556baa680120 msgr2=0x556baa680500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3580440488 shutdown_connections 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3580440488 wait complete. 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.107+0000 7fa1d8e4f740 1 -- start start 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1d8e4f740 1 Processor -- start 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1cf400640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1b40a2b80 0x7fa1b40a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa1c009bbf0 tx=0x7fa1c0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa68bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa68c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556baa6919e0 con 0x556baa68ed70 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556baa5a50f0 con 0x556baa6844b0 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556baa691b60 con 0x556baa683b90 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1d8e4f740 10 client.0 _ll_register_callbacks cb 0x556baa5969e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa68bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50504/0 (socket says 172.21.15.46:50504) 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.108+0000 7fa1cea00640 1 -- 172.21.15.46:0/2103092541 learned_addr learned my addr 172.21.15.46:0/2103092541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa68c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1cea00640 1 -- 172.21.15.46:0/2103092541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 msgr2=0x556baa68bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1cea00640 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa68bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1cea00640 1 -- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 msgr2=0x556baa68c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1cea00640 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa68c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1cea00640 1 -- 172.21.15.46:0/2103092541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556baa35b3c0 con 0x556baa6844b0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa68c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1cea00640 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68c220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009900 tx=0x7fa1bc0042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a96b67a2efa738 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2103092541 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc035040 con 0x556baa6844b0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2103092541 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa1bc005450 con 0x556baa6844b0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.109+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2103092541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c8003c30 con 0x556baa6844b0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.110+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2103092541 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa1bc005650 con 0x556baa6844b0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.110+0000 7fa1d8e4f740 10 client.25754 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.110+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2103092541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556baa69d710 con 0x556baa6844b0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2103092541 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c8003c30 con 0x556baa6844b0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2103092541 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7fa1bc0318e0 con 0x556baa6844b0 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1d8e4f740 10 client.25754 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1d8e4f740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1d8e4f740 20 client.25754 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556baa69ec70 0x556baa6a1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1d8e4f740 4 client.25754 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2103092541 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556baa5f7950 con 0x556baa69ec70 2024-09-10T07:38:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.111+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556baa69ec70 0x556baa6a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.112+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556baa69ec70 0x556baa6a1060 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2499bbf341fe9d server_cookie=fa743c7a608ffdf3 in_seq=0 out_seq=0 2024-09-10T07:38:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.112+0000 7fa1b8c00640 10 client.25754 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2103092541 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556baa5f7950 con 0x556baa69ec70 2024-09-10T07:38:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1b8c00640 10 client.25754 handle_command_reply: tid=0 2024-09-10T07:38:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 client.25754 shutdown 2024-09-10T07:38:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2103092541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 msgr2=0x556baa68c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68c220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009900 tx=0x7fa1bc0042e0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2103092541 shutdown_connections 2024-09-10T07:38:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556baa69ec70 0x556baa6a1060 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa68ed70 0x556baa68c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa6844b0 0x556baa68c220 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2103092541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa683b90 0x556baa68bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1b8c00640 0 client.25754 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2103092541 >> 172.21.15.46:0/2103092541 conn(0x556baa680120 msgr2=0x556baa685e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2103092541 shutdown_connections 2024-09-10T07:38:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2103092541 wait complete. 2024-09-10T07:38:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.113+0000 7fa1d8e4f740 20 client.25754 trim_cache size 0 max 16384 2024-09-10T07:38:12.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.131+0000 7fa1d8e4f740 1 Processor -- start 2024-09-10T07:38:12.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.131+0000 7fa1d8e4f740 1 -- start start 2024-09-10T07:38:12.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa683b90 0x556baa6a6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa6844b0 0x556baa6a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa68ed70 0x556baa6855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b51e0 con 0x556baa683b90 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b6de0 con 0x556baa68ed70 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b58b0 con 0x556baa6844b0 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa683b90 0x556baa6a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa683b90 0x556baa6a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56838/0 (socket says 172.21.15.46:56838) 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 -- 172.21.15.46:0/3777507223 learned_addr learned my addr 172.21.15.46:0/3777507223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cea00640 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa6844b0 0x556baa6a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa68ed70 0x556baa6855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 -- 172.21.15.46:0/3777507223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa6844b0 msgr2=0x556baa6a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa6844b0 0x556baa6a6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 -- 172.21.15.46:0/3777507223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa68ed70 msgr2=0x556baa6855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa68ed70 0x556baa6855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.132+0000 7fa1cfe00640 1 -- 172.21.15.46:0/3777507223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556baa6812b0 con 0x556baa683b90 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.133+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa683b90 0x556baa6a6930 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800f4f0 tx=0x7fa1c800f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4719d7c5c6e3bf1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.133+0000 7fa1bb400640 1 -- 172.21.15.46:0/3777507223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c801e070 con 0x556baa683b90 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.133+0000 7fa1bb400640 1 -- 172.21.15.46:0/3777507223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c801f040 con 0x556baa683b90 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.133+0000 7fa1bb400640 1 -- 172.21.15.46:0/3777507223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8033410 con 0x556baa683b90 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3777507223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa683b90 msgr2=0x556baa6a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa683b90 0x556baa6a6930 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800f4f0 tx=0x7fa1c800f480 comp rx=0 tx=0).stop 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3777507223 shutdown_connections 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa68ed70 0x556baa6855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa6844b0 0x556baa6a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/3777507223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa683b90 0x556baa6a6930 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3777507223 >> 172.21.15.46:0/3777507223 conn(0x556baa680120 msgr2=0x556baa68add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3777507223 shutdown_connections 2024-09-10T07:38:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/3777507223 wait complete. 2024-09-10T07:38:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.134+0000 7fa1d8e4f740 1 -- start start 2024-09-10T07:38:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1d8e4f740 1 Processor -- start 2024-09-10T07:38:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa683b90 0x556baa697510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa6844b0 0x556baa692890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1d8e4f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa68ed70 0x556baa692dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b6650 con 0x556baa6844b0 2024-09-10T07:38:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b5f80 con 0x556baa683b90 2024-09-10T07:38:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1d8e4f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b31c0 con 0x556baa68ed70 2024-09-10T07:38:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1d8e4f740 10 client.0 _ll_register_callbacks cb 0x7fa1bc005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa6844b0 0x556baa692890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa683b90 0x556baa697510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa683b90 0x556baa697510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50534/0 (socket says 172.21.15.46:50534) 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa6844b0 0x556baa692890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56850/0 (socket says 172.21.15.46:56850) 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.135+0000 7fa1cfe00640 1 -- 172.21.15.46:0/2184681691 learned_addr learned my addr 172.21.15.46:0/2184681691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa68ed70 0x556baa692dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1cea00640 1 -- 172.21.15.46:0/2184681691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa68ed70 msgr2=0x556baa692dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1cea00640 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa68ed70 0x556baa692dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1cea00640 1 -- 172.21.15.46:0/2184681691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa683b90 msgr2=0x556baa697510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1cea00640 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa683b90 0x556baa697510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1cea00640 1 -- 172.21.15.46:0/2184681691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556baa6935b0 con 0x556baa6844b0 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa68ed70 0x556baa692dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:38:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1cea00640 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa6844b0 0x556baa692890 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc031d30 tx=0x7fa1bc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4ea2e9421a08aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.136+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2184681691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc035050 con 0x556baa6844b0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2184681691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c801f040 con 0x556baa6844b0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2184681691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa1bc0319c0 con 0x556baa6844b0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2184681691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa1bc060050 con 0x556baa6844b0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 10 client.15663 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2184681691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556baa709920 con 0x556baa6844b0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2184681691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c801f040 con 0x556baa6844b0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2184681691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7fa1bc05b030 con 0x556baa6844b0 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 10 client.15663 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 20 client.15663 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556baa70a090 0x556baa70c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 4 client.15663 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:38:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.137+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2184681691 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556baa5f1e60 con 0x556baa70a090 2024-09-10T07:38:12.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.138+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556baa70a090 0x556baa70c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.138+0000 7fa1cfe00640 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556baa70a090 0x556baa70c4b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e2eeccb7cbd5868 server_cookie=61ebda417ede481f in_seq=0 out_seq=0 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.138+0000 7fa1b8c00640 10 client.15663 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.138+0000 7fa1b8c00640 1 -- 172.21.15.46:0/2184681691 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556baa5f1e60 con 0x556baa70a090 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.138+0000 7fa1b8c00640 10 client.15663 handle_command_reply: tid=0 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.138+0000 7fa1d8e4f740 1 client.15663 shutdown 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2184681691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa6844b0 msgr2=0x556baa692890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa6844b0 0x556baa692890 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc031d30 tx=0x7fa1bc003820 comp rx=0 tx=0).stop 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2184681691 shutdown_connections 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556baa70a090 0x556baa70c4b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556baa68ed70 0x556baa692dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556baa6844b0 0x556baa692890 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 --2- 172.21.15.46:0/2184681691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556baa683b90 0x556baa697510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2184681691 >> 172.21.15.46:0/2184681691 conn(0x556baa680120 msgr2=0x556baa6855e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1b8c00640 0 client.15663 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2184681691 shutdown_connections 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 1 -- 172.21.15.46:0/2184681691 wait complete. 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.139+0000 7fa1d8e4f740 20 client.15663 trim_cache size 0 max 16384 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:38:12.142 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:38:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 -- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1b40a2b80 msgr2=0x7fa1b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1b40a2b80 0x7fa1b40a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa1c009bbf0 tx=0x7fa1c0000bc0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 -- 172.21.15.46:0/2371834772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0076720 msgr2=0x7fa1d01a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0076720 0x7fa1d01a4700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9f0 tx=0x7fa1c000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 -- 172.21.15.46:0/2371834772 shutdown_connections 2024-09-10T07:38:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1b40a2b80 0x7fa1b40a5040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1d0112a30 0x7fa1d019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d0077040 0x7fa1d01a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 --2- 172.21.15.46:0/2371834772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d0076720 0x7fa1d01a4700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 -- 172.21.15.46:0/2371834772 >> 172.21.15.46:0/2371834772 conn(0x7fa1d00ffa90 msgr2=0x7fa1d0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 -- 172.21.15.46:0/2371834772 shutdown_connections 2024-09-10T07:38:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:12.140+0000 7fa1d6200640 1 -- 172.21.15.46:0/2371834772 wait complete. 2024-09-10T07:38:12.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:12 smithi181 ceph-mon[35397]: pgmap v446: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-09-10T07:38:12.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:12 smithi063 ceph-mon[35020]: pgmap v446: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-09-10T07:38:12.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:12 smithi046 ceph-mon[30705]: pgmap v446: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-09-10T07:38:13.688 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:38:13.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 -- 172.21.15.46:0/2091596688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a98104f50 msgr2=0x7f0a981053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 --2- 172.21.15.46:0/2091596688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a98104f50 0x7f0a981053d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009860 tx=0x7f0a8402a340 comp rx=0 tx=0).stop 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 -- 172.21.15.46:0/2091596688 shutdown_connections 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 --2- 172.21.15.46:0/2091596688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98105910 0x7f0a9810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 --2- 172.21.15.46:0/2091596688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a98104f50 0x7f0a981053d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 --2- 172.21.15.46:0/2091596688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a98103d60 0x7f0a98104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 -- 172.21.15.46:0/2091596688 >> 172.21.15.46:0/2091596688 conn(0x7f0a980ff4e0 msgr2=0x7f0a98101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 -- 172.21.15.46:0/2091596688 shutdown_connections 2024-09-10T07:38:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.975+0000 7f0aa0000640 1 -- 172.21.15.46:0/2091596688 wait complete. 2024-09-10T07:38:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.976+0000 7f0aa0000640 1 Processor -- start 2024-09-10T07:38:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0aa0000640 1 -- start start 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0aa0000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98103d60 0x7f0a98199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0aa0000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a98104f50 0x7f0a9819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0aa0000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a98105910 0x7f0a981a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0aa0000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810f9c0 con 0x7f0a98103d60 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0aa0000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810f840 con 0x7f0a98105910 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0aa0000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810fb40 con 0x7f0a98104f50 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0a9cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a98104f50 0x7f0a9819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0a9cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a98104f50 0x7f0a9819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46874/0 (socket says 172.21.15.46:46874) 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0a9d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98103d60 0x7f0a98199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0a9cc00640 1 -- 172.21.15.46:0/546951796 learned_addr learned my addr 172.21.15.46:0/546951796 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.977+0000 7f0a9e000640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a98105910 0x7f0a981a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a9cc00640 1 -- 172.21.15.46:0/546951796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a98105910 msgr2=0x7f0a981a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a98105910 0x7f0a981a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a9cc00640 1 -- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98103d60 msgr2=0x7f0a98199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98103d60 0x7f0a98199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a9cc00640 1 -- 172.21.15.46:0/546951796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a981a1bd0 con 0x7f0a98104f50 2024-09-10T07:38:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a98104f50 0x7f0a9819a4f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a84035040 tx=0x7f0a84033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a96000640 1 -- 172.21.15.46:0/546951796 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84037040 con 0x7f0a98104f50 2024-09-10T07:38:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a96000640 1 -- 172.21.15.46:0/546951796 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a84030850 con 0x7f0a98104f50 2024-09-10T07:38:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0aa0000640 1 -- 172.21.15.46:0/546951796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a981a1e60 con 0x7f0a98104f50 2024-09-10T07:38:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.978+0000 7f0a96000640 1 -- 172.21.15.46:0/546951796 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84004920 con 0x7f0a98104f50 2024-09-10T07:38:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.979+0000 7f0aa0000640 1 -- 172.21.15.46:0/546951796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a981a22d0 con 0x7f0a98104f50 2024-09-10T07:38:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0aa3054740 1 Processor -- start 2024-09-10T07:38:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0aa3054740 1 -- start start 2024-09-10T07:38:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0a96000640 1 -- 172.21.15.46:0/546951796 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a84004050 con 0x7f0a98104f50 2024-09-10T07:38:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93f3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad93f8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f8a60 0x55cad93faf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0aa3054740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cad9358100 con 0x55cad93f8a60 2024-09-10T07:38:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0aa3054740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cad93001b0 con 0x55cad93f39d0 2024-09-10T07:38:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0aa3054740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cad93540c0 con 0x55cad93f3090 2024-09-10T07:38:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0a9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f8a60 0x55cad93faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.982+0000 7f0a9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f8a60 0x55cad93faf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56876/0 (socket says 172.21.15.46:56876) 2024-09-10T07:38:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9cc00640 1 -- 172.21.15.46:0/1907429854 learned_addr learned my addr 172.21.15.46:0/1907429854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9d600640 1 --2- 172.21.15.46:0/1907429854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad93f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9e000640 1 -- 172.21.15.46:0/1907429854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 msgr2=0x55cad93f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9e000640 1 --2- 172.21.15.46:0/1907429854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad93f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9e000640 1 -- 172.21.15.46:0/1907429854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f8a60 msgr2=0x55cad93faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9e000640 1 --2- 172.21.15.46:0/1907429854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f8a60 0x55cad93faf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9e000640 1 -- 172.21.15.46:0/1907429854 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad906b210 con 0x55cad93f3090 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.983+0000 7f0a9e000640 1 --2- 172.21.15.46:0/1907429854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93f3490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a900099b0 tx=0x7f0a900095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e8af4e54f7e9899 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.984+0000 7f0a8a600640 1 -- 172.21.15.46:0/1907429854 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90003970 con 0x55cad93f3090 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.984+0000 7f0a96000640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a800a2cb0 0x7f0a800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.984+0000 7f0a8a600640 1 -- 172.21.15.46:0/1907429854 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9000d6b0 con 0x55cad93f3090 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.984+0000 7f0a8a600640 1 -- 172.21.15.46:0/1907429854 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90004280 con 0x55cad93f3090 2024-09-10T07:38:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.984+0000 7f0a96000640 1 -- 172.21.15.46:0/546951796 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0a840ef280 con 0x7f0a98104f50 2024-09-10T07:38:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.984+0000 7f0a9d600640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a800a2cb0 0x7f0a800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 -- 172.21.15.46:0/1907429854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 msgr2=0x55cad93f3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1907429854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93f3490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a900099b0 tx=0x7f0a900095c0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 -- 172.21.15.46:0/1907429854 shutdown_connections 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1907429854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f8a60 0x55cad93faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1907429854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad93f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1907429854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93f3490 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 -- 172.21.15.46:0/1907429854 >> 172.21.15.46:0/1907429854 conn(0x55cad93ee700 msgr2=0x55cad93eeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0a9d600640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a800a2cb0 0x7f0a800a5170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55cad93f4400 tx=0x7f0a8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 -- 172.21.15.46:0/1907429854 shutdown_connections 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 -- 172.21.15.46:0/1907429854 wait complete. 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 -- start start 2024-09-10T07:38:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.985+0000 7f0aa3054740 1 Processor -- start 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93fd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad93fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f8a60 0x55cad9404790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0aa3054740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cad93fba30 con 0x55cad93f39d0 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0aa3054740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cad92e1090 con 0x55cad93f8a60 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0aa3054740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cad93fbbb0 con 0x55cad93f3090 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0aa3054740 10 client.0 _ll_register_callbacks cb 0x55cad9335f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0a9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad93fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0a9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad93fd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56884/0 (socket says 172.21.15.46:56884) 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0a9d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0a9cc00640 1 -- 172.21.15.46:0/1755827036 learned_addr learned my addr 172.21.15.46:0/1755827036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.986+0000 7f0a9e000640 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f8a60 0x55cad9404790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a9cc00640 1 -- 172.21.15.46:0/1755827036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 msgr2=0x55cad93fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a9cc00640 1 -- 172.21.15.46:0/1755827036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f8a60 msgr2=0x55cad9404790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f8a60 0x55cad9404790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a9cc00640 1 -- 172.21.15.46:0/1755827036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad9137620 con 0x55cad93f39d0 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a9d600640 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a9e000640 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f8a60 0x55cad9404790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad93fd750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009250 tx=0x7f0a840b73a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d7c125103624d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1755827036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840ece60 con 0x55cad93f39d0 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1755827036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0a84004340 con 0x55cad93f39d0 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0aa3054740 1 -- 172.21.15.46:0/1755827036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a9000d6b0 con 0x55cad93f39d0 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1755827036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0a840bb4d0 con 0x55cad93f39d0 2024-09-10T07:38:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0aa3054740 10 client.15675 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.987+0000 7f0aa3054740 1 -- 172.21.15.46:0/1755827036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cad9407580 con 0x55cad93f39d0 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1755827036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9000d6b0 con 0x55cad93f39d0 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1755827036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f0a8403a070 con 0x55cad93f39d0 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0aa3054740 10 client.15675 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0aa3054740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0aa3054740 20 client.15675 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cad9408d10 0x55cad940b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0aa3054740 4 client.15675 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0aa3054740 1 -- 172.21.15.46:0/1755827036 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cad9365830 con 0x55cad9408d10 2024-09-10T07:38:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.988+0000 7f0a9d600640 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cad9408d10 0x55cad940b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.989+0000 7f0a9d600640 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cad9408d10 0x55cad940b150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fed6c7239215998 server_cookie=882b1abfeb2852b8 in_seq=0 out_seq=0 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.989+0000 7f0a7ea00640 10 client.15675 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.989+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1755827036 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cad9365830 con 0x55cad9408d10 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0a7ea00640 10 client.15675 handle_command_reply: tid=0 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 client.15675 shutdown 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 -- 172.21.15.46:0/1755827036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 msgr2=0x55cad93fd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad93fd750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009250 tx=0x7f0a840b73a0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 -- 172.21.15.46:0/1755827036 shutdown_connections 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cad9408d10 0x55cad940b150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f8a60 0x55cad9404790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad93fd750 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0a7ea00640 0 client.15675 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1755827036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f3090 0x55cad93fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 -- 172.21.15.46:0/1755827036 >> 172.21.15.46:0/1755827036 conn(0x55cad93ee700 msgr2=0x55cad93eff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 -- 172.21.15.46:0/1755827036 shutdown_connections 2024-09-10T07:38:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 1 -- 172.21.15.46:0/1755827036 wait complete. 2024-09-10T07:38:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:13.990+0000 7f0aa3054740 20 client.15675 trim_cache size 0 max 16384 2024-09-10T07:38:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.010+0000 7f0aa3054740 1 Processor -- start 2024-09-10T07:38:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.010+0000 7f0aa3054740 1 -- start start 2024-09-10T07:38:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.010+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f3090 0x55cad93e98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.010+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad9404750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 0x55cad93fc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0aa3054740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981ac840 con 0x55cad93f39d0 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0aa3054740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981ae440 con 0x55cad93f3090 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0aa3054740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981acf10 con 0x55cad93f8a60 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad9404750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad9404750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56892/0 (socket says 172.21.15.46:56892) 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9cc00640 1 -- 172.21.15.46:0/3745897740 learned_addr learned my addr 172.21.15.46:0/3745897740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9d600640 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f3090 0x55cad93e98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9e000640 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 0x55cad93fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9cc00640 1 -- 172.21.15.46:0/3745897740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 msgr2=0x55cad93fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 0x55cad93fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9cc00640 1 -- 172.21.15.46:0/3745897740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f3090 msgr2=0x55cad93e98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f3090 0x55cad93e98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.011+0000 7f0a9cc00640 1 -- 172.21.15.46:0/3745897740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad93f4660 con 0x55cad93f39d0 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.012+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad9404750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a840bb690 tx=0x7f0a84005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fed6645f5f72db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.012+0000 7f0a89c00640 1 -- 172.21.15.46:0/3745897740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840ece60 con 0x55cad93f39d0 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.012+0000 7f0a89c00640 1 -- 172.21.15.46:0/3745897740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a84005190 con 0x55cad93f39d0 2024-09-10T07:38:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.012+0000 7f0a89c00640 1 -- 172.21.15.46:0/3745897740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840bb060 con 0x55cad93f39d0 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 -- 172.21.15.46:0/3745897740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 msgr2=0x55cad9404750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad9404750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a840bb690 tx=0x7f0a84005ab0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 -- 172.21.15.46:0/3745897740 shutdown_connections 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 0x55cad93fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f39d0 0x55cad9404750 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 --2- 172.21.15.46:0/3745897740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f3090 0x55cad93e98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 -- 172.21.15.46:0/3745897740 >> 172.21.15.46:0/3745897740 conn(0x55cad93ee700 msgr2=0x55cad93fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 -- 172.21.15.46:0/3745897740 shutdown_connections 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.013+0000 7f0aa3054740 1 -- 172.21.15.46:0/3745897740 wait complete. 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 1 -- start start 2024-09-10T07:38:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 1 Processor -- start 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f3090 0x55cad94013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad9401930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 0x55cad94719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981adcb0 con 0x55cad93f3090 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981ad5e0 con 0x55cad93f39d0 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981aa820 con 0x55cad93f8a60 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0aa3054740 10 client.0 _ll_register_callbacks cb 0x7f0a84009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.014+0000 7f0a9d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f3090 0x55cad94013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f3090 0x55cad94013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56894/0 (socket says 172.21.15.46:56894) 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9d600640 1 -- 172.21.15.46:0/1497878937 learned_addr learned my addr 172.21.15.46:0/1497878937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad9401930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9e000640 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 0x55cad94719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9d600640 1 -- 172.21.15.46:0/1497878937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 msgr2=0x55cad94719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9d600640 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 0x55cad94719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9d600640 1 -- 172.21.15.46:0/1497878937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 msgr2=0x55cad9401930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9d600640 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad9401930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9d600640 1 -- 172.21.15.46:0/1497878937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad94049d0 con 0x55cad93f3090 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad9401930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a9d600640 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f3090 0x55cad94013f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55cad93ef580 tx=0x7f0a8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cc0afcf49e0297 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.015+0000 7f0a7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.016+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1497878937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c03b060 con 0x55cad93f3090 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.016+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1497878937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0a8c01ed40 con 0x55cad93f3090 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.016+0000 7f0aa3054740 1 -- 172.21.15.46:0/1497878937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cad935fb80 con 0x55cad93f3090 2024-09-10T07:38:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.016+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1497878937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c042050 con 0x55cad93f3090 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.016+0000 7f0aa3054740 10 client.15681 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.016+0000 7f0aa3054740 1 -- 172.21.15.46:0/1497878937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cad9472f90 con 0x55cad93f3090 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.016+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1497878937 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cad935fb80 con 0x55cad93f3090 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.016+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1497878937 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f0a8c02d5d0 con 0x55cad93f3090 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0aa3054740 10 client.15681 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0aa3054740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0aa3054740 20 client.15681 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cad9473a10 0x55cad9475e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0aa3054740 4 client.15681 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0aa3054740 1 -- 172.21.15.46:0/1497878937 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cad92dc7c0 con 0x55cad9473a10 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cad9473a10 0x55cad9475e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0a9cc00640 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cad9473a10 0x55cad9475e00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a476dcc85d5aecf server_cookie=bb294346375d530d in_seq=0 out_seq=0 2024-09-10T07:38:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.017+0000 7f0a7ea00640 10 client.15681 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.029+0000 7f0a7ea00640 1 -- 172.21.15.46:0/1497878937 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cad92dc7c0 con 0x55cad9473a10 2024-09-10T07:38:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.029+0000 7f0a7ea00640 10 client.15681 handle_command_reply: tid=0 2024-09-10T07:38:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.029+0000 7f0aa3054740 1 client.15681 shutdown 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 -- 172.21.15.46:0/1497878937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f3090 msgr2=0x55cad94013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f3090 0x55cad94013f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55cad93ef580 tx=0x7f0a8c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 -- 172.21.15.46:0/1497878937 shutdown_connections 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cad9473a10 0x55cad9475e00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cad93f8a60 0x55cad94719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cad93f39d0 0x55cad9401930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0a7ea00640 0 client.15681 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 --2- 172.21.15.46:0/1497878937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad93f3090 0x55cad94013f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 -- 172.21.15.46:0/1497878937 >> 172.21.15.46:0/1497878937 conn(0x55cad93ee700 msgr2=0x55cad93fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 -- 172.21.15.46:0/1497878937 shutdown_connections 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 1 -- 172.21.15.46:0/1497878937 wait complete. 2024-09-10T07:38:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.030+0000 7f0aa3054740 20 client.15681 trim_cache size 0 max 16384 2024-09-10T07:38:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:38:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:38:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:38:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 -- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a800a2cb0 msgr2=0x7f0a800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a800a2cb0 0x7f0a800a5170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55cad93f4400 tx=0x7f0a8c005f90 comp rx=0 tx=0).stop 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 -- 172.21.15.46:0/546951796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a98104f50 msgr2=0x7f0a9819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a98104f50 0x7f0a9819a4f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a84035040 tx=0x7f0a84033d90 comp rx=0 tx=0).stop 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 -- 172.21.15.46:0/546951796 shutdown_connections 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a98105910 0x7f0a981a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a98104f50 0x7f0a9819a4f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a800a2cb0 0x7f0a800a5170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 --2- 172.21.15.46:0/546951796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98103d60 0x7f0a98199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 -- 172.21.15.46:0/546951796 >> 172.21.15.46:0/546951796 conn(0x7f0a980ff4e0 msgr2=0x7f0a98108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 -- 172.21.15.46:0/546951796 shutdown_connections 2024-09-10T07:38:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:14.031+0000 7f0aa0000640 1 -- 172.21.15.46:0/546951796 wait complete. 2024-09-10T07:38:14.084 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:14 smithi046 ceph-mon[30705]: pgmap v447: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 314 op/s 2024-09-10T07:38:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:14 smithi181 ceph-mon[35397]: pgmap v447: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 314 op/s 2024-09-10T07:38:14.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:14 smithi063 ceph-mon[35020]: pgmap v447: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 314 op/s 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 -- 172.21.15.46:0/2422797172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128103e30 msgr2=0x7f0128104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 --2- 172.21.15.46:0/2422797172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128103e30 0x7f0128104210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f011c009900 tx=0x7f011c02a400 comp rx=0 tx=0).stop 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 -- 172.21.15.46:0/2422797172 shutdown_connections 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 --2- 172.21.15.46:0/2422797172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0128108f60 0x7f0128110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 --2- 172.21.15.46:0/2422797172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01281047e0 0x7f0128108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 --2- 172.21.15.46:0/2422797172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128103e30 0x7f0128104210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 -- 172.21.15.46:0/2422797172 >> 172.21.15.46:0/2422797172 conn(0x7f01280ff910 msgr2=0x7f0128101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 -- 172.21.15.46:0/2422797172 shutdown_connections 2024-09-10T07:38:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.660+0000 7f012f000640 1 -- 172.21.15.46:0/2422797172 wait complete. 2024-09-10T07:38:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.662+0000 7f012f000640 1 Processor -- start 2024-09-10T07:38:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.662+0000 7f012f000640 1 -- start start 2024-09-10T07:38:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.662+0000 7f012f000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0128103e30 0x7f012819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.662+0000 7f012f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01281047e0 0x7f012819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.662+0000 7f012f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128108f60 0x7f012819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.662+0000 7f012f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0128113fc0 con 0x7f01281047e0 2024-09-10T07:38:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.662+0000 7f012f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0128113e40 con 0x7f0128108f60 2024-09-10T07:38:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.662+0000 7f012f000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0128114140 con 0x7f0128103e30 2024-09-10T07:38:14.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01281047e0 0x7f012819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01281047e0 0x7f012819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56906/0 (socket says 172.21.15.46:56906) 2024-09-10T07:38:14.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012d200640 1 -- 172.21.15.46:0/592741090 learned_addr learned my addr 172.21.15.46:0/592741090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:14.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012e600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128108f60 0x7f012819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012dc00640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0128103e30 0x7f012819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012e600640 1 -- 172.21.15.46:0/592741090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0128103e30 msgr2=0x7f012819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012e600640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0128103e30 0x7f012819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012e600640 1 -- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01281047e0 msgr2=0x7f012819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012e600640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01281047e0 0x7f012819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012e600640 1 -- 172.21.15.46:0/592741090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012819f4d0 con 0x7f0128108f60 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.663+0000 7f012e600640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128108f60 0x7f012819edf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f012000ec50 tx=0x7f012000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.664+0000 7f0126000640 1 -- 172.21.15.46:0/592741090 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120003b70 con 0x7f0128108f60 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.664+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012819f7c0 con 0x7f0128108f60 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.664+0000 7f0126000640 1 -- 172.21.15.46:0/592741090 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0120003d10 con 0x7f0128108f60 2024-09-10T07:38:14.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.664+0000 7f0126000640 1 -- 172.21.15.46:0/592741090 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120010890 con 0x7f0128108f60 2024-09-10T07:38:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.664+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01281a70a0 con 0x7f0128108f60 2024-09-10T07:38:14.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.665+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0128105310 con 0x7f0128108f60 2024-09-10T07:38:14.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.667+0000 7f0126000640 1 -- 172.21.15.46:0/592741090 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0120004b40 con 0x7f0128108f60 2024-09-10T07:38:14.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.668+0000 7f0126000640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01100a2b80 0x7f01100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:14.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.668+0000 7f0126000640 1 -- 172.21.15.46:0/592741090 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f01200cf940 con 0x7f0128108f60 2024-09-10T07:38:14.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.668+0000 7f012dc00640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01100a2b80 0x7f01100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:14.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.669+0000 7f012dc00640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01100a2b80 0x7f01100a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f011c009230 tx=0x7f011c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.670+0000 7f0126000640 1 -- 172.21.15.46:0/592741090 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f012009c4c0 con 0x7f0128108f60 2024-09-10T07:38:14.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.895+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0128104210 con 0x7f0128108f60 2024-09-10T07:38:14.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.896+0000 7f0126000640 1 -- 172.21.15.46:0/592741090 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11979 (secure 0 0 0) 0x7f012009bdc0 con 0x7f0128108f60 2024-09-10T07:38:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:38:14.896 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"btime":"2024-09-10T07:37:35:593020+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:37:35.488899+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:38:14.897 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01100a2b80 msgr2=0x7f01100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01100a2b80 0x7f01100a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f011c009230 tx=0x7f011c0058d0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128108f60 msgr2=0x7f012819edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128108f60 0x7f012819edf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f012000ec50 tx=0x7f012000c6a0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 shutdown_connections 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0128108f60 0x7f012819edf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01100a2b80 0x7f01100a5040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01281047e0 0x7f012819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 --2- 172.21.15.46:0/592741090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0128103e30 0x7f012819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 >> 172.21.15.46:0/592741090 conn(0x7f01280ff910 msgr2=0x7f0128100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:14.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 shutdown_connections 2024-09-10T07:38:14.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:14.899+0000 7f012f000640 1 -- 172.21.15.46:0/592741090 wait complete. 2024-09-10T07:38:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 55.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:38:15.617 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:15 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/592741090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:15.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:15 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/592741090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:15.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:15 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/592741090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:15.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:38:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:16.422 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:16 smithi046 ceph-mon[30705]: pgmap v448: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 366 op/s 2024-09-10T07:38:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:16 smithi181 ceph-mon[35397]: pgmap v448: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 366 op/s 2024-09-10T07:38:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:16 smithi063 ceph-mon[35020]: pgmap v448: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 366 op/s 2024-09-10T07:38:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.661+0000 7f8639800640 1 -- 172.21.15.46:0/3807520843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634104010 msgr2=0x7f86341043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.661+0000 7f8639800640 1 --2- 172.21.15.46:0/3807520843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634104010 0x7f86341043f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8624009900 tx=0x7f862402a410 comp rx=0 tx=0).stop 2024-09-10T07:38:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.661+0000 7f8639800640 1 -- 172.21.15.46:0/3807520843 shutdown_connections 2024-09-10T07:38:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.661+0000 7f8639800640 1 --2- 172.21.15.46:0/3807520843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8634109140 0x7f8634110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.661+0000 7f8639800640 1 --2- 172.21.15.46:0/3807520843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86341049c0 0x7f8634108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.661+0000 7f8639800640 1 --2- 172.21.15.46:0/3807520843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634104010 0x7f86341043f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.661+0000 7f8639800640 1 -- 172.21.15.46:0/3807520843 >> 172.21.15.46:0/3807520843 conn(0x7f86340ffa50 msgr2=0x7f8634101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.662+0000 7f8639800640 1 -- 172.21.15.46:0/3807520843 shutdown_connections 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.662+0000 7f8639800640 1 -- 172.21.15.46:0/3807520843 wait complete. 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.662+0000 7f8639800640 1 Processor -- start 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8639800640 1 -- start start 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8639800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8634104010 0x7f863419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8639800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86341049c0 0x7f863419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8639800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634109140 0x7f863419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8639800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86341141c0 con 0x7f8634109140 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8639800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8634114040 con 0x7f86341049c0 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8639800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8634114340 con 0x7f8634104010 2024-09-10T07:38:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8633400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634109140 0x7f863419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8633400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634109140 0x7f863419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36804/0 (socket says 172.21.15.46:36804) 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8633400640 1 -- 172.21.15.46:0/2683569378 learned_addr learned my addr 172.21.15.46:0/2683569378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f862be00640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86341049c0 0x7f863419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.663+0000 7f8632a00640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8634104010 0x7f863419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f8633400640 1 -- 172.21.15.46:0/2683569378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8634104010 msgr2=0x7f863419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f8633400640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8634104010 0x7f863419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f8633400640 1 -- 172.21.15.46:0/2683569378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86341049c0 msgr2=0x7f863419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f8633400640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86341049c0 0x7f863419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f8633400640 1 -- 172.21.15.46:0/2683569378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863419f5f0 con 0x7f8634109140 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f8633400640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634109140 0x7f863419ef10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f862c00eb10 tx=0x7f862c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f862b400640 1 -- 172.21.15.46:0/2683569378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c004b20 con 0x7f8634109140 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f862b400640 1 -- 172.21.15.46:0/2683569378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862c004cc0 con 0x7f8634109140 2024-09-10T07:38:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f862b400640 1 -- 172.21.15.46:0/2683569378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c010760 con 0x7f8634109140 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.664+0000 7f8639800640 1 -- 172.21.15.46:0/2683569378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863419f8e0 con 0x7f8634109140 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.665+0000 7f8639800640 1 -- 172.21.15.46:0/2683569378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86341a71c0 con 0x7f8634109140 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f863c3bc740 1 Processor -- start 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f862b400640 1 -- 172.21.15.46:0/2683569378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f862c005020 con 0x7f8634109140 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f863c3bc740 1 -- start start 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84227b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227b590 0x55e842285910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 0x55e842288240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f863c3bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e8421e13e0 con 0x55e842285e50 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f863c3bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e842188d80 con 0x55e84227ac70 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.669+0000 7f863c3bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e8421dd3a0 con 0x55e84227b590 2024-09-10T07:38:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f862be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84227b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f862be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84227b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38470/0 (socket says 172.21.15.46:38470) 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f862be00640 1 -- 172.21.15.46:0/777274469 learned_addr learned my addr 172.21.15.46:0/777274469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f8632a00640 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227b590 0x55e842285910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f862be00640 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 0x55e842288240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f8632a00640 1 -- 172.21.15.46:0/777274469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 msgr2=0x55e84227b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f8632a00640 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84227b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f8632a00640 1 -- 172.21.15.46:0/777274469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 msgr2=0x55e842288240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f8632a00640 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 0x55e842288240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f8632a00640 1 -- 172.21.15.46:0/777274469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8421a7ae0 con 0x55e84227b590 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.670+0000 7f862be00640 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84227b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.671+0000 7f862b400640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86180a2b80 0x7f86180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.671+0000 7f8632a00640 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227b590 0x55e842285910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f862402aa00 tx=0x7f8624005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9631327b170fee28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.671+0000 7f862b400640 1 -- 172.21.15.46:0/2683569378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f862c0cfa60 con 0x7f8634109140 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.671+0000 7f862be00640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86180a2b80 0x7f86180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.671+0000 7f8623e00640 1 -- 172.21.15.46:0/777274469 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624035040 con 0x55e84227b590 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.671+0000 7f8623e00640 1 -- 172.21.15.46:0/777274469 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8624004400 con 0x55e84227b590 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.671+0000 7f8623e00640 1 -- 172.21.15.46:0/777274469 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240035f0 con 0x55e84227b590 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 -- 172.21.15.46:0/777274469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227b590 msgr2=0x55e842285910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f862be00640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86180a2b80 0x7f86180a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55e842278580 tx=0x7f861c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227b590 0x55e842285910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f862402aa00 tx=0x7f8624005de0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 -- 172.21.15.46:0/777274469 shutdown_connections 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 0x55e842288240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227b590 0x55e842285910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 --2- 172.21.15.46:0/777274469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84227b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 -- 172.21.15.46:0/777274469 >> 172.21.15.46:0/777274469 conn(0x55e842277200 msgr2=0x55e8422775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 -- 172.21.15.46:0/777274469 shutdown_connections 2024-09-10T07:38:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.672+0000 7f863c3bc740 1 -- 172.21.15.46:0/777274469 wait complete. 2024-09-10T07:38:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 1 -- start start 2024-09-10T07:38:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 1 Processor -- start 2024-09-10T07:38:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227ac70 0x55e842282d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842283260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e8422837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e842288a20 con 0x55e84227ac70 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e84219c0f0 con 0x55e84227b590 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e842288ba0 con 0x55e842285e50 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.673+0000 7f863c3bc740 10 client.0 _ll_register_callbacks cb 0x55e842183bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f862be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842283260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8632a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227ac70 0x55e842282d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8633400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e8422837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f862be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842283260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38474/0 (socket says 172.21.15.46:38474) 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f862be00640 1 -- 172.21.15.46:0/3779247334 learned_addr learned my addr 172.21.15.46:0/3779247334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8633400640 1 -- 172.21.15.46:0/3779247334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 msgr2=0x55e842283260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8633400640 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842283260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8633400640 1 -- 172.21.15.46:0/3779247334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227ac70 msgr2=0x55e842282d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8633400640 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227ac70 0x55e842282d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8633400640 1 -- 172.21.15.46:0/3779247334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e841f523c0 con 0x55e842285e50 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8633400640 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e8422837a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f862c0049d0 tx=0x7f862c09baa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e92ee808687627e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.674+0000 7f8620c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f8620c00640 1 -- 172.21.15.46:0/3779247334 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c005510 con 0x55e842285e50 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f863c3bc740 1 -- 172.21.15.46:0/3779247334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8624004400 con 0x55e842285e50 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f8620c00640 1 -- 172.21.15.46:0/3779247334 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f862c09b330 con 0x55e842285e50 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f8620c00640 1 -- 172.21.15.46:0/3779247334 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f862c09bee0 con 0x55e842285e50 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f863c3bc740 10 client.25795 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f863c3bc740 1 -- 172.21.15.46:0/3779247334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e842294a90 con 0x55e842285e50 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f8620c00640 1 -- 172.21.15.46:0/3779247334 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8624004400 con 0x55e842285e50 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f8620c00640 1 -- 172.21.15.46:0/3779247334 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f862c097cc0 con 0x55e842285e50 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.675+0000 7f863c3bc740 10 client.25795 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:16.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.676+0000 7f863c3bc740 10 client.25795 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.676+0000 7f863c3bc740 20 client.25795 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.676+0000 7f863c3bc740 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e842295ea0 0x55e842298310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.676+0000 7f863c3bc740 4 client.25795 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.676+0000 7f863c3bc740 1 -- 172.21.15.46:0/3779247334 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8421ee950 con 0x55e842295ea0 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.676+0000 7f8632a00640 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e842295ea0 0x55e842298310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.676+0000 7f8632a00640 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e842295ea0 0x55e842298310 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4825b0cc24973a4 server_cookie=6366b1dac8b3e6dd in_seq=0 out_seq=0 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.676+0000 7f8620c00640 10 client.25795 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.677+0000 7f8620c00640 1 -- 172.21.15.46:0/3779247334 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8421ee950 con 0x55e842295ea0 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.677+0000 7f8620c00640 10 client.25795 handle_command_reply: tid=0 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.677+0000 7f863c3bc740 1 client.25795 shutdown 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 -- 172.21.15.46:0/3779247334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 msgr2=0x55e8422837a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e8422837a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f862c0049d0 tx=0x7f862c09baa0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 -- 172.21.15.46:0/3779247334 shutdown_connections 2024-09-10T07:38:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e842295ea0 0x55e842298310 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e8422837a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842283260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 --2- 172.21.15.46:0/3779247334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227ac70 0x55e842282d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 -- 172.21.15.46:0/3779247334 >> 172.21.15.46:0/3779247334 conn(0x55e842277200 msgr2=0x55e84227cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f8620c00640 0 client.25795 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:38:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 -- 172.21.15.46:0/3779247334 shutdown_connections 2024-09-10T07:38:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 1 -- 172.21.15.46:0/3779247334 wait complete. 2024-09-10T07:38:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.678+0000 7f863c3bc740 20 client.25795 trim_cache size 0 max 16384 2024-09-10T07:38:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.692+0000 7f863c3bc740 1 Processor -- start 2024-09-10T07:38:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.692+0000 7f863c3bc740 1 -- start start 2024-09-10T07:38:16.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.692+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84229dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.692+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227b590 0x55e84229e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.692+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e84227ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.692+0000 7f863c3bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86341b0ea0 con 0x55e84227b590 2024-09-10T07:38:16.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.692+0000 7f863c3bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86341b2aa0 con 0x55e84227ac70 2024-09-10T07:38:16.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.692+0000 7f863c3bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86341b1570 con 0x55e842285e50 2024-09-10T07:38:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f862be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227b590 0x55e84229e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f8632a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84229dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f862be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227b590 0x55e84229e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36820/0 (socket says 172.21.15.46:36820) 2024-09-10T07:38:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f862be00640 1 -- 172.21.15.46:0/943537883 learned_addr learned my addr 172.21.15.46:0/943537883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f8633400640 1 --2- 172.21.15.46:0/943537883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e84227ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f8632a00640 1 -- 172.21.15.46:0/943537883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 msgr2=0x55e84227ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f8632a00640 1 --2- 172.21.15.46:0/943537883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e84227ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f8632a00640 1 -- 172.21.15.46:0/943537883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227b590 msgr2=0x55e84229e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f8632a00640 1 --2- 172.21.15.46:0/943537883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227b590 0x55e84229e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f8632a00640 1 -- 172.21.15.46:0/943537883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e842272770 con 0x55e84227ac70 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.693+0000 7f8632a00640 1 --2- 172.21.15.46:0/943537883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84229dbe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8624031550 tx=0x7f8624005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d93aa4e5d0cc7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f8623400640 1 -- 172.21.15.46:0/943537883 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624035040 con 0x55e84227ac70 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f8623400640 1 -- 172.21.15.46:0/943537883 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8624004200 con 0x55e84227ac70 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f8623400640 1 -- 172.21.15.46:0/943537883 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624003870 con 0x55e84227ac70 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f863c3bc740 1 -- 172.21.15.46:0/943537883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 msgr2=0x55e84229dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f863c3bc740 1 --2- 172.21.15.46:0/943537883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84229dbe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8624031550 tx=0x7f8624005b50 comp rx=0 tx=0).stop 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f863c3bc740 1 -- 172.21.15.46:0/943537883 shutdown_connections 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f863c3bc740 1 --2- 172.21.15.46:0/943537883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e842285e50 0x55e84227ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f863c3bc740 1 --2- 172.21.15.46:0/943537883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e84227b590 0x55e84229e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f863c3bc740 1 --2- 172.21.15.46:0/943537883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227ac70 0x55e84229dbe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f863c3bc740 1 -- 172.21.15.46:0/943537883 >> 172.21.15.46:0/943537883 conn(0x55e842277200 msgr2=0x55e842276380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.694+0000 7f863c3bc740 1 -- 172.21.15.46:0/943537883 shutdown_connections 2024-09-10T07:38:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.695+0000 7f863c3bc740 1 -- 172.21.15.46:0/943537883 wait complete. 2024-09-10T07:38:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.695+0000 7f863c3bc740 1 -- start start 2024-09-10T07:38:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.695+0000 7f863c3bc740 1 Processor -- start 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227ac70 0x55e84227fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842280240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f863c3bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 0x55e842280780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f863c3bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86341b2310 con 0x55e842285e50 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f863c3bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86341b1c40 con 0x55e84227b590 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f863c3bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86341aee80 con 0x55e84227ac70 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f863c3bc740 10 client.0 _ll_register_callbacks cb 0x7f862c09bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f862be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842280240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f8632a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227ac70 0x55e84227fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f862be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842280240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38494/0 (socket says 172.21.15.46:38494) 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f862be00640 1 -- 172.21.15.46:0/664448534 learned_addr learned my addr 172.21.15.46:0/664448534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f8633400640 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 0x55e842280780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f8632a00640 1 -- 172.21.15.46:0/664448534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 msgr2=0x55e842280240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f8632a00640 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842280240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f8632a00640 1 -- 172.21.15.46:0/664448534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 msgr2=0x55e842280780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f8632a00640 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 0x55e842280780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.696+0000 7f8632a00640 1 -- 172.21.15.46:0/664448534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8422ffbb0 con 0x55e84227ac70 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.697+0000 7f8632a00640 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227ac70 0x55e84227fd00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f862402a930 tx=0x7f8624009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e40975e642148287 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.697+0000 7f8620c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.697+0000 7f8620c00640 1 -- 172.21.15.46:0/664448534 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624035070 con 0x55e84227ac70 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.697+0000 7f8620c00640 1 -- 172.21.15.46:0/664448534 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8624005230 con 0x55e84227ac70 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.697+0000 7f863c3bc740 1 -- 172.21.15.46:0/664448534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8624004200 con 0x55e84227ac70 2024-09-10T07:38:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.697+0000 7f8620c00640 1 -- 172.21.15.46:0/664448534 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8624068050 con 0x55e84227ac70 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.697+0000 7f863c3bc740 10 client.25801 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.697+0000 7f863c3bc740 1 -- 172.21.15.46:0/664448534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e842300e50 con 0x55e84227ac70 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f8620c00640 1 -- 172.21.15.46:0/664448534 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8624004200 con 0x55e84227ac70 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f8620c00640 1 -- 172.21.15.46:0/664448534 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f8624031810 con 0x55e84227ac70 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f863c3bc740 10 client.25801 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f863c3bc740 10 client.25801 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f863c3bc740 20 client.25801 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f863c3bc740 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e842301120 0x55e842303510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f863c3bc740 4 client.25801 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f863c3bc740 1 -- 172.21.15.46:0/664448534 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8421e8e60 con 0x55e842301120 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.698+0000 7f862be00640 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e842301120 0x55e842303510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f862be00640 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e842301120 0x55e842303510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c577d0ea648d7cae server_cookie=91a3e8c3797fb3f2 in_seq=0 out_seq=0 2024-09-10T07:38:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f8620c00640 10 client.25801 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:38:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f8620c00640 1 -- 172.21.15.46:0/664448534 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8421e8e60 con 0x55e842301120 2024-09-10T07:38:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f8620c00640 10 client.25801 handle_command_reply: tid=0 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 client.25801 shutdown 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 -- 172.21.15.46:0/664448534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227ac70 msgr2=0x55e84227fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227ac70 0x55e84227fd00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f862402a930 tx=0x7f8624009640 comp rx=0 tx=0).stop 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 -- 172.21.15.46:0/664448534 shutdown_connections 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e842301120 0x55e842303510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e842285e50 0x55e842280780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e84227b590 0x55e842280240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 --2- 172.21.15.46:0/664448534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e84227ac70 0x55e84227fd00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 -- 172.21.15.46:0/664448534 >> 172.21.15.46:0/664448534 conn(0x55e842277200 msgr2=0x55e842284580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f8620c00640 0 client.25801 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 -- 172.21.15.46:0/664448534 shutdown_connections 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.699+0000 7f863c3bc740 1 -- 172.21.15.46:0/664448534 wait complete. 2024-09-10T07:38:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f863c3bc740 20 client.25801 trim_cache size 0 max 16384 2024-09-10T07:38:16.703 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:38:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 -- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86180a2b80 msgr2=0x7f86180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86180a2b80 0x7f86180a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55e842278580 tx=0x7f861c005f90 comp rx=0 tx=0).stop 2024-09-10T07:38:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 -- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634109140 msgr2=0x7f863419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634109140 0x7f863419ef10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f862c00eb10 tx=0x7f862c00c2d0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 -- 172.21.15.46:0/2683569378 shutdown_connections 2024-09-10T07:38:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86180a2b80 0x7f86180a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8634109140 0x7f863419ef10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86341049c0 0x7f863419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 --2- 172.21.15.46:0/2683569378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8634104010 0x7f863419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.700+0000 7f8639800640 1 -- 172.21.15.46:0/2683569378 >> 172.21.15.46:0/2683569378 conn(0x7f86340ffa50 msgr2=0x7f8634100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.701+0000 7f8639800640 1 -- 172.21.15.46:0/2683569378 shutdown_connections 2024-09-10T07:38:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:16.701+0000 7f8639800640 1 -- 172.21.15.46:0/2683569378 wait complete. 2024-09-10T07:38:17.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:38:18.032 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:18.555 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:18 smithi046 ceph-mon[30705]: pgmap v449: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 428 op/s 2024-09-10T07:38:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:18 smithi181 ceph-mon[35397]: pgmap v449: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 428 op/s 2024-09-10T07:38:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:18 smithi063 ceph-mon[35020]: pgmap v449: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 428 op/s 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 -- 172.21.15.46:0/4251628587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf81047e0 msgr2=0x7f0cf8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/4251628587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf81047e0 0x7f0cf8108830 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009900 tx=0x7f0ce002a410 comp rx=0 tx=0).stop 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 -- 172.21.15.46:0/4251628587 shutdown_connections 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/4251628587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf8108f60 0x7f0cf8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/4251628587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf81047e0 0x7f0cf8108830 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/4251628587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf8103e30 0x7f0cf8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 -- 172.21.15.46:0/4251628587 >> 172.21.15.46:0/4251628587 conn(0x7f0cf80ff8d0 msgr2=0x7f0cf8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 -- 172.21.15.46:0/4251628587 shutdown_connections 2024-09-10T07:38:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.672+0000 7f0cf7e00640 1 -- 172.21.15.46:0/4251628587 wait complete. 2024-09-10T07:38:18.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.673+0000 7f0cf7e00640 1 Processor -- start 2024-09-10T07:38:18.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.673+0000 7f0cf7e00640 1 -- start start 2024-09-10T07:38:18.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf8103e30 0x7f0cf819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf81047e0 0x7f0cf819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf8108f60 0x7f0cf819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8113f80 con 0x7f0cf8103e30 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8113e00 con 0x7f0cf81047e0 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf7e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8114100 con 0x7f0cf8108f60 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf8103e30 0x7f0cf819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf8103e30 0x7f0cf819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36834/0 (socket says 172.21.15.46:36834) 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf6a00640 1 -- 172.21.15.46:0/524122818 learned_addr learned my addr 172.21.15.46:0/524122818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cf7400640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf8108f60 0x7f0cf819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.674+0000 7f0cee000640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf81047e0 0x7f0cf819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cf6a00640 1 -- 172.21.15.46:0/524122818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf8108f60 msgr2=0x7f0cf819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cf6a00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf8108f60 0x7f0cf819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cf6a00640 1 -- 172.21.15.46:0/524122818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf81047e0 msgr2=0x7f0cf819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cf6a00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf81047e0 0x7f0cf819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cf6a00640 1 -- 172.21.15.46:0/524122818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf819f410 con 0x7f0cf8103e30 2024-09-10T07:38:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cf6a00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf8103e30 0x7f0cf819a460 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800e9f0 tx=0x7f0ce800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cfca00640 1 -- 172.21.15.46:0/524122818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800cd80 con 0x7f0cf8103e30 2024-09-10T07:38:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cfca00640 1 -- 172.21.15.46:0/524122818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce8010910 con 0x7f0cf8103e30 2024-09-10T07:38:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cf7e00640 1 -- 172.21.15.46:0/524122818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf819f700 con 0x7f0cf8103e30 2024-09-10T07:38:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cfca00640 1 -- 172.21.15.46:0/524122818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8018990 con 0x7f0cf8103e30 2024-09-10T07:38:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.675+0000 7f0cf7e00640 1 -- 172.21.15.46:0/524122818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81a6fe0 con 0x7f0cf8103e30 2024-09-10T07:38:18.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.678+0000 7f0d00862740 1 Processor -- start 2024-09-10T07:38:18.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0d00862740 1 -- start start 2024-09-10T07:38:18.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0cfca00640 1 -- 172.21.15.46:0/524122818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ce8010ac0 con 0x7f0cf8103e30 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0d00862740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d452220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0d00862740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0d00862740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 0x55fd0d45f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0d00862740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d3b83e0 con 0x55fd0d451e40 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0d00862740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d35fd80 con 0x55fd0d45d020 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0d00862740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d3b43a0 con 0x55fd0d452760 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0cf7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d452220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0cf7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d452220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36850/0 (socket says 172.21.15.46:36850) 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0cf7400640 1 -- 172.21.15.46:0/661478342 learned_addr learned my addr 172.21.15.46:0/661478342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0cf6a00640 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.679+0000 7f0cee000640 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 0x55fd0d45f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0cf7400640 1 -- 172.21.15.46:0/661478342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 msgr2=0x55fd0d45cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0cf7400640 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0cf7400640 1 -- 172.21.15.46:0/661478342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 msgr2=0x55fd0d45f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0cf7400640 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 0x55fd0d45f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0cf7400640 1 -- 172.21.15.46:0/661478342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0d37eae0 con 0x55fd0d451e40 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0cf6a00640 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0cf7400640 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d452220 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009f70 tx=0x7f0cf0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decc866d8fec9ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0ce7e00640 1 -- 172.21.15.46:0/661478342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0016070 con 0x55fd0d451e40 2024-09-10T07:38:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0ce7e00640 1 -- 172.21.15.46:0/661478342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0004d20 con 0x55fd0d451e40 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.680+0000 7f0ce7e00640 1 -- 172.21.15.46:0/661478342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00128f0 con 0x55fd0d451e40 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0cfca00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0cdc0a2c00 0x7f0cdc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0d00862740 1 -- 172.21.15.46:0/661478342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 msgr2=0x55fd0d452220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0d00862740 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d452220 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009f70 tx=0x7f0cf0010800 comp rx=0 tx=0).stop 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0cfca00640 1 -- 172.21.15.46:0/524122818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0ce80cf690 con 0x7f0cf8103e30 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0d00862740 1 -- 172.21.15.46:0/661478342 shutdown_connections 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0d00862740 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 0x55fd0d45f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0d00862740 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0d00862740 1 --2- 172.21.15.46:0/661478342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d452220 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0cf6a00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0cdc0a2c00 0x7f0cdc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.681+0000 7f0d00862740 1 -- 172.21.15.46:0/661478342 >> 172.21.15.46:0/661478342 conn(0x55fd0d44e3d0 msgr2=0x55fd0d44e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.682+0000 7f0d00862740 1 -- 172.21.15.46:0/661478342 shutdown_connections 2024-09-10T07:38:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.682+0000 7f0d00862740 1 -- 172.21.15.46:0/661478342 wait complete. 2024-09-10T07:38:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.682+0000 7f0cf6a00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0cdc0a2c00 0x7f0cdc0a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ce809b8c0 tx=0x7f0ce80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.682+0000 7f0d00862740 1 -- start start 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.682+0000 7f0d00862740 1 Processor -- start 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0d00862740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d459ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0d00862740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0d00862740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 0x55fd0d45a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0d00862740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d45fbf0 con 0x55fd0d451e40 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0d00862740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d3730f0 con 0x55fd0d45d020 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0d00862740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d45fd70 con 0x55fd0d452760 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0d00862740 10 client.0 _ll_register_callbacks cb 0x55fd0d35abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0cf7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d459ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0cf7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d459ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36866/0 (socket says 172.21.15.46:36866) 2024-09-10T07:38:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0cf7400640 1 -- 172.21.15.46:0/950710611 learned_addr learned my addr 172.21.15.46:0/950710611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0cee000640 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.683+0000 7f0cf7400640 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 0x55fd0d45a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0cee000640 1 -- 172.21.15.46:0/950710611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 msgr2=0x55fd0d45a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0cee000640 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 0x55fd0d45a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0cee000640 1 -- 172.21.15.46:0/950710611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 msgr2=0x55fd0d459ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0cee000640 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d459ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0cee000640 1 -- 172.21.15.46:0/950710611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0d1293c0 con 0x55fd0d452760 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0cee000640 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45a400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005940 tx=0x7f0ce00317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874e0a7f8bf09db0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0ce4c00640 1 -- 172.21.15.46:0/950710611 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0035040 con 0x55fd0d452760 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.684+0000 7f0d00862740 1 -- 172.21.15.46:0/950710611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf0004d20 con 0x55fd0d452760 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0ce4c00640 1 -- 172.21.15.46:0/950710611 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0ce0004240 con 0x55fd0d452760 2024-09-10T07:38:18.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0ce4c00640 1 -- 172.21.15.46:0/950710611 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ce004f050 con 0x55fd0d452760 2024-09-10T07:38:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0d00862740 10 client.25813 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0d00862740 1 -- 172.21.15.46:0/950710611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fd0d46bc30 con 0x55fd0d452760 2024-09-10T07:38:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0ce4c00640 1 -- 172.21.15.46:0/950710611 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0004d20 con 0x55fd0d452760 2024-09-10T07:38:18.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0ce4c00640 1 -- 172.21.15.46:0/950710611 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f0ce0004aa0 con 0x55fd0d452760 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0d00862740 10 client.25813 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0d00862740 10 client.25813 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0d00862740 20 client.25813 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.685+0000 7f0d00862740 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd0d46d040 0x55fd0d46f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.686+0000 7f0d00862740 4 client.25813 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.686+0000 7f0d00862740 1 -- 172.21.15.46:0/950710611 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd0d3c5950 con 0x55fd0d46d040 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.686+0000 7f0cf7400640 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd0d46d040 0x55fd0d46f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.686+0000 7f0cf7400640 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd0d46d040 0x55fd0d46f430 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f97bc7e4753a750 server_cookie=86e2c8330a024da5 in_seq=0 out_seq=0 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.686+0000 7f0ce4c00640 10 client.25813 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.687+0000 7f0ce4c00640 1 -- 172.21.15.46:0/950710611 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd0d3c5950 con 0x55fd0d46d040 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.687+0000 7f0ce4c00640 10 client.25813 handle_command_reply: tid=0 2024-09-10T07:38:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 client.25813 shutdown 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 -- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 msgr2=0x55fd0d45a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45a400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005940 tx=0x7f0ce00317e0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 -- 172.21.15.46:0/950710611 shutdown_connections 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d45d020 0x55fd0d45a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd0d46d040 0x55fd0d46f430 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d45a400 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 --2- 172.21.15.46:0/950710611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d459ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0ce4c00640 0 client.25813 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 -- 172.21.15.46:0/950710611 >> 172.21.15.46:0/950710611 conn(0x55fd0d44e3d0 msgr2=0x55fd0d44d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 -- 172.21.15.46:0/950710611 shutdown_connections 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 1 -- 172.21.15.46:0/950710611 wait complete. 2024-09-10T07:38:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.688+0000 7f0d00862740 20 client.25813 trim_cache size 0 max 16384 2024-09-10T07:38:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.702+0000 7f0d00862740 1 Processor -- start 2024-09-10T07:38:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.702+0000 7f0d00862740 1 -- start start 2024-09-10T07:38:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0d00862740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d451e40 0x55fd0d449300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0d00862740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d44d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0d00862740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d45d020 0x55fd0d474c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0d00862740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b0d40 con 0x55fd0d45d020 2024-09-10T07:38:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0d00862740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b2940 con 0x55fd0d451e40 2024-09-10T07:38:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0d00862740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b1410 con 0x55fd0d452760 2024-09-10T07:38:18.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d451e40 0x55fd0d449300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d451e40 0x55fd0d449300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38550/0 (socket says 172.21.15.46:38550) 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 -- 172.21.15.46:0/2158779348 learned_addr learned my addr 172.21.15.46:0/2158779348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cee000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d44d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d45d020 0x55fd0d474c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 -- 172.21.15.46:0/2158779348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 msgr2=0x55fd0d44d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d44d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 -- 172.21.15.46:0/2158779348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d45d020 msgr2=0x55fd0d474c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d45d020 0x55fd0d474c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cf7400640 1 -- 172.21.15.46:0/2158779348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0d451c70 con 0x55fd0d451e40 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.703+0000 7f0cee000640 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d44d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.704+0000 7f0cf7400640 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d451e40 0x55fd0d449300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0000f80 tx=0x7f0cf0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac655fab3544e674 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.704+0000 7f0ce7400640 1 -- 172.21.15.46:0/2158779348 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0016040 con 0x55fd0d451e40 2024-09-10T07:38:18.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.704+0000 7f0ce7400640 1 -- 172.21.15.46:0/2158779348 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0025040 con 0x55fd0d451e40 2024-09-10T07:38:18.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.704+0000 7f0ce7400640 1 -- 172.21.15.46:0/2158779348 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0024040 con 0x55fd0d451e40 2024-09-10T07:38:18.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 -- 172.21.15.46:0/2158779348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d451e40 msgr2=0x55fd0d449300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d451e40 0x55fd0d449300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0000f80 tx=0x7f0cf0010800 comp rx=0 tx=0).stop 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 -- 172.21.15.46:0/2158779348 shutdown_connections 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d45d020 0x55fd0d474c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d452760 0x55fd0d44d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 --2- 172.21.15.46:0/2158779348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d451e40 0x55fd0d449300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 -- 172.21.15.46:0/2158779348 >> 172.21.15.46:0/2158779348 conn(0x55fd0d44e3d0 msgr2=0x55fd0d44cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 -- 172.21.15.46:0/2158779348 shutdown_connections 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 -- 172.21.15.46:0/2158779348 wait complete. 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.705+0000 7f0d00862740 1 -- start start 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0d00862740 1 Processor -- start 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0d00862740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d461410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0d00862740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d452760 0x55fd0d461950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0d00862740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d45d020 0x55fd0d4d6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0d00862740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b21b0 con 0x55fd0d451e40 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0d00862740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b1ae0 con 0x55fd0d452760 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0d00862740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81aed20 con 0x55fd0d45d020 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0d00862740 10 client.0 _ll_register_callbacks cb 0x7f0ce0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0cf7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d461410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0cf7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d461410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36886/0 (socket says 172.21.15.46:36886) 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0cf7400640 1 -- 172.21.15.46:0/3230430671 learned_addr learned my addr 172.21.15.46:0/3230430671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.706+0000 7f0cee000640 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d452760 0x55fd0d461950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0cf7400640 1 -- 172.21.15.46:0/3230430671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d45d020 msgr2=0x55fd0d4d6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0cf7400640 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d45d020 0x55fd0d4d6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0cf7400640 1 -- 172.21.15.46:0/3230430671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d452760 msgr2=0x55fd0d461950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0cf7400640 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d452760 0x55fd0d461950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0cf7400640 1 -- 172.21.15.46:0/3230430671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0d4d7150 con 0x55fd0d451e40 2024-09-10T07:38:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0cee000640 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d452760 0x55fd0d461950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0cf7400640 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d461410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009f10 tx=0x7f0cf0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca872f0ab82958c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:18.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0ce4c00640 1 -- 172.21.15.46:0/3230430671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf003a070 con 0x55fd0d451e40 2024-09-10T07:38:18.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.707+0000 7f0ce4c00640 1 -- 172.21.15.46:0/3230430671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0cf002d3f0 con 0x55fd0d451e40 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 1 -- 172.21.15.46:0/3230430671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf0025040 con 0x55fd0d451e40 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0ce4c00640 1 -- 172.21.15.46:0/3230430671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0cf0024070 con 0x55fd0d451e40 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 10 client.15723 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 1 -- 172.21.15.46:0/3230430671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fd0d4d83f0 con 0x55fd0d451e40 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0ce4c00640 1 -- 172.21.15.46:0/3230430671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0025040 con 0x55fd0d451e40 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0ce4c00640 1 -- 172.21.15.46:0/3230430671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f0cf0034020 con 0x55fd0d451e40 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 10 client.15723 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 10 client.15723 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 20 client.15723 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd0d4d8870 0x55fd0d4dac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 4 client.15723 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.708+0000 7f0d00862740 1 -- 172.21.15.46:0/3230430671 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd0d3bfe60 con 0x55fd0d4d8870 2024-09-10T07:38:18.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.709+0000 7f0cee000640 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd0d4d8870 0x55fd0d4dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.709+0000 7f0cee000640 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd0d4d8870 0x55fd0d4dac60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=159d4c763d991594 server_cookie=f093ffb4c6a5523b in_seq=0 out_seq=0 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.709+0000 7f0ce4c00640 10 client.15723 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0ce4c00640 1 -- 172.21.15.46:0/3230430671 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd0d3bfe60 con 0x55fd0d4d8870 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0ce4c00640 10 client.15723 handle_command_reply: tid=0 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 client.15723 shutdown 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 -- 172.21.15.46:0/3230430671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 msgr2=0x55fd0d461410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d461410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009f10 tx=0x7f0cf0009c20 comp rx=0 tx=0).stop 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 -- 172.21.15.46:0/3230430671 shutdown_connections 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd0d4d8870 0x55fd0d4dac60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd0d45d020 0x55fd0d4d6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd0d452760 0x55fd0d461950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 --2- 172.21.15.46:0/3230430671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd0d451e40 0x55fd0d461410 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 -- 172.21.15.46:0/3230430671 >> 172.21.15.46:0/3230430671 conn(0x55fd0d44e3d0 msgr2=0x55fd0d44d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:18.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0ce4c00640 0 client.15723 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:38:18.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 -- 172.21.15.46:0/3230430671 shutdown_connections 2024-09-10T07:38:18.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 1 -- 172.21.15.46:0/3230430671 wait complete. 2024-09-10T07:38:18.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.710+0000 7f0d00862740 20 client.15723 trim_cache size 0 max 16384 2024-09-10T07:38:18.713 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 -- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0cdc0a2c00 msgr2=0x7f0cdc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0cdc0a2c00 0x7f0cdc0a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ce809b8c0 tx=0x7f0ce80d5530 comp rx=0 tx=0).stop 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 -- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf8103e30 msgr2=0x7f0cf819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf8103e30 0x7f0cf819a460 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800e9f0 tx=0x7f0ce800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 -- 172.21.15.46:0/524122818 shutdown_connections 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf8108f60 0x7f0cf819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf81047e0 0x7f0cf819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0cdc0a2c00 0x7f0cdc0a50c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 --2- 172.21.15.46:0/524122818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf8103e30 0x7f0cf819a460 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 -- 172.21.15.46:0/524122818 >> 172.21.15.46:0/524122818 conn(0x7f0cf80ff8d0 msgr2=0x7f0cf8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:18.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 -- 172.21.15.46:0/524122818 shutdown_connections 2024-09-10T07:38:18.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:18.711+0000 7f0cf7e00640 1 -- 172.21.15.46:0/524122818 wait complete. 2024-09-10T07:38:19.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:38:20.079 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:20.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:20 smithi046 ceph-mon[30705]: pgmap v450: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 277 op/s 2024-09-10T07:38:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 60.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:38:20.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:20 smithi181 ceph-mon[35397]: pgmap v450: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 277 op/s 2024-09-10T07:38:20.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:20 smithi063 ceph-mon[35020]: pgmap v450: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 277 op/s 2024-09-10T07:38:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.654+0000 7f4d3d800640 1 -- 172.21.15.46:0/895133287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074f40 msgr2=0x7f4d3810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.654+0000 7f4d3d800640 1 --2- 172.21.15.46:0/895133287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074f40 0x7f4d3810de80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009900 tx=0x7f4d2402a480 comp rx=0 tx=0).stop 2024-09-10T07:38:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.655+0000 7f4d3d800640 1 -- 172.21.15.46:0/895133287 shutdown_connections 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.655+0000 7f4d3d800640 1 --2- 172.21.15.46:0/895133287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d3810e3c0 0x7f4d38110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.655+0000 7f4d3d800640 1 --2- 172.21.15.46:0/895133287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074f40 0x7f4d3810de80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.655+0000 7f4d3d800640 1 --2- 172.21.15.46:0/895133287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d38074620 0x7f4d38074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.655+0000 7f4d3d800640 1 -- 172.21.15.46:0/895133287 >> 172.21.15.46:0/895133287 conn(0x7f4d380fd8c0 msgr2=0x7f4d380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.655+0000 7f4d3d800640 1 -- 172.21.15.46:0/895133287 shutdown_connections 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.655+0000 7f4d3d800640 1 -- 172.21.15.46:0/895133287 wait complete. 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.655+0000 7f4d3d800640 1 Processor -- start 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d3d800640 1 -- start start 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d3d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074620 0x7f4d3819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d3d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d38074f40 0x7f4d3819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d3d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d3810e3c0 0x7f4d3819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d3d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38114140 con 0x7f4d38074620 2024-09-10T07:38:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d3d800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38113fc0 con 0x7f4d38074f40 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d3d800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381142c0 con 0x7f4d3810e3c0 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074620 0x7f4d3819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d36000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d38074f40 0x7f4d3819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074620 0x7f4d3819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36900/0 (socket says 172.21.15.46:36900) 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d36a00640 1 -- 172.21.15.46:0/2571616433 learned_addr learned my addr 172.21.15.46:0/2571616433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.656+0000 7f4d37400640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d3810e3c0 0x7f4d3819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d36000640 1 -- 172.21.15.46:0/2571616433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d3810e3c0 msgr2=0x7f4d3819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d36000640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d3810e3c0 0x7f4d3819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d36000640 1 -- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074620 msgr2=0x7f4d3819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d36000640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074620 0x7f4d3819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d36000640 1 -- 172.21.15.46:0/2571616433 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3819f690 con 0x7f4d38074f40 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d36000640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d38074f40 0x7f4d3819ac20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4d24005940 tx=0x7f4d240040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d2f400640 1 -- 172.21.15.46:0/2571616433 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2403f030 con 0x7f4d38074f40 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d3d800640 1 -- 172.21.15.46:0/2571616433 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3819f920 con 0x7f4d38074f40 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d2f400640 1 -- 172.21.15.46:0/2571616433 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d240044c0 con 0x7f4d38074f40 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d2f400640 1 -- 172.21.15.46:0/2571616433 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24035070 con 0x7f4d38074f40 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.657+0000 7f4d3d800640 1 -- 172.21.15.46:0/2571616433 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d381a7260 con 0x7f4d38074f40 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.660+0000 7f4d402b8740 1 Processor -- start 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.660+0000 7f4d402b8740 1 -- start start 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d2f400640 1 -- 172.21.15.46:0/2571616433 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d24042030 con 0x7f4d38074f40 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034742050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 0x56503474f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d402b8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5650346a83e0 con 0x565034741c70 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d402b8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56503464fd80 con 0x565034742590 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d402b8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5650346a43a0 con 0x56503474ce50 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034742050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38578/0 (socket says 172.21.15.46:38578) 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d36a00640 1 -- 172.21.15.46:0/3358599941 learned_addr learned my addr 172.21.15.46:0/3358599941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.661+0000 7f4d36000640 1 --2- 172.21.15.46:0/3358599941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 0x56503474f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d36a00640 1 -- 172.21.15.46:0/3358599941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 msgr2=0x56503474f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d36a00640 1 --2- 172.21.15.46:0/3358599941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 0x56503474f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d36a00640 1 -- 172.21.15.46:0/3358599941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 msgr2=0x565034742050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d36a00640 1 --2- 172.21.15.46:0/3358599941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034742050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d36a00640 1 -- 172.21.15.46:0/3358599941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503466eae0 con 0x565034742590 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d36a00640 1 --2- 172.21.15.46:0/3358599941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474c910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009dc0 tx=0x7f4d28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=504015f2fdc30a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d2f400640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d23e00640 1 -- 172.21.15.46:0/3358599941 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28005220 con 0x565034742590 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d23e00640 1 -- 172.21.15.46:0/3358599941 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d28004c80 con 0x565034742590 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d23e00640 1 -- 172.21.15.46:0/3358599941 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28019910 con 0x565034742590 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d2f400640 1 -- 172.21.15.46:0/2571616433 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d240ef340 con 0x7f4d38074f40 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.662+0000 7f4d37400640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 -- 172.21.15.46:0/3358599941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 msgr2=0x56503474c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 --2- 172.21.15.46:0/3358599941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474c910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009dc0 tx=0x7f4d28011810 comp rx=0 tx=0).stop 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 -- 172.21.15.46:0/3358599941 shutdown_connections 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 --2- 172.21.15.46:0/3358599941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 0x56503474f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 --2- 172.21.15.46:0/3358599941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474c910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 --2- 172.21.15.46:0/3358599941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034742050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 -- 172.21.15.46:0/3358599941 >> 172.21.15.46:0/3358599941 conn(0x56503473e200 msgr2=0x56503473e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d37400640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d30005fd0 tx=0x7f4d30009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 -- 172.21.15.46:0/3358599941 shutdown_connections 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.663+0000 7f4d402b8740 1 -- 172.21.15.46:0/3358599941 wait complete. 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 1 -- start start 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 1 Processor -- start 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034749cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 0x56503474a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56503474fa20 con 0x565034741c70 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5650346630f0 con 0x565034742590 2024-09-10T07:38:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56503474fba0 con 0x56503474ce50 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.664+0000 7f4d402b8740 10 client.0 _ll_register_callbacks cb 0x56503464abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034749cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034749cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36918/0 (socket says 172.21.15.46:36918) 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36a00640 1 -- 172.21.15.46:0/69393673 learned_addr learned my addr 172.21.15.46:0/69393673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36000640 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d37400640 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 0x56503474a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36a00640 1 -- 172.21.15.46:0/69393673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 msgr2=0x56503474a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36a00640 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 0x56503474a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36a00640 1 -- 172.21.15.46:0/69393673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 msgr2=0x56503474a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36a00640 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36a00640 1 -- 172.21.15.46:0/69393673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650344193c0 con 0x565034741c70 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.665+0000 7f4d36000640 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.666+0000 7f4d36a00640 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034749cb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4d28019510 tx=0x7f4d280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26141a604b4f6a1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.666+0000 7f4d20c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.666+0000 7f4d20c00640 1 -- 172.21.15.46:0/69393673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28027070 con 0x565034741c70 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.666+0000 7f4d20c00640 1 -- 172.21.15.46:0/69393673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d28004450 con 0x565034741c70 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.666+0000 7f4d402b8740 1 -- 172.21.15.46:0/69393673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d28004c80 con 0x565034741c70 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.666+0000 7f4d20c00640 1 -- 172.21.15.46:0/69393673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4d28033050 con 0x565034741c70 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.666+0000 7f4d402b8740 10 client.15735 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.666+0000 7f4d402b8740 1 -- 172.21.15.46:0/69393673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56503475ba10 con 0x565034741c70 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d20c00640 1 -- 172.21.15.46:0/69393673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d28004c80 con 0x565034741c70 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d20c00640 1 -- 172.21.15.46:0/69393673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f4d28019c50 con 0x565034741c70 2024-09-10T07:38:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d402b8740 10 client.15735 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d402b8740 10 client.15735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d402b8740 20 client.15735 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d402b8740 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x56503475cd60 0x56503475f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d402b8740 4 client.15735 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d402b8740 1 -- 172.21.15.46:0/69393673 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650346b5950 con 0x56503475cd60 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.667+0000 7f4d36000640 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x56503475cd60 0x56503475f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.668+0000 7f4d36000640 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x56503475cd60 0x56503475f150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f22afcb5ffca6f9 server_cookie=652ff7344b5ec111 in_seq=0 out_seq=0 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.668+0000 7f4d20c00640 10 client.15735 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d20c00640 1 -- 172.21.15.46:0/69393673 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650346b5950 con 0x56503475cd60 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d20c00640 10 client.15735 handle_command_reply: tid=0 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 client.15735 shutdown 2024-09-10T07:38:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 -- 172.21.15.46:0/69393673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 msgr2=0x565034749cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034749cb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4d28019510 tx=0x7f4d280048f0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 -- 172.21.15.46:0/69393673 shutdown_connections 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x56503475cd60 0x56503475f150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56503474ce50 0x56503474a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034742590 0x56503474a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 --2- 172.21.15.46:0/69393673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034741c70 0x565034749cb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 -- 172.21.15.46:0/69393673 >> 172.21.15.46:0/69393673 conn(0x56503473e200 msgr2=0x56503473d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d20c00640 0 client.15735 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 -- 172.21.15.46:0/69393673 shutdown_connections 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 1 -- 172.21.15.46:0/69393673 wait complete. 2024-09-10T07:38:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.669+0000 7f4d402b8740 20 client.15735 trim_cache size 0 max 16384 2024-09-10T07:38:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.683+0000 7f4d402b8740 1 Processor -- start 2024-09-10T07:38:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.683+0000 7f4d402b8740 1 -- start start 2024-09-10T07:38:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034741c70 0x565034764960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034742590 0x565034764ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503474ce50 0x56503473cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d402b8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b0ec0 con 0x56503474ce50 2024-09-10T07:38:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d402b8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b2ac0 con 0x565034741c70 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d402b8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b1590 con 0x565034742590 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503474ce50 0x56503473cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034741c70 0x565034764960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034741c70 0x565034764960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38598/0 (socket says 172.21.15.46:38598) 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503474ce50 0x56503473cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36934/0 (socket says 172.21.15.46:36934) 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d37400640 1 -- 172.21.15.46:0/3962137135 learned_addr learned my addr 172.21.15.46:0/3962137135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.684+0000 7f4d36000640 1 --2- 172.21.15.46:0/3962137135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034742590 0x565034764ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d36a00640 1 -- 172.21.15.46:0/3962137135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034742590 msgr2=0x565034764ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d36a00640 1 --2- 172.21.15.46:0/3962137135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034742590 0x565034764ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d36a00640 1 -- 172.21.15.46:0/3962137135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503474ce50 msgr2=0x56503473cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d36a00640 1 --2- 172.21.15.46:0/3962137135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503474ce50 0x56503473cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d36a00640 1 -- 172.21.15.46:0/3962137135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503475f730 con 0x565034741c70 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d36a00640 1 --2- 172.21.15.46:0/3962137135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034741c70 0x565034764960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4d280037e0 tx=0x7f4d280059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c7a58f2370ba72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d23400640 1 -- 172.21.15.46:0/3962137135 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2803c070 con 0x565034741c70 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d23400640 1 -- 172.21.15.46:0/3962137135 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d28005380 con 0x565034741c70 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.685+0000 7f4d23400640 1 -- 172.21.15.46:0/3962137135 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28027050 con 0x565034741c70 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 -- 172.21.15.46:0/3962137135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034741c70 msgr2=0x565034764960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 --2- 172.21.15.46:0/3962137135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034741c70 0x565034764960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4d280037e0 tx=0x7f4d280059b0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 -- 172.21.15.46:0/3962137135 shutdown_connections 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 --2- 172.21.15.46:0/3962137135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503474ce50 0x56503473cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 --2- 172.21.15.46:0/3962137135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034742590 0x565034764ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 --2- 172.21.15.46:0/3962137135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565034741c70 0x565034764960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 -- 172.21.15.46:0/3962137135 >> 172.21.15.46:0/3962137135 conn(0x56503473e200 msgr2=0x565034748da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 -- 172.21.15.46:0/3962137135 shutdown_connections 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.686+0000 7f4d402b8740 1 -- 172.21.15.46:0/3962137135 wait complete. 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 1 -- start start 2024-09-10T07:38:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 1 Processor -- start 2024-09-10T07:38:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034741c70 0x565034750380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034742590 0x5650347508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56503474ce50 0x5650347c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b2330 con 0x565034742590 2024-09-10T07:38:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381b1c60 con 0x56503474ce50 2024-09-10T07:38:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381aeea0 con 0x565034741c70 2024-09-10T07:38:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.687+0000 7f4d402b8740 10 client.0 _ll_register_callbacks cb 0x7f4d30009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d36000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034742590 0x5650347508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56503474ce50 0x5650347c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034741c70 0x565034750380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56503474ce50 0x5650347c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38608/0 (socket says 172.21.15.46:38608) 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d36000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034742590 0x5650347508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36936/0 (socket says 172.21.15.46:36936) 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 -- 172.21.15.46:0/2589360406 learned_addr learned my addr 172.21.15.46:0/2589360406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 -- 172.21.15.46:0/2589360406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034741c70 msgr2=0x565034750380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034741c70 0x565034750380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 -- 172.21.15.46:0/2589360406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034742590 msgr2=0x5650347508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034742590 0x5650347508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 -- 172.21.15.46:0/2589360406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565034739770 con 0x56503474ce50 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d36000640 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034742590 0x5650347508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d37400640 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56503474ce50 0x5650347c5920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009f10 tx=0x7f4d30004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26267212337cd9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.688+0000 7f4d20c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d20c00640 1 -- 172.21.15.46:0/2589360406 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30004b20 con 0x56503474ce50 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 1 -- 172.21.15.46:0/2589360406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56503473edb0 con 0x56503474ce50 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d20c00640 1 -- 172.21.15.46:0/2589360406 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d30004cc0 con 0x56503474ce50 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d20c00640 1 -- 172.21.15.46:0/2589360406 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4d30034050 con 0x56503474ce50 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 10 client.25838 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 1 -- 172.21.15.46:0/2589360406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56503473e030 con 0x56503474ce50 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d20c00640 1 -- 172.21.15.46:0/2589360406 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56503473edb0 con 0x56503474ce50 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d20c00640 1 -- 172.21.15.46:0/2589360406 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f4d300056a0 con 0x56503474ce50 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 10 client.25838 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 10 client.25838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 20 client.25838 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5650347c76d0 0x5650347c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 4 client.25838 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.689+0000 7f4d402b8740 1 -- 172.21.15.46:0/2589360406 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650346afe60 con 0x5650347c76d0 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.690+0000 7f4d36a00640 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5650347c76d0 0x5650347c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.690+0000 7f4d36a00640 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5650347c76d0 0x5650347c9af0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f79ed133d8f9e52a server_cookie=ed6ac7e9452dca04 in_seq=0 out_seq=0 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.690+0000 7f4d20c00640 10 client.25838 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d20c00640 1 -- 172.21.15.46:0/2589360406 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650346afe60 con 0x5650347c76d0 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d20c00640 10 client.25838 handle_command_reply: tid=0 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 client.25838 shutdown 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 -- 172.21.15.46:0/2589360406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56503474ce50 msgr2=0x5650347c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56503474ce50 0x5650347c5920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009f10 tx=0x7f4d30004a40 comp rx=0 tx=0).stop 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 -- 172.21.15.46:0/2589360406 shutdown_connections 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5650347c76d0 0x5650347c9af0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56503474ce50 0x5650347c5920 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565034742590 0x5650347508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 --2- 172.21.15.46:0/2589360406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565034741c70 0x565034750380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d20c00640 0 client.25838 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 -- 172.21.15.46:0/2589360406 >> 172.21.15.46:0/2589360406 conn(0x56503473e200 msgr2=0x565034749c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 -- 172.21.15.46:0/2589360406 shutdown_connections 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 1 -- 172.21.15.46:0/2589360406 wait complete. 2024-09-10T07:38:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.691+0000 7f4d402b8740 20 client.25838 trim_cache size 0 max 16384 2024-09-10T07:38:20.697 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:38:20.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 -- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d1c0a2c00 msgr2=0x7f4d1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d30005fd0 tx=0x7f4d30009500 comp rx=0 tx=0).stop 2024-09-10T07:38:20.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 -- 172.21.15.46:0/2571616433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d38074f40 msgr2=0x7f4d3819ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:20.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d38074f40 0x7f4d3819ac20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4d24005940 tx=0x7f4d240040c0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 -- 172.21.15.46:0/2571616433 shutdown_connections 2024-09-10T07:38:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d3810e3c0 0x7f4d3819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d38074f40 0x7f4d3819ac20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d1c0a2c00 0x7f4d1c0a50c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 --2- 172.21.15.46:0/2571616433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d38074620 0x7f4d3819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 -- 172.21.15.46:0/2571616433 >> 172.21.15.46:0/2571616433 conn(0x7f4d380fd8c0 msgr2=0x7f4d380fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 -- 172.21.15.46:0/2571616433 shutdown_connections 2024-09-10T07:38:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:20.692+0000 7f4d3d800640 1 -- 172.21.15.46:0/2571616433 wait complete. 2024-09-10T07:38:21.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:38:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:22.450 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:22 smithi046 ceph-mon[30705]: pgmap v451: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 220 op/s 2024-09-10T07:38:22.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:22 smithi181 ceph-mon[35397]: pgmap v451: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 220 op/s 2024-09-10T07:38:22.653 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:22 smithi063 ceph-mon[35020]: pgmap v451: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 220 op/s 2024-09-10T07:38:22.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.707+0000 7f9e05200640 1 -- 172.21.15.46:0/3802368241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074510 msgr2=0x7f9e000748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.707+0000 7f9e05200640 1 --2- 172.21.15.46:0/3802368241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074510 0x7f9e000748f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009900 tx=0x7f9df002a480 comp rx=0 tx=0).stop 2024-09-10T07:38:22.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.707+0000 7f9e05200640 1 -- 172.21.15.46:0/3802368241 shutdown_connections 2024-09-10T07:38:22.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.707+0000 7f9e05200640 1 --2- 172.21.15.46:0/3802368241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e0010e280 0x7f9e00110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.707+0000 7f9e05200640 1 --2- 172.21.15.46:0/3802368241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e00074e30 0x7f9e0010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.707+0000 7f9e05200640 1 --2- 172.21.15.46:0/3802368241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074510 0x7f9e000748f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.707+0000 7f9e05200640 1 -- 172.21.15.46:0/3802368241 >> 172.21.15.46:0/3802368241 conn(0x7f9e000fd740 msgr2=0x7f9e000ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:22.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.708+0000 7f9e05200640 1 -- 172.21.15.46:0/3802368241 shutdown_connections 2024-09-10T07:38:22.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.708+0000 7f9e05200640 1 -- 172.21.15.46:0/3802368241 wait complete. 2024-09-10T07:38:22.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.709+0000 7f9e05200640 1 Processor -- start 2024-09-10T07:38:22.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.709+0000 7f9e05200640 1 -- start start 2024-09-10T07:38:22.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.709+0000 7f9e05200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e00074510 0x7f9e0019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.709+0000 7f9e05200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074e30 0x7f9e0019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.709+0000 7f9e05200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e0010e280 0x7f9e0019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.709+0000 7f9e05200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00113f70 con 0x7f9e00074e30 2024-09-10T07:38:22.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.709+0000 7f9e05200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00113df0 con 0x7f9e0010e280 2024-09-10T07:38:22.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.709+0000 7f9e05200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001140f0 con 0x7f9e00074510 2024-09-10T07:38:22.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dfea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074e30 0x7f9e0019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e00074510 0x7f9e0019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e00074510 0x7f9e0019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60138/0 (socket says 172.21.15.46:60138) 2024-09-10T07:38:22.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dfea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074e30 0x7f9e0019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36948/0 (socket says 172.21.15.46:36948) 2024-09-10T07:38:22.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dff400640 1 -- 172.21.15.46:0/3817587877 learned_addr learned my addr 172.21.15.46:0/3817587877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:22.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dffe00640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e0010e280 0x7f9e0019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dff400640 1 -- 172.21.15.46:0/3817587877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e0010e280 msgr2=0x7f9e0019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dff400640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e0010e280 0x7f9e0019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dff400640 1 -- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074e30 msgr2=0x7f9e0019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dff400640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074e30 0x7f9e0019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dff400640 1 -- 172.21.15.46:0/3817587877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0019f420 con 0x7f9e00074510 2024-09-10T07:38:22.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.710+0000 7f9dfea00640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074e30 0x7f9e0019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:22.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.711+0000 7f9dff400640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e00074510 0x7f9e0019a470 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006010 tx=0x7f9df00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:22.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.711+0000 7f9df7e00640 1 -- 172.21.15.46:0/3817587877 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df003f030 con 0x7f9e00074510 2024-09-10T07:38:22.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.711+0000 7f9e05200640 1 -- 172.21.15.46:0/3817587877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0019f6b0 con 0x7f9e00074510 2024-09-10T07:38:22.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.711+0000 7f9df7e00640 1 -- 172.21.15.46:0/3817587877 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df0003ce0 con 0x7f9e00074510 2024-09-10T07:38:22.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.711+0000 7f9df7e00640 1 -- 172.21.15.46:0/3817587877 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0035070 con 0x7f9e00074510 2024-09-10T07:38:22.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.711+0000 7f9e05200640 1 -- 172.21.15.46:0/3817587877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e001a6ff0 con 0x7f9e00074510 2024-09-10T07:38:22.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.714+0000 7f9df7e00640 1 -- 172.21.15.46:0/3817587877 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9df0042030 con 0x7f9e00074510 2024-09-10T07:38:22.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.715+0000 7f9e09474740 1 Processor -- start 2024-09-10T07:38:22.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9e09474740 1 -- start start 2024-09-10T07:38:22.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9df7e00640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9de40a2c00 0x7f9de40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9df7e00640 1 -- 172.21.15.46:0/3817587877 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9df00ef3a0 con 0x7f9e00074510 2024-09-10T07:38:22.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9e09474740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e16e40 0x55a307e17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9e09474740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e17760 0x55a307e21ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9e09474740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e24410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9e09474740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a307d7d3e0 con 0x55a307e17760 2024-09-10T07:38:22.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9e09474740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a307d24d80 con 0x55a307e22020 2024-09-10T07:38:22.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9e09474740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a307d793a0 con 0x55a307e16e40 2024-09-10T07:38:22.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.716+0000 7f9dffe00640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9de40a2c00 0x7f9de40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e16e40 0x55a307e17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e24410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e16e40 0x55a307e17220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60142/0 (socket says 172.21.15.46:60142) 2024-09-10T07:38:22.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dff400640 1 -- 172.21.15.46:0/2994556905 learned_addr learned my addr 172.21.15.46:0/2994556905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:22.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dfea00640 1 --2- 172.21.15.46:0/2994556905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e17760 0x55a307e21ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dff400640 1 -- 172.21.15.46:0/2994556905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 msgr2=0x55a307e24410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:22.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dff400640 1 --2- 172.21.15.46:0/2994556905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e24410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dff400640 1 -- 172.21.15.46:0/2994556905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e17760 msgr2=0x55a307e21ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:22.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dff400640 1 --2- 172.21.15.46:0/2994556905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e17760 0x55a307e21ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.717+0000 7f9dff400640 1 -- 172.21.15.46:0/2994556905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a307d43ae0 con 0x55a307e16e40 2024-09-10T07:38:22.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.718+0000 7f9dff400640 1 --2- 172.21.15.46:0/2994556905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e16e40 0x55a307e17220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9df00eca30 tx=0x7f9df0031a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d2b80f4b1c92e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:22.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.718+0000 7f9df4c00640 1 -- 172.21.15.46:0/2994556905 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0049070 con 0x55a307e16e40 2024-09-10T07:38:22.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.718+0000 7f9df4c00640 1 -- 172.21.15.46:0/2994556905 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df00bb990 con 0x55a307e16e40 2024-09-10T07:38:22.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.718+0000 7f9df4c00640 1 -- 172.21.15.46:0/2994556905 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df00c40c0 con 0x55a307e16e40 2024-09-10T07:38:22.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 -- 172.21.15.46:0/2994556905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e16e40 msgr2=0x55a307e17220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 --2- 172.21.15.46:0/2994556905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e16e40 0x55a307e17220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9df00eca30 tx=0x7f9df0031a60 comp rx=0 tx=0).stop 2024-09-10T07:38:22.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9dffe00640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9de40a2c00 0x7f9de40a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009fd0 tx=0x7f9df8009d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:22.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 -- 172.21.15.46:0/2994556905 shutdown_connections 2024-09-10T07:38:22.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 --2- 172.21.15.46:0/2994556905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 --2- 172.21.15.46:0/2994556905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e17760 0x55a307e21ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 --2- 172.21.15.46:0/2994556905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e16e40 0x55a307e17220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 -- 172.21.15.46:0/2994556905 >> 172.21.15.46:0/2994556905 conn(0x55a307e133d0 msgr2=0x55a307e137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:22.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 -- 172.21.15.46:0/2994556905 shutdown_connections 2024-09-10T07:38:22.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 -- 172.21.15.46:0/2994556905 wait complete. 2024-09-10T07:38:22.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.719+0000 7f9e09474740 1 -- start start 2024-09-10T07:38:22.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.720+0000 7f9e09474740 1 Processor -- start 2024-09-10T07:38:22.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.720+0000 7f9e09474740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e1efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.720+0000 7f9e09474740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 0x55a307e1f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.720+0000 7f9e09474740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e1fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.720+0000 7f9e09474740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a307e249b0 con 0x55a307e16e40 2024-09-10T07:38:22.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.720+0000 7f9e09474740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a307d380f0 con 0x55a307e22020 2024-09-10T07:38:22.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.720+0000 7f9e09474740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a307e24b30 con 0x55a307e17760 2024-09-10T07:38:22.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.720+0000 7f9e09474740 10 client.0 _ll_register_callbacks cb 0x55a307d1fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:22.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e1efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dfea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 0x55a307e1f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e1efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36972/0 (socket says 172.21.15.46:36972) 2024-09-10T07:38:22.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 -- 172.21.15.46:0/1502823392 learned_addr learned my addr 172.21.15.46:0/1502823392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:22.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dffe00640 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e1fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 -- 172.21.15.46:0/1502823392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 msgr2=0x55a307e1f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:22.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 0x55a307e1f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 -- 172.21.15.46:0/1502823392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 msgr2=0x55a307e1fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:22.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e1fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 -- 172.21.15.46:0/1502823392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a307aee3c0 con 0x55a307e16e40 2024-09-10T07:38:22.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9dff400640 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e1efc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9df00c4710 tx=0x7f9df00b9e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8e0773a4e0eea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:22.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.721+0000 7f9de9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:22.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.722+0000 7f9de9600640 1 -- 172.21.15.46:0/1502823392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0049070 con 0x55a307e16e40 2024-09-10T07:38:22.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.722+0000 7f9de9600640 1 -- 172.21.15.46:0/1502823392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9df00c42c0 con 0x55a307e16e40 2024-09-10T07:38:22.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.722+0000 7f9e09474740 1 -- 172.21.15.46:0/1502823392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df00bb990 con 0x55a307e16e40 2024-09-10T07:38:22.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.722+0000 7f9de9600640 1 -- 172.21.15.46:0/1502823392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9df00044e0 con 0x55a307e16e40 2024-09-10T07:38:22.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.722+0000 7f9e09474740 10 client.15753 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:22.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.722+0000 7f9e09474740 1 -- 172.21.15.46:0/1502823392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a307e249b0 con 0x55a307e16e40 2024-09-10T07:38:22.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.722+0000 7f9de9600640 1 -- 172.21.15.46:0/1502823392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df00bb990 con 0x55a307e16e40 2024-09-10T07:38:22.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.723+0000 7f9de9600640 1 -- 172.21.15.46:0/1502823392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f9df00a60e0 con 0x55a307e16e40 2024-09-10T07:38:22.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.723+0000 7f9e09474740 10 client.15753 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:22.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.723+0000 7f9e09474740 10 client.15753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:38:22.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.723+0000 7f9e09474740 20 client.15753 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:22.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.723+0000 7f9e09474740 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a307e31c10 0x55a307e34050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.723+0000 7f9e09474740 4 client.15753 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:22.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.723+0000 7f9e09474740 1 -- 172.21.15.46:0/1502823392 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a307d8a950 con 0x55a307e31c10 2024-09-10T07:38:22.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.723+0000 7f9dfea00640 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a307e31c10 0x55a307e34050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.724+0000 7f9dfea00640 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a307e31c10 0x55a307e34050 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72381083b4aad49 server_cookie=107f8370051ea0f3 in_seq=0 out_seq=0 2024-09-10T07:38:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.724+0000 7f9de9600640 10 client.15753 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:38:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9de9600640 1 -- 172.21.15.46:0/1502823392 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a307d8a950 con 0x55a307e31c10 2024-09-10T07:38:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9de9600640 10 client.15753 handle_command_reply: tid=0 2024-09-10T07:38:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 client.15753 shutdown 2024-09-10T07:38:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 -- 172.21.15.46:0/1502823392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 msgr2=0x55a307e1efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e1efc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9df00c4710 tx=0x7f9df00b9e30 comp rx=0 tx=0).stop 2024-09-10T07:38:22.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 -- 172.21.15.46:0/1502823392 shutdown_connections 2024-09-10T07:38:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a307e31c10 0x55a307e34050 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e1fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 0x55a307e1f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 --2- 172.21.15.46:0/1502823392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e1efc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9de9600640 0 client.15753 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:38:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 -- 172.21.15.46:0/1502823392 >> 172.21.15.46:0/1502823392 conn(0x55a307e133d0 msgr2=0x55a307e19100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 -- 172.21.15.46:0/1502823392 shutdown_connections 2024-09-10T07:38:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.725+0000 7f9e09474740 1 -- 172.21.15.46:0/1502823392 wait complete. 2024-09-10T07:38:22.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.726+0000 7f9e09474740 20 client.15753 trim_cache size 0 max 16384 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9e09474740 1 Processor -- start 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9e09474740 1 -- start start 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9e09474740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e0e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9e09474740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e17760 0x55a307e19860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9e09474740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e22020 0x55a307e39920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9e09474740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001b0d10 con 0x55a307e16e40 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9e09474740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001b2910 con 0x55a307e17760 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9e09474740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001b13e0 con 0x55a307e22020 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9dff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e0e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9dff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e0e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36978/0 (socket says 172.21.15.46:36978) 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9dff400640 1 -- 172.21.15.46:0/2249653953 learned_addr learned my addr 172.21.15.46:0/2249653953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9dfea00640 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e17760 0x55a307e19860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.740+0000 7f9dffe00640 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e22020 0x55a307e39920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.741+0000 7f9dfea00640 1 -- 172.21.15.46:0/2249653953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e22020 msgr2=0x55a307e39920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.741+0000 7f9dfea00640 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e22020 0x55a307e39920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.741+0000 7f9dfea00640 1 -- 172.21.15.46:0/2249653953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 msgr2=0x55a307e0e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.741+0000 7f9dfea00640 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e0e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.741+0000 7f9dfea00640 1 -- 172.21.15.46:0/2249653953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a307e1df70 con 0x55a307e17760 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.741+0000 7f9dff400640 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e0e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.741+0000 7f9dfea00640 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e17760 0x55a307e19860 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55a307e348c0 tx=0x7f9dec00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e13673ad76f2b027 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.741+0000 7f9debe00640 1 -- 172.21.15.46:0/2249653953 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec023070 con 0x55a307e17760 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9debe00640 1 -- 172.21.15.46:0/2249653953 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dec0108e0 con 0x55a307e17760 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9debe00640 1 -- 172.21.15.46:0/2249653953 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0054b0 con 0x55a307e17760 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9e09474740 1 -- 172.21.15.46:0/2249653953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e17760 msgr2=0x55a307e19860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9e09474740 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e17760 0x55a307e19860 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55a307e348c0 tx=0x7f9dec00b390 comp rx=0 tx=0).stop 2024-09-10T07:38:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9e09474740 1 -- 172.21.15.46:0/2249653953 shutdown_connections 2024-09-10T07:38:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9e09474740 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e22020 0x55a307e39920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9e09474740 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e17760 0x55a307e19860 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9e09474740 1 --2- 172.21.15.46:0/2249653953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e0e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9e09474740 1 -- 172.21.15.46:0/2249653953 >> 172.21.15.46:0/2249653953 conn(0x55a307e133d0 msgr2=0x55a307e2d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.742+0000 7f9e09474740 1 -- 172.21.15.46:0/2249653953 shutdown_connections 2024-09-10T07:38:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.743+0000 7f9e09474740 1 -- 172.21.15.46:0/2249653953 wait complete. 2024-09-10T07:38:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.743+0000 7f9e09474740 1 -- start start 2024-09-10T07:38:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.743+0000 7f9e09474740 1 Processor -- start 2024-09-10T07:38:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9e09474740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e2a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9e09474740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 0x55a307e255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9e09474740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e25ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9e09474740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001b2180 con 0x55a307e16e40 2024-09-10T07:38:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9e09474740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001b1ab0 con 0x55a307e22020 2024-09-10T07:38:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9e09474740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001aecf0 con 0x55a307e17760 2024-09-10T07:38:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9e09474740 10 client.0 _ll_register_callbacks cb 0x7f9df8013740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9dff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e2a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9dfea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 0x55a307e255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9dff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e2a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36984/0 (socket says 172.21.15.46:36984) 2024-09-10T07:38:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9dff400640 1 -- 172.21.15.46:0/3170915427 learned_addr learned my addr 172.21.15.46:0/3170915427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.744+0000 7f9dffe00640 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e25ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9dff400640 1 -- 172.21.15.46:0/3170915427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 msgr2=0x55a307e255a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9dff400640 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 0x55a307e255a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9dff400640 1 -- 172.21.15.46:0/3170915427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 msgr2=0x55a307e25ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9dff400640 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e25ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9dff400640 1 -- 172.21.15.46:0/3170915427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a307e260d0 con 0x55a307e16e40 2024-09-10T07:38:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9dff400640 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e2a220 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9df00c4710 tx=0x7f9df003bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea45ada16018ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9de9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9de9600640 1 -- 172.21.15.46:0/3170915427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0049070 con 0x55a307e16e40 2024-09-10T07:38:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9e09474740 1 -- 172.21.15.46:0/3170915427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dec0108e0 con 0x55a307e16e40 2024-09-10T07:38:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.745+0000 7f9de9600640 1 -- 172.21.15.46:0/3170915427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9df00c42c0 con 0x55a307e16e40 2024-09-10T07:38:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9de9600640 1 -- 172.21.15.46:0/3170915427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9df00bb4d0 con 0x55a307e16e40 2024-09-10T07:38:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9e09474740 10 client.15759 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9e09474740 1 -- 172.21.15.46:0/3170915427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a307e260d0 con 0x55a307e16e40 2024-09-10T07:38:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9de9600640 1 -- 172.21.15.46:0/3170915427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dec0108e0 con 0x55a307e16e40 2024-09-10T07:38:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9de9600640 1 -- 172.21.15.46:0/3170915427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f9df00b6080 con 0x55a307e16e40 2024-09-10T07:38:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9e09474740 10 client.15759 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9e09474740 10 client.15759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:38:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9e09474740 20 client.15759 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9e09474740 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a307e9c9e0 0x55a307e9edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9e09474740 4 client.15759 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:38:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.746+0000 7f9e09474740 1 -- 172.21.15.46:0/3170915427 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a307d84e60 con 0x55a307e9c9e0 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.747+0000 7f9dfea00640 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a307e9c9e0 0x55a307e9edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.747+0000 7f9dfea00640 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a307e9c9e0 0x55a307e9edd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e8f5d3df22977fc server_cookie=5a48cf835b3da9b8 in_seq=0 out_seq=0 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.747+0000 7f9de9600640 10 client.15759 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.747+0000 7f9de9600640 1 -- 172.21.15.46:0/3170915427 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a307d84e60 con 0x55a307e9c9e0 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.747+0000 7f9de9600640 10 client.15759 handle_command_reply: tid=0 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.747+0000 7f9e09474740 1 client.15759 shutdown 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 -- 172.21.15.46:0/3170915427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 msgr2=0x55a307e2a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e2a220 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9df00c4710 tx=0x7f9df003bde0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 -- 172.21.15.46:0/3170915427 shutdown_connections 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a307e9c9e0 0x55a307e9edd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a307e22020 0x55a307e25ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a307e17760 0x55a307e255a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 --2- 172.21.15.46:0/3170915427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a307e16e40 0x55a307e2a220 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 -- 172.21.15.46:0/3170915427 >> 172.21.15.46:0/3170915427 conn(0x55a307e133d0 msgr2=0x55a307e1ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9de9600640 0 client.15759 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:38:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 -- 172.21.15.46:0/3170915427 shutdown_connections 2024-09-10T07:38:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 1 -- 172.21.15.46:0/3170915427 wait complete. 2024-09-10T07:38:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.748+0000 7f9e09474740 20 client.15759 trim_cache size 0 max 16384 2024-09-10T07:38:22.758 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:38:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 -- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9de40a2c00 msgr2=0x7f9de40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9de40a2c00 0x7f9de40a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009fd0 tx=0x7f9df8009d80 comp rx=0 tx=0).stop 2024-09-10T07:38:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 -- 172.21.15.46:0/3817587877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e00074510 msgr2=0x7f9e0019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e00074510 0x7f9e0019a470 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006010 tx=0x7f9df00042f0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 -- 172.21.15.46:0/3817587877 shutdown_connections 2024-09-10T07:38:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e0010e280 0x7f9e0019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9de40a2c00 0x7f9de40a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e00074e30 0x7f9e0019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 --2- 172.21.15.46:0/3817587877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e00074510 0x7f9e0019a470 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 -- 172.21.15.46:0/3817587877 >> 172.21.15.46:0/3817587877 conn(0x7f9e000fd740 msgr2=0x7f9e000fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 -- 172.21.15.46:0/3817587877 shutdown_connections 2024-09-10T07:38:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:22.749+0000 7f9e05200640 1 -- 172.21.15.46:0/3817587877 wait complete. 2024-09-10T07:38:23.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:38:23.933 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:24.281 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:24 smithi046 ceph-mon[30705]: pgmap v452: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 214 op/s 2024-09-10T07:38:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 -- 172.21.15.46:0/810325955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38077040 msgr2=0x7efc38112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 --2- 172.21.15.46:0/810325955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38077040 0x7efc38112330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efc24009900 tx=0x7efc2402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 -- 172.21.15.46:0/810325955 shutdown_connections 2024-09-10T07:38:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 --2- 172.21.15.46:0/810325955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc38112870 0x7efc38114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 --2- 172.21.15.46:0/810325955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38077040 0x7efc38112330 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 --2- 172.21.15.46:0/810325955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc38076720 0x7efc38076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 -- 172.21.15.46:0/810325955 >> 172.21.15.46:0/810325955 conn(0x7efc380ff910 msgr2=0x7efc38101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 -- 172.21.15.46:0/810325955 shutdown_connections 2024-09-10T07:38:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.524+0000 7efc3e200640 1 -- 172.21.15.46:0/810325955 wait complete. 2024-09-10T07:38:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.525+0000 7efc3e200640 1 Processor -- start 2024-09-10T07:38:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3e200640 1 -- start start 2024-09-10T07:38:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc38076720 0x7efc381a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3e200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc38077040 0x7efc381a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38112870 0x7efc3819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc381183e0 con 0x7efc38112870 2024-09-10T07:38:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc38118260 con 0x7efc38076720 2024-09-10T07:38:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3e200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc38118560 con 0x7efc38077040 2024-09-10T07:38:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38112870 0x7efc3819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc38076720 0x7efc381a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38112870 0x7efc3819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37010/0 (socket says 172.21.15.46:37010) 2024-09-10T07:38:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.526+0000 7efc3d800640 1 -- 172.21.15.46:0/3816381383 learned_addr learned my addr 172.21.15.46:0/3816381383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc37e00640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc38077040 0x7efc381a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc3ce00640 1 -- 172.21.15.46:0/3816381383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc38077040 msgr2=0x7efc381a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc3ce00640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc38077040 0x7efc381a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc3ce00640 1 -- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38112870 msgr2=0x7efc3819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc3ce00640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38112870 0x7efc3819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc3ce00640 1 -- 172.21.15.46:0/3816381383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3819ef80 con 0x7efc38076720 2024-09-10T07:38:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc3d800640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38112870 0x7efc3819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc3ce00640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc38076720 0x7efc381a4640 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00e9f0 tx=0x7efc2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc35600640 1 -- 172.21.15.46:0/3816381383 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00ccc0 con 0x7efc38076720 2024-09-10T07:38:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc3e200640 1 -- 172.21.15.46:0/3816381383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3819f270 con 0x7efc38076720 2024-09-10T07:38:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc35600640 1 -- 172.21.15.46:0/3816381383 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc2c010910 con 0x7efc38076720 2024-09-10T07:38:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.527+0000 7efc35600640 1 -- 172.21.15.46:0/3816381383 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c018960 con 0x7efc38076720 2024-09-10T07:38:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.528+0000 7efc3e200640 1 -- 172.21.15.46:0/3816381383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc381ab5c0 con 0x7efc38076720 2024-09-10T07:38:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.530+0000 7efc421dd740 1 Processor -- start 2024-09-10T07:38:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc421dd740 1 -- start start 2024-09-10T07:38:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc421dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc35600640 1 -- 172.21.15.46:0/3816381383 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc2c010ac0 con 0x7efc38076720 2024-09-10T07:38:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc421dd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0e760 0x562b9bd18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc421dd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 0x562b9bd1b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc421dd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562b9bc743e0 con 0x562b9bd0e760 2024-09-10T07:38:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc421dd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b9bc1bd80 con 0x562b9bd19020 2024-09-10T07:38:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc421dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562b9bc703a0 con 0x562b9bd0de40 2024-09-10T07:38:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc3ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0e760 0x562b9bd18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc3d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc3d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60192/0 (socket says 172.21.15.46:60192) 2024-09-10T07:38:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc3d800640 1 -- 172.21.15.46:0/3312601292 learned_addr learned my addr 172.21.15.46:0/3312601292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.531+0000 7efc3ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0e760 0x562b9bd18ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37016/0 (socket says 172.21.15.46:37016) 2024-09-10T07:38:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc3d800640 1 -- 172.21.15.46:0/3312601292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 msgr2=0x562b9bd1b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc3d800640 1 --2- 172.21.15.46:0/3312601292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 0x562b9bd1b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc3d800640 1 -- 172.21.15.46:0/3312601292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0e760 msgr2=0x562b9bd18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc3d800640 1 --2- 172.21.15.46:0/3312601292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0e760 0x562b9bd18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc3d800640 1 -- 172.21.15.46:0/3312601292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9bc3aae0 con 0x562b9bd0de40 2024-09-10T07:38:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc3d800640 1 --2- 172.21.15.46:0/3312601292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0e220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc30011780 tx=0x7efc30011c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e62e20262714d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc2a000640 1 -- 172.21.15.46:0/3312601292 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3000fae0 con 0x562b9bd0de40 2024-09-10T07:38:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc2a000640 1 -- 172.21.15.46:0/3312601292 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc30011e90 con 0x562b9bd0de40 2024-09-10T07:38:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.532+0000 7efc2a000640 1 -- 172.21.15.46:0/3312601292 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3001a650 con 0x562b9bd0de40 2024-09-10T07:38:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc35600640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc200a2b80 0x7efc200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc35600640 1 -- 172.21.15.46:0/3816381383 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efc2c0cf7b0 con 0x7efc38076720 2024-09-10T07:38:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc3ce00640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc200a2b80 0x7efc200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc421dd740 1 -- 172.21.15.46:0/3312601292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 msgr2=0x562b9bd0e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc421dd740 1 --2- 172.21.15.46:0/3312601292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0e220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc30011780 tx=0x7efc30011c40 comp rx=0 tx=0).stop 2024-09-10T07:38:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc421dd740 1 -- 172.21.15.46:0/3312601292 shutdown_connections 2024-09-10T07:38:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc421dd740 1 --2- 172.21.15.46:0/3312601292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 0x562b9bd1b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc421dd740 1 --2- 172.21.15.46:0/3312601292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0e760 0x562b9bd18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc421dd740 1 --2- 172.21.15.46:0/3312601292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0e220 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc421dd740 1 -- 172.21.15.46:0/3312601292 >> 172.21.15.46:0/3312601292 conn(0x562b9bd0a3d0 msgr2=0x562b9bd0a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.533+0000 7efc421dd740 1 -- 172.21.15.46:0/3312601292 shutdown_connections 2024-09-10T07:38:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.534+0000 7efc421dd740 1 -- 172.21.15.46:0/3312601292 wait complete. 2024-09-10T07:38:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.534+0000 7efc3ce00640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc200a2b80 0x7efc200a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0986b0 tx=0x7efc2c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.534+0000 7efc421dd740 1 -- start start 2024-09-10T07:38:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.534+0000 7efc421dd740 1 Processor -- start 2024-09-10T07:38:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc421dd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd15fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc421dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0e760 0x562b9bd16500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc421dd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 0x562b9bd16a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc421dd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562b9bd1bbf0 con 0x562b9bd0de40 2024-09-10T07:38:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc421dd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b9bc2f0f0 con 0x562b9bd19020 2024-09-10T07:38:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc421dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562b9bd1bd70 con 0x562b9bd0e760 2024-09-10T07:38:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc421dd740 10 client.0 _ll_register_callbacks cb 0x562b9bc16bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc3d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd15fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc3d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd15fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37020/0 (socket says 172.21.15.46:37020) 2024-09-10T07:38:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc3d800640 1 -- 172.21.15.46:0/3270450225 learned_addr learned my addr 172.21.15.46:0/3270450225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc37e00640 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0e760 0x562b9bd16500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.535+0000 7efc3d800640 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 0x562b9bd16a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc3d800640 1 -- 172.21.15.46:0/3270450225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0e760 msgr2=0x562b9bd16500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc3d800640 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0e760 0x562b9bd16500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc3d800640 1 -- 172.21.15.46:0/3270450225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 msgr2=0x562b9bd16a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc3d800640 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 0x562b9bd16a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc3d800640 1 -- 172.21.15.46:0/3270450225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9b9e53c0 con 0x562b9bd0de40 2024-09-10T07:38:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc37e00640 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0e760 0x562b9bd16500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:24.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc3d800640 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd15fc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efc30012f00 tx=0x7efc30012d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682a02461b502d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:24.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:24.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc1ea00640 1 -- 172.21.15.46:0/3270450225 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3002e060 con 0x562b9bd0de40 2024-09-10T07:38:24.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc421dd740 1 -- 172.21.15.46:0/3270450225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc30011e90 con 0x562b9bd0de40 2024-09-10T07:38:24.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc1ea00640 1 -- 172.21.15.46:0/3270450225 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efc3000f8e0 con 0x562b9bd0de40 2024-09-10T07:38:24.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc1ea00640 1 -- 172.21.15.46:0/3270450225 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc30035050 con 0x562b9bd0de40 2024-09-10T07:38:24.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc421dd740 10 client.15771 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:24.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.536+0000 7efc421dd740 1 -- 172.21.15.46:0/3270450225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562b9bd27cd0 con 0x562b9bd0de40 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc1ea00640 1 -- 172.21.15.46:0/3270450225 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc30011e90 con 0x562b9bd0de40 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc1ea00640 1 -- 172.21.15.46:0/3270450225 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7efc3000f040 con 0x562b9bd0de40 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc421dd740 10 client.15771 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc421dd740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc421dd740 20 client.15771 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc421dd740 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b9bd290e0 0x562b9bd2b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc421dd740 4 client.15771 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc421dd740 1 -- 172.21.15.46:0/3270450225 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b9bc81950 con 0x562b9bd290e0 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.537+0000 7efc37e00640 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b9bd290e0 0x562b9bd2b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.538+0000 7efc37e00640 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b9bd290e0 0x562b9bd2b4d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18507c3b36d6b64b server_cookie=36dd1c59016c784d in_seq=0 out_seq=0 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.538+0000 7efc1ea00640 10 client.15771 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc1ea00640 1 -- 172.21.15.46:0/3270450225 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b9bc81950 con 0x562b9bd290e0 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc1ea00640 10 client.15771 handle_command_reply: tid=0 2024-09-10T07:38:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 client.15771 shutdown 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 -- 172.21.15.46:0/3270450225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 msgr2=0x562b9bd15fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd15fc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efc30012f00 tx=0x7efc30012d60 comp rx=0 tx=0).stop 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 -- 172.21.15.46:0/3270450225 shutdown_connections 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b9bd290e0 0x562b9bd2b4d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd19020 0x562b9bd16a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0e760 0x562b9bd16500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 --2- 172.21.15.46:0/3270450225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd15fc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc1ea00640 0 client.15771 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 -- 172.21.15.46:0/3270450225 >> 172.21.15.46:0/3270450225 conn(0x562b9bd0a3d0 msgr2=0x562b9bd10100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 -- 172.21.15.46:0/3270450225 shutdown_connections 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 1 -- 172.21.15.46:0/3270450225 wait complete. 2024-09-10T07:38:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.539+0000 7efc421dd740 20 client.15771 trim_cache size 0 max 16384 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.554+0000 7efc421dd740 1 Processor -- start 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc421dd740 1 -- start start 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc421dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc421dd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd30b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc421dd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd19020 0x562b9bd31040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc421dd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b51f0 con 0x562b9bd19020 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc421dd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b6df0 con 0x562b9bd0e760 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc421dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b58c0 con 0x562b9bd0de40 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd30b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc3d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd19020 0x562b9bd31040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd30b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38676/0 (socket says 172.21.15.46:38676) 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc37e00640 1 -- 172.21.15.46:0/3543056143 learned_addr learned my addr 172.21.15.46:0/3543056143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.555+0000 7efc3d800640 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc37e00640 1 -- 172.21.15.46:0/3543056143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 msgr2=0x562b9bd0ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc37e00640 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc37e00640 1 -- 172.21.15.46:0/3543056143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd19020 msgr2=0x562b9bd31040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc37e00640 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd19020 0x562b9bd31040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc37e00640 1 -- 172.21.15.46:0/3543056143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9bd0b060 con 0x562b9bd0e760 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc3d800640 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc37e00640 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd30b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc24009340 tx=0x7efc24000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2350497b4acfd613 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc29600640 1 -- 172.21.15.46:0/3543056143 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24035040 con 0x562b9bd0e760 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc29600640 1 -- 172.21.15.46:0/3543056143 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc24004310 con 0x562b9bd0e760 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.556+0000 7efc29600640 1 -- 172.21.15.46:0/3543056143 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc240047a0 con 0x562b9bd0e760 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 -- 172.21.15.46:0/3543056143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 msgr2=0x562b9bd30b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd30b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc24009340 tx=0x7efc24000bc0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 -- 172.21.15.46:0/3543056143 shutdown_connections 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd19020 0x562b9bd31040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd30b00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 --2- 172.21.15.46:0/3543056143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd0de40 0x562b9bd0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 -- 172.21.15.46:0/3543056143 >> 172.21.15.46:0/3543056143 conn(0x562b9bd0a3d0 msgr2=0x562b9bd0f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 -- 172.21.15.46:0/3543056143 shutdown_connections 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 -- 172.21.15.46:0/3543056143 wait complete. 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.557+0000 7efc421dd740 1 -- start start 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc421dd740 1 Processor -- start 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc421dd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd21690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc421dd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd1c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc421dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd19020 0x562b9bd1cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc421dd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b6660 con 0x562b9bd0de40 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc421dd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b5f90 con 0x562b9bd0e760 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc421dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b31d0 con 0x562b9bd19020 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc421dd740 10 client.0 _ll_register_callbacks cb 0x7efc2c0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc3d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd21690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc3d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd21690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37048/0 (socket says 172.21.15.46:37048) 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc3d800640 1 -- 172.21.15.46:0/1950199071 learned_addr learned my addr 172.21.15.46:0/1950199071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc37e00640 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd1c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.558+0000 7efc3d800640 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd19020 0x562b9bd1cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc3d800640 1 -- 172.21.15.46:0/1950199071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd19020 msgr2=0x562b9bd1cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc3d800640 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd19020 0x562b9bd1cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc3d800640 1 -- 172.21.15.46:0/1950199071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 msgr2=0x562b9bd1c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc3d800640 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd1c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc3d800640 1 -- 172.21.15.46:0/1950199071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9bd1d730 con 0x562b9bd0de40 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc3d800640 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd21690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efc30005b20 tx=0x7efc300049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd883877fe91a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc1ea00640 1 -- 172.21.15.46:0/1950199071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3002e070 con 0x562b9bd0de40 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc1ea00640 1 -- 172.21.15.46:0/1950199071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efc30004fc0 con 0x562b9bd0de40 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc421dd740 1 -- 172.21.15.46:0/1950199071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc24004310 con 0x562b9bd0de40 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc1ea00640 1 -- 172.21.15.46:0/1950199071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc3004b050 con 0x562b9bd0de40 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc421dd740 10 client.15777 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.559+0000 7efc421dd740 1 -- 172.21.15.46:0/1950199071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562b9bd939d0 con 0x562b9bd0de40 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc1ea00640 1 -- 172.21.15.46:0/1950199071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc24004310 con 0x562b9bd0de40 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc1ea00640 1 -- 172.21.15.46:0/1950199071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7efc30044060 con 0x562b9bd0de40 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc421dd740 10 client.15777 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc421dd740 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc421dd740 20 client.15777 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc421dd740 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b9bd94150 0x562b9bd96570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:24.592 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:38:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc421dd740 4 client.15777 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:38:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc421dd740 1 -- 172.21.15.46:0/1950199071 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b9bc7be60 con 0x562b9bd94150 2024-09-10T07:38:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.560+0000 7efc37e00640 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b9bd94150 0x562b9bd96570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc37e00640 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b9bd94150 0x562b9bd96570 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f0750c436a54fe server_cookie=7e440fbed3bd13f9 in_seq=0 out_seq=0 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc1ea00640 10 client.15777 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc1ea00640 1 -- 172.21.15.46:0/1950199071 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b9bc7be60 con 0x562b9bd94150 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc1ea00640 10 client.15777 handle_command_reply: tid=0 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 client.15777 shutdown 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 -- 172.21.15.46:0/1950199071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 msgr2=0x562b9bd21690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd21690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efc30005b20 tx=0x7efc300049d0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 -- 172.21.15.46:0/1950199071 shutdown_connections 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b9bd94150 0x562b9bd96570 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b9bd19020 0x562b9bd1cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b9bd0e760 0x562b9bd1c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 --2- 172.21.15.46:0/1950199071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b9bd0de40 0x562b9bd21690 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc421dd740 1 -- 172.21.15.46:0/1950199071 >> 172.21.15.46:0/1950199071 conn(0x562b9bd0a3d0 msgr2=0x562b9bd100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.561+0000 7efc1ea00640 0 client.15777 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.562+0000 7efc421dd740 1 -- 172.21.15.46:0/1950199071 shutdown_connections 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.562+0000 7efc421dd740 1 -- 172.21.15.46:0/1950199071 wait complete. 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.562+0000 7efc421dd740 20 client.15777 trim_cache size 0 max 16384 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 -- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc200a2b80 msgr2=0x7efc200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc200a2b80 0x7efc200a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0986b0 tx=0x7efc2c0d5530 comp rx=0 tx=0).stop 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 -- 172.21.15.46:0/3816381383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc38076720 msgr2=0x7efc381a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc38076720 0x7efc381a4640 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00e9f0 tx=0x7efc2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 -- 172.21.15.46:0/3816381383 shutdown_connections 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efc200a2b80 0x7efc200a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc38112870 0x7efc3819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc38077040 0x7efc381a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 --2- 172.21.15.46:0/3816381383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc38076720 0x7efc381a4640 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:24.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 -- 172.21.15.46:0/3816381383 >> 172.21.15.46:0/3816381383 conn(0x7efc380ff910 msgr2=0x7efc38101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:24.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 -- 172.21.15.46:0/3816381383 shutdown_connections 2024-09-10T07:38:24.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:24.563+0000 7efc3e200640 1 -- 172.21.15.46:0/3816381383 wait complete. 2024-09-10T07:38:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:24 smithi181 ceph-mon[35397]: pgmap v452: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 214 op/s 2024-09-10T07:38:24.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:24 smithi063 ceph-mon[35020]: pgmap v452: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 214 op/s 2024-09-10T07:38:25.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 65.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:38:26.465 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:38:26.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:38:26.522 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:26 smithi046 ceph-mon[30705]: pgmap v453: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.9 MiB/s wr, 197 op/s 2024-09-10T07:38:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:26 smithi181 ceph-mon[35397]: pgmap v453: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.9 MiB/s wr, 197 op/s 2024-09-10T07:38:26.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:26 smithi063 ceph-mon[35020]: pgmap v453: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.9 MiB/s wr, 197 op/s 2024-09-10T07:38:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 -- 172.21.15.46:0/1295467020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c103d60 msgr2=0x7ff38c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 --2- 172.21.15.46:0/1295467020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c103d60 0x7ff38c104160 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009900 tx=0x7ff37c02a410 comp rx=0 tx=0).stop 2024-09-10T07:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 -- 172.21.15.46:0/1295467020 shutdown_connections 2024-09-10T07:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 --2- 172.21.15.46:0/1295467020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c105910 0x7ff38c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 --2- 172.21.15.46:0/1295467020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff38c104f50 0x7ff38c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 --2- 172.21.15.46:0/1295467020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c103d60 0x7ff38c104160 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 -- 172.21.15.46:0/1295467020 >> 172.21.15.46:0/1295467020 conn(0x7ff38c0ff4e0 msgr2=0x7ff38c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 -- 172.21.15.46:0/1295467020 shutdown_connections 2024-09-10T07:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.149+0000 7ff392800640 1 -- 172.21.15.46:0/1295467020 wait complete. 2024-09-10T07:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.150+0000 7ff392800640 1 Processor -- start 2024-09-10T07:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.150+0000 7ff392800640 1 -- start start 2024-09-10T07:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff392800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff38c103d60 0x7ff38c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff392800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c104f50 0x7ff38c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff392800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c105910 0x7ff38c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff392800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c10fa30 con 0x7ff38c104f50 2024-09-10T07:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff392800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c10f8b0 con 0x7ff38c103d60 2024-09-10T07:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff392800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c10fbb0 con 0x7ff38c105910 2024-09-10T07:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff38b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff38c103d60 0x7ff38c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff382a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c104f50 0x7ff38c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff38b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff38c103d60 0x7ff38c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50156/0 (socket says 172.21.15.46:50156) 2024-09-10T07:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff38b400640 1 -- 172.21.15.46:0/3760034067 learned_addr learned my addr 172.21.15.46:0/3760034067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff38be00640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c105910 0x7ff38c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff382a00640 1 -- 172.21.15.46:0/3760034067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c105910 msgr2=0x7ff38c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff382a00640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c105910 0x7ff38c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff382a00640 1 -- 172.21.15.46:0/3760034067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff38c103d60 msgr2=0x7ff38c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff382a00640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff38c103d60 0x7ff38c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.151+0000 7ff382a00640 1 -- 172.21.15.46:0/3760034067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c1a1bf0 con 0x7ff38c104f50 2024-09-10T07:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.152+0000 7ff382a00640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c104f50 0x7ff38c19a510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff37400e9f0 tx=0x7ff37400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.152+0000 7ff388c00640 1 -- 172.21.15.46:0/3760034067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37400ccc0 con 0x7ff38c104f50 2024-09-10T07:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.152+0000 7ff388c00640 1 -- 172.21.15.46:0/3760034067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff374010910 con 0x7ff38c104f50 2024-09-10T07:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.152+0000 7ff388c00640 1 -- 172.21.15.46:0/3760034067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374018960 con 0x7ff38c104f50 2024-09-10T07:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.152+0000 7ff392800640 1 -- 172.21.15.46:0/3760034067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c1a1ee0 con 0x7ff38c104f50 2024-09-10T07:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.152+0000 7ff392800640 1 -- 172.21.15.46:0/3760034067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c1a2350 con 0x7ff38c104f50 2024-09-10T07:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.155+0000 7ff3956a1740 1 Processor -- start 2024-09-10T07:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.155+0000 7ff388c00640 1 -- 172.21.15.46:0/3760034067 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff374010ac0 con 0x7ff38c104f50 2024-09-10T07:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.155+0000 7ff3956a1740 1 -- start start 2024-09-10T07:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a41490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a419d0 0x55add9a46490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff3956a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55add99a6100 con 0x55add9a419d0 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff3956a1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55add994e1b0 con 0x55add9a41090 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff3956a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55add99a20c0 con 0x55add9a46a60 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a41490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a419d0 0x55add9a46490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a41490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50158/0 (socket says 172.21.15.46:50158) 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff382a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38be00640 1 -- 172.21.15.46:0/3676728948 learned_addr learned my addr 172.21.15.46:0/3676728948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38be00640 1 -- 172.21.15.46:0/3676728948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 msgr2=0x55add9a48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38be00640 1 --2- 172.21.15.46:0/3676728948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a48f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38be00640 1 -- 172.21.15.46:0/3676728948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a419d0 msgr2=0x55add9a46490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38be00640 1 --2- 172.21.15.46:0/3676728948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a419d0 0x55add9a46490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.156+0000 7ff38be00640 1 -- 172.21.15.46:0/3676728948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add96b9210 con 0x55add9a41090 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.157+0000 7ff38be00640 1 --2- 172.21.15.46:0/3676728948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a41490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff384010370 tx=0x7ff384010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4727c5bf6bb17771 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.157+0000 7ff388c00640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3700a2cb0 0x7ff3700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.157+0000 7ff380c00640 1 -- 172.21.15.46:0/3676728948 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384016070 con 0x55add9a41090 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.157+0000 7ff380c00640 1 -- 172.21.15.46:0/3676728948 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384004d20 con 0x55add9a41090 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.157+0000 7ff380c00640 1 -- 172.21.15.46:0/3676728948 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384005480 con 0x55add9a41090 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.157+0000 7ff388c00640 1 -- 172.21.15.46:0/3760034067 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff3740cf700 con 0x7ff38c104f50 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.157+0000 7ff38b400640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3700a2cb0 0x7ff3700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 -- 172.21.15.46:0/3676728948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 msgr2=0x55add9a41490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 --2- 172.21.15.46:0/3676728948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a41490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff384010370 tx=0x7ff384010830 comp rx=0 tx=0).stop 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 -- 172.21.15.46:0/3676728948 shutdown_connections 2024-09-10T07:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 --2- 172.21.15.46:0/3676728948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 --2- 172.21.15.46:0/3676728948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a419d0 0x55add9a46490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff38b400640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3700a2cb0 0x7ff3700a5170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000c00 tx=0x7ff37c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 --2- 172.21.15.46:0/3676728948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a41490 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 -- 172.21.15.46:0/3676728948 >> 172.21.15.46:0/3676728948 conn(0x55add9a3c700 msgr2=0x55add9a3cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 -- 172.21.15.46:0/3676728948 shutdown_connections 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 -- 172.21.15.46:0/3676728948 wait complete. 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.158+0000 7ff3956a1740 1 -- start start 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff3956a1740 1 Processor -- start 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a4b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 0x55add9a4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff3956a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55add9a49a30 con 0x55add9a46a60 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff3956a1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55add992f090 con 0x55add9a41090 2024-09-10T07:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff3956a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55add9a49bb0 con 0x55add9a419d0 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff3956a1740 10 client.0 _ll_register_callbacks cb 0x55add9983f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff38be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff38be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43496/0 (socket says 172.21.15.46:43496) 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff38be00640 1 -- 172.21.15.46:0/1860010592 learned_addr learned my addr 172.21.15.46:0/1860010592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff382a00640 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 0x55add9a4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.159+0000 7ff38b400640 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a4b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff38be00640 1 -- 172.21.15.46:0/1860010592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 msgr2=0x55add9a4b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff38be00640 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 0x55add9a4b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff38be00640 1 -- 172.21.15.46:0/1860010592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 msgr2=0x55add9a4b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff38be00640 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a4b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff38be00640 1 -- 172.21.15.46:0/1860010592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add9785620 con 0x55add9a46a60 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff38be00640 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a527a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff384009bf0 tx=0x7ff384003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cf8f104e352552 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff379600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff379600640 1 -- 172.21.15.46:0/1860010592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384029060 con 0x55add9a46a60 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff379600640 1 -- 172.21.15.46:0/1860010592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff384003cf0 con 0x55add9a46a60 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.160+0000 7ff3956a1740 1 -- 172.21.15.46:0/1860010592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff384004d20 con 0x55add9a46a60 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff379600640 1 -- 172.21.15.46:0/1860010592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff384030050 con 0x55add9a46a60 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff3956a1740 10 client.15789 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff3956a1740 1 -- 172.21.15.46:0/1860010592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55add9a55590 con 0x55add9a46a60 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff379600640 1 -- 172.21.15.46:0/1860010592 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384004d20 con 0x55add9a46a60 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff379600640 1 -- 172.21.15.46:0/1860010592 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7ff38400f040 con 0x55add9a46a60 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff3956a1740 10 client.15789 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff3956a1740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff3956a1740 20 client.15789 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff3956a1740 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55add9a569f0 0x55add9a58de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff3956a1740 4 client.15789 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.161+0000 7ff3956a1740 1 -- 172.21.15.46:0/1860010592 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55add99b3830 con 0x55add9a569f0 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.162+0000 7ff38b400640 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55add9a569f0 0x55add9a58de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.162+0000 7ff38b400640 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55add9a569f0 0x55add9a58de0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f399bc2a17fa50 server_cookie=b50218a1e538cbb0 in_seq=0 out_seq=0 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.162+0000 7ff379600640 10 client.15789 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff379600640 1 -- 172.21.15.46:0/1860010592 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55add99b3830 con 0x55add9a569f0 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff379600640 10 client.15789 handle_command_reply: tid=0 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 client.15789 shutdown 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 -- 172.21.15.46:0/1860010592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 msgr2=0x55add9a527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a527a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff384009bf0 tx=0x7ff384003970 comp rx=0 tx=0).stop 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 -- 172.21.15.46:0/1860010592 shutdown_connections 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55add9a569f0 0x55add9a58de0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a527a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 0x55add9a4b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 --2- 172.21.15.46:0/1860010592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a4b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff379600640 0 client.15789 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 -- 172.21.15.46:0/1860010592 >> 172.21.15.46:0/1860010592 conn(0x55add9a3c700 msgr2=0x55add9a3df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 -- 172.21.15.46:0/1860010592 shutdown_connections 2024-09-10T07:38:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 1 -- 172.21.15.46:0/1860010592 wait complete. 2024-09-10T07:38:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.163+0000 7ff3956a1740 20 client.15789 trim_cache size 0 max 16384 2024-09-10T07:38:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.180+0000 7ff3956a1740 1 Processor -- start 2024-09-10T07:38:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.180+0000 7ff3956a1740 1 -- start start 2024-09-10T07:38:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.180+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a41090 0x55add9a3b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.180+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a419d0 0x55add9a52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.180+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a3daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.180+0000 7ff3956a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1ac910 con 0x55add9a41090 2024-09-10T07:38:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.180+0000 7ff3956a1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1ae510 con 0x55add9a419d0 2024-09-10T07:38:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.180+0000 7ff3956a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1acfe0 con 0x55add9a46a60 2024-09-10T07:38:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff38be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a3daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a419d0 0x55add9a52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff38b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a41090 0x55add9a3b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a419d0 0x55add9a52760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50182/0 (socket says 172.21.15.46:50182) 2024-09-10T07:38:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 -- 172.21.15.46:0/2931524000 learned_addr learned my addr 172.21.15.46:0/2931524000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff38be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a3daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:42888/0 (socket says 172.21.15.46:42888) 2024-09-10T07:38:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 -- 172.21.15.46:0/2931524000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 msgr2=0x55add9a3daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 --2- 172.21.15.46:0/2931524000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a3daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 -- 172.21.15.46:0/2931524000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a41090 msgr2=0x55add9a3b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 --2- 172.21.15.46:0/2931524000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a41090 0x55add9a3b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 -- 172.21.15.46:0/2931524000 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add9a59440 con 0x55add9a419d0 2024-09-10T07:38:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff382a00640 1 --2- 172.21.15.46:0/2931524000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a419d0 0x55add9a52760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3740cce20 tx=0x7ff374009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ff4052d9b52456 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.181+0000 7ff37be00640 1 -- 172.21.15.46:0/2931524000 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37409b590 con 0x55add9a419d0 2024-09-10T07:38:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff37be00640 1 -- 172.21.15.46:0/2931524000 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37409bc30 con 0x55add9a419d0 2024-09-10T07:38:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff37be00640 1 -- 172.21.15.46:0/2931524000 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3740d1bc0 con 0x55add9a419d0 2024-09-10T07:38:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 -- 172.21.15.46:0/2931524000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a419d0 msgr2=0x55add9a52760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 --2- 172.21.15.46:0/2931524000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a419d0 0x55add9a52760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3740cce20 tx=0x7ff374009700 comp rx=0 tx=0).stop 2024-09-10T07:38:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 -- 172.21.15.46:0/2931524000 shutdown_connections 2024-09-10T07:38:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 --2- 172.21.15.46:0/2931524000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a46a60 0x55add9a3daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 --2- 172.21.15.46:0/2931524000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a419d0 0x55add9a52760 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 --2- 172.21.15.46:0/2931524000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a41090 0x55add9a3b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 -- 172.21.15.46:0/2931524000 >> 172.21.15.46:0/2931524000 conn(0x55add9a3c700 msgr2=0x55add9a3b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 -- 172.21.15.46:0/2931524000 shutdown_connections 2024-09-10T07:38:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.182+0000 7ff3956a1740 1 -- 172.21.15.46:0/2931524000 wait complete. 2024-09-10T07:38:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 1 -- start start 2024-09-10T07:38:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 1 Processor -- start 2024-09-10T07:38:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a50620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 0x55add9a50b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a51b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1add80 con 0x55add9a46a60 2024-09-10T07:38:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1ad6b0 con 0x55add9a41090 2024-09-10T07:38:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1aa8f0 con 0x55add9a419d0 2024-09-10T07:38:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.183+0000 7ff3956a1740 10 client.0 _ll_register_callbacks cb 0x7ff37c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a50620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a50620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50186/0 (socket says 172.21.15.46:50186) 2024-09-10T07:38:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 -- 172.21.15.46:0/339519963 learned_addr learned my addr 172.21.15.46:0/339519963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38be00640 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a51b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff382a00640 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 0x55add9a50b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 -- 172.21.15.46:0/339519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 msgr2=0x55add9a50b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 0x55add9a50b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 -- 172.21.15.46:0/339519963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 msgr2=0x55add9a51b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a51b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 -- 172.21.15.46:0/339519963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add9a52110 con 0x55add9a41090 2024-09-10T07:38:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff38b400640 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a50620 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff37c054fd0 tx=0x7ff37c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a31148b2fec5bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff379600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:38:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.184+0000 7ff379600640 1 -- 172.21.15.46:0/339519963 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c035040 con 0x55add9a41090 2024-09-10T07:38:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 1 -- 172.21.15.46:0/339519963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff37409bc30 con 0x55add9a41090 2024-09-10T07:38:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff379600640 1 -- 172.21.15.46:0/339519963 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff37c0037f0 con 0x55add9a41090 2024-09-10T07:38:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff379600640 1 -- 172.21.15.46:0/339519963 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff37c066050 con 0x55add9a41090 2024-09-10T07:38:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 10 client.25889 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 1 -- 172.21.15.46:0/339519963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55add9ac1e00 con 0x55add9a41090 2024-09-10T07:38:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff379600640 1 -- 172.21.15.46:0/339519963 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37409bc30 con 0x55add9a41090 2024-09-10T07:38:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff379600640 1 -- 172.21.15.46:0/339519963 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7ff37c054810 con 0x55add9a41090 2024-09-10T07:38:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 10 client.25889 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 10 client.25889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 20 client.25889 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55add9ac2850 0x55add9ac4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 4 client.25889 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:38:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.185+0000 7ff3956a1740 1 -- 172.21.15.46:0/339519963 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55add99adb80 con 0x55add9ac2850 2024-09-10T07:38:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.186+0000 7ff382a00640 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55add9ac2850 0x55add9ac4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.186+0000 7ff382a00640 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55add9ac2850 0x55add9ac4c70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f275cba45d45d5e4 server_cookie=fec5bc98d4aea6b2 in_seq=0 out_seq=0 2024-09-10T07:38:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.186+0000 7ff379600640 10 client.25889 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.196+0000 7ff379600640 1 -- 172.21.15.46:0/339519963 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55add99adb80 con 0x55add9ac2850 2024-09-10T07:38:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.196+0000 7ff379600640 10 client.25889 handle_command_reply: tid=0 2024-09-10T07:38:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.197+0000 7ff3956a1740 1 client.25889 shutdown 2024-09-10T07:38:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.197+0000 7ff3956a1740 1 -- 172.21.15.46:0/339519963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 msgr2=0x55add9a50620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.197+0000 7ff3956a1740 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a50620 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff37c054fd0 tx=0x7ff37c009260 comp rx=0 tx=0).stop 2024-09-10T07:38:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 1 -- 172.21.15.46:0/339519963 shutdown_connections 2024-09-10T07:38:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55add9ac2850 0x55add9ac4c70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add9a46a60 0x55add9a51b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55add9a419d0 0x55add9a50b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 1 --2- 172.21.15.46:0/339519963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55add9a41090 0x55add9a50620 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff379600640 0 client.25889 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 1 -- 172.21.15.46:0/339519963 >> 172.21.15.46:0/339519963 conn(0x55add9a3c700 msgr2=0x55add9a3b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 1 -- 172.21.15.46:0/339519963 shutdown_connections 2024-09-10T07:38:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 1 -- 172.21.15.46:0/339519963 wait complete. 2024-09-10T07:38:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.199+0000 7ff3956a1740 20 client.25889 trim_cache size 0 max 16384 2024-09-10T07:38:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:38:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:38:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:38:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:38:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 -- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3700a2cb0 msgr2=0x7ff3700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3700a2cb0 0x7ff3700a5170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000c00 tx=0x7ff37c0315b0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 -- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c104f50 msgr2=0x7ff38c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c104f50 0x7ff38c19a510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff37400e9f0 tx=0x7ff37400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 -- 172.21.15.46:0/3760034067 shutdown_connections 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff38c105910 0x7ff38c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3700a2cb0 0x7ff3700a5170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38c104f50 0x7ff38c19a510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 --2- 172.21.15.46:0/3760034067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff38c103d60 0x7ff38c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 -- 172.21.15.46:0/3760034067 >> 172.21.15.46:0/3760034067 conn(0x7ff38c0ff4e0 msgr2=0x7ff38c1081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 -- 172.21.15.46:0/3760034067 shutdown_connections 2024-09-10T07:38:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:27.200+0000 7ff392800640 1 -- 172.21.15.46:0/3760034067 wait complete. 2024-09-10T07:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 -- 172.21.15.46:0/2871311862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff600101df0 msgr2=0x7ff6001021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 --2- 172.21.15.46:0/2871311862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff600101df0 0x7ff6001021d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009900 tx=0x7ff5f002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 -- 172.21.15.46:0/2871311862 shutdown_connections 2024-09-10T07:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 --2- 172.21.15.46:0/2871311862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff600106f20 0x7ff60010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 --2- 172.21.15.46:0/2871311862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6001027a0 0x7ff6001067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 --2- 172.21.15.46:0/2871311862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff600101df0 0x7ff6001021d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 -- 172.21.15.46:0/2871311862 >> 172.21.15.46:0/2871311862 conn(0x7ff6000fd8b0 msgr2=0x7ff6000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 -- 172.21.15.46:0/2871311862 shutdown_connections 2024-09-10T07:38:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.460+0000 7ff606800640 1 -- 172.21.15.46:0/2871311862 wait complete. 2024-09-10T07:38:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.461+0000 7ff606800640 1 Processor -- start 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.461+0000 7ff606800640 1 -- start start 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.461+0000 7ff606800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff600101df0 0x7ff60019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.461+0000 7ff606800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6001027a0 0x7ff60019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.461+0000 7ff606800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff600106f20 0x7ff60019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.461+0000 7ff606800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff600111fd0 con 0x7ff6001027a0 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.461+0000 7ff606800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff600111e50 con 0x7ff600101df0 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.461+0000 7ff606800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff600112150 con 0x7ff600106f20 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff600101df0 0x7ff60019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff600101df0 0x7ff60019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50196/0 (socket says 172.21.15.46:50196) 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 -- 172.21.15.46:0/3262555151 learned_addr learned my addr 172.21.15.46:0/3262555151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6001027a0 0x7ff60019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff600106f20 0x7ff60019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 -- 172.21.15.46:0/3262555151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff600106f20 msgr2=0x7ff60019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff600106f20 0x7ff60019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 -- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6001027a0 msgr2=0x7ff60019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6001027a0 0x7ff60019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 -- 172.21.15.46:0/3262555151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60019f640 con 0x7ff600101df0 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.462+0000 7ff5ff400640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff600101df0 0x7ff60019a690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0004770 tx=0x7ff5f0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.463+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3262555151 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f003f070 con 0x7ff600101df0 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.463+0000 7ff606800640 1 -- 172.21.15.46:0/3262555151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60019f8d0 con 0x7ff600101df0 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.463+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3262555151 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f0031ca0 con 0x7ff600101df0 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.463+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3262555151 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0035030 con 0x7ff600101df0 2024-09-10T07:38:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.463+0000 7ff606800640 1 -- 172.21.15.46:0/3262555151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6001a7210 con 0x7ff600101df0 2024-09-10T07:38:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.466+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3262555151 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff5f0042070 con 0x7ff600101df0 2024-09-10T07:38:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.466+0000 7ff6091e3740 1 Processor -- start 2024-09-10T07:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.466+0000 7ff6091e3740 1 -- start start 2024-09-10T07:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158dac0 0x55dbf158dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158e3e0 0x55dbf1598760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf1598ca0 0x55dbf159b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff6091e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dbf14f4290 con 0x55dbf158e3e0 2024-09-10T07:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff6091e3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dbf149bb40 con 0x55dbf158dac0 2024-09-10T07:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff6091e3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dbf14f0250 con 0x55dbf1598ca0 2024-09-10T07:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158e3e0 0x55dbf1598760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158e3e0 0x55dbf1598760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43528/0 (socket says 172.21.15.46:43528) 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff5ff400640 1 -- 172.21.15.46:0/1255804510 learned_addr learned my addr 172.21.15.46:0/1255804510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf1598ca0 0x55dbf159b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.467+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158dac0 0x55dbf158dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5f7e00640 1 -- 172.21.15.46:0/1255804510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158dac0 msgr2=0x55dbf158dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158dac0 0x55dbf158dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5f7e00640 1 -- 172.21.15.46:0/1255804510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158e3e0 msgr2=0x55dbf1598760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158e3e0 0x55dbf1598760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5f7e00640 1 -- 172.21.15.46:0/1255804510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf12656a0 con 0x55dbf1598ca0 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5ff400640 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158e3e0 0x55dbf1598760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5fcc00640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5e40a2b80 0x7ff5e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5e40a2b80 0x7ff5e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf1598ca0 0x55dbf159b090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009bf0 tx=0x7ff5e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2d004bb82b7412 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3262555151 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff5f00ef9e0 con 0x7ff600101df0 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5f4c00640 1 -- 172.21.15.46:0/1255804510 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8004050 con 0x55dbf1598ca0 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.468+0000 7ff5f4c00640 1 -- 172.21.15.46:0/1255804510 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8005500 con 0x55dbf1598ca0 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff5f4c00640 1 -- 172.21.15.46:0/1255804510 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80035f0 con 0x55dbf1598ca0 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff6091e3740 1 -- 172.21.15.46:0/1255804510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf1598ca0 msgr2=0x55dbf159b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5e40a2b80 0x7ff5e40a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005fd0 tx=0x7ff5f8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf1598ca0 0x55dbf159b090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009bf0 tx=0x7ff5e8011810 comp rx=0 tx=0).stop 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff6091e3740 1 -- 172.21.15.46:0/1255804510 shutdown_connections 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf1598ca0 0x55dbf159b090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158e3e0 0x55dbf1598760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1255804510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158dac0 0x55dbf158dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff6091e3740 1 -- 172.21.15.46:0/1255804510 >> 172.21.15.46:0/1255804510 conn(0x55dbf158a050 msgr2=0x55dbf158a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.469+0000 7ff6091e3740 1 -- 172.21.15.46:0/1255804510 shutdown_connections 2024-09-10T07:38:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.470+0000 7ff6091e3740 1 -- 172.21.15.46:0/1255804510 wait complete. 2024-09-10T07:38:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.470+0000 7ff6091e3740 1 -- start start 2024-09-10T07:38:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.470+0000 7ff6091e3740 1 Processor -- start 2024-09-10T07:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1598560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf1595810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf1595d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff6091e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dbf159b910 con 0x55dbf1598ca0 2024-09-10T07:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff6091e3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dbf14aefa0 con 0x55dbf158e3e0 2024-09-10T07:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff6091e3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dbf159ba90 con 0x55dbf158dac0 2024-09-10T07:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff6091e3740 10 client.0 _ll_register_callbacks cb 0x55dbf1202ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf1595d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1598560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf1595d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43540/0 (socket says 172.21.15.46:43540) 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1598560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:42950/0 (socket says 172.21.15.46:42950) 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf1595810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.471+0000 7ff5ffe00640 1 -- 172.21.15.46:0/1844460604 learned_addr learned my addr 172.21.15.46:0/1844460604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5f7e00640 1 -- 172.21.15.46:0/1844460604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 msgr2=0x55dbf1598560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1598560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5f7e00640 1 -- 172.21.15.46:0/1844460604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 msgr2=0x55dbf1595d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf1595d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5f7e00640 1 -- 172.21.15.46:0/1844460604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf1596580 con 0x55dbf158e3e0 2024-09-10T07:38:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5ff400640 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1598560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf1595810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff5e801b040 tx=0x7ff5e8009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd4d585b8c961216 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5ed600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5ed600640 1 -- 172.21.15.46:0/1844460604 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8029070 con 0x55dbf158e3e0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff6091e3740 1 -- 172.21.15.46:0/1844460604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e8005500 con 0x55dbf158e3e0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.472+0000 7ff5ed600640 1 -- 172.21.15.46:0/1844460604 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff5e8005030 con 0x55dbf158e3e0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff5ed600640 1 -- 172.21.15.46:0/1844460604 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff5e8035050 con 0x55dbf158e3e0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff6091e3740 10 client.25901 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff6091e3740 1 -- 172.21.15.46:0/1844460604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dbf15a77b0 con 0x55dbf158e3e0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff5ed600640 1 -- 172.21.15.46:0/1844460604 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8005500 con 0x55dbf158e3e0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff5ed600640 1 -- 172.21.15.46:0/1844460604 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7ff5e8023430 con 0x55dbf158e3e0 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff6091e3740 10 client.25901 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff6091e3740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff6091e3740 20 client.25901 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dbf15a8b00 0x55dbf15aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff6091e3740 4 client.25901 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.473+0000 7ff6091e3740 1 -- 172.21.15.46:0/1844460604 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf1501800 con 0x55dbf15a8b00 2024-09-10T07:38:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.474+0000 7ff5ff400640 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dbf15a8b00 0x55dbf15aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.474+0000 7ff5ff400640 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dbf15a8b00 0x55dbf15aaef0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d43ce02e805d802 server_cookie=9626c48b3dca9863 in_seq=0 out_seq=0 2024-09-10T07:38:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.474+0000 7ff5ed600640 10 client.25901 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff5ed600640 1 -- 172.21.15.46:0/1844460604 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbf1501800 con 0x55dbf15a8b00 2024-09-10T07:38:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff5ed600640 10 client.25901 handle_command_reply: tid=0 2024-09-10T07:38:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 client.25901 shutdown 2024-09-10T07:38:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 -- 172.21.15.46:0/1844460604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 msgr2=0x55dbf1595810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf1595810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff5e801b040 tx=0x7ff5e8009fd0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 -- 172.21.15.46:0/1844460604 shutdown_connections 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dbf15a8b00 0x55dbf15aaef0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf1595d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf1595810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1844460604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1598560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 -- 172.21.15.46:0/1844460604 >> 172.21.15.46:0/1844460604 conn(0x55dbf158a050 msgr2=0x55dbf158fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff5ed600640 0 client.25901 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.475+0000 7ff6091e3740 1 -- 172.21.15.46:0/1844460604 shutdown_connections 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.476+0000 7ff6091e3740 1 -- 172.21.15.46:0/1844460604 wait complete. 2024-09-10T07:38:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.476+0000 7ff6091e3740 20 client.25901 trim_cache size 0 max 16384 2024-09-10T07:38:27.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.494+0000 7ff6091e3740 1 Processor -- start 2024-09-10T07:38:27.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff6091e3740 1 -- start start 2024-09-10T07:38:27.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1590d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf15b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf15b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff6091e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b0e40 con 0x55dbf1598ca0 2024-09-10T07:38:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff6091e3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b2a40 con 0x55dbf158e3e0 2024-09-10T07:38:27.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff6091e3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b1510 con 0x55dbf158dac0 2024-09-10T07:38:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff5f7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf15b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1590d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff5f7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf15b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50208/0 (socket says 172.21.15.46:50208) 2024-09-10T07:38:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff5f7e00640 1 -- 172.21.15.46:0/822467086 learned_addr learned my addr 172.21.15.46:0/822467086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.495+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf15b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5f7e00640 1 -- 172.21.15.46:0/822467086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 msgr2=0x55dbf1590d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1590d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5f7e00640 1 -- 172.21.15.46:0/822467086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 msgr2=0x55dbf15b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf15b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5f7e00640 1 -- 172.21.15.46:0/822467086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf158ac00 con 0x55dbf158e3e0 2024-09-10T07:38:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5ff400640 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1590d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf15b0790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80239f0 tx=0x7ff5e8004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606a6afc53838bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5efe00640 1 -- 172.21.15.46:0/822467086 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8029050 con 0x55dbf158e3e0 2024-09-10T07:38:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5efe00640 1 -- 172.21.15.46:0/822467086 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e80043d0 con 0x55dbf158e3e0 2024-09-10T07:38:27.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.496+0000 7ff5efe00640 1 -- 172.21.15.46:0/822467086 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8003850 con 0x55dbf158e3e0 2024-09-10T07:38:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 -- 172.21.15.46:0/822467086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 msgr2=0x55dbf15b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf15b0790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80239f0 tx=0x7ff5e8004a20 comp rx=0 tx=0).stop 2024-09-10T07:38:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 -- 172.21.15.46:0/822467086 shutdown_connections 2024-09-10T07:38:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf1598ca0 0x55dbf15b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf158e3e0 0x55dbf15b0790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 --2- 172.21.15.46:0/822467086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158dac0 0x55dbf1590d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 -- 172.21.15.46:0/822467086 >> 172.21.15.46:0/822467086 conn(0x55dbf158a050 msgr2=0x55dbf1590670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 -- 172.21.15.46:0/822467086 shutdown_connections 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.497+0000 7ff6091e3740 1 -- 172.21.15.46:0/822467086 wait complete. 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 1 -- start start 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 1 Processor -- start 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158dac0 0x55dbf159bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158e3e0 0x55dbf159c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf1598ca0 0x55dbf159c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b22b0 con 0x55dbf158dac0 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001b1be0 con 0x55dbf1598ca0 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001aee20 con 0x55dbf158e3e0 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.498+0000 7ff6091e3740 10 client.0 _ll_register_callbacks cb 0x7ff5f00f2f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158dac0 0x55dbf159bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158dac0 0x55dbf159bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43548/0 (socket says 172.21.15.46:43548) 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 -- 172.21.15.46:0/1351751 learned_addr learned my addr 172.21.15.46:0/1351751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158e3e0 0x55dbf159c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf1598ca0 0x55dbf159c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 -- 172.21.15.46:0/1351751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158e3e0 msgr2=0x55dbf159c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158e3e0 0x55dbf159c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 -- 172.21.15.46:0/1351751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf1598ca0 msgr2=0x55dbf159c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf1598ca0 0x55dbf159c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 -- 172.21.15.46:0/1351751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf1611a40 con 0x55dbf158dac0 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ff400640 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158dac0 0x55dbf159bd40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00dc0d0 tx=0x7ff5f00f5480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326b77ed116fa886 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.499+0000 7ff5ed600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff5ed600640 1 -- 172.21.15.46:0/1351751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0049040 con 0x55dbf158dac0 2024-09-10T07:38:27.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff5ed600640 1 -- 172.21.15.46:0/1351751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff5f003b460 con 0x55dbf158dac0 2024-09-10T07:38:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 1 -- 172.21.15.46:0/1351751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e80043d0 con 0x55dbf158dac0 2024-09-10T07:38:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff5ed600640 1 -- 172.21.15.46:0/1351751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff5f0031e50 con 0x55dbf158dac0 2024-09-10T07:38:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 10 client.15813 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 1 -- 172.21.15.46:0/1351751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dbf1612de0 con 0x55dbf158dac0 2024-09-10T07:38:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff5ed600640 1 -- 172.21.15.46:0/1351751 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e80043d0 con 0x55dbf158dac0 2024-09-10T07:38:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff5ed600640 1 -- 172.21.15.46:0/1351751 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7ff5f00b84d0 con 0x55dbf158dac0 2024-09-10T07:38:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 10 client.15813 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 20 client.15813 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dbf1613510 0x55dbf1615950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 4 client.15813 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:38:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.500+0000 7ff6091e3740 1 -- 172.21.15.46:0/1351751 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dbf14fbd10 con 0x55dbf1613510 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.501+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dbf1613510 0x55dbf1615950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.501+0000 7ff5f7e00640 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dbf1613510 0x55dbf1615950 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f53c101e9fb38f5 server_cookie=618ff422dd8be5ca in_seq=0 out_seq=0 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.501+0000 7ff5ed600640 10 client.15813 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.502+0000 7ff5ed600640 1 -- 172.21.15.46:0/1351751 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dbf14fbd10 con 0x55dbf1613510 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.502+0000 7ff5ed600640 10 client.15813 handle_command_reply: tid=0 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.503+0000 7ff6091e3740 1 client.15813 shutdown 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 -- 172.21.15.46:0/1351751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158dac0 msgr2=0x55dbf159bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158dac0 0x55dbf159bd40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00dc0d0 tx=0x7ff5f00f5480 comp rx=0 tx=0).stop 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 -- 172.21.15.46:0/1351751 shutdown_connections 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dbf1613510 0x55dbf1615950 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbf1598ca0 0x55dbf159c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dbf158e3e0 0x55dbf159c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 --2- 172.21.15.46:0/1351751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf158dac0 0x55dbf159bd40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff5ed600640 0 client.15813 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 -- 172.21.15.46:0/1351751 >> 172.21.15.46:0/1351751 conn(0x55dbf158a050 msgr2=0x55dbf1597890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 -- 172.21.15.46:0/1351751 shutdown_connections 2024-09-10T07:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 1 -- 172.21.15.46:0/1351751 wait complete. 2024-09-10T07:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.504+0000 7ff6091e3740 20 client.15813 trim_cache size 0 max 16384 2024-09-10T07:38:27.506 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:38:27.506 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:38:27.506 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "ef6d2902-4c7d-4155-b4c9-242cf4e442fd", 2024-09-10T07:38:27.506 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:38:27.506 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 -- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5e40a2b80 msgr2=0x7ff5e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5e40a2b80 0x7ff5e40a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005fd0 tx=0x7ff5f8009500 comp rx=0 tx=0).stop 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 -- 172.21.15.46:0/3262555151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff600101df0 msgr2=0x7ff60019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff600101df0 0x7ff60019a690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0004770 tx=0x7ff5f0003c10 comp rx=0 tx=0).stop 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 -- 172.21.15.46:0/3262555151 shutdown_connections 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff600106f20 0x7ff60019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5e40a2b80 0x7ff5e40a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6001027a0 0x7ff60019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 --2- 172.21.15.46:0/3262555151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff600101df0 0x7ff60019a690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.505+0000 7ff606800640 1 -- 172.21.15.46:0/3262555151 >> 172.21.15.46:0/3262555151 conn(0x7ff6000fd8b0 msgr2=0x7ff6000febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.506+0000 7ff606800640 1 -- 172.21.15.46:0/3262555151 shutdown_connections 2024-09-10T07:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:27.506+0000 7ff606800640 1 -- 172.21.15.46:0/3262555151 wait complete. 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: pgmap v454: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.7 MiB/s wr, 128 op/s 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:38:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:38:28.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:38:28.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: pgmap v454: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.7 MiB/s wr, 128 op/s 2024-09-10T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: pgmap v454: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.7 MiB/s wr, 128 op/s 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:28.982 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:29.377 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:29.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.599+0000 7f3c30e00640 1 -- 172.21.15.46:0/32369633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 msgr2=0x7f3c2c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.599+0000 7f3c30e00640 1 --2- 172.21.15.46:0/32369633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 0x7f3c2c10e890 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009cd0 tx=0x7f3c2402a8e0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.599+0000 7f3c30e00640 1 -- 172.21.15.46:0/32369633 shutdown_connections 2024-09-10T07:38:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.599+0000 7f3c30e00640 1 --2- 172.21.15.46:0/32369633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 0x7f3c2c10e890 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.599+0000 7f3c30e00640 1 --2- 172.21.15.46:0/32369633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c1019c0 0x7f3c2c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.599+0000 7f3c30e00640 1 --2- 172.21.15.46:0/32369633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c2c1079c0 0x7f3c2c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.599+0000 7f3c30e00640 1 -- 172.21.15.46:0/32369633 >> 172.21.15.46:0/32369633 conn(0x7f3c2c0fd740 msgr2=0x7f3c2c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.600+0000 7f3c30e00640 1 -- 172.21.15.46:0/32369633 shutdown_connections 2024-09-10T07:38:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.600+0000 7f3c30e00640 1 -- 172.21.15.46:0/32369633 wait complete. 2024-09-10T07:38:29.742 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:29 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:29.742 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:29 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:29 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:29.743 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:29 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c30e00640 1 Processor -- start 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c30e00640 1 -- start start 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c30e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c2c1019c0 0x7f3c2c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c30e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 0x7f3c2c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c30e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c1079c0 0x7f3c2c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c30e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c111dc0 con 0x7f3c2c102360 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c30e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c111c40 con 0x7f3c2c1019c0 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c30e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c111f40 con 0x7f3c2c1079c0 2024-09-10T07:38:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.601+0000 7f3c23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 0x7f3c2c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c2b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c2c1019c0 0x7f3c2c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 0x7f3c2c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43560/0 (socket says 172.21.15.46:43560) 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c23e00640 1 -- 172.21.15.46:0/845733756 learned_addr learned my addr 172.21.15.46:0/845733756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c2be00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c1079c0 0x7f3c2c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c23e00640 1 -- 172.21.15.46:0/845733756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c1079c0 msgr2=0x7f3c2c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c23e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c1079c0 0x7f3c2c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c23e00640 1 -- 172.21.15.46:0/845733756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c2c1019c0 msgr2=0x7f3c2c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c23e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c2c1019c0 0x7f3c2c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c23e00640 1 -- 172.21.15.46:0/845733756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c19f470 con 0x7f3c2c102360 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c2b400640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c2c1019c0 0x7f3c2c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.602+0000 7f3c23e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 0x7f3c2c19aa00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400e9f0 tx=0x7f3c1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.603+0000 7f3c28c00640 1 -- 172.21.15.46:0/845733756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400cd80 con 0x7f3c2c102360 2024-09-10T07:38:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.603+0000 7f3c28c00640 1 -- 172.21.15.46:0/845733756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c14005500 con 0x7f3c2c102360 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.603+0000 7f3c28c00640 1 -- 172.21.15.46:0/845733756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14010690 con 0x7f3c2c102360 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.603+0000 7f3c30e00640 1 -- 172.21.15.46:0/845733756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c19f760 con 0x7f3c2c102360 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.603+0000 7f3c30e00640 1 -- 172.21.15.46:0/845733756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c1a7040 con 0x7f3c2c102360 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.606+0000 7f3c3505a740 1 Processor -- start 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.606+0000 7f3c3505a740 1 -- start start 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c28c00640 1 -- 172.21.15.46:0/845733756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c140046c0 con 0x7f3c2c102360 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aa9e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aaab320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c3505a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c31aa043e0 con 0x55c31aa9dd50 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c3505a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c31a9abd80 con 0x55c31aaa8f30 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c3505a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c31aa003a0 con 0x55c31aa9e670 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c2b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c2b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:42988/0 (socket says 172.21.15.46:42988) 2024-09-10T07:38:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c2b400640 1 -- 172.21.15.46:0/3605846265 learned_addr learned my addr 172.21.15.46:0/3605846265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c2be00640 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aa9e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c23e00640 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aaab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c2b400640 1 -- 172.21.15.46:0/3605846265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 msgr2=0x55c31aaab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.607+0000 7f3c2b400640 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aaab320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c2b400640 1 -- 172.21.15.46:0/3605846265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 msgr2=0x55c31aa9e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c2b400640 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aa9e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c2b400640 1 -- 172.21.15.46:0/3605846265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31a9caae0 con 0x55c31aa9e670 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c2b400640 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa89f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0114f0 tx=0x7f3c1c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=208564d229273f5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c28c00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3c100a2b80 0x7f3c100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c20c00640 1 -- 172.21.15.46:0/3605846265 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c01e070 con 0x55c31aa9e670 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c20c00640 1 -- 172.21.15.46:0/3605846265 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c00faa0 con 0x55c31aa9e670 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c20c00640 1 -- 172.21.15.46:0/3605846265 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c019b90 con 0x55c31aa9e670 2024-09-10T07:38:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.608+0000 7f3c28c00640 1 -- 172.21.15.46:0/845733756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3c140cf800 con 0x7f3c2c102360 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c2be00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3c100a2b80 0x7f3c100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 -- 172.21.15.46:0/3605846265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 msgr2=0x55c31aaa89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa89f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0114f0 tx=0x7f3c1c0119b0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 -- 172.21.15.46:0/3605846265 shutdown_connections 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aaab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa89f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3605846265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aa9e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 -- 172.21.15.46:0/3605846265 >> 172.21.15.46:0/3605846265 conn(0x55c31aa9a2e0 msgr2=0x55c31aa9a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c2be00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3c100a2b80 0x7f3c100a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004ab0 tx=0x7f3c24004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 -- 172.21.15.46:0/3605846265 shutdown_connections 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.609+0000 7f3c3505a740 1 -- 172.21.15.46:0/3605846265 wait complete. 2024-09-10T07:38:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.610+0000 7f3c3505a740 1 -- start start 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.610+0000 7f3c3505a740 1 Processor -- start 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aa9dd50 0x55c31aaa5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aaa8f30 0x55c31aaa6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c3505a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c31aaabba0 con 0x55c31aaa8f30 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c3505a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c31a9bf0f0 con 0x55c31aa9dd50 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c3505a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c31aaabd20 con 0x55c31aa9e670 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c3505a740 10 client.0 _ll_register_callbacks cb 0x55c31a9b09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aa9dd50 0x55c31aaa5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aa9dd50 0x55c31aaa5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50240/0 (socket says 172.21.15.46:50240) 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2b400640 1 -- 172.21.15.46:0/3257737700 learned_addr learned my addr 172.21.15.46:0/3257737700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2be00640 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aaa8f30 0x55c31aaa6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c23e00640 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2b400640 1 -- 172.21.15.46:0/3257737700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 msgr2=0x55c31aaa62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2b400640 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2b400640 1 -- 172.21.15.46:0/3257737700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aaa8f30 msgr2=0x55c31aaa6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2b400640 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aaa8f30 0x55c31aaa6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.611+0000 7f3c2b400640 1 -- 172.21.15.46:0/3257737700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31a7753c0 con 0x55c31aa9dd50 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c2b400640 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aa9dd50 0x55c31aaa5da0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c01c040 tx=0x7f3c1c0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38235ebb53b41195 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c19600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c19600640 1 -- 172.21.15.46:0/3257737700 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c027060 con 0x55c31aa9dd50 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c3505a740 1 -- 172.21.15.46:0/3257737700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c1c00faa0 con 0x55c31aa9dd50 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c19600640 1 -- 172.21.15.46:0/3257737700 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3c1c00f5c0 con 0x55c31aa9dd50 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c19600640 1 -- 172.21.15.46:0/3257737700 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3c1c01e040 con 0x55c31aa9dd50 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c3505a740 10 client.25919 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c3505a740 1 -- 172.21.15.46:0/3257737700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c31aab7860 con 0x55c31aa9dd50 2024-09-10T07:38:29.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.612+0000 7f3c19600640 1 -- 172.21.15.46:0/3257737700 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c00faa0 con 0x55c31aa9dd50 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.613+0000 7f3c19600640 1 -- 172.21.15.46:0/3257737700 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f3c1c021030 con 0x55c31aa9dd50 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.613+0000 7f3c3505a740 10 client.25919 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.613+0000 7f3c3505a740 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.613+0000 7f3c3505a740 20 client.25919 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.613+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c31aab8c70 0x55c31aabb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.613+0000 7f3c3505a740 4 client.25919 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.613+0000 7f3c3505a740 1 -- 172.21.15.46:0/3257737700 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c31aa11950 con 0x55c31aab8c70 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.613+0000 7f3c23e00640 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c31aab8c70 0x55c31aabb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.614+0000 7f3c23e00640 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c31aab8c70 0x55c31aabb060 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753cd8c586857a17 server_cookie=db39c6006f08ef01 in_seq=0 out_seq=0 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.614+0000 7f3c19600640 10 client.25919 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.614+0000 7f3c19600640 1 -- 172.21.15.46:0/3257737700 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c31aa11950 con 0x55c31aab8c70 2024-09-10T07:38:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.614+0000 7f3c19600640 10 client.25919 handle_command_reply: tid=0 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.614+0000 7f3c3505a740 1 client.25919 shutdown 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 -- 172.21.15.46:0/3257737700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aa9dd50 msgr2=0x55c31aaa5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aa9dd50 0x55c31aaa5da0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c01c040 tx=0x7f3c1c0056d0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 -- 172.21.15.46:0/3257737700 shutdown_connections 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c31aab8c70 0x55c31aabb060 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aaa8f30 0x55c31aaa6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 --2- 172.21.15.46:0/3257737700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aa9dd50 0x55c31aaa5da0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c19600640 0 client.25919 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 -- 172.21.15.46:0/3257737700 >> 172.21.15.46:0/3257737700 conn(0x55c31aa9a2e0 msgr2=0x55c31aaa0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 -- 172.21.15.46:0/3257737700 shutdown_connections 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 1 -- 172.21.15.46:0/3257737700 wait complete. 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.615+0000 7f3c3505a740 20 client.25919 trim_cache size 0 max 16384 2024-09-10T07:38:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.633+0000 7f3c3505a740 1 Processor -- start 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.634+0000 7f3c3505a740 1 -- start start 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.634+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aaa07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.634+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.634+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aac0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.634+0000 7f3c3505a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c1b0d80 con 0x55c31aa9dd50 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.634+0000 7f3c3505a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c1b2980 con 0x55c31aaa8f30 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.634+0000 7f3c3505a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c1b1450 con 0x55c31aa9e670 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aaa07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aac0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aaa07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43594/0 (socket says 172.21.15.46:43594) 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2b400640 1 -- 172.21.15.46:0/1238968443 learned_addr learned my addr 172.21.15.46:0/1238968443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c23e00640 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2b400640 1 -- 172.21.15.46:0/1238968443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 msgr2=0x55c31aaa4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2b400640 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2b400640 1 -- 172.21.15.46:0/1238968443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 msgr2=0x55c31aac0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2b400640 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aac0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2b400640 1 -- 172.21.15.46:0/1238968443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31aabb5f0 con 0x55c31aa9dd50 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c23e00640 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.635+0000 7f3c2be00640 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aac0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.636+0000 7f3c2b400640 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aaa07b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c000c00 tx=0x7f3c1c009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d044694935d12ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.636+0000 7f3c1be00640 1 -- 172.21.15.46:0/1238968443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c003ac0 con 0x55c31aa9dd50 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.636+0000 7f3c1be00640 1 -- 172.21.15.46:0/1238968443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c003c60 con 0x55c31aa9dd50 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.636+0000 7f3c1be00640 1 -- 172.21.15.46:0/1238968443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00f840 con 0x55c31aa9dd50 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.636+0000 7f3c3505a740 1 -- 172.21.15.46:0/1238968443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 msgr2=0x55c31aaa07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.636+0000 7f3c3505a740 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aaa07b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c000c00 tx=0x7f3c1c009b90 comp rx=0 tx=0).stop 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.637+0000 7f3c3505a740 1 -- 172.21.15.46:0/1238968443 shutdown_connections 2024-09-10T07:38:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.637+0000 7f3c3505a740 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aac0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.637+0000 7f3c3505a740 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9e670 0x55c31aaa4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.637+0000 7f3c3505a740 1 --2- 172.21.15.46:0/1238968443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9dd50 0x55c31aaa07b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.637+0000 7f3c3505a740 1 -- 172.21.15.46:0/1238968443 >> 172.21.15.46:0/1238968443 conn(0x55c31aa9a2e0 msgr2=0x55c31aaa7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.637+0000 7f3c3505a740 1 -- 172.21.15.46:0/1238968443 shutdown_connections 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.637+0000 7f3c3505a740 1 -- 172.21.15.46:0/1238968443 wait complete. 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.637+0000 7f3c3505a740 1 -- start start 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c3505a740 1 Processor -- start 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9dd50 0x55c31ab1bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9e670 0x55c31ab1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c3505a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aaac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c3505a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c1b21f0 con 0x55c31aa9e670 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c3505a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c1b1b20 con 0x55c31aaa8f30 2024-09-10T07:38:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c3505a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c1aed60 con 0x55c31aa9dd50 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c3505a740 10 client.0 _ll_register_callbacks cb 0x7f3c1c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9e670 0x55c31ab1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c2b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9dd50 0x55c31ab1bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9e670 0x55c31ab1c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43604/0 (socket says 172.21.15.46:43604) 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.638+0000 7f3c23e00640 1 -- 172.21.15.46:0/2113165034 learned_addr learned my addr 172.21.15.46:0/2113165034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c2be00640 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aaac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c23e00640 1 -- 172.21.15.46:0/2113165034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9dd50 msgr2=0x55c31ab1bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c23e00640 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9dd50 0x55c31ab1bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c23e00640 1 -- 172.21.15.46:0/2113165034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 msgr2=0x55c31aaac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c23e00640 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aaac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c23e00640 1 -- 172.21.15.46:0/2113165034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31aa9a110 con 0x55c31aa9e670 2024-09-10T07:38:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c23e00640 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9e670 0x55c31ab1c090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c140ccf20 tx=0x7f3c14004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67f42e3600688a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c19600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.639+0000 7f3c19600640 1 -- 172.21.15.46:0/2113165034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c140a0030 con 0x55c31aa9e670 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c19600640 1 -- 172.21.15.46:0/2113165034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3c140982f0 con 0x55c31aa9e670 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 1 -- 172.21.15.46:0/2113165034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c1c003c60 con 0x55c31aa9e670 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c19600640 1 -- 172.21.15.46:0/2113165034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3c140a48f0 con 0x55c31aa9e670 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 10 client.15831 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 1 -- 172.21.15.46:0/2113165034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c31ab230f0 con 0x55c31aa9e670 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c19600640 1 -- 172.21.15.46:0/2113165034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c003c60 con 0x55c31aa9e670 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c19600640 1 -- 172.21.15.46:0/2113165034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f3c1409b0d0 con 0x55c31aa9e670 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 10 client.15831 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 20 client.15831 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c31ab238e0 0x55c31ab25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 4 client.15831 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.640+0000 7f3c3505a740 1 -- 172.21.15.46:0/2113165034 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c31aa0be60 con 0x55c31ab238e0 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.641+0000 7f3c2b400640 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c31ab238e0 0x55c31ab25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.641+0000 7f3c2b400640 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c31ab238e0 0x55c31ab25d00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87695c250e9de823 server_cookie=71bdd4e484d23477 in_seq=0 out_seq=0 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.641+0000 7f3c19600640 10 client.15831 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c19600640 1 -- 172.21.15.46:0/2113165034 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c31aa0be60 con 0x55c31ab238e0 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c19600640 10 client.15831 handle_command_reply: tid=0 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 client.15831 shutdown 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 -- 172.21.15.46:0/2113165034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9e670 msgr2=0x55c31ab1c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9e670 0x55c31ab1c090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c140ccf20 tx=0x7f3c14004ba0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 -- 172.21.15.46:0/2113165034 shutdown_connections 2024-09-10T07:38:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c31ab238e0 0x55c31ab25d00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c31aaa8f30 0x55c31aaac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c31aa9e670 0x55c31ab1c090 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 --2- 172.21.15.46:0/2113165034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c31aa9dd50 0x55c31ab1bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 -- 172.21.15.46:0/2113165034 >> 172.21.15.46:0/2113165034 conn(0x55c31aa9a2e0 msgr2=0x55c31aab3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c19600640 0 client.15831 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 -- 172.21.15.46:0/2113165034 shutdown_connections 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 1 -- 172.21.15.46:0/2113165034 wait complete. 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.642+0000 7f3c3505a740 20 client.15831 trim_cache size 0 max 16384 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:38:29.768 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:38:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:38:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "ef6d2902-4c7d-4155-b4c9-242cf4e442fd": { 2024-09-10T07:38:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:38:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "ef6d2902-4c7d-4155-b4c9-242cf4e442fd", 2024-09-10T07:38:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:38:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:29.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:29.769 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 -- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3c100a2b80 msgr2=0x7f3c100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3c100a2b80 0x7f3c100a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004ab0 tx=0x7f3c24004a40 comp rx=0 tx=0).stop 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 -- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 msgr2=0x7f3c2c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 0x7f3c2c19aa00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400e9f0 tx=0x7f3c1400edf0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 -- 172.21.15.46:0/845733756 shutdown_connections 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c2c1079c0 0x7f3c2c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3c100a2b80 0x7f3c100a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c102360 0x7f3c2c19aa00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 --2- 172.21.15.46:0/845733756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c2c1019c0 0x7f3c2c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 -- 172.21.15.46:0/845733756 >> 172.21.15.46:0/845733756 conn(0x7f3c2c0fd740 msgr2=0x7f3c2c10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 -- 172.21.15.46:0/845733756 shutdown_connections 2024-09-10T07:38:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:29.643+0000 7f3c30e00640 1 -- 172.21.15.46:0/845733756 wait complete. 2024-09-10T07:38:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 70.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:38:30.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:30 smithi181 ceph-mon[35397]: pgmap v455: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.6 KiB/s wr, 27 op/s 2024-09-10T07:38:30.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:30 smithi063 ceph-mon[35020]: pgmap v455: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.6 KiB/s wr, 27 op/s 2024-09-10T07:38:30.656 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:30 smithi046 ceph-mon[30705]: pgmap v455: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.6 KiB/s wr, 27 op/s 2024-09-10T07:38:30.696 INFO:tasks.cephfs.filesystem:scrub status for tag:ef6d2902-4c7d-4155-b4c9-242cf4e442fd - {'path': '/', 'tag': 'ef6d2902-4c7d-4155-b4c9-242cf4e442fd', 'options': 'recursive,force'} 2024-09-10T07:38:30.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:38:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:31.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.984+0000 7f32e5600640 1 -- 172.21.15.46:0/339871047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 msgr2=0x7f32e0112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.984+0000 7f32e5600640 1 --2- 172.21.15.46:0/339871047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 0x7f32e0112650 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009900 tx=0x7f32c802a480 comp rx=0 tx=0).stop 2024-09-10T07:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.984+0000 7f32e5600640 1 -- 172.21.15.46:0/339871047 shutdown_connections 2024-09-10T07:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.984+0000 7f32e5600640 1 --2- 172.21.15.46:0/339871047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e0112b90 0x7f32e0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.984+0000 7f32e5600640 1 --2- 172.21.15.46:0/339871047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 0x7f32e0112650 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.984+0000 7f32e5600640 1 --2- 172.21.15.46:0/339871047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e0074590 0x7f32e0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.985+0000 7f32e5600640 1 -- 172.21.15.46:0/339871047 >> 172.21.15.46:0/339871047 conn(0x7f32e00fd8c0 msgr2=0x7f32e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.985+0000 7f32e5600640 1 -- 172.21.15.46:0/339871047 shutdown_connections 2024-09-10T07:38:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.985+0000 7f32e5600640 1 -- 172.21.15.46:0/339871047 wait complete. 2024-09-10T07:38:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.985+0000 7f32e5600640 1 Processor -- start 2024-09-10T07:38:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32e5600640 1 -- start start 2024-09-10T07:38:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32e5600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e0074590 0x7f32e019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32e5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 0x7f32e019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32e5600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e0112b90 0x7f32e01a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32e5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32e01185b0 con 0x7f32e0074eb0 2024-09-10T07:38:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32e5600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32e0118430 con 0x7f32e0112b90 2024-09-10T07:38:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32e5600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32e0118730 con 0x7f32e0074590 2024-09-10T07:38:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32d7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 0x7f32e019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32df400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e0112b90 0x7f32e01a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32d7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 0x7f32e019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43618/0 (socket says 172.21.15.46:43618) 2024-09-10T07:38:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32d7e00640 1 -- 172.21.15.46:0/2551789995 learned_addr learned my addr 172.21.15.46:0/2551789995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.986+0000 7f32dea00640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e0074590 0x7f32e019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32df400640 1 -- 172.21.15.46:0/2551789995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e0074590 msgr2=0x7f32e019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32df400640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e0074590 0x7f32e019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32df400640 1 -- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 msgr2=0x7f32e019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32df400640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 0x7f32e019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32df400640 1 -- 172.21.15.46:0/2551789995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e01a3b70 con 0x7f32e0112b90 2024-09-10T07:38:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32df400640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e0112b90 0x7f32e01a3440 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f32d8005fd0 tx=0x7f32d800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32d7400640 1 -- 172.21.15.46:0/2551789995 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d800cdf0 con 0x7f32e0112b90 2024-09-10T07:38:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e01a3e60 con 0x7f32e0112b90 2024-09-10T07:38:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32d7400640 1 -- 172.21.15.46:0/2551789995 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32d8005020 con 0x7f32e0112b90 2024-09-10T07:38:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.987+0000 7f32d7400640 1 -- 172.21.15.46:0/2551789995 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8010690 con 0x7f32e0112b90 2024-09-10T07:38:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.988+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e01ab6f0 con 0x7f32e0112b90 2024-09-10T07:38:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.989+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e0074970 con 0x7f32e0112b90 2024-09-10T07:38:31.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.991+0000 7f32d7400640 1 -- 172.21.15.46:0/2551789995 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32d80054d0 con 0x7f32e0112b90 2024-09-10T07:38:31.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.992+0000 7f32d7400640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32c40a2c00 0x7f32c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:31.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.992+0000 7f32d7400640 1 -- 172.21.15.46:0/2551789995 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f32d80cf960 con 0x7f32e0112b90 2024-09-10T07:38:31.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.992+0000 7f32dea00640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32c40a2c00 0x7f32c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:31.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.993+0000 7f32dea00640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32c40a2c00 0x7f32c40a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009bd0 tx=0x7f32d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:31.994+0000 7f32d7400640 1 -- 172.21.15.46:0/2551789995 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32d809c4e0 con 0x7f32e0112b90 2024-09-10T07:38:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.216+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32e0105000 con 0x7f32e0112b90 2024-09-10T07:38:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.218+0000 7f32d7400640 1 -- 172.21.15.46:0/2551789995 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11979 (secure 0 0 0) 0x7f32d809bde0 con 0x7f32e0112b90 2024-09-10T07:38:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:38:32.218 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"btime":"2024-09-10T07:37:35:593020+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:37:35.488899+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:38:32.218 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32c40a2c00 msgr2=0x7f32c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32c40a2c00 0x7f32c40a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009bd0 tx=0x7f32d0005f90 comp rx=0 tx=0).stop 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e0112b90 msgr2=0x7f32e01a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e0112b90 0x7f32e01a3440 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f32d8005fd0 tx=0x7f32d800ef10 comp rx=0 tx=0).stop 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 shutdown_connections 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e0112b90 0x7f32e01a3440 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32c40a2c00 0x7f32c40a50c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e0074eb0 0x7f32e019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 --2- 172.21.15.46:0/2551789995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e0074590 0x7f32e019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 >> 172.21.15.46:0/2551789995 conn(0x7f32e00fd8c0 msgr2=0x7f32e00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:32.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 shutdown_connections 2024-09-10T07:38:32.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:32.220+0000 7f32e5600640 1 -- 172.21.15.46:0/2551789995 wait complete. 2024-09-10T07:38:32.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:32 smithi181 ceph-mon[35397]: pgmap v456: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.0 KiB/s wr, 71 op/s 2024-09-10T07:38:32.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:32 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2551789995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:32.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:32 smithi063 ceph-mon[35020]: pgmap v456: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.0 KiB/s wr, 71 op/s 2024-09-10T07:38:32.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:32 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2551789995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:32.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:32 smithi046 ceph-mon[30705]: pgmap v456: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.0 KiB/s wr, 71 op/s 2024-09-10T07:38:32.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:32 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2551789995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:38:33.128 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id 37a59a3d --release --await' 2024-09-10T07:38:33.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id 37a59a3d --release --await 2024-09-10T07:38:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 -- 172.21.15.46:0/2195567096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074e30 msgr2=0x7f76b810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 --2- 172.21.15.46:0/2195567096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074e30 0x7f76b810dd40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009900 tx=0x7f76a002a490 comp rx=0 tx=0).stop 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 -- 172.21.15.46:0/2195567096 shutdown_connections 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 --2- 172.21.15.46:0/2195567096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b810e280 0x7f76b8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 --2- 172.21.15.46:0/2195567096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074e30 0x7f76b810dd40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 --2- 172.21.15.46:0/2195567096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8074510 0x7f76b80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 -- 172.21.15.46:0/2195567096 >> 172.21.15.46:0/2195567096 conn(0x7f76b80fd6e0 msgr2=0x7f76b80ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 -- 172.21.15.46:0/2195567096 shutdown_connections 2024-09-10T07:38:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.163+0000 7f76b7200640 1 -- 172.21.15.46:0/2195567096 wait complete. 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.164+0000 7f76b7200640 1 Processor -- start 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.164+0000 7f76b7200640 1 -- start start 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.164+0000 7f76b7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074510 0x7f76b819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.164+0000 7f76b7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b8074e30 0x7f76b819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b810e280 0x7f76b819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8113fc0 con 0x7f76b810e280 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8113e40 con 0x7f76b8074510 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8114140 con 0x7f76b8074e30 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074510 0x7f76b819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074510 0x7f76b819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50290/0 (socket says 172.21.15.46:50290) 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b5e00640 1 -- 172.21.15.46:0/920281194 learned_addr learned my addr 172.21.15.46:0/920281194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b6800640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b810e280 0x7f76b819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76afe00640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b8074e30 0x7f76b819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b5e00640 1 -- 172.21.15.46:0/920281194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b8074e30 msgr2=0x7f76b819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b5e00640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b8074e30 0x7f76b819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b5e00640 1 -- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b810e280 msgr2=0x7f76b819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b5e00640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b810e280 0x7f76b819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.165+0000 7f76b5e00640 1 -- 172.21.15.46:0/920281194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b819f440 con 0x7f76b8074510 2024-09-10T07:38:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.166+0000 7f76b5e00640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074510 0x7f76b819a490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f76a800e9f0 tx=0x7f76a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:34.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.166+0000 7f76aea00640 1 -- 172.21.15.46:0/920281194 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a800cd80 con 0x7f76b8074510 2024-09-10T07:38:34.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.166+0000 7f76b7200640 1 -- 172.21.15.46:0/920281194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b819f730 con 0x7f76b8074510 2024-09-10T07:38:34.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.166+0000 7f76aea00640 1 -- 172.21.15.46:0/920281194 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a8010910 con 0x7f76b8074510 2024-09-10T07:38:34.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.166+0000 7f76aea00640 1 -- 172.21.15.46:0/920281194 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a80188f0 con 0x7f76b8074510 2024-09-10T07:38:34.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.166+0000 7f76b7200640 1 -- 172.21.15.46:0/920281194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81a7010 con 0x7f76b8074510 2024-09-10T07:38:34.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.170+0000 7f76bf9f3740 1 Processor -- start 2024-09-10T07:38:34.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.170+0000 7f76bf9f3740 1 -- start start 2024-09-10T07:38:34.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d26ae40 0x56329d2751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d275700 0x56329d277af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56329d1d0650 con 0x56329d26a520 2024-09-10T07:38:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56329d177f10 con 0x56329d275700 2024-09-10T07:38:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56329d1cc610 con 0x56329d26ae40 2024-09-10T07:38:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76aea00640 1 -- 172.21.15.46:0/920281194 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76a8010ac0 con 0x7f76b8074510 2024-09-10T07:38:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43650/0 (socket says 172.21.15.46:43650) 2024-09-10T07:38:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76b6800640 1 -- 172.21.15.46:0/1066453044 learned_addr learned my addr 172.21.15.46:0/1066453044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76afe00640 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d275700 0x56329d277af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.171+0000 7f76b5e00640 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d26ae40 0x56329d2751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76b6800640 1 -- 172.21.15.46:0/1066453044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d26ae40 msgr2=0x56329d2751c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76b6800640 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d26ae40 0x56329d2751c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76b6800640 1 -- 172.21.15.46:0/1066453044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d275700 msgr2=0x56329d277af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76b6800640 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d275700 0x56329d277af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76b6800640 1 -- 172.21.15.46:0/1066453044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329cedf580 con 0x56329d26a520 2024-09-10T07:38:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76b6800640 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26a900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009f40 tx=0x7f76b0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682e47dccc284e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76a7400640 1 -- 172.21.15.46:0/1066453044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0016070 con 0x56329d26a520 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76a7400640 1 -- 172.21.15.46:0/1066453044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b0004d20 con 0x56329d26a520 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.172+0000 7f76a7400640 1 -- 172.21.15.46:0/1066453044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0012850 con 0x56329d26a520 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76aea00640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f769c0a2c00 0x7f769c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76aea00640 1 -- 172.21.15.46:0/920281194 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f76a80cf800 con 0x7f76b8074510 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76b5e00640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f769c0a2c00 0x7f769c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76bf9f3740 1 -- 172.21.15.46:0/1066453044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 msgr2=0x56329d26a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26a900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009f40 tx=0x7f76b0010800 comp rx=0 tx=0).stop 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76bf9f3740 1 -- 172.21.15.46:0/1066453044 shutdown_connections 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d275700 0x56329d277af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d26ae40 0x56329d2751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/1066453044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26a900 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.173+0000 7f76bf9f3740 1 -- 172.21.15.46:0/1066453044 >> 172.21.15.46:0/1066453044 conn(0x56329d266ab0 msgr2=0x56329d266e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.174+0000 7f76bf9f3740 1 -- 172.21.15.46:0/1066453044 shutdown_connections 2024-09-10T07:38:34.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.174+0000 7f76bf9f3740 1 -- 172.21.15.46:0/1066453044 wait complete. 2024-09-10T07:38:34.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.174+0000 7f76b5e00640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f769c0a2c00 0x7f769c0a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f76a809b980 tx=0x7f76a80d5260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:34.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.174+0000 7f76bf9f3740 1 -- start start 2024-09-10T07:38:34.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.174+0000 7f76bf9f3740 1 Processor -- start 2024-09-10T07:38:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 0x56329d272650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d272b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d2730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56329d2782d0 con 0x56329d26ae40 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56329d18b350 con 0x56329d26a520 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56329d278450 con 0x56329d275700 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76bf9f3740 10 client.0 _ll_register_callbacks cb 0x56329d1ae290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d272b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d272b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43658/0 (socket says 172.21.15.46:43658) 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76afe00640 1 -- 172.21.15.46:0/2875522353 learned_addr learned my addr 172.21.15.46:0/2875522353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76b6800640 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 0x56329d272650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.175+0000 7f76b6800640 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d2730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f76afe00640 1 -- 172.21.15.46:0/2875522353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 msgr2=0x56329d2730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f76afe00640 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d2730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f76afe00640 1 -- 172.21.15.46:0/2875522353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 msgr2=0x56329d272650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f76afe00640 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 0x56329d272650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f76afe00640 1 -- 172.21.15.46:0/2875522353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329cfa9960 con 0x56329d26ae40 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f76afe00640 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d272b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f76a0033000 tx=0x7f76a0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd02138732b4f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f769be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f769be00640 1 -- 172.21.15.46:0/2875522353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0035040 con 0x56329d26ae40 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f76bf9f3740 1 -- 172.21.15.46:0/2875522353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b0004d20 con 0x56329d26ae40 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.176+0000 7f769be00640 1 -- 172.21.15.46:0/2875522353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f76a00048c0 con 0x56329d26ae40 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.177+0000 7f769be00640 1 -- 172.21.15.46:0/2875522353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f76a004e050 con 0x56329d26ae40 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.177+0000 7f76bf9f3740 10 client.15849 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.177+0000 7f76bf9f3740 1 -- 172.21.15.46:0/2875522353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56329d283fa0 con 0x56329d26ae40 2024-09-10T07:38:34.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.177+0000 7f769be00640 1 -- 172.21.15.46:0/2875522353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b0004d20 con 0x56329d26ae40 2024-09-10T07:38:34.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.177+0000 7f769be00640 1 -- 172.21.15.46:0/2875522353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f76a0031a70 con 0x56329d26ae40 2024-09-10T07:38:34.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.177+0000 7f76bf9f3740 10 client.15849 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:34.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.177+0000 7f76bf9f3740 10 client.15849 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:38:34.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.177+0000 7f76bf9f3740 20 client.15849 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:34.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.178+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56329d285420 0x56329d287870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.178+0000 7f76bf9f3740 4 client.15849 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:34.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.178+0000 7f76bf9f3740 1 -- 172.21.15.46:0/2875522353 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56329d1ddbc0 con 0x56329d285420 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.178+0000 7f76b6800640 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56329d285420 0x56329d287870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.178+0000 7f76b6800640 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56329d285420 0x56329d287870 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a45300dff9977d5 server_cookie=5c074278511e4554 in_seq=0 out_seq=0 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.178+0000 7f769be00640 10 client.15849 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f769be00640 1 -- 172.21.15.46:0/2875522353 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56329d1ddbc0 con 0x56329d285420 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f769be00640 10 client.15849 handle_command_reply: tid=0 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 client.15849 shutdown 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 -- 172.21.15.46:0/2875522353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 msgr2=0x56329d272b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d272b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f76a0033000 tx=0x7f76a0005250 comp rx=0 tx=0).stop 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 -- 172.21.15.46:0/2875522353 shutdown_connections 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56329d285420 0x56329d287870 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d2730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d272b90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/2875522353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 0x56329d272650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.179+0000 7f76bf9f3740 1 -- 172.21.15.46:0/2875522353 >> 172.21.15.46:0/2875522353 conn(0x56329d266ab0 msgr2=0x56329d265cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.180+0000 7f769be00640 0 client.15849 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.180+0000 7f76bf9f3740 1 -- 172.21.15.46:0/2875522353 shutdown_connections 2024-09-10T07:38:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.180+0000 7f76bf9f3740 1 -- 172.21.15.46:0/2875522353 wait complete. 2024-09-10T07:38:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.180+0000 7f76bf9f3740 20 client.15849 trim_cache size 0 max 16384 2024-09-10T07:38:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76bf9f3740 1 Processor -- start 2024-09-10T07:38:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76bf9f3740 1 -- start start 2024-09-10T07:38:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26ae40 0x56329d271590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d28d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81b0ce0 con 0x56329d26a520 2024-09-10T07:38:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81b28e0 con 0x56329d26ae40 2024-09-10T07:38:34.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81b13b0 con 0x56329d275700 2024-09-10T07:38:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43668/0 (socket says 172.21.15.46:43668) 2024-09-10T07:38:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76b6800640 1 -- 172.21.15.46:0/3663449322 learned_addr learned my addr 172.21.15.46:0/3663449322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.198+0000 7f76afe00640 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26ae40 0x56329d271590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76b6800640 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d28d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76afe00640 1 -- 172.21.15.46:0/3663449322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 msgr2=0x56329d28d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76afe00640 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d28d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76afe00640 1 -- 172.21.15.46:0/3663449322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 msgr2=0x56329d26db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76afe00640 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76afe00640 1 -- 172.21.15.46:0/3663449322 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329d270ba0 con 0x56329d26ae40 2024-09-10T07:38:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76b6800640 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76afe00640 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26ae40 0x56329d271590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f76a002a9b0 tx=0x7f76a002ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44379631a9ee0b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76a6a00640 1 -- 172.21.15.46:0/3663449322 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0035030 con 0x56329d26ae40 2024-09-10T07:38:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76a6a00640 1 -- 172.21.15.46:0/3663449322 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a0040070 con 0x56329d26ae40 2024-09-10T07:38:34.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.199+0000 7f76a6a00640 1 -- 172.21.15.46:0/3663449322 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00319a0 con 0x56329d26ae40 2024-09-10T07:38:34.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 -- 172.21.15.46:0/3663449322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26ae40 msgr2=0x56329d271590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26ae40 0x56329d271590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f76a002a9b0 tx=0x7f76a002ab90 comp rx=0 tx=0).stop 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 -- 172.21.15.46:0/3663449322 shutdown_connections 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d28d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26ae40 0x56329d271590 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/3663449322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26a520 0x56329d26db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 -- 172.21.15.46:0/3663449322 >> 172.21.15.46:0/3663449322 conn(0x56329d266ab0 msgr2=0x56329d265cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 -- 172.21.15.46:0/3663449322 shutdown_connections 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.200+0000 7f76bf9f3740 1 -- 172.21.15.46:0/3663449322 wait complete. 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.201+0000 7f76bf9f3740 1 -- start start 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.201+0000 7f76bf9f3740 1 Processor -- start 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.201+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 0x56329d27d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d27dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76bf9f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d27a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81b2150 con 0x56329d26ae40 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81b1a80 con 0x56329d26a520 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76bf9f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81aecc0 con 0x56329d275700 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76bf9f3740 10 client.0 _ll_register_callbacks cb 0x7f76a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d27dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76b6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 0x56329d27d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d27dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43674/0 (socket says 172.21.15.46:43674) 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76afe00640 1 -- 172.21.15.46:0/585328325 learned_addr learned my addr 172.21.15.46:0/585328325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76b6800640 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d27a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76afe00640 1 -- 172.21.15.46:0/585328325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 msgr2=0x56329d27a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:34.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76afe00640 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d27a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76afe00640 1 -- 172.21.15.46:0/585328325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 msgr2=0x56329d27d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:34.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76afe00640 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 0x56329d27d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.202+0000 7f76afe00640 1 -- 172.21.15.46:0/585328325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329d27e250 con 0x56329d26ae40 2024-09-10T07:38:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.203+0000 7f76afe00640 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d27dbe0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f76a0031f60 tx=0x7f76a0003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77a0344b8f4ae76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.203+0000 7f769be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:38:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.203+0000 7f769be00640 1 -- 172.21.15.46:0/585328325 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0035070 con 0x56329d26ae40 2024-09-10T07:38:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.203+0000 7f76bf9f3740 1 -- 172.21.15.46:0/585328325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76a0040070 con 0x56329d26ae40 2024-09-10T07:38:34.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.203+0000 7f769be00640 1 -- 172.21.15.46:0/585328325 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f76a0003c80 con 0x56329d26ae40 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.203+0000 7f769be00640 1 -- 172.21.15.46:0/585328325 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f76a0064050 con 0x56329d26ae40 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.203+0000 7f76bf9f3740 10 client.15855 fetch_fsmap learned FSMap version 34 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.203+0000 7f76bf9f3740 1 -- 172.21.15.46:0/585328325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56329d2f0760 con 0x56329d26ae40 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f769be00640 1 -- 172.21.15.46:0/585328325 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a0040070 con 0x56329d26ae40 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f769be00640 1 -- 172.21.15.46:0/585328325 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f76a00046c0 con 0x56329d26ae40 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f76bf9f3740 10 client.15855 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f76bf9f3740 10 client.15855 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f76bf9f3740 20 client.15855 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56329d2f0ee0 0x56329d2f3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f76bf9f3740 4 client.15855 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "37a59a3d", "release": true, "await": true}] 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f76bf9f3740 1 -- 172.21.15.46:0/585328325 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "37a59a3d", "release": true, "await": true}) -- 0x56329d1d80d0 con 0x56329d2f0ee0 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.204+0000 7f76b6800640 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56329d2f0ee0 0x56329d2f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.205+0000 7f76b6800640 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56329d2f0ee0 0x56329d2f3300 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e6d24e7058aa0a server_cookie=5deca856c08e9517 in_seq=0 out_seq=0 2024-09-10T07:38:34.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.205+0000 7f769be00640 10 client.15855 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.230+0000 7f769be00640 1 -- 172.21.15.46:0/585328325 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56329d1d80d0 con 0x56329d2f0ee0 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.230+0000 7f769be00640 10 client.15855 handle_command_reply: tid=0 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.230+0000 7f76bf9f3740 1 client.15855 shutdown 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.231+0000 7f76bf9f3740 1 -- 172.21.15.46:0/585328325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 msgr2=0x56329d27dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.231+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d27dbe0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f76a0031f60 tx=0x7f76a0003950 comp rx=0 tx=0).stop 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.231+0000 7f76bf9f3740 1 -- 172.21.15.46:0/585328325 shutdown_connections 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.231+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56329d2f0ee0 0x56329d2f3300 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.231+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56329d275700 0x56329d27a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.231+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56329d26ae40 0x56329d27dbe0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.231+0000 7f76bf9f3740 1 --2- 172.21.15.46:0/585328325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56329d26a520 0x56329d27d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.231+0000 7f76bf9f3740 1 -- 172.21.15.46:0/585328325 >> 172.21.15.46:0/585328325 conn(0x56329d266ab0 msgr2=0x56329d2725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.232+0000 7f769be00640 0 client.15855 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.240+0000 7f76bf9f3740 1 -- 172.21.15.46:0/585328325 shutdown_connections 2024-09-10T07:38:34.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.241+0000 7f76bf9f3740 1 -- 172.21.15.46:0/585328325 wait complete. 2024-09-10T07:38:34.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.241+0000 7f76bf9f3740 20 client.15855 trim_cache size 0 max 16384 2024-09-10T07:38:34.241 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:38:34.241 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 34, 2024-09-10T07:38:34.241 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:38:34.241 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 8, 2024-09-10T07:38:34.241 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "37a59a3d": { 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 8, 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 150.0, 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:38:34.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:38:34.243 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:38:34.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:34.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:34.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:34.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:34.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:38:34.243 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.242+0000 7f76b7200640 1 -- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f769c0a2c00 msgr2=0x7f769c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.242+0000 7f76b7200640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f769c0a2c00 0x7f769c0a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f76a809b980 tx=0x7f76a80d5260 comp rx=0 tx=0).stop 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.242+0000 7f76b7200640 1 -- 172.21.15.46:0/920281194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074510 msgr2=0x7f76b819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.242+0000 7f76b7200640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074510 0x7f76b819a490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f76a800e9f0 tx=0x7f76a800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.243+0000 7f76b7200640 1 -- 172.21.15.46:0/920281194 shutdown_connections 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.243+0000 7f76b7200640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f769c0a2c00 0x7f769c0a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.243+0000 7f76b7200640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b810e280 0x7f76b819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.243+0000 7f76b7200640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76b8074e30 0x7f76b819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.243+0000 7f76b7200640 1 --2- 172.21.15.46:0/920281194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8074510 0x7f76b819a490 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.243+0000 7f76b7200640 1 -- 172.21.15.46:0/920281194 >> 172.21.15.46:0/920281194 conn(0x7f76b80fd6e0 msgr2=0x7f76b80fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.244+0000 7f76b7200640 1 -- 172.21.15.46:0/920281194 shutdown_connections 2024-09-10T07:38:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:38:34.244+0000 7f76b7200640 1 -- 172.21.15.46:0/920281194 wait complete. 2024-09-10T07:38:34.340 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:34 smithi046 ceph-mon[30705]: pgmap v457: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.9 KiB/s wr, 101 op/s 2024-09-10T07:38:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:34 smithi181 ceph-mon[35397]: pgmap v457: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.9 KiB/s wr, 101 op/s 2024-09-10T07:38:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:34 smithi063 ceph-mon[35020]: pgmap v457: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.9 KiB/s wr, 101 op/s 2024-09-10T07:38:35.338 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '37a59a3d', total seconds elapsed: 34.4 2024-09-10T07:38:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 75.0 s, 549.8 tps, lat 1105.164 ms stddev 5440.362 2024-09-10T07:38:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:36 smithi181 ceph-mon[35397]: pgmap v458: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 10 KiB/s wr, 101 op/s 2024-09-10T07:38:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:38:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:38:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:38:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:38:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:38:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:36 smithi063 ceph-mon[35020]: pgmap v458: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 10 KiB/s wr, 101 op/s 2024-09-10T07:38:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:38:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:38:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:38:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:38:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:38:36.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:36 smithi046 ceph-mon[30705]: pgmap v458: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 10 KiB/s wr, 101 op/s 2024-09-10T07:38:36.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:38:36.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:38:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:38:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:38:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:38:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:38 smithi181 ceph-mon[35397]: pgmap v459: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 344 KiB/s wr, 144 op/s 2024-09-10T07:38:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:38 smithi063 ceph-mon[35020]: pgmap v459: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 344 KiB/s wr, 144 op/s 2024-09-10T07:38:38.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:38 smithi046 ceph-mon[30705]: pgmap v459: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 344 KiB/s wr, 144 op/s 2024-09-10T07:38:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:39 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:38:39.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:39 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:38:39.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:39 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:38:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.383+0000 7ff13b400640 1 -- 172.21.15.46:0/1933914957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff134102d00 msgr2=0x7ff134105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.383+0000 7ff13b400640 1 --2- 172.21.15.46:0/1933914957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff134102d00 0x7ff134105170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff120009860 tx=0x7ff12002a340 comp rx=0 tx=0).stop 2024-09-10T07:38:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.383+0000 7ff13b400640 1 -- 172.21.15.46:0/1933914957 shutdown_connections 2024-09-10T07:38:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.383+0000 7ff13b400640 1 --2- 172.21.15.46:0/1933914957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1341056b0 0x7ff134105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.383+0000 7ff13b400640 1 --2- 172.21.15.46:0/1933914957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff134102d00 0x7ff134105170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.383+0000 7ff13b400640 1 --2- 172.21.15.46:0/1933914957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1341003b0 0x7ff1341027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.383+0000 7ff13b400640 1 -- 172.21.15.46:0/1933914957 >> 172.21.15.46:0/1933914957 conn(0x7ff1340fc1e0 msgr2=0x7ff1340fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.385+0000 7ff13b400640 1 -- 172.21.15.46:0/1933914957 shutdown_connections 2024-09-10T07:38:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.385+0000 7ff13b400640 1 -- 172.21.15.46:0/1933914957 wait complete. 2024-09-10T07:38:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.386+0000 7ff13b400640 1 Processor -- start 2024-09-10T07:38:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.386+0000 7ff13b400640 1 -- start start 2024-09-10T07:38:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.386+0000 7ff13b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1341003b0 0x7ff13410b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.386+0000 7ff13b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff134102d00 0x7ff13410bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.386+0000 7ff13b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1341056b0 0x7ff1341ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.386+0000 7ff13b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff134113dc0 con 0x7ff134102d00 2024-09-10T07:38:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.386+0000 7ff13b400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff134113c40 con 0x7ff1341003b0 2024-09-10T07:38:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.386+0000 7ff13b400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff134113f40 con 0x7ff1341056b0 2024-09-10T07:38:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.387+0000 7ff139400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1341056b0 0x7ff1341ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.387+0000 7ff139400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1341056b0 0x7ff1341ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48450/0 (socket says 172.21.15.46:48450) 2024-09-10T07:38:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.387+0000 7ff139400640 1 -- 172.21.15.46:0/3587517865 learned_addr learned my addr 172.21.15.46:0/3587517865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.387+0000 7ff133e00640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff134102d00 0x7ff13410bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.387+0000 7ff138a00640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1341003b0 0x7ff13410b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.388+0000 7ff139400640 1 -- 172.21.15.46:0/3587517865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1341003b0 msgr2=0x7ff13410b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.388+0000 7ff139400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1341003b0 0x7ff13410b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.388+0000 7ff139400640 1 -- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff134102d00 msgr2=0x7ff13410bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.388+0000 7ff139400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff134102d00 0x7ff13410bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.388+0000 7ff139400640 1 -- 172.21.15.46:0/3587517865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13410ef60 con 0x7ff1341056b0 2024-09-10T07:38:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.388+0000 7ff139400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1341056b0 0x7ff1341ae4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009fd0 tx=0x7ff12c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.388+0000 7ff131600640 1 -- 172.21.15.46:0/3587517865 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c004b20 con 0x7ff1341056b0 2024-09-10T07:38:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.389+0000 7ff131600640 1 -- 172.21.15.46:0/3587517865 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff12c003e50 con 0x7ff1341056b0 2024-09-10T07:38:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.389+0000 7ff131600640 1 -- 172.21.15.46:0/3587517865 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00b630 con 0x7ff1341056b0 2024-09-10T07:38:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.389+0000 7ff13b400640 1 -- 172.21.15.46:0/3587517865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13406bcd0 con 0x7ff1341056b0 2024-09-10T07:38:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.389+0000 7ff13b400640 1 -- 172.21.15.46:0/3587517865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13406c1a0 con 0x7ff1341056b0 2024-09-10T07:38:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.393+0000 7ff131600640 1 -- 172.21.15.46:0/3587517865 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff12c01d020 con 0x7ff1341056b0 2024-09-10T07:38:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.393+0000 7ff13e1c4740 1 Processor -- start 2024-09-10T07:38:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.394+0000 7ff13e1c4740 1 -- start start 2024-09-10T07:38:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 0x55d3e45fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e4600f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d3e455e100 con 0x55d3e45f99d0 2024-09-10T07:38:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3e45061b0 con 0x55d3e45fea60 2024-09-10T07:38:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d3e455a0c0 con 0x55d3e45f9090 2024-09-10T07:38:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff133e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff133e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48460/0 (socket says 172.21.15.46:48460) 2024-09-10T07:38:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.395+0000 7ff133e00640 1 -- 172.21.15.46:0/3446381674 learned_addr learned my addr 172.21.15.46:0/3446381674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.396+0000 7ff133e00640 1 --2- 172.21.15.46:0/3446381674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e4600f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.396+0000 7ff133e00640 1 -- 172.21.15.46:0/3446381674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 msgr2=0x55d3e4600f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.396+0000 7ff131600640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff11c0a2d30 0x7ff11c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.396+0000 7ff133e00640 1 --2- 172.21.15.46:0/3446381674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e4600f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.396+0000 7ff133e00640 1 -- 172.21.15.46:0/3446381674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 msgr2=0x55d3e45fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.396+0000 7ff133e00640 1 --2- 172.21.15.46:0/3446381674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 0x55d3e45fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.396+0000 7ff133e00640 1 -- 172.21.15.46:0/3446381674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e4271210 con 0x55d3e45f9090 2024-09-10T07:38:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.397+0000 7ff131600640 1 -- 172.21.15.46:0/3587517865 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff12c0cf4b0 con 0x7ff1341056b0 2024-09-10T07:38:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.397+0000 7ff133e00640 1 --2- 172.21.15.46:0/3446381674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f9490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff120000c00 tx=0x7ff120005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=507ceb50b0878fed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.397+0000 7ff125c00640 1 -- 172.21.15.46:0/3446381674 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120037040 con 0x55d3e45f9090 2024-09-10T07:38:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.397+0000 7ff125c00640 1 -- 172.21.15.46:0/3446381674 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff120005050 con 0x55d3e45f9090 2024-09-10T07:38:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.397+0000 7ff125c00640 1 -- 172.21.15.46:0/3446381674 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004920 con 0x55d3e45f9090 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.398+0000 7ff139400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff11c0a2d30 0x7ff11c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.398+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3446381674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 msgr2=0x55d3e45f9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.398+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3446381674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f9490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff120000c00 tx=0x7ff120005a90 comp rx=0 tx=0).stop 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.399+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3446381674 shutdown_connections 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.399+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3446381674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e4600f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.399+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3446381674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 0x55d3e45fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.399+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3446381674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f9490 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.399+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3446381674 >> 172.21.15.46:0/3446381674 conn(0x55d3e45f4700 msgr2=0x55d3e45f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.399+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3446381674 shutdown_connections 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.399+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3446381674 wait complete. 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 1 -- start start 2024-09-10T07:38:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 1 Processor -- start 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff139400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff11c0a2d30 0x7ff11c0a51f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0cf790 tx=0x7ff12c0cf700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e46032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 0x55d3e46037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e460a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d3e4601a20 con 0x55d3e45f99d0 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3e44e7090 con 0x55d3e45fea60 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d3e4601ba0 con 0x55d3e45f9090 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.400+0000 7ff13e1c4740 10 client.0 _ll_register_callbacks cb 0x55d3e453bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e460a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e460a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43578/0 (socket says 172.21.15.46:43578) 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 -- 172.21.15.46:0/3604803263 learned_addr learned my addr 172.21.15.46:0/3604803263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e46032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff138a00640 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 0x55d3e46037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 -- 172.21.15.46:0/3604803263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 msgr2=0x55d3e46032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e46032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 -- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 msgr2=0x55d3e46037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 0x55d3e46037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff133e00640 1 -- 172.21.15.46:0/3604803263 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e433d620 con 0x55d3e45fea60 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.401+0000 7ff138a00640 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 0x55d3e46037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.402+0000 7ff133e00640 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e460a7b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55d3e4602870 tx=0x7ff120030570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5b6f764277f7d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.402+0000 7ff11a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.402+0000 7ff11a000640 1 -- 172.21.15.46:0/3604803263 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120037070 con 0x55d3e45fea60 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.402+0000 7ff11a000640 1 -- 172.21.15.46:0/3604803263 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff120005420 con 0x55d3e45fea60 2024-09-10T07:38:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.403+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3604803263 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff120005050 con 0x55d3e45fea60 2024-09-10T07:38:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.404+0000 7ff11a000640 1 -- 172.21.15.46:0/3604803263 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff1200046c0 con 0x55d3e45fea60 2024-09-10T07:38:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.404+0000 7ff13e1c4740 10 client.25961 fetch_fsmap learned FSMap version 35 2024-09-10T07:38:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.404+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3604803263 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d3e460d5a0 con 0x55d3e45fea60 2024-09-10T07:38:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.405+0000 7ff11a000640 1 -- 172.21.15.46:0/3604803263 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff120005050 con 0x55d3e45fea60 2024-09-10T07:38:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.405+0000 7ff11a000640 1 -- 172.21.15.46:0/3604803263 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8390+0+0 (secure 0 0 0) 0x7ff12004d040 con 0x55d3e45fea60 2024-09-10T07:38:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.406+0000 7ff13e1c4740 10 client.25961 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T07:38:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.406+0000 7ff13e1c4740 10 client.25961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.406+0000 7ff13e1c4740 20 client.25961 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.406+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3e460e8f0 0x55d3e4610ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.406+0000 7ff13e1c4740 4 client.25961 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.406+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3604803263 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3e456b830 con 0x55d3e460e8f0 2024-09-10T07:38:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.409+0000 7ff138a00640 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3e460e8f0 0x55d3e4610ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.411+0000 7ff11a000640 10 client.25961 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.411+0000 7ff138a00640 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3e460e8f0 0x55d3e4610ce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b78a451cd0f88d server_cookie=9d4dc4dd443aad6d in_seq=0 out_seq=0 2024-09-10T07:38:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.412+0000 7ff11a000640 1 -- 172.21.15.46:0/3604803263 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3e456b830 con 0x55d3e460e8f0 2024-09-10T07:38:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.412+0000 7ff11a000640 10 client.25961 handle_command_reply: tid=0 2024-09-10T07:38:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.412+0000 7ff13e1c4740 1 client.25961 shutdown 2024-09-10T07:38:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3604803263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 msgr2=0x55d3e460a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e460a7b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55d3e4602870 tx=0x7ff120030570 comp rx=0 tx=0).stop 2024-09-10T07:38:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3604803263 shutdown_connections 2024-09-10T07:38:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3e460e8f0 0x55d3e4610ce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45fea60 0x55d3e460a7b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45f99d0 0x55d3e46037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3604803263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e46032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff11a000640 0 client.25961 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.413+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3604803263 >> 172.21.15.46:0/3604803263 conn(0x55d3e45f4700 msgr2=0x55d3e45f5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.414+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3604803263 shutdown_connections 2024-09-10T07:38:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.414+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3604803263 wait complete. 2024-09-10T07:38:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.414+0000 7ff13e1c4740 20 client.25961 trim_cache size 0 max 16384 2024-09-10T07:38:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.462+0000 7ff13e1c4740 1 Processor -- start 2024-09-10T07:38:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.462+0000 7ff13e1c4740 1 -- start start 2024-09-10T07:38:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f99d0 0x55d3e460a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45fea60 0x55d3e4615500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341c1b80 con 0x55d3e45fea60 2024-09-10T07:38:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341c37b0 con 0x55d3e45f99d0 2024-09-10T07:38:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341c2250 con 0x55d3e45f9090 2024-09-10T07:38:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff138a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f99d0 0x55d3e460a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff138a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f99d0 0x55d3e460a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43582/0 (socket says 172.21.15.46:43582) 2024-09-10T07:38:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.463+0000 7ff138a00640 1 -- 172.21.15.46:0/3601681320 learned_addr learned my addr 172.21.15.46:0/3601681320 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.464+0000 7ff133e00640 1 --2- 172.21.15.46:0/3601681320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.464+0000 7ff138a00640 1 -- 172.21.15.46:0/3601681320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 msgr2=0x55d3e45f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.464+0000 7ff138a00640 1 --2- 172.21.15.46:0/3601681320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.464+0000 7ff138a00640 1 -- 172.21.15.46:0/3601681320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45fea60 msgr2=0x55d3e4615500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.464+0000 7ff138a00640 1 --2- 172.21.15.46:0/3601681320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45fea60 0x55d3e4615500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.464+0000 7ff138a00640 1 -- 172.21.15.46:0/3601681320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e45f4520 con 0x55d3e45f99d0 2024-09-10T07:38:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.465+0000 7ff138a00640 1 --2- 172.21.15.46:0/3601681320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f99d0 0x55d3e460a6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff1280045f0 tx=0x7ff12800c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=314f9145f857861d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.465+0000 7ff125200640 1 -- 172.21.15.46:0/3601681320 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128005020 con 0x55d3e45f99d0 2024-09-10T07:38:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.465+0000 7ff125200640 1 -- 172.21.15.46:0/3601681320 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff128004730 con 0x55d3e45f99d0 2024-09-10T07:38:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.465+0000 7ff125200640 1 -- 172.21.15.46:0/3601681320 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128019750 con 0x55d3e45f99d0 2024-09-10T07:38:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.467+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3601681320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f99d0 msgr2=0x55d3e460a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.467+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3601681320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f99d0 0x55d3e460a6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff1280045f0 tx=0x7ff12800c380 comp rx=0 tx=0).stop 2024-09-10T07:38:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.467+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3601681320 shutdown_connections 2024-09-10T07:38:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.467+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3601681320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e45fea60 0x55d3e4615500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.467+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3601681320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f99d0 0x55d3e460a6d0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff1280045f0 tx=0x7ff12800c380 comp rx=0 tx=0).stop 2024-09-10T07:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.467+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/3601681320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45f9090 0x55d3e45f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.467+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3601681320 >> 172.21.15.46:0/3601681320 conn(0x55d3e45f4700 msgr2=0x55d3e45f3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.468+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3601681320 shutdown_connections 2024-09-10T07:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.468+0000 7ff13e1c4740 1 -- 172.21.15.46:0/3601681320 wait complete. 2024-09-10T07:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.468+0000 7ff13e1c4740 1 -- start start 2024-09-10T07:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.469+0000 7ff13e1c4740 1 Processor -- start 2024-09-10T07:38:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.471+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f9090 0x55d3e4607c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.471+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45fea60 0x55d3e46081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.471+0000 7ff13e1c4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e46091d0 0x55d3e4609630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.471+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341c3020 con 0x55d3e46091d0 2024-09-10T07:38:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.471+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341c2950 con 0x55d3e45f9090 2024-09-10T07:38:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.471+0000 7ff13e1c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341bfb90 con 0x55d3e45fea60 2024-09-10T07:38:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.471+0000 7ff13e1c4740 10 client.0 _ll_register_callbacks cb 0x7ff12c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.472+0000 7ff138a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45fea60 0x55d3e46081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.472+0000 7ff138a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45fea60 0x55d3e46081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48500/0 (socket says 172.21.15.46:48500) 2024-09-10T07:38:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.472+0000 7ff138a00640 1 -- 172.21.15.46:0/4146305660 learned_addr learned my addr 172.21.15.46:0/4146305660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.472+0000 7ff133e00640 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e46091d0 0x55d3e4609630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.473+0000 7ff133e00640 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f9090 0x55d3e4607c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.474+0000 7ff138a00640 1 -- 172.21.15.46:0/4146305660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f9090 msgr2=0x55d3e4607c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.474+0000 7ff138a00640 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f9090 0x55d3e4607c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.474+0000 7ff138a00640 1 -- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e46091d0 msgr2=0x55d3e4609630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.474+0000 7ff138a00640 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e46091d0 0x55d3e4609630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.474+0000 7ff138a00640 1 -- 172.21.15.46:0/4146305660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e4609bc0 con 0x55d3e45fea60 2024-09-10T07:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.474+0000 7ff133e00640 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e46091d0 0x55d3e4609630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.475+0000 7ff138a00640 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45fea60 0x55d3e46081c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff128005e10 tx=0x7ff128004090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe56013b0781a4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.475+0000 7ff11a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.475+0000 7ff11a000640 1 -- 172.21.15.46:0/4146305660 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1280048e0 con 0x55d3e45fea60 2024-09-10T07:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.475+0000 7ff11a000640 1 -- 172.21.15.46:0/4146305660 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff128004a80 con 0x55d3e45fea60 2024-09-10T07:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.475+0000 7ff13e1c4740 1 -- 172.21.15.46:0/4146305660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff128004730 con 0x55d3e45fea60 2024-09-10T07:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.476+0000 7ff11a000640 1 -- 172.21.15.46:0/4146305660 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff128035050 con 0x55d3e45fea60 2024-09-10T07:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.476+0000 7ff13e1c4740 10 client.25963 fetch_fsmap learned FSMap version 35 2024-09-10T07:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.476+0000 7ff13e1c4740 1 -- 172.21.15.46:0/4146305660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d3e4679600 con 0x55d3e45fea60 2024-09-10T07:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.476+0000 7ff11a000640 1 -- 172.21.15.46:0/4146305660 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff128004730 con 0x55d3e45fea60 2024-09-10T07:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.479+0000 7ff11a000640 1 -- 172.21.15.46:0/4146305660 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 35) ==== 8390+0+0 (secure 0 0 0) 0x7ff12802a020 con 0x55d3e45fea60 2024-09-10T07:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.479+0000 7ff13e1c4740 10 client.25963 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T07:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.479+0000 7ff13e1c4740 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.479+0000 7ff13e1c4740 20 client.25963 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.479+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3e4679f20 0x55d3e467c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.479+0000 7ff13e1c4740 4 client.25963 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.479+0000 7ff13e1c4740 1 -- 172.21.15.46:0/4146305660 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3e4565b80 con 0x55d3e4679f20 2024-09-10T07:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.480+0000 7ff133e00640 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3e4679f20 0x55d3e467c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.481+0000 7ff11a000640 10 client.25963 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.481+0000 7ff133e00640 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3e4679f20 0x55d3e467c340 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3097f649cf20563 server_cookie=eae61bf626d6ba31 in_seq=0 out_seq=0 2024-09-10T07:38:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.503+0000 7ff11a000640 1 -- 172.21.15.46:0/4146305660 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3e4565b80 con 0x55d3e4679f20 2024-09-10T07:38:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.503+0000 7ff11a000640 10 client.25963 handle_command_reply: tid=0 2024-09-10T07:38:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 client.25963 shutdown 2024-09-10T07:38:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 -- 172.21.15.46:0/4146305660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45fea60 msgr2=0x55d3e46081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45fea60 0x55d3e46081c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff128005e10 tx=0x7ff128004090 comp rx=0 tx=0).stop 2024-09-10T07:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 -- 172.21.15.46:0/4146305660 shutdown_connections 2024-09-10T07:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3e4679f20 0x55d3e467c340 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3e46091d0 0x55d3e4609630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff11a000640 0 client.25963 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d3e45fea60 0x55d3e46081c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 --2- 172.21.15.46:0/4146305660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3e45f9090 0x55d3e4607c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 -- 172.21.15.46:0/4146305660 >> 172.21.15.46:0/4146305660 conn(0x55d3e45f4700 msgr2=0x55d3e45f3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.504+0000 7ff13e1c4740 1 -- 172.21.15.46:0/4146305660 shutdown_connections 2024-09-10T07:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.505+0000 7ff13e1c4740 1 -- 172.21.15.46:0/4146305660 wait complete. 2024-09-10T07:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.505+0000 7ff13e1c4740 20 client.25963 trim_cache size 0 max 16384 2024-09-10T07:38:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:38:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:38:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:38:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.505+0000 7ff13b400640 1 -- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff11c0a2d30 msgr2=0x7ff11c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.505+0000 7ff13b400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff11c0a2d30 0x7ff11c0a51f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0cf790 tx=0x7ff12c0cf700 comp rx=0 tx=0).stop 2024-09-10T07:38:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.506+0000 7ff13b400640 1 -- 172.21.15.46:0/3587517865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1341056b0 msgr2=0x7ff1341ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.506+0000 7ff13b400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1341056b0 0x7ff1341ae4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009fd0 tx=0x7ff12c00c800 comp rx=0 tx=0).stop 2024-09-10T07:38:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.506+0000 7ff13b400640 1 -- 172.21.15.46:0/3587517865 shutdown_connections 2024-09-10T07:38:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.506+0000 7ff13b400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1341056b0 0x7ff1341ae4b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.506+0000 7ff13b400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff11c0a2d30 0x7ff11c0a51f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.506+0000 7ff13b400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff134102d00 0x7ff13410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.506+0000 7ff13b400640 1 --2- 172.21.15.46:0/3587517865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1341003b0 0x7ff13410b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.506+0000 7ff13b400640 1 -- 172.21.15.46:0/3587517865 >> 172.21.15.46:0/3587517865 conn(0x7ff1340fc1e0 msgr2=0x7ff134100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.507+0000 7ff13b400640 1 -- 172.21.15.46:0/3587517865 shutdown_connections 2024-09-10T07:38:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:40.507+0000 7ff13b400640 1 -- 172.21.15.46:0/3587517865 wait complete. 2024-09-10T07:38:40.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 80.0 s, 1921.2 tps, lat 52.362 ms stddev 59.327 2024-09-10T07:38:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:40 smithi181 ceph-mon[35397]: pgmap v460: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-09-10T07:38:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:40 smithi063 ceph-mon[35020]: pgmap v460: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-09-10T07:38:40.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:40 smithi046 ceph-mon[30705]: pgmap v460: 129 pgs: 129 active+clean; 940 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-09-10T07:38:42.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:42 smithi181 ceph-mon[35397]: pgmap v461: 129 pgs: 129 active+clean; 941 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-09-10T07:38:42.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:42 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:38:42.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:42 smithi063 ceph-mon[35020]: pgmap v461: 129 pgs: 129 active+clean; 941 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-09-10T07:38:42.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:42 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:38:42.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:42 smithi046 ceph-mon[30705]: pgmap v461: 129 pgs: 129 active+clean; 941 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-09-10T07:38:42.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:42 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:38:44.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:44 smithi046 ceph-mon[30705]: pgmap v462: 129 pgs: 129 active+clean; 942 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.9 MiB/s wr, 352 op/s 2024-09-10T07:38:44.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:44 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:38:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:44 smithi181 ceph-mon[35397]: pgmap v462: 129 pgs: 129 active+clean; 942 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.9 MiB/s wr, 352 op/s 2024-09-10T07:38:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:44 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:38:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:38:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:38:44.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:38:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:44 smithi063 ceph-mon[35020]: pgmap v462: 129 pgs: 129 active+clean; 942 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.9 MiB/s wr, 352 op/s 2024-09-10T07:38:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:44 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:38:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:38:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:38:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:38:44.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:38:44.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:38:44.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:38:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 85.0 s, 2071.2 tps, lat 48.026 ms stddev 54.967 2024-09-10T07:38:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:46 smithi181 ceph-mon[35397]: pgmap v463: 129 pgs: 129 active+clean; 943 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-10T07:38:46.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:46 smithi063 ceph-mon[35020]: pgmap v463: 129 pgs: 129 active+clean; 943 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-10T07:38:46.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:46 smithi046 ceph-mon[30705]: pgmap v463: 129 pgs: 129 active+clean; 943 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-10T07:38:48.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:48 smithi181 ceph-mon[35397]: pgmap v464: 129 pgs: 129 active+clean; 944 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-09-10T07:38:48.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:48 smithi063 ceph-mon[35020]: pgmap v464: 129 pgs: 129 active+clean; 944 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-09-10T07:38:48.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:48 smithi046 ceph-mon[30705]: pgmap v464: 129 pgs: 129 active+clean; 944 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-09-10T07:38:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 90.0 s, 2012.8 tps, lat 49.922 ms stddev 59.951 2024-09-10T07:38:50.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:50 smithi181 ceph-mon[35397]: pgmap v465: 129 pgs: 129 active+clean; 948 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 620 op/s 2024-09-10T07:38:50.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:50 smithi063 ceph-mon[35020]: pgmap v465: 129 pgs: 129 active+clean; 948 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 620 op/s 2024-09-10T07:38:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:50 smithi046 ceph-mon[30705]: pgmap v465: 129 pgs: 129 active+clean; 948 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 620 op/s 2024-09-10T07:38:52.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:52 smithi181 ceph-mon[35397]: pgmap v466: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 627 op/s 2024-09-10T07:38:52.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:52 smithi063 ceph-mon[35020]: pgmap v466: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 627 op/s 2024-09-10T07:38:52.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:52 smithi046 ceph-mon[30705]: pgmap v466: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 627 op/s 2024-09-10T07:38:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.723+0000 7fd8e9600640 1 -- 172.21.15.46:0/1124569490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4101580 msgr2=0x7fd8e4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.723+0000 7fd8e9600640 1 --2- 172.21.15.46:0/1124569490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4101580 0x7fd8e4101a00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009900 tx=0x7fd8cc02a410 comp rx=0 tx=0).stop 2024-09-10T07:38:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.724+0000 7fd8e9600640 1 -- 172.21.15.46:0/1124569490 shutdown_connections 2024-09-10T07:38:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.724+0000 7fd8e9600640 1 --2- 172.21.15.46:0/1124569490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4101f40 0x7fd8e4109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.724+0000 7fd8e9600640 1 --2- 172.21.15.46:0/1124569490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4101580 0x7fd8e4101a00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.724+0000 7fd8e9600640 1 --2- 172.21.15.46:0/1124569490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4103050 0x7fd8e4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.724+0000 7fd8e9600640 1 -- 172.21.15.46:0/1124569490 >> 172.21.15.46:0/1124569490 conn(0x7fd8e40fd0d0 msgr2=0x7fd8e40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.725+0000 7fd8e9600640 1 -- 172.21.15.46:0/1124569490 shutdown_connections 2024-09-10T07:38:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.725+0000 7fd8e9600640 1 -- 172.21.15.46:0/1124569490 wait complete. 2024-09-10T07:38:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.726+0000 7fd8e9600640 1 Processor -- start 2024-09-10T07:38:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.727+0000 7fd8e9600640 1 -- start start 2024-09-10T07:38:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.728+0000 7fd8e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4101580 0x7fd8e4195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4101f40 0x7fd8e4196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e9600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4103050 0x7fd8e41ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e410d7c0 con 0x7fd8e4101580 2024-09-10T07:38:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e9600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e410d640 con 0x7fd8e4103050 2024-09-10T07:38:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e9600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e410d940 con 0x7fd8e4101f40 2024-09-10T07:38:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4101f40 0x7fd8e4196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4101f40 0x7fd8e4196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37498/0 (socket says 172.21.15.46:37498) 2024-09-10T07:38:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e3400640 1 -- 172.21.15.46:0/619596845 learned_addr learned my addr 172.21.15.46:0/619596845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.729+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4101580 0x7fd8e4195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.730+0000 7fd8e3400640 1 -- 172.21.15.46:0/619596845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4103050 msgr2=0x7fd8e41ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.730+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4103050 0x7fd8e41ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.730+0000 7fd8e3400640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4103050 0x7fd8e41ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.730+0000 7fd8e3400640 1 -- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4101580 msgr2=0x7fd8e4195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.730+0000 7fd8e3400640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4101580 0x7fd8e4195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.730+0000 7fd8e3400640 1 -- 172.21.15.46:0/619596845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e419b3c0 con 0x7fd8e4101f40 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.730+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4103050 0x7fd8e41ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.731+0000 7fd8e3400640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4101f40 0x7fd8e4196230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc031610 tx=0x7fd8cc032c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.731+0000 7fd8e0c00640 1 -- 172.21.15.46:0/619596845 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc03f030 con 0x7fd8e4101f40 2024-09-10T07:38:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.732+0000 7fd8e0c00640 1 -- 172.21.15.46:0/619596845 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8cc003ad0 con 0x7fd8e4101f40 2024-09-10T07:38:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.732+0000 7fd8e0c00640 1 -- 172.21.15.46:0/619596845 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc035070 con 0x7fd8e4101f40 2024-09-10T07:38:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.732+0000 7fd8e9600640 1 -- 172.21.15.46:0/619596845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e41aec10 con 0x7fd8e4101f40 2024-09-10T07:38:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.732+0000 7fd8e9600640 1 -- 172.21.15.46:0/619596845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e41af080 con 0x7fd8e4101f40 2024-09-10T07:38:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.737+0000 7fd8eda22740 1 Processor -- start 2024-09-10T07:38:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.737+0000 7fd8eda22740 1 -- start start 2024-09-10T07:38:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.738+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d91f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.738+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d91fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.738+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91fcc30 0x55e0d91ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.738+0000 7fd8eda22740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e0d915c100 con 0x55e0d91f7ba0 2024-09-10T07:38:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.738+0000 7fd8eda22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e0d91041b0 con 0x55e0d91f7260 2024-09-10T07:38:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.738+0000 7fd8eda22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e0d91580c0 con 0x55e0d91fcc30 2024-09-10T07:38:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d91fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d91f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d91f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55718/0 (socket says 172.21.15.46:55718) 2024-09-10T07:38:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e8c00640 1 -- 172.21.15.46:0/979520143 learned_addr learned my addr 172.21.15.46:0/979520143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e8c00640 1 -- 172.21.15.46:0/979520143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91fcc30 msgr2=0x55e0d91ff0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:38:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/979520143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91fcc30 0x55e0d91ff0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e8c00640 1 -- 172.21.15.46:0/979520143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 msgr2=0x55e0d91fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/979520143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d91fc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e8c00640 1 -- 172.21.15.46:0/979520143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0d8e6f210 con 0x55e0d91f7260 2024-09-10T07:38:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.739+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/979520143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d91fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:38:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.740+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/979520143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d91f7660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009b80 tx=0x7fd8dc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f12cdef92d15a87f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.740+0000 7fd8d9200640 1 -- 172.21.15.46:0/979520143 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc005020 con 0x55e0d91f7260 2024-09-10T07:38:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.740+0000 7fd8d9200640 1 -- 172.21.15.46:0/979520143 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8dc0047a0 con 0x55e0d91f7260 2024-09-10T07:38:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.740+0000 7fd8d9200640 1 -- 172.21.15.46:0/979520143 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc012740 con 0x55e0d91f7260 2024-09-10T07:38:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.742+0000 7fd8eda22740 1 -- 172.21.15.46:0/979520143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 msgr2=0x55e0d91f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.742+0000 7fd8eda22740 1 --2- 172.21.15.46:0/979520143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d91f7660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009b80 tx=0x7fd8dc010740 comp rx=0 tx=0).stop 2024-09-10T07:38:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.742+0000 7fd8eda22740 1 -- 172.21.15.46:0/979520143 shutdown_connections 2024-09-10T07:38:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.742+0000 7fd8eda22740 1 --2- 172.21.15.46:0/979520143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91fcc30 0x55e0d91ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.742+0000 7fd8eda22740 1 --2- 172.21.15.46:0/979520143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d91fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.742+0000 7fd8eda22740 1 --2- 172.21.15.46:0/979520143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d91f7660 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.742+0000 7fd8eda22740 1 -- 172.21.15.46:0/979520143 >> 172.21.15.46:0/979520143 conn(0x55e0d91f28d0 msgr2=0x55e0d91f2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.745+0000 7fd8e0c00640 1 -- 172.21.15.46:0/619596845 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd8cc042030 con 0x7fd8e4101f40 2024-09-10T07:38:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.747+0000 7fd8eda22740 1 -- 172.21.15.46:0/979520143 shutdown_connections 2024-09-10T07:38:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.747+0000 7fd8eda22740 1 -- 172.21.15.46:0/979520143 wait complete. 2024-09-10T07:38:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.748+0000 7fd8eda22740 1 -- start start 2024-09-10T07:38:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.749+0000 7fd8eda22740 1 Processor -- start 2024-09-10T07:38:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.751+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7260 0x55e0d92015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.751+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d9201b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.751+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d9208f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.751+0000 7fd8eda22740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e0d91ffc00 con 0x55e0d91f7ba0 2024-09-10T07:38:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.751+0000 7fd8eda22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e0d90e5090 con 0x55e0d9206b40 2024-09-10T07:38:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.751+0000 7fd8eda22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e0d91ffd80 con 0x55e0d91f7260 2024-09-10T07:38:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.751+0000 7fd8eda22740 10 client.0 _ll_register_callbacks cb 0x55e0d9139f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.752+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d9208f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.752+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d9208f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55724/0 (socket says 172.21.15.46:55724) 2024-09-10T07:38:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.752+0000 7fd8e3400640 1 -- 172.21.15.46:0/3060531059 learned_addr learned my addr 172.21.15.46:0/3060531059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.752+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7260 0x55e0d92015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.753+0000 7fd8e0c00640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8c80a2c20 0x7fd8c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.753+0000 7fd8e0c00640 1 -- 172.21.15.46:0/619596845 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd8cc0ef8d0 con 0x7fd8e4101f40 2024-09-10T07:38:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.754+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d9201b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.754+0000 7fd8e8c00640 1 -- 172.21.15.46:0/3060531059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 msgr2=0x55e0d9208f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.754+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d9208f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.754+0000 7fd8e8c00640 1 -- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 msgr2=0x55e0d9201b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.754+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d9201b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.754+0000 7fd8e8c00640 1 -- 172.21.15.46:0/3060531059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0d8f3b620 con 0x55e0d91f7260 2024-09-10T07:38:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.754+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d9201b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:38:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.755+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8c80a2c20 0x7fd8c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.756+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7260 0x55e0d92015d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc010c40 tx=0x7fd8dc005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b8aaece6da1e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.756+0000 7fd8d1600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:38:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.756+0000 7fd8d1600640 1 -- 172.21.15.46:0/3060531059 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc005550 con 0x55e0d91f7260 2024-09-10T07:38:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.757+0000 7fd8d1600640 1 -- 172.21.15.46:0/3060531059 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd8dc0124c0 con 0x55e0d91f7260 2024-09-10T07:38:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.758+0000 7fd8eda22740 1 -- 172.21.15.46:0/3060531059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8dc0047a0 con 0x55e0d91f7260 2024-09-10T07:38:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.758+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8c80a2c20 0x7fd8c80a50e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc004770 tx=0x7fd8dc02a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.758+0000 7fd8d1600640 1 -- 172.21.15.46:0/3060531059 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd8dc032410 con 0x55e0d91f7260 2024-09-10T07:38:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.758+0000 7fd8eda22740 10 client.25972 fetch_fsmap learned FSMap version 36 2024-09-10T07:38:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.758+0000 7fd8eda22740 1 -- 172.21.15.46:0/3060531059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e0d920bd80 con 0x55e0d91f7260 2024-09-10T07:38:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.758+0000 7fd8d1600640 1 -- 172.21.15.46:0/3060531059 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8dc0047a0 con 0x55e0d91f7260 2024-09-10T07:38:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.759+0000 7fd8d1600640 1 -- 172.21.15.46:0/3060531059 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fd8dc00f040 con 0x55e0d91f7260 2024-09-10T07:38:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.760+0000 7fd8eda22740 10 client.25972 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:38:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.760+0000 7fd8eda22740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.760+0000 7fd8eda22740 20 client.25972 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.760+0000 7fd8eda22740 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0d920d410 0x55e0d920f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.760+0000 7fd8eda22740 4 client.25972 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:38:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.761+0000 7fd8eda22740 1 -- 172.21.15.46:0/3060531059 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0d9169830 con 0x55e0d920d410 2024-09-10T07:38:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.762+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0d920d410 0x55e0d920f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.764+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0d920d410 0x55e0d920f830 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ec2184a96b3843 server_cookie=3a20d95b1cb965e0 in_seq=0 out_seq=0 2024-09-10T07:38:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.764+0000 7fd8d1600640 10 client.25972 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.767+0000 7fd8d1600640 1 -- 172.21.15.46:0/3060531059 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0d9169830 con 0x55e0d920d410 2024-09-10T07:38:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.768+0000 7fd8d1600640 10 client.25972 handle_command_reply: tid=0 2024-09-10T07:38:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.768+0000 7fd8eda22740 1 client.25972 shutdown 2024-09-10T07:38:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.769+0000 7fd8eda22740 1 -- 172.21.15.46:0/3060531059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7260 msgr2=0x55e0d92015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.769+0000 7fd8eda22740 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7260 0x55e0d92015d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc010c40 tx=0x7fd8dc005da0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.769+0000 7fd8eda22740 1 -- 172.21.15.46:0/3060531059 shutdown_connections 2024-09-10T07:38:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.770+0000 7fd8eda22740 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0d920d410 0x55e0d920f830 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.770+0000 7fd8eda22740 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d9208f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.770+0000 7fd8d1600640 0 client.25972 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.770+0000 7fd8eda22740 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d9201b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.770+0000 7fd8eda22740 1 --2- 172.21.15.46:0/3060531059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7260 0x55e0d92015d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.770+0000 7fd8eda22740 1 -- 172.21.15.46:0/3060531059 >> 172.21.15.46:0/3060531059 conn(0x55e0d91f28d0 msgr2=0x55e0d91f40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.770+0000 7fd8eda22740 1 -- 172.21.15.46:0/3060531059 shutdown_connections 2024-09-10T07:38:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.771+0000 7fd8eda22740 1 -- 172.21.15.46:0/3060531059 wait complete. 2024-09-10T07:38:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.771+0000 7fd8eda22740 20 client.25972 trim_cache size 0 max 16384 2024-09-10T07:38:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.820+0000 7fd8eda22740 1 Processor -- start 2024-09-10T07:38:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.820+0000 7fd8eda22740 1 -- start start 2024-09-10T07:38:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.820+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7260 0x55e0d9201a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.820+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7ba0 0x55e0d91f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.820+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d91f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.820+0000 7fd8eda22740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41b94a0 con 0x55e0d91f7260 2024-09-10T07:38:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.820+0000 7fd8eda22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41bb100 con 0x55e0d9206b40 2024-09-10T07:38:53.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.820+0000 7fd8eda22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41b9b70 con 0x55e0d91f7ba0 2024-09-10T07:38:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d91f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7260 0x55e0d9201a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7ba0 0x55e0d91f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d91f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55728/0 (socket says 172.21.15.46:55728) 2024-09-10T07:38:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e8c00640 1 -- 172.21.15.46:0/1410435656 learned_addr learned my addr 172.21.15.46:0/1410435656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e3400640 1 -- 172.21.15.46:0/1410435656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 msgr2=0x55e0d91f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e3400640 1 --2- 172.21.15.46:0/1410435656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d91f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e3400640 1 -- 172.21.15.46:0/1410435656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7260 msgr2=0x55e0d9201a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e3400640 1 --2- 172.21.15.46:0/1410435656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7260 0x55e0d9201a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.821+0000 7fd8e3400640 1 -- 172.21.15.46:0/1410435656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0d91f8970 con 0x55e0d91f7ba0 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.822+0000 7fd8e3400640 1 --2- 172.21.15.46:0/1410435656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7ba0 0x55e0d91f11f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc006010 tx=0x7fd8cc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7db142020a3f136 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.822+0000 7fd8d3e00640 1 -- 172.21.15.46:0/1410435656 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc049040 con 0x55e0d91f7ba0 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.822+0000 7fd8d3e00640 1 -- 172.21.15.46:0/1410435656 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8cc0bc4b0 con 0x55e0d91f7ba0 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.822+0000 7fd8d3e00640 1 -- 172.21.15.46:0/1410435656 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0ba0b0 con 0x55e0d91f7ba0 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 -- 172.21.15.46:0/1410435656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7ba0 msgr2=0x55e0d91f11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1410435656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7ba0 0x55e0d91f11f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc006010 tx=0x7fd8cc02a410 comp rx=0 tx=0).stop 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 -- 172.21.15.46:0/1410435656 shutdown_connections 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1410435656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d9206b40 0x55e0d91f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1410435656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d91f7ba0 0x55e0d91f11f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1410435656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7260 0x55e0d9201a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 -- 172.21.15.46:0/1410435656 >> 172.21.15.46:0/1410435656 conn(0x55e0d91f28d0 msgr2=0x55e0d9201380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 -- 172.21.15.46:0/1410435656 shutdown_connections 2024-09-10T07:38:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 -- 172.21.15.46:0/1410435656 wait complete. 2024-09-10T07:38:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 -- start start 2024-09-10T07:38:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.823+0000 7fd8eda22740 1 Processor -- start 2024-09-10T07:38:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d921ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d921ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8eda22740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d9206b40 0x55e0d921ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8eda22740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41ba970 con 0x55e0d91f7ba0 2024-09-10T07:38:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8eda22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41ba2a0 con 0x55e0d91f7260 2024-09-10T07:38:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8eda22740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41b7480 con 0x55e0d9206b40 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8eda22740 10 client.0 _ll_register_callbacks cb 0x7fd8dc009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d921ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d921ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55742/0 (socket says 172.21.15.46:55742) 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8e3e00640 1 -- 172.21.15.46:0/1618367552 learned_addr learned my addr 172.21.15.46:0/1618367552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.824+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d9206b40 0x55e0d921ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.825+0000 7fd8e3400640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d921ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.825+0000 7fd8e3e00640 1 -- 172.21.15.46:0/1618367552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d9206b40 msgr2=0x55e0d921ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.826+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d9206b40 0x55e0d921ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.826+0000 7fd8e3e00640 1 -- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 msgr2=0x55e0d921ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.826+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d921ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.826+0000 7fd8e3e00640 1 -- 172.21.15.46:0/1618367552 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0d9220520 con 0x55e0d91f7260 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.826+0000 7fd8e8c00640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d9206b40 0x55e0d921ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:38:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.826+0000 7fd8e3400640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d921ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:38:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.827+0000 7fd8e3e00640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d921ea10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55e0d91f3750 tx=0x7fd8d400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3202bd95dbcfc95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:38:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.827+0000 7fd8d1600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:38:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.827+0000 7fd8d1600640 1 -- 172.21.15.46:0/1618367552 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4021440 con 0x55e0d91f7260 2024-09-10T07:38:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.827+0000 7fd8eda22740 1 -- 172.21.15.46:0/1618367552 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8cc0bc4b0 con 0x55e0d91f7260 2024-09-10T07:38:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.828+0000 7fd8d1600640 1 -- 172.21.15.46:0/1618367552 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd8d40215e0 con 0x55e0d91f7260 2024-09-10T07:38:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.828+0000 7fd8d1600640 1 -- 172.21.15.46:0/1618367552 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd8d40217e0 con 0x55e0d91f7260 2024-09-10T07:38:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.829+0000 7fd8eda22740 10 client.25982 fetch_fsmap learned FSMap version 36 2024-09-10T07:38:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.829+0000 7fd8eda22740 1 -- 172.21.15.46:0/1618367552 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e0d9221900 con 0x55e0d91f7260 2024-09-10T07:38:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.829+0000 7fd8d1600640 1 -- 172.21.15.46:0/1618367552 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8cc0bc4b0 con 0x55e0d91f7260 2024-09-10T07:38:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.829+0000 7fd8d1600640 1 -- 172.21.15.46:0/1618367552 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fd8d40179b0 con 0x55e0d91f7260 2024-09-10T07:38:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.830+0000 7fd8eda22740 10 client.25982 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:38:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.830+0000 7fd8eda22740 10 client.25982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:38:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.830+0000 7fd8eda22740 20 client.25982 populate_metadata read hostname 'smithi046' 2024-09-10T07:38:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.830+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0d9278650 0x55e0d927aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:38:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.830+0000 7fd8eda22740 4 client.25982 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:38:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.830+0000 7fd8eda22740 1 -- 172.21.15.46:0/1618367552 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0d9163b80 con 0x55e0d9278650 2024-09-10T07:38:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.830+0000 7fd8e3400640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0d9278650 0x55e0d927aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:38:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.831+0000 7fd8e3400640 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0d9278650 0x55e0d927aa70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fae06067564dc97 server_cookie=d4edf1dccf8f6bc9 in_seq=0 out_seq=0 2024-09-10T07:38:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.832+0000 7fd8d1600640 10 client.25982 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.878+0000 7fd8d1600640 1 -- 172.21.15.46:0/1618367552 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0d9163b80 con 0x55e0d9278650 2024-09-10T07:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.878+0000 7fd8d1600640 10 client.25982 handle_command_reply: tid=0 2024-09-10T07:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.878+0000 7fd8eda22740 1 client.25982 shutdown 2024-09-10T07:38:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 -- 172.21.15.46:0/1618367552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 msgr2=0x55e0d921ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d921ea10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55e0d91f3750 tx=0x7fd8d400bf10 comp rx=0 tx=0).stop 2024-09-10T07:38:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 -- 172.21.15.46:0/1618367552 shutdown_connections 2024-09-10T07:38:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0d9278650 0x55e0d927aa70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e0d9206b40 0x55e0d921ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0d91f7ba0 0x55e0d921ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 --2- 172.21.15.46:0/1618367552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e0d91f7260 0x55e0d921ea10 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55e0d91f3750 tx=0x7fd8d400bf10 comp rx=0 tx=0).stop 2024-09-10T07:38:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 -- 172.21.15.46:0/1618367552 >> 172.21.15.46:0/1618367552 conn(0x55e0d91f28d0 msgr2=0x55e0d91f3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8d1600640 0 client.25982 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:38:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.879+0000 7fd8eda22740 1 -- 172.21.15.46:0/1618367552 shutdown_connections 2024-09-10T07:38:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.880+0000 7fd8eda22740 1 -- 172.21.15.46:0/1618367552 wait complete. 2024-09-10T07:38:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.880+0000 7fd8eda22740 20 client.25982 trim_cache size 0 max 16384 2024-09-10T07:38:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:38:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 -- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8c80a2c20 msgr2=0x7fd8c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8c80a2c20 0x7fd8c80a50e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc004770 tx=0x7fd8dc02a000 comp rx=0 tx=0).stop 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 -- 172.21.15.46:0/619596845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4101f40 msgr2=0x7fd8e4196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4101f40 0x7fd8e4196230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc031610 tx=0x7fd8cc032c80 comp rx=0 tx=0).stop 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 -- 172.21.15.46:0/619596845 shutdown_connections 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8e4103050 0x7fd8e41ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4101f40 0x7fd8e4196230 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8c80a2c20 0x7fd8c80a50e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 --2- 172.21.15.46:0/619596845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e4101580 0x7fd8e4195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 -- 172.21.15.46:0/619596845 >> 172.21.15.46:0/619596845 conn(0x7fd8e40fd0d0 msgr2=0x7fd8e4103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.882+0000 7fd8e9600640 1 -- 172.21.15.46:0/619596845 shutdown_connections 2024-09-10T07:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:38:53.883+0000 7fd8e9600640 1 -- 172.21.15.46:0/619596845 wait complete. 2024-09-10T07:38:54.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:54 smithi181 ceph-mon[35397]: pgmap v467: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 599 op/s 2024-09-10T07:38:54.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:54 smithi063 ceph-mon[35020]: pgmap v467: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 599 op/s 2024-09-10T07:38:54.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:54 smithi046 ceph-mon[30705]: pgmap v467: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 599 op/s 2024-09-10T07:38:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 95.0 s, 3239.6 tps, lat 30.875 ms stddev 32.434 2024-09-10T07:38:56.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:56 smithi181 ceph-mon[35397]: pgmap v468: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 615 op/s 2024-09-10T07:38:56.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:56 smithi063 ceph-mon[35020]: pgmap v468: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 615 op/s 2024-09-10T07:38:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:56 smithi046 ceph-mon[30705]: pgmap v468: 129 pgs: 129 active+clean; 961 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 615 op/s 2024-09-10T07:38:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:38:58 smithi181 ceph-mon[35397]: pgmap v469: 129 pgs: 129 active+clean; 961 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 802 op/s 2024-09-10T07:38:58.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:38:58 smithi063 ceph-mon[35020]: pgmap v469: 129 pgs: 129 active+clean; 961 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 802 op/s 2024-09-10T07:38:58.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:38:58 smithi046 ceph-mon[30705]: pgmap v469: 129 pgs: 129 active+clean; 961 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 802 op/s 2024-09-10T07:39:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 100.0 s, 2886.6 tps, lat 34.633 ms stddev 39.251 2024-09-10T07:39:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:00 smithi181 ceph-mon[35397]: pgmap v470: 129 pgs: 129 active+clean; 969 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 618 op/s 2024-09-10T07:39:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:00 smithi063 ceph-mon[35020]: pgmap v470: 129 pgs: 129 active+clean; 969 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 618 op/s 2024-09-10T07:39:00.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:00 smithi046 ceph-mon[30705]: pgmap v470: 129 pgs: 129 active+clean; 969 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 618 op/s 2024-09-10T07:39:00.697 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:39:01.397 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 -- 172.21.15.46:0/1428852549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37781068d0 msgr2=0x7f3778106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 --2- 172.21.15.46:0/1428852549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37781068d0 0x7f3778106cb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3768009890 tx=0x7f376802a360 comp rx=0 tx=0).stop 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 -- 172.21.15.46:0/1428852549 shutdown_connections 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 --2- 172.21.15.46:0/1428852549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3778101270 0x7f377810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 --2- 172.21.15.46:0/1428852549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37781008d0 0x7f3778100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 --2- 172.21.15.46:0/1428852549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37781068d0 0x7f3778106cb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 -- 172.21.15.46:0/1428852549 >> 172.21.15.46:0/1428852549 conn(0x7f37780fc610 msgr2=0x7f37780fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 -- 172.21.15.46:0/1428852549 shutdown_connections 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.129+0000 7f3777200640 1 -- 172.21.15.46:0/1428852549 wait complete. 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.130+0000 7f3777200640 1 Processor -- start 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.130+0000 7f3777200640 1 -- start start 2024-09-10T07:39:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.131+0000 7f3777200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37781008d0 0x7f37781982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.131+0000 7f3777200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3778101270 0x7f3778198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.131+0000 7f3777200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37781068d0 0x7f377819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.131+0000 7f3777200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3778110c90 con 0x7f3778101270 2024-09-10T07:39:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.131+0000 7f3777200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3778110b10 con 0x7f37781008d0 2024-09-10T07:39:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.131+0000 7f3777200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3778110e10 con 0x7f37781068d0 2024-09-10T07:39:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.132+0000 7f3775e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37781008d0 0x7f37781982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.132+0000 7f3775e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37781008d0 0x7f37781982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48162/0 (socket says 172.21.15.46:48162) 2024-09-10T07:39:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.132+0000 7f3775e00640 1 -- 172.21.15.46:0/1934311927 learned_addr learned my addr 172.21.15.46:0/1934311927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.133+0000 7f3775e00640 1 -- 172.21.15.46:0/1934311927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37781068d0 msgr2=0x7f377819cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T07:39:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.133+0000 7f3775e00640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37781068d0 0x7f377819cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.133+0000 7f3775e00640 1 -- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3778101270 msgr2=0x7f3778198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.133+0000 7f3775400640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3778101270 0x7f3778198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.134+0000 7f3775e00640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3778101270 0x7f3778198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.134+0000 7f3775e00640 1 -- 172.21.15.46:0/1934311927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377819d2a0 con 0x7f37781008d0 2024-09-10T07:39:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.134+0000 7f3775400640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3778101270 0x7f3778198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.134+0000 7f3775e00640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37781008d0 0x7f37781982f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3768033dc0 tx=0x7f3768033df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:02.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.135+0000 7f376ea00640 1 -- 172.21.15.46:0/1934311927 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768037040 con 0x7f37781008d0 2024-09-10T07:39:02.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.135+0000 7f376ea00640 1 -- 172.21.15.46:0/1934311927 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3768040070 con 0x7f37781008d0 2024-09-10T07:39:02.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.135+0000 7f376ea00640 1 -- 172.21.15.46:0/1934311927 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680039e0 con 0x7f37781008d0 2024-09-10T07:39:02.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.135+0000 7f3777200640 1 -- 172.21.15.46:0/1934311927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377819d530 con 0x7f37781008d0 2024-09-10T07:39:02.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.135+0000 7f3777200640 1 -- 172.21.15.46:0/1934311927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377819d9c0 con 0x7f37781008d0 2024-09-10T07:39:02.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.142+0000 7f377fdc1740 1 Processor -- start 2024-09-10T07:39:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.142+0000 7f376ea00640 1 -- 172.21.15.46:0/1934311927 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3768003b80 con 0x7f37781008d0 2024-09-10T07:39:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.143+0000 7f377fdc1740 1 -- start start 2024-09-10T07:39:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.143+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95077a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.143+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95077a670 0x55b95078d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.143+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.143+0000 7f377fdc1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b9506e03e0 con 0x55b95077a670 2024-09-10T07:39:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.143+0000 7f377fdc1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b950687d80 con 0x55b95078d760 2024-09-10T07:39:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.143+0000 7f377fdc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b9506dc3a0 con 0x55b950779d50 2024-09-10T07:39:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.144+0000 7f376ea00640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f375c0a2bf0 0x7f375c0a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.145+0000 7f376ea00640 1 -- 172.21.15.46:0/1934311927 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f37680eef30 con 0x7f37781008d0 2024-09-10T07:39:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.145+0000 7f3776800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95077a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.145+0000 7f3776800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95077a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:47050/0 (socket says 172.21.15.46:47050) 2024-09-10T07:39:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.145+0000 7f3776800640 1 -- 172.21.15.46:0/3039316226 learned_addr learned my addr 172.21.15.46:0/3039316226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.146+0000 7f3776800640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f375c0a2bf0 0x7f375c0a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.146+0000 7f3775400640 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.146+0000 7f3776800640 1 -- 172.21.15.46:0/3039316226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 msgr2=0x55b95078fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.146+0000 7f3776800640 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.146+0000 7f3776800640 1 -- 172.21.15.46:0/3039316226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95077a670 msgr2=0x55b95078d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.146+0000 7f3776800640 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95077a670 0x55b95078d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.146+0000 7f3776800640 1 -- 172.21.15.46:0/3039316226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9506a6ae0 con 0x55b950779d50 2024-09-10T07:39:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.146+0000 7f3775400640 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:39:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.148+0000 7f3776800640 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95077a130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3770014840 tx=0x7f3770014d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9c9125ed8b36ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:02.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.149+0000 7f3763400640 1 -- 172.21.15.46:0/3039316226 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004d20 con 0x55b950779d50 2024-09-10T07:39:02.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.149+0000 7f3763400640 1 -- 172.21.15.46:0/3039316226 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377000dd40 con 0x55b950779d50 2024-09-10T07:39:02.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.149+0000 7f3763400640 1 -- 172.21.15.46:0/3039316226 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377001cc90 con 0x55b950779d50 2024-09-10T07:39:02.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.149+0000 7f3776800640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f375c0a2bf0 0x7f375c0a50b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3770018000 tx=0x7f377001a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:02.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.150+0000 7f377fdc1740 1 -- 172.21.15.46:0/3039316226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 msgr2=0x55b95077a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:02.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.150+0000 7f377fdc1740 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95077a130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3770014840 tx=0x7f3770014d00 comp rx=0 tx=0).stop 2024-09-10T07:39:02.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.151+0000 7f377fdc1740 1 -- 172.21.15.46:0/3039316226 shutdown_connections 2024-09-10T07:39:02.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.151+0000 7f377fdc1740 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.151+0000 7f377fdc1740 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95077a670 0x55b95078d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.151+0000 7f377fdc1740 1 --2- 172.21.15.46:0/3039316226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95077a130 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.151+0000 7f377fdc1740 1 -- 172.21.15.46:0/3039316226 >> 172.21.15.46:0/3039316226 conn(0x55b950776220 msgr2=0x55b950776600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.151+0000 7f377fdc1740 1 -- 172.21.15.46:0/3039316226 shutdown_connections 2024-09-10T07:39:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.151+0000 7f377fdc1740 1 -- 172.21.15.46:0/3039316226 wait complete. 2024-09-10T07:39:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.152+0000 7f377fdc1740 1 -- start start 2024-09-10T07:39:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.152+0000 7f377fdc1740 1 Processor -- start 2024-09-10T07:39:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.154+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95078a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.154+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95077a670 0x55b95078ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.154+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.154+0000 7f377fdc1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b9507a1320 con 0x55b95077a670 2024-09-10T07:39:02.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.155+0000 7f3776800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.155+0000 7f3776800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48174/0 (socket says 172.21.15.46:48174) 2024-09-10T07:39:02.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.155+0000 7f3776800640 1 -- 172.21.15.46:0/1906243669 learned_addr learned my addr 172.21.15.46:0/1906243669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.154+0000 7f377fdc1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b95069b0f0 con 0x55b95078d760 2024-09-10T07:39:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.155+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b9507a14a0 con 0x55b950779d50 2024-09-10T07:39:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.156+0000 7f3776800640 1 -- 172.21.15.46:0/1906243669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 msgr2=0x55b95078a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.156+0000 7f3775e00640 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95078a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.156+0000 7f3776800640 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95078a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.156+0000 7f3776800640 1 -- 172.21.15.46:0/1906243669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95077a670 msgr2=0x55b95078ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.156+0000 7f3776800640 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95077a670 0x55b95078ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.156+0000 7f3776800640 1 -- 172.21.15.46:0/1906243669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9504513c0 con 0x55b95078d760 2024-09-10T07:39:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.157+0000 7f377fdc1740 10 client.0 _ll_register_callbacks cb 0x55b95068c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.157+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377000dd40 con 0x55b95078d760 2024-09-10T07:39:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.158+0000 7f3776800640 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078b220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f37700092f0 tx=0x7f377000db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6f93fe64fc63f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.158+0000 7f3757e00640 10 client.25994 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.159+0000 7f3757e00640 1 -- 172.21.15.46:0/1906243669 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770034050 con 0x55b95078d760 2024-09-10T07:39:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.159+0000 7f3757e00640 1 -- 172.21.15.46:0/1906243669 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3770014040 con 0x55b95078d760 2024-09-10T07:39:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.159+0000 7f3757e00640 1 -- 172.21.15.46:0/1906243669 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f377002a070 con 0x55b95078d760 2024-09-10T07:39:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.160+0000 7f377fdc1740 10 client.25994 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.160+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55b9507ad110 con 0x55b95078d760 2024-09-10T07:39:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.161+0000 7f3757e00640 1 -- 172.21.15.46:0/1906243669 <== mon.2 v2:172.21.15.63:3300/0 4 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f377001b070 con 0x55b95078d760 2024-09-10T07:39:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.161+0000 7f377fdc1740 10 client.25994 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.161+0000 7f377fdc1740 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.161+0000 7f377fdc1740 20 client.25994 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.161+0000 7f377fdc1740 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b9507ae460 0x55b9507b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.161+0000 7f377fdc1740 4 client.25994 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.161+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9506ed950 con 0x55b9507ae460 2024-09-10T07:39:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.162+0000 7f3775e00640 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b9507ae460 0x55b9507b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.163+0000 7f3775e00640 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b9507ae460 0x55b9507b0850 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab612b6a435cd6c server_cookie=25a7e1b44b0703f4 in_seq=0 out_seq=0 2024-09-10T07:39:02.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.163+0000 7f3757e00640 10 client.25994 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:02.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.164+0000 7f3757e00640 1 -- 172.21.15.46:0/1906243669 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9506ed950 con 0x55b9507ae460 2024-09-10T07:39:02.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.164+0000 7f3757e00640 10 client.25994 handle_command_reply: tid=0 2024-09-10T07:39:02.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.164+0000 7f377fdc1740 1 client.25994 shutdown 2024-09-10T07:39:02.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.164+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 msgr2=0x55b95078b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:02.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.164+0000 7f377fdc1740 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078b220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f37700092f0 tx=0x7f377000db10 comp rx=0 tx=0).stop 2024-09-10T07:39:02.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 shutdown_connections 2024-09-10T07:39:02.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f377fdc1740 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b9507ae460 0x55b9507b0850 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f377fdc1740 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95078d760 0x55b95078b220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f377fdc1740 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95077a670 0x55b95078ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f377fdc1740 1 --2- 172.21.15.46:0/1906243669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b950779d50 0x55b95078a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 >> 172.21.15.46:0/1906243669 conn(0x55b950776220 msgr2=0x55b95077c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:02.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f3757e00640 0 client.25994 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:02.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 shutdown_connections 2024-09-10T07:39:02.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.165+0000 7f377fdc1740 1 -- 172.21.15.46:0/1906243669 wait complete. 2024-09-10T07:39:02.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.166+0000 7f377fdc1740 20 client.25994 trim_cache size 0 max 16384 2024-09-10T07:39:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.202+0000 7f377fdc1740 1 Processor -- start 2024-09-10T07:39:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.202+0000 7f377fdc1740 1 -- start start 2024-09-10T07:39:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b950789070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b95078b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b950775270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f377fdc1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3778073780 con 0x55b950779d50 2024-09-10T07:39:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f377fdc1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37781a5e40 con 0x55b95077a670 2024-09-10T07:39:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f377fdc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3778073e50 con 0x55b95078d760 2024-09-10T07:39:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f3776800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b950775270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f3775e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b950789070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f3775e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b950789070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33768/0 (socket says 172.21.15.46:33768) 2024-09-10T07:39:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f3775e00640 1 -- 172.21.15.46:0/3071269495 learned_addr learned my addr 172.21.15.46:0/3071269495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.203+0000 7f3775400640 1 --2- 172.21.15.46:0/3071269495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b95078b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.204+0000 7f3776800640 1 -- 172.21.15.46:0/3071269495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 msgr2=0x55b95078b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.204+0000 7f3776800640 1 --2- 172.21.15.46:0/3071269495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b95078b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.204+0000 7f3776800640 1 -- 172.21.15.46:0/3071269495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 msgr2=0x55b950789070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.204+0000 7f3776800640 1 --2- 172.21.15.46:0/3071269495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b950789070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.204+0000 7f3776800640 1 -- 172.21.15.46:0/3071269495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b950789750 con 0x55b95078d760 2024-09-10T07:39:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.205+0000 7f3776800640 1 --2- 172.21.15.46:0/3071269495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b950775270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3770009260 tx=0x7f377001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd6ced52bf0817b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.205+0000 7f3762a00640 1 -- 172.21.15.46:0/3071269495 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770034040 con 0x55b95078d760 2024-09-10T07:39:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.206+0000 7f3762a00640 1 -- 172.21.15.46:0/3071269495 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377000dd40 con 0x55b95078d760 2024-09-10T07:39:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.206+0000 7f3762a00640 1 -- 172.21.15.46:0/3071269495 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004e40 con 0x55b95078d760 2024-09-10T07:39:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.207+0000 7f377fdc1740 1 -- 172.21.15.46:0/3071269495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 msgr2=0x55b950775270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.207+0000 7f377fdc1740 1 --2- 172.21.15.46:0/3071269495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b950775270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3770009260 tx=0x7f377001f000 comp rx=0 tx=0).stop 2024-09-10T07:39:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.207+0000 7f377fdc1740 1 -- 172.21.15.46:0/3071269495 shutdown_connections 2024-09-10T07:39:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.207+0000 7f377fdc1740 1 --2- 172.21.15.46:0/3071269495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b950775270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.207+0000 7f377fdc1740 1 --2- 172.21.15.46:0/3071269495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b95078b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.207+0000 7f377fdc1740 1 --2- 172.21.15.46:0/3071269495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b950789070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.207+0000 7f377fdc1740 1 -- 172.21.15.46:0/3071269495 >> 172.21.15.46:0/3071269495 conn(0x55b950776220 msgr2=0x55b950774b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.209+0000 7f377fdc1740 1 -- 172.21.15.46:0/3071269495 shutdown_connections 2024-09-10T07:39:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.209+0000 7f377fdc1740 1 -- 172.21.15.46:0/3071269495 wait complete. 2024-09-10T07:39:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.209+0000 7f377fdc1740 1 -- start start 2024-09-10T07:39:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.210+0000 7f377fdc1740 1 Processor -- start 2024-09-10T07:39:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.211+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b9507a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.212+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b9507a2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.212+0000 7f377fdc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b9507a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.212+0000 7f3776800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b9507a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.212+0000 7f3775400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b9507a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.212+0000 7f3776800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b9507a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:47078/0 (socket says 172.21.15.46:47078) 2024-09-10T07:39:02.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.212+0000 7f3775400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b9507a2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48200/0 (socket says 172.21.15.46:48200) 2024-09-10T07:39:02.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.212+0000 7f3775400640 1 -- 172.21.15.46:0/108310544 learned_addr learned my addr 172.21.15.46:0/108310544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:02.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.213+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37781a56b0 con 0x55b950779d50 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.213+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37781a4fe0 con 0x55b95077a670 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.213+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3778071760 con 0x55b95078d760 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.213+0000 7f3775e00640 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b9507a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.213+0000 7f377fdc1740 10 client.0 _ll_register_callbacks cb 0x7f377001c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.214+0000 7f3776800640 1 -- 172.21.15.46:0/108310544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 msgr2=0x55b9507a2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.214+0000 7f3776800640 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b9507a2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.214+0000 7f3776800640 1 -- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 msgr2=0x55b9507a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.214+0000 7f3776800640 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b9507a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.214+0000 7f3776800640 1 -- 172.21.15.46:0/108310544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b950818080 con 0x55b95078d760 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.214+0000 7f3775e00640 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b9507a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.214+0000 7f3776800640 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b9507a2ed0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3770011b20 tx=0x7f3770009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0efa64e478e15d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.214+0000 7f3757e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.215+0000 7f3757e00640 1 -- 172.21.15.46:0/108310544 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377003e070 con 0x55b95078d760 2024-09-10T07:39:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.215+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377000dd40 con 0x55b95078d760 2024-09-10T07:39:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.215+0000 7f3757e00640 1 -- 172.21.15.46:0/108310544 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3770014020 con 0x55b95078d760 2024-09-10T07:39:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.217+0000 7f3757e00640 1 -- 172.21.15.46:0/108310544 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f377004a050 con 0x55b95078d760 2024-09-10T07:39:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.217+0000 7f377fdc1740 10 client.25990 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.217+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b9508192b0 con 0x55b95078d760 2024-09-10T07:39:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.217+0000 7f3757e00640 1 -- 172.21.15.46:0/108310544 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377000dd40 con 0x55b95078d760 2024-09-10T07:39:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.218+0000 7f3757e00640 1 -- 172.21.15.46:0/108310544 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f377001b070 con 0x55b95078d760 2024-09-10T07:39:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.218+0000 7f377fdc1740 10 client.25990 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.218+0000 7f377fdc1740 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.218+0000 7f377fdc1740 20 client.25990 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.218+0000 7f377fdc1740 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b950819a60 0x55b95081be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.218+0000 7f377fdc1740 4 client.25990 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:39:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.218+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9506e7e60 con 0x55b950819a60 2024-09-10T07:39:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.219+0000 7f3775e00640 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b950819a60 0x55b95081be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.222+0000 7f3775e00640 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b950819a60 0x55b95081be80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a48bad350be9a server_cookie=53c3a39fd34988c5 in_seq=0 out_seq=0 2024-09-10T07:39:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.222+0000 7f3757e00640 10 client.25990 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.223+0000 7f3757e00640 1 -- 172.21.15.46:0/108310544 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b9506e7e60 con 0x55b950819a60 2024-09-10T07:39:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.223+0000 7f3757e00640 10 client.25990 handle_command_reply: tid=0 2024-09-10T07:39:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.223+0000 7f377fdc1740 1 client.25990 shutdown 2024-09-10T07:39:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.223+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 msgr2=0x55b9507a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.223+0000 7f377fdc1740 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b9507a2ed0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3770011b20 tx=0x7f3770009610 comp rx=0 tx=0).stop 2024-09-10T07:39:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.224+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 shutdown_connections 2024-09-10T07:39:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.224+0000 7f377fdc1740 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b950819a60 0x55b95081be80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.224+0000 7f377fdc1740 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95078d760 0x55b9507a2ed0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.224+0000 7f377fdc1740 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95077a670 0x55b9507a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.224+0000 7f377fdc1740 1 --2- 172.21.15.46:0/108310544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b950779d50 0x55b9507a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.224+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 >> 172.21.15.46:0/108310544 conn(0x55b950776220 msgr2=0x55b950774b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.224+0000 7f3757e00640 0 client.25990 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.225+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 shutdown_connections 2024-09-10T07:39:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.225+0000 7f377fdc1740 1 -- 172.21.15.46:0/108310544 wait complete. 2024-09-10T07:39:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.226+0000 7f377fdc1740 20 client.25990 trim_cache size 0 max 16384 2024-09-10T07:39:02.225 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:39:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:39:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:39:02.226 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:39:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.226+0000 7f3777200640 1 -- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f375c0a2bf0 msgr2=0x7f375c0a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.226+0000 7f3777200640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f375c0a2bf0 0x7f375c0a50b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3770018000 tx=0x7f377001a000 comp rx=0 tx=0).stop 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.226+0000 7f3777200640 1 -- 172.21.15.46:0/1934311927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37781008d0 msgr2=0x7f37781982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.226+0000 7f3777200640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37781008d0 0x7f37781982f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3768033dc0 tx=0x7f3768033df0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.227+0000 7f3777200640 1 -- 172.21.15.46:0/1934311927 shutdown_connections 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.227+0000 7f3777200640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37781068d0 0x7f377819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.227+0000 7f3777200640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f375c0a2bf0 0x7f375c0a50b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.227+0000 7f3777200640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3778101270 0x7f3778198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.227+0000 7f3777200640 1 --2- 172.21.15.46:0/1934311927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37781008d0 0x7f37781982f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.227+0000 7f3777200640 1 -- 172.21.15.46:0/1934311927 >> 172.21.15.46:0/1934311927 conn(0x7f37780fc610 msgr2=0x7f377806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.227+0000 7f3777200640 1 -- 172.21.15.46:0/1934311927 shutdown_connections 2024-09-10T07:39:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:02.227+0000 7f3777200640 1 -- 172.21.15.46:0/1934311927 wait complete. 2024-09-10T07:39:02.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:02 smithi046 ceph-mon[30705]: pgmap v471: 129 pgs: 129 active+clean; 974 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 741 op/s 2024-09-10T07:39:02.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:02 smithi181 ceph-mon[35397]: pgmap v471: 129 pgs: 129 active+clean; 974 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 741 op/s 2024-09-10T07:39:02.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:02 smithi063 ceph-mon[35020]: pgmap v471: 129 pgs: 129 active+clean; 974 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 741 op/s 2024-09-10T07:39:03.573 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:39:03.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:39:04.231 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:04.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:04 smithi046 ceph-mon[30705]: pgmap v472: 129 pgs: 129 active+clean; 978 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-09-10T07:39:04.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:04 smithi181 ceph-mon[35397]: pgmap v472: 129 pgs: 129 active+clean; 978 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-09-10T07:39:04.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:04 smithi063 ceph-mon[35020]: pgmap v472: 129 pgs: 129 active+clean; 978 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-09-10T07:39:04.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.979+0000 7f9c4e200640 1 -- 172.21.15.46:0/1219741156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c48101df0 msgr2=0x7f9c481021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.981+0000 7f9c4e200640 1 --2- 172.21.15.46:0/1219741156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c48101df0 0x7f9c481021d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009900 tx=0x7f9c3802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:39:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.981+0000 7f9c4e200640 1 -- 172.21.15.46:0/1219741156 shutdown_connections 2024-09-10T07:39:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.981+0000 7f9c4e200640 1 --2- 172.21.15.46:0/1219741156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48106f20 0x7f9c4810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.981+0000 7f9c4e200640 1 --2- 172.21.15.46:0/1219741156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c481027a0 0x7f9c481067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.981+0000 7f9c4e200640 1 --2- 172.21.15.46:0/1219741156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c48101df0 0x7f9c481021d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.981+0000 7f9c4e200640 1 -- 172.21.15.46:0/1219741156 >> 172.21.15.46:0/1219741156 conn(0x7f9c480fd8b0 msgr2=0x7f9c480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.982+0000 7f9c4e200640 1 -- 172.21.15.46:0/1219741156 shutdown_connections 2024-09-10T07:39:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.982+0000 7f9c4e200640 1 -- 172.21.15.46:0/1219741156 wait complete. 2024-09-10T07:39:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.985+0000 7f9c4e200640 1 Processor -- start 2024-09-10T07:39:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.986+0000 7f9c4e200640 1 -- start start 2024-09-10T07:39:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.987+0000 7f9c4e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c48101df0 0x7f9c4810c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.987+0000 7f9c4e200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c481027a0 0x7f9c4810caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.988+0000 7f9c4e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48106f20 0x7f9c481a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.988+0000 7f9c4e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48114160 con 0x7f9c48106f20 2024-09-10T07:39:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.988+0000 7f9c4e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48113fe0 con 0x7f9c48101df0 2024-09-10T07:39:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.988+0000 7f9c4e200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9c481142e0 con 0x7f9c481027a0 2024-09-10T07:39:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.988+0000 7f9c46a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c481027a0 0x7f9c4810caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.989+0000 7f9c47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c48101df0 0x7f9c4810c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.989+0000 7f9c47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c48101df0 0x7f9c4810c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48222/0 (socket says 172.21.15.46:48222) 2024-09-10T07:39:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.989+0000 7f9c47400640 1 -- 172.21.15.46:0/3426198015 learned_addr learned my addr 172.21.15.46:0/3426198015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.990+0000 7f9c47400640 1 -- 172.21.15.46:0/3426198015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c481027a0 msgr2=0x7f9c4810caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.990+0000 7f9c47400640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c481027a0 0x7f9c4810caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.990+0000 7f9c47400640 1 -- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48106f20 msgr2=0x7f9c481a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.990+0000 7f9c47400640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48106f20 0x7f9c481a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.990+0000 7f9c47400640 1 -- 172.21.15.46:0/3426198015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c481a13c0 con 0x7f9c48101df0 2024-09-10T07:39:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.990+0000 7f9c47400640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c48101df0 0x7f9c4810c5b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9c38006010 tx=0x7f9c38004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.991+0000 7f9c3fe00640 1 -- 172.21.15.46:0/3426198015 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3803f030 con 0x7f9c48101df0 2024-09-10T07:39:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.991+0000 7f9c3fe00640 1 -- 172.21.15.46:0/3426198015 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c38004440 con 0x7f9c48101df0 2024-09-10T07:39:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.991+0000 7f9c3fe00640 1 -- 172.21.15.46:0/3426198015 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38035070 con 0x7f9c48101df0 2024-09-10T07:39:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.991+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c481a1650 con 0x7f9c48101df0 2024-09-10T07:39:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.991+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c481122d0 con 0x7f9c48101df0 2024-09-10T07:39:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.997+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c481021d0 con 0x7f9c48101df0 2024-09-10T07:39:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:04.999+0000 7f9c3fe00640 1 -- 172.21.15.46:0/3426198015 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9c38042030 con 0x7f9c48101df0 2024-09-10T07:39:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.002+0000 7f9c3fe00640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c2c0a2d50 0x7f9c2c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.002+0000 7f9c3fe00640 1 -- 172.21.15.46:0/3426198015 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9c380ef760 con 0x7f9c48101df0 2024-09-10T07:39:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.003+0000 7f9c46a00640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c2c0a2d50 0x7f9c2c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:05.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.005+0000 7f9c3fe00640 1 -- 172.21.15.46:0/3426198015 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c38049030 con 0x7f9c48101df0 2024-09-10T07:39:05.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.007+0000 7f9c46a00640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c2c0a2d50 0x7f9c2c0a5210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9c48108600 tx=0x7f9c34005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.441+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c48112d80 con 0x7f9c48101df0 2024-09-10T07:39:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.441+0000 7f9c3fe00640 1 -- 172.21.15.46:0/3426198015 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11982 (secure 0 0 0) 0x7f9c380bc3a0 con 0x7f9c48101df0 2024-09-10T07:39:05.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:39:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"btime":"2024-09-10T07:38:43:331624+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:38:42.493709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:39:05.443 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-09-10T07:39:05.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c2c0a2d50 msgr2=0x7f9c2c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:05.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c2c0a2d50 0x7f9c2c0a5210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9c48108600 tx=0x7f9c34005f90 comp rx=0 tx=0).stop 2024-09-10T07:39:05.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c48101df0 msgr2=0x7f9c4810c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:05.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c48101df0 0x7f9c4810c5b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9c38006010 tx=0x7f9c38004040 comp rx=0 tx=0).stop 2024-09-10T07:39:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 shutdown_connections 2024-09-10T07:39:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c2c0a2d50 0x7f9c2c0a5210 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48106f20 0x7f9c481a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c481027a0 0x7f9c4810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.447+0000 7f9c4e200640 1 --2- 172.21.15.46:0/3426198015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c48101df0 0x7f9c4810c5b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.448+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 >> 172.21.15.46:0/3426198015 conn(0x7f9c480fd8b0 msgr2=0x7f9c480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.448+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 shutdown_connections 2024-09-10T07:39:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:05.448+0000 7f9c4e200640 1 -- 172.21.15.46:0/3426198015 wait complete. 2024-09-10T07:39:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 105.0 s, 1978.2 tps, lat 50.388 ms stddev 61.944 2024-09-10T07:39:06.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:39:06.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:06 smithi046 ceph-mon[30705]: pgmap v473: 129 pgs: 129 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 756 op/s 2024-09-10T07:39:06.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:06 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3426198015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:39:06.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:06 smithi181 ceph-mon[35397]: pgmap v473: 129 pgs: 129 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 756 op/s 2024-09-10T07:39:06.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:06 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3426198015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:39:06.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:06 smithi063 ceph-mon[35020]: pgmap v473: 129 pgs: 129 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 756 op/s 2024-09-10T07:39:06.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:06 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3426198015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:39:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.111+0000 7f5823800640 1 -- 172.21.15.46:0/494160375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103d60 msgr2=0x7f581c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.111+0000 7f5823800640 1 --2- 172.21.15.46:0/494160375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103d60 0x7f581c104160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5810009860 tx=0x7f581002a380 comp rx=0 tx=0).stop 2024-09-10T07:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.116+0000 7f5823800640 1 -- 172.21.15.46:0/494160375 shutdown_connections 2024-09-10T07:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.116+0000 7f5823800640 1 --2- 172.21.15.46:0/494160375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c105910 0x7f581c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.116+0000 7f5823800640 1 --2- 172.21.15.46:0/494160375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f581c104f50 0x7f581c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.116+0000 7f5823800640 1 --2- 172.21.15.46:0/494160375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103d60 0x7f581c104160 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.116+0000 7f5823800640 1 -- 172.21.15.46:0/494160375 >> 172.21.15.46:0/494160375 conn(0x7f581c0ff4e0 msgr2=0x7f581c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.116+0000 7f5823800640 1 -- 172.21.15.46:0/494160375 shutdown_connections 2024-09-10T07:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.116+0000 7f5823800640 1 -- 172.21.15.46:0/494160375 wait complete. 2024-09-10T07:39:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.117+0000 7f5823800640 1 Processor -- start 2024-09-10T07:39:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.118+0000 7f5823800640 1 -- start start 2024-09-10T07:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.118+0000 7f5823800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c103d60 0x7f581c19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.118+0000 7f5823800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c104f50 0x7f581c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.118+0000 7f5823800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f581c105910 0x7f581c1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.118+0000 7f5823800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f581c077590 con 0x7f581c103d60 2024-09-10T07:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.118+0000 7f5823800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f581c077410 con 0x7f581c105910 2024-09-10T07:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.118+0000 7f5823800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f581c077710 con 0x7f581c104f50 2024-09-10T07:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.119+0000 7f5821800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f581c105910 0x7f581c1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.119+0000 7f5821800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f581c105910 0x7f581c1a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43694/0 (socket says 172.21.15.46:43694) 2024-09-10T07:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.119+0000 7f5821800640 1 -- 172.21.15.46:0/3890897256 learned_addr learned my addr 172.21.15.46:0/3890897256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.119+0000 7f5821800640 1 -- 172.21.15.46:0/3890897256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c104f50 msgr2=0x7f581c19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.119+0000 7f5821800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c104f50 0x7f581c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.119+0000 7f5821800640 1 -- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c103d60 msgr2=0x7f581c19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.119+0000 7f5821800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c103d60 0x7f581c19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.119+0000 7f5821800640 1 -- 172.21.15.46:0/3890897256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c1a5fe0 con 0x7f581c105910 2024-09-10T07:39:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.120+0000 7f5821800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f581c105910 0x7f581c1a5970 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f581400eb10 tx=0x7f581400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.120+0000 7f5819600640 1 -- 172.21.15.46:0/3890897256 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814004b20 con 0x7f581c105910 2024-09-10T07:39:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.120+0000 7f5819600640 1 -- 172.21.15.46:0/3890897256 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5814004cc0 con 0x7f581c105910 2024-09-10T07:39:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.120+0000 7f5819600640 1 -- 172.21.15.46:0/3890897256 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814010790 con 0x7f581c105910 2024-09-10T07:39:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.120+0000 7f5823800640 1 -- 172.21.15.46:0/3890897256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581c1a62d0 con 0x7f581c105910 2024-09-10T07:39:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.120+0000 7f5823800640 1 -- 172.21.15.46:0/3890897256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581c1a67d0 con 0x7f581c105910 2024-09-10T07:39:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.131+0000 7f5819600640 1 -- 172.21.15.46:0/3890897256 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5814005020 con 0x7f581c105910 2024-09-10T07:39:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.135+0000 7f5819600640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f58040a2d30 0x7f58040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.136+0000 7f5819600640 1 -- 172.21.15.46:0/3890897256 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f58140cfb00 con 0x7f581c105910 2024-09-10T07:39:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.138+0000 7f5820e00640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f58040a2d30 0x7f58040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.140+0000 7f58265c9740 1 Processor -- start 2024-09-10T07:39:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.140+0000 7f58265c9740 1 -- start start 2024-09-10T07:39:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.141+0000 7f58265c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 0x5602c9dc7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.141+0000 7f58265c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9dcc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.141+0000 7f58265c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9dcef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.141+0000 7f58265c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5602c9d2c100 con 0x5602c9dc7090 2024-09-10T07:39:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.141+0000 7f58265c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5602c9cd41b0 con 0x5602c9dc79d0 2024-09-10T07:39:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.141+0000 7f58265c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5602c9d280c0 con 0x5602c9dcca60 2024-09-10T07:39:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f581be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9dcc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f581be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9dcc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43708/0 (socket says 172.21.15.46:43708) 2024-09-10T07:39:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f581be00640 1 -- 172.21.15.46:0/1842595571 learned_addr learned my addr 172.21.15.46:0/1842595571 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f5821800640 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9dcef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f581be00640 1 -- 172.21.15.46:0/1842595571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 msgr2=0x5602c9dcef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f581be00640 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9dcef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f581be00640 1 -- 172.21.15.46:0/1842595571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 msgr2=0x5602c9dc7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f581be00640 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 0x5602c9dc7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.143+0000 7f581be00640 1 -- 172.21.15.46:0/1842595571 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c9a3f210 con 0x5602c9dc79d0 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.144+0000 7f5821800640 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9dcef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.144+0000 7f581be00640 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9dcc490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5808004650 tx=0x7f580800e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28dfb9a14ca9b8c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.144+0000 7f580e000640 1 -- 172.21.15.46:0/1842595571 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580800ed80 con 0x5602c9dc79d0 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.145+0000 7f580e000640 1 -- 172.21.15.46:0/1842595571 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5808005500 con 0x5602c9dc79d0 2024-09-10T07:39:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.145+0000 7f580e000640 1 -- 172.21.15.46:0/1842595571 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808010690 con 0x5602c9dc79d0 2024-09-10T07:39:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f58265c9740 1 -- 172.21.15.46:0/1842595571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 msgr2=0x5602c9dcc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f5820e00640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f58040a2d30 0x7f58040a51f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58100057c0 tx=0x7f5810030cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f58265c9740 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9dcc490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5808004650 tx=0x7f580800e350 comp rx=0 tx=0).stop 2024-09-10T07:39:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f58265c9740 1 -- 172.21.15.46:0/1842595571 shutdown_connections 2024-09-10T07:39:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f58265c9740 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9dcef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f58265c9740 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9dcc490 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f58265c9740 1 --2- 172.21.15.46:0/1842595571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 0x5602c9dc7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f58265c9740 1 -- 172.21.15.46:0/1842595571 >> 172.21.15.46:0/1842595571 conn(0x5602c9dc2700 msgr2=0x5602c9dc2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.146+0000 7f58265c9740 1 -- 172.21.15.46:0/1842595571 shutdown_connections 2024-09-10T07:39:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.147+0000 7f58265c9740 1 -- 172.21.15.46:0/1842595571 wait complete. 2024-09-10T07:39:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.148+0000 7f58265c9740 1 -- start start 2024-09-10T07:39:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.148+0000 7f58265c9740 1 Processor -- start 2024-09-10T07:39:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f58265c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f58265c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f58265c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dcca60 0x5602c9de97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f58265c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5602c9de0a40 con 0x5602c9dcca60 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f58265c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5602c9cb5090 con 0x5602c9dc79d0 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f58265c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5602c9de0bc0 con 0x5602c9dc7090 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f58265c9740 10 client.0 _ll_register_callbacks cb 0x5602c9d09f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f581be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f581be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43722/0 (socket says 172.21.15.46:43722) 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f581be00640 1 -- 172.21.15.46:0/3936262544 learned_addr learned my addr 172.21.15.46:0/3936262544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f5820e00640 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f581be00640 1 -- 172.21.15.46:0/3936262544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dc7090 msgr2=0x5602c9de2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f581be00640 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f581be00640 1 -- 172.21.15.46:0/3936262544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dcca60 msgr2=0x5602c9de97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f581be00640 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dcca60 0x5602c9de97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.149+0000 7f581be00640 1 -- 172.21.15.46:0/3936262544 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c9b0b620 con 0x5602c9dc79d0 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.150+0000 7f581be00640 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de2790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5808000c00 tx=0x7f58080049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2dbc531897978e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.150+0000 7f5802a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.150+0000 7f5802a00640 1 -- 172.21.15.46:0/3936262544 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808026020 con 0x5602c9dc79d0 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.150+0000 7f5802a00640 1 -- 172.21.15.46:0/3936262544 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f58080051a0 con 0x5602c9dc79d0 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.150+0000 7f58265c9740 1 -- 172.21.15.46:0/3936262544 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5808005500 con 0x5602c9dc79d0 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.151+0000 7f5802a00640 1 -- 172.21.15.46:0/3936262544 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f580801d040 con 0x5602c9dc79d0 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.151+0000 7f58265c9740 10 client.26015 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.151+0000 7f58265c9740 1 -- 172.21.15.46:0/3936262544 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5602c9dfd5d0 con 0x5602c9dc79d0 2024-09-10T07:39:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.151+0000 7f5802a00640 1 -- 172.21.15.46:0/3936262544 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5808005500 con 0x5602c9dc79d0 2024-09-10T07:39:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.151+0000 7f5802a00640 1 -- 172.21.15.46:0/3936262544 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f5808010430 con 0x5602c9dc79d0 2024-09-10T07:39:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.151+0000 7f58265c9740 10 client.26015 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.151+0000 7f58265c9740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.152+0000 7f58265c9740 20 client.26015 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.152+0000 7f58265c9740 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5602c9dfe9e0 0x5602c9e00dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.152+0000 7f58265c9740 4 client.26015 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.152+0000 7f58265c9740 1 -- 172.21.15.46:0/3936262544 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602c9d39830 con 0x5602c9dfe9e0 2024-09-10T07:39:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.153+0000 7f5820e00640 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5602c9dfe9e0 0x5602c9e00dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.154+0000 7f5820e00640 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5602c9dfe9e0 0x5602c9e00dd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7106f3e0bad01c93 server_cookie=aadf79d7bff53b41 in_seq=0 out_seq=0 2024-09-10T07:39:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.154+0000 7f5802a00640 10 client.26015 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.156+0000 7f5802a00640 1 -- 172.21.15.46:0/3936262544 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602c9d39830 con 0x5602c9dfe9e0 2024-09-10T07:39:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.156+0000 7f5802a00640 10 client.26015 handle_command_reply: tid=0 2024-09-10T07:39:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.156+0000 7f58265c9740 1 client.26015 shutdown 2024-09-10T07:39:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.157+0000 7f58265c9740 1 -- 172.21.15.46:0/3936262544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 msgr2=0x5602c9de2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.157+0000 7f58265c9740 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de2790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5808000c00 tx=0x7f58080049d0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.158+0000 7f58265c9740 1 -- 172.21.15.46:0/3936262544 shutdown_connections 2024-09-10T07:39:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.158+0000 7f58265c9740 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5602c9dfe9e0 0x5602c9e00dd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.158+0000 7f58265c9740 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dcca60 0x5602c9de97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.158+0000 7f58265c9740 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de2790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.158+0000 7f58265c9740 1 --2- 172.21.15.46:0/3936262544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.158+0000 7f5802a00640 0 client.26015 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.158+0000 7f58265c9740 1 -- 172.21.15.46:0/3936262544 >> 172.21.15.46:0/3936262544 conn(0x5602c9dc2700 msgr2=0x5602c9dc3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.159+0000 7f58265c9740 1 -- 172.21.15.46:0/3936262544 shutdown_connections 2024-09-10T07:39:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.159+0000 7f58265c9740 1 -- 172.21.15.46:0/3936262544 wait complete. 2024-09-10T07:39:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.159+0000 7f58265c9740 20 client.26015 trim_cache size 0 max 16384 2024-09-10T07:39:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.211+0000 7f58265c9740 1 Processor -- start 2024-09-10T07:39:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.211+0000 7f58265c9740 1 -- start start 2024-09-10T07:39:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f58265c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f58265c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f58265c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f58265c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1105b0 con 0x5602c9dc7090 2024-09-10T07:39:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f58265c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f581c112210 con 0x5602c9dc79d0 2024-09-10T07:39:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f58265c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f581c110c80 con 0x5602c9dcca60 2024-09-10T07:39:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f581be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f581be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43736/0 (socket says 172.21.15.46:43736) 2024-09-10T07:39:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.212+0000 7f5821800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.213+0000 7f5821800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53506/0 (socket says 172.21.15.46:53506) 2024-09-10T07:39:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.213+0000 7f5821800640 1 -- 172.21.15.46:0/3299705681 learned_addr learned my addr 172.21.15.46:0/3299705681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.213+0000 7f581be00640 1 -- 172.21.15.46:0/3299705681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 msgr2=0x5602c9de2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.213+0000 7f581be00640 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.213+0000 7f581be00640 1 -- 172.21.15.46:0/3299705681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 msgr2=0x5602c9de2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.213+0000 7f5820e00640 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.214+0000 7f581be00640 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.214+0000 7f581be00640 1 -- 172.21.15.46:0/3299705681 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c9dbdd60 con 0x5602c9dc79d0 2024-09-10T07:39:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.214+0000 7f5821800640 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:39:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.214+0000 7f5820e00640 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.214+0000 7f581be00640 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de9790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5602c9de3730 tx=0x7f5808005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=494eed0c941f3ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.215+0000 7f580d600640 1 -- 172.21.15.46:0/3299705681 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808037070 con 0x5602c9dc79d0 2024-09-10T07:39:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.215+0000 7f580d600640 1 -- 172.21.15.46:0/3299705681 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580800ee90 con 0x5602c9dc79d0 2024-09-10T07:39:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.215+0000 7f580d600640 1 -- 172.21.15.46:0/3299705681 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808037070 con 0x5602c9dc79d0 2024-09-10T07:39:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.217+0000 7f58265c9740 1 -- 172.21.15.46:0/3299705681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 msgr2=0x5602c9de9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.217+0000 7f58265c9740 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de9790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5602c9de3730 tx=0x7f5808005a00 comp rx=0 tx=0).stop 2024-09-10T07:39:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.220+0000 7f58265c9740 1 -- 172.21.15.46:0/3299705681 shutdown_connections 2024-09-10T07:39:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.220+0000 7f58265c9740 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.220+0000 7f58265c9740 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc79d0 0x5602c9de9790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.220+0000 7f58265c9740 1 --2- 172.21.15.46:0/3299705681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc7090 0x5602c9de2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.220+0000 7f58265c9740 1 -- 172.21.15.46:0/3299705681 >> 172.21.15.46:0/3299705681 conn(0x5602c9dc2700 msgr2=0x5602c9dc8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.220+0000 7f58265c9740 1 -- 172.21.15.46:0/3299705681 shutdown_connections 2024-09-10T07:39:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.221+0000 7f58265c9740 1 -- 172.21.15.46:0/3299705681 wait complete. 2024-09-10T07:39:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.222+0000 7f58265c9740 1 -- start start 2024-09-10T07:39:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.222+0000 7f58265c9740 1 Processor -- start 2024-09-10T07:39:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.223+0000 7f58265c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc7090 0x5602c9de6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.223+0000 7f58265c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc79d0 0x5602c9de7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.223+0000 7f58265c9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.223+0000 7f58265c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f581c111a80 con 0x5602c9dc79d0 2024-09-10T07:39:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.223+0000 7f58265c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1113b0 con 0x5602c9dc7090 2024-09-10T07:39:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.223+0000 7f58265c9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f581c073f50 con 0x5602c9dcca60 2024-09-10T07:39:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.223+0000 7f58265c9740 10 client.0 _ll_register_callbacks cb 0x7f5808009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.223+0000 7f5821800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.224+0000 7f5821800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53522/0 (socket says 172.21.15.46:53522) 2024-09-10T07:39:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.224+0000 7f5821800640 1 -- 172.21.15.46:0/324487744 learned_addr learned my addr 172.21.15.46:0/324487744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.225+0000 7f5821800640 1 -- 172.21.15.46:0/324487744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc7090 msgr2=0x5602c9de6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.225+0000 7f581be00640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc79d0 0x5602c9de7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.225+0000 7f5820e00640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc7090 0x5602c9de6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.225+0000 7f5821800640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc7090 0x5602c9de6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.225+0000 7f5821800640 1 -- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc79d0 msgr2=0x5602c9de7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.225+0000 7f5821800640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc79d0 0x5602c9de7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.225+0000 7f5821800640 1 -- 172.21.15.46:0/324487744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c9de12c0 con 0x5602c9dcca60 2024-09-10T07:39:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.226+0000 7f581be00640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc79d0 0x5602c9de7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.226+0000 7f5820e00640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc7090 0x5602c9de6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.227+0000 7f5821800640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de8340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f581409bc80 tx=0x7f581409b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=792dd9167d59088c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.227+0000 7f5802a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.227+0000 7f5802a00640 1 -- 172.21.15.46:0/324487744 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58140a4960 con 0x5602c9dcca60 2024-09-10T07:39:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.227+0000 7f5802a00640 1 -- 172.21.15.46:0/324487744 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f58140babc0 con 0x5602c9dcca60 2024-09-10T07:39:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.228+0000 7f58265c9740 1 -- 172.21.15.46:0/324487744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580800ee90 con 0x5602c9dcca60 2024-09-10T07:39:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.228+0000 7f5802a00640 1 -- 172.21.15.46:0/324487744 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f58140a0330 con 0x5602c9dcca60 2024-09-10T07:39:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.228+0000 7f58265c9740 10 client.26011 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.228+0000 7f58265c9740 1 -- 172.21.15.46:0/324487744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5602c9de9a70 con 0x5602c9dcca60 2024-09-10T07:39:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.229+0000 7f5802a00640 1 -- 172.21.15.46:0/324487744 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580800ee90 con 0x5602c9dcca60 2024-09-10T07:39:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.229+0000 7f5802a00640 1 -- 172.21.15.46:0/324487744 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f5814097cd0 con 0x5602c9dcca60 2024-09-10T07:39:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.229+0000 7f58265c9740 10 client.26011 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.229+0000 7f58265c9740 10 client.26011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.229+0000 7f58265c9740 20 client.26011 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.229+0000 7f58265c9740 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5602c9e69d60 0x5602c9e6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.229+0000 7f58265c9740 4 client.26011 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:39:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.229+0000 7f58265c9740 1 -- 172.21.15.46:0/324487744 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602c9d33b80 con 0x5602c9e69d60 2024-09-10T07:39:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.230+0000 7f5820e00640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5602c9e69d60 0x5602c9e6c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.231+0000 7f5802a00640 10 client.26011 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.232+0000 7f5820e00640 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5602c9e69d60 0x5602c9e6c180 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476da6e5421abd0f server_cookie=ea5312ac3e952c36 in_seq=0 out_seq=0 2024-09-10T07:39:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.247+0000 7f5802a00640 1 -- 172.21.15.46:0/324487744 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602c9d33b80 con 0x5602c9e69d60 2024-09-10T07:39:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.247+0000 7f5802a00640 10 client.26011 handle_command_reply: tid=0 2024-09-10T07:39:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.247+0000 7f58265c9740 1 client.26011 shutdown 2024-09-10T07:39:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.247+0000 7f58265c9740 1 -- 172.21.15.46:0/324487744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 msgr2=0x5602c9de8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.248+0000 7f58265c9740 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de8340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f581409bc80 tx=0x7f581409b9a0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.248+0000 7f58265c9740 1 -- 172.21.15.46:0/324487744 shutdown_connections 2024-09-10T07:39:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.248+0000 7f58265c9740 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5602c9e69d60 0x5602c9e6c180 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.248+0000 7f58265c9740 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602c9dcca60 0x5602c9de8340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.248+0000 7f58265c9740 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602c9dc79d0 0x5602c9de7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.248+0000 7f58265c9740 1 --2- 172.21.15.46:0/324487744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602c9dc7090 0x5602c9de6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.248+0000 7f58265c9740 1 -- 172.21.15.46:0/324487744 >> 172.21.15.46:0/324487744 conn(0x5602c9dc2700 msgr2=0x5602c9dc8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.248+0000 7f5802a00640 0 client.26011 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.249+0000 7f58265c9740 1 -- 172.21.15.46:0/324487744 shutdown_connections 2024-09-10T07:39:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.249+0000 7f58265c9740 1 -- 172.21.15.46:0/324487744 wait complete. 2024-09-10T07:39:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.250+0000 7f58265c9740 20 client.26011 trim_cache size 0 max 16384 2024-09-10T07:39:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:39:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:39:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:39:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:39:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 -- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f58040a2d30 msgr2=0x7f58040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f58040a2d30 0x7f58040a51f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58100057c0 tx=0x7f5810030cc0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 -- 172.21.15.46:0/3890897256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f581c105910 msgr2=0x7f581c1a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f581c105910 0x7f581c1a5970 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f581400eb10 tx=0x7f581400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 -- 172.21.15.46:0/3890897256 shutdown_connections 2024-09-10T07:39:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f581c105910 0x7f581c1a5970 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c104f50 0x7f581c19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f58040a2d30 0x7f58040a51f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 --2- 172.21.15.46:0/3890897256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c103d60 0x7f581c19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.251+0000 7f5823800640 1 -- 172.21.15.46:0/3890897256 >> 172.21.15.46:0/3890897256 conn(0x7f581c0ff4e0 msgr2=0x7f581c07a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.252+0000 7f5823800640 1 -- 172.21.15.46:0/3890897256 shutdown_connections 2024-09-10T07:39:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:07.252+0000 7f5823800640 1 -- 172.21.15.46:0/3890897256 wait complete. 2024-09-10T07:39:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.646+0000 7f4d2ca00640 1 -- 172.21.15.46:0/431953011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 msgr2=0x7f4d28110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.646+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/431953011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 0x7f4d28110910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ca0 tx=0x7f4d2002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.646+0000 7f4d2ca00640 1 -- 172.21.15.46:0/431953011 shutdown_connections 2024-09-10T07:39:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.646+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/431953011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 0x7f4d28110910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.646+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/431953011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28101440 0x7f4d2810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.646+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/431953011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28100b20 0x7f4d28100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.647+0000 7f4d2ca00640 1 -- 172.21.15.46:0/431953011 >> 172.21.15.46:0/431953011 conn(0x7f4d280fc7d0 msgr2=0x7f4d280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.647+0000 7f4d2ca00640 1 -- 172.21.15.46:0/431953011 shutdown_connections 2024-09-10T07:39:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.647+0000 7f4d2ca00640 1 -- 172.21.15.46:0/431953011 wait complete. 2024-09-10T07:39:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.648+0000 7f4d2ca00640 1 Processor -- start 2024-09-10T07:39:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.649+0000 7f4d2ca00640 1 -- start start 2024-09-10T07:39:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.650+0000 7f4d2ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28100b20 0x7f4d28107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.650+0000 7f4d2ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28101440 0x7f4d28107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.650+0000 7f4d2ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 0x7f4d28102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.650+0000 7f4d2ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28114230 con 0x7f4d28100b20 2024-09-10T07:39:08.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.650+0000 7f4d2ca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281140b0 con 0x7f4d28101440 2024-09-10T07:39:08.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.650+0000 7f4d2ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281143b0 con 0x7f4d2810e500 2024-09-10T07:39:08.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.651+0000 7f4d25400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28101440 0x7f4d28107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.651+0000 7f4d25400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28101440 0x7f4d28107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43760/0 (socket says 172.21.15.46:43760) 2024-09-10T07:39:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.651+0000 7f4d25400640 1 -- 172.21.15.46:0/3159582431 learned_addr learned my addr 172.21.15.46:0/3159582431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.651+0000 7f4d26800640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 0x7f4d28102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.651+0000 7f4d25e00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28100b20 0x7f4d28107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.652+0000 7f4d25400640 1 -- 172.21.15.46:0/3159582431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 msgr2=0x7f4d28102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.652+0000 7f4d25400640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 0x7f4d28102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.652+0000 7f4d25400640 1 -- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28100b20 msgr2=0x7f4d28107560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.652+0000 7f4d25400640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28100b20 0x7f4d28107560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.652+0000 7f4d25400640 1 -- 172.21.15.46:0/3159582431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d28103390 con 0x7f4d28101440 2024-09-10T07:39:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.652+0000 7f4d26800640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 0x7f4d28102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:39:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.652+0000 7f4d25e00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28100b20 0x7f4d28107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:39:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.653+0000 7f4d25400640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28101440 0x7f4d28107aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e9c0 tx=0x7f4d1400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.654+0000 7f4d1ea00640 1 -- 172.21.15.46:0/3159582431 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400cd80 con 0x7f4d28101440 2024-09-10T07:39:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.654+0000 7f4d1ea00640 1 -- 172.21.15.46:0/3159582431 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d140035f0 con 0x7f4d28101440 2024-09-10T07:39:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.654+0000 7f4d2ca00640 1 -- 172.21.15.46:0/3159582431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d28103680 con 0x7f4d28101440 2024-09-10T07:39:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.654+0000 7f4d1ea00640 1 -- 172.21.15.46:0/3159582431 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140186d0 con 0x7f4d28101440 2024-09-10T07:39:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.654+0000 7f4d2ca00640 1 -- 172.21.15.46:0/3159582431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2810d450 con 0x7f4d28101440 2024-09-10T07:39:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.660+0000 7f4d2f412740 1 Processor -- start 2024-09-10T07:39:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.661+0000 7f4d2f412740 1 -- start start 2024-09-10T07:39:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.661+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b20250050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.661+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 0x560b2025a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.661+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b2025d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.661+0000 7f4d2f412740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b201b63e0 con 0x560b2025ae50 2024-09-10T07:39:08.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.661+0000 7f4d2f412740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560b2015dd80 con 0x560b2024fc70 2024-09-10T07:39:08.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.661+0000 7f4d2f412740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b201b23a0 con 0x560b20250590 2024-09-10T07:39:08.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.662+0000 7f4d26800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b20250050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.662+0000 7f4d26800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b20250050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43770/0 (socket says 172.21.15.46:43770) 2024-09-10T07:39:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.662+0000 7f4d26800640 1 -- 172.21.15.46:0/1491181495 learned_addr learned my addr 172.21.15.46:0/1491181495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.662+0000 7f4d26800640 1 -- 172.21.15.46:0/1491181495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 msgr2=0x560b2025a910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:39:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.662+0000 7f4d26800640 1 --2- 172.21.15.46:0/1491181495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 0x560b2025a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.662+0000 7f4d26800640 1 -- 172.21.15.46:0/1491181495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 msgr2=0x560b2025d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:39:08.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.662+0000 7f4d26800640 1 --2- 172.21.15.46:0/1491181495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b2025d240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.662+0000 7f4d26800640 1 -- 172.21.15.46:0/1491181495 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2017cae0 con 0x560b2024fc70 2024-09-10T07:39:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.663+0000 7f4d26800640 1 --2- 172.21.15.46:0/1491181495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b20250050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ca0 tx=0x7f4d200305c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd11dc536aebcd7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.663+0000 7f4d13000640 1 -- 172.21.15.46:0/1491181495 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20030a00 con 0x560b2024fc70 2024-09-10T07:39:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.663+0000 7f4d13000640 1 -- 172.21.15.46:0/1491181495 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d20036040 con 0x560b2024fc70 2024-09-10T07:39:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.663+0000 7f4d13000640 1 -- 172.21.15.46:0/1491181495 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20031970 con 0x560b2024fc70 2024-09-10T07:39:08.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.664+0000 7f4d1ea00640 1 -- 172.21.15.46:0/3159582431 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d140037a0 con 0x7f4d28101440 2024-09-10T07:39:08.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.665+0000 7f4d2f412740 1 -- 172.21.15.46:0/1491181495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 msgr2=0x560b20250050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.665+0000 7f4d2f412740 1 --2- 172.21.15.46:0/1491181495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b20250050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ca0 tx=0x7f4d200305c0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.665+0000 7f4d1ea00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2d90 0x7f4d0c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.666+0000 7f4d1ea00640 1 -- 172.21.15.46:0/3159582431 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d140cf700 con 0x7f4d28101440 2024-09-10T07:39:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.666+0000 7f4d26800640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2d90 0x7f4d0c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.666+0000 7f4d2f412740 1 -- 172.21.15.46:0/1491181495 shutdown_connections 2024-09-10T07:39:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.666+0000 7f4d2f412740 1 --2- 172.21.15.46:0/1491181495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b2025d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.666+0000 7f4d2f412740 1 --2- 172.21.15.46:0/1491181495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 0x560b2025a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.666+0000 7f4d2f412740 1 --2- 172.21.15.46:0/1491181495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b20250050 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.666+0000 7f4d2f412740 1 -- 172.21.15.46:0/1491181495 >> 172.21.15.46:0/1491181495 conn(0x560b2024c200 msgr2=0x560b2024c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.667+0000 7f4d2f412740 1 -- 172.21.15.46:0/1491181495 shutdown_connections 2024-09-10T07:39:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.667+0000 7f4d2f412740 1 -- 172.21.15.46:0/1491181495 wait complete. 2024-09-10T07:39:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.667+0000 7f4d26800640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2d90 0x7f4d0c0a5250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009960 tx=0x7f4d20031590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.668+0000 7f4d2f412740 1 -- start start 2024-09-10T07:39:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.668+0000 7f4d2f412740 1 Processor -- start 2024-09-10T07:39:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.669+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2024fc70 0x560b20257cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.669+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 0x560b20258200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.669+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2025ae50 0x560b20258740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.669+0000 7f4d2f412740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b2025d8a0 con 0x560b2024fc70 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.669+0000 7f4d2f412740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560b201710f0 con 0x560b20250590 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.669+0000 7f4d2f412740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b2025da20 con 0x560b2025ae50 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.669+0000 7f4d2f412740 10 client.0 _ll_register_callbacks cb 0x560b20158bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.670+0000 7f4d26800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2025ae50 0x560b20258740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.670+0000 7f4d26800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2025ae50 0x560b20258740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53536/0 (socket says 172.21.15.46:53536) 2024-09-10T07:39:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.670+0000 7f4d26800640 1 -- 172.21.15.46:0/339038584 learned_addr learned my addr 172.21.15.46:0/339038584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.670+0000 7f4d25400640 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 0x560b20258200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.670+0000 7f4d25e00640 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2024fc70 0x560b20257cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.671+0000 7f4d26800640 1 -- 172.21.15.46:0/339038584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 msgr2=0x560b20258200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.671+0000 7f4d26800640 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 0x560b20258200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.671+0000 7f4d26800640 1 -- 172.21.15.46:0/339038584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2024fc70 msgr2=0x560b20257cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.671+0000 7f4d26800640 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2024fc70 0x560b20257cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.671+0000 7f4d26800640 1 -- 172.21.15.46:0/339038584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1ff273c0 con 0x560b2025ae50 2024-09-10T07:39:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.671+0000 7f4d26800640 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2025ae50 0x560b20258740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009bf0 tx=0x7f4d200308b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=309165af7110e859 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.671+0000 7f4d07400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.672+0000 7f4d07400640 1 -- 172.21.15.46:0/339038584 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20060060 con 0x560b2025ae50 2024-09-10T07:39:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.672+0000 7f4d07400640 1 -- 172.21.15.46:0/339038584 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d20046b20 con 0x560b2025ae50 2024-09-10T07:39:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.672+0000 7f4d2f412740 1 -- 172.21.15.46:0/339038584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d20036040 con 0x560b2025ae50 2024-09-10T07:39:08.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.672+0000 7f4d07400640 1 -- 172.21.15.46:0/339038584 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4d20046d20 con 0x560b2025ae50 2024-09-10T07:39:08.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.672+0000 7f4d2f412740 10 client.26020 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:08.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.672+0000 7f4d2f412740 1 -- 172.21.15.46:0/339038584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560b2025d8a0 con 0x560b2025ae50 2024-09-10T07:39:08.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.673+0000 7f4d07400640 1 -- 172.21.15.46:0/339038584 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d20036040 con 0x560b2025ae50 2024-09-10T07:39:08.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.673+0000 7f4d07400640 1 -- 172.21.15.46:0/339038584 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f4d2005a070 con 0x560b2025ae50 2024-09-10T07:39:08.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.673+0000 7f4d2f412740 10 client.26020 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:08.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.673+0000 7f4d2f412740 10 client.26020 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:39:08.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.673+0000 7f4d2f412740 20 client.26020 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:08.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.673+0000 7f4d2f412740 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b2027ba20 0x560b2027de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.673+0000 7f4d2f412740 4 client.26020 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:08.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.673+0000 7f4d2f412740 1 -- 172.21.15.46:0/339038584 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b201c3950 con 0x560b2027ba20 2024-09-10T07:39:08.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.674+0000 7f4d25e00640 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b2027ba20 0x560b2027de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.675+0000 7f4d25e00640 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b2027ba20 0x560b2027de10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=faf0f312560c8e63 server_cookie=48a2f044600dd268 in_seq=0 out_seq=0 2024-09-10T07:39:08.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.675+0000 7f4d07400640 10 client.26020 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:39:08.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:08 smithi046 ceph-mon[30705]: pgmap v474: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 803 op/s 2024-09-10T07:39:08.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.675+0000 7f4d07400640 1 -- 172.21.15.46:0/339038584 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b201c3950 con 0x560b2027ba20 2024-09-10T07:39:08.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.675+0000 7f4d07400640 10 client.26020 handle_command_reply: tid=0 2024-09-10T07:39:08.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 client.26020 shutdown 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 -- 172.21.15.46:0/339038584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2025ae50 msgr2=0x560b20258740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2025ae50 0x560b20258740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009bf0 tx=0x7f4d200308b0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 -- 172.21.15.46:0/339038584 shutdown_connections 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b2027ba20 0x560b2027de10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2025ae50 0x560b20258740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 0x560b20258200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 --2- 172.21.15.46:0/339038584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2024fc70 0x560b20257cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d2f412740 1 -- 172.21.15.46:0/339038584 >> 172.21.15.46:0/339038584 conn(0x560b2024c200 msgr2=0x560b2024b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.676+0000 7f4d07400640 0 client.26020 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:39:08.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.678+0000 7f4d2f412740 1 -- 172.21.15.46:0/339038584 shutdown_connections 2024-09-10T07:39:08.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.678+0000 7f4d2f412740 1 -- 172.21.15.46:0/339038584 wait complete. 2024-09-10T07:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.678+0000 7f4d2f412740 20 client.26020 trim_cache size 0 max 16384 2024-09-10T07:39:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.699+0000 7f4d2f412740 1 Processor -- start 2024-09-10T07:39:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.699+0000 7f4d2f412740 1 -- start start 2024-09-10T07:39:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2024fc70 0x560b20258590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 0x560b20259c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b202835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d2f412740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b52a0 con 0x560b2025ae50 2024-09-10T07:39:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d2f412740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b6f00 con 0x560b20250590 2024-09-10T07:39:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d2f412740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b5970 con 0x560b2024fc70 2024-09-10T07:39:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2024fc70 0x560b20258590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2024fc70 0x560b20258590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53552/0 (socket says 172.21.15.46:53552) 2024-09-10T07:39:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.700+0000 7f4d25e00640 1 -- 172.21.15.46:0/2074975818 learned_addr learned my addr 172.21.15.46:0/2074975818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.701+0000 7f4d25e00640 1 -- 172.21.15.46:0/2074975818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 msgr2=0x560b20259c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.701+0000 7f4d25e00640 1 --2- 172.21.15.46:0/2074975818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 0x560b20259c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.701+0000 7f4d25e00640 1 -- 172.21.15.46:0/2074975818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 msgr2=0x560b202835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.701+0000 7f4d25e00640 1 --2- 172.21.15.46:0/2074975818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b202835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.701+0000 7f4d25e00640 1 -- 172.21.15.46:0/2074975818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2024faa0 con 0x560b2024fc70 2024-09-10T07:39:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.701+0000 7f4d25e00640 1 --2- 172.21.15.46:0/2074975818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2024fc70 0x560b20258590 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4d18011460 tx=0x7f4d18005660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c2802167cb39aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:08.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.702+0000 7f4d12600640 1 -- 172.21.15.46:0/2074975818 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18004220 con 0x560b2024fc70 2024-09-10T07:39:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.702+0000 7f4d12600640 1 -- 172.21.15.46:0/2074975818 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18004e10 con 0x560b2024fc70 2024-09-10T07:39:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.702+0000 7f4d12600640 1 -- 172.21.15.46:0/2074975818 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d180119b0 con 0x560b2024fc70 2024-09-10T07:39:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.703+0000 7f4d2f412740 1 -- 172.21.15.46:0/2074975818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2024fc70 msgr2=0x560b20258590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.703+0000 7f4d2f412740 1 --2- 172.21.15.46:0/2074975818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2024fc70 0x560b20258590 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4d18011460 tx=0x7f4d18005660 comp rx=0 tx=0).stop 2024-09-10T07:39:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.703+0000 7f4d2f412740 1 -- 172.21.15.46:0/2074975818 shutdown_connections 2024-09-10T07:39:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.703+0000 7f4d2f412740 1 --2- 172.21.15.46:0/2074975818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b202835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.703+0000 7f4d2f412740 1 --2- 172.21.15.46:0/2074975818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b20250590 0x560b20259c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.703+0000 7f4d2f412740 1 --2- 172.21.15.46:0/2074975818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2024fc70 0x560b20258590 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.703+0000 7f4d2f412740 1 -- 172.21.15.46:0/2074975818 >> 172.21.15.46:0/2074975818 conn(0x560b2024c200 msgr2=0x560b20259520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:08.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.704+0000 7f4d2f412740 1 -- 172.21.15.46:0/2074975818 shutdown_connections 2024-09-10T07:39:08.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.704+0000 7f4d2f412740 1 -- 172.21.15.46:0/2074975818 wait complete. 2024-09-10T07:39:08.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.705+0000 7f4d2f412740 1 -- start start 2024-09-10T07:39:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.705+0000 7f4d2f412740 1 Processor -- start 2024-09-10T07:39:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.706+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b202633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.707+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 0x560b2025e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.707+0000 7f4d2f412740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b2025ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.707+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b202633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.707+0000 7f4d25400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 0x560b2025e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.707+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b202633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43792/0 (socket says 172.21.15.46:43792) 2024-09-10T07:39:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.707+0000 7f4d25400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 0x560b2025e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53554/0 (socket says 172.21.15.46:53554) 2024-09-10T07:39:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.708+0000 7f4d2f412740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b6770 con 0x560b2025ae50 2024-09-10T07:39:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.708+0000 7f4d2f412740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b60a0 con 0x560b2024fc70 2024-09-10T07:39:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.708+0000 7f4d2f412740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28072760 con 0x560b20250590 2024-09-10T07:39:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.708+0000 7f4d2f412740 10 client.0 _ll_register_callbacks cb 0x7f4d20030f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.708+0000 7f4d25e00640 1 -- 172.21.15.46:0/561368274 learned_addr learned my addr 172.21.15.46:0/561368274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.708+0000 7f4d26800640 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b2025ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.709+0000 7f4d26800640 1 -- 172.21.15.46:0/561368274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 msgr2=0x560b2025e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.709+0000 7f4d26800640 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 0x560b2025e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.709+0000 7f4d26800640 1 -- 172.21.15.46:0/561368274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 msgr2=0x560b202633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.709+0000 7f4d26800640 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b202633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.709+0000 7f4d26800640 1 -- 172.21.15.46:0/561368274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b20252ae0 con 0x560b2025ae50 2024-09-10T07:39:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.710+0000 7f4d26800640 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b2025ec20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d2002aeb0 tx=0x7f4d20000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb131daf5bc2e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.710+0000 7f4d07400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:39:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.710+0000 7f4d07400640 1 -- 172.21.15.46:0/561368274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20060030 con 0x560b2025ae50 2024-09-10T07:39:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.711+0000 7f4d07400640 1 -- 172.21.15.46:0/561368274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4d2004b070 con 0x560b2025ae50 2024-09-10T07:39:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.711+0000 7f4d2f412740 1 -- 172.21.15.46:0/561368274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d18004e10 con 0x560b2025ae50 2024-09-10T07:39:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.711+0000 7f4d07400640 1 -- 172.21.15.46:0/561368274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4d20030770 con 0x560b2025ae50 2024-09-10T07:39:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.711+0000 7f4d2f412740 10 client.15897 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.711+0000 7f4d2f412740 1 -- 172.21.15.46:0/561368274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560b20247770 con 0x560b2025ae50 2024-09-10T07:39:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.712+0000 7f4d07400640 1 -- 172.21.15.46:0/561368274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18004e10 con 0x560b2025ae50 2024-09-10T07:39:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.712+0000 7f4d07400640 1 -- 172.21.15.46:0/561368274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f4d2005a040 con 0x560b2025ae50 2024-09-10T07:39:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.712+0000 7f4d2f412740 10 client.15897 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.712+0000 7f4d2f412740 10 client.15897 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:39:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.712+0000 7f4d2f412740 20 client.15897 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.712+0000 7f4d2f412740 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b202e6840 0x560b202e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.712+0000 7f4d2f412740 4 client.15897 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:39:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.712+0000 7f4d2f412740 1 -- 172.21.15.46:0/561368274 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b201bde60 con 0x560b202e6840 2024-09-10T07:39:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.713+0000 7f4d25e00640 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b202e6840 0x560b202e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:08.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.714+0000 7f4d07400640 10 client.15897 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:39:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.715+0000 7f4d25e00640 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b202e6840 0x560b202e8c60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3efacad88a76800 server_cookie=74d4f8fdc0fbb382 in_seq=0 out_seq=0 2024-09-10T07:39:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.715+0000 7f4d07400640 1 -- 172.21.15.46:0/561368274 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b201bde60 con 0x560b202e6840 2024-09-10T07:39:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.715+0000 7f4d07400640 10 client.15897 handle_command_reply: tid=0 2024-09-10T07:39:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.715+0000 7f4d2f412740 1 client.15897 shutdown 2024-09-10T07:39:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.715+0000 7f4d2f412740 1 -- 172.21.15.46:0/561368274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 msgr2=0x560b2025ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.715+0000 7f4d2f412740 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b2025ec20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d2002aeb0 tx=0x7f4d20000f40 comp rx=0 tx=0).stop 2024-09-10T07:39:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.716+0000 7f4d2f412740 1 -- 172.21.15.46:0/561368274 shutdown_connections 2024-09-10T07:39:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.716+0000 7f4d2f412740 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x560b202e6840 0x560b202e8c60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.716+0000 7f4d2f412740 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2025ae50 0x560b2025ec20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.716+0000 7f4d2f412740 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b20250590 0x560b2025e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.716+0000 7f4d2f412740 1 --2- 172.21.15.46:0/561368274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2024fc70 0x560b202633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.716+0000 7f4d2f412740 1 -- 172.21.15.46:0/561368274 >> 172.21.15.46:0/561368274 conn(0x560b2024c200 msgr2=0x560b20251ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.716+0000 7f4d07400640 0 client.15897 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:39:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.716+0000 7f4d2f412740 1 -- 172.21.15.46:0/561368274 shutdown_connections 2024-09-10T07:39:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.717+0000 7f4d2f412740 1 -- 172.21.15.46:0/561368274 wait complete. 2024-09-10T07:39:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.717+0000 7f4d2f412740 20 client.15897 trim_cache size 0 max 16384 2024-09-10T07:39:08.718 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:39:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.718+0000 7f4d2ca00640 1 -- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2d90 msgr2=0x7f4d0c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.718+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2d90 0x7f4d0c0a5250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009960 tx=0x7f4d20031590 comp rx=0 tx=0).stop 2024-09-10T07:39:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.718+0000 7f4d2ca00640 1 -- 172.21.15.46:0/3159582431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28101440 msgr2=0x7f4d28107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.718+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28101440 0x7f4d28107aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e9c0 tx=0x7f4d1400ee80 comp rx=0 tx=0).stop 2024-09-10T07:39:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.719+0000 7f4d2ca00640 1 -- 172.21.15.46:0/3159582431 shutdown_connections 2024-09-10T07:39:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.719+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2810e500 0x7f4d28102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.719+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d28101440 0x7f4d28107aa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.719+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4d0c0a2d90 0x7f4d0c0a5250 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.719+0000 7f4d2ca00640 1 --2- 172.21.15.46:0/3159582431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d28100b20 0x7f4d28107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.719+0000 7f4d2ca00640 1 -- 172.21.15.46:0/3159582431 >> 172.21.15.46:0/3159582431 conn(0x7f4d280fc7d0 msgr2=0x7f4d280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.719+0000 7f4d2ca00640 1 -- 172.21.15.46:0/3159582431 shutdown_connections 2024-09-10T07:39:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:08.719+0000 7f4d2ca00640 1 -- 172.21.15.46:0/3159582431 wait complete. 2024-09-10T07:39:08.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:08 smithi181 ceph-mon[35397]: pgmap v474: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 803 op/s 2024-09-10T07:39:08.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:08 smithi063 ceph-mon[35020]: pgmap v474: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 803 op/s 2024-09-10T07:39:09.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:39:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:10.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 110.0 s, 2178.0 tps, lat 45.773 ms stddev 53.538 2024-09-10T07:39:10.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:10 smithi046 ceph-mon[30705]: pgmap v475: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-09-10T07:39:10.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:10 smithi181 ceph-mon[35397]: pgmap v475: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-09-10T07:39:10.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:10 smithi063 ceph-mon[35020]: pgmap v475: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-09-10T07:39:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.437+0000 7f68c6000640 1 -- 172.21.15.46:0/2501582627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 msgr2=0x7f68c0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.437+0000 7f68c6000640 1 --2- 172.21.15.46:0/2501582627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 0x7f68c0106e60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f68b000d940 tx=0x7f68b002c3a0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.437+0000 7f68c6000640 1 -- 172.21.15.46:0/2501582627 shutdown_connections 2024-09-10T07:39:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.437+0000 7f68c6000640 1 --2- 172.21.15.46:0/2501582627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c0101420 0x7f68c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.437+0000 7f68c6000640 1 --2- 172.21.15.46:0/2501582627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c0100a80 0x7f68c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.437+0000 7f68c6000640 1 --2- 172.21.15.46:0/2501582627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 0x7f68c0106e60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.437+0000 7f68c6000640 1 -- 172.21.15.46:0/2501582627 >> 172.21.15.46:0/2501582627 conn(0x7f68c00fc7a0 msgr2=0x7f68c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:11.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.437+0000 7f68c6000640 1 -- 172.21.15.46:0/2501582627 shutdown_connections 2024-09-10T07:39:11.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.438+0000 7f68c6000640 1 -- 172.21.15.46:0/2501582627 wait complete. 2024-09-10T07:39:11.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.439+0000 7f68c6000640 1 Processor -- start 2024-09-10T07:39:11.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.441+0000 7f68c6000640 1 -- start start 2024-09-10T07:39:11.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.443+0000 7f68c6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c0100a80 0x7f68c019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.443+0000 7f68c6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c0101420 0x7f68c019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.443+0000 7f68c6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 0x7f68c0196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.443+0000 7f68bf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c0100a80 0x7f68c019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.443+0000 7f68bf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c0100a80 0x7f68c019cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43806/0 (socket says 172.21.15.46:43806) 2024-09-10T07:39:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68c6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68c010fdd0 con 0x7f68c0106a80 2024-09-10T07:39:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68c6000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68c010fc50 con 0x7f68c0100a80 2024-09-10T07:39:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68c6000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68c010ff50 con 0x7f68c0101420 2024-09-10T07:39:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c0101420 0x7f68c019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c0101420 0x7f68c019d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53568/0 (socket says 172.21.15.46:53568) 2024-09-10T07:39:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bea00640 1 -- 172.21.15.46:0/1202964684 learned_addr learned my addr 172.21.15.46:0/1202964684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bfe00640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 0x7f68c0196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bf400640 1 -- 172.21.15.46:0/1202964684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c0101420 msgr2=0x7f68c019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bf400640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c0101420 0x7f68c019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bf400640 1 -- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 msgr2=0x7f68c0196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bf400640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 0x7f68c0196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bf400640 1 -- 172.21.15.46:0/1202964684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c0196b00 con 0x7f68c0100a80 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.444+0000 7f68bfe00640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 0x7f68c0196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.445+0000 7f68bf400640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c0100a80 0x7f68c019cfd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68b002c990 tx=0x7f68b00346d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.445+0000 7f68b7e00640 1 -- 172.21.15.46:0/1202964684 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0034ad0 con 0x7f68c0100a80 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.445+0000 7f68b7e00640 1 -- 172.21.15.46:0/1202964684 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b0034c70 con 0x7f68c0100a80 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.445+0000 7f68b7e00640 1 -- 172.21.15.46:0/1202964684 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00333c0 con 0x7f68c0100a80 2024-09-10T07:39:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.445+0000 7f68c6000640 1 -- 172.21.15.46:0/1202964684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c0196d90 con 0x7f68c0100a80 2024-09-10T07:39:11.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.446+0000 7f68c6000640 1 -- 172.21.15.46:0/1202964684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c019acd0 con 0x7f68c0100a80 2024-09-10T07:39:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68b7e00640 1 -- 172.21.15.46:0/1202964684 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68b0043030 con 0x7f68c0100a80 2024-09-10T07:39:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68c8a25740 1 Processor -- start 2024-09-10T07:39:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68c8a25740 1 -- start start 2024-09-10T07:39:11.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527ac70 0x55e61527b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527b590 0x55e615285910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e615285e50 0x55e615288240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68c8a25740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e6151e13e0 con 0x55e61527b590 2024-09-10T07:39:11.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68c8a25740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e615188d80 con 0x55e61527ac70 2024-09-10T07:39:11.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.453+0000 7f68c8a25740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e6151dd3a0 con 0x55e615285e50 2024-09-10T07:39:11.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527ac70 0x55e61527b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527ac70 0x55e61527b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43816/0 (socket says 172.21.15.46:43816) 2024-09-10T07:39:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bfe00640 1 -- 172.21.15.46:0/3450286887 learned_addr learned my addr 172.21.15.46:0/3450286887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bea00640 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e615285e50 0x55e615288240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bfe00640 1 -- 172.21.15.46:0/3450286887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e615285e50 msgr2=0x55e615288240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bfe00640 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e615285e50 0x55e615288240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bfe00640 1 -- 172.21.15.46:0/3450286887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527b590 msgr2=0x55e615285910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:39:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bfe00640 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527b590 0x55e615285910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.454+0000 7f68bfe00640 1 -- 172.21.15.46:0/3450286887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6151a7ae0 con 0x55e61527ac70 2024-09-10T07:39:11.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.455+0000 7f68bfe00640 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527ac70 0x55e61527b050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009f10 tx=0x7f68b800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b00a905fcaf19d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:11.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.455+0000 7f68bea00640 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e615285e50 0x55e615288240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:11.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.455+0000 7f68b7e00640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68a40a2d50 0x7f68a40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.455+0000 7f68abe00640 1 -- 172.21.15.46:0/3450286887 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8003b70 con 0x55e61527ac70 2024-09-10T07:39:11.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.455+0000 7f68b7e00640 1 -- 172.21.15.46:0/1202964684 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f68b00eebf0 con 0x7f68c0100a80 2024-09-10T07:39:11.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.455+0000 7f68abe00640 1 -- 172.21.15.46:0/3450286887 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b8003d10 con 0x55e61527ac70 2024-09-10T07:39:11.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.455+0000 7f68abe00640 1 -- 172.21.15.46:0/3450286887 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8011ae0 con 0x55e61527ac70 2024-09-10T07:39:11.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.456+0000 7f68bf400640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68a40a2d50 0x7f68a40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.458+0000 7f68c8a25740 1 -- 172.21.15.46:0/3450286887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527ac70 msgr2=0x55e61527b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.458+0000 7f68c8a25740 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527ac70 0x55e61527b050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009f10 tx=0x7f68b800f730 comp rx=0 tx=0).stop 2024-09-10T07:39:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.458+0000 7f68bf400640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68a40a2d50 0x7f68a40a5210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f68b00c37b0 tx=0x7f68b003cd30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.458+0000 7f68c8a25740 1 -- 172.21.15.46:0/3450286887 shutdown_connections 2024-09-10T07:39:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.458+0000 7f68c8a25740 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e615285e50 0x55e615288240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.458+0000 7f68c8a25740 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527b590 0x55e615285910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.458+0000 7f68c8a25740 1 --2- 172.21.15.46:0/3450286887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527ac70 0x55e61527b050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.458+0000 7f68c8a25740 1 -- 172.21.15.46:0/3450286887 >> 172.21.15.46:0/3450286887 conn(0x55e615277200 msgr2=0x55e6152775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.459+0000 7f68c8a25740 1 -- 172.21.15.46:0/3450286887 shutdown_connections 2024-09-10T07:39:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.459+0000 7f68c8a25740 1 -- 172.21.15.46:0/3450286887 wait complete. 2024-09-10T07:39:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.460+0000 7f68c8a25740 1 -- start start 2024-09-10T07:39:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.460+0000 7f68c8a25740 1 Processor -- start 2024-09-10T07:39:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.462+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 0x55e615282dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.462+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e615283310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e615283850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68c8a25740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e615288a20 con 0x55e61527ac70 2024-09-10T07:39:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68c8a25740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e61519c0f0 con 0x55e615285e50 2024-09-10T07:39:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68c8a25740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e615288ba0 con 0x55e61527b590 2024-09-10T07:39:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 0x55e615282dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68c8a25740 10 client.0 _ll_register_callbacks cb 0x55e615183bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:11.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e615283850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e615283310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e615283850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43818/0 (socket says 172.21.15.46:43818) 2024-09-10T07:39:11.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bfe00640 1 -- 172.21.15.46:0/1396116135 learned_addr learned my addr 172.21.15.46:0/1396116135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bea00640 1 -- 172.21.15.46:0/1396116135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 msgr2=0x55e615283850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bea00640 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e615283850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bea00640 1 -- 172.21.15.46:0/1396116135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 msgr2=0x55e615282dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.463+0000 7f68bea00640 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 0x55e615282dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.464+0000 7f68bea00640 1 -- 172.21.15.46:0/1396116135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e614f523c0 con 0x55e61527b590 2024-09-10T07:39:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.464+0000 7f68bfe00640 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e615283850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:39:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.464+0000 7f68a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.464+0000 7f68bea00640 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e615283310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f68c0197820 tx=0x7f68ac013b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f24c3a980c08829c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.464+0000 7f68a8c00640 1 -- 172.21.15.46:0/1396116135 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac011e60 con 0x55e61527b590 2024-09-10T07:39:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.464+0000 7f68c8a25740 1 -- 172.21.15.46:0/1396116135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b8003d10 con 0x55e61527b590 2024-09-10T07:39:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.465+0000 7f68a8c00640 1 -- 172.21.15.46:0/1396116135 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f68ac011c60 con 0x55e61527b590 2024-09-10T07:39:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.466+0000 7f68a8c00640 1 -- 172.21.15.46:0/1396116135 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f68ac026050 con 0x55e61527b590 2024-09-10T07:39:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.466+0000 7f68c8a25740 10 client.26035 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.466+0000 7f68c8a25740 1 -- 172.21.15.46:0/1396116135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e615294b30 con 0x55e61527b590 2024-09-10T07:39:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.467+0000 7f68a8c00640 1 -- 172.21.15.46:0/1396116135 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b8003d10 con 0x55e61527b590 2024-09-10T07:39:11.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.467+0000 7f68a8c00640 1 -- 172.21.15.46:0/1396116135 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f68ac01b430 con 0x55e61527b590 2024-09-10T07:39:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.468+0000 7f68c8a25740 10 client.26035 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.468+0000 7f68c8a25740 10 client.26035 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:39:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.468+0000 7f68c8a25740 20 client.26035 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.468+0000 7f68c8a25740 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e615295f40 0x55e615298330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.468+0000 7f68c8a25740 4 client.26035 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.468+0000 7f68c8a25740 1 -- 172.21.15.46:0/1396116135 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6151ee950 con 0x55e615295f40 2024-09-10T07:39:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.469+0000 7f68bfe00640 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e615295f40 0x55e615298330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.469+0000 7f68bfe00640 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e615295f40 0x55e615298330 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19b319ae1413a0 server_cookie=515ecb578775e7db in_seq=0 out_seq=0 2024-09-10T07:39:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.469+0000 7f68a8c00640 10 client.26035 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:39:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.470+0000 7f68a8c00640 1 -- 172.21.15.46:0/1396116135 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6151ee950 con 0x55e615295f40 2024-09-10T07:39:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.470+0000 7f68a8c00640 10 client.26035 handle_command_reply: tid=0 2024-09-10T07:39:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.470+0000 7f68c8a25740 1 client.26035 shutdown 2024-09-10T07:39:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.470+0000 7f68c8a25740 1 -- 172.21.15.46:0/1396116135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 msgr2=0x55e615283310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.470+0000 7f68c8a25740 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e615283310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f68c0197820 tx=0x7f68ac013b20 comp rx=0 tx=0).stop 2024-09-10T07:39:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68c8a25740 1 -- 172.21.15.46:0/1396116135 shutdown_connections 2024-09-10T07:39:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68c8a25740 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e615283850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68c8a25740 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e615295f40 0x55e615298330 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68c8a25740 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e615283310 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68c8a25740 1 --2- 172.21.15.46:0/1396116135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 0x55e615282dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68c8a25740 1 -- 172.21.15.46:0/1396116135 >> 172.21.15.46:0/1396116135 conn(0x55e615277200 msgr2=0x55e61527cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68a8c00640 0 client.26035 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:39:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68c8a25740 1 -- 172.21.15.46:0/1396116135 shutdown_connections 2024-09-10T07:39:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.471+0000 7f68c8a25740 1 -- 172.21.15.46:0/1396116135 wait complete. 2024-09-10T07:39:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.472+0000 7f68c8a25740 20 client.26035 trim_cache size 0 max 16384 2024-09-10T07:39:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.495+0000 7f68c8a25740 1 Processor -- start 2024-09-10T07:39:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.495+0000 7f68c8a25740 1 -- start start 2024-09-10T07:39:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.495+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 0x55e61527d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.495+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e61529dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.495+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e61529e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.495+0000 7f68c8a25740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01bdb50 con 0x55e61527ac70 2024-09-10T07:39:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.495+0000 7f68c8a25740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01bf7b0 con 0x55e615285e50 2024-09-10T07:39:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.495+0000 7f68c8a25740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01be220 con 0x55e61527b590 2024-09-10T07:39:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.496+0000 7f68bea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e61529dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.496+0000 7f68bea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e61529dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53592/0 (socket says 172.21.15.46:53592) 2024-09-10T07:39:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.496+0000 7f68bea00640 1 -- 172.21.15.46:0/3691786935 learned_addr learned my addr 172.21.15.46:0/3691786935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.497+0000 7f68bea00640 1 -- 172.21.15.46:0/3691786935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 msgr2=0x55e61529e140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.497+0000 7f68bea00640 1 --2- 172.21.15.46:0/3691786935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e61529e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.497+0000 7f68bea00640 1 -- 172.21.15.46:0/3691786935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 msgr2=0x55e61527d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.497+0000 7f68bea00640 1 --2- 172.21.15.46:0/3691786935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 0x55e61527d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.497+0000 7f68bea00640 1 -- 172.21.15.46:0/3691786935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61527b3c0 con 0x55e61527b590 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.497+0000 7f68bea00640 1 --2- 172.21.15.46:0/3691786935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e61529dc00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68ac01d030 tx=0x7f68ac0097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44369093a3c291b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.497+0000 7f68ab400640 1 -- 172.21.15.46:0/3691786935 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac011e60 con 0x55e61527b590 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.498+0000 7f68ab400640 1 -- 172.21.15.46:0/3691786935 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac02f070 con 0x55e61527b590 2024-09-10T07:39:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.498+0000 7f68ab400640 1 -- 172.21.15.46:0/3691786935 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac011e60 con 0x55e61527b590 2024-09-10T07:39:11.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.498+0000 7f68c8a25740 1 -- 172.21.15.46:0/3691786935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 msgr2=0x55e61529dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.498+0000 7f68c8a25740 1 --2- 172.21.15.46:0/3691786935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e61529dc00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68ac01d030 tx=0x7f68ac0097f0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.499+0000 7f68c8a25740 1 -- 172.21.15.46:0/3691786935 shutdown_connections 2024-09-10T07:39:11.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.499+0000 7f68c8a25740 1 --2- 172.21.15.46:0/3691786935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e615285e50 0x55e61529e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.499+0000 7f68c8a25740 1 --2- 172.21.15.46:0/3691786935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527b590 0x55e61529dc00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.499+0000 7f68c8a25740 1 --2- 172.21.15.46:0/3691786935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61527ac70 0x55e61527d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.499+0000 7f68c8a25740 1 -- 172.21.15.46:0/3691786935 >> 172.21.15.46:0/3691786935 conn(0x55e615277200 msgr2=0x55e615284e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:11.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.499+0000 7f68c8a25740 1 -- 172.21.15.46:0/3691786935 shutdown_connections 2024-09-10T07:39:11.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.499+0000 7f68c8a25740 1 -- 172.21.15.46:0/3691786935 wait complete. 2024-09-10T07:39:11.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.500+0000 7f68c8a25740 1 -- start start 2024-09-10T07:39:11.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.500+0000 7f68c8a25740 1 Processor -- start 2024-09-10T07:39:11.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527ac70 0x55e6152fc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527b590 0x55e61528de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68c8a25740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e615285e50 0x55e61528e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68c8a25740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01bf020 con 0x55e615285e50 2024-09-10T07:39:11.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68c8a25740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01be950 con 0x55e61527b590 2024-09-10T07:39:11.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68c8a25740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01bbb30 con 0x55e61527ac70 2024-09-10T07:39:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68c8a25740 10 client.0 _ll_register_callbacks cb 0x7f68ac009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68bea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527b590 0x55e61528de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68bfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527ac70 0x55e6152fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68bea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527b590 0x55e61528de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43842/0 (socket says 172.21.15.46:43842) 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.501+0000 7f68bea00640 1 -- 172.21.15.46:0/872746021 learned_addr learned my addr 172.21.15.46:0/872746021 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.502+0000 7f68bea00640 1 -- 172.21.15.46:0/872746021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527ac70 msgr2=0x55e6152fc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.502+0000 7f68bea00640 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527ac70 0x55e6152fc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.502+0000 7f68bea00640 1 -- 172.21.15.46:0/872746021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e615285e50 msgr2=0x55e61528e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.502+0000 7f68bea00640 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e615285e50 0x55e61528e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.502+0000 7f68bea00640 1 -- 172.21.15.46:0/872746021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6152f96e0 con 0x55e61527b590 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.502+0000 7f68bea00640 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527b590 0x55e61528de20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68ac01b970 tx=0x7f68ac013b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce30da8c99f8761a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.502+0000 7f68a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.502+0000 7f68a8c00640 1 -- 172.21.15.46:0/872746021 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac038070 con 0x55e61527b590 2024-09-10T07:39:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.503+0000 7f68a8c00640 1 -- 172.21.15.46:0/872746021 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f68ac005450 con 0x55e61527b590 2024-09-10T07:39:11.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.503+0000 7f68c8a25740 1 -- 172.21.15.46:0/872746021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ac02f070 con 0x55e61527b590 2024-09-10T07:39:11.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.503+0000 7f68a8c00640 1 -- 172.21.15.46:0/872746021 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f68ac01ba40 con 0x55e61527b590 2024-09-10T07:39:11.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.503+0000 7f68c8a25740 10 client.26048 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:11.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.503+0000 7f68c8a25740 1 -- 172.21.15.46:0/872746021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e615311cd0 con 0x55e61527b590 2024-09-10T07:39:11.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.504+0000 7f68a8c00640 1 -- 172.21.15.46:0/872746021 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac02f070 con 0x55e61527b590 2024-09-10T07:39:11.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.504+0000 7f68a8c00640 1 -- 172.21.15.46:0/872746021 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f68ac02a430 con 0x55e61527b590 2024-09-10T07:39:11.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.504+0000 7f68c8a25740 10 client.26048 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:11.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.504+0000 7f68c8a25740 10 client.26048 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:39:11.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.505+0000 7f68c8a25740 20 client.26048 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:11.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.505+0000 7f68c8a25740 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e615312390 0x55e6153147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:11.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.505+0000 7f68c8a25740 4 client.26048 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:39:11.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.505+0000 7f68c8a25740 1 -- 172.21.15.46:0/872746021 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6151e8e60 con 0x55e615312390 2024-09-10T07:39:11.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.505+0000 7f68bfe00640 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e615312390 0x55e6153147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:11.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.507+0000 7f68bfe00640 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e615312390 0x55e6153147b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8139b0d9e02ebb77 server_cookie=d25ab287b7969b93 in_seq=0 out_seq=0 2024-09-10T07:39:11.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.507+0000 7f68a8c00640 10 client.26048 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:39:11.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.508+0000 7f68a8c00640 1 -- 172.21.15.46:0/872746021 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6151e8e60 con 0x55e615312390 2024-09-10T07:39:11.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.508+0000 7f68a8c00640 10 client.26048 handle_command_reply: tid=0 2024-09-10T07:39:11.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.508+0000 7f68c8a25740 1 client.26048 shutdown 2024-09-10T07:39:11.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.508+0000 7f68c8a25740 1 -- 172.21.15.46:0/872746021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527b590 msgr2=0x55e61528de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.508+0000 7f68c8a25740 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527b590 0x55e61528de20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68ac01b970 tx=0x7f68ac013b20 comp rx=0 tx=0).stop 2024-09-10T07:39:11.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.509+0000 7f68c8a25740 1 -- 172.21.15.46:0/872746021 shutdown_connections 2024-09-10T07:39:11.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.509+0000 7f68c8a25740 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e615285e50 0x55e61528e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.509+0000 7f68c8a25740 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61527b590 0x55e61528de20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.509+0000 7f68c8a25740 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e615312390 0x55e6153147b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.509+0000 7f68c8a25740 1 --2- 172.21.15.46:0/872746021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e61527ac70 0x55e6152fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.509+0000 7f68c8a25740 1 -- 172.21.15.46:0/872746021 >> 172.21.15.46:0/872746021 conn(0x55e615277200 msgr2=0x55e615275b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:11.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.509+0000 7f68a8c00640 0 client.26048 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:39:11.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.510+0000 7f68c8a25740 1 -- 172.21.15.46:0/872746021 shutdown_connections 2024-09-10T07:39:11.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.510+0000 7f68c8a25740 1 -- 172.21.15.46:0/872746021 wait complete. 2024-09-10T07:39:11.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.510+0000 7f68c8a25740 20 client.26048 trim_cache size 0 max 16384 2024-09-10T07:39:11.511 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:39:11.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 -- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68a40a2d50 msgr2=0x7f68a40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68a40a2d50 0x7f68a40a5210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f68b00c37b0 tx=0x7f68b003cd30 comp rx=0 tx=0).stop 2024-09-10T07:39:11.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 -- 172.21.15.46:0/1202964684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c0100a80 msgr2=0x7f68c019cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:11.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c0100a80 0x7f68c019cfd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68b002c990 tx=0x7f68b00346d0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 -- 172.21.15.46:0/1202964684 shutdown_connections 2024-09-10T07:39:11.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68a40a2d50 0x7f68a40a5210 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c0106a80 0x7f68c0196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c0101420 0x7f68c019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 --2- 172.21.15.46:0/1202964684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68c0100a80 0x7f68c019cfd0 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68b002c990 tx=0x7f68b00346d0 comp rx=0 tx=0).stop 2024-09-10T07:39:11.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 -- 172.21.15.46:0/1202964684 >> 172.21.15.46:0/1202964684 conn(0x7f68c00fc7a0 msgr2=0x7f68c00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:11.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 -- 172.21.15.46:0/1202964684 shutdown_connections 2024-09-10T07:39:11.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:11.511+0000 7f68c6000640 1 -- 172.21.15.46:0/1202964684 wait complete. 2024-09-10T07:39:12.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:39:12.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:12 smithi181 ceph-mon[35397]: pgmap v476: 129 pgs: 129 active+clean; 980 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 697 op/s 2024-09-10T07:39:12.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:12 smithi063 ceph-mon[35020]: pgmap v476: 129 pgs: 129 active+clean; 980 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 697 op/s 2024-09-10T07:39:12.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:12 smithi046 ceph-mon[30705]: pgmap v476: 129 pgs: 129 active+clean; 980 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 697 op/s 2024-09-10T07:39:13.228 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:14.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.100+0000 7fc30f000640 1 -- 172.21.15.46:0/1494120185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30810d530 msgr2=0x7fc3081150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.100+0000 7fc30f000640 1 --2- 172.21.15.46:0/1494120185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30810d530 0x7fc3081150b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc300009ca0 tx=0x7fc30002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.101+0000 7fc30f000640 1 -- 172.21.15.46:0/1494120185 shutdown_connections 2024-09-10T07:39:14.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.101+0000 7fc30f000640 1 --2- 172.21.15.46:0/1494120185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30810d530 0x7fc3081150b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.101+0000 7fc30f000640 1 --2- 172.21.15.46:0/1494120185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc308108e00 0x7fc30810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.101+0000 7fc30f000640 1 --2- 172.21.15.46:0/1494120185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc308108450 0x7fc308108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.101+0000 7fc30f000640 1 -- 172.21.15.46:0/1494120185 >> 172.21.15.46:0/1494120185 conn(0x7fc308077ba0 msgr2=0x7fc308079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:14.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.102+0000 7fc30f000640 1 -- 172.21.15.46:0/1494120185 shutdown_connections 2024-09-10T07:39:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.102+0000 7fc30f000640 1 -- 172.21.15.46:0/1494120185 wait complete. 2024-09-10T07:39:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.103+0000 7fc30f000640 1 Processor -- start 2024-09-10T07:39:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.104+0000 7fc30f000640 1 -- start start 2024-09-10T07:39:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.105+0000 7fc30f000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc308108450 0x7fc308073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.105+0000 7fc30f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc308108e00 0x7fc30806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.105+0000 7fc30f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30810d530 0x7fc30806e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.106+0000 7fc307400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc308108e00 0x7fc30806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.106+0000 7fc30f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc30806c660 con 0x7fc30810d530 2024-09-10T07:39:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.106+0000 7fc30f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc30806c4e0 con 0x7fc308108e00 2024-09-10T07:39:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.106+0000 7fc307e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc308108450 0x7fc308073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.106+0000 7fc307400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc308108e00 0x7fc30806dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43856/0 (socket says 172.21.15.46:43856) 2024-09-10T07:39:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.106+0000 7fc307400640 1 -- 172.21.15.46:0/2687351845 learned_addr learned my addr 172.21.15.46:0/2687351845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.106+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc30806c7e0 con 0x7fc308108450 2024-09-10T07:39:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.107+0000 7fc307e00640 1 -- 172.21.15.46:0/2687351845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc308108e00 msgr2=0x7fc30806dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.107+0000 7fc307e00640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc308108e00 0x7fc30806dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.107+0000 7fc30d000640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30810d530 0x7fc30806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.108+0000 7fc307e00640 1 -- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30810d530 msgr2=0x7fc30806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.108+0000 7fc307e00640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30810d530 0x7fc30806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.108+0000 7fc307e00640 1 -- 172.21.15.46:0/2687351845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30806e800 con 0x7fc308108450 2024-09-10T07:39:14.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.109+0000 7fc307400640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc308108e00 0x7fc30806dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:39:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.109+0000 7fc30d000640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30810d530 0x7fc30806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:14.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.113+0000 7fc307e00640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc308108450 0x7fc308073b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800e9f0 tx=0x7fc2f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:14.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.114+0000 7fc304c00640 1 -- 172.21.15.46:0/2687351845 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800ccc0 con 0x7fc308108450 2024-09-10T07:39:14.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.114+0000 7fc304c00640 1 -- 172.21.15.46:0/2687351845 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f8010910 con 0x7fc308108450 2024-09-10T07:39:14.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.114+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc308080a70 con 0x7fc308108450 2024-09-10T07:39:14.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.114+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc308081000 con 0x7fc308108450 2024-09-10T07:39:14.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.114+0000 7fc304c00640 1 -- 172.21.15.46:0/2687351845 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8018960 con 0x7fc308108450 2024-09-10T07:39:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.126+0000 7fc304c00640 1 -- 172.21.15.46:0/2687351845 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc2f8010ac0 con 0x7fc308108450 2024-09-10T07:39:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.128+0000 7fc304c00640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2ec0a2c00 0x7fc2ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.130+0000 7fc304c00640 1 -- 172.21.15.46:0/2687351845 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc2f80cf740 con 0x7fc308108450 2024-09-10T07:39:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.131+0000 7fc307400640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2ec0a2c00 0x7fc2ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.132+0000 7fc311ac4740 1 Processor -- start 2024-09-10T07:39:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.132+0000 7fc307400640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2ec0a2c00 0x7fc2ec0a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005e00 tx=0x7fc2f4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.132+0000 7fc311ac4740 1 -- start start 2024-09-10T07:39:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.133+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb27c70 0x562cbfb28050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.133+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb32910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.133+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb32e50 0x562cbfb35240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.133+0000 7fc311ac4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562cbfa8e3e0 con 0x562cbfb32e50 2024-09-10T07:39:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.133+0000 7fc311ac4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562cbfa35d80 con 0x562cbfb27c70 2024-09-10T07:39:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.133+0000 7fc311ac4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562cbfa8a3a0 con 0x562cbfb28590 2024-09-10T07:39:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb27c70 0x562cbfb28050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb27c70 0x562cbfb28050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43864/0 (socket says 172.21.15.46:43864) 2024-09-10T07:39:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307e00640 1 -- 172.21.15.46:0/459967767 learned_addr learned my addr 172.21.15.46:0/459967767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307400640 1 --2- 172.21.15.46:0/459967767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb32910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307e00640 1 -- 172.21.15.46:0/459967767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 msgr2=0x562cbfb32910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307e00640 1 --2- 172.21.15.46:0/459967767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb32910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307e00640 1 -- 172.21.15.46:0/459967767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb32e50 msgr2=0x562cbfb35240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307e00640 1 --2- 172.21.15.46:0/459967767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb32e50 0x562cbfb35240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.134+0000 7fc307e00640 1 -- 172.21.15.46:0/459967767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbfa54ae0 con 0x562cbfb27c70 2024-09-10T07:39:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.137+0000 7fc307e00640 1 --2- 172.21.15.46:0/459967767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb27c70 0x562cbfb28050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc2f809b920 tx=0x7fc2f809c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2837d45633ab9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.137+0000 7fc2fd600640 1 -- 172.21.15.46:0/459967767 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80a4890 con 0x562cbfb27c70 2024-09-10T07:39:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.137+0000 7fc2fd600640 1 -- 172.21.15.46:0/459967767 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f8099f20 con 0x562cbfb27c70 2024-09-10T07:39:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.138+0000 7fc311ac4740 1 -- 172.21.15.46:0/459967767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb27c70 msgr2=0x562cbfb28050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.138+0000 7fc311ac4740 1 --2- 172.21.15.46:0/459967767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb27c70 0x562cbfb28050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc2f809b920 tx=0x7fc2f809c320 comp rx=0 tx=0).stop 2024-09-10T07:39:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.139+0000 7fc311ac4740 1 -- 172.21.15.46:0/459967767 shutdown_connections 2024-09-10T07:39:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.139+0000 7fc311ac4740 1 --2- 172.21.15.46:0/459967767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb32e50 0x562cbfb35240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.139+0000 7fc311ac4740 1 --2- 172.21.15.46:0/459967767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb32910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.139+0000 7fc311ac4740 1 --2- 172.21.15.46:0/459967767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb27c70 0x562cbfb28050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.139+0000 7fc311ac4740 1 -- 172.21.15.46:0/459967767 >> 172.21.15.46:0/459967767 conn(0x562cbfb24200 msgr2=0x562cbfb245e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.139+0000 7fc311ac4740 1 -- 172.21.15.46:0/459967767 shutdown_connections 2024-09-10T07:39:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.139+0000 7fc311ac4740 1 -- 172.21.15.46:0/459967767 wait complete. 2024-09-10T07:39:14.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.141+0000 7fc311ac4740 1 -- start start 2024-09-10T07:39:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.142+0000 7fc311ac4740 1 Processor -- start 2024-09-10T07:39:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.142+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb30150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb32e50 0x562cbfb30690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc311ac4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562cbfb35ac0 con 0x562cbfb27c70 2024-09-10T07:39:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc311ac4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562cbfa490f0 con 0x562cbfb28590 2024-09-10T07:39:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc311ac4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562cbfb35c40 con 0x562cbfb32e50 2024-09-10T07:39:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc311ac4740 10 client.0 _ll_register_callbacks cb 0x562cbfa30bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc307400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb30150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc30d000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb32e50 0x562cbfb30690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc307400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb30150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43874/0 (socket says 172.21.15.46:43874) 2024-09-10T07:39:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.143+0000 7fc307400640 1 -- 172.21.15.46:0/2683781275 learned_addr learned my addr 172.21.15.46:0/2683781275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc30d000640 1 -- 172.21.15.46:0/2683781275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 msgr2=0x562cbfb30150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc30d000640 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb30150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc30d000640 1 -- 172.21.15.46:0/2683781275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 msgr2=0x562cbfb2fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc30d000640 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc30d000640 1 -- 172.21.15.46:0/2683781275 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbf7ff3c0 con 0x562cbfb32e50 2024-09-10T07:39:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc30d000640 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb32e50 0x562cbfb30690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc300004af0 tx=0x7fc30002f3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580375b3363d899 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc2f2000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc2f2000640 1 -- 172.21.15.46:0/2683781275 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300003b70 con 0x562cbfb32e50 2024-09-10T07:39:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.144+0000 7fc2f2000640 1 -- 172.21.15.46:0/2683781275 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc300003d10 con 0x562cbfb32e50 2024-09-10T07:39:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.145+0000 7fc311ac4740 1 -- 172.21.15.46:0/2683781275 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cbfa9b950 con 0x562cbfb32e50 2024-09-10T07:39:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.145+0000 7fc2f2000640 1 -- 172.21.15.46:0/2683781275 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc300031450 con 0x562cbfb32e50 2024-09-10T07:39:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.145+0000 7fc311ac4740 10 client.26053 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.145+0000 7fc311ac4740 1 -- 172.21.15.46:0/2683781275 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562cbfb41b50 con 0x562cbfb32e50 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc2f2000640 1 -- 172.21.15.46:0/2683781275 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562cbfa9b950 con 0x562cbfb32e50 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc2f2000640 1 -- 172.21.15.46:0/2683781275 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fc30002f610 con 0x562cbfb32e50 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc311ac4740 10 client.26053 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc311ac4740 10 client.26053 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc311ac4740 20 client.26053 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc311ac4740 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x562cbfb430b0 0x562cbfb454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc311ac4740 4 client.26053 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc311ac4740 1 -- 172.21.15.46:0/2683781275 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cbfa95e60 con 0x562cbfb430b0 2024-09-10T07:39:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.146+0000 7fc307e00640 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x562cbfb430b0 0x562cbfb454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.147+0000 7fc307e00640 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x562cbfb430b0 0x562cbfb454e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e23ccd8382c16cb6 server_cookie=8cb9fc5dd9e96911 in_seq=0 out_seq=0 2024-09-10T07:39:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.147+0000 7fc2f2000640 10 client.26053 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:39:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.148+0000 7fc2f2000640 1 -- 172.21.15.46:0/2683781275 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cbfa95e60 con 0x562cbfb430b0 2024-09-10T07:39:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.148+0000 7fc2f2000640 10 client.26053 handle_command_reply: tid=0 2024-09-10T07:39:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.148+0000 7fc311ac4740 1 client.26053 shutdown 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 -- 172.21.15.46:0/2683781275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb32e50 msgr2=0x562cbfb30690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb32e50 0x562cbfb30690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc300004af0 tx=0x7fc30002f3e0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 -- 172.21.15.46:0/2683781275 shutdown_connections 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x562cbfb430b0 0x562cbfb454e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb32e50 0x562cbfb30690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb30150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 --2- 172.21.15.46:0/2683781275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 -- 172.21.15.46:0/2683781275 >> 172.21.15.46:0/2683781275 conn(0x562cbfb24200 msgr2=0x562cbfb29f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc2f2000640 0 client.26053 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:39:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.149+0000 7fc311ac4740 1 -- 172.21.15.46:0/2683781275 shutdown_connections 2024-09-10T07:39:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.150+0000 7fc311ac4740 1 -- 172.21.15.46:0/2683781275 wait complete. 2024-09-10T07:39:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.151+0000 7fc311ac4740 20 client.26053 trim_cache size 0 max 16384 2024-09-10T07:39:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.184+0000 7fc311ac4740 1 Processor -- start 2024-09-10T07:39:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.184+0000 7fc311ac4740 1 -- start start 2024-09-10T07:39:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb27c70 0x562cbfb4adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb4b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb32e50 0x562cbfb22b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc311ac4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e800fc30 con 0x562cbfb32e50 2024-09-10T07:39:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc311ac4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8011890 con 0x562cbfb28590 2024-09-10T07:39:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc311ac4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8010300 con 0x562cbfb27c70 2024-09-10T07:39:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc307400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb4b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc307e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb27c70 0x562cbfb4adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc307400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb4b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43886/0 (socket says 172.21.15.46:43886) 2024-09-10T07:39:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.185+0000 7fc307400640 1 -- 172.21.15.46:0/3368346292 learned_addr learned my addr 172.21.15.46:0/3368346292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.186+0000 7fc307e00640 1 -- 172.21.15.46:0/3368346292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 msgr2=0x562cbfb4b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.186+0000 7fc307e00640 1 --2- 172.21.15.46:0/3368346292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb4b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.186+0000 7fc307e00640 1 -- 172.21.15.46:0/3368346292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb32e50 msgr2=0x562cbfb22b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.186+0000 7fc307e00640 1 --2- 172.21.15.46:0/3368346292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb32e50 0x562cbfb22b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.186+0000 7fc307e00640 1 -- 172.21.15.46:0/3368346292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbfb283c0 con 0x562cbfb27c70 2024-09-10T07:39:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.186+0000 7fc307400640 1 --2- 172.21.15.46:0/3368346292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb4b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:39:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.189+0000 7fc307e00640 1 --2- 172.21.15.46:0/3368346292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb27c70 0x562cbfb4adf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc2f809bde0 tx=0x7fc2f8009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4546a786ae278e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.189+0000 7fc2fcc00640 1 -- 172.21.15.46:0/3368346292 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8098290 con 0x562cbfb27c70 2024-09-10T07:39:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.189+0000 7fc2fcc00640 1 -- 172.21.15.46:0/3368346292 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f809b070 con 0x562cbfb27c70 2024-09-10T07:39:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.189+0000 7fc2fcc00640 1 -- 172.21.15.46:0/3368346292 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8097c60 con 0x562cbfb27c70 2024-09-10T07:39:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.190+0000 7fc311ac4740 1 -- 172.21.15.46:0/3368346292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb27c70 msgr2=0x562cbfb4adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.190+0000 7fc311ac4740 1 --2- 172.21.15.46:0/3368346292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb27c70 0x562cbfb4adf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc2f809bde0 tx=0x7fc2f8009700 comp rx=0 tx=0).stop 2024-09-10T07:39:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.190+0000 7fc311ac4740 1 -- 172.21.15.46:0/3368346292 shutdown_connections 2024-09-10T07:39:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.190+0000 7fc311ac4740 1 --2- 172.21.15.46:0/3368346292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb32e50 0x562cbfb22b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.190+0000 7fc311ac4740 1 --2- 172.21.15.46:0/3368346292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb28590 0x562cbfb4b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.190+0000 7fc311ac4740 1 --2- 172.21.15.46:0/3368346292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb27c70 0x562cbfb4adf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.190+0000 7fc311ac4740 1 -- 172.21.15.46:0/3368346292 >> 172.21.15.46:0/3368346292 conn(0x562cbfb24200 msgr2=0x562cbfb2ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.191+0000 7fc311ac4740 1 -- 172.21.15.46:0/3368346292 shutdown_connections 2024-09-10T07:39:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.191+0000 7fc311ac4740 1 -- 172.21.15.46:0/3368346292 wait complete. 2024-09-10T07:39:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.192+0000 7fc311ac4740 1 -- start start 2024-09-10T07:39:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.192+0000 7fc311ac4740 1 Processor -- start 2024-09-10T07:39:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.193+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.193+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb2c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.193+0000 7fc311ac4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb32e50 0x562cbfb2c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.193+0000 7fc311ac4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8011100 con 0x562cbfb27c70 2024-09-10T07:39:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.193+0000 7fc311ac4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8010a30 con 0x562cbfb32e50 2024-09-10T07:39:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.193+0000 7fc311ac4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e800dc10 con 0x562cbfb28590 2024-09-10T07:39:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.194+0000 7fc307400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb2c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.194+0000 7fc307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.194+0000 7fc311ac4740 10 client.0 _ll_register_callbacks cb 0x7fc300009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.194+0000 7fc307400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb2c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53644/0 (socket says 172.21.15.46:53644) 2024-09-10T07:39:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.194+0000 7fc307400640 1 -- 172.21.15.46:0/1225846806 learned_addr learned my addr 172.21.15.46:0/1225846806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc30d000640 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb32e50 0x562cbfb2c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.194+0000 7fc307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36494/0 (socket says 172.21.15.46:36494) 2024-09-10T07:39:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc307400640 1 -- 172.21.15.46:0/1225846806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb32e50 msgr2=0x562cbfb2c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc307400640 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb32e50 0x562cbfb2c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc307400640 1 -- 172.21.15.46:0/1225846806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 msgr2=0x562cbfb2bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc307400640 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc307400640 1 -- 172.21.15.46:0/1225846806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbfb3a070 con 0x562cbfb28590 2024-09-10T07:39:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc307e00640 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc30d000640 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb32e50 0x562cbfb2c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:39:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc307400640 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb2c460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x562cbfb28390 tx=0x7fc2f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a524f9a5a88c777 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.195+0000 7fc2f2000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.196+0000 7fc2f2000640 1 -- 172.21.15.46:0/1225846806 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4011440 con 0x562cbfb28590 2024-09-10T07:39:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.196+0000 7fc2f2000640 1 -- 172.21.15.46:0/1225846806 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc2f40115e0 con 0x562cbfb28590 2024-09-10T07:39:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.196+0000 7fc311ac4740 1 -- 172.21.15.46:0/1225846806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f809b070 con 0x562cbfb28590 2024-09-10T07:39:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.196+0000 7fc2f2000640 1 -- 172.21.15.46:0/1225846806 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc2f40117e0 con 0x562cbfb28590 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.196+0000 7fc311ac4740 10 client.26059 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.196+0000 7fc311ac4740 1 -- 172.21.15.46:0/1225846806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562cbfb3b400 con 0x562cbfb28590 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.196+0000 7fc2f2000640 1 -- 172.21.15.46:0/1225846806 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f809b070 con 0x562cbfb28590 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.197+0000 7fc2f2000640 1 -- 172.21.15.46:0/1225846806 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fc2f402bb00 con 0x562cbfb28590 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.197+0000 7fc311ac4740 10 client.26059 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.197+0000 7fc311ac4740 10 client.26059 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.197+0000 7fc311ac4740 20 client.26059 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.197+0000 7fc311ac4740 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x562cbfbaddc0 0x562cbfbb01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.197+0000 7fc311ac4740 4 client.26059 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.197+0000 7fc311ac4740 1 -- 172.21.15.46:0/1225846806 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cbfa12010 con 0x562cbfbaddc0 2024-09-10T07:39:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.198+0000 7fc307e00640 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x562cbfbaddc0 0x562cbfbb01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.200+0000 7fc307e00640 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x562cbfbaddc0 0x562cbfbb01b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=585f9ee879f649f4 server_cookie=23405ddda0a421fc in_seq=0 out_seq=0 2024-09-10T07:39:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.200+0000 7fc2f2000640 10 client.26059 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:39:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.200+0000 7fc2f2000640 1 -- 172.21.15.46:0/1225846806 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cbfa12010 con 0x562cbfbaddc0 2024-09-10T07:39:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.200+0000 7fc2f2000640 10 client.26059 handle_command_reply: tid=0 2024-09-10T07:39:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.201+0000 7fc311ac4740 1 client.26059 shutdown 2024-09-10T07:39:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.201+0000 7fc311ac4740 1 -- 172.21.15.46:0/1225846806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 msgr2=0x562cbfb2c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.201+0000 7fc311ac4740 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb2c460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x562cbfb28390 tx=0x7fc2f40046c0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.201+0000 7fc311ac4740 1 -- 172.21.15.46:0/1225846806 shutdown_connections 2024-09-10T07:39:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.201+0000 7fc311ac4740 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x562cbfbaddc0 0x562cbfbb01b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.201+0000 7fc311ac4740 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cbfb32e50 0x562cbfb2c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.201+0000 7fc311ac4740 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562cbfb28590 0x562cbfb2c460 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.201+0000 7fc311ac4740 1 --2- 172.21.15.46:0/1225846806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cbfb27c70 0x562cbfb2bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.202+0000 7fc311ac4740 1 -- 172.21.15.46:0/1225846806 >> 172.21.15.46:0/1225846806 conn(0x562cbfb24200 msgr2=0x562cbfb22b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.202+0000 7fc2f2000640 0 client.26059 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:39:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.203+0000 7fc311ac4740 1 -- 172.21.15.46:0/1225846806 shutdown_connections 2024-09-10T07:39:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.203+0000 7fc311ac4740 1 -- 172.21.15.46:0/1225846806 wait complete. 2024-09-10T07:39:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.203+0000 7fc311ac4740 20 client.26059 trim_cache size 0 max 16384 2024-09-10T07:39:14.204 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:39:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2ec0a2c00 msgr2=0x7fc2ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2ec0a2c00 0x7fc2ec0a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005e00 tx=0x7fc2f4005d30 comp rx=0 tx=0).stop 2024-09-10T07:39:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc308108450 msgr2=0x7fc308073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc308108450 0x7fc308073b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800e9f0 tx=0x7fc2f800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 shutdown_connections 2024-09-10T07:39:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2ec0a2c00 0x7fc2ec0a50c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30810d530 0x7fc30806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc308108e00 0x7fc30806dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 --2- 172.21.15.46:0/2687351845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc308108450 0x7fc308073b40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.205+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 >> 172.21.15.46:0/2687351845 conn(0x7fc308077ba0 msgr2=0x7fc308079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.206+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 shutdown_connections 2024-09-10T07:39:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:14.206+0000 7fc30f000640 1 -- 172.21.15.46:0/2687351845 wait complete. 2024-09-10T07:39:14.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:14 smithi046 ceph-mon[30705]: pgmap v477: 129 pgs: 129 active+clean; 981 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.8 MiB/s wr, 653 op/s 2024-09-10T07:39:14.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:14 smithi181 ceph-mon[35397]: pgmap v477: 129 pgs: 129 active+clean; 981 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.8 MiB/s wr, 653 op/s 2024-09-10T07:39:14.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:14 smithi063 ceph-mon[35020]: pgmap v477: 129 pgs: 129 active+clean; 981 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.8 MiB/s wr, 653 op/s 2024-09-10T07:39:15.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:39:15.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 115.0 s, 1408.4 tps, lat 70.599 ms stddev 84.511 2024-09-10T07:39:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:16.775 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:16 smithi046 ceph-mon[30705]: pgmap v478: 129 pgs: 129 active+clean; 982 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 668 op/s 2024-09-10T07:39:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.831+0000 7f3406200640 1 -- 172.21.15.46:0/2568996332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400112a50 msgr2=0x7f3400114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.831+0000 7f3406200640 1 --2- 172.21.15.46:0/2568996332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400112a50 0x7f3400114e40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009ca0 tx=0x7f33f802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.831+0000 7f3406200640 1 -- 172.21.15.46:0/2568996332 shutdown_connections 2024-09-10T07:39:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.831+0000 7f3406200640 1 --2- 172.21.15.46:0/2568996332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400112a50 0x7f3400114e40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.831+0000 7f3406200640 1 --2- 172.21.15.46:0/2568996332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400077040 0x7f3400112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.831+0000 7f3406200640 1 --2- 172.21.15.46:0/2568996332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3400076720 0x7f3400076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.831+0000 7f3406200640 1 -- 172.21.15.46:0/2568996332 >> 172.21.15.46:0/2568996332 conn(0x7f34000ffa50 msgr2=0x7f3400101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:16.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.831+0000 7f3406200640 1 -- 172.21.15.46:0/2568996332 shutdown_connections 2024-09-10T07:39:16.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.833+0000 7f3406200640 1 -- 172.21.15.46:0/2568996332 wait complete. 2024-09-10T07:39:16.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.835+0000 7f3406200640 1 Processor -- start 2024-09-10T07:39:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.836+0000 7f3406200640 1 -- start start 2024-09-10T07:39:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.836+0000 7f3406200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400076720 0x7f34001a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.836+0000 7f3406200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3400077040 0x7f34001a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.836+0000 7f3406200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400112a50 0x7f340019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.836+0000 7f3406200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3400118590 con 0x7f3400112a50 2024-09-10T07:39:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.836+0000 7f3406200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3400118410 con 0x7f3400076720 2024-09-10T07:39:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.836+0000 7f3406200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3400118710 con 0x7f3400077040 2024-09-10T07:39:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.837+0000 7f33ff400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400076720 0x7f34001a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.837+0000 7f33ff400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400076720 0x7f34001a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38020/0 (socket says 172.21.15.46:38020) 2024-09-10T07:39:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.837+0000 7f33ff400640 1 -- 172.21.15.46:0/2412341801 learned_addr learned my addr 172.21.15.46:0/2412341801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.837+0000 7f33fea00640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3400077040 0x7f34001a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.838+0000 7f33ff400640 1 -- 172.21.15.46:0/2412341801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3400077040 msgr2=0x7f34001a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.838+0000 7f33ff400640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3400077040 0x7f34001a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.838+0000 7f33ff400640 1 -- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400112a50 msgr2=0x7f340019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.838+0000 7f33ff400640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400112a50 0x7f340019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.838+0000 7f33ff400640 1 -- 172.21.15.46:0/2412341801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340019efc0 con 0x7f3400076720 2024-09-10T07:39:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.840+0000 7f33ff400640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400076720 0x7f34001a46d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f33f000d950 tx=0x7f33f000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.840+0000 7f33f7e00640 1 -- 172.21.15.46:0/2412341801 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0017070 con 0x7f3400076720 2024-09-10T07:39:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.840+0000 7f33f7e00640 1 -- 172.21.15.46:0/2412341801 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f0009c30 con 0x7f3400076720 2024-09-10T07:39:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.840+0000 7f33f7e00640 1 -- 172.21.15.46:0/2412341801 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f000f600 con 0x7f3400076720 2024-09-10T07:39:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.840+0000 7f3406200640 1 -- 172.21.15.46:0/2412341801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340019f2b0 con 0x7f3400076720 2024-09-10T07:39:16.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.840+0000 7f3406200640 1 -- 172.21.15.46:0/2412341801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340006bc10 con 0x7f3400076720 2024-09-10T07:39:16.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.845+0000 7f3408bef740 1 Processor -- start 2024-09-10T07:39:16.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.845+0000 7f3408bef740 1 -- start start 2024-09-10T07:39:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.845+0000 7f3408bef740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca691e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f3408bef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691e590 0x55dca6928910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f3408bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 0x55dca692b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f3408bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dca68843e0 con 0x55dca6928e50 2024-09-10T07:39:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f3408bef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dca682bd80 con 0x55dca691e590 2024-09-10T07:39:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f3408bef740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dca68803a0 con 0x55dca691dc70 2024-09-10T07:39:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca691e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca691e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40550/0 (socket says 172.21.15.46:40550) 2024-09-10T07:39:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f33ffe00640 1 -- 172.21.15.46:0/3033925009 learned_addr learned my addr 172.21.15.46:0/3033925009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.846+0000 7f33f7e00640 1 -- 172.21.15.46:0/2412341801 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f33f0010460 con 0x7f3400076720 2024-09-10T07:39:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.847+0000 7f33fea00640 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 0x55dca692b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.847+0000 7f33ffe00640 1 -- 172.21.15.46:0/3033925009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691e590 msgr2=0x55dca6928910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:39:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.847+0000 7f33ffe00640 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691e590 0x55dca6928910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.847+0000 7f33ffe00640 1 -- 172.21.15.46:0/3033925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 msgr2=0x55dca692b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.847+0000 7f33ffe00640 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 0x55dca692b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.847+0000 7f33ffe00640 1 -- 172.21.15.46:0/3033925009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca684aae0 con 0x55dca691dc70 2024-09-10T07:39:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.847+0000 7f33fea00640 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 0x55dca692b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.848+0000 7f33ffe00640 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca691e050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33f802ae30 tx=0x7f33f8009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec67330724f519a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.848+0000 7f33f7e00640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33e40a2cc0 0x7f33e40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.848+0000 7f33f7e00640 1 -- 172.21.15.46:0/2412341801 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f33f00cf0a0 con 0x7f3400076720 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.849+0000 7f33ebe00640 1 -- 172.21.15.46:0/3033925009 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8031db0 con 0x55dca691dc70 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.849+0000 7f33ebe00640 1 -- 172.21.15.46:0/3033925009 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f803f070 con 0x55dca691dc70 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.849+0000 7f33ebe00640 1 -- 172.21.15.46:0/3033925009 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f80042a0 con 0x55dca691dc70 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.849+0000 7f33ff400640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33e40a2cc0 0x7f33e40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.849+0000 7f3408bef740 1 -- 172.21.15.46:0/3033925009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 msgr2=0x55dca691e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.849+0000 7f3408bef740 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca691e050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33f802ae30 tx=0x7f33f8009800 comp rx=0 tx=0).stop 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.850+0000 7f3408bef740 1 -- 172.21.15.46:0/3033925009 shutdown_connections 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.850+0000 7f33ff400640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33e40a2cc0 0x7f33e40a5180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f33f00a3c60 tx=0x7f33f0097180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.850+0000 7f3408bef740 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 0x55dca692b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.850+0000 7f3408bef740 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691e590 0x55dca6928910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.850+0000 7f3408bef740 1 --2- 172.21.15.46:0/3033925009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca691e050 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.850+0000 7f3408bef740 1 -- 172.21.15.46:0/3033925009 >> 172.21.15.46:0/3033925009 conn(0x55dca691a200 msgr2=0x55dca691a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.850+0000 7f3408bef740 1 -- 172.21.15.46:0/3033925009 shutdown_connections 2024-09-10T07:39:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.850+0000 7f3408bef740 1 -- 172.21.15.46:0/3033925009 wait complete. 2024-09-10T07:39:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.851+0000 7f3408bef740 1 -- start start 2024-09-10T07:39:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.851+0000 7f3408bef740 1 Processor -- start 2024-09-10T07:39:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.851+0000 7f3408bef740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca6925dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.851+0000 7f3408bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691e590 0x55dca6926310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.851+0000 7f3408bef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 0x55dca6926850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.851+0000 7f3408bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dca692ba20 con 0x55dca691e590 2024-09-10T07:39:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.851+0000 7f3408bef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dca683f0f0 con 0x55dca6928e50 2024-09-10T07:39:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.851+0000 7f3408bef740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dca692bba0 con 0x55dca691dc70 2024-09-10T07:39:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f3408bef740 10 client.0 _ll_register_callbacks cb 0x55dca6826bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca6925dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca6925dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40560/0 (socket says 172.21.15.46:40560) 2024-09-10T07:39:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 -- 172.21.15.46:0/1972663008 learned_addr learned my addr 172.21.15.46:0/1972663008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 0x55dca6926850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 -- 172.21.15.46:0/1972663008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 msgr2=0x55dca6926850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 0x55dca6926850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 -- 172.21.15.46:0/1972663008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691e590 msgr2=0x55dca6926310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691e590 0x55dca6926310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.852+0000 7f33ffe00640 1 -- 172.21.15.46:0/1972663008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca65f53c0 con 0x55dca691dc70 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f33ffe00640 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca6925dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009620 tx=0x7f33f80318d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d5a140c9484fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f33e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f33e8c00640 1 -- 172.21.15.46:0/1972663008 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8004240 con 0x55dca691dc70 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f3408bef740 1 -- 172.21.15.46:0/1972663008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f803f070 con 0x55dca691dc70 2024-09-10T07:39:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f33e8c00640 1 -- 172.21.15.46:0/1972663008 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f33f80043e0 con 0x55dca691dc70 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f33e8c00640 1 -- 172.21.15.46:0/1972663008 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f33f80045e0 con 0x55dca691dc70 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f3408bef740 10 client.26071 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f3408bef740 1 -- 172.21.15.46:0/1972663008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dca692ba20 con 0x55dca691dc70 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.853+0000 7f33e8c00640 1 -- 172.21.15.46:0/1972663008 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f803f070 con 0x55dca691dc70 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.854+0000 7f33e8c00640 1 -- 172.21.15.46:0/1972663008 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f33f802f610 con 0x55dca691dc70 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.854+0000 7f3408bef740 10 client.26071 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.854+0000 7f3408bef740 10 client.26071 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.854+0000 7f3408bef740 20 client.26071 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.854+0000 7f3408bef740 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55dca6938bd0 0x55dca693afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.854+0000 7f3408bef740 4 client.26071 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.855+0000 7f3408bef740 1 -- 172.21.15.46:0/1972663008 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dca6891950 con 0x55dca6938bd0 2024-09-10T07:39:16.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.855+0000 7f33fea00640 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55dca6938bd0 0x55dca693afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.855+0000 7f33fea00640 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55dca6938bd0 0x55dca693afc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61d27fd47e445d83 server_cookie=9fcc43ccb7bfefb7 in_seq=0 out_seq=0 2024-09-10T07:39:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.855+0000 7f33e8c00640 10 client.26071 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:39:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.856+0000 7f33e8c00640 1 -- 172.21.15.46:0/1972663008 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dca6891950 con 0x55dca6938bd0 2024-09-10T07:39:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.856+0000 7f33e8c00640 10 client.26071 handle_command_reply: tid=0 2024-09-10T07:39:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.856+0000 7f3408bef740 1 client.26071 shutdown 2024-09-10T07:39:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 -- 172.21.15.46:0/1972663008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 msgr2=0x55dca6925dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca6925dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009620 tx=0x7f33f80318d0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 -- 172.21.15.46:0/1972663008 shutdown_connections 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55dca6938bd0 0x55dca693afc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 0x55dca6926850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691e590 0x55dca6926310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 --2- 172.21.15.46:0/1972663008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691dc70 0x55dca6925dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 -- 172.21.15.46:0/1972663008 >> 172.21.15.46:0/1972663008 conn(0x55dca691a200 msgr2=0x55dca6919380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f33e8c00640 0 client.26071 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 -- 172.21.15.46:0/1972663008 shutdown_connections 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 1 -- 172.21.15.46:0/1972663008 wait complete. 2024-09-10T07:39:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.857+0000 7f3408bef740 20 client.26071 trim_cache size 0 max 16384 2024-09-10T07:39:16.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.871+0000 7f3408bef740 1 Processor -- start 2024-09-10T07:39:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.871+0000 7f3408bef740 1 -- start start 2024-09-10T07:39:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f3408bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691dc70 0x55dca6920690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f3408bef740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca69405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f3408bef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 0x55dca6940b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f3408bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34001bdb40 con 0x55dca691dc70 2024-09-10T07:39:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f3408bef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34001bf7a0 con 0x55dca6928e50 2024-09-10T07:39:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f3408bef740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f34001be210 con 0x55dca691e590 2024-09-10T07:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691dc70 0x55dca6920690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33fea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca69405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691dc70 0x55dca6920690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37148/0 (socket says 172.21.15.46:37148) 2024-09-10T07:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33fea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca69405c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40564/0 (socket says 172.21.15.46:40564) 2024-09-10T07:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33ffe00640 1 -- 172.21.15.46:0/3262109610 learned_addr learned my addr 172.21.15.46:0/3262109610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33ffe00640 1 --2- 172.21.15.46:0/3262109610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 0x55dca6940b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33fea00640 1 -- 172.21.15.46:0/3262109610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 msgr2=0x55dca6940b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33fea00640 1 --2- 172.21.15.46:0/3262109610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 0x55dca6940b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33fea00640 1 -- 172.21.15.46:0/3262109610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691dc70 msgr2=0x55dca6920690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.872+0000 7f33fea00640 1 --2- 172.21.15.46:0/3262109610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691dc70 0x55dca6920690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.873+0000 7f33fea00640 1 -- 172.21.15.46:0/3262109610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca6915770 con 0x55dca691e590 2024-09-10T07:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.873+0000 7f33fea00640 1 --2- 172.21.15.46:0/3262109610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca69405c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55dca6925660 tx=0x7f33ec009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5c63efa1d8b7d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.873+0000 7f33eb400640 1 -- 172.21.15.46:0/3262109610 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec0130d0 con 0x55dca691e590 2024-09-10T07:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.873+0000 7f33eb400640 1 -- 172.21.15.46:0/3262109610 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33ec013270 con 0x55dca691e590 2024-09-10T07:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.873+0000 7f33eb400640 1 -- 172.21.15.46:0/3262109610 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec004e10 con 0x55dca691e590 2024-09-10T07:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 -- 172.21.15.46:0/3262109610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 msgr2=0x55dca69405c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 --2- 172.21.15.46:0/3262109610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca69405c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55dca6925660 tx=0x7f33ec009750 comp rx=0 tx=0).stop 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 -- 172.21.15.46:0/3262109610 shutdown_connections 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 --2- 172.21.15.46:0/3262109610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca6928e50 0x55dca6940b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 --2- 172.21.15.46:0/3262109610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca69405c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 --2- 172.21.15.46:0/3262109610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca691dc70 0x55dca6920690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 -- 172.21.15.46:0/3262109610 >> 172.21.15.46:0/3262109610 conn(0x55dca691a200 msgr2=0x55dca6919380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 -- 172.21.15.46:0/3262109610 shutdown_connections 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.874+0000 7f3408bef740 1 -- 172.21.15.46:0/3262109610 wait complete. 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.875+0000 7f3408bef740 1 -- start start 2024-09-10T07:39:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.875+0000 7f3408bef740 1 Processor -- start 2024-09-10T07:39:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f3408bef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691dc70 0x55dca692ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f3408bef740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca692d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f3408bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 0x55dca692d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f3408bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34001bf010 con 0x55dca6928e50 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f3408bef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34001be940 con 0x55dca691dc70 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f3408bef740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3400076b00 con 0x55dca691e590 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f3408bef740 10 client.0 _ll_register_callbacks cb 0x7f33f8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33fea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca692d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33fea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca692d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40568/0 (socket says 172.21.15.46:40568) 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33fea00640 1 -- 172.21.15.46:0/785833535 learned_addr learned my addr 172.21.15.46:0/785833535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33ffe00640 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691dc70 0x55dca692ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33fea00640 1 -- 172.21.15.46:0/785833535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691dc70 msgr2=0x55dca692ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33fea00640 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691dc70 0x55dca692ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33fea00640 1 -- 172.21.15.46:0/785833535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 msgr2=0x55dca692d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33fea00640 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 0x55dca692d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.876+0000 7f33fea00640 1 -- 172.21.15.46:0/785833535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca69a2700 con 0x55dca691e590 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.877+0000 7f33fea00640 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca692d1f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f33ec005f50 tx=0x7f33ec009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dce5250b1c5abd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.877+0000 7f33e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.877+0000 7f33e8c00640 1 -- 172.21.15.46:0/785833535 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec004b90 con 0x55dca691e590 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.877+0000 7f33e8c00640 1 -- 172.21.15.46:0/785833535 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f33ec004d30 con 0x55dca691e590 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.877+0000 7f3408bef740 1 -- 172.21.15.46:0/785833535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec013270 con 0x55dca691e590 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.877+0000 7f33e8c00640 1 -- 172.21.15.46:0/785833535 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f33ec036050 con 0x55dca691e590 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.877+0000 7f3408bef740 10 client.26077 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.877+0000 7f3408bef740 1 -- 172.21.15.46:0/785833535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dca69a39c0 con 0x55dca691e590 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f33e8c00640 1 -- 172.21.15.46:0/785833535 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33ec013270 con 0x55dca691e590 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f33e8c00640 1 -- 172.21.15.46:0/785833535 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f33ec02d840 con 0x55dca691e590 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f3408bef740 10 client.26077 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f3408bef740 10 client.26077 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f3408bef740 20 client.26077 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f3408bef740 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55dca69a4070 0x55dca69a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f3408bef740 4 client.26077 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f3408bef740 1 -- 172.21.15.46:0/785833535 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dca688be60 con 0x55dca69a4070 2024-09-10T07:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.878+0000 7f33ffe00640 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55dca69a4070 0x55dca69a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f33ffe00640 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55dca69a4070 0x55dca69a6490 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4278db4f0f2db3c server_cookie=147f926b34cdac41 in_seq=0 out_seq=0 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f33e8c00640 10 client.26077 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f33e8c00640 1 -- 172.21.15.46:0/785833535 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dca688be60 con 0x55dca69a4070 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f33e8c00640 10 client.26077 handle_command_reply: tid=0 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 client.26077 shutdown 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 -- 172.21.15.46:0/785833535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 msgr2=0x55dca692d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca692d1f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f33ec005f50 tx=0x7f33ec009c20 comp rx=0 tx=0).stop 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 -- 172.21.15.46:0/785833535 shutdown_connections 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55dca69a4070 0x55dca69a6490 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dca6928e50 0x55dca692d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f33e8c00640 0 client.26077 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dca691e590 0x55dca692d1f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 --2- 172.21.15.46:0/785833535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dca691dc70 0x55dca692ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.879+0000 7f3408bef740 1 -- 172.21.15.46:0/785833535 >> 172.21.15.46:0/785833535 conn(0x55dca691a200 msgr2=0x55dca6933c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.880+0000 7f3408bef740 1 -- 172.21.15.46:0/785833535 shutdown_connections 2024-09-10T07:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.880+0000 7f3408bef740 1 -- 172.21.15.46:0/785833535 wait complete. 2024-09-10T07:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.880+0000 7f3408bef740 20 client.26077 trim_cache size 0 max 16384 2024-09-10T07:39:16.883 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 -- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33e40a2cc0 msgr2=0x7f33e40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33e40a2cc0 0x7f33e40a5180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f33f00a3c60 tx=0x7f33f0097180 comp rx=0 tx=0).stop 2024-09-10T07:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 -- 172.21.15.46:0/2412341801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400076720 msgr2=0x7f34001a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400076720 0x7f34001a46d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f33f000d950 tx=0x7f33f000de10 comp rx=0 tx=0).stop 2024-09-10T07:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 -- 172.21.15.46:0/2412341801 shutdown_connections 2024-09-10T07:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33e40a2cc0 0x7f33e40a5180 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f33f00a3c60 tx=0x7f33f0097180 comp rx=0 tx=0).stop 2024-09-10T07:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400112a50 0x7f340019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3400077040 0x7f34001a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 --2- 172.21.15.46:0/2412341801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3400076720 0x7f34001a46d0 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f33f000d950 tx=0x7f33f000de10 comp rx=0 tx=0).stop 2024-09-10T07:39:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.881+0000 7f3406200640 1 -- 172.21.15.46:0/2412341801 >> 172.21.15.46:0/2412341801 conn(0x7f34000ffa50 msgr2=0x7f3400101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.882+0000 7f3406200640 1 -- 172.21.15.46:0/2412341801 shutdown_connections 2024-09-10T07:39:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:16.882+0000 7f3406200640 1 -- 172.21.15.46:0/2412341801 wait complete. 2024-09-10T07:39:16.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:16 smithi181 ceph-mon[35397]: pgmap v478: 129 pgs: 129 active+clean; 982 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 668 op/s 2024-09-10T07:39:16.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:16 smithi063 ceph-mon[35020]: pgmap v478: 129 pgs: 129 active+clean; 982 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 668 op/s 2024-09-10T07:39:17.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:39:18.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:18 smithi181 ceph-mon[35397]: pgmap v479: 129 pgs: 129 active+clean; 982 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T07:39:18.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:18 smithi063 ceph-mon[35020]: pgmap v479: 129 pgs: 129 active+clean; 982 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T07:39:18.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:18 smithi046 ceph-mon[30705]: pgmap v479: 129 pgs: 129 active+clean; 982 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T07:39:19.014 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.476+0000 7ff18a800640 1 -- 172.21.15.46:0/4163567491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 msgr2=0x7ff17c09fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.476+0000 7ff18a800640 1 --2- 172.21.15.46:0/4163567491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 0x7ff17c09fcb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff184067960 tx=0x7ff184096bd0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.477+0000 7ff18a800640 1 -- 172.21.15.46:0/4163567491 shutdown_connections 2024-09-10T07:39:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.477+0000 7ff18a800640 1 --2- 172.21.15.46:0/4163567491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 0x7ff17c09fcb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.477+0000 7ff18a800640 1 --2- 172.21.15.46:0/4163567491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c098a60 0x7ff17c098ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.477+0000 7ff18a800640 1 --2- 172.21.15.46:0/4163567491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c097870 0x7ff17c097c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.477+0000 7ff18a800640 1 -- 172.21.15.46:0/4163567491 >> 172.21.15.46:0/4163567491 conn(0x7ff17c092fb0 msgr2=0x7ff17c095410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.477+0000 7ff18a800640 1 -- 172.21.15.46:0/4163567491 shutdown_connections 2024-09-10T07:39:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.477+0000 7ff18a800640 1 -- 172.21.15.46:0/4163567491 wait complete. 2024-09-10T07:39:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.479+0000 7ff18a800640 1 Processor -- start 2024-09-10T07:39:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.480+0000 7ff18a800640 1 -- start start 2024-09-10T07:39:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.480+0000 7ff18a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c097870 0x7ff17c13ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.480+0000 7ff18a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c098a60 0x7ff17c13ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.480+0000 7ff18a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 0x7ff17c145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.480+0000 7ff18a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c0a3400 con 0x7ff17c098a60 2024-09-10T07:39:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.480+0000 7ff18a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c0a3280 con 0x7ff17c099420 2024-09-10T07:39:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.480+0000 7ff18a800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c0a3580 con 0x7ff17c097870 2024-09-10T07:39:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.481+0000 7ff189400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c097870 0x7ff17c13ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.481+0000 7ff189400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c097870 0x7ff17c13ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40578/0 (socket says 172.21.15.46:40578) 2024-09-10T07:39:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.481+0000 7ff189400640 1 -- 172.21.15.46:0/2686513082 learned_addr learned my addr 172.21.15.46:0/2686513082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.481+0000 7ff189e00640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 0x7ff17c145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.481+0000 7ff189e00640 1 -- 172.21.15.46:0/2686513082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c097870 msgr2=0x7ff17c13ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.482+0000 7ff189e00640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c097870 0x7ff17c13ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.482+0000 7ff188a00640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c098a60 0x7ff17c13ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.482+0000 7ff189e00640 1 -- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c098a60 msgr2=0x7ff17c13ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.483+0000 7ff189e00640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c098a60 0x7ff17c13ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.483+0000 7ff189e00640 1 -- 172.21.15.46:0/2686513082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c146630 con 0x7ff17c099420 2024-09-10T07:39:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.483+0000 7ff188a00640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c098a60 0x7ff17c13ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.484+0000 7ff189e00640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 0x7ff17c145fc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff184067ea0 tx=0x7ff184098850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.484+0000 7ff182000640 1 -- 172.21.15.46:0/2686513082 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1840a5020 con 0x7ff17c099420 2024-09-10T07:39:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.485+0000 7ff182000640 1 -- 172.21.15.46:0/2686513082 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18406b170 con 0x7ff17c099420 2024-09-10T07:39:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.485+0000 7ff182000640 1 -- 172.21.15.46:0/2686513082 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184097610 con 0x7ff17c099420 2024-09-10T07:39:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.485+0000 7ff18a800640 1 -- 172.21.15.46:0/2686513082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c1468c0 con 0x7ff17c099420 2024-09-10T07:39:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.485+0000 7ff18a800640 1 -- 172.21.15.46:0/2686513082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c146d30 con 0x7ff17c099420 2024-09-10T07:39:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.490+0000 7ff18ee3d740 1 Processor -- start 2024-09-10T07:39:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.491+0000 7ff182000640 1 -- 172.21.15.46:0/2686513082 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1840a8070 con 0x7ff17c099420 2024-09-10T07:39:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.492+0000 7ff18ee3d740 1 -- start start 2024-09-10T07:39:20.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.495+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.495+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.495+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 0x56448d3be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.495+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56448d313100 con 0x56448d3b6a70 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.495+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56448d2bb1b0 con 0x56448d3bc440 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.495+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56448d30f0c0 con 0x56448d3b73b0 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.496+0000 7ff188a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.496+0000 7ff188a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37194/0 (socket says 172.21.15.46:37194) 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.496+0000 7ff188a00640 1 -- 172.21.15.46:0/3961969157 learned_addr learned my addr 172.21.15.46:0/3961969157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.496+0000 7ff188a00640 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 0x56448d3be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.496+0000 7ff189400640 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.498+0000 7ff182000640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff16c0a2c10 0x7ff16c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.499+0000 7ff188a00640 1 -- 172.21.15.46:0/3961969157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 msgr2=0x56448d3bbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.499+0000 7ff188a00640 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3bbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.499+0000 7ff188a00640 1 -- 172.21.15.46:0/3961969157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 msgr2=0x56448d3be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.499+0000 7ff188a00640 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 0x56448d3be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.499+0000 7ff188a00640 1 -- 172.21.15.46:0/3961969157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448d026210 con 0x56448d3b6a70 2024-09-10T07:39:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.499+0000 7ff188a00640 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3b6e70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff1780127f0 tx=0x7ff17800d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64b858bb72db497 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.500+0000 7ff182000640 1 -- 172.21.15.46:0/2686513082 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff184152360 con 0x7ff17c099420 2024-09-10T07:39:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.501+0000 7ff189e00640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff16c0a2c10 0x7ff16c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.501+0000 7ff172600640 1 -- 172.21.15.46:0/3961969157 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178005220 con 0x56448d3b6a70 2024-09-10T07:39:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.501+0000 7ff172600640 1 -- 172.21.15.46:0/3961969157 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff178004c80 con 0x56448d3b6a70 2024-09-10T07:39:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.501+0000 7ff172600640 1 -- 172.21.15.46:0/3961969157 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17800ab10 con 0x56448d3b6a70 2024-09-10T07:39:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.502+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3961969157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 msgr2=0x56448d3b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.502+0000 7ff189e00640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff16c0a2c10 0x7ff16c0a50d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff18411b210 tx=0x7ff184068050 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.502+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3b6e70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff1780127f0 tx=0x7ff17800d3c0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.502+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3961969157 shutdown_connections 2024-09-10T07:39:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.502+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 0x56448d3be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.502+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.502+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/3961969157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3b6e70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.503+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3961969157 >> 172.21.15.46:0/3961969157 conn(0x56448d3a98d0 msgr2=0x56448d3a9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.503+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3961969157 shutdown_connections 2024-09-10T07:39:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.504+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3961969157 wait complete. 2024-09-10T07:39:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.504+0000 7ff18ee3d740 1 -- start start 2024-09-10T07:39:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.505+0000 7ff18ee3d740 1 Processor -- start 2024-09-10T07:39:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.506+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 0x56448d3b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56448d3abc10 con 0x56448d3bc440 2024-09-10T07:39:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56448d29c090 con 0x56448d3b73b0 2024-09-10T07:39:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56448d3abd90 con 0x56448d3b6a70 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff18ee3d740 10 client.0 _ll_register_callbacks cb 0x56448d2f0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff188a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff188a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40584/0 (socket says 172.21.15.46:40584) 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.507+0000 7ff188a00640 1 -- 172.21.15.46:0/2340994789 learned_addr learned my addr 172.21.15.46:0/2340994789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.509+0000 7ff189400640 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.509+0000 7ff189400640 1 -- 172.21.15.46:0/2340994789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 msgr2=0x56448d3af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.510+0000 7ff189400640 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.510+0000 7ff189400640 1 -- 172.21.15.46:0/2340994789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 msgr2=0x56448d3b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.510+0000 7ff189400640 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 0x56448d3b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.510+0000 7ff189400640 1 -- 172.21.15.46:0/2340994789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448d0f2620 con 0x56448d3b73b0 2024-09-10T07:39:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.511+0000 7ff188a00640 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:39:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.512+0000 7ff189400640 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x56448d3b7de0 tx=0x7ff1740128e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7b67497f894959 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.512+0000 7ff166a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.513+0000 7ff166a00640 1 -- 172.21.15.46:0/2340994789 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174005220 con 0x56448d3b73b0 2024-09-10T07:39:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.513+0000 7ff18ee3d740 1 -- 172.21.15.46:0/2340994789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff178004c80 con 0x56448d3b73b0 2024-09-10T07:39:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.513+0000 7ff166a00640 1 -- 172.21.15.46:0/2340994789 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff1740053c0 con 0x56448d3b73b0 2024-09-10T07:39:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.514+0000 7ff166a00640 1 -- 172.21.15.46:0/2340994789 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff174025050 con 0x56448d3b73b0 2024-09-10T07:39:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.515+0000 7ff18ee3d740 10 client.26093 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.515+0000 7ff18ee3d740 1 -- 172.21.15.46:0/2340994789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56448d3c2980 con 0x56448d3b73b0 2024-09-10T07:39:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.516+0000 7ff166a00640 1 -- 172.21.15.46:0/2340994789 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff178004c80 con 0x56448d3b73b0 2024-09-10T07:39:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.516+0000 7ff166a00640 1 -- 172.21.15.46:0/2340994789 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7ff174019030 con 0x56448d3b73b0 2024-09-10T07:39:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.516+0000 7ff18ee3d740 10 client.26093 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.516+0000 7ff18ee3d740 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.516+0000 7ff18ee3d740 20 client.26093 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.516+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56448d3c41c0 0x56448d3c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.516+0000 7ff18ee3d740 4 client.26093 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.516+0000 7ff18ee3d740 1 -- 172.21.15.46:0/2340994789 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56448d320830 con 0x56448d3c41c0 2024-09-10T07:39:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.518+0000 7ff188a00640 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56448d3c41c0 0x56448d3c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.522+0000 7ff188a00640 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56448d3c41c0 0x56448d3c65e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bafc6552b4d18ffc server_cookie=eeb27ad25474ff6 in_seq=0 out_seq=0 2024-09-10T07:39:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.522+0000 7ff166a00640 10 client.26093 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.525+0000 7ff166a00640 1 -- 172.21.15.46:0/2340994789 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56448d320830 con 0x56448d3c41c0 2024-09-10T07:39:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.525+0000 7ff166a00640 10 client.26093 handle_command_reply: tid=0 2024-09-10T07:39:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.526+0000 7ff18ee3d740 1 client.26093 shutdown 2024-09-10T07:39:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.527+0000 7ff18ee3d740 1 -- 172.21.15.46:0/2340994789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 msgr2=0x56448d3afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.527+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x56448d3b7de0 tx=0x7ff1740128e0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.528+0000 7ff18ee3d740 1 -- 172.21.15.46:0/2340994789 shutdown_connections 2024-09-10T07:39:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.528+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56448d3c41c0 0x56448d3c65e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.528+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 0x56448d3b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.528+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.528+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/2340994789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.528+0000 7ff18ee3d740 1 -- 172.21.15.46:0/2340994789 >> 172.21.15.46:0/2340994789 conn(0x56448d3a98d0 msgr2=0x56448d3ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.528+0000 7ff166a00640 0 client.26093 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.528+0000 7ff18ee3d740 1 -- 172.21.15.46:0/2340994789 shutdown_connections 2024-09-10T07:39:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.529+0000 7ff18ee3d740 1 -- 172.21.15.46:0/2340994789 wait complete. 2024-09-10T07:39:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.530+0000 7ff18ee3d740 20 client.26093 trim_cache size 0 max 16384 2024-09-10T07:39:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.562+0000 7ff18ee3d740 1 Processor -- start 2024-09-10T07:39:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff18ee3d740 1 -- start start 2024-09-10T07:39:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 0x56448d3ba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c14cf10 con 0x56448d3bc440 2024-09-10T07:39:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c14eb70 con 0x56448d3b73b0 2024-09-10T07:39:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c14d5e0 con 0x56448d3b6a70 2024-09-10T07:39:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff188a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.563+0000 7ff188a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40592/0 (socket says 172.21.15.46:40592) 2024-09-10T07:39:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.564+0000 7ff188a00640 1 -- 172.21.15.46:0/3146404983 learned_addr learned my addr 172.21.15.46:0/3146404983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.564+0000 7ff189400640 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.564+0000 7ff188a00640 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 0x56448d3ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.564+0000 7ff189400640 1 -- 172.21.15.46:0/3146404983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 msgr2=0x56448d3af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.564+0000 7ff189400640 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.564+0000 7ff189400640 1 -- 172.21.15.46:0/3146404983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 msgr2=0x56448d3ba610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.564+0000 7ff189400640 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 0x56448d3ba610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.564+0000 7ff189400640 1 -- 172.21.15.46:0/3146404983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448d3a96f0 con 0x56448d3b73b0 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.565+0000 7ff189400640 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff174010d40 tx=0x7ff17400bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86634383243fe242 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.565+0000 7ff171c00640 1 -- 172.21.15.46:0/3146404983 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17402f020 con 0x56448d3b73b0 2024-09-10T07:39:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.565+0000 7ff171c00640 1 -- 172.21.15.46:0/3146404983 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff174010760 con 0x56448d3b73b0 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.565+0000 7ff171c00640 1 -- 172.21.15.46:0/3146404983 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17402be60 con 0x56448d3b73b0 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3146404983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 msgr2=0x56448d3afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff174010d40 tx=0x7ff17400bb80 comp rx=0 tx=0).stop 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3146404983 shutdown_connections 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3bc440 0x56448d3ba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3b73b0 0x56448d3afa80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/3146404983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b6a70 0x56448d3af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3146404983 >> 172.21.15.46:0/3146404983 conn(0x56448d3a98d0 msgr2=0x56448d3a9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3146404983 shutdown_connections 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 -- 172.21.15.46:0/3146404983 wait complete. 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.566+0000 7ff18ee3d740 1 -- start start 2024-09-10T07:39:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff18ee3d740 1 Processor -- start 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3d5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff18ee3d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 0x56448d42ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c14e3e0 con 0x56448d3b6a70 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c14dd10 con 0x56448d3bc440 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff18ee3d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c14aef0 con 0x56448d3b73b0 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff18ee3d740 10 client.0 _ll_register_callbacks cb 0x7ff17400b880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.567+0000 7ff188a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3d5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff188a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 0x56448d42ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff188a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3d5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37212/0 (socket says 172.21.15.46:37212) 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff188a00640 1 -- 172.21.15.46:0/4108891616 learned_addr learned my addr 172.21.15.46:0/4108891616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff189400640 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff189400640 1 -- 172.21.15.46:0/4108891616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 msgr2=0x56448d42ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff189400640 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 0x56448d42ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff189400640 1 -- 172.21.15.46:0/4108891616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 msgr2=0x56448d3d5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff189400640 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3d5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.568+0000 7ff189400640 1 -- 172.21.15.46:0/4108891616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448d3d8750 con 0x56448d3b73b0 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.569+0000 7ff189400640 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3d5740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff174010d40 tx=0x7ff174010910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada1332eadcc563b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.569+0000 7ff166a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.569+0000 7ff166a00640 1 -- 172.21.15.46:0/4108891616 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17403b020 con 0x56448d3b73b0 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.569+0000 7ff166a00640 1 -- 172.21.15.46:0/4108891616 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff17402b850 con 0x56448d3b73b0 2024-09-10T07:39:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.570+0000 7ff18ee3d740 1 -- 172.21.15.46:0/4108891616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff174010760 con 0x56448d3b73b0 2024-09-10T07:39:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.570+0000 7ff166a00640 1 -- 172.21.15.46:0/4108891616 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff174005040 con 0x56448d3b73b0 2024-09-10T07:39:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.570+0000 7ff18ee3d740 10 client.26095 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.570+0000 7ff18ee3d740 1 -- 172.21.15.46:0/4108891616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56448d42e250 con 0x56448d3b73b0 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.570+0000 7ff166a00640 1 -- 172.21.15.46:0/4108891616 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff174010760 con 0x56448d3b73b0 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.570+0000 7ff166a00640 1 -- 172.21.15.46:0/4108891616 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7ff174019040 con 0x56448d3b73b0 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.571+0000 7ff18ee3d740 10 client.26095 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.571+0000 7ff18ee3d740 10 client.26095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.571+0000 7ff18ee3d740 20 client.26095 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.571+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56448d42eb80 0x56448d430fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.571+0000 7ff18ee3d740 4 client.26095 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.571+0000 7ff18ee3d740 1 -- 172.21.15.46:0/4108891616 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56448d31ab80 con 0x56448d42eb80 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.571+0000 7ff188a00640 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56448d42eb80 0x56448d430fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.572+0000 7ff188a00640 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56448d42eb80 0x56448d430fa0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb021705a0668d58 server_cookie=5c497a28c6c1b01d in_seq=0 out_seq=0 2024-09-10T07:39:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.572+0000 7ff166a00640 10 client.26095 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff166a00640 1 -- 172.21.15.46:0/4108891616 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56448d31ab80 con 0x56448d42eb80 2024-09-10T07:39:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff166a00640 10 client.26095 handle_command_reply: tid=0 2024-09-10T07:39:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff18ee3d740 1 client.26095 shutdown 2024-09-10T07:39:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff18ee3d740 1 -- 172.21.15.46:0/4108891616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 msgr2=0x56448d3d5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3d5740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff174010d40 tx=0x7ff174010910 comp rx=0 tx=0).stop 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff18ee3d740 1 -- 172.21.15.46:0/4108891616 shutdown_connections 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56448d42eb80 0x56448d430fa0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56448d3bc440 0x56448d42ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56448d3b73b0 0x56448d3d5740 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.587+0000 7ff18ee3d740 1 --2- 172.21.15.46:0/4108891616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56448d3b6a70 0x56448d3d5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.588+0000 7ff18ee3d740 1 -- 172.21.15.46:0/4108891616 >> 172.21.15.46:0/4108891616 conn(0x56448d3a98d0 msgr2=0x56448d3ba610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.588+0000 7ff166a00640 0 client.26095 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.588+0000 7ff18ee3d740 1 -- 172.21.15.46:0/4108891616 shutdown_connections 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.588+0000 7ff18ee3d740 1 -- 172.21.15.46:0/4108891616 wait complete. 2024-09-10T07:39:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.588+0000 7ff18ee3d740 20 client.26095 trim_cache size 0 max 16384 2024-09-10T07:39:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:39:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:39:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:39:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:39:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 -- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff16c0a2c10 msgr2=0x7ff16c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff16c0a2c10 0x7ff16c0a50d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff18411b210 tx=0x7ff184068050 comp rx=0 tx=0).stop 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 -- 172.21.15.46:0/2686513082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 msgr2=0x7ff17c145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 0x7ff17c145fc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff184067ea0 tx=0x7ff184098850 comp rx=0 tx=0).stop 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 -- 172.21.15.46:0/2686513082 shutdown_connections 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff17c099420 0x7ff17c145fc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff16c0a2c10 0x7ff16c0a50d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c098a60 0x7ff17c13ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 --2- 172.21.15.46:0/2686513082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c097870 0x7ff17c13ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.589+0000 7ff18a800640 1 -- 172.21.15.46:0/2686513082 >> 172.21.15.46:0/2686513082 conn(0x7ff17c092fb0 msgr2=0x7ff17c09be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.590+0000 7ff18a800640 1 -- 172.21.15.46:0/2686513082 shutdown_connections 2024-09-10T07:39:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:20.590+0000 7ff18a800640 1 -- 172.21.15.46:0/2686513082 wait complete. 2024-09-10T07:39:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 120.0 s, 1728.4 tps, lat 58.195 ms stddev 68.275 2024-09-10T07:39:20.710 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:20 smithi046 ceph-mon[30705]: pgmap v480: 129 pgs: 129 active+clean; 982 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 624 op/s 2024-09-10T07:39:20.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:20 smithi181 ceph-mon[35397]: pgmap v480: 129 pgs: 129 active+clean; 982 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 624 op/s 2024-09-10T07:39:20.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:20 smithi063 ceph-mon[35020]: pgmap v480: 129 pgs: 129 active+clean; 982 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 624 op/s 2024-09-10T07:39:20.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.901+0000 7faabfc00640 1 -- 172.21.15.46:0/1351223321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8104970 msgr2=0x7faab81089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.901+0000 7faabfc00640 1 --2- 172.21.15.46:0/1351223321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8104970 0x7faab81089c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009860 tx=0x7faaa402a340 comp rx=0 tx=0).stop 2024-09-10T07:39:20.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.902+0000 7faabfc00640 1 -- 172.21.15.46:0/1351223321 shutdown_connections 2024-09-10T07:39:20.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.902+0000 7faabfc00640 1 --2- 172.21.15.46:0/1351223321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab81090f0 0x7faab8110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.902+0000 7faabfc00640 1 --2- 172.21.15.46:0/1351223321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8104970 0x7faab81089c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.902+0000 7faabfc00640 1 --2- 172.21.15.46:0/1351223321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab8103fc0 0x7faab81043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.902+0000 7faabfc00640 1 -- 172.21.15.46:0/1351223321 >> 172.21.15.46:0/1351223321 conn(0x7faab80ffa40 msgr2=0x7faab8101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.902+0000 7faabfc00640 1 -- 172.21.15.46:0/1351223321 shutdown_connections 2024-09-10T07:39:20.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.902+0000 7faabfc00640 1 -- 172.21.15.46:0/1351223321 wait complete. 2024-09-10T07:39:20.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.905+0000 7faabfc00640 1 Processor -- start 2024-09-10T07:39:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.906+0000 7faabfc00640 1 -- start start 2024-09-10T07:39:20.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabfc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8103fc0 0x7faab8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabfc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab8104970 0x7faab8076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabfc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab81090f0 0x7faab8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabfc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faab8114140 con 0x7faab8104970 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabfc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faab8113fc0 con 0x7faab8103fc0 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabfc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faab81142c0 con 0x7faab81090f0 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabd200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8103fc0 0x7faab8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faab7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab8104970 0x7faab8076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabd200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8103fc0 0x7faab8076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38120/0 (socket says 172.21.15.46:38120) 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.907+0000 7faabd200640 1 -- 172.21.15.46:0/666658743 learned_addr learned my addr 172.21.15.46:0/666658743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.908+0000 7faabdc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab81090f0 0x7faab8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.908+0000 7faabd200640 1 -- 172.21.15.46:0/666658743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab81090f0 msgr2=0x7faab8077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.908+0000 7faabd200640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab81090f0 0x7faab8077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.908+0000 7faabd200640 1 -- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab8104970 msgr2=0x7faab8076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.908+0000 7faabd200640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab8104970 0x7faab8076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.908+0000 7faabd200640 1 -- 172.21.15.46:0/666658743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab81aadc0 con 0x7faab8103fc0 2024-09-10T07:39:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.909+0000 7faabd200640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8103fc0 0x7faab8076700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faaac01fa30 tx=0x7faaac01fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.910+0000 7faab5600640 1 -- 172.21.15.46:0/666658743 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac00cd80 con 0x7faab8103fc0 2024-09-10T07:39:20.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.910+0000 7faab5600640 1 -- 172.21.15.46:0/666658743 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac005500 con 0x7faab8103fc0 2024-09-10T07:39:20.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.910+0000 7faab5600640 1 -- 172.21.15.46:0/666658743 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0216a0 con 0x7faab8103fc0 2024-09-10T07:39:20.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.910+0000 7faabfc00640 1 -- 172.21.15.46:0/666658743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab806bc70 con 0x7faab8103fc0 2024-09-10T07:39:20.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.911+0000 7faabfc00640 1 -- 172.21.15.46:0/666658743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab806c130 con 0x7faab8103fc0 2024-09-10T07:39:20.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.917+0000 7faac2686740 1 Processor -- start 2024-09-10T07:39:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.917+0000 7faac2686740 1 -- start start 2024-09-10T07:39:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.917+0000 7faac2686740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 0x55559238d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.917+0000 7faac2686740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238d590 0x555592397910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.917+0000 7faac2686740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x55559239a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.917+0000 7faac2686740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5555922f33e0 con 0x55559238cc70 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.917+0000 7faac2686740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55559229ad80 con 0x55559238d590 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.917+0000 7faac2686740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5555922ef3a0 con 0x555592397e50 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.918+0000 7faab7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x55559239a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.918+0000 7faab7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x55559239a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40608/0 (socket says 172.21.15.46:40608) 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.918+0000 7faab7e00640 1 -- 172.21.15.46:0/3449976407 learned_addr learned my addr 172.21.15.46:0/3449976407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.918+0000 7faab5600640 1 -- 172.21.15.46:0/666658743 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faaac0046c0 con 0x7faab8103fc0 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.918+0000 7faab7e00640 1 -- 172.21.15.46:0/3449976407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238d590 msgr2=0x555592397910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.918+0000 7faab7e00640 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238d590 0x555592397910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.918+0000 7faab7e00640 1 -- 172.21.15.46:0/3449976407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 msgr2=0x55559238d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.918+0000 7faabdc00640 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 0x55559238d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.919+0000 7faab7e00640 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 0x55559238d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.919+0000 7faab7e00640 1 -- 172.21.15.46:0/3449976407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555922b9ae0 con 0x555592397e50 2024-09-10T07:39:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.919+0000 7faabdc00640 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 0x55559238d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.921+0000 7faab7e00640 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x55559239a240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009860 tx=0x7faaa40303e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49787653e18da46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.922+0000 7faaaa000640 1 -- 172.21.15.46:0/3449976407 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4041020 con 0x555592397e50 2024-09-10T07:39:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.923+0000 7faaaa000640 1 -- 172.21.15.46:0/3449976407 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa4030550 con 0x555592397e50 2024-09-10T07:39:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.923+0000 7faab5600640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faaa00a2d10 0x7faaa00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.923+0000 7faaaa000640 1 -- 172.21.15.46:0/3449976407 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4037040 con 0x555592397e50 2024-09-10T07:39:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.924+0000 7faab5600640 1 -- 172.21.15.46:0/666658743 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faaac0e0760 con 0x7faab8103fc0 2024-09-10T07:39:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.924+0000 7faabdc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faaa00a2d10 0x7faaa00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.926+0000 7faac2686740 1 -- 172.21.15.46:0/3449976407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 msgr2=0x55559239a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.926+0000 7faac2686740 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x55559239a240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009860 tx=0x7faaa40303e0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.927+0000 7faabdc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faaa00a2d10 0x7faaa00a51d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faab0005fd0 tx=0x7faab0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.928+0000 7faac2686740 1 -- 172.21.15.46:0/3449976407 shutdown_connections 2024-09-10T07:39:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.928+0000 7faac2686740 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x55559239a240 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.928+0000 7faac2686740 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238d590 0x555592397910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.928+0000 7faac2686740 1 --2- 172.21.15.46:0/3449976407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 0x55559238d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.928+0000 7faac2686740 1 -- 172.21.15.46:0/3449976407 >> 172.21.15.46:0/3449976407 conn(0x555592389200 msgr2=0x5555923895e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.935+0000 7faac2686740 1 -- 172.21.15.46:0/3449976407 shutdown_connections 2024-09-10T07:39:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.936+0000 7faac2686740 1 -- 172.21.15.46:0/3449976407 wait complete. 2024-09-10T07:39:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.936+0000 7faac2686740 1 -- start start 2024-09-10T07:39:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.937+0000 7faac2686740 1 Processor -- start 2024-09-10T07:39:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.938+0000 7faac2686740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x555592394da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.938+0000 7faac2686740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238d590 0x5555923952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.938+0000 7faac2686740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x555592395820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.939+0000 7faabdc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x555592395820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.939+0000 7faabdc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x555592395820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40624/0 (socket says 172.21.15.46:40624) 2024-09-10T07:39:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.939+0000 7faabd200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x555592394da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.940+0000 7faabdc00640 1 -- 172.21.15.46:0/3337354517 learned_addr learned my addr 172.21.15.46:0/3337354517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.940+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55559239aa20 con 0x55559238d590 2024-09-10T07:39:20.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.940+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5555922ae0f0 con 0x55559238cc70 2024-09-10T07:39:20.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.940+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55559239aba0 con 0x555592397e50 2024-09-10T07:39:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.941+0000 7faabdc00640 1 -- 172.21.15.46:0/3337354517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 msgr2=0x555592394da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.941+0000 7faabdc00640 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x555592394da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.941+0000 7faabdc00640 1 -- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238d590 msgr2=0x5555923952e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:39:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.941+0000 7faac2686740 10 client.0 _ll_register_callbacks cb 0x555592295bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.945+0000 7faab7e00640 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238d590 0x5555923952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.945+0000 7faabdc00640 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238d590 0x5555923952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.945+0000 7faabdc00640 1 -- 172.21.15.46:0/3337354517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555920643c0 con 0x555592397e50 2024-09-10T07:39:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.945+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa4030550 con 0x555592397e50 2024-09-10T07:39:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.945+0000 7faab7e00640 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238d590 0x5555923952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.945+0000 7faabdc00640 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x555592395820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faab0012c70 tx=0x7faab0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f5220aeb478785f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.945+0000 7faa9ea00640 10 client.26107 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.946+0000 7faa9ea00640 1 -- 172.21.15.46:0/3337354517 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab000fea0 con 0x555592397e50 2024-09-10T07:39:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.946+0000 7faa9ea00640 1 -- 172.21.15.46:0/3337354517 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faab001f620 con 0x555592397e50 2024-09-10T07:39:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.946+0000 7faa9ea00640 1 -- 172.21.15.46:0/3337354517 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faab001f880 con 0x555592397e50 2024-09-10T07:39:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.946+0000 7faac2686740 10 client.26107 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.946+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=36}) -- 0x5555923a6990 con 0x555592397e50 2024-09-10T07:39:20.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.947+0000 7faa9ea00640 1 -- 172.21.15.46:0/3337354517 <== mon.1 v2:172.21.15.181:3300/0 4 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7faab001fa20 con 0x555592397e50 2024-09-10T07:39:20.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.947+0000 7faac2686740 10 client.26107 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.947+0000 7faac2686740 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.947+0000 7faac2686740 20 client.26107 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.947+0000 7faac2686740 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5555923a8200 0x5555923aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.947+0000 7faac2686740 4 client.26107 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.947+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555592300950 con 0x5555923a8200 2024-09-10T07:39:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.949+0000 7faabd200640 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5555923a8200 0x5555923aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.950+0000 7faabd200640 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5555923a8200 0x5555923aa5f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7ab0cdb9f1e9fb server_cookie=92d5fbbaf1add737 in_seq=0 out_seq=0 2024-09-10T07:39:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.952+0000 7faa9ea00640 10 client.26107 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.952+0000 7faa9ea00640 1 -- 172.21.15.46:0/3337354517 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555592300950 con 0x5555923a8200 2024-09-10T07:39:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.952+0000 7faa9ea00640 10 client.26107 handle_command_reply: tid=0 2024-09-10T07:39:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.952+0000 7faac2686740 1 client.26107 shutdown 2024-09-10T07:39:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.953+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 msgr2=0x555592395820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.953+0000 7faac2686740 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x555592395820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faab0012c70 tx=0x7faab0004a40 comp rx=0 tx=0).stop 2024-09-10T07:39:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 shutdown_connections 2024-09-10T07:39:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faac2686740 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5555923a8200 0x5555923aa5f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faac2686740 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555592397e50 0x555592395820 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faac2686740 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238d590 0x5555923952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faac2686740 1 --2- 172.21.15.46:0/3337354517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x555592394da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 >> 172.21.15.46:0/3337354517 conn(0x555592389200 msgr2=0x55559238ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faa9ea00640 0 client.26107 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 shutdown_connections 2024-09-10T07:39:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.954+0000 7faac2686740 1 -- 172.21.15.46:0/3337354517 wait complete. 2024-09-10T07:39:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.955+0000 7faac2686740 20 client.26107 trim_cache size 0 max 16384 2024-09-10T07:39:20.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.983+0000 7faac2686740 1 Processor -- start 2024-09-10T07:39:20.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.984+0000 7faac2686740 1 -- start start 2024-09-10T07:39:20.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.984+0000 7faac2686740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x55559238ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.984+0000 7faac2686740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55559238d590 0x5555923ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.984+0000 7faac2686740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555592397e50 0x5555923ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faab7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55559238d590 0x5555923ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faabd200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x55559238ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faab7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55559238d590 0x5555923ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40628/0 (socket says 172.21.15.46:40628) 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faabd200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x55559238ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38140/0 (socket says 172.21.15.46:38140) 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faab7e00640 1 -- 172.21.15.46:0/111651808 learned_addr learned my addr 172.21.15.46:0/111651808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faac2686740 1 -- 172.21.15.46:0/111651808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faab81aca60 con 0x555592397e50 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faac2686740 1 -- 172.21.15.46:0/111651808 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ae6c0 con 0x55559238cc70 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faac2686740 1 -- 172.21.15.46:0/111651808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ad130 con 0x55559238d590 2024-09-10T07:39:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.985+0000 7faabdc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555592397e50 0x5555923ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.986+0000 7faab7e00640 1 -- 172.21.15.46:0/111651808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 msgr2=0x55559238ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.986+0000 7faab7e00640 1 --2- 172.21.15.46:0/111651808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x55559238ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.986+0000 7faab7e00640 1 -- 172.21.15.46:0/111651808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555592397e50 msgr2=0x5555923ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.986+0000 7faab7e00640 1 --2- 172.21.15.46:0/111651808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555592397e50 0x5555923ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.986+0000 7faab7e00640 1 -- 172.21.15.46:0/111651808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55559238caa0 con 0x55559238d590 2024-09-10T07:39:20.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.986+0000 7faab7e00640 1 --2- 172.21.15.46:0/111651808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55559238d590 0x5555923ad230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faaa40304f0 tx=0x7faaa4030ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f138457585e5f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.987+0000 7faaa9600640 1 -- 172.21.15.46:0/111651808 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4041070 con 0x55559238d590 2024-09-10T07:39:20.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.987+0000 7faaa9600640 1 -- 172.21.15.46:0/111651808 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa40309c0 con 0x55559238d590 2024-09-10T07:39:20.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.987+0000 7faaa9600640 1 -- 172.21.15.46:0/111651808 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4037040 con 0x55559238d590 2024-09-10T07:39:20.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.988+0000 7faac2686740 1 -- 172.21.15.46:0/111651808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55559238d590 msgr2=0x5555923ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:20.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.988+0000 7faac2686740 1 --2- 172.21.15.46:0/111651808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55559238d590 0x5555923ad230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faaa40304f0 tx=0x7faaa4030ec0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.988+0000 7faac2686740 1 -- 172.21.15.46:0/111651808 shutdown_connections 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.988+0000 7faac2686740 1 --2- 172.21.15.46:0/111651808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555592397e50 0x5555923ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.988+0000 7faac2686740 1 --2- 172.21.15.46:0/111651808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55559238d590 0x5555923ad230 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faaa40304f0 tx=0x7faaa4030ec0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.988+0000 7faac2686740 1 --2- 172.21.15.46:0/111651808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55559238cc70 0x55559238ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.988+0000 7faac2686740 1 -- 172.21.15.46:0/111651808 >> 172.21.15.46:0/111651808 conn(0x555592389200 msgr2=0x55559238e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.991+0000 7faac2686740 1 -- 172.21.15.46:0/111651808 shutdown_connections 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.991+0000 7faac2686740 1 -- 172.21.15.46:0/111651808 wait complete. 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.992+0000 7faac2686740 1 -- start start 2024-09-10T07:39:20.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.992+0000 7faac2686740 1 Processor -- start 2024-09-10T07:39:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.993+0000 7faac2686740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 0x5555923ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.994+0000 7faac2686740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555592397e50 0x5555923aad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.994+0000 7faac2686740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5555923ab2d0 0x5555923ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.994+0000 7faabdc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5555923ab2d0 0x5555923ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.994+0000 7faabdc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5555923ab2d0 0x5555923ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40640/0 (socket says 172.21.15.46:40640) 2024-09-10T07:39:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.994+0000 7faabdc00640 1 -- 172.21.15.46:0/1672940185 learned_addr learned my addr 172.21.15.46:0/1672940185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.994+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faab81adf30 con 0x55559238cc70 2024-09-10T07:39:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.994+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ad860 con 0x555592397e50 2024-09-10T07:39:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.994+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faab8073ec0 con 0x5555923ab2d0 2024-09-10T07:39:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.995+0000 7faac2686740 10 client.0 _ll_register_callbacks cb 0x7faab0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.995+0000 7faab7e00640 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555592397e50 0x5555923aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.995+0000 7faabdc00640 1 -- 172.21.15.46:0/1672940185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555592397e50 msgr2=0x5555923aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.995+0000 7faabdc00640 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555592397e50 0x5555923aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.995+0000 7faabdc00640 1 -- 172.21.15.46:0/1672940185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 msgr2=0x5555923ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.995+0000 7faabdc00640 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 0x5555923ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.995+0000 7faabdc00640 1 -- 172.21.15.46:0/1672940185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555592384770 con 0x5555923ab2d0 2024-09-10T07:39:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.996+0000 7faab7e00640 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555592397e50 0x5555923aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.996+0000 7faa9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.996+0000 7faabdc00640 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5555923ab2d0 0x5555923ab6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faab0026000 tx=0x7faab0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=168a0833a61953ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.996+0000 7faa9ea00640 1 -- 172.21.15.46:0/1672940185 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0039040 con 0x5555923ab2d0 2024-09-10T07:39:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.997+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa40309c0 con 0x5555923ab2d0 2024-09-10T07:39:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.997+0000 7faa9ea00640 1 -- 172.21.15.46:0/1672940185 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faab001dc40 con 0x5555923ab2d0 2024-09-10T07:39:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.997+0000 7faa9ea00640 1 -- 172.21.15.46:0/1672940185 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faab000fea0 con 0x5555923ab2d0 2024-09-10T07:39:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.997+0000 7faac2686740 10 client.26113 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.997+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555592412e20 con 0x5555923ab2d0 2024-09-10T07:39:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.998+0000 7faa9ea00640 1 -- 172.21.15.46:0/1672940185 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa40309c0 con 0x5555923ab2d0 2024-09-10T07:39:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.998+0000 7faa9ea00640 1 -- 172.21.15.46:0/1672940185 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7faab003c070 con 0x5555923ab2d0 2024-09-10T07:39:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.999+0000 7faac2686740 10 client.26113 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.999+0000 7faac2686740 10 client.26113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.999+0000 7faac2686740 20 client.26113 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.999+0000 7faac2686740 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555592413580 0x555592415a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.999+0000 7faac2686740 4 client.26113 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:39:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.999+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555922fae60 con 0x555592413580 2024-09-10T07:39:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:20.999+0000 7faabd200640 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555592413580 0x555592415a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:21.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.001+0000 7faabd200640 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555592413580 0x555592415a20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb52ff427ea0943e server_cookie=90a7137bcf4675c4 in_seq=0 out_seq=0 2024-09-10T07:39:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.001+0000 7faa9ea00640 10 client.26113 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.002+0000 7faa9ea00640 1 -- 172.21.15.46:0/1672940185 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555922fae60 con 0x555592413580 2024-09-10T07:39:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.002+0000 7faa9ea00640 10 client.26113 handle_command_reply: tid=0 2024-09-10T07:39:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.002+0000 7faac2686740 1 client.26113 shutdown 2024-09-10T07:39:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5555923ab2d0 msgr2=0x5555923ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faac2686740 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5555923ab2d0 0x5555923ab6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faab0026000 tx=0x7faab0018000 comp rx=0 tx=0).stop 2024-09-10T07:39:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 shutdown_connections 2024-09-10T07:39:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faac2686740 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555592413580 0x555592415a20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faa9ea00640 0 client.26113 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faac2686740 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5555923ab2d0 0x5555923ab6f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faac2686740 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555592397e50 0x5555923aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faac2686740 1 --2- 172.21.15.46:0/1672940185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55559238cc70 0x5555923ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.003+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 >> 172.21.15.46:0/1672940185 conn(0x555592389200 msgr2=0x555592396e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.004+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 shutdown_connections 2024-09-10T07:39:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.004+0000 7faac2686740 1 -- 172.21.15.46:0/1672940185 wait complete. 2024-09-10T07:39:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.004+0000 7faac2686740 20 client.26113 trim_cache size 0 max 16384 2024-09-10T07:39:21.005 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:39:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.005+0000 7faabfc00640 1 -- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faaa00a2d10 msgr2=0x7faaa00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.005+0000 7faabfc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faaa00a2d10 0x7faaa00a51d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faab0005fd0 tx=0x7faab0009500 comp rx=0 tx=0).stop 2024-09-10T07:39:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.005+0000 7faabfc00640 1 -- 172.21.15.46:0/666658743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8103fc0 msgr2=0x7faab8076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.005+0000 7faabfc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8103fc0 0x7faab8076700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faaac01fa30 tx=0x7faaac01fef0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.006+0000 7faabfc00640 1 -- 172.21.15.46:0/666658743 shutdown_connections 2024-09-10T07:39:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.006+0000 7faabfc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faab81090f0 0x7faab8077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.006+0000 7faabfc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faaa00a2d10 0x7faaa00a51d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.006+0000 7faabfc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab8104970 0x7faab8076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.006+0000 7faabfc00640 1 --2- 172.21.15.46:0/666658743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faab8103fc0 0x7faab8076700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.006+0000 7faabfc00640 1 -- 172.21.15.46:0/666658743 >> 172.21.15.46:0/666658743 conn(0x7faab80ffa40 msgr2=0x7faab8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.007+0000 7faabfc00640 1 -- 172.21.15.46:0/666658743 shutdown_connections 2024-09-10T07:39:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:21.007+0000 7faabfc00640 1 -- 172.21.15.46:0/666658743 wait complete. 2024-09-10T07:39:22.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:22 smithi181 ceph-mon[35397]: pgmap v481: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 680 op/s 2024-09-10T07:39:22.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:22 smithi063 ceph-mon[35020]: pgmap v481: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 680 op/s 2024-09-10T07:39:22.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:22 smithi046 ceph-mon[30705]: pgmap v481: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 680 op/s 2024-09-10T07:39:23.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:39:23.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:39:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:24.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:24 smithi181 ceph-mon[35397]: pgmap v482: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 650 op/s 2024-09-10T07:39:24.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:24 smithi063 ceph-mon[35020]: pgmap v482: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 650 op/s 2024-09-10T07:39:24.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:24 smithi046 ceph-mon[30705]: pgmap v482: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 650 op/s 2024-09-10T07:39:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 125.0 s, 1852.2 tps, lat 53.550 ms stddev 64.897 2024-09-10T07:39:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.213+0000 7f95e9a00640 1 -- 172.21.15.46:0/2613272649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e410e280 msgr2=0x7f95e4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.213+0000 7f95e9a00640 1 --2- 172.21.15.46:0/2613272649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e410e280 0x7f95e4110670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009900 tx=0x7f95dc02a490 comp rx=0 tx=0).stop 2024-09-10T07:39:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.214+0000 7f95e9a00640 1 -- 172.21.15.46:0/2613272649 shutdown_connections 2024-09-10T07:39:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.214+0000 7f95e9a00640 1 --2- 172.21.15.46:0/2613272649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e410e280 0x7f95e4110670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.214+0000 7f95e9a00640 1 --2- 172.21.15.46:0/2613272649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95e4074e30 0x7f95e410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.214+0000 7f95e9a00640 1 --2- 172.21.15.46:0/2613272649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e4074510 0x7f95e40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.214+0000 7f95e9a00640 1 -- 172.21.15.46:0/2613272649 >> 172.21.15.46:0/2613272649 conn(0x7f95e40fd740 msgr2=0x7f95e40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:26.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.214+0000 7f95e9a00640 1 -- 172.21.15.46:0/2613272649 shutdown_connections 2024-09-10T07:39:26.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.214+0000 7f95e9a00640 1 -- 172.21.15.46:0/2613272649 wait complete. 2024-09-10T07:39:26.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.215+0000 7f95e9a00640 1 Processor -- start 2024-09-10T07:39:26.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.216+0000 7f95e9a00640 1 -- start start 2024-09-10T07:39:26.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.216+0000 7f95e9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e4074510 0x7f95e419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e4074e30 0x7f95e419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95e410e280 0x7f95e419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4113fe0 con 0x7f95e4074e30 2024-09-10T07:39:26.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e9a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4113e60 con 0x7f95e410e280 2024-09-10T07:39:26.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e9a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4114160 con 0x7f95e4074510 2024-09-10T07:39:26.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e4074510 0x7f95e419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e4074510 0x7f95e419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57946/0 (socket says 172.21.15.46:57946) 2024-09-10T07:39:26.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e3e00640 1 -- 172.21.15.46:0/1951400550 learned_addr learned my addr 172.21.15.46:0/1951400550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:26.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.217+0000 7f95e9000640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95e410e280 0x7f95e419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.218+0000 7f95db400640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e4074e30 0x7f95e419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.218+0000 7f95e3e00640 1 -- 172.21.15.46:0/1951400550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95e410e280 msgr2=0x7f95e419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.218+0000 7f95e3e00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95e410e280 0x7f95e419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.218+0000 7f95e3e00640 1 -- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e4074e30 msgr2=0x7f95e419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.218+0000 7f95e3e00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e4074e30 0x7f95e419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.218+0000 7f95e3e00640 1 -- 172.21.15.46:0/1951400550 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e419f390 con 0x7f95e4074510 2024-09-10T07:39:26.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.218+0000 7f95e3e00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e4074510 0x7f95e419a3e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95d400cce0 tx=0x7f95d400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:26.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.219+0000 7f95e1600640 1 -- 172.21.15.46:0/1951400550 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d400f650 con 0x7f95e4074510 2024-09-10T07:39:26.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.219+0000 7f95e1600640 1 -- 172.21.15.46:0/1951400550 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d400fcd0 con 0x7f95e4074510 2024-09-10T07:39:26.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.219+0000 7f95e1600640 1 -- 172.21.15.46:0/1951400550 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4018c70 con 0x7f95e4074510 2024-09-10T07:39:26.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.219+0000 7f95e9a00640 1 -- 172.21.15.46:0/1951400550 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e419f680 con 0x7f95e4074510 2024-09-10T07:39:26.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.219+0000 7f95e9a00640 1 -- 172.21.15.46:0/1951400550 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e41a6f60 con 0x7f95e4074510 2024-09-10T07:39:26.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.225+0000 7f95e1600640 1 -- 172.21.15.46:0/1951400550 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f95d4004b20 con 0x7f95e4074510 2024-09-10T07:39:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95edc07740 1 Processor -- start 2024-09-10T07:39:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95edc07740 1 -- start start 2024-09-10T07:39:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95edc07740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cececb0 0x563b4cecf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95edc07740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4ced9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95edc07740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced9e90 0x563b4cedc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95edc07740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563b4ce35290 con 0x563b4cececb0 2024-09-10T07:39:26.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95edc07740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563b4cddcb40 con 0x563b4ced9e90 2024-09-10T07:39:26.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95edc07740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563b4ce31250 con 0x563b4cecf5d0 2024-09-10T07:39:26.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.226+0000 7f95e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4ced9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4ced9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57958/0 (socket says 172.21.15.46:57958) 2024-09-10T07:39:26.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced9e90 0x563b4cedc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95e3e00640 1 -- 172.21.15.46:0/804796673 learned_addr learned my addr 172.21.15.46:0/804796673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:26.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95db400640 1 -- 172.21.15.46:0/804796673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 msgr2=0x563b4ced9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:26.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95e1600640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f95c80a2cd0 0x7f95c80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95db400640 1 --2- 172.21.15.46:0/804796673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4ced9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95db400640 1 -- 172.21.15.46:0/804796673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cececb0 msgr2=0x563b4cecf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:26.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95db400640 1 --2- 172.21.15.46:0/804796673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cececb0 0x563b4cecf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95db400640 1 -- 172.21.15.46:0/804796673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4cba66a0 con 0x563b4ced9e90 2024-09-10T07:39:26.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95db400640 1 --2- 172.21.15.46:0/804796673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced9e90 0x563b4cedc280 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00bb30 tx=0x7f95cc00f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bcd8818560e90d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:26.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.227+0000 7f95e1600640 1 -- 172.21.15.46:0/1951400550 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f95d40cf600 con 0x7f95e4074510 2024-09-10T07:39:26.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.228+0000 7f95e9000640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f95c80a2cd0 0x7f95c80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.229+0000 7f95d9600640 1 -- 172.21.15.46:0/804796673 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc015070 con 0x563b4ced9e90 2024-09-10T07:39:26.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.229+0000 7f95d9600640 1 -- 172.21.15.46:0/804796673 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc00fd90 con 0x563b4ced9e90 2024-09-10T07:39:26.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.229+0000 7f95d9600640 1 -- 172.21.15.46:0/804796673 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc011440 con 0x563b4ced9e90 2024-09-10T07:39:26.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.230+0000 7f95edc07740 1 -- 172.21.15.46:0/804796673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced9e90 msgr2=0x563b4cedc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.230+0000 7f95edc07740 1 --2- 172.21.15.46:0/804796673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced9e90 0x563b4cedc280 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00bb30 tx=0x7f95cc00f360 comp rx=0 tx=0).stop 2024-09-10T07:39:26.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.230+0000 7f95e9000640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f95c80a2cd0 0x7f95c80a5190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f95dc004770 tx=0x7f95dc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:26.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.230+0000 7f95edc07740 1 -- 172.21.15.46:0/804796673 shutdown_connections 2024-09-10T07:39:26.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.230+0000 7f95edc07740 1 --2- 172.21.15.46:0/804796673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced9e90 0x563b4cedc280 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.230+0000 7f95edc07740 1 --2- 172.21.15.46:0/804796673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4ced9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.230+0000 7f95edc07740 1 --2- 172.21.15.46:0/804796673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cececb0 0x563b4cecf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.230+0000 7f95edc07740 1 -- 172.21.15.46:0/804796673 >> 172.21.15.46:0/804796673 conn(0x563b4cecb240 msgr2=0x563b4cecb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:26.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.231+0000 7f95edc07740 1 -- 172.21.15.46:0/804796673 shutdown_connections 2024-09-10T07:39:26.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.231+0000 7f95edc07740 1 -- 172.21.15.46:0/804796673 wait complete. 2024-09-10T07:39:26.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.231+0000 7f95edc07740 1 -- start start 2024-09-10T07:39:26.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.232+0000 7f95edc07740 1 Processor -- start 2024-09-10T07:39:26.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.232+0000 7f95edc07740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4ced6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.232+0000 7f95edc07740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 0x563b4ced71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.232+0000 7f95edc07740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4cee6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.232+0000 7f95edc07740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563b4cedcb00 con 0x563b4cecf5d0 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.232+0000 7f95edc07740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563b4cdeffa0 con 0x563b4ced76f0 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.232+0000 7f95edc07740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563b4cedcc80 con 0x563b4cececb0 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.232+0000 7f95edc07740 10 client.0 _ll_register_callbacks cb 0x563b4cb43ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95e9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4cee6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4ced6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95e9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4cee6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45118/0 (socket says 172.21.15.46:45118) 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95e9000640 1 -- 172.21.15.46:0/2393195603 learned_addr learned my addr 172.21.15.46:0/2393195603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95db400640 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 0x563b4ced71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95e9000640 1 -- 172.21.15.46:0/2393195603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 msgr2=0x563b4ced6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:26.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95e9000640 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4ced6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95e9000640 1 -- 172.21.15.46:0/2393195603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 msgr2=0x563b4ced71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:26.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.233+0000 7f95e9000640 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 0x563b4ced71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.234+0000 7f95e9000640 1 -- 172.21.15.46:0/2393195603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4cee6540 con 0x563b4ced76f0 2024-09-10T07:39:26.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.234+0000 7f95e9000640 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4cee6000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f95dc033000 tx=0x7f95dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da1f2b41802875a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.234+0000 7f95d2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.235+0000 7f95d2000640 1 -- 172.21.15.46:0/2393195603 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0315b0 con 0x563b4ced76f0 2024-09-10T07:39:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.235+0000 7f95d2000640 1 -- 172.21.15.46:0/2393195603 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f95dc031750 con 0x563b4ced76f0 2024-09-10T07:39:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.235+0000 7f95edc07740 1 -- 172.21.15.46:0/2393195603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95cc00fd90 con 0x563b4ced76f0 2024-09-10T07:39:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.236+0000 7f95d2000640 1 -- 172.21.15.46:0/2393195603 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f95dc031950 con 0x563b4ced76f0 2024-09-10T07:39:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.236+0000 7f95edc07740 10 client.26123 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.236+0000 7f95edc07740 1 -- 172.21.15.46:0/2393195603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563b4cee9060 con 0x563b4ced76f0 2024-09-10T07:39:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.236+0000 7f95d2000640 1 -- 172.21.15.46:0/2393195603 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc00fd90 con 0x563b4ced76f0 2024-09-10T07:39:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.237+0000 7f95d2000640 1 -- 172.21.15.46:0/2393195603 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f95dc043030 con 0x563b4ced76f0 2024-09-10T07:39:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.237+0000 7f95edc07740 10 client.26123 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:26.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.237+0000 7f95edc07740 10 client.26123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:26.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.237+0000 7f95edc07740 20 client.26123 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:26.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.237+0000 7f95edc07740 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563b4ceea5c0 0x563b4ceec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.237+0000 7f95edc07740 4 client.26123 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:26.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.237+0000 7f95edc07740 1 -- 172.21.15.46:0/2393195603 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b4ce42800 con 0x563b4ceea5c0 2024-09-10T07:39:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.238+0000 7f95e3e00640 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563b4ceea5c0 0x563b4ceec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.239+0000 7f95e3e00640 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563b4ceea5c0 0x563b4ceec9d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464501899fbf6bd server_cookie=8c2a91e34cf7afef in_seq=0 out_seq=0 2024-09-10T07:39:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.239+0000 7f95d2000640 10 client.26123 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.240+0000 7f95d2000640 1 -- 172.21.15.46:0/2393195603 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b4ce42800 con 0x563b4ceea5c0 2024-09-10T07:39:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.240+0000 7f95d2000640 10 client.26123 handle_command_reply: tid=0 2024-09-10T07:39:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.241+0000 7f95edc07740 1 client.26123 shutdown 2024-09-10T07:39:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 -- 172.21.15.46:0/2393195603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 msgr2=0x563b4cee6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4cee6000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f95dc033000 tx=0x7f95dc0046c0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 -- 172.21.15.46:0/2393195603 shutdown_connections 2024-09-10T07:39:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563b4ceea5c0 0x563b4ceec9d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4cee6000 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 0x563b4ced71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 --2- 172.21.15.46:0/2393195603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4ced6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 -- 172.21.15.46:0/2393195603 >> 172.21.15.46:0/2393195603 conn(0x563b4cecb240 msgr2=0x563b4ced0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95d2000640 0 client.26123 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 -- 172.21.15.46:0/2393195603 shutdown_connections 2024-09-10T07:39:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.242+0000 7f95edc07740 1 -- 172.21.15.46:0/2393195603 wait complete. 2024-09-10T07:39:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.243+0000 7f95edc07740 20 client.26123 trim_cache size 0 max 16384 2024-09-10T07:39:26.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.272+0000 7f95edc07740 1 Processor -- start 2024-09-10T07:39:26.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.272+0000 7f95edc07740 1 -- start start 2024-09-10T07:39:26.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.272+0000 7f95edc07740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4cef22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.272+0000 7f95edc07740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 0x563b4cef2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.272+0000 7f95edc07740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4ced3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.272+0000 7f95edc07740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95c000fb10 con 0x563b4cecf5d0 2024-09-10T07:39:26.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.272+0000 7f95edc07740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0011770 con 0x563b4ced76f0 2024-09-10T07:39:26.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.272+0000 7f95edc07740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95c00101e0 con 0x563b4cececb0 2024-09-10T07:39:26.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4cef22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4ced3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4ced3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45130/0 (socket says 172.21.15.46:45130) 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4cef22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57974/0 (socket says 172.21.15.46:57974) 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e9000640 1 -- 172.21.15.46:0/315508076 learned_addr learned my addr 172.21.15.46:0/315508076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e9000640 1 -- 172.21.15.46:0/315508076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 msgr2=0x563b4cef22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e9000640 1 --2- 172.21.15.46:0/315508076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4cef22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e9000640 1 -- 172.21.15.46:0/315508076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 msgr2=0x563b4cef2820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.273+0000 7f95e9000640 1 --2- 172.21.15.46:0/315508076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 0x563b4cef2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.274+0000 7f95e9000640 1 -- 172.21.15.46:0/315508076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4ced9a60 con 0x563b4ced76f0 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.274+0000 7f95e9000640 1 --2- 172.21.15.46:0/315508076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4ced3ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95dc02aa80 tx=0x7f95dc003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c9d57d2c96cbaff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.274+0000 7f95e3e00640 1 --2- 172.21.15.46:0/315508076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4cef22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T07:39:26.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.275+0000 7f95d8c00640 1 -- 172.21.15.46:0/315508076 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0317b0 con 0x563b4ced76f0 2024-09-10T07:39:26.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.275+0000 7f95d8c00640 1 -- 172.21.15.46:0/315508076 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc04b070 con 0x563b4ced76f0 2024-09-10T07:39:26.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.275+0000 7f95d8c00640 1 -- 172.21.15.46:0/315508076 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc03c440 con 0x563b4ced76f0 2024-09-10T07:39:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.277+0000 7f95edc07740 1 -- 172.21.15.46:0/315508076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 msgr2=0x563b4ced3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.277+0000 7f95edc07740 1 --2- 172.21.15.46:0/315508076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4ced3ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95dc02aa80 tx=0x7f95dc003af0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.277+0000 7f95edc07740 1 -- 172.21.15.46:0/315508076 shutdown_connections 2024-09-10T07:39:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.278+0000 7f95edc07740 1 --2- 172.21.15.46:0/315508076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4ced76f0 0x563b4ced3ed0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.278+0000 7f95edc07740 1 --2- 172.21.15.46:0/315508076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4cecf5d0 0x563b4cef2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.278+0000 7f95edc07740 1 --2- 172.21.15.46:0/315508076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cececb0 0x563b4cef22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.278+0000 7f95edc07740 1 -- 172.21.15.46:0/315508076 >> 172.21.15.46:0/315508076 conn(0x563b4cecb240 msgr2=0x563b4ceca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.279+0000 7f95edc07740 1 -- 172.21.15.46:0/315508076 shutdown_connections 2024-09-10T07:39:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.280+0000 7f95edc07740 1 -- 172.21.15.46:0/315508076 wait complete. 2024-09-10T07:39:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.281+0000 7f95edc07740 1 -- start start 2024-09-10T07:39:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.282+0000 7f95edc07740 1 Processor -- start 2024-09-10T07:39:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.285+0000 7f95edc07740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4cececb0 0x563b4cf502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.285+0000 7f95edc07740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4cf507e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.285+0000 7f95edc07740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4ced76f0 0x563b4cf50d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.285+0000 7f95edc07740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0010fe0 con 0x563b4ced76f0 2024-09-10T07:39:26.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.285+0000 7f95edc07740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0010910 con 0x563b4cececb0 2024-09-10T07:39:26.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.285+0000 7f95edc07740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95c000daf0 con 0x563b4cecf5d0 2024-09-10T07:39:26.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.285+0000 7f95edc07740 10 client.0 _ll_register_callbacks cb 0x7f95dc0314b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:26.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.286+0000 7f95db400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4cf507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.286+0000 7f95db400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4cf507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57984/0 (socket says 172.21.15.46:57984) 2024-09-10T07:39:26.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.286+0000 7f95db400640 1 -- 172.21.15.46:0/3528505518 learned_addr learned my addr 172.21.15.46:0/3528505518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:26.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.287+0000 7f95e3e00640 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4cececb0 0x563b4cf502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.287+0000 7f95e9000640 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4ced76f0 0x563b4cf50d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95e3e00640 1 -- 172.21.15.46:0/3528505518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 msgr2=0x563b4cf507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95e3e00640 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4cf507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95e3e00640 1 -- 172.21.15.46:0/3528505518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4ced76f0 msgr2=0x563b4cf50d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95e3e00640 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4ced76f0 0x563b4cf50d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95e3e00640 1 -- 172.21.15.46:0/3528505518 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4cee01c0 con 0x563b4cececb0 2024-09-10T07:39:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95e3e00640 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4cececb0 0x563b4cf502a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95d400afd0 tx=0x7f95d40d5320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8200b3b9f068259 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95d2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95d2000640 1 -- 172.21.15.46:0/3528505518 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d409c640 con 0x563b4cececb0 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95d2000640 1 -- 172.21.15.46:0/3528505518 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f95d40cf320 con 0x563b4cececb0 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.290+0000 7f95edc07740 1 -- 172.21.15.46:0/3528505518 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95dc04b070 con 0x563b4cececb0 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95d2000640 1 -- 172.21.15.46:0/3528505518 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f95d409d3e0 con 0x563b4cececb0 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95edc07740 10 client.26129 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95edc07740 1 -- 172.21.15.46:0/3528505518 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563b4cf66750 con 0x563b4cececb0 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95d2000640 1 -- 172.21.15.46:0/3528505518 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc04b070 con 0x563b4cececb0 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95d2000640 1 -- 172.21.15.46:0/3528505518 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f95d4099350 con 0x563b4cececb0 2024-09-10T07:39:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95edc07740 10 client.26129 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95edc07740 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95edc07740 20 client.26129 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95edc07740 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563b4cf66f30 0x563b4cf69350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95edc07740 4 client.26129 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:39:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.291+0000 7f95edc07740 1 -- 172.21.15.46:0/3528505518 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563b4ce3cd10 con 0x563b4cf66f30 2024-09-10T07:39:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.292+0000 7f95db400640 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563b4cf66f30 0x563b4cf69350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.293+0000 7f95db400640 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563b4cf66f30 0x563b4cf69350 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a859306a4ea2d8 server_cookie=fcb19f2fefd2b18 in_seq=0 out_seq=0 2024-09-10T07:39:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.294+0000 7f95d2000640 10 client.26129 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.297+0000 7f95d2000640 1 -- 172.21.15.46:0/3528505518 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563b4ce3cd10 con 0x563b4cf66f30 2024-09-10T07:39:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.297+0000 7f95d2000640 10 client.26129 handle_command_reply: tid=0 2024-09-10T07:39:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.297+0000 7f95edc07740 1 client.26129 shutdown 2024-09-10T07:39:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.297+0000 7f95edc07740 1 -- 172.21.15.46:0/3528505518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4cececb0 msgr2=0x563b4cf502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.297+0000 7f95edc07740 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4cececb0 0x563b4cf502a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95d400afd0 tx=0x7f95d40d5320 comp rx=0 tx=0).stop 2024-09-10T07:39:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.298+0000 7f95edc07740 1 -- 172.21.15.46:0/3528505518 shutdown_connections 2024-09-10T07:39:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.298+0000 7f95edc07740 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563b4cf66f30 0x563b4cf69350 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.298+0000 7f95edc07740 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b4ced76f0 0x563b4cf50d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.298+0000 7f95edc07740 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563b4cecf5d0 0x563b4cf507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.298+0000 7f95edc07740 1 --2- 172.21.15.46:0/3528505518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b4cececb0 0x563b4cf502a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.298+0000 7f95edc07740 1 -- 172.21.15.46:0/3528505518 >> 172.21.15.46:0/3528505518 conn(0x563b4cecb240 msgr2=0x563b4ced84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:26.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.298+0000 7f95d2000640 0 client.26129 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.299+0000 7f95edc07740 1 -- 172.21.15.46:0/3528505518 shutdown_connections 2024-09-10T07:39:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.300+0000 7f95edc07740 1 -- 172.21.15.46:0/3528505518 wait complete. 2024-09-10T07:39:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.300+0000 7f95edc07740 20 client.26129 trim_cache size 0 max 16384 2024-09-10T07:39:26.301 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:39:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:39:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "64b69cc7-deac-4d06-b271-47a425290bd1", 2024-09-10T07:39:26.301 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:39:26.301 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:39:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 -- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f95c80a2cd0 msgr2=0x7f95c80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f95c80a2cd0 0x7f95c80a5190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f95dc004770 tx=0x7f95dc005fd0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 -- 172.21.15.46:0/1951400550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e4074510 msgr2=0x7f95e419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e4074510 0x7f95e419a3e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95d400cce0 tx=0x7f95d400a620 comp rx=0 tx=0).stop 2024-09-10T07:39:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 -- 172.21.15.46:0/1951400550 shutdown_connections 2024-09-10T07:39:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95e410e280 0x7f95e419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f95c80a2cd0 0x7f95c80a5190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e4074e30 0x7f95e419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1951400550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e4074510 0x7f95e419a3e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.301+0000 7f95e9a00640 1 -- 172.21.15.46:0/1951400550 >> 172.21.15.46:0/1951400550 conn(0x7f95e40fd740 msgr2=0x7f95e40fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.302+0000 7f95e9a00640 1 -- 172.21.15.46:0/1951400550 shutdown_connections 2024-09-10T07:39:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:26.302+0000 7f95e9a00640 1 -- 172.21.15.46:0/1951400550 wait complete. 2024-09-10T07:39:26.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:26 smithi046 ceph-mon[30705]: pgmap v483: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 607 op/s 2024-09-10T07:39:26.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:26 smithi181 ceph-mon[35397]: pgmap v483: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 607 op/s 2024-09-10T07:39:26.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:26 smithi063 ceph-mon[35020]: pgmap v483: 129 pgs: 129 active+clean; 998 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 607 op/s 2024-09-10T07:39:27.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:39:27.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:27 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:39:27.901 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.902 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:27 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.912 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:27.912 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:27 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:28.801 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:28.802 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:28 smithi063 ceph-mon[35020]: pgmap v484: 129 pgs: 129 active+clean; 1000 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-09-10T07:39:28.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:28.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:28 smithi181 ceph-mon[35397]: pgmap v484: 129 pgs: 129 active+clean; 1000 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-09-10T07:39:28.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:39:28.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:28 smithi046 ceph-mon[30705]: pgmap v484: 129 pgs: 129 active+clean; 1000 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-09-10T07:39:29.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:39:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:39:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 130.0 s, 979.2 tps, lat 102.571 ms stddev 149.958 2024-09-10T07:39:30.765 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:30 smithi046 ceph-mon[30705]: pgmap v485: 129 pgs: 129 active+clean; 1000 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.2 MiB/s wr, 652 op/s 2024-09-10T07:39:30.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:30 smithi181 ceph-mon[35397]: pgmap v485: 129 pgs: 129 active+clean; 1000 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.2 MiB/s wr, 652 op/s 2024-09-10T07:39:30.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:30 smithi063 ceph-mon[35020]: pgmap v485: 129 pgs: 129 active+clean; 1000 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.2 MiB/s wr, 652 op/s 2024-09-10T07:39:31.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.144+0000 7f6d16a00640 1 -- 172.21.15.46:0/4031502378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 msgr2=0x7f6d1010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.144+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4031502378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 0x7f6d1010ea20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009890 tx=0x7f6d0002a360 comp rx=0 tx=0).stop 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.145+0000 7f6d16a00640 1 -- 172.21.15.46:0/4031502378 shutdown_connections 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.145+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4031502378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 0x7f6d1010ea20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.145+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4031502378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d100ffc00 0x7f6d10100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.145+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4031502378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d10106b80 0x7f6d10106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.145+0000 7f6d16a00640 1 -- 172.21.15.46:0/4031502378 >> 172.21.15.46:0/4031502378 conn(0x7f6d100fba90 msgr2=0x7f6d100fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.146+0000 7f6d16a00640 1 -- 172.21.15.46:0/4031502378 shutdown_connections 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.146+0000 7f6d16a00640 1 -- 172.21.15.46:0/4031502378 wait complete. 2024-09-10T07:39:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.147+0000 7f6d16a00640 1 Processor -- start 2024-09-10T07:39:31.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.148+0000 7f6d16a00640 1 -- start start 2024-09-10T07:39:31.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.148+0000 7f6d16a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d100ffc00 0x7f6d1010cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.148+0000 7f6d16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 0x7f6d1010d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d16a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d10106b80 0x7f6d101a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d16a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10111fa0 con 0x7f6d10106b80 2024-09-10T07:39:31.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d16a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10111e20 con 0x7f6d101005a0 2024-09-10T07:39:31.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d16a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10112120 con 0x7f6d100ffc00 2024-09-10T07:39:31.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 0x7f6d1010d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 0x7f6d1010d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45156/0 (socket says 172.21.15.46:45156) 2024-09-10T07:39:31.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0f400640 1 -- 172.21.15.46:0/4229459136 learned_addr learned my addr 172.21.15.46:0/4229459136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:31.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d100ffc00 0x7f6d1010cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0f400640 1 -- 172.21.15.46:0/4229459136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d100ffc00 msgr2=0x7f6d1010cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0f400640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d100ffc00 0x7f6d1010cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0f400640 1 -- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d10106b80 msgr2=0x7f6d101a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0f400640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d10106b80 0x7f6d101a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.149+0000 7f6d0f400640 1 -- 172.21.15.46:0/4229459136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d101a3a10 con 0x7f6d101005a0 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.150+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d100ffc00 0x7f6d1010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.150+0000 7f6d0f400640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 0x7f6d1010d0f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9f0 tx=0x7f6cfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.150+0000 7f6d0cc00640 1 -- 172.21.15.46:0/4229459136 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00cd80 con 0x7f6d101005a0 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.150+0000 7f6d0cc00640 1 -- 172.21.15.46:0/4229459136 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc010910 con 0x7f6d101005a0 2024-09-10T07:39:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.150+0000 7f6d16a00640 1 -- 172.21.15.46:0/4229459136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d101a3d00 con 0x7f6d101005a0 2024-09-10T07:39:31.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.150+0000 7f6d0cc00640 1 -- 172.21.15.46:0/4229459136 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc018990 con 0x7f6d101005a0 2024-09-10T07:39:31.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.150+0000 7f6d16a00640 1 -- 172.21.15.46:0/4229459136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d101ab5c0 con 0x7f6d101005a0 2024-09-10T07:39:31.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.156+0000 7f6d1944f740 1 Processor -- start 2024-09-10T07:39:31.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d1944f740 1 -- start start 2024-09-10T07:39:31.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb5ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb69830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb69d70 0x55ab3bb6c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d1944f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ab3bac53e0 con 0x55ab3bb5f4b0 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d1944f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ab3ba6cd80 con 0x55ab3bb5eb90 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d1944f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ab3bac13a0 con 0x55ab3bb69d70 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d14a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb5ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d14a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb5ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45158/0 (socket says 172.21.15.46:45158) 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.157+0000 7f6d14a00640 1 -- 172.21.15.46:0/3370834452 learned_addr learned my addr 172.21.15.46:0/3370834452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.158+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/3370834452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb69830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.158+0000 7f6d14a00640 1 -- 172.21.15.46:0/3370834452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb69d70 msgr2=0x55ab3bb6c160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.158+0000 7f6d14a00640 1 --2- 172.21.15.46:0/3370834452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb69d70 0x55ab3bb6c160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.158+0000 7f6d14a00640 1 -- 172.21.15.46:0/3370834452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 msgr2=0x55ab3bb69830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.158+0000 7f6d14a00640 1 --2- 172.21.15.46:0/3370834452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb69830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.158+0000 7f6d14a00640 1 -- 172.21.15.46:0/3370834452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3ba8bae0 con 0x55ab3bb5eb90 2024-09-10T07:39:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.159+0000 7f6d14a00640 1 --2- 172.21.15.46:0/3370834452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb5ef70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005b60 tx=0x7f6d00005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed7a57aa403d1ade server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.160+0000 7f6d0cc00640 1 -- 172.21.15.46:0/4229459136 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cfc010ac0 con 0x7f6d101005a0 2024-09-10T07:39:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.160+0000 7f6d05200640 1 -- 172.21.15.46:0/3370834452 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00037040 con 0x55ab3bb5eb90 2024-09-10T07:39:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.160+0000 7f6d05200640 1 -- 172.21.15.46:0/3370834452 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00040070 con 0x55ab3bb5eb90 2024-09-10T07:39:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.160+0000 7f6d05200640 1 -- 172.21.15.46:0/3370834452 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00037040 con 0x55ab3bb5eb90 2024-09-10T07:39:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.162+0000 7f6d1944f740 1 -- 172.21.15.46:0/3370834452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 msgr2=0x55ab3bb5ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.162+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3370834452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb5ef70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005b60 tx=0x7f6d00005c90 comp rx=0 tx=0).stop 2024-09-10T07:39:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.162+0000 7f6d0cc00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6cf40a2c80 0x7f6cf40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.162+0000 7f6d0cc00640 1 -- 172.21.15.46:0/4229459136 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6cfc0cf830 con 0x7f6d101005a0 2024-09-10T07:39:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.163+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6cf40a2c80 0x7f6cf40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.163+0000 7f6d1944f740 1 -- 172.21.15.46:0/3370834452 shutdown_connections 2024-09-10T07:39:31.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.163+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3370834452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb69d70 0x55ab3bb6c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.163+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3370834452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb69830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.163+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3370834452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb5ef70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.163+0000 7f6d1944f740 1 -- 172.21.15.46:0/3370834452 >> 172.21.15.46:0/3370834452 conn(0x55ab3bb5b120 msgr2=0x55ab3bb5b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:31.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.164+0000 7f6d1944f740 1 -- 172.21.15.46:0/3370834452 shutdown_connections 2024-09-10T07:39:31.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.164+0000 7f6d1944f740 1 -- 172.21.15.46:0/3370834452 wait complete. 2024-09-10T07:39:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.164+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6cf40a2c80 0x7f6cf40a5140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55ab3bb5bf80 tx=0x7f6d080095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.165+0000 7f6d1944f740 1 -- start start 2024-09-10T07:39:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.165+0000 7f6d1944f740 1 Processor -- start 2024-09-10T07:39:31.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb67660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d1944f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ab3bb7d9f0 con 0x55ab3bb5f4b0 2024-09-10T07:39:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d1944f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ab3ba800f0 con 0x55ab3bb69d70 2024-09-10T07:39:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d1944f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ab3bb7db70 con 0x55ab3bb5eb90 2024-09-10T07:39:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d1944f740 10 client.0 _ll_register_callbacks cb 0x55ab3ba719e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d0fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d0fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb66be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58014/0 (socket says 172.21.15.46:58014) 2024-09-10T07:39:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d0fe00640 1 -- 172.21.15.46:0/2225722226 learned_addr learned my addr 172.21.15.46:0/2225722226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d14a00640 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb67660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d0fe00640 1 -- 172.21.15.46:0/2225722226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 msgr2=0x55ab3bb67660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:31.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb67660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d0fe00640 1 -- 172.21.15.46:0/2225722226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 msgr2=0x55ab3bb67120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:31.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.166+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.167+0000 7f6d0fe00640 1 -- 172.21.15.46:0/2225722226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3b8363c0 con 0x55ab3bb5eb90 2024-09-10T07:39:31.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.167+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb66be0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6d08005c40 tx=0x7f6d08004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3473b4505eeda27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:31.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.167+0000 7f6cf9600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:39:31.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.167+0000 7f6cf9600640 1 -- 172.21.15.46:0/2225722226 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0802c060 con 0x55ab3bb5eb90 2024-09-10T07:39:31.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.167+0000 7f6cf9600640 1 -- 172.21.15.46:0/2225722226 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6d08016030 con 0x55ab3bb5eb90 2024-09-10T07:39:31.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.168+0000 7f6d1944f740 1 -- 172.21.15.46:0/2225722226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d00040070 con 0x55ab3bb5eb90 2024-09-10T07:39:31.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.168+0000 7f6cf9600640 1 -- 172.21.15.46:0/2225722226 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6d0801c920 con 0x55ab3bb5eb90 2024-09-10T07:39:31.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.168+0000 7f6d1944f740 10 client.26134 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:31.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.168+0000 7f6d1944f740 1 -- 172.21.15.46:0/2225722226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ab3bb896b0 con 0x55ab3bb5eb90 2024-09-10T07:39:31.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.170+0000 7f6cf9600640 1 -- 172.21.15.46:0/2225722226 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00040070 con 0x55ab3bb5eb90 2024-09-10T07:39:31.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.170+0000 7f6cf9600640 1 -- 172.21.15.46:0/2225722226 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f6d0800b040 con 0x55ab3bb5eb90 2024-09-10T07:39:31.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.170+0000 7f6d1944f740 10 client.26134 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:31.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.170+0000 7f6d1944f740 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:31.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.170+0000 7f6d1944f740 20 client.26134 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:31.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.170+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ab3bb8ab10 0x55ab3bb8cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.170+0000 7f6d1944f740 4 client.26134 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:31.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.170+0000 7f6d1944f740 1 -- 172.21.15.46:0/2225722226 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab3bad2950 con 0x55ab3bb8ab10 2024-09-10T07:39:31.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.171+0000 7f6d0f400640 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ab3bb8ab10 0x55ab3bb8cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.172+0000 7f6d0f400640 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ab3bb8ab10 0x55ab3bb8cfb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc6a7de39552a0d server_cookie=4198168ec982ba92 in_seq=0 out_seq=0 2024-09-10T07:39:31.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.172+0000 7f6cf9600640 10 client.26134 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:31.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.174+0000 7f6cf9600640 1 -- 172.21.15.46:0/2225722226 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab3bad2950 con 0x55ab3bb8ab10 2024-09-10T07:39:31.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.175+0000 7f6cf9600640 10 client.26134 handle_command_reply: tid=0 2024-09-10T07:39:31.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.175+0000 7f6d1944f740 1 client.26134 shutdown 2024-09-10T07:39:31.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 -- 172.21.15.46:0/2225722226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5eb90 msgr2=0x55ab3bb66be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb66be0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6d08005c40 tx=0x7f6d08004ab0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 -- 172.21.15.46:0/2225722226 shutdown_connections 2024-09-10T07:39:31.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ab3bb8ab10 0x55ab3bb8cfb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb67660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6cf9600640 0 client.26134 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2225722226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb66be0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 -- 172.21.15.46:0/2225722226 >> 172.21.15.46:0/2225722226 conn(0x55ab3bb5b120 msgr2=0x55ab3bb60e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 -- 172.21.15.46:0/2225722226 shutdown_connections 2024-09-10T07:39:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 1 -- 172.21.15.46:0/2225722226 wait complete. 2024-09-10T07:39:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.176+0000 7f6d1944f740 20 client.26134 trim_cache size 0 max 16384 2024-09-10T07:39:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.203+0000 7f6d1944f740 1 Processor -- start 2024-09-10T07:39:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.203+0000 7f6d1944f740 1 -- start start 2024-09-10T07:39:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.204+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.204+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.204+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb92e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.204+0000 7f6d1944f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101acb10 con 0x55ab3bb5eb90 2024-09-10T07:39:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.204+0000 7f6d1944f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101ae770 con 0x55ab3bb69d70 2024-09-10T07:39:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.204+0000 7f6d1944f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101ad1e0 con 0x55ab3bb5f4b0 2024-09-10T07:39:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.205+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.205+0000 7f6d14a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb92e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.205+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb928c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58030/0 (socket says 172.21.15.46:58030) 2024-09-10T07:39:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.205+0000 7f6d0f400640 1 -- 172.21.15.46:0/2394233931 learned_addr learned my addr 172.21.15.46:0/2394233931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.206+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/2394233931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.206+0000 7f6d14a00640 1 -- 172.21.15.46:0/2394233931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 msgr2=0x55ab3bb928c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.206+0000 7f6d14a00640 1 --2- 172.21.15.46:0/2394233931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb928c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.206+0000 7f6d14a00640 1 -- 172.21.15.46:0/2394233931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5eb90 msgr2=0x55ab3bb674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.206+0000 7f6d14a00640 1 --2- 172.21.15.46:0/2394233931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb674b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.206+0000 7f6d14a00640 1 -- 172.21.15.46:0/2394233931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3bb5af50 con 0x55ab3bb69d70 2024-09-10T07:39:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.207+0000 7f6d14a00640 1 --2- 172.21.15.46:0/2394233931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb92e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009890 tx=0x7f6d0002aaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef060c44300a1ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.207+0000 7f6cfbe00640 1 -- 172.21.15.46:0/2394233931 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00048070 con 0x55ab3bb69d70 2024-09-10T07:39:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.207+0000 7f6cfbe00640 1 -- 172.21.15.46:0/2394233931 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00040040 con 0x55ab3bb69d70 2024-09-10T07:39:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.207+0000 7f6cfbe00640 1 -- 172.21.15.46:0/2394233931 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00037040 con 0x55ab3bb69d70 2024-09-10T07:39:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.208+0000 7f6d1944f740 1 -- 172.21.15.46:0/2394233931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 msgr2=0x55ab3bb92e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.208+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2394233931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb92e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009890 tx=0x7f6d0002aaa0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.209+0000 7f6d1944f740 1 -- 172.21.15.46:0/2394233931 shutdown_connections 2024-09-10T07:39:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.209+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2394233931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb69d70 0x55ab3bb92e00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.209+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2394233931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.209+0000 7f6d1944f740 1 --2- 172.21.15.46:0/2394233931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.209+0000 7f6d1944f740 1 -- 172.21.15.46:0/2394233931 >> 172.21.15.46:0/2394233931 conn(0x55ab3bb5b120 msgr2=0x55ab3bb60e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.209+0000 7f6d1944f740 1 -- 172.21.15.46:0/2394233931 shutdown_connections 2024-09-10T07:39:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.209+0000 7f6d1944f740 1 -- 172.21.15.46:0/2394233931 wait complete. 2024-09-10T07:39:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.211+0000 7f6d1944f740 1 -- start start 2024-09-10T07:39:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.211+0000 7f6d1944f740 1 Processor -- start 2024-09-10T07:39:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.213+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb83700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.213+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb63a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.213+0000 7f6d1944f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb69d70 0x55ab3bb641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.214+0000 7f6d0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb83700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.214+0000 7f6d0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb83700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45190/0 (socket says 172.21.15.46:45190) 2024-09-10T07:39:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.215+0000 7f6d14a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb69d70 0x55ab3bb641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.215+0000 7f6d14a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb69d70 0x55ab3bb641b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34358/0 (socket says 172.21.15.46:34358) 2024-09-10T07:39:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.215+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb63a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.215+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb63a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58036/0 (socket says 172.21.15.46:58036) 2024-09-10T07:39:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.215+0000 7f6d0fe00640 1 -- 172.21.15.46:0/3289681698 learned_addr learned my addr 172.21.15.46:0/3289681698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.216+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101adfe0 con 0x55ab3bb69d70 2024-09-10T07:39:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.216+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101ad910 con 0x55ab3bb5eb90 2024-09-10T07:39:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.216+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10100060 con 0x55ab3bb5f4b0 2024-09-10T07:39:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.216+0000 7f6d1944f740 10 client.0 _ll_register_callbacks cb 0x7f6d0800ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.217+0000 7f6d0fe00640 1 -- 172.21.15.46:0/3289681698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 msgr2=0x55ab3bb63a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.217+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb63a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.217+0000 7f6d0fe00640 1 -- 172.21.15.46:0/3289681698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb69d70 msgr2=0x55ab3bb641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.217+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb69d70 0x55ab3bb641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.217+0000 7f6d0fe00640 1 -- 172.21.15.46:0/3289681698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3bbf4890 con 0x55ab3bb5eb90 2024-09-10T07:39:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.218+0000 7f6d0fe00640 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb83700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6d080096c0 tx=0x7f6d08030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2223b37e2363ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.218+0000 7f6cf9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.219+0000 7f6cf9600640 1 -- 172.21.15.46:0/3289681698 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0803f060 con 0x55ab3bb5eb90 2024-09-10T07:39:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.219+0000 7f6cf9600640 1 -- 172.21.15.46:0/3289681698 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6d08030a50 con 0x55ab3bb5eb90 2024-09-10T07:39:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.219+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d00040040 con 0x55ab3bb5eb90 2024-09-10T07:39:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.220+0000 7f6cf9600640 1 -- 172.21.15.46:0/3289681698 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6d0802c030 con 0x55ab3bb5eb90 2024-09-10T07:39:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.220+0000 7f6d1944f740 10 client.26147 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.220+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ab3bbf59e0 con 0x55ab3bb5eb90 2024-09-10T07:39:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.222+0000 7f6cf9600640 1 -- 172.21.15.46:0/3289681698 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00040040 con 0x55ab3bb5eb90 2024-09-10T07:39:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.222+0000 7f6cf9600640 1 -- 172.21.15.46:0/3289681698 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f6d08042050 con 0x55ab3bb5eb90 2024-09-10T07:39:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.222+0000 7f6d1944f740 10 client.26147 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.222+0000 7f6d1944f740 10 client.26147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.222+0000 7f6d1944f740 20 client.26147 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.222+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ab3bbf61c0 0x55ab3bbf8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.222+0000 7f6d1944f740 4 client.26147 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:39:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.222+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab3bacce60 con 0x55ab3bbf61c0 2024-09-10T07:39:31.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.223+0000 7f6d0f400640 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ab3bbf61c0 0x55ab3bbf8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:31.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.224+0000 7f6d0f400640 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ab3bbf61c0 0x55ab3bbf8600 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0e515e4fcc918 server_cookie=8befe768b6c5b8ca in_seq=0 out_seq=0 2024-09-10T07:39:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.224+0000 7f6cf9600640 10 client.26147 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:31.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.225+0000 7f6cf9600640 1 -- 172.21.15.46:0/3289681698 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab3bacce60 con 0x55ab3bbf61c0 2024-09-10T07:39:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.225+0000 7f6cf9600640 10 client.26147 handle_command_reply: tid=0 2024-09-10T07:39:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.225+0000 7f6d1944f740 1 client.26147 shutdown 2024-09-10T07:39:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 msgr2=0x55ab3bb83700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb83700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6d080096c0 tx=0x7f6d08030650 comp rx=0 tx=0).stop 2024-09-10T07:39:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 shutdown_connections 2024-09-10T07:39:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ab3bbf61c0 0x55ab3bbf8600 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab3bb69d70 0x55ab3bb641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3bb5f4b0 0x55ab3bb63a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6d1944f740 1 --2- 172.21.15.46:0/3289681698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3bb5eb90 0x55ab3bb83700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 >> 172.21.15.46:0/3289681698 conn(0x55ab3bb5b120 msgr2=0x55ab3bb599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.226+0000 7f6cf9600640 0 client.26147 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.227+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 shutdown_connections 2024-09-10T07:39:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.227+0000 7f6d1944f740 1 -- 172.21.15.46:0/3289681698 wait complete. 2024-09-10T07:39:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.227+0000 7f6d1944f740 20 client.26147 trim_cache size 0 max 16384 2024-09-10T07:39:31.228 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:39:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:39:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:39:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "64b69cc7-deac-4d06-b271-47a425290bd1": { 2024-09-10T07:39:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:39:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "64b69cc7-deac-4d06-b271-47a425290bd1", 2024-09-10T07:39:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:39:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:39:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:39:31.229 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:39:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 -- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6cf40a2c80 msgr2=0x7f6cf40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6cf40a2c80 0x7f6cf40a5140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55ab3bb5bf80 tx=0x7f6d080095c0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 -- 172.21.15.46:0/4229459136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 msgr2=0x7f6d1010d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 0x7f6d1010d0f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9f0 tx=0x7f6cfc00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 -- 172.21.15.46:0/4229459136 shutdown_connections 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6cf40a2c80 0x7f6cf40a5140 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d10106b80 0x7f6d101a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d101005a0 0x7f6d1010d0f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 --2- 172.21.15.46:0/4229459136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d100ffc00 0x7f6d1010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.228+0000 7f6d16a00640 1 -- 172.21.15.46:0/4229459136 >> 172.21.15.46:0/4229459136 conn(0x7f6d100fba90 msgr2=0x7f6d100fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.229+0000 7f6d16a00640 1 -- 172.21.15.46:0/4229459136 shutdown_connections 2024-09-10T07:39:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:39:31.229+0000 7f6d16a00640 1 -- 172.21.15.46:0/4229459136 wait complete. 2024-09-10T07:39:32.246 INFO:tasks.cephfs.filesystem:scrub status for tag:64b69cc7-deac-4d06-b271-47a425290bd1 - {'path': '/', 'tag': '64b69cc7-deac-4d06-b271-47a425290bd1', 'options': 'recursive,force'} 2024-09-10T07:39:32.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:32 smithi181 ceph-mon[35397]: pgmap v486: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 897 op/s 2024-09-10T07:39:32.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:32 smithi063 ceph-mon[35020]: pgmap v486: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 897 op/s 2024-09-10T07:39:32.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:32 smithi046 ceph-mon[30705]: pgmap v486: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 897 op/s 2024-09-10T07:39:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.787+0000 7f106a400640 1 -- 172.21.15.46:0/1514889118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 msgr2=0x7f10641020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.787+0000 7f106a400640 1 --2- 172.21.15.46:0/1514889118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 0x7f10641020b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1050009900 tx=0x7f105002a400 comp rx=0 tx=0).stop 2024-09-10T07:39:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.787+0000 7f106a400640 1 -- 172.21.15.46:0/1514889118 shutdown_connections 2024-09-10T07:39:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.787+0000 7f106a400640 1 --2- 172.21.15.46:0/1514889118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10641025f0 0x7f1064108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.787+0000 7f106a400640 1 --2- 172.21.15.46:0/1514889118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 0x7f10641020b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.787+0000 7f106a400640 1 --2- 172.21.15.46:0/1514889118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1064100a40 0x7f1064100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.787+0000 7f106a400640 1 -- 172.21.15.46:0/1514889118 >> 172.21.15.46:0/1514889118 conn(0x7f10640fc1e0 msgr2=0x7f10640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.788+0000 7f106a400640 1 -- 172.21.15.46:0/1514889118 shutdown_connections 2024-09-10T07:39:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.788+0000 7f106a400640 1 -- 172.21.15.46:0/1514889118 wait complete. 2024-09-10T07:39:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.789+0000 7f106a400640 1 Processor -- start 2024-09-10T07:39:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.789+0000 7f106a400640 1 -- start start 2024-09-10T07:39:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f106a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1064100a40 0x7f1064197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f106a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 0x7f1064198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f106a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10641025f0 0x7f106419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f106a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f106410d870 con 0x7f10641025f0 2024-09-10T07:39:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f106a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f106410d6f0 con 0x7f1064100a40 2024-09-10T07:39:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f106a400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f106410d9f0 con 0x7f1064101c30 2024-09-10T07:39:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f1062a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 0x7f1064198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f1062a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 0x7f1064198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58060/0 (socket says 172.21.15.46:58060) 2024-09-10T07:39:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.790+0000 7f1062a00640 1 -- 172.21.15.46:0/3928442830 learned_addr learned my addr 172.21.15.46:0/3928442830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.791+0000 7f1062a00640 1 -- 172.21.15.46:0/3928442830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1064100a40 msgr2=0x7f1064197dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T07:39:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.791+0000 7f1062a00640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1064100a40 0x7f1064197dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.791+0000 7f1062a00640 1 -- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10641025f0 msgr2=0x7f106419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.791+0000 7f1063e00640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10641025f0 0x7f106419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.791+0000 7f1062a00640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10641025f0 0x7f106419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.791+0000 7f1062a00640 1 -- 172.21.15.46:0/3928442830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1064109940 con 0x7f1064101c30 2024-09-10T07:39:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.792+0000 7f1063e00640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10641025f0 0x7f106419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.792+0000 7f1062a00640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 0x7f1064198310 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1050031440 tx=0x7f1050032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.793+0000 7f105be00640 1 -- 172.21.15.46:0/3928442830 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105003f030 con 0x7f1064101c30 2024-09-10T07:39:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.793+0000 7f105be00640 1 -- 172.21.15.46:0/3928442830 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1050003ce0 con 0x7f1064101c30 2024-09-10T07:39:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.793+0000 7f106a400640 1 -- 172.21.15.46:0/3928442830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1064109bd0 con 0x7f1064101c30 2024-09-10T07:39:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.794+0000 7f106a400640 1 -- 172.21.15.46:0/3928442830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106410a0c0 con 0x7f1064101c30 2024-09-10T07:39:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.794+0000 7f105be00640 1 -- 172.21.15.46:0/3928442830 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105003f030 con 0x7f1064101c30 2024-09-10T07:39:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.798+0000 7f105be00640 1 -- 172.21.15.46:0/3928442830 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1050042030 con 0x7f1064101c30 2024-09-10T07:39:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f106d23f740 1 Processor -- start 2024-09-10T07:39:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f106d23f740 1 -- start start 2024-09-10T07:39:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f106d23f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 0x55aad2ebe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f106d23f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ebe9d0 0x55aad2ec3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f106d23f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ec3a60 0x55aad2ec5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f106d23f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aad2e23100 con 0x55aad2ebe090 2024-09-10T07:39:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f106d23f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aad2dcb1b0 con 0x55aad2ec3a60 2024-09-10T07:39:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f106d23f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55aad2e1f0c0 con 0x55aad2ebe9d0 2024-09-10T07:39:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.802+0000 7f105be00640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10480a2e40 0x7f10480a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.803+0000 7f105be00640 1 -- 172.21.15.46:0/3928442830 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f10500f03e0 con 0x7f1064101c30 2024-09-10T07:39:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.803+0000 7f1063400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ebe9d0 0x55aad2ec3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.803+0000 7f1063400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ebe9d0 0x55aad2ec3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58070/0 (socket says 172.21.15.46:58070) 2024-09-10T07:39:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.803+0000 7f1063400640 1 -- 172.21.15.46:0/2844218333 learned_addr learned my addr 172.21.15.46:0/2844218333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.803+0000 7f1063e00640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10480a2e40 0x7f10480a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.803+0000 7f1062a00640 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ec3a60 0x55aad2ec5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.803+0000 7f1063e00640 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 0x55aad2ebe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.804+0000 7f1063400640 1 -- 172.21.15.46:0/2844218333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ec3a60 msgr2=0x55aad2ec5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.804+0000 7f1063400640 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ec3a60 0x55aad2ec5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.804+0000 7f1063400640 1 -- 172.21.15.46:0/2844218333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 msgr2=0x55aad2ebe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.804+0000 7f1063400640 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 0x55aad2ebe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.804+0000 7f1063400640 1 -- 172.21.15.46:0/2844218333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad2b36210 con 0x55aad2ebe9d0 2024-09-10T07:39:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.804+0000 7f1063400640 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ebe9d0 0x55aad2ec3490 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1054009b80 tx=0x7f105400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d459e3b2e183d50d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.804+0000 7f1063e00640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10480a2e40 0x7f10480a5300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f105c009fd0 tx=0x7f105c009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.804+0000 7f1058c00640 1 -- 172.21.15.46:0/2844218333 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054005020 con 0x55aad2ebe9d0 2024-09-10T07:39:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.805+0000 7f1058c00640 1 -- 172.21.15.46:0/2844218333 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10540047a0 con 0x55aad2ebe9d0 2024-09-10T07:39:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.805+0000 7f1058c00640 1 -- 172.21.15.46:0/2844218333 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054010690 con 0x55aad2ebe9d0 2024-09-10T07:39:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.807+0000 7f106d23f740 1 -- 172.21.15.46:0/2844218333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ebe9d0 msgr2=0x55aad2ec3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.807+0000 7f106d23f740 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ebe9d0 0x55aad2ec3490 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1054009b80 tx=0x7f105400e720 comp rx=0 tx=0).stop 2024-09-10T07:39:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.808+0000 7f106d23f740 1 -- 172.21.15.46:0/2844218333 shutdown_connections 2024-09-10T07:39:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.808+0000 7f106d23f740 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ec3a60 0x55aad2ec5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.808+0000 7f106d23f740 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ebe9d0 0x55aad2ec3490 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.808+0000 7f106d23f740 1 --2- 172.21.15.46:0/2844218333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 0x55aad2ebe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.808+0000 7f106d23f740 1 -- 172.21.15.46:0/2844218333 >> 172.21.15.46:0/2844218333 conn(0x55aad2eb9700 msgr2=0x55aad2eb9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.808+0000 7f106d23f740 1 -- 172.21.15.46:0/2844218333 shutdown_connections 2024-09-10T07:39:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.808+0000 7f106d23f740 1 -- 172.21.15.46:0/2844218333 wait complete. 2024-09-10T07:39:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.809+0000 7f106d23f740 1 -- start start 2024-09-10T07:39:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.809+0000 7f106d23f740 1 Processor -- start 2024-09-10T07:39:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.811+0000 7f106d23f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 0x55aad2ed0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.811+0000 7f106d23f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe9d0 0x55aad2ed0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f106d23f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec3a60 0x55aad2ed8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f106d23f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aad2ec6a30 con 0x55aad2ebe090 2024-09-10T07:39:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f106d23f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aad2dac090 con 0x55aad2ebe9d0 2024-09-10T07:39:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f106d23f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55aad2ec6bb0 con 0x55aad2ec3a60 2024-09-10T07:39:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f106d23f740 10 client.0 _ll_register_callbacks cb 0x55aad2e00f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1063e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec3a60 0x55aad2ed8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1062a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe9d0 0x55aad2ed0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1063e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec3a60 0x55aad2ed8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58078/0 (socket says 172.21.15.46:58078) 2024-09-10T07:39:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1063e00640 1 -- 172.21.15.46:0/2933147790 learned_addr learned my addr 172.21.15.46:0/2933147790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1063400640 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 0x55aad2ed0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1062a00640 1 -- 172.21.15.46:0/2933147790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec3a60 msgr2=0x55aad2ed8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1062a00640 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec3a60 0x55aad2ed8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1062a00640 1 -- 172.21.15.46:0/2933147790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 msgr2=0x55aad2ed0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1062a00640 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 0x55aad2ed0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.812+0000 7f1062a00640 1 -- 172.21.15.46:0/2933147790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad2c02620 con 0x55aad2ebe9d0 2024-09-10T07:39:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.813+0000 7f1062a00640 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe9d0 0x55aad2ed0fc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f10500bc7f0 tx=0x7f10500bc460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=881a943ff18df1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.813+0000 7f104d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.813+0000 7f104d600640 1 -- 172.21.15.46:0/2933147790 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050049030 con 0x55aad2ebe9d0 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.813+0000 7f104d600640 1 -- 172.21.15.46:0/2933147790 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f10500edc10 con 0x55aad2ebe9d0 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.814+0000 7f106d23f740 1 -- 172.21.15.46:0/2933147790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10540047a0 con 0x55aad2ebe9d0 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.814+0000 7f104d600640 1 -- 172.21.15.46:0/2933147790 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1050035040 con 0x55aad2ebe9d0 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.814+0000 7f106d23f740 10 client.26156 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.814+0000 7f106d23f740 1 -- 172.21.15.46:0/2933147790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55aad2edadf0 con 0x55aad2ebe9d0 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.815+0000 7f104d600640 1 -- 172.21.15.46:0/2933147790 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10540047a0 con 0x55aad2ebe9d0 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.815+0000 7f104d600640 1 -- 172.21.15.46:0/2933147790 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f105004c060 con 0x55aad2ebe9d0 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.816+0000 7f106d23f740 10 client.26156 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.816+0000 7f106d23f740 10 client.26156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.816+0000 7f106d23f740 20 client.26156 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.816+0000 7f106d23f740 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aad2edc550 0x55aad2ede9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.816+0000 7f106d23f740 4 client.26156 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.816+0000 7f106d23f740 1 -- 172.21.15.46:0/2933147790 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aad2e30830 con 0x55aad2edc550 2024-09-10T07:39:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.816+0000 7f1063400640 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aad2edc550 0x55aad2ede9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.817+0000 7f1063400640 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aad2edc550 0x55aad2ede9d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec8673e32c5a474 server_cookie=a5c8760d005d7e00 in_seq=0 out_seq=0 2024-09-10T07:39:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.817+0000 7f104d600640 10 client.26156 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.818+0000 7f104d600640 1 -- 172.21.15.46:0/2933147790 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aad2e30830 con 0x55aad2edc550 2024-09-10T07:39:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.818+0000 7f104d600640 10 client.26156 handle_command_reply: tid=0 2024-09-10T07:39:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.818+0000 7f106d23f740 1 client.26156 shutdown 2024-09-10T07:39:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f106d23f740 1 -- 172.21.15.46:0/2933147790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe9d0 msgr2=0x55aad2ed0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f106d23f740 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe9d0 0x55aad2ed0fc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f10500bc7f0 tx=0x7f10500bc460 comp rx=0 tx=0).stop 2024-09-10T07:39:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f106d23f740 1 -- 172.21.15.46:0/2933147790 shutdown_connections 2024-09-10T07:39:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f106d23f740 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aad2edc550 0x55aad2ede9d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f106d23f740 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec3a60 0x55aad2ed8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f106d23f740 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe9d0 0x55aad2ed0fc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f104d600640 0 client.26156 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f106d23f740 1 --2- 172.21.15.46:0/2933147790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ebe090 0x55aad2ed0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.819+0000 7f106d23f740 1 -- 172.21.15.46:0/2933147790 >> 172.21.15.46:0/2933147790 conn(0x55aad2eb9700 msgr2=0x55aad2ebaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.820+0000 7f106d23f740 1 -- 172.21.15.46:0/2933147790 shutdown_connections 2024-09-10T07:39:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.820+0000 7f106d23f740 1 -- 172.21.15.46:0/2933147790 wait complete. 2024-09-10T07:39:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.821+0000 7f106d23f740 20 client.26156 trim_cache size 0 max 16384 2024-09-10T07:39:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.855+0000 7f106d23f740 1 Processor -- start 2024-09-10T07:39:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.855+0000 7f106d23f740 1 -- start start 2024-09-10T07:39:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.856+0000 7f106d23f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ee32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.856+0000 7f106d23f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 0x55aad2ed7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.856+0000 7f106d23f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ee3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.856+0000 7f106d23f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10641a8470 con 0x55aad2ec3a60 2024-09-10T07:39:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.856+0000 7f106d23f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f10641aa0d0 con 0x55aad2ebe090 2024-09-10T07:39:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.856+0000 7f106d23f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f10641a8b40 con 0x55aad2ec0220 2024-09-10T07:39:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.857+0000 7f1063400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ee32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.857+0000 7f1063400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ee32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45232/0 (socket says 172.21.15.46:45232) 2024-09-10T07:39:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.857+0000 7f1063400640 1 -- 172.21.15.46:0/3477554292 learned_addr learned my addr 172.21.15.46:0/3477554292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.857+0000 7f1063e00640 1 --2- 172.21.15.46:0/3477554292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ee3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.858+0000 7f1063400640 1 -- 172.21.15.46:0/3477554292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 msgr2=0x55aad2ee3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.858+0000 7f1063400640 1 --2- 172.21.15.46:0/3477554292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ee3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.858+0000 7f1063400640 1 -- 172.21.15.46:0/3477554292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 msgr2=0x55aad2ed7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.858+0000 7f1063400640 1 --2- 172.21.15.46:0/3477554292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 0x55aad2ed7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.858+0000 7f1063400640 1 -- 172.21.15.46:0/3477554292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad2eb4d60 con 0x55aad2ebe090 2024-09-10T07:39:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.858+0000 7f1063400640 1 --2- 172.21.15.46:0/3477554292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ee32d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1054009b50 tx=0x7f1054009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4650e02cb618e116 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.861+0000 7f104fe00640 1 -- 172.21.15.46:0/3477554292 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105401d040 con 0x55aad2ebe090 2024-09-10T07:39:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.861+0000 7f104fe00640 1 -- 172.21.15.46:0/3477554292 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1054009d30 con 0x55aad2ebe090 2024-09-10T07:39:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.861+0000 7f104fe00640 1 -- 172.21.15.46:0/3477554292 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054005500 con 0x55aad2ebe090 2024-09-10T07:39:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.862+0000 7f106d23f740 1 -- 172.21.15.46:0/3477554292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 msgr2=0x55aad2ee32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.862+0000 7f106d23f740 1 --2- 172.21.15.46:0/3477554292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ee32d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1054009b50 tx=0x7f1054009940 comp rx=0 tx=0).stop 2024-09-10T07:39:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.862+0000 7f106d23f740 1 -- 172.21.15.46:0/3477554292 shutdown_connections 2024-09-10T07:39:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.862+0000 7f106d23f740 1 --2- 172.21.15.46:0/3477554292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ee3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.862+0000 7f106d23f740 1 --2- 172.21.15.46:0/3477554292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 0x55aad2ed7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.862+0000 7f106d23f740 1 --2- 172.21.15.46:0/3477554292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ee32d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.862+0000 7f106d23f740 1 -- 172.21.15.46:0/3477554292 >> 172.21.15.46:0/3477554292 conn(0x55aad2eb9700 msgr2=0x55aad2eb8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.864+0000 7f106d23f740 1 -- 172.21.15.46:0/3477554292 shutdown_connections 2024-09-10T07:39:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.864+0000 7f106d23f740 1 -- 172.21.15.46:0/3477554292 wait complete. 2024-09-10T07:39:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.864+0000 7f106d23f740 1 -- start start 2024-09-10T07:39:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.865+0000 7f106d23f740 1 Processor -- start 2024-09-10T07:39:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.866+0000 7f106d23f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ed79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.866+0000 7f106d23f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ed7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.866+0000 7f106d23f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 0x55aad2ed60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.867+0000 7f106d23f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10641a9940 con 0x55aad2ec3a60 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.867+0000 7f106d23f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f10641a9270 con 0x55aad2ebe090 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.867+0000 7f106d23f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f10641a6450 con 0x55aad2ec0220 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.867+0000 7f1062a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ed7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.867+0000 7f1062a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ed7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58100/0 (socket says 172.21.15.46:58100) 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.867+0000 7f1062a00640 1 -- 172.21.15.46:0/4178406429 learned_addr learned my addr 172.21.15.46:0/4178406429 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.868+0000 7f106d23f740 10 client.0 _ll_register_callbacks cb 0x7f10500b8440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.868+0000 7f1063e00640 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 0x55aad2ed60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.868+0000 7f1063400640 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ed79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.869+0000 7f1063400640 1 -- 172.21.15.46:0/4178406429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 msgr2=0x55aad2ed7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.869+0000 7f1063400640 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ed7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.869+0000 7f1063400640 1 -- 172.21.15.46:0/4178406429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 msgr2=0x55aad2ed60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.869+0000 7f1063400640 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 0x55aad2ed60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.869+0000 7f1063400640 1 -- 172.21.15.46:0/4178406429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad2ed6900 con 0x55aad2ebe090 2024-09-10T07:39:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.870+0000 7f1063400640 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ed79f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1054009b50 tx=0x7f10540096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4933251054fcb6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.870+0000 7f104d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.872+0000 7f104d600640 1 -- 172.21.15.46:0/4178406429 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054005280 con 0x55aad2ebe090 2024-09-10T07:39:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.872+0000 7f104d600640 1 -- 172.21.15.46:0/4178406429 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1054005420 con 0x55aad2ebe090 2024-09-10T07:39:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.872+0000 7f106d23f740 1 -- 172.21.15.46:0/4178406429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1054009d30 con 0x55aad2ebe090 2024-09-10T07:39:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.873+0000 7f104d600640 1 -- 172.21.15.46:0/4178406429 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1054005620 con 0x55aad2ebe090 2024-09-10T07:39:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.873+0000 7f106d23f740 10 client.26162 fetch_fsmap learned FSMap version 36 2024-09-10T07:39:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.873+0000 7f106d23f740 1 -- 172.21.15.46:0/4178406429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55aad2f47e80 con 0x55aad2ebe090 2024-09-10T07:39:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.876+0000 7f104d600640 1 -- 172.21.15.46:0/4178406429 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1054009d30 con 0x55aad2ebe090 2024-09-10T07:39:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.876+0000 7f104d600640 1 -- 172.21.15.46:0/4178406429 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f105403a020 con 0x55aad2ebe090 2024-09-10T07:39:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.877+0000 7f106d23f740 10 client.26162 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T07:39:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.877+0000 7f106d23f740 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.877+0000 7f106d23f740 20 client.26162 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.877+0000 7f106d23f740 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aad2f48660 0x55aad2f4aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.877+0000 7f106d23f740 4 client.26162 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:39:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.877+0000 7f106d23f740 1 -- 172.21.15.46:0/4178406429 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aad2ec37e0 con 0x55aad2f48660 2024-09-10T07:39:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.879+0000 7f1062a00640 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aad2f48660 0x55aad2f4aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.881+0000 7f1062a00640 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aad2f48660 0x55aad2f4aa80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3cc4302bf804c3 server_cookie=79aabd6a8a4a0554 in_seq=0 out_seq=0 2024-09-10T07:39:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.881+0000 7f104d600640 10 client.26162 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.912+0000 7f104d600640 1 -- 172.21.15.46:0/4178406429 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aad2ec37e0 con 0x55aad2f48660 2024-09-10T07:39:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.912+0000 7f104d600640 10 client.26162 handle_command_reply: tid=0 2024-09-10T07:39:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.912+0000 7f106d23f740 1 client.26162 shutdown 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 -- 172.21.15.46:0/4178406429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 msgr2=0x55aad2ed79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ed79f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1054009b50 tx=0x7f10540096a0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 -- 172.21.15.46:0/4178406429 shutdown_connections 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aad2f48660 0x55aad2f4aa80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aad2ec3a60 0x55aad2ed60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aad2ec0220 0x55aad2ed7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 --2- 172.21.15.46:0/4178406429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aad2ebe090 0x55aad2ed79f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 -- 172.21.15.46:0/4178406429 >> 172.21.15.46:0/4178406429 conn(0x55aad2eb9700 msgr2=0x55aad2eba390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f104d600640 0 client.26162 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.914+0000 7f106d23f740 1 -- 172.21.15.46:0/4178406429 shutdown_connections 2024-09-10T07:39:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.915+0000 7f106d23f740 1 -- 172.21.15.46:0/4178406429 wait complete. 2024-09-10T07:39:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.915+0000 7f106d23f740 20 client.26162 trim_cache size 0 max 16384 2024-09-10T07:39:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:39:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:39:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:39:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 -- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10480a2e40 msgr2=0x7f10480a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10480a2e40 0x7f10480a5300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f105c009fd0 tx=0x7f105c009550 comp rx=0 tx=0).stop 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 -- 172.21.15.46:0/3928442830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 msgr2=0x7f1064198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 0x7f1064198310 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1050031440 tx=0x7f1050032ce0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 -- 172.21.15.46:0/3928442830 shutdown_connections 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10480a2e40 0x7f10480a5300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10641025f0 0x7f106419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1064101c30 0x7f1064198310 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 --2- 172.21.15.46:0/3928442830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1064100a40 0x7f1064197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 -- 172.21.15.46:0/3928442830 >> 172.21.15.46:0/3928442830 conn(0x7f10640fc1e0 msgr2=0x7f1064104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.916+0000 7f106a400640 1 -- 172.21.15.46:0/3928442830 shutdown_connections 2024-09-10T07:39:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:33.917+0000 7f106a400640 1 -- 172.21.15.46:0/3928442830 wait complete. 2024-09-10T07:39:34.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:34 smithi181 ceph-mon[35397]: pgmap v487: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 16 MiB/s wr, 992 op/s 2024-09-10T07:39:34.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:34 smithi063 ceph-mon[35020]: pgmap v487: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 16 MiB/s wr, 992 op/s 2024-09-10T07:39:34.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:34 smithi046 ceph-mon[30705]: pgmap v487: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 16 MiB/s wr, 992 op/s 2024-09-10T07:39:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 135.0 s, 1671.0 tps, lat 59.897 ms stddev 75.846 2024-09-10T07:39:36.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:36 smithi181 ceph-mon[35397]: pgmap v488: 129 pgs: 129 active+clean; 1001 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-10T07:39:36.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:39:36.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:39:36.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:39:36.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:39:36.900 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:39:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:36 smithi063 ceph-mon[35020]: pgmap v488: 129 pgs: 129 active+clean; 1001 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-10T07:39:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:39:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:39:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:39:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:39:36.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:39:36.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:36 smithi046 ceph-mon[30705]: pgmap v488: 129 pgs: 129 active+clean; 1001 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-10T07:39:36.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:39:36.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:39:36.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:39:36.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:39:36.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:39:38.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:38 smithi181 ceph-mon[35397]: pgmap v489: 129 pgs: 129 active+clean; 1001 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-09-10T07:39:38.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:38 smithi063 ceph-mon[35020]: pgmap v489: 129 pgs: 129 active+clean; 1001 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-09-10T07:39:38.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:38 smithi046 ceph-mon[30705]: pgmap v489: 129 pgs: 129 active+clean; 1001 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-09-10T07:39:39.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:39 smithi181 ceph-mon[35397]: pgmap v490: 129 pgs: 129 active+clean; 1009 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 16 MiB/s wr, 961 op/s 2024-09-10T07:39:39.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:39 smithi063 ceph-mon[35020]: pgmap v490: 129 pgs: 129 active+clean; 1009 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 16 MiB/s wr, 961 op/s 2024-09-10T07:39:39.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:39 smithi046 ceph-mon[30705]: pgmap v490: 129 pgs: 129 active+clean; 1009 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 16 MiB/s wr, 961 op/s 2024-09-10T07:39:40.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 140.0 s, 2080.2 tps, lat 48.180 ms stddev 57.566 2024-09-10T07:39:41.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:41 smithi046 ceph-mon[30705]: pgmap v491: 129 pgs: 129 active+clean; 1018 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-10T07:39:42.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:41 smithi181 ceph-mon[35397]: pgmap v491: 129 pgs: 129 active+clean; 1018 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-10T07:39:42.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:41 smithi063 ceph-mon[35020]: pgmap v491: 129 pgs: 129 active+clean; 1018 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-10T07:39:44.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:43 smithi181 ceph-mon[35397]: pgmap v492: 129 pgs: 129 active+clean; 1018 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 811 op/s 2024-09-10T07:39:44.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:43 smithi063 ceph-mon[35020]: pgmap v492: 129 pgs: 129 active+clean; 1018 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 811 op/s 2024-09-10T07:39:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:43 smithi046 ceph-mon[30705]: pgmap v492: 129 pgs: 129 active+clean; 1018 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 811 op/s 2024-09-10T07:39:45.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:44 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:39:45.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:44 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:39:45.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:44 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:39:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 145.0 s, 1712.2 tps, lat 58.138 ms stddev 64.657 2024-09-10T07:39:46.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:45 smithi181 ceph-mon[35397]: pgmap v493: 129 pgs: 129 active+clean; 1018 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.5 MiB/s wr, 709 op/s 2024-09-10T07:39:46.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:39:46.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:39:46.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:45 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:39:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:45 smithi063 ceph-mon[35020]: pgmap v493: 129 pgs: 129 active+clean; 1018 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.5 MiB/s wr, 709 op/s 2024-09-10T07:39:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:39:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:39:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:45 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:39:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:45 smithi046 ceph-mon[30705]: pgmap v493: 129 pgs: 129 active+clean; 1018 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.5 MiB/s wr, 709 op/s 2024-09-10T07:39:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:39:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:39:46.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:45 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:39:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.117+0000 7feb97a00640 1 -- 172.21.15.46:0/1566381660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90107880 msgr2=0x7feb90109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.117+0000 7feb97a00640 1 --2- 172.21.15.46:0/1566381660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90107880 0x7feb90109c70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feb88009ca0 tx=0x7feb8802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.118+0000 7feb97a00640 1 -- 172.21.15.46:0/1566381660 shutdown_connections 2024-09-10T07:39:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.118+0000 7feb97a00640 1 --2- 172.21.15.46:0/1566381660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90107880 0x7feb90109c70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.118+0000 7feb97a00640 1 --2- 172.21.15.46:0/1566381660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90100120 0x7feb90107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.118+0000 7feb97a00640 1 --2- 172.21.15.46:0/1566381660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb900ff770 0x7feb900ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.118+0000 7feb97a00640 1 -- 172.21.15.46:0/1566381660 >> 172.21.15.46:0/1566381660 conn(0x7feb900fb500 msgr2=0x7feb900fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.118+0000 7feb97a00640 1 -- 172.21.15.46:0/1566381660 shutdown_connections 2024-09-10T07:39:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.118+0000 7feb97a00640 1 -- 172.21.15.46:0/1566381660 wait complete. 2024-09-10T07:39:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.119+0000 7feb97a00640 1 Processor -- start 2024-09-10T07:39:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb97a00640 1 -- start start 2024-09-10T07:39:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb97a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb900ff770 0x7feb90110da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb97a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90100120 0x7feb901112e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb97a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90107880 0x7feb90116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb97a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010b5f0 con 0x7feb90107880 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb97a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010b470 con 0x7feb900ff770 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb97a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010b770 con 0x7feb90100120 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb8fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90100120 0x7feb901112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb95000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb900ff770 0x7feb90110da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb95000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb900ff770 0x7feb90110da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48866/0 (socket says 172.21.15.46:48866) 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb8fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90100120 0x7feb901112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39306/0 (socket says 172.21.15.46:39306) 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.121+0000 7feb95000640 1 -- 172.21.15.46:0/1326876519 learned_addr learned my addr 172.21.15.46:0/1326876519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.122+0000 7feb95a00640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90107880 0x7feb90116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.122+0000 7feb95000640 1 -- 172.21.15.46:0/1326876519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90100120 msgr2=0x7feb901112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.122+0000 7feb95000640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90100120 0x7feb901112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.122+0000 7feb95000640 1 -- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90107880 msgr2=0x7feb90116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.122+0000 7feb95000640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90107880 0x7feb90116340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.122+0000 7feb95000640 1 -- 172.21.15.46:0/1326876519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb901169b0 con 0x7feb900ff770 2024-09-10T07:39:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.122+0000 7feb8fe00640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90100120 0x7feb901112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T07:39:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.122+0000 7feb95000640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb900ff770 0x7feb90110da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb8400d950 tx=0x7feb8400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.123+0000 7feb8d600640 1 -- 172.21.15.46:0/1326876519 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84017070 con 0x7feb900ff770 2024-09-10T07:39:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.123+0000 7feb8d600640 1 -- 172.21.15.46:0/1326876519 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb84009c30 con 0x7feb900ff770 2024-09-10T07:39:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.123+0000 7feb97a00640 1 -- 172.21.15.46:0/1326876519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb90116ca0 con 0x7feb900ff770 2024-09-10T07:39:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.123+0000 7feb97a00640 1 -- 172.21.15.46:0/1326876519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901171f0 con 0x7feb900ff770 2024-09-10T07:39:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.123+0000 7feb8d600640 1 -- 172.21.15.46:0/1326876519 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8400f6b0 con 0x7feb900ff770 2024-09-10T07:39:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.130+0000 7feb9a8bc740 1 Processor -- start 2024-09-10T07:39:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.131+0000 7feb8d600640 1 -- 172.21.15.46:0/1326876519 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb84010460 con 0x7feb900ff770 2024-09-10T07:39:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.131+0000 7feb9a8bc740 1 -- start start 2024-09-10T07:39:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.132+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.132+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb939f0 0x55f3ffb984b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.132+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb98a80 0x55f3ffb9af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.132+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3ffaf8100 con 0x55f3ffb98a80 2024-09-10T07:39:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.132+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3ffaa01b0 con 0x55f3ffb939f0 2024-09-10T07:39:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.132+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f3ffaf40c0 con 0x55f3ffb930b0 2024-09-10T07:39:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.133+0000 7feb95a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.133+0000 7feb95a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39314/0 (socket says 172.21.15.46:39314) 2024-09-10T07:39:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.133+0000 7feb95a00640 1 -- 172.21.15.46:0/1861183277 learned_addr learned my addr 172.21.15.46:0/1861183277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.133+0000 7feb8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb98a80 0x55f3ffb9af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.135+0000 7feb95000640 1 --2- 172.21.15.46:0/1861183277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb939f0 0x55f3ffb984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.135+0000 7feb95a00640 1 -- 172.21.15.46:0/1861183277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb939f0 msgr2=0x55f3ffb984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.135+0000 7feb95a00640 1 --2- 172.21.15.46:0/1861183277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb939f0 0x55f3ffb984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.135+0000 7feb95a00640 1 -- 172.21.15.46:0/1861183277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb98a80 msgr2=0x55f3ffb9af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.135+0000 7feb95a00640 1 --2- 172.21.15.46:0/1861183277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb98a80 0x55f3ffb9af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.135+0000 7feb95a00640 1 -- 172.21.15.46:0/1861183277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ff80b210 con 0x55f3ffb930b0 2024-09-10T07:39:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.136+0000 7feb8d600640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb780a2e20 0x7feb780a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.136+0000 7feb95a00640 1 --2- 172.21.15.46:0/1861183277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb934b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb880097b0 tx=0x7feb88009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98ec6933ae2e6d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.137+0000 7feb8d600640 1 -- 172.21.15.46:0/1326876519 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feb840cf090 con 0x7feb900ff770 2024-09-10T07:39:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.138+0000 7feb82000640 1 -- 172.21.15.46:0/1861183277 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88031650 con 0x55f3ffb930b0 2024-09-10T07:39:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.138+0000 7feb95000640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb780a2e20 0x7feb780a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.138+0000 7feb82000640 1 -- 172.21.15.46:0/1861183277 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb88036070 con 0x55f3ffb930b0 2024-09-10T07:39:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.138+0000 7feb82000640 1 -- 172.21.15.46:0/1861183277 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004040 con 0x55f3ffb930b0 2024-09-10T07:39:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.139+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1861183277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 msgr2=0x55f3ffb934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.139+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1861183277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb934b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb880097b0 tx=0x7feb88009a20 comp rx=0 tx=0).stop 2024-09-10T07:39:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.139+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1861183277 shutdown_connections 2024-09-10T07:39:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.139+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1861183277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb98a80 0x55f3ffb9af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.139+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1861183277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb939f0 0x55f3ffb984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.139+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1861183277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb934b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.139+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1861183277 >> 172.21.15.46:0/1861183277 conn(0x55f3ffb8e720 msgr2=0x55f3ffb8eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.140+0000 7feb95000640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb780a2e20 0x7feb780a52e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb84097970 tx=0x7feb840d2b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.140+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1861183277 shutdown_connections 2024-09-10T07:39:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.140+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1861183277 wait complete. 2024-09-10T07:39:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.141+0000 7feb9a8bc740 1 -- start start 2024-09-10T07:39:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.142+0000 7feb9a8bc740 1 Processor -- start 2024-09-10T07:39:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb930b0 0x55f3ffb9d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb939f0 0x55f3ffb9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffba4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3ffb9b9f0 con 0x55f3ffb930b0 2024-09-10T07:39:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3ffa81090 con 0x55f3ffb98a80 2024-09-10T07:39:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f3ffb9bb70 con 0x55f3ffb939f0 2024-09-10T07:39:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb9a8bc740 10 client.0 _ll_register_callbacks cb 0x55f3ffad5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb95a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffba4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb95a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffba4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48884/0 (socket says 172.21.15.46:48884) 2024-09-10T07:39:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.144+0000 7feb95a00640 1 -- 172.21.15.46:0/1988542915 learned_addr learned my addr 172.21.15.46:0/1988542915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.145+0000 7feb95a00640 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb930b0 0x55f3ffb9d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.145+0000 7feb95a00640 1 -- 172.21.15.46:0/1988542915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb939f0 msgr2=0x55f3ffb9d740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:39:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.145+0000 7feb95a00640 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb939f0 0x55f3ffb9d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.145+0000 7feb95a00640 1 -- 172.21.15.46:0/1988542915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb930b0 msgr2=0x55f3ffb9d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.145+0000 7feb95a00640 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb930b0 0x55f3ffb9d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.145+0000 7feb95a00640 1 -- 172.21.15.46:0/1988542915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ff8d7620 con 0x55f3ffb98a80 2024-09-10T07:39:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.147+0000 7feb95a00640 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffba4780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feb88031c10 tx=0x7feb88005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5145dd143e819fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.147+0000 7feb76a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.147+0000 7feb76a00640 1 -- 172.21.15.46:0/1988542915 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88049040 con 0x55f3ffb98a80 2024-09-10T07:39:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.147+0000 7feb76a00640 1 -- 172.21.15.46:0/1988542915 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feb88030c00 con 0x55f3ffb98a80 2024-09-10T07:39:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.147+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1988542915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb88036070 con 0x55f3ffb98a80 2024-09-10T07:39:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.148+0000 7feb76a00640 1 -- 172.21.15.46:0/1988542915 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feb88030e00 con 0x55f3ffb98a80 2024-09-10T07:39:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.148+0000 7feb9a8bc740 10 client.26174 fetch_fsmap learned FSMap version 37 2024-09-10T07:39:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.148+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1988542915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f3ffb9b9f0 con 0x55f3ffb98a80 2024-09-10T07:39:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.148+0000 7feb76a00640 1 -- 172.21.15.46:0/1988542915 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb88036070 con 0x55f3ffb98a80 2024-09-10T07:39:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.150+0000 7feb76a00640 1 -- 172.21.15.46:0/1988542915 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7feb88052470 con 0x55f3ffb98a80 2024-09-10T07:39:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.150+0000 7feb9a8bc740 10 client.26174 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T07:39:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.150+0000 7feb9a8bc740 10 client.26174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.150+0000 7feb9a8bc740 20 client.26174 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.150+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3ffba8820 0x55f3ffbaac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.150+0000 7feb9a8bc740 4 client.26174 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:39:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.150+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1988542915 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ffb05830 con 0x55f3ffba8820 2024-09-10T07:39:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.151+0000 7feb8fe00640 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3ffba8820 0x55f3ffbaac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.152+0000 7feb8fe00640 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3ffba8820 0x55f3ffbaac40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6561d0b25c3bdfd server_cookie=fdd971d6261f8207 in_seq=0 out_seq=0 2024-09-10T07:39:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.152+0000 7feb76a00640 10 client.26174 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.154+0000 7feb76a00640 1 -- 172.21.15.46:0/1988542915 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3ffb05830 con 0x55f3ffba8820 2024-09-10T07:39:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.154+0000 7feb76a00640 10 client.26174 handle_command_reply: tid=0 2024-09-10T07:39:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.154+0000 7feb9a8bc740 1 client.26174 shutdown 2024-09-10T07:39:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.154+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1988542915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 msgr2=0x55f3ffba4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.154+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffba4780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feb88031c10 tx=0x7feb88005df0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.155+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1988542915 shutdown_connections 2024-09-10T07:39:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.155+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3ffba8820 0x55f3ffbaac40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.155+0000 7feb76a00640 0 client.26174 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.155+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffba4780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.155+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb939f0 0x55f3ffb9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.155+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1988542915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb930b0 0x55f3ffb9d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.156+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1988542915 >> 172.21.15.46:0/1988542915 conn(0x55f3ffb8e720 msgr2=0x55f3ffb8ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.156+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1988542915 shutdown_connections 2024-09-10T07:39:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.156+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1988542915 wait complete. 2024-09-10T07:39:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.156+0000 7feb9a8bc740 20 client.26174 trim_cache size 0 max 16384 2024-09-10T07:39:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.185+0000 7feb9a8bc740 1 Processor -- start 2024-09-10T07:39:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.185+0000 7feb9a8bc740 1 -- start start 2024-09-10T07:39:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.186+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb8d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.186+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.186+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffb8d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.186+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b9270 con 0x55f3ffb939f0 2024-09-10T07:39:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.186+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb901baea0 con 0x55f3ffb98a80 2024-09-10T07:39:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.186+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b9940 con 0x55f3ffb930b0 2024-09-10T07:39:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.187+0000 7feb8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.187+0000 7feb8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52620/0 (socket says 172.21.15.46:52620) 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.187+0000 7feb8fe00640 1 -- 172.21.15.46:0/1507083260 learned_addr learned my addr 172.21.15.46:0/1507083260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.187+0000 7feb95a00640 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffb8d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.187+0000 7feb95a00640 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb8d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.188+0000 7feb8fe00640 1 -- 172.21.15.46:0/1507083260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 msgr2=0x55f3ffb8d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.188+0000 7feb8fe00640 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb8d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.188+0000 7feb8fe00640 1 -- 172.21.15.46:0/1507083260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 msgr2=0x55f3ffb8d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.188+0000 7feb8fe00640 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffb8d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.188+0000 7feb8fe00640 1 -- 172.21.15.46:0/1507083260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ffb94680 con 0x55f3ffb939f0 2024-09-10T07:39:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.188+0000 7feb8fe00640 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba46a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feb7c005f00 tx=0x7feb7c00b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecb3863a49d5066 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.189+0000 7feb81600640 1 -- 172.21.15.46:0/1507083260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0110d0 con 0x55f3ffb939f0 2024-09-10T07:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.189+0000 7feb81600640 1 -- 172.21.15.46:0/1507083260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7c011270 con 0x55f3ffb939f0 2024-09-10T07:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.189+0000 7feb81600640 1 -- 172.21.15.46:0/1507083260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0193a0 con 0x55f3ffb939f0 2024-09-10T07:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.190+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1507083260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 msgr2=0x55f3ffba46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.190+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba46a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feb7c005f00 tx=0x7feb7c00b470 comp rx=0 tx=0).stop 2024-09-10T07:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.190+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1507083260 shutdown_connections 2024-09-10T07:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.190+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb98a80 0x55f3ffb8d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.190+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba46a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.190+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/1507083260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb930b0 0x55f3ffb8d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.190+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1507083260 >> 172.21.15.46:0/1507083260 conn(0x55f3ffb8e720 msgr2=0x55f3ffb9d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.191+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1507083260 shutdown_connections 2024-09-10T07:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.191+0000 7feb9a8bc740 1 -- 172.21.15.46:0/1507083260 wait complete. 2024-09-10T07:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.191+0000 7feb9a8bc740 1 -- start start 2024-09-10T07:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.192+0000 7feb9a8bc740 1 Processor -- start 2024-09-10T07:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.192+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb930b0 0x55f3ffba17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.192+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.192+0000 7feb9a8bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb98a80 0x55f3ffc114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.192+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb901ba740 con 0x55f3ffb939f0 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.192+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb901ba070 con 0x55f3ffb930b0 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.192+0000 7feb9a8bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b7250 con 0x55f3ffb98a80 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.192+0000 7feb9a8bc740 10 client.0 _ll_register_callbacks cb 0x7feb7c00b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb930b0 0x55f3ffba17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb930b0 0x55f3ffba17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48902/0 (socket says 172.21.15.46:48902) 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 -- 172.21.15.46:0/257453958 learned_addr learned my addr 172.21.15.46:0/257453958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb8fe00640 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb98a80 0x55f3ffc114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 -- 172.21.15.46:0/257453958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb98a80 msgr2=0x55f3ffc114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb98a80 0x55f3ffc114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 -- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 msgr2=0x55f3ffba1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb95a00640 1 -- 172.21.15.46:0/257453958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ffc11a20 con 0x55f3ffb930b0 2024-09-10T07:39:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.193+0000 7feb8fe00640 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:39:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.194+0000 7feb95a00640 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb930b0 0x55f3ffba17b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feb88002230 tx=0x7feb88005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615798e7769213dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:39:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.194+0000 7feb76a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:39:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.194+0000 7feb76a00640 1 -- 172.21.15.46:0/257453958 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88052a00 con 0x55f3ffb930b0 2024-09-10T07:39:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.194+0000 7feb76a00640 1 -- 172.21.15.46:0/257453958 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feb88052ba0 con 0x55f3ffb930b0 2024-09-10T07:39:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.195+0000 7feb9a8bc740 1 -- 172.21.15.46:0/257453958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7c011270 con 0x55f3ffb930b0 2024-09-10T07:39:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.195+0000 7feb76a00640 1 -- 172.21.15.46:0/257453958 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feb88052da0 con 0x55f3ffb930b0 2024-09-10T07:39:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.196+0000 7feb9a8bc740 10 client.26180 fetch_fsmap learned FSMap version 37 2024-09-10T07:39:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.196+0000 7feb9a8bc740 1 -- 172.21.15.46:0/257453958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f3ffc12d10 con 0x55f3ffb930b0 2024-09-10T07:39:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.197+0000 7feb76a00640 1 -- 172.21.15.46:0/257453958 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7c011270 con 0x55f3ffb930b0 2024-09-10T07:39:47.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.197+0000 7feb76a00640 1 -- 172.21.15.46:0/257453958 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7feb880045d0 con 0x55f3ffb930b0 2024-09-10T07:39:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.198+0000 7feb9a8bc740 10 client.26180 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T07:39:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.198+0000 7feb9a8bc740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:39:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.198+0000 7feb9a8bc740 20 client.26180 populate_metadata read hostname 'smithi046' 2024-09-10T07:39:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.198+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3ffba8820 0x55f3ffc15650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:39:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.198+0000 7feb9a8bc740 4 client.26180 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:39:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.198+0000 7feb9a8bc740 1 -- 172.21.15.46:0/257453958 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3ffaffb80 con 0x55f3ffba8820 2024-09-10T07:39:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.198+0000 7feb8fe00640 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3ffba8820 0x55f3ffc15650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:39:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.200+0000 7feb8fe00640 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3ffba8820 0x55f3ffc15650 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281a3c5884776784 server_cookie=29e4ac8ca0667462 in_seq=0 out_seq=0 2024-09-10T07:39:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.200+0000 7feb76a00640 10 client.26180 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.215+0000 7feb76a00640 1 -- 172.21.15.46:0/257453958 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3ffaffb80 con 0x55f3ffba8820 2024-09-10T07:39:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb76a00640 10 client.26180 handle_command_reply: tid=0 2024-09-10T07:39:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 client.26180 shutdown 2024-09-10T07:39:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 -- 172.21.15.46:0/257453958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb930b0 msgr2=0x55f3ffba17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb930b0 0x55f3ffba17b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feb88002230 tx=0x7feb88005df0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 -- 172.21.15.46:0/257453958 shutdown_connections 2024-09-10T07:39:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3ffba8820 0x55f3ffc15650 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3ffb98a80 0x55f3ffc114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3ffb939f0 0x55f3ffba1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 --2- 172.21.15.46:0/257453958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ffb930b0 0x55f3ffba17b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 -- 172.21.15.46:0/257453958 >> 172.21.15.46:0/257453958 conn(0x55f3ffb8e720 msgr2=0x55f3ffb8f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb76a00640 0 client.26180 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:39:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.216+0000 7feb9a8bc740 1 -- 172.21.15.46:0/257453958 shutdown_connections 2024-09-10T07:39:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.217+0000 7feb9a8bc740 1 -- 172.21.15.46:0/257453958 wait complete. 2024-09-10T07:39:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.217+0000 7feb9a8bc740 20 client.26180 trim_cache size 0 max 16384 2024-09-10T07:39:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:39:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:39:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:39:47.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:39:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 -- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb780a2e20 msgr2=0x7feb780a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb780a2e20 0x7feb780a52e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb84097970 tx=0x7feb840d2b30 comp rx=0 tx=0).stop 2024-09-10T07:39:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 -- 172.21.15.46:0/1326876519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb900ff770 msgr2=0x7feb90110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:39:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb900ff770 0x7feb90110da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb8400d950 tx=0x7feb8400de10 comp rx=0 tx=0).stop 2024-09-10T07:39:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 -- 172.21.15.46:0/1326876519 shutdown_connections 2024-09-10T07:39:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb780a2e20 0x7feb780a52e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90107880 0x7feb90116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90100120 0x7feb901112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 --2- 172.21.15.46:0/1326876519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb900ff770 0x7feb90110da0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:39:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.218+0000 7feb97a00640 1 -- 172.21.15.46:0/1326876519 >> 172.21.15.46:0/1326876519 conn(0x7feb900fb500 msgr2=0x7feb90103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:39:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.219+0000 7feb97a00640 1 -- 172.21.15.46:0/1326876519 shutdown_connections 2024-09-10T07:39:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:39:47.220+0000 7feb97a00640 1 -- 172.21.15.46:0/1326876519 wait complete. 2024-09-10T07:39:47.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:47 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:39:47.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:47 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:39:47.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:47 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:39:48.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:48 smithi181 ceph-mon[35397]: pgmap v494: 129 pgs: 129 active+clean; 1019 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 720 op/s 2024-09-10T07:39:48.653 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:48 smithi063 ceph-mon[35020]: pgmap v494: 129 pgs: 129 active+clean; 1019 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 720 op/s 2024-09-10T07:39:48.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:48 smithi046 ceph-mon[30705]: pgmap v494: 129 pgs: 129 active+clean; 1019 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 720 op/s 2024-09-10T07:39:50.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 150.0 s, 1965.4 tps, lat 50.907 ms stddev 60.844 2024-09-10T07:39:50.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:50 smithi181 ceph-mon[35397]: pgmap v495: 129 pgs: 129 active+clean; 1019 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.0 MiB/s wr, 529 op/s 2024-09-10T07:39:50.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:50 smithi063 ceph-mon[35020]: pgmap v495: 129 pgs: 129 active+clean; 1019 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.0 MiB/s wr, 529 op/s 2024-09-10T07:39:50.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:50 smithi046 ceph-mon[30705]: pgmap v495: 129 pgs: 129 active+clean; 1019 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.0 MiB/s wr, 529 op/s 2024-09-10T07:39:52.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:52 smithi181 ceph-mon[35397]: pgmap v496: 129 pgs: 129 active+clean; 1019 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 670 op/s 2024-09-10T07:39:52.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:52 smithi063 ceph-mon[35020]: pgmap v496: 129 pgs: 129 active+clean; 1019 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 670 op/s 2024-09-10T07:39:52.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:52 smithi046 ceph-mon[30705]: pgmap v496: 129 pgs: 129 active+clean; 1019 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 670 op/s 2024-09-10T07:39:54.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:54 smithi181 ceph-mon[35397]: pgmap v497: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 603 op/s 2024-09-10T07:39:54.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:54 smithi063 ceph-mon[35020]: pgmap v497: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 603 op/s 2024-09-10T07:39:54.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:54 smithi046 ceph-mon[30705]: pgmap v497: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 603 op/s 2024-09-10T07:39:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 155.0 s, 1756.7 tps, lat 54.723 ms stddev 65.436 2024-09-10T07:39:56.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:56 smithi181 ceph-mon[35397]: pgmap v498: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 643 op/s 2024-09-10T07:39:56.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:56 smithi063 ceph-mon[35020]: pgmap v498: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 643 op/s 2024-09-10T07:39:56.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:56 smithi046 ceph-mon[30705]: pgmap v498: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 643 op/s 2024-09-10T07:39:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:39:58 smithi181 ceph-mon[35397]: pgmap v499: 129 pgs: 129 active+clean; 1019 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-09-10T07:39:58.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:39:58 smithi063 ceph-mon[35020]: pgmap v499: 129 pgs: 129 active+clean; 1019 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-09-10T07:39:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:39:58 smithi046 ceph-mon[30705]: pgmap v499: 129 pgs: 129 active+clean; 1019 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-09-10T07:40:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.419+0000 7f6aab200640 1 -- 172.21.15.46:0/4130304889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 msgr2=0x7f6aac101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.419+0000 7f6aab200640 1 --2- 172.21.15.46:0/4130304889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 0x7f6aac101a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009900 tx=0x7f6a9802a400 comp rx=0 tx=0).stop 2024-09-10T07:40:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.420+0000 7f6aab200640 1 -- 172.21.15.46:0/4130304889 shutdown_connections 2024-09-10T07:40:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.420+0000 7f6aab200640 1 --2- 172.21.15.46:0/4130304889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aac101f40 0x7f6aac109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.420+0000 7f6aab200640 1 --2- 172.21.15.46:0/4130304889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 0x7f6aac101a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.420+0000 7f6aab200640 1 --2- 172.21.15.46:0/4130304889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac103050 0x7f6aac103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.420+0000 7f6aab200640 1 -- 172.21.15.46:0/4130304889 >> 172.21.15.46:0/4130304889 conn(0x7f6aac0fd0d0 msgr2=0x7f6aac0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.420+0000 7f6aab200640 1 -- 172.21.15.46:0/4130304889 shutdown_connections 2024-09-10T07:40:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.420+0000 7f6aab200640 1 -- 172.21.15.46:0/4130304889 wait complete. 2024-09-10T07:40:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.421+0000 7f6aab200640 1 Processor -- start 2024-09-10T07:40:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.422+0000 7f6aab200640 1 -- start start 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aab200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 0x7f6aac19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aab200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac101f40 0x7f6aac19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aab200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aac103050 0x7f6aac1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aab200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac076100 con 0x7f6aac101f40 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aab200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac075f80 con 0x7f6aac103050 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aab200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac076280 con 0x7f6aac101580 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 0x7f6aac19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 0x7f6aac19e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40112/0 (socket says 172.21.15.46:40112) 2024-09-10T07:40:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aa9e00640 1 -- 172.21.15.46:0/2993902873 learned_addr learned my addr 172.21.15.46:0/2993902873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.423+0000 7f6aaa800640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aac103050 0x7f6aac1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.424+0000 7f6aa9e00640 1 -- 172.21.15.46:0/2993902873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aac103050 msgr2=0x7f6aac1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.424+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aac103050 0x7f6aac1a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.424+0000 7f6aa9e00640 1 -- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac101f40 msgr2=0x7f6aac19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.424+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac101f40 0x7f6aac19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.424+0000 7f6aa9e00640 1 -- 172.21.15.46:0/2993902873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac1a5fe0 con 0x7f6aac101580 2024-09-10T07:40:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.424+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 0x7f6aac19e3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00d950 tx=0x7f6a9c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.425+0000 7f6aa2a00640 1 -- 172.21.15.46:0/2993902873 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c017020 con 0x7f6aac101580 2024-09-10T07:40:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.425+0000 7f6aa2a00640 1 -- 172.21.15.46:0/2993902873 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9c009c30 con 0x7f6aac101580 2024-09-10T07:40:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.425+0000 7f6aab200640 1 -- 172.21.15.46:0/2993902873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac1a62d0 con 0x7f6aac101580 2024-09-10T07:40:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.425+0000 7f6aa2a00640 1 -- 172.21.15.46:0/2993902873 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00f720 con 0x7f6aac101580 2024-09-10T07:40:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.425+0000 7f6aab200640 1 -- 172.21.15.46:0/2993902873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac1a6870 con 0x7f6aac101580 2024-09-10T07:40:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.431+0000 7f6ab3e81740 1 Processor -- start 2024-09-10T07:40:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.431+0000 7f6ab3e81740 1 -- start start 2024-09-10T07:40:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.432+0000 7f6aa2a00640 1 -- 172.21.15.46:0/2993902873 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a9c01a020 con 0x7f6aac101580 2024-09-10T07:40:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.432+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.432+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.432+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.432+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a9ce30a100 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.432+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a9ce2b21b0 con 0x55a9ce3aac30 2024-09-10T07:40:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.432+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a9ce3060c0 con 0x55a9ce3a5260 2024-09-10T07:40:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.433+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.433+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.433+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.433+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3ad0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44254/0 (socket says 172.21.15.46:44254) 2024-09-10T07:40:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.434+0000 7f6aa9400640 1 -- 172.21.15.46:0/3477257813 learned_addr learned my addr 172.21.15.46:0/3477257813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.434+0000 7f6aa9400640 1 -- 172.21.15.46:0/3477257813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 msgr2=0x55a9ce3a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.435+0000 7f6aa9400640 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.435+0000 7f6aa9400640 1 -- 172.21.15.46:0/3477257813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 msgr2=0x55a9ce3aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.435+0000 7f6aa9400640 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3aa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.435+0000 7f6aa9400640 1 -- 172.21.15.46:0/3477257813 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9ce01d210 con 0x55a9ce3aac30 2024-09-10T07:40:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.435+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.435+0000 7f6aa9400640 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3ad0f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6a98031610 tx=0x7f6a98000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c88ed5651094405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.436+0000 7f6aaa800640 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.436+0000 7f6a97400640 1 -- 172.21.15.46:0/3477257813 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9803e030 con 0x55a9ce3aac30 2024-09-10T07:40:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.436+0000 7f6a97400640 1 -- 172.21.15.46:0/3477257813 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a980316b0 con 0x55a9ce3aac30 2024-09-10T07:40:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.436+0000 7f6a97400640 1 -- 172.21.15.46:0/3477257813 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98035040 con 0x55a9ce3aac30 2024-09-10T07:40:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.437+0000 7f6aa2a00640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a900a2cc0 0x7f6a900a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.438+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3477257813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 msgr2=0x55a9ce3ad0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.438+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3ad0f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6a98031610 tx=0x7f6a98000bc0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.438+0000 7f6aaa800640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a900a2cc0 0x7f6a900a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.439+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3477257813 shutdown_connections 2024-09-10T07:40:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.439+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3ad0f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.439+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.439+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3477257813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.439+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3477257813 >> 172.21.15.46:0/3477257813 conn(0x55a9ce3a08d0 msgr2=0x55a9ce3a0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.439+0000 7f6aa2a00640 1 -- 172.21.15.46:0/2993902873 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6a9c0cf320 con 0x7f6aac101580 2024-09-10T07:40:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.439+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3477257813 shutdown_connections 2024-09-10T07:40:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.440+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3477257813 wait complete. 2024-09-10T07:40:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.440+0000 7f6ab3e81740 1 -- start start 2024-09-10T07:40:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.441+0000 7f6ab3e81740 1 Processor -- start 2024-09-10T07:40:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.441+0000 7f6aaa800640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a900a2cc0 0x7f6a900a5180 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4005fd0 tx=0x7f6aa4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.441+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5260 0x55a9ce3af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.441+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.441+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.441+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a9ce3adba0 con 0x55a9ce3a5260 2024-09-10T07:40:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.442+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a9ce293090 con 0x55a9ce3aac30 2024-09-10T07:40:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.442+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a9ce3add20 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.442+0000 7f6ab3e81740 10 client.0 _ll_register_callbacks cb 0x55a9ce2e7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.442+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.442+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44270/0 (socket says 172.21.15.46:44270) 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.442+0000 7f6aaa800640 1 -- 172.21.15.46:0/3416664488 learned_addr learned my addr 172.21.15.46:0/3416664488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.442+0000 7f6aa9400640 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.443+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5260 0x55a9ce3af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.443+0000 7f6aaa800640 1 -- 172.21.15.46:0/3416664488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 msgr2=0x55a9ce3af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.443+0000 7f6aaa800640 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.443+0000 7f6aaa800640 1 -- 172.21.15.46:0/3416664488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5260 msgr2=0x55a9ce3af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.443+0000 7f6aaa800640 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5260 0x55a9ce3af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.443+0000 7f6aaa800640 1 -- 172.21.15.46:0/3416664488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9ce0e9620 con 0x55a9ce3aac30 2024-09-10T07:40:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.443+0000 7f6aa9400640 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.444+0000 7f6aaa800640 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b6900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4010ad0 tx=0x7f6aa4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f305e74048a5ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.444+0000 7f6a8be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.444+0000 7f6a8be00640 1 -- 172.21.15.46:0/3416664488 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa402c060 con 0x55a9ce3aac30 2024-09-10T07:40:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.444+0000 7f6a8be00640 1 -- 172.21.15.46:0/3416664488 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6aa4017030 con 0x55a9ce3aac30 2024-09-10T07:40:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.445+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3416664488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a980316b0 con 0x55a9ce3aac30 2024-09-10T07:40:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.445+0000 7f6a8be00640 1 -- 172.21.15.46:0/3416664488 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6aa401fc80 con 0x55a9ce3aac30 2024-09-10T07:40:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.445+0000 7f6ab3e81740 10 client.26192 fetch_fsmap learned FSMap version 37 2024-09-10T07:40:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.445+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3416664488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a9ce3ca790 con 0x55a9ce3aac30 2024-09-10T07:40:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.446+0000 7f6a8be00640 1 -- 172.21.15.46:0/3416664488 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a980316b0 con 0x55a9ce3aac30 2024-09-10T07:40:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.446+0000 7f6a8be00640 1 -- 172.21.15.46:0/3416664488 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f6aa401c590 con 0x55a9ce3aac30 2024-09-10T07:40:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.446+0000 7f6ab3e81740 10 client.26192 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T07:40:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.446+0000 7f6ab3e81740 10 client.26192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.446+0000 7f6ab3e81740 20 client.26192 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.446+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a9ce3cc2d0 0x55a9ce3ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.446+0000 7f6ab3e81740 4 client.26192 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.446+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3416664488 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9ce317830 con 0x55a9ce3cc2d0 2024-09-10T07:40:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.447+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a9ce3cc2d0 0x55a9ce3ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.448+0000 7f6a8be00640 10 client.26192 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.449+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a9ce3cc2d0 0x55a9ce3ce730 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232784cfe26339d6 server_cookie=af9374d7b56cce14 in_seq=0 out_seq=0 2024-09-10T07:40:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.450+0000 7f6a8be00640 1 -- 172.21.15.46:0/3416664488 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9ce317830 con 0x55a9ce3cc2d0 2024-09-10T07:40:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.450+0000 7f6a8be00640 10 client.26192 handle_command_reply: tid=0 2024-09-10T07:40:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.450+0000 7f6ab3e81740 1 client.26192 shutdown 2024-09-10T07:40:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3416664488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 msgr2=0x55a9ce3b6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b6900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4010ad0 tx=0x7f6aa4004a40 comp rx=0 tx=0).stop 2024-09-10T07:40:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3416664488 shutdown_connections 2024-09-10T07:40:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a9ce3cc2d0 0x55a9ce3ce730 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b6900 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3416664488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5260 0x55a9ce3af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3416664488 >> 172.21.15.46:0/3416664488 conn(0x55a9ce3a08d0 msgr2=0x55a9ce3a20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.451+0000 7f6a8be00640 0 client.26192 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.452+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3416664488 shutdown_connections 2024-09-10T07:40:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.452+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3416664488 wait complete. 2024-09-10T07:40:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.452+0000 7f6ab3e81740 20 client.26192 trim_cache size 0 max 16384 2024-09-10T07:40:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.496+0000 7f6ab3e81740 1 Processor -- start 2024-09-10T07:40:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.497+0000 7f6ab3e81740 1 -- start start 2024-09-10T07:40:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.497+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.497+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.497+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.497+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1a8430 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.497+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1aa060 con 0x55a9ce3aac30 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.497+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1a8b00 con 0x55a9ce3a5260 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.498+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.499+0000 7f6aaa800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44286/0 (socket says 172.21.15.46:44286) 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.499+0000 7f6aaa800640 1 -- 172.21.15.46:0/424950693 learned_addr learned my addr 172.21.15.46:0/424950693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.499+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/424950693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.499+0000 7f6aaa800640 1 -- 172.21.15.46:0/424950693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 msgr2=0x55a9ce3af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.500+0000 7f6aaa800640 1 --2- 172.21.15.46:0/424950693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.500+0000 7f6aaa800640 1 -- 172.21.15.46:0/424950693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 msgr2=0x55a9ce3b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.500+0000 7f6aaa800640 1 --2- 172.21.15.46:0/424950693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.500+0000 7f6aaa800640 1 -- 172.21.15.46:0/424950693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9ce3a06f0 con 0x55a9ce3aac30 2024-09-10T07:40:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.500+0000 7f6aaa800640 1 --2- 172.21.15.46:0/424950693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3af8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4011010 tx=0x7f6aa40114b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc5c640ab9ea612d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.501+0000 7f6a96a00640 1 -- 172.21.15.46:0/424950693 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4036080 con 0x55a9ce3aac30 2024-09-10T07:40:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.501+0000 7f6a96a00640 1 -- 172.21.15.46:0/424950693 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa4004ab0 con 0x55a9ce3aac30 2024-09-10T07:40:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.501+0000 7f6a96a00640 1 -- 172.21.15.46:0/424950693 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4036080 con 0x55a9ce3aac30 2024-09-10T07:40:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.502+0000 7f6ab3e81740 1 -- 172.21.15.46:0/424950693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 msgr2=0x55a9ce3af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.502+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/424950693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3af8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4011010 tx=0x7f6aa40114b0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.503+0000 7f6ab3e81740 1 -- 172.21.15.46:0/424950693 shutdown_connections 2024-09-10T07:40:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.504+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/424950693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3aac30 0x55a9ce3af8c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.504+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/424950693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.504+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/424950693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5260 0x55a9ce3af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.504+0000 7f6ab3e81740 1 -- 172.21.15.46:0/424950693 >> 172.21.15.46:0/424950693 conn(0x55a9ce3a08d0 msgr2=0x55a9ce39b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.504+0000 7f6ab3e81740 1 -- 172.21.15.46:0/424950693 shutdown_connections 2024-09-10T07:40:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.504+0000 7f6ab3e81740 1 -- 172.21.15.46:0/424950693 wait complete. 2024-09-10T07:40:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.505+0000 7f6ab3e81740 1 -- start start 2024-09-10T07:40:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.505+0000 7f6ab3e81740 1 Processor -- start 2024-09-10T07:40:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.506+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3a5260 0x55a9ce3b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.506+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.506+0000 7f6ab3e81740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.506+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1a9900 con 0x55a9ce3aac30 2024-09-10T07:40:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.506+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1a9230 con 0x55a9ce3a5260 2024-09-10T07:40:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.506+0000 7f6ab3e81740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac073ca0 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.506+0000 7f6ab3e81740 10 client.0 _ll_register_callbacks cb 0x7f6aa4009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.507+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3a5260 0x55a9ce3b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.507+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.507+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40148/0 (socket says 172.21.15.46:40148) 2024-09-10T07:40:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.507+0000 7f6aa9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3a5260 0x55a9ce3b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44294/0 (socket says 172.21.15.46:44294) 2024-09-10T07:40:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.507+0000 7f6aa9400640 1 -- 172.21.15.46:0/3045828597 learned_addr learned my addr 172.21.15.46:0/3045828597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.507+0000 7f6aaa800640 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.508+0000 7f6aa9400640 1 -- 172.21.15.46:0/3045828597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3a5260 msgr2=0x55a9ce3b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.508+0000 7f6aa9400640 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3a5260 0x55a9ce3b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.508+0000 7f6aa9400640 1 -- 172.21.15.46:0/3045828597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3aac30 msgr2=0x55a9ce3b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.508+0000 7f6aa9400640 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.508+0000 7f6aa9400640 1 -- 172.21.15.46:0/3045828597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9ce3b5950 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.508+0000 7f6aa9400640 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b4330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a98031610 tx=0x7f6a980318e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e046ab37fe0e9cba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.508+0000 7f6a8be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.509+0000 7f6a8be00640 1 -- 172.21.15.46:0/3045828597 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9804b070 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.509+0000 7f6a8be00640 1 -- 172.21.15.46:0/3045828597 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6a98031c50 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.509+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3045828597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa4004ab0 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.510+0000 7f6a8be00640 1 -- 172.21.15.46:0/3045828597 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a98035040 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.511+0000 7f6ab3e81740 10 client.26188 fetch_fsmap learned FSMap version 37 2024-09-10T07:40:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.511+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3045828597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a9ce3bce70 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.515+0000 7f6a8be00640 1 -- 172.21.15.46:0/3045828597 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa4004ab0 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.515+0000 7f6a8be00640 1 -- 172.21.15.46:0/3045828597 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f6a98043520 con 0x55a9ce3a5ba0 2024-09-10T07:40:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.516+0000 7f6ab3e81740 10 client.26188 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T07:40:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.516+0000 7f6ab3e81740 10 client.26188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.516+0000 7f6ab3e81740 20 client.26188 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.516+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a9ce3bd350 0x55a9ce3bf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.516+0000 7f6ab3e81740 4 client.26188 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:40:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.516+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3045828597 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9ce311b80 con 0x55a9ce3bd350 2024-09-10T07:40:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.521+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a9ce3bd350 0x55a9ce3bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.523+0000 7f6aa9e00640 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a9ce3bd350 0x55a9ce3bf770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76411009fa05826f server_cookie=c4149197b475f5e9 in_seq=0 out_seq=0 2024-09-10T07:40:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.523+0000 7f6a8be00640 10 client.26188 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:00.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:40:00.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:40:00.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:40:00.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:40:00.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:40:00.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:40:00.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:40:00.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:00 smithi046 ceph-mon[30705]: overall HEALTH_OK 2024-09-10T07:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6a8be00640 1 -- 172.21.15.46:0/3045828597 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9ce311b80 con 0x55a9ce3bd350 2024-09-10T07:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6a8be00640 10 client.26188 handle_command_reply: tid=0 2024-09-10T07:40:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 client.26188 shutdown 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3045828597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 msgr2=0x55a9ce3b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b4330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a98031610 tx=0x7f6a980318e0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3045828597 shutdown_connections 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a9ce3bd350 0x55a9ce3bf770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9ce3aac30 0x55a9ce3b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9ce3a5ba0 0x55a9ce3b4330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 --2- 172.21.15.46:0/3045828597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a9ce3a5260 0x55a9ce3b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.577+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3045828597 >> 172.21.15.46:0/3045828597 conn(0x55a9ce3a08d0 msgr2=0x55a9ce3af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.578+0000 7f6a8be00640 0 client.26188 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.578+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3045828597 shutdown_connections 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.578+0000 7f6ab3e81740 1 -- 172.21.15.46:0/3045828597 wait complete. 2024-09-10T07:40:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.578+0000 7f6ab3e81740 20 client.26188 trim_cache size 0 max 16384 2024-09-10T07:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:40:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:40:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 -- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a900a2cc0 msgr2=0x7f6a900a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a900a2cc0 0x7f6a900a5180 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4005fd0 tx=0x7f6aa4009500 comp rx=0 tx=0).stop 2024-09-10T07:40:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 -- 172.21.15.46:0/2993902873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 msgr2=0x7f6aac19e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 0x7f6aac19e3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00d950 tx=0x7f6a9c00de10 comp rx=0 tx=0).stop 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 -- 172.21.15.46:0/2993902873 shutdown_connections 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aac103050 0x7f6aac1a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a900a2cc0 0x7f6a900a5180 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac101f40 0x7f6aac19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 --2- 172.21.15.46:0/2993902873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aac101580 0x7f6aac19e3c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.580+0000 7f6aab200640 1 -- 172.21.15.46:0/2993902873 >> 172.21.15.46:0/2993902873 conn(0x7f6aac0fd0d0 msgr2=0x7f6aac10e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.581+0000 7f6aab200640 1 -- 172.21.15.46:0/2993902873 shutdown_connections 2024-09-10T07:40:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:00.581+0000 7f6aab200640 1 -- 172.21.15.46:0/2993902873 wait complete. 2024-09-10T07:40:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 160.0 s, 1291.1 tps, lat 79.415 ms stddev 118.260 2024-09-10T07:40:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:40:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:40:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:40:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:40:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:40:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:00 smithi181 ceph-mon[35397]: overall HEALTH_OK 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:40:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:00 smithi063 ceph-mon[35020]: overall HEALTH_OK 2024-09-10T07:40:02.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:40:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:03.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.860+0000 7fde41400640 1 -- 172.21.15.46:0/2741889723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c10ac90 msgr2=0x7fde3c10d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.860+0000 7fde41400640 1 --2- 172.21.15.46:0/2741889723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c10ac90 0x7fde3c10d080 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fde34009cd0 tx=0x7fde3402a910 comp rx=0 tx=0).stop 2024-09-10T07:40:03.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.864+0000 7fde41400640 1 -- 172.21.15.46:0/2741889723 shutdown_connections 2024-09-10T07:40:03.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.864+0000 7fde41400640 1 --2- 172.21.15.46:0/2741889723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c10ac90 0x7fde3c10d080 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.864+0000 7fde41400640 1 --2- 172.21.15.46:0/2741889723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c104f30 0x7fde3c105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.864+0000 7fde41400640 1 --2- 172.21.15.46:0/2741889723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde3c100620 0x7fde3c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.864+0000 7fde41400640 1 -- 172.21.15.46:0/2741889723 >> 172.21.15.46:0/2741889723 conn(0x7fde3c0fba90 msgr2=0x7fde3c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:03.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.864+0000 7fde41400640 1 -- 172.21.15.46:0/2741889723 shutdown_connections 2024-09-10T07:40:03.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.864+0000 7fde41400640 1 -- 172.21.15.46:0/2741889723 wait complete. 2024-09-10T07:40:03.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.866+0000 7fde41400640 1 Processor -- start 2024-09-10T07:40:03.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.866+0000 7fde41400640 1 -- start start 2024-09-10T07:40:03.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.867+0000 7fde41400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c100620 0x7fde3c10e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.867+0000 7fde41400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde3c104f30 0x7fde3c10ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.867+0000 7fde41400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10ac90 0x7fde3c10f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.867+0000 7fde41400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c112020 con 0x7fde3c104f30 2024-09-10T07:40:03.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.867+0000 7fde41400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c111ea0 con 0x7fde3c10ac90 2024-09-10T07:40:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.867+0000 7fde41400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1121a0 con 0x7fde3c100620 2024-09-10T07:40:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.867+0000 7fde3b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10ac90 0x7fde3c10f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.868+0000 7fde3a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c100620 0x7fde3c10e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.868+0000 7fde3a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c100620 0x7fde3c10e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40170/0 (socket says 172.21.15.46:40170) 2024-09-10T07:40:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.868+0000 7fde3a800640 1 -- 172.21.15.46:0/604362356 learned_addr learned my addr 172.21.15.46:0/604362356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.868+0000 7fde33e00640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde3c104f30 0x7fde3c10ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.869+0000 7fde3b200640 1 -- 172.21.15.46:0/604362356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c100620 msgr2=0x7fde3c10e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.869+0000 7fde3b200640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c100620 0x7fde3c10e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.869+0000 7fde3b200640 1 -- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde3c104f30 msgr2=0x7fde3c10ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.869+0000 7fde3b200640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde3c104f30 0x7fde3c10ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.869+0000 7fde3b200640 1 -- 172.21.15.46:0/604362356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c1982e0 con 0x7fde3c10ac90 2024-09-10T07:40:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.869+0000 7fde33e00640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde3c104f30 0x7fde3c10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.870+0000 7fde3a800640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c100620 0x7fde3c10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.870+0000 7fde3b200640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10ac90 0x7fde3c10f220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fde34009ca0 tx=0x7fde34003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.870+0000 7fde33400640 1 -- 172.21.15.46:0/604362356 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34031820 con 0x7fde3c10ac90 2024-09-10T07:40:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.871+0000 7fde33400640 1 -- 172.21.15.46:0/604362356 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde34004520 con 0x7fde3c10ac90 2024-09-10T07:40:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.871+0000 7fde33400640 1 -- 172.21.15.46:0/604362356 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3402fd40 con 0x7fde3c10ac90 2024-09-10T07:40:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.871+0000 7fde41400640 1 -- 172.21.15.46:0/604362356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c198570 con 0x7fde3c10ac90 2024-09-10T07:40:03.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.871+0000 7fde41400640 1 -- 172.21.15.46:0/604362356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c198a30 con 0x7fde3c10ac90 2024-09-10T07:40:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.875+0000 7fde43eb4740 1 Processor -- start 2024-09-10T07:40:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.878+0000 7fde33400640 1 -- 172.21.15.46:0/604362356 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde340305d0 con 0x7fde3c10ac90 2024-09-10T07:40:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.878+0000 7fde43eb4740 1 -- start start 2024-09-10T07:40:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a4496f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44a1d70 0x5651a44a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde43eb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5651a43fd3e0 con 0x5651a44974b0 2024-09-10T07:40:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde43eb4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5651a43a4d80 con 0x5651a4496b90 2024-09-10T07:40:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde43eb4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5651a43f93a0 con 0x5651a44a1d70 2024-09-10T07:40:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde33e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a4496f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde33e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a4496f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44320/0 (socket says 172.21.15.46:44320) 2024-09-10T07:40:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.879+0000 7fde33e00640 1 -- 172.21.15.46:0/3587971132 learned_addr learned my addr 172.21.15.46:0/3587971132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.880+0000 7fde33e00640 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44a1d70 0x5651a44a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.880+0000 7fde3a800640 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.880+0000 7fde33e00640 1 -- 172.21.15.46:0/3587971132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 msgr2=0x5651a4496f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.880+0000 7fde33e00640 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a4496f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.880+0000 7fde33e00640 1 -- 172.21.15.46:0/3587971132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 msgr2=0x5651a44a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.880+0000 7fde33e00640 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.881+0000 7fde33e00640 1 -- 172.21.15.46:0/3587971132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a43c3ae0 con 0x5651a44a1d70 2024-09-10T07:40:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.881+0000 7fde3a800640 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.882+0000 7fde33400640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde200a2c00 0x7fde200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.882+0000 7fde33e00640 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44a1d70 0x5651a44a4160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde24015600 tx=0x7fde240046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f910a6ad844b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.882+0000 7fde33400640 1 -- 172.21.15.46:0/604362356 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde340f0880 con 0x7fde3c10ac90 2024-09-10T07:40:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.882+0000 7fde3a800640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde200a2c00 0x7fde200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.882+0000 7fde2b400640 1 -- 172.21.15.46:0/3587971132 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2400f040 con 0x5651a44a1d70 2024-09-10T07:40:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.883+0000 7fde2b400640 1 -- 172.21.15.46:0/3587971132 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde2400ae00 con 0x5651a44a1d70 2024-09-10T07:40:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.883+0000 7fde2b400640 1 -- 172.21.15.46:0/3587971132 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24013810 con 0x5651a44a1d70 2024-09-10T07:40:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.883+0000 7fde43eb4740 1 -- 172.21.15.46:0/3587971132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44a1d70 msgr2=0x5651a44a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.883+0000 7fde43eb4740 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44a1d70 0x5651a44a4160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde24015600 tx=0x7fde240046e0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.883+0000 7fde3a800640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde200a2c00 0x7fde200a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00afd0 tx=0x7fde2c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.884+0000 7fde43eb4740 1 -- 172.21.15.46:0/3587971132 shutdown_connections 2024-09-10T07:40:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.884+0000 7fde43eb4740 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44a1d70 0x5651a44a4160 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.884+0000 7fde43eb4740 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.884+0000 7fde43eb4740 1 --2- 172.21.15.46:0/3587971132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a4496f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.884+0000 7fde43eb4740 1 -- 172.21.15.46:0/3587971132 >> 172.21.15.46:0/3587971132 conn(0x5651a4493120 msgr2=0x5651a4493500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.884+0000 7fde43eb4740 1 -- 172.21.15.46:0/3587971132 shutdown_connections 2024-09-10T07:40:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.884+0000 7fde43eb4740 1 -- 172.21.15.46:0/3587971132 wait complete. 2024-09-10T07:40:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.885+0000 7fde43eb4740 1 -- start start 2024-09-10T07:40:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.886+0000 7fde43eb4740 1 Processor -- start 2024-09-10T07:40:03.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.887+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a449ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.887+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44974b0 0x5651a449f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.887+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 0x5651a449f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.887+0000 7fde43eb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5651a44a49e0 con 0x5651a44a1d70 2024-09-10T07:40:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.887+0000 7fde43eb4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5651a43b80f0 con 0x5651a44974b0 2024-09-10T07:40:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.887+0000 7fde43eb4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5651a44a4b60 con 0x5651a4496b90 2024-09-10T07:40:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.887+0000 7fde43eb4740 10 client.0 _ll_register_callbacks cb 0x5651a43a99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde3a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a449ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde3a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a449ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40198/0 (socket says 172.21.15.46:40198) 2024-09-10T07:40:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde3a800640 1 -- 172.21.15.46:0/1953521970 learned_addr learned my addr 172.21.15.46:0/1953521970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde33e00640 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44974b0 0x5651a449f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde3a800640 1 -- 172.21.15.46:0/1953521970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44974b0 msgr2=0x5651a449f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde3a800640 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44974b0 0x5651a449f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde3a800640 1 -- 172.21.15.46:0/1953521970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 msgr2=0x5651a449f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde3a800640 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 0x5651a449f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde3a800640 1 -- 172.21.15.46:0/1953521970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a416e3c0 con 0x5651a4496b90 2024-09-10T07:40:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.888+0000 7fde33e00640 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44974b0 0x5651a449f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.889+0000 7fde3a800640 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a449ec20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde2c010eb0 tx=0x7fde2c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260c68ce59385fdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.889+0000 7fde1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.889+0000 7fde1fe00640 1 -- 172.21.15.46:0/1953521970 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c005220 con 0x5651a4496b90 2024-09-10T07:40:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.889+0000 7fde1fe00640 1 -- 172.21.15.46:0/1953521970 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde2c0053c0 con 0x5651a4496b90 2024-09-10T07:40:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.890+0000 7fde43eb4740 1 -- 172.21.15.46:0/1953521970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde2400ae00 con 0x5651a4496b90 2024-09-10T07:40:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.891+0000 7fde1fe00640 1 -- 172.21.15.46:0/1953521970 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fde2c0055c0 con 0x5651a4496b90 2024-09-10T07:40:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.891+0000 7fde43eb4740 10 client.26200 fetch_fsmap learned FSMap version 37 2024-09-10T07:40:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.891+0000 7fde43eb4740 1 -- 172.21.15.46:0/1953521970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5651a44b06c0 con 0x5651a4496b90 2024-09-10T07:40:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.891+0000 7fde1fe00640 1 -- 172.21.15.46:0/1953521970 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde2400ae00 con 0x5651a4496b90 2024-09-10T07:40:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.892+0000 7fde1fe00640 1 -- 172.21.15.46:0/1953521970 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7fde2c01ba90 con 0x5651a4496b90 2024-09-10T07:40:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.892+0000 7fde43eb4740 10 client.26200 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T07:40:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.892+0000 7fde43eb4740 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.892+0000 7fde43eb4740 20 client.26200 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.892+0000 7fde43eb4740 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651a44b1be0 0x5651a44b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.892+0000 7fde43eb4740 4 client.26200 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.892+0000 7fde43eb4740 1 -- 172.21.15.46:0/1953521970 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a440a950 con 0x5651a44b1be0 2024-09-10T07:40:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.893+0000 7fde33e00640 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651a44b1be0 0x5651a44b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.894+0000 7fde33e00640 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651a44b1be0 0x5651a44b4080 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f333b02b401afaad server_cookie=a24cdf0a405d4a8f in_seq=0 out_seq=0 2024-09-10T07:40:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.894+0000 7fde1fe00640 10 client.26200 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.896+0000 7fde1fe00640 1 -- 172.21.15.46:0/1953521970 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651a440a950 con 0x5651a44b1be0 2024-09-10T07:40:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.896+0000 7fde1fe00640 10 client.26200 handle_command_reply: tid=0 2024-09-10T07:40:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.896+0000 7fde43eb4740 1 client.26200 shutdown 2024-09-10T07:40:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.897+0000 7fde43eb4740 1 -- 172.21.15.46:0/1953521970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 msgr2=0x5651a449ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.897+0000 7fde43eb4740 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a449ec20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde2c010eb0 tx=0x7fde2c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde43eb4740 1 -- 172.21.15.46:0/1953521970 shutdown_connections 2024-09-10T07:40:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde43eb4740 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651a44b1be0 0x5651a44b4080 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde43eb4740 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 0x5651a449f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde43eb4740 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44974b0 0x5651a449f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde43eb4740 1 --2- 172.21.15.46:0/1953521970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a449ec20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde43eb4740 1 -- 172.21.15.46:0/1953521970 >> 172.21.15.46:0/1953521970 conn(0x5651a4493120 msgr2=0x5651a4498e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde1fe00640 0 client.26200 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde43eb4740 1 -- 172.21.15.46:0/1953521970 shutdown_connections 2024-09-10T07:40:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.898+0000 7fde43eb4740 1 -- 172.21.15.46:0/1953521970 wait complete. 2024-09-10T07:40:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.899+0000 7fde43eb4740 20 client.26200 trim_cache size 0 max 16384 2024-09-10T07:40:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.932+0000 7fde43eb4740 1 Processor -- start 2024-09-10T07:40:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.932+0000 7fde43eb4740 1 -- start start 2024-09-10T07:40:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.933+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a44b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.933+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44974b0 0x5651a4498cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.933+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 0x5651a44b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.933+0000 7fde43eb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b5280 con 0x5651a44a1d70 2024-09-10T07:40:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.933+0000 7fde43eb4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b6eb0 con 0x5651a4496b90 2024-09-10T07:40:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.933+0000 7fde43eb4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b5950 con 0x5651a44974b0 2024-09-10T07:40:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.933+0000 7fde3b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 0x5651a44b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.934+0000 7fde3b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 0x5651a44b9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41068/0 (socket says 172.21.15.46:41068) 2024-09-10T07:40:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.934+0000 7fde3b200640 1 -- 172.21.15.46:0/478558579 learned_addr learned my addr 172.21.15.46:0/478558579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.934+0000 7fde33e00640 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44974b0 0x5651a4498cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.934+0000 7fde3a800640 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a44b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.934+0000 7fde3a800640 1 -- 172.21.15.46:0/478558579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44974b0 msgr2=0x5651a4498cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.934+0000 7fde3a800640 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44974b0 0x5651a4498cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.935+0000 7fde3a800640 1 -- 172.21.15.46:0/478558579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 msgr2=0x5651a44b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:03.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.935+0000 7fde3a800640 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 0x5651a44b9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.935+0000 7fde3a800640 1 -- 172.21.15.46:0/478558579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a449dbd0 con 0x5651a4496b90 2024-09-10T07:40:03.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.936+0000 7fde3a800640 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a44b3bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde2c031510 tx=0x7fde2c00a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94feaff465ab5e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.937+0000 7fde2aa00640 1 -- 172.21.15.46:0/478558579 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c01b860 con 0x5651a4496b90 2024-09-10T07:40:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.937+0000 7fde2aa00640 1 -- 172.21.15.46:0/478558579 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde2c01ba00 con 0x5651a4496b90 2024-09-10T07:40:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.937+0000 7fde2aa00640 1 -- 172.21.15.46:0/478558579 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c03a670 con 0x5651a4496b90 2024-09-10T07:40:03.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.938+0000 7fde43eb4740 1 -- 172.21.15.46:0/478558579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 msgr2=0x5651a44b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.938+0000 7fde43eb4740 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a44b3bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde2c031510 tx=0x7fde2c00a550 comp rx=0 tx=0).stop 2024-09-10T07:40:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.938+0000 7fde43eb4740 1 -- 172.21.15.46:0/478558579 shutdown_connections 2024-09-10T07:40:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.938+0000 7fde43eb4740 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44a1d70 0x5651a44b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.938+0000 7fde43eb4740 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a44974b0 0x5651a4498cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.938+0000 7fde43eb4740 1 --2- 172.21.15.46:0/478558579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a4496b90 0x5651a44b3bf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.938+0000 7fde43eb4740 1 -- 172.21.15.46:0/478558579 >> 172.21.15.46:0/478558579 conn(0x5651a4493120 msgr2=0x5651a44985e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.940+0000 7fde43eb4740 1 -- 172.21.15.46:0/478558579 shutdown_connections 2024-09-10T07:40:03.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.940+0000 7fde43eb4740 1 -- 172.21.15.46:0/478558579 wait complete. 2024-09-10T07:40:03.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.940+0000 7fde43eb4740 1 -- start start 2024-09-10T07:40:03.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.941+0000 7fde43eb4740 1 Processor -- start 2024-09-10T07:40:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.941+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a44aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde43eb4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44a1d70 0x5651a44a6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde43eb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b6750 con 0x5651a44974b0 2024-09-10T07:40:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde43eb4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b6080 con 0x5651a44a1d70 2024-09-10T07:40:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde43eb4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b3260 con 0x5651a4496b90 2024-09-10T07:40:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde43eb4740 10 client.0 _ll_register_callbacks cb 0x7fde2c010b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:03.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde3a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a44aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde3a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a44aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40216/0 (socket says 172.21.15.46:40216) 2024-09-10T07:40:03.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde3a800640 1 -- 172.21.15.46:0/865300930 learned_addr learned my addr 172.21.15.46:0/865300930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:03.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde33e00640 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.942+0000 7fde3b200640 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44a1d70 0x5651a44a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.943+0000 7fde3a800640 1 -- 172.21.15.46:0/865300930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44a1d70 msgr2=0x5651a44a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:03.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.943+0000 7fde3a800640 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44a1d70 0x5651a44a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.943+0000 7fde3a800640 1 -- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 msgr2=0x5651a44a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.943+0000 7fde3a800640 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.943+0000 7fde3a800640 1 -- 172.21.15.46:0/865300930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a451b960 con 0x5651a4496b90 2024-09-10T07:40:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.944+0000 7fde33e00640 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.944+0000 7fde3a800640 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a44aa6d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde2c031df0 tx=0x7fde2c01bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69425ad5e1ab7813 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.945+0000 7fde1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:03.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.946+0000 7fde1fe00640 1 -- 172.21.15.46:0/865300930 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c041020 con 0x5651a4496b90 2024-09-10T07:40:03.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.946+0000 7fde1fe00640 1 -- 172.21.15.46:0/865300930 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde2c031a10 con 0x5651a4496b90 2024-09-10T07:40:03.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.947+0000 7fde43eb4740 1 -- 172.21.15.46:0/865300930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde2c01ba00 con 0x5651a4496b90 2024-09-10T07:40:03.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.947+0000 7fde1fe00640 1 -- 172.21.15.46:0/865300930 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fde2c03a410 con 0x5651a4496b90 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.947+0000 7fde43eb4740 10 client.26206 fetch_fsmap learned FSMap version 37 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.947+0000 7fde43eb4740 1 -- 172.21.15.46:0/865300930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5651a451cb50 con 0x5651a4496b90 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.948+0000 7fde1fe00640 1 -- 172.21.15.46:0/865300930 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde2c01ba00 con 0x5651a4496b90 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.948+0000 7fde1fe00640 1 -- 172.21.15.46:0/865300930 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7fde2c027760 con 0x5651a4496b90 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.948+0000 7fde43eb4740 10 client.26206 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.948+0000 7fde43eb4740 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.948+0000 7fde43eb4740 20 client.26206 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.949+0000 7fde43eb4740 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651a451d110 0x5651a451f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.949+0000 7fde43eb4740 4 client.26206 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:40:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.949+0000 7fde43eb4740 1 -- 172.21.15.46:0/865300930 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651a4404e60 con 0x5651a451d110 2024-09-10T07:40:03.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.949+0000 7fde33e00640 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651a451d110 0x5651a451f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:03.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.951+0000 7fde33e00640 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651a451d110 0x5651a451f530 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d97e41e87a55563 server_cookie=a6ab0735b2232e1d in_seq=0 out_seq=0 2024-09-10T07:40:03.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.951+0000 7fde1fe00640 10 client.26206 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.951+0000 7fde1fe00640 1 -- 172.21.15.46:0/865300930 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651a4404e60 con 0x5651a451d110 2024-09-10T07:40:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.951+0000 7fde1fe00640 10 client.26206 handle_command_reply: tid=0 2024-09-10T07:40:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.952+0000 7fde43eb4740 1 client.26206 shutdown 2024-09-10T07:40:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.952+0000 7fde43eb4740 1 -- 172.21.15.46:0/865300930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 msgr2=0x5651a44aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.952+0000 7fde43eb4740 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a44aa6d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde2c031df0 tx=0x7fde2c01bde0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.956+0000 7fde43eb4740 1 -- 172.21.15.46:0/865300930 shutdown_connections 2024-09-10T07:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.956+0000 7fde43eb4740 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5651a451d110 0x5651a451f530 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.956+0000 7fde43eb4740 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a44a1d70 0x5651a44a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.956+0000 7fde43eb4740 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651a44974b0 0x5651a44a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.956+0000 7fde43eb4740 1 --2- 172.21.15.46:0/865300930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651a4496b90 0x5651a44aa6d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.956+0000 7fde43eb4740 1 -- 172.21.15.46:0/865300930 >> 172.21.15.46:0/865300930 conn(0x5651a4493120 msgr2=0x5651a4498e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.957+0000 7fde1fe00640 0 client.26206 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.957+0000 7fde43eb4740 1 -- 172.21.15.46:0/865300930 shutdown_connections 2024-09-10T07:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.957+0000 7fde43eb4740 1 -- 172.21.15.46:0/865300930 wait complete. 2024-09-10T07:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.957+0000 7fde43eb4740 20 client.26206 trim_cache size 0 max 16384 2024-09-10T07:40:03.957 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:40:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:40:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:40:03.958 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:40:03.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.960+0000 7fde41400640 1 -- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde200a2c00 msgr2=0x7fde200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.960+0000 7fde41400640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde200a2c00 0x7fde200a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00afd0 tx=0x7fde2c00a380 comp rx=0 tx=0).stop 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 -- 172.21.15.46:0/604362356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10ac90 msgr2=0x7fde3c10f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10ac90 0x7fde3c10f220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fde34009ca0 tx=0x7fde34003f10 comp rx=0 tx=0).stop 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 -- 172.21.15.46:0/604362356 shutdown_connections 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10ac90 0x7fde3c10f220 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde200a2c00 0x7fde200a50c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde3c104f30 0x7fde3c10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 --2- 172.21.15.46:0/604362356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde3c100620 0x7fde3c10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 -- 172.21.15.46:0/604362356 >> 172.21.15.46:0/604362356 conn(0x7fde3c0fba90 msgr2=0x7fde3c10b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 -- 172.21.15.46:0/604362356 shutdown_connections 2024-09-10T07:40:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:03.961+0000 7fde41400640 1 -- 172.21.15.46:0/604362356 wait complete. 2024-09-10T07:40:04.365 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:04 smithi046 ceph-mon[30705]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 685 op/s 2024-09-10T07:40:04.365 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:04 smithi046 ceph-mon[30705]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.1 MiB/s wr, 630 op/s 2024-09-10T07:40:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:04 smithi181 ceph-mon[35397]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 685 op/s 2024-09-10T07:40:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:04 smithi181 ceph-mon[35397]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.1 MiB/s wr, 630 op/s 2024-09-10T07:40:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:04 smithi063 ceph-mon[35020]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 685 op/s 2024-09-10T07:40:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:04 smithi063 ceph-mon[35020]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.1 MiB/s wr, 630 op/s 2024-09-10T07:40:04.971 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:40:04.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:40:05.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:05 smithi046 ceph-mon[30705]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 623 op/s 2024-09-10T07:40:05.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:05 smithi181 ceph-mon[35397]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 623 op/s 2024-09-10T07:40:05.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:05 smithi063 ceph-mon[35020]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 623 op/s 2024-09-10T07:40:05.602 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 165.0 s, 1659.1 tps, lat 60.203 ms stddev 72.008 2024-09-10T07:40:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.410+0000 7ff9bae00640 1 -- 172.21.15.46:0/2220992040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b4101dd0 msgr2=0x7ff9b41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.410+0000 7ff9bae00640 1 --2- 172.21.15.46:0/2220992040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b4101dd0 0x7ff9b41021b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009900 tx=0x7ff9a402a420 comp rx=0 tx=0).stop 2024-09-10T07:40:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.411+0000 7ff9bae00640 1 -- 172.21.15.46:0/2220992040 shutdown_connections 2024-09-10T07:40:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.411+0000 7ff9bae00640 1 --2- 172.21.15.46:0/2220992040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b4106f00 0x7ff9b410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.411+0000 7ff9bae00640 1 --2- 172.21.15.46:0/2220992040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b4102780 0x7ff9b41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.411+0000 7ff9bae00640 1 --2- 172.21.15.46:0/2220992040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b4101dd0 0x7ff9b41021b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.411+0000 7ff9bae00640 1 -- 172.21.15.46:0/2220992040 >> 172.21.15.46:0/2220992040 conn(0x7ff9b40fd890 msgr2=0x7ff9b40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.411+0000 7ff9bae00640 1 -- 172.21.15.46:0/2220992040 shutdown_connections 2024-09-10T07:40:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.411+0000 7ff9bae00640 1 -- 172.21.15.46:0/2220992040 wait complete. 2024-09-10T07:40:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.412+0000 7ff9bae00640 1 Processor -- start 2024-09-10T07:40:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.412+0000 7ff9bae00640 1 -- start start 2024-09-10T07:40:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.413+0000 7ff9bae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b4101dd0 0x7ff9b419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.413+0000 7ff9bae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b4102780 0x7ff9b419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.413+0000 7ff9bae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b4106f00 0x7ff9b419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.413+0000 7ff9bae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4114290 con 0x7ff9b4101dd0 2024-09-10T07:40:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.413+0000 7ff9bae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4114110 con 0x7ff9b4102780 2024-09-10T07:40:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.413+0000 7ff9bae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4114410 con 0x7ff9b4106f00 2024-09-10T07:40:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.413+0000 7ff9b8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b4106f00 0x7ff9b419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b4102780 0x7ff9b419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b4102780 0x7ff9b419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45458/0 (socket says 172.21.15.46:45458) 2024-09-10T07:40:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3400640 1 -- 172.21.15.46:0/3527723557 learned_addr learned my addr 172.21.15.46:0/3527723557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3e00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b4101dd0 0x7ff9b419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3400640 1 -- 172.21.15.46:0/3527723557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b4106f00 msgr2=0x7ff9b419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3400640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b4106f00 0x7ff9b419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3400640 1 -- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b4101dd0 msgr2=0x7ff9b419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3400640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b4101dd0 0x7ff9b419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.414+0000 7ff9b3400640 1 -- 172.21.15.46:0/3527723557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b419f6e0 con 0x7ff9b4102780 2024-09-10T07:40:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.415+0000 7ff9b3400640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b4102780 0x7ff9b419ac70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b4a0 tx=0x7ff9a000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.415+0000 7ff9b0c00640 1 -- 172.21.15.46:0/3527723557 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0014070 con 0x7ff9b4102780 2024-09-10T07:40:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.416+0000 7ff9b0c00640 1 -- 172.21.15.46:0/3527723557 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a0010af0 con 0x7ff9b4102780 2024-09-10T07:40:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.416+0000 7ff9b0c00640 1 -- 172.21.15.46:0/3527723557 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0017d10 con 0x7ff9b4102780 2024-09-10T07:40:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.416+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b419f9d0 con 0x7ff9b4102780 2024-09-10T07:40:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.416+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b410f5d0 con 0x7ff9b4102780 2024-09-10T07:40:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.418+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b41032b0 con 0x7ff9b4102780 2024-09-10T07:40:06.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.424+0000 7ff9b0c00640 1 -- 172.21.15.46:0/3527723557 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9a0017070 con 0x7ff9b4102780 2024-09-10T07:40:06.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.427+0000 7ff9b0c00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9980a2c00 0x7ff9980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:06.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.427+0000 7ff9b0c00640 1 -- 172.21.15.46:0/3527723557 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff9a00ce730 con 0x7ff9b4102780 2024-09-10T07:40:06.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.429+0000 7ff9b3e00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9980a2c00 0x7ff9980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:06.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.429+0000 7ff9b0c00640 1 -- 172.21.15.46:0/3527723557 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9a009b310 con 0x7ff9b4102780 2024-09-10T07:40:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.430+0000 7ff9b3e00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9980a2c00 0x7ff9980a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4000c00 tx=0x7ff9a4042040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.868+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9b41021b0 con 0x7ff9b4102780 2024-09-10T07:40:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.869+0000 7ff9b0c00640 1 -- 172.21.15.46:0/3527723557 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11976 (secure 0 0 0) 0x7ff9b41021b0 con 0x7ff9b4102780 2024-09-10T07:40:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:40:06.869 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"btime":"2024-09-10T07:40:06:106302+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:40:05.500206+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:40:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 38 2024-09-10T07:40:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9980a2c00 msgr2=0x7ff9980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9980a2c00 0x7ff9980a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4000c00 tx=0x7ff9a4042040 comp rx=0 tx=0).stop 2024-09-10T07:40:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b4102780 msgr2=0x7ff9b419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b4102780 0x7ff9b419ac70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b4a0 tx=0x7ff9a000b960 comp rx=0 tx=0).stop 2024-09-10T07:40:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 shutdown_connections 2024-09-10T07:40:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b4106f00 0x7ff9b419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b4102780 0x7ff9b419ac70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9980a2c00 0x7ff9980a50c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 --2- 172.21.15.46:0/3527723557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b4101dd0 0x7ff9b419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 >> 172.21.15.46:0/3527723557 conn(0x7ff9b40fd890 msgr2=0x7ff9b40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.874+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 shutdown_connections 2024-09-10T07:40:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:06.875+0000 7ff9bae00640 1 -- 172.21.15.46:0/3527723557 wait complete. 2024-09-10T07:40:07.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:07 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:40:07.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:07 smithi046 ceph-mon[30705]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 715 op/s 2024-09-10T07:40:07.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:07 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3527723557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:07.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:07 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:40:07.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:07 smithi181 ceph-mon[35397]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 715 op/s 2024-09-10T07:40:07.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:07 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3527723557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:07.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:07 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:40:07.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:07 smithi063 ceph-mon[35020]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 715 op/s 2024-09-10T07:40:07.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:07 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3527723557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:07.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:40:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.327+0000 7fa06ba00640 1 -- 172.21.15.46:0/216071359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0641090f0 msgr2=0x7fa064110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.327+0000 7fa06ba00640 1 --2- 172.21.15.46:0/216071359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0641090f0 0x7fa064110c70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa054009860 tx=0x7fa05402a340 comp rx=0 tx=0).stop 2024-09-10T07:40:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.328+0000 7fa06ba00640 1 -- 172.21.15.46:0/216071359 shutdown_connections 2024-09-10T07:40:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.328+0000 7fa06ba00640 1 --2- 172.21.15.46:0/216071359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0641090f0 0x7fa064110c70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.328+0000 7fa06ba00640 1 --2- 172.21.15.46:0/216071359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa064104970 0x7fa0641089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.328+0000 7fa06ba00640 1 --2- 172.21.15.46:0/216071359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa064103fc0 0x7fa0641043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.328+0000 7fa06ba00640 1 -- 172.21.15.46:0/216071359 >> 172.21.15.46:0/216071359 conn(0x7fa0640ffa20 msgr2=0x7fa064101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.328+0000 7fa06ba00640 1 -- 172.21.15.46:0/216071359 shutdown_connections 2024-09-10T07:40:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.328+0000 7fa06ba00640 1 -- 172.21.15.46:0/216071359 wait complete. 2024-09-10T07:40:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.330+0000 7fa06ba00640 1 Processor -- start 2024-09-10T07:40:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.330+0000 7fa06ba00640 1 -- start start 2024-09-10T07:40:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa06ba00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa064103fc0 0x7fa06419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa06ba00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa064104970 0x7fa06419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa06ba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0641090f0 0x7fa06419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa06ba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa064114100 con 0x7fa0641090f0 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa06ba00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa064113f80 con 0x7fa064104970 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa06ba00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa064114280 con 0x7fa064103fc0 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa063e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa064104970 0x7fa06419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa069000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa064103fc0 0x7fa06419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa063e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa064104970 0x7fa06419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45474/0 (socket says 172.21.15.46:45474) 2024-09-10T07:40:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa063e00640 1 -- 172.21.15.46:0/2841342162 learned_addr learned my addr 172.21.15.46:0/2841342162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa069a00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0641090f0 0x7fa06419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa069000640 1 -- 172.21.15.46:0/2841342162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa064104970 msgr2=0x7fa06419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa069000640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa064104970 0x7fa06419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa069000640 1 -- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0641090f0 msgr2=0x7fa06419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa069000640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0641090f0 0x7fa06419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.331+0000 7fa069000640 1 -- 172.21.15.46:0/2841342162 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06419f5f0 con 0x7fa064103fc0 2024-09-10T07:40:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.332+0000 7fa069000640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa064103fc0 0x7fa06419a640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa05000d6d0 tx=0x7fa05000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.332+0000 7fa061600640 1 -- 172.21.15.46:0/2841342162 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050017070 con 0x7fa064103fc0 2024-09-10T07:40:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.332+0000 7fa061600640 1 -- 172.21.15.46:0/2841342162 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa050009820 con 0x7fa064103fc0 2024-09-10T07:40:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.332+0000 7fa061600640 1 -- 172.21.15.46:0/2841342162 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050004d00 con 0x7fa064103fc0 2024-09-10T07:40:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.333+0000 7fa06ba00640 1 -- 172.21.15.46:0/2841342162 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06419f8e0 con 0x7fa064103fc0 2024-09-10T07:40:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.333+0000 7fa06ba00640 1 -- 172.21.15.46:0/2841342162 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa064076630 con 0x7fa064103fc0 2024-09-10T07:40:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.339+0000 7fa06e5bf740 1 Processor -- start 2024-09-10T07:40:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.339+0000 7fa06e5bf740 1 -- start start 2024-09-10T07:40:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.341+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b04050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.341+0000 7fa061600640 1 -- 172.21.15.46:0/2841342162 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa0500099d0 con 0x7fa064103fc0 2024-09-10T07:40:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.342+0000 7fa069a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b04050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.342+0000 7fa069a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b04050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53436/0 (socket says 172.21.15.46:53436) 2024-09-10T07:40:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.343+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b0e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.343+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b11240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.343+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a453a6a3e0 con 0x55a453b03c70 2024-09-10T07:40:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.343+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a453a11d80 con 0x55a453b0ee50 2024-09-10T07:40:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.343+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a453a663a0 con 0x55a453b04590 2024-09-10T07:40:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.343+0000 7fa069a00640 1 -- 172.21.15.46:0/3808951843 learned_addr learned my addr 172.21.15.46:0/3808951843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:09.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.343+0000 7fa061600640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0480a2c40 0x7fa0480a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.343+0000 7fa069a00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0480a2c40 0x7fa0480a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa063e00640 1 --2- 172.21.15.46:0/3808951843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b11240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa069a00640 1 -- 172.21.15.46:0/3808951843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 msgr2=0x55a453b0e910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa069a00640 1 --2- 172.21.15.46:0/3808951843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b0e910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa069a00640 1 -- 172.21.15.46:0/3808951843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 msgr2=0x55a453b11240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa069a00640 1 --2- 172.21.15.46:0/3808951843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b11240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa069a00640 1 -- 172.21.15.46:0/3808951843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a453a30ae0 con 0x55a453b03c70 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa061600640 1 -- 172.21.15.46:0/2841342162 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa0500ce960 con 0x7fa064103fc0 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa069a00640 1 --2- 172.21.15.46:0/3808951843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b04050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa054035040 tx=0x7fa054030ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b511e5cc80397b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.344+0000 7fa069a00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0480a2c40 0x7fa0480a5100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa054044000 tx=0x7fa054046000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.345+0000 7fa05a000640 1 -- 172.21.15.46:0/3808951843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054037040 con 0x55a453b03c70 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.345+0000 7fa05a000640 1 -- 172.21.15.46:0/3808951843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0540043e0 con 0x55a453b03c70 2024-09-10T07:40:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.345+0000 7fa05a000640 1 -- 172.21.15.46:0/3808951843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05404a660 con 0x55a453b03c70 2024-09-10T07:40:09.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.347+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3808951843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 msgr2=0x55a453b04050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.347+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3808951843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b04050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa054035040 tx=0x7fa054030ee0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.348+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3808951843 shutdown_connections 2024-09-10T07:40:09.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.348+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3808951843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b11240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.348+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3808951843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b0e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.348+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3808951843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b04050 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.348+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3808951843 >> 172.21.15.46:0/3808951843 conn(0x55a453b00200 msgr2=0x55a453b005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:09.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.348+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3808951843 shutdown_connections 2024-09-10T07:40:09.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.348+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3808951843 wait complete. 2024-09-10T07:40:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.350+0000 7fa06e5bf740 1 -- start start 2024-09-10T07:40:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.351+0000 7fa06e5bf740 1 Processor -- start 2024-09-10T07:40:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.351+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b03c70 0x55a453b0be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.351+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b0c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.351+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.351+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a453b11a20 con 0x55a453b04590 2024-09-10T07:40:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.351+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a453a250f0 con 0x55a453b0ee50 2024-09-10T07:40:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.351+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a453b11ba0 con 0x55a453b03c70 2024-09-10T07:40:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.352+0000 7fa069000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b03c70 0x55a453b0be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.352+0000 7fa069000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b03c70 0x55a453b0be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40604/0 (socket says 172.21.15.46:40604) 2024-09-10T07:40:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.352+0000 7fa069000640 1 -- 172.21.15.46:0/571538525 learned_addr learned my addr 172.21.15.46:0/571538525 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.352+0000 7fa06e5bf740 10 client.0 _ll_register_callbacks cb 0x55a453a0cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.352+0000 7fa063e00640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b0c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.352+0000 7fa069000640 1 -- 172.21.15.46:0/571538525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 msgr2=0x55a453b0c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.353+0000 7fa069a00640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.353+0000 7fa069000640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b0c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.353+0000 7fa069000640 1 -- 172.21.15.46:0/571538525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 msgr2=0x55a453b0c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.353+0000 7fa069000640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b0c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.353+0000 7fa069000640 1 -- 172.21.15.46:0/571538525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4537db3c0 con 0x55a453b03c70 2024-09-10T07:40:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.353+0000 7fa069a00640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b0c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.353+0000 7fa063e00640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b0c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.354+0000 7fa069000640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b03c70 0x55a453b0be00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa050099d00 tx=0x7fa050009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b04c72cc14df760 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.354+0000 7fa04ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.355+0000 7fa04ea00640 1 -- 172.21.15.46:0/571538525 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05009d070 con 0x55a453b03c70 2024-09-10T07:40:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.355+0000 7fa04ea00640 1 -- 172.21.15.46:0/571538525 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa050099400 con 0x55a453b03c70 2024-09-10T07:40:09.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.355+0000 7fa06e5bf740 1 -- 172.21.15.46:0/571538525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0540043e0 con 0x55a453b03c70 2024-09-10T07:40:09.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.356+0000 7fa04ea00640 1 -- 172.21.15.46:0/571538525 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa050020040 con 0x55a453b03c70 2024-09-10T07:40:09.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.356+0000 7fa06e5bf740 10 client.26218 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:09.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.356+0000 7fa06e5bf740 1 -- 172.21.15.46:0/571538525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a453b1db20 con 0x55a453b03c70 2024-09-10T07:40:09.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.356+0000 7fa04ea00640 1 -- 172.21.15.46:0/571538525 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0540043e0 con 0x55a453b03c70 2024-09-10T07:40:09.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.356+0000 7fa04ea00640 1 -- 172.21.15.46:0/571538525 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fa050023060 con 0x55a453b03c70 2024-09-10T07:40:09.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.356+0000 7fa06e5bf740 10 client.26218 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.356+0000 7fa06e5bf740 10 client.26218 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:40:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.357+0000 7fa06e5bf740 20 client.26218 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.357+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55a453b1ef30 0x55a453b21360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.357+0000 7fa06e5bf740 4 client.26218 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.357+0000 7fa06e5bf740 1 -- 172.21.15.46:0/571538525 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a453a77950 con 0x55a453b1ef30 2024-09-10T07:40:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.357+0000 7fa063e00640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55a453b1ef30 0x55a453b21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.358+0000 7fa063e00640 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55a453b1ef30 0x55a453b21360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=18900bfecfc74f8f server_cookie=1b7cd30f52164ed4 in_seq=0 out_seq=0 2024-09-10T07:40:09.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.358+0000 7fa04ea00640 10 client.26218 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:40:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.359+0000 7fa04ea00640 1 -- 172.21.15.46:0/571538525 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a453a77950 con 0x55a453b1ef30 2024-09-10T07:40:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.359+0000 7fa04ea00640 10 client.26218 handle_command_reply: tid=0 2024-09-10T07:40:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.359+0000 7fa06e5bf740 1 client.26218 shutdown 2024-09-10T07:40:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.360+0000 7fa06e5bf740 1 -- 172.21.15.46:0/571538525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b03c70 msgr2=0x55a453b0be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.360+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b03c70 0x55a453b0be00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa050099d00 tx=0x7fa050009f70 comp rx=0 tx=0).stop 2024-09-10T07:40:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.361+0000 7fa06e5bf740 1 -- 172.21.15.46:0/571538525 shutdown_connections 2024-09-10T07:40:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.361+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55a453b1ef30 0x55a453b21360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.361+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b0c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.361+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b0c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.361+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/571538525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b03c70 0x55a453b0be00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.361+0000 7fa06e5bf740 1 -- 172.21.15.46:0/571538525 >> 172.21.15.46:0/571538525 conn(0x55a453b00200 msgr2=0x55a453b05ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.361+0000 7fa04ea00640 0 client.26218 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:40:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.361+0000 7fa06e5bf740 1 -- 172.21.15.46:0/571538525 shutdown_connections 2024-09-10T07:40:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.362+0000 7fa06e5bf740 1 -- 172.21.15.46:0/571538525 wait complete. 2024-09-10T07:40:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.362+0000 7fa06e5bf740 20 client.26218 trim_cache size 0 max 16384 2024-09-10T07:40:09.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.386+0000 7fa06e5bf740 1 Processor -- start 2024-09-10T07:40:09.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.387+0000 7fa06e5bf740 1 -- start start 2024-09-10T07:40:09.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.387+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b0a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.387+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.387+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b26c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.387+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa064071630 con 0x55a453b03c70 2024-09-10T07:40:09.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.387+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa064073200 con 0x55a453b0ee50 2024-09-10T07:40:09.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.387+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa064071d00 con 0x55a453b04590 2024-09-10T07:40:09.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa063e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa063e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40620/0 (socket says 172.21.15.46:40620) 2024-09-10T07:40:09.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa063e00640 1 -- 172.21.15.46:0/193214092 learned_addr learned my addr 172.21.15.46:0/193214092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:09.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa069000640 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b0a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa069a00640 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b26c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa063e00640 1 -- 172.21.15.46:0/193214092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 msgr2=0x55a453b26c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:09.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa063e00640 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b26c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa063e00640 1 -- 172.21.15.46:0/193214092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 msgr2=0x55a453b0a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:09.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa063e00640 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b0a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.388+0000 7fa063e00640 1 -- 172.21.15.46:0/193214092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a453b0adb0 con 0x55a453b04590 2024-09-10T07:40:09.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.389+0000 7fa063e00640 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b066d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55a453b01580 tx=0x7fa05c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da1ec5f14c333c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:09.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.389+0000 7fa059600640 1 -- 172.21.15.46:0/193214092 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c003b70 con 0x55a453b04590 2024-09-10T07:40:09.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.390+0000 7fa059600640 1 -- 172.21.15.46:0/193214092 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05c020070 con 0x55a453b04590 2024-09-10T07:40:09.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.390+0000 7fa059600640 1 -- 172.21.15.46:0/193214092 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00e910 con 0x55a453b04590 2024-09-10T07:40:09.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.391+0000 7fa06e5bf740 1 -- 172.21.15.46:0/193214092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 msgr2=0x55a453b066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.391+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b066d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55a453b01580 tx=0x7fa05c0095c0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.391+0000 7fa06e5bf740 1 -- 172.21.15.46:0/193214092 shutdown_connections 2024-09-10T07:40:09.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.391+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b0ee50 0x55a453b26c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.391+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b04590 0x55a453b066d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.391+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/193214092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b03c70 0x55a453b0a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.391+0000 7fa06e5bf740 1 -- 172.21.15.46:0/193214092 >> 172.21.15.46:0/193214092 conn(0x55a453b00200 msgr2=0x55a453b0c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:09.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.392+0000 7fa06e5bf740 1 -- 172.21.15.46:0/193214092 shutdown_connections 2024-09-10T07:40:09.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.392+0000 7fa06e5bf740 1 -- 172.21.15.46:0/193214092 wait complete. 2024-09-10T07:40:09.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.393+0000 7fa06e5bf740 1 -- start start 2024-09-10T07:40:09.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.393+0000 7fa06e5bf740 1 Processor -- start 2024-09-10T07:40:09.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.393+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b03c70 0x55a453b12be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.393+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b13120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.393+0000 7fa06e5bf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b0ee50 0x55a453b13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.393+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa064072b00 con 0x55a453b04590 2024-09-10T07:40:09.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.393+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa064072430 con 0x55a453b03c70 2024-09-10T07:40:09.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.393+0000 7fa06e5bf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa06406f610 con 0x55a453b0ee50 2024-09-10T07:40:09.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.394+0000 7fa06e5bf740 10 client.0 _ll_register_callbacks cb 0x7fa050005810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:09.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.394+0000 7fa069a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b0ee50 0x55a453b13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.394+0000 7fa069a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b0ee50 0x55a453b13660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40634/0 (socket says 172.21.15.46:40634) 2024-09-10T07:40:09.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.394+0000 7fa069a00640 1 -- 172.21.15.46:0/3708854910 learned_addr learned my addr 172.21.15.46:0/3708854910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:09.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.394+0000 7fa069000640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b03c70 0x55a453b12be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.395+0000 7fa069a00640 1 -- 172.21.15.46:0/3708854910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b03c70 msgr2=0x55a453b12be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:09.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.395+0000 7fa063e00640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b13120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.395+0000 7fa069a00640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b03c70 0x55a453b12be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.395+0000 7fa069a00640 1 -- 172.21.15.46:0/3708854910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 msgr2=0x55a453b13120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:09.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.395+0000 7fa069a00640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b13120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.395+0000 7fa069a00640 1 -- 172.21.15.46:0/3708854910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a453b88cb0 con 0x55a453b0ee50 2024-09-10T07:40:09.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.395+0000 7fa069000640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b03c70 0x55a453b12be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:09.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.395+0000 7fa063e00640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b13120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:09.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.396+0000 7fa069a00640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b0ee50 0x55a453b13660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa054003af0 tx=0x7fa05402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b34119b4a0fb90bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:09.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.396+0000 7fa04ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.397+0000 7fa04ea00640 1 -- 172.21.15.46:0/3708854910 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054037040 con 0x55a453b0ee50 2024-09-10T07:40:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.397+0000 7fa04ea00640 1 -- 172.21.15.46:0/3708854910 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa0540043e0 con 0x55a453b0ee50 2024-09-10T07:40:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.397+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3708854910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c020070 con 0x55a453b0ee50 2024-09-10T07:40:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.398+0000 7fa04ea00640 1 -- 172.21.15.46:0/3708854910 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa054003c50 con 0x55a453b0ee50 2024-09-10T07:40:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.398+0000 7fa06e5bf740 10 client.26224 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.398+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3708854910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a453b89fa0 con 0x55a453b0ee50 2024-09-10T07:40:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa04ea00640 1 -- 172.21.15.46:0/3708854910 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05c020070 con 0x55a453b0ee50 2024-09-10T07:40:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa04ea00640 1 -- 172.21.15.46:0/3708854910 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fa054058020 con 0x55a453b0ee50 2024-09-10T07:40:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa06e5bf740 10 client.26224 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa06e5bf740 10 client.26224 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:40:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa06e5bf740 20 client.26224 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55a453b8a360 0x55a453b8c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa06e5bf740 4 client.26224 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:40:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3708854910 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a453a71e60 con 0x55a453b8a360 2024-09-10T07:40:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.399+0000 7fa069000640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55a453b8a360 0x55a453b8c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.400+0000 7fa04ea00640 10 client.26224 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:40:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.400+0000 7fa069000640 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55a453b8a360 0x55a453b8c780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d853d87ef0090019 server_cookie=9fd2b86b074ac10f in_seq=0 out_seq=0 2024-09-10T07:40:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.400+0000 7fa04ea00640 1 -- 172.21.15.46:0/3708854910 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a453a71e60 con 0x55a453b8a360 2024-09-10T07:40:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.400+0000 7fa04ea00640 10 client.26224 handle_command_reply: tid=0 2024-09-10T07:40:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.400+0000 7fa06e5bf740 1 client.26224 shutdown 2024-09-10T07:40:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3708854910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b0ee50 msgr2=0x55a453b13660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b0ee50 0x55a453b13660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa054003af0 tx=0x7fa05402a340 comp rx=0 tx=0).stop 2024-09-10T07:40:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3708854910 shutdown_connections 2024-09-10T07:40:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55a453b8a360 0x55a453b8c780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a453b0ee50 0x55a453b13660 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a453b04590 0x55a453b13120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa06e5bf740 1 --2- 172.21.15.46:0/3708854910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a453b03c70 0x55a453b12be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3708854910 >> 172.21.15.46:0/3708854910 conn(0x55a453b00200 msgr2=0x55a453b0c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.401+0000 7fa04ea00640 0 client.26224 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:40:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.402+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3708854910 shutdown_connections 2024-09-10T07:40:09.403 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.402+0000 7fa06e5bf740 1 -- 172.21.15.46:0/3708854910 wait complete. 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.402+0000 7fa06e5bf740 20 client.26224 trim_cache size 0 max 16384 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.403+0000 7fa06ba00640 1 -- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0480a2c40 msgr2=0x7fa0480a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.403+0000 7fa06ba00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0480a2c40 0x7fa0480a5100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa054044000 tx=0x7fa054046000 comp rx=0 tx=0).stop 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.404+0000 7fa06ba00640 1 -- 172.21.15.46:0/2841342162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa064103fc0 msgr2=0x7fa06419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.404+0000 7fa06ba00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa064103fc0 0x7fa06419a640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa05000d6d0 tx=0x7fa05000db90 comp rx=0 tx=0).stop 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.404+0000 7fa06ba00640 1 -- 172.21.15.46:0/2841342162 shutdown_connections 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.404+0000 7fa06ba00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0480a2c40 0x7fa0480a5100 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.404+0000 7fa06ba00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0641090f0 0x7fa06419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.404+0000 7fa06ba00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa064104970 0x7fa06419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.404+0000 7fa06ba00640 1 --2- 172.21.15.46:0/2841342162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa064103fc0 0x7fa06419a640 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.404+0000 7fa06ba00640 1 -- 172.21.15.46:0/2841342162 >> 172.21.15.46:0/2841342162 conn(0x7fa0640ffa20 msgr2=0x7fa064100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.405+0000 7fa06ba00640 1 -- 172.21.15.46:0/2841342162 shutdown_connections 2024-09-10T07:40:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:09.409+0000 7fa06ba00640 1 -- 172.21.15.46:0/2841342162 wait complete. 2024-09-10T07:40:10.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:09 smithi181 ceph-mon[35397]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 583 op/s 2024-09-10T07:40:10.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:09 smithi063 ceph-mon[35020]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 583 op/s 2024-09-10T07:40:10.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:09 smithi046 ceph-mon[30705]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 583 op/s 2024-09-10T07:40:10.399 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:40:10.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 170.0 s, 1412.9 tps, lat 71.142 ms stddev 88.796 2024-09-10T07:40:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.862+0000 7fb577800640 1 -- 172.21.15.46:0/3913495469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570102a50 msgr2=0x7fb570102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.862+0000 7fb577800640 1 --2- 172.21.15.46:0/3913495469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570102a50 0x7fb570102e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb564009900 tx=0x7fb56402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.863+0000 7fb577800640 1 -- 172.21.15.46:0/3913495469 shutdown_connections 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.863+0000 7fb577800640 1 --2- 172.21.15.46:0/3913495469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb570100c80 0x7fb570110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.863+0000 7fb577800640 1 --2- 172.21.15.46:0/3913495469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb570100100 0x7fb570100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.863+0000 7fb577800640 1 --2- 172.21.15.46:0/3913495469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570102a50 0x7fb570102e30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.863+0000 7fb577800640 1 -- 172.21.15.46:0/3913495469 >> 172.21.15.46:0/3913495469 conn(0x7fb570077230 msgr2=0x7fb5700fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.863+0000 7fb577800640 1 -- 172.21.15.46:0/3913495469 shutdown_connections 2024-09-10T07:40:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.863+0000 7fb577800640 1 -- 172.21.15.46:0/3913495469 wait complete. 2024-09-10T07:40:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.864+0000 7fb577800640 1 Processor -- start 2024-09-10T07:40:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.864+0000 7fb577800640 1 -- start start 2024-09-10T07:40:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb577800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570100100 0x7fb570110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb577800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb570100c80 0x7fb57010d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb577800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb570102a50 0x7fb57010dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb577800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb570114150 con 0x7fb570102a50 2024-09-10T07:40:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb577800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb570113fd0 con 0x7fb570100c80 2024-09-10T07:40:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb577800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701142d0 con 0x7fb570100100 2024-09-10T07:40:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb574e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570100100 0x7fb570110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb570100c80 0x7fb57010d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb574e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570100100 0x7fb570110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40664/0 (socket says 172.21.15.46:40664) 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb574e00640 1 -- 172.21.15.46:0/3592444906 learned_addr learned my addr 172.21.15.46:0/3592444906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.865+0000 7fb575800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb570102a50 0x7fb57010dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.866+0000 7fb574e00640 1 -- 172.21.15.46:0/3592444906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb570100c80 msgr2=0x7fb57010d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.866+0000 7fb574e00640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb570100c80 0x7fb57010d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.866+0000 7fb574e00640 1 -- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb570102a50 msgr2=0x7fb57010dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.866+0000 7fb574e00640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb570102a50 0x7fb57010dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.866+0000 7fb574e00640 1 -- 172.21.15.46:0/3592444906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5701aae90 con 0x7fb570100100 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.866+0000 7fb56fe00640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb570100c80 0x7fb57010d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.866+0000 7fb574e00640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570100100 0x7fb570110720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb56402a960 tx=0x7fb564003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.867+0000 7fb56d600640 1 -- 172.21.15.46:0/3592444906 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56403f070 con 0x7fb570100100 2024-09-10T07:40:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.867+0000 7fb56d600640 1 -- 172.21.15.46:0/3592444906 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb564031c90 con 0x7fb570100100 2024-09-10T07:40:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.867+0000 7fb56d600640 1 -- 172.21.15.46:0/3592444906 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564035030 con 0x7fb570100100 2024-09-10T07:40:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.867+0000 7fb577800640 1 -- 172.21.15.46:0/3592444906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5701ab060 con 0x7fb570100100 2024-09-10T07:40:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.867+0000 7fb577800640 1 -- 172.21.15.46:0/3592444906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701ab400 con 0x7fb570100100 2024-09-10T07:40:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.874+0000 7fb57a458740 1 Processor -- start 2024-09-10T07:40:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb57a458740 1 -- start start 2024-09-10T07:40:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb57a458740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c55c70 0x55c7f1c56050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb57a458740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c60910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb57a458740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 0x55c7f1c63240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb57a458740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1bbc3e0 con 0x55c7f1c60e50 2024-09-10T07:40:11.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb57a458740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1b63d80 con 0x55c7f1c55c70 2024-09-10T07:40:11.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb57a458740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1bb83a0 con 0x55c7f1c56590 2024-09-10T07:40:11.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb575800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c55c70 0x55c7f1c56050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb575800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c55c70 0x55c7f1c56050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45540/0 (socket says 172.21.15.46:45540) 2024-09-10T07:40:11.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb575800640 1 -- 172.21.15.46:0/3325459344 learned_addr learned my addr 172.21.15.46:0/3325459344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:11.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.875+0000 7fb56fe00640 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 0x55c7f1c63240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb575800640 1 -- 172.21.15.46:0/3325459344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 msgr2=0x55c7f1c60910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb575800640 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c60910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb575800640 1 -- 172.21.15.46:0/3325459344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 msgr2=0x55c7f1c63240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb575800640 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 0x55c7f1c63240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb575800640 1 -- 172.21.15.46:0/3325459344 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f1b82ae0 con 0x55c7f1c55c70 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb56fe00640 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 0x55c7f1c63240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb575800640 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c55c70 0x55c7f1c56050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb568009f00 tx=0x7fb5680107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2636e2f9223801d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb562000640 1 -- 172.21.15.46:0/3325459344 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568003b70 con 0x55c7f1c55c70 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb562000640 1 -- 172.21.15.46:0/3325459344 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb568003d10 con 0x55c7f1c55c70 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb562000640 1 -- 172.21.15.46:0/3325459344 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568011a50 con 0x55c7f1c55c70 2024-09-10T07:40:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.876+0000 7fb56d600640 1 -- 172.21.15.46:0/3592444906 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb564042070 con 0x7fb570100100 2024-09-10T07:40:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.877+0000 7fb57a458740 1 -- 172.21.15.46:0/3325459344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c55c70 msgr2=0x55c7f1c56050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.877+0000 7fb57a458740 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c55c70 0x55c7f1c56050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb568009f00 tx=0x7fb5680107a0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.877+0000 7fb57a458740 1 -- 172.21.15.46:0/3325459344 shutdown_connections 2024-09-10T07:40:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.877+0000 7fb57a458740 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 0x55c7f1c63240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.877+0000 7fb57a458740 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c60910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.877+0000 7fb57a458740 1 --2- 172.21.15.46:0/3325459344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c55c70 0x55c7f1c56050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.878+0000 7fb57a458740 1 -- 172.21.15.46:0/3325459344 >> 172.21.15.46:0/3325459344 conn(0x55c7f1c52200 msgr2=0x55c7f1c525e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.880+0000 7fb57a458740 1 -- 172.21.15.46:0/3325459344 shutdown_connections 2024-09-10T07:40:11.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.880+0000 7fb57a458740 1 -- 172.21.15.46:0/3325459344 wait complete. 2024-09-10T07:40:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.881+0000 7fb57a458740 1 -- start start 2024-09-10T07:40:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.881+0000 7fb57a458740 1 Processor -- start 2024-09-10T07:40:11.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.883+0000 7fb56d600640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5580a2b50 0x7fb5580a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.883+0000 7fb57a458740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c55c70 0x55c7f1c5dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb57a458740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb57a458740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 0x55c7f1c5e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb57a458740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1c63a20 con 0x55c7f1c60e50 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb57a458740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1b770f0 con 0x55c7f1c56590 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb57a458740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1c63ba0 con 0x55c7f1c55c70 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb575800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5580a2b50 0x7fb5580a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb57a458740 10 client.0 _ll_register_callbacks cb 0x55c7f1b5ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb574e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c55c70 0x55c7f1c5dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb574e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c55c70 0x55c7f1c5dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40666/0 (socket says 172.21.15.46:40666) 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb574e00640 1 -- 172.21.15.46:0/3506327754 learned_addr learned my addr 172.21.15.46:0/3506327754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb56d600640 1 -- 172.21.15.46:0/3592444906 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb5640ef800 con 0x7fb570100100 2024-09-10T07:40:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb574e00640 1 -- 172.21.15.46:0/3506327754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c56590 msgr2=0x55c7f1c5e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb574e00640 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb574e00640 1 -- 172.21.15.46:0/3506327754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 msgr2=0x55c7f1c5e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.884+0000 7fb574e00640 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 0x55c7f1c5e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb574e00640 1 -- 172.21.15.46:0/3506327754 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f192d3c0 con 0x55c7f1c55c70 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb56fe00640 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb574e00640 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c55c70 0x55c7f1c5dcf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5640b8090 tx=0x7fb56403b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9da4c0bae0c2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb575800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5580a2b50 0x7fb5580a5010 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb568000f80 tx=0x7fb568011610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb556a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb556a00640 1 -- 172.21.15.46:0/3506327754 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564049070 con 0x55c7f1c55c70 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb556a00640 1 -- 172.21.15.46:0/3506327754 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb5640b82f0 con 0x55c7f1c55c70 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb57a458740 1 -- 172.21.15.46:0/3506327754 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb568003d10 con 0x55c7f1c55c70 2024-09-10T07:40:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.885+0000 7fb556a00640 1 -- 172.21.15.46:0/3506327754 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb5640bbc80 con 0x55c7f1c55c70 2024-09-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.889+0000 7fb57a458740 10 client.26233 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.889+0000 7fb57a458740 1 -- 172.21.15.46:0/3506327754 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c7f1c63a20 con 0x55c7f1c55c70 2024-09-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.890+0000 7fb556a00640 1 -- 172.21.15.46:0/3506327754 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb568003d10 con 0x55c7f1c55c70 2024-09-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.890+0000 7fb556a00640 1 -- 172.21.15.46:0/3506327754 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fb5640b61a0 con 0x55c7f1c55c70 2024-09-10T07:40:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.890+0000 7fb57a458740 10 client.26233 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.890+0000 7fb57a458740 10 client.26233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:40:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.890+0000 7fb57a458740 20 client.26233 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.891+0000 7fb57a458740 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55c7f1c70e70 0x55c7f1c73290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.891+0000 7fb57a458740 4 client.26233 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.891+0000 7fb57a458740 1 -- 172.21.15.46:0/3506327754 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7f1bc9950 con 0x55c7f1c70e70 2024-09-10T07:40:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.891+0000 7fb56fe00640 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55c7f1c70e70 0x55c7f1c73290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.892+0000 7fb56fe00640 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55c7f1c70e70 0x55c7f1c73290 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10cab4d615221abc server_cookie=2094f528a4833bed in_seq=0 out_seq=0 2024-09-10T07:40:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.892+0000 7fb556a00640 10 client.26233 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:40:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.893+0000 7fb556a00640 1 -- 172.21.15.46:0/3506327754 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7f1bc9950 con 0x55c7f1c70e70 2024-09-10T07:40:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.893+0000 7fb556a00640 10 client.26233 handle_command_reply: tid=0 2024-09-10T07:40:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.893+0000 7fb57a458740 1 client.26233 shutdown 2024-09-10T07:40:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 -- 172.21.15.46:0/3506327754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c55c70 msgr2=0x55c7f1c5dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c55c70 0x55c7f1c5dcf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5640b8090 tx=0x7fb56403b720 comp rx=0 tx=0).stop 2024-09-10T07:40:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 -- 172.21.15.46:0/3506327754 shutdown_connections 2024-09-10T07:40:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c60e50 0x55c7f1c5e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55c7f1c70e70 0x55c7f1c73290 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 --2- 172.21.15.46:0/3506327754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c55c70 0x55c7f1c5dcf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 -- 172.21.15.46:0/3506327754 >> 172.21.15.46:0/3506327754 conn(0x55c7f1c52200 msgr2=0x55c7f1c51380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb556a00640 0 client.26233 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:40:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.894+0000 7fb57a458740 1 -- 172.21.15.46:0/3506327754 shutdown_connections 2024-09-10T07:40:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.895+0000 7fb57a458740 1 -- 172.21.15.46:0/3506327754 wait complete. 2024-09-10T07:40:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.895+0000 7fb57a458740 20 client.26233 trim_cache size 0 max 16384 2024-09-10T07:40:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.920+0000 7fb57a458740 1 Processor -- start 2024-09-10T07:40:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.920+0000 7fb57a458740 1 -- start start 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb57a458740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb57a458740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c60e50 0x55c7f1c78b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb57a458740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c790c0 0x55c7f1c5af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb57a458740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701b5260 con 0x55c7f1c56590 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb57a458740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701b6e30 con 0x55c7f1c790c0 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb57a458740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701b5930 con 0x55c7f1c60e50 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb575800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c790c0 0x55c7f1c5af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c60e50 0x55c7f1c78b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb575800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c790c0 0x55c7f1c5af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45560/0 (socket says 172.21.15.46:45560) 2024-09-10T07:40:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.921+0000 7fb575800640 1 -- 172.21.15.46:0/4184323305 learned_addr learned my addr 172.21.15.46:0/4184323305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb574e00640 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb56fe00640 1 -- 172.21.15.46:0/4184323305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c790c0 msgr2=0x55c7f1c5af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb56fe00640 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c790c0 0x55c7f1c5af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb56fe00640 1 -- 172.21.15.46:0/4184323305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c56590 msgr2=0x55c7f1c5e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb56fe00640 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb56fe00640 1 -- 172.21.15.46:0/4184323305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f1c4d770 con 0x55c7f1c60e50 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb574e00640 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb575800640 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c790c0 0x55c7f1c5af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.922+0000 7fb56fe00640 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c60e50 0x55c7f1c78b80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55c7f1c5f330 tx=0x7fb55c00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73102e538a63dce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.923+0000 7fb561600640 1 -- 172.21.15.46:0/4184323305 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c023070 con 0x55c7f1c60e50 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.923+0000 7fb561600640 1 -- 172.21.15.46:0/4184323305 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb55c012e30 con 0x55c7f1c60e50 2024-09-10T07:40:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.923+0000 7fb561600640 1 -- 172.21.15.46:0/4184323305 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c010cc0 con 0x55c7f1c60e50 2024-09-10T07:40:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.924+0000 7fb57a458740 1 -- 172.21.15.46:0/4184323305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c60e50 msgr2=0x55c7f1c78b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.924+0000 7fb57a458740 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c60e50 0x55c7f1c78b80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55c7f1c5f330 tx=0x7fb55c00b390 comp rx=0 tx=0).stop 2024-09-10T07:40:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.924+0000 7fb57a458740 1 -- 172.21.15.46:0/4184323305 shutdown_connections 2024-09-10T07:40:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.924+0000 7fb57a458740 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c790c0 0x55c7f1c5af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.924+0000 7fb57a458740 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c60e50 0x55c7f1c78b80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.924+0000 7fb57a458740 1 --2- 172.21.15.46:0/4184323305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c56590 0x55c7f1c5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.924+0000 7fb57a458740 1 -- 172.21.15.46:0/4184323305 >> 172.21.15.46:0/4184323305 conn(0x55c7f1c52200 msgr2=0x55c7f1c57ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:11.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:11 smithi046 ceph-mon[30705]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 714 op/s 2024-09-10T07:40:11.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.926+0000 7fb57a458740 1 -- 172.21.15.46:0/4184323305 shutdown_connections 2024-09-10T07:40:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.926+0000 7fb57a458740 1 -- 172.21.15.46:0/4184323305 wait complete. 2024-09-10T07:40:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.927+0000 7fb57a458740 1 -- start start 2024-09-10T07:40:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.927+0000 7fb57a458740 1 Processor -- start 2024-09-10T07:40:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.928+0000 7fb57a458740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.928+0000 7fb57a458740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c60e50 0x55c7f1c658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.928+0000 7fb57a458740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c790c0 0x55c7f1cdab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.928+0000 7fb57a458740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701b6730 con 0x55c7f1c790c0 2024-09-10T07:40:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.928+0000 7fb57a458740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701b6060 con 0x55c7f1c60e50 2024-09-10T07:40:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.928+0000 7fb57a458740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701b3240 con 0x55c7f1c56590 2024-09-10T07:40:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.928+0000 7fb57a458740 10 client.0 _ll_register_callbacks cb 0x7fb5640b6640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.929+0000 7fb574e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.929+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c60e50 0x55c7f1c658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.929+0000 7fb574e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40684/0 (socket says 172.21.15.46:40684) 2024-09-10T07:40:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.929+0000 7fb574e00640 1 -- 172.21.15.46:0/3717407466 learned_addr learned my addr 172.21.15.46:0/3717407466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.929+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c60e50 0x55c7f1c658e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45572/0 (socket says 172.21.15.46:45572) 2024-09-10T07:40:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb574e00640 1 -- 172.21.15.46:0/3717407466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c60e50 msgr2=0x55c7f1c658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb574e00640 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c60e50 0x55c7f1c658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb574e00640 1 -- 172.21.15.46:0/3717407466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c790c0 msgr2=0x55c7f1cdab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb574e00640 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c790c0 0x55c7f1cdab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb574e00640 1 -- 172.21.15.46:0/3717407466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f1cec0d0 con 0x55c7f1c56590 2024-09-10T07:40:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb574e00640 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c653a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb564005ab0 tx=0x7fb564031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e366ae80c6fdee09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb556a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb556a00640 1 -- 172.21.15.46:0/3717407466 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56409f060 con 0x55c7f1c56590 2024-09-10T07:40:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb556a00640 1 -- 172.21.15.46:0/3717407466 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb5640317b0 con 0x55c7f1c56590 2024-09-10T07:40:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.930+0000 7fb57a458740 1 -- 172.21.15.46:0/3717407466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb55c012e30 con 0x55c7f1c56590 2024-09-10T07:40:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb556a00640 1 -- 172.21.15.46:0/3717407466 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb564049070 con 0x55c7f1c56590 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb57a458740 10 client.26239 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb57a458740 1 -- 172.21.15.46:0/3717407466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c7f1cec0d0 con 0x55c7f1c56590 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb556a00640 1 -- 172.21.15.46:0/3717407466 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb55c012e30 con 0x55c7f1c56590 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb556a00640 1 -- 172.21.15.46:0/3717407466 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fb5640c4530 con 0x55c7f1c56590 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb57a458740 10 client.26239 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb57a458740 10 client.26239 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb57a458740 20 client.26239 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb57a458740 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55c7f1ced710 0x55c7f1cefb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb57a458740 4 client.26239 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.931+0000 7fb57a458740 1 -- 172.21.15.46:0/3717407466 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7f1bc3e60 con 0x55c7f1ced710 2024-09-10T07:40:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.932+0000 7fb56fe00640 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55c7f1ced710 0x55c7f1cefb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.933+0000 7fb556a00640 10 client.26239 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:40:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.933+0000 7fb56fe00640 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55c7f1ced710 0x55c7f1cefb40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c34f22b9f3a89098 server_cookie=8b3273888db69948 in_seq=0 out_seq=0 2024-09-10T07:40:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.934+0000 7fb556a00640 1 -- 172.21.15.46:0/3717407466 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7f1bc3e60 con 0x55c7f1ced710 2024-09-10T07:40:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.934+0000 7fb556a00640 10 client.26239 handle_command_reply: tid=0 2024-09-10T07:40:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.934+0000 7fb57a458740 1 client.26239 shutdown 2024-09-10T07:40:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.934+0000 7fb57a458740 1 -- 172.21.15.46:0/3717407466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 msgr2=0x55c7f1c653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.934+0000 7fb57a458740 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c653a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb564005ab0 tx=0x7fb564031400 comp rx=0 tx=0).stop 2024-09-10T07:40:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 1 -- 172.21.15.46:0/3717407466 shutdown_connections 2024-09-10T07:40:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7f1c790c0 0x55c7f1cdab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c7f1c60e50 0x55c7f1c658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55c7f1ced710 0x55c7f1cefb40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 1 --2- 172.21.15.46:0/3717407466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7f1c56590 0x55c7f1c653a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb556a00640 0 client.26239 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:40:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 1 -- 172.21.15.46:0/3717407466 >> 172.21.15.46:0/3717407466 conn(0x55c7f1c52200 msgr2=0x55c7f1c576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 1 -- 172.21.15.46:0/3717407466 shutdown_connections 2024-09-10T07:40:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 1 -- 172.21.15.46:0/3717407466 wait complete. 2024-09-10T07:40:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.935+0000 7fb57a458740 20 client.26239 trim_cache size 0 max 16384 2024-09-10T07:40:11.936 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:40:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.936+0000 7fb577800640 1 -- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5580a2b50 msgr2=0x7fb5580a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.936+0000 7fb577800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5580a2b50 0x7fb5580a5010 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb568000f80 tx=0x7fb568011610 comp rx=0 tx=0).stop 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.936+0000 7fb577800640 1 -- 172.21.15.46:0/3592444906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570100100 msgr2=0x7fb570110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.936+0000 7fb577800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570100100 0x7fb570110720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb56402a960 tx=0x7fb564003b90 comp rx=0 tx=0).stop 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.937+0000 7fb577800640 1 -- 172.21.15.46:0/3592444906 shutdown_connections 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.937+0000 7fb577800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5580a2b50 0x7fb5580a5010 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.937+0000 7fb577800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb570102a50 0x7fb57010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.937+0000 7fb577800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb570100c80 0x7fb57010d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.937+0000 7fb577800640 1 --2- 172.21.15.46:0/3592444906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb570100100 0x7fb570110720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.937+0000 7fb577800640 1 -- 172.21.15.46:0/3592444906 >> 172.21.15.46:0/3592444906 conn(0x7fb570077230 msgr2=0x7fb5700fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.937+0000 7fb577800640 1 -- 172.21.15.46:0/3592444906 shutdown_connections 2024-09-10T07:40:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:11.937+0000 7fb577800640 1 -- 172.21.15.46:0/3592444906 wait complete. 2024-09-10T07:40:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:11 smithi181 ceph-mon[35397]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 714 op/s 2024-09-10T07:40:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:11 smithi063 ceph-mon[35020]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 714 op/s 2024-09-10T07:40:13.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:40:13.713 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.802+0000 7ff838a00640 1 -- 172.21.15.46:0/3767359289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff834104f70 msgr2=0x7ff8341053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.802+0000 7ff838a00640 1 --2- 172.21.15.46:0/3767359289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff834104f70 0x7ff8341053f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff824009900 tx=0x7ff82402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.803+0000 7ff838a00640 1 -- 172.21.15.46:0/3767359289 shutdown_connections 2024-09-10T07:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.803+0000 7ff838a00640 1 --2- 172.21.15.46:0/3767359289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff834105930 0x7ff83410c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.803+0000 7ff838a00640 1 --2- 172.21.15.46:0/3767359289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff834104f70 0x7ff8341053f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.803+0000 7ff838a00640 1 --2- 172.21.15.46:0/3767359289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff834103d80 0x7ff834104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.803+0000 7ff838a00640 1 -- 172.21.15.46:0/3767359289 >> 172.21.15.46:0/3767359289 conn(0x7ff8340ff4e0 msgr2=0x7ff834101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.804+0000 7ff838a00640 1 -- 172.21.15.46:0/3767359289 shutdown_connections 2024-09-10T07:40:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.804+0000 7ff838a00640 1 -- 172.21.15.46:0/3767359289 wait complete. 2024-09-10T07:40:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.806+0000 7ff838a00640 1 Processor -- start 2024-09-10T07:40:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.806+0000 7ff838a00640 1 -- start start 2024-09-10T07:40:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.807+0000 7ff838a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff834103d80 0x7ff83419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.807+0000 7ff838a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff834104f70 0x7ff83419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.807+0000 7ff838a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff834105930 0x7ff8341a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.807+0000 7ff838a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff83410fa30 con 0x7ff834103d80 2024-09-10T07:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.807+0000 7ff838a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff83410f8b0 con 0x7ff834104f70 2024-09-10T07:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.807+0000 7ff838a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff83410fbb0 con 0x7ff834105930 2024-09-10T07:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff831400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff834104f70 0x7ff83419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff831e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff834103d80 0x7ff83419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff831e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff834103d80 0x7ff83419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53526/0 (socket says 172.21.15.46:53526) 2024-09-10T07:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff831e00640 1 -- 172.21.15.46:0/118576466 learned_addr learned my addr 172.21.15.46:0/118576466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff832800640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff834105930 0x7ff8341a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff832800640 1 -- 172.21.15.46:0/118576466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff834104f70 msgr2=0x7ff83419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff832800640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff834104f70 0x7ff83419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff832800640 1 -- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff834103d80 msgr2=0x7ff83419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff832800640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff834103d80 0x7ff83419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.808+0000 7ff832800640 1 -- 172.21.15.46:0/118576466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8341a1ca0 con 0x7ff834105930 2024-09-10T07:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.809+0000 7ff832800640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff834105930 0x7ff8341a1630 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00e940 tx=0x7ff82c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.809+0000 7ff82aa00640 1 -- 172.21.15.46:0/118576466 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c004d20 con 0x7ff834105930 2024-09-10T07:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.809+0000 7ff838a00640 1 -- 172.21.15.46:0/118576466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff834076690 con 0x7ff834105930 2024-09-10T07:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.809+0000 7ff838a00640 1 -- 172.21.15.46:0/118576466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff834076bb0 con 0x7ff834105930 2024-09-10T07:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.809+0000 7ff82aa00640 1 -- 172.21.15.46:0/118576466 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff82c005020 con 0x7ff834105930 2024-09-10T07:40:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.809+0000 7ff82aa00640 1 -- 172.21.15.46:0/118576466 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0116b0 con 0x7ff834105930 2024-09-10T07:40:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.817+0000 7ff83b9fe740 1 Processor -- start 2024-09-10T07:40:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.817+0000 7ff82aa00640 1 -- 172.21.15.46:0/118576466 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff82c0054d0 con 0x7ff834105930 2024-09-10T07:40:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.817+0000 7ff83b9fe740 1 -- start start 2024-09-10T07:40:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.818+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a090 0x558efab3a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.818+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a9d0 0x558efab3f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.818+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3fa60 0x558efab41f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.818+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558efaa9f100 con 0x558efab3a090 2024-09-10T07:40:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.818+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558efaa471b0 con 0x558efab3a9d0 2024-09-10T07:40:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.818+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558efaa9b0c0 con 0x558efab3fa60 2024-09-10T07:40:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.818+0000 7ff82aa00640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8180a2d30 0x7ff8180a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.818+0000 7ff831400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a090 0x558efab3a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.819+0000 7ff831e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a9d0 0x558efab3f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.819+0000 7ff831400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a090 0x558efab3a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53536/0 (socket says 172.21.15.46:53536) 2024-09-10T07:40:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.819+0000 7ff831400640 1 -- 172.21.15.46:0/2677696060 learned_addr learned my addr 172.21.15.46:0/2677696060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.819+0000 7ff82aa00640 1 -- 172.21.15.46:0/118576466 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff82c0cf440 con 0x7ff834105930 2024-09-10T07:40:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.819+0000 7ff831400640 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3fa60 0x558efab41f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.820+0000 7ff832800640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8180a2d30 0x7ff8180a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.820+0000 7ff831e00640 1 -- 172.21.15.46:0/2677696060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3fa60 msgr2=0x558efab41f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.820+0000 7ff831e00640 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3fa60 0x558efab41f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.820+0000 7ff831e00640 1 -- 172.21.15.46:0/2677696060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a090 msgr2=0x558efab3a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.821+0000 7ff831e00640 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a090 0x558efab3a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.821+0000 7ff831e00640 1 -- 172.21.15.46:0/2677696060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558efa7b2210 con 0x558efab3a9d0 2024-09-10T07:40:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.822+0000 7ff831400640 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a090 0x558efab3a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.822+0000 7ff831e00640 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a9d0 0x558efab3f490 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff82000bc90 tx=0x7ff8200107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf82c971491dc6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.823+0000 7ff832800640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8180a2d30 0x7ff8180a51f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff82c005940 tx=0x7ff82c00ef30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.824+0000 7ff81f000640 1 -- 172.21.15.46:0/2677696060 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820004220 con 0x558efab3a9d0 2024-09-10T07:40:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.824+0000 7ff81f000640 1 -- 172.21.15.46:0/2677696060 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff82000be40 con 0x558efab3a9d0 2024-09-10T07:40:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.824+0000 7ff81f000640 1 -- 172.21.15.46:0/2677696060 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820004cc0 con 0x558efab3a9d0 2024-09-10T07:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 -- 172.21.15.46:0/2677696060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a9d0 msgr2=0x558efab3f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a9d0 0x558efab3f490 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff82000bc90 tx=0x7ff8200107a0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 -- 172.21.15.46:0/2677696060 shutdown_connections 2024-09-10T07:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3fa60 0x558efab41f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a9d0 0x558efab3f490 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/2677696060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a090 0x558efab3a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 -- 172.21.15.46:0/2677696060 >> 172.21.15.46:0/2677696060 conn(0x558efab35700 msgr2=0x558efab35b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 -- 172.21.15.46:0/2677696060 shutdown_connections 2024-09-10T07:40:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.825+0000 7ff83b9fe740 1 -- 172.21.15.46:0/2677696060 wait complete. 2024-09-10T07:40:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.826+0000 7ff83b9fe740 1 -- start start 2024-09-10T07:40:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.826+0000 7ff83b9fe740 1 Processor -- start 2024-09-10T07:40:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 0x558efab5c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558efab53a40 con 0x558efab3a9d0 2024-09-10T07:40:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558efaa28090 con 0x558efab3fa60 2024-09-10T07:40:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558efab53bc0 con 0x558efab3a090 2024-09-10T07:40:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff831400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff831400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40718/0 (socket says 172.21.15.46:40718) 2024-09-10T07:40:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.829+0000 7ff831400640 1 -- 172.21.15.46:0/1910949187 learned_addr learned my addr 172.21.15.46:0/1910949187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff83b9fe740 10 client.0 _ll_register_callbacks cb 0x558efaa7cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff831400640 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 0x558efab5c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff831400640 1 -- 172.21.15.46:0/1910949187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 msgr2=0x558efab5c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff831400640 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 0x558efab5c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff831400640 1 -- 172.21.15.46:0/1910949187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 msgr2=0x558efab557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff831400640 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff831400640 1 -- 172.21.15.46:0/1910949187 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558efa87e620 con 0x558efab3a090 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff831400640 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab552b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff82402a9e0 tx=0x7ff8240314e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83c2cc12c4bfce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.830+0000 7ff813400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.831+0000 7ff813400640 1 -- 172.21.15.46:0/1910949187 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824035070 con 0x558efab3a090 2024-09-10T07:40:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.831+0000 7ff813400640 1 -- 172.21.15.46:0/1910949187 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff824004ad0 con 0x558efab3a090 2024-09-10T07:40:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.831+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1910949187 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558efaaac830 con 0x558efab3a090 2024-09-10T07:40:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.831+0000 7ff813400640 1 -- 172.21.15.46:0/1910949187 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff824004cd0 con 0x558efab3a090 2024-09-10T07:40:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff83b9fe740 10 client.26251 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1910949187 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558efab5f670 con 0x558efab3a090 2024-09-10T07:40:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff813400640 1 -- 172.21.15.46:0/1910949187 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558efaaac830 con 0x558efab3a090 2024-09-10T07:40:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff813400640 1 -- 172.21.15.46:0/1910949187 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff824049060 con 0x558efab3a090 2024-09-10T07:40:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff83b9fe740 10 client.26251 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff83b9fe740 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff83b9fe740 20 client.26251 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558efab60d40 0x558efab63190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff83b9fe740 4 client.26251 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.832+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1910949187 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558efaaa6b80 con 0x558efab60d40 2024-09-10T07:40:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.833+0000 7ff831e00640 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558efab60d40 0x558efab63190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.834+0000 7ff813400640 10 client.26251 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.834+0000 7ff831e00640 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558efab60d40 0x558efab63190 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad104f04df292528 server_cookie=a7ddb0b8f78679db in_seq=0 out_seq=0 2024-09-10T07:40:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.835+0000 7ff813400640 1 -- 172.21.15.46:0/1910949187 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558efaaa6b80 con 0x558efab60d40 2024-09-10T07:40:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.835+0000 7ff813400640 10 client.26251 handle_command_reply: tid=0 2024-09-10T07:40:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.835+0000 7ff83b9fe740 1 client.26251 shutdown 2024-09-10T07:40:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.836+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1910949187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 msgr2=0x558efab552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.837+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab552b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff82402a9e0 tx=0x7ff8240314e0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.837+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1910949187 shutdown_connections 2024-09-10T07:40:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.837+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558efab60d40 0x558efab63190 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.837+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 0x558efab5c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.837+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.837+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1910949187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab552b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.837+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1910949187 >> 172.21.15.46:0/1910949187 conn(0x558efab35700 msgr2=0x558efab36f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.837+0000 7ff813400640 0 client.26251 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.838+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1910949187 shutdown_connections 2024-09-10T07:40:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.839+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1910949187 wait complete. 2024-09-10T07:40:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.839+0000 7ff83b9fe740 20 client.26251 trim_cache size 0 max 16384 2024-09-10T07:40:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.869+0000 7ff83b9fe740 1 Processor -- start 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.869+0000 7ff83b9fe740 1 -- start start 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.869+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.869+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab5c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.869+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 0x558efab36af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.869+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff81400fbf0 con 0x558efab3a9d0 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.869+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8140117c0 con 0x558efab3fa60 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.869+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff8140102c0 con 0x558efab3a090 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab5c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab5c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53550/0 (socket says 172.21.15.46:53550) 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831e00640 1 -- 172.21.15.46:0/1878309115 learned_addr learned my addr 172.21.15.46:0/1878309115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831400640 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831400640 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 0x558efab36af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831400640 1 -- 172.21.15.46:0/1878309115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 msgr2=0x558efab36af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831400640 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 0x558efab36af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831400640 1 -- 172.21.15.46:0/1878309115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 msgr2=0x558efab5c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.870+0000 7ff831400640 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab5c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.871+0000 7ff831400640 1 -- 172.21.15.46:0/1878309115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558efab35520 con 0x558efab3a090 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.871+0000 7ff831e00640 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab5c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.871+0000 7ff831400640 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab365b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff82402a910 tx=0x7ff824042040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57658d4e3ae286e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.871+0000 7ff81e600640 1 -- 172.21.15.46:0/1878309115 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824035070 con 0x558efab3a090 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.871+0000 7ff81e600640 1 -- 172.21.15.46:0/1878309115 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8240048d0 con 0x558efab3a090 2024-09-10T07:40:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.871+0000 7ff81e600640 1 -- 172.21.15.46:0/1878309115 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824031df0 con 0x558efab3a090 2024-09-10T07:40:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.872+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1878309115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 msgr2=0x558efab365b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.872+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab365b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff82402a910 tx=0x7ff824042040 comp rx=0 tx=0).stop 2024-09-10T07:40:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.872+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1878309115 shutdown_connections 2024-09-10T07:40:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.872+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3fa60 0x558efab36af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.872+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3a9d0 0x558efab5c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.872+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/1878309115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a090 0x558efab365b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.872+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1878309115 >> 172.21.15.46:0/1878309115 conn(0x558efab35700 msgr2=0x558efab54e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.872+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1878309115 shutdown_connections 2024-09-10T07:40:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.873+0000 7ff83b9fe740 1 -- 172.21.15.46:0/1878309115 wait complete. 2024-09-10T07:40:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.873+0000 7ff83b9fe740 1 -- start start 2024-09-10T07:40:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.873+0000 7ff83b9fe740 1 Processor -- start 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.874+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a090 0x558efab59860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.874+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a9d0 0x558efab59da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.875+0000 7ff83b9fe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3fa60 0x558efabc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.875+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8140110c0 con 0x558efab3fa60 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.875+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8140109f0 con 0x558efab3a090 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.875+0000 7ff83b9fe740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff81400dbd0 con 0x558efab3a9d0 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.875+0000 7ff83b9fe740 10 client.0 _ll_register_callbacks cb 0x7ff824009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.875+0000 7ff831400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3fa60 0x558efabc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.875+0000 7ff831e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a9d0 0x558efab59da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.876+0000 7ff831400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a090 0x558efab59860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.875+0000 7ff831e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a9d0 0x558efab59da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40732/0 (socket says 172.21.15.46:40732) 2024-09-10T07:40:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.876+0000 7ff831e00640 1 -- 172.21.15.46:0/3098750635 learned_addr learned my addr 172.21.15.46:0/3098750635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.877+0000 7ff831e00640 1 -- 172.21.15.46:0/3098750635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a090 msgr2=0x558efab59860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.877+0000 7ff831e00640 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a090 0x558efab59860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.877+0000 7ff831e00640 1 -- 172.21.15.46:0/3098750635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3fa60 msgr2=0x558efabc9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.877+0000 7ff831e00640 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3fa60 0x558efabc9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.877+0000 7ff831e00640 1 -- 172.21.15.46:0/3098750635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558efabca050 con 0x558efab3a9d0 2024-09-10T07:40:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.877+0000 7ff831400640 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3fa60 0x558efabc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.878+0000 7ff813400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.878+0000 7ff831e00640 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a9d0 0x558efab59da0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff82000bc90 tx=0x7ff82000b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c110b71f5d355c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.878+0000 7ff813400640 1 -- 172.21.15.46:0/3098750635 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820005240 con 0x558efab3a9d0 2024-09-10T07:40:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.878+0000 7ff813400640 1 -- 172.21.15.46:0/3098750635 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff8200053e0 con 0x558efab3a9d0 2024-09-10T07:40:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.878+0000 7ff83b9fe740 1 -- 172.21.15.46:0/3098750635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8240048d0 con 0x558efab3a9d0 2024-09-10T07:40:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.879+0000 7ff813400640 1 -- 172.21.15.46:0/3098750635 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff820005600 con 0x558efab3a9d0 2024-09-10T07:40:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.879+0000 7ff83b9fe740 10 client.26257 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.879+0000 7ff83b9fe740 1 -- 172.21.15.46:0/3098750635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558efabcb360 con 0x558efab3a9d0 2024-09-10T07:40:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.879+0000 7ff813400640 1 -- 172.21.15.46:0/3098750635 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff820004ba0 con 0x558efab3a9d0 2024-09-10T07:40:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.880+0000 7ff813400640 1 -- 172.21.15.46:0/3098750635 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff82002a870 con 0x558efab3a9d0 2024-09-10T07:40:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.880+0000 7ff83b9fe740 10 client.26257 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.880+0000 7ff83b9fe740 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.880+0000 7ff83b9fe740 20 client.26257 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.880+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558efabcba90 0x558efabcdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.880+0000 7ff83b9fe740 4 client.26257 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:40:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.880+0000 7ff83b9fe740 1 -- 172.21.15.46:0/3098750635 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff8240048d0 con 0x558efabcba90 2024-09-10T07:40:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.881+0000 7ff831400640 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558efabcba90 0x558efabcdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.881+0000 7ff831400640 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558efabcba90 0x558efabcdeb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b793b7f1a7ddf9a server_cookie=b39bf25f381db388 in_seq=0 out_seq=0 2024-09-10T07:40:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.881+0000 7ff813400640 10 client.26257 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.965+0000 7ff813400640 1 -- 172.21.15.46:0/3098750635 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff8240048d0 con 0x558efabcba90 2024-09-10T07:40:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.965+0000 7ff813400640 10 client.26257 handle_command_reply: tid=0 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.965+0000 7ff83b9fe740 1 client.26257 shutdown 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff83b9fe740 1 -- 172.21.15.46:0/3098750635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a9d0 msgr2=0x558efab59da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a9d0 0x558efab59da0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff82000bc90 tx=0x7ff82000b810 comp rx=0 tx=0).stop 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff83b9fe740 1 -- 172.21.15.46:0/3098750635 shutdown_connections 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558efabcba90 0x558efabcdeb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558efab3fa60 0x558efabc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558efab3a9d0 0x558efab59da0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff83b9fe740 1 --2- 172.21.15.46:0/3098750635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558efab3a090 0x558efab59860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff83b9fe740 1 -- 172.21.15.46:0/3098750635 >> 172.21.15.46:0/3098750635 conn(0x558efab35700 msgr2=0x558efab34060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.966+0000 7ff813400640 0 client.26257 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.967+0000 7ff83b9fe740 1 -- 172.21.15.46:0/3098750635 shutdown_connections 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.967+0000 7ff83b9fe740 1 -- 172.21.15.46:0/3098750635 wait complete. 2024-09-10T07:40:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.967+0000 7ff83b9fe740 20 client.26257 trim_cache size 0 max 16384 2024-09-10T07:40:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:40:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:40:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:40:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:40:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 -- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8180a2d30 msgr2=0x7ff8180a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8180a2d30 0x7ff8180a51f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff82c005940 tx=0x7ff82c00ef30 comp rx=0 tx=0).stop 2024-09-10T07:40:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 -- 172.21.15.46:0/118576466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff834105930 msgr2=0x7ff8341a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff834105930 0x7ff8341a1630 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00e940 tx=0x7ff82c00ee00 comp rx=0 tx=0).stop 2024-09-10T07:40:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 -- 172.21.15.46:0/118576466 shutdown_connections 2024-09-10T07:40:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff834105930 0x7ff8341a1630 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff834104f70 0x7ff83419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8180a2d30 0x7ff8180a51f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 --2- 172.21.15.46:0/118576466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff834103d80 0x7ff83419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 -- 172.21.15.46:0/118576466 >> 172.21.15.46:0/118576466 conn(0x7ff8340ff4e0 msgr2=0x7ff834108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 -- 172.21.15.46:0/118576466 shutdown_connections 2024-09-10T07:40:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:13.970+0000 7ff838a00640 1 -- 172.21.15.46:0/118576466 wait complete. 2024-09-10T07:40:14.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:13 smithi046 ceph-mon[30705]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 666 op/s 2024-09-10T07:40:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:13 smithi181 ceph-mon[35397]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 666 op/s 2024-09-10T07:40:14.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:13 smithi063 ceph-mon[35020]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 666 op/s 2024-09-10T07:40:14.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 -- 172.21.15.46:0/3457921214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 msgr2=0x7f9774106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 --2- 172.21.15.46:0/3457921214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 0x7f9774106cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9768009900 tx=0x7f976802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 -- 172.21.15.46:0/3457921214 shutdown_connections 2024-09-10T07:40:14.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 --2- 172.21.15.46:0/3457921214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9774101270 0x7f9774069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 --2- 172.21.15.46:0/3457921214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97741008d0 0x7f9774100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 --2- 172.21.15.46:0/3457921214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 0x7f9774106cb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 -- 172.21.15.46:0/3457921214 >> 172.21.15.46:0/3457921214 conn(0x7f97740fc610 msgr2=0x7f97740fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:14.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 -- 172.21.15.46:0/3457921214 shutdown_connections 2024-09-10T07:40:14.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.631+0000 7f977aa00640 1 -- 172.21.15.46:0/3457921214 wait complete. 2024-09-10T07:40:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.633+0000 7f977aa00640 1 Processor -- start 2024-09-10T07:40:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.634+0000 7f977aa00640 1 -- start start 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.634+0000 7f977aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97741008d0 0x7f9774109a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.634+0000 7f977aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9774101270 0x7f9774109f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.634+0000 7f977aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 0x7f977410a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.634+0000 7f977aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111e40 con 0x7f9774101270 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.634+0000 7f977aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111cc0 con 0x7f97741008d0 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.634+0000 7f977aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111fc0 con 0x7f97741068d0 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.635+0000 7f9778c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9774101270 0x7f9774109f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.635+0000 7f9778c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9774101270 0x7f9774109f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53592/0 (socket says 172.21.15.46:53592) 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.635+0000 7f9778c00640 1 -- 172.21.15.46:0/4165743089 learned_addr learned my addr 172.21.15.46:0/4165743089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.635+0000 7f9779600640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97741008d0 0x7f9774109a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.636+0000 7f977a000640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 0x7f977410a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.636+0000 7f9779600640 1 -- 172.21.15.46:0/4165743089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 msgr2=0x7f977410a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.636+0000 7f9779600640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 0x7f977410a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.636+0000 7f9779600640 1 -- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9774101270 msgr2=0x7f9774109f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.636+0000 7f9779600640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9774101270 0x7f9774109f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.636+0000 7f9779600640 1 -- 172.21.15.46:0/4165743089 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977419a2f0 con 0x7f97741008d0 2024-09-10T07:40:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.636+0000 7f977a000640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 0x7f977410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.636+0000 7f9778c00640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9774101270 0x7f9774109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.637+0000 7f9779600640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97741008d0 0x7f9774109a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9768009230 tx=0x7f9768003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.637+0000 7f9772000640 1 -- 172.21.15.46:0/4165743089 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976803f070 con 0x7f97741008d0 2024-09-10T07:40:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.637+0000 7f977aa00640 1 -- 172.21.15.46:0/4165743089 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977419a580 con 0x7f97741008d0 2024-09-10T07:40:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.637+0000 7f9772000640 1 -- 172.21.15.46:0/4165743089 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9768031c90 con 0x7f97741008d0 2024-09-10T07:40:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.637+0000 7f9772000640 1 -- 172.21.15.46:0/4165743089 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768035030 con 0x7f97741008d0 2024-09-10T07:40:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.637+0000 7f977aa00640 1 -- 172.21.15.46:0/4165743089 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977419aab0 con 0x7f97741008d0 2024-09-10T07:40:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.641+0000 7f977ec72740 1 Processor -- start 2024-09-10T07:40:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.641+0000 7f977ec72740 1 -- start start 2024-09-10T07:40:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.642+0000 7f977ec72740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfa9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.642+0000 7f9772000640 1 -- 172.21.15.46:0/4165743089 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9768042070 con 0x7f97741008d0 2024-09-10T07:40:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.642+0000 7f977ec72740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfb3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.642+0000 7f977a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfa9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.642+0000 7f977ec72740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfb6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.642+0000 7f977ec72740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563afdf0f3e0 con 0x563afdfb4020 2024-09-10T07:40:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.642+0000 7f977ec72740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563afdeb6d80 con 0x563afdfa8e40 2024-09-10T07:40:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.643+0000 7f9779600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfb3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.643+0000 7f9779600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfb3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40762/0 (socket says 172.21.15.46:40762) 2024-09-10T07:40:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.643+0000 7f9779600640 1 -- 172.21.15.46:0/62078483 learned_addr learned my addr 172.21.15.46:0/62078483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.643+0000 7f977ec72740 1 -- 172.21.15.46:0/62078483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563afdf0b3a0 con 0x563afdfa9760 2024-09-10T07:40:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.644+0000 7f9778c00640 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfb6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.644+0000 7f9779600640 1 -- 172.21.15.46:0/62078483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 msgr2=0x563afdfa9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.644+0000 7f9779600640 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfa9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.644+0000 7f9779600640 1 -- 172.21.15.46:0/62078483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 msgr2=0x563afdfb6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.644+0000 7f9779600640 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfb6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.644+0000 7f9779600640 1 -- 172.21.15.46:0/62078483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563afded5ae0 con 0x563afdfa9760 2024-09-10T07:40:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.644+0000 7f9778c00640 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfb6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.644+0000 7f9779600640 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfb3ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97680f3ef0 tx=0x7f97680f8440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c3d06c00419eac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.645+0000 7f9772000640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f975c0a2c00 0x7f975c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.645+0000 7f9766600640 1 -- 172.21.15.46:0/62078483 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680ff070 con 0x563afdfa9760 2024-09-10T07:40:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.645+0000 7f9766600640 1 -- 172.21.15.46:0/62078483 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97680bc940 con 0x563afdfa9760 2024-09-10T07:40:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.645+0000 7f9766600640 1 -- 172.21.15.46:0/62078483 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680c4eb0 con 0x563afdfa9760 2024-09-10T07:40:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.646+0000 7f977ec72740 1 -- 172.21.15.46:0/62078483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 msgr2=0x563afdfb3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.646+0000 7f977ec72740 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfb3ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97680f3ef0 tx=0x7f97680f8440 comp rx=0 tx=0).stop 2024-09-10T07:40:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.647+0000 7f9772000640 1 -- 172.21.15.46:0/4165743089 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f97680efdc0 con 0x7f97741008d0 2024-09-10T07:40:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.647+0000 7f977a000640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f975c0a2c00 0x7f975c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.647+0000 7f977ec72740 1 -- 172.21.15.46:0/62078483 shutdown_connections 2024-09-10T07:40:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.647+0000 7f977ec72740 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfb6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.647+0000 7f977ec72740 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfb3ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.647+0000 7f977ec72740 1 --2- 172.21.15.46:0/62078483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfa9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.647+0000 7f977ec72740 1 -- 172.21.15.46:0/62078483 >> 172.21.15.46:0/62078483 conn(0x563afdfa53d0 msgr2=0x563afdfa57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:14.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.648+0000 7f977ec72740 1 -- 172.21.15.46:0/62078483 shutdown_connections 2024-09-10T07:40:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.648+0000 7f977a000640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f975c0a2c00 0x7f975c0a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f976c005fd0 tx=0x7f976c0095a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.648+0000 7f977ec72740 1 -- 172.21.15.46:0/62078483 wait complete. 2024-09-10T07:40:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.649+0000 7f977ec72740 1 -- start start 2024-09-10T07:40:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.649+0000 7f977ec72740 1 Processor -- start 2024-09-10T07:40:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f977ec72740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa8e40 0x563afdfb0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f977ec72740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa9760 0x563afdfb1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f977ec72740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfb1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f977ec72740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563afdfb69b0 con 0x563afdfa8e40 2024-09-10T07:40:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f977ec72740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563afdeca0f0 con 0x563afdfa9760 2024-09-10T07:40:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f977ec72740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563afdfb6b30 con 0x563afdfb4020 2024-09-10T07:40:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f977ec72740 10 client.0 _ll_register_callbacks cb 0x563afdeb1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f9778c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa9760 0x563afdfb1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f9778c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa9760 0x563afdfb1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45664/0 (socket says 172.21.15.46:45664) 2024-09-10T07:40:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f9778c00640 1 -- 172.21.15.46:0/205606910 learned_addr learned my addr 172.21.15.46:0/205606910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.650+0000 7f977a000640 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfb1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.651+0000 7f9779600640 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa8e40 0x563afdfb0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.651+0000 7f9778c00640 1 -- 172.21.15.46:0/205606910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 msgr2=0x563afdfb1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.651+0000 7f9778c00640 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfb1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.651+0000 7f9778c00640 1 -- 172.21.15.46:0/205606910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa8e40 msgr2=0x563afdfb0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:14.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.651+0000 7f9778c00640 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa8e40 0x563afdfb0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.651+0000 7f9778c00640 1 -- 172.21.15.46:0/205606910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563afdc803c0 con 0x563afdfa9760 2024-09-10T07:40:14.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.651+0000 7f9779600640 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa8e40 0x563afdfb0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:14.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.655+0000 7f9778c00640 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa9760 0x563afdfb1530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9760013d20 tx=0x7f97600117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d6936075735c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.655+0000 7f975aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.655+0000 7f975aa00640 1 -- 172.21.15.46:0/205606910 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760020060 con 0x563afdfa9760 2024-09-10T07:40:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.655+0000 7f977ec72740 1 -- 172.21.15.46:0/205606910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97680bc940 con 0x563afdfa9760 2024-09-10T07:40:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.655+0000 7f975aa00640 1 -- 172.21.15.46:0/205606910 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f976001b430 con 0x563afdfa9760 2024-09-10T07:40:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.656+0000 7f975aa00640 1 -- 172.21.15.46:0/205606910 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9760020060 con 0x563afdfa9760 2024-09-10T07:40:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.656+0000 7f977ec72740 10 client.26276 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.656+0000 7f977ec72740 1 -- 172.21.15.46:0/205606910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563afdfc2a50 con 0x563afdfa9760 2024-09-10T07:40:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.657+0000 7f975aa00640 1 -- 172.21.15.46:0/205606910 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97680bc940 con 0x563afdfa9760 2024-09-10T07:40:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.657+0000 7f975aa00640 1 -- 172.21.15.46:0/205606910 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f976000f830 con 0x563afdfa9760 2024-09-10T07:40:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.657+0000 7f977ec72740 10 client.26276 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.657+0000 7f977ec72740 10 client.26276 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:40:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.657+0000 7f977ec72740 20 client.26276 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.658+0000 7f977ec72740 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563afdfc3f00 0x563afdfc6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.658+0000 7f977ec72740 4 client.26276 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.658+0000 7f977ec72740 1 -- 172.21.15.46:0/205606910 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563afdf1c950 con 0x563afdfc3f00 2024-09-10T07:40:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.658+0000 7f9779600640 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563afdfc3f00 0x563afdfc6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.658+0000 7f975aa00640 10 client.26276 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:40:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.658+0000 7f9779600640 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563afdfc3f00 0x563afdfc6340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5717ff57749af5ba server_cookie=c5516b9bb39d0856 in_seq=0 out_seq=0 2024-09-10T07:40:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.659+0000 7f975aa00640 1 -- 172.21.15.46:0/205606910 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563afdf1c950 con 0x563afdfc3f00 2024-09-10T07:40:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.659+0000 7f975aa00640 10 client.26276 handle_command_reply: tid=0 2024-09-10T07:40:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.659+0000 7f977ec72740 1 client.26276 shutdown 2024-09-10T07:40:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.660+0000 7f977ec72740 1 -- 172.21.15.46:0/205606910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa9760 msgr2=0x563afdfb1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.660+0000 7f977ec72740 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa9760 0x563afdfb1530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9760013d20 tx=0x7f97600117b0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.661+0000 7f977ec72740 1 -- 172.21.15.46:0/205606910 shutdown_connections 2024-09-10T07:40:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.661+0000 7f977ec72740 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563afdfc3f00 0x563afdfc6340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.661+0000 7f977ec72740 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfb1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.661+0000 7f977ec72740 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa9760 0x563afdfb1530 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.661+0000 7f975aa00640 0 client.26276 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:40:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.661+0000 7f977ec72740 1 --2- 172.21.15.46:0/205606910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa8e40 0x563afdfb0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.661+0000 7f977ec72740 1 -- 172.21.15.46:0/205606910 >> 172.21.15.46:0/205606910 conn(0x563afdfa53d0 msgr2=0x563afdfab0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.662+0000 7f977ec72740 1 -- 172.21.15.46:0/205606910 shutdown_connections 2024-09-10T07:40:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.662+0000 7f977ec72740 1 -- 172.21.15.46:0/205606910 wait complete. 2024-09-10T07:40:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.662+0000 7f977ec72740 20 client.26276 trim_cache size 0 max 16384 2024-09-10T07:40:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.686+0000 7f977ec72740 1 Processor -- start 2024-09-10T07:40:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.686+0000 7f977ec72740 1 -- start start 2024-09-10T07:40:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977ec72740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfcbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977ec72740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa9760 0x563afdfcc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977ec72740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977ec72740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97741bd9a0 con 0x563afdfa9760 2024-09-10T07:40:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977ec72740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97741bf570 con 0x563afdfa8e40 2024-09-10T07:40:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977ec72740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97741be070 con 0x563afdfb4020 2024-09-10T07:40:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40782/0 (socket says 172.21.15.46:40782) 2024-09-10T07:40:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.687+0000 7f977a000640 1 -- 172.21.15.46:0/1856508522 learned_addr learned my addr 172.21.15.46:0/1856508522 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.688+0000 7f9779600640 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfcbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.688+0000 7f9778c00640 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa9760 0x563afdfcc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.688+0000 7f977a000640 1 -- 172.21.15.46:0/1856508522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 msgr2=0x563afdfcbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.688+0000 7f977a000640 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfcbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.688+0000 7f977a000640 1 -- 172.21.15.46:0/1856508522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa9760 msgr2=0x563afdfcc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.688+0000 7f977a000640 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa9760 0x563afdfcc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.688+0000 7f977a000640 1 -- 172.21.15.46:0/1856508522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563afdfa6060 con 0x563afdfb4020 2024-09-10T07:40:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.689+0000 7f9779600640 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfcbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.689+0000 7f977a000640 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfae270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f976c012a20 tx=0x7f976c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=773ad9d6147d3c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.689+0000 7f9765c00640 1 -- 172.21.15.46:0/1856508522 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c020bb0 con 0x563afdfb4020 2024-09-10T07:40:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.690+0000 7f9765c00640 1 -- 172.21.15.46:0/1856508522 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c020d50 con 0x563afdfb4020 2024-09-10T07:40:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.690+0000 7f9765c00640 1 -- 172.21.15.46:0/1856508522 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c01b8a0 con 0x563afdfb4020 2024-09-10T07:40:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.691+0000 7f977ec72740 1 -- 172.21.15.46:0/1856508522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 msgr2=0x563afdfae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.691+0000 7f977ec72740 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfae270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f976c012a20 tx=0x7f976c014000 comp rx=0 tx=0).stop 2024-09-10T07:40:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.692+0000 7f977ec72740 1 -- 172.21.15.46:0/1856508522 shutdown_connections 2024-09-10T07:40:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.692+0000 7f977ec72740 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfb4020 0x563afdfae270 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.692+0000 7f977ec72740 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfa9760 0x563afdfcc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.692+0000 7f977ec72740 1 --2- 172.21.15.46:0/1856508522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfcbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.692+0000 7f977ec72740 1 -- 172.21.15.46:0/1856508522 >> 172.21.15.46:0/1856508522 conn(0x563afdfa53d0 msgr2=0x563afdfb00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.692+0000 7f977ec72740 1 -- 172.21.15.46:0/1856508522 shutdown_connections 2024-09-10T07:40:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.692+0000 7f977ec72740 1 -- 172.21.15.46:0/1856508522 wait complete. 2024-09-10T07:40:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.692+0000 7f977ec72740 1 -- start start 2024-09-10T07:40:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.693+0000 7f977ec72740 1 Processor -- start 2024-09-10T07:40:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.693+0000 7f977ec72740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfadfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.693+0000 7f977ec72740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfae500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.693+0000 7f977ec72740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfaea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.693+0000 7f977ec72740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97741bee70 con 0x563afdfb4020 2024-09-10T07:40:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.693+0000 7f977ec72740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97741be7a0 con 0x563afdfa8e40 2024-09-10T07:40:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.693+0000 7f977ec72740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f977410af80 con 0x563afdfa9760 2024-09-10T07:40:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.693+0000 7f977ec72740 10 client.0 _ll_register_callbacks cb 0x7f9760009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfadfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfadfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45686/0 (socket says 172.21.15.46:45686) 2024-09-10T07:40:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 -- 172.21.15.46:0/1266059938 learned_addr learned my addr 172.21.15.46:0/1266059938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f977a000640 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfaea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9778c00640 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 -- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 msgr2=0x563afdfae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 -- 172.21.15.46:0/1266059938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 msgr2=0x563afdfaea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfaea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 -- 172.21.15.46:0/1266059938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563afe02dbd0 con 0x563afdfa8e40 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9778c00640 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f9779600640 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfadfc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f97680b86a0 tx=0x7f976802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f8f7935eaad610 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.694+0000 7f975aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.695+0000 7f975aa00640 1 -- 172.21.15.46:0/1266059938 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768049040 con 0x563afdfa8e40 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.695+0000 7f975aa00640 1 -- 172.21.15.46:0/1266059938 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f97680ba540 con 0x563afdfa8e40 2024-09-10T07:40:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.695+0000 7f977ec72740 1 -- 172.21.15.46:0/1266059938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976c020d50 con 0x563afdfa8e40 2024-09-10T07:40:14.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.697+0000 7f975aa00640 1 -- 172.21.15.46:0/1266059938 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f97680ff040 con 0x563afdfa8e40 2024-09-10T07:40:14.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.697+0000 7f977ec72740 10 client.26282 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:14.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.697+0000 7f977ec72740 1 -- 172.21.15.46:0/1266059938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563afe02eed0 con 0x563afdfa8e40 2024-09-10T07:40:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.698+0000 7f975aa00640 1 -- 172.21.15.46:0/1266059938 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c020d50 con 0x563afdfa8e40 2024-09-10T07:40:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.698+0000 7f975aa00640 1 -- 172.21.15.46:0/1266059938 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f97680b88b0 con 0x563afdfa8e40 2024-09-10T07:40:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.699+0000 7f977ec72740 10 client.26282 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.699+0000 7f977ec72740 10 client.26282 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:40:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.699+0000 7f977ec72740 20 client.26282 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.699+0000 7f977ec72740 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563afe02f640 0x563afe031a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.699+0000 7f977ec72740 4 client.26282 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:40:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.699+0000 7f977ec72740 1 -- 172.21.15.46:0/1266059938 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563afdf16e60 con 0x563afe02f640 2024-09-10T07:40:14.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.700+0000 7f9778c00640 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563afe02f640 0x563afe031a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:14.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.701+0000 7f9778c00640 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563afe02f640 0x563afe031a60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d49853ff670770ea server_cookie=6b27c69e54243b7a in_seq=0 out_seq=0 2024-09-10T07:40:14.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.701+0000 7f975aa00640 10 client.26282 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:40:14.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.701+0000 7f975aa00640 1 -- 172.21.15.46:0/1266059938 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563afdf16e60 con 0x563afe02f640 2024-09-10T07:40:14.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.701+0000 7f975aa00640 10 client.26282 handle_command_reply: tid=0 2024-09-10T07:40:14.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.701+0000 7f977ec72740 1 client.26282 shutdown 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 -- 172.21.15.46:0/1266059938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 msgr2=0x563afdfadfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfadfc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f97680b86a0 tx=0x7f976802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 -- 172.21.15.46:0/1266059938 shutdown_connections 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563afe02f640 0x563afe031a60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afdfb4020 0x563afdfaea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563afdfa9760 0x563afdfae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 --2- 172.21.15.46:0/1266059938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563afdfa8e40 0x563afdfadfc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 -- 172.21.15.46:0/1266059938 >> 172.21.15.46:0/1266059938 conn(0x563afdfa53d0 msgr2=0x563afdfa3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f975aa00640 0 client.26282 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.702+0000 7f977ec72740 1 -- 172.21.15.46:0/1266059938 shutdown_connections 2024-09-10T07:40:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.703+0000 7f977ec72740 1 -- 172.21.15.46:0/1266059938 wait complete. 2024-09-10T07:40:14.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.703+0000 7f977ec72740 20 client.26282 trim_cache size 0 max 16384 2024-09-10T07:40:14.703 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:40:14.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.704+0000 7f977aa00640 1 -- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f975c0a2c00 msgr2=0x7f975c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.704+0000 7f977aa00640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f975c0a2c00 0x7f975c0a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f976c005fd0 tx=0x7f976c0095a0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.704+0000 7f977aa00640 1 -- 172.21.15.46:0/4165743089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97741008d0 msgr2=0x7f9774109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:14.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.704+0000 7f977aa00640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97741008d0 0x7f9774109a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9768009230 tx=0x7f9768003b90 comp rx=0 tx=0).stop 2024-09-10T07:40:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.705+0000 7f977aa00640 1 -- 172.21.15.46:0/4165743089 shutdown_connections 2024-09-10T07:40:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.705+0000 7f977aa00640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97741068d0 0x7f977410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.705+0000 7f977aa00640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f975c0a2c00 0x7f975c0a50c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.705+0000 7f977aa00640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9774101270 0x7f9774109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.705+0000 7f977aa00640 1 --2- 172.21.15.46:0/4165743089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97741008d0 0x7f9774109a50 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9768009230 tx=0x7f9768003b90 comp rx=0 tx=0).stop 2024-09-10T07:40:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.705+0000 7f977aa00640 1 -- 172.21.15.46:0/4165743089 >> 172.21.15.46:0/4165743089 conn(0x7f97740fc610 msgr2=0x7f977410e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.707+0000 7f977aa00640 1 -- 172.21.15.46:0/4165743089 shutdown_connections 2024-09-10T07:40:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:14.707+0000 7f977aa00640 1 -- 172.21.15.46:0/4165743089 wait complete. 2024-09-10T07:40:15.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:40:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 175.0 s, 1141.8 tps, lat 87.942 ms stddev 100.833 2024-09-10T07:40:15.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:15 smithi046 ceph-mon[30705]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-09-10T07:40:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:15 smithi181 ceph-mon[35397]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-09-10T07:40:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:15 smithi063 ceph-mon[35020]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-09-10T07:40:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.956+0000 7feb16400640 1 -- 172.21.15.46:0/2008791840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb10101df0 msgr2=0x7feb101021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.956+0000 7feb16400640 1 --2- 172.21.15.46:0/2008791840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb10101df0 0x7feb101021d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feb00009900 tx=0x7feb0002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.956+0000 7feb16400640 1 -- 172.21.15.46:0/2008791840 shutdown_connections 2024-09-10T07:40:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.956+0000 7feb16400640 1 --2- 172.21.15.46:0/2008791840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10106f20 0x7feb1010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.956+0000 7feb16400640 1 --2- 172.21.15.46:0/2008791840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb101027a0 0x7feb101067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.956+0000 7feb16400640 1 --2- 172.21.15.46:0/2008791840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb10101df0 0x7feb101021d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.956+0000 7feb16400640 1 -- 172.21.15.46:0/2008791840 >> 172.21.15.46:0/2008791840 conn(0x7feb100fd8b0 msgr2=0x7feb100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.956+0000 7feb16400640 1 -- 172.21.15.46:0/2008791840 shutdown_connections 2024-09-10T07:40:16.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.957+0000 7feb16400640 1 -- 172.21.15.46:0/2008791840 wait complete. 2024-09-10T07:40:16.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.958+0000 7feb16400640 1 Processor -- start 2024-09-10T07:40:16.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.963+0000 7feb16400640 1 -- start start 2024-09-10T07:40:16.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb16400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10101df0 0x7feb100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb16400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb101027a0 0x7feb10074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb16400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10106f20 0x7feb10074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb16400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb10111fd0 con 0x7feb10101df0 2024-09-10T07:40:16.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb16400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb10111e50 con 0x7feb101027a0 2024-09-10T07:40:16.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb16400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb10112150 con 0x7feb10106f20 2024-09-10T07:40:16.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb0ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb101027a0 0x7feb10074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb0ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb101027a0 0x7feb10074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49726/0 (socket says 172.21.15.46:49726) 2024-09-10T07:40:16.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb0ea00640 1 -- 172.21.15.46:0/2618078209 learned_addr learned my addr 172.21.15.46:0/2618078209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:16.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.964+0000 7feb0fe00640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10106f20 0x7feb10074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.965+0000 7feb0ea00640 1 -- 172.21.15.46:0/2618078209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10106f20 msgr2=0x7feb10074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:16.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.965+0000 7feb0ea00640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10106f20 0x7feb10074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.965+0000 7feb0ea00640 1 -- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10101df0 msgr2=0x7feb100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.965+0000 7feb0ea00640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10101df0 0x7feb100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.965+0000 7feb0ea00640 1 -- 172.21.15.46:0/2618078209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb101a6830 con 0x7feb101027a0 2024-09-10T07:40:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.965+0000 7feb0fe00640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10106f20 0x7feb10074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.966+0000 7feb0ea00640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb101027a0 0x7feb10074a30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e9f0 tx=0x7feafc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.967+0000 7feb07e00640 1 -- 172.21.15.46:0/2618078209 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc00cd80 con 0x7feb101027a0 2024-09-10T07:40:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.967+0000 7feb16400640 1 -- 172.21.15.46:0/2618078209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb101a6b20 con 0x7feb101027a0 2024-09-10T07:40:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.967+0000 7feb16400640 1 -- 172.21.15.46:0/2618078209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101a70b0 con 0x7feb101027a0 2024-09-10T07:40:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.967+0000 7feb07e00640 1 -- 172.21.15.46:0/2618078209 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feafc010910 con 0x7feb101027a0 2024-09-10T07:40:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.967+0000 7feb07e00640 1 -- 172.21.15.46:0/2618078209 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc0188f0 con 0x7feb101027a0 2024-09-10T07:40:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.976+0000 7feb18e76740 1 Processor -- start 2024-09-10T07:40:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.977+0000 7feb18e76740 1 -- start start 2024-09-10T07:40:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.977+0000 7feb18e76740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb86319f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.977+0000 7feb18e76740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8632d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.977+0000 7feb18e76740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8632fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.977+0000 7feb18e76740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eb862803e0 con 0x55eb8631a4b0 2024-09-10T07:40:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.977+0000 7feb18e76740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb86227d80 con 0x55eb86319b90 2024-09-10T07:40:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.977+0000 7feb18e76740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eb8627c3a0 con 0x55eb8632d660 2024-09-10T07:40:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.978+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb86319f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.978+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb86319f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49742/0 (socket says 172.21.15.46:49742) 2024-09-10T07:40:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.978+0000 7feb0fe00640 1 -- 172.21.15.46:0/1164381039 learned_addr learned my addr 172.21.15.46:0/1164381039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:16.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.978+0000 7feb07e00640 1 -- 172.21.15.46:0/2618078209 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feafc018bb0 con 0x7feb101027a0 2024-09-10T07:40:16.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.979+0000 7feb0ea00640 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8632fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.979+0000 7feb0f400640 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8632d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.979+0000 7feb0fe00640 1 -- 172.21.15.46:0/1164381039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 msgr2=0x55eb8632fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.979+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8632fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.979+0000 7feb0fe00640 1 -- 172.21.15.46:0/1164381039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 msgr2=0x55eb8632d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.979+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8632d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.979+0000 7feb0fe00640 1 -- 172.21.15.46:0/1164381039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb86246ae0 con 0x55eb86319b90 2024-09-10T07:40:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.980+0000 7feb0f400640 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8632d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.980+0000 7feb0ea00640 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8632fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.980+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb86319f70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feb0800f790 tx=0x7feb0800fc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb7fc7d076333778 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.981+0000 7feb04c00640 1 -- 172.21.15.46:0/1164381039 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08004d20 con 0x55eb86319b90 2024-09-10T07:40:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.981+0000 7feb04c00640 1 -- 172.21.15.46:0/1164381039 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0800de00 con 0x55eb86319b90 2024-09-10T07:40:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.981+0000 7feb04c00640 1 -- 172.21.15.46:0/1164381039 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08004d20 con 0x55eb86319b90 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.983+0000 7feb07e00640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feaf40a2c40 0x7feaf40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.984+0000 7feb18e76740 1 -- 172.21.15.46:0/1164381039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 msgr2=0x55eb86319f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.984+0000 7feb18e76740 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb86319f70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feb0800f790 tx=0x7feb0800fc50 comp rx=0 tx=0).stop 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.984+0000 7feb0f400640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feaf40a2c40 0x7feaf40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.984+0000 7feb18e76740 1 -- 172.21.15.46:0/1164381039 shutdown_connections 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.984+0000 7feb18e76740 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8632fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.984+0000 7feb18e76740 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8632d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.984+0000 7feb18e76740 1 --2- 172.21.15.46:0/1164381039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb86319f70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.984+0000 7feb18e76740 1 -- 172.21.15.46:0/1164381039 >> 172.21.15.46:0/1164381039 conn(0x55eb86316200 msgr2=0x55eb863165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.985+0000 7feb18e76740 1 -- 172.21.15.46:0/1164381039 shutdown_connections 2024-09-10T07:40:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.985+0000 7feb07e00640 1 -- 172.21.15.46:0/2618078209 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feafc0cf690 con 0x7feb101027a0 2024-09-10T07:40:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.985+0000 7feb18e76740 1 -- 172.21.15.46:0/1164381039 wait complete. 2024-09-10T07:40:16.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.987+0000 7feb18e76740 1 -- start start 2024-09-10T07:40:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.987+0000 7feb18e76740 1 Processor -- start 2024-09-10T07:40:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.988+0000 7feb0f400640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feaf40a2c40 0x7feaf40a5100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb00031550 tx=0x7feb00000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.989+0000 7feb18e76740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 0x55eb86327060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.989+0000 7feb18e76740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.989+0000 7feb18e76740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb8632cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.989+0000 7feb0ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.990+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb8632cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.990+0000 7feb0f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 0x55eb86327060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.990+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb8632cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49754/0 (socket says 172.21.15.46:49754) 2024-09-10T07:40:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.990+0000 7feb0fe00640 1 -- 172.21.15.46:0/1283659689 learned_addr learned my addr 172.21.15.46:0/1283659689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.990+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eb8631caf0 con 0x55eb86319b90 2024-09-10T07:40:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.990+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb8623b0f0 con 0x55eb8632d660 2024-09-10T07:40:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.990+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eb8631cc70 con 0x55eb8631a4b0 2024-09-10T07:40:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.990+0000 7feb18e76740 10 client.0 _ll_register_callbacks cb 0x55eb86222bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.991+0000 7feb0fe00640 1 -- 172.21.15.46:0/1283659689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 msgr2=0x55eb863275a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.991+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863275a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.991+0000 7feb0fe00640 1 -- 172.21.15.46:0/1283659689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 msgr2=0x55eb86327060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.991+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 0x55eb86327060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.991+0000 7feb0fe00640 1 -- 172.21.15.46:0/1283659689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb85ff13c0 con 0x55eb8632d660 2024-09-10T07:40:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.993+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb8632cde0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feb08004d20 tx=0x7feb08004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1694eb693b8361d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.993+0000 7feaf9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.994+0000 7feaf9600640 1 -- 172.21.15.46:0/1283659689 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08028060 con 0x55eb8632d660 2024-09-10T07:40:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.994+0000 7feaf9600640 1 -- 172.21.15.46:0/1283659689 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feb08003970 con 0x55eb8632d660 2024-09-10T07:40:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.994+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0800de00 con 0x55eb8632d660 2024-09-10T07:40:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.995+0000 7feaf9600640 1 -- 172.21.15.46:0/1283659689 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7feb0801d040 con 0x55eb8632d660 2024-09-10T07:40:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.995+0000 7feb18e76740 10 client.26294 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:16.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.995+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eb863449e0 con 0x55eb8632d660 2024-09-10T07:40:16.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.996+0000 7feaf9600640 1 -- 172.21.15.46:0/1283659689 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0800de00 con 0x55eb8632d660 2024-09-10T07:40:16.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.997+0000 7feaf9600640 1 -- 172.21.15.46:0/1283659689 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7feb08004510 con 0x55eb8632d660 2024-09-10T07:40:16.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.997+0000 7feb18e76740 10 client.26294 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:16.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.997+0000 7feb18e76740 10 client.26294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:40:16.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.998+0000 7feb18e76740 20 client.26294 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:16.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.998+0000 7feb18e76740 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55eb86345d90 0x55eb86348180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.998+0000 7feb18e76740 4 client.26294 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.998+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb8628d950 con 0x55eb86345d90 2024-09-10T07:40:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.998+0000 7feb0f400640 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55eb86345d90 0x55eb86348180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:16.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.999+0000 7feb0f400640 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55eb86345d90 0x55eb86348180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b4bdd6fb640bd37 server_cookie=9134c569fa48af21 in_seq=0 out_seq=0 2024-09-10T07:40:16.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:16.999+0000 7feaf9600640 10 client.26294 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:40:17.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.000+0000 7feaf9600640 1 -- 172.21.15.46:0/1283659689 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb8628d950 con 0x55eb86345d90 2024-09-10T07:40:17.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.000+0000 7feaf9600640 10 client.26294 handle_command_reply: tid=0 2024-09-10T07:40:17.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.000+0000 7feb18e76740 1 client.26294 shutdown 2024-09-10T07:40:17.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.001+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 msgr2=0x55eb8632cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:17.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.001+0000 7feb18e76740 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb8632cde0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feb08004d20 tx=0x7feb08004d50 comp rx=0 tx=0).stop 2024-09-10T07:40:17.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.002+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 shutdown_connections 2024-09-10T07:40:17.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.002+0000 7feb18e76740 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55eb86345d90 0x55eb86348180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.003+0000 7feb18e76740 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb8632cde0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.003+0000 7feb18e76740 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863275a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.003+0000 7feb18e76740 1 --2- 172.21.15.46:0/1283659689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 0x55eb86327060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.003+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 >> 172.21.15.46:0/1283659689 conn(0x55eb86316200 msgr2=0x55eb86315380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.003+0000 7feaf9600640 0 client.26294 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:40:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.003+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 shutdown_connections 2024-09-10T07:40:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.004+0000 7feb18e76740 1 -- 172.21.15.46:0/1283659689 wait complete. 2024-09-10T07:40:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.004+0000 7feb18e76740 20 client.26294 trim_cache size 0 max 16384 2024-09-10T07:40:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.029+0000 7feb18e76740 1 Processor -- start 2024-09-10T07:40:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.029+0000 7feb18e76740 1 -- start start 2024-09-10T07:40:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.029+0000 7feb18e76740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb8632cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.029+0000 7feb18e76740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8634adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.029+0000 7feb18e76740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8634b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb18e76740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feaf000fbf0 con 0x55eb8631a4b0 2024-09-10T07:40:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb18e76740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feaf00117c0 con 0x55eb86319b90 2024-09-10T07:40:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb18e76740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feaf00102c0 con 0x55eb8632d660 2024-09-10T07:40:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb8632cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb8632cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49758/0 (socket says 172.21.15.46:49758) 2024-09-10T07:40:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0f400640 1 -- 172.21.15.46:0/1531891090 learned_addr learned my addr 172.21.15.46:0/1531891090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0ea00640 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8634adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8634b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0fe00640 1 -- 172.21.15.46:0/1531891090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 msgr2=0x55eb8632cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb8632cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0fe00640 1 -- 172.21.15.46:0/1531891090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 msgr2=0x55eb8634adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.030+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8634adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.031+0000 7feb0fe00640 1 -- 172.21.15.46:0/1531891090 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb8631ccf0 con 0x55eb8632d660 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.031+0000 7feb0f400640 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb8632cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.031+0000 7feb0fe00640 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8634b300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb08003f50 tx=0x7feb080077a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d63df7dca3c2e77c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:17.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.031+0000 7feafbe00640 1 -- 172.21.15.46:0/1531891090 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08028070 con 0x55eb8632d660 2024-09-10T07:40:17.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.031+0000 7feafbe00640 1 -- 172.21.15.46:0/1531891090 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb08003970 con 0x55eb8632d660 2024-09-10T07:40:17.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.031+0000 7feafbe00640 1 -- 172.21.15.46:0/1531891090 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08032080 con 0x55eb8632d660 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.033+0000 7feb18e76740 1 -- 172.21.15.46:0/1531891090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 msgr2=0x55eb8634b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.033+0000 7feb18e76740 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8634b300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb08003f50 tx=0x7feb080077a0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.034+0000 7feb18e76740 1 -- 172.21.15.46:0/1531891090 shutdown_connections 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.034+0000 7feb18e76740 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8632d660 0x55eb8634b300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.034+0000 7feb18e76740 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb8631a4b0 0x55eb8634adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.034+0000 7feb18e76740 1 --2- 172.21.15.46:0/1531891090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb86319b90 0x55eb8632cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.034+0000 7feb18e76740 1 -- 172.21.15.46:0/1531891090 >> 172.21.15.46:0/1531891090 conn(0x55eb86316200 msgr2=0x55eb8631d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.034+0000 7feb18e76740 1 -- 172.21.15.46:0/1531891090 shutdown_connections 2024-09-10T07:40:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.034+0000 7feb18e76740 1 -- 172.21.15.46:0/1531891090 wait complete. 2024-09-10T07:40:17.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.035+0000 7feb18e76740 1 -- start start 2024-09-10T07:40:17.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.035+0000 7feb18e76740 1 Processor -- start 2024-09-10T07:40:17.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.036+0000 7feb18e76740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 0x55eb86348da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.036+0000 7feb18e76740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.036+0000 7feb18e76740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb86349820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.036+0000 7feb18e76740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feaf00110c0 con 0x55eb86319b90 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.036+0000 7feb18e76740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feaf00109f0 con 0x55eb8632d660 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.036+0000 7feb18e76740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feaf000dbd0 con 0x55eb8631a4b0 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.036+0000 7feb18e76740 10 client.0 _ll_register_callbacks cb 0x7feb0800f5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53696/0 (socket says 172.21.15.46:53696) 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 -- 172.21.15.46:0/2348313558 learned_addr learned my addr 172.21.15.46:0/2348313558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0f400640 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 0x55eb86348da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0fe00640 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb86349820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 -- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 msgr2=0x55eb86349820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb86349820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 -- 172.21.15.46:0/2348313558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 msgr2=0x55eb86348da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 0x55eb86348da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 -- 172.21.15.46:0/2348313558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb86311770 con 0x55eb8631a4b0 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0fe00640 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb86349820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feb0ea00640 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863492e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c6a0 tx=0x7feafc09c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a01fe1d799cb403b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.037+0000 7feaf9600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.038+0000 7feaf9600640 1 -- 172.21.15.46:0/2348313558 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc0a47e0 con 0x55eb8631a4b0 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.038+0000 7feaf9600640 1 -- 172.21.15.46:0/2348313558 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feafc0cfb40 con 0x55eb8631a4b0 2024-09-10T07:40:17.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.038+0000 7feb18e76740 1 -- 172.21.15.46:0/2348313558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb08003970 con 0x55eb8631a4b0 2024-09-10T07:40:17.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.038+0000 7feaf9600640 1 -- 172.21.15.46:0/2348313558 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7feafc0cfd80 con 0x55eb8631a4b0 2024-09-10T07:40:17.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.038+0000 7feb18e76740 10 client.26290 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:17.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.038+0000 7feb18e76740 1 -- 172.21.15.46:0/2348313558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eb86316030 con 0x55eb8631a4b0 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feaf9600640 1 -- 172.21.15.46:0/2348313558 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb08003970 con 0x55eb8631a4b0 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feaf9600640 1 -- 172.21.15.46:0/2348313558 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7feafc0978c0 con 0x55eb8631a4b0 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feb18e76740 10 client.26290 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feb18e76740 10 client.26290 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feb18e76740 20 client.26290 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feb18e76740 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55eb863b1010 0x55eb863b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feb18e76740 4 client.26290 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feb18e76740 1 -- 172.21.15.46:0/2348313558 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb86287e60 con 0x55eb863b1010 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.039+0000 7feb0f400640 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55eb863b1010 0x55eb863b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.040+0000 7feb0f400640 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55eb863b1010 0x55eb863b3430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cecd22b6c012d0de server_cookie=a95f5d66c25f2e0d in_seq=0 out_seq=0 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.040+0000 7feaf9600640 10 client.26290 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.040+0000 7feaf9600640 1 -- 172.21.15.46:0/2348313558 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb86287e60 con 0x55eb863b1010 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.040+0000 7feaf9600640 10 client.26290 handle_command_reply: tid=0 2024-09-10T07:40:17.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 client.26290 shutdown 2024-09-10T07:40:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 -- 172.21.15.46:0/2348313558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 msgr2=0x55eb863492e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863492e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c6a0 tx=0x7feafc09c270 comp rx=0 tx=0).stop 2024-09-10T07:40:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 -- 172.21.15.46:0/2348313558 shutdown_connections 2024-09-10T07:40:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55eb863b1010 0x55eb863b3430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb8632d660 0x55eb86349820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eb8631a4b0 0x55eb863492e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 --2- 172.21.15.46:0/2348313558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb86319b90 0x55eb86348da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feb18e76740 1 -- 172.21.15.46:0/2348313558 >> 172.21.15.46:0/2348313558 conn(0x55eb86316200 msgr2=0x55eb8631b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:17.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.041+0000 7feaf9600640 0 client.26290 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:40:17.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.042+0000 7feb18e76740 1 -- 172.21.15.46:0/2348313558 shutdown_connections 2024-09-10T07:40:17.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.042+0000 7feb18e76740 1 -- 172.21.15.46:0/2348313558 wait complete. 2024-09-10T07:40:17.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.043+0000 7feb18e76740 20 client.26290 trim_cache size 0 max 16384 2024-09-10T07:40:17.043 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:40:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.044+0000 7feb16400640 1 -- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feaf40a2c40 msgr2=0x7feaf40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.044+0000 7feb16400640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feaf40a2c40 0x7feaf40a5100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb00031550 tx=0x7feb00000bc0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 -- 172.21.15.46:0/2618078209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb101027a0 msgr2=0x7feb10074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb101027a0 0x7feb10074a30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e9f0 tx=0x7feafc00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 -- 172.21.15.46:0/2618078209 shutdown_connections 2024-09-10T07:40:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10106f20 0x7feb10074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb101027a0 0x7feb10074a30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feaf40a2c40 0x7feaf40a5100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 --2- 172.21.15.46:0/2618078209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb10101df0 0x7feb100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 -- 172.21.15.46:0/2618078209 >> 172.21.15.46:0/2618078209 conn(0x7feb100fd8b0 msgr2=0x7feb100ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 -- 172.21.15.46:0/2618078209 shutdown_connections 2024-09-10T07:40:17.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:17.045+0000 7feb16400640 1 -- 172.21.15.46:0/2618078209 wait complete. 2024-09-10T07:40:18.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:17 smithi181 ceph-mon[35397]: pgmap v509: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.5 MiB/s wr, 704 op/s 2024-09-10T07:40:18.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:17 smithi063 ceph-mon[35020]: pgmap v509: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.5 MiB/s wr, 704 op/s 2024-09-10T07:40:18.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:17 smithi046 ceph-mon[30705]: pgmap v509: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.5 MiB/s wr, 704 op/s 2024-09-10T07:40:19.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:40:20.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:19 smithi181 ceph-mon[35397]: pgmap v510: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-09-10T07:40:20.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:19 smithi063 ceph-mon[35020]: pgmap v510: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-09-10T07:40:20.157 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:20.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:19 smithi046 ceph-mon[30705]: pgmap v510: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-09-10T07:40:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 180.0 s, 1072.7 tps, lat 93.563 ms stddev 137.236 2024-09-10T07:40:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 -- 172.21.15.46:0/4187869273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac114ab0 msgr2=0x7f27ac116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 --2- 172.21.15.46:0/4187869273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac114ab0 0x7f27ac116ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009be0 tx=0x7f27a402ead0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 -- 172.21.15.46:0/4187869273 shutdown_connections 2024-09-10T07:40:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 --2- 172.21.15.46:0/4187869273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac114ab0 0x7f27ac116ea0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 --2- 172.21.15.46:0/4187869273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac109ba0 0x7f27ac114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 --2- 172.21.15.46:0/4187869273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ac1091f0 0x7f27ac1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 -- 172.21.15.46:0/4187869273 >> 172.21.15.46:0/4187869273 conn(0x7f27ac06e2d0 msgr2=0x7f27ac06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 -- 172.21.15.46:0/4187869273 shutdown_connections 2024-09-10T07:40:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.951+0000 7f27b2a00640 1 -- 172.21.15.46:0/4187869273 wait complete. 2024-09-10T07:40:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.953+0000 7f27b2a00640 1 Processor -- start 2024-09-10T07:40:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.953+0000 7f27b2a00640 1 -- start start 2024-09-10T07:40:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.954+0000 7f27b2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac1091f0 0x7f27ac1a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.954+0000 7f27b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ac109ba0 0x7f27ac1a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.954+0000 7f27b2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac114ab0 0x7f27ac1ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.954+0000 7f27b2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac120d70 con 0x7f27ac109ba0 2024-09-10T07:40:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.954+0000 7f27b2a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac120bf0 con 0x7f27ac114ab0 2024-09-10T07:40:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.954+0000 7f27b2a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac120ef0 con 0x7f27ac1091f0 2024-09-10T07:40:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.955+0000 7f27ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ac109ba0 0x7f27ac1a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.956+0000 7f27abe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac1091f0 0x7f27ac1a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.956+0000 7f27abe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac1091f0 0x7f27ac1a70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53720/0 (socket says 172.21.15.46:53720) 2024-09-10T07:40:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.956+0000 7f27abe00640 1 -- 172.21.15.46:0/2151273391 learned_addr learned my addr 172.21.15.46:0/2151273391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.956+0000 7f27b0a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac114ab0 0x7f27ac1ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.957+0000 7f27b0a00640 1 -- 172.21.15.46:0/2151273391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac1091f0 msgr2=0x7f27ac1a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.957+0000 7f27b0a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac1091f0 0x7f27ac1a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.957+0000 7f27b0a00640 1 -- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ac109ba0 msgr2=0x7f27ac1a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.957+0000 7f27b0a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ac109ba0 0x7f27ac1a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.957+0000 7f27b0a00640 1 -- 172.21.15.46:0/2151273391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac1ac140 con 0x7f27ac114ab0 2024-09-10T07:40:20.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.960+0000 7f27b0a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac114ab0 0x7f27ac1ab9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f27a402f0c0 tx=0x7f27a4031630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:20.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.960+0000 7f27a8c00640 1 -- 172.21.15.46:0/2151273391 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4031720 con 0x7f27ac114ab0 2024-09-10T07:40:20.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.960+0000 7f27a8c00640 1 -- 172.21.15.46:0/2151273391 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27a40318c0 con 0x7f27ac114ab0 2024-09-10T07:40:20.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.960+0000 7f27a8c00640 1 -- 172.21.15.46:0/2151273391 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40046c0 con 0x7f27ac114ab0 2024-09-10T07:40:20.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.960+0000 7f27b2a00640 1 -- 172.21.15.46:0/2151273391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ac1ac3d0 con 0x7f27ac114ab0 2024-09-10T07:40:20.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.961+0000 7f27b2a00640 1 -- 172.21.15.46:0/2151273391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ac1b3c70 con 0x7f27ac114ab0 2024-09-10T07:40:20.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.965+0000 7f27b5414740 1 Processor -- start 2024-09-10T07:40:20.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.971+0000 7f27a8c00640 1 -- 172.21.15.46:0/2151273391 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f27a4030a20 con 0x7f27ac114ab0 2024-09-10T07:40:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.971+0000 7f27b5414740 1 -- start start 2024-09-10T07:40:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27b5414740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4ac70 0x557f79c4b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27b5414740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 0x557f79c55910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27b5414740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c55e50 0x557f79c58240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27b5414740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557f79bb13e0 con 0x557f79c4b590 2024-09-10T07:40:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27b5414740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f79b58d80 con 0x557f79c4ac70 2024-09-10T07:40:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27b5414740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557f79bad3a0 con 0x557f79c55e50 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27ab400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c55e50 0x557f79c58240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27ab400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c55e50 0x557f79c58240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53724/0 (socket says 172.21.15.46:53724) 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.973+0000 7f27ab400640 1 -- 172.21.15.46:0/629257833 learned_addr learned my addr 172.21.15.46:0/629257833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.974+0000 7f27ab400640 1 --2- 172.21.15.46:0/629257833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4ac70 0x557f79c4b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.974+0000 7f27ab400640 1 -- 172.21.15.46:0/629257833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4ac70 msgr2=0x557f79c4b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.974+0000 7f27ab400640 1 --2- 172.21.15.46:0/629257833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4ac70 0x557f79c4b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.974+0000 7f27ab400640 1 -- 172.21.15.46:0/629257833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 msgr2=0x557f79c55910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.974+0000 7f27ab400640 1 --2- 172.21.15.46:0/629257833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 0x557f79c55910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.974+0000 7f27ab400640 1 -- 172.21.15.46:0/629257833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f79b77ae0 con 0x557f79c55e50 2024-09-10T07:40:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.976+0000 7f27ab400640 1 --2- 172.21.15.46:0/629257833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c55e50 0x557f79c58240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f27980058c0 tx=0x7f2798005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cec747ebbf6b6f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.976+0000 7f27a8c00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f27900a2d90 0x7f27900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.976+0000 7f27a8c00640 1 -- 172.21.15.46:0/2151273391 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f27a40f0610 con 0x7f27ac114ab0 2024-09-10T07:40:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.977+0000 7f27a1200640 1 -- 172.21.15.46:0/629257833 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798004030 con 0x557f79c55e50 2024-09-10T07:40:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.977+0000 7f27a1200640 1 -- 172.21.15.46:0/629257833 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2798005500 con 0x557f79c55e50 2024-09-10T07:40:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.977+0000 7f27a1200640 1 -- 172.21.15.46:0/629257833 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798004d90 con 0x557f79c55e50 2024-09-10T07:40:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.978+0000 7f27b5414740 1 -- 172.21.15.46:0/629257833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c55e50 msgr2=0x557f79c58240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.978+0000 7f27b5414740 1 --2- 172.21.15.46:0/629257833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c55e50 0x557f79c58240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f27980058c0 tx=0x7f2798005850 comp rx=0 tx=0).stop 2024-09-10T07:40:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.978+0000 7f27abe00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f27900a2d90 0x7f27900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.978+0000 7f27b5414740 1 -- 172.21.15.46:0/629257833 shutdown_connections 2024-09-10T07:40:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.978+0000 7f27b5414740 1 --2- 172.21.15.46:0/629257833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c55e50 0x557f79c58240 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.978+0000 7f27b5414740 1 --2- 172.21.15.46:0/629257833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 0x557f79c55910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.978+0000 7f27b5414740 1 --2- 172.21.15.46:0/629257833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4ac70 0x557f79c4b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.978+0000 7f27b5414740 1 -- 172.21.15.46:0/629257833 >> 172.21.15.46:0/629257833 conn(0x557f79c47200 msgr2=0x557f79c475e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.979+0000 7f27b5414740 1 -- 172.21.15.46:0/629257833 shutdown_connections 2024-09-10T07:40:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.980+0000 7f27b5414740 1 -- 172.21.15.46:0/629257833 wait complete. 2024-09-10T07:40:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.981+0000 7f27b5414740 1 -- start start 2024-09-10T07:40:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.981+0000 7f27abe00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f27900a2d90 0x7f27900a5250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x557f79c48060 tx=0x7f279c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.981+0000 7f27b5414740 1 Processor -- start 2024-09-10T07:40:20.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27b5414740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c52d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27b5414740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27b5414740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 0x557f79c53800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27b5414740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557f79c58a20 con 0x557f79c55e50 2024-09-10T07:40:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27b5414740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f79b6c0f0 con 0x557f79c4b590 2024-09-10T07:40:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27b5414740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557f79c58ba0 con 0x557f79c4ac70 2024-09-10T07:40:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27b5414740 10 client.0 _ll_register_callbacks cb 0x557f79b53bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27ab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.982+0000 7f27ab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49796/0 (socket says 172.21.15.46:49796) 2024-09-10T07:40:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.983+0000 7f27ab400640 1 -- 172.21.15.46:0/3966972487 learned_addr learned my addr 172.21.15.46:0/3966972487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.983+0000 7f27abe00640 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c52d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.983+0000 7f27ab400640 1 -- 172.21.15.46:0/3966972487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 msgr2=0x557f79c52d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.983+0000 7f27ab400640 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c52d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.983+0000 7f27ab400640 1 -- 172.21.15.46:0/3966972487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 msgr2=0x557f79c53800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:20.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.983+0000 7f27ab400640 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 0x557f79c53800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.983+0000 7f27ab400640 1 -- 172.21.15.46:0/3966972487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f799223c0 con 0x557f79c4b590 2024-09-10T07:40:20.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.986+0000 7f2795600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:20.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.986+0000 7f27ab400640 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c532c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2798009670 tx=0x7f27980115f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5bd93d94b78751b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:20.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.986+0000 7f2795600640 1 -- 172.21.15.46:0/3966972487 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980046c0 con 0x557f79c4b590 2024-09-10T07:40:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.986+0000 7f2795600640 1 -- 172.21.15.46:0/3966972487 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f279800f790 con 0x557f79c4b590 2024-09-10T07:40:20.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.987+0000 7f27b5414740 1 -- 172.21.15.46:0/3966972487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2798005500 con 0x557f79c4b590 2024-09-10T07:40:20.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.988+0000 7f2795600640 1 -- 172.21.15.46:0/3966972487 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f279802f050 con 0x557f79c4b590 2024-09-10T07:40:20.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.989+0000 7f27b5414740 10 client.26309 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:20.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.989+0000 7f27b5414740 1 -- 172.21.15.46:0/3966972487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557f79c64ab0 con 0x557f79c4b590 2024-09-10T07:40:20.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.989+0000 7f2795600640 1 -- 172.21.15.46:0/3966972487 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2798005500 con 0x557f79c4b590 2024-09-10T07:40:20.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.990+0000 7f2795600640 1 -- 172.21.15.46:0/3966972487 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f279800f9b0 con 0x557f79c4b590 2024-09-10T07:40:20.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.990+0000 7f27b5414740 10 client.26309 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:20.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.990+0000 7f27b5414740 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:20.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.991+0000 7f27b5414740 20 client.26309 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:20.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.991+0000 7f27b5414740 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557f79c65e00 0x557f79c681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:20.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.991+0000 7f27b5414740 4 client.26309 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:20.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.991+0000 7f27b5414740 1 -- 172.21.15.46:0/3966972487 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f79bbe950 con 0x557f79c65e00 2024-09-10T07:40:20.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.992+0000 7f27abe00640 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557f79c65e00 0x557f79c681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.994+0000 7f27abe00640 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557f79c65e00 0x557f79c681f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2752cc86bc989e03 server_cookie=62e2819e7dcf40fe in_seq=0 out_seq=0 2024-09-10T07:40:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.994+0000 7f2795600640 10 client.26309 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.995+0000 7f2795600640 1 -- 172.21.15.46:0/3966972487 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f79bbe950 con 0x557f79c65e00 2024-09-10T07:40:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.995+0000 7f2795600640 10 client.26309 handle_command_reply: tid=0 2024-09-10T07:40:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.996+0000 7f27b5414740 1 client.26309 shutdown 2024-09-10T07:40:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.996+0000 7f27b5414740 1 -- 172.21.15.46:0/3966972487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 msgr2=0x557f79c532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.996+0000 7f27b5414740 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c532c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2798009670 tx=0x7f27980115f0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.996+0000 7f27b5414740 1 -- 172.21.15.46:0/3966972487 shutdown_connections 2024-09-10T07:40:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.997+0000 7f27b5414740 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557f79c65e00 0x557f79c681f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.997+0000 7f27b5414740 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 0x557f79c53800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.997+0000 7f27b5414740 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c532c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.997+0000 7f27b5414740 1 --2- 172.21.15.46:0/3966972487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c52d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.997+0000 7f27b5414740 1 -- 172.21.15.46:0/3966972487 >> 172.21.15.46:0/3966972487 conn(0x557f79c47200 msgr2=0x557f79c4cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.997+0000 7f2795600640 0 client.26309 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.997+0000 7f27b5414740 1 -- 172.21.15.46:0/3966972487 shutdown_connections 2024-09-10T07:40:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.997+0000 7f27b5414740 1 -- 172.21.15.46:0/3966972487 wait complete. 2024-09-10T07:40:20.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:20.998+0000 7f27b5414740 20 client.26309 trim_cache size 0 max 16384 2024-09-10T07:40:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.022+0000 7f27b5414740 1 Processor -- start 2024-09-10T07:40:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.022+0000 7f27b5414740 1 -- start start 2024-09-10T07:40:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b5414740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c6da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b5414740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 0x557f79c6df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b5414740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c55e50 0x557f79c4c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b5414740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac1bda50 con 0x557f79c4b590 2024-09-10T07:40:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b5414740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac1bf620 con 0x557f79c55e50 2024-09-10T07:40:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b5414740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac1be120 con 0x557f79c4ac70 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c55e50 0x557f79c4c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c55e50 0x557f79c4c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49798/0 (socket says 172.21.15.46:49798) 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27b0a00640 1 -- 172.21.15.46:0/2434440483 learned_addr learned my addr 172.21.15.46:0/2434440483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27ab400640 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 0x557f79c6df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.023+0000 7f27abe00640 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c6da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f27b0a00640 1 -- 172.21.15.46:0/2434440483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 msgr2=0x557f79c6da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f27b0a00640 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c6da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f27b0a00640 1 -- 172.21.15.46:0/2434440483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 msgr2=0x557f79c6df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f27b0a00640 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 0x557f79c6df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f27b0a00640 1 -- 172.21.15.46:0/2434440483 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f79c42770 con 0x557f79c55e50 2024-09-10T07:40:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f27b0a00640 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c55e50 0x557f79c4c6c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f27a40bca20 tx=0x7f27a40b8610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acfe1dc48ab2e6f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f2797e00640 1 -- 172.21.15.46:0/2434440483 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40d4020 con 0x557f79c55e50 2024-09-10T07:40:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f2797e00640 1 -- 172.21.15.46:0/2434440483 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27a40bd190 con 0x557f79c55e50 2024-09-10T07:40:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.024+0000 7f2797e00640 1 -- 172.21.15.46:0/2434440483 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40f0300 con 0x557f79c55e50 2024-09-10T07:40:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.025+0000 7f27b5414740 1 -- 172.21.15.46:0/2434440483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c55e50 msgr2=0x557f79c4c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.025+0000 7f27b5414740 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c55e50 0x557f79c4c6c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f27a40bca20 tx=0x7f27a40b8610 comp rx=0 tx=0).stop 2024-09-10T07:40:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.026+0000 7f27b5414740 1 -- 172.21.15.46:0/2434440483 shutdown_connections 2024-09-10T07:40:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.026+0000 7f27b5414740 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c55e50 0x557f79c4c6c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.026+0000 7f27b5414740 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c4b590 0x557f79c6df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.026+0000 7f27b5414740 1 --2- 172.21.15.46:0/2434440483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c6da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.026+0000 7f27b5414740 1 -- 172.21.15.46:0/2434440483 >> 172.21.15.46:0/2434440483 conn(0x557f79c47200 msgr2=0x557f79c475e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.026+0000 7f27b5414740 1 -- 172.21.15.46:0/2434440483 shutdown_connections 2024-09-10T07:40:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.026+0000 7f27b5414740 1 -- 172.21.15.46:0/2434440483 wait complete. 2024-09-10T07:40:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.027+0000 7f27b5414740 1 -- start start 2024-09-10T07:40:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.027+0000 7f27b5414740 1 Processor -- start 2024-09-10T07:40:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.029+0000 7f27b5414740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c5ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.029+0000 7f27b5414740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.029+0000 7f27b5414740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 0x557f79c5adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.029+0000 7f27b5414740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac1bef20 con 0x557f79c55e50 2024-09-10T07:40:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.029+0000 7f27b5414740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac1be850 con 0x557f79c4b590 2024-09-10T07:40:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.029+0000 7f27b5414740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac1bb9f0 con 0x557f79c4ac70 2024-09-10T07:40:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.029+0000 7f27b5414740 10 client.0 _ll_register_callbacks cb 0x7f2798009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:21.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27b0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 0x557f79c5adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:21.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27abe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c5ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27abe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c5ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53744/0 (socket says 172.21.15.46:53744) 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27abe00640 1 -- 172.21.15.46:0/403227591 learned_addr learned my addr 172.21.15.46:0/403227591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27ab400640 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c5e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27b0a00640 1 -- 172.21.15.46:0/403227591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 msgr2=0x557f79c5ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27b0a00640 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c5ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27b0a00640 1 -- 172.21.15.46:0/403227591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 msgr2=0x557f79c5e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.030+0000 7f27b0a00640 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c5e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.031+0000 7f27b0a00640 1 -- 172.21.15.46:0/403227591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f79c5ea20 con 0x557f79c55e50 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.031+0000 7f27ab400640 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c5e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.031+0000 7f27b0a00640 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 0x557f79c5adc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x557f79b53c20 tx=0x7f27a40bc640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5277dbbe2c191a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.031+0000 7f2795600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.032+0000 7f2795600640 1 -- 172.21.15.46:0/403227591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40d4070 con 0x557f79c55e50 2024-09-10T07:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.032+0000 7f2795600640 1 -- 172.21.15.46:0/403227591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f27a40f0300 con 0x557f79c55e50 2024-09-10T07:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.032+0000 7f27b5414740 1 -- 172.21.15.46:0/403227591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a40bd190 con 0x557f79c55e50 2024-09-10T07:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.032+0000 7f2795600640 1 -- 172.21.15.46:0/403227591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f27a40db050 con 0x557f79c55e50 2024-09-10T07:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.033+0000 7f27b5414740 10 client.16065 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.033+0000 7f27b5414740 1 -- 172.21.15.46:0/403227591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557f79cd1030 con 0x557f79c55e50 2024-09-10T07:40:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.034+0000 7f2795600640 1 -- 172.21.15.46:0/403227591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27a40bd190 con 0x557f79c55e50 2024-09-10T07:40:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.034+0000 7f2795600640 1 -- 172.21.15.46:0/403227591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f27a4030040 con 0x557f79c55e50 2024-09-10T07:40:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.034+0000 7f27b5414740 10 client.16065 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.034+0000 7f27b5414740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.034+0000 7f27b5414740 20 client.16065 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.034+0000 7f27b5414740 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557f79cd1640 0x557f79cd3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.034+0000 7f27b5414740 4 client.16065 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:40:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.034+0000 7f27b5414740 1 -- 172.21.15.46:0/403227591 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f79bb8e60 con 0x557f79cd1640 2024-09-10T07:40:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.036+0000 7f27abe00640 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557f79cd1640 0x557f79cd3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.039+0000 7f27abe00640 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557f79cd1640 0x557f79cd3a60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ced203e1431908 server_cookie=95e6d40af33d517c in_seq=0 out_seq=0 2024-09-10T07:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.039+0000 7f2795600640 10 client.16065 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f2795600640 1 -- 172.21.15.46:0/403227591 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f79bb8e60 con 0x557f79cd1640 2024-09-10T07:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f2795600640 10 client.16065 handle_command_reply: tid=0 2024-09-10T07:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 client.16065 shutdown 2024-09-10T07:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 -- 172.21.15.46:0/403227591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 msgr2=0x557f79c5adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 0x557f79c5adc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x557f79b53c20 tx=0x7f27a40bc640 comp rx=0 tx=0).stop 2024-09-10T07:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 -- 172.21.15.46:0/403227591 shutdown_connections 2024-09-10T07:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557f79cd1640 0x557f79cd3a60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f79c55e50 0x557f79c5adc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f79c4b590 0x557f79c5e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 --2- 172.21.15.46:0/403227591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557f79c4ac70 0x557f79c5ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f2795600640 0 client.16065 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.041+0000 7f27b5414740 1 -- 172.21.15.46:0/403227591 >> 172.21.15.46:0/403227591 conn(0x557f79c47200 msgr2=0x557f79c545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.042+0000 7f27b5414740 1 -- 172.21.15.46:0/403227591 shutdown_connections 2024-09-10T07:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.042+0000 7f27b5414740 1 -- 172.21.15.46:0/403227591 wait complete. 2024-09-10T07:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.042+0000 7f27b5414740 20 client.16065 trim_cache size 0 max 16384 2024-09-10T07:40:21.044 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.043+0000 7f27b2a00640 1 -- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f27900a2d90 msgr2=0x7f27900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.043+0000 7f27b2a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f27900a2d90 0x7f27900a5250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x557f79c48060 tx=0x7f279c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.043+0000 7f27b2a00640 1 -- 172.21.15.46:0/2151273391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac114ab0 msgr2=0x7f27ac1ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.043+0000 7f27b2a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac114ab0 0x7f27ac1ab9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f27a402f0c0 tx=0x7f27a4031630 comp rx=0 tx=0).stop 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.044+0000 7f27b2a00640 1 -- 172.21.15.46:0/2151273391 shutdown_connections 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.044+0000 7f27b2a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac114ab0 0x7f27ac1ab9c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.044+0000 7f27b2a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f27900a2d90 0x7f27900a5250 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.044+0000 7f27b2a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ac109ba0 0x7f27ac1a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.044+0000 7f27b2a00640 1 --2- 172.21.15.46:0/2151273391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ac1091f0 0x7f27ac1a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.044+0000 7f27b2a00640 1 -- 172.21.15.46:0/2151273391 >> 172.21.15.46:0/2151273391 conn(0x7f27ac06e2d0 msgr2=0x7f27ac0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.044+0000 7f27b2a00640 1 -- 172.21.15.46:0/2151273391 shutdown_connections 2024-09-10T07:40:21.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:21.045+0000 7f27b2a00640 1 -- 172.21.15.46:0/2151273391 wait complete. 2024-09-10T07:40:22.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:21 smithi181 ceph-mon[35397]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 634 op/s 2024-09-10T07:40:22.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:21 smithi063 ceph-mon[35020]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 634 op/s 2024-09-10T07:40:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:21 smithi046 ceph-mon[30705]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 634 op/s 2024-09-10T07:40:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:23 smithi181 ceph-mon[35397]: pgmap v512: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 MiB/s wr, 615 op/s 2024-09-10T07:40:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:23 smithi063 ceph-mon[35020]: pgmap v512: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 MiB/s wr, 615 op/s 2024-09-10T07:40:24.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:23 smithi046 ceph-mon[30705]: pgmap v512: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 MiB/s wr, 615 op/s 2024-09-10T07:40:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 185.0 s, 1174.3 tps, lat 84.504 ms stddev 118.206 2024-09-10T07:40:26.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:25 smithi181 ceph-mon[35397]: pgmap v513: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-09-10T07:40:26.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:25 smithi063 ceph-mon[35020]: pgmap v513: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-09-10T07:40:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:25 smithi046 ceph-mon[30705]: pgmap v513: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-09-10T07:40:26.336 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:40:26.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:40:26.768 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T07:40:26.768 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T07:40:26.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:40:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 -- 172.21.15.46:0/1421643855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c109930 msgr2=0x7efd1c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 --2- 172.21.15.46:0/1421643855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c109930 0x7efd1c109d30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efd14009ca0 tx=0x7efd1402a8d0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 -- 172.21.15.46:0/1421643855 shutdown_connections 2024-09-10T07:40:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 --2- 172.21.15.46:0/1421643855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1c0737d0 0x7efd1c112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 --2- 172.21.15.46:0/1421643855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c072d20 0x7efd1c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 --2- 172.21.15.46:0/1421643855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c109930 0x7efd1c109d30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 -- 172.21.15.46:0/1421643855 >> 172.21.15.46:0/1421643855 conn(0x7efd1c06ece0 msgr2=0x7efd1c071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 -- 172.21.15.46:0/1421643855 shutdown_connections 2024-09-10T07:40:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.209+0000 7efd20e00640 1 -- 172.21.15.46:0/1421643855 wait complete. 2024-09-10T07:40:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.210+0000 7efd20e00640 1 Processor -- start 2024-09-10T07:40:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.211+0000 7efd20e00640 1 -- start start 2024-09-10T07:40:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.211+0000 7efd20e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1c072d20 0x7efd1c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.211+0000 7efd20e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0737d0 0x7efd1c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.211+0000 7efd20e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c109930 0x7efd1c1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.211+0000 7efd20e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c113e10 con 0x7efd1c072d20 2024-09-10T07:40:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.211+0000 7efd20e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c113c90 con 0x7efd1c0737d0 2024-09-10T07:40:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.211+0000 7efd20e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c113f90 con 0x7efd1c109930 2024-09-10T07:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0737d0 0x7efd1c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1c072d20 0x7efd1c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0737d0 0x7efd1c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:46950/0 (socket says 172.21.15.46:46950) 2024-09-10T07:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 -- 172.21.15.46:0/1176414906 learned_addr learned my addr 172.21.15.46:0/1176414906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd1a800640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c109930 0x7efd1c1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 -- 172.21.15.46:0/1176414906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c109930 msgr2=0x7efd1c1a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c109930 0x7efd1c1a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 -- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1c072d20 msgr2=0x7efd1c19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1c072d20 0x7efd1c19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 -- 172.21.15.46:0/1176414906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c1a5fb0 con 0x7efd1c0737d0 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd1a800640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c109930 0x7efd1c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.212+0000 7efd19400640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0737d0 0x7efd1c19e8d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e9f0 tx=0x7efd0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.213+0000 7efd12a00640 1 -- 172.21.15.46:0/1176414906 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0800cd80 con 0x7efd1c0737d0 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.213+0000 7efd12a00640 1 -- 172.21.15.46:0/1176414906 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd08010910 con 0x7efd1c0737d0 2024-09-10T07:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.213+0000 7efd12a00640 1 -- 172.21.15.46:0/1176414906 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08018980 con 0x7efd1c0737d0 2024-09-10T07:40:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.213+0000 7efd20e00640 1 -- 172.21.15.46:0/1176414906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c1a62a0 con 0x7efd1c0737d0 2024-09-10T07:40:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.214+0000 7efd20e00640 1 -- 172.21.15.46:0/1176414906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c1a6770 con 0x7efd1c0737d0 2024-09-10T07:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.221+0000 7efd23c06740 1 Processor -- start 2024-09-10T07:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.221+0000 7efd23c06740 1 -- start start 2024-09-10T07:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.221+0000 7efd23c06740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a058a0 0x5584b4a05ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.221+0000 7efd23c06740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a061e0 0x5584b4a0aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.221+0000 7efd23c06740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a0d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.221+0000 7efd23c06740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5584b4962100 con 0x5584b4a0b270 2024-09-10T07:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.221+0000 7efd23c06740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5584b490a1b0 con 0x5584b4a058a0 2024-09-10T07:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.221+0000 7efd23c06740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5584b495e0c0 con 0x5584b4a061e0 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd12a00640 1 -- 172.21.15.46:0/1176414906 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd08010ac0 con 0x7efd1c0737d0 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd1a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a058a0 0x5584b4a05ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd1a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a058a0 0x5584b4a05ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:46960/0 (socket says 172.21.15.46:46960) 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd1a800640 1 -- 172.21.15.46:0/147187178 learned_addr learned my addr 172.21.15.46:0/147187178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd19e00640 1 --2- 172.21.15.46:0/147187178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a061e0 0x5584b4a0aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd19e00640 1 -- 172.21.15.46:0/147187178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a058a0 msgr2=0x5584b4a05ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd19e00640 1 --2- 172.21.15.46:0/147187178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a058a0 0x5584b4a05ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd19e00640 1 -- 172.21.15.46:0/147187178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 msgr2=0x5584b4a0d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd19e00640 1 --2- 172.21.15.46:0/147187178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a0d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.222+0000 7efd19e00640 1 -- 172.21.15.46:0/147187178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b4675210 con 0x5584b4a061e0 2024-09-10T07:40:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.223+0000 7efd19e00640 1 --2- 172.21.15.46:0/147187178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a061e0 0x5584b4a0aca0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd1402aec0 tx=0x7efd14005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27170d8f189ac3bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.225+0000 7efd07400640 1 -- 172.21.15.46:0/147187178 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14030a30 con 0x5584b4a061e0 2024-09-10T07:40:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.225+0000 7efd07400640 1 -- 172.21.15.46:0/147187178 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14036030 con 0x5584b4a061e0 2024-09-10T07:40:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.225+0000 7efd07400640 1 -- 172.21.15.46:0/147187178 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14004430 con 0x5584b4a061e0 2024-09-10T07:40:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.225+0000 7efd12a00640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efd000a2c30 0x7efd000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.225+0000 7efd12a00640 1 -- 172.21.15.46:0/1176414906 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efd080cf990 con 0x7efd1c0737d0 2024-09-10T07:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.226+0000 7efd1a800640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efd000a2c30 0x7efd000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.226+0000 7efd23c06740 1 -- 172.21.15.46:0/147187178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a061e0 msgr2=0x5584b4a0aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.226+0000 7efd23c06740 1 --2- 172.21.15.46:0/147187178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a061e0 0x5584b4a0aca0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd1402aec0 tx=0x7efd14005fa0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.227+0000 7efd1a800640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efd000a2c30 0x7efd000a50f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00be10 tx=0x7efd0c00b5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.227+0000 7efd23c06740 1 -- 172.21.15.46:0/147187178 shutdown_connections 2024-09-10T07:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.227+0000 7efd23c06740 1 --2- 172.21.15.46:0/147187178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a0d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.227+0000 7efd23c06740 1 --2- 172.21.15.46:0/147187178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a061e0 0x5584b4a0aca0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.227+0000 7efd23c06740 1 --2- 172.21.15.46:0/147187178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a058a0 0x5584b4a05ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.227+0000 7efd23c06740 1 -- 172.21.15.46:0/147187178 >> 172.21.15.46:0/147187178 conn(0x5584b4a00f10 msgr2=0x5584b4a01310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.227+0000 7efd23c06740 1 -- 172.21.15.46:0/147187178 shutdown_connections 2024-09-10T07:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.227+0000 7efd23c06740 1 -- 172.21.15.46:0/147187178 wait complete. 2024-09-10T07:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.228+0000 7efd23c06740 1 -- start start 2024-09-10T07:40:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.229+0000 7efd23c06740 1 Processor -- start 2024-09-10T07:40:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.231+0000 7efd23c06740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a0fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.231+0000 7efd23c06740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a0ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.231+0000 7efd23c06740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a17000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.231+0000 7efd23c06740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5584b4a0e1e0 con 0x5584b4a0b270 2024-09-10T07:40:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.232+0000 7efd19e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a0fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.232+0000 7efd23c06740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5584b48eb090 con 0x5584b4a061e0 2024-09-10T07:40:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.232+0000 7efd23c06740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5584b4a0e360 con 0x5584b4a058a0 2024-09-10T07:40:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.232+0000 7efd19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a0ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.232+0000 7efd19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a0ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:46972/0 (socket says 172.21.15.46:46972) 2024-09-10T07:40:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.232+0000 7efd19400640 1 -- 172.21.15.46:0/644379078 learned_addr learned my addr 172.21.15.46:0/644379078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.232+0000 7efd23c06740 10 client.0 _ll_register_callbacks cb 0x5584b493ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.233+0000 7efd1a800640 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a17000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.233+0000 7efd19400640 1 -- 172.21.15.46:0/644379078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 msgr2=0x5584b4a0fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.233+0000 7efd19400640 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a0fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.233+0000 7efd19400640 1 -- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 msgr2=0x5584b4a17000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.233+0000 7efd19400640 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a17000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.233+0000 7efd19400640 1 -- 172.21.15.46:0/644379078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b4741620 con 0x5584b4a061e0 2024-09-10T07:40:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.234+0000 7efd1a800640 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a17000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.234+0000 7efcfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.235+0000 7efd19400640 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a0ffc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efd08098270 tx=0x7efd0809c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67e538895432256a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.235+0000 7efcfbe00640 1 -- 172.21.15.46:0/644379078 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0809c270 con 0x5584b4a061e0 2024-09-10T07:40:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.235+0000 7efcfbe00640 1 -- 172.21.15.46:0/644379078 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efd080a45f0 con 0x5584b4a061e0 2024-09-10T07:40:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.235+0000 7efd23c06740 1 -- 172.21.15.46:0/644379078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd14036030 con 0x5584b4a061e0 2024-09-10T07:40:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.236+0000 7efcfbe00640 1 -- 172.21.15.46:0/644379078 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efd080a4a80 con 0x5584b4a061e0 2024-09-10T07:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.236+0000 7efd23c06740 10 client.26324 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.236+0000 7efd23c06740 1 -- 172.21.15.46:0/644379078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5584b4a19f10 con 0x5584b4a061e0 2024-09-10T07:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.237+0000 7efcfbe00640 1 -- 172.21.15.46:0/644379078 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14036030 con 0x5584b4a061e0 2024-09-10T07:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.237+0000 7efcfbe00640 1 -- 172.21.15.46:0/644379078 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7efd08010430 con 0x5584b4a061e0 2024-09-10T07:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.237+0000 7efd23c06740 10 client.26324 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.237+0000 7efd23c06740 10 client.26324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.237+0000 7efd23c06740 20 client.26324 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.237+0000 7efd23c06740 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5584b4a1b5e0 0x5584b4a1d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.237+0000 7efd23c06740 4 client.26324 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.237+0000 7efd23c06740 1 -- 172.21.15.46:0/644379078 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584b496f830 con 0x5584b4a1b5e0 2024-09-10T07:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.238+0000 7efd19e00640 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5584b4a1b5e0 0x5584b4a1d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.239+0000 7efd19e00640 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5584b4a1b5e0 0x5584b4a1d9d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e5b5c3f46c3e1 server_cookie=dec537bed08c4f35 in_seq=0 out_seq=0 2024-09-10T07:40:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.239+0000 7efcfbe00640 10 client.26324 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.240+0000 7efcfbe00640 1 -- 172.21.15.46:0/644379078 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584b496f830 con 0x5584b4a1b5e0 2024-09-10T07:40:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.240+0000 7efcfbe00640 10 client.26324 handle_command_reply: tid=0 2024-09-10T07:40:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.240+0000 7efd23c06740 1 client.26324 shutdown 2024-09-10T07:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.241+0000 7efd23c06740 1 -- 172.21.15.46:0/644379078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 msgr2=0x5584b4a0ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.242+0000 7efd23c06740 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a0ffc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efd08098270 tx=0x7efd0809c510 comp rx=0 tx=0).stop 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.242+0000 7efd23c06740 1 -- 172.21.15.46:0/644379078 shutdown_connections 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.242+0000 7efd23c06740 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5584b4a1b5e0 0x5584b4a1d9d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.242+0000 7efd23c06740 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a17000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.242+0000 7efd23c06740 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a0ffc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.242+0000 7efd23c06740 1 --2- 172.21.15.46:0/644379078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a0fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.242+0000 7efd23c06740 1 -- 172.21.15.46:0/644379078 >> 172.21.15.46:0/644379078 conn(0x5584b4a00f10 msgr2=0x5584b4a02720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.242+0000 7efcfbe00640 0 client.26324 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.243+0000 7efd23c06740 1 -- 172.21.15.46:0/644379078 shutdown_connections 2024-09-10T07:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.243+0000 7efd23c06740 1 -- 172.21.15.46:0/644379078 wait complete. 2024-09-10T07:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.243+0000 7efd23c06740 20 client.26324 trim_cache size 0 max 16384 2024-09-10T07:40:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.267+0000 7efd23c06740 1 Processor -- start 2024-09-10T07:40:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.270+0000 7efd23c06740 1 -- start start 2024-09-10T07:40:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.270+0000 7efd23c06740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a17dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.270+0000 7efd23c06740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a16fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.271+0000 7efd23c06740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a01dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.271+0000 7efd23c06740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1b0c70 con 0x5584b4a0b270 2024-09-10T07:40:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.271+0000 7efd23c06740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1b2840 con 0x5584b4a061e0 2024-09-10T07:40:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.271+0000 7efd23c06740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1b1340 con 0x5584b4a058a0 2024-09-10T07:40:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.271+0000 7efd19e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a17dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.271+0000 7efd19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a16fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.272+0000 7efd19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a16fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:46980/0 (socket says 172.21.15.46:46980) 2024-09-10T07:40:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.272+0000 7efd19400640 1 -- 172.21.15.46:0/1103481506 learned_addr learned my addr 172.21.15.46:0/1103481506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.272+0000 7efd19400640 1 -- 172.21.15.46:0/1103481506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 msgr2=0x5584b4a17dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.273+0000 7efd19400640 1 --2- 172.21.15.46:0/1103481506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a17dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.273+0000 7efd19400640 1 -- 172.21.15.46:0/1103481506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 msgr2=0x5584b4a01dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.273+0000 7efd19400640 1 --2- 172.21.15.46:0/1103481506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a01dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.273+0000 7efd19400640 1 -- 172.21.15.46:0/1103481506 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b4a066c0 con 0x5584b4a061e0 2024-09-10T07:40:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.273+0000 7efd19e00640 1 --2- 172.21.15.46:0/1103481506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a17dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.274+0000 7efd19400640 1 --2- 172.21.15.46:0/1103481506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a16fc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efd0809bb10 tx=0x7efd08018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ce6bb362c94ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.276+0000 7efd06a00640 1 -- 172.21.15.46:0/1103481506 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08010430 con 0x5584b4a061e0 2024-09-10T07:40:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.276+0000 7efd06a00640 1 -- 172.21.15.46:0/1103481506 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd080a45f0 con 0x5584b4a061e0 2024-09-10T07:40:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.276+0000 7efd06a00640 1 -- 172.21.15.46:0/1103481506 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08010430 con 0x5584b4a061e0 2024-09-10T07:40:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.277+0000 7efd23c06740 1 -- 172.21.15.46:0/1103481506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 msgr2=0x5584b4a16fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.277+0000 7efd23c06740 1 --2- 172.21.15.46:0/1103481506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a16fc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efd0809bb10 tx=0x7efd08018420 comp rx=0 tx=0).stop 2024-09-10T07:40:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.277+0000 7efd23c06740 1 -- 172.21.15.46:0/1103481506 shutdown_connections 2024-09-10T07:40:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.277+0000 7efd23c06740 1 --2- 172.21.15.46:0/1103481506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a01dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.278+0000 7efd23c06740 1 --2- 172.21.15.46:0/1103481506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a16fc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.278+0000 7efd23c06740 1 --2- 172.21.15.46:0/1103481506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a17dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.278+0000 7efd23c06740 1 -- 172.21.15.46:0/1103481506 >> 172.21.15.46:0/1103481506 conn(0x5584b4a00f10 msgr2=0x5584b4a176b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.278+0000 7efd23c06740 1 -- 172.21.15.46:0/1103481506 shutdown_connections 2024-09-10T07:40:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.278+0000 7efd23c06740 1 -- 172.21.15.46:0/1103481506 wait complete. 2024-09-10T07:40:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.279+0000 7efd23c06740 1 -- start start 2024-09-10T07:40:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.281+0000 7efd23c06740 1 Processor -- start 2024-09-10T07:40:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.282+0000 7efd23c06740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.282+0000 7efd23c06740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a14ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.282+0000 7efd23c06740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a15a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.282+0000 7efd23c06740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1b2140 con 0x5584b4a0b270 2024-09-10T07:40:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.282+0000 7efd23c06740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1b1a70 con 0x5584b4a061e0 2024-09-10T07:40:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.282+0000 7efd23c06740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1aec50 con 0x5584b4a058a0 2024-09-10T07:40:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.282+0000 7efd23c06740 10 client.0 _ll_register_callbacks cb 0x7efd08009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.284+0000 7efd1a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a15a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.284+0000 7efd19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a14ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.285+0000 7efd19e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.285+0000 7efd19400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a14ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:46990/0 (socket says 172.21.15.46:46990) 2024-09-10T07:40:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.285+0000 7efd19400640 1 -- 172.21.15.46:0/690066545 learned_addr learned my addr 172.21.15.46:0/690066545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.286+0000 7efd19400640 1 -- 172.21.15.46:0/690066545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 msgr2=0x5584b4a14580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.286+0000 7efd19400640 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a14580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.286+0000 7efd19400640 1 -- 172.21.15.46:0/690066545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 msgr2=0x5584b4a15a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.286+0000 7efd19400640 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a15a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.286+0000 7efd19400640 1 -- 172.21.15.46:0/690066545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b4a16040 con 0x5584b4a061e0 2024-09-10T07:40:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.287+0000 7efd19400640 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a14ac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd0809bda0 tx=0x7efd080cfc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a9269ac7a839b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.287+0000 7efcfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.288+0000 7efcfbe00640 1 -- 172.21.15.46:0/690066545 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0809c2f0 con 0x5584b4a061e0 2024-09-10T07:40:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.289+0000 7efcfbe00640 1 -- 172.21.15.46:0/690066545 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7efd080850d0 con 0x5584b4a061e0 2024-09-10T07:40:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.289+0000 7efd23c06740 1 -- 172.21.15.46:0/690066545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd080a45f0 con 0x5584b4a061e0 2024-09-10T07:40:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.289+0000 7efcfbe00640 1 -- 172.21.15.46:0/690066545 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efd0809b260 con 0x5584b4a061e0 2024-09-10T07:40:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.291+0000 7efd23c06740 10 client.26330 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.291+0000 7efd23c06740 1 -- 172.21.15.46:0/690066545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5584b49f8120 con 0x5584b4a061e0 2024-09-10T07:40:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.291+0000 7efcfbe00640 1 -- 172.21.15.46:0/690066545 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd080a45f0 con 0x5584b4a061e0 2024-09-10T07:40:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.291+0000 7efcfbe00640 1 -- 172.21.15.46:0/690066545 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7efd080a1350 con 0x5584b4a061e0 2024-09-10T07:40:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.292+0000 7efd23c06740 10 client.26330 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.292+0000 7efd23c06740 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.292+0000 7efd23c06740 20 client.26330 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.292+0000 7efd23c06740 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5584b49f87a0 0x5584b49fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.292+0000 7efd23c06740 4 client.26330 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:40:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.292+0000 7efd23c06740 1 -- 172.21.15.46:0/690066545 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584b4969b80 con 0x5584b49f87a0 2024-09-10T07:40:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.293+0000 7efd19e00640 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5584b49f87a0 0x5584b49fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.297+0000 7efd19e00640 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5584b49f87a0 0x5584b49fabc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d38222b2bef70b server_cookie=9c1389bb9d236ffc in_seq=0 out_seq=0 2024-09-10T07:40:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.297+0000 7efcfbe00640 10 client.26330 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.344+0000 7efcfbe00640 1 -- 172.21.15.46:0/690066545 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584b4969b80 con 0x5584b49f87a0 2024-09-10T07:40:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efcfbe00640 10 client.26330 handle_command_reply: tid=0 2024-09-10T07:40:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 client.26330 shutdown 2024-09-10T07:40:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 -- 172.21.15.46:0/690066545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 msgr2=0x5584b4a14ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a14ac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd0809bda0 tx=0x7efd080cfc80 comp rx=0 tx=0).stop 2024-09-10T07:40:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 -- 172.21.15.46:0/690066545 shutdown_connections 2024-09-10T07:40:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5584b49f87a0 0x5584b49fabc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584b4a0b270 0x5584b4a15a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584b4a061e0 0x5584b4a14ac0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 --2- 172.21.15.46:0/690066545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584b4a058a0 0x5584b4a14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.345+0000 7efd23c06740 1 -- 172.21.15.46:0/690066545 >> 172.21.15.46:0/690066545 conn(0x5584b4a00f10 msgr2=0x5584b49f7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.346+0000 7efcfbe00640 0 client.26330 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:40:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:40:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:40:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:40:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.347+0000 7efd23c06740 1 -- 172.21.15.46:0/690066545 shutdown_connections 2024-09-10T07:40:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.347+0000 7efd23c06740 1 -- 172.21.15.46:0/690066545 wait complete. 2024-09-10T07:40:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.347+0000 7efd23c06740 20 client.26330 trim_cache size 0 max 16384 2024-09-10T07:40:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 -- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efd000a2c30 msgr2=0x7efd000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efd000a2c30 0x7efd000a50f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00be10 tx=0x7efd0c00b5c0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 -- 172.21.15.46:0/1176414906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0737d0 msgr2=0x7efd1c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0737d0 0x7efd1c19e8d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e9f0 tx=0x7efd0800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 -- 172.21.15.46:0/1176414906 shutdown_connections 2024-09-10T07:40:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c109930 0x7efd1c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0737d0 0x7efd1c19e8d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7efd000a2c30 0x7efd000a50f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 --2- 172.21.15.46:0/1176414906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1c072d20 0x7efd1c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.349+0000 7efd20e00640 1 -- 172.21.15.46:0/1176414906 >> 172.21.15.46:0/1176414906 conn(0x7efd1c06ece0 msgr2=0x7efd1c10e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.350+0000 7efd20e00640 1 -- 172.21.15.46:0/1176414906 shutdown_connections 2024-09-10T07:40:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:27.350+0000 7efd20e00640 1 -- 172.21.15.46:0/1176414906 wait complete. 2024-09-10T07:40:27.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:27 smithi046 ceph-mon[30705]: pgmap v514: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.0 MiB/s wr, 729 op/s 2024-09-10T07:40:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.939+0000 7ff30ce00640 1 -- 172.21.15.46:0/2926898406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101270 msgr2=0x7ff30810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.939+0000 7ff30ce00640 1 --2- 172.21.15.46:0/2926898406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101270 0x7ff30810df30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009860 tx=0x7ff2f402a340 comp rx=0 tx=0).stop 2024-09-10T07:40:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.939+0000 7ff30ce00640 1 -- 172.21.15.46:0/2926898406 shutdown_connections 2024-09-10T07:40:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.939+0000 7ff30ce00640 1 --2- 172.21.15.46:0/2926898406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30810e470 0x7ff308110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.939+0000 7ff30ce00640 1 --2- 172.21.15.46:0/2926898406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101270 0x7ff30810df30 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009860 tx=0x7ff2f402a340 comp rx=0 tx=0).stop 2024-09-10T07:40:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.939+0000 7ff30ce00640 1 --2- 172.21.15.46:0/2926898406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3081008c0 0x7ff308100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.939+0000 7ff30ce00640 1 -- 172.21.15.46:0/2926898406 >> 172.21.15.46:0/2926898406 conn(0x7ff3080fc790 msgr2=0x7ff3080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.940+0000 7ff30ce00640 1 -- 172.21.15.46:0/2926898406 shutdown_connections 2024-09-10T07:40:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.940+0000 7ff30ce00640 1 -- 172.21.15.46:0/2926898406 wait complete. 2024-09-10T07:40:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.941+0000 7ff30ce00640 1 Processor -- start 2024-09-10T07:40:27.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.942+0000 7ff30ce00640 1 -- start start 2024-09-10T07:40:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.942+0000 7ff30ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3081008c0 0x7ff3081a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.942+0000 7ff30ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30810e470 0x7ff3081a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.942+0000 7ff30ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff30819a770 0x7ff30819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.942+0000 7ff305400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30810e470 0x7ff3081a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.943+0000 7ff30ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081141d0 con 0x7ff3081008c0 2024-09-10T07:40:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.943+0000 7ff30ce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff308114050 con 0x7ff30819a770 2024-09-10T07:40:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.943+0000 7ff30ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff308114350 con 0x7ff30810e470 2024-09-10T07:40:27.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.943+0000 7ff305400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30810e470 0x7ff3081a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:54752/0 (socket says 172.21.15.46:54752) 2024-09-10T07:40:27.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.943+0000 7ff306800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff30819a770 0x7ff30819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.943+0000 7ff305400640 1 -- 172.21.15.46:0/1081025989 learned_addr learned my addr 172.21.15.46:0/1081025989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:27.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.943+0000 7ff305e00640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3081008c0 0x7ff3081a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.945+0000 7ff306800640 1 -- 172.21.15.46:0/1081025989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30810e470 msgr2=0x7ff3081a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.945+0000 7ff306800640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30810e470 0x7ff3081a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.945+0000 7ff306800640 1 -- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3081008c0 msgr2=0x7ff3081a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.945+0000 7ff306800640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3081008c0 0x7ff3081a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.945+0000 7ff306800640 1 -- 172.21.15.46:0/1081025989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30819b320 con 0x7ff30819a770 2024-09-10T07:40:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.947+0000 7ff306800640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff30819a770 0x7ff30819abf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff308102aa0 tx=0x7ff30000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.948+0000 7ff2fea00640 1 -- 172.21.15.46:0/1081025989 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300003b70 con 0x7ff30819a770 2024-09-10T07:40:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.948+0000 7ff2fea00640 1 -- 172.21.15.46:0/1081025989 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30000ce40 con 0x7ff30819a770 2024-09-10T07:40:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.948+0000 7ff2fea00640 1 -- 172.21.15.46:0/1081025989 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000106d0 con 0x7ff30819a770 2024-09-10T07:40:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.948+0000 7ff30ce00640 1 -- 172.21.15.46:0/1081025989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3081a7450 con 0x7ff30819a770 2024-09-10T07:40:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.948+0000 7ff30ce00640 1 -- 172.21.15.46:0/1081025989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3081a77f0 con 0x7ff30819a770 2024-09-10T07:40:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.955+0000 7ff30f9ca740 1 Processor -- start 2024-09-10T07:40:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.956+0000 7ff2fea00640 1 -- 172.21.15.46:0/1081025989 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff300010870 con 0x7ff30819a770 2024-09-10T07:40:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.959+0000 7ff2fea00640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2ec0a2c40 0x7ff2ec0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.960+0000 7ff2fea00640 1 -- 172.21.15.46:0/1081025989 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff3000cf7e0 con 0x7ff30819a770 2024-09-10T07:40:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.960+0000 7ff30f9ca740 1 -- start start 2024-09-10T07:40:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.962+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x5648887f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.962+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888805f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.962+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x5648888088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.962+0000 7ff305400640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2ec0a2c40 0x7ff2ec0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.962+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564888759290 con 0x5648887f29c0 2024-09-10T07:40:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.962+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564888700b40 con 0x5648887f32e0 2024-09-10T07:40:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.962+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564888755250 con 0x5648888064b0 2024-09-10T07:40:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.963+0000 7ff305e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x5648887f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.963+0000 7ff306800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x5648888088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.963+0000 7ff306800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x5648888088a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:54760/0 (socket says 172.21.15.46:54760) 2024-09-10T07:40:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.963+0000 7ff306800640 1 -- 172.21.15.46:0/798339221 learned_addr learned my addr 172.21.15.46:0/798339221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.963+0000 7ff305e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x5648887f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44226/0 (socket says 172.21.15.46:44226) 2024-09-10T07:40:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.963+0000 7ff305400640 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888805f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.964+0000 7ff305400640 1 -- 172.21.15.46:0/798339221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 msgr2=0x5648888088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.965+0000 7ff305400640 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x5648888088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.965+0000 7ff305400640 1 -- 172.21.15.46:0/798339221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 msgr2=0x5648887f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.965+0000 7ff305400640 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x5648887f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.965+0000 7ff305400640 1 -- 172.21.15.46:0/798339221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648884ca6a0 con 0x5648887f32e0 2024-09-10T07:40:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.965+0000 7ff305e00640 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x5648887f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.965+0000 7ff306800640 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x5648888088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.966+0000 7ff305400640 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888805f70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009830 tx=0x7ff2f4004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3219e3f73b2442bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.966+0000 7ff2f3400640 1 -- 172.21.15.46:0/798339221 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4037040 con 0x5648887f32e0 2024-09-10T07:40:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.966+0000 7ff305400640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2ec0a2c40 0x7ff2ec0a5100 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4035040 tx=0x7ff2f4039000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.966+0000 7ff2f3400640 1 -- 172.21.15.46:0/798339221 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4003ad0 con 0x5648887f32e0 2024-09-10T07:40:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.966+0000 7ff2f3400640 1 -- 172.21.15.46:0/798339221 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4037040 con 0x5648887f32e0 2024-09-10T07:40:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.968+0000 7ff30f9ca740 1 -- 172.21.15.46:0/798339221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 msgr2=0x564888805f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.969+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888805f70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009830 tx=0x7ff2f4004300 comp rx=0 tx=0).stop 2024-09-10T07:40:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.970+0000 7ff30f9ca740 1 -- 172.21.15.46:0/798339221 shutdown_connections 2024-09-10T07:40:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.970+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x5648888088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.970+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888805f70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.970+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/798339221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x5648887f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.970+0000 7ff30f9ca740 1 -- 172.21.15.46:0/798339221 >> 172.21.15.46:0/798339221 conn(0x5648887ef050 msgr2=0x5648887ef430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.970+0000 7ff30f9ca740 1 -- 172.21.15.46:0/798339221 shutdown_connections 2024-09-10T07:40:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.970+0000 7ff30f9ca740 1 -- 172.21.15.46:0/798339221 wait complete. 2024-09-10T07:40:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.971+0000 7ff30f9ca740 1 -- start start 2024-09-10T07:40:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.972+0000 7ff30f9ca740 1 Processor -- start 2024-09-10T07:40:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f29c0 0x564888805ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888803180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648888064b0 0x564888803790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564888809120 con 0x5648888064b0 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564888713fa0 con 0x5648887f32e0 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648888092a0 con 0x5648887f29c0 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff30f9ca740 10 client.0 _ll_register_callbacks cb 0x564888467ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff305400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888803180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff305400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888803180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47018/0 (socket says 172.21.15.46:47018) 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff305400640 1 -- 172.21.15.46:0/3492059772 learned_addr learned my addr 172.21.15.46:0/3492059772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.973+0000 7ff305e00640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f29c0 0x564888805ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.974+0000 7ff305400640 1 -- 172.21.15.46:0/3492059772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f29c0 msgr2=0x564888805ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.974+0000 7ff305400640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f29c0 0x564888805ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.974+0000 7ff305400640 1 -- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648888064b0 msgr2=0x564888803790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.974+0000 7ff306800640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648888064b0 0x564888803790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.974+0000 7ff305400640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648888064b0 0x564888803790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.974+0000 7ff305400640 1 -- 172.21.15.46:0/3492059772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564888803fc0 con 0x5648887f32e0 2024-09-10T07:40:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.975+0000 7ff305e00640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f29c0 0x564888805ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.975+0000 7ff306800640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648888064b0 0x564888803790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.975+0000 7ff305400640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888803180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009250 tx=0x7ff2f40039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1270fd55267e92f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.975+0000 7ff2e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.976+0000 7ff2e7e00640 1 -- 172.21.15.46:0/3492059772 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f404f070 con 0x5648887f32e0 2024-09-10T07:40:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.976+0000 7ff2e7e00640 1 -- 172.21.15.46:0/3492059772 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff2f4032530 con 0x5648887f32e0 2024-09-10T07:40:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.976+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3492059772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f4003ad0 con 0x5648887f32e0 2024-09-10T07:40:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.977+0000 7ff2e7e00640 1 -- 172.21.15.46:0/3492059772 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff2f4037050 con 0x5648887f32e0 2024-09-10T07:40:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.977+0000 7ff30f9ca740 10 client.26342 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.977+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3492059772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564888809120 con 0x5648887f32e0 2024-09-10T07:40:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.978+0000 7ff2e7e00640 1 -- 172.21.15.46:0/3492059772 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4003ad0 con 0x5648887f32e0 2024-09-10T07:40:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.978+0000 7ff2e7e00640 1 -- 172.21.15.46:0/3492059772 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff2f404a070 con 0x5648887f32e0 2024-09-10T07:40:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.979+0000 7ff30f9ca740 10 client.26342 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.979+0000 7ff30f9ca740 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.979+0000 7ff30f9ca740 20 client.26342 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.979+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5648888162c0 0x5648888186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.979+0000 7ff30f9ca740 4 client.26342 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.979+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3492059772 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564888766800 con 0x5648888162c0 2024-09-10T07:40:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.981+0000 7ff305e00640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5648888162c0 0x5648888186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.983+0000 7ff2e7e00640 10 client.26342 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.983+0000 7ff305e00640 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5648888162c0 0x5648888186b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f8219b20ecbd55 server_cookie=6aef8b527d8b3348 in_seq=0 out_seq=0 2024-09-10T07:40:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.985+0000 7ff2e7e00640 1 -- 172.21.15.46:0/3492059772 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564888766800 con 0x5648888162c0 2024-09-10T07:40:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.985+0000 7ff2e7e00640 10 client.26342 handle_command_reply: tid=0 2024-09-10T07:40:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.986+0000 7ff30f9ca740 1 client.26342 shutdown 2024-09-10T07:40:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.986+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3492059772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 msgr2=0x564888803180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.986+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888803180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009250 tx=0x7ff2f40039e0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3492059772 shutdown_connections 2024-09-10T07:40:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5648888162c0 0x5648888186b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648888064b0 0x564888803790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f32e0 0x564888803180 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3492059772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f29c0 0x564888805ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff2e7e00640 0 client.26342 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3492059772 >> 172.21.15.46:0/3492059772 conn(0x5648887ef050 msgr2=0x5648887f4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3492059772 shutdown_connections 2024-09-10T07:40:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.987+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3492059772 wait complete. 2024-09-10T07:40:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:27.988+0000 7ff30f9ca740 20 client.26342 trim_cache size 0 max 16384 2024-09-10T07:40:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.022+0000 7ff30f9ca740 1 Processor -- start 2024-09-10T07:40:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.022+0000 7ff30f9ca740 1 -- start start 2024-09-10T07:40:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f29c0 0x5648888050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f32e0 0x56488881ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x56488881e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff305e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f29c0 0x5648888050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ad1a0 con 0x5648887f32e0 2024-09-10T07:40:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081aed70 con 0x5648887f29c0 2024-09-10T07:40:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff306800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x56488881e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff306800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x56488881e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:54770/0 (socket says 172.21.15.46:54770) 2024-09-10T07:40:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff305400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f32e0 0x56488881ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff306800640 1 -- 172.21.15.46:0/3749871328 learned_addr learned my addr 172.21.15.46:0/3749871328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3749871328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ad870 con 0x5648888064b0 2024-09-10T07:40:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.023+0000 7ff305e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f29c0 0x5648888050f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47032/0 (socket says 172.21.15.46:47032) 2024-09-10T07:40:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff305400640 1 -- 172.21.15.46:0/3749871328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 msgr2=0x56488881e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff305400640 1 --2- 172.21.15.46:0/3749871328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x56488881e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff305400640 1 -- 172.21.15.46:0/3749871328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f29c0 msgr2=0x5648888050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff305400640 1 --2- 172.21.15.46:0/3749871328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f29c0 0x5648888050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff305400640 1 -- 172.21.15.46:0/3749871328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648888088a0 con 0x5648887f32e0 2024-09-10T07:40:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.024+0000 7ff306800640 1 --2- 172.21.15.46:0/3749871328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x56488881e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.025+0000 7ff305400640 1 --2- 172.21.15.46:0/3749871328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f32e0 0x56488881ddf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x564888804730 tx=0x7ff2f4030460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6ae9bc8285608a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.026+0000 7ff2f2a00640 1 -- 172.21.15.46:0/3749871328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f404f040 con 0x5648887f32e0 2024-09-10T07:40:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.026+0000 7ff2f2a00640 1 -- 172.21.15.46:0/3749871328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f403e870 con 0x5648887f32e0 2024-09-10T07:40:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.026+0000 7ff2f2a00640 1 -- 172.21.15.46:0/3749871328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4032530 con 0x5648887f32e0 2024-09-10T07:40:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.028+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3749871328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f32e0 msgr2=0x56488881ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.028+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3749871328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f32e0 0x56488881ddf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x564888804730 tx=0x7ff2f4030460 comp rx=0 tx=0).stop 2024-09-10T07:40:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.029+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3749871328 shutdown_connections 2024-09-10T07:40:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.029+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3749871328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648888064b0 0x56488881e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.029+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3749871328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f32e0 0x56488881ddf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.029+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/3749871328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648887f29c0 0x5648888050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.029+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3749871328 >> 172.21.15.46:0/3749871328 conn(0x5648887ef050 msgr2=0x5648888049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.030+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3749871328 shutdown_connections 2024-09-10T07:40:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.030+0000 7ff30f9ca740 1 -- 172.21.15.46:0/3749871328 wait complete. 2024-09-10T07:40:28.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.031+0000 7ff30f9ca740 1 -- start start 2024-09-10T07:40:28.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.031+0000 7ff30f9ca740 1 Processor -- start 2024-09-10T07:40:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.032+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x56488880bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.033+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f32e0 0x56488880c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.033+0000 7ff30f9ca740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648888064b0 0x5648887fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.033+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ae670 con 0x5648887f29c0 2024-09-10T07:40:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.033+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081adfa0 con 0x5648888064b0 2024-09-10T07:40:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.033+0000 7ff30f9ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ab180 con 0x5648887f32e0 2024-09-10T07:40:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.033+0000 7ff30f9ca740 10 client.0 _ll_register_callbacks cb 0x7ff2f80097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff305e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x56488880bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff305400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f32e0 0x56488880c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff306800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648888064b0 0x5648887fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff306800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648888064b0 0x5648887fe7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47034/0 (socket says 172.21.15.46:47034) 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff305400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f32e0 0x56488880c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:54778/0 (socket says 172.21.15.46:54778) 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff306800640 1 -- 172.21.15.46:0/4162928288 learned_addr learned my addr 172.21.15.46:0/4162928288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff305e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x56488880bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44262/0 (socket says 172.21.15.46:44262) 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff306800640 1 -- 172.21.15.46:0/4162928288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f32e0 msgr2=0x56488880c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff306800640 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f32e0 0x56488880c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.034+0000 7ff306800640 1 -- 172.21.15.46:0/4162928288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 msgr2=0x56488880bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.035+0000 7ff306800640 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x56488880bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.035+0000 7ff306800640 1 -- 172.21.15.46:0/4162928288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648887f7720 con 0x5648888064b0 2024-09-10T07:40:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.035+0000 7ff305e00640 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x56488880bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.036+0000 7ff306800640 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648888064b0 0x5648887fe7f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff300098120 tx=0x7ff30009a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c927a7e9706729d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.036+0000 7ff2e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.036+0000 7ff2e7e00640 1 -- 172.21.15.46:0/4162928288 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30009bcc0 con 0x5648888064b0 2024-09-10T07:40:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.036+0000 7ff2e7e00640 1 -- 172.21.15.46:0/4162928288 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff3000cd070 con 0x5648888064b0 2024-09-10T07:40:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.036+0000 7ff30f9ca740 1 -- 172.21.15.46:0/4162928288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f403e870 con 0x5648888064b0 2024-09-10T07:40:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.036+0000 7ff2e7e00640 1 -- 172.21.15.46:0/4162928288 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff3000a0070 con 0x5648888064b0 2024-09-10T07:40:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff30f9ca740 10 client.26348 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff30f9ca740 1 -- 172.21.15.46:0/4162928288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564888889eb0 con 0x5648888064b0 2024-09-10T07:40:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff2e7e00640 1 -- 172.21.15.46:0/4162928288 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f403e870 con 0x5648888064b0 2024-09-10T07:40:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff2e7e00640 1 -- 172.21.15.46:0/4162928288 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7ff300010b60 con 0x5648888064b0 2024-09-10T07:40:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff30f9ca740 10 client.26348 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff30f9ca740 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff30f9ca740 20 client.26348 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56488888a630 0x56488888ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff30f9ca740 4 client.26348 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:40:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.037+0000 7ff30f9ca740 1 -- 172.21.15.46:0/4162928288 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564888760d10 con 0x56488888a630 2024-09-10T07:40:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.038+0000 7ff305e00640 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56488888a630 0x56488888ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.040+0000 7ff305e00640 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56488888a630 0x56488888ca50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=292a2b1a7a3fb2af server_cookie=213a526cf309fe3e in_seq=0 out_seq=0 2024-09-10T07:40:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.040+0000 7ff2e7e00640 10 client.26348 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.041+0000 7ff2e7e00640 1 -- 172.21.15.46:0/4162928288 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564888760d10 con 0x56488888a630 2024-09-10T07:40:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff2e7e00640 10 client.26348 handle_command_reply: tid=0 2024-09-10T07:40:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 client.26348 shutdown 2024-09-10T07:40:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 -- 172.21.15.46:0/4162928288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648888064b0 msgr2=0x5648887fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648888064b0 0x5648887fe7f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff300098120 tx=0x7ff30009a370 comp rx=0 tx=0).stop 2024-09-10T07:40:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 -- 172.21.15.46:0/4162928288 shutdown_connections 2024-09-10T07:40:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56488888a630 0x56488888ca50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648888064b0 0x5648887fe7f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648887f32e0 0x56488880c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 --2- 172.21.15.46:0/4162928288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648887f29c0 0x56488880bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 -- 172.21.15.46:0/4162928288 >> 172.21.15.46:0/4162928288 conn(0x5648887ef050 msgr2=0x5648887f4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff2e7e00640 0 client.26348 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 -- 172.21.15.46:0/4162928288 shutdown_connections 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.042+0000 7ff30f9ca740 1 -- 172.21.15.46:0/4162928288 wait complete. 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.043+0000 7ff30f9ca740 20 client.26348 trim_cache size 0 max 16384 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "7b82bb22-f98e-4ddc-bf86-585bdcabd035", 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:40:28.043 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:40:28.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.044+0000 7ff30ce00640 1 -- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2ec0a2c40 msgr2=0x7ff2ec0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:28.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.044+0000 7ff30ce00640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2ec0a2c40 0x7ff2ec0a5100 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4035040 tx=0x7ff2f4039000 comp rx=0 tx=0).stop 2024-09-10T07:40:28.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.044+0000 7ff30ce00640 1 -- 172.21.15.46:0/1081025989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff30819a770 msgr2=0x7ff30819abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:28.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.044+0000 7ff30ce00640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff30819a770 0x7ff30819abf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff308102aa0 tx=0x7ff30000c6a0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.045+0000 7ff30ce00640 1 -- 172.21.15.46:0/1081025989 shutdown_connections 2024-09-10T07:40:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.045+0000 7ff30ce00640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff30819a770 0x7ff30819abf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.045+0000 7ff30ce00640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30810e470 0x7ff3081a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.045+0000 7ff30ce00640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2ec0a2c40 0x7ff2ec0a5100 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.045+0000 7ff30ce00640 1 --2- 172.21.15.46:0/1081025989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3081008c0 0x7ff3081a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.045+0000 7ff30ce00640 1 -- 172.21.15.46:0/1081025989 >> 172.21.15.46:0/1081025989 conn(0x7ff3080fc790 msgr2=0x7ff3080feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.046+0000 7ff30ce00640 1 -- 172.21.15.46:0/1081025989 shutdown_connections 2024-09-10T07:40:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:28.047+0000 7ff30ce00640 1 -- 172.21.15.46:0/1081025989 wait complete. 2024-09-10T07:40:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:27 smithi181 ceph-mon[35397]: pgmap v514: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.0 MiB/s wr, 729 op/s 2024-09-10T07:40:28.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:27 smithi063 ceph-mon[35020]: pgmap v514: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.0 MiB/s wr, 729 op/s 2024-09-10T07:40:28.159 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: 2024-09-10T07:40:28.088+0000 7f775d800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004a7 [...2,head] /volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/base/13494/ auth{1=1,2=1,3=3,4=1} fragtree_t(*^3) v4376 ap=2 f(v1 m2024-09-10T07:39:34.768666+0000 310=310+0) n(v14 rc2024-09-10T07:40:26.255113+0000 b531673647 311=310+1) (inest sync->mix g=1,2,3,4) caps={15174=pAsLsXsFs/pAsLsXsFsx@458} mcw={1=AsLsXsFs,2=AsLsXsFs,3=AsLsXsFs,4=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55c133f69b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004a7:[<0x10000000014/13494 v1858>,<0x10000000006/base v348>,<0x10000000005/data v78>,<0x100000001fb/postgres v13>,<0x100000001fa/a8e9ad88-14b3-4732-92a7-3dc20544565e v31>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v36>,<0x1/volumes v36>]//[]","memoryvalue":"(2)0x100000004a7:[<0x10000000014/13494 v4376>,<0x10000000006/base v1293>,<0x10000000005/data v318>,<0x100000001fb/postgres v165>,<0x100000001fa/a8e9ad88-14b3-4732-92a7-3dc20544565e v179>,<0x10000000001/sv_1 v206>,<0x10000000000/qa v189>,<0x1/volumes v179>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:37:20.954092+0000 309=309+0)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:40:26.255113+0000 b531673647 311=310+1)","memory_value.dirstat":"f(v1 m2024-09-10T07:39:34.768666+0000 310=310+0)","memory_value.rstat":"n(v14 rc2024-09-10T07:40:26.255113+0000 b531673647 311=310+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T07:40:28.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:40:28.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:28.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:40:28.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/base/13494) see mds.a log and `damage ls` output for details 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:28.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:28 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.051 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:40:29.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:40:29.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:40:29.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:40:29.052 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:40:29.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:40:29.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:40:29.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:40:29.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.053 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/base/13494) see mds.a log and `damage ls` output for details 2024-09-10T07:40:29.054 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.054 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:28 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:40:29.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:40:29.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:40:29.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:40:29.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:40:29.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:40:29.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:40:29.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:40:29.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/base/13494) see mds.a log and `damage ls` output for details 2024-09-10T07:40:29.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:28 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.088+0000 7f6355600640 1 -- 172.21.15.46:0/22018776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63501139e0 msgr2=0x7f6350115e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.088+0000 7f6355600640 1 --2- 172.21.15.46:0/22018776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63501139e0 0x7f6350115e70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6348009ca0 tx=0x7f634802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.088+0000 7f6355600640 1 -- 172.21.15.46:0/22018776 shutdown_connections 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.088+0000 7f6355600640 1 --2- 172.21.15.46:0/22018776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63501139e0 0x7f6350115e70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.088+0000 7f6355600640 1 --2- 172.21.15.46:0/22018776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f635010ae80 0x7f6350113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.088+0000 7f6355600640 1 --2- 172.21.15.46:0/22018776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f635010a560 0x7f635010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.089+0000 7f6355600640 1 -- 172.21.15.46:0/22018776 >> 172.21.15.46:0/22018776 conn(0x7f635006e2d0 msgr2=0x7f635006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.089+0000 7f6355600640 1 -- 172.21.15.46:0/22018776 shutdown_connections 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.089+0000 7f6355600640 1 -- 172.21.15.46:0/22018776 wait complete. 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.091+0000 7f6355600640 1 Processor -- start 2024-09-10T07:40:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.091+0000 7f6355600640 1 -- start start 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.092+0000 7f6355600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f635010a560 0x7f63501a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.092+0000 7f6355600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f635010ae80 0x7f63501a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.092+0000 7f6355600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63501139e0 0x7f63501a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.092+0000 7f6355600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f635010a150 con 0x7f635010ae80 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.092+0000 7f6355600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6350109fd0 con 0x7f635010a560 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.092+0000 7f6355600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f635011aa10 con 0x7f63501139e0 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.093+0000 7f634ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f635010a560 0x7f63501a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.093+0000 7f634f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63501139e0 0x7f63501a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.093+0000 7f634ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f635010a560 0x7f63501a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47052/0 (socket says 172.21.15.46:47052) 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.093+0000 7f634ea00640 1 -- 172.21.15.46:0/1622092665 learned_addr learned my addr 172.21.15.46:0/1622092665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.093+0000 7f634e000640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f635010ae80 0x7f63501a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.094+0000 7f634ea00640 1 -- 172.21.15.46:0/1622092665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63501139e0 msgr2=0x7f63501a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.094+0000 7f634ea00640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63501139e0 0x7f63501a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.094+0000 7f634ea00640 1 -- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f635010ae80 msgr2=0x7f63501a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.094+0000 7f634ea00640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f635010ae80 0x7f63501a6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.094+0000 7f634ea00640 1 -- 172.21.15.46:0/1622092665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63501a1220 con 0x7f635010a560 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.094+0000 7f634e000640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f635010ae80 0x7f63501a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.095+0000 7f634ea00640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f635010a560 0x7f63501a6930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f634000e9f0 tx=0x7f634000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.096+0000 7f6347400640 1 -- 172.21.15.46:0/1622092665 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634000cd80 con 0x7f635010a560 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.096+0000 7f6347400640 1 -- 172.21.15.46:0/1622092665 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6340005500 con 0x7f635010a560 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.096+0000 7f6347400640 1 -- 172.21.15.46:0/1622092665 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340010690 con 0x7f635010a560 2024-09-10T07:40:29.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.096+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63501a1510 con 0x7f635010a560 2024-09-10T07:40:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.096+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6350118800 con 0x7f635010a560 2024-09-10T07:40:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.098+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635010a940 con 0x7f635010a560 2024-09-10T07:40:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.101+0000 7f6347400640 1 -- 172.21.15.46:0/1622092665 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f63400046c0 con 0x7f635010a560 2024-09-10T07:40:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.103+0000 7f6347400640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63340a2c40 0x7f63340a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.103+0000 7f6347400640 1 -- 172.21.15.46:0/1622092665 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f63400cf890 con 0x7f635010a560 2024-09-10T07:40:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.103+0000 7f634e000640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63340a2c40 0x7f63340a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.106+0000 7f634e000640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63340a2c40 0x7f63340a5100 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6350073280 tx=0x7f633c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:29.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.108+0000 7f6347400640 1 -- 172.21.15.46:0/1622092665 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f634009c470 con 0x7f635010a560 2024-09-10T07:40:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.487+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63501a1fe0 con 0x7f635010a560 2024-09-10T07:40:29.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.488+0000 7f6347400640 1 -- 172.21.15.46:0/1622092665 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11976 (secure 0 0 0) 0x7f634009bd70 con 0x7f635010a560 2024-09-10T07:40:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:40:29.489 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"btime":"2024-09-10T07:40:06:106302+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:40:05.500206+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:40:29.489 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 38 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.491+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63340a2c40 msgr2=0x7f63340a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.491+0000 7f6355600640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63340a2c40 0x7f63340a5100 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6350073280 tx=0x7f633c008040 comp rx=0 tx=0).stop 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.491+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f635010a560 msgr2=0x7f63501a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.491+0000 7f6355600640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f635010a560 0x7f63501a6930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f634000e9f0 tx=0x7f634000edf0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.492+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 shutdown_connections 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.492+0000 7f6355600640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63501139e0 0x7f63501a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.492+0000 7f6355600640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63340a2c40 0x7f63340a5100 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.492+0000 7f6355600640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f635010ae80 0x7f63501a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.492+0000 7f6355600640 1 --2- 172.21.15.46:0/1622092665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f635010a560 0x7f63501a6930 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f634000e9f0 tx=0x7f634000edf0 comp rx=0 tx=0).stop 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.492+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 >> 172.21.15.46:0/1622092665 conn(0x7f635006e2d0 msgr2=0x7f63500708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.492+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 shutdown_connections 2024-09-10T07:40:29.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:29.492+0000 7f6355600640 1 -- 172.21.15.46:0/1622092665 wait complete. 2024-09-10T07:40:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:29 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:29 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:29 smithi181 ceph-mon[35397]: pgmap v515: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.8 MiB/s wr, 575 op/s 2024-09-10T07:40:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:29 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1622092665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:29 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:29 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:29 smithi063 ceph-mon[35020]: pgmap v515: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.8 MiB/s wr, 575 op/s 2024-09-10T07:40:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:29 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1622092665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:30.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:30.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:40:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:29 smithi046 ceph-mon[30705]: pgmap v515: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.8 MiB/s wr, 575 op/s 2024-09-10T07:40:30.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:29 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1622092665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 190.0 s, 1135.5 tps, lat 87.197 ms stddev 137.023 2024-09-10T07:40:31.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:31 smithi046 ceph-mon[30705]: pgmap v516: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 928 op/s 2024-09-10T07:40:32.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:40:32.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:31 smithi181 ceph-mon[35397]: pgmap v516: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 928 op/s 2024-09-10T07:40:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:31 smithi063 ceph-mon[35020]: pgmap v516: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 928 op/s 2024-09-10T07:40:32.860 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T07:40:32.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T07:40:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 -- 172.21.15.46:0/3952004264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c112880 msgr2=0x7f245c114c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 --2- 172.21.15.46:0/3952004264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c112880 0x7f245c114c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2454009ca0 tx=0x7f245402a950 comp rx=0 tx=0).stop 2024-09-10T07:40:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 -- 172.21.15.46:0/3952004264 shutdown_connections 2024-09-10T07:40:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 --2- 172.21.15.46:0/3952004264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c112880 0x7f245c114c70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 --2- 172.21.15.46:0/3952004264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f245c100510 0x7f245c112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 --2- 172.21.15.46:0/3952004264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f245c0ffbf0 0x7f245c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 -- 172.21.15.46:0/3952004264 >> 172.21.15.46:0/3952004264 conn(0x7f245c0fba90 msgr2=0x7f245c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 -- 172.21.15.46:0/3952004264 shutdown_connections 2024-09-10T07:40:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.723+0000 7f2463a00640 1 -- 172.21.15.46:0/3952004264 wait complete. 2024-09-10T07:40:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.724+0000 7f2463a00640 1 Processor -- start 2024-09-10T07:40:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.725+0000 7f2463a00640 1 -- start start 2024-09-10T07:40:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.725+0000 7f2463a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c0ffbf0 0x7f245c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.725+0000 7f2463a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f245c100510 0x7f245c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.725+0000 7f2463a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f245c112880 0x7f245c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.725+0000 7f2463a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f245c118540 con 0x7f245c100510 2024-09-10T07:40:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.725+0000 7f2463a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f245c1183c0 con 0x7f245c112880 2024-09-10T07:40:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.725+0000 7f2463a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f245c1186c0 con 0x7f245c0ffbf0 2024-09-10T07:40:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.725+0000 7f2461000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c0ffbf0 0x7f245c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f2461000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c0ffbf0 0x7f245c1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:54804/0 (socket says 172.21.15.46:54804) 2024-09-10T07:40:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f2461000640 1 -- 172.21.15.46:0/2669370339 learned_addr learned my addr 172.21.15.46:0/2669370339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f2461a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f245c112880 0x7f245c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f2461a00640 1 -- 172.21.15.46:0/2669370339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c0ffbf0 msgr2=0x7f245c1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f2461a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c0ffbf0 0x7f245c1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f2461a00640 1 -- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f245c100510 msgr2=0x7f245c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f245be00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f245c100510 0x7f245c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f2461a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f245c100510 0x7f245c1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.726+0000 7f2461a00640 1 -- 172.21.15.46:0/2669370339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c19efe0 con 0x7f245c112880 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.727+0000 7f245be00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f245c100510 0x7f245c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.727+0000 7f2461a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f245c112880 0x7f245c19e840 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f245402af40 tx=0x7f2454003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.727+0000 7f2459600640 1 -- 172.21.15.46:0/2669370339 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454034040 con 0x7f245c112880 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.727+0000 7f2463a00640 1 -- 172.21.15.46:0/2669370339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c19f270 con 0x7f245c112880 2024-09-10T07:40:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.727+0000 7f2463a00640 1 -- 172.21.15.46:0/2669370339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c1ab520 con 0x7f245c112880 2024-09-10T07:40:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.728+0000 7f2459600640 1 -- 172.21.15.46:0/2669370339 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24540098d0 con 0x7f245c112880 2024-09-10T07:40:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.728+0000 7f2459600640 1 -- 172.21.15.46:0/2669370339 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540047e0 con 0x7f245c112880 2024-09-10T07:40:33.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.733+0000 7f24664a2740 1 Processor -- start 2024-09-10T07:40:33.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.735+0000 7f2459600640 1 -- 172.21.15.46:0/2669370339 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2454037020 con 0x7f245c112880 2024-09-10T07:40:33.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.736+0000 7f24664a2740 1 -- start start 2024-09-10T07:40:33.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.736+0000 7f24664a2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69db90 0x55758a69df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.736+0000 7f24664a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69e4b0 0x55758a6a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.737+0000 7f24664a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.737+0000 7f24664a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55758a6043e0 con 0x55758a69e4b0 2024-09-10T07:40:33.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.737+0000 7f245be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69db90 0x55758a69df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.737+0000 7f24664a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55758a5abd80 con 0x55758a6a8d70 2024-09-10T07:40:33.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.737+0000 7f245be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69db90 0x55758a69df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:54816/0 (socket says 172.21.15.46:54816) 2024-09-10T07:40:33.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.737+0000 7f245be00640 1 -- 172.21.15.46:0/936503810 learned_addr learned my addr 172.21.15.46:0/936503810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:33.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.737+0000 7f24664a2740 1 -- 172.21.15.46:0/936503810 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55758a6003a0 con 0x55758a69db90 2024-09-10T07:40:33.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.739+0000 7f2459600640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f24440a2c40 0x7f24440a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.739+0000 7f2461000640 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69e4b0 0x55758a6a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.740+0000 7f245be00640 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.740+0000 7f245be00640 1 -- 172.21.15.46:0/936503810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 msgr2=0x55758a6ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:33.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.740+0000 7f245be00640 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.740+0000 7f245be00640 1 -- 172.21.15.46:0/936503810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69e4b0 msgr2=0x55758a6a8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:33.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.740+0000 7f245be00640 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69e4b0 0x55758a6a8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.740+0000 7f245be00640 1 -- 172.21.15.46:0/936503810 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758a5caae0 con 0x55758a69db90 2024-09-10T07:40:33.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.740+0000 7f2459600640 1 -- 172.21.15.46:0/2669370339 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f24540f0100 con 0x7f245c112880 2024-09-10T07:40:33.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.741+0000 7f2461a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f24440a2c40 0x7f24440a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.742+0000 7f245be00640 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69db90 0x55758a69df70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f245000bfd0 tx=0x7f245000b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1af02946ae8ba0cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:33.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.742+0000 7f2461a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f24440a2c40 0x7f24440a5100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f24540043b0 tx=0x7f2454003f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:33.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.742+0000 7f244a000640 1 -- 172.21.15.46:0/936503810 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450005020 con 0x55758a69db90 2024-09-10T07:40:33.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.742+0000 7f244a000640 1 -- 172.21.15.46:0/936503810 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24500047a0 con 0x55758a69db90 2024-09-10T07:40:33.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.742+0000 7f244a000640 1 -- 172.21.15.46:0/936503810 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245000f7d0 con 0x55758a69db90 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.743+0000 7f24664a2740 1 -- 172.21.15.46:0/936503810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69db90 msgr2=0x55758a69df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.743+0000 7f24664a2740 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69db90 0x55758a69df70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f245000bfd0 tx=0x7f245000b7d0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.744+0000 7f24664a2740 1 -- 172.21.15.46:0/936503810 shutdown_connections 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.744+0000 7f24664a2740 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.744+0000 7f24664a2740 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69e4b0 0x55758a6a8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.744+0000 7f24664a2740 1 --2- 172.21.15.46:0/936503810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69db90 0x55758a69df70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.744+0000 7f24664a2740 1 -- 172.21.15.46:0/936503810 >> 172.21.15.46:0/936503810 conn(0x55758a69a120 msgr2=0x55758a69a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.744+0000 7f24664a2740 1 -- 172.21.15.46:0/936503810 shutdown_connections 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.744+0000 7f24664a2740 1 -- 172.21.15.46:0/936503810 wait complete. 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.745+0000 7f24664a2740 1 -- start start 2024-09-10T07:40:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.745+0000 7f24664a2740 1 Processor -- start 2024-09-10T07:40:33.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f24664a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69db90 0x55758a6a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f24664a2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69e4b0 0x55758a6a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f24664a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f24664a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55758a6ab9e0 con 0x55758a69db90 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f24664a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55758a5bf0f0 con 0x55758a6a8d70 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f24664a2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55758a6abb60 con 0x55758a69e4b0 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f24664a2740 10 client.0 _ll_register_callbacks cb 0x55758a5b09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f2461000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69e4b0 0x55758a6a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f2461000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69e4b0 0x55758a6a61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:54826/0 (socket says 172.21.15.46:54826) 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.746+0000 7f2461000640 1 -- 172.21.15.46:0/1675611996 learned_addr learned my addr 172.21.15.46:0/1675611996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.747+0000 7f245be00640 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69db90 0x55758a6a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.747+0000 7f2461000640 1 -- 172.21.15.46:0/1675611996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 msgr2=0x55758a6a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.747+0000 7f245be00640 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.747+0000 7f2461000640 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.748+0000 7f2461000640 1 -- 172.21.15.46:0/1675611996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69db90 msgr2=0x55758a6a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.748+0000 7f2461000640 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69db90 0x55758a6a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.748+0000 7f2461000640 1 -- 172.21.15.46:0/1675611996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758a3753c0 con 0x55758a69e4b0 2024-09-10T07:40:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.748+0000 7f245be00640 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.748+0000 7f2461000640 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69e4b0 0x55758a6a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f244c012890 tx=0x7f244c012d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db21d93e29f738d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.748+0000 7f243ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.749+0000 7f243ea00640 1 -- 172.21.15.46:0/1675611996 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c00f920 con 0x55758a69e4b0 2024-09-10T07:40:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.749+0000 7f243ea00640 1 -- 172.21.15.46:0/1675611996 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f244c00fac0 con 0x55758a69e4b0 2024-09-10T07:40:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.749+0000 7f24664a2740 1 -- 172.21.15.46:0/1675611996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24500047a0 con 0x55758a69e4b0 2024-09-10T07:40:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.750+0000 7f243ea00640 1 -- 172.21.15.46:0/1675611996 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f244c00fd00 con 0x55758a69e4b0 2024-09-10T07:40:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.750+0000 7f24664a2740 10 client.26356 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.750+0000 7f24664a2740 1 -- 172.21.15.46:0/1675611996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55758a6b77a0 con 0x55758a69e4b0 2024-09-10T07:40:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.751+0000 7f243ea00640 1 -- 172.21.15.46:0/1675611996 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24500047a0 con 0x55758a69e4b0 2024-09-10T07:40:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.751+0000 7f243ea00640 1 -- 172.21.15.46:0/1675611996 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f244c0104c0 con 0x55758a69e4b0 2024-09-10T07:40:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.751+0000 7f24664a2740 10 client.26356 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.751+0000 7f24664a2740 10 client.26356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:33.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.751+0000 7f24664a2740 20 client.26356 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:33.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.751+0000 7f24664a2740 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55758a6b8c20 0x55758a6bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.751+0000 7f24664a2740 4 client.26356 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:33.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.751+0000 7f24664a2740 1 -- 172.21.15.46:0/1675611996 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55758a611950 con 0x55758a6b8c20 2024-09-10T07:40:33.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.754+0000 7f245be00640 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55758a6b8c20 0x55758a6bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.755+0000 7f245be00640 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55758a6b8c20 0x55758a6bb080 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c64e4bd963e002 server_cookie=2f335ce2ed61b655 in_seq=0 out_seq=0 2024-09-10T07:40:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.755+0000 7f243ea00640 10 client.26356 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.756+0000 7f243ea00640 1 -- 172.21.15.46:0/1675611996 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55758a611950 con 0x55758a6b8c20 2024-09-10T07:40:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.756+0000 7f243ea00640 10 client.26356 handle_command_reply: tid=0 2024-09-10T07:40:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.756+0000 7f24664a2740 1 client.26356 shutdown 2024-09-10T07:40:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 -- 172.21.15.46:0/1675611996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69e4b0 msgr2=0x55758a6a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69e4b0 0x55758a6a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f244c012890 tx=0x7f244c012d50 comp rx=0 tx=0).stop 2024-09-10T07:40:33.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 -- 172.21.15.46:0/1675611996 shutdown_connections 2024-09-10T07:40:33.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55758a6b8c20 0x55758a6bb080 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a6a8d70 0x55758a6a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a69e4b0 0x55758a6a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 --2- 172.21.15.46:0/1675611996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a69db90 0x55758a6a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 -- 172.21.15.46:0/1675611996 >> 172.21.15.46:0/1675611996 conn(0x55758a69a120 msgr2=0x55758a69fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:33.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f243ea00640 0 client.26356 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:33.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.757+0000 7f24664a2740 1 -- 172.21.15.46:0/1675611996 shutdown_connections 2024-09-10T07:40:33.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.758+0000 7f24664a2740 1 -- 172.21.15.46:0/1675611996 wait complete. 2024-09-10T07:40:33.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.758+0000 7f24664a2740 20 client.26356 trim_cache size 0 max 16384 2024-09-10T07:40:33.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.795+0000 7f24664a2740 1 Processor -- start 2024-09-10T07:40:33.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.796+0000 7f24664a2740 1 -- start start 2024-09-10T07:40:33.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.796+0000 7f24664a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.796+0000 7f24664a2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a6a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.796+0000 7f24664a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 0x55758a6990d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.796+0000 7f24664a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f244000fb50 con 0x55758a6a4d90 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.796+0000 7f24664a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2440011720 con 0x55758a69db90 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.796+0000 7f24664a2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2440010220 con 0x55758a6a8d70 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f245be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f2461000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a6a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f2461000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a6a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:54840/0 (socket says 172.21.15.46:54840) 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f2461000640 1 -- 172.21.15.46:0/3913156467 learned_addr learned my addr 172.21.15.46:0/3913156467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f245be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47094/0 (socket says 172.21.15.46:47094) 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f2461000640 1 -- 172.21.15.46:0/3913156467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 msgr2=0x55758a6a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:33.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f2461000640 1 --2- 172.21.15.46:0/3913156467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f2461000640 1 -- 172.21.15.46:0/3913156467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 msgr2=0x55758a6990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:33.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.797+0000 7f2461000640 1 --2- 172.21.15.46:0/3913156467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 0x55758a6990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.798+0000 7f2461000640 1 -- 172.21.15.46:0/3913156467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758a6a41f0 con 0x55758a6a8d70 2024-09-10T07:40:33.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.798+0000 7f2461000640 1 --2- 172.21.15.46:0/3913156467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a6a11e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f244c012040 tx=0x7f244c00fea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef055b5dd5514a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:33.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.799+0000 7f2449600640 1 -- 172.21.15.46:0/3913156467 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c02b030 con 0x55758a6a8d70 2024-09-10T07:40:33.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:33 smithi046 ceph-mon[30705]: pgmap v517: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-10T07:40:33.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.799+0000 7f2449600640 1 -- 172.21.15.46:0/3913156467 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f244c005510 con 0x55758a6a8d70 2024-09-10T07:40:33.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.799+0000 7f2449600640 1 -- 172.21.15.46:0/3913156467 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c00fb50 con 0x55758a6a8d70 2024-09-10T07:40:33.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.800+0000 7f24664a2740 1 -- 172.21.15.46:0/3913156467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 msgr2=0x55758a6a11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.800+0000 7f24664a2740 1 --2- 172.21.15.46:0/3913156467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a6a11e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f244c012040 tx=0x7f244c00fea0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.801+0000 7f24664a2740 1 -- 172.21.15.46:0/3913156467 shutdown_connections 2024-09-10T07:40:33.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.801+0000 7f24664a2740 1 --2- 172.21.15.46:0/3913156467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 0x55758a6990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.801+0000 7f24664a2740 1 --2- 172.21.15.46:0/3913156467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a6a11e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.801+0000 7f24664a2740 1 --2- 172.21.15.46:0/3913156467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.801+0000 7f24664a2740 1 -- 172.21.15.46:0/3913156467 >> 172.21.15.46:0/3913156467 conn(0x55758a69a120 msgr2=0x55758a6989c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:33.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.801+0000 7f24664a2740 1 -- 172.21.15.46:0/3913156467 shutdown_connections 2024-09-10T07:40:33.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.801+0000 7f24664a2740 1 -- 172.21.15.46:0/3913156467 wait complete. 2024-09-10T07:40:33.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.801+0000 7f24664a2740 1 -- start start 2024-09-10T07:40:33.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.802+0000 7f24664a2740 1 Processor -- start 2024-09-10T07:40:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f24664a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f24664a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 0x55758a6ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f24664a2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a7229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f24664a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2440011020 con 0x55758a6a4d90 2024-09-10T07:40:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f24664a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2440010950 con 0x55758a69db90 2024-09-10T07:40:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f24664a2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f244000db30 con 0x55758a6a8d70 2024-09-10T07:40:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f24664a2740 10 client.0 _ll_register_callbacks cb 0x7f244c012470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f245be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f245be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47106/0 (socket says 172.21.15.46:47106) 2024-09-10T07:40:33.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f245be00640 1 -- 172.21.15.46:0/3678710527 learned_addr learned my addr 172.21.15.46:0/3678710527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:33.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.806+0000 7f2461000640 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 0x55758a6ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.807+0000 7f245be00640 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a7229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.807+0000 7f2461000640 1 -- 172.21.15.46:0/3678710527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 msgr2=0x55758a7229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:33.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.807+0000 7f2461000640 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a7229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.807+0000 7f2461000640 1 -- 172.21.15.46:0/3678710527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 msgr2=0x55758a6ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.807+0000 7f2461000640 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.807+0000 7f2461000640 1 -- 172.21.15.46:0/3678710527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758a722ef0 con 0x55758a6a4d90 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.808+0000 7f2461000640 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 0x55758a6ad720 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f244c012040 tx=0x7f244c005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cbf7dc16cceb48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.808+0000 7f243ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.808+0000 7f243ea00640 1 -- 172.21.15.46:0/3678710527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c02b040 con 0x55758a6a4d90 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.808+0000 7f243ea00640 1 -- 172.21.15.46:0/3678710527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f244c010430 con 0x55758a6a4d90 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.808+0000 7f24664a2740 1 -- 172.21.15.46:0/3678710527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f244c005510 con 0x55758a6a4d90 2024-09-10T07:40:33.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.808+0000 7f243ea00640 1 -- 172.21.15.46:0/3678710527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f244c010630 con 0x55758a6a4d90 2024-09-10T07:40:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.808+0000 7f24664a2740 10 client.16101 fetch_fsmap learned FSMap version 38 2024-09-10T07:40:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.808+0000 7f24664a2740 1 -- 172.21.15.46:0/3678710527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55758a724100 con 0x55758a6a4d90 2024-09-10T07:40:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.809+0000 7f243ea00640 1 -- 172.21.15.46:0/3678710527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f244c005510 con 0x55758a6a4d90 2024-09-10T07:40:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.809+0000 7f243ea00640 1 -- 172.21.15.46:0/3678710527 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f244c01c420 con 0x55758a6a4d90 2024-09-10T07:40:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.809+0000 7f24664a2740 10 client.16101 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T07:40:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.809+0000 7f24664a2740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.809+0000 7f24664a2740 20 client.16101 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.809+0000 7f24664a2740 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55758a7246b0 0x55758a726aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.809+0000 7f24664a2740 4 client.16101 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:40:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.809+0000 7f24664a2740 1 -- 172.21.15.46:0/3678710527 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55758a6a18c0 con 0x55758a7246b0 2024-09-10T07:40:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.810+0000 7f245be00640 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55758a7246b0 0x55758a726aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.811+0000 7f245be00640 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55758a7246b0 0x55758a726aa0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea99f7cc1a52c7bf server_cookie=6045c67308b9e13d in_seq=0 out_seq=0 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.811+0000 7f243ea00640 10 client.16101 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f243ea00640 1 -- 172.21.15.46:0/3678710527 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55758a6a18c0 con 0x55758a7246b0 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f243ea00640 10 client.16101 handle_command_reply: tid=0 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 client.16101 shutdown 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 -- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 msgr2=0x55758a6ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 0x55758a6ad720 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f244c012040 tx=0x7f244c005cc0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 -- 172.21.15.46:0/3678710527 shutdown_connections 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55758a7246b0 0x55758a726aa0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55758a6a8d70 0x55758a7229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55758a6a4d90 0x55758a6ad720 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 --2- 172.21.15.46:0/3678710527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55758a69db90 0x55758a6ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f24664a2740 1 -- 172.21.15.46:0/3678710527 >> 172.21.15.46:0/3678710527 conn(0x55758a69a120 msgr2=0x55758a6a7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.812+0000 7f243ea00640 0 client.16101 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.813+0000 7f24664a2740 1 -- 172.21.15.46:0/3678710527 shutdown_connections 2024-09-10T07:40:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.813+0000 7f24664a2740 1 -- 172.21.15.46:0/3678710527 wait complete. 2024-09-10T07:40:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.814+0000 7f24664a2740 20 client.16101 trim_cache size 0 max 16384 2024-09-10T07:40:33.814 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:40:33.814 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:40:33.815 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:40:33.815 INFO:teuthology.orchestra.run.smithi046.stdout: "7b82bb22-f98e-4ddc-bf86-585bdcabd035": { 2024-09-10T07:40:33.815 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:40:33.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "7b82bb22-f98e-4ddc-bf86-585bdcabd035", 2024-09-10T07:40:33.815 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:40:33.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:33.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:33.815 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 -- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f24440a2c40 msgr2=0x7f24440a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f24440a2c40 0x7f24440a5100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f24540043b0 tx=0x7f2454003f10 comp rx=0 tx=0).stop 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 -- 172.21.15.46:0/2669370339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f245c112880 msgr2=0x7f245c19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f245c112880 0x7f245c19e840 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f245402af40 tx=0x7f2454003c10 comp rx=0 tx=0).stop 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 -- 172.21.15.46:0/2669370339 shutdown_connections 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f245c112880 0x7f245c19e840 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f24440a2c40 0x7f24440a5100 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f245c100510 0x7f245c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 --2- 172.21.15.46:0/2669370339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245c0ffbf0 0x7f245c1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.816+0000 7f2463a00640 1 -- 172.21.15.46:0/2669370339 >> 172.21.15.46:0/2669370339 conn(0x7f245c0fba90 msgr2=0x7f245c113480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.817+0000 7f2463a00640 1 -- 172.21.15.46:0/2669370339 shutdown_connections 2024-09-10T07:40:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:33.817+0000 7f2463a00640 1 -- 172.21.15.46:0/2669370339 wait complete. 2024-09-10T07:40:34.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:33 smithi181 ceph-mon[35397]: pgmap v517: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-10T07:40:34.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:33 smithi063 ceph-mon[35020]: pgmap v517: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-10T07:40:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 195.0 s, 1140.2 tps, lat 85.516 ms stddev 109.091 2024-09-10T07:40:36.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:35 smithi181 ceph-mon[35397]: pgmap v518: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-10T07:40:36.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:35 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:40:36.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:35 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:40:36.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:35 smithi063 ceph-mon[35020]: pgmap v518: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-10T07:40:36.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:35 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:40:36.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:35 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:40:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:35 smithi046 ceph-mon[30705]: pgmap v518: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-10T07:40:36.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:35 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:40:36.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:35 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:40:37.021 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:40:37.021 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:40:37.021 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:40:37.021 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:36 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:40:37.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:40:37.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:40:37.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:40:37.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:36 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:40:37.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:40:37.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:40:37.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:40:37.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:36 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:40:37.307 INFO:tasks.cephfs.filesystem:scrub status for tag:7b82bb22-f98e-4ddc-bf86-585bdcabd035 - {'path': '/', 'tag': '7b82bb22-f98e-4ddc-bf86-585bdcabd035', 'options': 'recursive,force'} 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 -- 172.21.15.46:0/957222644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384109120 msgr2=0x7fb384110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 --2- 172.21.15.46:0/957222644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384109120 0x7fb384110ca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009ca0 tx=0x7fb37c02a8d0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 -- 172.21.15.46:0/957222644 shutdown_connections 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 --2- 172.21.15.46:0/957222644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384109120 0x7fb384110ca0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 --2- 172.21.15.46:0/957222644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3841049a0 0x7fb3841089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 --2- 172.21.15.46:0/957222644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb384103ff0 0x7fb3841043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 -- 172.21.15.46:0/957222644 >> 172.21.15.46:0/957222644 conn(0x7fb3840ffa90 msgr2=0x7fb384101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 -- 172.21.15.46:0/957222644 shutdown_connections 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.993+0000 7fb38aa00640 1 -- 172.21.15.46:0/957222644 wait complete. 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.995+0000 7fb38aa00640 1 Processor -- start 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.995+0000 7fb38aa00640 1 -- start start 2024-09-10T07:40:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.996+0000 7fb38aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384103ff0 0x7fb38419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.996+0000 7fb38aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3841049a0 0x7fb38419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.996+0000 7fb38aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb384109120 0x7fb38419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.996+0000 7fb38aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb384114180 con 0x7fb3841049a0 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.996+0000 7fb38aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb384114000 con 0x7fb384103ff0 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.996+0000 7fb38aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb384114300 con 0x7fb384109120 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.998+0000 7fb383e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384103ff0 0x7fb38419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.998+0000 7fb383e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384103ff0 0x7fb38419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42564/0 (socket says 172.21.15.46:42564) 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.998+0000 7fb388a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb384109120 0x7fb38419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.998+0000 7fb388a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb384109120 0x7fb38419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59608/0 (socket says 172.21.15.46:59608) 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.998+0000 7fb388a00640 1 -- 172.21.15.46:0/3288986519 learned_addr learned my addr 172.21.15.46:0/3288986519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.999+0000 7fb388a00640 1 -- 172.21.15.46:0/3288986519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384103ff0 msgr2=0x7fb38419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.999+0000 7fb388a00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384103ff0 0x7fb38419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.999+0000 7fb388a00640 1 -- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3841049a0 msgr2=0x7fb38419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.999+0000 7fb388a00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3841049a0 0x7fb38419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.999+0000 7fb388a00640 1 -- 172.21.15.46:0/3288986519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38419f490 con 0x7fb384109120 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:36.999+0000 7fb388a00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb384109120 0x7fb38419ed90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb37c006010 tx=0x7fb37c031a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.001+0000 7fb380c00640 1 -- 172.21.15.46:0/3288986519 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c003db0 con 0x7fb384109120 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.001+0000 7fb380c00640 1 -- 172.21.15.46:0/3288986519 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb37c004520 con 0x7fb384109120 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.001+0000 7fb380c00640 1 -- 172.21.15.46:0/3288986519 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c02fdd0 con 0x7fb384109120 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.002+0000 7fb38aa00640 1 -- 172.21.15.46:0/3288986519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38419f720 con 0x7fb384109120 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.002+0000 7fb38aa00640 1 -- 172.21.15.46:0/3288986519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38406bc10 con 0x7fb384109120 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.015+0000 7fb38d427740 1 Processor -- start 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.015+0000 7fb380c00640 1 -- 172.21.15.46:0/3288986519 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb37c004040 con 0x7fb384109120 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.018+0000 7fb380c00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb3680a2c00 0x7fb3680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.019+0000 7fb38d427740 1 -- start start 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.019+0000 7fb38d427740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d675c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.019+0000 7fb38d427740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.019+0000 7fb38d427740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d67722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.019+0000 7fb380c00640 1 -- 172.21.15.46:0/3288986519 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb37c02f620 con 0x7fb384109120 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.020+0000 7fb388a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d67722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.020+0000 7fb388a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d67722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42568/0 (socket says 172.21.15.46:42568) 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.020+0000 7fb388a00640 1 -- 172.21.15.46:0/2334960519 learned_addr learned my addr 172.21.15.46:0/2334960519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.020+0000 7fb38d427740 1 -- 172.21.15.46:0/2334960519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5579d66c1260 con 0x5579d675cd20 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.020+0000 7fb38d427740 1 -- 172.21.15.46:0/2334960519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5579d6668500 con 0x5579d676fed0 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.020+0000 7fb38d427740 1 -- 172.21.15.46:0/2334960519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5579d66bd220 con 0x5579d675c400 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.021+0000 7fb383e00640 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d675c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.021+0000 7fb383400640 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.022+0000 7fb388a00640 1 -- 172.21.15.46:0/2334960519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 msgr2=0x5579d675c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.022+0000 7fb388a00640 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d675c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.022+0000 7fb388a00640 1 -- 172.21.15.46:0/2334960519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 msgr2=0x5579d676f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.022+0000 7fb388a00640 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.023+0000 7fb388a00640 1 -- 172.21.15.46:0/2334960519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579d6432ba0 con 0x5579d676fed0 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.023+0000 7fb383400640 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.023+0000 7fb383400640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb3680a2c00 0x7fb3680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.023+0000 7fb388a00640 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d67722c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb37c02ae90 tx=0x7fb37c0c4500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9c01d4ba08f5fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.023+0000 7fb383e00640 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d675c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.024+0000 7fb379200640 1 -- 172.21.15.46:0/2334960519 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0b9870 con 0x5579d676fed0 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.024+0000 7fb379200640 1 -- 172.21.15.46:0/2334960519 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb37c036070 con 0x5579d676fed0 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.026+0000 7fb38d427740 1 -- 172.21.15.46:0/2334960519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 msgr2=0x5579d67722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.026+0000 7fb38d427740 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d67722c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb37c02ae90 tx=0x7fb37c0c4500 comp rx=0 tx=0).stop 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.027+0000 7fb383400640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb3680a2c00 0x7fb3680a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb370005940 tx=0x7fb370005b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.027+0000 7fb38d427740 1 -- 172.21.15.46:0/2334960519 shutdown_connections 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.027+0000 7fb38d427740 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d67722c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.027+0000 7fb38d427740 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.027+0000 7fb38d427740 1 --2- 172.21.15.46:0/2334960519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d675c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.027+0000 7fb38d427740 1 -- 172.21.15.46:0/2334960519 >> 172.21.15.46:0/2334960519 conn(0x5579d6758a70 msgr2=0x5579d6758e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.030+0000 7fb38d427740 1 -- 172.21.15.46:0/2334960519 shutdown_connections 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.030+0000 7fb38d427740 1 -- 172.21.15.46:0/2334960519 wait complete. 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.031+0000 7fb38d427740 1 -- start start 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.031+0000 7fb38d427740 1 Processor -- start 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.033+0000 7fb38d427740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d676cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.033+0000 7fb38d427740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.033+0000 7fb383e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d676cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.033+0000 7fb383e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d676cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59620/0 (socket says 172.21.15.46:59620) 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.033+0000 7fb383e00640 1 -- 172.21.15.46:0/1536790374 learned_addr learned my addr 172.21.15.46:0/1536790374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.034+0000 7fb38d427740 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d676d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.034+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5579d6772800 con 0x5579d675cd20 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.034+0000 7fb383400640 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.034+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5579d667bbd0 con 0x5579d676fed0 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.034+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5579d6772980 con 0x5579d675c400 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.034+0000 7fb388a00640 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d676d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.034+0000 7fb38d427740 10 client.0 _ll_register_callbacks cb 0x5579d63d00a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.035+0000 7fb388a00640 1 -- 172.21.15.46:0/1536790374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 msgr2=0x5579d676cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.035+0000 7fb388a00640 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d676cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.035+0000 7fb388a00640 1 -- 172.21.15.46:0/1536790374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 msgr2=0x5579d676d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.035+0000 7fb388a00640 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.035+0000 7fb388a00640 1 -- 172.21.15.46:0/1536790374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579d677bd00 con 0x5579d676fed0 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.037+0000 7fb383e00640 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d676cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.037+0000 7fb388a00640 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d676d920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0ba0c0 tx=0x7fb37c02a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afec164929a88dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.037+0000 7fb36d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.038+0000 7fb36d600640 1 -- 172.21.15.46:0/1536790374 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0bf070 con 0x5579d676fed0 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.038+0000 7fb36d600640 1 -- 172.21.15.46:0/1536790374 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb37c0bc980 con 0x5579d676fed0 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.038+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5579d66ce990 con 0x5579d676fed0 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.038+0000 7fb36d600640 1 -- 172.21.15.46:0/1536790374 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb37c0b9bb0 con 0x5579d676fed0 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.038+0000 7fb38d427740 10 client.26381 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.038+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5579d6772800 con 0x5579d676fed0 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.039+0000 7fb36d600640 1 -- 172.21.15.46:0/1536790374 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5579d66ce990 con 0x5579d676fed0 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.039+0000 7fb36d600640 1 -- 172.21.15.46:0/1536790374 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fb37c004b40 con 0x5579d676fed0 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.039+0000 7fb38d427740 10 client.26381 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.039+0000 7fb38d427740 10 client.26381 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:40:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.039+0000 7fb38d427740 20 client.26381 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.039+0000 7fb38d427740 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5579d677f860 0x5579d6781c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.040+0000 7fb38d427740 4 client.26381 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.040+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579d66c8ce0 con 0x5579d677f860 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.040+0000 7fb383e00640 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5579d677f860 0x5579d6781c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.042+0000 7fb383e00640 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5579d677f860 0x5579d6781c50 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b477bcd47040193b server_cookie=ea14bc4e465e1aca in_seq=0 out_seq=0 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.042+0000 7fb36d600640 10 client.26381 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb36d600640 1 -- 172.21.15.46:0/1536790374 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579d66c8ce0 con 0x5579d677f860 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb36d600640 10 client.26381 handle_command_reply: tid=0 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 client.26381 shutdown 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 msgr2=0x5579d676d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d676d920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0ba0c0 tx=0x7fb37c02a8d0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 shutdown_connections 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5579d677f860 0x5579d6781c50 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d676d920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d676d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 --2- 172.21.15.46:0/1536790374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d676cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 >> 172.21.15.46:0/1536790374 conn(0x5579d6758a70 msgr2=0x5579d6757bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.044+0000 7fb36d600640 0 client.26381 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.045+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 shutdown_connections 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.045+0000 7fb38d427740 1 -- 172.21.15.46:0/1536790374 wait complete. 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.045+0000 7fb38d427740 20 client.26381 trim_cache size 0 max 16384 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.080+0000 7fb38d427740 1 Processor -- start 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.080+0000 7fb38d427740 1 -- start start 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb38d427740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675c400 0x5579d676bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb38d427740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d675cd20 0x5579d676d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb38d427740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d676fed0 0x5579d6787520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb38d427740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb384071510 con 0x5579d675c400 2024-09-10T07:40:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb38d427740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb3840730b0 con 0x5579d675cd20 2024-09-10T07:40:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb38d427740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb384071be0 con 0x5579d676fed0 2024-09-10T07:40:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d675cd20 0x5579d676d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d675cd20 0x5579d676d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42588/0 (socket says 172.21.15.46:42588) 2024-09-10T07:40:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675c400 0x5579d676bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383400640 1 -- 172.21.15.46:0/341157166 learned_addr learned my addr 172.21.15.46:0/341157166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb388a00640 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d676fed0 0x5579d6787520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383400640 1 -- 172.21.15.46:0/341157166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d676fed0 msgr2=0x5579d6787520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383400640 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d676fed0 0x5579d6787520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383400640 1 -- 172.21.15.46:0/341157166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675c400 msgr2=0x5579d676bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383400640 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675c400 0x5579d676bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383400640 1 -- 172.21.15.46:0/341157166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579d6752fe0 con 0x5579d675cd20 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.081+0000 7fb383e00640 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675c400 0x5579d676bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.082+0000 7fb383400640 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d675cd20 0x5579d676d770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb37000bf60 tx=0x7fb370005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcefc38d4e63c4fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:37.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.082+0000 7fb36fe00640 1 -- 172.21.15.46:0/341157166 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370023030 con 0x5579d675cd20 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.082+0000 7fb36fe00640 1 -- 172.21.15.46:0/341157166 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb370017070 con 0x5579d675cd20 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.082+0000 7fb36fe00640 1 -- 172.21.15.46:0/341157166 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37001e800 con 0x5579d675cd20 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 -- 172.21.15.46:0/341157166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d675cd20 msgr2=0x5579d676d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d675cd20 0x5579d676d770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb37000bf60 tx=0x7fb370005ce0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 -- 172.21.15.46:0/341157166 shutdown_connections 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d676fed0 0x5579d6787520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d675cd20 0x5579d676d770 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 --2- 172.21.15.46:0/341157166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675c400 0x5579d676bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 -- 172.21.15.46:0/341157166 >> 172.21.15.46:0/341157166 conn(0x5579d6758a70 msgr2=0x5579d676ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 -- 172.21.15.46:0/341157166 shutdown_connections 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.083+0000 7fb38d427740 1 -- 172.21.15.46:0/341157166 wait complete. 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.084+0000 7fb38d427740 1 -- start start 2024-09-10T07:40:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.084+0000 7fb38d427740 1 Processor -- start 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.084+0000 7fb38d427740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d67781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.084+0000 7fb38d427740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d6773500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.084+0000 7fb38d427740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d6773a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.084+0000 7fb38d427740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb3840729e0 con 0x5579d675cd20 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.084+0000 7fb38d427740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb384072310 con 0x5579d676fed0 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.084+0000 7fb38d427740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb38406f530 con 0x5579d675c400 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb38d427740 10 client.0 _ll_register_callbacks cb 0x7fb37400c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d67781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d67781d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59640/0 (socket says 172.21.15.46:59640) 2024-09-10T07:40:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383e00640 1 -- 172.21.15.46:0/3294511654 learned_addr learned my addr 172.21.15.46:0/3294511654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383400640 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d6773500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383e00640 1 -- 172.21.15.46:0/3294511654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 msgr2=0x5579d6773a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383e00640 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d6773a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383e00640 1 -- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 msgr2=0x5579d6773500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383e00640 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d6773500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383e00640 1 -- 172.21.15.46:0/3294511654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579d6774130 con 0x5579d675c400 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.085+0000 7fb383400640 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d6773500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.086+0000 7fb383e00640 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d67781d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5579d6782490 tx=0x7fb3740039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f2c1851b33bb1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.086+0000 7fb36d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.086+0000 7fb36d600640 1 -- 172.21.15.46:0/3294511654 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37401c620 con 0x5579d675c400 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.086+0000 7fb38d427740 1 -- 172.21.15.46:0/3294511654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb370017070 con 0x5579d675c400 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.087+0000 7fb36d600640 1 -- 172.21.15.46:0/3294511654 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb37401c7c0 con 0x5579d675c400 2024-09-10T07:40:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.089+0000 7fb36d600640 1 -- 172.21.15.46:0/3294511654 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb37402d050 con 0x5579d675c400 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.089+0000 7fb38d427740 10 client.26377 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.089+0000 7fb38d427740 1 -- 172.21.15.46:0/3294511654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5579d67ea420 con 0x5579d675c400 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb36d600640 1 -- 172.21.15.46:0/3294511654 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb370017070 con 0x5579d675c400 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb36d600640 1 -- 172.21.15.46:0/3294511654 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fb37401b3a0 con 0x5579d675c400 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb38d427740 10 client.26377 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb38d427740 10 client.26377 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb38d427740 20 client.26377 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb38d427740 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5579d67eab60 0x5579d67ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb38d427740 4 client.26377 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb38d427740 1 -- 172.21.15.46:0/3294511654 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x5579d6644730 con 0x5579d67eab60 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.091+0000 7fb383400640 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5579d67eab60 0x5579d67ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.092+0000 7fb383400640 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5579d67eab60 0x5579d67ecf80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187403f4ef8fde8e server_cookie=7aabafc1d092da27 in_seq=0 out_seq=0 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.092+0000 7fb36d600640 10 client.26377 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb36d600640 1 -- 172.21.15.46:0/3294511654 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5579d6644730 con 0x5579d67eab60 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb36d600640 10 client.26377 handle_command_reply: tid=0 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 client.26377 shutdown 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 -- 172.21.15.46:0/3294511654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 msgr2=0x5579d67781d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d67781d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5579d6782490 tx=0x7fb3740039c0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 -- 172.21.15.46:0/3294511654 shutdown_connections 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5579d67eab60 0x5579d67ecf80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579d676fed0 0x5579d6773a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579d675cd20 0x5579d6773500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 --2- 172.21.15.46:0/3294511654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579d675c400 0x5579d67781d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb36d600640 0 client.26377 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.268+0000 7fb38d427740 1 -- 172.21.15.46:0/3294511654 >> 172.21.15.46:0/3294511654 conn(0x5579d6758a70 msgr2=0x5579d67573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.269+0000 7fb38d427740 1 -- 172.21.15.46:0/3294511654 shutdown_connections 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.269+0000 7fb38d427740 1 -- 172.21.15.46:0/3294511654 wait complete. 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.269+0000 7fb38d427740 20 client.26377 trim_cache size 0 max 16384 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 39, 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 10, 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "8ac40f1b": { 2024-09-10T07:40:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 10, 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 142.8, 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:40:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.270+0000 7fb38aa00640 1 -- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb3680a2c00 msgr2=0x7fb3680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.270+0000 7fb38aa00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb3680a2c00 0x7fb3680a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb370005940 tx=0x7fb370005b80 comp rx=0 tx=0).stop 2024-09-10T07:40:37.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.270+0000 7fb38aa00640 1 -- 172.21.15.46:0/3288986519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb384109120 msgr2=0x7fb38419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.270+0000 7fb38aa00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb384109120 0x7fb38419ed90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb37c006010 tx=0x7fb37c031a00 comp rx=0 tx=0).stop 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.271+0000 7fb38aa00640 1 -- 172.21.15.46:0/3288986519 shutdown_connections 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.271+0000 7fb38aa00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb384109120 0x7fb38419ed90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.271+0000 7fb38aa00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb3680a2c00 0x7fb3680a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.271+0000 7fb38aa00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3841049a0 0x7fb38419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.271+0000 7fb38aa00640 1 --2- 172.21.15.46:0/3288986519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb384103ff0 0x7fb38419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.271+0000 7fb38aa00640 1 -- 172.21.15.46:0/3288986519 >> 172.21.15.46:0/3288986519 conn(0x7fb3840ffa90 msgr2=0x7fb384100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.273+0000 7fb38aa00640 1 -- 172.21.15.46:0/3288986519 shutdown_connections 2024-09-10T07:40:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:37.273+0000 7fb38aa00640 1 -- 172.21.15.46:0/3288986519 wait complete. 2024-09-10T07:40:38.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:37 smithi181 ceph-mon[35397]: pgmap v519: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-09-10T07:40:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:37 smithi063 ceph-mon[35020]: pgmap v519: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-09-10T07:40:38.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:37 smithi046 ceph-mon[30705]: pgmap v519: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-09-10T07:40:38.468 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8ac40f1b', quiesce took 11.7 seconds. Will release after: 11.05 2024-09-10T07:40:40.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:39 smithi181 ceph-mon[35397]: pgmap v520: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 12 MiB/s wr, 873 op/s 2024-09-10T07:40:40.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:39 smithi063 ceph-mon[35020]: pgmap v520: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 12 MiB/s wr, 873 op/s 2024-09-10T07:40:40.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:39 smithi046 ceph-mon[30705]: pgmap v520: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 12 MiB/s wr, 873 op/s 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 -- 172.21.15.46:0/1545206131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff38103d60 msgr2=0x7eff38104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 --2- 172.21.15.46:0/1545206131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff38103d60 0x7eff38104160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7eff28009900 tx=0x7eff2802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 -- 172.21.15.46:0/1545206131 shutdown_connections 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 --2- 172.21.15.46:0/1545206131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff38105910 0x7eff3810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 --2- 172.21.15.46:0/1545206131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff38104f50 0x7eff381053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 --2- 172.21.15.46:0/1545206131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff38103d60 0x7eff38104160 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 -- 172.21.15.46:0/1545206131 >> 172.21.15.46:0/1545206131 conn(0x7eff380ff4e0 msgr2=0x7eff38101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 -- 172.21.15.46:0/1545206131 shutdown_connections 2024-09-10T07:40:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.490+0000 7eff3fa00640 1 -- 172.21.15.46:0/1545206131 wait complete. 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.491+0000 7eff3fa00640 1 Processor -- start 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.491+0000 7eff3fa00640 1 -- start start 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3fa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff38103d60 0x7eff38199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff38104f50 0x7eff3819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3fa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff38105910 0x7eff381a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810fa30 con 0x7eff38104f50 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3fa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810f8b0 con 0x7eff38103d60 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3fa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810fbb0 con 0x7eff38105910 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff38103d60 0x7eff38199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff38104f50 0x7eff3819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff38103d60 0x7eff38199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42602/0 (socket says 172.21.15.46:42602) 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3d000640 1 -- 172.21.15.46:0/4114455192 learned_addr learned my addr 172.21.15.46:0/4114455192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3da00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff38105910 0x7eff381a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3d000640 1 -- 172.21.15.46:0/4114455192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff38105910 msgr2=0x7eff381a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3d000640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff38105910 0x7eff381a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3d000640 1 -- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff38104f50 msgr2=0x7eff3819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3d000640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff38104f50 0x7eff3819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.492+0000 7eff3d000640 1 -- 172.21.15.46:0/4114455192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff381a1be0 con 0x7eff38103d60 2024-09-10T07:40:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.493+0000 7eff3d000640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff38103d60 0x7eff38199fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7eff28031440 tx=0x7eff28032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.493+0000 7eff36000640 1 -- 172.21.15.46:0/4114455192 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2803f030 con 0x7eff38103d60 2024-09-10T07:40:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.493+0000 7eff3fa00640 1 -- 172.21.15.46:0/4114455192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff381a1e70 con 0x7eff38103d60 2024-09-10T07:40:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.493+0000 7eff36000640 1 -- 172.21.15.46:0/4114455192 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff28003ce0 con 0x7eff38103d60 2024-09-10T07:40:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.493+0000 7eff36000640 1 -- 172.21.15.46:0/4114455192 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28035070 con 0x7eff38103d60 2024-09-10T07:40:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.493+0000 7eff3fa00640 1 -- 172.21.15.46:0/4114455192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff381a22d0 con 0x7eff38103d60 2024-09-10T07:40:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff36000640 1 -- 172.21.15.46:0/4114455192 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff28042030 con 0x7eff38103d60 2024-09-10T07:40:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff429e4740 1 Processor -- start 2024-09-10T07:40:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff429e4740 1 -- start start 2024-09-10T07:40:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff429e4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff429e4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff429e4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff429e4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5638e4817100 con 0x5638e48b7a60 2024-09-10T07:40:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff429e4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5638e47bf1b0 con 0x5638e48b2090 2024-09-10T07:40:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.497+0000 7eff429e4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5638e48130c0 con 0x5638e48b29d0 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff3d000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff3da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff3da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42612/0 (socket says 172.21.15.46:42612) 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff3da00640 1 -- 172.21.15.46:0/3218919919 learned_addr learned my addr 172.21.15.46:0/3218919919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48b9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43094/0 (socket says 172.21.15.46:43094) 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff2fe00640 1 -- 172.21.15.46:0/3218919919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 msgr2=0x5638e48b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff2fe00640 1 --2- 172.21.15.46:0/3218919919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff2fe00640 1 -- 172.21.15.46:0/3218919919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 msgr2=0x5638e48b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff2fe00640 1 --2- 172.21.15.46:0/3218919919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff2fe00640 1 -- 172.21.15.46:0/3218919919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e452a210 con 0x5638e48b7a60 2024-09-10T07:40:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff3da00640 1 --2- 172.21.15.46:0/3218919919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff36000640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff1c0a2cb0 0x7eff1c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.498+0000 7eff2fe00640 1 --2- 172.21.15.46:0/3218919919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48b9f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7eff20009bf0 tx=0x7eff20011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d5e7ef6788027c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.499+0000 7eff36000640 1 -- 172.21.15.46:0/4114455192 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7eff280ef800 con 0x7eff38103d60 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.499+0000 7eff3da00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff1c0a2cb0 0x7eff1c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.499+0000 7eff2e000640 1 -- 172.21.15.46:0/3218919919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2001e070 con 0x5638e48b7a60 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.499+0000 7eff2e000640 1 -- 172.21.15.46:0/3218919919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff20009da0 con 0x5638e48b7a60 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.499+0000 7eff2e000640 1 -- 172.21.15.46:0/3218919919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20018870 con 0x5638e48b7a60 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 -- 172.21.15.46:0/3218919919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 msgr2=0x5638e48b9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff3da00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff1c0a2cb0 0x7eff1c0a5170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff30011890 tx=0x7eff30009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 --2- 172.21.15.46:0/3218919919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48b9f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7eff20009bf0 tx=0x7eff20011810 comp rx=0 tx=0).stop 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 -- 172.21.15.46:0/3218919919 shutdown_connections 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 --2- 172.21.15.46:0/3218919919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48b9f20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 --2- 172.21.15.46:0/3218919919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 --2- 172.21.15.46:0/3218919919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 -- 172.21.15.46:0/3218919919 >> 172.21.15.46:0/3218919919 conn(0x5638e48ad700 msgr2=0x5638e48adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 -- 172.21.15.46:0/3218919919 shutdown_connections 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 -- 172.21.15.46:0/3218919919 wait complete. 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.500+0000 7eff429e4740 1 -- start start 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff429e4740 1 Processor -- start 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff429e4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b2090 0x5638e48bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff429e4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b29d0 0x5638e48bc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff429e4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b7a60 0x5638e48c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff429e4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5638e48baa30 con 0x5638e48b2090 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff429e4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5638e47a0090 con 0x5638e48b29d0 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff429e4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5638e48babb0 con 0x5638e48b7a60 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff429e4740 10 client.0 _ll_register_callbacks cb 0x5638e47f4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff3d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b2090 0x5638e48bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff3d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b2090 0x5638e48bc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43106/0 (socket says 172.21.15.46:43106) 2024-09-10T07:40:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.501+0000 7eff3d000640 1 -- 172.21.15.46:0/4208964958 learned_addr learned my addr 172.21.15.46:0/4208964958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff2fe00640 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b29d0 0x5638e48bc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff3da00640 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b7a60 0x5638e48c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff3d000640 1 -- 172.21.15.46:0/4208964958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b7a60 msgr2=0x5638e48c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff3d000640 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b7a60 0x5638e48c3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff3d000640 1 -- 172.21.15.46:0/4208964958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b29d0 msgr2=0x5638e48bc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff3d000640 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b29d0 0x5638e48bc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff3d000640 1 -- 172.21.15.46:0/4208964958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e45f6620 con 0x5638e48b2090 2024-09-10T07:40:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff3d000640 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b2090 0x5638e48bc2b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7eff28006010 tx=0x7eff28009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e84bca5ff14dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff26a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff26a00640 1 -- 172.21.15.46:0/4208964958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28049070 con 0x5638e48b2090 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff26a00640 1 -- 172.21.15.46:0/4208964958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7eff2803bcf0 con 0x5638e48b2090 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.502+0000 7eff429e4740 1 -- 172.21.15.46:0/4208964958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff20009da0 con 0x5638e48b2090 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff26a00640 1 -- 172.21.15.46:0/4208964958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7eff280bb870 con 0x5638e48b2090 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff429e4740 10 client.16122 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff429e4740 1 -- 172.21.15.46:0/4208964958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5638e48c6620 con 0x5638e48b2090 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff26a00640 1 -- 172.21.15.46:0/4208964958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff20009da0 con 0x5638e48b2090 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff26a00640 1 -- 172.21.15.46:0/4208964958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7eff280b83a0 con 0x5638e48b2090 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff429e4740 10 client.16122 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff429e4740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff429e4740 20 client.16122 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff429e4740 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5638e48c7d00 0x5638e48ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff429e4740 4 client.16122 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff429e4740 1 -- 172.21.15.46:0/4208964958 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638e4824830 con 0x5638e48c7d00 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.503+0000 7eff2fe00640 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5638e48c7d00 0x5638e48ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.504+0000 7eff2fe00640 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5638e48c7d00 0x5638e48ca150 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653b381390d1e038 server_cookie=c7c84e7a40127aae in_seq=0 out_seq=0 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.504+0000 7eff26a00640 10 client.16122 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff26a00640 1 -- 172.21.15.46:0/4208964958 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638e4824830 con 0x5638e48c7d00 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff26a00640 10 client.16122 handle_command_reply: tid=0 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 client.16122 shutdown 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 -- 172.21.15.46:0/4208964958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b2090 msgr2=0x5638e48bc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b2090 0x5638e48bc2b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7eff28006010 tx=0x7eff28009620 comp rx=0 tx=0).stop 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 -- 172.21.15.46:0/4208964958 shutdown_connections 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5638e48c7d00 0x5638e48ca150 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b7a60 0x5638e48c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b29d0 0x5638e48bc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 --2- 172.21.15.46:0/4208964958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b2090 0x5638e48bc2b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff26a00640 0 client.16122 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 -- 172.21.15.46:0/4208964958 >> 172.21.15.46:0/4208964958 conn(0x5638e48ad700 msgr2=0x5638e48aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 -- 172.21.15.46:0/4208964958 shutdown_connections 2024-09-10T07:40:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 1 -- 172.21.15.46:0/4208964958 wait complete. 2024-09-10T07:40:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.505+0000 7eff429e4740 20 client.16122 trim_cache size 0 max 16384 2024-09-10T07:40:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff429e4740 1 Processor -- start 2024-09-10T07:40:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff429e4740 1 -- start start 2024-09-10T07:40:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff429e4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48bafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff429e4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff429e4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff429e4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff381ac830 con 0x5638e48b7a60 2024-09-10T07:40:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff429e4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff381ae3d0 con 0x5638e48b2090 2024-09-10T07:40:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff429e4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff381acf00 con 0x5638e48b29d0 2024-09-10T07:40:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff3da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff3da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43120/0 (socket says 172.21.15.46:43120) 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff3da00640 1 -- 172.21.15.46:0/279023914 learned_addr learned my addr 172.21.15.46:0/279023914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.521+0000 7eff2fe00640 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff3d000640 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48bafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff3da00640 1 -- 172.21.15.46:0/279023914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 msgr2=0x5638e48c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff3da00640 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff3da00640 1 -- 172.21.15.46:0/279023914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 msgr2=0x5638e48bafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff3da00640 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48bafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff3da00640 1 -- 172.21.15.46:0/279023914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e48ad520 con 0x5638e48b7a60 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff3da00640 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48ae5b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7eff300039f0 tx=0x7eff30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184958a78d91b17b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff2d600640 1 -- 172.21.15.46:0/279023914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30004b20 con 0x5638e48b7a60 2024-09-10T07:40:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff2d600640 1 -- 172.21.15.46:0/279023914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30004cc0 con 0x5638e48b7a60 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.522+0000 7eff2d600640 1 -- 172.21.15.46:0/279023914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30005020 con 0x5638e48b7a60 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 -- 172.21.15.46:0/279023914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 msgr2=0x5638e48ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48ae5b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7eff300039f0 tx=0x7eff30016000 comp rx=0 tx=0).stop 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 -- 172.21.15.46:0/279023914 shutdown_connections 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48ae5b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 --2- 172.21.15.46:0/279023914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48bafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 -- 172.21.15.46:0/279023914 >> 172.21.15.46:0/279023914 conn(0x5638e48ad700 msgr2=0x5638e48bc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 -- 172.21.15.46:0/279023914 shutdown_connections 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.523+0000 7eff429e4740 1 -- 172.21.15.46:0/279023914 wait complete. 2024-09-10T07:40:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 1 -- start start 2024-09-10T07:40:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 1 Processor -- start 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48c0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff381add00 con 0x5638e48b7a60 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff381ad630 con 0x5638e48b2090 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff381aa810 con 0x5638e48b29d0 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff429e4740 10 client.0 _ll_register_callbacks cb 0x7eff280f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff3da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff3da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48c20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43126/0 (socket says 172.21.15.46:43126) 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff3da00640 1 -- 172.21.15.46:0/2952724234 learned_addr learned my addr 172.21.15.46:0/2952724234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff2fe00640 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.524+0000 7eff3d000640 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48c0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff3da00640 1 -- 172.21.15.46:0/2952724234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 msgr2=0x5638e48c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff3da00640 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48c10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff3da00640 1 -- 172.21.15.46:0/2952724234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 msgr2=0x5638e48c0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff3da00640 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48c0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff3da00640 1 -- 172.21.15.46:0/2952724234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e48c2700 con 0x5638e48b7a60 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff3da00640 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48c20f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7eff30029040 tx=0x7eff3000ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80dab9092c2496f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff26a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff26a00640 1 -- 172.21.15.46:0/2952724234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30009c80 con 0x5638e48b7a60 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff429e4740 1 -- 172.21.15.46:0/2952724234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff30004cc0 con 0x5638e48b7a60 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.525+0000 7eff26a00640 1 -- 172.21.15.46:0/2952724234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7eff30003fe0 con 0x5638e48b7a60 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff26a00640 1 -- 172.21.15.46:0/2952724234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7eff30030480 con 0x5638e48b7a60 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff429e4740 10 client.16128 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff429e4740 1 -- 172.21.15.46:0/2952724234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5638e48c3ad0 con 0x5638e48b7a60 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff26a00640 1 -- 172.21.15.46:0/2952724234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30004cc0 con 0x5638e48b7a60 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff26a00640 1 -- 172.21.15.46:0/2952724234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7eff30026070 con 0x5638e48b7a60 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff429e4740 10 client.16128 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff429e4740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff429e4740 20 client.16128 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff429e4740 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5638e4933120 0x5638e4935510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff429e4740 4 client.16128 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff429e4740 1 -- 172.21.15.46:0/2952724234 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638e481eb80 con 0x5638e4933120 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.526+0000 7eff3d000640 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5638e4933120 0x5638e4935510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.527+0000 7eff3d000640 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5638e4933120 0x5638e4935510 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f020e6404086ec server_cookie=8145c6ab12d73e93 in_seq=0 out_seq=0 2024-09-10T07:40:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.527+0000 7eff26a00640 10 client.16128 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.539+0000 7eff26a00640 1 -- 172.21.15.46:0/2952724234 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638e481eb80 con 0x5638e4933120 2024-09-10T07:40:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.539+0000 7eff26a00640 10 client.16128 handle_command_reply: tid=0 2024-09-10T07:40:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.539+0000 7eff429e4740 1 client.16128 shutdown 2024-09-10T07:40:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 -- 172.21.15.46:0/2952724234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 msgr2=0x5638e48c20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48c20f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7eff30029040 tx=0x7eff3000ff80 comp rx=0 tx=0).stop 2024-09-10T07:40:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 -- 172.21.15.46:0/2952724234 shutdown_connections 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5638e4933120 0x5638e4935510 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638e48b7a60 0x5638e48c20f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638e48b29d0 0x5638e48c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 --2- 172.21.15.46:0/2952724234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638e48b2090 0x5638e48c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 -- 172.21.15.46:0/2952724234 >> 172.21.15.46:0/2952724234 conn(0x5638e48ad700 msgr2=0x5638e48bc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff26a00640 0 client.16128 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 -- 172.21.15.46:0/2952724234 shutdown_connections 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 1 -- 172.21.15.46:0/2952724234 wait complete. 2024-09-10T07:40:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.540+0000 7eff429e4740 20 client.16128 trim_cache size 0 max 16384 2024-09-10T07:40:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:40:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:40:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:40:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:40:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 -- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff1c0a2cb0 msgr2=0x7eff1c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff1c0a2cb0 0x7eff1c0a5170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff30011890 tx=0x7eff30009500 comp rx=0 tx=0).stop 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 -- 172.21.15.46:0/4114455192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff38103d60 msgr2=0x7eff38199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff38103d60 0x7eff38199fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7eff28031440 tx=0x7eff28032ce0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 -- 172.21.15.46:0/4114455192 shutdown_connections 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff38105910 0x7eff381a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff1c0a2cb0 0x7eff1c0a5170 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff38104f50 0x7eff3819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 --2- 172.21.15.46:0/4114455192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff38103d60 0x7eff38199fc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 -- 172.21.15.46:0/4114455192 >> 172.21.15.46:0/4114455192 conn(0x7eff380ff4e0 msgr2=0x7eff38108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 -- 172.21.15.46:0/4114455192 shutdown_connections 2024-09-10T07:40:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:40.541+0000 7eff3fa00640 1 -- 172.21.15.46:0/4114455192 wait complete. 2024-09-10T07:40:40.602 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 200.0 s, 271.4 tps, lat 127.499 ms stddev 201.707 2024-09-10T07:40:42.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:41 smithi046 ceph-mon[30705]: pgmap v521: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-10T07:40:42.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:41 smithi181 ceph-mon[35397]: pgmap v521: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-10T07:40:42.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:41 smithi063 ceph-mon[35020]: pgmap v521: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-10T07:40:44.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:43 smithi181 ceph-mon[35397]: pgmap v522: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-09-10T07:40:44.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:43 smithi063 ceph-mon[35020]: pgmap v522: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-09-10T07:40:44.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:43 smithi046 ceph-mon[30705]: pgmap v522: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2024-09-10T07:40:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 205.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:40:46.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:45 smithi181 ceph-mon[35397]: pgmap v523: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.8 MiB/s wr, 440 op/s 2024-09-10T07:40:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:45 smithi063 ceph-mon[35020]: pgmap v523: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.8 MiB/s wr, 440 op/s 2024-09-10T07:40:46.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:45 smithi046 ceph-mon[30705]: pgmap v523: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.8 MiB/s wr, 440 op/s 2024-09-10T07:40:48.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:47 smithi181 ceph-mon[35397]: pgmap v524: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.5 MiB/s wr, 471 op/s 2024-09-10T07:40:48.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:47 smithi063 ceph-mon[35020]: pgmap v524: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.5 MiB/s wr, 471 op/s 2024-09-10T07:40:48.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:47 smithi046 ceph-mon[30705]: pgmap v524: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.5 MiB/s wr, 471 op/s 2024-09-10T07:40:49.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:40:49.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:49 smithi046 ceph-mon[30705]: pgmap v525: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 320 op/s 2024-09-10T07:40:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:50.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:49 smithi181 ceph-mon[35397]: pgmap v525: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 320 op/s 2024-09-10T07:40:50.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:49 smithi063 ceph-mon[35020]: pgmap v525: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 320 op/s 2024-09-10T07:40:50.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 -- 172.21.15.46:0/2406256352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64f4104010 msgr2=0x7f64f41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 --2- 172.21.15.46:0/2406256352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64f4104010 0x7f64f41043f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009ca0 tx=0x7f64e802a8b0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 -- 172.21.15.46:0/2406256352 shutdown_connections 2024-09-10T07:40:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 --2- 172.21.15.46:0/2406256352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f4109140 0x7f64f4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 --2- 172.21.15.46:0/2406256352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f41049c0 0x7f64f4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 --2- 172.21.15.46:0/2406256352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64f4104010 0x7f64f41043f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 -- 172.21.15.46:0/2406256352 >> 172.21.15.46:0/2406256352 conn(0x7f64f40ffa50 msgr2=0x7f64f4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 -- 172.21.15.46:0/2406256352 shutdown_connections 2024-09-10T07:40:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.488+0000 7f64fb400640 1 -- 172.21.15.46:0/2406256352 wait complete. 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.489+0000 7f64fb400640 1 Processor -- start 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.489+0000 7f64fb400640 1 -- start start 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.489+0000 7f64fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f4104010 0x7f64f419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f41049c0 0x7f64f419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64fb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64f4109140 0x7f64f419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64fb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4114190 con 0x7f64f41049c0 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64fb400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4114010 con 0x7f64f4109140 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64fb400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4114310 con 0x7f64f4104010 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f4104010 0x7f64f419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f41049c0 0x7f64f419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f4104010 0x7f64f419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37938/0 (socket says 172.21.15.46:37938) 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f8a00640 1 -- 172.21.15.46:0/1295304921 learned_addr learned my addr 172.21.15.46:0/1295304921 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f9400640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64f4109140 0x7f64f419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f8a00640 1 -- 172.21.15.46:0/1295304921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64f4109140 msgr2=0x7f64f419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f8a00640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64f4109140 0x7f64f419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f8a00640 1 -- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f41049c0 msgr2=0x7f64f419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f8a00640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f41049c0 0x7f64f419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.490+0000 7f64f8a00640 1 -- 172.21.15.46:0/1295304921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f419f5e0 con 0x7f64f4104010 2024-09-10T07:40:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.491+0000 7f64f3e00640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f41049c0 0x7f64f419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.491+0000 7f64f8a00640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f4104010 0x7f64f419a630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64e8005e30 tx=0x7f64e8003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.491+0000 7f64f1600640 1 -- 172.21.15.46:0/1295304921 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8030ec0 con 0x7f64f4104010 2024-09-10T07:40:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.491+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f419f870 con 0x7f64f4104010 2024-09-10T07:40:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.491+0000 7f64f1600640 1 -- 172.21.15.46:0/1295304921 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e8040070 con 0x7f64f4104010 2024-09-10T07:40:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.491+0000 7f64f1600640 1 -- 172.21.15.46:0/1295304921 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8045b80 con 0x7f64f4104010 2024-09-10T07:40:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.491+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f41a71b0 con 0x7f64f4104010 2024-09-10T07:40:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.492+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f41043f0 con 0x7f64f4104010 2024-09-10T07:40:50.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.495+0000 7f64f1600640 1 -- 172.21.15.46:0/1295304921 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64e8004b40 con 0x7f64f4104010 2024-09-10T07:40:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.497+0000 7f64f1600640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64dc0a2da0 0x7f64dc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:50.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.497+0000 7f64f1600640 1 -- 172.21.15.46:0/1295304921 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f64e80f0670 con 0x7f64f4104010 2024-09-10T07:40:50.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.497+0000 7f64f3e00640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64dc0a2da0 0x7f64dc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:50.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.498+0000 7f64f3e00640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64dc0a2da0 0x7f64dc0a5260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64f40ffe70 tx=0x7f64e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:50.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.498+0000 7f64f1600640 1 -- 172.21.15.46:0/1295304921 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64e80bd1f0 con 0x7f64f4104010 2024-09-10T07:40:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 210.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:40:50.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.733+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64f419b990 con 0x7f64f4104010 2024-09-10T07:40:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.735+0000 7f64f1600640 1 -- 172.21.15.46:0/1295304921 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11975 (secure 0 0 0) 0x7f64e80bcaf0 con 0x7f64f4104010 2024-09-10T07:40:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:40:50.736 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"btime":"2024-09-10T07:40:35:758945+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:40:35.499710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:40:50.736 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 39 2024-09-10T07:40:50.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.737+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64dc0a2da0 msgr2=0x7f64dc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:50.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.737+0000 7f64fb400640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64dc0a2da0 0x7f64dc0a5260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64f40ffe70 tx=0x7f64e000b390 comp rx=0 tx=0).stop 2024-09-10T07:40:50.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.737+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f4104010 msgr2=0x7f64f419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:50.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.737+0000 7f64fb400640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f4104010 0x7f64f419a630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64e8005e30 tx=0x7f64e8003f90 comp rx=0 tx=0).stop 2024-09-10T07:40:50.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.738+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 shutdown_connections 2024-09-10T07:40:50.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.738+0000 7f64fb400640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64f4109140 0x7f64f419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.738+0000 7f64fb400640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64dc0a2da0 0x7f64dc0a5260 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.738+0000 7f64fb400640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f41049c0 0x7f64f419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.738+0000 7f64fb400640 1 --2- 172.21.15.46:0/1295304921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f4104010 0x7f64f419a630 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.738+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 >> 172.21.15.46:0/1295304921 conn(0x7f64f40ffa50 msgr2=0x7f64f4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.738+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 shutdown_connections 2024-09-10T07:40:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:50.738+0000 7f64fb400640 1 -- 172.21.15.46:0/1295304921 wait complete. 2024-09-10T07:40:51.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:50 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1295304921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:51.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1295304921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:51.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:50 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1295304921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:51.651 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id 8ac40f1b --release --await' 2024-09-10T07:40:51.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id 8ac40f1b --release --await 2024-09-10T07:40:51.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:51 smithi046 ceph-mon[30705]: pgmap v526: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 317 op/s 2024-09-10T07:40:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:52.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:51 smithi181 ceph-mon[35397]: pgmap v526: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 317 op/s 2024-09-10T07:40:52.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:51 smithi063 ceph-mon[35020]: pgmap v526: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 317 op/s 2024-09-10T07:40:52.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 -- 172.21.15.46:0/1021546330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 msgr2=0x7f56a01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 --2- 172.21.15.46:0/1021546330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 0x7f56a01043d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5690009900 tx=0x7f569002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 -- 172.21.15.46:0/1021546330 shutdown_connections 2024-09-10T07:40:52.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 --2- 172.21.15.46:0/1021546330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56a0109120 0x7f56a0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 --2- 172.21.15.46:0/1021546330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a01049a0 0x7f56a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 --2- 172.21.15.46:0/1021546330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 0x7f56a01043d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 -- 172.21.15.46:0/1021546330 >> 172.21.15.46:0/1021546330 conn(0x7f56a00ffa90 msgr2=0x7f56a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:52.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 -- 172.21.15.46:0/1021546330 shutdown_connections 2024-09-10T07:40:52.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.646+0000 7f56a6800640 1 -- 172.21.15.46:0/1021546330 wait complete. 2024-09-10T07:40:52.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.647+0000 7f56a6800640 1 Processor -- start 2024-09-10T07:40:52.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.647+0000 7f56a6800640 1 -- start start 2024-09-10T07:40:52.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f56a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 0x7f56a019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f56a6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56a01049a0 0x7f56a019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f56a6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a0109120 0x7f56a019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f56a6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0114170 con 0x7f56a0103ff0 2024-09-10T07:40:52.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f56a6800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0113ff0 con 0x7f56a0109120 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f56a6800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01142f0 con 0x7f56a01049a0 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f569f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 0x7f56a019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f569f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 0x7f56a019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37264/0 (socket says 172.21.15.46:37264) 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f569f400640 1 -- 172.21.15.46:0/1036641697 learned_addr learned my addr 172.21.15.46:0/1036641697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f569ea00640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56a01049a0 0x7f56a019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.648+0000 7f569fe00640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a0109120 0x7f56a019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f569f400640 1 -- 172.21.15.46:0/1036641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56a01049a0 msgr2=0x7f56a019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f569f400640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56a01049a0 0x7f56a019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f569f400640 1 -- 172.21.15.46:0/1036641697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a0109120 msgr2=0x7f56a019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f569f400640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a0109120 0x7f56a019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f569f400640 1 -- 172.21.15.46:0/1036641697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a019f680 con 0x7f56a0103ff0 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f569fe00640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a0109120 0x7f56a019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f569ea00640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56a01049a0 0x7f56a019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f569f400640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 0x7f56a019a6d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5690004770 tx=0x7f5690003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f5697e00640 1 -- 172.21.15.46:0/1036641697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569003f070 con 0x7f56a0103ff0 2024-09-10T07:40:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f5697e00640 1 -- 172.21.15.46:0/1036641697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5690031ca0 con 0x7f56a0103ff0 2024-09-10T07:40:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f5697e00640 1 -- 172.21.15.46:0/1036641697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690035030 con 0x7f56a0103ff0 2024-09-10T07:40:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f56a6800640 1 -- 172.21.15.46:0/1036641697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a019f910 con 0x7f56a0103ff0 2024-09-10T07:40:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.649+0000 7f56a6800640 1 -- 172.21.15.46:0/1036641697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a7250 con 0x7f56a0103ff0 2024-09-10T07:40:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f56a9290740 1 Processor -- start 2024-09-10T07:40:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f5697e00640 1 -- 172.21.15.46:0/1036641697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5690042070 con 0x7f56a0103ff0 2024-09-10T07:40:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f56a9290740 1 -- start start 2024-09-10T07:40:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f56a9290740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ff310 0x5643712ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f56a9290740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371309fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f56a9290740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f56a9290740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564371265650 con 0x56437130a4f0 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f56a9290740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56437120cf10 con 0x5643712ff310 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.653+0000 7f56a9290740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564371261610 con 0x5643712ffc30 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ff310 0x5643712ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ff310 0x5643712ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37316/0 (socket says 172.21.15.46:37316) 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 -- 172.21.15.46:0/1655883165 learned_addr learned my addr 172.21.15.46:0/1655883165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569f400640 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371309fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569ea00640 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 -- 172.21.15.46:0/1655883165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 msgr2=0x564371309fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371309fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 -- 172.21.15.46:0/1655883165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 msgr2=0x56437130c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 -- 172.21.15.46:0/1655883165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564370f74580 con 0x5643712ff310 2024-09-10T07:40:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.654+0000 7f569fe00640 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ff310 0x5643712ff6f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5698009f30 tx=0x7f5698010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b593aca710bbea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.655+0000 7f5697e00640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56840a2c00 0x7f56840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.655+0000 7f5694c00640 1 -- 172.21.15.46:0/1655883165 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698016070 con 0x5643712ff310 2024-09-10T07:40:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.655+0000 7f5697e00640 1 -- 172.21.15.46:0/1036641697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f56900efb90 con 0x7f56a0103ff0 2024-09-10T07:40:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.655+0000 7f5694c00640 1 -- 172.21.15.46:0/1655883165 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569801bb40 con 0x5643712ff310 2024-09-10T07:40:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.655+0000 7f5694c00640 1 -- 172.21.15.46:0/1655883165 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698012c50 con 0x5643712ff310 2024-09-10T07:40:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.655+0000 7f569f400640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56840a2c00 0x7f56840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 -- 172.21.15.46:0/1655883165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ff310 msgr2=0x5643712ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f569f400640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56840a2c00 0x7f56840a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f56900b89e0 tx=0x7f56900f35d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ff310 0x5643712ff6f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5698009f30 tx=0x7f5698010800 comp rx=0 tx=0).stop 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 -- 172.21.15.46:0/1655883165 shutdown_connections 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371309fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 --2- 172.21.15.46:0/1655883165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ff310 0x5643712ff6f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 -- 172.21.15.46:0/1655883165 >> 172.21.15.46:0/1655883165 conn(0x5643712fb8a0 msgr2=0x5643712fbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 -- 172.21.15.46:0/1655883165 shutdown_connections 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 -- 172.21.15.46:0/1655883165 wait complete. 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.656+0000 7f56a9290740 1 -- start start 2024-09-10T07:40:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f56a9290740 1 Processor -- start 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f56a9290740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x564371307440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f56a9290740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x564371307980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f56a9290740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x564371307ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f56a9290740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56437130d0c0 con 0x56437130a4f0 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f56a9290740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564371220350 con 0x5643712ffc30 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f56a9290740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56437130d240 con 0x5643712ff310 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f56a9290740 10 client.0 _ll_register_callbacks cb 0x564371243290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f569ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x564371307980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f569fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x564371307ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.657+0000 7f569ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x564371307980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37322/0 (socket says 172.21.15.46:37322) 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569ea00640 1 -- 172.21.15.46:0/205080028 learned_addr learned my addr 172.21.15.46:0/205080028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569fe00640 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x564371307440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569ea00640 1 -- 172.21.15.46:0/205080028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 msgr2=0x564371307440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569ea00640 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x564371307440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569ea00640 1 -- 172.21.15.46:0/205080028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 msgr2=0x564371307ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569ea00640 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x564371307ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569ea00640 1 -- 172.21.15.46:0/205080028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56437103e960 con 0x5643712ffc30 2024-09-10T07:40:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569fe00640 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x564371307440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f569ea00640 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x564371307980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f568c0117c0 tx=0x7f568c011c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a6439882aa5fa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.658+0000 7f5689600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.659+0000 7f5689600640 1 -- 172.21.15.46:0/205080028 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c010c60 con 0x5643712ffc30 2024-09-10T07:40:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.659+0000 7f5689600640 1 -- 172.21.15.46:0/205080028 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f568c010e00 con 0x5643712ffc30 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.659+0000 7f56a9290740 1 -- 172.21.15.46:0/205080028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569801bb40 con 0x5643712ffc30 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.659+0000 7f5689600640 1 -- 172.21.15.46:0/205080028 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f568c025050 con 0x5643712ffc30 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.659+0000 7f56a9290740 10 client.26420 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.659+0000 7f56a9290740 1 -- 172.21.15.46:0/205080028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564371318d40 con 0x5643712ffc30 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.659+0000 7f5689600640 1 -- 172.21.15.46:0/205080028 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569801bb40 con 0x5643712ffc30 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f5689600640 1 -- 172.21.15.46:0/205080028 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f568c00f620 con 0x5643712ffc30 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f56a9290740 10 client.26420 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f56a9290740 10 client.26420 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f56a9290740 20 client.26420 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f56a9290740 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56437131a2a0 0x56437131c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f56a9290740 4 client.26420 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f56a9290740 1 -- 172.21.15.46:0/205080028 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564371272bc0 con 0x56437131a2a0 2024-09-10T07:40:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f569fe00640 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56437131a2a0 0x56437131c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f569fe00640 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56437131a2a0 0x56437131c6c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7954149218d66ce6 server_cookie=4e0dfd3eb8a3863a in_seq=0 out_seq=0 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.660+0000 7f5689600640 10 client.26420 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.661+0000 7f5689600640 1 -- 172.21.15.46:0/205080028 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564371272bc0 con 0x56437131a2a0 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.661+0000 7f5689600640 10 client.26420 handle_command_reply: tid=0 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.661+0000 7f56a9290740 1 client.26420 shutdown 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.661+0000 7f56a9290740 1 -- 172.21.15.46:0/205080028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 msgr2=0x564371307980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.661+0000 7f56a9290740 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x564371307980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f568c0117c0 tx=0x7f568c011c80 comp rx=0 tx=0).stop 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 1 -- 172.21.15.46:0/205080028 shutdown_connections 2024-09-10T07:40:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56437131a2a0 0x56437131c6c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x564371307ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x564371307980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 1 --2- 172.21.15.46:0/205080028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x564371307440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f5689600640 0 client.26420 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 1 -- 172.21.15.46:0/205080028 >> 172.21.15.46:0/205080028 conn(0x5643712fb8a0 msgr2=0x5643712faaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 1 -- 172.21.15.46:0/205080028 shutdown_connections 2024-09-10T07:40:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 1 -- 172.21.15.46:0/205080028 wait complete. 2024-09-10T07:40:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.662+0000 7f56a9290740 20 client.26420 trim_cache size 0 max 16384 2024-09-10T07:40:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f56a9290740 1 Processor -- start 2024-09-10T07:40:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f56a9290740 1 -- start start 2024-09-10T07:40:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f56a9290740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643712ff310 0x564371307d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f56a9290740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371321f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f56a9290740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56437130a4f0 0x5643713224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f56a9290740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b0db0 con 0x5643712ff310 2024-09-10T07:40:52.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f56a9290740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b2950 con 0x56437130a4f0 2024-09-10T07:40:52.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f56a9290740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b1480 con 0x5643712ffc30 2024-09-10T07:40:52.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.680+0000 7f569fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643712ff310 0x564371307d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371321f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643712ff310 0x564371307d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37294/0 (socket says 172.21.15.46:37294) 2024-09-10T07:40:52.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569fe00640 1 -- 172.21.15.46:0/2263945822 learned_addr learned my addr 172.21.15.46:0/2263945822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:52.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569fe00640 1 --2- 172.21.15.46:0/2263945822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56437130a4f0 0x5643713224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569ea00640 1 -- 172.21.15.46:0/2263945822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56437130a4f0 msgr2=0x5643713224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569ea00640 1 --2- 172.21.15.46:0/2263945822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56437130a4f0 0x5643713224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569ea00640 1 -- 172.21.15.46:0/2263945822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643712ff310 msgr2=0x564371307d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569ea00640 1 --2- 172.21.15.46:0/2263945822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643712ff310 0x564371307d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569ea00640 1 -- 172.21.15.46:0/2263945822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643712fb340 con 0x5643712ffc30 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f569ea00640 1 --2- 172.21.15.46:0/2263945822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371321f70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f568c00fc00 tx=0x7f568c00ff30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d1554a9327f501c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.681+0000 7f568be00640 1 -- 172.21.15.46:0/2263945822 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c02d020 con 0x5643712ffc30 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f568be00640 1 -- 172.21.15.46:0/2263945822 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c010c60 con 0x5643712ffc30 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f568be00640 1 -- 172.21.15.46:0/2263945822 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c0053f0 con 0x5643712ffc30 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 -- 172.21.15.46:0/2263945822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 msgr2=0x564371321f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 --2- 172.21.15.46:0/2263945822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371321f70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f568c00fc00 tx=0x7f568c00ff30 comp rx=0 tx=0).stop 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 -- 172.21.15.46:0/2263945822 shutdown_connections 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 --2- 172.21.15.46:0/2263945822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56437130a4f0 0x5643713224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 --2- 172.21.15.46:0/2263945822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ffc30 0x564371321f70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 --2- 172.21.15.46:0/2263945822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643712ff310 0x564371307d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 -- 172.21.15.46:0/2263945822 >> 172.21.15.46:0/2263945822 conn(0x5643712fb8a0 msgr2=0x5643713093b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 -- 172.21.15.46:0/2263945822 shutdown_connections 2024-09-10T07:40:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.682+0000 7f56a9290740 1 -- 172.21.15.46:0/2263945822 wait complete. 2024-09-10T07:40:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.683+0000 7f56a9290740 1 -- start start 2024-09-10T07:40:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.683+0000 7f56a9290740 1 Processor -- start 2024-09-10T07:40:52.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.683+0000 7f56a9290740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x56437130e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.683+0000 7f56a9290740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x56437130e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.683+0000 7f56a9290740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.683+0000 7f56a9290740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b2280 con 0x56437130a4f0 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.683+0000 7f56a9290740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01b1bb0 con 0x5643712ffc30 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.683+0000 7f56a9290740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01aed90 con 0x5643712ff310 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f56a9290740 10 client.0 _ll_register_callbacks cb 0x7f568c00bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x56437130e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x56437130e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x56437130e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37332/0 (socket says 172.21.15.46:37332) 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569ea00640 1 -- 172.21.15.46:0/4289799122 learned_addr learned my addr 172.21.15.46:0/4289799122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569fe00640 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569ea00640 1 -- 172.21.15.46:0/4289799122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 msgr2=0x56437130e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569ea00640 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x56437130e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569ea00640 1 -- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 msgr2=0x56437130edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569ea00640 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569ea00640 1 -- 172.21.15.46:0/4289799122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643713840e0 con 0x5643712ffc30 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.684+0000 7f569fe00640 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.685+0000 7f569ea00640 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x56437130e8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f568c000c00 tx=0x7f568c00f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c395bb5e283f965 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.685+0000 7f5689600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.685+0000 7f5689600640 1 -- 172.21.15.46:0/4289799122 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c036070 con 0x5643712ffc30 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.685+0000 7f5689600640 1 -- 172.21.15.46:0/4289799122 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f568c00fc30 con 0x5643712ffc30 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.685+0000 7f56a9290740 1 -- 172.21.15.46:0/4289799122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568c010c60 con 0x5643712ffc30 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.685+0000 7f5689600640 1 -- 172.21.15.46:0/4289799122 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f568c041050 con 0x5643712ffc30 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.685+0000 7f56a9290740 10 client.26426 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.685+0000 7f56a9290740 1 -- 172.21.15.46:0/4289799122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5643713853e0 con 0x5643712ffc30 2024-09-10T07:40:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f5689600640 1 -- 172.21.15.46:0/4289799122 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c010c60 con 0x5643712ffc30 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f5689600640 1 -- 172.21.15.46:0/4289799122 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f568c004950 con 0x5643712ffc30 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f56a9290740 10 client.26426 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f56a9290740 10 client.26426 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f56a9290740 20 client.26426 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f56a9290740 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5643713859f0 0x564371387e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f56a9290740 4 client.26426 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8ac40f1b", "release": true, "await": true}] 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f56a9290740 1 -- 172.21.15.46:0/4289799122 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8ac40f1b", "release": true, "await": true}) -- 0x56437126d0d0 con 0x5643713859f0 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.686+0000 7f569fe00640 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5643713859f0 0x564371387e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.687+0000 7f569fe00640 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5643713859f0 0x564371387e10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410963a19ab3ba7e server_cookie=c41900fdfabf9d0f in_seq=0 out_seq=0 2024-09-10T07:40:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.687+0000 7f5689600640 10 client.26426 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.730+0000 7f5689600640 1 -- 172.21.15.46:0/4289799122 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56437126d0d0 con 0x5643713859f0 2024-09-10T07:40:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.730+0000 7f5689600640 10 client.26426 handle_command_reply: tid=0 2024-09-10T07:40:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.730+0000 7f56a9290740 1 client.26426 shutdown 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f56a9290740 1 -- 172.21.15.46:0/4289799122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 msgr2=0x56437130e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f56a9290740 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x56437130e8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f568c000c00 tx=0x7f568c00f670 comp rx=0 tx=0).stop 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f56a9290740 1 -- 172.21.15.46:0/4289799122 shutdown_connections 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f56a9290740 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5643713859f0 0x564371387e10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f56a9290740 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56437130a4f0 0x56437130edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f56a9290740 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643712ffc30 0x56437130e8b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f56a9290740 1 --2- 172.21.15.46:0/4289799122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5643712ff310 0x56437130e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f56a9290740 1 -- 172.21.15.46:0/4289799122 >> 172.21.15.46:0/4289799122 conn(0x5643712fb8a0 msgr2=0x564371300d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.732+0000 7f5689600640 0 client.26426 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.733+0000 7f56a9290740 1 -- 172.21.15.46:0/4289799122 shutdown_connections 2024-09-10T07:40:52.736 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:40:52.736 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 39, 2024-09-10T07:40:52.736 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:40:52.736 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 12, 2024-09-10T07:40:52.736 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:40:52.736 INFO:teuthology.orchestra.run.smithi046.stdout: "8ac40f1b": { 2024-09-10T07:40:52.736 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 12, 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 142.8, 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:40:52.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.733+0000 7f56a9290740 1 -- 172.21.15.46:0/4289799122 wait complete. 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.733+0000 7f56a9290740 20 client.26426 trim_cache size 0 max 16384 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 -- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56840a2c00 msgr2=0x7f56840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56840a2c00 0x7f56840a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f56900b89e0 tx=0x7f56900f35d0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 -- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 msgr2=0x7f56a019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 0x7f56a019a6d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5690004770 tx=0x7f5690003c10 comp rx=0 tx=0).stop 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 -- 172.21.15.46:0/1036641697 shutdown_connections 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a0109120 0x7f56a019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56a01049a0 0x7f56a019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56840a2c00 0x7f56840a50c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 --2- 172.21.15.46:0/1036641697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56a0103ff0 0x7f56a019a6d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.735+0000 7f56a6800640 1 -- 172.21.15.46:0/1036641697 >> 172.21.15.46:0/1036641697 conn(0x7f56a00ffa90 msgr2=0x7f56a0100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.736+0000 7f56a6800640 1 -- 172.21.15.46:0/1036641697 shutdown_connections 2024-09-10T07:40:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:52.736+0000 7f56a6800640 1 -- 172.21.15.46:0/1036641697 wait complete. 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 -- 172.21.15.46:0/3696808815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103d60 msgr2=0x7f1740104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 --2- 172.21.15.46:0/3696808815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103d60 0x7f1740104160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1738009ca0 tx=0x7f173802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 -- 172.21.15.46:0/3696808815 shutdown_connections 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 --2- 172.21.15.46:0/3696808815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1740105910 0x7f174010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 --2- 172.21.15.46:0/3696808815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1740104f50 0x7f17401053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 --2- 172.21.15.46:0/3696808815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740103d60 0x7f1740104160 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 -- 172.21.15.46:0/3696808815 >> 172.21.15.46:0/3696808815 conn(0x7f17400ff4e0 msgr2=0x7f1740101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 -- 172.21.15.46:0/3696808815 shutdown_connections 2024-09-10T07:40:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.719+0000 7f1747e00640 1 -- 172.21.15.46:0/3696808815 wait complete. 2024-09-10T07:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.720+0000 7f1747e00640 1 Processor -- start 2024-09-10T07:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1747e00640 1 -- start start 2024-09-10T07:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1747e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1740103d60 0x7f174019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1747e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1740104f50 0x7f174019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1747e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740105910 0x7f17401a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1747e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1740077990 con 0x7f1740103d60 2024-09-10T07:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1747e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1740077810 con 0x7f1740105910 2024-09-10T07:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1747e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1740077b10 con 0x7f1740104f50 2024-09-10T07:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1745e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740105910 0x7f17401a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1745e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740105910 0x7f17401a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37340/0 (socket says 172.21.15.46:37340) 2024-09-10T07:40:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1745e00640 1 -- 172.21.15.46:0/2609832836 learned_addr learned my addr 172.21.15.46:0/2609832836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1744a00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1740104f50 0x7f174019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.721+0000 7f1745400640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1740103d60 0x7f174019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.722+0000 7f1745e00640 1 -- 172.21.15.46:0/2609832836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1740104f50 msgr2=0x7f174019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.722+0000 7f1745e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1740104f50 0x7f174019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.722+0000 7f1745e00640 1 -- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1740103d60 msgr2=0x7f174019e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.722+0000 7f1745e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1740103d60 0x7f174019e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.722+0000 7f1745e00640 1 -- 172.21.15.46:0/2609832836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17401a5f90 con 0x7f1740105910 2024-09-10T07:40:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.722+0000 7f1745e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740105910 0x7f17401a5920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f172c00e970 tx=0x7f172c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.723+0000 7f173e000640 1 -- 172.21.15.46:0/2609832836 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c00ccc0 con 0x7f1740105910 2024-09-10T07:40:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.723+0000 7f173e000640 1 -- 172.21.15.46:0/2609832836 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c00ce60 con 0x7f1740105910 2024-09-10T07:40:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.723+0000 7f173e000640 1 -- 172.21.15.46:0/2609832836 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c010490 con 0x7f1740105910 2024-09-10T07:40:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.723+0000 7f1747e00640 1 -- 172.21.15.46:0/2609832836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17401a6280 con 0x7f1740105910 2024-09-10T07:40:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.723+0000 7f1747e00640 1 -- 172.21.15.46:0/2609832836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401a6780 con 0x7f1740105910 2024-09-10T07:40:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.727+0000 7f174abd3740 1 Processor -- start 2024-09-10T07:40:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.731+0000 7f173e000640 1 -- 172.21.15.46:0/2609832836 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f172c005020 con 0x7f1740105910 2024-09-10T07:40:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.732+0000 7f174abd3740 1 -- start start 2024-09-10T07:40:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.732+0000 7f174abd3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91df490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.732+0000 7f174abd3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91e4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.732+0000 7f174abd3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb91e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.732+0000 7f174abd3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dcb9144100 con 0x55dcb91e4a60 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.732+0000 7f174abd3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dcb90ec1b0 con 0x55dcb91df9d0 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.732+0000 7f174abd3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dcb91400c0 con 0x55dcb91df090 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.733+0000 7f1745400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.733+0000 7f1745400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91e4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37348/0 (socket says 172.21.15.46:37348) 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.733+0000 7f1745400640 1 -- 172.21.15.46:0/2219056182 learned_addr learned my addr 172.21.15.46:0/2219056182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.733+0000 7f1744a00640 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb91e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.733+0000 7f1745400640 1 -- 172.21.15.46:0/2219056182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 msgr2=0x55dcb91df490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:40:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1744a00640 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91df490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1745400640 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91df490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1745400640 1 -- 172.21.15.46:0/2219056182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 msgr2=0x55dcb91e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1745400640 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb91e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1745400640 1 -- 172.21.15.46:0/2219056182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcb8e57210 con 0x55dcb91df9d0 2024-09-10T07:40:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1744a00640 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91df490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f173e000640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17280a2d90 0x7f17280a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1745400640 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91e4490 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1740104db0 tx=0x7f173802adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92ebd3b9258540b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f173e000640 1 -- 172.21.15.46:0/2609832836 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f172c0cf7c0 con 0x7f1740105910 2024-09-10T07:40:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1732600640 1 -- 172.21.15.46:0/2219056182 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738049070 con 0x55dcb91df9d0 2024-09-10T07:40:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1732600640 1 -- 172.21.15.46:0/2219056182 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1738036040 con 0x55dcb91df9d0 2024-09-10T07:40:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.734+0000 7f1732600640 1 -- 172.21.15.46:0/2219056182 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173802fd90 con 0x55dcb91df9d0 2024-09-10T07:40:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.735+0000 7f1744a00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17280a2d90 0x7f17280a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f174abd3740 1 -- 172.21.15.46:0/2219056182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 msgr2=0x55dcb91e4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f1744a00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17280a2d90 0x7f17280a5250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f174019f880 tx=0x7f1734014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f174abd3740 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91e4490 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1740104db0 tx=0x7f173802adc0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f174abd3740 1 -- 172.21.15.46:0/2219056182 shutdown_connections 2024-09-10T07:40:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f174abd3740 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb91e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f174abd3740 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91e4490 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f174abd3740 1 --2- 172.21.15.46:0/2219056182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91df490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f174abd3740 1 -- 172.21.15.46:0/2219056182 >> 172.21.15.46:0/2219056182 conn(0x55dcb91da700 msgr2=0x55dcb91dab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.736+0000 7f174abd3740 1 -- 172.21.15.46:0/2219056182 shutdown_connections 2024-09-10T07:40:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.737+0000 7f174abd3740 1 -- 172.21.15.46:0/2219056182 wait complete. 2024-09-10T07:40:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.739+0000 7f174abd3740 1 -- start start 2024-09-10T07:40:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.739+0000 7f174abd3740 1 Processor -- start 2024-09-10T07:40:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.740+0000 7f174abd3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.740+0000 7f174abd3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df9d0 0x55dcb91fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.740+0000 7f1745400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.740+0000 7f1745400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91fa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37354/0 (socket says 172.21.15.46:37354) 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.740+0000 7f1745400640 1 -- 172.21.15.46:0/219417699 learned_addr learned my addr 172.21.15.46:0/219417699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f174abd3740 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91e4a60 0x55dcb9201620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dcb91f8860 con 0x55dcb91df090 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dcb90cd090 con 0x55dcb91e4a60 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dcb91f89e0 con 0x55dcb91df9d0 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f174abd3740 10 client.0 _ll_register_callbacks cb 0x55dcb9121f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1744a00640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df9d0 0x55dcb91fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1745e00640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91e4a60 0x55dcb9201620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1745400640 1 -- 172.21.15.46:0/219417699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df9d0 msgr2=0x55dcb91fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1745400640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df9d0 0x55dcb91fa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1745400640 1 -- 172.21.15.46:0/219417699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91e4a60 msgr2=0x55dcb9201620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1745400640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91e4a60 0x55dcb9201620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1745400640 1 -- 172.21.15.46:0/219417699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcb8f23620 con 0x55dcb91df090 2024-09-10T07:40:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1744a00640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df9d0 0x55dcb91fa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:40:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.741+0000 7f1745e00640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91e4a60 0x55dcb9201620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.742+0000 7f1745400640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91fa0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1738009ca0 tx=0x7f17380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2264bff980b745 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.742+0000 7f1726a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:40:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.742+0000 7f1726a00640 1 -- 172.21.15.46:0/219417699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738054060 con 0x55dcb91df090 2024-09-10T07:40:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.742+0000 7f1726a00640 1 -- 172.21.15.46:0/219417699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1738031450 con 0x55dcb91df090 2024-09-10T07:40:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.742+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1738036040 con 0x55dcb91df090 2024-09-10T07:40:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.743+0000 7f1726a00640 1 -- 172.21.15.46:0/219417699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f173805b050 con 0x55dcb91df090 2024-09-10T07:40:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.743+0000 7f174abd3740 10 client.16164 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.743+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dcb91f8860 con 0x55dcb91df090 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.744+0000 7f1726a00640 1 -- 172.21.15.46:0/219417699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1738036040 con 0x55dcb91df090 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.744+0000 7f1726a00640 1 -- 172.21.15.46:0/219417699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f173803b490 con 0x55dcb91df090 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.744+0000 7f174abd3740 10 client.16164 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.744+0000 7f174abd3740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.744+0000 7f174abd3740 20 client.16164 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.744+0000 7f174abd3740 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dcb9205740 0x55dcb9207b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.744+0000 7f174abd3740 4 client.16164 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.744+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcb9151830 con 0x55dcb9205740 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.745+0000 7f1744a00640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dcb9205740 0x55dcb9207b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.746+0000 7f1744a00640 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dcb9205740 0x55dcb9207b50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef61e7484bb1ea5 server_cookie=f33104d461a7ee84 in_seq=0 out_seq=0 2024-09-10T07:40:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.746+0000 7f1726a00640 10 client.16164 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f1726a00640 1 -- 172.21.15.46:0/219417699 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcb9151830 con 0x55dcb9205740 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f1726a00640 10 client.16164 handle_command_reply: tid=0 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 client.16164 shutdown 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 msgr2=0x55dcb91fa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91fa0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1738009ca0 tx=0x7f17380095c0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 shutdown_connections 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dcb9205740 0x55dcb9207b50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91e4a60 0x55dcb9201620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df9d0 0x55dcb91fa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 --2- 172.21.15.46:0/219417699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91fa0a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 >> 172.21.15.46:0/219417699 conn(0x55dcb91da700 msgr2=0x55dcb91dbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f1726a00640 0 client.16164 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.748+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 shutdown_connections 2024-09-10T07:40:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.749+0000 7f174abd3740 1 -- 172.21.15.46:0/219417699 wait complete. 2024-09-10T07:40:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.749+0000 7f174abd3740 20 client.16164 trim_cache size 0 max 16384 2024-09-10T07:40:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.792+0000 7f174abd3740 1 Processor -- start 2024-09-10T07:40:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.792+0000 7f174abd3740 1 -- start start 2024-09-10T07:40:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.792+0000 7f174abd3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91dbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.792+0000 7f174abd3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb92015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.792+0000 7f174abd3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91e4a60 0x55dcb91dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.792+0000 7f174abd3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17401a8480 con 0x55dcb91df090 2024-09-10T07:40:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.792+0000 7f174abd3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17401aa020 con 0x55dcb91df9d0 2024-09-10T07:40:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.792+0000 7f174abd3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17401a8b50 con 0x55dcb91e4a60 2024-09-10T07:40:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.793+0000 7f1744a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb92015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.793+0000 7f1744a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb92015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37362/0 (socket says 172.21.15.46:37362) 2024-09-10T07:40:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.793+0000 7f1744a00640 1 -- 172.21.15.46:0/2942398801 learned_addr learned my addr 172.21.15.46:0/2942398801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.793+0000 7f1744a00640 1 -- 172.21.15.46:0/2942398801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91e4a60 msgr2=0x55dcb91dc030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:40:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.793+0000 7f1744a00640 1 --2- 172.21.15.46:0/2942398801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91e4a60 0x55dcb91dc030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.793+0000 7f1744a00640 1 -- 172.21.15.46:0/2942398801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 msgr2=0x55dcb91dbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.793+0000 7f1744a00640 1 --2- 172.21.15.46:0/2942398801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91dbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.793+0000 7f1744a00640 1 -- 172.21.15.46:0/2942398801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcb92081d0 con 0x55dcb91df9d0 2024-09-10T07:40:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.794+0000 7f1744a00640 1 --2- 172.21.15.46:0/2942398801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb92015e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55dcb91fb580 tx=0x7f1734016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e141353c7b1b122e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.794+0000 7f1731c00640 1 -- 172.21.15.46:0/2942398801 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734020a10 con 0x55dcb91df9d0 2024-09-10T07:40:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.795+0000 7f1731c00640 1 -- 172.21.15.46:0/2942398801 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1734020bb0 con 0x55dcb91df9d0 2024-09-10T07:40:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.795+0000 7f1731c00640 1 -- 172.21.15.46:0/2942398801 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734003e50 con 0x55dcb91df9d0 2024-09-10T07:40:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.795+0000 7f174abd3740 1 -- 172.21.15.46:0/2942398801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 msgr2=0x55dcb92015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.795+0000 7f174abd3740 1 --2- 172.21.15.46:0/2942398801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb92015e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55dcb91fb580 tx=0x7f1734016000 comp rx=0 tx=0).stop 2024-09-10T07:40:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.796+0000 7f174abd3740 1 -- 172.21.15.46:0/2942398801 shutdown_connections 2024-09-10T07:40:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.796+0000 7f174abd3740 1 --2- 172.21.15.46:0/2942398801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91e4a60 0x55dcb91dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.796+0000 7f174abd3740 1 --2- 172.21.15.46:0/2942398801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb92015e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.796+0000 7f174abd3740 1 --2- 172.21.15.46:0/2942398801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91df090 0x55dcb91dbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.796+0000 7f174abd3740 1 -- 172.21.15.46:0/2942398801 >> 172.21.15.46:0/2942398801 conn(0x55dcb91da700 msgr2=0x55dcb91d9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.796+0000 7f174abd3740 1 -- 172.21.15.46:0/2942398801 shutdown_connections 2024-09-10T07:40:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.796+0000 7f174abd3740 1 -- 172.21.15.46:0/2942398801 wait complete. 2024-09-10T07:40:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.797+0000 7f174abd3740 1 -- start start 2024-09-10T07:40:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.797+0000 7f174abd3740 1 Processor -- start 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f174abd3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f174abd3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91ff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f174abd3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb9200870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f174abd3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17401a9950 con 0x55dcb91e4a60 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f174abd3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17401a9280 con 0x55dcb91df9d0 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f174abd3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1740073de0 con 0x55dcb91df090 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f174abd3740 10 client.0 _ll_register_callbacks cb 0x7f1738047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f1745400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f1745400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91ff320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38024/0 (socket says 172.21.15.46:38024) 2024-09-10T07:40:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f1745400640 1 -- 172.21.15.46:0/3745696629 learned_addr learned my addr 172.21.15.46:0/3745696629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.798+0000 7f1745e00640 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb9200870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.799+0000 7f1744a00640 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.799+0000 7f1745e00640 1 -- 172.21.15.46:0/3745696629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 msgr2=0x55dcb91ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.799+0000 7f1745e00640 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91ff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.799+0000 7f1745e00640 1 -- 172.21.15.46:0/3745696629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 msgr2=0x55dcb91ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.799+0000 7f1745e00640 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.799+0000 7f1745e00640 1 -- 172.21.15.46:0/3745696629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcb91d9f90 con 0x55dcb91e4a60 2024-09-10T07:40:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.800+0000 7f1726a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:40:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.800+0000 7f1745e00640 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb9200870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ccee0 tx=0x7f172c09c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf10736605fc633 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.800+0000 7f1726a00640 1 -- 172.21.15.46:0/3745696629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0ccff0 con 0x55dcb91e4a60 2024-09-10T07:40:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.800+0000 7f1726a00640 1 -- 172.21.15.46:0/3745696629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f172c005310 con 0x55dcb91e4a60 2024-09-10T07:40:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.800+0000 7f174abd3740 1 -- 172.21.15.46:0/3745696629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1734020bb0 con 0x55dcb91e4a60 2024-09-10T07:40:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.801+0000 7f1726a00640 1 -- 172.21.15.46:0/3745696629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f172c0a4420 con 0x55dcb91e4a60 2024-09-10T07:40:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.802+0000 7f174abd3740 10 client.16167 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.802+0000 7f174abd3740 1 -- 172.21.15.46:0/3745696629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dcb9270830 con 0x55dcb91e4a60 2024-09-10T07:40:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.803+0000 7f1726a00640 1 -- 172.21.15.46:0/3745696629 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1734020bb0 con 0x55dcb91e4a60 2024-09-10T07:40:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.803+0000 7f1726a00640 1 -- 172.21.15.46:0/3745696629 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f172c0b4030 con 0x55dcb91e4a60 2024-09-10T07:40:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.803+0000 7f174abd3740 10 client.16167 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.803+0000 7f174abd3740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:40:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.803+0000 7f174abd3740 20 client.16167 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.803+0000 7f174abd3740 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dcb9271230 0x55dcb9273650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.803+0000 7f174abd3740 4 client.16167 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:40:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.803+0000 7f174abd3740 1 -- 172.21.15.46:0/3745696629 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcb914bb80 con 0x55dcb9271230 2024-09-10T07:40:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.804+0000 7f1745400640 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dcb9271230 0x55dcb9273650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.805+0000 7f1745400640 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dcb9271230 0x55dcb9273650 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28bae7936a89e3d server_cookie=836750e5da45252b in_seq=0 out_seq=0 2024-09-10T07:40:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.805+0000 7f1726a00640 10 client.16167 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.880+0000 7f1726a00640 1 -- 172.21.15.46:0/3745696629 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcb914bb80 con 0x55dcb9271230 2024-09-10T07:40:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.880+0000 7f1726a00640 10 client.16167 handle_command_reply: tid=0 2024-09-10T07:40:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.881+0000 7f174abd3740 1 client.16167 shutdown 2024-09-10T07:40:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.881+0000 7f174abd3740 1 -- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 msgr2=0x55dcb9200870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.881+0000 7f174abd3740 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb9200870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ccee0 tx=0x7f172c09c480 comp rx=0 tx=0).stop 2024-09-10T07:40:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f174abd3740 1 -- 172.21.15.46:0/3745696629 shutdown_connections 2024-09-10T07:40:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f174abd3740 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dcb9271230 0x55dcb9273650 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f174abd3740 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dcb91e4a60 0x55dcb9200870 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f174abd3740 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dcb91df9d0 0x55dcb91ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f1726a00640 0 client.16167 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f174abd3740 1 --2- 172.21.15.46:0/3745696629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb91df090 0x55dcb91ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f174abd3740 1 -- 172.21.15.46:0/3745696629 >> 172.21.15.46:0/3745696629 conn(0x55dcb91da700 msgr2=0x55dcb91d9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f174abd3740 1 -- 172.21.15.46:0/3745696629 shutdown_connections 2024-09-10T07:40:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.882+0000 7f174abd3740 1 -- 172.21.15.46:0/3745696629 wait complete. 2024-09-10T07:40:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.883+0000 7f174abd3740 20 client.16167 trim_cache size 0 max 16384 2024-09-10T07:40:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:40:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:40:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:40:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.883+0000 7f1747e00640 1 -- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17280a2d90 msgr2=0x7f17280a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.883+0000 7f1747e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17280a2d90 0x7f17280a5250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f174019f880 tx=0x7f1734014040 comp rx=0 tx=0).stop 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.883+0000 7f1747e00640 1 -- 172.21.15.46:0/2609832836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740105910 msgr2=0x7f17401a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.883+0000 7f1747e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740105910 0x7f17401a5920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f172c00e970 tx=0x7f172c00ed70 comp rx=0 tx=0).stop 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.884+0000 7f1747e00640 1 -- 172.21.15.46:0/2609832836 shutdown_connections 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.884+0000 7f1747e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1740105910 0x7f17401a5920 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.884+0000 7f1747e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1740104f50 0x7f174019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.884+0000 7f1747e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17280a2d90 0x7f17280a5250 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.884+0000 7f1747e00640 1 --2- 172.21.15.46:0/2609832836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1740103d60 0x7f174019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.884+0000 7f1747e00640 1 -- 172.21.15.46:0/2609832836 >> 172.21.15.46:0/2609832836 conn(0x7f17400ff4e0 msgr2=0x7f1740106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.884+0000 7f1747e00640 1 -- 172.21.15.46:0/2609832836 shutdown_connections 2024-09-10T07:40:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:40:53.884+0000 7f1747e00640 1 -- 172.21.15.46:0/2609832836 wait complete. 2024-09-10T07:40:53.906 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8ac40f1b', total seconds elapsed: 27.1 2024-09-10T07:40:53.906 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T07:40:53.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:40:54.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:53 smithi181 ceph-mon[35397]: pgmap v527: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 257 op/s 2024-09-10T07:40:54.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:53 smithi063 ceph-mon[35020]: pgmap v527: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 257 op/s 2024-09-10T07:40:54.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:53 smithi046 ceph-mon[30705]: pgmap v527: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 257 op/s 2024-09-10T07:40:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.218+0000 7fd499e00640 1 -- 172.21.15.46:0/532229703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd494112a30 msgr2=0x7fd494114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.218+0000 7fd499e00640 1 --2- 172.21.15.46:0/532229703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd494112a30 0x7fd494114e20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009ca0 tx=0x7fd48c02a8b0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.219+0000 7fd499e00640 1 -- 172.21.15.46:0/532229703 shutdown_connections 2024-09-10T07:40:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.219+0000 7fd499e00640 1 --2- 172.21.15.46:0/532229703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd494112a30 0x7fd494114e20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.219+0000 7fd499e00640 1 --2- 172.21.15.46:0/532229703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd494077040 0x7fd4941124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.219+0000 7fd499e00640 1 --2- 172.21.15.46:0/532229703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd494076720 0x7fd494076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.219+0000 7fd499e00640 1 -- 172.21.15.46:0/532229703 >> 172.21.15.46:0/532229703 conn(0x7fd4940ffa90 msgr2=0x7fd494101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.221+0000 7fd499e00640 1 -- 172.21.15.46:0/532229703 shutdown_connections 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.222+0000 7fd499e00640 1 -- 172.21.15.46:0/532229703 wait complete. 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.224+0000 7fd499e00640 1 Processor -- start 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.228+0000 7fd499e00640 1 -- start start 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.230+0000 7fd499e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd494076720 0x7fd494111fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.230+0000 7fd499e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd494077040 0x7fd4941124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.231+0000 7fd492000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd494077040 0x7fd4941124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.231+0000 7fd492000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd494077040 0x7fd4941124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38044/0 (socket says 172.21.15.46:38044) 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.231+0000 7fd492000640 1 -- 172.21.15.46:0/1371398813 learned_addr learned my addr 172.21.15.46:0/1371398813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.231+0000 7fd499e00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd494112a30 0x7fd49410b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.232+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd494118570 con 0x7fd494076720 2024-09-10T07:40:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.232+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941183f0 con 0x7fd494112a30 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.232+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941186f0 con 0x7fd494077040 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.232+0000 7fd493400640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd494112a30 0x7fd49410b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.232+0000 7fd492a00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd494076720 0x7fd494111fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.233+0000 7fd492a00640 1 -- 172.21.15.46:0/1371398813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd494077040 msgr2=0x7fd4941124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.233+0000 7fd492a00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd494077040 0x7fd4941124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.233+0000 7fd492a00640 1 -- 172.21.15.46:0/1371398813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd494112a30 msgr2=0x7fd49410b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.233+0000 7fd492a00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd494112a30 0x7fd49410b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.233+0000 7fd492a00640 1 -- 172.21.15.46:0/1371398813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49410bad0 con 0x7fd494076720 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.235+0000 7fd492a00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd494076720 0x7fd494111fa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d6d0 tx=0x7fd48400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.238+0000 7fd498a00640 1 -- 172.21.15.46:0/1371398813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484028070 con 0x7fd494076720 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.238+0000 7fd498a00640 1 -- 172.21.15.46:0/1371398813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd484009820 con 0x7fd494076720 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.238+0000 7fd498a00640 1 -- 172.21.15.46:0/1371398813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484004d00 con 0x7fd494076720 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.238+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49410bdc0 con 0x7fd494076720 2024-09-10T07:40:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.238+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49410fd60 con 0x7fd494076720 2024-09-10T07:40:55.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.245+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd474005290 con 0x7fd494076720 2024-09-10T07:40:55.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.247+0000 7fd498a00640 1 -- 172.21.15.46:0/1371398813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd484005220 con 0x7fd494076720 2024-09-10T07:40:55.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.249+0000 7fd498a00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4780a2cd0 0x7fd4780a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:55.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.249+0000 7fd498a00640 1 -- 172.21.15.46:0/1371398813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd4840dfb00 con 0x7fd494076720 2024-09-10T07:40:55.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.256+0000 7fd492000640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4780a2cd0 0x7fd4780a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:55.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.260+0000 7fd492000640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4780a2cd0 0x7fd4780a5190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd49410c7f0 tx=0x7fd48000b630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:55.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.268+0000 7fd498a00640 1 -- 172.21.15.46:0/1371398813 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd484031030 con 0x7fd494076720 2024-09-10T07:40:55.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 215.0 s, 713.3 tps, lat 519.369 ms stddev 2588.515 2024-09-10T07:40:55.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.685+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd474005580 con 0x7fd494076720 2024-09-10T07:40:55.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.692+0000 7fd498a00640 1 -- 172.21.15.46:0/1371398813 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11975 (secure 0 0 0) 0x7fd4840ac760 con 0x7fd494076720 2024-09-10T07:40:55.696 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:40:55.696 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 39 2024-09-10T07:40:55.696 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"btime":"2024-09-10T07:40:35:758945+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:40:35.499710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:40:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.696+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4780a2cd0 msgr2=0x7fd4780a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.696+0000 7fd499e00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4780a2cd0 0x7fd4780a5190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd49410c7f0 tx=0x7fd48000b630 comp rx=0 tx=0).stop 2024-09-10T07:40:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.696+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd494076720 msgr2=0x7fd494111fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.696+0000 7fd499e00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd494076720 0x7fd494111fa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d6d0 tx=0x7fd48400db90 comp rx=0 tx=0).stop 2024-09-10T07:40:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.697+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 shutdown_connections 2024-09-10T07:40:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.697+0000 7fd499e00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd494112a30 0x7fd49410b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.697+0000 7fd499e00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd494077040 0x7fd4941124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.697+0000 7fd499e00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd4780a2cd0 0x7fd4780a5190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.697+0000 7fd499e00640 1 --2- 172.21.15.46:0/1371398813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd494076720 0x7fd494111fa0 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d6d0 tx=0x7fd48400db90 comp rx=0 tx=0).stop 2024-09-10T07:40:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.697+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 >> 172.21.15.46:0/1371398813 conn(0x7fd4940ffa90 msgr2=0x7fd494101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.697+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 shutdown_connections 2024-09-10T07:40:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:55.697+0000 7fd499e00640 1 -- 172.21.15.46:0/1371398813 wait complete. 2024-09-10T07:40:56.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:55 smithi181 ceph-mon[35397]: pgmap v528: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 229 op/s 2024-09-10T07:40:56.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:55 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1371398813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:56.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:55 smithi063 ceph-mon[35020]: pgmap v528: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 229 op/s 2024-09-10T07:40:56.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:55 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1371398813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:56.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:55 smithi046 ceph-mon[30705]: pgmap v528: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 229 op/s 2024-09-10T07:40:56.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:55 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1371398813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:40:57.983 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T07:40:57.983 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T07:40:58.084 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:57 smithi046 ceph-mon[30705]: pgmap v529: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.8 MiB/s wr, 260 op/s 2024-09-10T07:40:58.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:57 smithi181 ceph-mon[35397]: pgmap v529: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.8 MiB/s wr, 260 op/s 2024-09-10T07:40:58.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:57 smithi063 ceph-mon[35020]: pgmap v529: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.8 MiB/s wr, 260 op/s 2024-09-10T07:40:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.420+0000 7f3153e00640 1 -- 172.21.15.46:0/54663180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3154102b90 msgr2=0x7f3154102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.420+0000 7f3153e00640 1 --2- 172.21.15.46:0/54663180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3154102b90 0x7f3154102f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3144009900 tx=0x7f314402a480 comp rx=0 tx=0).stop 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.420+0000 7f3153e00640 1 -- 172.21.15.46:0/54663180 shutdown_connections 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.420+0000 7f3153e00640 1 --2- 172.21.15.46:0/54663180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315410bfc0 0x7f315410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.420+0000 7f3153e00640 1 --2- 172.21.15.46:0/54663180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31541034b0 0x7f315410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.420+0000 7f3153e00640 1 --2- 172.21.15.46:0/54663180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3154102b90 0x7f3154102f70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.420+0000 7f3153e00640 1 -- 172.21.15.46:0/54663180 >> 172.21.15.46:0/54663180 conn(0x7f31540fc610 msgr2=0x7f31540fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.421+0000 7f3153e00640 1 -- 172.21.15.46:0/54663180 shutdown_connections 2024-09-10T07:40:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.421+0000 7f3153e00640 1 -- 172.21.15.46:0/54663180 wait complete. 2024-09-10T07:40:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.423+0000 7f3153e00640 1 Processor -- start 2024-09-10T07:40:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.423+0000 7f3153e00640 1 -- start start 2024-09-10T07:40:59.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.424+0000 7f3153e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3154102b90 0x7f31541007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.425+0000 7f3152a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3154102b90 0x7f31541007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.425+0000 7f3153e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31541034b0 0x7f3154100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.425+0000 7f3153e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315410bfc0 0x7f3154101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.425+0000 7f3153e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3154111e10 con 0x7f315410bfc0 2024-09-10T07:40:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.425+0000 7f3153e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3154111c90 con 0x7f3154102b90 2024-09-10T07:40:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.425+0000 7f3153e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3154111f90 con 0x7f31541034b0 2024-09-10T07:40:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.425+0000 7f3152a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3154102b90 0x7f31541007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55080/0 (socket says 172.21.15.46:55080) 2024-09-10T07:40:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.425+0000 7f3152a00640 1 -- 172.21.15.46:0/482957168 learned_addr learned my addr 172.21.15.46:0/482957168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.427+0000 7f3153400640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315410bfc0 0x7f3154101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.428+0000 7f3153400640 1 -- 172.21.15.46:0/482957168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31541034b0 msgr2=0x7f3154100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.428+0000 7f3153400640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31541034b0 0x7f3154100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.428+0000 7f3153400640 1 -- 172.21.15.46:0/482957168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3154102b90 msgr2=0x7f31541007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.428+0000 7f3153400640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3154102b90 0x7f31541007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.428+0000 7f3153400640 1 -- 172.21.15.46:0/482957168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31541a66f0 con 0x7f315410bfc0 2024-09-10T07:40:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.428+0000 7f3152a00640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3154102b90 0x7f31541007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:40:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.429+0000 7f3153400640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315410bfc0 0x7f3154101270 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f314c00cab0 tx=0x7f314c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:59.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.432+0000 7f3158a00640 1 -- 172.21.15.46:0/482957168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c00ae20 con 0x7f315410bfc0 2024-09-10T07:40:59.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.432+0000 7f3153e00640 1 -- 172.21.15.46:0/482957168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31541a69e0 con 0x7f315410bfc0 2024-09-10T07:40:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.432+0000 7f3153e00640 1 -- 172.21.15.46:0/482957168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31541a6f20 con 0x7f315410bfc0 2024-09-10T07:40:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.432+0000 7f3158a00640 1 -- 172.21.15.46:0/482957168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314c00f930 con 0x7f315410bfc0 2024-09-10T07:40:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.432+0000 7f3158a00640 1 -- 172.21.15.46:0/482957168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c018b30 con 0x7f315410bfc0 2024-09-10T07:40:59.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.440+0000 7f315c8ad740 1 Processor -- start 2024-09-10T07:40:59.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.440+0000 7f315c8ad740 1 -- start start 2024-09-10T07:40:59.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.441+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c21216c0 0x5590c2121aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.442+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c212c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.443+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 0x5590c212ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.443+0000 7f3152000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c21216c0 0x5590c2121aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.443+0000 7f315c8ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5590c2086260 con 0x5590c212c8a0 2024-09-10T07:40:59.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.443+0000 7f315c8ad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590c202d500 con 0x5590c2121fe0 2024-09-10T07:40:59.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.443+0000 7f315c8ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5590c2082220 con 0x5590c21216c0 2024-09-10T07:40:59.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.444+0000 7f3152000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 0x5590c212ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.444+0000 7f3152000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 0x5590c212ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36108/0 (socket says 172.21.15.46:36108) 2024-09-10T07:40:59.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.444+0000 7f3152000640 1 -- 172.21.15.46:0/1200873117 learned_addr learned my addr 172.21.15.46:0/1200873117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:59.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.445+0000 7f3152000640 1 -- 172.21.15.46:0/1200873117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 msgr2=0x5590c212c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:59.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.445+0000 7f3152000640 1 --2- 172.21.15.46:0/1200873117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c212c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.445+0000 7f3152000640 1 -- 172.21.15.46:0/1200873117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 msgr2=0x5590c212ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:59.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.445+0000 7f3152000640 1 --2- 172.21.15.46:0/1200873117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 0x5590c212ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.445+0000 7f3152000640 1 -- 172.21.15.46:0/1200873117 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c1df7ba0 con 0x5590c21216c0 2024-09-10T07:40:59.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.446+0000 7f3152000640 1 --2- 172.21.15.46:0/1200873117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c21216c0 0x5590c2121aa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3140013760 tx=0x7f3140009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed2db92ef527202 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:59.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.446+0000 7f313fe00640 1 -- 172.21.15.46:0/1200873117 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140004230 con 0x5590c21216c0 2024-09-10T07:40:59.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.447+0000 7f313fe00640 1 -- 172.21.15.46:0/1200873117 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31400043d0 con 0x5590c21216c0 2024-09-10T07:40:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.448+0000 7f315c8ad740 1 -- 172.21.15.46:0/1200873117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c21216c0 msgr2=0x5590c2121aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.448+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1200873117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c21216c0 0x5590c2121aa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3140013760 tx=0x7f3140009420 comp rx=0 tx=0).stop 2024-09-10T07:40:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.448+0000 7f315c8ad740 1 -- 172.21.15.46:0/1200873117 shutdown_connections 2024-09-10T07:40:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.448+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1200873117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 0x5590c212ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.448+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1200873117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c212c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.448+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1200873117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c21216c0 0x5590c2121aa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.448+0000 7f315c8ad740 1 -- 172.21.15.46:0/1200873117 >> 172.21.15.46:0/1200873117 conn(0x5590c211dc50 msgr2=0x5590c211e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.449+0000 7f315c8ad740 1 -- 172.21.15.46:0/1200873117 shutdown_connections 2024-09-10T07:40:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.452+0000 7f3158a00640 1 -- 172.21.15.46:0/482957168 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f314c00fae0 con 0x7f315410bfc0 2024-09-10T07:40:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.452+0000 7f315c8ad740 1 -- 172.21.15.46:0/1200873117 wait complete. 2024-09-10T07:40:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.453+0000 7f315c8ad740 1 -- start start 2024-09-10T07:40:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.453+0000 7f315c8ad740 1 Processor -- start 2024-09-10T07:40:59.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.457+0000 7f3158a00640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31380a2ac0 0x7f31380a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.457+0000 7f3158a00640 1 -- 172.21.15.46:0/482957168 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f314c0cf8a0 con 0x7f315410bfc0 2024-09-10T07:40:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.459+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c212c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.459+0000 7f3152000640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31380a2ac0 0x7f31380a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.459+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c212c8a0 0x5590c2129440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.459+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2129dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.460+0000 7f3152000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c212c8a0 0x5590c2129440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.460+0000 7f3153400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2129dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.460+0000 7f315c8ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5590c212f510 con 0x5590c2121fe0 2024-09-10T07:40:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.460+0000 7f315c8ad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590c2040bd0 con 0x5590c212c8a0 2024-09-10T07:40:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.460+0000 7f315c8ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5590c212f690 con 0x5590c2129980 2024-09-10T07:40:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.460+0000 7f3153400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2129dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:42564/0 (socket says 172.21.15.46:42564) 2024-09-10T07:40:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.460+0000 7f3153400640 1 -- 172.21.15.46:0/4241125998 learned_addr learned my addr 172.21.15.46:0/4241125998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.460+0000 7f315c8ad740 10 client.0 _ll_register_callbacks cb 0x5590c1d950a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:59.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.462+0000 7f3152a00640 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c212c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.463+0000 7f3153400640 1 -- 172.21.15.46:0/4241125998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c212c8a0 msgr2=0x5590c2129440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.463+0000 7f3153400640 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c212c8a0 0x5590c2129440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.463+0000 7f3153400640 1 -- 172.21.15.46:0/4241125998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 msgr2=0x5590c212c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.463+0000 7f3153400640 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c212c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.463+0000 7f3153400640 1 -- 172.21.15.46:0/4241125998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c2138b60 con 0x5590c2129980 2024-09-10T07:40:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.463+0000 7f3153400640 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2129dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f314c00a6e0 tx=0x7f314c09c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf6cec3eb4516 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.463+0000 7f313cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.464+0000 7f313cc00640 1 -- 172.21.15.46:0/4241125998 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c0a49f0 con 0x5590c2129980 2024-09-10T07:40:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.464+0000 7f3152000640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31380a2ac0 0x7f31380a4f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f31400099f0 tx=0x7f3140009c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.464+0000 7f313cc00640 1 -- 172.21.15.46:0/4241125998 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f314c0a4560 con 0x5590c2129980 2024-09-10T07:40:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.464+0000 7f315c8ad740 1 -- 172.21.15.46:0/4241125998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590c2093990 con 0x5590c2129980 2024-09-10T07:40:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.466+0000 7f313cc00640 1 -- 172.21.15.46:0/4241125998 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f314c0b9050 con 0x5590c2129980 2024-09-10T07:40:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.466+0000 7f315c8ad740 10 client.26446 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.466+0000 7f315c8ad740 1 -- 172.21.15.46:0/4241125998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5590c213b5e0 con 0x5590c2129980 2024-09-10T07:40:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.467+0000 7f313cc00640 1 -- 172.21.15.46:0/4241125998 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5590c2093990 con 0x5590c2129980 2024-09-10T07:40:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.467+0000 7f313cc00640 1 -- 172.21.15.46:0/4241125998 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f314c09b1d0 con 0x5590c2129980 2024-09-10T07:40:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.467+0000 7f315c8ad740 10 client.26446 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.467+0000 7f315c8ad740 10 client.26446 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:40:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.467+0000 7f315c8ad740 20 client.26446 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.467+0000 7f315c8ad740 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c213caa0 0x5590c213ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.467+0000 7f315c8ad740 4 client.26446 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:40:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.467+0000 7f315c8ad740 1 -- 172.21.15.46:0/4241125998 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590c208dce0 con 0x5590c213caa0 2024-09-10T07:40:59.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.471+0000 7f3152a00640 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c213caa0 0x5590c213ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.472+0000 7f3152a00640 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c213caa0 0x5590c213ee90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175bf8edd0f81869 server_cookie=a6c92d52a85b099e in_seq=0 out_seq=0 2024-09-10T07:40:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.472+0000 7f313cc00640 10 client.26446 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f313cc00640 1 -- 172.21.15.46:0/4241125998 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590c208dce0 con 0x5590c213caa0 2024-09-10T07:40:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f313cc00640 10 client.26446 handle_command_reply: tid=0 2024-09-10T07:40:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 client.26446 shutdown 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 -- 172.21.15.46:0/4241125998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 msgr2=0x5590c2129dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2129dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f314c00a6e0 tx=0x7f314c09c540 comp rx=0 tx=0).stop 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 -- 172.21.15.46:0/4241125998 shutdown_connections 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c213caa0 0x5590c213ee90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2129dc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c212c8a0 0x5590c2129440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 --2- 172.21.15.46:0/4241125998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c212c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f313cc00640 0 client.26446 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.474+0000 7f315c8ad740 1 -- 172.21.15.46:0/4241125998 >> 172.21.15.46:0/4241125998 conn(0x5590c211dc50 msgr2=0x5590c2123190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.475+0000 7f315c8ad740 1 -- 172.21.15.46:0/4241125998 shutdown_connections 2024-09-10T07:40:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.475+0000 7f315c8ad740 1 -- 172.21.15.46:0/4241125998 wait complete. 2024-09-10T07:40:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.475+0000 7f315c8ad740 20 client.26446 trim_cache size 0 max 16384 2024-09-10T07:40:59.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.522+0000 7f315c8ad740 1 Processor -- start 2024-09-10T07:40:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.526+0000 7f315c8ad740 1 -- start start 2024-09-10T07:40:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.526+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c2124840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.526+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.526+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 0x5590c2128250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.526+0000 7f315c8ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f313400fc80 con 0x5590c212c8a0 2024-09-10T07:40:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.526+0000 7f315c8ad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3134011820 con 0x5590c2121fe0 2024-09-10T07:40:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.526+0000 7f315c8ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3134010350 con 0x5590c2129980 2024-09-10T07:40:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.526+0000 7f3152a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c2124840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.527+0000 7f3152a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c2124840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55092/0 (socket says 172.21.15.46:55092) 2024-09-10T07:40:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.527+0000 7f3152a00640 1 -- 172.21.15.46:0/2456032542 learned_addr learned my addr 172.21.15.46:0/2456032542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.527+0000 7f3152000640 1 --2- 172.21.15.46:0/2456032542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.527+0000 7f3152a00640 1 -- 172.21.15.46:0/2456032542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 msgr2=0x5590c2127d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.527+0000 7f3152a00640 1 --2- 172.21.15.46:0/2456032542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2127d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.527+0000 7f3152a00640 1 -- 172.21.15.46:0/2456032542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 msgr2=0x5590c2128250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.527+0000 7f3152a00640 1 --2- 172.21.15.46:0/2456032542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 0x5590c2128250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.527+0000 7f3152a00640 1 -- 172.21.15.46:0/2456032542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c213f200 con 0x5590c2121fe0 2024-09-10T07:40:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.528+0000 7f3152a00640 1 --2- 172.21.15.46:0/2456032542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c2124840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f314402aa40 tx=0x7f3144005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c289f1fed2b8871a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.528+0000 7f313f400640 1 -- 172.21.15.46:0/2456032542 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144035040 con 0x5590c2121fe0 2024-09-10T07:40:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.528+0000 7f313f400640 1 -- 172.21.15.46:0/2456032542 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3144031bd0 con 0x5590c2121fe0 2024-09-10T07:40:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.528+0000 7f313f400640 1 -- 172.21.15.46:0/2456032542 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144045370 con 0x5590c2121fe0 2024-09-10T07:40:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.529+0000 7f315c8ad740 1 -- 172.21.15.46:0/2456032542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 msgr2=0x5590c2124840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.529+0000 7f315c8ad740 1 --2- 172.21.15.46:0/2456032542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c2124840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f314402aa40 tx=0x7f3144005d50 comp rx=0 tx=0).stop 2024-09-10T07:40:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.529+0000 7f315c8ad740 1 -- 172.21.15.46:0/2456032542 shutdown_connections 2024-09-10T07:40:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.529+0000 7f315c8ad740 1 --2- 172.21.15.46:0/2456032542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c212c8a0 0x5590c2128250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.529+0000 7f315c8ad740 1 --2- 172.21.15.46:0/2456032542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c2129980 0x5590c2127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.529+0000 7f315c8ad740 1 --2- 172.21.15.46:0/2456032542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2121fe0 0x5590c2124840 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.529+0000 7f315c8ad740 1 -- 172.21.15.46:0/2456032542 >> 172.21.15.46:0/2456032542 conn(0x5590c211dc50 msgr2=0x5590c211e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.530+0000 7f315c8ad740 1 -- 172.21.15.46:0/2456032542 shutdown_connections 2024-09-10T07:40:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.530+0000 7f315c8ad740 1 -- 172.21.15.46:0/2456032542 wait complete. 2024-09-10T07:40:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.530+0000 7f315c8ad740 1 -- start start 2024-09-10T07:40:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.531+0000 7f315c8ad740 1 Processor -- start 2024-09-10T07:40:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.531+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c21349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.531+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2129980 0x5590c2134f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.531+0000 7f315c8ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c212c8a0 0x5590c21256b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.531+0000 7f315c8ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3134011150 con 0x5590c2121fe0 2024-09-10T07:40:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.531+0000 7f315c8ad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3134010a80 con 0x5590c2129980 2024-09-10T07:40:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.531+0000 7f315c8ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f313400dc60 con 0x5590c212c8a0 2024-09-10T07:40:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.531+0000 7f315c8ad740 10 client.0 _ll_register_callbacks cb 0x7f314c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:40:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.532+0000 7f3153400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c212c8a0 0x5590c21256b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.532+0000 7f3153400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c212c8a0 0x5590c21256b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:42596/0 (socket says 172.21.15.46:42596) 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.532+0000 7f3153400640 1 -- 172.21.15.46:0/1901203022 learned_addr learned my addr 172.21.15.46:0/1901203022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.532+0000 7f3152000640 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2129980 0x5590c2134f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.532+0000 7f3152a00640 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c21349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f3153400640 1 -- 172.21.15.46:0/1901203022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2129980 msgr2=0x5590c2134f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f3153400640 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2129980 0x5590c2134f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f3153400640 1 -- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 msgr2=0x5590c21349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f3153400640 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c21349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f3153400640 1 -- 172.21.15.46:0/1901203022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c2125bf0 con 0x5590c212c8a0 2024-09-10T07:40:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f3152a00640 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c21349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:40:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f3153400640 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c212c8a0 0x5590c21256b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f314c09ba80 tx=0x7f314c09bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b470f2c5f819e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:40:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f313cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:40:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f313cc00640 1 -- 172.21.15.46:0/1901203022 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c09b8c0 con 0x5590c212c8a0 2024-09-10T07:40:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.533+0000 7f313cc00640 1 -- 172.21.15.46:0/1901203022 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f314c004b40 con 0x5590c212c8a0 2024-09-10T07:40:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.534+0000 7f315c8ad740 1 -- 172.21.15.46:0/1901203022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3144031bd0 con 0x5590c212c8a0 2024-09-10T07:40:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.534+0000 7f313cc00640 1 -- 172.21.15.46:0/1901203022 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f314c0a49f0 con 0x5590c212c8a0 2024-09-10T07:40:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.534+0000 7f315c8ad740 10 client.26452 fetch_fsmap learned FSMap version 39 2024-09-10T07:40:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.534+0000 7f315c8ad740 1 -- 172.21.15.46:0/1901203022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5590c2126ed0 con 0x5590c212c8a0 2024-09-10T07:40:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.534+0000 7f313cc00640 1 -- 172.21.15.46:0/1901203022 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3144031bd0 con 0x5590c212c8a0 2024-09-10T07:40:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.536+0000 7f313cc00640 1 -- 172.21.15.46:0/1901203022 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f314c09ac40 con 0x5590c212c8a0 2024-09-10T07:40:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.536+0000 7f315c8ad740 10 client.26452 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:40:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.536+0000 7f315c8ad740 10 client.26452 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:40:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.536+0000 7f315c8ad740 20 client.26452 populate_metadata read hostname 'smithi046' 2024-09-10T07:40:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.536+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c21b0e60 0x5590c21b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:40:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.536+0000 7f315c8ad740 4 client.26452 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T07:40:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.536+0000 7f315c8ad740 1 -- 172.21.15.46:0/1901203022 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x5590c2009730 con 0x5590c21b0e60 2024-09-10T07:40:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.536+0000 7f3152a00640 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c21b0e60 0x5590c21b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:40:59.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.537+0000 7f3152a00640 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c21b0e60 0x5590c21b3250 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39079c64e13ba1fe server_cookie=26d68182fe4a314 in_seq=0 out_seq=0 2024-09-10T07:40:59.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.537+0000 7f313cc00640 10 client.26452 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.733+0000 7f313cc00640 1 -- 172.21.15.46:0/1901203022 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5590c2009730 con 0x5590c21b0e60 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.733+0000 7f313cc00640 10 client.26452 handle_command_reply: tid=0 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.733+0000 7f315c8ad740 1 client.26452 shutdown 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.733+0000 7f315c8ad740 1 -- 172.21.15.46:0/1901203022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c212c8a0 msgr2=0x5590c21256b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.733+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c212c8a0 0x5590c21256b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f314c09ba80 tx=0x7f314c09bc50 comp rx=0 tx=0).stop 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.733+0000 7f315c8ad740 1 -- 172.21.15.46:0/1901203022 shutdown_connections 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.733+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c21b0e60 0x5590c21b3250 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.734+0000 7f313cc00640 0 client.26452 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.733+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c212c8a0 0x5590c21256b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.734+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c2129980 0x5590c2134f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.734+0000 7f315c8ad740 1 --2- 172.21.15.46:0/1901203022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c2121fe0 0x5590c21349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.734+0000 7f315c8ad740 1 -- 172.21.15.46:0/1901203022 >> 172.21.15.46:0/1901203022 conn(0x5590c211dc50 msgr2=0x5590c2127c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.734+0000 7f315c8ad740 1 -- 172.21.15.46:0/1901203022 shutdown_connections 2024-09-10T07:40:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.734+0000 7f315c8ad740 1 -- 172.21.15.46:0/1901203022 wait complete. 2024-09-10T07:40:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.734+0000 7f315c8ad740 20 client.26452 trim_cache size 0 max 16384 2024-09-10T07:40:59.736 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 39, 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 14, 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "93c5c7e4": { 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 14, 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:40:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 142.8, 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:59.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:40:59.739 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:40:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 -- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31380a2ac0 msgr2=0x7f31380a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31380a2ac0 0x7f31380a4f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f31400099f0 tx=0x7f3140009c40 comp rx=0 tx=0).stop 2024-09-10T07:40:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 -- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315410bfc0 msgr2=0x7f3154101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:40:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315410bfc0 0x7f3154101270 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f314c00cab0 tx=0x7f314c00cf70 comp rx=0 tx=0).stop 2024-09-10T07:40:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 -- 172.21.15.46:0/482957168 shutdown_connections 2024-09-10T07:40:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31380a2ac0 0x7f31380a4f80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315410bfc0 0x7f3154101270 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31541034b0 0x7f3154100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 --2- 172.21.15.46:0/482957168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3154102b90 0x7f31541007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:40:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 -- 172.21.15.46:0/482957168 >> 172.21.15.46:0/482957168 conn(0x7f31540fc610 msgr2=0x7f31540fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:40:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.735+0000 7f3153e00640 1 -- 172.21.15.46:0/482957168 shutdown_connections 2024-09-10T07:40:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:40:59.736+0000 7f3153e00640 1 -- 172.21.15.46:0/482957168 wait complete. 2024-09-10T07:41:00.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:59 smithi181 ceph-mon[35397]: pgmap v530: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-10T07:41:00.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:40:59 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:41:00.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:59 smithi063 ceph-mon[35020]: pgmap v530: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-10T07:41:00.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:40:59 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:41:00.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:59 smithi046 ceph-mon[30705]: pgmap v530: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-10T07:41:00.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:40:59 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:41:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 220.0 s, 916.1 tps, lat 83.853 ms stddev 103.159 2024-09-10T07:41:01.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:41:01.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:41:01.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:00 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:41:01.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:41:01.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:41:01.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:00 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:41:01.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:41:01.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:41:01.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:00 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:41:01.622 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '93c5c7e4', quiesce took 7.7 seconds. Will release after: 15.05 2024-09-10T07:41:02.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:01 smithi181 ceph-mon[35397]: pgmap v531: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-10T07:41:02.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:01 smithi063 ceph-mon[35020]: pgmap v531: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-10T07:41:02.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:01 smithi046 ceph-mon[30705]: pgmap v531: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-10T07:41:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:03 smithi181 ceph-mon[35397]: pgmap v532: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-09-10T07:41:04.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:03 smithi063 ceph-mon[35020]: pgmap v532: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-09-10T07:41:04.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:03 smithi046 ceph-mon[30705]: pgmap v532: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-09-10T07:41:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 225.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:41:06.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:05 smithi181 ceph-mon[35397]: pgmap v533: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-09-10T07:41:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:05 smithi063 ceph-mon[35020]: pgmap v533: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-09-10T07:41:06.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:05 smithi046 ceph-mon[30705]: pgmap v533: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-09-10T07:41:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 -- 172.21.15.46:0/1258657349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c10e110 msgr2=0x7ff69c110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 --2- 172.21.15.46:0/1258657349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c10e110 0x7ff69c110500 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff694009be0 tx=0x7ff69402a6d0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 -- 172.21.15.46:0/1258657349 shutdown_connections 2024-09-10T07:41:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 --2- 172.21.15.46:0/1258657349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c10e110 0x7ff69c110500 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 --2- 172.21.15.46:0/1258657349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff69c074c50 0x7ff69c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 --2- 172.21.15.46:0/1258657349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff69c076230 0x7ff69c074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 -- 172.21.15.46:0/1258657349 >> 172.21.15.46:0/1258657349 conn(0x7ff69c0fd2e0 msgr2=0x7ff69c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 -- 172.21.15.46:0/1258657349 shutdown_connections 2024-09-10T07:41:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.024+0000 7ff6a1000640 1 -- 172.21.15.46:0/1258657349 wait complete. 2024-09-10T07:41:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.025+0000 7ff6a1000640 1 Processor -- start 2024-09-10T07:41:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.025+0000 7ff6a1000640 1 -- start start 2024-09-10T07:41:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.025+0000 7ff6a1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c074c50 0x7ff69c19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.025+0000 7ff6a1000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff69c076230 0x7ff69c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.025+0000 7ff6a1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff69c10e110 0x7ff69c1a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.025+0000 7ff6a1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c113d80 con 0x7ff69c074c50 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff6a1000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c113c00 con 0x7ff69c10e110 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff6a1000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c113f00 con 0x7ff69c076230 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c074c50 0x7ff69c19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c074c50 0x7ff69c19e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48436/0 (socket says 172.21.15.46:48436) 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 -- 172.21.15.46:0/2080237477 learned_addr learned my addr 172.21.15.46:0/2080237477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699400640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff69c076230 0x7ff69c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff69a800640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff69c10e110 0x7ff69c1a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 -- 172.21.15.46:0/2080237477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff69c076230 msgr2=0x7ff69c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff69c076230 0x7ff69c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 -- 172.21.15.46:0/2080237477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff69c10e110 msgr2=0x7ff69c1a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff69c10e110 0x7ff69c1a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 -- 172.21.15.46:0/2080237477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c1a5f60 con 0x7ff69c074c50 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff69a800640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff69c10e110 0x7ff69c1a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.026+0000 7ff699e00640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c074c50 0x7ff69c19e340 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00e9f0 tx=0x7ff68c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.027+0000 7ff692a00640 1 -- 172.21.15.46:0/2080237477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c00cd80 con 0x7ff69c074c50 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.027+0000 7ff692a00640 1 -- 172.21.15.46:0/2080237477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff68c010910 con 0x7ff69c074c50 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.027+0000 7ff692a00640 1 -- 172.21.15.46:0/2080237477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c018990 con 0x7ff69c074c50 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.027+0000 7ff6a1000640 1 -- 172.21.15.46:0/2080237477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c1a6250 con 0x7ff69c074c50 2024-09-10T07:41:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.027+0000 7ff6a1000640 1 -- 172.21.15.46:0/2080237477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a67a0 con 0x7ff69c074c50 2024-09-10T07:41:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.031+0000 7ff6a400f740 1 Processor -- start 2024-09-10T07:41:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.031+0000 7ff6a400f740 1 -- start start 2024-09-10T07:41:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.031+0000 7ff692a00640 1 -- 172.21.15.46:0/2080237477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff68c010ac0 con 0x7ff69c074c50 2024-09-10T07:41:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd79090 0x5574bfd79490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd7e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd80f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff6a400f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5574bfcde100 con 0x5574bfd799d0 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff6a400f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5574bfc861b0 con 0x5574bfd79090 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff6a400f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5574bfcda0c0 con 0x5574bfd7ea60 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd7e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd7e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48448/0 (socket says 172.21.15.46:48448) 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699e00640 1 -- 172.21.15.46:0/4129051516 learned_addr learned my addr 172.21.15.46:0/4129051516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff69a800640 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd79090 0x5574bfd79490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699400640 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd80f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699e00640 1 -- 172.21.15.46:0/4129051516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 msgr2=0x5574bfd80f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699e00640 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd80f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699e00640 1 -- 172.21.15.46:0/4129051516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd79090 msgr2=0x5574bfd79490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699e00640 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd79090 0x5574bfd79490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.032+0000 7ff699e00640 1 -- 172.21.15.46:0/4129051516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574bf9f1210 con 0x5574bfd799d0 2024-09-10T07:41:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.033+0000 7ff699e00640 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd7e490 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0d5c10 tx=0x7ff68c0d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6305e12a21ed41f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.033+0000 7ff687400640 1 -- 172.21.15.46:0/4129051516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0de070 con 0x5574bfd799d0 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.033+0000 7ff687400640 1 -- 172.21.15.46:0/4129051516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff68c0dab20 con 0x5574bfd799d0 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.033+0000 7ff687400640 1 -- 172.21.15.46:0/4129051516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0d1870 con 0x5574bfd799d0 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.033+0000 7ff692a00640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff6800a2c30 0x7ff6800a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.033+0000 7ff692a00640 1 -- 172.21.15.46:0/2080237477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff68c0cf780 con 0x7ff69c074c50 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff69a800640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff6800a2c30 0x7ff6800a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 -- 172.21.15.46:0/4129051516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 msgr2=0x5574bfd7e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd7e490 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0d5c10 tx=0x7ff68c0d7690 comp rx=0 tx=0).stop 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 -- 172.21.15.46:0/4129051516 shutdown_connections 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd80f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd7e490 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 --2- 172.21.15.46:0/4129051516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd79090 0x5574bfd79490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 -- 172.21.15.46:0/4129051516 >> 172.21.15.46:0/4129051516 conn(0x5574bfd74700 msgr2=0x5574bfd74b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 -- 172.21.15.46:0/4129051516 shutdown_connections 2024-09-10T07:41:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff69a800640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff6800a2c30 0x7ff6800a50f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff694005cc0 tx=0x7ff694030a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.034+0000 7ff6a400f740 1 -- 172.21.15.46:0/4129051516 wait complete. 2024-09-10T07:41:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 1 -- start start 2024-09-10T07:41:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 1 Processor -- start 2024-09-10T07:41:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd79090 0x5574bfd83250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd799d0 0x5574bfd83790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd8a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5574bfd81a30 con 0x5574bfd79090 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5574bfc67090 con 0x5574bfd799d0 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5574bfd81bb0 con 0x5574bfd7ea60 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.035+0000 7ff6a400f740 10 client.0 _ll_register_callbacks cb 0x5574bfcbbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd79090 0x5574bfd83250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd799d0 0x5574bfd83790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd79090 0x5574bfd83250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48464/0 (socket says 172.21.15.46:48464) 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 -- 172.21.15.46:0/2184818171 learned_addr learned my addr 172.21.15.46:0/2184818171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd799d0 0x5574bfd83790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36522/0 (socket says 172.21.15.46:36522) 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff69a800640 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd8a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 -- 172.21.15.46:0/2184818171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 msgr2=0x5574bfd8a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd8a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 -- 172.21.15.46:0/2184818171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd799d0 msgr2=0x5574bfd83790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd799d0 0x5574bfd83790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 -- 172.21.15.46:0/2184818171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574bfabd620 con 0x5574bfd79090 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff69a800640 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd8a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699400640 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd799d0 0x5574bfd83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff699e00640 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd79090 0x5574bfd83250 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff68c098630 tx=0x7ff68c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991166882681e583 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.036+0000 7ff67be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.037+0000 7ff67be00640 1 -- 172.21.15.46:0/2184818171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0de070 con 0x5574bfd79090 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.037+0000 7ff67be00640 1 -- 172.21.15.46:0/2184818171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff68c099f60 con 0x5574bfd79090 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.037+0000 7ff6a400f740 1 -- 172.21.15.46:0/2184818171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff68c0dab20 con 0x5574bfd79090 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.037+0000 7ff67be00640 1 -- 172.21.15.46:0/2184818171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff68c097f00 con 0x5574bfd79090 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.037+0000 7ff6a400f740 10 client.16194 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.037+0000 7ff6a400f740 1 -- 172.21.15.46:0/2184818171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5574bfd8d5c0 con 0x5574bfd79090 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.037+0000 7ff67be00640 1 -- 172.21.15.46:0/2184818171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff68c0dab20 con 0x5574bfd79090 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.038+0000 7ff67be00640 1 -- 172.21.15.46:0/2184818171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7ff68c09bec0 con 0x5574bfd79090 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.038+0000 7ff6a400f740 10 client.16194 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.038+0000 7ff6a400f740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.038+0000 7ff6a400f740 20 client.16194 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.038+0000 7ff6a400f740 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574bfd8e910 0x5574bfd90d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.038+0000 7ff6a400f740 4 client.16194 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.038+0000 7ff6a400f740 1 -- 172.21.15.46:0/2184818171 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574bfceb830 con 0x5574bfd8e910 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.038+0000 7ff699400640 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574bfd8e910 0x5574bfd90d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.039+0000 7ff699400640 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574bfd8e910 0x5574bfd90d00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6edca3ecf3e202a server_cookie=ac06bc83ac6c8628 in_seq=0 out_seq=0 2024-09-10T07:41:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.039+0000 7ff67be00640 10 client.16194 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.039+0000 7ff67be00640 1 -- 172.21.15.46:0/2184818171 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574bfceb830 con 0x5574bfd8e910 2024-09-10T07:41:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.039+0000 7ff67be00640 10 client.16194 handle_command_reply: tid=0 2024-09-10T07:41:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.039+0000 7ff6a400f740 1 client.16194 shutdown 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 -- 172.21.15.46:0/2184818171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd79090 msgr2=0x5574bfd83250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd79090 0x5574bfd83250 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff68c098630 tx=0x7ff68c00cf20 comp rx=0 tx=0).stop 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 -- 172.21.15.46:0/2184818171 shutdown_connections 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574bfd8e910 0x5574bfd90d00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd7ea60 0x5574bfd8a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd799d0 0x5574bfd83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 --2- 172.21.15.46:0/2184818171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd79090 0x5574bfd83250 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff67be00640 0 client.16194 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 -- 172.21.15.46:0/2184818171 >> 172.21.15.46:0/2184818171 conn(0x5574bfd74700 msgr2=0x5574bfd75f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 -- 172.21.15.46:0/2184818171 shutdown_connections 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 1 -- 172.21.15.46:0/2184818171 wait complete. 2024-09-10T07:41:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.040+0000 7ff6a400f740 20 client.16194 trim_cache size 0 max 16384 2024-09-10T07:41:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.055+0000 7ff6a400f740 1 Processor -- start 2024-09-10T07:41:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.055+0000 7ff6a400f740 1 -- start start 2024-09-10T07:41:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd83000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd8a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd83540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff6a400f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1b0cf0 con 0x5574bfd799d0 2024-09-10T07:41:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff6a400f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1b2890 con 0x5574bfd74700 2024-09-10T07:41:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff6a400f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1b13c0 con 0x5574bfd79090 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff69a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd83540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd8a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd8a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53702/0 (socket says 172.21.15.46:53702) 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699400640 1 -- 172.21.15.46:0/3751183519 learned_addr learned my addr 172.21.15.46:0/3751183519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699e00640 1 --2- 172.21.15.46:0/3751183519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd83000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699400640 1 -- 172.21.15.46:0/3751183519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 msgr2=0x5574bfd83000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699400640 1 --2- 172.21.15.46:0/3751183519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd83000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699400640 1 -- 172.21.15.46:0/3751183519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 msgr2=0x5574bfd83540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699400640 1 --2- 172.21.15.46:0/3751183519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd83540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.056+0000 7ff699400640 1 -- 172.21.15.46:0/3751183519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574bfd74520 con 0x5574bfd79090 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.057+0000 7ff699400640 1 --2- 172.21.15.46:0/3751183519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd8a790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff688005fd0 tx=0x7ff688009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ab73dd15dbe883 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.057+0000 7ff686a00640 1 -- 172.21.15.46:0/3751183519 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688004220 con 0x5574bfd79090 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.057+0000 7ff686a00640 1 -- 172.21.15.46:0/3751183519 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6880043c0 con 0x5574bfd79090 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.057+0000 7ff686a00640 1 -- 172.21.15.46:0/3751183519 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688003850 con 0x5574bfd79090 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.057+0000 7ff6a400f740 1 -- 172.21.15.46:0/3751183519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 msgr2=0x5574bfd8a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.057+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3751183519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd8a790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff688005fd0 tx=0x7ff688009260 comp rx=0 tx=0).stop 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 -- 172.21.15.46:0/3751183519 shutdown_connections 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3751183519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfd83540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3751183519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd8a790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3751183519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd83000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 -- 172.21.15.46:0/3751183519 >> 172.21.15.46:0/3751183519 conn(0x5574bfd7a490 msgr2=0x5574bfd7abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 -- 172.21.15.46:0/3751183519 shutdown_connections 2024-09-10T07:41:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 -- 172.21.15.46:0/3751183519 wait complete. 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 -- start start 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 Processor -- start 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.058+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd87970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff6a400f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfdf73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff6a400f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1b21c0 con 0x5574bfd799d0 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff6a400f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1b1af0 con 0x5574bfd74700 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff6a400f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1aecd0 con 0x5574bfd79090 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff6a400f740 10 client.0 _ll_register_callbacks cb 0x7ff694034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff69a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfdf73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff699400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd87970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff699e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff699400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd87970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53714/0 (socket says 172.21.15.46:53714) 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff699400640 1 -- 172.21.15.46:0/3144500729 learned_addr learned my addr 172.21.15.46:0/3144500729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff69a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfdf73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48484/0 (socket says 172.21.15.46:48484) 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff69a800640 1 -- 172.21.15.46:0/3144500729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 msgr2=0x5574bfd87970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff69a800640 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd87970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff69a800640 1 -- 172.21.15.46:0/3144500729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 msgr2=0x5574bfd87430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff69a800640 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd87430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff69a800640 1 -- 172.21.15.46:0/3144500729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574bfd87f80 con 0x5574bfd799d0 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.059+0000 7ff699e00640 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd87430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.060+0000 7ff69a800640 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfdf73e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff69402abf0 tx=0x7ff69402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbdd27faf997cef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.060+0000 7ff67be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.060+0000 7ff67be00640 1 -- 172.21.15.46:0/3144500729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694036040 con 0x5574bfd799d0 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.060+0000 7ff67be00640 1 -- 172.21.15.46:0/3144500729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff694004690 con 0x5574bfd799d0 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.060+0000 7ff6a400f740 1 -- 172.21.15.46:0/3144500729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6880043c0 con 0x5574bfd799d0 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.060+0000 7ff67be00640 1 -- 172.21.15.46:0/3144500729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff694031440 con 0x5574bfd799d0 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.060+0000 7ff6a400f740 10 client.16200 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.060+0000 7ff6a400f740 1 -- 172.21.15.46:0/3144500729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5574bfdf8a30 con 0x5574bfd799d0 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff67be00640 1 -- 172.21.15.46:0/3144500729 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6880043c0 con 0x5574bfd799d0 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff67be00640 1 -- 172.21.15.46:0/3144500729 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7ff69403c040 con 0x5574bfd799d0 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff6a400f740 10 client.16200 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff6a400f740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff6a400f740 20 client.16200 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574bfd7ea60 0x5574bfdfb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff6a400f740 4 client.16200 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:41:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff6a400f740 1 -- 172.21.15.46:0/3144500729 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574bfce5b80 con 0x5574bfd7ea60 2024-09-10T07:41:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.061+0000 7ff699e00640 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574bfd7ea60 0x5574bfdfb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.062+0000 7ff699e00640 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574bfd7ea60 0x5574bfdfb420 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8400b11a8308574 server_cookie=af39402af2654e5b in_seq=0 out_seq=0 2024-09-10T07:41:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.062+0000 7ff67be00640 10 client.16200 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff67be00640 1 -- 172.21.15.46:0/3144500729 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574bfce5b80 con 0x5574bfd7ea60 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff67be00640 10 client.16200 handle_command_reply: tid=0 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 client.16200 shutdown 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 -- 172.21.15.46:0/3144500729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 msgr2=0x5574bfdf73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfdf73e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff69402abf0 tx=0x7ff69402a6d0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 -- 172.21.15.46:0/3144500729 shutdown_connections 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574bfd7ea60 0x5574bfdfb420 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574bfd799d0 0x5574bfdf73e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574bfd79090 0x5574bfd87970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff67be00640 0 client.16200 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 --2- 172.21.15.46:0/3144500729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574bfd74700 0x5574bfd87430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 -- 172.21.15.46:0/3144500729 >> 172.21.15.46:0/3144500729 conn(0x5574bfd7a490 msgr2=0x5574bfd82db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 -- 172.21.15.46:0/3144500729 shutdown_connections 2024-09-10T07:41:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 1 -- 172.21.15.46:0/3144500729 wait complete. 2024-09-10T07:41:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a400f740 20 client.16200 trim_cache size 0 max 16384 2024-09-10T07:41:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:41:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:41:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:41:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 -- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff6800a2c30 msgr2=0x7ff6800a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff6800a2c30 0x7ff6800a50f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff694005cc0 tx=0x7ff694030a30 comp rx=0 tx=0).stop 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 -- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c074c50 msgr2=0x7ff69c19e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c074c50 0x7ff69c19e340 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00e9f0 tx=0x7ff68c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 -- 172.21.15.46:0/2080237477 shutdown_connections 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff69c10e110 0x7ff69c1a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff69c076230 0x7ff69c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff6800a2c30 0x7ff6800a50f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 --2- 172.21.15.46:0/2080237477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69c074c50 0x7ff69c19e340 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.076+0000 7ff6a1000640 1 -- 172.21.15.46:0/2080237477 >> 172.21.15.46:0/2080237477 conn(0x7ff69c0fd2e0 msgr2=0x7ff69c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.077+0000 7ff6a1000640 1 -- 172.21.15.46:0/2080237477 shutdown_connections 2024-09-10T07:41:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:07.077+0000 7ff6a1000640 1 -- 172.21.15.46:0/2080237477 wait complete. 2024-09-10T07:41:07.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:41:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:08.107 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:07 smithi046 ceph-mon[30705]: pgmap v534: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 476 op/s 2024-09-10T07:41:08.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:07 smithi181 ceph-mon[35397]: pgmap v534: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 476 op/s 2024-09-10T07:41:08.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:07 smithi063 ceph-mon[35020]: pgmap v534: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 476 op/s 2024-09-10T07:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.473+0000 7f7b64c00640 1 -- 172.21.15.46:0/2998492815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6010e7e0 msgr2=0x7f7b60115130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.473+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2998492815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6010e7e0 0x7f7b60115130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009cd0 tx=0x7f7b5802a8f0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.474+0000 7f7b64c00640 1 -- 172.21.15.46:0/2998492815 shutdown_connections 2024-09-10T07:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.474+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2998492815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6010e7e0 0x7f7b60115130 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.474+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2998492815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b60109c40 0x7f7b6010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.474+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2998492815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b60109290 0x7f7b60109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.474+0000 7f7b64c00640 1 -- 172.21.15.46:0/2998492815 >> 172.21.15.46:0/2998492815 conn(0x7f7b60077bf0 msgr2=0x7f7b6007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.474+0000 7f7b64c00640 1 -- 172.21.15.46:0/2998492815 shutdown_connections 2024-09-10T07:41:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.474+0000 7f7b64c00640 1 -- 172.21.15.46:0/2998492815 wait complete. 2024-09-10T07:41:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.478+0000 7f7b64c00640 1 Processor -- start 2024-09-10T07:41:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.479+0000 7f7b64c00640 1 -- start start 2024-09-10T07:41:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.482+0000 7f7b64c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b60109290 0x7f7b601a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.482+0000 7f7b64c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b60109c40 0x7f7b601a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.483+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b60109c40 0x7f7b601a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.484+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b60109c40 0x7f7b601a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36548/0 (socket says 172.21.15.46:36548) 2024-09-10T07:41:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.484+0000 7f7b5d400640 1 -- 172.21.15.46:0/2145535133 learned_addr learned my addr 172.21.15.46:0/2145535133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.483+0000 7f7b5de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b60109290 0x7f7b601a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.485+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b6010e7e0 0x7f7b6019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.485+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7b60118640 con 0x7f7b6010e7e0 2024-09-10T07:41:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.485+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b601184c0 con 0x7f7b60109c40 2024-09-10T07:41:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.485+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b601187c0 con 0x7f7b60109290 2024-09-10T07:41:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.487+0000 7f7b5de00640 1 -- 172.21.15.46:0/2145535133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b60109c40 msgr2=0x7f7b601a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.487+0000 7f7b5de00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b60109c40 0x7f7b601a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.487+0000 7f7b5de00640 1 -- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b6010e7e0 msgr2=0x7f7b6019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.487+0000 7f7b5de00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b6010e7e0 0x7f7b6019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.487+0000 7f7b5de00640 1 -- 172.21.15.46:0/2145535133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6019f090 con 0x7f7b60109290 2024-09-10T07:41:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.490+0000 7f7b5de00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b60109290 0x7f7b601a47a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000b3f0 tx=0x7f7b5000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.490+0000 7f7b56a00640 1 -- 172.21.15.46:0/2145535133 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50015020 con 0x7f7b60109290 2024-09-10T07:41:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.491+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6019f380 con 0x7f7b60109290 2024-09-10T07:41:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.491+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b601ab6f0 con 0x7f7b60109290 2024-09-10T07:41:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.491+0000 7f7b56a00640 1 -- 172.21.15.46:0/2145535133 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b50011d10 con 0x7f7b60109290 2024-09-10T07:41:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.491+0000 7f7b56a00640 1 -- 172.21.15.46:0/2145535133 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50010e40 con 0x7f7b60109290 2024-09-10T07:41:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.497+0000 7f7b56a00640 1 -- 172.21.15.46:0/2145535133 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b50018020 con 0x7f7b60109290 2024-09-10T07:41:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.499+0000 7f7b67879740 1 Processor -- start 2024-09-10T07:41:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.499+0000 7f7b56a00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b440a2cc0 0x7f7b440a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.499+0000 7f7b56a00640 1 -- 172.21.15.46:0/2145535133 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7b500ceb50 con 0x7f7b60109290 2024-09-10T07:41:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.500+0000 7f7b5e800640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b440a2cc0 0x7f7b440a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.502+0000 7f7b67879740 1 -- start start 2024-09-10T07:41:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b67879740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8041f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b67879740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da804c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b67879740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da804f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b67879740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560da7fa83e0 con 0x560da80424b0 2024-09-10T07:41:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b67879740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560da7f4fd80 con 0x560da8041b90 2024-09-10T07:41:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b67879740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560da7fa43a0 con 0x560da804cd70 2024-09-10T07:41:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b5de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8041f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b5de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8041f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36560/0 (socket says 172.21.15.46:36560) 2024-09-10T07:41:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.503+0000 7f7b5de00640 1 -- 172.21.15.46:0/4179113351 learned_addr learned my addr 172.21.15.46:0/4179113351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.504+0000 7f7b5e800640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b440a2cc0 0x7f7b440a5180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7b5802aee0 tx=0x7f7b58009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.504+0000 7f7b5de00640 1 -- 172.21.15.46:0/4179113351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 msgr2=0x560da804f160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:41:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.504+0000 7f7b5de00640 1 --2- 172.21.15.46:0/4179113351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da804f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.504+0000 7f7b5de00640 1 -- 172.21.15.46:0/4179113351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 msgr2=0x560da804c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.504+0000 7f7b5de00640 1 --2- 172.21.15.46:0/4179113351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da804c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.504+0000 7f7b5de00640 1 -- 172.21.15.46:0/4179113351 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da7f6eae0 con 0x560da8041b90 2024-09-10T07:41:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.504+0000 7f7b5de00640 1 --2- 172.21.15.46:0/4179113351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8041f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b5009acd0 tx=0x7f7b500047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed698b8522eb291b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.506+0000 7f7b4b000640 1 -- 172.21.15.46:0/4179113351 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5009f380 con 0x560da8041b90 2024-09-10T07:41:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.506+0000 7f7b4b000640 1 -- 172.21.15.46:0/4179113351 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b500a3c40 con 0x560da8041b90 2024-09-10T07:41:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.506+0000 7f7b4b000640 1 -- 172.21.15.46:0/4179113351 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5009f380 con 0x560da8041b90 2024-09-10T07:41:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.507+0000 7f7b67879740 1 -- 172.21.15.46:0/4179113351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 msgr2=0x560da8041f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.507+0000 7f7b67879740 1 --2- 172.21.15.46:0/4179113351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8041f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b5009acd0 tx=0x7f7b500047d0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.508+0000 7f7b67879740 1 -- 172.21.15.46:0/4179113351 shutdown_connections 2024-09-10T07:41:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.508+0000 7f7b67879740 1 --2- 172.21.15.46:0/4179113351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da804f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.508+0000 7f7b67879740 1 --2- 172.21.15.46:0/4179113351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da804c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.508+0000 7f7b67879740 1 --2- 172.21.15.46:0/4179113351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8041f70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.508+0000 7f7b67879740 1 -- 172.21.15.46:0/4179113351 >> 172.21.15.46:0/4179113351 conn(0x560da803e120 msgr2=0x560da803e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.509+0000 7f7b67879740 1 -- 172.21.15.46:0/4179113351 shutdown_connections 2024-09-10T07:41:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.509+0000 7f7b67879740 1 -- 172.21.15.46:0/4179113351 wait complete. 2024-09-10T07:41:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.510+0000 7f7b67879740 1 -- start start 2024-09-10T07:41:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.510+0000 7f7b67879740 1 Processor -- start 2024-09-10T07:41:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.511+0000 7f7b67879740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da8041b90 0x560da8049cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.511+0000 7f7b67879740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da80424b0 0x560da804a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.512+0000 7f7b67879740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da804cd70 0x560da804a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.512+0000 7f7b5de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da8041b90 0x560da8049cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.512+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da80424b0 0x560da804a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.512+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da80424b0 0x560da804a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53758/0 (socket says 172.21.15.46:53758) 2024-09-10T07:41:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.512+0000 7f7b5d400640 1 -- 172.21.15.46:0/2253781303 learned_addr learned my addr 172.21.15.46:0/2253781303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.513+0000 7f7b5e800640 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da804cd70 0x560da804a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.514+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560da804f9e0 con 0x560da8041b90 2024-09-10T07:41:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.514+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560da7f630f0 con 0x560da804cd70 2024-09-10T07:41:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.514+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560da804fb60 con 0x560da80424b0 2024-09-10T07:41:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.514+0000 7f7b67879740 10 client.0 _ll_register_callbacks cb 0x560da7f549e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.515+0000 7f7b5d400640 1 -- 172.21.15.46:0/2253781303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da804cd70 msgr2=0x560da804a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.515+0000 7f7b5d400640 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da804cd70 0x560da804a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.515+0000 7f7b5d400640 1 -- 172.21.15.46:0/2253781303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da8041b90 msgr2=0x560da8049cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.515+0000 7f7b5d400640 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da8041b90 0x560da8049cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.515+0000 7f7b5d400640 1 -- 172.21.15.46:0/2253781303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da7d193c0 con 0x560da80424b0 2024-09-10T07:41:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.516+0000 7f7b5d400640 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da80424b0 0x560da804a210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c007fb0 tx=0x7f7b4c00d5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df991c7a7bd860c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.516+0000 7f7b3f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:41:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.518+0000 7f7b3f400640 1 -- 172.21.15.46:0/2253781303 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c013480 con 0x560da80424b0 2024-09-10T07:41:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.518+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b500a3c40 con 0x560da80424b0 2024-09-10T07:41:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.518+0000 7f7b3f400640 1 -- 172.21.15.46:0/2253781303 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7b4c00f040 con 0x560da80424b0 2024-09-10T07:41:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.518+0000 7f7b3f400640 1 -- 172.21.15.46:0/2253781303 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b4c021050 con 0x560da80424b0 2024-09-10T07:41:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.519+0000 7f7b67879740 10 client.26476 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.519+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560da805b830 con 0x560da80424b0 2024-09-10T07:41:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.519+0000 7f7b3f400640 1 -- 172.21.15.46:0/2253781303 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b500a3c40 con 0x560da80424b0 2024-09-10T07:41:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.520+0000 7f7b3f400640 1 -- 172.21.15.46:0/2253781303 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7b4c0129b0 con 0x560da80424b0 2024-09-10T07:41:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.520+0000 7f7b67879740 10 client.26476 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.520+0000 7f7b67879740 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.520+0000 7f7b67879740 20 client.26476 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.520+0000 7f7b67879740 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560da805cc90 0x560da805f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.521+0000 7f7b67879740 4 client.26476 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.521+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560da7fb5950 con 0x560da805cc90 2024-09-10T07:41:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.521+0000 7f7b5de00640 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560da805cc90 0x560da805f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.523+0000 7f7b5de00640 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560da805cc90 0x560da805f0e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a83c27656999913 server_cookie=84a65c6e7965970d in_seq=0 out_seq=0 2024-09-10T07:41:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.523+0000 7f7b3f400640 10 client.26476 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.525+0000 7f7b3f400640 1 -- 172.21.15.46:0/2253781303 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560da7fb5950 con 0x560da805cc90 2024-09-10T07:41:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.525+0000 7f7b3f400640 10 client.26476 handle_command_reply: tid=0 2024-09-10T07:41:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.525+0000 7f7b67879740 1 client.26476 shutdown 2024-09-10T07:41:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da80424b0 msgr2=0x560da804a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b67879740 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da80424b0 0x560da804a210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c007fb0 tx=0x7f7b4c00d5f0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 shutdown_connections 2024-09-10T07:41:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b67879740 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560da805cc90 0x560da805f0e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b67879740 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da804cd70 0x560da804a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b67879740 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da80424b0 0x560da804a210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b67879740 1 --2- 172.21.15.46:0/2253781303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da8041b90 0x560da8049cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 >> 172.21.15.46:0/2253781303 conn(0x560da803e120 msgr2=0x560da8043e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.526+0000 7f7b3f400640 0 client.26476 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.527+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 shutdown_connections 2024-09-10T07:41:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.527+0000 7f7b67879740 1 -- 172.21.15.46:0/2253781303 wait complete. 2024-09-10T07:41:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.527+0000 7f7b67879740 20 client.26476 trim_cache size 0 max 16384 2024-09-10T07:41:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.568+0000 7f7b67879740 1 Processor -- start 2024-09-10T07:41:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.568+0000 7f7b67879740 1 -- start start 2024-09-10T07:41:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b67879740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da804a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b67879740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da8064950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b67879740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da8064e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b67879740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4000fb90 con 0x560da80424b0 2024-09-10T07:41:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b67879740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40011730 con 0x560da8041b90 2024-09-10T07:41:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b67879740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40010260 con 0x560da804cd70 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da8064950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b5de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da804a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b5de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da804a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36584/0 (socket says 172.21.15.46:36584) 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.569+0000 7f7b5de00640 1 -- 172.21.15.46:0/68681463 learned_addr learned my addr 172.21.15.46:0/68681463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.570+0000 7f7b5e800640 1 --2- 172.21.15.46:0/68681463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da8064e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.570+0000 7f7b5de00640 1 -- 172.21.15.46:0/68681463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 msgr2=0x560da8064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.570+0000 7f7b5de00640 1 --2- 172.21.15.46:0/68681463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da8064e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.570+0000 7f7b5de00640 1 -- 172.21.15.46:0/68681463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 msgr2=0x560da8064950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.570+0000 7f7b5de00640 1 --2- 172.21.15.46:0/68681463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da8064950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.570+0000 7f7b5de00640 1 -- 172.21.15.46:0/68681463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da803d9d0 con 0x560da8041b90 2024-09-10T07:41:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.571+0000 7f7b5de00640 1 --2- 172.21.15.46:0/68681463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da804a5a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7b5009f540 tx=0x7f7b5009adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722fe10181fc8c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.572+0000 7f7b4a600640 1 -- 172.21.15.46:0/68681463 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5007f070 con 0x560da8041b90 2024-09-10T07:41:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.572+0000 7f7b4a600640 1 -- 172.21.15.46:0/68681463 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b500cc400 con 0x560da8041b90 2024-09-10T07:41:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.572+0000 7f7b4a600640 1 -- 172.21.15.46:0/68681463 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5007f070 con 0x560da8041b90 2024-09-10T07:41:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.574+0000 7f7b67879740 1 -- 172.21.15.46:0/68681463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 msgr2=0x560da804a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.575+0000 7f7b67879740 1 --2- 172.21.15.46:0/68681463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da804a5a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7b5009f540 tx=0x7f7b5009adc0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.575+0000 7f7b67879740 1 -- 172.21.15.46:0/68681463 shutdown_connections 2024-09-10T07:41:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.575+0000 7f7b67879740 1 --2- 172.21.15.46:0/68681463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da8064e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.575+0000 7f7b67879740 1 --2- 172.21.15.46:0/68681463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da8064950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.575+0000 7f7b67879740 1 --2- 172.21.15.46:0/68681463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da804a5a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.575+0000 7f7b67879740 1 -- 172.21.15.46:0/68681463 >> 172.21.15.46:0/68681463 conn(0x560da803e120 msgr2=0x560da8048dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.575+0000 7f7b67879740 1 -- 172.21.15.46:0/68681463 shutdown_connections 2024-09-10T07:41:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.576+0000 7f7b67879740 1 -- 172.21.15.46:0/68681463 wait complete. 2024-09-10T07:41:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.576+0000 7f7b67879740 1 -- start start 2024-09-10T07:41:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.577+0000 7f7b67879740 1 Processor -- start 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.577+0000 7f7b67879740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8055190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.577+0000 7f7b67879740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da80556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.577+0000 7f7b67879740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da8050760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.577+0000 7f7b67879740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40011060 con 0x560da80424b0 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.577+0000 7f7b67879740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40010990 con 0x560da8041b90 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.577+0000 7f7b67879740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4000db70 con 0x560da804cd70 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.577+0000 7f7b67879740 10 client.0 _ll_register_callbacks cb 0x7f7b4c007b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.579+0000 7f7b5de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8055190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.579+0000 7f7b5de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8055190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36600/0 (socket says 172.21.15.46:36600) 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.579+0000 7f7b5de00640 1 -- 172.21.15.46:0/1788126643 learned_addr learned my addr 172.21.15.46:0/1788126643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.579+0000 7f7b5e800640 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da8050760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.579+0000 7f7b5d400640 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da80556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.580+0000 7f7b5de00640 1 -- 172.21.15.46:0/1788126643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 msgr2=0x560da8050760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.580+0000 7f7b5de00640 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da8050760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.580+0000 7f7b5de00640 1 -- 172.21.15.46:0/1788126643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 msgr2=0x560da80556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.580+0000 7f7b5de00640 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da80556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.580+0000 7f7b5de00640 1 -- 172.21.15.46:0/1788126643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da8050e90 con 0x560da8041b90 2024-09-10T07:41:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.581+0000 7f7b5de00640 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8055190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7b50010a50 tx=0x7f7b500993f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a1dc9c12adb6769 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.581+0000 7f7b3f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.581+0000 7f7b3f400640 1 -- 172.21.15.46:0/1788126643 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5007f030 con 0x560da8041b90 2024-09-10T07:41:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.582+0000 7f7b3f400640 1 -- 172.21.15.46:0/1788126643 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7b500b5040 con 0x560da8041b90 2024-09-10T07:41:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.582+0000 7f7b67879740 1 -- 172.21.15.46:0/1788126643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b500cc400 con 0x560da8041b90 2024-09-10T07:41:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.583+0000 7f7b3f400640 1 -- 172.21.15.46:0/1788126643 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b500a9050 con 0x560da8041b90 2024-09-10T07:41:08.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.584+0000 7f7b67879740 10 client.26498 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:08.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.584+0000 7f7b67879740 1 -- 172.21.15.46:0/1788126643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560da80d8580 con 0x560da8041b90 2024-09-10T07:41:08.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.584+0000 7f7b3f400640 1 -- 172.21.15.46:0/1788126643 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b500cc400 con 0x560da8041b90 2024-09-10T07:41:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.586+0000 7f7b3f400640 1 -- 172.21.15.46:0/1788126643 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7b50011830 con 0x560da8041b90 2024-09-10T07:41:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.586+0000 7f7b67879740 10 client.26498 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.586+0000 7f7b67879740 10 client.26498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.586+0000 7f7b67879740 20 client.26498 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.586+0000 7f7b67879740 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560da80d8b40 0x560da80daf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:08.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.586+0000 7f7b67879740 4 client.26498 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:41:08.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.586+0000 7f7b67879740 1 -- 172.21.15.46:0/1788126643 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560da7fafe60 con 0x560da80d8b40 2024-09-10T07:41:08.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.587+0000 7f7b5d400640 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560da80d8b40 0x560da80daf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:08.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.589+0000 7f7b3f400640 10 client.26498 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:08.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.589+0000 7f7b5d400640 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560da80d8b40 0x560da80daf30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd54e32daad160c server_cookie=30da12efb6babd23 in_seq=0 out_seq=0 2024-09-10T07:41:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.589+0000 7f7b3f400640 1 -- 172.21.15.46:0/1788126643 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560da7fafe60 con 0x560da80d8b40 2024-09-10T07:41:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.589+0000 7f7b3f400640 10 client.26498 handle_command_reply: tid=0 2024-09-10T07:41:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.589+0000 7f7b67879740 1 client.26498 shutdown 2024-09-10T07:41:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b67879740 1 -- 172.21.15.46:0/1788126643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 msgr2=0x560da8055190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b67879740 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8055190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7b50010a50 tx=0x7f7b500993f0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b67879740 1 -- 172.21.15.46:0/1788126643 shutdown_connections 2024-09-10T07:41:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b67879740 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560da80d8b40 0x560da80daf30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b67879740 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560da804cd70 0x560da8050760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b67879740 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560da80424b0 0x560da80556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b67879740 1 --2- 172.21.15.46:0/1788126643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560da8041b90 0x560da8055190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b3f400640 0 client.26498 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.590+0000 7f7b67879740 1 -- 172.21.15.46:0/1788126643 >> 172.21.15.46:0/1788126643 conn(0x560da803e120 msgr2=0x560da803c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.593+0000 7f7b67879740 1 -- 172.21.15.46:0/1788126643 shutdown_connections 2024-09-10T07:41:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.593+0000 7f7b67879740 1 -- 172.21.15.46:0/1788126643 wait complete. 2024-09-10T07:41:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.594+0000 7f7b67879740 20 client.26498 trim_cache size 0 max 16384 2024-09-10T07:41:08.594 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:41:08.595 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:41:08.595 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:41:08.595 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:41:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b440a2cc0 msgr2=0x7f7b440a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b440a2cc0 0x7f7b440a5180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7b5802aee0 tx=0x7f7b58009800 comp rx=0 tx=0).stop 2024-09-10T07:41:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b60109290 msgr2=0x7f7b601a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b60109290 0x7f7b601a47a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000b3f0 tx=0x7f7b5000b8b0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 shutdown_connections 2024-09-10T07:41:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b440a2cc0 0x7f7b440a5180 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b6010e7e0 0x7f7b6019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b60109c40 0x7f7b601a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 --2- 172.21.15.46:0/2145535133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b60109290 0x7f7b601a47a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 >> 172.21.15.46:0/2145535133 conn(0x7f7b60077bf0 msgr2=0x7f7b6010c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 shutdown_connections 2024-09-10T07:41:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:08.595+0000 7f7b64c00640 1 -- 172.21.15.46:0/2145535133 wait complete. 2024-09-10T07:41:09.640 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:41:09.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:41:10.019 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:10.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:09 smithi181 ceph-mon[35397]: pgmap v535: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-09-10T07:41:10.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:09 smithi063 ceph-mon[35020]: pgmap v535: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-09-10T07:41:10.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:09 smithi046 ceph-mon[30705]: pgmap v535: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-09-10T07:41:10.602 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 230.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:41:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.578+0000 7f9227400640 1 -- 172.21.15.46:0/2206019274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220074590 msgr2=0x7f9220074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.578+0000 7f9227400640 1 --2- 172.21.15.46:0/2206019274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220074590 0x7f9220074970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9210009900 tx=0x7f921002a480 comp rx=0 tx=0).stop 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.579+0000 7f9227400640 1 -- 172.21.15.46:0/2206019274 shutdown_connections 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.579+0000 7f9227400640 1 --2- 172.21.15.46:0/2206019274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9220112b90 0x7f9220114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.579+0000 7f9227400640 1 --2- 172.21.15.46:0/2206019274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9220074eb0 0x7f9220112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.579+0000 7f9227400640 1 --2- 172.21.15.46:0/2206019274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220074590 0x7f9220074970 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.579+0000 7f9227400640 1 -- 172.21.15.46:0/2206019274 >> 172.21.15.46:0/2206019274 conn(0x7f92200fd8c0 msgr2=0x7f92200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.579+0000 7f9227400640 1 -- 172.21.15.46:0/2206019274 shutdown_connections 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.579+0000 7f9227400640 1 -- 172.21.15.46:0/2206019274 wait complete. 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.580+0000 7f9227400640 1 Processor -- start 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.580+0000 7f9227400640 1 -- start start 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.580+0000 7f9227400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9220074590 0x7f92201a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.580+0000 7f9227400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9220074eb0 0x7f92201a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.580+0000 7f9227400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220112b90 0x7f922019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.580+0000 7f9227400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92201185a0 con 0x7f9220112b90 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.580+0000 7f9227400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9220118420 con 0x7f9220074590 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.580+0000 7f9227400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9220118720 con 0x7f9220074eb0 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9220074590 0x7f92201a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9217e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9220074eb0 0x7f92201a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9220074590 0x7f92201a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36610/0 (socket says 172.21.15.46:36610) 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9225400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220112b90 0x7f922019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 -- 172.21.15.46:0/69594191 learned_addr learned my addr 172.21.15.46:0/69594191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 -- 172.21.15.46:0/69594191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9220074eb0 msgr2=0x7f92201a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9220074eb0 0x7f92201a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 -- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220112b90 msgr2=0x7f922019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220112b90 0x7f922019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 -- 172.21.15.46:0/69594191 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922019f080 con 0x7f9220074590 2024-09-10T07:41:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.581+0000 7f9224a00640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9220074590 0x7f92201a4790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9210004770 tx=0x7f9210003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.582+0000 7f921e000640 1 -- 172.21.15.46:0/69594191 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921003f070 con 0x7f9220074590 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.582+0000 7f921e000640 1 -- 172.21.15.46:0/69594191 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9210031ca0 con 0x7f9220074590 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.582+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922019f310 con 0x7f9220074590 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.582+0000 7f921e000640 1 -- 172.21.15.46:0/69594191 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210035030 con 0x7f9220074590 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.582+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201ab660 con 0x7f9220074590 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.583+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9220074970 con 0x7f9220074590 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.585+0000 7f921e000640 1 -- 172.21.15.46:0/69594191 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9210042070 con 0x7f9220074590 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.586+0000 7f921e000640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92040a2c00 0x7f92040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.587+0000 7f921e000640 1 -- 172.21.15.46:0/69594191 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f92100efff0 con 0x7f9220074590 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.587+0000 7f9217e00640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92040a2c00 0x7f92040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.587+0000 7f9217e00640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92040a2c00 0x7f92040a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f92200fff90 tx=0x7f920800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.589+0000 7f921e000640 1 -- 172.21.15.46:0/69594191 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92100bc540 con 0x7f9220074590 2024-09-10T07:41:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.812+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9220104fd0 con 0x7f9220074590 2024-09-10T07:41:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.813+0000 7f921e000640 1 -- 172.21.15.46:0/69594191 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11975 (secure 0 0 0) 0x7f92100c0840 con 0x7f9220074590 2024-09-10T07:41:10.813 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:41:10.813 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"btime":"2024-09-10T07:40:35:758945+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:40:35.499710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:41:10.813 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 39 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92040a2c00 msgr2=0x7f92040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92040a2c00 0x7f92040a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f92200fff90 tx=0x7f920800b410 comp rx=0 tx=0).stop 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9220074590 msgr2=0x7f92201a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9220074590 0x7f92201a4790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9210004770 tx=0x7f9210003c10 comp rx=0 tx=0).stop 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 shutdown_connections 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92040a2c00 0x7f92040a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220112b90 0x7f922019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9220074eb0 0x7f92201a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 --2- 172.21.15.46:0/69594191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9220074590 0x7f92201a4790 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 >> 172.21.15.46:0/69594191 conn(0x7f92200fd8c0 msgr2=0x7f92200ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 shutdown_connections 2024-09-10T07:41:10.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:10.815+0000 7f9227400640 1 -- 172.21.15.46:0/69594191 wait complete. 2024-09-10T07:41:11.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:10 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/69594191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:41:11.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:10 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/69594191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:41:11.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:10 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/69594191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:41:12.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:41:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:11 smithi181 ceph-mon[35397]: pgmap v536: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 674 op/s 2024-09-10T07:41:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:11 smithi063 ceph-mon[35020]: pgmap v536: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 674 op/s 2024-09-10T07:41:12.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:11 smithi046 ceph-mon[30705]: pgmap v536: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 674 op/s 2024-09-10T07:41:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 -- 172.21.15.46:0/2354972116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36dc1049a0 msgr2=0x7f36dc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 --2- 172.21.15.46:0/2354972116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36dc1049a0 0x7f36dc1089f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009900 tx=0x7f36c402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 -- 172.21.15.46:0/2354972116 shutdown_connections 2024-09-10T07:41:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 --2- 172.21.15.46:0/2354972116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36dc109120 0x7f36dc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 --2- 172.21.15.46:0/2354972116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36dc1049a0 0x7f36dc1089f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 --2- 172.21.15.46:0/2354972116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36dc103ff0 0x7f36dc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 -- 172.21.15.46:0/2354972116 >> 172.21.15.46:0/2354972116 conn(0x7f36dc0ffa90 msgr2=0x7f36dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 -- 172.21.15.46:0/2354972116 shutdown_connections 2024-09-10T07:41:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.002+0000 7f36e2400640 1 -- 172.21.15.46:0/2354972116 wait complete. 2024-09-10T07:41:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.003+0000 7f36e2400640 1 Processor -- start 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.003+0000 7f36e2400640 1 -- start start 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36e2400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36dc103ff0 0x7f36dc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36e2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36dc1049a0 0x7f36dc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36e2400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36dc109120 0x7f36dc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36e2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc114190 con 0x7f36dc1049a0 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36e2400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc114010 con 0x7f36dc109120 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36e2400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc114310 con 0x7f36dc103ff0 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36db400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36dc103ff0 0x7f36dc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36db400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36dc103ff0 0x7f36dc19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53812/0 (socket says 172.21.15.46:53812) 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36dc1049a0 0x7f36dc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36db400640 1 -- 172.21.15.46:0/437406345 learned_addr learned my addr 172.21.15.46:0/437406345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.004+0000 7f36dbe00640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36dc109120 0x7f36dc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36db400640 1 -- 172.21.15.46:0/437406345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36dc109120 msgr2=0x7f36dc19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36db400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36dc109120 0x7f36dc19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36db400640 1 -- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36dc1049a0 msgr2=0x7f36dc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36db400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36dc1049a0 0x7f36dc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36db400640 1 -- 172.21.15.46:0/437406345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc19f5a0 con 0x7f36dc103ff0 2024-09-10T07:41:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36db400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36dc103ff0 0x7f36dc19a5f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00ea60 tx=0x7f36cc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36d8c00640 1 -- 172.21.15.46:0/437406345 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00cde0 con 0x7f36dc103ff0 2024-09-10T07:41:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36e2400640 1 -- 172.21.15.46:0/437406345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc19f890 con 0x7f36dc103ff0 2024-09-10T07:41:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36d8c00640 1 -- 172.21.15.46:0/437406345 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc0035f0 con 0x7f36dc103ff0 2024-09-10T07:41:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36d8c00640 1 -- 172.21.15.46:0/437406345 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc010670 con 0x7f36dc103ff0 2024-09-10T07:41:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.005+0000 7f36e2400640 1 -- 172.21.15.46:0/437406345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc1a7170 con 0x7f36dc103ff0 2024-09-10T07:41:13.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.009+0000 7f36e4e42740 1 Processor -- start 2024-09-10T07:41:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.009+0000 7f36e4e42740 1 -- start start 2024-09-10T07:41:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f372050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36d8c00640 1 -- 172.21.15.46:0/437406345 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36cc0037a0 con 0x7f36dc103ff0 2024-09-10T07:41:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36e4e42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c91f2d83e0 con 0x55c91f37ce50 2024-09-10T07:41:13.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36e4e42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c91f27fd80 con 0x55c91f372590 2024-09-10T07:41:13.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36e4e42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c91f2d43a0 con 0x55c91f371c70 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f372050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48576/0 (socket says 172.21.15.46:48576) 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36d3e00640 1 -- 172.21.15.46:0/3216122382 learned_addr learned my addr 172.21.15.46:0/3216122382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.010+0000 7f36db400640 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36dbe00640 1 -- 172.21.15.46:0/3216122382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 msgr2=0x55c91f37c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36dbe00640 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36dbe00640 1 -- 172.21.15.46:0/3216122382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 msgr2=0x55c91f37f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36dbe00640 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36dbe00640 1 -- 172.21.15.46:0/3216122382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91f29eae0 con 0x55c91f371c70 2024-09-10T07:41:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36db400640 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36dbe00640 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f372050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009f00 tx=0x7f36d4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c77264b930a02c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36d0c00640 1 -- 172.21.15.46:0/3216122382 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4004d20 con 0x55c91f371c70 2024-09-10T07:41:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36d0c00640 1 -- 172.21.15.46:0/3216122382 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d4004050 con 0x55c91f371c70 2024-09-10T07:41:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.011+0000 7f36d0c00640 1 -- 172.21.15.46:0/3216122382 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4005280 con 0x55c91f371c70 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36d8c00640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f36c00a2b80 0x7f36c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 -- 172.21.15.46:0/3216122382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 msgr2=0x55c91f372050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f372050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009f00 tx=0x7f36d4011810 comp rx=0 tx=0).stop 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36d8c00640 1 -- 172.21.15.46:0/437406345 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f36cc0cf640 con 0x7f36dc103ff0 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 -- 172.21.15.46:0/3216122382 shutdown_connections 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 --2- 172.21.15.46:0/3216122382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f372050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 -- 172.21.15.46:0/3216122382 >> 172.21.15.46:0/3216122382 conn(0x55c91f36e200 msgr2=0x55c91f36e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36db400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f36c00a2b80 0x7f36c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 -- 172.21.15.46:0/3216122382 shutdown_connections 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.012+0000 7f36e4e42740 1 -- 172.21.15.46:0/3216122382 wait complete. 2024-09-10T07:41:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.013+0000 7f36e4e42740 1 -- start start 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.013+0000 7f36db400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f36c00a2b80 0x7f36c00a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0cd130 tx=0x7f36cc003f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.013+0000 7f36e4e42740 1 Processor -- start 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f379d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36e4e42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c91f37fa20 con 0x55c91f37ce50 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36e4e42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c91f2930f0 con 0x55c91f372590 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36e4e42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c91f37fba0 con 0x55c91f371c70 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36e4e42740 10 client.0 _ll_register_callbacks cb 0x55c91f27abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f379d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f379d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53824/0 (socket says 172.21.15.46:53824) 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36dbe00640 1 -- 172.21.15.46:0/2343534356 learned_addr learned my addr 172.21.15.46:0/2343534356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36dbe00640 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.014+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36d3e00640 1 -- 172.21.15.46:0/2343534356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 msgr2=0x55c91f379d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f379d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36d3e00640 1 -- 172.21.15.46:0/2343534356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 msgr2=0x55c91f37a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36d3e00640 1 -- 172.21.15.46:0/2343534356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91f0493c0 con 0x55c91f372590 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37a270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f36c4031550 tx=0x7f36c40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc20ec19da8fd03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36c9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36c9600640 1 -- 172.21.15.46:0/2343534356 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4035040 con 0x55c91f372590 2024-09-10T07:41:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36e4e42740 1 -- 172.21.15.46:0/2343534356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d4004050 con 0x55c91f372590 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.015+0000 7f36c9600640 1 -- 172.21.15.46:0/2343534356 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f36c4005450 con 0x55c91f372590 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36c9600640 1 -- 172.21.15.46:0/2343534356 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f36c40046c0 con 0x55c91f372590 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36e4e42740 10 client.26516 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36e4e42740 1 -- 172.21.15.46:0/2343534356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c91f38bab0 con 0x55c91f372590 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36c9600640 1 -- 172.21.15.46:0/2343534356 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d4004050 con 0x55c91f372590 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36c9600640 1 -- 172.21.15.46:0/2343534356 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f36c40039c0 con 0x55c91f372590 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36e4e42740 10 client.26516 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36e4e42740 10 client.26516 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36e4e42740 20 client.26516 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36e4e42740 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55c91f38cec0 0x55c91f38f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36e4e42740 4 client.26516 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.016+0000 7f36e4e42740 1 -- 172.21.15.46:0/2343534356 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c91f2e5950 con 0x55c91f38cec0 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.017+0000 7f36dbe00640 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55c91f38cec0 0x55c91f38f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.017+0000 7f36dbe00640 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55c91f38cec0 0x55c91f38f2e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8dd2b66752420d70 server_cookie=bda54b7b676ad3eb in_seq=0 out_seq=0 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.017+0000 7f36c9600640 10 client.26516 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36c9600640 1 -- 172.21.15.46:0/2343534356 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c91f2e5950 con 0x55c91f38cec0 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36c9600640 10 client.26516 handle_command_reply: tid=0 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 client.26516 shutdown 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 -- 172.21.15.46:0/2343534356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 msgr2=0x55c91f37a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37a270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f36c4031550 tx=0x7f36c40043b0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 -- 172.21.15.46:0/2343534356 shutdown_connections 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55c91f38cec0 0x55c91f38f2e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f37a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f37a270 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 --2- 172.21.15.46:0/2343534356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f379d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36c9600640 0 client.26516 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:41:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 -- 172.21.15.46:0/2343534356 >> 172.21.15.46:0/2343534356 conn(0x55c91f36e200 msgr2=0x55c91f36d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 -- 172.21.15.46:0/2343534356 shutdown_connections 2024-09-10T07:41:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.018+0000 7f36e4e42740 1 -- 172.21.15.46:0/2343534356 wait complete. 2024-09-10T07:41:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.019+0000 7f36e4e42740 20 client.26516 trim_cache size 0 max 16384 2024-09-10T07:41:13.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.032+0000 7f36e4e42740 1 Processor -- start 2024-09-10T07:41:13.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.032+0000 7f36e4e42740 1 -- start start 2024-09-10T07:41:13.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f371c70 0x55c91f378600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f372590 0x55c91f37a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f394bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36e4e42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1b0dc0 con 0x55c91f37ce50 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36e4e42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1b2960 con 0x55c91f371c70 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36e4e42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1b1490 con 0x55c91f372590 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f372590 0x55c91f37a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f372590 0x55c91f37a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53840/0 (socket says 172.21.15.46:53840) 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f371c70 0x55c91f378600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36d3e00640 1 -- 172.21.15.46:0/1779944318 learned_addr learned my addr 172.21.15.46:0/1779944318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36dbe00640 1 --2- 172.21.15.46:0/1779944318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f394bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36d3e00640 1 -- 172.21.15.46:0/1779944318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f371c70 msgr2=0x55c91f378600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36d3e00640 1 --2- 172.21.15.46:0/1779944318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f371c70 0x55c91f378600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36d3e00640 1 -- 172.21.15.46:0/1779944318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 msgr2=0x55c91f394bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36d3e00640 1 --2- 172.21.15.46:0/1779944318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f394bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.033+0000 7f36d3e00640 1 -- 172.21.15.46:0/1779944318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91f369770 con 0x55c91f372590 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.034+0000 7f36d3e00640 1 --2- 172.21.15.46:0/1779944318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f372590 0x55c91f37a600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f36c4003de0 tx=0x7f36c402ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f00ec6b0f62abb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.034+0000 7f36cbe00640 1 -- 172.21.15.46:0/1779944318 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4035030 con 0x55c91f372590 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.034+0000 7f36cbe00640 1 -- 172.21.15.46:0/1779944318 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c4041070 con 0x55c91f372590 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.034+0000 7f36cbe00640 1 -- 172.21.15.46:0/1779944318 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4003ac0 con 0x55c91f372590 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 -- 172.21.15.46:0/1779944318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f372590 msgr2=0x55c91f37a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1779944318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f372590 0x55c91f37a600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f36c4003de0 tx=0x7f36c402ab70 comp rx=0 tx=0).stop 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 -- 172.21.15.46:0/1779944318 shutdown_connections 2024-09-10T07:41:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1779944318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f394bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1779944318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f372590 0x55c91f37a600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1779944318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f371c70 0x55c91f378600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 -- 172.21.15.46:0/1779944318 >> 172.21.15.46:0/1779944318 conn(0x55c91f36e200 msgr2=0x55c91f37bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 -- 172.21.15.46:0/1779944318 shutdown_connections 2024-09-10T07:41:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 -- 172.21.15.46:0/1779944318 wait complete. 2024-09-10T07:41:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.035+0000 7f36e4e42740 1 -- start start 2024-09-10T07:41:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36e4e42740 1 Processor -- start 2024-09-10T07:41:13.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f385590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f3808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36e4e42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f380e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36e4e42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1b2290 con 0x55c91f37ce50 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36e4e42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1b1bc0 con 0x55c91f372590 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36e4e42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1aeda0 con 0x55c91f371c70 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36e4e42740 10 client.0 _ll_register_callbacks cb 0x7f36c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f380e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f380e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48596/0 (socket says 172.21.15.46:48596) 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36dbe00640 1 -- 172.21.15.46:0/1266169903 learned_addr learned my addr 172.21.15.46:0/1266169903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36d3e00640 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f3808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.036+0000 7f36dbe00640 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f385590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36dbe00640 1 -- 172.21.15.46:0/1266169903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 msgr2=0x55c91f385590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36dbe00640 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f385590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36dbe00640 1 -- 172.21.15.46:0/1266169903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 msgr2=0x55c91f3808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36dbe00640 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f3808c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36dbe00640 1 -- 172.21.15.46:0/1266169903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91f381630 con 0x55c91f37ce50 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36d3e00640 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f3808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36dbe00640 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f380e00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f36d4011e10 tx=0x7f36d4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f535f288fcb80443 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36c9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.037+0000 7f36c9600640 1 -- 172.21.15.46:0/1266169903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40043e0 con 0x55c91f37ce50 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 1 -- 172.21.15.46:0/1266169903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c4041070 con 0x55c91f37ce50 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36c9600640 1 -- 172.21.15.46:0/1266169903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f36d4004580 con 0x55c91f37ce50 2024-09-10T07:41:13.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36c9600640 1 -- 172.21.15.46:0/1266169903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f36d4043050 con 0x55c91f37ce50 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 10 client.16233 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 1 -- 172.21.15.46:0/1266169903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c91f3f7c40 con 0x55c91f37ce50 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36c9600640 1 -- 172.21.15.46:0/1266169903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c4041070 con 0x55c91f37ce50 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36c9600640 1 -- 172.21.15.46:0/1266169903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f36d4019430 con 0x55c91f37ce50 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 10 client.16233 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 10 client.16233 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 20 client.16233 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55c91f3f8180 0x55c91f3fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 4 client.16233 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.038+0000 7f36e4e42740 1 -- 172.21.15.46:0/1266169903 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c91f2dfe60 con 0x55c91f3f8180 2024-09-10T07:41:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.039+0000 7f36d3e00640 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55c91f3f8180 0x55c91f3fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.039+0000 7f36d3e00640 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55c91f3f8180 0x55c91f3fa570 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca495c0903509596 server_cookie=e7474416d274f15f in_seq=0 out_seq=0 2024-09-10T07:41:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.039+0000 7f36c9600640 10 client.16233 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:41:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36c9600640 1 -- 172.21.15.46:0/1266169903 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c91f2dfe60 con 0x55c91f3f8180 2024-09-10T07:41:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36c9600640 10 client.16233 handle_command_reply: tid=0 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 client.16233 shutdown 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 -- 172.21.15.46:0/1266169903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 msgr2=0x55c91f380e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f380e00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f36d4011e10 tx=0x7f36d4011810 comp rx=0 tx=0).stop 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 -- 172.21.15.46:0/1266169903 shutdown_connections 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55c91f3f8180 0x55c91f3fa570 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f37ce50 0x55c91f380e00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c91f372590 0x55c91f3808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 --2- 172.21.15.46:0/1266169903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c91f371c70 0x55c91f385590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 -- 172.21.15.46:0/1266169903 >> 172.21.15.46:0/1266169903 conn(0x55c91f36e200 msgr2=0x55c91f36cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36c9600640 0 client.16233 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 -- 172.21.15.46:0/1266169903 shutdown_connections 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 1 -- 172.21.15.46:0/1266169903 wait complete. 2024-09-10T07:41:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.040+0000 7f36e4e42740 20 client.16233 trim_cache size 0 max 16384 2024-09-10T07:41:13.044 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:41:13.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 -- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f36c00a2b80 msgr2=0x7f36c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f36c00a2b80 0x7f36c00a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0cd130 tx=0x7f36cc003f30 comp rx=0 tx=0).stop 2024-09-10T07:41:13.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 -- 172.21.15.46:0/437406345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36dc103ff0 msgr2=0x7f36dc19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36dc103ff0 0x7f36dc19a5f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00ea60 tx=0x7f36cc00ef20 comp rx=0 tx=0).stop 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 -- 172.21.15.46:0/437406345 shutdown_connections 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36dc109120 0x7f36dc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f36c00a2b80 0x7f36c00a5040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36dc1049a0 0x7f36dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 --2- 172.21.15.46:0/437406345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36dc103ff0 0x7f36dc19a5f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 -- 172.21.15.46:0/437406345 >> 172.21.15.46:0/437406345 conn(0x7f36dc0ffa90 msgr2=0x7f36dc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 -- 172.21.15.46:0/437406345 shutdown_connections 2024-09-10T07:41:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:13.041+0000 7f36e2400640 1 -- 172.21.15.46:0/437406345 wait complete. 2024-09-10T07:41:14.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:41:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:13 smithi181 ceph-mon[35397]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 913 op/s 2024-09-10T07:41:14.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:13 smithi063 ceph-mon[35020]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 913 op/s 2024-09-10T07:41:14.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:13 smithi046 ceph-mon[30705]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 913 op/s 2024-09-10T07:41:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 -- 172.21.15.46:0/1669877226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 msgr2=0x7f4ff81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 --2- 172.21.15.46:0/1669877226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 0x7f4ff81021d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009ca0 tx=0x7f4fec02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 -- 172.21.15.46:0/1669877226 shutdown_connections 2024-09-10T07:41:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 --2- 172.21.15.46:0/1669877226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff8106f20 0x7f4ff810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 --2- 172.21.15.46:0/1669877226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ff81027a0 0x7f4ff81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 --2- 172.21.15.46:0/1669877226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 0x7f4ff81021d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 -- 172.21.15.46:0/1669877226 >> 172.21.15.46:0/1669877226 conn(0x7f4ff80fd8b0 msgr2=0x7f4ff80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 -- 172.21.15.46:0/1669877226 shutdown_connections 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.035+0000 7f4fff800640 1 -- 172.21.15.46:0/1669877226 wait complete. 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.036+0000 7f4fff800640 1 Processor -- start 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.036+0000 7f4fff800640 1 -- start start 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.036+0000 7f4fff800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 0x7f4ff80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4fff800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff81027a0 0x7f4ff8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4fff800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ff8106f20 0x7f4ff8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4fff800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8111f40 con 0x7f4ff8101df0 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4fff800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8111dc0 con 0x7f4ff8106f20 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4fff800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81120c0 con 0x7f4ff81027a0 2024-09-10T07:41:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ffce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 0x7f4ff80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ffce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 0x7f4ff80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48614/0 (socket says 172.21.15.46:48614) 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ff7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff81027a0 0x7f4ff8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ffce00640 1 -- 172.21.15.46:0/1121003635 learned_addr learned my addr 172.21.15.46:0/1121003635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ffd800640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ff8106f20 0x7f4ff8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ff7e00640 1 -- 172.21.15.46:0/1121003635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ff8106f20 msgr2=0x7f4ff8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ff8106f20 0x7f4ff8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ff7e00640 1 -- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 msgr2=0x7f4ff80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 0x7f4ff80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ff7e00640 1 -- 172.21.15.46:0/1121003635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff81aaeb0 con 0x7f4ff81027a0 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.037+0000 7f4ffce00640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 0x7f4ff80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.038+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff81027a0 0x7f4ff8074a30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400e9f0 tx=0x7f4fe400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.038+0000 7f4ff5600640 1 -- 172.21.15.46:0/1121003635 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe400cd80 con 0x7f4ff81027a0 2024-09-10T07:41:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.038+0000 7f4fff800640 1 -- 172.21.15.46:0/1121003635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff81ab0e0 con 0x7f4ff81027a0 2024-09-10T07:41:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.038+0000 7f4ff5600640 1 -- 172.21.15.46:0/1121003635 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe40035f0 con 0x7f4ff81027a0 2024-09-10T07:41:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.038+0000 7f4ff5600640 1 -- 172.21.15.46:0/1121003635 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4010670 con 0x7f4ff81027a0 2024-09-10T07:41:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.038+0000 7f4fff800640 1 -- 172.21.15.46:0/1121003635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff81ab670 con 0x7f4ff81027a0 2024-09-10T07:41:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.041+0000 7f500221a740 1 Processor -- start 2024-09-10T07:41:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.041+0000 7f500221a740 1 -- start start 2024-09-10T07:41:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f4ff5600640 1 -- 172.21.15.46:0/1121003635 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4fe40037a0 con 0x7f4ff81027a0 2024-09-10T07:41:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f500221a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b666c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f500221a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b6676910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f500221a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 0x5644b6679240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f500221a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5644b65d23e0 con 0x5644b666bc70 2024-09-10T07:41:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f500221a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5644b6579d80 con 0x5644b666c590 2024-09-10T07:41:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f500221a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5644b65ce3a0 con 0x5644b6676e50 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f4ffd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b666c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f4ffd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b666c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48630/0 (socket says 172.21.15.46:48630) 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f4ffd800640 1 -- 172.21.15.46:0/787317091 learned_addr learned my addr 172.21.15.46:0/787317091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f4ffce00640 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b6676910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.042+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 0x5644b6679240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4ffd800640 1 -- 172.21.15.46:0/787317091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 msgr2=0x5644b6679240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4ffd800640 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 0x5644b6679240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4ffd800640 1 -- 172.21.15.46:0/787317091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 msgr2=0x5644b6676910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4ffd800640 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b6676910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4ffd800640 1 -- 172.21.15.46:0/787317091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b6598ae0 con 0x5644b666bc70 2024-09-10T07:41:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4ffce00640 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b6676910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4ffd800640 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b666c050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009c20 tx=0x7f4ff0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d3e1421b8637e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4fea000640 1 -- 172.21.15.46:0/787317091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff001e070 con 0x5644b666bc70 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4fea000640 1 -- 172.21.15.46:0/787317091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0019b10 con 0x5644b666bc70 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.043+0000 7f4fea000640 1 -- 172.21.15.46:0/787317091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000ea80 con 0x5644b666bc70 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f4ff5600640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4fe00a2c00 0x7f4fe00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f4ff5600640 1 -- 172.21.15.46:0/1121003635 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4fe40cf640 con 0x7f4ff81027a0 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f4ffce00640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4fe00a2c00 0x7f4fe00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f500221a740 1 -- 172.21.15.46:0/787317091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 msgr2=0x5644b666c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f500221a740 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b666c050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009c20 tx=0x7f4ff0011810 comp rx=0 tx=0).stop 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f500221a740 1 -- 172.21.15.46:0/787317091 shutdown_connections 2024-09-10T07:41:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f500221a740 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 0x5644b6679240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f500221a740 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b6676910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f500221a740 1 --2- 172.21.15.46:0/787317091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b666c050 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.044+0000 7f500221a740 1 -- 172.21.15.46:0/787317091 >> 172.21.15.46:0/787317091 conn(0x5644b6668200 msgr2=0x5644b66685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.045+0000 7f500221a740 1 -- 172.21.15.46:0/787317091 shutdown_connections 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.045+0000 7f500221a740 1 -- 172.21.15.46:0/787317091 wait complete. 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.045+0000 7f4ffce00640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4fe00a2c00 0x7f4fe00a50c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5644b6669060 tx=0x7f4fec03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.045+0000 7f500221a740 1 -- start start 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.045+0000 7f500221a740 1 Processor -- start 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f500221a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b6673d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f500221a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b66742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f500221a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 0x5644b6674810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f500221a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5644b6679a20 con 0x5644b666bc70 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f500221a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5644b658d0f0 con 0x5644b666c590 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f500221a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5644b6679ba0 con 0x5644b6676e50 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f500221a740 10 client.0 _ll_register_callbacks cb 0x5644b6574bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f4ffce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b6673d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f4ffce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b6673d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48640/0 (socket says 172.21.15.46:48640) 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f4ffce00640 1 -- 172.21.15.46:0/736269095 learned_addr learned my addr 172.21.15.46:0/736269095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b66742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.046+0000 7f4ffd800640 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 0x5644b6674810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4ffce00640 1 -- 172.21.15.46:0/736269095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 msgr2=0x5644b6674810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4ffce00640 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 0x5644b6674810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4ffce00640 1 -- 172.21.15.46:0/736269095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 msgr2=0x5644b66742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:15.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4ffce00640 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b66742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4ffce00640 1 -- 172.21.15.46:0/736269095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b63433c0 con 0x5644b666bc70 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b66742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4ffce00640 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b6673d90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0097b0 tx=0x7f4fec02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46495d17dc038e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4fdea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4fdea00640 1 -- 172.21.15.46:0/736269095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0308c0 con 0x5644b666bc70 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f4fdea00640 1 -- 172.21.15.46:0/736269095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4fec030a60 con 0x5644b666bc70 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.047+0000 7f500221a740 1 -- 172.21.15.46:0/736269095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff0019b10 con 0x5644b666bc70 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f4fdea00640 1 -- 172.21.15.46:0/736269095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4fec030c60 con 0x5644b666bc70 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f500221a740 10 client.16245 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f500221a740 1 -- 172.21.15.46:0/736269095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5644b6685ab0 con 0x5644b666bc70 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f4fdea00640 1 -- 172.21.15.46:0/736269095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0019b10 con 0x5644b666bc70 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f4fdea00640 1 -- 172.21.15.46:0/736269095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f4fec050400 con 0x5644b666bc70 2024-09-10T07:41:15.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f500221a740 10 client.16245 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:15.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f500221a740 10 client.16245 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:41:15.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f500221a740 20 client.16245 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:15.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f500221a740 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5644b6686f70 0x5644b6689430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f500221a740 4 client.16245 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:15.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.048+0000 7f500221a740 1 -- 172.21.15.46:0/736269095 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b65df950 con 0x5644b6686f70 2024-09-10T07:41:15.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.049+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5644b6686f70 0x5644b6689430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.049+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5644b6686f70 0x5644b6689430 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dce65b4bbb51bb0c server_cookie=a5a79669a46c2e1b in_seq=0 out_seq=0 2024-09-10T07:41:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.049+0000 7f4fdea00640 10 client.16245 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:41:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.050+0000 7f4fdea00640 1 -- 172.21.15.46:0/736269095 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644b65df950 con 0x5644b6686f70 2024-09-10T07:41:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.050+0000 7f4fdea00640 10 client.16245 handle_command_reply: tid=0 2024-09-10T07:41:15.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.050+0000 7f500221a740 1 client.16245 shutdown 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 -- 172.21.15.46:0/736269095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 msgr2=0x5644b6673d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b6673d90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0097b0 tx=0x7f4fec02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 -- 172.21.15.46:0/736269095 shutdown_connections 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5644b6686f70 0x5644b6689430 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b6676e50 0x5644b6674810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666c590 0x5644b66742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 --2- 172.21.15.46:0/736269095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666bc70 0x5644b6673d90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 -- 172.21.15.46:0/736269095 >> 172.21.15.46:0/736269095 conn(0x5644b6668200 msgr2=0x5644b6667380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f4fdea00640 0 client.16245 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 -- 172.21.15.46:0/736269095 shutdown_connections 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 1 -- 172.21.15.46:0/736269095 wait complete. 2024-09-10T07:41:15.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.051+0000 7f500221a740 20 client.16245 trim_cache size 0 max 16384 2024-09-10T07:41:15.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f500221a740 1 Processor -- start 2024-09-10T07:41:15.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f500221a740 1 -- start start 2024-09-10T07:41:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f500221a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666bc70 0x5644b666e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f500221a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666c590 0x5644b668ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f500221a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b6676e50 0x5644b668f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f500221a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b5220 con 0x5644b6676e50 2024-09-10T07:41:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f500221a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b6dc0 con 0x5644b666bc70 2024-09-10T07:41:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f500221a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b58f0 con 0x5644b666c590 2024-09-10T07:41:15.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f4ffd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b6676e50 0x5644b668f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f4ffd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b6676e50 0x5644b668f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48648/0 (socket says 172.21.15.46:48648) 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f4ffd800640 1 -- 172.21.15.46:0/4282329809 learned_addr learned my addr 172.21.15.46:0/4282329809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f4ffce00640 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666bc70 0x5644b666e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.065+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666c590 0x5644b668ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4ffce00640 1 -- 172.21.15.46:0/4282329809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666c590 msgr2=0x5644b668ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4ffce00640 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666c590 0x5644b668ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4ffce00640 1 -- 172.21.15.46:0/4282329809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b6676e50 msgr2=0x5644b668f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4ffce00640 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b6676e50 0x5644b668f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4ffce00640 1 -- 172.21.15.46:0/4282329809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b666baa0 con 0x5644b666bc70 2024-09-10T07:41:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4ffce00640 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666bc70 0x5644b666e720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4fec050a60 tx=0x7f4fec009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998690e63d0e3f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4fe9600640 1 -- 172.21.15.46:0/4282329809 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec04a070 con 0x5644b666bc70 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4fe9600640 1 -- 172.21.15.46:0/4282329809 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fec036040 con 0x5644b666bc70 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.066+0000 7f4fe9600640 1 -- 172.21.15.46:0/4282329809 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec050400 con 0x5644b666bc70 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 -- 172.21.15.46:0/4282329809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666bc70 msgr2=0x5644b666e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666bc70 0x5644b666e720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4fec050a60 tx=0x7f4fec009540 comp rx=0 tx=0).stop 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 -- 172.21.15.46:0/4282329809 shutdown_connections 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b6676e50 0x5644b668f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666c590 0x5644b668ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 --2- 172.21.15.46:0/4282329809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b666bc70 0x5644b666e720 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 -- 172.21.15.46:0/4282329809 >> 172.21.15.46:0/4282329809 conn(0x5644b6668200 msgr2=0x5644b6672e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 -- 172.21.15.46:0/4282329809 shutdown_connections 2024-09-10T07:41:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 -- 172.21.15.46:0/4282329809 wait complete. 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.067+0000 7f500221a740 1 -- start start 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f500221a740 1 Processor -- start 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f500221a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666bc70 0x5644b667f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f500221a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666c590 0x5644b667f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f500221a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b6676e50 0x5644b667a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f500221a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b66f0 con 0x5644b666c590 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f500221a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b6020 con 0x5644b6676e50 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f500221a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b3200 con 0x5644b666bc70 2024-09-10T07:41:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f500221a740 10 client.0 _ll_register_callbacks cb 0x7f4fec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f4ff7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666c590 0x5644b667f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f4ffd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b6676e50 0x5644b667a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.068+0000 7f4ff7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666c590 0x5644b667f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48652/0 (socket says 172.21.15.46:48652) 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ff7e00640 1 -- 172.21.15.46:0/1422899983 learned_addr learned my addr 172.21.15.46:0/1422899983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ffce00640 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666bc70 0x5644b667f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ff7e00640 1 -- 172.21.15.46:0/1422899983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666bc70 msgr2=0x5644b667f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666bc70 0x5644b667f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ff7e00640 1 -- 172.21.15.46:0/1422899983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b6676e50 msgr2=0x5644b667a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b6676e50 0x5644b667a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ff7e00640 1 -- 172.21.15.46:0/1422899983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b66692b0 con 0x5644b666c590 2024-09-10T07:41:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ffd800640 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b6676e50 0x5644b667a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4ff7e00640 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666c590 0x5644b667f6e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4097f20 tx=0x7f4fe409c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ff396f2c2affec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4fdea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4fdea00640 1 -- 172.21.15.46:0/1422899983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4099dc0 con 0x5644b666c590 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f4fdea00640 1 -- 172.21.15.46:0/1422899983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4fe40cce70 con 0x5644b666c590 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.069+0000 7f500221a740 1 -- 172.21.15.46:0/1422899983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fec036040 con 0x5644b666c590 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f4fdea00640 1 -- 172.21.15.46:0/1422899983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4fe40cf330 con 0x5644b666c590 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f500221a740 10 client.16251 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f500221a740 1 -- 172.21.15.46:0/1422899983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5644b6663770 con 0x5644b666c590 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f4fdea00640 1 -- 172.21.15.46:0/1422899983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fec036040 con 0x5644b666c590 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f4fdea00640 1 -- 172.21.15.46:0/1422899983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f4fe40a42a0 con 0x5644b666c590 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f500221a740 10 client.16251 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f500221a740 10 client.16251 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f500221a740 20 client.16251 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f500221a740 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5644b66f1d80 0x5644b66f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f500221a740 4 client.16251 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f500221a740 1 -- 172.21.15.46:0/1422899983 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644b65d9e60 con 0x5644b66f1d80 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.070+0000 7f4ffce00640 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5644b66f1d80 0x5644b66f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.071+0000 7f4ffce00640 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5644b66f1d80 0x5644b66f41a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7a22335ece588c0 server_cookie=5991faed179b31fb in_seq=0 out_seq=0 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.071+0000 7f4fdea00640 10 client.16251 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.071+0000 7f4fdea00640 1 -- 172.21.15.46:0/1422899983 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644b65d9e60 con 0x5644b66f1d80 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.071+0000 7f4fdea00640 10 client.16251 handle_command_reply: tid=0 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.071+0000 7f500221a740 1 client.16251 shutdown 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.071+0000 7f500221a740 1 -- 172.21.15.46:0/1422899983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666c590 msgr2=0x5644b667f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.071+0000 7f500221a740 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666c590 0x5644b667f6e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4097f20 tx=0x7f4fe409c330 comp rx=0 tx=0).stop 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 1 -- 172.21.15.46:0/1422899983 shutdown_connections 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644b6676e50 0x5644b667a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b666c590 0x5644b667f6e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5644b66f1d80 0x5644b66f41a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 1 --2- 172.21.15.46:0/1422899983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5644b666bc70 0x5644b667f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 1 -- 172.21.15.46:0/1422899983 >> 172.21.15.46:0/1422899983 conn(0x5644b6668200 msgr2=0x5644b6673d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:15.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f4fdea00640 0 client.16251 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:41:15.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 1 -- 172.21.15.46:0/1422899983 shutdown_connections 2024-09-10T07:41:15.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 1 -- 172.21.15.46:0/1422899983 wait complete. 2024-09-10T07:41:15.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.072+0000 7f500221a740 20 client.16251 trim_cache size 0 max 16384 2024-09-10T07:41:15.076 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 -- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4fe00a2c00 msgr2=0x7f4fe00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4fe00a2c00 0x7f4fe00a50c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5644b6669060 tx=0x7f4fec03d040 comp rx=0 tx=0).stop 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 -- 172.21.15.46:0/1121003635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff81027a0 msgr2=0x7f4ff8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff81027a0 0x7f4ff8074a30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400e9f0 tx=0x7f4fe400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 -- 172.21.15.46:0/1121003635 shutdown_connections 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ff8106f20 0x7f4ff8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff81027a0 0x7f4ff8074a30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4fe00a2c00 0x7f4fe00a50c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 --2- 172.21.15.46:0/1121003635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff8101df0 0x7f4ff80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:15.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 -- 172.21.15.46:0/1121003635 >> 172.21.15.46:0/1121003635 conn(0x7f4ff80fd8b0 msgr2=0x7f4ff80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:15.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 -- 172.21.15.46:0/1121003635 shutdown_connections 2024-09-10T07:41:15.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:15.073+0000 7f4fff800640 1 -- 172.21.15.46:0/1121003635 wait complete. 2024-09-10T07:41:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 235.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:41:15.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:41:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:15 smithi181 ceph-mon[35397]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 869 op/s 2024-09-10T07:41:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:15 smithi063 ceph-mon[35020]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 869 op/s 2024-09-10T07:41:16.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:15 smithi046 ceph-mon[30705]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 869 op/s 2024-09-10T07:41:16.328 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:16.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:41:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 -- 172.21.15.46:0/1350387033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 msgr2=0x7f230c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 --2- 172.21.15.46:0/1350387033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 0x7f230c0748f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009900 tx=0x7f22fc02a480 comp rx=0 tx=0).stop 2024-09-10T07:41:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 -- 172.21.15.46:0/1350387033 shutdown_connections 2024-09-10T07:41:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 --2- 172.21.15.46:0/1350387033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f230c10e280 0x7f230c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 --2- 172.21.15.46:0/1350387033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c074e30 0x7f230c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 --2- 172.21.15.46:0/1350387033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 0x7f230c0748f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 -- 172.21.15.46:0/1350387033 >> 172.21.15.46:0/1350387033 conn(0x7f230c0fd740 msgr2=0x7f230c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 -- 172.21.15.46:0/1350387033 shutdown_connections 2024-09-10T07:41:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.971+0000 7f230b200640 1 -- 172.21.15.46:0/1350387033 wait complete. 2024-09-10T07:41:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.972+0000 7f230b200640 1 Processor -- start 2024-09-10T07:41:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.972+0000 7f230b200640 1 -- start start 2024-09-10T07:41:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f230b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 0x7f230c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f230b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f230c074e30 0x7f230c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f230b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c10e280 0x7f230c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f230b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f230c114000 con 0x7f230c074510 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f230b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f230c113e80 con 0x7f230c074e30 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f230b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f230c114180 con 0x7f230c10e280 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f2309e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 0x7f230c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f2309e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 0x7f230c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56772/0 (socket says 172.21.15.46:56772) 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f2309e00640 1 -- 172.21.15.46:0/1340734249 learned_addr learned my addr 172.21.15.46:0/1340734249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f2309400640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f230c074e30 0x7f230c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.973+0000 7f230a800640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c10e280 0x7f230c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f2309e00640 1 -- 172.21.15.46:0/1340734249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c10e280 msgr2=0x7f230c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f2309e00640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c10e280 0x7f230c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f2309e00640 1 -- 172.21.15.46:0/1340734249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f230c074e30 msgr2=0x7f230c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f2309e00640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f230c074e30 0x7f230c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f2309e00640 1 -- 172.21.15.46:0/1340734249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c19f450 con 0x7f230c074510 2024-09-10T07:41:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f2309e00640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 0x7f230c19a4a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22fc004770 tx=0x7f22fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f2302a00640 1 -- 172.21.15.46:0/1340734249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc03f070 con 0x7f230c074510 2024-09-10T07:41:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f2302a00640 1 -- 172.21.15.46:0/1340734249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc031ca0 con 0x7f230c074510 2024-09-10T07:41:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f230b200640 1 -- 172.21.15.46:0/1340734249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c19f6e0 con 0x7f230c074510 2024-09-10T07:41:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.974+0000 7f230b200640 1 -- 172.21.15.46:0/1340734249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c1a7020 con 0x7f230c074510 2024-09-10T07:41:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.975+0000 7f2302a00640 1 -- 172.21.15.46:0/1340734249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc035030 con 0x7f230c074510 2024-09-10T07:41:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f23139c6740 1 Processor -- start 2024-09-10T07:41:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f2302a00640 1 -- 172.21.15.46:0/1340734249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f22fc042070 con 0x7f230c074510 2024-09-10T07:41:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f23139c6740 1 -- start start 2024-09-10T07:41:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f23139c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f6b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f23139c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f6b680 0x560050f7e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f23139c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f7e830 0x560050f80c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f23139c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560050ed13e0 con 0x560050f6ad60 2024-09-10T07:41:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f23139c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560050e78d80 con 0x560050f7e830 2024-09-10T07:41:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.978+0000 7f23139c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560050ecd3a0 con 0x560050f6b680 2024-09-10T07:41:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f230a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f6b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f2309400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f7e830 0x560050f80c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f2309400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f7e830 0x560050f80c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53652/0 (socket says 172.21.15.46:53652) 2024-09-10T07:41:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f230a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f6b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56776/0 (socket says 172.21.15.46:56776) 2024-09-10T07:41:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f2309400640 1 -- 172.21.15.46:0/4139746854 learned_addr learned my addr 172.21.15.46:0/4139746854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f2309e00640 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f6b680 0x560050f7e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f230a800640 1 -- 172.21.15.46:0/4139746854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f6b680 msgr2=0x560050f7e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f230a800640 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f6b680 0x560050f7e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f230a800640 1 -- 172.21.15.46:0/4139746854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f7e830 msgr2=0x560050f80c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f230a800640 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f7e830 0x560050f80c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.979+0000 7f230a800640 1 -- 172.21.15.46:0/4139746854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560050e97ae0 con 0x560050f6ad60 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.980+0000 7f2309e00640 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f6b680 0x560050f7e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.980+0000 7f230a800640 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f6b140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2304005fd0 tx=0x7f2304011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6938594e41de3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.980+0000 7f2302a00640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.980+0000 7f22f7400640 1 -- 172.21.15.46:0/4139746854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304004d20 con 0x560050f6ad60 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.980+0000 7f22f7400640 1 -- 172.21.15.46:0/4139746854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2304004050 con 0x560050f6ad60 2024-09-10T07:41:16.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.980+0000 7f22f7400640 1 -- 172.21.15.46:0/4139746854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304005280 con 0x560050f6ad60 2024-09-10T07:41:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.980+0000 7f2302a00640 1 -- 172.21.15.46:0/1340734249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f22fc0efaa0 con 0x7f230c074510 2024-09-10T07:41:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.981+0000 7f2309e00640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.981+0000 7f23139c6740 1 -- 172.21.15.46:0/4139746854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 msgr2=0x560050f6b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.981+0000 7f23139c6740 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f6b140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2304005fd0 tx=0x7f2304011810 comp rx=0 tx=0).stop 2024-09-10T07:41:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 -- 172.21.15.46:0/4139746854 shutdown_connections 2024-09-10T07:41:16.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f7e830 0x560050f80c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f6b680 0x560050f7e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 --2- 172.21.15.46:0/4139746854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f6b140 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 -- 172.21.15.46:0/4139746854 >> 172.21.15.46:0/4139746854 conn(0x560050f673d0 msgr2=0x560050f677b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 -- 172.21.15.46:0/4139746854 shutdown_connections 2024-09-10T07:41:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 -- 172.21.15.46:0/4139746854 wait complete. 2024-09-10T07:41:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 -- start start 2024-09-10T07:41:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.982+0000 7f23139c6740 1 Processor -- start 2024-09-10T07:41:16.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.983+0000 7f2309e00640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22f00a2b80 0x7f22f00a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b89a0 tx=0x7f22fc0f34e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.983+0000 7f23139c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f7b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.983+0000 7f23139c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6b680 0x560050f7bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.983+0000 7f23139c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f7c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.983+0000 7f23139c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560050f813f0 con 0x560050f6ad60 2024-09-10T07:41:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.983+0000 7f23139c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560050e8c0f0 con 0x560050f6b680 2024-09-10T07:41:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.983+0000 7f23139c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560050f81570 con 0x560050f7e830 2024-09-10T07:41:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.983+0000 7f23139c6740 10 client.0 _ll_register_callbacks cb 0x560050e73bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f230a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f7b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f230a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f7b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56792/0 (socket says 172.21.15.46:56792) 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f230a800640 1 -- 172.21.15.46:0/2910444458 learned_addr learned my addr 172.21.15.46:0/2910444458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f2309400640 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6b680 0x560050f7bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f230a800640 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f7c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f2309400640 1 -- 172.21.15.46:0/2910444458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 msgr2=0x560050f7c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f2309400640 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f7c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f2309400640 1 -- 172.21.15.46:0/2910444458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 msgr2=0x560050f7b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f2309400640 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f7b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f2309400640 1 -- 172.21.15.46:0/2910444458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560050c423c0 con 0x560050f6b680 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.984+0000 7f230a800640 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f7b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.985+0000 7f2309400640 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6b680 0x560050f7bcb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f22f800b6c0 tx=0x7f22f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b848071225b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:16.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.985+0000 7f22ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.985+0000 7f22ebe00640 1 -- 172.21.15.46:0/2910444458 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8004020 con 0x560050f6b680 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.985+0000 7f22ebe00640 1 -- 172.21.15.46:0/2910444458 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f22f80041c0 con 0x560050f6b680 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.985+0000 7f23139c6740 1 -- 172.21.15.46:0/2910444458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2304004050 con 0x560050f6b680 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.985+0000 7f22ebe00640 1 -- 172.21.15.46:0/2910444458 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f22f80043c0 con 0x560050f6b680 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.985+0000 7f23139c6740 10 client.26552 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.985+0000 7f23139c6740 1 -- 172.21.15.46:0/2910444458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560050f8d4c0 con 0x560050f6b680 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.986+0000 7f22ebe00640 1 -- 172.21.15.46:0/2910444458 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2304004050 con 0x560050f6b680 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.986+0000 7f22ebe00640 1 -- 172.21.15.46:0/2910444458 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f22f8010640 con 0x560050f6b680 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.986+0000 7f23139c6740 10 client.26552 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.986+0000 7f23139c6740 10 client.26552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.986+0000 7f23139c6740 20 client.26552 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.986+0000 7f23139c6740 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560050f8ea20 0x560050f90e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:16.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.986+0000 7f23139c6740 4 client.26552 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.986+0000 7f23139c6740 1 -- 172.21.15.46:0/2910444458 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560050ede950 con 0x560050f8ea20 2024-09-10T07:41:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.987+0000 7f230a800640 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560050f8ea20 0x560050f90e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.987+0000 7f230a800640 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560050f8ea20 0x560050f90e70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cb9b3a2549a7397 server_cookie=3c3a2301a8b8e5a8 in_seq=0 out_seq=0 2024-09-10T07:41:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.987+0000 7f22ebe00640 10 client.26552 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.988+0000 7f22ebe00640 1 -- 172.21.15.46:0/2910444458 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560050ede950 con 0x560050f8ea20 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.988+0000 7f22ebe00640 10 client.26552 handle_command_reply: tid=0 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.988+0000 7f23139c6740 1 client.26552 shutdown 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.988+0000 7f23139c6740 1 -- 172.21.15.46:0/2910444458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6b680 msgr2=0x560050f7bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.988+0000 7f23139c6740 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6b680 0x560050f7bcb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f22f800b6c0 tx=0x7f22f8011810 comp rx=0 tx=0).stop 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 1 -- 172.21.15.46:0/2910444458 shutdown_connections 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560050f8ea20 0x560050f90e70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f7c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6b680 0x560050f7bcb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 1 --2- 172.21.15.46:0/2910444458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6ad60 0x560050f7b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 1 -- 172.21.15.46:0/2910444458 >> 172.21.15.46:0/2910444458 conn(0x560050f673d0 msgr2=0x560050f6d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f22ebe00640 0 client.26552 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:41:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 1 -- 172.21.15.46:0/2910444458 shutdown_connections 2024-09-10T07:41:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 1 -- 172.21.15.46:0/2910444458 wait complete. 2024-09-10T07:41:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:16.989+0000 7f23139c6740 20 client.26552 trim_cache size 0 max 16384 2024-09-10T07:41:17.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.017+0000 7f23139c6740 1 Processor -- start 2024-09-10T07:41:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.017+0000 7f23139c6740 1 -- start start 2024-09-10T07:41:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.017+0000 7f23139c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f8add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.017+0000 7f23139c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f7c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.017+0000 7f23139c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f6cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.017+0000 7f23139c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1ac890 con 0x560050f6b680 2024-09-10T07:41:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.017+0000 7f23139c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1ae430 con 0x560050f6ad60 2024-09-10T07:41:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.017+0000 7f23139c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1acf60 con 0x560050f7e830 2024-09-10T07:41:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f7c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f230a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f8add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f7c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56794/0 (socket says 172.21.15.46:56794) 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 -- 172.21.15.46:0/276169826 learned_addr learned my addr 172.21.15.46:0/276169826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f230a800640 1 --2- 172.21.15.46:0/276169826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f6cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 -- 172.21.15.46:0/276169826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 msgr2=0x560050f6cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 --2- 172.21.15.46:0/276169826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f6cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 -- 172.21.15.46:0/276169826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 msgr2=0x560050f8add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 --2- 172.21.15.46:0/276169826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f8add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 -- 172.21.15.46:0/276169826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560050f6c240 con 0x560050f6b680 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.018+0000 7f2309400640 1 --2- 172.21.15.46:0/276169826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f7c040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f22f8001eb0 tx=0x7f22f80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a39c6dfc59d358 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.019+0000 7f22f6a00640 1 -- 172.21.15.46:0/276169826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f802d030 con 0x560050f6b680 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.019+0000 7f22f6a00640 1 -- 172.21.15.46:0/276169826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f800b480 con 0x560050f6b680 2024-09-10T07:41:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.019+0000 7f22f6a00640 1 -- 172.21.15.46:0/276169826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8004660 con 0x560050f6b680 2024-09-10T07:41:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 -- 172.21.15.46:0/276169826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 msgr2=0x560050f7c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 --2- 172.21.15.46:0/276169826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f7c040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f22f8001eb0 tx=0x7f22f80058d0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 -- 172.21.15.46:0/276169826 shutdown_connections 2024-09-10T07:41:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 --2- 172.21.15.46:0/276169826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f6cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 --2- 172.21.15.46:0/276169826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f7c040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 --2- 172.21.15.46:0/276169826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f8add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 -- 172.21.15.46:0/276169826 >> 172.21.15.46:0/276169826 conn(0x560050f673d0 msgr2=0x560050f6c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 -- 172.21.15.46:0/276169826 shutdown_connections 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.020+0000 7f23139c6740 1 -- 172.21.15.46:0/276169826 wait complete. 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 1 -- start start 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 1 Processor -- start 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f86d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1add60 con 0x560050f6b680 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1ad690 con 0x560050f6ad60 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1aa870 con 0x560050f7e830 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.021+0000 7f23139c6740 10 client.0 _ll_register_callbacks cb 0x7f22f8005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f2309400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f2309400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f820c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56800/0 (socket says 172.21.15.46:56800) 2024-09-10T07:41:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f2309400640 1 -- 172.21.15.46:0/406056928 learned_addr learned my addr 172.21.15.46:0/406056928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:17.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f230a800640 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f86d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:17.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f230a800640 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f230a800640 1 -- 172.21.15.46:0/406056928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 msgr2=0x560050f82600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f230a800640 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f82600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f230a800640 1 -- 172.21.15.46:0/406056928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 msgr2=0x560050f820c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.022+0000 7f230a800640 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f820c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.023+0000 7f230a800640 1 -- 172.21.15.46:0/406056928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560050f82c40 con 0x560050f6ad60 2024-09-10T07:41:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.023+0000 7f2309400640 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.023+0000 7f230a800640 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f86d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2304009d40 tx=0x7f2304011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953457fd6a2a8c5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.023+0000 7f22ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.023+0000 7f22ebe00640 1 -- 172.21.15.46:0/406056928 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304003e40 con 0x560050f6ad60 2024-09-10T07:41:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f22ebe00640 1 -- 172.21.15.46:0/406056928 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2304028450 con 0x560050f6ad60 2024-09-10T07:41:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f23139c6740 1 -- 172.21.15.46:0/406056928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f800b480 con 0x560050f6ad60 2024-09-10T07:41:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f22ebe00640 1 -- 172.21.15.46:0/406056928 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2304028650 con 0x560050f6ad60 2024-09-10T07:41:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f23139c6740 10 client.26558 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f23139c6740 1 -- 172.21.15.46:0/406056928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560050ff95a0 con 0x560050f6ad60 2024-09-10T07:41:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f22ebe00640 1 -- 172.21.15.46:0/406056928 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f800b480 con 0x560050f6ad60 2024-09-10T07:41:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f22ebe00640 1 -- 172.21.15.46:0/406056928 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f2304025040 con 0x560050f6ad60 2024-09-10T07:41:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f23139c6740 10 client.26558 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.024+0000 7f23139c6740 10 client.26558 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:41:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.025+0000 7f23139c6740 20 client.26558 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.025+0000 7f23139c6740 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560050ff9ab0 0x560050ffbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.025+0000 7f23139c6740 4 client.26558 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:41:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.025+0000 7f23139c6740 1 -- 172.21.15.46:0/406056928 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560050ed8e60 con 0x560050ff9ab0 2024-09-10T07:41:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.025+0000 7f2309400640 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560050ff9ab0 0x560050ffbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.025+0000 7f2309400640 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560050ff9ab0 0x560050ffbed0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46d2b46b4b3c372c server_cookie=b174b83a8605f3a0 in_seq=0 out_seq=0 2024-09-10T07:41:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.025+0000 7f22ebe00640 10 client.26558 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:41:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f22ebe00640 1 -- 172.21.15.46:0/406056928 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560050ed8e60 con 0x560050ff9ab0 2024-09-10T07:41:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f22ebe00640 10 client.26558 handle_command_reply: tid=0 2024-09-10T07:41:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 client.26558 shutdown 2024-09-10T07:41:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 -- 172.21.15.46:0/406056928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 msgr2=0x560050f86d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f86d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2304009d40 tx=0x7f2304011810 comp rx=0 tx=0).stop 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 -- 172.21.15.46:0/406056928 shutdown_connections 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560050ff9ab0 0x560050ffbed0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560050f7e830 0x560050f82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f22ebe00640 0 client.26558 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560050f6b680 0x560050f820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 --2- 172.21.15.46:0/406056928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560050f6ad60 0x560050f86d90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 -- 172.21.15.46:0/406056928 >> 172.21.15.46:0/406056928 conn(0x560050f673d0 msgr2=0x560050f6cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 -- 172.21.15.46:0/406056928 shutdown_connections 2024-09-10T07:41:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 1 -- 172.21.15.46:0/406056928 wait complete. 2024-09-10T07:41:17.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.026+0000 7f23139c6740 20 client.26558 trim_cache size 0 max 16384 2024-09-10T07:41:17.032 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 -- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22f00a2b80 msgr2=0x7f22f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22f00a2b80 0x7f22f00a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b89a0 tx=0x7f22fc0f34e0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 -- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 msgr2=0x7f230c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 0x7f230c19a4a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22fc004770 tx=0x7f22fc003c10 comp rx=0 tx=0).stop 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 -- 172.21.15.46:0/1340734249 shutdown_connections 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230c10e280 0x7f230c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f230c074e30 0x7f230c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 --2- 172.21.15.46:0/1340734249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c074510 0x7f230c19a4a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 -- 172.21.15.46:0/1340734249 >> 172.21.15.46:0/1340734249 conn(0x7f230c0fd740 msgr2=0x7f230c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.027+0000 7f230b200640 1 -- 172.21.15.46:0/1340734249 shutdown_connections 2024-09-10T07:41:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:17.028+0000 7f230b200640 1 -- 172.21.15.46:0/1340734249 wait complete. 2024-09-10T07:41:17.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:41:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:17.963 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:17 smithi046 ceph-mon[30705]: pgmap v539: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 796 op/s 2024-09-10T07:41:18.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:17 smithi181 ceph-mon[35397]: pgmap v539: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 796 op/s 2024-09-10T07:41:18.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:17 smithi063 ceph-mon[35020]: pgmap v539: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 796 op/s 2024-09-10T07:41:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.574+0000 7fa5a7800640 1 -- 172.21.15.46:0/1659691484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 msgr2=0x7fa5a0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.574+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1659691484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 0x7fa5a0103110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa590009900 tx=0x7fa59002a480 comp rx=0 tx=0).stop 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.574+0000 7fa5a7800640 1 -- 172.21.15.46:0/1659691484 shutdown_connections 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.574+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1659691484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5a010c160 0x7fa5a010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.574+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1659691484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a0103650 0x7fa5a010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.574+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1659691484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 0x7fa5a0103110 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.574+0000 7fa5a7800640 1 -- 172.21.15.46:0/1659691484 >> 172.21.15.46:0/1659691484 conn(0x7fa5a00fc7d0 msgr2=0x7fa5a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.575+0000 7fa5a7800640 1 -- 172.21.15.46:0/1659691484 shutdown_connections 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.575+0000 7fa5a7800640 1 -- 172.21.15.46:0/1659691484 wait complete. 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.576+0000 7fa5a7800640 1 Processor -- start 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.576+0000 7fa5a7800640 1 -- start start 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 0x7fa5a019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a7800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5a0103650 0x7fa5a019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a7800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a010c160 0x7fa5a019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0111f60 con 0x7fa5a0102d30 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a7800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0111de0 con 0x7fa5a0103650 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a7800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01120e0 con 0x7fa5a010c160 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 0x7fa5a019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 0x7fa5a019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56816/0 (socket says 172.21.15.46:56816) 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a4e00640 1 -- 172.21.15.46:0/1624716366 learned_addr learned my addr 172.21.15.46:0/1624716366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa597e00640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5a0103650 0x7fa5a019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a5800640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a010c160 0x7fa5a019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a4e00640 1 -- 172.21.15.46:0/1624716366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a010c160 msgr2=0x7fa5a019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a4e00640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a010c160 0x7fa5a019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a4e00640 1 -- 172.21.15.46:0/1624716366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5a0103650 msgr2=0x7fa5a019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a4e00640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5a0103650 0x7fa5a019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a4e00640 1 -- 172.21.15.46:0/1624716366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a019f630 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.577+0000 7fa5a5800640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a010c160 0x7fa5a019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.578+0000 7fa5a4e00640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 0x7fa5a019a680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa590006010 tx=0x7fa5900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.578+0000 7fa59e000640 1 -- 172.21.15.46:0/1624716366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59003f030 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.578+0000 7fa59e000640 1 -- 172.21.15.46:0/1624716366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa590003ce0 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.578+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a019f8c0 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.578+0000 7fa59e000640 1 -- 172.21.15.46:0/1624716366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590035070 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.578+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a01a7200 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.579+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a006b070 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.581+0000 7fa59e000640 1 -- 172.21.15.46:0/1624716366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa590042030 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.583+0000 7fa59e000640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa5840a2b80 0x7fa5840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.584+0000 7fa59e000640 1 -- 172.21.15.46:0/1624716366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa5900efbd0 con 0x7fa5a0102d30 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.584+0000 7fa597e00640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa5840a2b80 0x7fa5840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.584+0000 7fa597e00640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa5840a2b80 0x7fa5840a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa5a01029b0 tx=0x7fa588005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:18.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.586+0000 7fa59e000640 1 -- 172.21.15.46:0/1624716366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5900bc810 con 0x7fa5a0102d30 2024-09-10T07:41:18.656 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:18.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.820+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5a01a73d0 con 0x7fa5a0102d30 2024-09-10T07:41:18.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.821+0000 7fa59e000640 1 -- 172.21.15.46:0/1624716366 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11975 (secure 0 0 0) 0x7fa5900bc110 con 0x7fa5a0102d30 2024-09-10T07:41:18.821 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:41:18.821 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"btime":"2024-09-10T07:40:35:758945+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:40:35.499710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:41:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 39 2024-09-10T07:41:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa5840a2b80 msgr2=0x7fa5840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa5840a2b80 0x7fa5840a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa5a01029b0 tx=0x7fa588005f90 comp rx=0 tx=0).stop 2024-09-10T07:41:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 msgr2=0x7fa5a019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 0x7fa5a019a680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa590006010 tx=0x7fa5900042f0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 shutdown_connections 2024-09-10T07:41:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a010c160 0x7fa5a019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5a0103650 0x7fa5a019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa5840a2b80 0x7fa5840a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 --2- 172.21.15.46:0/1624716366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a0102d30 0x7fa5a019a680 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 >> 172.21.15.46:0/1624716366 conn(0x7fa5a00fc7d0 msgr2=0x7fa5a00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 shutdown_connections 2024-09-10T07:41:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:18.824+0000 7fa5a7800640 1 -- 172.21.15.46:0/1624716366 wait complete. 2024-09-10T07:41:19.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:18 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1624716366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:41:19.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:18 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1624716366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:41:19.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:18 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1624716366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:41:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 -- 172.21.15.46:0/798746452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc077040 msgr2=0x7f69cc112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 --2- 172.21.15.46:0/798746452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc077040 0x7f69cc112330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009900 tx=0x7f69b402a410 comp rx=0 tx=0).stop 2024-09-10T07:41:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 -- 172.21.15.46:0/798746452 shutdown_connections 2024-09-10T07:41:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 --2- 172.21.15.46:0/798746452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69cc112870 0x7f69cc114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 --2- 172.21.15.46:0/798746452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc077040 0x7f69cc112330 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 --2- 172.21.15.46:0/798746452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69cc076720 0x7f69cc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 -- 172.21.15.46:0/798746452 >> 172.21.15.46:0/798746452 conn(0x7f69cc0ff8d0 msgr2=0x7f69cc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 -- 172.21.15.46:0/798746452 shutdown_connections 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.400+0000 7f69cb200640 1 -- 172.21.15.46:0/798746452 wait complete. 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.401+0000 7f69cb200640 1 Processor -- start 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69cb200640 1 -- start start 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69cb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69cc076720 0x7f69cc1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69cb200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69cc077040 0x7f69cc1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69cb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc112870 0x7f69cc19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69cb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc118400 con 0x7f69cc112870 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69cb200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc118280 con 0x7f69cc077040 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69cb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc118580 con 0x7f69cc076720 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc112870 0x7f69cc19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc112870 0x7f69cc19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56832/0 (socket says 172.21.15.46:56832) 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69ca800640 1 -- 172.21.15.46:0/1989348083 learned_addr learned my addr 172.21.15.46:0/1989348083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69c9e00640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69cc076720 0x7f69cc1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.402+0000 7f69c1400640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69cc077040 0x7f69cc1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69ca800640 1 -- 172.21.15.46:0/1989348083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69cc076720 msgr2=0x7f69cc1a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69ca800640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69cc076720 0x7f69cc1a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69ca800640 1 -- 172.21.15.46:0/1989348083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69cc077040 msgr2=0x7f69cc1a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69ca800640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69cc077040 0x7f69cc1a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69ca800640 1 -- 172.21.15.46:0/1989348083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc19efc0 con 0x7f69cc112870 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69ca800640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc112870 0x7f69cc19e750 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e940 tx=0x7f69c400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69c3400640 1 -- 172.21.15.46:0/1989348083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c400ccc0 con 0x7f69cc112870 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69c3400640 1 -- 172.21.15.46:0/1989348083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c4005500 con 0x7f69cc112870 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69c3400640 1 -- 172.21.15.46:0/1989348083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4010690 con 0x7f69cc112870 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.403+0000 7f69cb200640 1 -- 172.21.15.46:0/1989348083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc19f2b0 con 0x7f69cc112870 2024-09-10T07:41:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.404+0000 7f69cb200640 1 -- 172.21.15.46:0/1989348083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc1ab430 con 0x7f69cc112870 2024-09-10T07:41:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69c3400640 1 -- 172.21.15.46:0/1989348083 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f69c40046c0 con 0x7f69cc112870 2024-09-10T07:41:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69d37ca740 1 Processor -- start 2024-09-10T07:41:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69d37ca740 1 -- start start 2024-09-10T07:41:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774d6020 0x55b4774d8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69d37ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b4774313e0 con 0x55b4774d6020 2024-09-10T07:41:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69d37ca740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b4773d8d80 con 0x55b4774cb760 2024-09-10T07:41:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.407+0000 7f69d37ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b47742d3a0 con 0x55b4774cae40 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774d6020 0x55b4774d8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53710/0 (socket says 172.21.15.46:53710) 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c9e00640 1 -- 172.21.15.46:0/1196307374 learned_addr learned my addr 172.21.15.46:0/1196307374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c1400640 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c9e00640 1 -- 172.21.15.46:0/1196307374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 msgr2=0x55b4774cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c9e00640 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c9e00640 1 -- 172.21.15.46:0/1196307374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774d6020 msgr2=0x55b4774d8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c9e00640 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774d6020 0x55b4774d8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c9e00640 1 -- 172.21.15.46:0/1196307374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4773f7ae0 con 0x55b4774cb760 2024-09-10T07:41:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.408+0000 7f69c1400640 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.409+0000 7f69c3400640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69b00a2c00 0x7f69b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.409+0000 7f69c9e00640 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d5ae0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0114f0 tx=0x7f69bc0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa25cd328702125 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.409+0000 7f69c3400640 1 -- 172.21.15.46:0/1989348083 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f69c40cf650 con 0x7f69cc112870 2024-09-10T07:41:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.409+0000 7f69c1400640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69b00a2c00 0x7f69b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.409+0000 7f69bb400640 1 -- 172.21.15.46:0/1196307374 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc004d20 con 0x55b4774cb760 2024-09-10T07:41:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.409+0000 7f69bb400640 1 -- 172.21.15.46:0/1196307374 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc004050 con 0x55b4774cb760 2024-09-10T07:41:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.409+0000 7f69bb400640 1 -- 172.21.15.46:0/1196307374 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00f7c0 con 0x55b4774cb760 2024-09-10T07:41:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69d37ca740 1 -- 172.21.15.46:0/1196307374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 msgr2=0x55b4774d5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69c1400640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69b00a2c00 0x7f69b00a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009230 tx=0x7f69b4041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69d37ca740 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d5ae0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0114f0 tx=0x7f69bc0119b0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69d37ca740 1 -- 172.21.15.46:0/1196307374 shutdown_connections 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69d37ca740 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774d6020 0x55b4774d8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69d37ca740 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d5ae0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69d37ca740 1 --2- 172.21.15.46:0/1196307374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69d37ca740 1 -- 172.21.15.46:0/1196307374 >> 172.21.15.46:0/1196307374 conn(0x55b4774c73d0 msgr2=0x55b4774c77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.410+0000 7f69d37ca740 1 -- 172.21.15.46:0/1196307374 shutdown_connections 2024-09-10T07:41:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.411+0000 7f69d37ca740 1 -- 172.21.15.46:0/1196307374 wait complete. 2024-09-10T07:41:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.411+0000 7f69d37ca740 1 -- start start 2024-09-10T07:41:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.411+0000 7f69d37ca740 1 Processor -- start 2024-09-10T07:41:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774d6020 0x55b4774d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69d37ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b4774d8bf0 con 0x55b4774cb760 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69d37ca740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b4773ec0f0 con 0x55b4774d6020 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69d37ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b4774d8d70 con 0x55b4774cae40 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69d37ca740 10 client.0 _ll_register_callbacks cb 0x55b4773d3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69c9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774d3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56864/0 (socket says 172.21.15.46:56864) 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69c9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57418/0 (socket says 172.21.15.46:57418) 2024-09-10T07:41:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69c1400640 1 -- 172.21.15.46:0/2570730366 learned_addr learned my addr 172.21.15.46:0/2570730366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.412+0000 7f69ca800640 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774d6020 0x55b4774d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.413+0000 7f69c9e00640 1 -- 172.21.15.46:0/2570730366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774d6020 msgr2=0x55b4774d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.413+0000 7f69c9e00640 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774d6020 0x55b4774d39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.413+0000 7f69c9e00640 1 -- 172.21.15.46:0/2570730366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 msgr2=0x55b4774d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.413+0000 7f69c9e00640 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.413+0000 7f69c9e00640 1 -- 172.21.15.46:0/2570730366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4771a23c0 con 0x55b4774cae40 2024-09-10T07:41:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.413+0000 7f69c9e00640 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774d2f50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f69bc006010 tx=0x7f69bc004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8f3afeb699a5d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.413+0000 7f69afe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:41:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.413+0000 7f69afe00640 1 -- 172.21.15.46:0/2570730366 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc028020 con 0x55b4774cae40 2024-09-10T07:41:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.414+0000 7f69afe00640 1 -- 172.21.15.46:0/2570730366 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f69bc004410 con 0x55b4774cae40 2024-09-10T07:41:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.414+0000 7f69d37ca740 1 -- 172.21.15.46:0/2570730366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc004050 con 0x55b4774cae40 2024-09-10T07:41:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.414+0000 7f69afe00640 1 -- 172.21.15.46:0/2570730366 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f69bc033050 con 0x55b4774cae40 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.414+0000 7f69d37ca740 10 client.26557 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.414+0000 7f69d37ca740 1 -- 172.21.15.46:0/2570730366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b4774e4c80 con 0x55b4774cae40 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.414+0000 7f69afe00640 1 -- 172.21.15.46:0/2570730366 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc004050 con 0x55b4774cae40 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.415+0000 7f69afe00640 1 -- 172.21.15.46:0/2570730366 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f69bc0234e0 con 0x55b4774cae40 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.415+0000 7f69d37ca740 10 client.26557 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.415+0000 7f69d37ca740 10 client.26557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.415+0000 7f69d37ca740 20 client.26557 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.415+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b4774e60e0 0x55b4774e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.415+0000 7f69d37ca740 4 client.26557 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.415+0000 7f69d37ca740 1 -- 172.21.15.46:0/2570730366 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b47743e950 con 0x55b4774e60e0 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.415+0000 7f69c1400640 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b4774e60e0 0x55b4774e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.416+0000 7f69c1400640 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b4774e60e0 0x55b4774e84d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a944f44f36f5c6a7 server_cookie=9bb5ce618b2bf7f2 in_seq=0 out_seq=0 2024-09-10T07:41:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.416+0000 7f69afe00640 10 client.26557 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69afe00640 1 -- 172.21.15.46:0/2570730366 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b47743e950 con 0x55b4774e60e0 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69afe00640 10 client.26557 handle_command_reply: tid=0 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 client.26557 shutdown 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 -- 172.21.15.46:0/2570730366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 msgr2=0x55b4774d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774d2f50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f69bc006010 tx=0x7f69bc004c70 comp rx=0 tx=0).stop 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 -- 172.21.15.46:0/2570730366 shutdown_connections 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b4774e60e0 0x55b4774e84d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774d6020 0x55b4774d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2570730366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774cae40 0x55b4774d2f50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69afe00640 0 client.26557 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:41:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 -- 172.21.15.46:0/2570730366 >> 172.21.15.46:0/2570730366 conn(0x55b4774c73d0 msgr2=0x55b4774cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 -- 172.21.15.46:0/2570730366 shutdown_connections 2024-09-10T07:41:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.417+0000 7f69d37ca740 1 -- 172.21.15.46:0/2570730366 wait complete. 2024-09-10T07:41:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.418+0000 7f69d37ca740 20 client.26557 trim_cache size 0 max 16384 2024-09-10T07:41:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.432+0000 7f69d37ca740 1 Processor -- start 2024-09-10T07:41:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.432+0000 7f69d37ca740 1 -- start start 2024-09-10T07:41:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cae40 0x55b4774edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69d37ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b50a0 con 0x55b4774cb760 2024-09-10T07:41:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69d37ca740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b6c40 con 0x55b4774cae40 2024-09-10T07:41:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69d37ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b5770 con 0x55b4774d6020 2024-09-10T07:41:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774ee240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56878/0 (socket says 172.21.15.46:56878) 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69c1400640 1 -- 172.21.15.46:0/2456519128 learned_addr learned my addr 172.21.15.46:0/2456519128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69c9e00640 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cae40 0x55b4774edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69ca800640 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69c1400640 1 -- 172.21.15.46:0/2456519128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 msgr2=0x55b4774cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.433+0000 7f69c1400640 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69c1400640 1 -- 172.21.15.46:0/2456519128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cae40 msgr2=0x55b4774edd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69c1400640 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cae40 0x55b4774edd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69c1400640 1 -- 172.21.15.46:0/2456519128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4774c6c80 con 0x55b4774cb760 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69c9e00640 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cae40 0x55b4774edd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69ca800640 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69c1400640 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774ee240 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006010 tx=0x7f69b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42298c1462b2043e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69baa00640 1 -- 172.21.15.46:0/2456519128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b405b070 con 0x55b4774cb760 2024-09-10T07:41:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69baa00640 1 -- 172.21.15.46:0/2456519128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b4004480 con 0x55b4774cb760 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.434+0000 7f69baa00640 1 -- 172.21.15.46:0/2456519128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40569e0 con 0x55b4774cb760 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 -- 172.21.15.46:0/2456519128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 msgr2=0x55b4774ee240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774ee240 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006010 tx=0x7f69b4009260 comp rx=0 tx=0).stop 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 -- 172.21.15.46:0/2456519128 shutdown_connections 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cb760 0x55b4774ee240 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 --2- 172.21.15.46:0/2456519128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cae40 0x55b4774edd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 -- 172.21.15.46:0/2456519128 >> 172.21.15.46:0/2456519128 conn(0x55b4774c73d0 msgr2=0x55b4774d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 -- 172.21.15.46:0/2456519128 shutdown_connections 2024-09-10T07:41:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.435+0000 7f69d37ca740 1 -- 172.21.15.46:0/2456519128 wait complete. 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 1 -- start start 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 1 Processor -- start 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cae40 0x55b4774de3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b6570 con 0x55b4774cae40 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b5ea0 con 0x55b4774cb760 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b3080 con 0x55b4774d6020 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.436+0000 7f69d37ca740 10 client.0 _ll_register_callbacks cb 0x7f69bc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cae40 0x55b4774de3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53736/0 (socket says 172.21.15.46:53736) 2024-09-10T07:41:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 -- 172.21.15.46:0/4029558075 learned_addr learned my addr 172.21.15.46:0/4029558075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69ca800640 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 -- 172.21.15.46:0/4029558075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 msgr2=0x55b4774d9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774d9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 -- 172.21.15.46:0/4029558075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cae40 msgr2=0x55b4774de3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cae40 0x55b4774de3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 -- 172.21.15.46:0/4029558075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4774da1e0 con 0x55b4774cb760 2024-09-10T07:41:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c9e00640 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cae40 0x55b4774de3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69c1400640 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d9760 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f69b4059040 tx=0x7f69b40095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da64ef5ac7c1bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.437+0000 7f69afe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.438+0000 7f69afe00640 1 -- 172.21.15.46:0/4029558075 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4063020 con 0x55b4774cb760 2024-09-10T07:41:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.438+0000 7f69d37ca740 1 -- 172.21.15.46:0/4029558075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b4004480 con 0x55b4774cb760 2024-09-10T07:41:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.438+0000 7f69afe00640 1 -- 172.21.15.46:0/4029558075 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f69b4056760 con 0x55b4774cb760 2024-09-10T07:41:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.438+0000 7f69afe00640 1 -- 172.21.15.46:0/4029558075 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f69b406e050 con 0x55b4774cb760 2024-09-10T07:41:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.438+0000 7f69d37ca740 10 client.26582 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.438+0000 7f69d37ca740 1 -- 172.21.15.46:0/4029558075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b4775509a0 con 0x55b4774cb760 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.438+0000 7f69afe00640 1 -- 172.21.15.46:0/4029558075 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b4004480 con 0x55b4774cb760 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.439+0000 7f69afe00640 1 -- 172.21.15.46:0/4029558075 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f69b4003af0 con 0x55b4774cb760 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.439+0000 7f69d37ca740 10 client.26582 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.439+0000 7f69d37ca740 10 client.26582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.439+0000 7f69d37ca740 20 client.26582 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.439+0000 7f69d37ca740 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b477551030 0x55b477553450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.439+0000 7f69d37ca740 4 client.26582 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.439+0000 7f69d37ca740 1 -- 172.21.15.46:0/4029558075 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b477438e60 con 0x55b477551030 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.439+0000 7f69c9e00640 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b477551030 0x55b477553450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69c9e00640 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b477551030 0x55b477553450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=428a0b255f9c728b server_cookie=f12712be33e66e6b in_seq=0 out_seq=0 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69afe00640 10 client.26582 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69afe00640 1 -- 172.21.15.46:0/4029558075 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b477438e60 con 0x55b477551030 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69afe00640 10 client.26582 handle_command_reply: tid=0 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 client.26582 shutdown 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 -- 172.21.15.46:0/4029558075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 msgr2=0x55b4774d9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d9760 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f69b4059040 tx=0x7f69b40095d0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 -- 172.21.15.46:0/4029558075 shutdown_connections 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b477551030 0x55b477553450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b4774d6020 0x55b4774d9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4774cb760 0x55b4774d9760 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 --2- 172.21.15.46:0/4029558075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4774cae40 0x55b4774de3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69afe00640 0 client.26582 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:41:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 -- 172.21.15.46:0/4029558075 >> 172.21.15.46:0/4029558075 conn(0x55b4774c73d0 msgr2=0x55b4774cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 -- 172.21.15.46:0/4029558075 shutdown_connections 2024-09-10T07:41:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 1 -- 172.21.15.46:0/4029558075 wait complete. 2024-09-10T07:41:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.440+0000 7f69d37ca740 20 client.26582 trim_cache size 0 max 16384 2024-09-10T07:41:19.445 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:41:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 -- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69b00a2c00 msgr2=0x7f69b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69b00a2c00 0x7f69b00a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009230 tx=0x7f69b4041040 comp rx=0 tx=0).stop 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 -- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc112870 msgr2=0x7f69cc19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc112870 0x7f69cc19e750 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e940 tx=0x7f69c400ee00 comp rx=0 tx=0).stop 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 -- 172.21.15.46:0/1989348083 shutdown_connections 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f69b00a2c00 0x7f69b00a50c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc112870 0x7f69cc19e750 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69cc077040 0x7f69cc1a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 --2- 172.21.15.46:0/1989348083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69cc076720 0x7f69cc1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 -- 172.21.15.46:0/1989348083 >> 172.21.15.46:0/1989348083 conn(0x7f69cc0ff8d0 msgr2=0x7f69cc101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 -- 172.21.15.46:0/1989348083 shutdown_connections 2024-09-10T07:41:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:19.441+0000 7f69cb200640 1 -- 172.21.15.46:0/1989348083 wait complete. 2024-09-10T07:41:19.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:19 smithi046 ceph-mon[30705]: pgmap v540: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.9 MiB/s wr, 711 op/s 2024-09-10T07:41:20.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:19 smithi181 ceph-mon[35397]: pgmap v540: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.9 MiB/s wr, 711 op/s 2024-09-10T07:41:20.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:19 smithi063 ceph-mon[35020]: pgmap v540: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.9 MiB/s wr, 711 op/s 2024-09-10T07:41:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.195+0000 7f4eca800640 1 -- 172.21.15.46:0/3433412318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 msgr2=0x7f4ec41053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.195+0000 7f4eca800640 1 --2- 172.21.15.46:0/3433412318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 0x7f4ec41053d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009900 tx=0x7f4eb002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.195+0000 7f4eca800640 1 -- 172.21.15.46:0/3433412318 shutdown_connections 2024-09-10T07:41:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.195+0000 7f4eca800640 1 --2- 172.21.15.46:0/3433412318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec4105910 0x7f4ec410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.195+0000 7f4eca800640 1 --2- 172.21.15.46:0/3433412318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 0x7f4ec41053d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.195+0000 7f4eca800640 1 --2- 172.21.15.46:0/3433412318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ec4103d60 0x7f4ec4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.195+0000 7f4eca800640 1 -- 172.21.15.46:0/3433412318 >> 172.21.15.46:0/3433412318 conn(0x7f4ec40ff4e0 msgr2=0x7f4ec4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.196+0000 7f4eca800640 1 -- 172.21.15.46:0/3433412318 shutdown_connections 2024-09-10T07:41:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.196+0000 7f4eca800640 1 -- 172.21.15.46:0/3433412318 wait complete. 2024-09-10T07:41:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.196+0000 7f4eca800640 1 Processor -- start 2024-09-10T07:41:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4eca800640 1 -- start start 2024-09-10T07:41:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4eca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec4103d60 0x7f4ec4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4eca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 0x7f4ec419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4eca800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ec4105910 0x7f4ec41a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4eca800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec410f9c0 con 0x7f4ec4104f50 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4eca800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec410f840 con 0x7f4ec4105910 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4eca800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec410fb40 con 0x7f4ec4103d60 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 0x7f4ec419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 0x7f4ec419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56918/0 (socket says 172.21.15.46:56918) 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec2a00640 1 -- 172.21.15.46:0/3706435026 learned_addr learned my addr 172.21.15.46:0/3706435026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec3400640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec4103d60 0x7f4ec4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ec4105910 0x7f4ec41a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec2a00640 1 -- 172.21.15.46:0/3706435026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec4103d60 msgr2=0x7f4ec4199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec4103d60 0x7f4ec4199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec2a00640 1 -- 172.21.15.46:0/3706435026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ec4105910 msgr2=0x7f4ec41a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ec4105910 0x7f4ec41a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec2a00640 1 -- 172.21.15.46:0/3706435026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec41a1c00 con 0x7f4ec4104f50 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.197+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ec4105910 0x7f4ec41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.198+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 0x7f4ec419a520 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009230 tx=0x7f4eb0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.198+0000 7f4ebbe00640 1 -- 172.21.15.46:0/3706435026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb003f030 con 0x7f4ec4104f50 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.198+0000 7f4ebbe00640 1 -- 172.21.15.46:0/3706435026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb0003ad0 con 0x7f4ec4104f50 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.198+0000 7f4ebbe00640 1 -- 172.21.15.46:0/3706435026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0035070 con 0x7f4ec4104f50 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.198+0000 7f4eca800640 1 -- 172.21.15.46:0/3706435026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec41a1e90 con 0x7f4ec4104f50 2024-09-10T07:41:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.198+0000 7f4eca800640 1 -- 172.21.15.46:0/3706435026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec41a22a0 con 0x7f4ec4104f50 2024-09-10T07:41:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.201+0000 7f4ecd6ad740 1 Processor -- start 2024-09-10T07:41:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.201+0000 7f4ebbe00640 1 -- 172.21.15.46:0/3706435026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4eb0042030 con 0x7f4ec4104f50 2024-09-10T07:41:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.201+0000 7f4ecd6ad740 1 -- start start 2024-09-10T07:41:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4282490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4287490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4289f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fc41e7100 con 0x557fc4287a60 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557fc418f1b0 con 0x557fc4282090 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557fc41e30c0 con 0x557fc42829d0 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4289f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4282490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4289f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56934/0 (socket says 172.21.15.46:56934) 2024-09-10T07:41:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4287490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4282490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53772/0 (socket says 172.21.15.46:53772) 2024-09-10T07:41:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec2a00640 1 -- 172.21.15.46:0/1362627719 learned_addr learned my addr 172.21.15.46:0/1362627719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec3400640 1 -- 172.21.15.46:0/1362627719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 msgr2=0x557fc4282490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec3400640 1 --2- 172.21.15.46:0/1362627719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4282490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec3400640 1 -- 172.21.15.46:0/1362627719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 msgr2=0x557fc4289f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec3400640 1 --2- 172.21.15.46:0/1362627719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4289f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec3400640 1 -- 172.21.15.46:0/1362627719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc3efa210 con 0x557fc42829d0 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.202+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/1362627719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4289f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.203+0000 7f4ec3400640 1 --2- 172.21.15.46:0/1362627719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4287490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009bf0 tx=0x7f4eb4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6550a5e98e6129f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.203+0000 7f4ebbe00640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ea80a2c30 0x7f4ea80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.203+0000 7f4eb8c00640 1 -- 172.21.15.46:0/1362627719 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4009da0 con 0x557fc42829d0 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.203+0000 7f4ebbe00640 1 -- 172.21.15.46:0/3706435026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4eb00ef9a0 con 0x7f4ec4104f50 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.203+0000 7f4eb8c00640 1 -- 172.21.15.46:0/1362627719 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb4005020 con 0x557fc42829d0 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.203+0000 7f4eb8c00640 1 -- 172.21.15.46:0/1362627719 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4004b20 con 0x557fc42829d0 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.203+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ea80a2c30 0x7f4ea80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ea80a2c30 0x7f4ea80a50f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc005fd0 tx=0x7f4ebc0098a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/1362627719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 msgr2=0x557fc4287490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/1362627719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4287490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009bf0 tx=0x7f4eb4011810 comp rx=0 tx=0).stop 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/1362627719 shutdown_connections 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/1362627719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4289f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/1362627719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4287490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/1362627719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4282490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/1362627719 >> 172.21.15.46:0/1362627719 conn(0x557fc427d700 msgr2=0x557fc427db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/1362627719 shutdown_connections 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.204+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/1362627719 wait complete. 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 1 -- start start 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 1 Processor -- start 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc428c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc428c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4293760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fc428aa30 con 0x557fc4287a60 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557fc4170090 con 0x557fc4282090 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557fc428abb0 con 0x557fc42829d0 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.205+0000 7f4ecd6ad740 10 client.0 _ll_register_callbacks cb 0x557fc41c4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc428c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc428c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57492/0 (socket says 172.21.15.46:57492) 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 -- 172.21.15.46:0/719878739 learned_addr learned my addr 172.21.15.46:0/719878739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec3400640 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc428c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4293760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 -- 172.21.15.46:0/719878739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 msgr2=0x557fc428c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc428c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 -- 172.21.15.46:0/719878739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 msgr2=0x557fc4293760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4293760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 -- 172.21.15.46:0/719878739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc3fc6620 con 0x557fc42829d0 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc428c720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00f52d0 tx=0x7f4eb0009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983d8d8381381d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.206+0000 7f4ead600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ead600640 1 -- 172.21.15.46:0/719878739 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00f1e60 con 0x557fc42829d0 2024-09-10T07:41:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ead600640 1 -- 172.21.15.46:0/719878739 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4eb00b8490 con 0x557fc42829d0 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/719878739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb4005020 con 0x557fc42829d0 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ead600640 1 -- 172.21.15.46:0/719878739 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4eb0049030 con 0x557fc42829d0 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 10 client.26575 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/719878739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557fc428ab00 con 0x557fc42829d0 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ead600640 1 -- 172.21.15.46:0/719878739 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb4005020 con 0x557fc42829d0 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ead600640 1 -- 172.21.15.46:0/719878739 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f4eb0003c80 con 0x557fc42829d0 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 10 client.26575 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 10 client.26575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 20 client.26575 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc4297d00 0x557fc429a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 4 client.26575 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.207+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/719878739 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fc41f4830 con 0x557fc4297d00 2024-09-10T07:41:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.208+0000 7f4ec3400640 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc4297d00 0x557fc429a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.208+0000 7f4ec3400640 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc4297d00 0x557fc429a150 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6028082847c350a server_cookie=c68cbcfb69666dc in_seq=0 out_seq=0 2024-09-10T07:41:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.208+0000 7f4ead600640 10 client.26575 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ead600640 1 -- 172.21.15.46:0/719878739 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fc41f4830 con 0x557fc4297d00 2024-09-10T07:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ead600640 10 client.26575 handle_command_reply: tid=0 2024-09-10T07:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 client.26575 shutdown 2024-09-10T07:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/719878739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 msgr2=0x557fc428c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc428c720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00f52d0 tx=0x7f4eb0009680 comp rx=0 tx=0).stop 2024-09-10T07:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/719878739 shutdown_connections 2024-09-10T07:41:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc4297d00 0x557fc429a150 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc4293760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc428c720 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ead600640 0 client.26575 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/719878739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc428c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.209+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/719878739 >> 172.21.15.46:0/719878739 conn(0x557fc427d700 msgr2=0x557fc427ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.210+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/719878739 shutdown_connections 2024-09-10T07:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.210+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/719878739 wait complete. 2024-09-10T07:41:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.210+0000 7f4ecd6ad740 20 client.26575 trim_cache size 0 max 16384 2024-09-10T07:41:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.225+0000 7f4ecd6ad740 1 Processor -- start 2024-09-10T07:41:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.225+0000 7f4ecd6ad740 1 -- start start 2024-09-10T07:41:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.225+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc427e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.225+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4293720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.225+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc427eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.225+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41ac830 con 0x557fc4287a60 2024-09-10T07:41:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.225+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41ae3d0 con 0x557fc4282090 2024-09-10T07:41:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.225+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41acf00 con 0x557fc42829d0 2024-09-10T07:41:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc427eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc427eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56956/0 (socket says 172.21.15.46:56956) 2024-09-10T07:41:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 -- 172.21.15.46:0/2602300691 learned_addr learned my addr 172.21.15.46:0/2602300691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3400640 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc427e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4293720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 -- 172.21.15.46:0/2602300691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 msgr2=0x557fc4293720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4293720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 -- 172.21.15.46:0/2602300691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 msgr2=0x557fc427e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc427e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 -- 172.21.15.46:0/2602300691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc428a910 con 0x557fc4287a60 2024-09-10T07:41:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3400640 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc427e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.226+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc427eaf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc010860 tx=0x7f4ebc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3f9a92967f07d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4eafe00640 1 -- 172.21.15.46:0/2602300691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc01c430 con 0x557fc4287a60 2024-09-10T07:41:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4eafe00640 1 -- 172.21.15.46:0/2602300691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ebc01cab0 con 0x557fc4287a60 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4eafe00640 1 -- 172.21.15.46:0/2602300691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc01fba0 con 0x557fc4287a60 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/2602300691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 msgr2=0x557fc427eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc427eaf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc010860 tx=0x7f4ebc015000 comp rx=0 tx=0).stop 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/2602300691 shutdown_connections 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc4287a60 0x557fc427eaf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc42829d0 0x557fc4293720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/2602300691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc427e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/2602300691 >> 172.21.15.46:0/2602300691 conn(0x557fc427d700 msgr2=0x557fc4283b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/2602300691 shutdown_connections 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.227+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/2602300691 wait complete. 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 1 -- start start 2024-09-10T07:41:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 1 Processor -- start 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4291500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc42829d0 0x557fc4291a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc4287a60 0x557fc4292990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41add00 con 0x557fc42829d0 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41ad630 con 0x557fc4282090 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41aa810 con 0x557fc4287a60 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.228+0000 7f4ecd6ad740 10 client.0 _ll_register_callbacks cb 0x7f4eb00f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4291500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc42829d0 0x557fc4291a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc4287a60 0x557fc4292990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4291500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53800/0 (socket says 172.21.15.46:53800) 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3400640 1 -- 172.21.15.46:0/3302358853 learned_addr learned my addr 172.21.15.46:0/3302358853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc42829d0 0x557fc4291a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56966/0 (socket says 172.21.15.46:56966) 2024-09-10T07:41:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3e00640 1 -- 172.21.15.46:0/3302358853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 msgr2=0x557fc4291500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4291500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3e00640 1 -- 172.21.15.46:0/3302358853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc42829d0 msgr2=0x557fc4291a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc42829d0 0x557fc4291a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3e00640 1 -- 172.21.15.46:0/3302358853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc4292fa0 con 0x557fc4287a60 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec2a00640 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc42829d0 0x557fc4291a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ec3e00640 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc4287a60 0x557fc4292990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc010d20 tx=0x7f4ebc01ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6202b7a3b8c36c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.229+0000 7f4ead600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.230+0000 7f4ead600640 1 -- 172.21.15.46:0/3302358853 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc01c640 con 0x557fc4287a60 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.230+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/3302358853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ebc01cab0 con 0x557fc4287a60 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.230+0000 7f4ead600640 1 -- 172.21.15.46:0/3302358853 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4ebc02e070 con 0x557fc4287a60 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.230+0000 7f4ead600640 1 -- 172.21.15.46:0/3302358853 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ebc03f050 con 0x557fc4287a60 2024-09-10T07:41:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.230+0000 7f4ecd6ad740 10 client.26581 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.230+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/3302358853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557fc4303020 con 0x557fc4287a60 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.230+0000 7f4ead600640 1 -- 172.21.15.46:0/3302358853 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ebc01cab0 con 0x557fc4287a60 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.230+0000 7f4ead600640 1 -- 172.21.15.46:0/3302358853 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f4ebc0298d0 con 0x557fc4287a60 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ecd6ad740 10 client.26581 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ecd6ad740 10 client.26581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ecd6ad740 20 client.26581 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc4303970 0x557fc4305d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ecd6ad740 4 client.26581 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/3302358853 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557fc41eeb80 con 0x557fc4303970 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ec3400640 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc4303970 0x557fc4305d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ec3400640 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc4303970 0x557fc4305d90 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5a1a8f01a493c6 server_cookie=68d4eabe41acc8b2 in_seq=0 out_seq=0 2024-09-10T07:41:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.231+0000 7f4ead600640 10 client.26581 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ead600640 1 -- 172.21.15.46:0/3302358853 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557fc41eeb80 con 0x557fc4303970 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ead600640 10 client.26581 handle_command_reply: tid=0 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 client.26581 shutdown 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/3302358853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc4287a60 msgr2=0x557fc4292990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc4287a60 0x557fc4292990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc010d20 tx=0x7f4ebc01ce90 comp rx=0 tx=0).stop 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/3302358853 shutdown_connections 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc4303970 0x557fc4305d90 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc4287a60 0x557fc4292990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ead600640 0 client.26581 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc42829d0 0x557fc4291a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 --2- 172.21.15.46:0/3302358853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc4282090 0x557fc4291500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/3302358853 >> 172.21.15.46:0/3302358853 conn(0x557fc427d700 msgr2=0x557fc427c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/3302358853 shutdown_connections 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 1 -- 172.21.15.46:0/3302358853 wait complete. 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.244+0000 7f4ecd6ad740 20 client.26581 trim_cache size 0 max 16384 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:41:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:41:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 -- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ea80a2c30 msgr2=0x7f4ea80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ea80a2c30 0x7f4ea80a50f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc005fd0 tx=0x7f4ebc0098a0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 -- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 msgr2=0x7f4ec419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 0x7f4ec419a520 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009230 tx=0x7f4eb0031b60 comp rx=0 tx=0).stop 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 -- 172.21.15.46:0/3706435026 shutdown_connections 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ec4105910 0x7f4ec41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4ea80a2c30 0x7f4ea80a50f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4104f50 0x7f4ec419a520 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 --2- 172.21.15.46:0/3706435026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec4103d60 0x7f4ec4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 -- 172.21.15.46:0/3706435026 >> 172.21.15.46:0/3706435026 conn(0x7f4ec40ff4e0 msgr2=0x7f4ec4108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 -- 172.21.15.46:0/3706435026 shutdown_connections 2024-09-10T07:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:20.245+0000 7f4eca800640 1 -- 172.21.15.46:0/3706435026 wait complete. 2024-09-10T07:41:20.501 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id 93c5c7e4 --release --await' 2024-09-10T07:41:20.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id 93c5c7e4 --release --await 2024-09-10T07:41:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 240.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:41:20.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:41:20.989 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 -- 172.21.15.46:0/1433124670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c1027a0 msgr2=0x7fef3c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 --2- 172.21.15.46:0/1433124670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c1027a0 0x7fef3c1067f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef28009900 tx=0x7fef2802a410 comp rx=0 tx=0).stop 2024-09-10T07:41:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 -- 172.21.15.46:0/1433124670 shutdown_connections 2024-09-10T07:41:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 --2- 172.21.15.46:0/1433124670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef3c106f20 0x7fef3c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 --2- 172.21.15.46:0/1433124670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c1027a0 0x7fef3c1067f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 --2- 172.21.15.46:0/1433124670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef3c101df0 0x7fef3c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 -- 172.21.15.46:0/1433124670 >> 172.21.15.46:0/1433124670 conn(0x7fef3c0fd8f0 msgr2=0x7fef3c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 -- 172.21.15.46:0/1433124670 shutdown_connections 2024-09-10T07:41:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.605+0000 7fef40e00640 1 -- 172.21.15.46:0/1433124670 wait complete. 2024-09-10T07:41:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.606+0000 7fef40e00640 1 Processor -- start 2024-09-10T07:41:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.606+0000 7fef40e00640 1 -- start start 2024-09-10T07:41:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef40e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c101df0 0x7fef3c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef40e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef3c1027a0 0x7fef3c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef40e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef3c106f20 0x7fef3c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef40e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c111f40 con 0x7fef3c1027a0 2024-09-10T07:41:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef40e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c111dc0 con 0x7fef3c101df0 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef40e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1120c0 con 0x7fef3c106f20 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef3c1027a0 0x7fef3c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef3c1027a0 0x7fef3c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56980/0 (socket says 172.21.15.46:56980) 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef39400640 1 -- 172.21.15.46:0/3767082975 learned_addr learned my addr 172.21.15.46:0/3767082975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef39e00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c101df0 0x7fef3c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.607+0000 7fef3a800640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef3c106f20 0x7fef3c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef39400640 1 -- 172.21.15.46:0/3767082975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef3c106f20 msgr2=0x7fef3c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef39400640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef3c106f20 0x7fef3c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef39400640 1 -- 172.21.15.46:0/3767082975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c101df0 msgr2=0x7fef3c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef39400640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c101df0 0x7fef3c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef39400640 1 -- 172.21.15.46:0/3767082975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3c19f630 con 0x7fef3c1027a0 2024-09-10T07:41:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef39e00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c101df0 0x7fef3c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef39400640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef3c1027a0 0x7fef3c19abc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fef28006010 tx=0x7fef280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef32a00640 1 -- 172.21.15.46:0/3767082975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2803f030 con 0x7fef3c1027a0 2024-09-10T07:41:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef32a00640 1 -- 172.21.15.46:0/3767082975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef28003ce0 con 0x7fef3c1027a0 2024-09-10T07:41:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.608+0000 7fef32a00640 1 -- 172.21.15.46:0/3767082975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28035070 con 0x7fef3c1027a0 2024-09-10T07:41:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.609+0000 7fef40e00640 1 -- 172.21.15.46:0/3767082975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3c19f8c0 con 0x7fef3c1027a0 2024-09-10T07:41:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.609+0000 7fef40e00640 1 -- 172.21.15.46:0/3767082975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3c19fd50 con 0x7fef3c1027a0 2024-09-10T07:41:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef439bc740 1 Processor -- start 2024-09-10T07:41:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef32a00640 1 -- 172.21.15.46:0/3767082975 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fef28042030 con 0x7fef3c1027a0 2024-09-10T07:41:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef439bc740 1 -- start start 2024-09-10T07:41:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef439bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef439bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ed6c30 0x556b47ee0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef439bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ee14f0 0x556b47ee38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef439bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556b47e3c650 con 0x556b47ee14f0 2024-09-10T07:41:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef439bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556b47de3f10 con 0x556b47ed6c30 2024-09-10T07:41:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.613+0000 7fef439bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556b47e38610 con 0x556b47ed6310 2024-09-10T07:41:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ed6c30 0x556b47ee0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ed6c30 0x556b47ee0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53830/0 (socket says 172.21.15.46:53830) 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39e00640 1 -- 172.21.15.46:0/2717723145 learned_addr learned my addr 172.21.15.46:0/2717723145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39400640 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ee14f0 0x556b47ee38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef3a800640 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39e00640 1 -- 172.21.15.46:0/2717723145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 msgr2=0x556b47ed66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39e00640 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39e00640 1 -- 172.21.15.46:0/2717723145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ee14f0 msgr2=0x556b47ee38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39e00640 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ee14f0 0x556b47ee38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef39e00640 1 -- 172.21.15.46:0/2717723145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b47b4b580 con 0x556b47ed6c30 2024-09-10T07:41:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.614+0000 7fef3a800640 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.615+0000 7fef32a00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fef200a2c00 0x7fef200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.615+0000 7fef39e00640 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ed6c30 0x556b47ee0fb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0115f0 tx=0x7fef2c011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65757edcf3e37333 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.615+0000 7fef32a00640 1 -- 172.21.15.46:0/3767082975 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fef280ef7d0 con 0x7fef3c1027a0 2024-09-10T07:41:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.615+0000 7fef27400640 1 -- 172.21.15.46:0/2717723145 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c01e070 con 0x556b47ed6c30 2024-09-10T07:41:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.615+0000 7fef27400640 1 -- 172.21.15.46:0/2717723145 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef2c00fb20 con 0x556b47ed6c30 2024-09-10T07:41:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.615+0000 7fef27400640 1 -- 172.21.15.46:0/2717723145 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c019cb0 con 0x556b47ed6c30 2024-09-10T07:41:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.615+0000 7fef3a800640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fef200a2c00 0x7fef200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef3a800640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fef200a2c00 0x7fef200a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef34005fd0 tx=0x7fef340096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 -- 172.21.15.46:0/2717723145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ed6c30 msgr2=0x556b47ee0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ed6c30 0x556b47ee0fb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0115f0 tx=0x7fef2c011ab0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 -- 172.21.15.46:0/2717723145 shutdown_connections 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ee14f0 0x556b47ee38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ed6c30 0x556b47ee0fb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 --2- 172.21.15.46:0/2717723145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 -- 172.21.15.46:0/2717723145 >> 172.21.15.46:0/2717723145 conn(0x556b47ed28a0 msgr2=0x556b47ed2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 -- 172.21.15.46:0/2717723145 shutdown_connections 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.616+0000 7fef439bc740 1 -- 172.21.15.46:0/2717723145 wait complete. 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.617+0000 7fef439bc740 1 -- start start 2024-09-10T07:41:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.617+0000 7fef439bc740 1 Processor -- start 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef439bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6310 0x556b47ede390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef439bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6c30 0x556b47ede8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef439bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47edee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef439bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556b47ee40c0 con 0x556b47ed6310 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef439bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556b47df7350 con 0x556b47ee14f0 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef439bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556b47ee4240 con 0x556b47ed6c30 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef439bc740 10 client.0 _ll_register_callbacks cb 0x556b47e1a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef39e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6310 0x556b47ede390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef39e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6310 0x556b47ede390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57004/0 (socket says 172.21.15.46:57004) 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef39e00640 1 -- 172.21.15.46:0/2894395727 learned_addr learned my addr 172.21.15.46:0/2894395727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.618+0000 7fef39400640 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6c30 0x556b47ede8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.619+0000 7fef3a800640 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47edee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.619+0000 7fef39e00640 1 -- 172.21.15.46:0/2894395727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6c30 msgr2=0x556b47ede8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.619+0000 7fef39e00640 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6c30 0x556b47ede8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.619+0000 7fef39e00640 1 -- 172.21.15.46:0/2894395727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 msgr2=0x556b47edee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.619+0000 7fef39e00640 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47edee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.619+0000 7fef39e00640 1 -- 172.21.15.46:0/2894395727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b47c15960 con 0x556b47ed6310 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.619+0000 7fef39e00640 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6310 0x556b47ede390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fef2c010000 tx=0x7fef2c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d7ba9f40dbf706 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.619+0000 7fef1be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:41:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.620+0000 7fef1be00640 1 -- 172.21.15.46:0/2894395727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c01e040 con 0x556b47ed6310 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.620+0000 7fef439bc740 1 -- 172.21.15.46:0/2894395727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2c00fb20 con 0x556b47ed6310 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.620+0000 7fef1be00640 1 -- 172.21.15.46:0/2894395727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fef2c00f640 con 0x556b47ed6310 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.620+0000 7fef1be00640 1 -- 172.21.15.46:0/2894395727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef2c02d050 con 0x556b47ed6310 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.620+0000 7fef439bc740 10 client.16323 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.620+0000 7fef439bc740 1 -- 172.21.15.46:0/2894395727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b47eefcc0 con 0x556b47ed6310 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.620+0000 7fef1be00640 1 -- 172.21.15.46:0/2894395727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef2c00fb20 con 0x556b47ed6310 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.621+0000 7fef1be00640 1 -- 172.21.15.46:0/2894395727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fef2c005070 con 0x556b47ed6310 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.621+0000 7fef439bc740 10 client.16323 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.621+0000 7fef439bc740 10 client.16323 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.621+0000 7fef439bc740 20 client.16323 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.621+0000 7fef439bc740 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556b47ef1120 0x556b47ef3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.621+0000 7fef439bc740 4 client.16323 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.621+0000 7fef439bc740 1 -- 172.21.15.46:0/2894395727 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b47e49bc0 con 0x556b47ef1120 2024-09-10T07:41:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.621+0000 7fef39400640 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556b47ef1120 0x556b47ef3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.622+0000 7fef39400640 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556b47ef1120 0x556b47ef3510 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5902d18dac0d9ef3 server_cookie=3905f1338625d12d in_seq=0 out_seq=0 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.622+0000 7fef1be00640 10 client.16323 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.622+0000 7fef1be00640 1 -- 172.21.15.46:0/2894395727 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b47e49bc0 con 0x556b47ef1120 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.622+0000 7fef1be00640 10 client.16323 handle_command_reply: tid=0 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 client.16323 shutdown 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 -- 172.21.15.46:0/2894395727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6310 msgr2=0x556b47ede390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6310 0x556b47ede390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fef2c010000 tx=0x7fef2c0196e0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 -- 172.21.15.46:0/2894395727 shutdown_connections 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556b47ef1120 0x556b47ef3510 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47edee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6c30 0x556b47ede8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 --2- 172.21.15.46:0/2894395727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6310 0x556b47ede390 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 -- 172.21.15.46:0/2894395727 >> 172.21.15.46:0/2894395727 conn(0x556b47ed28a0 msgr2=0x556b47ed1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef1be00640 0 client.16323 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 -- 172.21.15.46:0/2894395727 shutdown_connections 2024-09-10T07:41:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 1 -- 172.21.15.46:0/2894395727 wait complete. 2024-09-10T07:41:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.623+0000 7fef439bc740 20 client.16323 trim_cache size 0 max 16384 2024-09-10T07:41:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef439bc740 1 Processor -- start 2024-09-10T07:41:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef439bc740 1 -- start start 2024-09-10T07:41:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef439bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef439bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ed1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef439bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ef8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef439bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b0ec0 con 0x556b47ed6c30 2024-09-10T07:41:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef439bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b2a60 con 0x556b47ee14f0 2024-09-10T07:41:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef439bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b1590 con 0x556b47ed6310 2024-09-10T07:41:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ed1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef39e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ed1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57006/0 (socket says 172.21.15.46:57006) 2024-09-10T07:41:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef39e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57564/0 (socket says 172.21.15.46:57564) 2024-09-10T07:41:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.645+0000 7fef39400640 1 -- 172.21.15.46:0/687856588 learned_addr learned my addr 172.21.15.46:0/687856588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.646+0000 7fef3a800640 1 --2- 172.21.15.46:0/687856588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ef8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.646+0000 7fef39400640 1 -- 172.21.15.46:0/687856588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 msgr2=0x556b47ed8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.646+0000 7fef39400640 1 --2- 172.21.15.46:0/687856588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.646+0000 7fef39400640 1 -- 172.21.15.46:0/687856588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 msgr2=0x556b47ef8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.646+0000 7fef39400640 1 --2- 172.21.15.46:0/687856588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ef8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.646+0000 7fef39400640 1 -- 172.21.15.46:0/687856588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b47ed26d0 con 0x556b47ed6c30 2024-09-10T07:41:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.646+0000 7fef39400640 1 --2- 172.21.15.46:0/687856588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ed1aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fef280b86e0 tx=0x7fef280f5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d85ba9c17267716 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.646+0000 7fef26a00640 1 -- 172.21.15.46:0/687856588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef280b7fb0 con 0x556b47ed6c30 2024-09-10T07:41:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef26a00640 1 -- 172.21.15.46:0/687856588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef280d4e50 con 0x556b47ed6c30 2024-09-10T07:41:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef26a00640 1 -- 172.21.15.46:0/687856588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef280bb670 con 0x556b47ed6c30 2024-09-10T07:41:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef439bc740 1 -- 172.21.15.46:0/687856588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 msgr2=0x556b47ed1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef439bc740 1 --2- 172.21.15.46:0/687856588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ed1aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fef280b86e0 tx=0x7fef280f5530 comp rx=0 tx=0).stop 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef439bc740 1 -- 172.21.15.46:0/687856588 shutdown_connections 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef439bc740 1 --2- 172.21.15.46:0/687856588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ef8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef439bc740 1 --2- 172.21.15.46:0/687856588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ed1aa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef439bc740 1 --2- 172.21.15.46:0/687856588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ed8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.647+0000 7fef439bc740 1 -- 172.21.15.46:0/687856588 >> 172.21.15.46:0/687856588 conn(0x556b47ed28a0 msgr2=0x556b47ee0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.648+0000 7fef439bc740 1 -- 172.21.15.46:0/687856588 shutdown_connections 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.648+0000 7fef439bc740 1 -- 172.21.15.46:0/687856588 wait complete. 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.648+0000 7fef439bc740 1 -- start start 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.648+0000 7fef439bc740 1 Processor -- start 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef439bc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ee9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef439bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ee4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef439bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ee5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef439bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b2390 con 0x556b47ed6c30 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef439bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1b1cc0 con 0x556b47ee14f0 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef439bc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c1aeea0 con 0x556b47ed6310 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef439bc740 10 client.0 _ll_register_callbacks cb 0x7fef2c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef39e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ee9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef3a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ee5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ee4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef3a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ee5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53854/0 (socket says 172.21.15.46:53854) 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef3a800640 1 -- 172.21.15.46:0/4293396052 learned_addr learned my addr 172.21.15.46:0/4293396052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef3a800640 1 -- 172.21.15.46:0/4293396052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 msgr2=0x556b47ee9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef3a800640 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ee9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef3a800640 1 -- 172.21.15.46:0/4293396052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 msgr2=0x556b47ee4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef3a800640 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ee4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.649+0000 7fef3a800640 1 -- 172.21.15.46:0/4293396052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b47ee5c60 con 0x556b47ee14f0 2024-09-10T07:41:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.650+0000 7fef3a800640 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ee5480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef34010930 tx=0x7fef34014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4752daa803d87c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.650+0000 7fef1be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.650+0000 7fef1be00640 1 -- 172.21.15.46:0/4293396052 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3402e060 con 0x556b47ee14f0 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.650+0000 7fef439bc740 1 -- 172.21.15.46:0/4293396052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef280d4e50 con 0x556b47ee14f0 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.650+0000 7fef1be00640 1 -- 172.21.15.46:0/4293396052 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fef3401d8e0 con 0x556b47ee14f0 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef1be00640 1 -- 172.21.15.46:0/4293396052 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef34035050 con 0x556b47ee14f0 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef439bc740 10 client.26618 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef439bc740 1 -- 172.21.15.46:0/4293396052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b47f5bd50 con 0x556b47ee14f0 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef1be00640 1 -- 172.21.15.46:0/4293396052 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef280d4e50 con 0x556b47ee14f0 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef1be00640 1 -- 172.21.15.46:0/4293396052 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fef3401c400 con 0x556b47ee14f0 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef439bc740 10 client.26618 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef439bc740 10 client.26618 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef439bc740 20 client.26618 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef439bc740 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556b47f5c480 0x556b47f5e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef439bc740 4 client.26618 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "93c5c7e4", "release": true, "await": true}] 2024-09-10T07:41:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef439bc740 1 -- 172.21.15.46:0/4293396052 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "93c5c7e4", "release": true, "await": true}) -- 0x556b47e440d0 con 0x556b47f5c480 2024-09-10T07:41:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.651+0000 7fef39e00640 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556b47f5c480 0x556b47f5e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.652+0000 7fef39e00640 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556b47f5c480 0x556b47f5e8a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef0205f49fa59d79 server_cookie=efded24c5c09b7cc in_seq=0 out_seq=0 2024-09-10T07:41:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.652+0000 7fef1be00640 10 client.26618 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:21.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.680+0000 7fef1be00640 1 -- 172.21.15.46:0/4293396052 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556b47e440d0 con 0x556b47f5c480 2024-09-10T07:41:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.680+0000 7fef1be00640 10 client.26618 handle_command_reply: tid=0 2024-09-10T07:41:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.680+0000 7fef439bc740 1 client.26618 shutdown 2024-09-10T07:41:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.680+0000 7fef439bc740 1 -- 172.21.15.46:0/4293396052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 msgr2=0x556b47ee5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.680+0000 7fef439bc740 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ee5480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef34010930 tx=0x7fef34014040 comp rx=0 tx=0).stop 2024-09-10T07:41:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef439bc740 1 -- 172.21.15.46:0/4293396052 shutdown_connections 2024-09-10T07:41:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef439bc740 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556b47f5c480 0x556b47f5e8a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef439bc740 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b47ee14f0 0x556b47ee5480 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef439bc740 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b47ed6c30 0x556b47ee4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef1be00640 0 client.26618 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef439bc740 1 --2- 172.21.15.46:0/4293396052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b47ed6310 0x556b47ee9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef439bc740 1 -- 172.21.15.46:0/4293396052 >> 172.21.15.46:0/4293396052 conn(0x556b47ed28a0 msgr2=0x556b47ed1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef439bc740 1 -- 172.21.15.46:0/4293396052 shutdown_connections 2024-09-10T07:41:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.681+0000 7fef439bc740 1 -- 172.21.15.46:0/4293396052 wait complete. 2024-09-10T07:41:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.682+0000 7fef439bc740 20 client.26618 trim_cache size 0 max 16384 2024-09-10T07:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 39, 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 16, 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "93c5c7e4": { 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 16, 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 142.8, 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:41:21.685 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:41:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 -- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fef200a2c00 msgr2=0x7fef200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fef200a2c00 0x7fef200a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef34005fd0 tx=0x7fef340096a0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 -- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef3c1027a0 msgr2=0x7fef3c19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef3c1027a0 0x7fef3c19abc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fef28006010 tx=0x7fef280042f0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 -- 172.21.15.46:0/3767082975 shutdown_connections 2024-09-10T07:41:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef3c106f20 0x7fef3c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fef200a2c00 0x7fef200a50c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef3c1027a0 0x7fef3c19abc0 secure :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fef28006010 tx=0x7fef280042f0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 --2- 172.21.15.46:0/3767082975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3c101df0 0x7fef3c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.685+0000 7fef40e00640 1 -- 172.21.15.46:0/3767082975 >> 172.21.15.46:0/3767082975 conn(0x7fef3c0fd8f0 msgr2=0x7fef3c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.687+0000 7fef40e00640 1 -- 172.21.15.46:0/3767082975 shutdown_connections 2024-09-10T07:41:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:21.688+0000 7fef40e00640 1 -- 172.21.15.46:0/3767082975 wait complete. 2024-09-10T07:41:22.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:21 smithi046 ceph-mon[30705]: pgmap v541: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 713 op/s 2024-09-10T07:41:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:21 smithi181 ceph-mon[35397]: pgmap v541: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 713 op/s 2024-09-10T07:41:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:21 smithi063 ceph-mon[35020]: pgmap v541: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 713 op/s 2024-09-10T07:41:23.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.247+0000 7f7b54a00640 1 -- 172.21.15.46:0/2836872956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 msgr2=0x7f7b50103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.247+0000 7f7b54a00640 1 --2- 172.21.15.46:0/2836872956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 0x7f7b50103110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009900 tx=0x7f7b3c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.248+0000 7f7b54a00640 1 -- 172.21.15.46:0/2836872956 shutdown_connections 2024-09-10T07:41:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.248+0000 7f7b54a00640 1 --2- 172.21.15.46:0/2836872956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b501082a0 0x7f7b5010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.248+0000 7f7b54a00640 1 --2- 172.21.15.46:0/2836872956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b501036e0 0x7f7b50107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.248+0000 7f7b54a00640 1 --2- 172.21.15.46:0/2836872956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 0x7f7b50103110 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.248+0000 7f7b54a00640 1 -- 172.21.15.46:0/2836872956 >> 172.21.15.46:0/2836872956 conn(0x7f7b500fc7d0 msgr2=0x7f7b500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.249+0000 7f7b54a00640 1 -- 172.21.15.46:0/2836872956 shutdown_connections 2024-09-10T07:41:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.249+0000 7f7b54a00640 1 -- 172.21.15.46:0/2836872956 wait complete. 2024-09-10T07:41:23.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.250+0000 7f7b54a00640 1 Processor -- start 2024-09-10T07:41:23.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.251+0000 7f7b54a00640 1 -- start start 2024-09-10T07:41:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.251+0000 7f7b54a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 0x7f7b50196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.251+0000 7f7b54a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b501036e0 0x7f7b50196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.251+0000 7f7b54a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b501082a0 0x7f7b5019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 0x7f7b50196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b501082a0 0x7f7b5019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 0x7f7b50196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57608/0 (socket says 172.21.15.46:57608) 2024-09-10T07:41:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4de00640 1 -- 172.21.15.46:0/3603349476 learned_addr learned my addr 172.21.15.46:0/3603349476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b501036e0 0x7f7b50196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010fe90 con 0x7f7b501082a0 2024-09-10T07:41:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010fd10 con 0x7f7b501036e0 2024-09-10T07:41:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b50110010 con 0x7f7b50102d30 2024-09-10T07:41:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4d400640 1 -- 172.21.15.46:0/3603349476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 msgr2=0x7f7b50196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4d400640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 0x7f7b50196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4d400640 1 -- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b501082a0 msgr2=0x7f7b5019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4d400640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b501082a0 0x7f7b5019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.252+0000 7f7b4d400640 1 -- 172.21.15.46:0/3603349476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5019b3b0 con 0x7f7b501036e0 2024-09-10T07:41:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.253+0000 7f7b4de00640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 0x7f7b50196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.253+0000 7f7b4d400640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b501036e0 0x7f7b50196940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7b500697d0 tx=0x7f7b40008600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.253+0000 7f7b46a00640 1 -- 172.21.15.46:0/3603349476 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4001b070 con 0x7f7b501036e0 2024-09-10T07:41:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.253+0000 7f7b46a00640 1 -- 172.21.15.46:0/3603349476 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4000bc40 con 0x7f7b501036e0 2024-09-10T07:41:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.253+0000 7f7b46a00640 1 -- 172.21.15.46:0/3603349476 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40015760 con 0x7f7b501036e0 2024-09-10T07:41:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.253+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5019b6a0 con 0x7f7b501036e0 2024-09-10T07:41:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.254+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b501a2f80 con 0x7f7b501036e0 2024-09-10T07:41:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.259+0000 7f7b575c6740 1 Processor -- start 2024-09-10T07:41:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.259+0000 7f7b575c6740 1 -- start start 2024-09-10T07:41:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.259+0000 7f7b46a00640 1 -- 172.21.15.46:0/3603349476 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b4001e070 con 0x7f7b501036e0 2024-09-10T07:41:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496bdfac70 0x56496bdfb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496be05e50 0x56496be08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b575c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56496bd613e0 con 0x56496bdfac70 2024-09-10T07:41:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b575c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56496bd08d80 con 0x56496bdfb590 2024-09-10T07:41:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b575c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56496bd5d3a0 con 0x56496be05e50 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b4d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496be05e50 0x56496be08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b4d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496be05e50 0x56496be08240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57622/0 (socket says 172.21.15.46:57622) 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b4d400640 1 -- 172.21.15.46:0/2046908469 learned_addr learned my addr 172.21.15.46:0/2046908469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b4de00640 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.260+0000 7f7b4e800640 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496bdfac70 0x56496bdfb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b4d400640 1 -- 172.21.15.46:0/2046908469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 msgr2=0x56496be05910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b4d400640 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be05910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b4d400640 1 -- 172.21.15.46:0/2046908469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496bdfac70 msgr2=0x56496bdfb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b4d400640 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496bdfac70 0x56496bdfb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b4d400640 1 -- 172.21.15.46:0/2046908469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56496bd27ae0 con 0x56496be05e50 2024-09-10T07:41:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b4e800640 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496bdfac70 0x56496bdfb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b4d400640 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496be05e50 0x56496be08240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b400d2ef0 tx=0x7f7b400d74b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efae5d640234f2e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b46a00640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b340a2c00 0x7f7b340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b3b000640 1 -- 172.21.15.46:0/2046908469 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b400e2070 con 0x56496be05e50 2024-09-10T07:41:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.261+0000 7f7b46a00640 1 -- 172.21.15.46:0/3603349476 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7b400cee90 con 0x7f7b501036e0 2024-09-10T07:41:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.262+0000 7f7b3b000640 1 -- 172.21.15.46:0/2046908469 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4009b420 con 0x56496be05e50 2024-09-10T07:41:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.262+0000 7f7b3b000640 1 -- 172.21.15.46:0/2046908469 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4009af60 con 0x56496be05e50 2024-09-10T07:41:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.262+0000 7f7b4e800640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b340a2c00 0x7f7b340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.263+0000 7f7b575c6740 1 -- 172.21.15.46:0/2046908469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496be05e50 msgr2=0x56496be08240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.263+0000 7f7b575c6740 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496be05e50 0x56496be08240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b400d2ef0 tx=0x7f7b400d74b0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.265+0000 7f7b4e800640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b340a2c00 0x7f7b340a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b48005fd0 tx=0x7f7b48015040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.265+0000 7f7b575c6740 1 -- 172.21.15.46:0/2046908469 shutdown_connections 2024-09-10T07:41:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.265+0000 7f7b575c6740 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496be05e50 0x56496be08240 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.265+0000 7f7b575c6740 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be05910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.265+0000 7f7b575c6740 1 --2- 172.21.15.46:0/2046908469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496bdfac70 0x56496bdfb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.265+0000 7f7b575c6740 1 -- 172.21.15.46:0/2046908469 >> 172.21.15.46:0/2046908469 conn(0x56496bdf7200 msgr2=0x56496bdf75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.265+0000 7f7b575c6740 1 -- 172.21.15.46:0/2046908469 shutdown_connections 2024-09-10T07:41:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.266+0000 7f7b575c6740 1 -- 172.21.15.46:0/2046908469 wait complete. 2024-09-10T07:41:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.266+0000 7f7b575c6740 1 -- start start 2024-09-10T07:41:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.266+0000 7f7b575c6740 1 Processor -- start 2024-09-10T07:41:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.267+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfac70 0x56496be02cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.267+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfb590 0x56496be03230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.267+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be03770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.267+0000 7f7b575c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56496be08a20 con 0x56496be05e50 2024-09-10T07:41:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.267+0000 7f7b575c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56496bd1c0f0 con 0x56496bdfac70 2024-09-10T07:41:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.267+0000 7f7b575c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56496be08ba0 con 0x56496bdfb590 2024-09-10T07:41:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.267+0000 7f7b575c6740 10 client.0 _ll_register_callbacks cb 0x56496bd03bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfac70 0x56496be02cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfac70 0x56496be02cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53900/0 (socket says 172.21.15.46:53900) 2024-09-10T07:41:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4de00640 1 -- 172.21.15.46:0/3530490334 learned_addr learned my addr 172.21.15.46:0/3530490334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4d400640 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfb590 0x56496be03230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4de00640 1 -- 172.21.15.46:0/3530490334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfb590 msgr2=0x56496be03230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4de00640 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfb590 0x56496be03230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4de00640 1 -- 172.21.15.46:0/3530490334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 msgr2=0x56496be03770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4de00640 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be03770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.268+0000 7f7b4de00640 1 -- 172.21.15.46:0/3530490334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56496bad23c0 con 0x56496bdfac70 2024-09-10T07:41:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.269+0000 7f7b4de00640 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfac70 0x56496be02cf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009a20 tx=0x7f7b3c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad9e74b4458db85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.269+0000 7f7b2f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.269+0000 7f7b2f400640 1 -- 172.21.15.46:0/3530490334 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c035040 con 0x56496bdfac70 2024-09-10T07:41:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.270+0000 7f7b2f400640 1 -- 172.21.15.46:0/3530490334 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7b3c031540 con 0x56496bdfac70 2024-09-10T07:41:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.270+0000 7f7b575c6740 1 -- 172.21.15.46:0/3530490334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b4009b420 con 0x56496bdfac70 2024-09-10T07:41:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.270+0000 7f7b2f400640 1 -- 172.21.15.46:0/3530490334 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b3c04f050 con 0x56496bdfac70 2024-09-10T07:41:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.270+0000 7f7b575c6740 10 client.26630 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.270+0000 7f7b575c6740 1 -- 172.21.15.46:0/3530490334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56496be14aa0 con 0x56496bdfac70 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.270+0000 7f7b2f400640 1 -- 172.21.15.46:0/3530490334 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b4009b420 con 0x56496bdfac70 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.271+0000 7f7b2f400640 1 -- 172.21.15.46:0/3530490334 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7b3c046780 con 0x56496bdfac70 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.271+0000 7f7b575c6740 10 client.26630 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.271+0000 7f7b575c6740 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.271+0000 7f7b575c6740 20 client.26630 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.271+0000 7f7b575c6740 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56496be15f60 0x56496be183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.271+0000 7f7b575c6740 4 client.26630 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.271+0000 7f7b575c6740 1 -- 172.21.15.46:0/3530490334 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56496bd6e950 con 0x56496be15f60 2024-09-10T07:41:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.272+0000 7f7b4d400640 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56496be15f60 0x56496be183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.273+0000 7f7b4d400640 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56496be15f60 0x56496be183d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd1545694aae8043 server_cookie=970cb1b6a89e6a62 in_seq=0 out_seq=0 2024-09-10T07:41:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.273+0000 7f7b2f400640 10 client.26630 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.274+0000 7f7b2f400640 1 -- 172.21.15.46:0/3530490334 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56496bd6e950 con 0x56496be15f60 2024-09-10T07:41:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.274+0000 7f7b2f400640 10 client.26630 handle_command_reply: tid=0 2024-09-10T07:41:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.274+0000 7f7b575c6740 1 client.26630 shutdown 2024-09-10T07:41:23.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.274+0000 7f7b575c6740 1 -- 172.21.15.46:0/3530490334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfac70 msgr2=0x56496be02cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.274+0000 7f7b575c6740 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfac70 0x56496be02cf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009a20 tx=0x7f7b3c003820 comp rx=0 tx=0).stop 2024-09-10T07:41:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.275+0000 7f7b575c6740 1 -- 172.21.15.46:0/3530490334 shutdown_connections 2024-09-10T07:41:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.275+0000 7f7b575c6740 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56496be15f60 0x56496be183d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.275+0000 7f7b575c6740 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be03770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.275+0000 7f7b575c6740 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfb590 0x56496be03230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.275+0000 7f7b575c6740 1 --2- 172.21.15.46:0/3530490334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfac70 0x56496be02cf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.275+0000 7f7b575c6740 1 -- 172.21.15.46:0/3530490334 >> 172.21.15.46:0/3530490334 conn(0x56496bdf7200 msgr2=0x56496bdfcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.275+0000 7f7b2f400640 0 client.26630 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.276+0000 7f7b575c6740 1 -- 172.21.15.46:0/3530490334 shutdown_connections 2024-09-10T07:41:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.276+0000 7f7b575c6740 1 -- 172.21.15.46:0/3530490334 wait complete. 2024-09-10T07:41:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.276+0000 7f7b575c6740 20 client.26630 trim_cache size 0 max 16384 2024-09-10T07:41:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.295+0000 7f7b575c6740 1 Processor -- start 2024-09-10T07:41:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b575c6740 1 -- start start 2024-09-10T07:41:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496bdfce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be1dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be1e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b575c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501acab0 con 0x56496be05e50 2024-09-10T07:41:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b575c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501ae650 con 0x56496bdfb590 2024-09-10T07:41:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b575c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501ad180 con 0x56496bdfac70 2024-09-10T07:41:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be1dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b4de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496bdfce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be1dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53910/0 (socket says 172.21.15.46:53910) 2024-09-10T07:41:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.296+0000 7f7b4d400640 1 -- 172.21.15.46:0/1629818088 learned_addr learned my addr 172.21.15.46:0/1629818088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.297+0000 7f7b4de00640 1 -- 172.21.15.46:0/1629818088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 msgr2=0x56496be1dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.297+0000 7f7b4de00640 1 --2- 172.21.15.46:0/1629818088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be1dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.297+0000 7f7b4de00640 1 -- 172.21.15.46:0/1629818088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 msgr2=0x56496be1e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.297+0000 7f7b4de00640 1 --2- 172.21.15.46:0/1629818088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be1e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.297+0000 7f7b4de00640 1 -- 172.21.15.46:0/1629818088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56496be18740 con 0x56496bdfac70 2024-09-10T07:41:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.297+0000 7f7b4d400640 1 --2- 172.21.15.46:0/1629818088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be1dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.297+0000 7f7b4de00640 1 --2- 172.21.15.46:0/1629818088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496bdfce20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c006010 tx=0x7f7b3c0058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadc860dfc5b4a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.298+0000 7f7b3a600640 1 -- 172.21.15.46:0/1629818088 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c058070 con 0x56496bdfac70 2024-09-10T07:41:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.298+0000 7f7b3a600640 1 -- 172.21.15.46:0/1629818088 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b3c046960 con 0x56496bdfac70 2024-09-10T07:41:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.298+0000 7f7b3a600640 1 -- 172.21.15.46:0/1629818088 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c035030 con 0x56496bdfac70 2024-09-10T07:41:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 -- 172.21.15.46:0/1629818088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 msgr2=0x56496bdfce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1629818088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496bdfce20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c006010 tx=0x7f7b3c0058d0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 -- 172.21.15.46:0/1629818088 shutdown_connections 2024-09-10T07:41:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1629818088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be1e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1629818088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be1dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1629818088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496bdfce20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 -- 172.21.15.46:0/1629818088 >> 172.21.15.46:0/1629818088 conn(0x56496bdf7200 msgr2=0x56496bdfc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 -- 172.21.15.46:0/1629818088 shutdown_connections 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 -- 172.21.15.46:0/1629818088 wait complete. 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.299+0000 7f7b575c6740 1 -- start start 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.300+0000 7f7b575c6740 1 Processor -- start 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.300+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496be7c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.300+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be0de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.300+0000 7f7b575c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be0e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.300+0000 7f7b575c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501adf80 con 0x56496be05e50 2024-09-10T07:41:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.300+0000 7f7b575c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501ad8b0 con 0x56496bdfb590 2024-09-10T07:41:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.300+0000 7f7b575c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501aaa90 con 0x56496bdfac70 2024-09-10T07:41:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.300+0000 7f7b575c6740 10 client.0 _ll_register_callbacks cb 0x7f7b3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.301+0000 7f7b4e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be0e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.301+0000 7f7b4e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be0e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57078/0 (socket says 172.21.15.46:57078) 2024-09-10T07:41:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.301+0000 7f7b4e800640 1 -- 172.21.15.46:0/1984400543 learned_addr learned my addr 172.21.15.46:0/1984400543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.301+0000 7f7b4d400640 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be0de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b4e800640 1 -- 172.21.15.46:0/1984400543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 msgr2=0x56496be7c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b4de00640 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496be7c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b4e800640 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496be7c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b4e800640 1 -- 172.21.15.46:0/1984400543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 msgr2=0x56496be0de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b4e800640 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be0de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b4e800640 1 -- 172.21.15.46:0/1984400543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56496be0e930 con 0x56496be05e50 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b4de00640 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496be7c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b4e800640 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be0e3c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b48013d90 tx=0x7f7b48023850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e38b6ef021da34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.302+0000 7f7b2f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.303+0000 7f7b2f400640 1 -- 172.21.15.46:0/1984400543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48031060 con 0x56496be05e50 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.303+0000 7f7b2f400640 1 -- 172.21.15.46:0/1984400543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7b48023e00 con 0x56496be05e50 2024-09-10T07:41:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.303+0000 7f7b575c6740 1 -- 172.21.15.46:0/1984400543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b3c046960 con 0x56496be05e50 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.304+0000 7f7b2f400640 1 -- 172.21.15.46:0/1984400543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b480232e0 con 0x56496be05e50 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.304+0000 7f7b575c6740 10 client.16341 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.304+0000 7f7b575c6740 1 -- 172.21.15.46:0/1984400543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56496be91c80 con 0x56496be05e50 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.305+0000 7f7b2f400640 1 -- 172.21.15.46:0/1984400543 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b3c046960 con 0x56496be05e50 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.305+0000 7f7b2f400640 1 -- 172.21.15.46:0/1984400543 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7b4801f720 con 0x56496be05e50 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.305+0000 7f7b575c6740 10 client.16341 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.305+0000 7f7b575c6740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.305+0000 7f7b575c6740 20 client.16341 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.305+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56496be922b0 0x56496be946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.305+0000 7f7b575c6740 4 client.16341 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:41:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.305+0000 7f7b575c6740 1 -- 172.21.15.46:0/1984400543 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56496bd68e60 con 0x56496be922b0 2024-09-10T07:41:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.306+0000 7f7b4de00640 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56496be922b0 0x56496be946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.307+0000 7f7b4de00640 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56496be922b0 0x56496be946d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fdfe7deea61de76 server_cookie=bf9fba7ca09cb0e2 in_seq=0 out_seq=0 2024-09-10T07:41:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.307+0000 7f7b2f400640 10 client.16341 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.307+0000 7f7b2f400640 1 -- 172.21.15.46:0/1984400543 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56496bd68e60 con 0x56496be922b0 2024-09-10T07:41:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.307+0000 7f7b2f400640 10 client.16341 handle_command_reply: tid=0 2024-09-10T07:41:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.307+0000 7f7b575c6740 1 client.16341 shutdown 2024-09-10T07:41:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b575c6740 1 -- 172.21.15.46:0/1984400543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 msgr2=0x56496be0e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be0e3c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b48013d90 tx=0x7f7b48023850 comp rx=0 tx=0).stop 2024-09-10T07:41:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b575c6740 1 -- 172.21.15.46:0/1984400543 shutdown_connections 2024-09-10T07:41:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56496be922b0 0x56496be946d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56496be05e50 0x56496be0e3c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56496bdfb590 0x56496be0de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b2f400640 0 client.16341 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b575c6740 1 --2- 172.21.15.46:0/1984400543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56496bdfac70 0x56496be7c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.308+0000 7f7b575c6740 1 -- 172.21.15.46:0/1984400543 >> 172.21.15.46:0/1984400543 conn(0x56496bdf7200 msgr2=0x56496bdf6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.309+0000 7f7b575c6740 1 -- 172.21.15.46:0/1984400543 shutdown_connections 2024-09-10T07:41:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.309+0000 7f7b575c6740 1 -- 172.21.15.46:0/1984400543 wait complete. 2024-09-10T07:41:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.309+0000 7f7b575c6740 20 client.16341 trim_cache size 0 max 16384 2024-09-10T07:41:23.311 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:41:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b340a2c00 msgr2=0x7f7b340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b340a2c00 0x7f7b340a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b48005fd0 tx=0x7f7b48015040 comp rx=0 tx=0).stop 2024-09-10T07:41:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b501036e0 msgr2=0x7f7b50196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b501036e0 0x7f7b50196940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7b500697d0 tx=0x7f7b40008600 comp rx=0 tx=0).stop 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 shutdown_connections 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7b340a2c00 0x7f7b340a50c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b501082a0 0x7f7b5019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b501036e0 0x7f7b50196940 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 --2- 172.21.15.46:0/3603349476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b50102d30 0x7f7b50196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 >> 172.21.15.46:0/3603349476 conn(0x7f7b500fc7d0 msgr2=0x7f7b50105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 shutdown_connections 2024-09-10T07:41:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:23.311+0000 7f7b54a00640 1 -- 172.21.15.46:0/3603349476 wait complete. 2024-09-10T07:41:24.216 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:23 smithi046 ceph-mon[30705]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T07:41:24.221 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '93c5c7e4', total seconds elapsed: 30.3 2024-09-10T07:41:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:23 smithi181 ceph-mon[35397]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T07:41:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:23 smithi063 ceph-mon[35020]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T07:41:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 245.0 s, 1074.0 tps, lat 486.389 ms stddev 3019.982 2024-09-10T07:41:26.160 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:41:26.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:41:26.232 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:25 smithi046 ceph-mon[30705]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 860 KiB/s wr, 87 op/s 2024-09-10T07:41:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:25 smithi181 ceph-mon[35397]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 860 KiB/s wr, 87 op/s 2024-09-10T07:41:26.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:25 smithi063 ceph-mon[35020]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 860 KiB/s wr, 87 op/s 2024-09-10T07:41:26.823 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:28.266 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:27 smithi046 ceph-mon[30705]: pgmap v544: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-10T07:41:28.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:27 smithi181 ceph-mon[35397]: pgmap v544: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-10T07:41:28.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:27 smithi063 ceph-mon[35020]: pgmap v544: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-10T07:41:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.675+0000 7f882fe00640 1 -- 172.21.15.46:0/3218671023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 msgr2=0x7f883010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.675+0000 7f882fe00640 1 --2- 172.21.15.46:0/3218671023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 0x7f883010d7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f881c009930 tx=0x7f881c02a420 comp rx=0 tx=0).stop 2024-09-10T07:41:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.676+0000 7f882fe00640 1 -- 172.21.15.46:0/3218671023 shutdown_connections 2024-09-10T07:41:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.676+0000 7f882fe00640 1 --2- 172.21.15.46:0/3218671023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 0x7f883010d7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.676+0000 7f882fe00640 1 --2- 172.21.15.46:0/3218671023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88301008d0 0x7f8830100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.676+0000 7f882fe00640 1 --2- 172.21.15.46:0/3218671023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88301068d0 0x7f8830106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.676+0000 7f882fe00640 1 -- 172.21.15.46:0/3218671023 >> 172.21.15.46:0/3218671023 conn(0x7f88300fc610 msgr2=0x7f88300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.676+0000 7f882fe00640 1 -- 172.21.15.46:0/3218671023 shutdown_connections 2024-09-10T07:41:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.676+0000 7f882fe00640 1 -- 172.21.15.46:0/3218671023 wait complete. 2024-09-10T07:41:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.678+0000 7f882fe00640 1 Processor -- start 2024-09-10T07:41:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.678+0000 7f882fe00640 1 -- start start 2024-09-10T07:41:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.679+0000 7f882fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88301008d0 0x7f88301982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.897 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:41:28.898 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:41:28.898 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6a3d67ec-1bd8-497f-a2d5-b52de31dc00d", 2024-09-10T07:41:28.898 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:41:28.898 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:41:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.679+0000 7f882fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 0x7f8830198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.679+0000 7f882fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88301068d0 0x7f883019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.679+0000 7f882ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88301008d0 0x7f88301982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.679+0000 7f882ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88301008d0 0x7f88301982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34040/0 (socket says 172.21.15.46:34040) 2024-09-10T07:41:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.679+0000 7f882e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 0x7f8830198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.679+0000 7f882e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 0x7f8830198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58514/0 (socket says 172.21.15.46:58514) 2024-09-10T07:41:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.680+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f883006a440 con 0x7f88301068d0 2024-09-10T07:41:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.680+0000 7f882ea00640 1 -- 172.21.15.46:0/2232636473 learned_addr learned my addr 172.21.15.46:0/2232636473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.680+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f883006a2c0 con 0x7f88301008d0 2024-09-10T07:41:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.680+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f883006a5c0 con 0x7f8830101270 2024-09-10T07:41:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.681+0000 7f882e000640 1 -- 172.21.15.46:0/2232636473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88301008d0 msgr2=0x7f88301982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.681+0000 7f882e000640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88301008d0 0x7f88301982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.681+0000 7f882e000640 1 -- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88301068d0 msgr2=0x7f883019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.681+0000 7f882e000640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88301068d0 0x7f883019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.681+0000 7f882e000640 1 -- 172.21.15.46:0/2232636473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883019d270 con 0x7f8830101270 2024-09-10T07:41:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.682+0000 7f882e000640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 0x7f8830198800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f882000b3f0 tx=0x7f882000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.682+0000 7f8827400640 1 -- 172.21.15.46:0/2232636473 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820015020 con 0x7f8830101270 2024-09-10T07:41:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.683+0000 7f8827400640 1 -- 172.21.15.46:0/2232636473 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8820011d10 con 0x7f8830101270 2024-09-10T07:41:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.683+0000 7f8827400640 1 -- 172.21.15.46:0/2232636473 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882000f2e0 con 0x7f8830101270 2024-09-10T07:41:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.683+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883019d560 con 0x7f8830101270 2024-09-10T07:41:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.684+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8830110870 con 0x7f8830101270 2024-09-10T07:41:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.688+0000 7f88383dc740 1 Processor -- start 2024-09-10T07:41:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.688+0000 7f88383dc740 1 -- start start 2024-09-10T07:41:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.689+0000 7f88383dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456dcb0 0x556ab456e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.689+0000 7f88383dc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456e5d0 0x556ab4578950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.689+0000 7f88383dc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab457b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.689+0000 7f88383dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ab44d4290 con 0x556ab456dcb0 2024-09-10T07:41:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.689+0000 7f88383dc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556ab447bb40 con 0x556ab456e5d0 2024-09-10T07:41:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.689+0000 7f88383dc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556ab44d0250 con 0x556ab4578e90 2024-09-10T07:41:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456dcb0 0x556ab456e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456dcb0 0x556ab456e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34988/0 (socket says 172.21.15.46:34988) 2024-09-10T07:41:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882f400640 1 -- 172.21.15.46:0/460404782 learned_addr learned my addr 172.21.15.46:0/460404782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882ea00640 1 --2- 172.21.15.46:0/460404782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456e5d0 0x556ab4578950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882f400640 1 -- 172.21.15.46:0/460404782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 msgr2=0x556ab457b280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:41:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882f400640 1 --2- 172.21.15.46:0/460404782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab457b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882f400640 1 -- 172.21.15.46:0/460404782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456e5d0 msgr2=0x556ab4578950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882f400640 1 --2- 172.21.15.46:0/460404782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456e5d0 0x556ab4578950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f882f400640 1 -- 172.21.15.46:0/460404782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab42456a0 con 0x556ab456dcb0 2024-09-10T07:41:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.690+0000 7f8827400640 1 -- 172.21.15.46:0/2232636473 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8820018020 con 0x7f8830101270 2024-09-10T07:41:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.691+0000 7f882f400640 1 --2- 172.21.15.46:0/460404782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456dcb0 0x556ab456e090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f881c005c20 tx=0x7f881c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ae0868529c3f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.691+0000 7f881be00640 1 -- 172.21.15.46:0/460404782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c03e030 con 0x556ab456dcb0 2024-09-10T07:41:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.692+0000 7f881be00640 1 -- 172.21.15.46:0/460404782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f881c0035f0 con 0x556ab456dcb0 2024-09-10T07:41:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.693+0000 7f8827400640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88140a2c00 0x7f88140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.693+0000 7f88383dc740 1 -- 172.21.15.46:0/460404782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456dcb0 msgr2=0x556ab456e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.693+0000 7f881be00640 1 -- 172.21.15.46:0/460404782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c035040 con 0x556ab456dcb0 2024-09-10T07:41:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.693+0000 7f8827400640 1 -- 172.21.15.46:0/2232636473 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f88200ceae0 con 0x7f8830101270 2024-09-10T07:41:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.693+0000 7f88383dc740 1 --2- 172.21.15.46:0/460404782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456dcb0 0x556ab456e090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f881c005c20 tx=0x7f881c005e00 comp rx=0 tx=0).stop 2024-09-10T07:41:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.693+0000 7f882ea00640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88140a2c00 0x7f88140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.694+0000 7f88383dc740 1 -- 172.21.15.46:0/460404782 shutdown_connections 2024-09-10T07:41:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.694+0000 7f88383dc740 1 --2- 172.21.15.46:0/460404782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab457b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.694+0000 7f88383dc740 1 --2- 172.21.15.46:0/460404782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456e5d0 0x556ab4578950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.694+0000 7f88383dc740 1 --2- 172.21.15.46:0/460404782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456dcb0 0x556ab456e090 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.694+0000 7f88383dc740 1 -- 172.21.15.46:0/460404782 >> 172.21.15.46:0/460404782 conn(0x556ab456a240 msgr2=0x556ab456a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.694+0000 7f88383dc740 1 -- 172.21.15.46:0/460404782 shutdown_connections 2024-09-10T07:41:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.694+0000 7f88383dc740 1 -- 172.21.15.46:0/460404782 wait complete. 2024-09-10T07:41:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.695+0000 7f88383dc740 1 -- start start 2024-09-10T07:41:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.695+0000 7f882ea00640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88140a2c00 0x7f88140a50c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8828010fd0 tx=0x7f8828012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.695+0000 7f88383dc740 1 Processor -- start 2024-09-10T07:41:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f88383dc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4578750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f88383dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 0x556ab4575a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f88383dc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab4575f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f88383dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ab457bb50 con 0x556ab456e5d0 2024-09-10T07:41:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f88383dc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556ab448efa0 con 0x556ab456dcb0 2024-09-10T07:41:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f88383dc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556ab457bcd0 con 0x556ab4578e90 2024-09-10T07:41:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f88383dc740 10 client.0 _ll_register_callbacks cb 0x556ab41e2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f882e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 0x556ab4575a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f882e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 0x556ab4575a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35004/0 (socket says 172.21.15.46:35004) 2024-09-10T07:41:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f882f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab4575f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f882f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab4575f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58528/0 (socket says 172.21.15.46:58528) 2024-09-10T07:41:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.696+0000 7f882f400640 1 -- 172.21.15.46:0/2431023204 learned_addr learned my addr 172.21.15.46:0/2431023204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.697+0000 7f882ea00640 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4578750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.697+0000 7f882f400640 1 -- 172.21.15.46:0/2431023204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 msgr2=0x556ab4578750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.697+0000 7f882f400640 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4578750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.697+0000 7f882f400640 1 -- 172.21.15.46:0/2431023204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 msgr2=0x556ab4575a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.697+0000 7f882f400640 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 0x556ab4575a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.697+0000 7f882f400640 1 -- 172.21.15.46:0/2431023204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab4576770 con 0x556ab4578e90 2024-09-10T07:41:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.697+0000 7f882f400640 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab4575f40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f881c009850 tx=0x7f881c009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c401bf158d4c9642 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.697+0000 7f8818c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:41:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.698+0000 7f8818c00640 1 -- 172.21.15.46:0/2431023204 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c047070 con 0x556ab4578e90 2024-09-10T07:41:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.698+0000 7f8818c00640 1 -- 172.21.15.46:0/2431023204 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f881c004400 con 0x556ab4578e90 2024-09-10T07:41:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.698+0000 7f88383dc740 1 -- 172.21.15.46:0/2431023204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881c0035f0 con 0x556ab4578e90 2024-09-10T07:41:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.699+0000 7f8818c00640 1 -- 172.21.15.46:0/2431023204 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f881c055050 con 0x556ab4578e90 2024-09-10T07:41:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.699+0000 7f88383dc740 10 client.26623 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.699+0000 7f88383dc740 1 -- 172.21.15.46:0/2431023204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556ab457bba0 con 0x556ab4578e90 2024-09-10T07:41:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.699+0000 7f8818c00640 1 -- 172.21.15.46:0/2431023204 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f881c0035f0 con 0x556ab4578e90 2024-09-10T07:41:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.700+0000 7f8818c00640 1 -- 172.21.15.46:0/2431023204 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f881c04c7a0 con 0x556ab4578e90 2024-09-10T07:41:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.700+0000 7f88383dc740 10 client.26623 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.700+0000 7f88383dc740 10 client.26623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.700+0000 7f88383dc740 20 client.26623 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.700+0000 7f88383dc740 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ab45914b0 0x556ab45938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.700+0000 7f88383dc740 4 client.26623 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.700+0000 7f88383dc740 1 -- 172.21.15.46:0/2431023204 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ab44e1800 con 0x556ab45914b0 2024-09-10T07:41:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.702+0000 7f882ea00640 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ab45914b0 0x556ab45938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.703+0000 7f882ea00640 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ab45914b0 0x556ab45938a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d78c656f26992af server_cookie=c2b58854ad3ffe23 in_seq=0 out_seq=0 2024-09-10T07:41:28.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.703+0000 7f8818c00640 10 client.26623 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.704+0000 7f8818c00640 1 -- 172.21.15.46:0/2431023204 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ab44e1800 con 0x556ab45914b0 2024-09-10T07:41:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.704+0000 7f8818c00640 10 client.26623 handle_command_reply: tid=0 2024-09-10T07:41:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.705+0000 7f88383dc740 1 client.26623 shutdown 2024-09-10T07:41:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.705+0000 7f88383dc740 1 -- 172.21.15.46:0/2431023204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 msgr2=0x556ab4575f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.705+0000 7f88383dc740 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab4575f40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f881c009850 tx=0x7f881c009520 comp rx=0 tx=0).stop 2024-09-10T07:41:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.707+0000 7f88383dc740 1 -- 172.21.15.46:0/2431023204 shutdown_connections 2024-09-10T07:41:28.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.707+0000 7f88383dc740 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ab45914b0 0x556ab45938a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.707+0000 7f88383dc740 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab4578e90 0x556ab4575f40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.707+0000 7f88383dc740 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 0x556ab4575a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.707+0000 7f88383dc740 1 --2- 172.21.15.46:0/2431023204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4578750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.707+0000 7f88383dc740 1 -- 172.21.15.46:0/2431023204 >> 172.21.15.46:0/2431023204 conn(0x556ab456a240 msgr2=0x556ab456ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:28.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.707+0000 7f8818c00640 0 client.26623 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:28.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.707+0000 7f88383dc740 1 -- 172.21.15.46:0/2431023204 shutdown_connections 2024-09-10T07:41:28.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.708+0000 7f88383dc740 1 -- 172.21.15.46:0/2431023204 wait complete. 2024-09-10T07:41:28.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.708+0000 7f88383dc740 20 client.26623 trim_cache size 0 max 16384 2024-09-10T07:41:28.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.752+0000 7f88383dc740 1 Processor -- start 2024-09-10T07:41:28.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.752+0000 7f88383dc740 1 -- start start 2024-09-10T07:41:28.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f88383dc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456dcb0 0x556ab45748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f88383dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 0x556ab4598fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f88383dc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab4578e90 0x556ab4599500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f88383dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88301ac890 con 0x556ab456e5d0 2024-09-10T07:41:28.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f88383dc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88301ae430 con 0x556ab4578e90 2024-09-10T07:41:28.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f88383dc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88301acf60 con 0x556ab456dcb0 2024-09-10T07:41:28.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f882ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456dcb0 0x556ab45748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f882ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456dcb0 0x556ab45748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58534/0 (socket says 172.21.15.46:58534) 2024-09-10T07:41:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f882ea00640 1 -- 172.21.15.46:0/2169738986 learned_addr learned my addr 172.21.15.46:0/2169738986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.753+0000 7f882f400640 1 --2- 172.21.15.46:0/2169738986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab4578e90 0x556ab4599500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.754+0000 7f882f400640 1 -- 172.21.15.46:0/2169738986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456dcb0 msgr2=0x556ab45748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.754+0000 7f882f400640 1 --2- 172.21.15.46:0/2169738986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456dcb0 0x556ab45748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.754+0000 7f882f400640 1 -- 172.21.15.46:0/2169738986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 msgr2=0x556ab4598fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.754+0000 7f882f400640 1 --2- 172.21.15.46:0/2169738986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 0x556ab4598fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.754+0000 7f882f400640 1 -- 172.21.15.46:0/2169738986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab456e400 con 0x556ab4578e90 2024-09-10T07:41:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.755+0000 7f882f400640 1 --2- 172.21.15.46:0/2169738986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab4578e90 0x556ab4599500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f881c04cba0 tx=0x7f881c031500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a3be9953df17d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.755+0000 7f881b400640 1 -- 172.21.15.46:0/2169738986 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c05e070 con 0x556ab4578e90 2024-09-10T07:41:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.756+0000 7f881b400640 1 -- 172.21.15.46:0/2169738986 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f881c0035f0 con 0x556ab4578e90 2024-09-10T07:41:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.756+0000 7f881b400640 1 -- 172.21.15.46:0/2169738986 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c047030 con 0x556ab4578e90 2024-09-10T07:41:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.757+0000 7f88383dc740 1 -- 172.21.15.46:0/2169738986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab4578e90 msgr2=0x556ab4599500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.757+0000 7f88383dc740 1 --2- 172.21.15.46:0/2169738986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab4578e90 0x556ab4599500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f881c04cba0 tx=0x7f881c031500 comp rx=0 tx=0).stop 2024-09-10T07:41:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.758+0000 7f88383dc740 1 -- 172.21.15.46:0/2169738986 shutdown_connections 2024-09-10T07:41:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.758+0000 7f88383dc740 1 --2- 172.21.15.46:0/2169738986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab4578e90 0x556ab4599500 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.758+0000 7f88383dc740 1 --2- 172.21.15.46:0/2169738986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab456e5d0 0x556ab4598fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.758+0000 7f88383dc740 1 --2- 172.21.15.46:0/2169738986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456dcb0 0x556ab45748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.758+0000 7f88383dc740 1 -- 172.21.15.46:0/2169738986 >> 172.21.15.46:0/2169738986 conn(0x556ab456a240 msgr2=0x556ab45741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.758+0000 7f88383dc740 1 -- 172.21.15.46:0/2169738986 shutdown_connections 2024-09-10T07:41:28.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.758+0000 7f88383dc740 1 -- 172.21.15.46:0/2169738986 wait complete. 2024-09-10T07:41:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.759+0000 7f88383dc740 1 -- start start 2024-09-10T07:41:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.759+0000 7f88383dc740 1 Processor -- start 2024-09-10T07:41:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.761+0000 7f88383dc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4588830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.761+0000 7f88383dc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456e5d0 0x556ab4588d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.761+0000 7f88383dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab4578e90 0x556ab45894e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.761+0000 7f88383dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88301add60 con 0x556ab4578e90 2024-09-10T07:41:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.761+0000 7f88383dc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88301ad690 con 0x556ab456dcb0 2024-09-10T07:41:28.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.761+0000 7f88383dc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88301aa870 con 0x556ab456e5d0 2024-09-10T07:41:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.761+0000 7f88383dc740 10 client.0 _ll_register_callbacks cb 0x7f8828010d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.762+0000 7f882ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4588830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.762+0000 7f882ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4588830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34076/0 (socket says 172.21.15.46:34076) 2024-09-10T07:41:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.762+0000 7f882ea00640 1 -- 172.21.15.46:0/609877081 learned_addr learned my addr 172.21.15.46:0/609877081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.762+0000 7f882e000640 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456e5d0 0x556ab4588d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.762+0000 7f882f400640 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab4578e90 0x556ab45894e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.763+0000 7f882ea00640 1 -- 172.21.15.46:0/609877081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456e5d0 msgr2=0x556ab4588d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.763+0000 7f882ea00640 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456e5d0 0x556ab4588d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.763+0000 7f882ea00640 1 -- 172.21.15.46:0/609877081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab4578e90 msgr2=0x556ab45894e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.763+0000 7f882ea00640 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab4578e90 0x556ab45894e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.763+0000 7f882ea00640 1 -- 172.21.15.46:0/609877081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab4589d10 con 0x556ab456dcb0 2024-09-10T07:41:28.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.763+0000 7f882ea00640 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4588830 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8828010280 tx=0x7f8828004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b557f8dfefcb57f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:28.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.763+0000 7f8818c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:28.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.764+0000 7f8818c00640 1 -- 172.21.15.46:0/609877081 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882800f870 con 0x556ab456dcb0 2024-09-10T07:41:28.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.764+0000 7f8818c00640 1 -- 172.21.15.46:0/609877081 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f882800fa10 con 0x556ab456dcb0 2024-09-10T07:41:28.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.764+0000 7f88383dc740 1 -- 172.21.15.46:0/609877081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881c0035f0 con 0x556ab456dcb0 2024-09-10T07:41:28.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.765+0000 7f8818c00640 1 -- 172.21.15.46:0/609877081 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f882800fc10 con 0x556ab456dcb0 2024-09-10T07:41:28.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.765+0000 7f88383dc740 10 client.26654 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:28.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.765+0000 7f88383dc740 1 -- 172.21.15.46:0/609877081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556ab460d960 con 0x556ab456dcb0 2024-09-10T07:41:28.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.766+0000 7f8818c00640 1 -- 172.21.15.46:0/609877081 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f881c0035f0 con 0x556ab456dcb0 2024-09-10T07:41:28.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.766+0000 7f8818c00640 1 -- 172.21.15.46:0/609877081 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f8828031070 con 0x556ab456dcb0 2024-09-10T07:41:28.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.766+0000 7f88383dc740 10 client.26654 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:28.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.766+0000 7f88383dc740 10 client.26654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:28.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.766+0000 7f88383dc740 20 client.26654 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:28.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.766+0000 7f88383dc740 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ab460e110 0x556ab4610530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:28.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.766+0000 7f88383dc740 4 client.26654 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:41:28.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.766+0000 7f88383dc740 1 -- 172.21.15.46:0/609877081 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556ab44dbd10 con 0x556ab460e110 2024-09-10T07:41:28.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.767+0000 7f882e000640 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ab460e110 0x556ab4610530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:28.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.768+0000 7f882e000640 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ab460e110 0x556ab4610530 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e7353e7d14b395 server_cookie=9e96c293fd2868ff in_seq=0 out_seq=0 2024-09-10T07:41:28.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.768+0000 7f8818c00640 10 client.26654 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:28.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.770+0000 7f8818c00640 1 -- 172.21.15.46:0/609877081 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556ab44dbd10 con 0x556ab460e110 2024-09-10T07:41:28.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.770+0000 7f8818c00640 10 client.26654 handle_command_reply: tid=0 2024-09-10T07:41:28.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.772+0000 7f88383dc740 1 client.26654 shutdown 2024-09-10T07:41:28.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 -- 172.21.15.46:0/609877081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 msgr2=0x556ab4588830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4588830 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8828010280 tx=0x7f8828004ab0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 -- 172.21.15.46:0/609877081 shutdown_connections 2024-09-10T07:41:28.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ab460e110 0x556ab4610530 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ab4578e90 0x556ab45894e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ab456e5d0 0x556ab4588d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 --2- 172.21.15.46:0/609877081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ab456dcb0 0x556ab4588830 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 -- 172.21.15.46:0/609877081 >> 172.21.15.46:0/609877081 conn(0x556ab456a240 msgr2=0x556ab45749e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f8818c00640 0 client.26654 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 -- 172.21.15.46:0/609877081 shutdown_connections 2024-09-10T07:41:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.773+0000 7f88383dc740 1 -- 172.21.15.46:0/609877081 wait complete. 2024-09-10T07:41:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.774+0000 7f88383dc740 20 client.26654 trim_cache size 0 max 16384 2024-09-10T07:41:28.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88140a2c00 msgr2=0x7f88140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88140a2c00 0x7f88140a50c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8828010fd0 tx=0x7f8828012040 comp rx=0 tx=0).stop 2024-09-10T07:41:28.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 msgr2=0x7f8830198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:28.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 0x7f8830198800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f882000b3f0 tx=0x7f882000b8b0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 shutdown_connections 2024-09-10T07:41:28.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88140a2c00 0x7f88140a50c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88301068d0 0x7f883019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830101270 0x7f8830198800 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 --2- 172.21.15.46:0/2232636473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88301008d0 0x7f88301982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:28.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.775+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 >> 172.21.15.46:0/2232636473 conn(0x7f88300fc610 msgr2=0x7f88300fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:28.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.776+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 shutdown_connections 2024-09-10T07:41:28.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:28.776+0000 7f882fe00640 1 -- 172.21.15.46:0/2232636473 wait complete. 2024-09-10T07:41:29.159 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:41:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: 2024-09-10T07:41:28.798+0000 7f775d800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/pg_wal/ auth{1=1} v1344 ap=2 f(v8 m2024-09-10T07:40:26.281113+0000 34=33+1) n(v15 rc2024-09-10T07:40:59.542756+0000 b553648128 35=33+2) (inest sync->mix g=1) caps={15174=pAsLsXsFs/pAsLsXsFsx@196} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55c133889600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000007:[<0x10000000006/pg_wal v274>,<0x10000000005/data v78>,<0x100000001fb/postgres v13>,<0x100000001fa/a8e9ad88-14b3-4732-92a7-3dc20544565e v31>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v36>,<0x1/volumes v36>]//[]","memoryvalue":"(2)0x10000000007:[<0x10000000006/pg_wal v1344>,<0x10000000005/data v358>,<0x100000001fb/postgres v205>,<0x100000001fa/a8e9ad88-14b3-4732-92a7-3dc20544565e v218>,<0x10000000001/sv_1 v245>,<0x10000000000/qa v225>,<0x1/volumes v219>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:39:58.655410+0000 33=32+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:40:59.542756+0000 b553648128 35=33+2)","memory_value.dirstat":"f(v8 m2024-09-10T07:40:26.281113+0000 34=33+1)","memory_value.rstat":"n(v15 rc2024-09-10T07:40:59.542756+0000 b553648128 35=33+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T07:41:29.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:41:30.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: pgmap v545: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-09-10T07:41:30.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:41:30.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:41:30.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:41:30.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/pg_wal) see mds.a log and `damage ls` output for details 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:29 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: pgmap v545: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-09-10T07:41:30.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:41:30.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:41:30.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:41:30.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:41:30.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:41:30.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:41:30.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:41:30.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/pg_wal) see mds.a log and `damage ls` output for details 2024-09-10T07:41:30.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.402 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:30 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: pgmap v545: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-09-10T07:41:30.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:41:30.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:41:30.404 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:41:30.404 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.404 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:41:30.404 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:41:30.404 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:41:30.404 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:41:30.405 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/pg_wal) see mds.a log and `damage ls` output for details 2024-09-10T07:41:30.405 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.405 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.405 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.405 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.405 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:30 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:41:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:41:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 250.0 s, 845.8 tps, lat 115.085 ms stddev 131.706 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.328+0000 7fd092600640 1 -- 172.21.15.46:0/868755907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109c40 msgr2=0x7fd08c10e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.328+0000 7fd092600640 1 --2- 172.21.15.46:0/868755907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109c40 0x7fd08c10e080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009900 tx=0x7fd07c02a400 comp rx=0 tx=0).stop 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.329+0000 7fd092600640 1 -- 172.21.15.46:0/868755907 shutdown_connections 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.329+0000 7fd092600640 1 --2- 172.21.15.46:0/868755907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08c10e7b0 0x7fd08c115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.329+0000 7fd092600640 1 --2- 172.21.15.46:0/868755907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109c40 0x7fd08c10e080 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.329+0000 7fd092600640 1 --2- 172.21.15.46:0/868755907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd08c109290 0x7fd08c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.329+0000 7fd092600640 1 -- 172.21.15.46:0/868755907 >> 172.21.15.46:0/868755907 conn(0x7fd08c077c10 msgr2=0x7fd08c07a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.329+0000 7fd092600640 1 -- 172.21.15.46:0/868755907 shutdown_connections 2024-09-10T07:41:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.330+0000 7fd092600640 1 -- 172.21.15.46:0/868755907 wait complete. 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.332+0000 7fd092600640 1 Processor -- start 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.333+0000 7fd092600640 1 -- start start 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.333+0000 7fd092600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109290 0x7fd08c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd092600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08c109c40 0x7fd08c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd092600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd08c10e7b0 0x7fd08c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd092600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c118590 con 0x7fd08c109c40 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd092600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c118410 con 0x7fd08c10e7b0 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd092600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c118710 con 0x7fd08c109290 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd08b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109290 0x7fd08c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd08be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd08c10e7b0 0x7fd08c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd08b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109290 0x7fd08c1a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58562/0 (socket says 172.21.15.46:58562) 2024-09-10T07:41:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.334+0000 7fd08b400640 1 -- 172.21.15.46:0/3759722806 learned_addr learned my addr 172.21.15.46:0/3759722806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.335+0000 7fd08be00640 1 -- 172.21.15.46:0/3759722806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109290 msgr2=0x7fd08c1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.335+0000 7fd08be00640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109290 0x7fd08c1a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.335+0000 7fd08be00640 1 -- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08c109c40 msgr2=0x7fd08c1a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.335+0000 7fd08be00640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08c109c40 0x7fd08c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.335+0000 7fd08be00640 1 -- 172.21.15.46:0/3759722806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c19ef80 con 0x7fd08c10e7b0 2024-09-10T07:41:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.335+0000 7fd08be00640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd08c10e7b0 0x7fd08c19e7e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd07800c740 tx=0x7fd07800cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.336+0000 7fd083e00640 1 -- 172.21.15.46:0/3759722806 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078007bb0 con 0x7fd08c10e7b0 2024-09-10T07:41:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.336+0000 7fd083e00640 1 -- 172.21.15.46:0/3759722806 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07800ce50 con 0x7fd08c10e7b0 2024-09-10T07:41:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.336+0000 7fd083e00640 1 -- 172.21.15.46:0/3759722806 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078018730 con 0x7fd08c10e7b0 2024-09-10T07:41:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.336+0000 7fd092600640 1 -- 172.21.15.46:0/3759722806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c19f270 con 0x7fd08c10e7b0 2024-09-10T07:41:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.337+0000 7fd092600640 1 -- 172.21.15.46:0/3759722806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c1ab4c0 con 0x7fd08c10e7b0 2024-09-10T07:41:31.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.346+0000 7fd083e00640 1 -- 172.21.15.46:0/3759722806 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0780188d0 con 0x7fd08c10e7b0 2024-09-10T07:41:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.347+0000 7fd094ff6740 1 Processor -- start 2024-09-10T07:41:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.347+0000 7fd094ff6740 1 -- start start 2024-09-10T07:41:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.348+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea874f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.349+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.349+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea882160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.349+0000 7fd094ff6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562fea7db3e0 con 0x562fea874b90 2024-09-10T07:41:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.349+0000 7fd094ff6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562fea782d80 con 0x562fea8754b0 2024-09-10T07:41:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.349+0000 7fd094ff6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562fea7d73a0 con 0x562fea87fd70 2024-09-10T07:41:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.350+0000 7fd083e00640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd0700a2cd0 0x7fd0700a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.351+0000 7fd083e00640 1 -- 172.21.15.46:0/3759722806 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd0780ced30 con 0x7fd08c10e7b0 2024-09-10T07:41:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.351+0000 7fd08be00640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd0700a2cd0 0x7fd0700a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.351+0000 7fd08b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.352+0000 7fd08b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34118/0 (socket says 172.21.15.46:34118) 2024-09-10T07:41:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.352+0000 7fd08b400640 1 -- 172.21.15.46:0/3520938465 learned_addr learned my addr 172.21.15.46:0/3520938465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.352+0000 7fd08be00640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd0700a2cd0 0x7fd0700a5190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd07809b220 tx=0x7fd078007f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.352+0000 7fd08aa00640 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea882160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.354+0000 7fd08b400640 1 -- 172.21.15.46:0/3520938465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 msgr2=0x562fea882160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.354+0000 7fd08b400640 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea882160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.354+0000 7fd08aa00640 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea874f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.354+0000 7fd08b400640 1 -- 172.21.15.46:0/3520938465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 msgr2=0x562fea874f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:31.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.354+0000 7fd08b400640 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea874f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.354+0000 7fd08b400640 1 -- 172.21.15.46:0/3520938465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fea7a1ae0 con 0x562fea8754b0 2024-09-10T07:41:31.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.354+0000 7fd08aa00640 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea874f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:41:31.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.354+0000 7fd08b400640 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87f830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd0840103a0 tx=0x7fd084010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b77479a7a1f1ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:31.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.355+0000 7fd080c00640 1 -- 172.21.15.46:0/3520938465 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084016070 con 0x562fea8754b0 2024-09-10T07:41:31.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.355+0000 7fd080c00640 1 -- 172.21.15.46:0/3520938465 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd084012bf0 con 0x562fea8754b0 2024-09-10T07:41:31.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.355+0000 7fd080c00640 1 -- 172.21.15.46:0/3520938465 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08401be60 con 0x562fea8754b0 2024-09-10T07:41:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.356+0000 7fd094ff6740 1 -- 172.21.15.46:0/3520938465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 msgr2=0x562fea87f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.356+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87f830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd0840103a0 tx=0x7fd084010860 comp rx=0 tx=0).stop 2024-09-10T07:41:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.356+0000 7fd094ff6740 1 -- 172.21.15.46:0/3520938465 shutdown_connections 2024-09-10T07:41:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.356+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea882160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.356+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87f830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.356+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3520938465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea874f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.356+0000 7fd094ff6740 1 -- 172.21.15.46:0/3520938465 >> 172.21.15.46:0/3520938465 conn(0x562fea871120 msgr2=0x562fea871500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.357+0000 7fd094ff6740 1 -- 172.21.15.46:0/3520938465 shutdown_connections 2024-09-10T07:41:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.357+0000 7fd094ff6740 1 -- 172.21.15.46:0/3520938465 wait complete. 2024-09-10T07:41:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.357+0000 7fd094ff6740 1 -- start start 2024-09-10T07:41:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.358+0000 7fd094ff6740 1 Processor -- start 2024-09-10T07:41:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.358+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea87ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea87d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd094ff6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562fea8829e0 con 0x562fea874b90 2024-09-10T07:41:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd094ff6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562fea7960f0 con 0x562fea8754b0 2024-09-10T07:41:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd094ff6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562fea882b60 con 0x562fea87fd70 2024-09-10T07:41:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd094ff6740 10 client.0 _ll_register_callbacks cb 0x562fea7879e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd08aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea87d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd08aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea87d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58592/0 (socket says 172.21.15.46:58592) 2024-09-10T07:41:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd08aa00640 1 -- 172.21.15.46:0/2165511252 learned_addr learned my addr 172.21.15.46:0/2165511252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.359+0000 7fd08aa00640 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea87ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.360+0000 7fd08b400640 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.361+0000 7fd08aa00640 1 -- 172.21.15.46:0/2165511252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 msgr2=0x562fea87d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:31.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.361+0000 7fd08aa00640 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.361+0000 7fd08aa00640 1 -- 172.21.15.46:0/2165511252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 msgr2=0x562fea87ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:31.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.361+0000 7fd08aa00640 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea87ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.361+0000 7fd08aa00640 1 -- 172.21.15.46:0/2165511252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fea54c3c0 con 0x562fea87fd70 2024-09-10T07:41:31.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.364+0000 7fd08aa00640 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea87d750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd07c005ab0 tx=0x7fd07c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495e4bbb7604288e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:31.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.364+0000 7fd075600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:41:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.365+0000 7fd075600640 1 -- 172.21.15.46:0/2165511252 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c035070 con 0x562fea87fd70 2024-09-10T07:41:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.365+0000 7fd094ff6740 1 -- 172.21.15.46:0/2165511252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd084012bf0 con 0x562fea87fd70 2024-09-10T07:41:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.365+0000 7fd075600640 1 -- 172.21.15.46:0/2165511252 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd07c03d650 con 0x562fea87fd70 2024-09-10T07:41:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.368+0000 7fd075600640 1 -- 172.21.15.46:0/2165511252 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd07c051050 con 0x562fea87fd70 2024-09-10T07:41:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.368+0000 7fd094ff6740 10 client.26641 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.368+0000 7fd094ff6740 1 -- 172.21.15.46:0/2165511252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562fea88e700 con 0x562fea87fd70 2024-09-10T07:41:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.369+0000 7fd075600640 1 -- 172.21.15.46:0/2165511252 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd084012bf0 con 0x562fea87fd70 2024-09-10T07:41:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.369+0000 7fd075600640 1 -- 172.21.15.46:0/2165511252 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd07c03a040 con 0x562fea87fd70 2024-09-10T07:41:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.372+0000 7fd094ff6740 10 client.26641 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.372+0000 7fd094ff6740 10 client.26641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.372+0000 7fd094ff6740 20 client.26641 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.372+0000 7fd094ff6740 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562fea88fb10 0x562fea891f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.372+0000 7fd094ff6740 4 client.26641 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.372+0000 7fd094ff6740 1 -- 172.21.15.46:0/2165511252 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fea7e8950 con 0x562fea88fb10 2024-09-10T07:41:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.374+0000 7fd08b400640 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562fea88fb10 0x562fea891f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.375+0000 7fd08b400640 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562fea88fb10 0x562fea891f00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afcf7f46001e5dc3 server_cookie=daa9e008a2c2ce63 in_seq=0 out_seq=0 2024-09-10T07:41:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.376+0000 7fd075600640 10 client.26641 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.377+0000 7fd075600640 1 -- 172.21.15.46:0/2165511252 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fea7e8950 con 0x562fea88fb10 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.377+0000 7fd075600640 10 client.26641 handle_command_reply: tid=0 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.377+0000 7fd094ff6740 1 client.26641 shutdown 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 -- 172.21.15.46:0/2165511252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 msgr2=0x562fea87d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea87d750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd07c005ab0 tx=0x7fd07c005850 comp rx=0 tx=0).stop 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 -- 172.21.15.46:0/2165511252 shutdown_connections 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562fea88fb10 0x562fea891f00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea87d750 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea87d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 --2- 172.21.15.46:0/2165511252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea87ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 -- 172.21.15.46:0/2165511252 >> 172.21.15.46:0/2165511252 conn(0x562fea871120 msgr2=0x562fea876e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd075600640 0 client.26641 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 -- 172.21.15.46:0/2165511252 shutdown_connections 2024-09-10T07:41:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.378+0000 7fd094ff6740 1 -- 172.21.15.46:0/2165511252 wait complete. 2024-09-10T07:41:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.379+0000 7fd094ff6740 20 client.26641 trim_cache size 0 max 16384 2024-09-10T07:41:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.404+0000 7fd094ff6740 1 Processor -- start 2024-09-10T07:41:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.405+0000 7fd094ff6740 1 -- start start 2024-09-10T07:41:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.405+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea87d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.405+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea897750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.405+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea897c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.405+0000 7fd094ff6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b52d0 con 0x562fea874b90 2024-09-10T07:41:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.405+0000 7fd094ff6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b6e70 con 0x562fea8754b0 2024-09-10T07:41:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.405+0000 7fd094ff6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b59a0 con 0x562fea87fd70 2024-09-10T07:41:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.406+0000 7fd08b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea897750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.406+0000 7fd08b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea897750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34146/0 (socket says 172.21.15.46:34146) 2024-09-10T07:41:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.406+0000 7fd08b400640 1 -- 172.21.15.46:0/1612326131 learned_addr learned my addr 172.21.15.46:0/1612326131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:31.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.407+0000 7fd08aa00640 1 --2- 172.21.15.46:0/1612326131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea87d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.407+0000 7fd08b400640 1 -- 172.21.15.46:0/1612326131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 msgr2=0x562fea897c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:31.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.407+0000 7fd08b400640 1 --2- 172.21.15.46:0/1612326131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea897c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.407+0000 7fd08b400640 1 -- 172.21.15.46:0/1612326131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 msgr2=0x562fea87d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.407+0000 7fd08b400640 1 --2- 172.21.15.46:0/1612326131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea87d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.407+0000 7fd08b400640 1 -- 172.21.15.46:0/1612326131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fea871db0 con 0x562fea8754b0 2024-09-10T07:41:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.407+0000 7fd08b400640 1 --2- 172.21.15.46:0/1612326131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea897750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0840103a0 tx=0x7fd084009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe5e5de551911ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.407+0000 7fd077e00640 1 -- 172.21.15.46:0/1612326131 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084016040 con 0x562fea8754b0 2024-09-10T07:41:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.408+0000 7fd077e00640 1 -- 172.21.15.46:0/1612326131 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd084003970 con 0x562fea8754b0 2024-09-10T07:41:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.408+0000 7fd077e00640 1 -- 172.21.15.46:0/1612326131 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084012a50 con 0x562fea8754b0 2024-09-10T07:41:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 -- 172.21.15.46:0/1612326131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 msgr2=0x562fea897750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 --2- 172.21.15.46:0/1612326131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea897750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0840103a0 tx=0x7fd084009920 comp rx=0 tx=0).stop 2024-09-10T07:41:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 -- 172.21.15.46:0/1612326131 shutdown_connections 2024-09-10T07:41:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 --2- 172.21.15.46:0/1612326131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea87fd70 0x562fea897c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 --2- 172.21.15.46:0/1612326131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea8754b0 0x562fea897750 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 --2- 172.21.15.46:0/1612326131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea874b90 0x562fea87d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 -- 172.21.15.46:0/1612326131 >> 172.21.15.46:0/1612326131 conn(0x562fea871120 msgr2=0x562fea87b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 -- 172.21.15.46:0/1612326131 shutdown_connections 2024-09-10T07:41:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.409+0000 7fd094ff6740 1 -- 172.21.15.46:0/1612326131 wait complete. 2024-09-10T07:41:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.410+0000 7fd094ff6740 1 -- start start 2024-09-10T07:41:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.410+0000 7fd094ff6740 1 Processor -- start 2024-09-10T07:41:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.411+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea874b90 0x562fea879e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.412+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea8754b0 0x562fea87a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.412+0000 7fd094ff6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea87fd70 0x562fea87a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.412+0000 7fd094ff6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b67a0 con 0x562fea87fd70 2024-09-10T07:41:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.412+0000 7fd094ff6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b60d0 con 0x562fea874b90 2024-09-10T07:41:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.412+0000 7fd094ff6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b32b0 con 0x562fea8754b0 2024-09-10T07:41:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.412+0000 7fd094ff6740 10 client.0 _ll_register_callbacks cb 0x7fd07c031870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.412+0000 7fd08aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea874b90 0x562fea879e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.413+0000 7fd08aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea874b90 0x562fea879e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34156/0 (socket says 172.21.15.46:34156) 2024-09-10T07:41:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.413+0000 7fd08aa00640 1 -- 172.21.15.46:0/3024707314 learned_addr learned my addr 172.21.15.46:0/3024707314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.413+0000 7fd08b400640 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea8754b0 0x562fea87a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.414+0000 7fd08aa00640 1 -- 172.21.15.46:0/3024707314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea8754b0 msgr2=0x562fea87a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.414+0000 7fd08aa00640 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea8754b0 0x562fea87a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.414+0000 7fd08aa00640 1 -- 172.21.15.46:0/3024707314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea87fd70 msgr2=0x562fea87a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.414+0000 7fd08aa00640 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea87fd70 0x562fea87a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.414+0000 7fd08aa00640 1 -- 172.21.15.46:0/3024707314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fea8f9850 con 0x562fea874b90 2024-09-10T07:41:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.415+0000 7fd08aa00640 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea874b90 0x562fea879e30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd07c005a80 tx=0x7fd07c03c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeac196b0ec3da4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.415+0000 7fd075600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.415+0000 7fd075600640 1 -- 172.21.15.46:0/3024707314 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c035070 con 0x562fea874b90 2024-09-10T07:41:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.416+0000 7fd075600640 1 -- 172.21.15.46:0/3024707314 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd07c03cc90 con 0x562fea874b90 2024-09-10T07:41:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.416+0000 7fd094ff6740 1 -- 172.21.15.46:0/3024707314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd084003970 con 0x562fea874b90 2024-09-10T07:41:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.416+0000 7fd075600640 1 -- 172.21.15.46:0/3024707314 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd07c05a050 con 0x562fea874b90 2024-09-10T07:41:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.416+0000 7fd094ff6740 10 client.26672 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.416+0000 7fd094ff6740 1 -- 172.21.15.46:0/3024707314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562fea8faae0 con 0x562fea874b90 2024-09-10T07:41:31.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.417+0000 7fd075600640 1 -- 172.21.15.46:0/3024707314 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd084003970 con 0x562fea874b90 2024-09-10T07:41:31.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.418+0000 7fd075600640 1 -- 172.21.15.46:0/3024707314 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd07c005490 con 0x562fea874b90 2024-09-10T07:41:31.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.420+0000 7fd094ff6740 10 client.26672 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:31.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.420+0000 7fd094ff6740 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:31.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.420+0000 7fd094ff6740 20 client.26672 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:31.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.420+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562fea8fb1b0 0x562fea8fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:31.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.420+0000 7fd094ff6740 4 client.26672 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:41:31.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.420+0000 7fd094ff6740 1 -- 172.21.15.46:0/3024707314 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562fea7e2e60 con 0x562fea8fb1b0 2024-09-10T07:41:31.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.421+0000 7fd08b400640 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562fea8fb1b0 0x562fea8fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:31.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.422+0000 7fd08b400640 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562fea8fb1b0 0x562fea8fd5d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c047ae5fc2f69bd3 server_cookie=8bc3820a72115171 in_seq=0 out_seq=0 2024-09-10T07:41:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.422+0000 7fd075600640 10 client.26672 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.422+0000 7fd075600640 1 -- 172.21.15.46:0/3024707314 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562fea7e2e60 con 0x562fea8fb1b0 2024-09-10T07:41:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.422+0000 7fd075600640 10 client.26672 handle_command_reply: tid=0 2024-09-10T07:41:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.422+0000 7fd094ff6740 1 client.26672 shutdown 2024-09-10T07:41:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd094ff6740 1 -- 172.21.15.46:0/3024707314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea874b90 msgr2=0x562fea879e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea874b90 0x562fea879e30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd07c005a80 tx=0x7fd07c03c6f0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd094ff6740 1 -- 172.21.15.46:0/3024707314 shutdown_connections 2024-09-10T07:41:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562fea8fb1b0 0x562fea8fd5d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562fea87fd70 0x562fea87a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562fea8754b0 0x562fea87a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd094ff6740 1 --2- 172.21.15.46:0/3024707314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fea874b90 0x562fea879e30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd094ff6740 1 -- 172.21.15.46:0/3024707314 >> 172.21.15.46:0/3024707314 conn(0x562fea871120 msgr2=0x562fea86f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.423+0000 7fd075600640 0 client.26672 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.424+0000 7fd094ff6740 1 -- 172.21.15.46:0/3024707314 shutdown_connections 2024-09-10T07:41:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.424+0000 7fd094ff6740 1 -- 172.21.15.46:0/3024707314 wait complete. 2024-09-10T07:41:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.424+0000 7fd094ff6740 20 client.26672 trim_cache size 0 max 16384 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "6a3d67ec-1bd8-497f-a2d5-b52de31dc00d": { 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6a3d67ec-1bd8-497f-a2d5-b52de31dc00d", 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:41:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:41:31.426 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:41:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 -- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd0700a2cd0 msgr2=0x7fd0700a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd0700a2cd0 0x7fd0700a5190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd07809b220 tx=0x7fd078007f50 comp rx=0 tx=0).stop 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 -- 172.21.15.46:0/3759722806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd08c10e7b0 msgr2=0x7fd08c19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd08c10e7b0 0x7fd08c19e7e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd07800c740 tx=0x7fd07800cc00 comp rx=0 tx=0).stop 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 -- 172.21.15.46:0/3759722806 shutdown_connections 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd08c10e7b0 0x7fd08c19e7e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd0700a2cd0 0x7fd0700a5190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd08c109c40 0x7fd08c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 --2- 172.21.15.46:0/3759722806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd08c109290 0x7fd08c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.425+0000 7fd092600640 1 -- 172.21.15.46:0/3759722806 >> 172.21.15.46:0/3759722806 conn(0x7fd08c077c10 msgr2=0x7fd08c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.426+0000 7fd092600640 1 -- 172.21.15.46:0/3759722806 shutdown_connections 2024-09-10T07:41:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:41:31.426+0000 7fd092600640 1 -- 172.21.15.46:0/3759722806 wait complete. 2024-09-10T07:41:32.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:32 smithi046 ceph-mon[30705]: pgmap v546: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.6 MiB/s wr, 360 op/s 2024-09-10T07:41:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:32 smithi181 ceph-mon[35397]: pgmap v546: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.6 MiB/s wr, 360 op/s 2024-09-10T07:41:32.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:32 smithi063 ceph-mon[35020]: pgmap v546: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.6 MiB/s wr, 360 op/s 2024-09-10T07:41:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.418+0000 7f8c5ca00640 1 -- 172.21.15.46:0/3622781047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c58109d40 msgr2=0x7f8c581105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.418+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/3622781047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c58109d40 0x7f8c581105d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009900 tx=0x7f8c4802a400 comp rx=0 tx=0).stop 2024-09-10T07:41:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.419+0000 7f8c5ca00640 1 -- 172.21.15.46:0/3622781047 shutdown_connections 2024-09-10T07:41:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.419+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/3622781047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c58109d40 0x7f8c581105d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.419+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/3622781047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c58109380 0x7f8c58109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.419+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/3622781047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c581084c0 0x7f8c581088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.419+0000 7f8c5ca00640 1 -- 172.21.15.46:0/3622781047 >> 172.21.15.46:0/3622781047 conn(0x7f8c58077490 msgr2=0x7f8c580798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.419+0000 7f8c5ca00640 1 -- 172.21.15.46:0/3622781047 shutdown_connections 2024-09-10T07:41:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.419+0000 7f8c5ca00640 1 -- 172.21.15.46:0/3622781047 wait complete. 2024-09-10T07:41:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.421+0000 7f8c5ca00640 1 Processor -- start 2024-09-10T07:41:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.421+0000 7f8c5ca00640 1 -- start start 2024-09-10T07:41:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.421+0000 7f8c5ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c581084c0 0x7f8c5819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.421+0000 7f8c5ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c58109380 0x7f8c5819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.421+0000 7f8c5ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c58109d40 0x7f8c581a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.421+0000 7f8c5ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58113ec0 con 0x7f8c58109d40 2024-09-10T07:41:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.421+0000 7f8c5ca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58113d40 con 0x7f8c581084c0 2024-09-10T07:41:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.421+0000 7f8c5ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58114040 con 0x7f8c58109380 2024-09-10T07:41:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c581084c0 0x7f8c5819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c581084c0 0x7f8c5819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34178/0 (socket says 172.21.15.46:34178) 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 -- 172.21.15.46:0/511259005 learned_addr learned my addr 172.21.15.46:0/511259005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55400640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c58109380 0x7f8c5819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c56800640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c58109d40 0x7f8c581a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 -- 172.21.15.46:0/511259005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c58109380 msgr2=0x7f8c5819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c58109380 0x7f8c5819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 -- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c58109d40 msgr2=0x7f8c581a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c58109d40 0x7f8c581a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 -- 172.21.15.46:0/511259005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c581a6010 con 0x7f8c581084c0 2024-09-10T07:41:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.422+0000 7f8c55e00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c581084c0 0x7f8c5819e3f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000b450 tx=0x7f8c4000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.423+0000 7f8c4ea00640 1 -- 172.21.15.46:0/511259005 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40015020 con 0x7f8c581084c0 2024-09-10T07:41:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.423+0000 7f8c4ea00640 1 -- 172.21.15.46:0/511259005 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c40011d70 con 0x7f8c581084c0 2024-09-10T07:41:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.423+0000 7f8c4ea00640 1 -- 172.21.15.46:0/511259005 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000f340 con 0x7f8c581084c0 2024-09-10T07:41:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.423+0000 7f8c5ca00640 1 -- 172.21.15.46:0/511259005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c581a6300 con 0x7f8c581084c0 2024-09-10T07:41:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.423+0000 7f8c5ca00640 1 -- 172.21.15.46:0/511259005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c581a6880 con 0x7f8c581084c0 2024-09-10T07:41:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.428+0000 7f8c5f81e740 1 Processor -- start 2024-09-10T07:41:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.429+0000 7f8c5f81e740 1 -- start start 2024-09-10T07:41:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.430+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef124490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.430+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef129490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.430+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 0x55c4ef12bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.430+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef089100 con 0x55c4ef129a60 2024-09-10T07:41:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.430+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef0311b0 con 0x55c4ef124090 2024-09-10T07:41:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.430+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef0850c0 con 0x55c4ef1249d0 2024-09-10T07:41:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.431+0000 7f8c56800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef124490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.431+0000 7f8c56800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef124490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34186/0 (socket says 172.21.15.46:34186) 2024-09-10T07:41:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.431+0000 7f8c56800640 1 -- 172.21.15.46:0/2387945665 learned_addr learned my addr 172.21.15.46:0/2387945665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.431+0000 7f8c56800640 1 -- 172.21.15.46:0/2387945665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 msgr2=0x55c4ef129490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:41:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.431+0000 7f8c56800640 1 --2- 172.21.15.46:0/2387945665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef129490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.432+0000 7f8c56800640 1 -- 172.21.15.46:0/2387945665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 msgr2=0x55c4ef12bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.432+0000 7f8c56800640 1 --2- 172.21.15.46:0/2387945665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 0x55c4ef12bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.432+0000 7f8c56800640 1 -- 172.21.15.46:0/2387945665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4eed9c210 con 0x55c4ef124090 2024-09-10T07:41:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.433+0000 7f8c4ea00640 1 -- 172.21.15.46:0/511259005 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c40018020 con 0x7f8c581084c0 2024-09-10T07:41:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.433+0000 7f8c56800640 1 --2- 172.21.15.46:0/2387945665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef124490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8c48033000 tx=0x7f8c480040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b646b92be135d2e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.435+0000 7f8c47000640 1 -- 172.21.15.46:0/2387945665 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48035030 con 0x55c4ef124090 2024-09-10T07:41:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.435+0000 7f8c47000640 1 -- 172.21.15.46:0/2387945665 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48005520 con 0x55c4ef124090 2024-09-10T07:41:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.435+0000 7f8c47000640 1 -- 172.21.15.46:0/2387945665 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480318a0 con 0x55c4ef124090 2024-09-10T07:41:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.436+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2387945665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 msgr2=0x55c4ef124490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.436+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2387945665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef124490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8c48033000 tx=0x7f8c480040d0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.436+0000 7f8c4ea00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8c3c0a2cb0 0x7f8c3c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c4ea00640 1 -- 172.21.15.46:0/511259005 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8c400cef50 con 0x7f8c581084c0 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2387945665 shutdown_connections 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2387945665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 0x55c4ef12bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2387945665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef129490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2387945665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef124490 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2387945665 >> 172.21.15.46:0/2387945665 conn(0x55c4ef11f700 msgr2=0x55c4ef11fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c56800640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8c3c0a2cb0 0x7f8c3c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2387945665 shutdown_connections 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.437+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2387945665 wait complete. 2024-09-10T07:41:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.438+0000 7f8c5f81e740 1 -- start start 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.438+0000 7f8c5f81e740 1 Processor -- start 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.438+0000 7f8c56800640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8c3c0a2cb0 0x7f8c3c0a5170 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8c48031470 tx=0x7f8c4802a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef13f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef13f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef129a60 0x55c4ef146800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef13d9e0 con 0x55c4ef124090 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef012090 con 0x55c4ef129a60 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef13db60 con 0x55c4ef1249d0 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c5f81e740 10 client.0 _ll_register_callbacks cb 0x55c4ef066f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c55400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef13f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c55400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef13f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:58638/0 (socket says 172.21.15.46:58638) 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c55400640 1 -- 172.21.15.46:0/2834124180 learned_addr learned my addr 172.21.15.46:0/2834124180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c55e00640 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef13f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.439+0000 7f8c56800640 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef129a60 0x55c4ef146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.440+0000 7f8c55400640 1 -- 172.21.15.46:0/2834124180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef129a60 msgr2=0x55c4ef146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.440+0000 7f8c55400640 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef129a60 0x55c4ef146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.440+0000 7f8c55400640 1 -- 172.21.15.46:0/2834124180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 msgr2=0x55c4ef13f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.440+0000 7f8c55400640 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef13f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.440+0000 7f8c55400640 1 -- 172.21.15.46:0/2834124180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4eee68620 con 0x55c4ef1249d0 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.440+0000 7f8c55400640 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef13f7c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8c50010820 tx=0x7f8c50010c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87bff8ccf9d1290c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.441+0000 7f8c56800640 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef129a60 0x55c4ef146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.441+0000 7f8c3b400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.441+0000 7f8c3b400640 1 -- 172.21.15.46:0/2834124180 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50017070 con 0x55c4ef1249d0 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.441+0000 7f8c3b400640 1 -- 172.21.15.46:0/2834124180 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8c5000ecc0 con 0x55c4ef1249d0 2024-09-10T07:41:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.441+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2834124180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c48005520 con 0x55c4ef1249d0 2024-09-10T07:41:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.441+0000 7f8c3b400640 1 -- 172.21.15.46:0/2834124180 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c50010e70 con 0x55c4ef1249d0 2024-09-10T07:41:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.441+0000 7f8c5f81e740 10 client.26653 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.441+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2834124180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c4ef149710 con 0x55c4ef1249d0 2024-09-10T07:41:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.442+0000 7f8c3b400640 1 -- 172.21.15.46:0/2834124180 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48005520 con 0x55c4ef1249d0 2024-09-10T07:41:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.442+0000 7f8c3b400640 1 -- 172.21.15.46:0/2834124180 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f8c500046d0 con 0x55c4ef1249d0 2024-09-10T07:41:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.442+0000 7f8c5f81e740 10 client.26653 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.442+0000 7f8c5f81e740 10 client.26653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.442+0000 7f8c5f81e740 20 client.26653 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.442+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c4ef14ade0 0x55c4ef14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.442+0000 7f8c5f81e740 4 client.26653 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.442+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2834124180 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4ef096830 con 0x55c4ef14ade0 2024-09-10T07:41:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.443+0000 7f8c55e00640 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c4ef14ade0 0x55c4ef14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.443+0000 7f8c55e00640 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c4ef14ade0 0x55c4ef14d1d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dece68d907551068 server_cookie=35b7d5a799793351 in_seq=0 out_seq=0 2024-09-10T07:41:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.443+0000 7f8c3b400640 10 client.26653 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.444+0000 7f8c3b400640 1 -- 172.21.15.46:0/2834124180 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4ef096830 con 0x55c4ef14ade0 2024-09-10T07:41:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.444+0000 7f8c3b400640 10 client.26653 handle_command_reply: tid=0 2024-09-10T07:41:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.444+0000 7f8c5f81e740 1 client.26653 shutdown 2024-09-10T07:41:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2834124180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 msgr2=0x55c4ef13f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef13f7c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8c50010820 tx=0x7f8c50010c20 comp rx=0 tx=0).stop 2024-09-10T07:41:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2834124180 shutdown_connections 2024-09-10T07:41:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c4ef14ade0 0x55c4ef14d1d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c3b400640 0 client.26653 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef129a60 0x55c4ef146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef13f7c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2834124180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef13f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2834124180 >> 172.21.15.46:0/2834124180 conn(0x55c4ef11f700 msgr2=0x55c4ef120f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2834124180 shutdown_connections 2024-09-10T07:41:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.445+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2834124180 wait complete. 2024-09-10T07:41:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.446+0000 7f8c5f81e740 20 client.26653 trim_cache size 0 max 16384 2024-09-10T07:41:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.471+0000 7f8c5f81e740 1 Processor -- start 2024-09-10T07:41:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.471+0000 7f8c5f81e740 1 -- start start 2024-09-10T07:41:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef11e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef1467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 0x55c4ef120af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581a8690 con 0x55c4ef129a60 2024-09-10T07:41:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581aa230 con 0x55c4ef124090 2024-09-10T07:41:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581a8d60 con 0x55c4ef1249d0 2024-09-10T07:41:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c55e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef11e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c55e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef11e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34198/0 (socket says 172.21.15.46:34198) 2024-09-10T07:41:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c55400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef1467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.472+0000 7f8c55e00640 1 -- 172.21.15.46:0/2463621281 learned_addr learned my addr 172.21.15.46:0/2463621281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.473+0000 7f8c55400640 1 -- 172.21.15.46:0/2463621281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 msgr2=0x55c4ef11e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.473+0000 7f8c55400640 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef11e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.473+0000 7f8c55400640 1 -- 172.21.15.46:0/2463621281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 msgr2=0x55c4ef120af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:41:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.473+0000 7f8c56800640 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 0x55c4ef120af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.473+0000 7f8c55400640 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 0x55c4ef120af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.473+0000 7f8c55400640 1 -- 172.21.15.46:0/2463621281 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ef11ad60 con 0x55c4ef1249d0 2024-09-10T07:41:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.473+0000 7f8c55e00640 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef11e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.473+0000 7f8c56800640 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 0x55c4ef120af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:41:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.474+0000 7f8c55400640 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef1467c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8c50006010 tx=0x7f8c50011a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e296968e8f3944 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.474+0000 7f8c46600640 1 -- 172.21.15.46:0/2463621281 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50017040 con 0x55c4ef1249d0 2024-09-10T07:41:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.474+0000 7f8c46600640 1 -- 172.21.15.46:0/2463621281 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c50011d60 con 0x55c4ef1249d0 2024-09-10T07:41:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.474+0000 7f8c46600640 1 -- 172.21.15.46:0/2463621281 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50005310 con 0x55c4ef1249d0 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.475+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2463621281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 msgr2=0x55c4ef1467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.475+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef1467c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8c50006010 tx=0x7f8c50011a90 comp rx=0 tx=0).stop 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.475+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2463621281 shutdown_connections 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.475+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef129a60 0x55c4ef120af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.475+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef1249d0 0x55c4ef1467c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.475+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/2463621281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef124090 0x55c4ef11e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.475+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2463621281 >> 172.21.15.46:0/2463621281 conn(0x55c4ef11f700 msgr2=0x55c4ef11e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.475+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2463621281 shutdown_connections 2024-09-10T07:41:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.476+0000 7f8c5f81e740 1 -- 172.21.15.46:0/2463621281 wait complete. 2024-09-10T07:41:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.476+0000 7f8c5f81e740 1 -- start start 2024-09-10T07:41:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.476+0000 7f8c5f81e740 1 Processor -- start 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef144190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef1249d0 0x55c4ef1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c5f81e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef129a60 0x55c4ef1456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581a9b60 con 0x55c4ef124090 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581a9490 con 0x55c4ef1249d0 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c5f81e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58082d20 con 0x55c4ef129a60 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c5f81e740 10 client.0 _ll_register_callbacks cb 0x7f8c500099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef144190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35124/0 (socket says 172.21.15.46:35124) 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c55e00640 1 -- 172.21.15.46:0/3285069140 learned_addr learned my addr 172.21.15.46:0/3285069140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c55400640 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef1249d0 0x55c4ef1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.477+0000 7f8c56800640 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef129a60 0x55c4ef1456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.478+0000 7f8c55e00640 1 -- 172.21.15.46:0/3285069140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef129a60 msgr2=0x55c4ef1456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.478+0000 7f8c55e00640 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef129a60 0x55c4ef1456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.478+0000 7f8c55e00640 1 -- 172.21.15.46:0/3285069140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef1249d0 msgr2=0x55c4ef1446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.478+0000 7f8c55e00640 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef1249d0 0x55c4ef1446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.478+0000 7f8c55e00640 1 -- 172.21.15.46:0/3285069140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ef145cf0 con 0x55c4ef124090 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.478+0000 7f8c55e00640 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef144190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8c4009b0d0 tx=0x7f8c400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e3b9534fac6884 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.478+0000 7f8c3b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.479+0000 7f8c3b400640 1 -- 172.21.15.46:0/3285069140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4009b3d0 con 0x55c4ef124090 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.479+0000 7f8c3b400640 1 -- 172.21.15.46:0/3285069140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8c400996d0 con 0x55c4ef124090 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.479+0000 7f8c5f81e740 1 -- 172.21.15.46:0/3285069140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c50011d60 con 0x55c4ef124090 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.479+0000 7f8c3b400640 1 -- 172.21.15.46:0/3285069140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c40010600 con 0x55c4ef124090 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.479+0000 7f8c5f81e740 10 client.16362 fetch_fsmap learned FSMap version 39 2024-09-10T07:41:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.479+0000 7f8c5f81e740 1 -- 172.21.15.46:0/3285069140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c4ef1b5d90 con 0x55c4ef124090 2024-09-10T07:41:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c3b400640 1 -- 172.21.15.46:0/3285069140 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c50011d60 con 0x55c4ef124090 2024-09-10T07:41:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c3b400640 1 -- 172.21.15.46:0/3285069140 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f8c400ccc30 con 0x55c4ef124090 2024-09-10T07:41:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c5f81e740 10 client.16362 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:41:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c5f81e740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c5f81e740 20 client.16362 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c4ef1b6670 0x55c4ef1b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c5f81e740 4 client.16362 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:41:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c5f81e740 1 -- 172.21.15.46:0/3285069140 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4ef129820 con 0x55c4ef1b6670 2024-09-10T07:41:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.480+0000 7f8c55400640 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c4ef1b6670 0x55c4ef1b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.481+0000 7f8c55400640 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c4ef1b6670 0x55c4ef1b8a90 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffb787ecd5c30f2 server_cookie=a06bc76b86365951 in_seq=0 out_seq=0 2024-09-10T07:41:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.481+0000 7f8c3b400640 10 client.16362 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.498+0000 7f8c3b400640 1 -- 172.21.15.46:0/3285069140 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4ef129820 con 0x55c4ef1b6670 2024-09-10T07:41:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.498+0000 7f8c3b400640 10 client.16362 handle_command_reply: tid=0 2024-09-10T07:41:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.498+0000 7f8c5f81e740 1 client.16362 shutdown 2024-09-10T07:41:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.498+0000 7f8c5f81e740 1 -- 172.21.15.46:0/3285069140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 msgr2=0x55c4ef144190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.498+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef144190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8c4009b0d0 tx=0x7f8c400046c0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 1 -- 172.21.15.46:0/3285069140 shutdown_connections 2024-09-10T07:41:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c4ef1b6670 0x55c4ef1b8a90 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4ef129a60 0x55c4ef1456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c4ef1249d0 0x55c4ef1446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 1 --2- 172.21.15.46:0/3285069140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4ef124090 0x55c4ef144190 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 1 -- 172.21.15.46:0/3285069140 >> 172.21.15.46:0/3285069140 conn(0x55c4ef11f700 msgr2=0x55c4ef11e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c3b400640 0 client.16362 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 1 -- 172.21.15.46:0/3285069140 shutdown_connections 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 1 -- 172.21.15.46:0/3285069140 wait complete. 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.499+0000 7f8c5f81e740 20 client.16362 trim_cache size 0 max 16384 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:41:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:41:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 -- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8c3c0a2cb0 msgr2=0x7f8c3c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8c3c0a2cb0 0x7f8c3c0a5170 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8c48031470 tx=0x7f8c4802a400 comp rx=0 tx=0).stop 2024-09-10T07:41:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 -- 172.21.15.46:0/511259005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c581084c0 msgr2=0x7f8c5819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c581084c0 0x7f8c5819e3f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000b450 tx=0x7f8c4000b910 comp rx=0 tx=0).stop 2024-09-10T07:41:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 -- 172.21.15.46:0/511259005 shutdown_connections 2024-09-10T07:41:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8c3c0a2cb0 0x7f8c3c0a5170 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c58109d40 0x7f8c581a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c58109380 0x7f8c5819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 --2- 172.21.15.46:0/511259005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c581084c0 0x7f8c5819e3f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.500+0000 7f8c5ca00640 1 -- 172.21.15.46:0/511259005 >> 172.21.15.46:0/511259005 conn(0x7f8c58077490 msgr2=0x7f8c5810c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.501+0000 7f8c5ca00640 1 -- 172.21.15.46:0/511259005 shutdown_connections 2024-09-10T07:41:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:33.501+0000 7f8c5ca00640 1 -- 172.21.15.46:0/511259005 wait complete. 2024-09-10T07:41:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:34 smithi181 ceph-mon[35397]: pgmap v547: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 3.6 MiB/s wr, 505 op/s 2024-09-10T07:41:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:34 smithi063 ceph-mon[35020]: pgmap v547: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 3.6 MiB/s wr, 505 op/s 2024-09-10T07:41:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:34 smithi046 ceph-mon[30705]: pgmap v547: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 3.6 MiB/s wr, 505 op/s 2024-09-10T07:41:34.893 INFO:tasks.cephfs.filesystem:scrub status for tag:6a3d67ec-1bd8-497f-a2d5-b52de31dc00d - {'path': '/', 'tag': '6a3d67ec-1bd8-497f-a2d5-b52de31dc00d', 'options': 'recursive,force'} 2024-09-10T07:41:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 255.0 s, 775.4 tps, lat 128.928 ms stddev 191.845 2024-09-10T07:41:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:36 smithi181 ceph-mon[35397]: pgmap v548: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 562 op/s 2024-09-10T07:41:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:41:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:36 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:41:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:36 smithi063 ceph-mon[35020]: pgmap v548: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 562 op/s 2024-09-10T07:41:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:41:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:36 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:41:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:36 smithi046 ceph-mon[30705]: pgmap v548: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 562 op/s 2024-09-10T07:41:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:41:36.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:36 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:41:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:37 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:41:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:37 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:41:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:37 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:41:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:37 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:41:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:37 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:41:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:37 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:41:37.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:37 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:41:37.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:37 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:41:37.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:37 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:41:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:38 smithi181 ceph-mon[35397]: pgmap v549: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.1 MiB/s wr, 692 op/s 2024-09-10T07:41:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:38 smithi063 ceph-mon[35020]: pgmap v549: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.1 MiB/s wr, 692 op/s 2024-09-10T07:41:38.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:38 smithi046 ceph-mon[30705]: pgmap v549: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.1 MiB/s wr, 692 op/s 2024-09-10T07:41:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:40 smithi181 ceph-mon[35397]: pgmap v550: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 3.7 MiB/s wr, 543 op/s 2024-09-10T07:41:40.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:40 smithi063 ceph-mon[35020]: pgmap v550: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 3.7 MiB/s wr, 543 op/s 2024-09-10T07:41:40.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:40 smithi046 ceph-mon[30705]: pgmap v550: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 3.7 MiB/s wr, 543 op/s 2024-09-10T07:41:40.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 260.0 s, 1716.6 tps, lat 60.632 ms stddev 103.753 2024-09-10T07:41:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:42 smithi181 ceph-mon[35397]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.9 MiB/s wr, 707 op/s 2024-09-10T07:41:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:42 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:41:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:42 smithi063 ceph-mon[35020]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.9 MiB/s wr, 707 op/s 2024-09-10T07:41:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:42 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:41:42.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:42 smithi046 ceph-mon[30705]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.9 MiB/s wr, 707 op/s 2024-09-10T07:41:42.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:42 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:41:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:44 smithi181 ceph-mon[35397]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.5 MiB/s wr, 642 op/s 2024-09-10T07:41:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:44 smithi063 ceph-mon[35020]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.5 MiB/s wr, 642 op/s 2024-09-10T07:41:44.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:44 smithi046 ceph-mon[30705]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.5 MiB/s wr, 642 op/s 2024-09-10T07:41:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 265.0 s, 1968.6 tps, lat 50.613 ms stddev 59.753 2024-09-10T07:41:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:46 smithi181 ceph-mon[35397]: pgmap v553: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 522 op/s 2024-09-10T07:41:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:46 smithi063 ceph-mon[35020]: pgmap v553: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 522 op/s 2024-09-10T07:41:46.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:46 smithi046 ceph-mon[30705]: pgmap v553: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 522 op/s 2024-09-10T07:41:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.700+0000 7fdff0800640 1 -- 172.21.15.46:0/2597406911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8076bb0 msgr2=0x7fdfe8076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.700+0000 7fdff0800640 1 --2- 172.21.15.46:0/2597406911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8076bb0 0x7fdfe8076fb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009900 tx=0x7fdfdc02a410 comp rx=0 tx=0).stop 2024-09-10T07:41:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.700+0000 7fdff0800640 1 -- 172.21.15.46:0/2597406911 shutdown_connections 2024-09-10T07:41:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.700+0000 7fdff0800640 1 --2- 172.21.15.46:0/2597406911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe8078790 0x7fdfe8079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.700+0000 7fdff0800640 1 --2- 172.21.15.46:0/2597406911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe8077df0 0x7fdfe8078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.700+0000 7fdff0800640 1 --2- 172.21.15.46:0/2597406911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8076bb0 0x7fdfe8076fb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.700+0000 7fdff0800640 1 -- 172.21.15.46:0/2597406911 >> 172.21.15.46:0/2597406911 conn(0x7fdfe80ff4e0 msgr2=0x7fdfe8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.701+0000 7fdff0800640 1 -- 172.21.15.46:0/2597406911 shutdown_connections 2024-09-10T07:41:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.701+0000 7fdff0800640 1 -- 172.21.15.46:0/2597406911 wait complete. 2024-09-10T07:41:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.702+0000 7fdff0800640 1 Processor -- start 2024-09-10T07:41:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.702+0000 7fdff0800640 1 -- start start 2024-09-10T07:41:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdff0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe8076bb0 0x7fdfe819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdff0800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe8077df0 0x7fdfe819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdff0800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8078790 0x7fdfe81a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdff0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8113ea0 con 0x7fdfe8076bb0 2024-09-10T07:41:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdff0800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8113d20 con 0x7fdfe8077df0 2024-09-10T07:41:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdff0800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8114020 con 0x7fdfe8078790 2024-09-10T07:41:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdfee800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8078790 0x7fdfe81a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdfed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe8077df0 0x7fdfe819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdfee800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8078790 0x7fdfe81a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39078/0 (socket says 172.21.15.46:39078) 2024-09-10T07:41:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.703+0000 7fdfee800640 1 -- 172.21.15.46:0/2891694867 learned_addr learned my addr 172.21.15.46:0/2891694867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.704+0000 7fdfed400640 1 -- 172.21.15.46:0/2891694867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8078790 msgr2=0x7fdfe81a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.704+0000 7fdfed400640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8078790 0x7fdfe81a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.704+0000 7fdfed400640 1 -- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe8076bb0 msgr2=0x7fdfe819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.704+0000 7fdfed400640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe8076bb0 0x7fdfe819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.704+0000 7fdfed400640 1 -- 172.21.15.46:0/2891694867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81a6090 con 0x7fdfe8077df0 2024-09-10T07:41:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.704+0000 7fdfee800640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8078790 0x7fdfe81a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:41:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.705+0000 7fdfed400640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe8077df0 0x7fdfe819e9b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800e9f0 tx=0x7fdfd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.705+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2891694867 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800ccc0 con 0x7fdfe8077df0 2024-09-10T07:41:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.705+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2891694867 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd8010910 con 0x7fdfe8077df0 2024-09-10T07:41:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.705+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2891694867 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8018960 con 0x7fdfe8077df0 2024-09-10T07:41:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.705+0000 7fdff0800640 1 -- 172.21.15.46:0/2891694867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe81a6380 con 0x7fdfe8077df0 2024-09-10T07:41:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.706+0000 7fdff0800640 1 -- 172.21.15.46:0/2891694867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe81a6900 con 0x7fdfe8077df0 2024-09-10T07:41:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.710+0000 7fdff36c5740 1 Processor -- start 2024-09-10T07:41:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.711+0000 7fdff36c5740 1 -- start start 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.711+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d824490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d8249d0 0x561f1d829490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 0x561f1d82bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdff36c5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f1d789100 con 0x561f1d824090 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdff36c5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f1d7311b0 con 0x561f1d8249d0 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdff36c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561f1d7850c0 con 0x561f1d829a60 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdfee800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d824490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2891694867 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfd8010ac0 con 0x7fdfe8077df0 2024-09-10T07:41:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdfee800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d824490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35458/0 (socket says 172.21.15.46:35458) 2024-09-10T07:41:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdfee800640 1 -- 172.21.15.46:0/103110596 learned_addr learned my addr 172.21.15.46:0/103110596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdfee800640 1 -- 172.21.15.46:0/103110596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 msgr2=0x561f1d82bf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:41:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdfee800640 1 --2- 172.21.15.46:0/103110596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 0x561f1d82bf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdfee800640 1 -- 172.21.15.46:0/103110596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d8249d0 msgr2=0x561f1d829490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:41:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.712+0000 7fdfee800640 1 --2- 172.21.15.46:0/103110596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d8249d0 0x561f1d829490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.713+0000 7fdfee800640 1 -- 172.21.15.46:0/103110596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f1d49c210 con 0x561f1d824090 2024-09-10T07:41:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.716+0000 7fdfe2a00640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdfd00a2cb0 0x7fdfd00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.717+0000 7fdfee800640 1 --2- 172.21.15.46:0/103110596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d824490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e5b0 tx=0x7fdfe4009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85872f34c63dd72f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.717+0000 7fdfede00640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdfd00a2cb0 0x7fdfd00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.717+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2891694867 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdfd80cf760 con 0x7fdfe8077df0 2024-09-10T07:41:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.718+0000 7fdfede00640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdfd00a2cb0 0x7fdfd00a5170 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0057c0 tx=0x7fdfdc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.719+0000 7fdfd7000640 1 -- 172.21.15.46:0/103110596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4028070 con 0x561f1d824090 2024-09-10T07:41:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.719+0000 7fdfd7000640 1 -- 172.21.15.46:0/103110596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe4024ce0 con 0x561f1d824090 2024-09-10T07:41:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.719+0000 7fdfd7000640 1 -- 172.21.15.46:0/103110596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4007da0 con 0x561f1d824090 2024-09-10T07:41:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.720+0000 7fdff36c5740 1 -- 172.21.15.46:0/103110596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 msgr2=0x561f1d824490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.720+0000 7fdff36c5740 1 --2- 172.21.15.46:0/103110596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d824490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e5b0 tx=0x7fdfe4009710 comp rx=0 tx=0).stop 2024-09-10T07:41:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.721+0000 7fdff36c5740 1 -- 172.21.15.46:0/103110596 shutdown_connections 2024-09-10T07:41:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.721+0000 7fdff36c5740 1 --2- 172.21.15.46:0/103110596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 0x561f1d82bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.721+0000 7fdff36c5740 1 --2- 172.21.15.46:0/103110596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d8249d0 0x561f1d829490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.721+0000 7fdff36c5740 1 --2- 172.21.15.46:0/103110596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d824490 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.721+0000 7fdff36c5740 1 -- 172.21.15.46:0/103110596 >> 172.21.15.46:0/103110596 conn(0x561f1d81f700 msgr2=0x561f1d81fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.722+0000 7fdff36c5740 1 -- 172.21.15.46:0/103110596 shutdown_connections 2024-09-10T07:41:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.722+0000 7fdff36c5740 1 -- 172.21.15.46:0/103110596 wait complete. 2024-09-10T07:41:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.723+0000 7fdff36c5740 1 -- start start 2024-09-10T07:41:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.723+0000 7fdff36c5740 1 Processor -- start 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.724+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d82e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.724+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d82e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.724+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d8357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.725+0000 7fdff36c5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f1d82ca30 con 0x561f1d824090 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.725+0000 7fdff36c5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f1d712090 con 0x561f1d829a60 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.725+0000 7fdff36c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561f1d82cbb0 con 0x561f1d8249d0 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.725+0000 7fdff36c5740 10 client.0 _ll_register_callbacks cb 0x561f1d766f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfed400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d82e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d8357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d8357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:54584/0 (socket says 172.21.15.46:54584) 2024-09-10T07:41:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfed400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d82e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39090/0 (socket says 172.21.15.46:39090) 2024-09-10T07:41:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfee800640 1 -- 172.21.15.46:0/3390167961 learned_addr learned my addr 172.21.15.46:0/3390167961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfede00640 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d82e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfee800640 1 -- 172.21.15.46:0/3390167961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 msgr2=0x561f1d82e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfee800640 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d82e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfee800640 1 -- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 msgr2=0x561f1d82e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfee800640 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d82e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.726+0000 7fdfee800640 1 -- 172.21.15.46:0/3390167961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f1d568620 con 0x561f1d829a60 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.727+0000 7fdfede00640 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d82e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.727+0000 7fdfee800640 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d8357d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400f010 tx=0x7fdfe4007e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885f4ca12fc4e0f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.727+0000 7fdfcb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.727+0000 7fdfcb400640 1 -- 172.21.15.46:0/3390167961 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe403c060 con 0x561f1d829a60 2024-09-10T07:41:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.727+0000 7fdfcb400640 1 -- 172.21.15.46:0/3390167961 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdfe4024800 con 0x561f1d829a60 2024-09-10T07:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.728+0000 7fdff36c5740 1 -- 172.21.15.46:0/3390167961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe4024ce0 con 0x561f1d829a60 2024-09-10T07:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.728+0000 7fdfcb400640 1 -- 172.21.15.46:0/3390167961 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdfe4028040 con 0x561f1d829a60 2024-09-10T07:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.728+0000 7fdff36c5740 10 client.26696 fetch_fsmap learned FSMap version 41 2024-09-10T07:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.728+0000 7fdff36c5740 1 -- 172.21.15.46:0/3390167961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561f1d8385c0 con 0x561f1d829a60 2024-09-10T07:41:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.729+0000 7fdfcb400640 1 -- 172.21.15.46:0/3390167961 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe4024ce0 con 0x561f1d829a60 2024-09-10T07:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.729+0000 7fdfcb400640 1 -- 172.21.15.46:0/3390167961 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fdfe40403a0 con 0x561f1d829a60 2024-09-10T07:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.730+0000 7fdff36c5740 10 client.26696 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T07:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.730+0000 7fdff36c5740 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.730+0000 7fdff36c5740 20 client.26696 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.730+0000 7fdff36c5740 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561f1d839a20 0x561f1d83be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.730+0000 7fdff36c5740 4 client.26696 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.730+0000 7fdff36c5740 1 -- 172.21.15.46:0/3390167961 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f1d796830 con 0x561f1d839a20 2024-09-10T07:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.730+0000 7fdfede00640 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561f1d839a20 0x561f1d83be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.734+0000 7fdfede00640 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561f1d839a20 0x561f1d83be10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e6faac250e8cd0 server_cookie=e21b0c4b331fa8be in_seq=0 out_seq=0 2024-09-10T07:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.734+0000 7fdfcb400640 10 client.26696 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.735+0000 7fdfcb400640 1 -- 172.21.15.46:0/3390167961 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f1d796830 con 0x561f1d839a20 2024-09-10T07:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.735+0000 7fdfcb400640 10 client.26696 handle_command_reply: tid=0 2024-09-10T07:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 client.26696 shutdown 2024-09-10T07:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 -- 172.21.15.46:0/3390167961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 msgr2=0x561f1d8357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d8357d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400f010 tx=0x7fdfe4007e20 comp rx=0 tx=0).stop 2024-09-10T07:41:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 -- 172.21.15.46:0/3390167961 shutdown_connections 2024-09-10T07:41:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561f1d839a20 0x561f1d83be10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d8357d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d82e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 --2- 172.21.15.46:0/3390167961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d82e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.736+0000 7fdff36c5740 1 -- 172.21.15.46:0/3390167961 >> 172.21.15.46:0/3390167961 conn(0x561f1d81f700 msgr2=0x561f1d820f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.737+0000 7fdfcb400640 0 client.26696 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.737+0000 7fdff36c5740 1 -- 172.21.15.46:0/3390167961 shutdown_connections 2024-09-10T07:41:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.737+0000 7fdff36c5740 1 -- 172.21.15.46:0/3390167961 wait complete. 2024-09-10T07:41:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.738+0000 7fdff36c5740 20 client.26696 trim_cache size 0 max 16384 2024-09-10T07:41:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.769+0000 7fdff36c5740 1 Processor -- start 2024-09-10T07:41:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.769+0000 7fdff36c5740 1 -- start start 2024-09-10T07:41:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d81fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d835790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d820020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdff36c5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a8770 con 0x561f1d824090 2024-09-10T07:41:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdff36c5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81aa370 con 0x561f1d829a60 2024-09-10T07:41:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdff36c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a8e40 con 0x561f1d8249d0 2024-09-10T07:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdfee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d820020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdfed400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d835790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdfed400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d835790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39106/0 (socket says 172.21.15.46:39106) 2024-09-10T07:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdfed400640 1 -- 172.21.15.46:0/4054898672 learned_addr learned my addr 172.21.15.46:0/4054898672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.770+0000 7fdfede00640 1 --2- 172.21.15.46:0/4054898672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d81fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.771+0000 7fdfed400640 1 -- 172.21.15.46:0/4054898672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 msgr2=0x561f1d820020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.771+0000 7fdfed400640 1 --2- 172.21.15.46:0/4054898672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d820020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.771+0000 7fdfed400640 1 -- 172.21.15.46:0/4054898672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 msgr2=0x561f1d81fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.771+0000 7fdfed400640 1 --2- 172.21.15.46:0/4054898672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d81fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.771+0000 7fdfed400640 1 -- 172.21.15.46:0/4054898672 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f1d83c440 con 0x561f1d8249d0 2024-09-10T07:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.772+0000 7fdfed400640 1 --2- 172.21.15.46:0/4054898672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d835790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80980a0 tx=0x7fdfd809c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d26d0c3ca1a59f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.773+0000 7fdfd6600640 1 -- 172.21.15.46:0/4054898672 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80a48b0 con 0x561f1d8249d0 2024-09-10T07:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.773+0000 7fdfd6600640 1 -- 172.21.15.46:0/4054898672 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd8099ce0 con 0x561f1d8249d0 2024-09-10T07:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.773+0000 7fdfd6600640 1 -- 172.21.15.46:0/4054898672 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd809b090 con 0x561f1d8249d0 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 -- 172.21.15.46:0/4054898672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 msgr2=0x561f1d835790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 --2- 172.21.15.46:0/4054898672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d835790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80980a0 tx=0x7fdfd809c400 comp rx=0 tx=0).stop 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 -- 172.21.15.46:0/4054898672 shutdown_connections 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 --2- 172.21.15.46:0/4054898672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d829a60 0x561f1d820020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 --2- 172.21.15.46:0/4054898672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d8249d0 0x561f1d835790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 --2- 172.21.15.46:0/4054898672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d81fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 -- 172.21.15.46:0/4054898672 >> 172.21.15.46:0/4054898672 conn(0x561f1d81f700 msgr2=0x561f1d829230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 -- 172.21.15.46:0/4054898672 shutdown_connections 2024-09-10T07:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.774+0000 7fdff36c5740 1 -- 172.21.15.46:0/4054898672 wait complete. 2024-09-10T07:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.775+0000 7fdff36c5740 1 -- start start 2024-09-10T07:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.775+0000 7fdff36c5740 1 Processor -- start 2024-09-10T07:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.775+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d832db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 0x561f1d8332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdff36c5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d834300 0x561f1d834760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdff36c5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a9c40 con 0x561f1d824090 2024-09-10T07:41:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdff36c5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a9570 con 0x561f1d834300 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdff36c5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8074010 con 0x561f1d829a60 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdff36c5740 10 client.0 _ll_register_callbacks cb 0x7fdfe4007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d834300 0x561f1d834760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d832db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d834300 0x561f1d834760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:54606/0 (socket says 172.21.15.46:54606) 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfee800640 1 -- 172.21.15.46:0/409509089 learned_addr learned my addr 172.21.15.46:0/409509089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfed400640 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 0x561f1d8332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfed400640 1 -- 172.21.15.46:0/409509089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d834300 msgr2=0x561f1d834760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfed400640 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d834300 0x561f1d834760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfed400640 1 -- 172.21.15.46:0/409509089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 msgr2=0x561f1d832db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfed400640 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d832db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.776+0000 7fdfed400640 1 -- 172.21.15.46:0/409509089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f1d81ad60 con 0x561f1d829a60 2024-09-10T07:41:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.777+0000 7fdfed400640 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 0x561f1d8332f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80a4820 tx=0x7fdfd8099e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=515c9b3fe837fa8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.777+0000 7fdfcb400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.777+0000 7fdfcb400640 1 -- 172.21.15.46:0/409509089 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80b8070 con 0x561f1d829a60 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.777+0000 7fdfcb400640 1 -- 172.21.15.46:0/409509089 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdfd80982b0 con 0x561f1d829a60 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.777+0000 7fdff36c5740 1 -- 172.21.15.46:0/409509089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd8099ce0 con 0x561f1d829a60 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.777+0000 7fdfcb400640 1 -- 172.21.15.46:0/409509089 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdfd80a48b0 con 0x561f1d829a60 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.777+0000 7fdff36c5740 10 client.26674 fetch_fsmap learned FSMap version 41 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.777+0000 7fdff36c5740 1 -- 172.21.15.46:0/409509089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561f1d8a4830 con 0x561f1d829a60 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.778+0000 7fdfcb400640 1 -- 172.21.15.46:0/409509089 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd8099ce0 con 0x561f1d829a60 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.778+0000 7fdfcb400640 1 -- 172.21.15.46:0/409509089 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fdfd8097990 con 0x561f1d829a60 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.778+0000 7fdff36c5740 10 client.26674 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.778+0000 7fdff36c5740 10 client.26674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:41:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.778+0000 7fdff36c5740 20 client.26674 populate_metadata read hostname 'smithi046' 2024-09-10T07:41:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.778+0000 7fdff36c5740 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561f1d8a5160 0x561f1d8a7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.778+0000 7fdff36c5740 4 client.26674 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:41:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.778+0000 7fdff36c5740 1 -- 172.21.15.46:0/409509089 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f1d790b80 con 0x561f1d8a5160 2024-09-10T07:41:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.779+0000 7fdfede00640 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561f1d8a5160 0x561f1d8a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.784+0000 7fdfede00640 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561f1d8a5160 0x561f1d8a7580 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf3a36c771133f1 server_cookie=fd97c371f6e5ab36 in_seq=0 out_seq=0 2024-09-10T07:41:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.784+0000 7fdfcb400640 10 client.26674 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.800+0000 7fdfcb400640 1 -- 172.21.15.46:0/409509089 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f1d790b80 con 0x561f1d8a5160 2024-09-10T07:41:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.800+0000 7fdfcb400640 10 client.26674 handle_command_reply: tid=0 2024-09-10T07:41:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 client.26674 shutdown 2024-09-10T07:41:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 -- 172.21.15.46:0/409509089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 msgr2=0x561f1d8332f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 0x561f1d8332f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80a4820 tx=0x7fdfd8099e90 comp rx=0 tx=0).stop 2024-09-10T07:41:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 -- 172.21.15.46:0/409509089 shutdown_connections 2024-09-10T07:41:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561f1d8a5160 0x561f1d8a7580 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f1d834300 0x561f1d834760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f1d829a60 0x561f1d8332f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 --2- 172.21.15.46:0/409509089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f1d824090 0x561f1d832db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 -- 172.21.15.46:0/409509089 >> 172.21.15.46:0/409509089 conn(0x561f1d81f700 msgr2=0x561f1d81e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdfcb400640 0 client.26674 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:41:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 -- 172.21.15.46:0/409509089 shutdown_connections 2024-09-10T07:41:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 1 -- 172.21.15.46:0/409509089 wait complete. 2024-09-10T07:41:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.801+0000 7fdff36c5740 20 client.26674 trim_cache size 0 max 16384 2024-09-10T07:41:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:41:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:41:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:41:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:41:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.802+0000 7fdff0800640 1 -- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdfd00a2cb0 msgr2=0x7fdfd00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.802+0000 7fdff0800640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdfd00a2cb0 0x7fdfd00a5170 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0057c0 tx=0x7fdfdc031400 comp rx=0 tx=0).stop 2024-09-10T07:41:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.802+0000 7fdff0800640 1 -- 172.21.15.46:0/2891694867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe8077df0 msgr2=0x7fdfe819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.802+0000 7fdff0800640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe8077df0 0x7fdfe819e9b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800e9f0 tx=0x7fdfd800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.803+0000 7fdff0800640 1 -- 172.21.15.46:0/2891694867 shutdown_connections 2024-09-10T07:41:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.803+0000 7fdff0800640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfe8078790 0x7fdfe81a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.803+0000 7fdff0800640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe8077df0 0x7fdfe819e9b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.803+0000 7fdff0800640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdfd00a2cb0 0x7fdfd00a5170 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.803+0000 7fdff0800640 1 --2- 172.21.15.46:0/2891694867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe8076bb0 0x7fdfe819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.803+0000 7fdff0800640 1 -- 172.21.15.46:0/2891694867 >> 172.21.15.46:0/2891694867 conn(0x7fdfe80ff4e0 msgr2=0x7fdfe8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.803+0000 7fdff0800640 1 -- 172.21.15.46:0/2891694867 shutdown_connections 2024-09-10T07:41:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:46.803+0000 7fdff0800640 1 -- 172.21.15.46:0/2891694867 wait complete. 2024-09-10T07:41:47.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:47 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:41:47.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:47 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:41:47.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:47 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:41:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:48 smithi181 ceph-mon[35397]: pgmap v554: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.5 MiB/s wr, 711 op/s 2024-09-10T07:41:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:48 smithi063 ceph-mon[35020]: pgmap v554: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.5 MiB/s wr, 711 op/s 2024-09-10T07:41:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:48 smithi046 ceph-mon[30705]: pgmap v554: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.5 MiB/s wr, 711 op/s 2024-09-10T07:41:49.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:49 smithi181 ceph-mon[35397]: pgmap v555: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-09-10T07:41:49.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:49 smithi063 ceph-mon[35020]: pgmap v555: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-09-10T07:41:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:49 smithi046 ceph-mon[30705]: pgmap v555: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-09-10T07:41:50.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 270.0 s, 997.2 tps, lat 97.949 ms stddev 122.150 2024-09-10T07:41:52.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:51 smithi181 ceph-mon[35397]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.2 MiB/s wr, 679 op/s 2024-09-10T07:41:52.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:51 smithi063 ceph-mon[35020]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.2 MiB/s wr, 679 op/s 2024-09-10T07:41:52.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:51 smithi046 ceph-mon[30705]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.2 MiB/s wr, 679 op/s 2024-09-10T07:41:54.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:53 smithi181 ceph-mon[35397]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.0 MiB/s wr, 621 op/s 2024-09-10T07:41:54.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:53 smithi063 ceph-mon[35020]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.0 MiB/s wr, 621 op/s 2024-09-10T07:41:54.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:53 smithi046 ceph-mon[30705]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.0 MiB/s wr, 621 op/s 2024-09-10T07:41:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 275.0 s, 789.4 tps, lat 128.179 ms stddev 152.705 2024-09-10T07:41:56.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:55 smithi181 ceph-mon[35397]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-09-10T07:41:56.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:55 smithi063 ceph-mon[35020]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-09-10T07:41:56.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:55 smithi046 ceph-mon[30705]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-09-10T07:41:58.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:57 smithi181 ceph-mon[35397]: pgmap v559: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.9 MiB/s wr, 793 op/s 2024-09-10T07:41:58.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:57 smithi063 ceph-mon[35020]: pgmap v559: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.9 MiB/s wr, 793 op/s 2024-09-10T07:41:58.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:57 smithi046 ceph-mon[30705]: pgmap v559: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.9 MiB/s wr, 793 op/s 2024-09-10T07:41:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 -- 172.21.15.46:0/1908937375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10103050 msgr2=0x7fae10103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 --2- 172.21.15.46:0/1908937375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10103050 0x7fae10103450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae08009ca0 tx=0x7fae0802a8d0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 -- 172.21.15.46:0/1908937375 shutdown_connections 2024-09-10T07:41:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 --2- 172.21.15.46:0/1908937375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae10101f40 0x7fae10109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 --2- 172.21.15.46:0/1908937375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae10101580 0x7fae10101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 --2- 172.21.15.46:0/1908937375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10103050 0x7fae10103450 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 -- 172.21.15.46:0/1908937375 >> 172.21.15.46:0/1908937375 conn(0x7fae100fd0d0 msgr2=0x7fae100ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 -- 172.21.15.46:0/1908937375 shutdown_connections 2024-09-10T07:41:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.974+0000 7fae16a00640 1 -- 172.21.15.46:0/1908937375 wait complete. 2024-09-10T07:41:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.976+0000 7fae16a00640 1 Processor -- start 2024-09-10T07:41:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.976+0000 7fae16a00640 1 -- start start 2024-09-10T07:41:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae10101580 0x7fae1019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae16a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae10101f40 0x7fae1019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae16a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10103050 0x7fae101a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae16a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae10113d40 con 0x7fae10101f40 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae16a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae10113bc0 con 0x7fae10101580 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae16a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fae10113ec0 con 0x7fae10103050 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae15600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae10101580 0x7fae1019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae15600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae10101580 0x7fae1019e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:39116/0 (socket says 172.21.15.46:39116) 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae15600640 1 -- 172.21.15.46:0/43379180 learned_addr learned my addr 172.21.15.46:0/43379180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae14c00640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae10101f40 0x7fae1019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.977+0000 7fae16000640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10103050 0x7fae101a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.978+0000 7fae15600640 1 -- 172.21.15.46:0/43379180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10103050 msgr2=0x7fae101a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.978+0000 7fae15600640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10103050 0x7fae101a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.978+0000 7fae15600640 1 -- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae10101f40 msgr2=0x7fae1019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.978+0000 7fae15600640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae10101f40 0x7fae1019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.978+0000 7fae15600640 1 -- 172.21.15.46:0/43379180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae101a5f60 con 0x7fae10101580 2024-09-10T07:41:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.978+0000 7fae15600640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae10101580 0x7fae1019e340 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae08005e30 tx=0x7fae08003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.980+0000 7fae0e000640 1 -- 172.21.15.46:0/43379180 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08004040 con 0x7fae10101580 2024-09-10T07:41:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.980+0000 7fae0e000640 1 -- 172.21.15.46:0/43379180 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae080046c0 con 0x7fae10101580 2024-09-10T07:41:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.980+0000 7fae0e000640 1 -- 172.21.15.46:0/43379180 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08030eb0 con 0x7fae10101580 2024-09-10T07:41:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.980+0000 7fae16a00640 1 -- 172.21.15.46:0/43379180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae101a61f0 con 0x7fae10101580 2024-09-10T07:41:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.981+0000 7fae16a00640 1 -- 172.21.15.46:0/43379180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae101a66f0 con 0x7fae10101580 2024-09-10T07:41:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.987+0000 7fae0e000640 1 -- 172.21.15.46:0/43379180 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae080305d0 con 0x7fae10101580 2024-09-10T07:41:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.987+0000 7fae1ae98740 1 Processor -- start 2024-09-10T07:41:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.988+0000 7fae1ae98740 1 -- start start 2024-09-10T07:41:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.991+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7705660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.991+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c770a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.991+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 0x55f7c770d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.991+0000 7fae1ae98740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f7c766a100 con 0x55f7c770ac30 2024-09-10T07:41:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.991+0000 7fae1ae98740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f7c76121b0 con 0x55f7c7705260 2024-09-10T07:41:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.991+0000 7fae1ae98740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f7c76660c0 con 0x55f7c7705ba0 2024-09-10T07:41:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.992+0000 7fae15600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c770a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.992+0000 7fae15600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c770a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57730/0 (socket says 172.21.15.46:57730) 2024-09-10T07:41:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.992+0000 7fae15600640 1 -- 172.21.15.46:0/894209181 learned_addr learned my addr 172.21.15.46:0/894209181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:41:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.992+0000 7fae16000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7705660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.992+0000 7fae14c00640 1 --2- 172.21.15.46:0/894209181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 0x55f7c770d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.993+0000 7fae16000640 1 -- 172.21.15.46:0/894209181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 msgr2=0x55f7c770a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.993+0000 7fae16000640 1 --2- 172.21.15.46:0/894209181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c770a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.993+0000 7fae16000640 1 -- 172.21.15.46:0/894209181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 msgr2=0x55f7c770d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:41:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.993+0000 7fae16000640 1 --2- 172.21.15.46:0/894209181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 0x55f7c770d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.993+0000 7fae16000640 1 -- 172.21.15.46:0/894209181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c737d210 con 0x55f7c7705260 2024-09-10T07:41:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.993+0000 7fae0e000640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fadf80a2c80 0x7fadf80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.993+0000 7fae16000640 1 --2- 172.21.15.46:0/894209181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7705660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00dc00 tx=0x7fadfc00d360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ee8911a71b61f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.994+0000 7fae02600640 1 -- 172.21.15.46:0/894209181 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc01c070 con 0x55f7c7705260 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.994+0000 7fae0e000640 1 -- 172.21.15.46:0/43379180 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fae080f0a10 con 0x7fae10101580 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.994+0000 7fae02600640 1 -- 172.21.15.46:0/894209181 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadfc004250 con 0x55f7c7705260 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.994+0000 7fae02600640 1 -- 172.21.15.46:0/894209181 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc01c070 con 0x55f7c7705260 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.994+0000 7fae15600640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fadf80a2c80 0x7fadf80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.995+0000 7fae1ae98740 1 -- 172.21.15.46:0/894209181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 msgr2=0x55f7c7705660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.995+0000 7fae1ae98740 1 --2- 172.21.15.46:0/894209181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7705660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00dc00 tx=0x7fadfc00d360 comp rx=0 tx=0).stop 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.995+0000 7fae1ae98740 1 -- 172.21.15.46:0/894209181 shutdown_connections 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.995+0000 7fae1ae98740 1 --2- 172.21.15.46:0/894209181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 0x55f7c770d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.995+0000 7fae1ae98740 1 --2- 172.21.15.46:0/894209181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c770a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.995+0000 7fae1ae98740 1 --2- 172.21.15.46:0/894209181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7705660 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:41:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.995+0000 7fae1ae98740 1 -- 172.21.15.46:0/894209181 >> 172.21.15.46:0/894209181 conn(0x55f7c77008d0 msgr2=0x55f7c7700cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:41:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.996+0000 7fae1ae98740 1 -- 172.21.15.46:0/894209181 shutdown_connections 2024-09-10T07:41:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.997+0000 7fae15600640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fadf80a2c80 0x7fadf80a5140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae080bcb90 tx=0x7fae0802a8d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:41:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.997+0000 7fae1ae98740 1 -- 172.21.15.46:0/894209181 wait complete. 2024-09-10T07:41:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.998+0000 7fae1ae98740 1 -- start start 2024-09-10T07:41:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.998+0000 7fae1ae98740 1 Processor -- start 2024-09-10T07:41:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c770f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705ba0 0x55f7c770fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:41:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c77169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae1ae98740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f7c770dc00 con 0x55f7c7705ba0 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae1ae98740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f7c75f3090 con 0x55f7c7705260 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae1ae98740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f7c770dd80 con 0x55f7c770ac30 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae1ae98740 10 client.0 _ll_register_callbacks cb 0x55f7c7647f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae16000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c770f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae16000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c770f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:39138/0 (socket says 172.21.15.46:39138) 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae16000640 1 -- 172.21.15.46:0/3834610916 learned_addr learned my addr 172.21.15.46:0/3834610916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae16000640 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c77169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:41:59.999+0000 7fae14c00640 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705ba0 0x55f7c770fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fae16000640 1 -- 172.21.15.46:0/3834610916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 msgr2=0x55f7c77169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fae16000640 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c77169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fae16000640 1 -- 172.21.15.46:0/3834610916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705ba0 msgr2=0x55f7c770fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fae16000640 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705ba0 0x55f7c770fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fae16000640 1 -- 172.21.15.46:0/3834610916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c7449620 con 0x55f7c7705260 2024-09-10T07:42:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fae16000640 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c770f4e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fadfc005940 tx=0x7fadfc00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b451c84cf9261369 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fadf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fadf6a00640 1 -- 172.21.15.46:0/3834610916 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc01c070 con 0x55f7c7705260 2024-09-10T07:42:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fadf6a00640 1 -- 172.21.15.46:0/3834610916 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fadfc005460 con 0x55f7c7705260 2024-09-10T07:42:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.000+0000 7fae1ae98740 1 -- 172.21.15.46:0/3834610916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadfc004250 con 0x55f7c7705260 2024-09-10T07:42:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fadf6a00640 1 -- 172.21.15.46:0/3834610916 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fadfc003f00 con 0x55f7c7705260 2024-09-10T07:42:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fae1ae98740 10 client.26714 fetch_fsmap learned FSMap version 41 2024-09-10T07:42:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fae1ae98740 1 -- 172.21.15.46:0/3834610916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f7c7719810 con 0x55f7c7705260 2024-09-10T07:42:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fadf6a00640 1 -- 172.21.15.46:0/3834610916 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadfc004250 con 0x55f7c7705260 2024-09-10T07:42:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fadf6a00640 1 -- 172.21.15.46:0/3834610916 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fadfc0177b0 con 0x55f7c7705260 2024-09-10T07:42:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fae1ae98740 10 client.26714 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T07:42:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fae1ae98740 10 client.26714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fae1ae98740 20 client.26714 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f7c771ae00 0x55f7c771d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fae1ae98740 4 client.26714 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.001+0000 7fae1ae98740 1 -- 172.21.15.46:0/3834610916 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7c7677830 con 0x55f7c771ae00 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.002+0000 7fae14c00640 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f7c771ae00 0x55f7c771d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.002+0000 7fae14c00640 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f7c771ae00 0x55f7c771d1f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd09ac43d7f60ae0 server_cookie=b703324c89ac7887 in_seq=0 out_seq=0 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.002+0000 7fadf6a00640 10 client.26714 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fadf6a00640 1 -- 172.21.15.46:0/3834610916 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7c7677830 con 0x55f7c771ae00 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fadf6a00640 10 client.26714 handle_command_reply: tid=0 2024-09-10T07:42:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 client.26714 shutdown 2024-09-10T07:42:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 -- 172.21.15.46:0/3834610916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 msgr2=0x55f7c770f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c770f4e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fadfc005940 tx=0x7fadfc00bf10 comp rx=0 tx=0).stop 2024-09-10T07:42:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 -- 172.21.15.46:0/3834610916 shutdown_connections 2024-09-10T07:42:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f7c771ae00 0x55f7c771d1f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c77169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705ba0 0x55f7c770fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3834610916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c770f4e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fadf6a00640 0 client.26714 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.003+0000 7fae1ae98740 1 -- 172.21.15.46:0/3834610916 >> 172.21.15.46:0/3834610916 conn(0x55f7c77008d0 msgr2=0x55f7c77020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.004+0000 7fae1ae98740 1 -- 172.21.15.46:0/3834610916 shutdown_connections 2024-09-10T07:42:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.004+0000 7fae1ae98740 1 -- 172.21.15.46:0/3834610916 wait complete. 2024-09-10T07:42:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.004+0000 7fae1ae98740 20 client.26714 trim_cache size 0 max 16384 2024-09-10T07:42:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.023+0000 7fae1ae98740 1 Processor -- start 2024-09-10T07:42:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.023+0000 7fae1ae98740 1 -- start start 2024-09-10T07:42:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705260 0x55f7c770f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705ba0 0x55f7c77169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c7701780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae1ae98740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadf000fb50 con 0x55f7c7705260 2024-09-10T07:42:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae1ae98740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0011750 con 0x55f7c7705ba0 2024-09-10T07:42:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae1ae98740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0010220 con 0x55f7c770ac30 2024-09-10T07:42:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705260 0x55f7c770f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae14c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705ba0 0x55f7c77169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae14c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705ba0 0x55f7c77169b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:39150/0 (socket says 172.21.15.46:39150) 2024-09-10T07:42:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae14c00640 1 -- 172.21.15.46:0/784074095 learned_addr learned my addr 172.21.15.46:0/784074095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705260 0x55f7c770f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43952/0 (socket says 172.21.15.46:43952) 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae16000640 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c7701780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae14c00640 1 -- 172.21.15.46:0/784074095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 msgr2=0x55f7c7701780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae14c00640 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c7701780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae14c00640 1 -- 172.21.15.46:0/784074095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705260 msgr2=0x55f7c770f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae14c00640 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705260 0x55f7c770f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.024+0000 7fae14c00640 1 -- 172.21.15.46:0/784074095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c771d560 con 0x55f7c7705ba0 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.025+0000 7fae16000640 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c7701780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.025+0000 7fae14c00640 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705ba0 0x55f7c77169b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55f7c771da80 tx=0x7fae04013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef82ff2b447f922 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.025+0000 7fae01c00640 1 -- 172.21.15.46:0/784074095 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04011440 con 0x55f7c7705ba0 2024-09-10T07:42:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.025+0000 7fae01c00640 1 -- 172.21.15.46:0/784074095 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae04011ac0 con 0x55f7c7705ba0 2024-09-10T07:42:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.025+0000 7fae01c00640 1 -- 172.21.15.46:0/784074095 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0401cb10 con 0x55f7c7705ba0 2024-09-10T07:42:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 -- 172.21.15.46:0/784074095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705ba0 msgr2=0x55f7c77169b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705ba0 0x55f7c77169b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55f7c771da80 tx=0x7fae04013830 comp rx=0 tx=0).stop 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 -- 172.21.15.46:0/784074095 shutdown_connections 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c770ac30 0x55f7c7701780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705ba0 0x55f7c77169b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 --2- 172.21.15.46:0/784074095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c7705260 0x55f7c770f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 -- 172.21.15.46:0/784074095 >> 172.21.15.46:0/784074095 conn(0x55f7c77008d0 msgr2=0x55f7c770f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 -- 172.21.15.46:0/784074095 shutdown_connections 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 -- 172.21.15.46:0/784074095 wait complete. 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.026+0000 7fae1ae98740 1 -- start start 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae1ae98740 1 Processor -- start 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7713b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c7714040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae1ae98740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 0x55f7c7715050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae1ae98740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0011020 con 0x55f7c770ac30 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae1ae98740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0010950 con 0x55f7c7705260 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae1ae98740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fadf000db30 con 0x55f7c7705ba0 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae1ae98740 10 client.0 _ll_register_callbacks cb 0x7fadfc01a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae16000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7713b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae16000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7713b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:39152/0 (socket says 172.21.15.46:39152) 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae16000640 1 -- 172.21.15.46:0/3529651717 learned_addr learned my addr 172.21.15.46:0/3529651717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.027+0000 7fae14c00640 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c7714040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.028+0000 7fae16000640 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 0x55f7c7715050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.028+0000 7fae14c00640 1 -- 172.21.15.46:0/3529651717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 msgr2=0x55f7c7713b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.028+0000 7fae14c00640 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7713b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.028+0000 7fae14c00640 1 -- 172.21.15.46:0/3529651717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 msgr2=0x55f7c7715050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.028+0000 7fae14c00640 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 0x55f7c7715050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.028+0000 7fae14c00640 1 -- 172.21.15.46:0/3529651717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c7715660 con 0x55f7c7705ba0 2024-09-10T07:42:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.029+0000 7fae14c00640 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c7714040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55f7c770a3f0 tx=0x7fae040131d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=283ce85fcc84b6ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.029+0000 7fadf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.029+0000 7fadf6a00640 1 -- 172.21.15.46:0/3529651717 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0402b070 con 0x55f7c7705ba0 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.029+0000 7fadf6a00640 1 -- 172.21.15.46:0/3529651717 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fae04011850 con 0x55f7c7705ba0 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.029+0000 7fae1ae98740 1 -- 172.21.15.46:0/3529651717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae04011ac0 con 0x55f7c7705ba0 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.029+0000 7fadf6a00640 1 -- 172.21.15.46:0/3529651717 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae0401c890 con 0x55f7c7705ba0 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.029+0000 7fae1ae98740 10 client.26689 fetch_fsmap learned FSMap version 41 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.029+0000 7fae1ae98740 1 -- 172.21.15.46:0/3529651717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f7c7716980 con 0x55f7c7705ba0 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fadf6a00640 1 -- 172.21.15.46:0/3529651717 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae04011ac0 con 0x55f7c7705ba0 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fadf6a00640 1 -- 172.21.15.46:0/3529651717 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fae04004cd0 con 0x55f7c7705ba0 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fae1ae98740 10 client.26689 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fae1ae98740 10 client.26689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fae1ae98740 20 client.26689 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f7c7785f40 0x55f7c7788360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fae1ae98740 4 client.26689 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:42:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fae1ae98740 1 -- 172.21.15.46:0/3529651717 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7c7671b80 con 0x55f7c7785f40 2024-09-10T07:42:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.030+0000 7fae16000640 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f7c7785f40 0x55f7c7788360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.031+0000 7fae16000640 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f7c7785f40 0x55f7c7788360 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca8ed49aa417e46c server_cookie=f9583fa6bc18ad2d in_seq=0 out_seq=0 2024-09-10T07:42:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.031+0000 7fadf6a00640 10 client.26689 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:00.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:41:59 smithi181 ceph-mon[35397]: pgmap v560: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-09-10T07:42:00.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:41:59 smithi063 ceph-mon[35020]: pgmap v560: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-09-10T07:42:00.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:41:59 smithi046 ceph-mon[30705]: pgmap v560: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-09-10T07:42:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.359+0000 7fadf6a00640 1 -- 172.21.15.46:0/3529651717 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7c7671b80 con 0x55f7c7785f40 2024-09-10T07:42:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.359+0000 7fadf6a00640 10 client.26689 handle_command_reply: tid=0 2024-09-10T07:42:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 client.26689 shutdown 2024-09-10T07:42:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 -- 172.21.15.46:0/3529651717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 msgr2=0x55f7c7714040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c7714040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55f7c770a3f0 tx=0x7fae040131d0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 -- 172.21.15.46:0/3529651717 shutdown_connections 2024-09-10T07:42:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f7c7785f40 0x55f7c7788360 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7c770ac30 0x55f7c7715050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7c7705ba0 0x55f7c7714040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 --2- 172.21.15.46:0/3529651717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7c7705260 0x55f7c7713b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fadf6a00640 0 client.26689 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.360+0000 7fae1ae98740 1 -- 172.21.15.46:0/3529651717 >> 172.21.15.46:0/3529651717 conn(0x55f7c77008d0 msgr2=0x55f7c770f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.361+0000 7fae1ae98740 1 -- 172.21.15.46:0/3529651717 shutdown_connections 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.361+0000 7fae1ae98740 1 -- 172.21.15.46:0/3529651717 wait complete. 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.361+0000 7fae1ae98740 20 client.26689 trim_cache size 0 max 16384 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:42:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:42:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:42:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 -- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fadf80a2c80 msgr2=0x7fadf80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fadf80a2c80 0x7fadf80a5140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae080bcb90 tx=0x7fae0802a8d0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 -- 172.21.15.46:0/43379180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae10101580 msgr2=0x7fae1019e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae10101580 0x7fae1019e340 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae08005e30 tx=0x7fae08003c00 comp rx=0 tx=0).stop 2024-09-10T07:42:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 -- 172.21.15.46:0/43379180 shutdown_connections 2024-09-10T07:42:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10103050 0x7fae101a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fadf80a2c80 0x7fadf80a5140 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae10101f40 0x7fae1019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 --2- 172.21.15.46:0/43379180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae10101580 0x7fae1019e340 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae08005e30 tx=0x7fae08003c00 comp rx=0 tx=0).stop 2024-09-10T07:42:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.363+0000 7fae16a00640 1 -- 172.21.15.46:0/43379180 >> 172.21.15.46:0/43379180 conn(0x7fae100fd0d0 msgr2=0x7fae10105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.365+0000 7fae16a00640 1 -- 172.21.15.46:0/43379180 shutdown_connections 2024-09-10T07:42:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:00.365+0000 7fae16a00640 1 -- 172.21.15.46:0/43379180 wait complete. 2024-09-10T07:42:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 280.0 s, 960.6 tps, lat 103.680 ms stddev 128.773 2024-09-10T07:42:02.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:01 smithi181 ceph-mon[35397]: pgmap v561: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 661 op/s 2024-09-10T07:42:02.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:01 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:42:02.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:01 smithi063 ceph-mon[35020]: pgmap v561: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 661 op/s 2024-09-10T07:42:02.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:01 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:42:02.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:01 smithi046 ceph-mon[30705]: pgmap v561: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 661 op/s 2024-09-10T07:42:02.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:01 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:42:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:42:03.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:42:03.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:02 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:42:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:42:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:42:03.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:02 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:42:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:42:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:42:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:42:03.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:02 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:42:04.153 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:03 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi046 to 177.5M 2024-09-10T07:42:04.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:03 smithi181 ceph-mon[35397]: Unable to set osd_memory_target on smithi046 to 186147942: error parsing value: Value '186147942' is below minimum 939524096 2024-09-10T07:42:04.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:03 smithi181 ceph-mon[35397]: pgmap v562: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.9 MiB/s wr, 653 op/s 2024-09-10T07:42:04.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:03 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:42:04.154 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:03 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi046 to 177.5M 2024-09-10T07:42:04.154 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:03 smithi063 ceph-mon[35020]: Unable to set osd_memory_target on smithi046 to 186147942: error parsing value: Value '186147942' is below minimum 939524096 2024-09-10T07:42:04.154 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:03 smithi063 ceph-mon[35020]: pgmap v562: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.9 MiB/s wr, 653 op/s 2024-09-10T07:42:04.154 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:03 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:42:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:03 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi046 to 177.5M 2024-09-10T07:42:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:03 smithi046 ceph-mon[30705]: Unable to set osd_memory_target on smithi046 to 186147942: error parsing value: Value '186147942' is below minimum 939524096 2024-09-10T07:42:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:03 smithi046 ceph-mon[30705]: pgmap v562: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.9 MiB/s wr, 653 op/s 2024-09-10T07:42:04.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:03 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:42:04.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:42:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 285.0 s, 880.2 tps, lat 115.040 ms stddev 129.506 2024-09-10T07:42:06.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:05 smithi181 ceph-mon[35397]: pgmap v563: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 MiB/s wr, 570 op/s 2024-09-10T07:42:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:05 smithi063 ceph-mon[35020]: pgmap v563: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 MiB/s wr, 570 op/s 2024-09-10T07:42:06.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:05 smithi046 ceph-mon[30705]: pgmap v563: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 MiB/s wr, 570 op/s 2024-09-10T07:42:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.642+0000 7faccfe00640 1 -- 172.21.15.46:0/2182511088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8101de0 msgr2=0x7facc81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.642+0000 7facc7e00640 1 -- 172.21.15.46:0/2182511088 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc004230 con 0x7facc8101de0 2024-09-10T07:42:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.642+0000 7faccfe00640 1 --2- 172.21.15.46:0/2182511088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8101de0 0x7facc81021c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7facbc009900 tx=0x7facbc02a420 comp rx=0 tx=0).stop 2024-09-10T07:42:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.645+0000 7faccfe00640 1 -- 172.21.15.46:0/2182511088 shutdown_connections 2024-09-10T07:42:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.645+0000 7faccfe00640 1 --2- 172.21.15.46:0/2182511088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8106f10 0x7facc810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.645+0000 7faccfe00640 1 --2- 172.21.15.46:0/2182511088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8102790 0x7facc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.645+0000 7faccfe00640 1 --2- 172.21.15.46:0/2182511088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8101de0 0x7facc81021c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.645+0000 7faccfe00640 1 -- 172.21.15.46:0/2182511088 >> 172.21.15.46:0/2182511088 conn(0x7facc80fd8c0 msgr2=0x7facc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.645+0000 7faccfe00640 1 -- 172.21.15.46:0/2182511088 shutdown_connections 2024-09-10T07:42:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.645+0000 7faccfe00640 1 -- 172.21.15.46:0/2182511088 wait complete. 2024-09-10T07:42:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.647+0000 7faccfe00640 1 Processor -- start 2024-09-10T07:42:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.648+0000 7faccfe00640 1 -- start start 2024-09-10T07:42:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.649+0000 7faccfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8101de0 0x7facc81a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.649+0000 7faccfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8102790 0x7facc81a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.649+0000 7faccfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8106f10 0x7facc810d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8101de0 0x7facc81a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7facc8111fe0 con 0x7facc8101de0 2024-09-10T07:42:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8102790 0x7facc81a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8106f10 0x7facc810d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8106f10 0x7facc810d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37460/0 (socket says 172.21.15.46:37460) 2024-09-10T07:42:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8102790 0x7facc81a3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49882/0 (socket says 172.21.15.46:49882) 2024-09-10T07:42:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccfe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7facc8111e60 con 0x7facc8102790 2024-09-10T07:42:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccfe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7facc8112160 con 0x7facc8106f10 2024-09-10T07:42:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccca00640 1 -- 172.21.15.46:0/711379455 learned_addr learned my addr 172.21.15.46:0/711379455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.650+0000 7faccd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8101de0 0x7facc81a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47980/0 (socket says 172.21.15.46:47980) 2024-09-10T07:42:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.651+0000 7faccde00640 1 -- 172.21.15.46:0/711379455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8102790 msgr2=0x7facc81a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.651+0000 7faccde00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8102790 0x7facc81a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.651+0000 7faccde00640 1 -- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8101de0 msgr2=0x7facc81a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.651+0000 7faccde00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8101de0 0x7facc81a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.652+0000 7faccde00640 1 -- 172.21.15.46:0/711379455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc810e140 con 0x7facc8106f10 2024-09-10T07:42:07.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.653+0000 7faccde00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8106f10 0x7facc810d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7facc000cab0 tx=0x7facc000cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.653+0000 7facc6000640 1 -- 172.21.15.46:0/711379455 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc000ae20 con 0x7facc8106f10 2024-09-10T07:42:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.654+0000 7facc6000640 1 -- 172.21.15.46:0/711379455 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc000f930 con 0x7facc8106f10 2024-09-10T07:42:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.654+0000 7facc6000640 1 -- 172.21.15.46:0/711379455 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0018ad0 con 0x7facc8106f10 2024-09-10T07:42:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.655+0000 7faccfe00640 1 -- 172.21.15.46:0/711379455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc810e430 con 0x7facc8106f10 2024-09-10T07:42:07.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.655+0000 7faccfe00640 1 -- 172.21.15.46:0/711379455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81ab3a0 con 0x7facc8106f10 2024-09-10T07:42:07.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.662+0000 7facd27cc740 1 Processor -- start 2024-09-10T07:42:07.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.663+0000 7facc6000640 1 -- 172.21.15.46:0/711379455 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7facc000fae0 con 0x7facc8106f10 2024-09-10T07:42:07.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.663+0000 7facd27cc740 1 -- start start 2024-09-10T07:42:07.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7facd27cc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5128b90 0x55acd5128f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7facd27cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5133830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7facd27cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5133d70 0x55acd5136160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7facd27cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55acd508f3e0 con 0x55acd51294b0 2024-09-10T07:42:07.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7facd27cc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55acd5036d80 con 0x55acd5128b90 2024-09-10T07:42:07.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7facd27cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55acd508b3a0 con 0x55acd5133d70 2024-09-10T07:42:07.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7faccca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5128b90 0x55acd5128f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7faccca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5128b90 0x55acd5128f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49894/0 (socket says 172.21.15.46:49894) 2024-09-10T07:42:07.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7faccca00640 1 -- 172.21.15.46:0/3821984388 learned_addr learned my addr 172.21.15.46:0/3821984388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:07.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.664+0000 7faccca00640 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5133d70 0x55acd5136160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.665+0000 7faccd400640 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.665+0000 7faccca00640 1 -- 172.21.15.46:0/3821984388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5133d70 msgr2=0x55acd5136160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.665+0000 7faccca00640 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5133d70 0x55acd5136160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.665+0000 7faccca00640 1 -- 172.21.15.46:0/3821984388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 msgr2=0x55acd5133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.665+0000 7faccca00640 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5133830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.665+0000 7facc6000640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facb00a2d20 0x7facb00a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.665+0000 7faccca00640 1 -- 172.21.15.46:0/3821984388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd5055ae0 con 0x55acd5128b90 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.666+0000 7faccca00640 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5128b90 0x55acd5128f70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7facb4006aa0 tx=0x7facb4007220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=832c62c9a163fcd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.666+0000 7facc6000640 1 -- 172.21.15.46:0/711379455 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7facc00cfca0 con 0x7facc8106f10 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.666+0000 7facba600640 1 -- 172.21.15.46:0/3821984388 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb402e070 con 0x55acd5128b90 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.666+0000 7facba600640 1 -- 172.21.15.46:0/3821984388 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb4010040 con 0x55acd5128b90 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.666+0000 7facba600640 1 -- 172.21.15.46:0/3821984388 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4029a00 con 0x55acd5128b90 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.666+0000 7faccd400640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facb00a2d20 0x7facb00a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.667+0000 7facd27cc740 1 -- 172.21.15.46:0/3821984388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5128b90 msgr2=0x55acd5128f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.667+0000 7facd27cc740 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5128b90 0x55acd5128f70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7facb4006aa0 tx=0x7facb4007220 comp rx=0 tx=0).stop 2024-09-10T07:42:07.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.668+0000 7facd27cc740 1 -- 172.21.15.46:0/3821984388 shutdown_connections 2024-09-10T07:42:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.668+0000 7facd27cc740 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5133d70 0x55acd5136160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.668+0000 7facd27cc740 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.668+0000 7facd27cc740 1 --2- 172.21.15.46:0/3821984388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5128b90 0x55acd5128f70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.668+0000 7facd27cc740 1 -- 172.21.15.46:0/3821984388 >> 172.21.15.46:0/3821984388 conn(0x55acd5125120 msgr2=0x55acd5125500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.668+0000 7facd27cc740 1 -- 172.21.15.46:0/3821984388 shutdown_connections 2024-09-10T07:42:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.669+0000 7faccd400640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facb00a2d20 0x7facb00a51e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55acd5125f80 tx=0x7facbc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.669+0000 7facd27cc740 1 -- 172.21.15.46:0/3821984388 wait complete. 2024-09-10T07:42:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.669+0000 7facd27cc740 1 -- start start 2024-09-10T07:42:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.670+0000 7facd27cc740 1 Processor -- start 2024-09-10T07:42:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.670+0000 7facd27cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5128b90 0x55acd5130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.670+0000 7facd27cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd51294b0 0x55acd51312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.671+0000 7facd27cc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd51317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.671+0000 7facd27cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55acd51369e0 con 0x55acd5128b90 2024-09-10T07:42:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.671+0000 7facd27cc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55acd504a0f0 con 0x55acd5133d70 2024-09-10T07:42:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.671+0000 7facd27cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55acd5136b60 con 0x55acd51294b0 2024-09-10T07:42:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.671+0000 7faccca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd51294b0 0x55acd51312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.671+0000 7facd27cc740 10 client.0 _ll_register_callbacks cb 0x55acd503b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.671+0000 7faccde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd51317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.672+0000 7faccde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd51317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49908/0 (socket says 172.21.15.46:49908) 2024-09-10T07:42:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.672+0000 7faccde00640 1 -- 172.21.15.46:0/2927178405 learned_addr learned my addr 172.21.15.46:0/2927178405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.672+0000 7faccde00640 1 -- 172.21.15.46:0/2927178405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd51294b0 msgr2=0x55acd51312b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.672+0000 7faccde00640 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd51294b0 0x55acd51312b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.672+0000 7faccde00640 1 -- 172.21.15.46:0/2927178405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5128b90 msgr2=0x55acd5130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.672+0000 7faccde00640 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5128b90 0x55acd5130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.672+0000 7faccde00640 1 -- 172.21.15.46:0/2927178405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd4e003c0 con 0x55acd5133d70 2024-09-10T07:42:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.672+0000 7faccca00640 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd51294b0 0x55acd51312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.673+0000 7faccde00640 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd51317f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7facc0098b70 tx=0x7facc009c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b906c4cff2c6bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.673+0000 7facaea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.673+0000 7facaea00640 1 -- 172.21.15.46:0/2927178405 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc00b5070 con 0x55acd5133d70 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.673+0000 7facaea00640 1 -- 172.21.15.46:0/2927178405 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7facc0018c70 con 0x55acd5133d70 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.673+0000 7facd27cc740 1 -- 172.21.15.46:0/2927178405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb4010040 con 0x55acd5133d70 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.674+0000 7facaea00640 1 -- 172.21.15.46:0/2927178405 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7facc00a4e10 con 0x55acd5133d70 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.674+0000 7facd27cc740 10 client.26729 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.674+0000 7facd27cc740 1 -- 172.21.15.46:0/2927178405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55acd5142810 con 0x55acd5133d70 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.674+0000 7facaea00640 1 -- 172.21.15.46:0/2927178405 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb4010040 con 0x55acd5133d70 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.674+0000 7facaea00640 1 -- 172.21.15.46:0/2927178405 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7facc0020070 con 0x55acd5133d70 2024-09-10T07:42:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.674+0000 7facd27cc740 10 client.26729 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.674+0000 7facd27cc740 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.674+0000 7facd27cc740 20 client.26729 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.675+0000 7facd27cc740 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55acd5143d20 0x55acd51461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.675+0000 7facd27cc740 4 client.26729 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.675+0000 7facd27cc740 1 -- 172.21.15.46:0/2927178405 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acd509c950 con 0x55acd5143d20 2024-09-10T07:42:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.675+0000 7faccd400640 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55acd5143d20 0x55acd51461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.676+0000 7faccd400640 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55acd5143d20 0x55acd51461c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0195f97e3432bc3 server_cookie=974d5bd6d25df7e4 in_seq=0 out_seq=0 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.676+0000 7facaea00640 10 client.26729 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.676+0000 7facaea00640 1 -- 172.21.15.46:0/2927178405 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55acd509c950 con 0x55acd5143d20 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.676+0000 7facaea00640 10 client.26729 handle_command_reply: tid=0 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 client.26729 shutdown 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 -- 172.21.15.46:0/2927178405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 msgr2=0x55acd51317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd51317f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7facc0098b70 tx=0x7facc009c960 comp rx=0 tx=0).stop 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 -- 172.21.15.46:0/2927178405 shutdown_connections 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55acd5143d20 0x55acd51461c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd51317f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd51294b0 0x55acd51312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 --2- 172.21.15.46:0/2927178405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5128b90 0x55acd5130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facd27cc740 1 -- 172.21.15.46:0/2927178405 >> 172.21.15.46:0/2927178405 conn(0x55acd5125120 msgr2=0x55acd512ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:07.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.677+0000 7facaea00640 0 client.26729 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:07.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.678+0000 7facd27cc740 1 -- 172.21.15.46:0/2927178405 shutdown_connections 2024-09-10T07:42:07.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.678+0000 7facd27cc740 1 -- 172.21.15.46:0/2927178405 wait complete. 2024-09-10T07:42:07.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.678+0000 7facd27cc740 20 client.26729 trim_cache size 0 max 16384 2024-09-10T07:42:07.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.708+0000 7facd27cc740 1 Processor -- start 2024-09-10T07:42:07.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.708+0000 7facd27cc740 1 -- start start 2024-09-10T07:42:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.708+0000 7facd27cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd512b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.708+0000 7facd27cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5131440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.708+0000 7facd27cc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd514bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.708+0000 7facd27cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7facc81ac970 con 0x55acd51294b0 2024-09-10T07:42:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.708+0000 7facd27cc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7facc81ae510 con 0x55acd5133d70 2024-09-10T07:42:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.708+0000 7facd27cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7facc81ad040 con 0x55acd5128b90 2024-09-10T07:42:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.709+0000 7faccd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd512b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.709+0000 7faccd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd512b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37488/0 (socket says 172.21.15.46:37488) 2024-09-10T07:42:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.709+0000 7faccd400640 1 -- 172.21.15.46:0/3361998387 learned_addr learned my addr 172.21.15.46:0/3361998387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.709+0000 7faccde00640 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd514bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.709+0000 7faccca00640 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5131440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.710+0000 7faccd400640 1 -- 172.21.15.46:0/3361998387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 msgr2=0x55acd514bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.710+0000 7faccd400640 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd514bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.710+0000 7faccd400640 1 -- 172.21.15.46:0/3361998387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 msgr2=0x55acd5131440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.710+0000 7faccde00640 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd514bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.710+0000 7faccd400640 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5131440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.710+0000 7faccd400640 1 -- 172.21.15.46:0/3361998387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd51261d0 con 0x55acd5128b90 2024-09-10T07:42:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.710+0000 7faccca00640 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5131440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.711+0000 7faccd400640 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd512b640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facbc000c00 tx=0x7facbc041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37484f25fb9ff66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.711+0000 7facb9c00640 1 -- 172.21.15.46:0/3361998387 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc059070 con 0x55acd5128b90 2024-09-10T07:42:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.713+0000 7facb9c00640 1 -- 172.21.15.46:0/3361998387 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facbc0054a0 con 0x55acd5128b90 2024-09-10T07:42:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.713+0000 7facb9c00640 1 -- 172.21.15.46:0/3361998387 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc03f6a0 con 0x55acd5128b90 2024-09-10T07:42:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.714+0000 7facd27cc740 1 -- 172.21.15.46:0/3361998387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 msgr2=0x55acd512b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.714+0000 7facd27cc740 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd512b640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facbc000c00 tx=0x7facbc041000 comp rx=0 tx=0).stop 2024-09-10T07:42:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.721+0000 7facd27cc740 1 -- 172.21.15.46:0/3361998387 shutdown_connections 2024-09-10T07:42:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.721+0000 7facd27cc740 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd5133d70 0x55acd514bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.721+0000 7facd27cc740 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd51294b0 0x55acd5131440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.721+0000 7facd27cc740 1 --2- 172.21.15.46:0/3361998387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd512b640 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.721+0000 7facd27cc740 1 -- 172.21.15.46:0/3361998387 >> 172.21.15.46:0/3361998387 conn(0x55acd5125120 msgr2=0x55acd5130d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.724+0000 7facd27cc740 1 -- 172.21.15.46:0/3361998387 shutdown_connections 2024-09-10T07:42:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.724+0000 7facd27cc740 1 -- 172.21.15.46:0/3361998387 wait complete. 2024-09-10T07:42:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.724+0000 7facd27cc740 1 -- start start 2024-09-10T07:42:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.724+0000 7facd27cc740 1 Processor -- start 2024-09-10T07:42:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.726+0000 7facd27cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd513c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.726+0000 7facd27cc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd51294b0 0x55acd5137810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.726+0000 7facd27cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5133d70 0x55acd5137d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.726+0000 7facd27cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7facc81adde0 con 0x55acd5133d70 2024-09-10T07:42:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.726+0000 7facd27cc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7facc81ad710 con 0x55acd51294b0 2024-09-10T07:42:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.726+0000 7facd27cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7facc81032c0 con 0x55acd5128b90 2024-09-10T07:42:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.726+0000 7facd27cc740 10 client.0 _ll_register_callbacks cb 0x7facc009bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.726+0000 7faccde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5133d70 0x55acd5137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.727+0000 7faccd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd513c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.727+0000 7faccca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd51294b0 0x55acd5137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.727+0000 7faccd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd513c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37496/0 (socket says 172.21.15.46:37496) 2024-09-10T07:42:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.727+0000 7faccd400640 1 -- 172.21.15.46:0/298671502 learned_addr learned my addr 172.21.15.46:0/298671502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.727+0000 7faccd400640 1 -- 172.21.15.46:0/298671502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd51294b0 msgr2=0x55acd5137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.727+0000 7faccd400640 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd51294b0 0x55acd5137810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.727+0000 7faccd400640 1 -- 172.21.15.46:0/298671502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5133d70 msgr2=0x55acd5137d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.728+0000 7faccd400640 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5133d70 0x55acd5137d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.728+0000 7faccd400640 1 -- 172.21.15.46:0/298671502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd5125db0 con 0x55acd5128b90 2024-09-10T07:42:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.728+0000 7faccde00640 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5133d70 0x55acd5137d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.730+0000 7faccd400640 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd513c4e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7facbc009900 tx=0x7facbc02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc7a4fb9124be1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.730+0000 7facaea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.731+0000 7facaea00640 1 -- 172.21.15.46:0/298671502 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc062060 con 0x55acd5128b90 2024-09-10T07:42:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.731+0000 7facd27cc740 1 -- 172.21.15.46:0/298671502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facbc0054a0 con 0x55acd5128b90 2024-09-10T07:42:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.731+0000 7facaea00640 1 -- 172.21.15.46:0/298671502 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7facbc005650 con 0x55acd5128b90 2024-09-10T07:42:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.732+0000 7facaea00640 1 -- 172.21.15.46:0/298671502 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7facbc059040 con 0x55acd5128b90 2024-09-10T07:42:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.732+0000 7facd27cc740 10 client.26707 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.732+0000 7facd27cc740 1 -- 172.21.15.46:0/298671502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55acd51ae8e0 con 0x55acd5128b90 2024-09-10T07:42:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.733+0000 7facaea00640 1 -- 172.21.15.46:0/298671502 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facbc0054a0 con 0x55acd5128b90 2024-09-10T07:42:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.733+0000 7facaea00640 1 -- 172.21.15.46:0/298671502 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7facbc05c070 con 0x55acd5128b90 2024-09-10T07:42:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.733+0000 7facd27cc740 10 client.26707 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.733+0000 7facd27cc740 10 client.26707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.733+0000 7facd27cc740 20 client.26707 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.733+0000 7facd27cc740 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55acd51aeea0 0x55acd51b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.733+0000 7facd27cc740 4 client.26707 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:42:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.733+0000 7facd27cc740 1 -- 172.21.15.46:0/298671502 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55acd5096e60 con 0x55acd51aeea0 2024-09-10T07:42:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.735+0000 7faccca00640 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55acd51aeea0 0x55acd51b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:07.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.737+0000 7faccca00640 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55acd51aeea0 0x55acd51b12c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26aa0075ab60ed6c server_cookie=d16033c2404ee4fb in_seq=0 out_seq=0 2024-09-10T07:42:07.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.737+0000 7facaea00640 10 client.26707 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:07.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.739+0000 7facaea00640 1 -- 172.21.15.46:0/298671502 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55acd5096e60 con 0x55acd51aeea0 2024-09-10T07:42:07.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.739+0000 7facaea00640 10 client.26707 handle_command_reply: tid=0 2024-09-10T07:42:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.739+0000 7facd27cc740 1 client.26707 shutdown 2024-09-10T07:42:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.739+0000 7facd27cc740 1 -- 172.21.15.46:0/298671502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 msgr2=0x55acd513c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.739+0000 7facd27cc740 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd513c4e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7facbc009900 tx=0x7facbc02a420 comp rx=0 tx=0).stop 2024-09-10T07:42:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facd27cc740 1 -- 172.21.15.46:0/298671502 shutdown_connections 2024-09-10T07:42:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facd27cc740 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55acd51aeea0 0x55acd51b12c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facd27cc740 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acd5133d70 0x55acd5137d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facd27cc740 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55acd51294b0 0x55acd5137810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facaea00640 0 client.26707 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facd27cc740 1 --2- 172.21.15.46:0/298671502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55acd5128b90 0x55acd513c4e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facd27cc740 1 -- 172.21.15.46:0/298671502 >> 172.21.15.46:0/298671502 conn(0x55acd5125120 msgr2=0x55acd51241e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facd27cc740 1 -- 172.21.15.46:0/298671502 shutdown_connections 2024-09-10T07:42:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.741+0000 7facd27cc740 1 -- 172.21.15.46:0/298671502 wait complete. 2024-09-10T07:42:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.742+0000 7facd27cc740 20 client.26707 trim_cache size 0 max 16384 2024-09-10T07:42:07.742 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:42:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:42:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:42:07.743 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:42:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 -- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facb00a2d20 msgr2=0x7facb00a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facb00a2d20 0x7facb00a51e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55acd5125f80 tx=0x7facbc005850 comp rx=0 tx=0).stop 2024-09-10T07:42:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 -- 172.21.15.46:0/711379455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8106f10 msgr2=0x7facc810d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8106f10 0x7facc810d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7facc000cab0 tx=0x7facc000cf70 comp rx=0 tx=0).stop 2024-09-10T07:42:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 -- 172.21.15.46:0/711379455 shutdown_connections 2024-09-10T07:42:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc8106f10 0x7facc810d9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8102790 0x7facc81a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facb00a2d20 0x7facb00a51e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 --2- 172.21.15.46:0/711379455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8101de0 0x7facc81a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.743+0000 7faccfe00640 1 -- 172.21.15.46:0/711379455 >> 172.21.15.46:0/711379455 conn(0x7facc80fd8c0 msgr2=0x7facc80fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.744+0000 7faccfe00640 1 -- 172.21.15.46:0/711379455 shutdown_connections 2024-09-10T07:42:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:07.744+0000 7faccfe00640 1 -- 172.21.15.46:0/711379455 wait complete. 2024-09-10T07:42:08.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:07 smithi181 ceph-mon[35397]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 MiB/s wr, 714 op/s 2024-09-10T07:42:08.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:07 smithi063 ceph-mon[35020]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 MiB/s wr, 714 op/s 2024-09-10T07:42:08.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:07 smithi046 ceph-mon[30705]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 MiB/s wr, 714 op/s 2024-09-10T07:42:08.869 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:42:08.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:42:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:10.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:09 smithi046 ceph-mon[30705]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-09-10T07:42:10.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:09 smithi181 ceph-mon[35397]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-09-10T07:42:10.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:09 smithi063 ceph-mon[35020]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-09-10T07:42:10.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.386+0000 7f185c600640 1 -- 172.21.15.46:0/2843316147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1854074620 msgr2=0x7f1854074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:10.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.387+0000 7f185c600640 1 --2- 172.21.15.46:0/2843316147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1854074620 0x7f1854074a00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f184c009be0 tx=0x7f184c02a750 comp rx=0 tx=0).stop 2024-09-10T07:42:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.389+0000 7f185c600640 1 -- 172.21.15.46:0/2843316147 shutdown_connections 2024-09-10T07:42:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.389+0000 7f185c600640 1 --2- 172.21.15.46:0/2843316147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f185410e3b0 0x7f1854110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.389+0000 7f185c600640 1 --2- 172.21.15.46:0/2843316147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1854074f40 0x7f185410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.389+0000 7f185c600640 1 --2- 172.21.15.46:0/2843316147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1854074620 0x7f1854074a00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.389+0000 7f185c600640 1 -- 172.21.15.46:0/2843316147 >> 172.21.15.46:0/2843316147 conn(0x7f18540fd890 msgr2=0x7f18540ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.389+0000 7f185c600640 1 -- 172.21.15.46:0/2843316147 shutdown_connections 2024-09-10T07:42:10.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.389+0000 7f185c600640 1 -- 172.21.15.46:0/2843316147 wait complete. 2024-09-10T07:42:10.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.391+0000 7f185c600640 1 Processor -- start 2024-09-10T07:42:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.391+0000 7f185c600640 1 -- start start 2024-09-10T07:42:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.391+0000 7f185c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1854074620 0x7f1854109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.391+0000 7f185c600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1854074f40 0x7f18541095b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.391+0000 7f185c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185410e3b0 0x7f1854104190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.391+0000 7f185c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1854102d20 con 0x7f1854074620 2024-09-10T07:42:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.391+0000 7f185c600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1854102ba0 con 0x7f1854074f40 2024-09-10T07:42:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.391+0000 7f185c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1854102ea0 con 0x7f185410e3b0 2024-09-10T07:42:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.392+0000 7f185a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185410e3b0 0x7f1854104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.392+0000 7f185a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185410e3b0 0x7f1854104190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37506/0 (socket says 172.21.15.46:37506) 2024-09-10T07:42:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.392+0000 7f185a600640 1 -- 172.21.15.46:0/143963453 learned_addr learned my addr 172.21.15.46:0/143963453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.392+0000 7f1859c00640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1854074620 0x7f1854109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.392+0000 7f1859200640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1854074f40 0x7f18541095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:10.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.393+0000 7f185a600640 1 -- 172.21.15.46:0/143963453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1854074f40 msgr2=0x7f18541095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:10.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.393+0000 7f185a600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1854074f40 0x7f18541095b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.393+0000 7f185a600640 1 -- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1854074620 msgr2=0x7f1854109070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:10.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.393+0000 7f185a600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1854074620 0x7f1854109070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.393+0000 7f185a600640 1 -- 172.21.15.46:0/143963453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1854104a50 con 0x7f185410e3b0 2024-09-10T07:42:10.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.393+0000 7f185a600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185410e3b0 0x7f1854104190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f184800e940 tx=0x7f184800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:10.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.394+0000 7f1852000640 1 -- 172.21.15.46:0/143963453 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800cd80 con 0x7f185410e3b0 2024-09-10T07:42:10.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.394+0000 7f1852000640 1 -- 172.21.15.46:0/143963453 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18480035f0 con 0x7f185410e3b0 2024-09-10T07:42:10.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.394+0000 7f1852000640 1 -- 172.21.15.46:0/143963453 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848010690 con 0x7f185410e3b0 2024-09-10T07:42:10.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.394+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1854104d40 con 0x7f185410e3b0 2024-09-10T07:42:10.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.394+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185410cf50 con 0x7f185410e3b0 2024-09-10T07:42:10.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.396+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1834005290 con 0x7f185410e3b0 2024-09-10T07:42:10.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.400+0000 7f1852000640 1 -- 172.21.15.46:0/143963453 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1848004030 con 0x7f185410e3b0 2024-09-10T07:42:10.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.402+0000 7f1852000640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f183c0a2cc0 0x7f183c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:10.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.403+0000 7f1852000640 1 -- 172.21.15.46:0/143963453 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f18480cf700 con 0x7f185410e3b0 2024-09-10T07:42:10.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.403+0000 7f1859c00640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f183c0a2cc0 0x7f183c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:10.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.404+0000 7f1859c00640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f183c0a2cc0 0x7f183c0a5180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f184c000f80 tx=0x7f184c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:10.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.405+0000 7f1852000640 1 -- 172.21.15.46:0/143963453 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f184809c280 con 0x7f185410e3b0 2024-09-10T07:42:10.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 290.0 s, 751.8 tps, lat 127.295 ms stddev 213.539 2024-09-10T07:42:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.824+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1834005580 con 0x7f185410e3b0 2024-09-10T07:42:10.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.825+0000 7f1852000640 1 -- 172.21.15.46:0/143963453 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11975 (secure 0 0 0) 0x7f184809bb80 con 0x7f185410e3b0 2024-09-10T07:42:10.826 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:42:10.826 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"btime":"2024-09-10T07:42:03:230374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:42:02.499291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:42:10.826 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2024-09-10T07:42:10.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.829+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f183c0a2cc0 msgr2=0x7f183c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:10.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.829+0000 7f185c600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f183c0a2cc0 0x7f183c0a5180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f184c000f80 tx=0x7f184c005e50 comp rx=0 tx=0).stop 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.829+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185410e3b0 msgr2=0x7f1854104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.829+0000 7f185c600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185410e3b0 0x7f1854104190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f184800e940 tx=0x7f184800ee00 comp rx=0 tx=0).stop 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.830+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 shutdown_connections 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.830+0000 7f185c600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185410e3b0 0x7f1854104190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.830+0000 7f185c600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1854074f40 0x7f18541095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.830+0000 7f185c600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f183c0a2cc0 0x7f183c0a5180 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.830+0000 7f185c600640 1 --2- 172.21.15.46:0/143963453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1854074620 0x7f1854109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.830+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 >> 172.21.15.46:0/143963453 conn(0x7f18540fd890 msgr2=0x7f18540ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.830+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 shutdown_connections 2024-09-10T07:42:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:10.830+0000 7f185c600640 1 -- 172.21.15.46:0/143963453 wait complete. 2024-09-10T07:42:11.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:11 smithi046 ceph-mon[30705]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 606 op/s 2024-09-10T07:42:11.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:11 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/143963453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:42:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:11 smithi181 ceph-mon[35397]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 606 op/s 2024-09-10T07:42:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:11 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/143963453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:42:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:11 smithi063 ceph-mon[35020]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 606 op/s 2024-09-10T07:42:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:11 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/143963453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:42:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.578+0000 7fb16fc00640 1 -- 172.21.15.46:0/3487546668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb168104cf0 msgr2=0x7fb1681070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.578+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3487546668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb168104cf0 0x7fb1681070e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb154009900 tx=0x7fb15402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.579+0000 7fb16fc00640 1 -- 172.21.15.46:0/3487546668 shutdown_connections 2024-09-10T07:42:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.579+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3487546668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168107620 0x7fb168109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.579+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3487546668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb168104cf0 0x7fb1681070e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.579+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3487546668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1681023c0 0x7fb1681047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.579+0000 7fb16fc00640 1 -- 172.21.15.46:0/3487546668 >> 172.21.15.46:0/3487546668 conn(0x7fb1680fbfa0 msgr2=0x7fb1680fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.580+0000 7fb16fc00640 1 -- 172.21.15.46:0/3487546668 shutdown_connections 2024-09-10T07:42:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.580+0000 7fb16fc00640 1 -- 172.21.15.46:0/3487546668 wait complete. 2024-09-10T07:42:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.582+0000 7fb16fc00640 1 Processor -- start 2024-09-10T07:42:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.584+0000 7fb16fc00640 1 -- start start 2024-09-10T07:42:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.584+0000 7fb16fc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1681023c0 0x7fb168195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.584+0000 7fb16fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168104cf0 0x7fb1681961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.585+0000 7fb16fc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168107620 0x7fb16819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.585+0000 7fb167e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168104cf0 0x7fb1681961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.585+0000 7fb167e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168104cf0 0x7fb1681961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48056/0 (socket says 172.21.15.46:48056) 2024-09-10T07:42:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.585+0000 7fb16d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1681023c0 0x7fb168195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.585+0000 7fb16d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1681023c0 0x7fb168195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49972/0 (socket says 172.21.15.46:49972) 2024-09-10T07:42:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.585+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168107620 0x7fb16819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.585+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168107620 0x7fb16819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37528/0 (socket says 172.21.15.46:37528) 2024-09-10T07:42:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.585+0000 7fb167e00640 1 -- 172.21.15.46:0/3172990008 learned_addr learned my addr 172.21.15.46:0/3172990008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.586+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb16810b510 con 0x7fb168104cf0 2024-09-10T07:42:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.586+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb16810b390 con 0x7fb1681023c0 2024-09-10T07:42:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.586+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb16810b690 con 0x7fb168107620 2024-09-10T07:42:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.587+0000 7fb16d200640 1 -- 172.21.15.46:0/3172990008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168107620 msgr2=0x7fb16819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.587+0000 7fb16d200640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168107620 0x7fb16819d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.587+0000 7fb16d200640 1 -- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168104cf0 msgr2=0x7fb1681961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.587+0000 7fb16d200640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168104cf0 0x7fb1681961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.587+0000 7fb16d200640 1 -- 172.21.15.46:0/3172990008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16819d870 con 0x7fb1681023c0 2024-09-10T07:42:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.587+0000 7fb16dc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168107620 0x7fb16819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.588+0000 7fb16d200640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1681023c0 0x7fb168195c60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b3f0 tx=0x7fb15c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.589+0000 7fb165600640 1 -- 172.21.15.46:0/3172990008 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c015020 con 0x7fb1681023c0 2024-09-10T07:42:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.589+0000 7fb165600640 1 -- 172.21.15.46:0/3172990008 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c011d10 con 0x7fb1681023c0 2024-09-10T07:42:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.589+0000 7fb165600640 1 -- 172.21.15.46:0/3172990008 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00f2e0 con 0x7fb1681023c0 2024-09-10T07:42:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.589+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16819db60 con 0x7fb1681023c0 2024-09-10T07:42:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.589+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16819e060 con 0x7fb1681023c0 2024-09-10T07:42:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.595+0000 7fb1729be740 1 Processor -- start 2024-09-10T07:42:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.598+0000 7fb165600640 1 -- 172.21.15.46:0/3172990008 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb15c018020 con 0x7fb1681023c0 2024-09-10T07:42:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.598+0000 7fb1729be740 1 -- start start 2024-09-10T07:42:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.598+0000 7fb1729be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8e99e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.599+0000 7fb1729be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9a3b0 0x5574c8e9ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.599+0000 7fb1729be740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8ea1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.599+0000 7fb1729be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5574c8df6100 con 0x5574c8e99a70 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.599+0000 7fb1729be740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5574c8d9e1b0 con 0x5574c8e9f440 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.599+0000 7fb1729be740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5574c8df20c0 con 0x5574c8e9a3b0 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.599+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8e99e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.599+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8e99e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48070/0 (socket says 172.21.15.46:48070) 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.599+0000 7fb16dc00640 1 -- 172.21.15.46:0/26686401 learned_addr learned my addr 172.21.15.46:0/26686401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.600+0000 7fb16d200640 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9a3b0 0x5574c8e9ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.600+0000 7fb165600640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1500a2c00 0x7fb1500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.600+0000 7fb167e00640 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8ea1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.601+0000 7fb167e00640 1 -- 172.21.15.46:0/26686401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9a3b0 msgr2=0x5574c8e9ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.601+0000 7fb167e00640 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9a3b0 0x5574c8e9ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.601+0000 7fb167e00640 1 -- 172.21.15.46:0/26686401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 msgr2=0x5574c8e99e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.601+0000 7fb167e00640 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8e99e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.601+0000 7fb167e00640 1 -- 172.21.15.46:0/26686401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c8b09210 con 0x5574c8e9f440 2024-09-10T07:42:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.601+0000 7fb16dc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1500a2c00 0x7fb1500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.602+0000 7fb167e00640 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8ea1900 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb154006010 tx=0x7fb1540043d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb40420be29ebf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.603+0000 7fb15a000640 1 -- 172.21.15.46:0/26686401 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154040070 con 0x5574c8e9f440 2024-09-10T07:42:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.603+0000 7fb15a000640 1 -- 172.21.15.46:0/26686401 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1540047a0 con 0x5574c8e9f440 2024-09-10T07:42:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.603+0000 7fb15a000640 1 -- 172.21.15.46:0/26686401 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154035040 con 0x5574c8e9f440 2024-09-10T07:42:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.605+0000 7fb1729be740 1 -- 172.21.15.46:0/26686401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 msgr2=0x5574c8ea1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.605+0000 7fb1729be740 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8ea1900 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb154006010 tx=0x7fb1540043d0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.607+0000 7fb165600640 1 -- 172.21.15.46:0/3172990008 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb15c0cebb0 con 0x7fb1681023c0 2024-09-10T07:42:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.607+0000 7fb16dc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1500a2c00 0x7fb1500a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb160000b60 tx=0x7fb16000a450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.607+0000 7fb1729be740 1 -- 172.21.15.46:0/26686401 shutdown_connections 2024-09-10T07:42:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.607+0000 7fb1729be740 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8ea1900 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.607+0000 7fb1729be740 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9a3b0 0x5574c8e9ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.607+0000 7fb1729be740 1 --2- 172.21.15.46:0/26686401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8e99e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.607+0000 7fb1729be740 1 -- 172.21.15.46:0/26686401 >> 172.21.15.46:0/26686401 conn(0x5574c8e8c8d0 msgr2=0x5574c8e8ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.608+0000 7fb1729be740 1 -- 172.21.15.46:0/26686401 shutdown_connections 2024-09-10T07:42:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.608+0000 7fb1729be740 1 -- 172.21.15.46:0/26686401 wait complete. 2024-09-10T07:42:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.609+0000 7fb1729be740 1 -- start start 2024-09-10T07:42:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.609+0000 7fb1729be740 1 Processor -- start 2024-09-10T07:42:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.610+0000 7fb1729be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb1729be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb1729be740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb1729be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5574c8e91280 con 0x5574c8e9a3b0 2024-09-10T07:42:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb1729be740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5574c8d7f090 con 0x5574c8e9f440 2024-09-10T07:42:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb1729be740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5574c8e91400 con 0x5574c8e99a70 2024-09-10T07:42:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb1729be740 10 client.0 _ll_register_callbacks cb 0x5574c8dd3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb16d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb16d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37546/0 (socket says 172.21.15.46:37546) 2024-09-10T07:42:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49988/0 (socket says 172.21.15.46:49988) 2024-09-10T07:42:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.611+0000 7fb16d200640 1 -- 172.21.15.46:0/3968129413 learned_addr learned my addr 172.21.15.46:0/3968129413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.612+0000 7fb167e00640 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.612+0000 7fb16d200640 1 -- 172.21.15.46:0/3968129413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 msgr2=0x5574c8e980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.612+0000 7fb16d200640 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.612+0000 7fb16d200640 1 -- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 msgr2=0x5574c8e930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.612+0000 7fb16d200640 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.612+0000 7fb16d200640 1 -- 172.21.15.46:0/3968129413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c8bd5620 con 0x5574c8e99a70 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.613+0000 7fb167e00640 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.614+0000 7fb16d200640 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92b80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb15c09b250 tx=0x7fb15c010440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c13379d33314d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.615+0000 7fb14ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.615+0000 7fb14ea00640 1 -- 172.21.15.46:0/3968129413 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c009200 con 0x5574c8e99a70 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.615+0000 7fb14ea00640 1 -- 172.21.15.46:0/3968129413 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb15c011830 con 0x5574c8e99a70 2024-09-10T07:42:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.616+0000 7fb1729be740 1 -- 172.21.15.46:0/3968129413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1540047a0 con 0x5574c8e99a70 2024-09-10T07:42:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.616+0000 7fb14ea00640 1 -- 172.21.15.46:0/3968129413 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb15c09bc50 con 0x5574c8e99a70 2024-09-10T07:42:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.616+0000 7fb1729be740 10 client.26722 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.616+0000 7fb1729be740 1 -- 172.21.15.46:0/3968129413 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5574c8ea57d0 con 0x5574c8e99a70 2024-09-10T07:42:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.617+0000 7fb14ea00640 1 -- 172.21.15.46:0/3968129413 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1540047a0 con 0x5574c8e99a70 2024-09-10T07:42:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.619+0000 7fb14ea00640 1 -- 172.21.15.46:0/3968129413 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fb15c09a9a0 con 0x5574c8e99a70 2024-09-10T07:42:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.620+0000 7fb1729be740 10 client.26722 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.620+0000 7fb1729be740 10 client.26722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.620+0000 7fb1729be740 20 client.26722 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.620+0000 7fb1729be740 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574c8ea6ea0 0x5574c8ea92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.620+0000 7fb1729be740 4 client.26722 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.620+0000 7fb1729be740 1 -- 172.21.15.46:0/3968129413 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c8e03830 con 0x5574c8ea6ea0 2024-09-10T07:42:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.622+0000 7fb167e00640 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574c8ea6ea0 0x5574c8ea92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.623+0000 7fb167e00640 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574c8ea6ea0 0x5574c8ea92b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed80bc2d87986d7 server_cookie=f270737d9fa0f62 in_seq=0 out_seq=0 2024-09-10T07:42:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.623+0000 7fb14ea00640 10 client.26722 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.624+0000 7fb14ea00640 1 -- 172.21.15.46:0/3968129413 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574c8e03830 con 0x5574c8ea6ea0 2024-09-10T07:42:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.624+0000 7fb14ea00640 10 client.26722 handle_command_reply: tid=0 2024-09-10T07:42:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.625+0000 7fb1729be740 1 client.26722 shutdown 2024-09-10T07:42:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 -- 172.21.15.46:0/3968129413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 msgr2=0x5574c8e92b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92b80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb15c09b250 tx=0x7fb15c010440 comp rx=0 tx=0).stop 2024-09-10T07:42:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 -- 172.21.15.46:0/3968129413 shutdown_connections 2024-09-10T07:42:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574c8ea6ea0 0x5574c8ea92b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 --2- 172.21.15.46:0/3968129413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92b80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb14ea00640 0 client.26722 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 -- 172.21.15.46:0/3968129413 >> 172.21.15.46:0/3968129413 conn(0x5574c8e8c8d0 msgr2=0x5574c8e8e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.626+0000 7fb1729be740 1 -- 172.21.15.46:0/3968129413 shutdown_connections 2024-09-10T07:42:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.627+0000 7fb1729be740 1 -- 172.21.15.46:0/3968129413 wait complete. 2024-09-10T07:42:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.627+0000 7fb1729be740 20 client.26722 trim_cache size 0 max 16384 2024-09-10T07:42:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.663+0000 7fb1729be740 1 Processor -- start 2024-09-10T07:42:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.663+0000 7fb1729be740 1 -- start start 2024-09-10T07:42:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.663+0000 7fb1729be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.663+0000 7fb1729be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e92e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.663+0000 7fb1729be740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e8d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.663+0000 7fb1729be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a85d0 con 0x5574c8e9a3b0 2024-09-10T07:42:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.663+0000 7fb1729be740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681aa170 con 0x5574c8e9f440 2024-09-10T07:42:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.665+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e8d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.665+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e8d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49992/0 (socket says 172.21.15.46:49992) 2024-09-10T07:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.665+0000 7fb16dc00640 1 -- 172.21.15.46:0/3337595905 learned_addr learned my addr 172.21.15.46:0/3337595905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.665+0000 7fb167e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e92e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.665+0000 7fb1729be740 1 -- 172.21.15.46:0/3337595905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a8ca0 con 0x5574c8e99a70 2024-09-10T07:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.666+0000 7fb16d200640 1 --2- 172.21.15.46:0/3337595905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.666+0000 7fb167e00640 1 -- 172.21.15.46:0/3337595905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 msgr2=0x5574c8e92930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.666+0000 7fb167e00640 1 --2- 172.21.15.46:0/3337595905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.666+0000 7fb167e00640 1 -- 172.21.15.46:0/3337595905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 msgr2=0x5574c8e8d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.666+0000 7fb167e00640 1 --2- 172.21.15.46:0/3337595905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e8d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.666+0000 7fb167e00640 1 -- 172.21.15.46:0/3337595905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c8e8c6f0 con 0x5574c8e9a3b0 2024-09-10T07:42:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.667+0000 7fb167e00640 1 --2- 172.21.15.46:0/3337595905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e92e70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb154005ec0 tx=0x7fb154031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1080a2629b269f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.668+0000 7fb159600640 1 -- 172.21.15.46:0/3337595905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154040050 con 0x5574c8e9a3b0 2024-09-10T07:42:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.668+0000 7fb159600640 1 -- 172.21.15.46:0/3337595905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1540047a0 con 0x5574c8e9a3b0 2024-09-10T07:42:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.668+0000 7fb159600640 1 -- 172.21.15.46:0/3337595905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154035040 con 0x5574c8e9a3b0 2024-09-10T07:42:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.670+0000 7fb1729be740 1 -- 172.21.15.46:0/3337595905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 msgr2=0x5574c8e92e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.670+0000 7fb1729be740 1 --2- 172.21.15.46:0/3337595905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e92e70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb154005ec0 tx=0x7fb154031400 comp rx=0 tx=0).stop 2024-09-10T07:42:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.671+0000 7fb1729be740 1 -- 172.21.15.46:0/3337595905 shutdown_connections 2024-09-10T07:42:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.671+0000 7fb1729be740 1 --2- 172.21.15.46:0/3337595905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9f440 0x5574c8e8d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.671+0000 7fb1729be740 1 --2- 172.21.15.46:0/3337595905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e9a3b0 0x5574c8e92e70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.671+0000 7fb1729be740 1 --2- 172.21.15.46:0/3337595905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e99a70 0x5574c8e92930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.671+0000 7fb1729be740 1 -- 172.21.15.46:0/3337595905 >> 172.21.15.46:0/3337595905 conn(0x5574c8e8c8d0 msgr2=0x5574c8e876a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.673+0000 7fb1729be740 1 -- 172.21.15.46:0/3337595905 shutdown_connections 2024-09-10T07:42:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.673+0000 7fb1729be740 1 -- 172.21.15.46:0/3337595905 wait complete. 2024-09-10T07:42:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.674+0000 7fb1729be740 1 -- start start 2024-09-10T07:42:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.674+0000 7fb1729be740 1 Processor -- start 2024-09-10T07:42:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.675+0000 7fb1729be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8f0e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.675+0000 7fb1729be740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9a3b0 0x5574c8f0e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.675+0000 7fb1729be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9f440 0x5574c8eb98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.675+0000 7fb167e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9a3b0 0x5574c8f0e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb167e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9a3b0 0x5574c8f0e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50000/0 (socket says 172.21.15.46:50000) 2024-09-10T07:42:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb167e00640 1 -- 172.21.15.46:0/1905961638 learned_addr learned my addr 172.21.15.46:0/1905961638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb16dc00640 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9f440 0x5574c8eb98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a9a40 con 0x5574c8e99a70 2024-09-10T07:42:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a9370 con 0x5574c8e9a3b0 2024-09-10T07:42:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a65b0 con 0x5574c8e9f440 2024-09-10T07:42:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb16d200640 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8f0e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb16dc00640 1 -- 172.21.15.46:0/1905961638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9a3b0 msgr2=0x5574c8f0e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb16dc00640 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9a3b0 0x5574c8f0e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb16dc00640 1 -- 172.21.15.46:0/1905961638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 msgr2=0x5574c8f0e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb16dc00640 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8f0e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb16dc00640 1 -- 172.21.15.46:0/1905961638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c8eb9e10 con 0x5574c8e9f440 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb1729be740 10 client.0 _ll_register_callbacks cb 0x7fb15c010810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.676+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1540047a0 con 0x5574c8e9f440 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.677+0000 7fb16dc00640 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9f440 0x5574c8eb98d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb160010040 tx=0x7fb1600012e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341bd2d94cf0f81f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.677+0000 7fb14ea00640 10 client.26728 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.677+0000 7fb14ea00640 1 -- 172.21.15.46:0/1905961638 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16000e6d0 con 0x5574c8e9f440 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.677+0000 7fb14ea00640 1 -- 172.21.15.46:0/1905961638 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb16000e870 con 0x5574c8e9f440 2024-09-10T07:42:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.677+0000 7fb14ea00640 1 -- 172.21.15.46:0/1905961638 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb16000ea90 con 0x5574c8e9f440 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.677+0000 7fb1729be740 10 client.26728 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.677+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=42}) -- 0x5574c8ebb0c0 con 0x5574c8e9f440 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.678+0000 7fb14ea00640 1 -- 172.21.15.46:0/1905961638 <== mon.1 v2:172.21.15.181:3300/0 4 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fb16000ec30 con 0x5574c8e9f440 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.678+0000 7fb1729be740 10 client.26728 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.678+0000 7fb1729be740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.678+0000 7fb1729be740 20 client.26728 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.678+0000 7fb1729be740 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574c8f22cc0 0x5574c8f250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.678+0000 7fb1729be740 4 client.26728 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:42:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.678+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574c8dfdb80 con 0x5574c8f22cc0 2024-09-10T07:42:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.679+0000 7fb16d200640 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574c8f22cc0 0x5574c8f250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.680+0000 7fb16d200640 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574c8f22cc0 0x5574c8f250e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4422561143fb70f server_cookie=f5c5988a28308420 in_seq=0 out_seq=0 2024-09-10T07:42:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.680+0000 7fb14ea00640 10 client.26728 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.710+0000 7fb14ea00640 1 -- 172.21.15.46:0/1905961638 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574c8dfdb80 con 0x5574c8f22cc0 2024-09-10T07:42:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.710+0000 7fb14ea00640 10 client.26728 handle_command_reply: tid=0 2024-09-10T07:42:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.710+0000 7fb1729be740 1 client.26728 shutdown 2024-09-10T07:42:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.714+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9f440 msgr2=0x5574c8eb98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.714+0000 7fb1729be740 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9f440 0x5574c8eb98d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb160010040 tx=0x7fb1600012e0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.715+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 shutdown_connections 2024-09-10T07:42:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.715+0000 7fb1729be740 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5574c8f22cc0 0x5574c8f250e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.715+0000 7fb1729be740 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5574c8e9f440 0x5574c8eb98d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.715+0000 7fb1729be740 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574c8e9a3b0 0x5574c8f0e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.715+0000 7fb1729be740 1 --2- 172.21.15.46:0/1905961638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c8e99a70 0x5574c8f0e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.715+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 >> 172.21.15.46:0/1905961638 conn(0x5574c8e8c8d0 msgr2=0x5574c8e92720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.715+0000 7fb14ea00640 0 client.26728 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.716+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 shutdown_connections 2024-09-10T07:42:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.717+0000 7fb1729be740 1 -- 172.21.15.46:0/1905961638 wait complete. 2024-09-10T07:42:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.717+0000 7fb1729be740 20 client.26728 trim_cache size 0 max 16384 2024-09-10T07:42:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:42:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:42:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:42:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:42:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1500a2c00 msgr2=0x7fb1500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1500a2c00 0x7fb1500a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb160000b60 tx=0x7fb16000a450 comp rx=0 tx=0).stop 2024-09-10T07:42:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1681023c0 msgr2=0x7fb168195c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1681023c0 0x7fb168195c60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b3f0 tx=0x7fb15c00b8b0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 shutdown_connections 2024-09-10T07:42:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb168107620 0x7fb16819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1500a2c00 0x7fb1500a50c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168104cf0 0x7fb1681961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 --2- 172.21.15.46:0/3172990008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1681023c0 0x7fb168195c60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.719+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 >> 172.21.15.46:0/3172990008 conn(0x7fb1680fbfa0 msgr2=0x7fb168102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.721+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 shutdown_connections 2024-09-10T07:42:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:13.721+0000 7fb16fc00640 1 -- 172.21.15.46:0/3172990008 wait complete. 2024-09-10T07:42:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:13 smithi181 ceph-mon[35397]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-09-10T07:42:14.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:13 smithi063 ceph-mon[35020]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-09-10T07:42:14.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:13 smithi046 ceph-mon[30705]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-09-10T07:42:14.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:42:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 295.0 s, 730.6 tps, lat 139.828 ms stddev 198.110 2024-09-10T07:42:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:16.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:15 smithi046 ceph-mon[30705]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 502 op/s 2024-09-10T07:42:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:15 smithi181 ceph-mon[35397]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 502 op/s 2024-09-10T07:42:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:15 smithi063 ceph-mon[35020]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 502 op/s 2024-09-10T07:42:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 -- 172.21.15.46:0/2818103459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde98108e00 msgr2=0x7fde9810ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 --2- 172.21.15.46:0/2818103459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde98108e00 0x7fde9810ce00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fde80009900 tx=0x7fde8002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 -- 172.21.15.46:0/2818103459 shutdown_connections 2024-09-10T07:42:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 --2- 172.21.15.46:0/2818103459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde9810d530 0x7fde981150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 --2- 172.21.15.46:0/2818103459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde98108e00 0x7fde9810ce00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 --2- 172.21.15.46:0/2818103459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108450 0x7fde98108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 -- 172.21.15.46:0/2818103459 >> 172.21.15.46:0/2818103459 conn(0x7fde98077ba0 msgr2=0x7fde98079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 -- 172.21.15.46:0/2818103459 shutdown_connections 2024-09-10T07:42:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.447+0000 7fde9dc00640 1 -- 172.21.15.46:0/2818103459 wait complete. 2024-09-10T07:42:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.449+0000 7fde9dc00640 1 Processor -- start 2024-09-10T07:42:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.450+0000 7fde9dc00640 1 -- start start 2024-09-10T07:42:16.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108450 0x7fde981a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde9dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde98108e00 0x7fde981a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde9dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde9810d530 0x7fde9819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde9dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde981185a0 con 0x7fde98108450 2024-09-10T07:42:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde9dc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde98118420 con 0x7fde98108e00 2024-09-10T07:42:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde9dc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fde98118720 con 0x7fde9810d530 2024-09-10T07:42:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde97400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde9810d530 0x7fde9819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde97400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde9810d530 0x7fde9819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37106/0 (socket says 172.21.15.46:37106) 2024-09-10T07:42:16.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde97400640 1 -- 172.21.15.46:0/1776817995 learned_addr learned my addr 172.21.15.46:0/1776817995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde8e000640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde98108e00 0x7fde981a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.451+0000 7fde96a00640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108450 0x7fde981a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde97400640 1 -- 172.21.15.46:0/1776817995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde98108e00 msgr2=0x7fde981a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde97400640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde98108e00 0x7fde981a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde97400640 1 -- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108450 msgr2=0x7fde981a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde97400640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108450 0x7fde981a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde97400640 1 -- 172.21.15.46:0/1776817995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9819efe0 con 0x7fde9810d530 2024-09-10T07:42:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde97400640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde9810d530 0x7fde9819e8b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde9000e970 tx=0x7fde9000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde8fe00640 1 -- 172.21.15.46:0/1776817995 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9000ccc0 con 0x7fde9810d530 2024-09-10T07:42:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde9dc00640 1 -- 172.21.15.46:0/1776817995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9819f2d0 con 0x7fde9810d530 2024-09-10T07:42:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde8fe00640 1 -- 172.21.15.46:0/1776817995 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde9000ce60 con 0x7fde9810d530 2024-09-10T07:42:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde8fe00640 1 -- 172.21.15.46:0/1776817995 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90010430 con 0x7fde9810d530 2024-09-10T07:42:16.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.452+0000 7fde9dc00640 1 -- 172.21.15.46:0/1776817995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981ab5c0 con 0x7fde9810d530 2024-09-10T07:42:16.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.457+0000 7fdea07d6740 1 Processor -- start 2024-09-10T07:42:16.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.459+0000 7fde8fe00640 1 -- 172.21.15.46:0/1776817995 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde90005020 con 0x7fde9810d530 2024-09-10T07:42:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.460+0000 7fdea07d6740 1 -- start start 2024-09-10T07:42:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.460+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e1c70 0x5587641e2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.460+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e2590 0x5587641ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.460+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.460+0000 7fdea07d6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5587641483e0 con 0x5587641e1c70 2024-09-10T07:42:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.460+0000 7fdea07d6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5587640efd80 con 0x5587641e2590 2024-09-10T07:42:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.460+0000 7fdea07d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5587641443a0 con 0x5587641ece50 2024-09-10T07:42:16.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.461+0000 7fde96a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e2590 0x5587641ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.461+0000 7fde96a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e2590 0x5587641ec910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53248/0 (socket says 172.21.15.46:53248) 2024-09-10T07:42:16.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.461+0000 7fde96a00640 1 -- 172.21.15.46:0/4221298579 learned_addr learned my addr 172.21.15.46:0/4221298579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:16.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.461+0000 7fde8e000640 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.462+0000 7fde8e000640 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e1c70 0x5587641e2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.462+0000 7fde96a00640 1 -- 172.21.15.46:0/4221298579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 msgr2=0x5587641ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.462+0000 7fde96a00640 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.462+0000 7fde96a00640 1 -- 172.21.15.46:0/4221298579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e1c70 msgr2=0x5587641e2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.462+0000 7fde96a00640 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e1c70 0x5587641e2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.462+0000 7fde96a00640 1 -- 172.21.15.46:0/4221298579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876410eae0 con 0x5587641e2590 2024-09-10T07:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.463+0000 7fde8fe00640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.463+0000 7fde96a00640 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e2590 0x5587641ec910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fde8800ae50 tx=0x7fde8800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec629e8acba6783 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:16.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.463+0000 7fde8fe00640 1 -- 172.21.15.46:0/1776817995 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde900cf710 con 0x7fde9810d530 2024-09-10T07:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.463+0000 7fde8e000640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.463+0000 7fde87e00640 1 -- 172.21.15.46:0/4221298579 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8800fea0 con 0x5587641e2590 2024-09-10T07:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.463+0000 7fde87e00640 1 -- 172.21.15.46:0/4221298579 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde88004d20 con 0x5587641e2590 2024-09-10T07:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.463+0000 7fde87e00640 1 -- 172.21.15.46:0/4221298579 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880056c0 con 0x5587641e2590 2024-09-10T07:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 -- 172.21.15.46:0/4221298579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e2590 msgr2=0x5587641ec910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fde8e000640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fde80000c00 tx=0x7fde80005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e2590 0x5587641ec910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fde8800ae50 tx=0x7fde8800f730 comp rx=0 tx=0).stop 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 -- 172.21.15.46:0/4221298579 shutdown_connections 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e2590 0x5587641ec910 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 --2- 172.21.15.46:0/4221298579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e1c70 0x5587641e2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 -- 172.21.15.46:0/4221298579 >> 172.21.15.46:0/4221298579 conn(0x5587641de200 msgr2=0x5587641de5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 -- 172.21.15.46:0/4221298579 shutdown_connections 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.464+0000 7fdea07d6740 1 -- 172.21.15.46:0/4221298579 wait complete. 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.465+0000 7fdea07d6740 1 -- start start 2024-09-10T07:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.465+0000 7fdea07d6740 1 Processor -- start 2024-09-10T07:42:16.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.466+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x5587641e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.466+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fdea07d6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558764200a30 con 0x5587641e2590 2024-09-10T07:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fdea07d6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5587641030f0 con 0x5587641e1c70 2024-09-10T07:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fdea07d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558764200bb0 con 0x5587641ece50 2024-09-10T07:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fdea07d6740 10 client.0 _ll_register_callbacks cb 0x5587640eabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fde97400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fde97400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ea770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37134/0 (socket says 172.21.15.46:37134) 2024-09-10T07:42:16.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fde97400640 1 -- 172.21.15.46:0/1065957005 learned_addr learned my addr 172.21.15.46:0/1065957005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fde96a00640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x5587641e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fde8e000640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.467+0000 7fde97400640 1 -- 172.21.15.46:0/1065957005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 msgr2=0x5587641e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde97400640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x5587641e9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde97400640 1 -- 172.21.15.46:0/1065957005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 msgr2=0x5587641ea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:16.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde97400640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde97400640 1 -- 172.21.15.46:0/1065957005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558763eb93c0 con 0x5587641ece50 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde96a00640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x5587641e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde8e000640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde97400640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ea770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fde900a4330 tx=0x7fde9009c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4fe84c10f83e9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde84c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde84c00640 1 -- 172.21.15.46:0/1065957005 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900b2070 con 0x5587641ece50 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fde84c00640 1 -- 172.21.15.46:0/1065957005 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde90016df0 con 0x5587641ece50 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.468+0000 7fdea07d6740 1 -- 172.21.15.46:0/1065957005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde88004d20 con 0x5587641ece50 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.469+0000 7fde84c00640 1 -- 172.21.15.46:0/1065957005 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde900a4860 con 0x5587641ece50 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.469+0000 7fdea07d6740 10 client.26740 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.469+0000 7fdea07d6740 1 -- 172.21.15.46:0/1065957005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55876420ca80 con 0x5587641ece50 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.469+0000 7fde84c00640 1 -- 172.21.15.46:0/1065957005 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde88004d20 con 0x5587641ece50 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.469+0000 7fde84c00640 1 -- 172.21.15.46:0/1065957005 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fde9009b040 con 0x5587641ece50 2024-09-10T07:42:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.470+0000 7fdea07d6740 10 client.26740 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.470+0000 7fdea07d6740 10 client.26740 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:42:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.470+0000 7fdea07d6740 20 client.26740 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.470+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55876420de90 0x5587642102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.470+0000 7fdea07d6740 4 client.26740 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.470+0000 7fdea07d6740 1 -- 172.21.15.46:0/1065957005 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558764155950 con 0x55876420de90 2024-09-10T07:42:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.470+0000 7fde96a00640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55876420de90 0x5587642102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.471+0000 7fde96a00640 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55876420de90 0x5587642102c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cadae52ca68f64ee server_cookie=cbf0deff35c05464 in_seq=0 out_seq=0 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.471+0000 7fde84c00640 10 client.26740 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.471+0000 7fde84c00640 1 -- 172.21.15.46:0/1065957005 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558764155950 con 0x55876420de90 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.471+0000 7fde84c00640 10 client.26740 handle_command_reply: tid=0 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 client.26740 shutdown 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 -- 172.21.15.46:0/1065957005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 msgr2=0x5587641ea770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ea770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fde900a4330 tx=0x7fde9009c3b0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 -- 172.21.15.46:0/1065957005 shutdown_connections 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55876420de90 0x5587642102c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x5587641ea770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1065957005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x5587641e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 -- 172.21.15.46:0/1065957005 >> 172.21.15.46:0/1065957005 conn(0x5587641de200 msgr2=0x5587641e3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:16.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fde84c00640 0 client.26740 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:42:16.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.472+0000 7fdea07d6740 1 -- 172.21.15.46:0/1065957005 shutdown_connections 2024-09-10T07:42:16.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.473+0000 7fdea07d6740 1 -- 172.21.15.46:0/1065957005 wait complete. 2024-09-10T07:42:16.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.473+0000 7fdea07d6740 20 client.26740 trim_cache size 0 max 16384 2024-09-10T07:42:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.493+0000 7fdea07d6740 1 Processor -- start 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.494+0000 7fdea07d6740 1 -- start start 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.494+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641e1c70 0x5587641e4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.494+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.494+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641ece50 0x5587641ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.494+0000 7fdea07d6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde981acba0 con 0x5587641e2590 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.494+0000 7fdea07d6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ae740 con 0x5587641ece50 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.494+0000 7fdea07d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ad270 con 0x5587641e1c70 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde96a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641e1c70 0x5587641e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde96a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641e1c70 0x5587641e4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37144/0 (socket says 172.21.15.46:37144) 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde96a00640 1 -- 172.21.15.46:0/1927192850 learned_addr learned my addr 172.21.15.46:0/1927192850 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde97400640 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641ece50 0x5587641ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde8e000640 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde96a00640 1 -- 172.21.15.46:0/1927192850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641ece50 msgr2=0x5587641ebc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde96a00640 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641ece50 0x5587641ebc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde96a00640 1 -- 172.21.15.46:0/1927192850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 msgr2=0x5587641ea5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde96a00640 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde96a00640 1 -- 172.21.15.46:0/1927192850 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587641de030 con 0x5587641e1c70 2024-09-10T07:42:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.495+0000 7fde8e000640 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.496+0000 7fde96a00640 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641e1c70 0x5587641e4720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fde88005c40 tx=0x7fde8800abc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7c2f4c4a19efdc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.496+0000 7fde87400640 1 -- 172.21.15.46:0/1927192850 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88003b70 con 0x5587641e1c70 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.496+0000 7fde87400640 1 -- 172.21.15.46:0/1927192850 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde88003d10 con 0x5587641e1c70 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.496+0000 7fde87400640 1 -- 172.21.15.46:0/1927192850 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88005940 con 0x5587641e1c70 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 -- 172.21.15.46:0/1927192850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641e1c70 msgr2=0x5587641e4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641e1c70 0x5587641e4720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fde88005c40 tx=0x7fde8800abc0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 -- 172.21.15.46:0/1927192850 shutdown_connections 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641ece50 0x5587641ebc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x5587641ea5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 --2- 172.21.15.46:0/1927192850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641e1c70 0x5587641e4720 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 -- 172.21.15.46:0/1927192850 >> 172.21.15.46:0/1927192850 conn(0x5587641de200 msgr2=0x5587641eb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 -- 172.21.15.46:0/1927192850 shutdown_connections 2024-09-10T07:42:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.497+0000 7fdea07d6740 1 -- 172.21.15.46:0/1927192850 wait complete. 2024-09-10T07:42:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.498+0000 7fdea07d6740 1 -- start start 2024-09-10T07:42:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.499+0000 7fdea07d6740 1 Processor -- start 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.500+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x558764201540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.500+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x558764201a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.500+0000 7fdea07d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x558764270b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.500+0000 7fdea07d6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ae010 con 0x5587641e2590 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.500+0000 7fdea07d6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ad940 con 0x5587641e1c70 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.500+0000 7fdea07d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fde980737b0 con 0x5587641ece50 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.500+0000 7fdea07d6740 10 client.0 _ll_register_callbacks cb 0x7fde90005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.501+0000 7fde97400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x558764270b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.501+0000 7fde97400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x558764270b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37146/0 (socket says 172.21.15.46:37146) 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.501+0000 7fde97400640 1 -- 172.21.15.46:0/2291315854 learned_addr learned my addr 172.21.15.46:0/2291315854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.501+0000 7fde96a00640 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x558764201540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.502+0000 7fde8e000640 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x558764201a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.502+0000 7fde97400640 1 -- 172.21.15.46:0/2291315854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 msgr2=0x558764201540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.502+0000 7fde97400640 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x558764201540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.502+0000 7fde97400640 1 -- 172.21.15.46:0/2291315854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 msgr2=0x558764201a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.503+0000 7fde97400640 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x558764201a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.503+0000 7fde97400640 1 -- 172.21.15.46:0/2291315854 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558764271150 con 0x5587641ece50 2024-09-10T07:42:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.503+0000 7fde8e000640 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x558764201a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.503+0000 7fde97400640 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x558764270b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fde90098050 tx=0x7fde900a00f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0a18384f52b8ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.503+0000 7fde84c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.504+0000 7fde84c00640 1 -- 172.21.15.46:0/2291315854 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900b2070 con 0x5587641ece50 2024-09-10T07:42:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.504+0000 7fde84c00640 1 -- 172.21.15.46:0/2291315854 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fde90005310 con 0x5587641ece50 2024-09-10T07:42:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.504+0000 7fdea07d6740 1 -- 172.21.15.46:0/2291315854 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde88003d10 con 0x5587641ece50 2024-09-10T07:42:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.504+0000 7fde84c00640 1 -- 172.21.15.46:0/2291315854 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde900a9050 con 0x5587641ece50 2024-09-10T07:42:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.504+0000 7fdea07d6740 10 client.26746 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.504+0000 7fdea07d6740 1 -- 172.21.15.46:0/2291315854 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558764289510 con 0x5587641ece50 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.504+0000 7fde84c00640 1 -- 172.21.15.46:0/2291315854 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde88003d10 con 0x5587641ece50 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.505+0000 7fde84c00640 1 -- 172.21.15.46:0/2291315854 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fde90081020 con 0x5587641ece50 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.505+0000 7fdea07d6740 10 client.26746 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.505+0000 7fdea07d6740 10 client.26746 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.505+0000 7fdea07d6740 20 client.26746 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.505+0000 7fdea07d6740 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x558764289aa0 0x55876428be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.505+0000 7fdea07d6740 4 client.26746 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:42:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.505+0000 7fdea07d6740 1 -- 172.21.15.46:0/2291315854 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55876414fe60 con 0x558764289aa0 2024-09-10T07:42:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.505+0000 7fde96a00640 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x558764289aa0 0x55876428be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.506+0000 7fde96a00640 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x558764289aa0 0x55876428be90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98bd754499c85b84 server_cookie=7af90768fac8a5bc in_seq=0 out_seq=0 2024-09-10T07:42:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.506+0000 7fde84c00640 10 client.26746 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:42:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.506+0000 7fde84c00640 1 -- 172.21.15.46:0/2291315854 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55876414fe60 con 0x558764289aa0 2024-09-10T07:42:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.506+0000 7fde84c00640 10 client.26746 handle_command_reply: tid=0 2024-09-10T07:42:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.506+0000 7fdea07d6740 1 client.26746 shutdown 2024-09-10T07:42:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 -- 172.21.15.46:0/2291315854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 msgr2=0x558764270b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x558764270b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fde90098050 tx=0x7fde900a00f0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 -- 172.21.15.46:0/2291315854 shutdown_connections 2024-09-10T07:42:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x558764289aa0 0x55876428be90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587641ece50 0x558764270b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587641e2590 0x558764201a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 --2- 172.21.15.46:0/2291315854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587641e1c70 0x558764201540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 -- 172.21.15.46:0/2291315854 >> 172.21.15.46:0/2291315854 conn(0x5587641de200 msgr2=0x5587641dd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fde84c00640 0 client.26746 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 -- 172.21.15.46:0/2291315854 shutdown_connections 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.507+0000 7fdea07d6740 1 -- 172.21.15.46:0/2291315854 wait complete. 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.508+0000 7fdea07d6740 20 client.26746 trim_cache size 0 max 16384 2024-09-10T07:42:16.509 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:42:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 -- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde7c0a2c00 msgr2=0x7fde7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fde80000c00 tx=0x7fde80005a20 comp rx=0 tx=0).stop 2024-09-10T07:42:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 -- 172.21.15.46:0/1776817995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde9810d530 msgr2=0x7fde9819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde9810d530 0x7fde9819e8b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde9000e970 tx=0x7fde9000ee30 comp rx=0 tx=0).stop 2024-09-10T07:42:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 -- 172.21.15.46:0/1776817995 shutdown_connections 2024-09-10T07:42:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde9810d530 0x7fde9819e8b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde98108e00 0x7fde981a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 --2- 172.21.15.46:0/1776817995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108450 0x7fde981a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 -- 172.21.15.46:0/1776817995 >> 172.21.15.46:0/1776817995 conn(0x7fde98077ba0 msgr2=0x7fde98078e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.509+0000 7fde9dc00640 1 -- 172.21.15.46:0/1776817995 shutdown_connections 2024-09-10T07:42:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:16.510+0000 7fde9dc00640 1 -- 172.21.15.46:0/1776817995 wait complete. 2024-09-10T07:42:18.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:17 smithi181 ceph-mon[35397]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-10T07:42:18.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:17 smithi063 ceph-mon[35020]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-10T07:42:18.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:17 smithi046 ceph-mon[30705]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-10T07:42:20.086 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:19 smithi046 ceph-mon[30705]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-10T07:42:20.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:42:20.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:19 smithi181 ceph-mon[35397]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-10T07:42:20.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:19 smithi063 ceph-mon[35020]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-10T07:42:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 300.0 s, 701.4 tps, lat 142.833 ms stddev 206.326 2024-09-10T07:42:20.827 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.556+0000 7f0126a00640 1 -- 172.21.15.46:0/1498217961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120103fe0 msgr2=0x7f01201043c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.556+0000 7f0126a00640 1 --2- 172.21.15.46:0/1498217961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120103fe0 0x7f01201043c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0118009ca0 tx=0x7f011802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.556+0000 7f0126a00640 1 -- 172.21.15.46:0/1498217961 shutdown_connections 2024-09-10T07:42:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.556+0000 7f0126a00640 1 --2- 172.21.15.46:0/1498217961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0120109110 0x7f0120110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.556+0000 7f0126a00640 1 --2- 172.21.15.46:0/1498217961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120104990 0x7f01201089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.556+0000 7f0126a00640 1 --2- 172.21.15.46:0/1498217961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120103fe0 0x7f01201043c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.556+0000 7f0126a00640 1 -- 172.21.15.46:0/1498217961 >> 172.21.15.46:0/1498217961 conn(0x7f01200ffa20 msgr2=0x7f0120101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.558+0000 7f0126a00640 1 -- 172.21.15.46:0/1498217961 shutdown_connections 2024-09-10T07:42:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.558+0000 7f0126a00640 1 -- 172.21.15.46:0/1498217961 wait complete. 2024-09-10T07:42:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.559+0000 7f0126a00640 1 Processor -- start 2024-09-10T07:42:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.559+0000 7f0126a00640 1 -- start start 2024-09-10T07:42:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f0126a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120103fe0 0x7f0120110b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f0126a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0120104990 0x7f012010dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f0126a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120109110 0x7f012010e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f0126a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0120114160 con 0x7f0120103fe0 2024-09-10T07:42:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f0126a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0120113fe0 con 0x7f0120109110 2024-09-10T07:42:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f0126a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01201142e0 con 0x7f0120104990 2024-09-10T07:42:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f011f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0120104990 0x7f012010dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f011f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0120104990 0x7f012010dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37170/0 (socket says 172.21.15.46:37170) 2024-09-10T07:42:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.560+0000 7f011f400640 1 -- 172.21.15.46:0/1205417402 learned_addr learned my addr 172.21.15.46:0/1205417402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.561+0000 7f0124a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120109110 0x7f012010e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.561+0000 7f0124a00640 1 -- 172.21.15.46:0/1205417402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0120104990 msgr2=0x7f012010dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.561+0000 7f0124a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0120104990 0x7f012010dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.561+0000 7f0124a00640 1 -- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120103fe0 msgr2=0x7f0120110b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.561+0000 7f0124a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120103fe0 0x7f0120110b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.561+0000 7f0124a00640 1 -- 172.21.15.46:0/1205417402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012010ea20 con 0x7f0120109110 2024-09-10T07:42:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.561+0000 7f011f400640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0120104990 0x7f012010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.562+0000 7f0124a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120109110 0x7f012010e160 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e970 tx=0x7f010c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.562+0000 7f011cc00640 1 -- 172.21.15.46:0/1205417402 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00ccc0 con 0x7f0120109110 2024-09-10T07:42:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.562+0000 7f011cc00640 1 -- 172.21.15.46:0/1205417402 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c005020 con 0x7f0120109110 2024-09-10T07:42:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.562+0000 7f011cc00640 1 -- 172.21.15.46:0/1205417402 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c004a00 con 0x7f0120109110 2024-09-10T07:42:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.562+0000 7f0126a00640 1 -- 172.21.15.46:0/1205417402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01201a07a0 con 0x7f0120109110 2024-09-10T07:42:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.563+0000 7f0126a00640 1 -- 172.21.15.46:0/1205417402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201a0c30 con 0x7f0120109110 2024-09-10T07:42:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.572+0000 7f0129427740 1 Processor -- start 2024-09-10T07:42:21.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.574+0000 7f011cc00640 1 -- 172.21.15.46:0/1205417402 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f010c0051d0 con 0x7f0120109110 2024-09-10T07:42:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.578+0000 7f011cc00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01040a2cc0 0x7f01040a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.578+0000 7f011cc00640 1 -- 172.21.15.46:0/1205417402 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f010c0cf2f0 con 0x7f0120109110 2024-09-10T07:42:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.579+0000 7f011f400640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01040a2cc0 0x7f01040a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.580+0000 7f011f400640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01040a2cc0 0x7f01040a5180 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f012010f3c0 tx=0x7f0110005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.582+0000 7f0129427740 1 -- start start 2024-09-10T07:42:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.582+0000 7f0129427740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e8c70 0x5615888e9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.582+0000 7f0129427740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x5615888f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.582+0000 7f0129427740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.582+0000 7f0129427740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56158884f3e0 con 0x5615888e8c70 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.582+0000 7f0129427740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5615887f6d80 con 0x5615888f3e50 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.582+0000 7f0129427740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56158884b3a0 con 0x5615888e9590 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.583+0000 7f011f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x5615888f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.583+0000 7f0124a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.583+0000 7f0124a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53290/0 (socket says 172.21.15.46:53290) 2024-09-10T07:42:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.583+0000 7f0124a00640 1 -- 172.21.15.46:0/2478547146 learned_addr learned my addr 172.21.15.46:0/2478547146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.585+0000 7f0124a00640 1 -- 172.21.15.46:0/2478547146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 msgr2=0x5615888f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.585+0000 7f011fe00640 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e8c70 0x5615888e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.585+0000 7f0124a00640 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x5615888f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.585+0000 7f0124a00640 1 -- 172.21.15.46:0/2478547146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e8c70 msgr2=0x5615888e9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.586+0000 7f0124a00640 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e8c70 0x5615888e9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.586+0000 7f0124a00640 1 -- 172.21.15.46:0/2478547146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561588815ae0 con 0x5615888f3e50 2024-09-10T07:42:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.586+0000 7f011f400640 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x5615888f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.587+0000 7f011fe00640 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e8c70 0x5615888e9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.587+0000 7f0124a00640 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f6240 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f010c09b700 tx=0x7f010c09b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2a7903de342305e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.588+0000 7f0115200640 1 -- 172.21.15.46:0/2478547146 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c01c040 con 0x5615888f3e50 2024-09-10T07:42:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.588+0000 7f0115200640 1 -- 172.21.15.46:0/2478547146 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c09fb20 con 0x5615888f3e50 2024-09-10T07:42:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.588+0000 7f0115200640 1 -- 172.21.15.46:0/2478547146 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c0a3f50 con 0x5615888f3e50 2024-09-10T07:42:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.589+0000 7f0129427740 1 -- 172.21.15.46:0/2478547146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 msgr2=0x5615888f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.589+0000 7f0129427740 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f6240 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f010c09b700 tx=0x7f010c09b670 comp rx=0 tx=0).stop 2024-09-10T07:42:21.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.591+0000 7f0129427740 1 -- 172.21.15.46:0/2478547146 shutdown_connections 2024-09-10T07:42:21.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.591+0000 7f0129427740 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f6240 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.591+0000 7f0129427740 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x5615888f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.591+0000 7f0129427740 1 --2- 172.21.15.46:0/2478547146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e8c70 0x5615888e9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.591+0000 7f0129427740 1 -- 172.21.15.46:0/2478547146 >> 172.21.15.46:0/2478547146 conn(0x5615888e5200 msgr2=0x5615888e55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.592+0000 7f0129427740 1 -- 172.21.15.46:0/2478547146 shutdown_connections 2024-09-10T07:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.592+0000 7f0129427740 1 -- 172.21.15.46:0/2478547146 wait complete. 2024-09-10T07:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.592+0000 7f0129427740 1 -- start start 2024-09-10T07:42:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.593+0000 7f0129427740 1 Processor -- start 2024-09-10T07:42:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.593+0000 7f0129427740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e8c70 0x5615888f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0129427740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0129427740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0129427740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5615888f6ac0 con 0x5615888e9590 2024-09-10T07:42:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0129427740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56158880a0f0 con 0x5615888f3e50 2024-09-10T07:42:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0129427740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5615888f6c40 con 0x5615888e8c70 2024-09-10T07:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0124a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0124a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53292/0 (socket says 172.21.15.46:53292) 2024-09-10T07:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0124a00640 1 -- 172.21.15.46:0/1062639837 learned_addr learned my addr 172.21.15.46:0/1062639837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f0129427740 10 client.0 _ll_register_callbacks cb 0x5615887f1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f011fe00640 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e8c70 0x5615888f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.594+0000 7f011f400640 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.595+0000 7f011fe00640 1 -- 172.21.15.46:0/1062639837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 msgr2=0x5615888f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.595+0000 7f011fe00640 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.595+0000 7f011fe00640 1 -- 172.21.15.46:0/1062639837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 msgr2=0x5615888f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.595+0000 7f011fe00640 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.595+0000 7f011fe00640 1 -- 172.21.15.46:0/1062639837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615885c03c0 con 0x5615888e8c70 2024-09-10T07:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.597+0000 7f0124a00640 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.597+0000 7f011fe00640 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e8c70 0x5615888f0cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f011802aeb0 tx=0x7f011802f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae48ac741143f57e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.597+0000 7f0109600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.597+0000 7f0109600640 1 -- 172.21.15.46:0/1062639837 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118003b70 con 0x5615888e8c70 2024-09-10T07:42:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.597+0000 7f0109600640 1 -- 172.21.15.46:0/1062639837 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0118003d10 con 0x5615888e8c70 2024-09-10T07:42:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.598+0000 7f0129427740 1 -- 172.21.15.46:0/1062639837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010c09fb20 con 0x5615888e8c70 2024-09-10T07:42:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.598+0000 7f0109600640 1 -- 172.21.15.46:0/1062639837 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0118031450 con 0x5615888e8c70 2024-09-10T07:42:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0129427740 10 client.26758 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0129427740 1 -- 172.21.15.46:0/1062639837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5615889029f0 con 0x5615888e8c70 2024-09-10T07:42:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0109600640 1 -- 172.21.15.46:0/1062639837 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c09fb20 con 0x5615888e8c70 2024-09-10T07:42:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0109600640 1 -- 172.21.15.46:0/1062639837 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f011802f610 con 0x5615888e8c70 2024-09-10T07:42:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0129427740 10 client.26758 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0129427740 10 client.26758 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:42:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0129427740 20 client.26758 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0129427740 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x561588904260 0x561588906700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0129427740 4 client.26758 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.599+0000 7f0129427740 1 -- 172.21.15.46:0/1062639837 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56158885c950 con 0x561588904260 2024-09-10T07:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.600+0000 7f011f400640 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x561588904260 0x561588906700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.601+0000 7f0109600640 10 client.26758 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.601+0000 7f011f400640 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x561588904260 0x561588906700 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d6201a4452c1741 server_cookie=85dd4c6ef01739f1 in_seq=0 out_seq=0 2024-09-10T07:42:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.602+0000 7f0109600640 1 -- 172.21.15.46:0/1062639837 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56158885c950 con 0x561588904260 2024-09-10T07:42:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.602+0000 7f0109600640 10 client.26758 handle_command_reply: tid=0 2024-09-10T07:42:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.602+0000 7f0129427740 1 client.26758 shutdown 2024-09-10T07:42:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.602+0000 7f0129427740 1 -- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e8c70 msgr2=0x5615888f0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.602+0000 7f0129427740 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e8c70 0x5615888f0cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f011802aeb0 tx=0x7f011802f380 comp rx=0 tx=0).stop 2024-09-10T07:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0129427740 1 -- 172.21.15.46:0/1062639837 shutdown_connections 2024-09-10T07:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0129427740 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888f3e50 0x5615888f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0129427740 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0129427740 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x561588904260 0x561588906700 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0129427740 1 --2- 172.21.15.46:0/1062639837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e8c70 0x5615888f0cb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0109600640 0 client.26758 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:42:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0129427740 1 -- 172.21.15.46:0/1062639837 >> 172.21.15.46:0/1062639837 conn(0x5615888e5200 msgr2=0x5615888eaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0129427740 1 -- 172.21.15.46:0/1062639837 shutdown_connections 2024-09-10T07:42:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.603+0000 7f0129427740 1 -- 172.21.15.46:0/1062639837 wait complete. 2024-09-10T07:42:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.604+0000 7f0129427740 20 client.26758 trim_cache size 0 max 16384 2024-09-10T07:42:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.627+0000 7f0129427740 1 Processor -- start 2024-09-10T07:42:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.628+0000 7f0129427740 1 -- start start 2024-09-10T07:42:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.628+0000 7f0129427740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888eae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.628+0000 7f0129427740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x56158890c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.628+0000 7f0129427740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888f3e50 0x56158890c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.628+0000 7f0129427740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01201b5460 con 0x5615888f3e50 2024-09-10T07:42:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.628+0000 7f0129427740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01201b7000 con 0x5615888e8c70 2024-09-10T07:42:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.628+0000 7f0129427740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01201b5b30 con 0x5615888e9590 2024-09-10T07:42:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.629+0000 7f011fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.629+0000 7f011fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888eae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53294/0 (socket says 172.21.15.46:53294) 2024-09-10T07:42:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.629+0000 7f011fe00640 1 -- 172.21.15.46:0/2858513864 learned_addr learned my addr 172.21.15.46:0/2858513864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.629+0000 7f0124a00640 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888f3e50 0x56158890c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.630+0000 7f011f400640 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x56158890c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.630+0000 7f011f400640 1 -- 172.21.15.46:0/2858513864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 msgr2=0x5615888eae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.630+0000 7f011f400640 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888eae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.630+0000 7f011f400640 1 -- 172.21.15.46:0/2858513864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888f3e50 msgr2=0x56158890c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.630+0000 7f011f400640 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888f3e50 0x56158890c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.630+0000 7f011f400640 1 -- 172.21.15.46:0/2858513864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615888e5030 con 0x5615888e9590 2024-09-10T07:42:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.631+0000 7f011f400640 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x56158890c010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0110009f10 tx=0x7f0110014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c5a96b78d64dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.632+0000 7f010be00640 1 -- 172.21.15.46:0/2858513864 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011002daf0 con 0x5615888e9590 2024-09-10T07:42:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.632+0000 7f010be00640 1 -- 172.21.15.46:0/2858513864 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011002dc90 con 0x5615888e9590 2024-09-10T07:42:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.632+0000 7f010be00640 1 -- 172.21.15.46:0/2858513864 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110039610 con 0x5615888e9590 2024-09-10T07:42:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.633+0000 7f0129427740 1 -- 172.21.15.46:0/2858513864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 msgr2=0x56158890c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.633+0000 7f0129427740 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x56158890c010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0110009f10 tx=0x7f0110014000 comp rx=0 tx=0).stop 2024-09-10T07:42:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.633+0000 7f0129427740 1 -- 172.21.15.46:0/2858513864 shutdown_connections 2024-09-10T07:42:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.633+0000 7f0129427740 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888f3e50 0x56158890c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.633+0000 7f0129427740 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888e9590 0x56158890c010 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.633+0000 7f0129427740 1 --2- 172.21.15.46:0/2858513864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888eae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.633+0000 7f0129427740 1 -- 172.21.15.46:0/2858513864 >> 172.21.15.46:0/2858513864 conn(0x5615888e5200 msgr2=0x5615888ea710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.634+0000 7f0129427740 1 -- 172.21.15.46:0/2858513864 shutdown_connections 2024-09-10T07:42:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.634+0000 7f0129427740 1 -- 172.21.15.46:0/2858513864 wait complete. 2024-09-10T07:42:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.634+0000 7f0129427740 1 -- start start 2024-09-10T07:42:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.635+0000 7f0129427740 1 Processor -- start 2024-09-10T07:42:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.635+0000 7f0129427740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.635+0000 7f0129427740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888ee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.635+0000 7f0129427740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888f3e50 0x5615888ee740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.635+0000 7f0129427740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01201b68d0 con 0x5615888e9590 2024-09-10T07:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.635+0000 7f0129427740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01201b6200 con 0x5615888e8c70 2024-09-10T07:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.635+0000 7f0129427740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0120072a00 con 0x5615888f3e50 2024-09-10T07:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.635+0000 7f0129427740 10 client.0 _ll_register_callbacks cb 0x7f0118009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f011fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f011fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888edcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53310/0 (socket says 172.21.15.46:53310) 2024-09-10T07:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f011fe00640 1 -- 172.21.15.46:0/1218107677 learned_addr learned my addr 172.21.15.46:0/1218107677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f0124a00640 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888f3e50 0x5615888ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f011f400640 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f0124a00640 1 -- 172.21.15.46:0/1218107677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 msgr2=0x5615888edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f0124a00640 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f0124a00640 1 -- 172.21.15.46:0/1218107677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 msgr2=0x5615888ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f0124a00640 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888ee200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f0124a00640 1 -- 172.21.15.46:0/1218107677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615888e0770 con 0x5615888f3e50 2024-09-10T07:42:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.636+0000 7f011f400640 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.638+0000 7f0124a00640 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888f3e50 0x5615888ee740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f010c09b470 tx=0x7f010c09b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d54862cbb1a3cef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.638+0000 7f0109600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.639+0000 7f0109600640 1 -- 172.21.15.46:0/1218107677 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c01c040 con 0x5615888f3e50 2024-09-10T07:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.639+0000 7f0109600640 1 -- 172.21.15.46:0/1218107677 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f010c097de0 con 0x5615888f3e50 2024-09-10T07:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.639+0000 7f0129427740 1 -- 172.21.15.46:0/1218107677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f011002dc90 con 0x5615888f3e50 2024-09-10T07:42:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.639+0000 7f0109600640 1 -- 172.21.15.46:0/1218107677 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f010c09fb20 con 0x5615888f3e50 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.639+0000 7f0129427740 10 client.26764 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.639+0000 7f0129427740 1 -- 172.21.15.46:0/1218107677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56158896f0d0 con 0x5615888f3e50 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.640+0000 7f0109600640 1 -- 172.21.15.46:0/1218107677 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011002dc90 con 0x5615888f3e50 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.640+0000 7f0109600640 1 -- 172.21.15.46:0/1218107677 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f010c004ba0 con 0x5615888f3e50 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.640+0000 7f0129427740 10 client.26764 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.640+0000 7f0129427740 10 client.26764 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.640+0000 7f0129427740 20 client.26764 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.640+0000 7f0129427740 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x56158896f400 0x5615889717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.640+0000 7f0129427740 4 client.26764 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:42:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.640+0000 7f0129427740 1 -- 172.21.15.46:0/1218107677 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561588856e60 con 0x56158896f400 2024-09-10T07:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.641+0000 7f011fe00640 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x56158896f400 0x5615889717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.642+0000 7f011fe00640 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x56158896f400 0x5615889717f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fdbd3fa3b04927d1 server_cookie=22f24b8b96b2dff8 in_seq=0 out_seq=0 2024-09-10T07:42:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.642+0000 7f0109600640 10 client.26764 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.642+0000 7f0109600640 1 -- 172.21.15.46:0/1218107677 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561588856e60 con 0x56158896f400 2024-09-10T07:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.642+0000 7f0109600640 10 client.26764 handle_command_reply: tid=0 2024-09-10T07:42:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.642+0000 7f0129427740 1 client.26764 shutdown 2024-09-10T07:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 -- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888f3e50 msgr2=0x5615888ee740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888f3e50 0x5615888ee740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f010c09b470 tx=0x7f010c09b8f0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 -- 172.21.15.46:0/1218107677 shutdown_connections 2024-09-10T07:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x56158896f400 0x5615889717f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0109600640 0 client.26764 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:42:21.645 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615888f3e50 0x5615888ee740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615888e9590 0x5615888ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 --2- 172.21.15.46:0/1218107677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615888e8c70 0x5615888edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 -- 172.21.15.46:0/1218107677 >> 172.21.15.46:0/1218107677 conn(0x5615888e5200 msgr2=0x5615888e3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 -- 172.21.15.46:0/1218107677 shutdown_connections 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.643+0000 7f0129427740 1 -- 172.21.15.46:0/1218107677 wait complete. 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0129427740 20 client.26764 trim_cache size 0 max 16384 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 -- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01040a2cc0 msgr2=0x7f01040a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01040a2cc0 0x7f01040a5180 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f012010f3c0 tx=0x7f0110005f90 comp rx=0 tx=0).stop 2024-09-10T07:42:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 -- 172.21.15.46:0/1205417402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120109110 msgr2=0x7f012010e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120109110 0x7f012010e160 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e970 tx=0x7f010c00ee30 comp rx=0 tx=0).stop 2024-09-10T07:42:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 -- 172.21.15.46:0/1205417402 shutdown_connections 2024-09-10T07:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0120109110 0x7f012010e160 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0120104990 0x7f012010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f01040a2cc0 0x7f01040a5180 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 --2- 172.21.15.46:0/1205417402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120103fe0 0x7f0120110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.644+0000 7f0126a00640 1 -- 172.21.15.46:0/1205417402 >> 172.21.15.46:0/1205417402 conn(0x7f01200ffa20 msgr2=0x7f0120100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.645+0000 7f0126a00640 1 -- 172.21.15.46:0/1205417402 shutdown_connections 2024-09-10T07:42:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:21.645+0000 7f0126a00640 1 -- 172.21.15.46:0/1205417402 wait complete. 2024-09-10T07:42:21.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:21 smithi046 ceph-mon[30705]: pgmap v571: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 670 op/s 2024-09-10T07:42:22.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:21 smithi181 ceph-mon[35397]: pgmap v571: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 670 op/s 2024-09-10T07:42:22.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:21 smithi063 ceph-mon[35020]: pgmap v571: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 670 op/s 2024-09-10T07:42:22.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:42:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:24.104 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:23 smithi046 ceph-mon[30705]: pgmap v572: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 646 op/s 2024-09-10T07:42:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:23 smithi181 ceph-mon[35397]: pgmap v572: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 646 op/s 2024-09-10T07:42:24.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:23 smithi063 ceph-mon[35020]: pgmap v572: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 646 op/s 2024-09-10T07:42:25.004 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:42:25.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.517+0000 7f5c46600640 1 -- 172.21.15.46:0/2017426474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 msgr2=0x7f5c401043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.517+0000 7f5c46600640 1 --2- 172.21.15.46:0/2017426474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 0x7f5c401043a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002a440 comp rx=0 tx=0).stop 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.519+0000 7f5c46600640 1 -- 172.21.15.46:0/2017426474 shutdown_connections 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.519+0000 7f5c46600640 1 --2- 172.21.15.46:0/2017426474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c401090f0 0x7f5c40110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.519+0000 7f5c46600640 1 --2- 172.21.15.46:0/2017426474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c40104970 0x7f5c401089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.519+0000 7f5c46600640 1 --2- 172.21.15.46:0/2017426474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 0x7f5c401043a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.519+0000 7f5c46600640 1 -- 172.21.15.46:0/2017426474 >> 172.21.15.46:0/2017426474 conn(0x7f5c400ffa40 msgr2=0x7f5c40101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.519+0000 7f5c46600640 1 -- 172.21.15.46:0/2017426474 shutdown_connections 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.520+0000 7f5c46600640 1 -- 172.21.15.46:0/2017426474 wait complete. 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.521+0000 7f5c46600640 1 Processor -- start 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.521+0000 7f5c46600640 1 -- start start 2024-09-10T07:42:25.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.522+0000 7f5c46600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 0x7f5c40073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.522+0000 7f5c46600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c40104970 0x7f5c4006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.522+0000 7f5c46600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c401090f0 0x7f5c4006c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.522+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 0x7f5c40073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.522+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 0x7f5c40073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37208/0 (socket says 172.21.15.46:37208) 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.522+0000 7f5c3f400640 1 -- 172.21.15.46:0/1083630418 learned_addr learned my addr 172.21.15.46:0/1083630418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40114140 con 0x7f5c401090f0 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40113fc0 con 0x7f5c40104970 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401142c0 con 0x7f5c40103fc0 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c36a00640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c40104970 0x7f5c4006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c3fe00640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c401090f0 0x7f5c4006c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c3f400640 1 -- 172.21.15.46:0/1083630418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c40104970 msgr2=0x7f5c4006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c3f400640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c40104970 0x7f5c4006bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c3f400640 1 -- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c401090f0 msgr2=0x7f5c4006c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c3f400640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c401090f0 0x7f5c4006c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.523+0000 7f5c3f400640 1 -- 172.21.15.46:0/1083630418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4006ca40 con 0x7f5c40103fc0 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.524+0000 7f5c36a00640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c40104970 0x7f5c4006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.524+0000 7f5c3f400640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 0x7f5c40073d10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5c30033040 tx=0x7f5c300053d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.525+0000 7f5c3cc00640 1 -- 172.21.15.46:0/1083630418 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30041020 con 0x7f5c40103fc0 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.525+0000 7f5c3cc00640 1 -- 172.21.15.46:0/1083630418 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c3003dd70 con 0x7f5c40103fc0 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.525+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4019a1f0 con 0x7f5c40103fc0 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.525+0000 7f5c3cc00640 1 -- 172.21.15.46:0/1083630418 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30031c70 con 0x7f5c40103fc0 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.525+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4019a780 con 0x7f5c40103fc0 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.532+0000 7f5c491e5740 1 Processor -- start 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.533+0000 7f5c3cc00640 1 -- 172.21.15.46:0/1083630418 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c30044020 con 0x7f5c40103fc0 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.534+0000 7f5c491e5740 1 -- start start 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.536+0000 7f5c3cc00640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c240a2c80 0x7f5c240a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.536+0000 7f5c3cc00640 1 -- 172.21.15.46:0/1083630418 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5c300eff80 con 0x7f5c40103fc0 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.536+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90ee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.536+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa90f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.536+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 0x557fa90fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.536+0000 7f5c491e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fa904c3e0 con 0x557fa90f9660 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.536+0000 7f5c491e5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557fa8ff3d80 con 0x557fa90ee480 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.536+0000 7f5c491e5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557fa90483a0 con 0x557fa90eeda0 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90ee860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53340/0 (socket says 172.21.15.46:53340) 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c3f400640 1 -- 172.21.15.46:0/3607483450 learned_addr learned my addr 172.21.15.46:0/3607483450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa90f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c3fe00640 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 0x557fa90fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c36a00640 1 -- 172.21.15.46:0/3607483450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 msgr2=0x557fa90ee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90ee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c36a00640 1 -- 172.21.15.46:0/3607483450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 msgr2=0x557fa90fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.537+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 0x557fa90fba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.538+0000 7f5c36a00640 1 -- 172.21.15.46:0/3607483450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa9012ae0 con 0x557fa90eeda0 2024-09-10T07:42:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.539+0000 7f5c3f400640 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.539+0000 7f5c3fe00640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c240a2c80 0x7f5c240a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.539+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa90f9120 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5c4006d530 tx=0x7f5c28030300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=861ffe49247b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.539+0000 7f5c34c00640 1 -- 172.21.15.46:0/3607483450 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28038070 con 0x557fa90eeda0 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.540+0000 7f5c34c00640 1 -- 172.21.15.46:0/3607483450 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28005500 con 0x557fa90eeda0 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.540+0000 7f5c34c00640 1 -- 172.21.15.46:0/3607483450 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28033480 con 0x557fa90eeda0 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.542+0000 7f5c491e5740 1 -- 172.21.15.46:0/3607483450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 msgr2=0x557fa90f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.542+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa90f9120 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5c4006d530 tx=0x7f5c28030300 comp rx=0 tx=0).stop 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.543+0000 7f5c3fe00640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c240a2c80 0x7f5c240a5140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x557fa90ebd90 tx=0x7f5c38004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.545+0000 7f5c491e5740 1 -- 172.21.15.46:0/3607483450 shutdown_connections 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.545+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 0x557fa90fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.545+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa90f9120 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.545+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3607483450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.545+0000 7f5c491e5740 1 -- 172.21.15.46:0/3607483450 >> 172.21.15.46:0/3607483450 conn(0x557fa90eaa10 msgr2=0x557fa90eadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.545+0000 7f5c491e5740 1 -- 172.21.15.46:0/3607483450 shutdown_connections 2024-09-10T07:42:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.545+0000 7f5c491e5740 1 -- 172.21.15.46:0/3607483450 wait complete. 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.545+0000 7f5c491e5740 1 -- start start 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.546+0000 7f5c491e5740 1 Processor -- start 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 0x557fa90f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90f9660 0x557fa90f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c491e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fa90fc230 con 0x557fa90eeda0 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c491e5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557fa90070f0 con 0x557fa90ee480 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c491e5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557fa90fc3b0 con 0x557fa90f9660 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c491e5740 10 client.0 _ll_register_callbacks cb 0x557fa8feebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53354/0 (socket says 172.21.15.46:53354) 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c3f400640 1 -- 172.21.15.46:0/2660391855 learned_addr learned my addr 172.21.15.46:0/2660391855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.547+0000 7f5c36a00640 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 0x557fa90f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.548+0000 7f5c3f400640 1 -- 172.21.15.46:0/2660391855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90f9660 msgr2=0x557fa90f7020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.549+0000 7f5c3f400640 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90f9660 0x557fa90f7020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.549+0000 7f5c3f400640 1 -- 172.21.15.46:0/2660391855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 msgr2=0x557fa90f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.549+0000 7f5c3f400640 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 0x557fa90f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.549+0000 7f5c3f400640 1 -- 172.21.15.46:0/2660391855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa8dbd3c0 con 0x557fa90ee480 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.549+0000 7f5c36a00640 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 0x557fa90f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.550+0000 7f5c2d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.550+0000 7f5c3f400640 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90f65a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005d50 tx=0x7f5c30005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf155189251d653 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.550+0000 7f5c2d600640 1 -- 172.21.15.46:0/2660391855 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300d5060 con 0x557fa90ee480 2024-09-10T07:42:25.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.550+0000 7f5c2d600640 1 -- 172.21.15.46:0/2660391855 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5c300ed7b0 con 0x557fa90ee480 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.551+0000 7f5c491e5740 1 -- 172.21.15.46:0/2660391855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c28005500 con 0x557fa90ee480 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.551+0000 7f5c2d600640 1 -- 172.21.15.46:0/2660391855 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5c300da050 con 0x557fa90ee480 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.551+0000 7f5c491e5740 10 client.26798 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.552+0000 7f5c491e5740 1 -- 172.21.15.46:0/2660391855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557fa90fc230 con 0x557fa90ee480 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.552+0000 7f5c2d600640 1 -- 172.21.15.46:0/2660391855 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c300bc400 con 0x557fa90ee480 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.553+0000 7f5c2d600640 1 -- 172.21.15.46:0/2660391855 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f5c3004e060 con 0x557fa90ee480 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.554+0000 7f5c491e5740 10 client.26798 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.554+0000 7f5c491e5740 10 client.26798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.554+0000 7f5c491e5740 20 client.26798 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.554+0000 7f5c491e5740 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557fa90e30e0 0x557fa90e5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.554+0000 7f5c491e5740 4 client.26798 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.554+0000 7f5c491e5740 1 -- 172.21.15.46:0/2660391855 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5c28005500 con 0x557fa90e30e0 2024-09-10T07:42:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.554+0000 7f5c36a00640 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557fa90e30e0 0x557fa90e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.556+0000 7f5c2d600640 10 client.26798 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:42:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.556+0000 7f5c36a00640 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557fa90e30e0 0x557fa90e5510 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b61c15fd03619a1f server_cookie=27dd9ab1ec3b0a3 in_seq=0 out_seq=0 2024-09-10T07:42:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.557+0000 7f5c2d600640 1 -- 172.21.15.46:0/2660391855 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5c28005500 con 0x557fa90e30e0 2024-09-10T07:42:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.557+0000 7f5c2d600640 10 client.26798 handle_command_reply: tid=0 2024-09-10T07:42:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.557+0000 7f5c491e5740 1 client.26798 shutdown 2024-09-10T07:42:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 -- 172.21.15.46:0/2660391855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 msgr2=0x557fa90f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90f65a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005d50 tx=0x7f5c30005fd0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 -- 172.21.15.46:0/2660391855 shutdown_connections 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557fa90e30e0 0x557fa90e5510 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90f9660 0x557fa90f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 0x557fa90f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 --2- 172.21.15.46:0/2660391855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90ee480 0x557fa90f65a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 -- 172.21.15.46:0/2660391855 >> 172.21.15.46:0/2660391855 conn(0x557fa90eaa10 msgr2=0x557fa90e1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c2d600640 0 client.26798 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 -- 172.21.15.46:0/2660391855 shutdown_connections 2024-09-10T07:42:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.559+0000 7f5c491e5740 1 -- 172.21.15.46:0/2660391855 wait complete. 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.560+0000 7f5c491e5740 20 client.26798 trim_cache size 0 max 16384 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.581+0000 7f5c491e5740 1 Processor -- start 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.582+0000 7f5c491e5740 1 -- start start 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.582+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 0x557fa90f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.582+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90f9660 0x557fa90f8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.582+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa910d230 0x557fa910d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.582+0000 7f5c491e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401a0470 con 0x557fa90eeda0 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.582+0000 7f5c491e5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401a2010 con 0x557fa90f9660 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.582+0000 7f5c491e5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401a0b40 con 0x557fa910d230 2024-09-10T07:42:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c3fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa910d230 0x557fa910d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c36a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90f9660 0x557fa90f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c3fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa910d230 0x557fa910d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37222/0 (socket says 172.21.15.46:37222) 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c3fe00640 1 -- 172.21.15.46:0/3211564281 learned_addr learned my addr 172.21.15.46:0/3211564281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c36a00640 1 -- 172.21.15.46:0/3211564281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa910d230 msgr2=0x557fa910d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3211564281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa910d230 0x557fa910d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c36a00640 1 -- 172.21.15.46:0/3211564281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 msgr2=0x557fa90f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3211564281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 0x557fa90f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c36a00640 1 -- 172.21.15.46:0/3211564281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa90dd770 con 0x557fa90f9660 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.583+0000 7f5c3fe00640 1 --2- 172.21.15.46:0/3211564281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa910d230 0x557fa910d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.584+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3211564281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90f9660 0x557fa90f8510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000c00 tx=0x7f5c28004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f893f0268eaaefbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.584+0000 7f5c2fe00640 1 -- 172.21.15.46:0/3211564281 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28038040 con 0x557fa90f9660 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.584+0000 7f5c2fe00640 1 -- 172.21.15.46:0/3211564281 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28030e90 con 0x557fa90f9660 2024-09-10T07:42:25.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.584+0000 7f5c2fe00640 1 -- 172.21.15.46:0/3211564281 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280035f0 con 0x557fa90f9660 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.585+0000 7f5c491e5740 1 -- 172.21.15.46:0/3211564281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90f9660 msgr2=0x557fa90f8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.585+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3211564281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90f9660 0x557fa90f8510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000c00 tx=0x7f5c28004c60 comp rx=0 tx=0).stop 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.585+0000 7f5c491e5740 1 -- 172.21.15.46:0/3211564281 shutdown_connections 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.585+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3211564281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa910d230 0x557fa910d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.585+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3211564281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa90f9660 0x557fa90f8510 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.585+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3211564281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90eeda0 0x557fa90f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.585+0000 7f5c491e5740 1 -- 172.21.15.46:0/3211564281 >> 172.21.15.46:0/3211564281 conn(0x557fa90eaa10 msgr2=0x557fa90f7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.586+0000 7f5c491e5740 1 -- 172.21.15.46:0/3211564281 shutdown_connections 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.586+0000 7f5c491e5740 1 -- 172.21.15.46:0/3211564281 wait complete. 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.586+0000 7f5c491e5740 1 -- start start 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.587+0000 7f5c491e5740 1 Processor -- start 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.587+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa910a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.587+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 0x557fa910a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.587+0000 7f5c491e5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa910d230 0x557fa910ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.587+0000 7f5c491e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401a18e0 con 0x557fa90f9660 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.587+0000 7f5c491e5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401a1210 con 0x557fa910d230 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.587+0000 7f5c491e5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4019e470 con 0x557fa90eeda0 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.587+0000 7f5c491e5740 10 client.0 _ll_register_callbacks cb 0x7f5c300057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa910a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa910d230 0x557fa910ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 0x557fa910a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa910a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37228/0 (socket says 172.21.15.46:37228) 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa910d230 0x557fa910ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53364/0 (socket says 172.21.15.46:53364) 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 -- 172.21.15.46:0/3185280217 learned_addr learned my addr 172.21.15.46:0/3185280217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 -- 172.21.15.46:0/3185280217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa910d230 msgr2=0x557fa910ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa910d230 0x557fa910ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 -- 172.21.15.46:0/3185280217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 msgr2=0x557fa910a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 0x557fa910a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 -- 172.21.15.46:0/3185280217 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa916b500 con 0x557fa90eeda0 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.588+0000 7f5c3f400640 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa910a380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009850 tx=0x7f5c300ba910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde29d46fa002f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.589+0000 7f5c2d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.589+0000 7f5c2d600640 1 -- 172.21.15.46:0/3185280217 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300da050 con 0x557fa90eeda0 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.589+0000 7f5c2d600640 1 -- 172.21.15.46:0/3185280217 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5c300c4be0 con 0x557fa90eeda0 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.589+0000 7f5c491e5740 1 -- 172.21.15.46:0/3185280217 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c28030e90 con 0x557fa90eeda0 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.589+0000 7f5c2d600640 1 -- 172.21.15.46:0/3185280217 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5c300a3070 con 0x557fa90eeda0 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.590+0000 7f5c491e5740 10 client.26779 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.590+0000 7f5c491e5740 1 -- 172.21.15.46:0/3185280217 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557fa916b500 con 0x557fa90eeda0 2024-09-10T07:42:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.590+0000 7f5c2d600640 1 -- 172.21.15.46:0/3185280217 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28030e90 con 0x557fa90eeda0 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.590+0000 7f5c2d600640 1 -- 172.21.15.46:0/3185280217 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f5c300edb60 con 0x557fa90eeda0 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.591+0000 7f5c491e5740 10 client.26779 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.591+0000 7f5c491e5740 10 client.26779 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.591+0000 7f5c491e5740 20 client.26779 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.591+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557fa916cb30 0x557fa916ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.591+0000 7f5c491e5740 4 client.26779 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.591+0000 7f5c491e5740 1 -- 172.21.15.46:0/3185280217 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fa9059950 con 0x557fa916cb30 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.592+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557fa916cb30 0x557fa916ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c36a00640 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557fa916cb30 0x557fa916ef20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d58ec6b2caf6744a server_cookie=166732e9c8e2ce46 in_seq=0 out_seq=0 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c2d600640 10 client.26779 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c2d600640 1 -- 172.21.15.46:0/3185280217 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fa9059950 con 0x557fa916cb30 2024-09-10T07:42:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c2d600640 10 client.26779 handle_command_reply: tid=0 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 client.26779 shutdown 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 -- 172.21.15.46:0/3185280217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 msgr2=0x557fa910a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa910a380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009850 tx=0x7f5c300ba910 comp rx=0 tx=0).stop 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 -- 172.21.15.46:0/3185280217 shutdown_connections 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557fa916cb30 0x557fa916ef20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fa910d230 0x557fa910ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fa90f9660 0x557fa910a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 --2- 172.21.15.46:0/3185280217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fa90eeda0 0x557fa910a380 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c491e5740 1 -- 172.21.15.46:0/3185280217 >> 172.21.15.46:0/3185280217 conn(0x557fa90eaa10 msgr2=0x557fa9104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.593+0000 7f5c2d600640 0 client.26779 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.594+0000 7f5c491e5740 1 -- 172.21.15.46:0/3185280217 shutdown_connections 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.594+0000 7f5c491e5740 1 -- 172.21.15.46:0/3185280217 wait complete. 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.594+0000 7f5c491e5740 20 client.26779 trim_cache size 0 max 16384 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c240a2c80 msgr2=0x7f5c240a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c240a2c80 0x7f5c240a5140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x557fa90ebd90 tx=0x7f5c38004ab0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 msgr2=0x7f5c40073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 0x7f5c40073d10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5c30033040 tx=0x7f5c300053d0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 shutdown_connections 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5c240a2c80 0x7f5c240a5140 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c401090f0 0x7f5c4006c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c40104970 0x7f5c4006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 --2- 172.21.15.46:0/1083630418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c40103fc0 0x7f5c40073d10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.595+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 >> 172.21.15.46:0/1083630418 conn(0x7f5c400ffa40 msgr2=0x7f5c40100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.597+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 shutdown_connections 2024-09-10T07:42:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:24.598+0000 7f5c46600640 1 -- 172.21.15.46:0/1083630418 wait complete. 2024-09-10T07:42:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 305.0 s, 812.0 tps, lat 120.345 ms stddev 155.289 2024-09-10T07:42:26.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:42:26.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:25 smithi181 ceph-mon[35397]: pgmap v573: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 556 op/s 2024-09-10T07:42:26.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:25 smithi063 ceph-mon[35020]: pgmap v573: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 556 op/s 2024-09-10T07:42:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:25 smithi046 ceph-mon[30705]: pgmap v573: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 556 op/s 2024-09-10T07:42:26.789 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:26.998+0000 7fce94c00640 1 -- 172.21.15.46:0/4286719745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90109d10 msgr2=0x7fce9010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:26.998+0000 7fce94c00640 1 --2- 172.21.15.46:0/4286719745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90109d10 0x7fce9010a5b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fce88009c10 tx=0x7fce8802a710 comp rx=0 tx=0).stop 2024-09-10T07:42:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:26.999+0000 7fce94c00640 1 -- 172.21.15.46:0/4286719745 shutdown_connections 2024-09-10T07:42:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:26.999+0000 7fce94c00640 1 --2- 172.21.15.46:0/4286719745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90109d10 0x7fce9010a5b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:26.999+0000 7fce94c00640 1 --2- 172.21.15.46:0/4286719745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce90109370 0x7fce901097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:26.999+0000 7fce94c00640 1 --2- 172.21.15.46:0/4286719745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce90073460 0x7fce90073860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:26.999+0000 7fce94c00640 1 -- 172.21.15.46:0/4286719745 >> 172.21.15.46:0/4286719745 conn(0x7fce9006ef20 msgr2=0x7fce90071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.000+0000 7fce94c00640 1 -- 172.21.15.46:0/4286719745 shutdown_connections 2024-09-10T07:42:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.000+0000 7fce94c00640 1 -- 172.21.15.46:0/4286719745 wait complete. 2024-09-10T07:42:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.001+0000 7fce94c00640 1 Processor -- start 2024-09-10T07:42:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.002+0000 7fce94c00640 1 -- start start 2024-09-10T07:42:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.002+0000 7fce94c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90073460 0x7fce901a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.002+0000 7fce94c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce90109370 0x7fce901a7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.002+0000 7fce94c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce90109d10 0x7fce901ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.002+0000 7fce94c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce901181f0 con 0x7fce90109370 2024-09-10T07:42:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.002+0000 7fce94c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fce90118070 con 0x7fce90073460 2024-09-10T07:42:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.002+0000 7fce94c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fce90118370 con 0x7fce90109d10 2024-09-10T07:42:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.003+0000 7fce8de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90073460 0x7fce901a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.003+0000 7fce8e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce90109d10 0x7fce901ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.003+0000 7fce8de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90073460 0x7fce901a6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57816/0 (socket says 172.21.15.46:57816) 2024-09-10T07:42:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.003+0000 7fce8de00640 1 -- 172.21.15.46:0/4275318944 learned_addr learned my addr 172.21.15.46:0/4275318944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.004+0000 7fce8d400640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce90109370 0x7fce901a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.005+0000 7fce8e800640 1 -- 172.21.15.46:0/4275318944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90073460 msgr2=0x7fce901a6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.005+0000 7fce8e800640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90073460 0x7fce901a6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.005+0000 7fce8e800640 1 -- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce90109370 msgr2=0x7fce901a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.005+0000 7fce8e800640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce90109370 0x7fce901a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.005+0000 7fce8e800640 1 -- 172.21.15.46:0/4275318944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce901ae7f0 con 0x7fce90109d10 2024-09-10T07:42:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.005+0000 7fce8e800640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce90109d10 0x7fce901ae180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fce88034040 tx=0x7fce88031670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.006+0000 7fce86a00640 1 -- 172.21.15.46:0/4275318944 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88041030 con 0x7fce90109d10 2024-09-10T07:42:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.006+0000 7fce86a00640 1 -- 172.21.15.46:0/4275318944 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce88003e50 con 0x7fce90109d10 2024-09-10T07:42:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.006+0000 7fce86a00640 1 -- 172.21.15.46:0/4275318944 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88036040 con 0x7fce90109d10 2024-09-10T07:42:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.006+0000 7fce94c00640 1 -- 172.21.15.46:0/4275318944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce901aea80 con 0x7fce90109d10 2024-09-10T07:42:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.006+0000 7fce94c00640 1 -- 172.21.15.46:0/4275318944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901aef90 con 0x7fce90109d10 2024-09-10T07:42:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.014+0000 7fce86a00640 1 -- 172.21.15.46:0/4275318944 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fce88044070 con 0x7fce90109d10 2024-09-10T07:42:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.015+0000 7fce97ac2740 1 Processor -- start 2024-09-10T07:42:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.017+0000 7fce86a00640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fce740a2cb0 0x7fce740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.017+0000 7fce86a00640 1 -- 172.21.15.46:0/4275318944 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fce880f01d0 con 0x7fce90109d10 2024-09-10T07:42:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.018+0000 7fce97ac2740 1 -- start start 2024-09-10T07:42:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.019+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcde2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.019+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcde7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.019+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcde9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.019+0000 7fce97ac2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5647dcd47100 con 0x5647dcde7a60 2024-09-10T07:42:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.019+0000 7fce97ac2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5647dccef1b0 con 0x5647dcde29d0 2024-09-10T07:42:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.019+0000 7fce97ac2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5647dcd430c0 con 0x5647dcde2090 2024-09-10T07:42:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.019+0000 7fce8de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcde2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.020+0000 7fce8d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcde7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.020+0000 7fce8d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcde7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57830/0 (socket says 172.21.15.46:57830) 2024-09-10T07:42:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.020+0000 7fce8d400640 1 -- 172.21.15.46:0/2132248099 learned_addr learned my addr 172.21.15.46:0/2132248099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.021+0000 7fce8d400640 1 -- 172.21.15.46:0/2132248099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 msgr2=0x5647dcde2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.021+0000 7fce8d400640 1 --2- 172.21.15.46:0/2132248099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcde2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.021+0000 7fce8d400640 1 -- 172.21.15.46:0/2132248099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 msgr2=0x5647dcde9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.021+0000 7fce8d400640 1 --2- 172.21.15.46:0/2132248099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcde9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.021+0000 7fce8d400640 1 -- 172.21.15.46:0/2132248099 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647dca5a210 con 0x5647dcde29d0 2024-09-10T07:42:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.021+0000 7fce8d400640 1 --2- 172.21.15.46:0/2132248099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcde7490 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009d00 tx=0x7fce7c012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2671e36a644ba55f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.023+0000 7fce7b000640 1 -- 172.21.15.46:0/2132248099 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c01b640 con 0x5647dcde29d0 2024-09-10T07:42:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.023+0000 7fce8d400640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fce740a2cb0 0x7fce740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.023+0000 7fce7b000640 1 -- 172.21.15.46:0/2132248099 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7c00f910 con 0x5647dcde29d0 2024-09-10T07:42:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.023+0000 7fce7b000640 1 -- 172.21.15.46:0/2132248099 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c01a870 con 0x5647dcde29d0 2024-09-10T07:42:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.024+0000 7fce97ac2740 1 -- 172.21.15.46:0/2132248099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 msgr2=0x5647dcde7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.024+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2132248099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcde7490 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009d00 tx=0x7fce7c012820 comp rx=0 tx=0).stop 2024-09-10T07:42:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.025+0000 7fce97ac2740 1 -- 172.21.15.46:0/2132248099 shutdown_connections 2024-09-10T07:42:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.025+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2132248099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcde9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.025+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2132248099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcde7490 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.025+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2132248099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcde2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.025+0000 7fce97ac2740 1 -- 172.21.15.46:0/2132248099 >> 172.21.15.46:0/2132248099 conn(0x5647dcddd700 msgr2=0x5647dcdddb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.026+0000 7fce97ac2740 1 -- 172.21.15.46:0/2132248099 shutdown_connections 2024-09-10T07:42:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.026+0000 7fce97ac2740 1 -- 172.21.15.46:0/2132248099 wait complete. 2024-09-10T07:42:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.027+0000 7fce97ac2740 1 -- start start 2024-09-10T07:42:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.027+0000 7fce97ac2740 1 Processor -- start 2024-09-10T07:42:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.027+0000 7fce8d400640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fce740a2cb0 0x7fce740a5170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009d00 tx=0x7fce7c012820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcdec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcdf3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce97ac2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5647dcdeaa30 con 0x5647dcde7a60 2024-09-10T07:42:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce97ac2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5647dccd0090 con 0x5647dcde29d0 2024-09-10T07:42:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce97ac2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5647dcdeabb0 con 0x5647dcde2090 2024-09-10T07:42:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce97ac2740 10 client.0 _ll_register_callbacks cb 0x5647dcd24f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce8de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcdec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.028+0000 7fce8d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.029+0000 7fce8de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcdec2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57408/0 (socket says 172.21.15.46:57408) 2024-09-10T07:42:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.029+0000 7fce8de00640 1 -- 172.21.15.46:0/2767223567 learned_addr learned my addr 172.21.15.46:0/2767223567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.029+0000 7fce8e800640 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcdf3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.029+0000 7fce8de00640 1 -- 172.21.15.46:0/2767223567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 msgr2=0x5647dcdec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.029+0000 7fce8de00640 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.029+0000 7fce8de00640 1 -- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 msgr2=0x5647dcdf3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.029+0000 7fce8de00640 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcdf3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.029+0000 7fce8de00640 1 -- 172.21.15.46:0/2767223567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647dcb26620 con 0x5647dcde2090 2024-09-10T07:42:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.030+0000 7fce8d400640 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.030+0000 7fce8e800640 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcdf3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.030+0000 7fce8de00640 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcdec2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fce80012840 tx=0x7fce80012d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edb1f6d8c6c5ea08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.030+0000 7fce6f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.030+0000 7fce6f400640 1 -- 172.21.15.46:0/2767223567 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80010dc0 con 0x5647dcde2090 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.030+0000 7fce6f400640 1 -- 172.21.15.46:0/2767223567 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fce80005020 con 0x5647dcde2090 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.030+0000 7fce97ac2740 1 -- 172.21.15.46:0/2767223567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce7c00f910 con 0x5647dcde2090 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.031+0000 7fce6f400640 1 -- 172.21.15.46:0/2767223567 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fce80025050 con 0x5647dcde2090 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.031+0000 7fce97ac2740 10 client.26788 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.031+0000 7fce97ac2740 1 -- 172.21.15.46:0/2767223567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5647dcdf66f0 con 0x5647dcde2090 2024-09-10T07:42:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.031+0000 7fce6f400640 1 -- 172.21.15.46:0/2767223567 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7c00f910 con 0x5647dcde2090 2024-09-10T07:42:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.031+0000 7fce6f400640 1 -- 172.21.15.46:0/2767223567 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fce8001a430 con 0x5647dcde2090 2024-09-10T07:42:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.031+0000 7fce97ac2740 10 client.26788 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.032+0000 7fce97ac2740 10 client.26788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.032+0000 7fce97ac2740 20 client.26788 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.032+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5647dcdf7ef0 0x5647dcdfa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.032+0000 7fce97ac2740 4 client.26788 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.032+0000 7fce97ac2740 1 -- 172.21.15.46:0/2767223567 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647dcd54830 con 0x5647dcdf7ef0 2024-09-10T07:42:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.033+0000 7fce8d400640 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5647dcdf7ef0 0x5647dcdfa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.035+0000 7fce8d400640 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5647dcdf7ef0 0x5647dcdfa320 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69df3efdd7c5a29e server_cookie=2f481a7190606d1b in_seq=0 out_seq=0 2024-09-10T07:42:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.035+0000 7fce6f400640 10 client.26788 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.039+0000 7fce6f400640 1 -- 172.21.15.46:0/2767223567 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647dcd54830 con 0x5647dcdf7ef0 2024-09-10T07:42:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.039+0000 7fce6f400640 10 client.26788 handle_command_reply: tid=0 2024-09-10T07:42:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.039+0000 7fce97ac2740 1 client.26788 shutdown 2024-09-10T07:42:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.040+0000 7fce97ac2740 1 -- 172.21.15.46:0/2767223567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 msgr2=0x5647dcdec2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.040+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcdec2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fce80012840 tx=0x7fce80012d00 comp rx=0 tx=0).stop 2024-09-10T07:42:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce97ac2740 1 -- 172.21.15.46:0/2767223567 shutdown_connections 2024-09-10T07:42:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5647dcdf7ef0 0x5647dcdfa320 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcdf3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2767223567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde2090 0x5647dcdec2b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce6f400640 0 client.26788 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce97ac2740 1 -- 172.21.15.46:0/2767223567 >> 172.21.15.46:0/2767223567 conn(0x5647dcddd700 msgr2=0x5647dcddef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce97ac2740 1 -- 172.21.15.46:0/2767223567 shutdown_connections 2024-09-10T07:42:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.041+0000 7fce97ac2740 1 -- 172.21.15.46:0/2767223567 wait complete. 2024-09-10T07:42:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.042+0000 7fce97ac2740 20 client.26788 trim_cache size 0 max 16384 2024-09-10T07:42:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.068+0000 7fce97ac2740 1 Processor -- start 2024-09-10T07:42:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.068+0000 7fce97ac2740 1 -- start start 2024-09-10T07:42:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde2090 0x5647dcdec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdf37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde7a60 0x5647dcde0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce97ac2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b9440 con 0x5647dcde2090 2024-09-10T07:42:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce97ac2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fce901bafe0 con 0x5647dcde29d0 2024-09-10T07:42:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce97ac2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b9b10 con 0x5647dcde7a60 2024-09-10T07:42:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce8d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdf37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce8e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde7a60 0x5647dcde0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce8e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde7a60 0x5647dcde0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57412/0 (socket says 172.21.15.46:57412) 2024-09-10T07:42:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce8e800640 1 -- 172.21.15.46:0/1325138228 learned_addr learned my addr 172.21.15.46:0/1325138228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.069+0000 7fce8d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdf37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57852/0 (socket says 172.21.15.46:57852) 2024-09-10T07:42:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.070+0000 7fce8de00640 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde2090 0x5647dcdec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.070+0000 7fce8e800640 1 -- 172.21.15.46:0/1325138228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 msgr2=0x5647dcdf37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.070+0000 7fce8e800640 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdf37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.070+0000 7fce8e800640 1 -- 172.21.15.46:0/1325138228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde2090 msgr2=0x5647dcdec6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.070+0000 7fce8e800640 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde2090 0x5647dcdec6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.070+0000 7fce8e800640 1 -- 172.21.15.46:0/1325138228 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647dcdfa9a0 con 0x5647dcde7a60 2024-09-10T07:42:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.071+0000 7fce8d400640 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdf37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.071+0000 7fce8de00640 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde2090 0x5647dcdec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.071+0000 7fce8e800640 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde7a60 0x5647dcde0b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fce880bd210 tx=0x7fce8803c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c250cd648479245c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.072+0000 7fce7a600640 1 -- 172.21.15.46:0/1325138228 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce880bdc10 con 0x5647dcde7a60 2024-09-10T07:42:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.072+0000 7fce7a600640 1 -- 172.21.15.46:0/1325138228 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce880c18c0 con 0x5647dcde7a60 2024-09-10T07:42:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.072+0000 7fce7a600640 1 -- 172.21.15.46:0/1325138228 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce880b9b80 con 0x5647dcde7a60 2024-09-10T07:42:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.073+0000 7fce97ac2740 1 -- 172.21.15.46:0/1325138228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde7a60 msgr2=0x5647dcde0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.073+0000 7fce97ac2740 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde7a60 0x5647dcde0b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fce880bd210 tx=0x7fce8803c000 comp rx=0 tx=0).stop 2024-09-10T07:42:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.074+0000 7fce97ac2740 1 -- 172.21.15.46:0/1325138228 shutdown_connections 2024-09-10T07:42:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.074+0000 7fce97ac2740 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde7a60 0x5647dcde0b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.074+0000 7fce97ac2740 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde29d0 0x5647dcdf37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.074+0000 7fce97ac2740 1 --2- 172.21.15.46:0/1325138228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde2090 0x5647dcdec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.074+0000 7fce97ac2740 1 -- 172.21.15.46:0/1325138228 >> 172.21.15.46:0/1325138228 conn(0x5647dcddd700 msgr2=0x5647dcddc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.074+0000 7fce97ac2740 1 -- 172.21.15.46:0/1325138228 shutdown_connections 2024-09-10T07:42:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.074+0000 7fce97ac2740 1 -- 172.21.15.46:0/1325138228 wait complete. 2024-09-10T07:42:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.075+0000 7fce97ac2740 1 -- start start 2024-09-10T07:42:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.075+0000 7fce97ac2740 1 Processor -- start 2024-09-10T07:42:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.079+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde2090 0x5647dcdf0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde29d0 0x5647dcdf0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce97ac2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcdf1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce97ac2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ba8b0 con 0x5647dcde7a60 2024-09-10T07:42:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce97ac2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ba1e0 con 0x5647dcde2090 2024-09-10T07:42:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce97ac2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b7420 con 0x5647dcde29d0 2024-09-10T07:42:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce97ac2740 10 client.0 _ll_register_callbacks cb 0x7fce8000a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce8d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde29d0 0x5647dcdf0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce8d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde29d0 0x5647dcdf0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57420/0 (socket says 172.21.15.46:57420) 2024-09-10T07:42:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce8d400640 1 -- 172.21.15.46:0/2597778236 learned_addr learned my addr 172.21.15.46:0/2597778236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.080+0000 7fce8de00640 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde2090 0x5647dcdf0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.081+0000 7fce8d400640 1 -- 172.21.15.46:0/2597778236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde2090 msgr2=0x5647dcdf0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.081+0000 7fce8d400640 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde2090 0x5647dcdf0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.081+0000 7fce8d400640 1 -- 172.21.15.46:0/2597778236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 msgr2=0x5647dcdf1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.081+0000 7fce8d400640 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcdf1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.081+0000 7fce8d400640 1 -- 172.21.15.46:0/2597778236 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647dcdf25b0 con 0x5647dcde29d0 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.081+0000 7fce8de00640 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde2090 0x5647dcdf0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.082+0000 7fce6f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.082+0000 7fce8d400640 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde29d0 0x5647dcdf0f90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fce7c01a660 tx=0x7fce7c009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c8f8eb077855888 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.082+0000 7fce6f400640 1 -- 172.21.15.46:0/2597778236 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c020040 con 0x5647dcde29d0 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.083+0000 7fce6f400640 1 -- 172.21.15.46:0/2597778236 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fce7c01c8a0 con 0x5647dcde29d0 2024-09-10T07:42:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.083+0000 7fce97ac2740 1 -- 172.21.15.46:0/2597778236 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce880c18c0 con 0x5647dcde29d0 2024-09-10T07:42:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.084+0000 7fce6f400640 1 -- 172.21.15.46:0/2597778236 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fce7c040050 con 0x5647dcde29d0 2024-09-10T07:42:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.084+0000 7fce97ac2740 10 client.26794 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.084+0000 7fce97ac2740 1 -- 172.21.15.46:0/2597778236 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5647dcdf38d0 con 0x5647dcde29d0 2024-09-10T07:42:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.084+0000 7fce6f400640 1 -- 172.21.15.46:0/2597778236 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce880c18c0 con 0x5647dcde29d0 2024-09-10T07:42:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.086+0000 7fce6f400640 1 -- 172.21.15.46:0/2597778236 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fce7c019070 con 0x5647dcde29d0 2024-09-10T07:42:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.086+0000 7fce97ac2740 10 client.26794 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.086+0000 7fce97ac2740 10 client.26794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.086+0000 7fce97ac2740 20 client.26794 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.086+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5647dce631a0 0x5647dce655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.086+0000 7fce97ac2740 4 client.26794 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:42:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.086+0000 7fce97ac2740 1 -- 172.21.15.46:0/2597778236 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647dcd4eb80 con 0x5647dce631a0 2024-09-10T07:42:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.092+0000 7fce8de00640 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5647dce631a0 0x5647dce655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.093+0000 7fce8de00640 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5647dce631a0 0x5647dce655c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f8ecc3fd1f7104 server_cookie=8176c5cf33906b2d in_seq=0 out_seq=0 2024-09-10T07:42:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.093+0000 7fce6f400640 10 client.26794 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.146+0000 7fce6f400640 1 -- 172.21.15.46:0/2597778236 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647dcd4eb80 con 0x5647dce631a0 2024-09-10T07:42:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.146+0000 7fce6f400640 10 client.26794 handle_command_reply: tid=0 2024-09-10T07:42:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.147+0000 7fce97ac2740 1 client.26794 shutdown 2024-09-10T07:42:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.147+0000 7fce97ac2740 1 -- 172.21.15.46:0/2597778236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde29d0 msgr2=0x5647dcdf0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.147+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde29d0 0x5647dcdf0f90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fce7c01a660 tx=0x7fce7c009790 comp rx=0 tx=0).stop 2024-09-10T07:42:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.148+0000 7fce97ac2740 1 -- 172.21.15.46:0/2597778236 shutdown_connections 2024-09-10T07:42:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.148+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5647dce631a0 0x5647dce655c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.148+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647dcde7a60 0x5647dcdf1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.148+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647dcde29d0 0x5647dcdf0f90 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fce7c01a660 tx=0x7fce7c009790 comp rx=0 tx=0).stop 2024-09-10T07:42:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.148+0000 7fce97ac2740 1 --2- 172.21.15.46:0/2597778236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647dcde2090 0x5647dcdf0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.148+0000 7fce97ac2740 1 -- 172.21.15.46:0/2597778236 >> 172.21.15.46:0/2597778236 conn(0x5647dcddd700 msgr2=0x5647dcddfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.148+0000 7fce6f400640 0 client.26794 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.148+0000 7fce97ac2740 1 -- 172.21.15.46:0/2597778236 shutdown_connections 2024-09-10T07:42:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.149+0000 7fce97ac2740 1 -- 172.21.15.46:0/2597778236 wait complete. 2024-09-10T07:42:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.149+0000 7fce97ac2740 20 client.26794 trim_cache size 0 max 16384 2024-09-10T07:42:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:42:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:42:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:42:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:42:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.150+0000 7fce94c00640 1 -- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fce740a2cb0 msgr2=0x7fce740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.150+0000 7fce94c00640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fce740a2cb0 0x7fce740a5170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009d00 tx=0x7fce7c012820 comp rx=0 tx=0).stop 2024-09-10T07:42:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.150+0000 7fce94c00640 1 -- 172.21.15.46:0/4275318944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce90109d10 msgr2=0x7fce901ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.150+0000 7fce94c00640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce90109d10 0x7fce901ae180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fce88034040 tx=0x7fce88031670 comp rx=0 tx=0).stop 2024-09-10T07:42:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.151+0000 7fce94c00640 1 -- 172.21.15.46:0/4275318944 shutdown_connections 2024-09-10T07:42:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.151+0000 7fce94c00640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce90109d10 0x7fce901ae180 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.151+0000 7fce94c00640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fce740a2cb0 0x7fce740a5170 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.151+0000 7fce94c00640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce90109370 0x7fce901a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.151+0000 7fce94c00640 1 --2- 172.21.15.46:0/4275318944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce90073460 0x7fce901a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.151+0000 7fce94c00640 1 -- 172.21.15.46:0/4275318944 >> 172.21.15.46:0/4275318944 conn(0x7fce9006ef20 msgr2=0x7fce900709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.151+0000 7fce94c00640 1 -- 172.21.15.46:0/4275318944 shutdown_connections 2024-09-10T07:42:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:27.151+0000 7fce94c00640 1 -- 172.21.15.46:0/4275318944 wait complete. 2024-09-10T07:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 -- 172.21.15.46:0/1233490636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98102780 msgr2=0x7fdf981067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 --2- 172.21.15.46:0/1233490636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98102780 0x7fdf981067d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009900 tx=0x7fdf8002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 -- 172.21.15.46:0/1233490636 shutdown_connections 2024-09-10T07:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 --2- 172.21.15.46:0/1233490636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf98106f00 0x7fdf9810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 --2- 172.21.15.46:0/1233490636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98102780 0x7fdf981067d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 --2- 172.21.15.46:0/1233490636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98101dd0 0x7fdf981021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 -- 172.21.15.46:0/1233490636 >> 172.21.15.46:0/1233490636 conn(0x7fdf980fd890 msgr2=0x7fdf980ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 -- 172.21.15.46:0/1233490636 shutdown_connections 2024-09-10T07:42:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.035+0000 7fdf9e400640 1 -- 172.21.15.46:0/1233490636 wait complete. 2024-09-10T07:42:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf9e400640 1 Processor -- start 2024-09-10T07:42:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf9e400640 1 -- start start 2024-09-10T07:42:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf9e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98101dd0 0x7fdf981a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf9e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf98102780 0x7fdf981a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf9e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98106f00 0x7fdf9819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf9e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf98118560 con 0x7fdf98101dd0 2024-09-10T07:42:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf9e400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981183e0 con 0x7fdf98102780 2024-09-10T07:42:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf9e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981186e0 con 0x7fdf98106f00 2024-09-10T07:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98106f00 0x7fdf9819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf97400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98101dd0 0x7fdf981a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.037+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98106f00 0x7fdf9819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57436/0 (socket says 172.21.15.46:57436) 2024-09-10T07:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf97e00640 1 -- 172.21.15.46:0/3835682715 learned_addr learned my addr 172.21.15.46:0/3835682715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf98102780 0x7fdf981a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf97400640 1 -- 172.21.15.46:0/3835682715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98106f00 msgr2=0x7fdf9819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf97400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98106f00 0x7fdf9819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf97400640 1 -- 172.21.15.46:0/3835682715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf98102780 msgr2=0x7fdf981a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf97400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf98102780 0x7fdf981a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf97400640 1 -- 172.21.15.46:0/3835682715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9819eff0 con 0x7fdf98101dd0 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf97e00640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98106f00 0x7fdf9819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.038+0000 7fdf97400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98101dd0 0x7fdf981a46d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009a10 tx=0x7fdf88009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.039+0000 7fdf94c00640 1 -- 172.21.15.46:0/3835682715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800f5e0 con 0x7fdf98101dd0 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.039+0000 7fdf94c00640 1 -- 172.21.15.46:0/3835682715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8800eba0 con 0x7fdf98101dd0 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.039+0000 7fdf94c00640 1 -- 172.21.15.46:0/3835682715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88017410 con 0x7fdf98101dd0 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.039+0000 7fdf9e400640 1 -- 172.21.15.46:0/3835682715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9819f2e0 con 0x7fdf98101dd0 2024-09-10T07:42:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.039+0000 7fdf9e400640 1 -- 172.21.15.46:0/3835682715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf981ab530 con 0x7fdf98101dd0 2024-09-10T07:42:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.042+0000 7fdfa1052740 1 Processor -- start 2024-09-10T07:42:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdfa1052740 1 -- start start 2024-09-10T07:42:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c88c70 0x557636c89050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c93910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636c96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdfa1052740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557636bef3e0 con 0x557636c89590 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdfa1052740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557636b96d80 con 0x557636c93e50 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdfa1052740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557636beb3a0 con 0x557636c88c70 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdf8ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636c96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c88c70 0x557636c89050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdf8ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636c96240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57890/0 (socket says 172.21.15.46:57890) 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.043+0000 7fdf8ea00640 1 -- 172.21.15.46:0/265819475 learned_addr learned my addr 172.21.15.46:0/265819475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.044+0000 7fdf94c00640 1 -- 172.21.15.46:0/3835682715 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf8801e020 con 0x7fdf98101dd0 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.044+0000 7fdf8ea00640 1 -- 172.21.15.46:0/265819475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c88c70 msgr2=0x557636c89050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.044+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c88c70 0x557636c89050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.044+0000 7fdf8ea00640 1 -- 172.21.15.46:0/265819475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 msgr2=0x557636c93910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.044+0000 7fdf97400640 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c93910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.044+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c93910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.044+0000 7fdf8ea00640 1 -- 172.21.15.46:0/265819475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557636bb5ae0 con 0x557636c93e50 2024-09-10T07:42:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.044+0000 7fdf97400640 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c93910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.045+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636c96240 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009900 tx=0x7fdf80005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1317fadcca4df01f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.045+0000 7fdf8cc00640 1 -- 172.21.15.46:0/265819475 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80040070 con 0x557636c93e50 2024-09-10T07:42:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.045+0000 7fdf8cc00640 1 -- 172.21.15.46:0/265819475 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf800317b0 con 0x557636c93e50 2024-09-10T07:42:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.045+0000 7fdf8cc00640 1 -- 172.21.15.46:0/265819475 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80035040 con 0x557636c93e50 2024-09-10T07:42:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.046+0000 7fdf94c00640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf7c0a2ef0 0x7fdf7c0a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.046+0000 7fdfa1052740 1 -- 172.21.15.46:0/265819475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 msgr2=0x557636c96240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.046+0000 7fdfa1052740 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636c96240 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009900 tx=0x7fdf80005be0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.046+0000 7fdf94c00640 1 -- 172.21.15.46:0/3835682715 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdf880ceb80 con 0x7fdf98101dd0 2024-09-10T07:42:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.046+0000 7fdf97e00640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf7c0a2ef0 0x7fdf7c0a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.047+0000 7fdfa1052740 1 -- 172.21.15.46:0/265819475 shutdown_connections 2024-09-10T07:42:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.047+0000 7fdfa1052740 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636c96240 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.047+0000 7fdfa1052740 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c93910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.047+0000 7fdfa1052740 1 --2- 172.21.15.46:0/265819475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c88c70 0x557636c89050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.047+0000 7fdfa1052740 1 -- 172.21.15.46:0/265819475 >> 172.21.15.46:0/265819475 conn(0x557636c85200 msgr2=0x557636c855e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.047+0000 7fdfa1052740 1 -- 172.21.15.46:0/265819475 shutdown_connections 2024-09-10T07:42:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.047+0000 7fdfa1052740 1 -- 172.21.15.46:0/265819475 wait complete. 2024-09-10T07:42:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.047+0000 7fdf97e00640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf7c0a2ef0 0x7fdf7c0a53b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf900049d0 tx=0x7fdf90009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.048+0000 7fdfa1052740 1 -- start start 2024-09-10T07:42:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.048+0000 7fdfa1052740 1 Processor -- start 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.048+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c88c70 0x557636c90c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.048+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.048+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c93e50 0x557636c91710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.048+0000 7fdfa1052740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557636c96a20 con 0x557636c89590 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.048+0000 7fdfa1052740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557636baa0f0 con 0x557636c88c70 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.048+0000 7fdfa1052740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557636c96ba0 con 0x557636c93e50 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdfa1052740 10 client.0 _ll_register_callbacks cb 0x557636b91bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf8ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf97400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c88c70 0x557636c90c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf8ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c911d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55818/0 (socket says 172.21.15.46:55818) 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf8ea00640 1 -- 172.21.15.46:0/1688509737 learned_addr learned my addr 172.21.15.46:0/1688509737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf97e00640 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c93e50 0x557636c91710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf97400640 1 -- 172.21.15.46:0/1688509737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c93e50 msgr2=0x557636c91710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf97400640 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c93e50 0x557636c91710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf97400640 1 -- 172.21.15.46:0/1688509737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 msgr2=0x557636c911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf97400640 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c911d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.049+0000 7fdf97400640 1 -- 172.21.15.46:0/1688509737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576369603c0 con 0x557636c88c70 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdf97400640 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c88c70 0x557636c90c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x557636c86060 tx=0x7fdf8809a960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba4eaea89e21c426 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdf85600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdf85600640 1 -- 172.21.15.46:0/1688509737 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880bb0e0 con 0x557636c88c70 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdf85600640 1 -- 172.21.15.46:0/1688509737 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdf880a37e0 con 0x557636c88c70 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdfa1052740 1 -- 172.21.15.46:0/1688509737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf800317b0 con 0x557636c88c70 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdf85600640 1 -- 172.21.15.46:0/1688509737 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdf8809f3b0 con 0x557636c88c70 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdfa1052740 10 client.26831 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.050+0000 7fdfa1052740 1 -- 172.21.15.46:0/1688509737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557636ca2980 con 0x557636c88c70 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdf85600640 1 -- 172.21.15.46:0/1688509737 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf800317b0 con 0x557636c88c70 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdf85600640 1 -- 172.21.15.46:0/1688509737 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fdf880b7050 con 0x557636c88c70 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdfa1052740 10 client.26831 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdfa1052740 10 client.26831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdfa1052740 20 client.26831 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdfa1052740 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557636ca3e40 0x557636ca6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdfa1052740 4 client.26831 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdfa1052740 1 -- 172.21.15.46:0/1688509737 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557636bfc950 con 0x557636ca3e40 2024-09-10T07:42:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.051+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557636ca3e40 0x557636ca6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.052+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557636ca3e40 0x557636ca6250 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27b678355a428a5a server_cookie=dab21b46db74a857 in_seq=0 out_seq=0 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.052+0000 7fdf85600640 10 client.26831 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdf85600640 1 -- 172.21.15.46:0/1688509737 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557636bfc950 con 0x557636ca3e40 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdf85600640 10 client.26831 handle_command_reply: tid=0 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 client.26831 shutdown 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 -- 172.21.15.46:0/1688509737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c88c70 msgr2=0x557636c90c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c88c70 0x557636c90c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x557636c86060 tx=0x7fdf8809a960 comp rx=0 tx=0).stop 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 -- 172.21.15.46:0/1688509737 shutdown_connections 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557636ca3e40 0x557636ca6250 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c93e50 0x557636c91710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636c911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 --2- 172.21.15.46:0/1688509737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c88c70 0x557636c90c90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 -- 172.21.15.46:0/1688509737 >> 172.21.15.46:0/1688509737 conn(0x557636c85200 msgr2=0x557636c8af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdf85600640 0 client.26831 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:42:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.053+0000 7fdfa1052740 1 -- 172.21.15.46:0/1688509737 shutdown_connections 2024-09-10T07:42:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.054+0000 7fdfa1052740 1 -- 172.21.15.46:0/1688509737 wait complete. 2024-09-10T07:42:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.054+0000 7fdfa1052740 20 client.26831 trim_cache size 0 max 16384 2024-09-10T07:42:28.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.067+0000 7fdfa1052740 1 Processor -- start 2024-09-10T07:42:28.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdfa1052740 1 -- start start 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636cabb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636cac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c83fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdfa1052740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981b5200 con 0x557636c89590 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdfa1052740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981b6da0 con 0x557636c93e50 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdfa1052740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981b58d0 con 0x557636c83b60 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdf8ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636cac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdf8ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636cac060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57920/0 (socket says 172.21.15.46:57920) 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdf8ea00640 1 -- 172.21.15.46:0/642704489 learned_addr learned my addr 172.21.15.46:0/642704489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:28.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.068+0000 7fdf97400640 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636cabb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf97e00640 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c83fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf8ea00640 1 -- 172.21.15.46:0/642704489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 msgr2=0x557636c83fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c83fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf8ea00640 1 -- 172.21.15.46:0/642704489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 msgr2=0x557636cabb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636cabb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf8ea00640 1 -- 172.21.15.46:0/642704489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557636c85030 con 0x557636c93e50 2024-09-10T07:42:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf97e00640 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c83fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf97400640 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636cabb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.069+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636cac060 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005390 tx=0x7fdf800053c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae1f0e903978da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.070+0000 7fdf87e00640 1 -- 172.21.15.46:0/642704489 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80040040 con 0x557636c93e50 2024-09-10T07:42:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.070+0000 7fdf87e00640 1 -- 172.21.15.46:0/642704489 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf80004030 con 0x557636c93e50 2024-09-10T07:42:28.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.070+0000 7fdf87e00640 1 -- 172.21.15.46:0/642704489 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80035040 con 0x557636c93e50 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.070+0000 7fdfa1052740 1 -- 172.21.15.46:0/642704489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 msgr2=0x557636cac060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.070+0000 7fdfa1052740 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636cac060 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005390 tx=0x7fdf800053c0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 -- 172.21.15.46:0/642704489 shutdown_connections 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c83fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c93e50 0x557636cac060 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 --2- 172.21.15.46:0/642704489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c89590 0x557636cabb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 -- 172.21.15.46:0/642704489 >> 172.21.15.46:0/642704489 conn(0x557636c85200 msgr2=0x557636c8fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 -- 172.21.15.46:0/642704489 shutdown_connections 2024-09-10T07:42:28.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 -- 172.21.15.46:0/642704489 wait complete. 2024-09-10T07:42:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 -- start start 2024-09-10T07:42:28.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.071+0000 7fdfa1052740 1 Processor -- start 2024-09-10T07:42:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c97d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c89590 0x557636c98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdfa1052740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c93e50 0x557636c987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdfa1052740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981b6670 con 0x557636c93e50 2024-09-10T07:42:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdfa1052740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981b5fa0 con 0x557636c89590 2024-09-10T07:42:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdfa1052740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981b31e0 con 0x557636c83b60 2024-09-10T07:42:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdfa1052740 10 client.0 _ll_register_callbacks cb 0x7fdf880d3040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdf8ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c89590 0x557636c98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdf8ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c89590 0x557636c98270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57928/0 (socket says 172.21.15.46:57928) 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdf8ea00640 1 -- 172.21.15.46:0/4146760790 learned_addr learned my addr 172.21.15.46:0/4146760790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.072+0000 7fdf97e00640 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c93e50 0x557636c987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.073+0000 7fdf97400640 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c97d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.073+0000 7fdf8ea00640 1 -- 172.21.15.46:0/4146760790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 msgr2=0x557636c97d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.073+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c97d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.073+0000 7fdf8ea00640 1 -- 172.21.15.46:0/4146760790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c93e50 msgr2=0x557636c987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.073+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c93e50 0x557636c987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.073+0000 7fdf8ea00640 1 -- 172.21.15.46:0/4146760790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557636c80770 con 0x557636c89590 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.073+0000 7fdf8ea00640 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c89590 0x557636c98270 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005360 tx=0x7fdf80004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae8b33612f8a3d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.073+0000 7fdf85600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:28.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.074+0000 7fdf85600640 1 -- 172.21.15.46:0/4146760790 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80040030 con 0x557636c89590 2024-09-10T07:42:28.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.074+0000 7fdf85600640 1 -- 172.21.15.46:0/4146760790 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdf800043e0 con 0x557636c89590 2024-09-10T07:42:28.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.074+0000 7fdfa1052740 1 -- 172.21.15.46:0/4146760790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf80004030 con 0x557636c89590 2024-09-10T07:42:28.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.074+0000 7fdf85600640 1 -- 172.21.15.46:0/4146760790 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdf80035070 con 0x557636c89590 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.074+0000 7fdfa1052740 10 client.26837 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.074+0000 7fdfa1052740 1 -- 172.21.15.46:0/4146760790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557636d0ede0 con 0x557636c89590 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.074+0000 7fdf85600640 1 -- 172.21.15.46:0/4146760790 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf80004030 con 0x557636c89590 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.075+0000 7fdf85600640 1 -- 172.21.15.46:0/4146760790 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fdf80003820 con 0x557636c89590 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.075+0000 7fdfa1052740 10 client.26837 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.075+0000 7fdfa1052740 10 client.26837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.075+0000 7fdfa1052740 20 client.26837 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.075+0000 7fdfa1052740 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557636d0f070 0x557636d11490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.075+0000 7fdfa1052740 4 client.26837 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.075+0000 7fdfa1052740 1 -- 172.21.15.46:0/4146760790 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557636bf6e60 con 0x557636d0f070 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.075+0000 7fdf97400640 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557636d0f070 0x557636d11490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdf97400640 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557636d0f070 0x557636d11490 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3dbf7135d0461e3f server_cookie=f3e399d3f785bf96 in_seq=0 out_seq=0 2024-09-10T07:42:28.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdf85600640 10 client.26837 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdf85600640 1 -- 172.21.15.46:0/4146760790 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557636bf6e60 con 0x557636d0f070 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdf85600640 10 client.26837 handle_command_reply: tid=0 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 client.26837 shutdown 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 -- 172.21.15.46:0/4146760790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c89590 msgr2=0x557636c98270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c89590 0x557636c98270 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005360 tx=0x7fdf80004790 comp rx=0 tx=0).stop 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 -- 172.21.15.46:0/4146760790 shutdown_connections 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x557636d0f070 0x557636d11490 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdf85600640 0 client.26837 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557636c93e50 0x557636c987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557636c89590 0x557636c98270 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 --2- 172.21.15.46:0/4146760790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557636c83b60 0x557636c97d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 -- 172.21.15.46:0/4146760790 >> 172.21.15.46:0/4146760790 conn(0x557636c85200 msgr2=0x557636c8fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.076+0000 7fdfa1052740 1 -- 172.21.15.46:0/4146760790 shutdown_connections 2024-09-10T07:42:28.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.077+0000 7fdfa1052740 1 -- 172.21.15.46:0/4146760790 wait complete. 2024-09-10T07:42:28.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.077+0000 7fdfa1052740 20 client.26837 trim_cache size 0 max 16384 2024-09-10T07:42:28.080 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:42:28.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.077+0000 7fdf9e400640 1 -- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf7c0a2ef0 msgr2=0x7fdf7c0a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.077+0000 7fdf9e400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf7c0a2ef0 0x7fdf7c0a53b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf900049d0 tx=0x7fdf90009500 comp rx=0 tx=0).stop 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.077+0000 7fdf9e400640 1 -- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98101dd0 msgr2=0x7fdf981a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.077+0000 7fdf9e400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98101dd0 0x7fdf981a46d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009a10 tx=0x7fdf88009ed0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.078+0000 7fdf9e400640 1 -- 172.21.15.46:0/3835682715 shutdown_connections 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.078+0000 7fdf9e400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf98106f00 0x7fdf9819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.078+0000 7fdf9e400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf98102780 0x7fdf981a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.078+0000 7fdf9e400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf7c0a2ef0 0x7fdf7c0a53b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.078+0000 7fdf9e400640 1 --2- 172.21.15.46:0/3835682715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98101dd0 0x7fdf981a46d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.078+0000 7fdf9e400640 1 -- 172.21.15.46:0/3835682715 >> 172.21.15.46:0/3835682715 conn(0x7fdf980fd890 msgr2=0x7fdf980fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.078+0000 7fdf9e400640 1 -- 172.21.15.46:0/3835682715 shutdown_connections 2024-09-10T07:42:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:28.078+0000 7fdf9e400640 1 -- 172.21.15.46:0/3835682715 wait complete. 2024-09-10T07:42:28.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:27 smithi181 ceph-mon[35397]: pgmap v574: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 809 op/s 2024-09-10T07:42:28.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:27 smithi063 ceph-mon[35020]: pgmap v574: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 809 op/s 2024-09-10T07:42:28.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:27 smithi046 ceph-mon[30705]: pgmap v574: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 809 op/s 2024-09-10T07:42:29.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:42:29.829 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:29.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:29 smithi046 ceph-mon[30705]: pgmap v575: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 626 op/s 2024-09-10T07:42:30.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:29 smithi181 ceph-mon[35397]: pgmap v575: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 626 op/s 2024-09-10T07:42:30.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:29 smithi063 ceph-mon[35020]: pgmap v575: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 626 op/s 2024-09-10T07:42:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 310.0 s, 972.6 tps, lat 104.867 ms stddev 155.395 2024-09-10T07:42:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.035+0000 7fcb1e200640 1 -- 172.21.15.46:0/2572183073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb1810e430 msgr2=0x7fcb18110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.036+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2572183073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb1810e430 0x7fcb18110820 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009900 tx=0x7fcb0402a4d0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.036+0000 7fcb1e200640 1 -- 172.21.15.46:0/2572183073 shutdown_connections 2024-09-10T07:42:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.036+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2572183073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb1810e430 0x7fcb18110820 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.036+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2572183073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb18074ee0 0x7fcb1810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.036+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2572183073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb180745c0 0x7fcb180749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.036+0000 7fcb1e200640 1 -- 172.21.15.46:0/2572183073 >> 172.21.15.46:0/2572183073 conn(0x7fcb180fd8b0 msgr2=0x7fcb180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.038+0000 7fcb1e200640 1 -- 172.21.15.46:0/2572183073 shutdown_connections 2024-09-10T07:42:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.039+0000 7fcb1e200640 1 -- 172.21.15.46:0/2572183073 wait complete. 2024-09-10T07:42:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.043+0000 7fcb1e200640 1 Processor -- start 2024-09-10T07:42:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.044+0000 7fcb1e200640 1 -- start start 2024-09-10T07:42:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.044+0000 7fcb1e200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb180745c0 0x7fcb18101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.044+0000 7fcb1e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb1810e430 0x7fcb18102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.044+0000 7fcb1e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb181025d0 0x7fcb181aac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.045+0000 7fcb1e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18114230 con 0x7fcb1810e430 2024-09-10T07:42:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.045+0000 7fcb1e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181140b0 con 0x7fcb181025d0 2024-09-10T07:42:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.045+0000 7fcb1e200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181143b0 con 0x7fcb180745c0 2024-09-10T07:42:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.045+0000 7fcb17400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb180745c0 0x7fcb18101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.045+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb181025d0 0x7fcb181aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.045+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb181025d0 0x7fcb181aac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57944/0 (socket says 172.21.15.46:57944) 2024-09-10T07:42:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.045+0000 7fcb17e00640 1 -- 172.21.15.46:0/2758863320 learned_addr learned my addr 172.21.15.46:0/2758863320 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.046+0000 7fcb17e00640 1 -- 172.21.15.46:0/2758863320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb180745c0 msgr2=0x7fcb18101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.046+0000 7fcb17e00640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb180745c0 0x7fcb18101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.046+0000 7fcb17e00640 1 -- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb1810e430 msgr2=0x7fcb18102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.046+0000 7fcb17e00640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb1810e430 0x7fcb18102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.046+0000 7fcb17e00640 1 -- 172.21.15.46:0/2758863320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb181ab170 con 0x7fcb181025d0 2024-09-10T07:42:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.047+0000 7fcb17e00640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb181025d0 0x7fcb181aac30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcb04033040 tx=0x7fcb040051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.047+0000 7fcb0fe00640 1 -- 172.21.15.46:0/2758863320 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04041020 con 0x7fcb181025d0 2024-09-10T07:42:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.047+0000 7fcb1e200640 1 -- 172.21.15.46:0/2758863320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb181ab400 con 0x7fcb181025d0 2024-09-10T07:42:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.047+0000 7fcb1e200640 1 -- 172.21.15.46:0/2758863320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb181ab960 con 0x7fcb181025d0 2024-09-10T07:42:31.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.051+0000 7fcb0fe00640 1 -- 172.21.15.46:0/2758863320 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0403d960 con 0x7fcb181025d0 2024-09-10T07:42:31.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.051+0000 7fcb0fe00640 1 -- 172.21.15.46:0/2758863320 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04031940 con 0x7fcb181025d0 2024-09-10T07:42:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.055+0000 7fcb0fe00640 1 -- 172.21.15.46:0/2758863320 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb04044020 con 0x7fcb181025d0 2024-09-10T07:42:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.055+0000 7fcb20c7a740 1 Processor -- start 2024-09-10T07:42:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.057+0000 7fcb0fe00640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcafc0a2c00 0x7fcafc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.058+0000 7fcb20c7a740 1 -- start start 2024-09-10T07:42:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.059+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.059+0000 7fcb16a00640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcafc0a2c00 0x7fcafc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.059+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.059+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.060+0000 7fcb16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.060+0000 7fcb16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57948/0 (socket says 172.21.15.46:57948) 2024-09-10T07:42:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.061+0000 7fcb20c7a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4159103e0 con 0x55a4159b4e50 2024-09-10T07:42:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.061+0000 7fcb20c7a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a4158b7d80 con 0x55a4159aa590 2024-09-10T07:42:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.061+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.062+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55864/0 (socket says 172.21.15.46:55864) 2024-09-10T07:42:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.062+0000 7fcb17e00640 1 -- 172.21.15.46:0/1679589517 learned_addr learned my addr 172.21.15.46:0/1679589517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:31.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.062+0000 7fcb17400640 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.062+0000 7fcb20c7a740 1 -- 172.21.15.46:0/1679589517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a41590c3a0 con 0x55a4159a9c70 2024-09-10T07:42:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.062+0000 7fcb0fe00640 1 -- 172.21.15.46:0/2758863320 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcb040bb3b0 con 0x7fcb181025d0 2024-09-10T07:42:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.063+0000 7fcb17e00640 1 -- 172.21.15.46:0/1679589517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 msgr2=0x55a4159aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.063+0000 7fcb17e00640 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.064+0000 7fcb17e00640 1 -- 172.21.15.46:0/1679589517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 msgr2=0x55a4159b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.064+0000 7fcb17e00640 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.064+0000 7fcb17e00640 1 -- 172.21.15.46:0/1679589517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4158d6ae0 con 0x55a4159b4e50 2024-09-10T07:42:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.064+0000 7fcb17400640 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.065+0000 7fcb16a00640 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.068+0000 7fcb16a00640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcafc0a2c00 0x7fcafc0a50c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcb18103310 tx=0x7fcb08009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.068+0000 7fcb17e00640 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159b7240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcb04005ab0 tx=0x7fcb040c49d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab0138a90e74ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.069+0000 7fcb03e00640 1 -- 172.21.15.46:0/1679589517 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb040b9d40 con 0x55a4159b4e50 2024-09-10T07:42:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.069+0000 7fcb03e00640 1 -- 172.21.15.46:0/1679589517 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb040bda10 con 0x55a4159b4e50 2024-09-10T07:42:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.070+0000 7fcb20c7a740 1 -- 172.21.15.46:0/1679589517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 msgr2=0x55a4159b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.070+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159b7240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcb04005ab0 tx=0x7fcb040c49d0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.071+0000 7fcb20c7a740 1 -- 172.21.15.46:0/1679589517 shutdown_connections 2024-09-10T07:42:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.071+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159b7240 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.071+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.071+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/1679589517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.071+0000 7fcb20c7a740 1 -- 172.21.15.46:0/1679589517 >> 172.21.15.46:0/1679589517 conn(0x55a4159a6200 msgr2=0x55a4159a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.071+0000 7fcb20c7a740 1 -- 172.21.15.46:0/1679589517 shutdown_connections 2024-09-10T07:42:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.071+0000 7fcb20c7a740 1 -- 172.21.15.46:0/1679589517 wait complete. 2024-09-10T07:42:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.072+0000 7fcb20c7a740 1 -- start start 2024-09-10T07:42:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.072+0000 7fcb20c7a740 1 Processor -- start 2024-09-10T07:42:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.073+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 0x55a4159b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57952/0 (socket says 172.21.15.46:57952) 2024-09-10T07:42:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb16a00640 1 -- 172.21.15.46:0/794909612 learned_addr learned my addr 172.21.15.46:0/794909612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4159103e0 con 0x55a4159a9c70 2024-09-10T07:42:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a4158cb0f0 con 0x55a4159aa590 2024-09-10T07:42:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a4159b77e0 con 0x55a4159b4e50 2024-09-10T07:42:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.075+0000 7fcb20c7a740 10 client.0 _ll_register_callbacks cb 0x55a4158b2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.074+0000 7fcb17e00640 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 0x55a4159b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.075+0000 7fcb16a00640 1 -- 172.21.15.46:0/794909612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 msgr2=0x55a4159b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.075+0000 7fcb16a00640 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 0x55a4159b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.075+0000 7fcb16a00640 1 -- 172.21.15.46:0/794909612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 msgr2=0x55a4159b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.075+0000 7fcb16a00640 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.075+0000 7fcb16a00640 1 -- 172.21.15.46:0/794909612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4156813c0 con 0x55a4159aa590 2024-09-10T07:42:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.075+0000 7fcb16a00640 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b2190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcb080199e0 tx=0x7fcb08004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1b514c0e81c147 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.075+0000 7fcb00c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.076+0000 7fcb00c00640 1 -- 172.21.15.46:0/794909612 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800c070 con 0x55a4159aa590 2024-09-10T07:42:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.076+0000 7fcb00c00640 1 -- 172.21.15.46:0/794909612 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcb08016700 con 0x55a4159aa590 2024-09-10T07:42:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.076+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb040bda10 con 0x55a4159aa590 2024-09-10T07:42:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.076+0000 7fcb00c00640 1 -- 172.21.15.46:0/794909612 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcb08016900 con 0x55a4159aa590 2024-09-10T07:42:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.077+0000 7fcb20c7a740 10 client.26849 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.077+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a4159c3640 con 0x55a4159aa590 2024-09-10T07:42:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb00c00640 1 -- 172.21.15.46:0/794909612 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb08010040 con 0x55a4159aa590 2024-09-10T07:42:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb00c00640 1 -- 172.21.15.46:0/794909612 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fcb08015070 con 0x55a4159aa590 2024-09-10T07:42:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb20c7a740 10 client.26849 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb20c7a740 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb20c7a740 20 client.26849 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a4159c4b10 0x55a4159c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb20c7a740 4 client.26849 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcb040bda10 con 0x55a4159c4b10 2024-09-10T07:42:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.080+0000 7fcb17400640 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a4159c4b10 0x55a4159c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.081+0000 7fcb17400640 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a4159c4b10 0x55a4159c6fb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3675d5da0e68c1 server_cookie=85e3786a9a61a32a in_seq=0 out_seq=0 2024-09-10T07:42:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.081+0000 7fcb00c00640 10 client.26849 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.083+0000 7fcb00c00640 1 -- 172.21.15.46:0/794909612 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcb040bda10 con 0x55a4159c4b10 2024-09-10T07:42:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.083+0000 7fcb00c00640 10 client.26849 handle_command_reply: tid=0 2024-09-10T07:42:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.083+0000 7fcb20c7a740 1 client.26849 shutdown 2024-09-10T07:42:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.083+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 msgr2=0x55a4159b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.083+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b2190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcb080199e0 tx=0x7fcb08004840 comp rx=0 tx=0).stop 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 shutdown_connections 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a4159c4b10 0x55a4159c6fb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 0x55a4159b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159b2190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/794909612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 >> 172.21.15.46:0/794909612 conn(0x55a4159a6200 msgr2=0x55a4159abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb00c00640 0 client.26849 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 shutdown_connections 2024-09-10T07:42:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 1 -- 172.21.15.46:0/794909612 wait complete. 2024-09-10T07:42:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.084+0000 7fcb20c7a740 20 client.26849 trim_cache size 0 max 16384 2024-09-10T07:42:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.106+0000 7fcb20c7a740 1 Processor -- start 2024-09-10T07:42:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.106+0000 7fcb20c7a740 1 -- start start 2024-09-10T07:42:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.106+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.106+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.106+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.106+0000 7fcb20c7a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf800fbb0 con 0x55a4159b4e50 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.106+0000 7fcb20c7a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8011750 con 0x55a4159aa590 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.106+0000 7fcb20c7a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8010280 con 0x55a4159a9c70 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57960/0 (socket says 172.21.15.46:57960) 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb16a00640 1 -- 172.21.15.46:0/4107522771 learned_addr learned my addr 172.21.15.46:0/4107522771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb17400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb17e00640 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb16a00640 1 -- 172.21.15.46:0/4107522771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 msgr2=0x55a4159abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb16a00640 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb16a00640 1 -- 172.21.15.46:0/4107522771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 msgr2=0x55a4159cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb16a00640 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb16a00640 1 -- 172.21.15.46:0/4107522771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4159c7630 con 0x55a4159aa590 2024-09-10T07:42:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.107+0000 7fcb17e00640 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.108+0000 7fcb16a00640 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159cc8c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb08012a90 tx=0x7fcb08006d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c9afa57f302a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.108+0000 7fcb03400640 1 -- 172.21.15.46:0/4107522771 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800c040 con 0x55a4159aa590 2024-09-10T07:42:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.108+0000 7fcb03400640 1 -- 172.21.15.46:0/4107522771 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb08010030 con 0x55a4159aa590 2024-09-10T07:42:31.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.108+0000 7fcb03400640 1 -- 172.21.15.46:0/4107522771 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08022680 con 0x55a4159aa590 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.109+0000 7fcb20c7a740 1 -- 172.21.15.46:0/4107522771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 msgr2=0x55a4159cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.109+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159cc8c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb08012a90 tx=0x7fcb08006d00 comp rx=0 tx=0).stop 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.109+0000 7fcb20c7a740 1 -- 172.21.15.46:0/4107522771 shutdown_connections 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.109+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159b4e50 0x55a4159cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.109+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159cc8c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.109+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/4107522771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159a9c70 0x55a4159abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.109+0000 7fcb20c7a740 1 -- 172.21.15.46:0/4107522771 >> 172.21.15.46:0/4107522771 conn(0x55a4159a6200 msgr2=0x55a4159ab6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.110+0000 7fcb20c7a740 1 -- 172.21.15.46:0/4107522771 shutdown_connections 2024-09-10T07:42:31.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.110+0000 7fcb20c7a740 1 -- 172.21.15.46:0/4107522771 wait complete. 2024-09-10T07:42:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.111+0000 7fcb20c7a740 1 -- start start 2024-09-10T07:42:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.111+0000 7fcb20c7a740 1 Processor -- start 2024-09-10T07:42:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb20c7a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 0x55a4159b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb20c7a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8011020 con 0x55a4159a9c70 2024-09-10T07:42:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb20c7a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8010950 con 0x55a4159aa590 2024-09-10T07:42:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb20c7a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf800db90 con 0x55a4159b4e50 2024-09-10T07:42:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb20c7a740 10 client.0 _ll_register_callbacks cb 0x7fcb08019680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159bcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55890/0 (socket says 172.21.15.46:55890) 2024-09-10T07:42:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.112+0000 7fcb17400640 1 -- 172.21.15.46:0/3827823881 learned_addr learned my addr 172.21.15.46:0/3827823881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:31.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.113+0000 7fcb17e00640 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 0x55a4159b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.113+0000 7fcb16a00640 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.113+0000 7fcb16a00640 1 -- 172.21.15.46:0/3827823881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 msgr2=0x55a4159b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:31.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.113+0000 7fcb16a00640 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 0x55a4159b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.113+0000 7fcb16a00640 1 -- 172.21.15.46:0/3827823881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 msgr2=0x55a4159bcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:31.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.114+0000 7fcb16a00640 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159bcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.114+0000 7fcb16a00640 1 -- 172.21.15.46:0/3827823881 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4159a6030 con 0x55a4159aa590 2024-09-10T07:42:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.114+0000 7fcb17400640 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.114+0000 7fcb16a00640 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159bd290 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55a4159ab100 tx=0x7fcb08005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7b4333a308e034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:31.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.114+0000 7fcb00c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.115+0000 7fcb00c00640 1 -- 172.21.15.46:0/3827823881 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08036070 con 0x55a4159aa590 2024-09-10T07:42:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.115+0000 7fcb00c00640 1 -- 172.21.15.46:0/3827823881 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcb080215f0 con 0x55a4159aa590 2024-09-10T07:42:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.115+0000 7fcb20c7a740 1 -- 172.21.15.46:0/3827823881 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb08010030 con 0x55a4159aa590 2024-09-10T07:42:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.116+0000 7fcb00c00640 1 -- 172.21.15.46:0/3827823881 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcb08043050 con 0x55a4159aa590 2024-09-10T07:42:31.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.116+0000 7fcb20c7a740 10 client.26855 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:31.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.116+0000 7fcb20c7a740 1 -- 172.21.15.46:0/3827823881 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a4159bc440 con 0x55a4159aa590 2024-09-10T07:42:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.117+0000 7fcb00c00640 1 -- 172.21.15.46:0/3827823881 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb08010030 con 0x55a4159aa590 2024-09-10T07:42:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.117+0000 7fcb00c00640 1 -- 172.21.15.46:0/3827823881 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fcb08015040 con 0x55a4159aa590 2024-09-10T07:42:31.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.118+0000 7fcb20c7a740 10 client.26855 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.118+0000 7fcb20c7a740 10 client.26855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.118+0000 7fcb20c7a740 20 client.26855 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.118+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a415a2f890 0x55a415a31c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.118+0000 7fcb20c7a740 4 client.26855 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:42:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.118+0000 7fcb20c7a740 1 -- 172.21.15.46:0/3827823881 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a41591d950 con 0x55a415a2f890 2024-09-10T07:42:31.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.119+0000 7fcb17400640 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a415a2f890 0x55a415a31c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:31.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.121+0000 7fcb17400640 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a415a2f890 0x55a415a31c80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b7854c84467dd8 server_cookie=ce87b78da62bdbbd in_seq=0 out_seq=0 2024-09-10T07:42:31.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.121+0000 7fcb00c00640 10 client.26855 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.121+0000 7fcb00c00640 1 -- 172.21.15.46:0/3827823881 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a41591d950 con 0x55a415a2f890 2024-09-10T07:42:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.121+0000 7fcb00c00640 10 client.26855 handle_command_reply: tid=0 2024-09-10T07:42:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.121+0000 7fcb20c7a740 1 client.26855 shutdown 2024-09-10T07:42:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb20c7a740 1 -- 172.21.15.46:0/3827823881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 msgr2=0x55a4159bd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159bd290 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55a4159ab100 tx=0x7fcb08005c60 comp rx=0 tx=0).stop 2024-09-10T07:42:31.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb20c7a740 1 -- 172.21.15.46:0/3827823881 shutdown_connections 2024-09-10T07:42:31.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a415a2f890 0x55a415a31c80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4159b4e50 0x55a4159b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4159aa590 0x55a4159bd290 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb20c7a740 1 --2- 172.21.15.46:0/3827823881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4159a9c70 0x55a4159bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb20c7a740 1 -- 172.21.15.46:0/3827823881 >> 172.21.15.46:0/3827823881 conn(0x55a4159a6200 msgr2=0x55a4159b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.122+0000 7fcb00c00640 0 client.26855 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.123+0000 7fcb20c7a740 1 -- 172.21.15.46:0/3827823881 shutdown_connections 2024-09-10T07:42:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.123+0000 7fcb20c7a740 1 -- 172.21.15.46:0/3827823881 wait complete. 2024-09-10T07:42:31.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.123+0000 7fcb20c7a740 20 client.26855 trim_cache size 0 max 16384 2024-09-10T07:42:31.124 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 -- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcafc0a2c00 msgr2=0x7fcafc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcafc0a2c00 0x7fcafc0a50c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcb18103310 tx=0x7fcb08009310 comp rx=0 tx=0).stop 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 -- 172.21.15.46:0/2758863320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb181025d0 msgr2=0x7fcb181aac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb181025d0 0x7fcb181aac30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcb04033040 tx=0x7fcb040051f0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 -- 172.21.15.46:0/2758863320 shutdown_connections 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb181025d0 0x7fcb181aac30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcafc0a2c00 0x7fcafc0a50c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb1810e430 0x7fcb18102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 --2- 172.21.15.46:0/2758863320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb180745c0 0x7fcb18101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.124+0000 7fcb1e200640 1 -- 172.21.15.46:0/2758863320 >> 172.21.15.46:0/2758863320 conn(0x7fcb180fd8b0 msgr2=0x7fcb18110630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:31.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.125+0000 7fcb1e200640 1 -- 172.21.15.46:0/2758863320 shutdown_connections 2024-09-10T07:42:31.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:31.129+0000 7fcb1e200640 1 -- 172.21.15.46:0/2758863320 wait complete. 2024-09-10T07:42:32.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:31 smithi181 ceph-mon[35397]: pgmap v576: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 734 op/s 2024-09-10T07:42:32.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:31 smithi063 ceph-mon[35020]: pgmap v576: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 734 op/s 2024-09-10T07:42:32.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:31 smithi046 ceph-mon[30705]: pgmap v576: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 734 op/s 2024-09-10T07:42:33.135 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:42:33.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:34.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:33 smithi181 ceph-mon[35397]: pgmap v577: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 686 op/s 2024-09-10T07:42:34.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:33 smithi063 ceph-mon[35020]: pgmap v577: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 686 op/s 2024-09-10T07:42:34.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:33 smithi046 ceph-mon[30705]: pgmap v577: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 686 op/s 2024-09-10T07:42:34.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3531466675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a41019c0 msgr2=0x7ff9a4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3531466675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a41019c0 0x7ff9a4101e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff990009900 tx=0x7ff99002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3531466675 shutdown_connections 2024-09-10T07:42:34.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3531466675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a4102360 0x7ff9a410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3531466675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a41019c0 0x7ff9a4101e20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3531466675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a41079c0 0x7ff9a4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3531466675 >> 172.21.15.46:0/3531466675 conn(0x7ff9a40fd740 msgr2=0x7ff9a40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:34.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3531466675 shutdown_connections 2024-09-10T07:42:34.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.525+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3531466675 wait complete. 2024-09-10T07:42:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.526+0000 7ff9a8a00640 1 Processor -- start 2024-09-10T07:42:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a8a00640 1 -- start start 2024-09-10T07:42:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a41019c0 0x7ff9a4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a4102360 0x7ff9a41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a41079c0 0x7ff9a419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4111d90 con 0x7ff9a41079c0 2024-09-10T07:42:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a8a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4111c10 con 0x7ff9a4102360 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a8a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4111f10 con 0x7ff9a41019c0 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a4102360 0x7ff9a41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a41019c0 0x7ff9a4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a4102360 0x7ff9a41987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58006/0 (socket says 172.21.15.46:58006) 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a2a00640 1 -- 172.21.15.46:0/3639367475 learned_addr learned my addr 172.21.15.46:0/3639367475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.527+0000 7ff9a3e00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a41079c0 0x7ff9a419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff9a3400640 1 -- 172.21.15.46:0/3639367475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a4102360 msgr2=0x7ff9a41987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff9a3400640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a4102360 0x7ff9a41987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff9a3400640 1 -- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a41079c0 msgr2=0x7ff9a419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff9a3400640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a41079c0 0x7ff9a419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff9a3400640 1 -- 172.21.15.46:0/3639367475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a419d240 con 0x7ff9a41019c0 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff9a3400640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a41019c0 0x7ff9a4198290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff99400e9f0 tx=0x7ff99400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff99be00640 1 -- 172.21.15.46:0/3639367475 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99400ccc0 con 0x7ff9a41019c0 2024-09-10T07:42:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3639367475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a419d530 con 0x7ff9a41019c0 2024-09-10T07:42:34.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff99be00640 1 -- 172.21.15.46:0/3639367475 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff994010910 con 0x7ff9a41019c0 2024-09-10T07:42:34.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.528+0000 7ff99be00640 1 -- 172.21.15.46:0/3639367475 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994018960 con 0x7ff9a41019c0 2024-09-10T07:42:34.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.529+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3639367475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a4074420 con 0x7ff9a41019c0 2024-09-10T07:42:34.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.532+0000 7ff99be00640 1 -- 172.21.15.46:0/3639367475 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff994010ac0 con 0x7ff9a41019c0 2024-09-10T07:42:34.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.532+0000 7ff9acc36740 1 Processor -- start 2024-09-10T07:42:34.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.532+0000 7ff9acc36740 1 -- start start 2024-09-10T07:42:34.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b60cb0 0x55e216b61090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b615d0 0x55e216b6b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b6e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9acc36740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e216ac7290 con 0x55e216b615d0 2024-09-10T07:42:34.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9acc36740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e216a6eb40 con 0x55e216b6be90 2024-09-10T07:42:34.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9acc36740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e216ac3250 con 0x55e216b60cb0 2024-09-10T07:42:34.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b6e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b60cb0 0x55e216b61090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b6e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58020/0 (socket says 172.21.15.46:58020) 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9a2a00640 1 -- 172.21.15.46:0/2999443135 learned_addr learned my addr 172.21.15.46:0/2999443135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.533+0000 7ff9a3400640 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b615d0 0x55e216b6b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff9a2a00640 1 -- 172.21.15.46:0/2999443135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b60cb0 msgr2=0x55e216b61090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b60cb0 0x55e216b61090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff9a2a00640 1 -- 172.21.15.46:0/2999443135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b615d0 msgr2=0x55e216b6b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b615d0 0x55e216b6b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff99be00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9880a2c00 0x7ff9880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff9a2a00640 1 -- 172.21.15.46:0/2999443135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2168386a0 con 0x55e216b6be90 2024-09-10T07:42:34.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff9a3400640 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b615d0 0x55e216b6b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:34.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff99be00640 1 -- 172.21.15.46:0/3639367475 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff9940cf6d0 con 0x7ff9a41019c0 2024-09-10T07:42:34.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b6e280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c00 tx=0x7ff990005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b395d0b1932d9f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:34.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.534+0000 7ff9a3e00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9880a2c00 0x7ff9880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.535+0000 7ff98fe00640 1 -- 172.21.15.46:0/2999443135 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990040070 con 0x55e216b6be90 2024-09-10T07:42:34.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.535+0000 7ff98fe00640 1 -- 172.21.15.46:0/2999443135 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff990004480 con 0x55e216b6be90 2024-09-10T07:42:34.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.535+0000 7ff98fe00640 1 -- 172.21.15.46:0/2999443135 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990035040 con 0x55e216b6be90 2024-09-10T07:42:34.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 -- 172.21.15.46:0/2999443135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 msgr2=0x55e216b6e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b6e280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c00 tx=0x7ff990005e10 comp rx=0 tx=0).stop 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9a3e00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9880a2c00 0x7ff9880a50c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005fd0 tx=0x7ff99c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 -- 172.21.15.46:0/2999443135 shutdown_connections 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b6e280 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b615d0 0x55e216b6b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 --2- 172.21.15.46:0/2999443135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b60cb0 0x55e216b61090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 -- 172.21.15.46:0/2999443135 >> 172.21.15.46:0/2999443135 conn(0x55e216b5d240 msgr2=0x55e216b5d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 -- 172.21.15.46:0/2999443135 shutdown_connections 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 -- 172.21.15.46:0/2999443135 wait complete. 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.536+0000 7ff9acc36740 1 -- start start 2024-09-10T07:42:34.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.537+0000 7ff9acc36740 1 Processor -- start 2024-09-10T07:42:34.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.537+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b60cb0 0x55e216b6b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.537+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b68a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.537+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b6be90 0x55e216b68f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.537+0000 7ff9acc36740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e216b6eb00 con 0x55e216b6be90 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.537+0000 7ff9acc36740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e216a81fa0 con 0x55e216b60cb0 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.537+0000 7ff9acc36740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e216b6ec80 con 0x55e216b615d0 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.537+0000 7ff9acc36740 10 client.0 _ll_register_callbacks cb 0x55e2167d5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.538+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b68a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.538+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b60cb0 0x55e216b6b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.538+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b68a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57572/0 (socket says 172.21.15.46:57572) 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.538+0000 7ff9a2a00640 1 -- 172.21.15.46:0/344756132 learned_addr learned my addr 172.21.15.46:0/344756132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.538+0000 7ff9a3e00640 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b6be90 0x55e216b68f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.539+0000 7ff9a3400640 1 -- 172.21.15.46:0/344756132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 msgr2=0x55e216b68a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.539+0000 7ff9a3400640 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b68a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.539+0000 7ff9a3400640 1 -- 172.21.15.46:0/344756132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b6be90 msgr2=0x55e216b68f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:34.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.539+0000 7ff9a3400640 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b6be90 0x55e216b68f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.539+0000 7ff9a3400640 1 -- 172.21.15.46:0/344756132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e216b69750 con 0x55e216b60cb0 2024-09-10T07:42:34.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff9a3400640 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b60cb0 0x55e216b6b780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55e216b5e0a0 tx=0x7ff994097f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4759decb9d36857 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:34.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff98cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff98cc00640 1 -- 172.21.15.46:0/344756132 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99409b560 con 0x55e216b60cb0 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff98cc00640 1 -- 172.21.15.46:0/344756132 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff9940d1bb0 con 0x55e216b60cb0 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff9acc36740 1 -- 172.21.15.46:0/344756132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff990004480 con 0x55e216b60cb0 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff98cc00640 1 -- 172.21.15.46:0/344756132 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff9940b3050 con 0x55e216b60cb0 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff9acc36740 10 client.26867 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff9acc36740 1 -- 172.21.15.46:0/344756132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e216b7a9c0 con 0x55e216b60cb0 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.540+0000 7ff98cc00640 1 -- 172.21.15.46:0/344756132 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff990004480 con 0x55e216b60cb0 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.541+0000 7ff98cc00640 1 -- 172.21.15.46:0/344756132 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7ff99409b170 con 0x55e216b60cb0 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.541+0000 7ff9acc36740 10 client.26867 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.541+0000 7ff9acc36740 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.541+0000 7ff9acc36740 20 client.26867 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:34.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.541+0000 7ff9acc36740 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e216b7be80 0x55e216b7e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.541+0000 7ff9acc36740 4 client.26867 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.541+0000 7ff9acc36740 1 -- 172.21.15.46:0/344756132 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e216ad4800 con 0x55e216b7be80 2024-09-10T07:42:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.541+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e216b7be80 0x55e216b7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.542+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e216b7be80 0x55e216b7e340 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d457c2364bfbf9 server_cookie=8628afc619b77664 in_seq=0 out_seq=0 2024-09-10T07:42:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.542+0000 7ff98cc00640 10 client.26867 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.543+0000 7ff98cc00640 1 -- 172.21.15.46:0/344756132 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff990050e80 con 0x55e216b7be80 2024-09-10T07:42:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.543+0000 7ff98cc00640 10 client.26867 handle_command_reply: tid=0 2024-09-10T07:42:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.543+0000 7ff9acc36740 1 client.26867 shutdown 2024-09-10T07:42:34.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 -- 172.21.15.46:0/344756132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b60cb0 msgr2=0x55e216b6b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b60cb0 0x55e216b6b780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55e216b5e0a0 tx=0x7ff994097f60 comp rx=0 tx=0).stop 2024-09-10T07:42:34.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 -- 172.21.15.46:0/344756132 shutdown_connections 2024-09-10T07:42:34.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e216b7be80 0x55e216b7e340 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b6be90 0x55e216b68f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff98cc00640 0 client.26867 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:34.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b68a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 --2- 172.21.15.46:0/344756132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b60cb0 0x55e216b6b780 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 -- 172.21.15.46:0/344756132 >> 172.21.15.46:0/344756132 conn(0x55e216b5d240 msgr2=0x55e216b62f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:34.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 -- 172.21.15.46:0/344756132 shutdown_connections 2024-09-10T07:42:34.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.544+0000 7ff9acc36740 1 -- 172.21.15.46:0/344756132 wait complete. 2024-09-10T07:42:34.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.545+0000 7ff9acc36740 20 client.26867 trim_cache size 0 max 16384 2024-09-10T07:42:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.566+0000 7ff9acc36740 1 Processor -- start 2024-09-10T07:42:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.566+0000 7ff9acc36740 1 -- start start 2024-09-10T07:42:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 0x55e216b64000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b83c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b84150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9acc36740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b0cc0 con 0x55e216b60cb0 2024-09-10T07:42:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9acc36740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b2860 con 0x55e216b6be90 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9acc36740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b1390 con 0x55e216b615d0 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b83c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b83c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57574/0 (socket says 172.21.15.46:57574) 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9a2a00640 1 -- 172.21.15.46:0/3479135898 learned_addr learned my addr 172.21.15.46:0/3479135898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.567+0000 7ff9a3e00640 1 --2- 172.21.15.46:0/3479135898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b84150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.568+0000 7ff9a2a00640 1 -- 172.21.15.46:0/3479135898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 msgr2=0x55e216b84150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.568+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/3479135898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b84150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.568+0000 7ff9a2a00640 1 -- 172.21.15.46:0/3479135898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 msgr2=0x55e216b64000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.568+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/3479135898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 0x55e216b64000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.568+0000 7ff9a2a00640 1 -- 172.21.15.46:0/3479135898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e216b67a20 con 0x55e216b615d0 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.568+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/3479135898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b83c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c00 tx=0x7ff990005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=949871cb764e6b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.569+0000 7ff98f400640 1 -- 172.21.15.46:0/3479135898 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990077070 con 0x55e216b615d0 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.569+0000 7ff98f400640 1 -- 172.21.15.46:0/3479135898 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e216ad4800 con 0x55e216b615d0 2024-09-10T07:42:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.569+0000 7ff98f400640 1 -- 172.21.15.46:0/3479135898 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990035040 con 0x55e216b615d0 2024-09-10T07:42:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.570+0000 7ff9acc36740 1 -- 172.21.15.46:0/3479135898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 msgr2=0x55e216b83c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.570+0000 7ff9acc36740 1 --2- 172.21.15.46:0/3479135898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b83c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c00 tx=0x7ff990005e80 comp rx=0 tx=0).stop 2024-09-10T07:42:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.571+0000 7ff9acc36740 1 -- 172.21.15.46:0/3479135898 shutdown_connections 2024-09-10T07:42:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.571+0000 7ff9acc36740 1 --2- 172.21.15.46:0/3479135898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b6be90 0x55e216b84150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.571+0000 7ff9acc36740 1 --2- 172.21.15.46:0/3479135898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b615d0 0x55e216b83c10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.571+0000 7ff9acc36740 1 --2- 172.21.15.46:0/3479135898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 0x55e216b64000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.571+0000 7ff9acc36740 1 -- 172.21.15.46:0/3479135898 >> 172.21.15.46:0/3479135898 conn(0x55e216b5d240 msgr2=0x55e216b638f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.572+0000 7ff9acc36740 1 -- 172.21.15.46:0/3479135898 shutdown_connections 2024-09-10T07:42:34.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.572+0000 7ff9acc36740 1 -- 172.21.15.46:0/3479135898 wait complete. 2024-09-10T07:42:34.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.573+0000 7ff9acc36740 1 -- start start 2024-09-10T07:42:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.573+0000 7ff9acc36740 1 Processor -- start 2024-09-10T07:42:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.577+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 0x55e216b70800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.577+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b615d0 0x55e216b70d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.578+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 0x55e216b70800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.578+0000 7ff9acc36740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b6be90 0x55e216bdeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.579+0000 7ff9acc36740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b2130 con 0x55e216b60cb0 2024-09-10T07:42:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.579+0000 7ff9acc36740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b1a60 con 0x55e216b615d0 2024-09-10T07:42:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.580+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 0x55e216b70800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55924/0 (socket says 172.21.15.46:55924) 2024-09-10T07:42:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.580+0000 7ff9a3400640 1 -- 172.21.15.46:0/169460308 learned_addr learned my addr 172.21.15.46:0/169460308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.580+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41aeca0 con 0x55e216b6be90 2024-09-10T07:42:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.581+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b615d0 0x55e216b70d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.582+0000 7ff9acc36740 10 client.0 _ll_register_callbacks cb 0x7ff99400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.583+0000 7ff9a3e00640 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b6be90 0x55e216bdeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.584+0000 7ff9a2a00640 1 -- 172.21.15.46:0/169460308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b6be90 msgr2=0x55e216bdeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.584+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b6be90 0x55e216bdeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.584+0000 7ff9a2a00640 1 -- 172.21.15.46:0/169460308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 msgr2=0x55e216b70800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.584+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 0x55e216b70800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.584+0000 7ff9a2a00640 1 -- 172.21.15.46:0/169460308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e216bdf370 con 0x55e216b615d0 2024-09-10T07:42:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.584+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e216ad4800 con 0x55e216b615d0 2024-09-10T07:42:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.586+0000 7ff9a2a00640 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b615d0 0x55e216b70d40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55e216a82010 tx=0x7ff9900095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17667dbdb5f8ec94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.586+0000 7ff98cc00640 10 client.26873 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.587+0000 7ff98cc00640 1 -- 172.21.15.46:0/169460308 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990080060 con 0x55e216b615d0 2024-09-10T07:42:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.587+0000 7ff98cc00640 1 -- 172.21.15.46:0/169460308 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff9900037a0 con 0x55e216b615d0 2024-09-10T07:42:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.587+0000 7ff98cc00640 1 -- 172.21.15.46:0/169460308 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff990035070 con 0x55e216b615d0 2024-09-10T07:42:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.587+0000 7ff9acc36740 10 client.26873 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.587+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55e216be7650 con 0x55e216b615d0 2024-09-10T07:42:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.588+0000 7ff98cc00640 1 -- 172.21.15.46:0/169460308 <== mon.2 v2:172.21.15.63:3300/0 4 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7ff99007a030 con 0x55e216b615d0 2024-09-10T07:42:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.589+0000 7ff9acc36740 10 client.26873 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.589+0000 7ff9acc36740 10 client.26873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.589+0000 7ff9acc36740 20 client.26873 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.589+0000 7ff9acc36740 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e216be7820 0x55e216be9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.589+0000 7ff9acc36740 4 client.26873 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:42:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.589+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e216aced10 con 0x55e216be7820 2024-09-10T07:42:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.590+0000 7ff9a3400640 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e216be7820 0x55e216be9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.592+0000 7ff98cc00640 10 client.26873 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.592+0000 7ff9a3400640 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e216be7820 0x55e216be9c10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd20a7a6e158244 server_cookie=ad01eaf57dcc4a1e in_seq=0 out_seq=0 2024-09-10T07:42:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff98cc00640 1 -- 172.21.15.46:0/169460308 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e216aced10 con 0x55e216be7820 2024-09-10T07:42:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff98cc00640 10 client.26873 handle_command_reply: tid=0 2024-09-10T07:42:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 client.26873 shutdown 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b615d0 msgr2=0x55e216b70d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b615d0 0x55e216b70d40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55e216a82010 tx=0x7ff9900095b0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 shutdown_connections 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e216be7820 0x55e216be9c10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e216b6be90 0x55e216bdeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e216b615d0 0x55e216b70d40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 --2- 172.21.15.46:0/169460308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e216b60cb0 0x55e216b70800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 >> 172.21.15.46:0/169460308 conn(0x55e216b5d240 msgr2=0x55e216b6aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.595+0000 7ff98cc00640 0 client.26873 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.597+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 shutdown_connections 2024-09-10T07:42:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.597+0000 7ff9acc36740 1 -- 172.21.15.46:0/169460308 wait complete. 2024-09-10T07:42:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.597+0000 7ff9acc36740 20 client.26873 trim_cache size 0 max 16384 2024-09-10T07:42:34.598 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:42:34.598 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:42:34.598 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "dec8de88-4713-4bf6-88f5-67333b21a4f6", 2024-09-10T07:42:34.598 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:42:34.599 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:42:34.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.600+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9880a2c00 msgr2=0x7ff9880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.600+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9880a2c00 0x7ff9880a50c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005fd0 tx=0x7ff99c009500 comp rx=0 tx=0).stop 2024-09-10T07:42:34.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.600+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3639367475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a41019c0 msgr2=0x7ff9a4198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:34.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.600+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a41019c0 0x7ff9a4198290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff99400e9f0 tx=0x7ff99400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.602+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3639367475 shutdown_connections 2024-09-10T07:42:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.602+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff9880a2c00 0x7ff9880a50c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.602+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a41079c0 0x7ff9a419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.602+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a4102360 0x7ff9a41987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.602+0000 7ff9a8a00640 1 --2- 172.21.15.46:0/3639367475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9a41019c0 0x7ff9a4198290 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.602+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3639367475 >> 172.21.15.46:0/3639367475 conn(0x7ff9a40fd740 msgr2=0x7ff9a40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.602+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3639367475 shutdown_connections 2024-09-10T07:42:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:34.602+0000 7ff9a8a00640 1 -- 172.21.15.46:0/3639367475 wait complete. 2024-09-10T07:42:35.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:34 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:42:35.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:35.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:42:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:34 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:42:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:42:35.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:34 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:42:35.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:35.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 315.0 s, 699.6 tps, lat 145.385 ms stddev 196.177 2024-09-10T07:42:36.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:35 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:42:36.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:42:36.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: pgmap v578: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 591 op/s 2024-09-10T07:42:36.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:42:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:42:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: pgmap v578: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 591 op/s 2024-09-10T07:42:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: pgmap v578: 129 pgs: 129 active+clean; 1.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 591 op/s 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:42:36.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:42:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:38.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:37 smithi046 ceph-mon[30705]: pgmap v579: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 19 MiB/s wr, 817 op/s 2024-09-10T07:42:38.230 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:37 smithi181 ceph-mon[35397]: pgmap v579: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 19 MiB/s wr, 817 op/s 2024-09-10T07:42:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:37 smithi063 ceph-mon[35020]: pgmap v579: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 19 MiB/s wr, 817 op/s 2024-09-10T07:42:38.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.865+0000 7f6281200640 1 -- 172.21.15.46:0/2492969104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 msgr2=0x7f627c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.866+0000 7f6281200640 1 --2- 172.21.15.46:0/2492969104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 0x7f627c110ae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6274009ca0 tx=0x7f627402a8e0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.866+0000 7f6281200640 1 -- 172.21.15.46:0/2492969104 shutdown_connections 2024-09-10T07:42:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.866+0000 7f6281200640 1 --2- 172.21.15.46:0/2492969104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 0x7f627c110ae0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.866+0000 7f6281200640 1 --2- 172.21.15.46:0/2492969104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c1047e0 0x7f627c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.866+0000 7f6281200640 1 --2- 172.21.15.46:0/2492969104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f627c103e30 0x7f627c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.866+0000 7f6281200640 1 -- 172.21.15.46:0/2492969104 >> 172.21.15.46:0/2492969104 conn(0x7f627c0ff910 msgr2=0x7f627c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:38.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.867+0000 7f6281200640 1 -- 172.21.15.46:0/2492969104 shutdown_connections 2024-09-10T07:42:38.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.868+0000 7f6281200640 1 -- 172.21.15.46:0/2492969104 wait complete. 2024-09-10T07:42:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.871+0000 7f6281200640 1 Processor -- start 2024-09-10T07:42:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.872+0000 7f6281200640 1 -- start start 2024-09-10T07:42:38.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.872+0000 7f6281200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f627c103e30 0x7f627c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.873+0000 7f6281200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c1047e0 0x7f627c076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.873+0000 7f6281200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 0x7f627c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.873+0000 7f627b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f627c103e30 0x7f627c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.874+0000 7f627b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f627c103e30 0x7f627c076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36362/0 (socket says 172.21.15.46:36362) 2024-09-10T07:42:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.874+0000 7f627b400640 1 -- 172.21.15.46:0/1109285622 learned_addr learned my addr 172.21.15.46:0/1109285622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.874+0000 7f627be00640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 0x7f627c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.874+0000 7f627aa00640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c1047e0 0x7f627c076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.875+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f627c113fc0 con 0x7f627c103e30 2024-09-10T07:42:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.875+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f627c113e40 con 0x7f627c108f60 2024-09-10T07:42:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.875+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f627c114140 con 0x7f627c1047e0 2024-09-10T07:42:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.876+0000 7f627be00640 1 -- 172.21.15.46:0/1109285622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c1047e0 msgr2=0x7f627c076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.876+0000 7f627be00640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c1047e0 0x7f627c076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.876+0000 7f627be00640 1 -- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f627c103e30 msgr2=0x7f627c076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.876+0000 7f627be00640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f627c103e30 0x7f627c076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.876+0000 7f627be00640 1 -- 172.21.15.46:0/1109285622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c1aaa40 con 0x7f627c108f60 2024-09-10T07:42:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.876+0000 7f627aa00640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c1047e0 0x7f627c076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.876+0000 7f627b400640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f627c103e30 0x7f627c076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.876+0000 7f627be00640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 0x7f627c077180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6274009ca0 tx=0x7f6274030880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.877+0000 7f6273e00640 1 -- 172.21.15.46:0/1109285622 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627403f070 con 0x7f627c108f60 2024-09-10T07:42:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.877+0000 7f6273e00640 1 -- 172.21.15.46:0/1109285622 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6274030e40 con 0x7f627c108f60 2024-09-10T07:42:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.877+0000 7f6273e00640 1 -- 172.21.15.46:0/1109285622 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627403b900 con 0x7f627c108f60 2024-09-10T07:42:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.877+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627c1aacd0 con 0x7f627c108f60 2024-09-10T07:42:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.877+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f627c1ab1c0 con 0x7f627c108f60 2024-09-10T07:42:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.883+0000 7f628528c740 1 Processor -- start 2024-09-10T07:42:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.883+0000 7f628528c740 1 -- start start 2024-09-10T07:42:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.883+0000 7f6273e00640 1 -- 172.21.15.46:0/1109285622 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f627403baa0 con 0x7f627c108f60 2024-09-10T07:42:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.884+0000 7f628528c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x555706504130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.884+0000 7f628528c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706504670 0x55570650e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.884+0000 7f628528c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 0x555706511320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.884+0000 7f628528c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55570646a3e0 con 0x55570650ef30 2024-09-10T07:42:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.884+0000 7f628528c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555706411d80 con 0x555706503d50 2024-09-10T07:42:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.884+0000 7f628528c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5557064663a0 con 0x555706504670 2024-09-10T07:42:38.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.884+0000 7f627aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x555706504130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.885+0000 7f627aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x555706504130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45092/0 (socket says 172.21.15.46:45092) 2024-09-10T07:42:38.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.885+0000 7f627aa00640 1 -- 172.21.15.46:0/1838141617 learned_addr learned my addr 172.21.15.46:0/1838141617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:38.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.885+0000 7f627aa00640 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 0x555706511320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.885+0000 7f627b400640 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706504670 0x55570650e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.886+0000 7f627aa00640 1 -- 172.21.15.46:0/1838141617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706504670 msgr2=0x55570650e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.886+0000 7f627aa00640 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706504670 0x55570650e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.886+0000 7f627aa00640 1 -- 172.21.15.46:0/1838141617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 msgr2=0x555706511320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.886+0000 7f627aa00640 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 0x555706511320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.886+0000 7f627aa00640 1 -- 172.21.15.46:0/1838141617 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555706430ae0 con 0x555706503d50 2024-09-10T07:42:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.886+0000 7f627aa00640 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x555706504130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6268005940 tx=0x7f626800a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd32e944be1992cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.887+0000 7f6270c00640 1 -- 172.21.15.46:0/1838141617 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268005220 con 0x555706503d50 2024-09-10T07:42:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.887+0000 7f6270c00640 1 -- 172.21.15.46:0/1838141617 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6268004c80 con 0x555706503d50 2024-09-10T07:42:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.887+0000 7f6270c00640 1 -- 172.21.15.46:0/1838141617 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268011840 con 0x555706503d50 2024-09-10T07:42:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.888+0000 7f6273e00640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f62600a2cc0 0x7f62600a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.888+0000 7f628528c740 1 -- 172.21.15.46:0/1838141617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 msgr2=0x555706504130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.888+0000 7f628528c740 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x555706504130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6268005940 tx=0x7f626800a5f0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.889+0000 7f6273e00640 1 -- 172.21.15.46:0/1109285622 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f62740f0de0 con 0x7f627c108f60 2024-09-10T07:42:38.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.889+0000 7f627b400640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f62600a2cc0 0x7f62600a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.890+0000 7f627b400640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f62600a2cc0 0x7f62600a5180 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x555706501140 tx=0x7f626c00c330 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.890+0000 7f628528c740 1 -- 172.21.15.46:0/1838141617 shutdown_connections 2024-09-10T07:42:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.890+0000 7f628528c740 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 0x555706511320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.890+0000 7f628528c740 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706504670 0x55570650e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.890+0000 7f628528c740 1 --2- 172.21.15.46:0/1838141617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x555706504130 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.890+0000 7f628528c740 1 -- 172.21.15.46:0/1838141617 >> 172.21.15.46:0/1838141617 conn(0x5557065002e0 msgr2=0x5557065006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.891+0000 7f628528c740 1 -- 172.21.15.46:0/1838141617 shutdown_connections 2024-09-10T07:42:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.891+0000 7f628528c740 1 -- 172.21.15.46:0/1838141617 wait complete. 2024-09-10T07:42:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.892+0000 7f628528c740 1 -- start start 2024-09-10T07:42:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.892+0000 7f628528c740 1 Processor -- start 2024-09-10T07:42:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.894+0000 7f628528c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x55570650be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.894+0000 7f628528c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706504670 0x55570650c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.894+0000 7f628528c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x55570650c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.894+0000 7f628528c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555706522bb0 con 0x555706504670 2024-09-10T07:42:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.894+0000 7f628528c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5557064250f0 con 0x555706503d50 2024-09-10T07:42:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.894+0000 7f628528c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555706522d30 con 0x55570650ef30 2024-09-10T07:42:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.894+0000 7f628528c740 10 client.0 _ll_register_callbacks cb 0x5557064169e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.894+0000 7f627b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x55570650be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.895+0000 7f627b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x55570650be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45094/0 (socket says 172.21.15.46:45094) 2024-09-10T07:42:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.895+0000 7f627b400640 1 -- 172.21.15.46:0/3430801498 learned_addr learned my addr 172.21.15.46:0/3430801498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.895+0000 7f627b400640 1 -- 172.21.15.46:0/3430801498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 msgr2=0x55570650c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.895+0000 7f627b400640 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x55570650c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.895+0000 7f627b400640 1 -- 172.21.15.46:0/3430801498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706504670 msgr2=0x55570650c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.895+0000 7f627b400640 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706504670 0x55570650c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.895+0000 7f627b400640 1 -- 172.21.15.46:0/3430801498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557061db3c0 con 0x555706503d50 2024-09-10T07:42:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.896+0000 7f627b400640 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x55570650be90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f626c011e70 tx=0x7f626c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d20338bc5ee6bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.896+0000 7f6265600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.897+0000 7f6265600640 1 -- 172.21.15.46:0/3430801498 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c005020 con 0x555706503d50 2024-09-10T07:42:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.897+0000 7f6265600640 1 -- 172.21.15.46:0/3430801498 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f626c0051c0 con 0x555706503d50 2024-09-10T07:42:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.897+0000 7f628528c740 1 -- 172.21.15.46:0/3430801498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6268004c80 con 0x555706503d50 2024-09-10T07:42:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.898+0000 7f6265600640 1 -- 172.21.15.46:0/3430801498 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f626c0053c0 con 0x555706503d50 2024-09-10T07:42:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.898+0000 7f628528c740 10 client.26885 fetch_fsmap learned FSMap version 42 2024-09-10T07:42:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.898+0000 7f628528c740 1 -- 172.21.15.46:0/3430801498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55570652e940 con 0x555706503d50 2024-09-10T07:42:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.899+0000 7f6265600640 1 -- 172.21.15.46:0/3430801498 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6268004c80 con 0x555706503d50 2024-09-10T07:42:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.899+0000 7f6265600640 1 -- 172.21.15.46:0/3430801498 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f626c030650 con 0x555706503d50 2024-09-10T07:42:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.900+0000 7f628528c740 10 client.26885 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T07:42:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.901+0000 7f628528c740 10 client.26885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.901+0000 7f628528c740 20 client.26885 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.901+0000 7f628528c740 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55570652fef0 0x555706532380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.901+0000 7f628528c740 4 client.26885 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.901+0000 7f628528c740 1 -- 172.21.15.46:0/3430801498 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555706477950 con 0x55570652fef0 2024-09-10T07:42:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.903+0000 7f627aa00640 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55570652fef0 0x555706532380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.904+0000 7f627aa00640 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55570652fef0 0x555706532380 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81de7cdee4384492 server_cookie=fb94020ff5da9cd1 in_seq=0 out_seq=0 2024-09-10T07:42:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.904+0000 7f6265600640 10 client.26885 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.905+0000 7f6265600640 1 -- 172.21.15.46:0/3430801498 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555706477950 con 0x55570652fef0 2024-09-10T07:42:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.905+0000 7f6265600640 10 client.26885 handle_command_reply: tid=0 2024-09-10T07:42:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.906+0000 7f628528c740 1 client.26885 shutdown 2024-09-10T07:42:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.906+0000 7f628528c740 1 -- 172.21.15.46:0/3430801498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 msgr2=0x55570650be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.906+0000 7f628528c740 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x55570650be90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f626c011e70 tx=0x7f626c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.907+0000 7f628528c740 1 -- 172.21.15.46:0/3430801498 shutdown_connections 2024-09-10T07:42:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.907+0000 7f628528c740 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55570652fef0 0x555706532380 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.907+0000 7f628528c740 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x55570650c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.907+0000 7f628528c740 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706504670 0x55570650c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.907+0000 7f628528c740 1 --2- 172.21.15.46:0/3430801498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706503d50 0x55570650be90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.907+0000 7f628528c740 1 -- 172.21.15.46:0/3430801498 >> 172.21.15.46:0/3430801498 conn(0x5557065002e0 msgr2=0x555706505fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.907+0000 7f6265600640 0 client.26885 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.908+0000 7f628528c740 1 -- 172.21.15.46:0/3430801498 shutdown_connections 2024-09-10T07:42:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.908+0000 7f628528c740 1 -- 172.21.15.46:0/3430801498 wait complete. 2024-09-10T07:42:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.909+0000 7f628528c740 20 client.26885 trim_cache size 0 max 16384 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.945+0000 7f628528c740 1 Processor -- start 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.945+0000 7f628528c740 1 -- start start 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f628528c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706503d50 0x55570650c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f628528c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x55570650de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f628528c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 0x555706537c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f628528c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ac7e0 con 0x55570650ef30 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f628528c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ae380 con 0x555706504670 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f628528c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1aceb0 con 0x555706503d50 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f627b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706503d50 0x55570650c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f627b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706503d50 0x55570650c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34048/0 (socket says 172.21.15.46:34048) 2024-09-10T07:42:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.946+0000 7f627b400640 1 -- 172.21.15.46:0/3155987381 learned_addr learned my addr 172.21.15.46:0/3155987381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.947+0000 7f627aa00640 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x55570650de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.947+0000 7f627be00640 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 0x555706537c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.947+0000 7f627b400640 1 -- 172.21.15.46:0/3155987381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 msgr2=0x55570650de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.947+0000 7f627b400640 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x55570650de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.947+0000 7f627b400640 1 -- 172.21.15.46:0/3155987381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 msgr2=0x555706537c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.947+0000 7f627b400640 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 0x555706537c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.947+0000 7f627b400640 1 -- 172.21.15.46:0/3155987381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555706504ef0 con 0x555706503d50 2024-09-10T07:42:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.948+0000 7f627b400640 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706503d50 0x55570650c760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f626c010cd0 tx=0x7f626c005350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e7cecedbd98750 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.948+0000 7f6267e00640 1 -- 172.21.15.46:0/3155987381 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c0053c0 con 0x555706503d50 2024-09-10T07:42:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.948+0000 7f6267e00640 1 -- 172.21.15.46:0/3155987381 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f626c03f070 con 0x555706503d50 2024-09-10T07:42:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.948+0000 7f6267e00640 1 -- 172.21.15.46:0/3155987381 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c004970 con 0x555706503d50 2024-09-10T07:42:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.949+0000 7f628528c740 1 -- 172.21.15.46:0/3155987381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706503d50 msgr2=0x55570650c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.949+0000 7f628528c740 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706503d50 0x55570650c760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f626c010cd0 tx=0x7f626c005350 comp rx=0 tx=0).stop 2024-09-10T07:42:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.950+0000 7f628528c740 1 -- 172.21.15.46:0/3155987381 shutdown_connections 2024-09-10T07:42:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.950+0000 7f628528c740 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55570650ef30 0x555706537c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.950+0000 7f628528c740 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x55570650de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.950+0000 7f628528c740 1 --2- 172.21.15.46:0/3155987381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555706503d50 0x55570650c760 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.950+0000 7f628528c740 1 -- 172.21.15.46:0/3155987381 >> 172.21.15.46:0/3155987381 conn(0x5557065002e0 msgr2=0x55570650d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.950+0000 7f628528c740 1 -- 172.21.15.46:0/3155987381 shutdown_connections 2024-09-10T07:42:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.950+0000 7f628528c740 1 -- 172.21.15.46:0/3155987381 wait complete. 2024-09-10T07:42:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.951+0000 7f628528c740 1 -- start start 2024-09-10T07:42:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.951+0000 7f628528c740 1 Processor -- start 2024-09-10T07:42:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.953+0000 7f628528c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706503d50 0x5557065236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.953+0000 7f628528c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x555706523c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.953+0000 7f628528c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x555706598c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.954+0000 7f627aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x555706523c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.954+0000 7f628528c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1adc50 con 0x555706503d50 2024-09-10T07:42:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.954+0000 7f628528c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ad580 con 0x555706504670 2024-09-10T07:42:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.954+0000 7f627be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x555706598c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.955+0000 7f627be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x555706598c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34050/0 (socket says 172.21.15.46:34050) 2024-09-10T07:42:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.955+0000 7f627be00640 1 -- 172.21.15.46:0/3906533707 learned_addr learned my addr 172.21.15.46:0/3906533707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.955+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f627c0736f0 con 0x55570650ef30 2024-09-10T07:42:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.955+0000 7f627b400640 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706503d50 0x5557065236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.955+0000 7f628528c740 10 client.0 _ll_register_callbacks cb 0x7f626c011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.956+0000 7f627be00640 1 -- 172.21.15.46:0/3906533707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 msgr2=0x555706523c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.956+0000 7f627be00640 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x555706523c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.956+0000 7f627be00640 1 -- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706503d50 msgr2=0x5557065236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.956+0000 7f627be00640 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706503d50 0x5557065236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.956+0000 7f627be00640 1 -- 172.21.15.46:0/3906533707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555706599210 con 0x55570650ef30 2024-09-10T07:42:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.956+0000 7f627b400640 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706503d50 0x5557065236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.956+0000 7f627aa00640 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x555706523c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.957+0000 7f627be00640 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x555706598c50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f62740c6930 tx=0x7f62740be8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c223cdecf0bce37d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.957+0000 7f6265600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.957+0000 7f6265600640 1 -- 172.21.15.46:0/3906533707 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740be1f0 con 0x55570650ef30 2024-09-10T07:42:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.957+0000 7f6265600640 1 -- 172.21.15.46:0/3906533707 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f62740c2170 con 0x55570650ef30 2024-09-10T07:42:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.957+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f626c03f070 con 0x55570650ef30 2024-09-10T07:42:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.957+0000 7f6265600640 1 -- 172.21.15.46:0/3906533707 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f62740c2370 con 0x55570650ef30 2024-09-10T07:42:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.958+0000 7f628528c740 10 client.26857 fetch_fsmap learned FSMap version 43 2024-09-10T07:42:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.958+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55570659a4e0 con 0x55570650ef30 2024-09-10T07:42:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.958+0000 7f6265600640 1 -- 172.21.15.46:0/3906533707 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f626c03f070 con 0x55570650ef30 2024-09-10T07:42:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.958+0000 7f6265600640 1 -- 172.21.15.46:0/3906533707 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f627404c070 con 0x55570650ef30 2024-09-10T07:42:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.959+0000 7f628528c740 10 client.26857 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:42:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.959+0000 7f628528c740 10 client.26857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.959+0000 7f628528c740 20 client.26857 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.959+0000 7f628528c740 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55570659abc0 0x55570659cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.959+0000 7f628528c740 4 client.26857 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:42:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.959+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555706471e60 con 0x55570659abc0 2024-09-10T07:42:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.960+0000 7f627b400640 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55570659abc0 0x55570659cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.961+0000 7f627b400640 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55570659abc0 0x55570659cfe0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b310e1f766997be server_cookie=7d52b91b8ea49ee4 in_seq=0 out_seq=0 2024-09-10T07:42:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.961+0000 7f6265600640 10 client.26857 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:38.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f6265600640 1 -- 172.21.15.46:0/3906533707 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555706471e60 con 0x55570659abc0 2024-09-10T07:42:38.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f6265600640 10 client.26857 handle_command_reply: tid=0 2024-09-10T07:42:38.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 client.26857 shutdown 2024-09-10T07:42:38.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 msgr2=0x555706598c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x555706598c50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f62740c6930 tx=0x7f62740be8f0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 shutdown_connections 2024-09-10T07:42:38.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55570659abc0 0x55570659cfe0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55570650ef30 0x555706598c50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555706504670 0x555706523c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 --2- 172.21.15.46:0/3906533707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555706503d50 0x5557065236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f6265600640 0 client.26857 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:38.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 >> 172.21.15.46:0/3906533707 conn(0x5557065002e0 msgr2=0x55570650a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:38.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.962+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 shutdown_connections 2024-09-10T07:42:38.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.965+0000 7f628528c740 1 -- 172.21.15.46:0/3906533707 wait complete. 2024-09-10T07:42:38.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.965+0000 7f628528c740 20 client.26857 trim_cache size 0 max 16384 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout: "dec8de88-4713-4bf6-88f5-67333b21a4f6": { 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "dec8de88-4713-4bf6-88f5-67333b21a4f6", 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:42:38.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:42:38.968 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:42:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f62600a2cc0 msgr2=0x7f62600a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f62600a2cc0 0x7f62600a5180 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x555706501140 tx=0x7f626c00c330 comp rx=0 tx=0).stop 2024-09-10T07:42:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 msgr2=0x7f627c077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 0x7f627c077180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6274009ca0 tx=0x7f6274030880 comp rx=0 tx=0).stop 2024-09-10T07:42:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 shutdown_connections 2024-09-10T07:42:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f627c108f60 0x7f627c077180 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c1047e0 0x7f627c076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f62600a2cc0 0x7f62600a5180 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 --2- 172.21.15.46:0/1109285622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f627c103e30 0x7f627c076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:38.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 >> 172.21.15.46:0/1109285622 conn(0x7f627c0ff910 msgr2=0x7f627c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:38.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.967+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 shutdown_connections 2024-09-10T07:42:38.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:38.968+0000 7f6281200640 1 -- 172.21.15.46:0/1109285622 wait complete. 2024-09-10T07:42:39.984 INFO:tasks.cephfs.filesystem:scrub status for tag:dec8de88-4713-4bf6-88f5-67333b21a4f6 - {'path': '/', 'tag': 'dec8de88-4713-4bf6-88f5-67333b21a4f6', 'options': 'recursive,force'} 2024-09-10T07:42:40.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:39 smithi046 ceph-mon[30705]: pgmap v580: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 14 MiB/s wr, 580 op/s 2024-09-10T07:42:40.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:39 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:42:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.349+0000 7f7ab1600640 1 -- 172.21.15.46:0/311460779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac1099f0 msgr2=0x7f7aac10beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.349+0000 7f7ab1600640 1 --2- 172.21.15.46:0/311460779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac1099f0 0x7f7aac10beb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009ca0 tx=0x7f7aa402a940 comp rx=0 tx=0).stop 2024-09-10T07:42:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.349+0000 7f7ab1600640 1 -- 172.21.15.46:0/311460779 shutdown_connections 2024-09-10T07:42:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.349+0000 7f7ab1600640 1 --2- 172.21.15.46:0/311460779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac1099f0 0x7f7aac10beb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.349+0000 7f7ab1600640 1 --2- 172.21.15.46:0/311460779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac074bf0 0x7f7aac075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.349+0000 7f7ab1600640 1 --2- 172.21.15.46:0/311460779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac0761d0 0x7f7aac074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.349+0000 7f7ab1600640 1 -- 172.21.15.46:0/311460779 >> 172.21.15.46:0/311460779 conn(0x7f7aac0fd2c0 msgr2=0x7f7aac0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.351+0000 7f7ab1600640 1 -- 172.21.15.46:0/311460779 shutdown_connections 2024-09-10T07:42:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.351+0000 7f7ab1600640 1 -- 172.21.15.46:0/311460779 wait complete. 2024-09-10T07:42:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.353+0000 7f7ab1600640 1 Processor -- start 2024-09-10T07:42:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.354+0000 7f7ab1600640 1 -- start start 2024-09-10T07:42:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.355+0000 7f7ab1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac074bf0 0x7f7aac199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.355+0000 7f7ab1600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac0761d0 0x7f7aac19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.357+0000 7f7ab1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac1099f0 0x7f7aac1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.357+0000 7f7ab1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac10fa20 con 0x7f7aac074bf0 2024-09-10T07:42:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.357+0000 7f7ab1600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac10f8a0 con 0x7f7aac0761d0 2024-09-10T07:42:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.357+0000 7f7ab1600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac10fba0 con 0x7f7aac1099f0 2024-09-10T07:42:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.357+0000 7f7aaa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac0761d0 0x7f7aac19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.357+0000 7f7aaa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac0761d0 0x7f7aac19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45146/0 (socket says 172.21.15.46:45146) 2024-09-10T07:42:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.357+0000 7f7aaa000640 1 -- 172.21.15.46:0/271089722 learned_addr learned my addr 172.21.15.46:0/271089722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.357+0000 7f7aab400640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac1099f0 0x7f7aac1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.358+0000 7f7aaa000640 1 -- 172.21.15.46:0/271089722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac1099f0 msgr2=0x7f7aac1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.358+0000 7f7aaa000640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac1099f0 0x7f7aac1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.358+0000 7f7aaa000640 1 -- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac074bf0 msgr2=0x7f7aac199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.358+0000 7f7aaa000640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac074bf0 0x7f7aac199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.358+0000 7f7aaa000640 1 -- 172.21.15.46:0/271089722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac1a1b70 con 0x7f7aac0761d0 2024-09-10T07:42:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.358+0000 7f7aab400640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac1099f0 0x7f7aac1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.358+0000 7f7aaa000640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac0761d0 0x7f7aac19a490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800d950 tx=0x7f7a9800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.360+0000 7f7aa3400640 1 -- 172.21.15.46:0/271089722 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98017070 con 0x7f7aac0761d0 2024-09-10T07:42:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.360+0000 7f7aa3400640 1 -- 172.21.15.46:0/271089722 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a98009c30 con 0x7f7aac0761d0 2024-09-10T07:42:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.360+0000 7f7aa3400640 1 -- 172.21.15.46:0/271089722 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9800f6b0 con 0x7f7aac0761d0 2024-09-10T07:42:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.360+0000 7f7ab1600640 1 -- 172.21.15.46:0/271089722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac1a1e60 con 0x7f7aac0761d0 2024-09-10T07:42:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.360+0000 7f7ab1600640 1 -- 172.21.15.46:0/271089722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac1a22d0 con 0x7f7aac0761d0 2024-09-10T07:42:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.365+0000 7f7ab443f740 1 Processor -- start 2024-09-10T07:42:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.367+0000 7f7aa3400640 1 -- 172.21.15.46:0/271089722 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a9800f850 con 0x7f7aac0761d0 2024-09-10T07:42:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.368+0000 7f7ab443f740 1 -- start start 2024-09-10T07:42:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.369+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b34862d090 0x55b34862d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.369+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d9d0 0x55b348632490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.369+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b348632a60 0x55b348634f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.369+0000 7f7aa3400640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2cb0 0x7f7a900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.369+0000 7f7ab443f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b348592100 con 0x55b34862d090 2024-09-10T07:42:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.369+0000 7f7ab443f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b34853a1b0 con 0x55b34862d9d0 2024-09-10T07:42:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.369+0000 7f7ab443f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b34858e0c0 con 0x55b348632a60 2024-09-10T07:42:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.369+0000 7f7aa3400640 1 -- 172.21.15.46:0/271089722 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7a980cefc0 con 0x7f7aac0761d0 2024-09-10T07:42:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d9d0 0x55b348632490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d9d0 0x55b348632490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45154/0 (socket says 172.21.15.46:45154) 2024-09-10T07:42:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 -- 172.21.15.46:0/2064236639 learned_addr learned my addr 172.21.15.46:0/2064236639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 -- 172.21.15.46:0/2064236639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b348632a60 msgr2=0x55b348634f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/2064236639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b348632a60 0x55b348634f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 -- 172.21.15.46:0/2064236639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b34862d090 msgr2=0x55b34862d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/2064236639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b34862d090 0x55b34862d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 -- 172.21.15.46:0/2064236639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3482a5210 con 0x55b34862d9d0 2024-09-10T07:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.370+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/2064236639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d9d0 0x55b348632490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009c40 tx=0x7f7a9c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76e32d301dcc6211 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.371+0000 7f7a97400640 1 -- 172.21.15.46:0/2064236639 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c005020 con 0x55b34862d9d0 2024-09-10T07:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.371+0000 7f7a97400640 1 -- 172.21.15.46:0/2064236639 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0047a0 con 0x55b34862d9d0 2024-09-10T07:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.371+0000 7f7a97400640 1 -- 172.21.15.46:0/2064236639 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c010690 con 0x55b34862d9d0 2024-09-10T07:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.372+0000 7f7aab400640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2cb0 0x7f7a900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.372+0000 7f7ab443f740 1 -- 172.21.15.46:0/2064236639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d9d0 msgr2=0x55b348632490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.372+0000 7f7ab443f740 1 --2- 172.21.15.46:0/2064236639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d9d0 0x55b348632490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009c40 tx=0x7f7a9c00e720 comp rx=0 tx=0).stop 2024-09-10T07:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.373+0000 7f7ab443f740 1 -- 172.21.15.46:0/2064236639 shutdown_connections 2024-09-10T07:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.373+0000 7f7ab443f740 1 --2- 172.21.15.46:0/2064236639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b348632a60 0x55b348634f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.373+0000 7f7ab443f740 1 --2- 172.21.15.46:0/2064236639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d9d0 0x55b348632490 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.373+0000 7f7ab443f740 1 --2- 172.21.15.46:0/2064236639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b34862d090 0x55b34862d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.373+0000 7f7ab443f740 1 -- 172.21.15.46:0/2064236639 >> 172.21.15.46:0/2064236639 conn(0x55b348628700 msgr2=0x55b348628b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.373+0000 7f7ab443f740 1 -- 172.21.15.46:0/2064236639 shutdown_connections 2024-09-10T07:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.374+0000 7f7ab443f740 1 -- 172.21.15.46:0/2064236639 wait complete. 2024-09-10T07:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.374+0000 7f7aab400640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2cb0 0x7f7a900a5170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006010 tx=0x7f7aa403e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.374+0000 7f7ab443f740 1 -- start start 2024-09-10T07:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.375+0000 7f7ab443f740 1 Processor -- start 2024-09-10T07:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.375+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348648300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.375+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b348648840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.376+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348648300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.376+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348648300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45162/0 (socket says 172.21.15.46:45162) 2024-09-10T07:42:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.376+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aaa000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b348648840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aaa000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b348648840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34084/0 (socket says 172.21.15.46:34084) 2024-09-10T07:42:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36424/0 (socket says 172.21.15.46:36424) 2024-09-10T07:42:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7ab443f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b348635a30 con 0x55b348632a60 2024-09-10T07:42:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7ab443f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b34851b090 con 0x55b34862d090 2024-09-10T07:42:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7ab443f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b348635bb0 con 0x55b34862d9d0 2024-09-10T07:42:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7ab443f740 10 client.0 _ll_register_callbacks cb 0x55b34856ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aaaa00640 1 -- 172.21.15.46:0/3134202522 learned_addr learned my addr 172.21.15.46:0/3134202522 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aaa000640 1 -- 172.21.15.46:0/3134202522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 msgr2=0x55b348648300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aaa000640 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348648300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aaa000640 1 -- 172.21.15.46:0/3134202522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 msgr2=0x55b34864f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.378+0000 7f7aaa000640 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.379+0000 7f7aaa000640 1 -- 172.21.15.46:0/3134202522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b348371620 con 0x55b34862d9d0 2024-09-10T07:42:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.379+0000 7f7aaa000640 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b348648840 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7a980cc790 tx=0x7f7a980109d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b487a0bf98546b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.379+0000 7f7a8be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.380+0000 7f7a8be00640 1 -- 172.21.15.46:0/3134202522 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98020040 con 0x55b34862d9d0 2024-09-10T07:42:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.380+0000 7f7a8be00640 1 -- 172.21.15.46:0/3134202522 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7a9809a8f0 con 0x55b34862d9d0 2024-09-10T07:42:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.380+0000 7f7ab443f740 1 -- 172.21.15.46:0/3134202522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9c0047a0 con 0x55b34862d9d0 2024-09-10T07:42:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.380+0000 7f7a8be00640 1 -- 172.21.15.46:0/3134202522 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7a9809b4a0 con 0x55b34862d9d0 2024-09-10T07:42:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.380+0000 7f7ab443f740 10 client.26866 fetch_fsmap learned FSMap version 43 2024-09-10T07:42:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.380+0000 7f7ab443f740 1 -- 172.21.15.46:0/3134202522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b3486526e0 con 0x55b34862d9d0 2024-09-10T07:42:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.381+0000 7f7a8be00640 1 -- 172.21.15.46:0/3134202522 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0047a0 con 0x55b34862d9d0 2024-09-10T07:42:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.381+0000 7f7a8be00640 1 -- 172.21.15.46:0/3134202522 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f7a980b4200 con 0x55b34862d9d0 2024-09-10T07:42:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.381+0000 7f7ab443f740 10 client.26866 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:42:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.381+0000 7f7ab443f740 10 client.26866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.382+0000 7f7ab443f740 20 client.26866 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.382+0000 7f7ab443f740 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b348653db0 0x55b3486561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.382+0000 7f7ab443f740 4 client.26866 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.382+0000 7f7ab443f740 1 -- 172.21.15.46:0/3134202522 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b34859f830 con 0x55b348653db0 2024-09-10T07:42:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.385+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b348653db0 0x55b3486561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.387+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b348653db0 0x55b3486561a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452f602fbdeb69cb server_cookie=f2fec92b1e9af2d1 in_seq=0 out_seq=0 2024-09-10T07:42:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.387+0000 7f7a8be00640 10 client.26866 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.388+0000 7f7a8be00640 1 -- 172.21.15.46:0/3134202522 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b34859f830 con 0x55b348653db0 2024-09-10T07:42:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.388+0000 7f7a8be00640 10 client.26866 handle_command_reply: tid=0 2024-09-10T07:42:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.389+0000 7f7ab443f740 1 client.26866 shutdown 2024-09-10T07:42:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.390+0000 7f7ab443f740 1 -- 172.21.15.46:0/3134202522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 msgr2=0x55b348648840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.390+0000 7f7ab443f740 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b348648840 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7a980cc790 tx=0x7f7a980109d0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.391+0000 7f7ab443f740 1 -- 172.21.15.46:0/3134202522 shutdown_connections 2024-09-10T07:42:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.391+0000 7f7ab443f740 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b348653db0 0x55b3486561a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.391+0000 7f7ab443f740 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.391+0000 7f7ab443f740 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b348648840 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.391+0000 7f7ab443f740 1 --2- 172.21.15.46:0/3134202522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348648300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.391+0000 7f7ab443f740 1 -- 172.21.15.46:0/3134202522 >> 172.21.15.46:0/3134202522 conn(0x55b348628700 msgr2=0x55b348629f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.391+0000 7f7a8be00640 0 client.26866 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.393+0000 7f7ab443f740 1 -- 172.21.15.46:0/3134202522 shutdown_connections 2024-09-10T07:42:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.394+0000 7f7ab443f740 1 -- 172.21.15.46:0/3134202522 wait complete. 2024-09-10T07:42:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.394+0000 7f7ab443f740 20 client.26866 trim_cache size 0 max 16384 2024-09-10T07:42:40.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:39 smithi181 ceph-mon[35397]: pgmap v580: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 14 MiB/s wr, 580 op/s 2024-09-10T07:42:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:39 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:42:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:39 smithi063 ceph-mon[35020]: pgmap v580: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 14 MiB/s wr, 580 op/s 2024-09-10T07:42:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:39 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:42:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7ab443f740 1 Processor -- start 2024-09-10T07:42:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7ab443f740 1 -- start start 2024-09-10T07:42:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348629af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34862a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7ab443f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac071c70 con 0x55b348632a60 2024-09-10T07:42:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7ab443f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac073840 con 0x55b34862d090 2024-09-10T07:42:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7ab443f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac072340 con 0x55b34862d9d0 2024-09-10T07:42:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.440+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348629af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaa000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348629af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45174/0 (socket says 172.21.15.46:45174) 2024-09-10T07:42:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaaa00640 1 -- 172.21.15.46:0/310708607 learned_addr learned my addr 172.21.15.46:0/310708607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aab400640 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34862a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaaa00640 1 -- 172.21.15.46:0/310708607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 msgr2=0x55b34864f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaaa00640 1 -- 172.21.15.46:0/310708607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 msgr2=0x55b34862a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34862a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaaa00640 1 -- 172.21.15.46:0/310708607 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b348652510 con 0x55b34862d090 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.441+0000 7f7aaa000640 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.442+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348629af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009c10 tx=0x7f7a9c009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa16750dc8a185a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.442+0000 7f7a96a00640 1 -- 172.21.15.46:0/310708607 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c01d040 con 0x55b34862d090 2024-09-10T07:42:40.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.442+0000 7f7a96a00640 1 -- 172.21.15.46:0/310708607 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c009840 con 0x55b34862d090 2024-09-10T07:42:40.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.442+0000 7f7a96a00640 1 -- 172.21.15.46:0/310708607 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c02e480 con 0x55b34862d090 2024-09-10T07:42:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.446+0000 7f7ab443f740 1 -- 172.21.15.46:0/310708607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 msgr2=0x55b348629af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.446+0000 7f7ab443f740 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348629af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009c10 tx=0x7f7a9c009df0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.447+0000 7f7ab443f740 1 -- 172.21.15.46:0/310708607 shutdown_connections 2024-09-10T07:42:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.447+0000 7f7ab443f740 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34862a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.447+0000 7f7ab443f740 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.447+0000 7f7ab443f740 1 --2- 172.21.15.46:0/310708607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b348629af0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.447+0000 7f7ab443f740 1 -- 172.21.15.46:0/310708607 >> 172.21.15.46:0/310708607 conn(0x55b348628700 msgr2=0x55b348627880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.448+0000 7f7ab443f740 1 -- 172.21.15.46:0/310708607 shutdown_connections 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.448+0000 7f7ab443f740 1 -- 172.21.15.46:0/310708607 wait complete. 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.448+0000 7f7ab443f740 1 -- start start 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.449+0000 7f7ab443f740 1 Processor -- start 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.450+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b34864ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.450+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.450+0000 7f7ab443f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7ab443f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac073110 con 0x55b348632a60 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b34864ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b34864ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45180/0 (socket says 172.21.15.46:45180) 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7aaaa00640 1 -- 172.21.15.46:0/1122125589 learned_addr learned my addr 172.21.15.46:0/1122125589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7aab400640 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7aaa000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac072a10 con 0x55b34862d090 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac06fc50 con 0x55b34862d9d0 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.451+0000 7f7ab443f740 10 client.0 _ll_register_callbacks cb 0x7f7a9800df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.452+0000 7f7aaaa00640 1 -- 172.21.15.46:0/1122125589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 msgr2=0x55b34864d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.452+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.452+0000 7f7aaaa00640 1 -- 172.21.15.46:0/1122125589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 msgr2=0x55b34864e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.452+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.453+0000 7f7aaaa00640 1 -- 172.21.15.46:0/1122125589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34864e9a0 con 0x55b34862d090 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.454+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b34864ce40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00ec20 tx=0x7f7a9c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e978851a29127cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.454+0000 7f7a8be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.454+0000 7f7a8be00640 1 -- 172.21.15.46:0/1122125589 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004030 con 0x55b34862d090 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.455+0000 7f7a8be00640 1 -- 172.21.15.46:0/1122125589 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7a9c0041d0 con 0x55b34862d090 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.455+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9c009840 con 0x55b34862d090 2024-09-10T07:42:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.455+0000 7f7a8be00640 1 -- 172.21.15.46:0/1122125589 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c0043d0 con 0x55b34862d090 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.455+0000 7f7ab443f740 10 client.26906 fetch_fsmap learned FSMap version 43 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.455+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b3486bea60 con 0x55b34862d090 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.456+0000 7f7a8be00640 1 -- 172.21.15.46:0/1122125589 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c009840 con 0x55b34862d090 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.456+0000 7f7a8be00640 1 -- 172.21.15.46:0/1122125589 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f7a9c005010 con 0x55b34862d090 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.456+0000 7f7ab443f740 10 client.26906 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.456+0000 7f7ab443f740 10 client.26906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.456+0000 7f7ab443f740 20 client.26906 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.456+0000 7f7ab443f740 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b3486bf0c0 0x55b3486c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.456+0000 7f7ab443f740 4 client.26906 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.456+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b348599b80 con 0x55b3486bf0c0 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.457+0000 7f7aaa000640 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b3486bf0c0 0x55b3486c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.459+0000 7f7aaa000640 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b3486bf0c0 0x55b3486c14e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f945c72fc31cf5b4 server_cookie=626c67656c403f65 in_seq=0 out_seq=0 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.459+0000 7f7a8be00640 10 client.26906 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.482+0000 7f7a8be00640 1 -- 172.21.15.46:0/1122125589 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b348599b80 con 0x55b3486bf0c0 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.482+0000 7f7a8be00640 10 client.26906 handle_command_reply: tid=0 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.482+0000 7f7ab443f740 1 client.26906 shutdown 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.482+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 msgr2=0x55b34864ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.482+0000 7f7ab443f740 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b34864ce40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00ec20 tx=0x7f7a9c009770 comp rx=0 tx=0).stop 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.483+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 shutdown_connections 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.483+0000 7f7ab443f740 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b3486bf0c0 0x55b3486c14e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.483+0000 7f7ab443f740 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b348632a60 0x55b34864e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.483+0000 7f7ab443f740 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b34862d9d0 0x55b34864d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.483+0000 7f7ab443f740 1 --2- 172.21.15.46:0/1122125589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b34862d090 0x55b34864ce40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.483+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 >> 172.21.15.46:0/1122125589 conn(0x55b348628700 msgr2=0x55b348628ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.483+0000 7f7a8be00640 0 client.26906 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.484+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 shutdown_connections 2024-09-10T07:42:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.484+0000 7f7ab443f740 1 -- 172.21.15.46:0/1122125589 wait complete. 2024-09-10T07:42:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.486+0000 7f7ab443f740 20 client.26906 trim_cache size 0 max 16384 2024-09-10T07:42:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:42:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:42:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:42:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:42:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 -- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2cb0 msgr2=0x7f7a900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2cb0 0x7f7a900a5170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006010 tx=0x7f7aa403e040 comp rx=0 tx=0).stop 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 -- 172.21.15.46:0/271089722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac0761d0 msgr2=0x7f7aac19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac0761d0 0x7f7aac19a490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800d950 tx=0x7f7a9800de10 comp rx=0 tx=0).stop 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 -- 172.21.15.46:0/271089722 shutdown_connections 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac1099f0 0x7f7aac1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac0761d0 0x7f7aac19a490 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2cb0 0x7f7a900a5170 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 --2- 172.21.15.46:0/271089722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac074bf0 0x7f7aac199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.487+0000 7f7ab1600640 1 -- 172.21.15.46:0/271089722 >> 172.21.15.46:0/271089722 conn(0x7f7aac0fd2c0 msgr2=0x7f7aac105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.488+0000 7f7ab1600640 1 -- 172.21.15.46:0/271089722 shutdown_connections 2024-09-10T07:42:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:40.488+0000 7f7ab1600640 1 -- 172.21.15.46:0/271089722 wait complete. 2024-09-10T07:42:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 320.0 s, 858.6 tps, lat 115.361 ms stddev 150.020 2024-09-10T07:42:41.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:40 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:42:41.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:40 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:42:41.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:40 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:42:41.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:40 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:42:41.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:40 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:42:41.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:40 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:42:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:41 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:42:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:41 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:42:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:41 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:42:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:41 smithi181 ceph-mon[35397]: pgmap v581: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 696 op/s 2024-09-10T07:42:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:41 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:42:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:41 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:42:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:41 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:42:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:41 smithi063 ceph-mon[35020]: pgmap v581: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 696 op/s 2024-09-10T07:42:42.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:41 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:42:42.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:41 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:42:42.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:41 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:42:42.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:41 smithi046 ceph-mon[30705]: pgmap v581: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 696 op/s 2024-09-10T07:42:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:43 smithi181 ceph-mon[35397]: pgmap v582: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 651 op/s 2024-09-10T07:42:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:43 smithi063 ceph-mon[35020]: pgmap v582: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 651 op/s 2024-09-10T07:42:44.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:43 smithi046 ceph-mon[30705]: pgmap v582: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 651 op/s 2024-09-10T07:42:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 325.0 s, 1126.2 tps, lat 87.384 ms stddev 110.407 2024-09-10T07:42:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:45 smithi181 ceph-mon[35397]: pgmap v583: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-10T07:42:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:45 smithi063 ceph-mon[35020]: pgmap v583: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-10T07:42:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:45 smithi046 ceph-mon[30705]: pgmap v583: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-10T07:42:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:47 smithi181 ceph-mon[35397]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-09-10T07:42:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:47 smithi063 ceph-mon[35020]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-09-10T07:42:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:47 smithi046 ceph-mon[30705]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 17 MiB/s wr, 741 op/s 2024-09-10T07:42:48.722 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T07:42:48.722 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T07:42:48.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:42:49.422 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:50.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:49 smithi046 ceph-mon[30705]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 518 op/s 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.270+0000 7fbceb200640 1 -- 172.21.15.46:0/823874939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec10e7f0 msgr2=0x7fbcec10ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.270+0000 7fbceb200640 1 --2- 172.21.15.46:0/823874939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec10e7f0 0x7fbcec10ec80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009be0 tx=0x7fbce402a6c0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.271+0000 7fbceb200640 1 -- 172.21.15.46:0/823874939 shutdown_connections 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.271+0000 7fbceb200640 1 --2- 172.21.15.46:0/823874939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec10e7f0 0x7fbcec10ec80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.271+0000 7fbceb200640 1 --2- 172.21.15.46:0/823874939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec109d70 0x7fbcec10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.271+0000 7fbceb200640 1 --2- 172.21.15.46:0/823874939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcec109450 0x7fbcec109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.271+0000 7fbceb200640 1 -- 172.21.15.46:0/823874939 >> 172.21.15.46:0/823874939 conn(0x7fbcec073660 msgr2=0x7fbcec073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.271+0000 7fbceb200640 1 -- 172.21.15.46:0/823874939 shutdown_connections 2024-09-10T07:42:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.272+0000 7fbceb200640 1 -- 172.21.15.46:0/823874939 wait complete. 2024-09-10T07:42:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.276+0000 7fbceb200640 1 Processor -- start 2024-09-10T07:42:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.277+0000 7fbceb200640 1 -- start start 2024-09-10T07:42:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.279+0000 7fbceb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec109450 0x7fbcec11c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.280+0000 7fbceb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec109d70 0x7fbcec1194d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.280+0000 7fbce9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec109450 0x7fbcec11c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.281+0000 7fbceb200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcec10e7f0 0x7fbcec119a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.281+0000 7fbceb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1163c0 con 0x7fbcec109450 2024-09-10T07:42:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.281+0000 7fbceb200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec116240 con 0x7fbcec10e7f0 2024-09-10T07:42:50.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.281+0000 7fbceb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec11ca90 con 0x7fbcec109d70 2024-09-10T07:42:50.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.281+0000 7fbce9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec109450 0x7fbcec11c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50874/0 (socket says 172.21.15.46:50874) 2024-09-10T07:42:50.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.281+0000 7fbce9e00640 1 -- 172.21.15.46:0/4098771395 learned_addr learned my addr 172.21.15.46:0/4098771395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:50.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.282+0000 7fbce9400640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec109d70 0x7fbcec1194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:50.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.282+0000 7fbce9e00640 1 -- 172.21.15.46:0/4098771395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec109d70 msgr2=0x7fbcec1194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:50.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.283+0000 7fbcea800640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcec10e7f0 0x7fbcec119a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:50.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.285+0000 7fbce9e00640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec109d70 0x7fbcec1194d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.285+0000 7fbce9e00640 1 -- 172.21.15.46:0/4098771395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcec10e7f0 msgr2=0x7fbcec119a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:50.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.285+0000 7fbce9e00640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcec10e7f0 0x7fbcec119a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.285+0000 7fbce9e00640 1 -- 172.21.15.46:0/4098771395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec11a2f0 con 0x7fbcec109450 2024-09-10T07:42:50.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.285+0000 7fbcea800640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcec10e7f0 0x7fbcec119a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:50.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.285+0000 7fbce9400640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec109d70 0x7fbcec1194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:50.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.286+0000 7fbce9e00640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec109450 0x7fbcec11c420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800e9c0 tx=0x7fbcd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:50.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.286+0000 7fbce2a00640 1 -- 172.21.15.46:0/4098771395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800ccc0 con 0x7fbcec109450 2024-09-10T07:42:50.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.286+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec1a8e60 con 0x7fbcec109450 2024-09-10T07:42:50.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.286+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec1a93f0 con 0x7fbcec109450 2024-09-10T07:42:50.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.286+0000 7fbce2a00640 1 -- 172.21.15.46:0/4098771395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd8005500 con 0x7fbcec109450 2024-09-10T07:42:50.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.286+0000 7fbce2a00640 1 -- 172.21.15.46:0/4098771395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8010430 con 0x7fbcec109450 2024-09-10T07:42:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.288+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc005290 con 0x7fbcec109450 2024-09-10T07:42:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.295+0000 7fbce2a00640 1 -- 172.21.15.46:0/4098771395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbcd8005000 con 0x7fbcec109450 2024-09-10T07:42:50.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.297+0000 7fbce2a00640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcd00a2bc0 0x7fbcd00a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:50.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.298+0000 7fbce2a00640 1 -- 172.21.15.46:0/4098771395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbcd80cf840 con 0x7fbcec109450 2024-09-10T07:42:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.301+0000 7fbce2a00640 1 -- 172.21.15.46:0/4098771395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcd80541b0 con 0x7fbcec109450 2024-09-10T07:42:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.301+0000 7fbce9400640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcd00a2bc0 0x7fbcd00a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.302+0000 7fbce9400640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcd00a2bc0 0x7fbcd00a5080 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcec11ac90 tx=0x7fbcdc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:49 smithi181 ceph-mon[35397]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 518 op/s 2024-09-10T07:42:50.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:49 smithi063 ceph-mon[35020]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 518 op/s 2024-09-10T07:42:50.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 330.0 s, 815.2 tps, lat 124.453 ms stddev 140.714 2024-09-10T07:42:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.643+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbccc005580 con 0x7fbcec109450 2024-09-10T07:42:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.644+0000 7fbce2a00640 1 -- 172.21.15.46:0/4098771395 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11976 (secure 0 0 0) 0x7fbcd809c420 con 0x7fbcec109450 2024-09-10T07:42:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:42:50.645 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"btime":"2024-09-10T07:42:38:910267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:42:38.047718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:42:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 43 2024-09-10T07:42:50.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcd00a2bc0 msgr2=0x7fbcd00a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:50.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcd00a2bc0 0x7fbcd00a5080 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcec11ac90 tx=0x7fbcdc005f90 comp rx=0 tx=0).stop 2024-09-10T07:42:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec109450 msgr2=0x7fbcec11c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec109450 0x7fbcec11c420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800e9c0 tx=0x7fbcd800ee80 comp rx=0 tx=0).stop 2024-09-10T07:42:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 shutdown_connections 2024-09-10T07:42:50.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcec10e7f0 0x7fbcec119a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec109d70 0x7fbcec1194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcd00a2bc0 0x7fbcd00a5080 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 --2- 172.21.15.46:0/4098771395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcec109450 0x7fbcec11c420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.647+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 >> 172.21.15.46:0/4098771395 conn(0x7fbcec073660 msgr2=0x7fbcec06d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.649+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 shutdown_connections 2024-09-10T07:42:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:50.649+0000 7fbceb200640 1 -- 172.21.15.46:0/4098771395 wait complete. 2024-09-10T07:42:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/4098771395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:42:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:50 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/4098771395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:42:51.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:50 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/4098771395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:42:52.315 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:52 smithi046 ceph-mon[30705]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 622 op/s 2024-09-10T07:42:52.379 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T07:42:52.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T07:42:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:52 smithi181 ceph-mon[35397]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 622 op/s 2024-09-10T07:42:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:52 smithi063 ceph-mon[35020]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 622 op/s 2024-09-10T07:42:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:42:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.697+0000 7f6825e00640 1 -- 172.21.15.46:0/1120879534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820105900 msgr2=0x7f682010c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.697+0000 7f6825e00640 1 --2- 172.21.15.46:0/1120879534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820105900 0x7f682010c190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6818009ca0 tx=0x7f681802a8f0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.697+0000 7f6825e00640 1 -- 172.21.15.46:0/1120879534 shutdown_connections 2024-09-10T07:42:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.697+0000 7f6825e00640 1 --2- 172.21.15.46:0/1120879534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820105900 0x7f682010c190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.697+0000 7f6825e00640 1 --2- 172.21.15.46:0/1120879534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820104f40 0x7f68201053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.697+0000 7f6825e00640 1 --2- 172.21.15.46:0/1120879534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820103d50 0x7f6820104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.697+0000 7f6825e00640 1 -- 172.21.15.46:0/1120879534 >> 172.21.15.46:0/1120879534 conn(0x7f68200ff4b0 msgr2=0x7f68201018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.700+0000 7f6825e00640 1 -- 172.21.15.46:0/1120879534 shutdown_connections 2024-09-10T07:42:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.700+0000 7f6825e00640 1 -- 172.21.15.46:0/1120879534 wait complete. 2024-09-10T07:42:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.701+0000 7f6825e00640 1 Processor -- start 2024-09-10T07:42:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.702+0000 7f6825e00640 1 -- start start 2024-09-10T07:42:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.702+0000 7f6825e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820103d50 0x7f682019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.702+0000 7f6825e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820104f40 0x7f682019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.704+0000 7f681ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820103d50 0x7f682019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.704+0000 7f681ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820103d50 0x7f682019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50878/0 (socket says 172.21.15.46:50878) 2024-09-10T07:42:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.704+0000 7f681ea00640 1 -- 172.21.15.46:0/4120329758 learned_addr learned my addr 172.21.15.46:0/4120329758 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f6825e00640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105900 0x7f68201a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f682010faf0 con 0x7f6820103d50 2024-09-10T07:42:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f682010f970 con 0x7f6820105900 2024-09-10T07:42:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f682010fc70 con 0x7f6820104f40 2024-09-10T07:42:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f681e000640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820104f40 0x7f682019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f681f400640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105900 0x7f68201a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f681e000640 1 -- 172.21.15.46:0/4120329758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105900 msgr2=0x7f68201a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f681e000640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105900 0x7f68201a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f681e000640 1 -- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820103d50 msgr2=0x7f682019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f681e000640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820103d50 0x7f682019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.705+0000 7f681e000640 1 -- 172.21.15.46:0/4120329758 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68201a1d30 con 0x7f6820104f40 2024-09-10T07:42:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.706+0000 7f681e000640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820104f40 0x7f682019a650 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f680c009fd0 tx=0x7f680c00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.706+0000 7f6824a00640 1 -- 172.21.15.46:0/4120329758 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c015020 con 0x7f6820104f40 2024-09-10T07:42:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.706+0000 7f6824a00640 1 -- 172.21.15.46:0/4120329758 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c011d10 con 0x7f6820104f40 2024-09-10T07:42:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.706+0000 7f6824a00640 1 -- 172.21.15.46:0/4120329758 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c00f2e0 con 0x7f6820104f40 2024-09-10T07:42:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.706+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68201a1fc0 con 0x7f6820104f40 2024-09-10T07:42:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.707+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201a24b0 con 0x7f6820104f40 2024-09-10T07:42:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.715+0000 7f6828cc2740 1 Processor -- start 2024-09-10T07:42:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.716+0000 7f6828cc2740 1 -- start start 2024-09-10T07:42:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.716+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a68c0 0x5563c55a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.716+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.716+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 0x5563c55ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.716+0000 7f6828cc2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5563c5503100 con 0x5563c55a68c0 2024-09-10T07:42:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.716+0000 7f6828cc2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5563c54ab1b0 con 0x5563c55a7200 2024-09-10T07:42:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.716+0000 7f6828cc2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5563c54ff0c0 con 0x5563c55ac290 2024-09-10T07:42:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34342/0 (socket says 172.21.15.46:34342) 2024-09-10T07:42:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681ea00640 1 -- 172.21.15.46:0/4225588387 learned_addr learned my addr 172.21.15.46:0/4225588387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f6824a00640 1 -- 172.21.15.46:0/4120329758 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f680c018020 con 0x7f6820104f40 2024-09-10T07:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681e000640 1 --2- 172.21.15.46:0/4225588387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 0x5563c55ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681ea00640 1 -- 172.21.15.46:0/4225588387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 msgr2=0x5563c55ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681ea00640 1 --2- 172.21.15.46:0/4225588387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 0x5563c55ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681ea00640 1 -- 172.21.15.46:0/4225588387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a68c0 msgr2=0x5563c55a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681ea00640 1 --2- 172.21.15.46:0/4225588387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a68c0 0x5563c55a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.717+0000 7f681ea00640 1 -- 172.21.15.46:0/4225588387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c5216210 con 0x5563c55a7200 2024-09-10T07:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.718+0000 7f681ea00640 1 --2- 172.21.15.46:0/4225588387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55abcc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6810009ae0 tx=0x7f681000e1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c473cdea0fadad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.718+0000 7f680be00640 1 -- 172.21.15.46:0/4225588387 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810017070 con 0x5563c55a7200 2024-09-10T07:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.718+0000 7f680be00640 1 -- 172.21.15.46:0/4225588387 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f681000ebb0 con 0x5563c55a7200 2024-09-10T07:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.718+0000 7f680be00640 1 -- 172.21.15.46:0/4225588387 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810012630 con 0x5563c55a7200 2024-09-10T07:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 -- 172.21.15.46:0/4225588387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 msgr2=0x5563c55abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 --2- 172.21.15.46:0/4225588387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55abcc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6810009ae0 tx=0x7f681000e1d0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 -- 172.21.15.46:0/4225588387 shutdown_connections 2024-09-10T07:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 --2- 172.21.15.46:0/4225588387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 0x5563c55ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 --2- 172.21.15.46:0/4225588387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55abcc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 --2- 172.21.15.46:0/4225588387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a68c0 0x5563c55a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 -- 172.21.15.46:0/4225588387 >> 172.21.15.46:0/4225588387 conn(0x5563c55a1f30 msgr2=0x5563c55a2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 -- 172.21.15.46:0/4225588387 shutdown_connections 2024-09-10T07:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.720+0000 7f6828cc2740 1 -- 172.21.15.46:0/4225588387 wait complete. 2024-09-10T07:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.721+0000 7f6828cc2740 1 -- start start 2024-09-10T07:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.724+0000 7f6828cc2740 1 Processor -- start 2024-09-10T07:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.724+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a68c0 0x5563c55c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.724+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 0x5563c55c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.725+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 0x5563c55c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.725+0000 7f6828cc2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5563c55c0230 con 0x5563c55a7200 2024-09-10T07:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.725+0000 7f6828cc2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5563c548c090 con 0x5563c55a68c0 2024-09-10T07:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.725+0000 7f6828cc2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5563c55c03b0 con 0x5563c55ac290 2024-09-10T07:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.725+0000 7f6828cc2740 10 client.0 _ll_register_callbacks cb 0x5563c54e0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.725+0000 7f681f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a68c0 0x5563c55c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.725+0000 7f681f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a68c0 0x5563c55c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34348/0 (socket says 172.21.15.46:34348) 2024-09-10T07:42:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.725+0000 7f681f400640 1 -- 172.21.15.46:0/3436627680 learned_addr learned my addr 172.21.15.46:0/3436627680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.726+0000 7f681ea00640 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 0x5563c55c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.727+0000 7f681e000640 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 0x5563c55c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.727+0000 7f6824a00640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68040a2c20 0x7f68040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f681f400640 1 -- 172.21.15.46:0/3436627680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 msgr2=0x5563c55c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f681f400640 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 0x5563c55c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f681f400640 1 -- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 msgr2=0x5563c55c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f681f400640 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 0x5563c55c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f681f400640 1 -- 172.21.15.46:0/3436627680 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c52e2620 con 0x5563c55a68c0 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f6824a00640 1 -- 172.21.15.46:0/4120329758 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f680c0cecd0 con 0x7f6820104f40 2024-09-10T07:42:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f681ea00640 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 0x5563c55c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:42:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f681f400640 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a68c0 0x5563c55c1a60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6818009ca0 tx=0x7f6818031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc96dec2b79e0299 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f6808c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:42:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.728+0000 7f6808c00640 1 -- 172.21.15.46:0/3436627680 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68180318e0 con 0x5563c55a68c0 2024-09-10T07:42:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.729+0000 7f6808c00640 1 -- 172.21.15.46:0/3436627680 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6818030c30 con 0x5563c55a68c0 2024-09-10T07:42:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.729+0000 7f6828cc2740 1 -- 172.21.15.46:0/3436627680 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f681000ebb0 con 0x5563c55a68c0 2024-09-10T07:42:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.730+0000 7f6808c00640 1 -- 172.21.15.46:0/3436627680 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6818030e30 con 0x5563c55a68c0 2024-09-10T07:42:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.730+0000 7f681f400640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68040a2c20 0x7f68040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.730+0000 7f6828cc2740 10 client.26918 fetch_fsmap learned FSMap version 43 2024-09-10T07:42:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.730+0000 7f6828cc2740 1 -- 172.21.15.46:0/3436627680 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563c55cbe40 con 0x5563c55a68c0 2024-09-10T07:42:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.730+0000 7f6808c00640 1 -- 172.21.15.46:0/3436627680 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f681000ebb0 con 0x5563c55a68c0 2024-09-10T07:42:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.730+0000 7f6808c00640 1 -- 172.21.15.46:0/3436627680 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f6818054020 con 0x5563c55a68c0 2024-09-10T07:42:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.731+0000 7f6828cc2740 10 client.26918 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:42:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.731+0000 7f6828cc2740 10 client.26918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.731+0000 7f6828cc2740 20 client.26918 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.731+0000 7f6828cc2740 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5563c55cd5c0 0x5563c55cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.731+0000 7f6828cc2740 4 client.26918 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.731+0000 7f6828cc2740 1 -- 172.21.15.46:0/3436627680 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563c5510830 con 0x5563c55cd5c0 2024-09-10T07:42:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.732+0000 7f681ea00640 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5563c55cd5c0 0x5563c55cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.733+0000 7f681ea00640 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5563c55cd5c0 0x5563c55cfa00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a0184010afe70ae server_cookie=c8b83a27cb06644b in_seq=0 out_seq=0 2024-09-10T07:42:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.733+0000 7f6808c00640 10 client.26918 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.736+0000 7f681f400640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68040a2c20 0x7f68040a50e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6818054fd0 tx=0x7f6818045b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.737+0000 7f6808c00640 1 -- 172.21.15.46:0/3436627680 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563c5510830 con 0x5563c55cd5c0 2024-09-10T07:42:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.737+0000 7f6808c00640 10 client.26918 handle_command_reply: tid=0 2024-09-10T07:42:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.737+0000 7f6828cc2740 1 client.26918 shutdown 2024-09-10T07:42:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.738+0000 7f6828cc2740 1 -- 172.21.15.46:0/3436627680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a68c0 msgr2=0x5563c55c1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.738+0000 7f6828cc2740 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a68c0 0x5563c55c1a60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6818009ca0 tx=0x7f6818031770 comp rx=0 tx=0).stop 2024-09-10T07:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 1 -- 172.21.15.46:0/3436627680 shutdown_connections 2024-09-10T07:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5563c55cd5c0 0x5563c55cfa00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55ac290 0x5563c55c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 0x5563c55c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 1 --2- 172.21.15.46:0/3436627680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a68c0 0x5563c55c1a60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 1 -- 172.21.15.46:0/3436627680 >> 172.21.15.46:0/3436627680 conn(0x5563c55a1f30 msgr2=0x5563c55a3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6808c00640 0 client.26918 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 1 -- 172.21.15.46:0/3436627680 shutdown_connections 2024-09-10T07:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 1 -- 172.21.15.46:0/3436627680 wait complete. 2024-09-10T07:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.740+0000 7f6828cc2740 20 client.26918 trim_cache size 0 max 16384 2024-09-10T07:42:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.766+0000 7f6828cc2740 1 Processor -- start 2024-09-10T07:42:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.766+0000 7f6828cc2740 1 -- start start 2024-09-10T07:42:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.766+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55a68c0 0x5563c55c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.766+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 0x5563c55c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.766+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55ac290 0x5563c55c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.766+0000 7f6828cc2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6820071a40 con 0x5563c55a7200 2024-09-10T07:42:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.766+0000 7f6828cc2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6820073610 con 0x5563c55ac290 2024-09-10T07:42:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.766+0000 7f6828cc2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6820072110 con 0x5563c55a68c0 2024-09-10T07:42:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.767+0000 7f681f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55ac290 0x5563c55c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.767+0000 7f681f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55ac290 0x5563c55c1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34352/0 (socket says 172.21.15.46:34352) 2024-09-10T07:42:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.767+0000 7f681f400640 1 -- 172.21.15.46:0/212839259 learned_addr learned my addr 172.21.15.46:0/212839259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.767+0000 7f681f400640 1 -- 172.21.15.46:0/212839259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55a68c0 msgr2=0x5563c55c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.767+0000 7f681f400640 1 --2- 172.21.15.46:0/212839259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55a68c0 0x5563c55c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.767+0000 7f681f400640 1 -- 172.21.15.46:0/212839259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 msgr2=0x5563c55c8fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.767+0000 7f681f400640 1 --2- 172.21.15.46:0/212839259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 0x5563c55c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.768+0000 7f681f400640 1 -- 172.21.15.46:0/212839259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c55a7e90 con 0x5563c55ac290 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.768+0000 7f681f400640 1 --2- 172.21.15.46:0/212839259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55ac290 0x5563c55c1d50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6818006010 tx=0x7f6818056000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d96512d18fe3011 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.768+0000 7f680b400640 1 -- 172.21.15.46:0/212839259 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818054b10 con 0x5563c55ac290 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.768+0000 7f680b400640 1 -- 172.21.15.46:0/212839259 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818036040 con 0x5563c55ac290 2024-09-10T07:42:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.768+0000 7f680b400640 1 -- 172.21.15.46:0/212839259 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818030c90 con 0x5563c55ac290 2024-09-10T07:42:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.769+0000 7f6828cc2740 1 -- 172.21.15.46:0/212839259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55ac290 msgr2=0x5563c55c1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.769+0000 7f6828cc2740 1 --2- 172.21.15.46:0/212839259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55ac290 0x5563c55c1d50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6818006010 tx=0x7f6818056000 comp rx=0 tx=0).stop 2024-09-10T07:42:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.769+0000 7f6828cc2740 1 -- 172.21.15.46:0/212839259 shutdown_connections 2024-09-10T07:42:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.769+0000 7f6828cc2740 1 --2- 172.21.15.46:0/212839259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55ac290 0x5563c55c1d50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.769+0000 7f6828cc2740 1 --2- 172.21.15.46:0/212839259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55a7200 0x5563c55c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.769+0000 7f6828cc2740 1 --2- 172.21.15.46:0/212839259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55a68c0 0x5563c55c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.769+0000 7f6828cc2740 1 -- 172.21.15.46:0/212839259 >> 172.21.15.46:0/212839259 conn(0x5563c55a1f30 msgr2=0x5563c55a83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.770+0000 7f6828cc2740 1 -- 172.21.15.46:0/212839259 shutdown_connections 2024-09-10T07:42:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.770+0000 7f6828cc2740 1 -- 172.21.15.46:0/212839259 wait complete. 2024-09-10T07:42:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.770+0000 7f6828cc2740 1 -- start start 2024-09-10T07:42:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.771+0000 7f6828cc2740 1 Processor -- start 2024-09-10T07:42:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.771+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55a68c0 0x5563c55c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f6828cc2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55ac290 0x5563c56362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f6828cc2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6820072ee0 con 0x5563c55ac290 2024-09-10T07:42:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f6828cc2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68200727e0 con 0x5563c55a7200 2024-09-10T07:42:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f6828cc2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f682006fa40 con 0x5563c55a68c0 2024-09-10T07:42:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f6828cc2740 10 client.0 _ll_register_callbacks cb 0x7f6818009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f681f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55ac290 0x5563c56362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f681f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55ac290 0x5563c56362d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50918/0 (socket says 172.21.15.46:50918) 2024-09-10T07:42:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.772+0000 7f681f400640 1 -- 172.21.15.46:0/250915863 learned_addr learned my addr 172.21.15.46:0/250915863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.773+0000 7f681e000640 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.773+0000 7f681f400640 1 -- 172.21.15.46:0/250915863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55a68c0 msgr2=0x5563c55c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.773+0000 7f681f400640 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55a68c0 0x5563c55c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.773+0000 7f681f400640 1 -- 172.21.15.46:0/250915863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 msgr2=0x5563c55c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.773+0000 7f681f400640 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.773+0000 7f681f400640 1 -- 172.21.15.46:0/250915863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c55c6860 con 0x5563c55ac290 2024-09-10T07:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.774+0000 7f681e000640 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.774+0000 7f681f400640 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55ac290 0x5563c56362d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6818005d50 tx=0x7f681802adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3517baa9fd61ace6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.774+0000 7f6808c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.774+0000 7f6808c00640 1 -- 172.21.15.46:0/250915863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681806d030 con 0x5563c55ac290 2024-09-10T07:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.774+0000 7f6808c00640 1 -- 172.21.15.46:0/250915863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6818064070 con 0x5563c55ac290 2024-09-10T07:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.774+0000 7f6828cc2740 1 -- 172.21.15.46:0/250915863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6818036040 con 0x5563c55ac290 2024-09-10T07:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.775+0000 7f6808c00640 1 -- 172.21.15.46:0/250915863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6818079050 con 0x5563c55ac290 2024-09-10T07:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.775+0000 7f6828cc2740 10 client.16524 fetch_fsmap learned FSMap version 43 2024-09-10T07:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.775+0000 7f6828cc2740 1 -- 172.21.15.46:0/250915863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563c56378f0 con 0x5563c55ac290 2024-09-10T07:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.776+0000 7f6808c00640 1 -- 172.21.15.46:0/250915863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818036040 con 0x5563c55ac290 2024-09-10T07:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.776+0000 7f6808c00640 1 -- 172.21.15.46:0/250915863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f6818054040 con 0x5563c55ac290 2024-09-10T07:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.776+0000 7f6828cc2740 10 client.16524 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.776+0000 7f6828cc2740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.776+0000 7f6828cc2740 20 client.16524 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.776+0000 7f6828cc2740 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5563c5638240 0x5563c563a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.776+0000 7f6828cc2740 4 client.16524 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.776+0000 7f6828cc2740 1 -- 172.21.15.46:0/250915863 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563c550ab80 con 0x5563c5638240 2024-09-10T07:42:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.777+0000 7f681ea00640 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5563c5638240 0x5563c563a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.778+0000 7f681ea00640 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5563c5638240 0x5563c563a660 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aabf6e2f50d4cb1 server_cookie=be97af45368c0b48 in_seq=0 out_seq=0 2024-09-10T07:42:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.778+0000 7f6808c00640 10 client.16524 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.921+0000 7f6808c00640 1 -- 172.21.15.46:0/250915863 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563c550ab80 con 0x5563c5638240 2024-09-10T07:42:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.921+0000 7f6808c00640 10 client.16524 handle_command_reply: tid=0 2024-09-10T07:42:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.921+0000 7f6828cc2740 1 client.16524 shutdown 2024-09-10T07:42:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.922+0000 7f6828cc2740 1 -- 172.21.15.46:0/250915863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55ac290 msgr2=0x5563c56362d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.922+0000 7f6828cc2740 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55ac290 0x5563c56362d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6818005d50 tx=0x7f681802adf0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.923+0000 7f6828cc2740 1 -- 172.21.15.46:0/250915863 shutdown_connections 2024-09-10T07:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.923+0000 7f6828cc2740 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5563c5638240 0x5563c563a660 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.923+0000 7f6828cc2740 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c55ac290 0x5563c56362d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.923+0000 7f6808c00640 0 client.16524 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.923+0000 7f6828cc2740 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563c55a7200 0x5563c55c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.923+0000 7f6828cc2740 1 --2- 172.21.15.46:0/250915863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563c55a68c0 0x5563c55c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.923+0000 7f6828cc2740 1 -- 172.21.15.46:0/250915863 >> 172.21.15.46:0/250915863 conn(0x5563c55a1f30 msgr2=0x5563c55c1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.924+0000 7f6828cc2740 1 -- 172.21.15.46:0/250915863 shutdown_connections 2024-09-10T07:42:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.924+0000 7f6828cc2740 1 -- 172.21.15.46:0/250915863 wait complete. 2024-09-10T07:42:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.924+0000 7f6828cc2740 20 client.16524 trim_cache size 0 max 16384 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.910+0000 7f2903800640 1 -- 172.21.15.46:0/4088458023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 msgr2=0x7f28fc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.910+0000 7f2903800640 1 --2- 172.21.15.46:0/4088458023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 0x7f28fc1089f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f28e8011f90 tx=0x7f28e803af70 comp rx=0 tx=0).stop 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.911+0000 7f2903800640 1 -- 172.21.15.46:0/4088458023 shutdown_connections 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.911+0000 7f2903800640 1 --2- 172.21.15.46:0/4088458023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc109120 0x7f28fc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.911+0000 7f2903800640 1 --2- 172.21.15.46:0/4088458023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 0x7f28fc1089f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.911+0000 7f2903800640 1 --2- 172.21.15.46:0/4088458023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28fc103ff0 0x7f28fc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.911+0000 7f2903800640 1 -- 172.21.15.46:0/4088458023 >> 172.21.15.46:0/4088458023 conn(0x7f28fc0ffa90 msgr2=0x7f28fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.911+0000 7f2903800640 1 -- 172.21.15.46:0/4088458023 shutdown_connections 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.911+0000 7f2903800640 1 -- 172.21.15.46:0/4088458023 wait complete. 2024-09-10T07:42:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.912+0000 7f2903800640 1 Processor -- start 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.913+0000 7f2903800640 1 -- start start 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.913+0000 7f2903800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28fc103ff0 0x7f28fc116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.914+0000 7f2903800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 0x7f28fc1167b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.914+0000 7f2903800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc109120 0x7f28fc1117f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.914+0000 7f2903800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc079010 con 0x7f28fc103ff0 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.914+0000 7f2903800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc078e90 con 0x7f28fc1049a0 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.914+0000 7f2903800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc079190 con 0x7f28fc109120 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.915+0000 7f28fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 0x7f28fc1167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.915+0000 7f2901800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc109120 0x7f28fc1117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.915+0000 7f2901800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc109120 0x7f28fc1117f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43572/0 (socket says 172.21.15.46:43572) 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.915+0000 7f2901800640 1 -- 172.21.15.46:0/3575674549 learned_addr learned my addr 172.21.15.46:0/3575674549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.916+0000 7f2900e00640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28fc103ff0 0x7f28fc116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.917+0000 7f28fbe00640 1 -- 172.21.15.46:0/3575674549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc109120 msgr2=0x7f28fc1117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.917+0000 7f28fbe00640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc109120 0x7f28fc1117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.917+0000 7f28fbe00640 1 -- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28fc103ff0 msgr2=0x7f28fc116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.917+0000 7f28fbe00640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28fc103ff0 0x7f28fc116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.917+0000 7f28fbe00640 1 -- 172.21.15.46:0/3575674549 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc1120b0 con 0x7f28fc1049a0 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.917+0000 7f28fbe00640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 0x7f28fc1167b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28e8013120 tx=0x7f28e8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.918+0000 7f28f9600640 1 -- 172.21.15.46:0/3575674549 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e80112d0 con 0x7f28fc1049a0 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.918+0000 7f28f9600640 1 -- 172.21.15.46:0/3575674549 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e80035f0 con 0x7f28fc1049a0 2024-09-10T07:42:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.918+0000 7f28f9600640 1 -- 172.21.15.46:0/3575674549 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8043620 con 0x7f28fc1049a0 2024-09-10T07:42:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:42:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:42:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:42:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:42:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.928+0000 7f2903800640 1 -- 172.21.15.46:0/3575674549 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28fc112340 con 0x7f28fc1049a0 2024-09-10T07:42:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.929+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68040a2c20 msgr2=0x7f68040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.929+0000 7f6825e00640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68040a2c20 0x7f68040a50e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6818054fd0 tx=0x7f6818045b50 comp rx=0 tx=0).stop 2024-09-10T07:42:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.929+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820104f40 msgr2=0x7f682019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.929+0000 7f6825e00640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820104f40 0x7f682019a650 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f680c009fd0 tx=0x7f680c00b870 comp rx=0 tx=0).stop 2024-09-10T07:42:53.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.929+0000 7f2903800640 1 -- 172.21.15.46:0/3575674549 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28fc11dfa0 con 0x7f28fc1049a0 2024-09-10T07:42:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.930+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 shutdown_connections 2024-09-10T07:42:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.930+0000 7f6825e00640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105900 0x7f68201a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.930+0000 7f6825e00640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6820104f40 0x7f682019a650 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.930+0000 7f6825e00640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f68040a2c20 0x7f68040a50e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.930+0000 7f6825e00640 1 --2- 172.21.15.46:0/4120329758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820103d50 0x7f682019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.930+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 >> 172.21.15.46:0/4120329758 conn(0x7f68200ff4b0 msgr2=0x7f6820108310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.930+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 shutdown_connections 2024-09-10T07:42:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:42:53.931+0000 7f6825e00640 1 -- 172.21.15.46:0/4120329758 wait complete. 2024-09-10T07:42:53.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.939+0000 7f28f9600640 1 -- 172.21.15.46:0/3575674549 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f28e8022060 con 0x7f28fc1049a0 2024-09-10T07:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.942+0000 7f28f9600640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28e40a2c40 0x7f28e40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.943+0000 7f29061e8740 1 Processor -- start 2024-09-10T07:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.944+0000 7f29061e8740 1 -- start start 2024-09-10T07:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.944+0000 7f28f9600640 1 -- 172.21.15.46:0/3575674549 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28e8020070 con 0x7f28fc1049a0 2024-09-10T07:42:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.945+0000 7f2900e00640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28e40a2c40 0x7f28e40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.945+0000 7f29061e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b5171790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.945+0000 7f29061e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 0x55d1b517c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.945+0000 7f29061e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b517c590 0x55d1b517e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.946+0000 7f29061e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1b50d7530 con 0x55d1b51713b0 2024-09-10T07:42:53.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.946+0000 7f29061e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d1b507e860 con 0x55d1b517c590 2024-09-10T07:42:53.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.946+0000 7f29061e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d1b50d34f0 con 0x55d1b5171cd0 2024-09-10T07:42:53.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.947+0000 7f2901800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b517c590 0x55d1b517e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.947+0000 7f2901800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b517c590 0x55d1b517e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34384/0 (socket says 172.21.15.46:34384) 2024-09-10T07:42:53.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.947+0000 7f2901800640 1 -- 172.21.15.46:0/141996030 learned_addr learned my addr 172.21.15.46:0/141996030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.947+0000 7f2900e00640 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b5171790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.948+0000 7f2901800640 1 -- 172.21.15.46:0/141996030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 msgr2=0x55d1b517c050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:42:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.948+0000 7f2901800640 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 0x55d1b517c050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.948+0000 7f2901800640 1 -- 172.21.15.46:0/141996030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 msgr2=0x55d1b5171790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.948+0000 7f2901800640 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b5171790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.948+0000 7f2901800640 1 -- 172.21.15.46:0/141996030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b4e48b90 con 0x55d1b517c590 2024-09-10T07:42:53.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.950+0000 7f2900e00640 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b5171790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:42:53.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.950+0000 7f2900e00640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28e40a2c40 0x7f28e40a5100 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009b60 tx=0x7f28f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.950+0000 7f2901800640 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b517c590 0x55d1b517e980 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009fb0 tx=0x7f28f4011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913c4ca7d6aa8115 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.956+0000 7f28ee000640 1 -- 172.21.15.46:0/141996030 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f40127d0 con 0x55d1b517c590 2024-09-10T07:42:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.957+0000 7f28ee000640 1 -- 172.21.15.46:0/141996030 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f4012e50 con 0x55d1b517c590 2024-09-10T07:42:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.957+0000 7f28ee000640 1 -- 172.21.15.46:0/141996030 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f40193a0 con 0x55d1b517c590 2024-09-10T07:42:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.958+0000 7f29061e8740 1 -- 172.21.15.46:0/141996030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b517c590 msgr2=0x55d1b517e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.958+0000 7f29061e8740 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b517c590 0x55d1b517e980 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009fb0 tx=0x7f28f4011870 comp rx=0 tx=0).stop 2024-09-10T07:42:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.958+0000 7f29061e8740 1 -- 172.21.15.46:0/141996030 shutdown_connections 2024-09-10T07:42:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.958+0000 7f29061e8740 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b517c590 0x55d1b517e980 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.958+0000 7f29061e8740 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 0x55d1b517c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.958+0000 7f29061e8740 1 --2- 172.21.15.46:0/141996030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b5171790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.958+0000 7f29061e8740 1 -- 172.21.15.46:0/141996030 >> 172.21.15.46:0/141996030 conn(0x55d1b516d940 msgr2=0x55d1b516dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.959+0000 7f29061e8740 1 -- 172.21.15.46:0/141996030 shutdown_connections 2024-09-10T07:42:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.959+0000 7f29061e8740 1 -- 172.21.15.46:0/141996030 wait complete. 2024-09-10T07:42:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.960+0000 7f29061e8740 1 -- start start 2024-09-10T07:42:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.960+0000 7f29061e8740 1 Processor -- start 2024-09-10T07:42:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.961+0000 7f29061e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b5179590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.961+0000 7f29061e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 0x55d1b5179ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.962+0000 7f29061e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b517a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.962+0000 7f29061e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1b517f0d0 con 0x55d1b51713b0 2024-09-10T07:42:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.962+0000 7f29061e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d1b5091e60 con 0x55d1b5171cd0 2024-09-10T07:42:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.962+0000 7f29061e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d1b517f250 con 0x55d1b517c590 2024-09-10T07:42:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.962+0000 7f29061e8740 10 client.0 _ll_register_callbacks cb 0x55d1b5096cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.962+0000 7f2901800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b517a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.962+0000 7f2901800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b517a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43584/0 (socket says 172.21.15.46:43584) 2024-09-10T07:42:53.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.962+0000 7f2901800640 1 -- 172.21.15.46:0/902138759 learned_addr learned my addr 172.21.15.46:0/902138759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.963+0000 7f28fbe00640 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 0x55d1b5179ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.963+0000 7f2901800640 1 -- 172.21.15.46:0/902138759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 msgr2=0x55d1b5179ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.963+0000 7f2901800640 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 0x55d1b5179ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.963+0000 7f2901800640 1 -- 172.21.15.46:0/902138759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 msgr2=0x55d1b5179590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.963+0000 7f2901800640 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b5179590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.963+0000 7f2901800640 1 -- 172.21.15.46:0/902138759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b509d990 con 0x55d1b517c590 2024-09-10T07:42:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.965+0000 7f2901800640 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b517a010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f28f401c040 tx=0x7f28f401af70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be37e8cd184d4436 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.965+0000 7f28e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.966+0000 7f28e2a00640 1 -- 172.21.15.46:0/902138759 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4028070 con 0x55d1b517c590 2024-09-10T07:42:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.966+0000 7f28e2a00640 1 -- 172.21.15.46:0/902138759 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28f401a5f0 con 0x55d1b517c590 2024-09-10T07:42:53.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.966+0000 7f29061e8740 1 -- 172.21.15.46:0/902138759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f4012e50 con 0x55d1b517c590 2024-09-10T07:42:53.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.967+0000 7f28e2a00640 1 -- 172.21.15.46:0/902138759 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f28f4032050 con 0x55d1b517c590 2024-09-10T07:42:53.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.967+0000 7f29061e8740 10 client.26899 fetch_fsmap learned FSMap version 43 2024-09-10T07:42:53.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.967+0000 7f29061e8740 1 -- 172.21.15.46:0/902138759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d1b518ae50 con 0x55d1b517c590 2024-09-10T07:42:53.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.968+0000 7f28e2a00640 1 -- 172.21.15.46:0/902138759 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f4012e50 con 0x55d1b517c590 2024-09-10T07:42:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.972+0000 7f28e2a00640 1 -- 172.21.15.46:0/902138759 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f28f400f040 con 0x55d1b517c590 2024-09-10T07:42:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.973+0000 7f29061e8740 10 client.26899 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:42:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.973+0000 7f29061e8740 10 client.26899 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:42:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.973+0000 7f29061e8740 20 client.26899 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.973+0000 7f29061e8740 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1b518c200 0x55d1b518e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.973+0000 7f29061e8740 4 client.26899 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:42:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.973+0000 7f29061e8740 1 -- 172.21.15.46:0/902138759 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b507a310 con 0x55d1b518c200 2024-09-10T07:42:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.974+0000 7f2900e00640 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1b518c200 0x55d1b518e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:53.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.977+0000 7f2900e00640 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1b518c200 0x55d1b518e5f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34bde5d7eb82b1d6 server_cookie=25e6b900055d242d in_seq=0 out_seq=0 2024-09-10T07:42:53.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.977+0000 7f28e2a00640 10 client.26899 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.982+0000 7f28e2a00640 1 -- 172.21.15.46:0/902138759 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1b507a310 con 0x55d1b518c200 2024-09-10T07:42:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.982+0000 7f28e2a00640 10 client.26899 handle_command_reply: tid=0 2024-09-10T07:42:53.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.982+0000 7f29061e8740 1 client.26899 shutdown 2024-09-10T07:42:53.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f29061e8740 1 -- 172.21.15.46:0/902138759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 msgr2=0x55d1b517a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:53.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f29061e8740 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b517a010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f28f401c040 tx=0x7f28f401af70 comp rx=0 tx=0).stop 2024-09-10T07:42:53.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f29061e8740 1 -- 172.21.15.46:0/902138759 shutdown_connections 2024-09-10T07:42:53.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f29061e8740 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1b518c200 0x55d1b518e5f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f29061e8740 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b517a010 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f29061e8740 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 0x55d1b5179ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f29061e8740 1 --2- 172.21.15.46:0/902138759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b5179590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f29061e8740 1 -- 172.21.15.46:0/902138759 >> 172.21.15.46:0/902138759 conn(0x55d1b516d940 msgr2=0x55d1b516cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.983+0000 7f28e2a00640 0 client.26899 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.984+0000 7f29061e8740 1 -- 172.21.15.46:0/902138759 shutdown_connections 2024-09-10T07:42:53.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.986+0000 7f29061e8740 1 -- 172.21.15.46:0/902138759 wait complete. 2024-09-10T07:42:53.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:53.986+0000 7f29061e8740 20 client.26899 trim_cache size 0 max 16384 2024-09-10T07:42:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.022+0000 7f29061e8740 1 Processor -- start 2024-09-10T07:42:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.022+0000 7f29061e8740 1 -- start start 2024-09-10T07:42:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.022+0000 7f29061e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b516ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.022+0000 7f29061e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 0x55d1b5193c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.022+0000 7f29061e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b5194160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.022+0000 7f29061e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc127cf0 con 0x55d1b51713b0 2024-09-10T07:42:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.022+0000 7f29061e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc1298c0 con 0x55d1b5171cd0 2024-09-10T07:42:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.022+0000 7f29061e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc1283c0 con 0x55d1b517c590 2024-09-10T07:42:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.023+0000 7f2901800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b5194160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.023+0000 7f2901800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b5194160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43596/0 (socket says 172.21.15.46:43596) 2024-09-10T07:42:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.023+0000 7f2901800640 1 -- 172.21.15.46:0/268844562 learned_addr learned my addr 172.21.15.46:0/268844562 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.023+0000 7f2900e00640 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b516ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.023+0000 7f28fbe00640 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 0x55d1b5193c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.024+0000 7f2901800640 1 -- 172.21.15.46:0/268844562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 msgr2=0x55d1b5193c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.024+0000 7f2901800640 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 0x55d1b5193c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.024+0000 7f2901800640 1 -- 172.21.15.46:0/268844562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 msgr2=0x55d1b516ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.024+0000 7f2901800640 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b516ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.024+0000 7f2901800640 1 -- 172.21.15.46:0/268844562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b517f250 con 0x55d1b517c590 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.024+0000 7f2901800640 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b5194160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009830 tx=0x7f28f4009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c07fb552feb738f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.025+0000 7f28ed600640 1 -- 172.21.15.46:0/268844562 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4028040 con 0x55d1b517c590 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.025+0000 7f28ed600640 1 -- 172.21.15.46:0/268844562 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f4011d70 con 0x55d1b517c590 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.025+0000 7f28ed600640 1 -- 172.21.15.46:0/268844562 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f40054e0 con 0x55d1b517c590 2024-09-10T07:42:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.026+0000 7f29061e8740 1 -- 172.21.15.46:0/268844562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 msgr2=0x55d1b5194160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.026+0000 7f29061e8740 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b5194160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009830 tx=0x7f28f4009230 comp rx=0 tx=0).stop 2024-09-10T07:42:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.026+0000 7f29061e8740 1 -- 172.21.15.46:0/268844562 shutdown_connections 2024-09-10T07:42:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.026+0000 7f29061e8740 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b517c590 0x55d1b5194160 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.026+0000 7f29061e8740 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b5171cd0 0x55d1b5193c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.026+0000 7f29061e8740 1 --2- 172.21.15.46:0/268844562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b51713b0 0x55d1b516ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.026+0000 7f29061e8740 1 -- 172.21.15.46:0/268844562 >> 172.21.15.46:0/268844562 conn(0x55d1b516d940 msgr2=0x55d1b5178680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.027+0000 7f29061e8740 1 -- 172.21.15.46:0/268844562 shutdown_connections 2024-09-10T07:42:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.027+0000 7f29061e8740 1 -- 172.21.15.46:0/268844562 wait complete. 2024-09-10T07:42:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.027+0000 7f29061e8740 1 -- start start 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.027+0000 7f29061e8740 1 Processor -- start 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.028+0000 7f29061e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b51713b0 0x55d1b5184580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.028+0000 7f29061e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 0x55d1b5184ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.028+0000 7f29061e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b517c590 0x55d1b5181550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.028+0000 7f29061e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc129190 con 0x55d1b517c590 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.028+0000 7f29061e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc128a90 con 0x55d1b51713b0 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.028+0000 7f29061e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc1043d0 con 0x55d1b5171cd0 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.028+0000 7f29061e8740 10 client.0 _ll_register_callbacks cb 0x7f28f4009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:42:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.028+0000 7f2900e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b51713b0 0x55d1b5184580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f28fbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 0x55d1b5184ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f2900e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b51713b0 0x55d1b5184580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34408/0 (socket says 172.21.15.46:34408) 2024-09-10T07:42:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f2900e00640 1 -- 172.21.15.46:0/2817322617 learned_addr learned my addr 172.21.15.46:0/2817322617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:42:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f2901800640 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b517c590 0x55d1b5181550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f28fbe00640 1 -- 172.21.15.46:0/2817322617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b51713b0 msgr2=0x55d1b5184580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f28fbe00640 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b51713b0 0x55d1b5184580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f28fbe00640 1 -- 172.21.15.46:0/2817322617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b517c590 msgr2=0x55d1b5181550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:42:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f28fbe00640 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b517c590 0x55d1b5181550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.029+0000 7f28fbe00640 1 -- 172.21.15.46:0/2817322617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b516ead0 con 0x55d1b5171cd0 2024-09-10T07:42:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.030+0000 7f28fbe00640 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 0x55d1b5184ac0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55d1b516e7a0 tx=0x7f28e80b6cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad77f48e87086a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:42:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.030+0000 7f28e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:42:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.030+0000 7f28e2a00640 1 -- 172.21.15.46:0/2817322617 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e80d3060 con 0x55d1b5171cd0 2024-09-10T07:42:54.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.031+0000 7f28e2a00640 1 -- 172.21.15.46:0/2817322617 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28e80b6f50 con 0x55d1b5171cd0 2024-09-10T07:42:54.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.031+0000 7f29061e8740 1 -- 172.21.15.46:0/2817322617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f4011d70 con 0x55d1b5171cd0 2024-09-10T07:42:54.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.031+0000 7f28e2a00640 1 -- 172.21.15.46:0/2817322617 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f28e801f040 con 0x55d1b5171cd0 2024-09-10T07:42:54.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.031+0000 7f29061e8740 10 client.26905 fetch_fsmap learned FSMap version 43 2024-09-10T07:42:54.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.031+0000 7f29061e8740 1 -- 172.21.15.46:0/2817322617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d1b51f7360 con 0x55d1b5171cd0 2024-09-10T07:42:54.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.032+0000 7f28e2a00640 1 -- 172.21.15.46:0/2817322617 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f4011d70 con 0x55d1b5171cd0 2024-09-10T07:42:54.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.032+0000 7f28e2a00640 1 -- 172.21.15.46:0/2817322617 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f28e801a030 con 0x55d1b5171cd0 2024-09-10T07:42:54.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.032+0000 7f29061e8740 10 client.26905 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:42:54.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.032+0000 7f29061e8740 10 client.26905 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:42:54.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.032+0000 7f29061e8740 20 client.26905 populate_metadata read hostname 'smithi046' 2024-09-10T07:42:54.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.032+0000 7f29061e8740 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1b51f7b60 0x55d1b51f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:42:54.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.032+0000 7f29061e8740 4 client.26905 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T07:42:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.032+0000 7f29061e8740 1 -- 172.21.15.46:0/2817322617 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x55d1b50e4c60 con 0x55d1b51f7b60 2024-09-10T07:42:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.033+0000 7f2900e00640 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1b51f7b60 0x55d1b51f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:42:54.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.035+0000 7f28e2a00640 10 client.26905 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:54.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.036+0000 7f2900e00640 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1b51f7b60 0x55d1b51f9f80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b8b8db83fe18db server_cookie=e139eb9dbe536d30 in_seq=0 out_seq=0 2024-09-10T07:42:54.121 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:54 smithi046 ceph-mon[30705]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 559 op/s 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.346+0000 7f28e2a00640 1 -- 172.21.15.46:0/2817322617 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d1b50e4c60 con 0x55d1b51f7b60 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.346+0000 7f28e2a00640 10 client.26905 handle_command_reply: tid=0 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 client.26905 shutdown 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 -- 172.21.15.46:0/2817322617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 msgr2=0x55d1b5184ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 0x55d1b5184ac0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55d1b516e7a0 tx=0x7f28e80b6cb0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 -- 172.21.15.46:0/2817322617 shutdown_connections 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1b51f7b60 0x55d1b51f9f80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b517c590 0x55d1b5181550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1b5171cd0 0x55d1b5184ac0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f28e2a00640 0 client.26905 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:42:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 --2- 172.21.15.46:0/2817322617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1b51713b0 0x55d1b5184580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 -- 172.21.15.46:0/2817322617 >> 172.21.15.46:0/2817322617 conn(0x55d1b516d940 msgr2=0x55d1b517abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 -- 172.21.15.46:0/2817322617 shutdown_connections 2024-09-10T07:42:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 1 -- 172.21.15.46:0/2817322617 wait complete. 2024-09-10T07:42:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.347+0000 7f29061e8740 20 client.26905 trim_cache size 0 max 16384 2024-09-10T07:42:54.348 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:42:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 43, 2024-09-10T07:42:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 18, 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "6a5933b6": { 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 18, 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 140.1, 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:42:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:42:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 -- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28e40a2c40 msgr2=0x7f28e40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28e40a2c40 0x7f28e40a5100 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009b60 tx=0x7f28f0005f90 comp rx=0 tx=0).stop 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 -- 172.21.15.46:0/3575674549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 msgr2=0x7f28fc1167b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 0x7f28fc1167b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28e8013120 tx=0x7f28e8004270 comp rx=0 tx=0).stop 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 -- 172.21.15.46:0/3575674549 shutdown_connections 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc109120 0x7f28fc1117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28fc1049a0 0x7f28fc1167b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28e40a2c40 0x7f28e40a5100 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 --2- 172.21.15.46:0/3575674549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28fc103ff0 0x7f28fc116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.348+0000 7f2903800640 1 -- 172.21.15.46:0/3575674549 >> 172.21.15.46:0/3575674549 conn(0x7f28fc0ffa90 msgr2=0x7f28fc1079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.349+0000 7f2903800640 1 -- 172.21.15.46:0/3575674549 shutdown_connections 2024-09-10T07:42:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:42:54.349+0000 7f2903800640 1 -- 172.21.15.46:0/3575674549 wait complete. 2024-09-10T07:42:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:54 smithi181 ceph-mon[35397]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 559 op/s 2024-09-10T07:42:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:54 smithi063 ceph-mon[35020]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 559 op/s 2024-09-10T07:42:55.252 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6a5933b6', quiesce took 6.5 seconds. Will release after: 13.600000000000001 2024-09-10T07:42:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 335.0 s, 554.2 tps, lat 120.783 ms stddev 154.408 2024-09-10T07:42:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:56 smithi181 ceph-mon[35397]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 491 op/s 2024-09-10T07:42:56.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:56 smithi063 ceph-mon[35020]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 491 op/s 2024-09-10T07:42:56.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:56 smithi046 ceph-mon[30705]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 491 op/s 2024-09-10T07:42:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:42:58 smithi181 ceph-mon[35397]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 668 op/s 2024-09-10T07:42:58.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:42:58 smithi063 ceph-mon[35020]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 668 op/s 2024-09-10T07:42:58.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:42:58 smithi046 ceph-mon[30705]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 668 op/s 2024-09-10T07:43:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:00 smithi181 ceph-mon[35397]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 493 op/s 2024-09-10T07:43:00.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:00 smithi063 ceph-mon[35020]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 493 op/s 2024-09-10T07:43:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:00 smithi046 ceph-mon[30705]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 493 op/s 2024-09-10T07:43:00.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 340.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:43:02.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:02 smithi046 ceph-mon[30705]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 469 op/s 2024-09-10T07:43:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:02 smithi181 ceph-mon[35397]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 469 op/s 2024-09-10T07:43:02.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:02 smithi063 ceph-mon[35020]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 469 op/s 2024-09-10T07:43:03.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:43:03.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:43:03.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:03.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:43:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:04 smithi181 ceph-mon[35397]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 329 op/s 2024-09-10T07:43:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:04 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-10T07:43:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:04 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi063 to 176.0M 2024-09-10T07:43:04.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:04 smithi181 ceph-mon[35397]: Unable to set osd_memory_target on smithi181 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-10T07:43:04.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:04 smithi181 ceph-mon[35397]: Unable to set osd_memory_target on smithi063 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T07:43:04.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:43:04.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:43:04.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:43:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:04 smithi063 ceph-mon[35020]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 329 op/s 2024-09-10T07:43:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:04 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-10T07:43:04.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:04 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi063 to 176.0M 2024-09-10T07:43:04.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:04 smithi063 ceph-mon[35020]: Unable to set osd_memory_target on smithi181 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-10T07:43:04.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:04 smithi063 ceph-mon[35020]: Unable to set osd_memory_target on smithi063 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T07:43:04.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:43:04.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:43:04.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:43:04.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:04 smithi046 ceph-mon[30705]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 329 op/s 2024-09-10T07:43:04.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:04 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-10T07:43:04.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:04 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi063 to 176.0M 2024-09-10T07:43:04.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:04 smithi046 ceph-mon[30705]: Unable to set osd_memory_target on smithi181 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-10T07:43:04.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:04 smithi046 ceph-mon[30705]: Unable to set osd_memory_target on smithi063 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T07:43:04.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:43:04.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:43:04.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:43:05.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 345.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:43:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:06 smithi181 ceph-mon[35397]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 245 op/s 2024-09-10T07:43:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:06 smithi063 ceph-mon[35020]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 245 op/s 2024-09-10T07:43:06.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:06 smithi046 ceph-mon[30705]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 245 op/s 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 -- 172.21.15.46:0/1483704229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530103d60 msgr2=0x7f0530104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 --2- 172.21.15.46:0/1483704229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530103d60 0x7f0530104160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0520009ca0 tx=0x7f052002a8b0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 -- 172.21.15.46:0/1483704229 shutdown_connections 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 --2- 172.21.15.46:0/1483704229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530105910 0x7f053010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 --2- 172.21.15.46:0/1483704229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0530104f50 0x7f05301053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 --2- 172.21.15.46:0/1483704229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530103d60 0x7f0530104160 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 -- 172.21.15.46:0/1483704229 >> 172.21.15.46:0/1483704229 conn(0x7f05300ff4e0 msgr2=0x7f0530101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 -- 172.21.15.46:0/1483704229 shutdown_connections 2024-09-10T07:43:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.072+0000 7f0535000640 1 -- 172.21.15.46:0/1483704229 wait complete. 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.073+0000 7f0535000640 1 Processor -- start 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.073+0000 7f0535000640 1 -- start start 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0535000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530103d60 0x7f0530199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0535000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530104f50 0x7f053019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0535000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0530105910 0x7f05301a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0535000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f053010fa30 con 0x7f0530104f50 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0535000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f053010f8b0 con 0x7f0530105910 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0535000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f053010fbb0 con 0x7f0530103d60 2024-09-10T07:43:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0525400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530104f50 0x7f053019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0525400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530104f50 0x7f053019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40638/0 (socket says 172.21.15.46:40638) 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0525400640 1 -- 172.21.15.46:0/1565220077 learned_addr learned my addr 172.21.15.46:0/1565220077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f052de00640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530103d60 0x7f0530199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f052e800640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0530105910 0x7f05301a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0525400640 1 -- 172.21.15.46:0/1565220077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530103d60 msgr2=0x7f0530199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0525400640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530103d60 0x7f0530199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0525400640 1 -- 172.21.15.46:0/1565220077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0530105910 msgr2=0x7f05301a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.074+0000 7f0525400640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0530105910 0x7f05301a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.075+0000 7f0525400640 1 -- 172.21.15.46:0/1565220077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05301a1bf0 con 0x7f0530104f50 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.075+0000 7f052e800640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0530105910 0x7f05301a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.075+0000 7f0525400640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530104f50 0x7f053019a510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f051800e9f0 tx=0x7f051800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.075+0000 7f0527400640 1 -- 172.21.15.46:0/1565220077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051800cd80 con 0x7f0530104f50 2024-09-10T07:43:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.075+0000 7f0527400640 1 -- 172.21.15.46:0/1565220077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0518010910 con 0x7f0530104f50 2024-09-10T07:43:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.075+0000 7f0535000640 1 -- 172.21.15.46:0/1565220077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05301a1ee0 con 0x7f0530104f50 2024-09-10T07:43:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.075+0000 7f0527400640 1 -- 172.21.15.46:0/1565220077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518018990 con 0x7f0530104f50 2024-09-10T07:43:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.075+0000 7f0535000640 1 -- 172.21.15.46:0/1565220077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05301a23a0 con 0x7f0530104f50 2024-09-10T07:43:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.078+0000 7f0537def740 1 Processor -- start 2024-09-10T07:43:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.078+0000 7f0527400640 1 -- 172.21.15.46:0/1565220077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0518010ac0 con 0x7f0530104f50 2024-09-10T07:43:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.078+0000 7f0537def740 1 -- start start 2024-09-10T07:43:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f0537def740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adf1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f0537def740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf19d0 0x55af0adf6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f0537def740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf6a60 0x55af0adf8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f0537def740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55af0ad56100 con 0x55af0adf1090 2024-09-10T07:43:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f0537def740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af0acfe1b0 con 0x55af0adf19d0 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f0537def740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55af0ad520c0 con 0x55af0adf6a60 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f052e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adf1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f052e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adf1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40644/0 (socket says 172.21.15.46:40644) 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f052e800640 1 -- 172.21.15.46:0/3202795091 learned_addr learned my addr 172.21.15.46:0/3202795091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f052de00640 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf19d0 0x55af0adf6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f0525400640 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf6a60 0x55af0adf8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f052e800640 1 -- 172.21.15.46:0/3202795091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf6a60 msgr2=0x55af0adf8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f052e800640 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf6a60 0x55af0adf8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.079+0000 7f052e800640 1 -- 172.21.15.46:0/3202795091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf19d0 msgr2=0x55af0adf6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f052e800640 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf19d0 0x55af0adf6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f052e800640 1 -- 172.21.15.46:0/3202795091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af0aa69210 con 0x55af0adf1090 2024-09-10T07:43:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f052de00640 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf19d0 0x55af0adf6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f052e800640 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adf1490 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0528011540 tx=0x7f0528011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98961f596d72446e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f0527400640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05140a2c30 0x7f05140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f051f400640 1 -- 172.21.15.46:0/3202795091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052800f5c0 con 0x55af0adf1090 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f051f400640 1 -- 172.21.15.46:0/3202795091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052800fc40 con 0x55af0adf1090 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f051f400640 1 -- 172.21.15.46:0/3202795091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528019cb0 con 0x55af0adf1090 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.080+0000 7f0527400640 1 -- 172.21.15.46:0/1565220077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f05180cf740 con 0x7f0530104f50 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f052de00640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05140a2c30 0x7f05140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 -- 172.21.15.46:0/3202795091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 msgr2=0x55af0adf1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adf1490 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0528011540 tx=0x7f0528011a00 comp rx=0 tx=0).stop 2024-09-10T07:43:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 -- 172.21.15.46:0/3202795091 shutdown_connections 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf6a60 0x55af0adf8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf19d0 0x55af0adf6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 --2- 172.21.15.46:0/3202795091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adf1490 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 -- 172.21.15.46:0/3202795091 >> 172.21.15.46:0/3202795091 conn(0x55af0adec700 msgr2=0x55af0adecb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 -- 172.21.15.46:0/3202795091 shutdown_connections 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f052de00640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05140a2c30 0x7f05140a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0520000f80 tx=0x7f0520005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.081+0000 7f0537def740 1 -- 172.21.15.46:0/3202795091 wait complete. 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.082+0000 7f0537def740 1 -- start start 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.082+0000 7f0537def740 1 Processor -- start 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.082+0000 7f0537def740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adfb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.082+0000 7f0537def740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 0x55af0adfb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.082+0000 7f0537def740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0ae02790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.082+0000 7f0537def740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55af0adf9a30 con 0x55af0adf1090 2024-09-10T07:43:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.082+0000 7f0537def740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af0acdf090 con 0x55af0adf6a60 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.082+0000 7f0537def740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55af0adf9bb0 con 0x55af0adf19d0 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f0537def740 10 client.0 _ll_register_callbacks cb 0x55af0ad33f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adfb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0ae02790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adfb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40658/0 (socket says 172.21.15.46:40658) 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052de00640 1 -- 172.21.15.46:0/2370118743 learned_addr learned my addr 172.21.15.46:0/2370118743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0ae02790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60050/0 (socket says 172.21.15.46:60050) 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f0525400640 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 0x55af0adfb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052e800640 1 -- 172.21.15.46:0/2370118743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 msgr2=0x55af0adfb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052e800640 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 0x55af0adfb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052e800640 1 -- 172.21.15.46:0/2370118743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 msgr2=0x55af0adfb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052e800640 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adfb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.083+0000 7f052e800640 1 -- 172.21.15.46:0/2370118743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af0ab35620 con 0x55af0adf6a60 2024-09-10T07:43:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.084+0000 7f052e800640 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0ae02790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f052801b040 tx=0x7f0528004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5594b55955ca96c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.084+0000 7f0513e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.084+0000 7f0513e00640 1 -- 172.21.15.46:0/2370118743 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528026020 con 0x55af0adf6a60 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.084+0000 7f0537def740 1 -- 172.21.15.46:0/2370118743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f052800fc40 con 0x55af0adf6a60 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.084+0000 7f0513e00640 1 -- 172.21.15.46:0/2370118743 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f05280199b0 con 0x55af0adf6a60 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.084+0000 7f0513e00640 1 -- 172.21.15.46:0/2370118743 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0528031050 con 0x55af0adf6a60 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.084+0000 7f0537def740 10 client.26954 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.084+0000 7f0537def740 1 -- 172.21.15.46:0/2370118743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55af0ae055d0 con 0x55af0adf6a60 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f0513e00640 1 -- 172.21.15.46:0/2370118743 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052800fc40 con 0x55af0adf6a60 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f0513e00640 1 -- 172.21.15.46:0/2370118743 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f05280037f0 con 0x55af0adf6a60 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f0537def740 10 client.26954 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f0537def740 10 client.26954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f0537def740 20 client.26954 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f0537def740 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af0ae06bc0 0x55af0ae08fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f0537def740 4 client.26954 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f0537def740 1 -- 172.21.15.46:0/2370118743 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af0ad63830 con 0x55af0ae06bc0 2024-09-10T07:43:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.085+0000 7f052de00640 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af0ae06bc0 0x55af0ae08fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.086+0000 7f052de00640 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af0ae06bc0 0x55af0ae08fb0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977b2053cb24bb35 server_cookie=d01de2d273b8ae50 in_seq=0 out_seq=0 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.086+0000 7f0513e00640 10 client.26954 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0513e00640 1 -- 172.21.15.46:0/2370118743 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af0ad63830 con 0x55af0ae06bc0 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0513e00640 10 client.26954 handle_command_reply: tid=0 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 client.26954 shutdown 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 -- 172.21.15.46:0/2370118743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 msgr2=0x55af0ae02790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0ae02790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f052801b040 tx=0x7f0528004040 comp rx=0 tx=0).stop 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 -- 172.21.15.46:0/2370118743 shutdown_connections 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af0ae06bc0 0x55af0ae08fb0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0ae02790 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 0x55af0adfb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 --2- 172.21.15.46:0/2370118743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf1090 0x55af0adfb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0513e00640 0 client.26954 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 -- 172.21.15.46:0/2370118743 >> 172.21.15.46:0/2370118743 conn(0x55af0adec700 msgr2=0x55af0adedf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 -- 172.21.15.46:0/2370118743 shutdown_connections 2024-09-10T07:43:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 1 -- 172.21.15.46:0/2370118743 wait complete. 2024-09-10T07:43:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.087+0000 7f0537def740 20 client.26954 trim_cache size 0 max 16384 2024-09-10T07:43:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.102+0000 7f0537def740 1 Processor -- start 2024-09-10T07:43:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.102+0000 7f0537def740 1 -- start start 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0537def740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf1090 0x55af0adecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0537def740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf19d0 0x55af0ae026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0537def740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0aded020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0537def740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05301ac890 con 0x55af0adf19d0 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0537def740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05301ae460 con 0x55af0adf6a60 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0537def740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f05301acf60 con 0x55af0adf1090 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0525400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf19d0 0x55af0ae026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0525400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf19d0 0x55af0ae026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40670/0 (socket says 172.21.15.46:40670) 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0525400640 1 -- 172.21.15.46:0/4104636294 learned_addr learned my addr 172.21.15.46:0/4104636294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f052e800640 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0aded020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f052de00640 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf1090 0x55af0adecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0525400640 1 -- 172.21.15.46:0/4104636294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf1090 msgr2=0x55af0adecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0525400640 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf1090 0x55af0adecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0525400640 1 -- 172.21.15.46:0/4104636294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 msgr2=0x55af0aded020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0525400640 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0aded020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.103+0000 7f0525400640 1 -- 172.21.15.46:0/4104636294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af0ae09610 con 0x55af0adf19d0 2024-09-10T07:43:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.104+0000 7f0525400640 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf19d0 0x55af0ae026b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f05180cf9e0 tx=0x7f0518009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebee0e610456482 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.104+0000 7f051ea00640 1 -- 172.21.15.46:0/4104636294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051809b5d0 con 0x55af0adf19d0 2024-09-10T07:43:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.104+0000 7f051ea00640 1 -- 172.21.15.46:0/4104636294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051809bc70 con 0x55af0adf19d0 2024-09-10T07:43:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.104+0000 7f051ea00640 1 -- 172.21.15.46:0/4104636294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05180cfb30 con 0x55af0adf19d0 2024-09-10T07:43:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 -- 172.21.15.46:0/4104636294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf19d0 msgr2=0x55af0ae026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf19d0 0x55af0ae026b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f05180cf9e0 tx=0x7f0518009700 comp rx=0 tx=0).stop 2024-09-10T07:43:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 -- 172.21.15.46:0/4104636294 shutdown_connections 2024-09-10T07:43:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf6a60 0x55af0aded020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf19d0 0x55af0ae026b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 --2- 172.21.15.46:0/4104636294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf1090 0x55af0adecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 -- 172.21.15.46:0/4104636294 >> 172.21.15.46:0/4104636294 conn(0x55af0adec700 msgr2=0x55af0adfb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 -- 172.21.15.46:0/4104636294 shutdown_connections 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 -- 172.21.15.46:0/4104636294 wait complete. 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 -- start start 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.105+0000 7f0537def740 1 Processor -- start 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f0537def740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf1090 0x55af0adff970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f0537def740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 0x55af0adffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f0537def740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf6a60 0x55af0ae00ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f0537def740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05301add30 con 0x55af0adf6a60 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f0537def740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05301ad630 con 0x55af0adf1090 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f0537def740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f05301aa870 con 0x55af0adf19d0 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f0537def740 10 client.0 _ll_register_callbacks cb 0x7f0528005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf6a60 0x55af0ae00ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf6a60 0x55af0ae00ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40672/0 (socket says 172.21.15.46:40672) 2024-09-10T07:43:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052e800640 1 -- 172.21.15.46:0/331745777 learned_addr learned my addr 172.21.15.46:0/331745777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052de00640 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf1090 0x55af0adff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f0525400640 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 0x55af0adffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052e800640 1 -- 172.21.15.46:0/331745777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 msgr2=0x55af0adffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052e800640 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 0x55af0adffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052e800640 1 -- 172.21.15.46:0/331745777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf1090 msgr2=0x55af0adff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052e800640 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf1090 0x55af0adff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.106+0000 7f052e800640 1 -- 172.21.15.46:0/331745777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af0ae014d0 con 0x55af0adf6a60 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f052e800640 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf6a60 0x55af0ae00ec0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0528011a40 tx=0x7f0528011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56290b53e8aaa142 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f0513e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f0513e00640 1 -- 172.21.15.46:0/331745777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528026040 con 0x55af0adf6a60 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f0513e00640 1 -- 172.21.15.46:0/331745777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0528005040 con 0x55af0adf6a60 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f0537def740 1 -- 172.21.15.46:0/331745777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f051809bc70 con 0x55af0adf6a60 2024-09-10T07:43:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f0513e00640 1 -- 172.21.15.46:0/331745777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0528005850 con 0x55af0adf6a60 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f0537def740 10 client.16551 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f0537def740 1 -- 172.21.15.46:0/331745777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55af0ae027e0 con 0x55af0adf6a60 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.107+0000 7f0513e00640 1 -- 172.21.15.46:0/331745777 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051809bc70 con 0x55af0adf6a60 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.108+0000 7f0513e00640 1 -- 172.21.15.46:0/331745777 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f052800fa60 con 0x55af0adf6a60 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.108+0000 7f0537def740 10 client.16551 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.108+0000 7f0537def740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.108+0000 7f0537def740 20 client.16551 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.108+0000 7f0537def740 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af0ae71d50 0x55af0ae74170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.108+0000 7f0537def740 4 client.16551 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.108+0000 7f0537def740 1 -- 172.21.15.46:0/331745777 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af0ad5db80 con 0x55af0ae71d50 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.108+0000 7f052de00640 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af0ae71d50 0x55af0ae74170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.109+0000 7f052de00640 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af0ae71d50 0x55af0ae74170 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f3d31807840bf9 server_cookie=930ba08cf486f222 in_seq=0 out_seq=0 2024-09-10T07:43:07.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.109+0000 7f0513e00640 10 client.16551 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.124+0000 7f0513e00640 1 -- 172.21.15.46:0/331745777 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af0ad5db80 con 0x55af0ae71d50 2024-09-10T07:43:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.124+0000 7f0513e00640 10 client.16551 handle_command_reply: tid=0 2024-09-10T07:43:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.124+0000 7f0537def740 1 client.16551 shutdown 2024-09-10T07:43:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 -- 172.21.15.46:0/331745777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf6a60 msgr2=0x55af0ae00ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf6a60 0x55af0ae00ec0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0528011a40 tx=0x7f0528011440 comp rx=0 tx=0).stop 2024-09-10T07:43:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 -- 172.21.15.46:0/331745777 shutdown_connections 2024-09-10T07:43:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af0ae71d50 0x55af0ae74170 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af0adf6a60 0x55af0ae00ec0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af0adf19d0 0x55af0adffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 --2- 172.21.15.46:0/331745777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af0adf1090 0x55af0adff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0513e00640 0 client.16551 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 -- 172.21.15.46:0/331745777 >> 172.21.15.46:0/331745777 conn(0x55af0adec700 msgr2=0x55af0adeb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 -- 172.21.15.46:0/331745777 shutdown_connections 2024-09-10T07:43:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 1 -- 172.21.15.46:0/331745777 wait complete. 2024-09-10T07:43:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.125+0000 7f0537def740 20 client.16551 trim_cache size 0 max 16384 2024-09-10T07:43:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:43:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:43:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:43:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:43:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 -- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05140a2c30 msgr2=0x7f05140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05140a2c30 0x7f05140a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0520000f80 tx=0x7f0520005f30 comp rx=0 tx=0).stop 2024-09-10T07:43:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 -- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530104f50 msgr2=0x7f053019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530104f50 0x7f053019a510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f051800e9f0 tx=0x7f051800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 -- 172.21.15.46:0/1565220077 shutdown_connections 2024-09-10T07:43:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0530105910 0x7f05301a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f05140a2c30 0x7f05140a50f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0530104f50 0x7f053019a510 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 --2- 172.21.15.46:0/1565220077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0530103d60 0x7f0530199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 -- 172.21.15.46:0/1565220077 >> 172.21.15.46:0/1565220077 conn(0x7f05300ff4e0 msgr2=0x7f05301081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 -- 172.21.15.46:0/1565220077 shutdown_connections 2024-09-10T07:43:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:07.126+0000 7f0535000640 1 -- 172.21.15.46:0/1565220077 wait complete. 2024-09-10T07:43:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:08 smithi181 ceph-mon[35397]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-09-10T07:43:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:08 smithi063 ceph-mon[35020]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-09-10T07:43:08.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:08 smithi046 ceph-mon[30705]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-09-10T07:43:08.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:43:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 -- 172.21.15.46:0/1820162820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 msgr2=0x7f563c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 --2- 172.21.15.46:0/1820162820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 0x7f563c076b00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f562c009900 tx=0x7f562c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 -- 172.21.15.46:0/1820162820 shutdown_connections 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 --2- 172.21.15.46:0/1820162820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f563c112a30 0x7f563c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 --2- 172.21.15.46:0/1820162820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563c077040 0x7f563c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 --2- 172.21.15.46:0/1820162820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 0x7f563c076b00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 -- 172.21.15.46:0/1820162820 >> 172.21.15.46:0/1820162820 conn(0x7f563c0ffa90 msgr2=0x7f563c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 -- 172.21.15.46:0/1820162820 shutdown_connections 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.817+0000 7f5642400640 1 -- 172.21.15.46:0/1820162820 wait complete. 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.818+0000 7f5642400640 1 Processor -- start 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.818+0000 7f5642400640 1 -- start start 2024-09-10T07:43:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f5642400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 0x7f563c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f5642400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f563c077040 0x7f563c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f5642400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563c112a30 0x7f563c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f5642400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1185c0 con 0x7f563c076720 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f5642400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f563c118440 con 0x7f563c077040 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f5642400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f563c118740 con 0x7f563c112a30 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f563b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 0x7f563c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f563b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 0x7f563c1a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40682/0 (socket says 172.21.15.46:40682) 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f563b400640 1 -- 172.21.15.46:0/1068302357 learned_addr learned my addr 172.21.15.46:0/1068302357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f563aa00640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f563c077040 0x7f563c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.819+0000 7f563be00640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563c112a30 0x7f563c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f563aa00640 1 -- 172.21.15.46:0/1068302357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563c112a30 msgr2=0x7f563c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f563aa00640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563c112a30 0x7f563c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f563aa00640 1 -- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 msgr2=0x7f563c1a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f563aa00640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 0x7f563c1a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:09.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f563aa00640 1 -- 172.21.15.46:0/1068302357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c19efe0 con 0x7f563c077040 2024-09-10T07:43:09.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f563b400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 0x7f563c1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:09.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f563aa00640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f563c077040 0x7f563c1a4c30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f562800e9f0 tx=0x7f562800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:09.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f5633e00640 1 -- 172.21.15.46:0/1068302357 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562800ccc0 con 0x7f563c077040 2024-09-10T07:43:09.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c19f2d0 con 0x7f563c077040 2024-09-10T07:43:09.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f5633e00640 1 -- 172.21.15.46:0/1068302357 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5628005020 con 0x7f563c077040 2024-09-10T07:43:09.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.820+0000 7f5633e00640 1 -- 172.21.15.46:0/1068302357 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628004a00 con 0x7f563c077040 2024-09-10T07:43:09.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.821+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c1ab5c0 con 0x7f563c077040 2024-09-10T07:43:09.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.822+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563c076b00 con 0x7f563c077040 2024-09-10T07:43:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.825+0000 7f5633e00640 1 -- 172.21.15.46:0/1068302357 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f56280051d0 con 0x7f563c077040 2024-09-10T07:43:09.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.826+0000 7f5633e00640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56200a2b80 0x7f56200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:09.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.826+0000 7f5633e00640 1 -- 172.21.15.46:0/1068302357 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f56280cf3d0 con 0x7f563c077040 2024-09-10T07:43:09.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.826+0000 7f563b400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56200a2b80 0x7f56200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.827+0000 7f563b400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56200a2b80 0x7f56200a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f562c031550 tx=0x7f562c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:09.827+0000 7f5633e00640 1 -- 172.21.15.46:0/1068302357 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f562809c040 con 0x7f563c077040 2024-09-10T07:43:09.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:43:10.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.051+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f563c19fde0 con 0x7f563c077040 2024-09-10T07:43:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.052+0000 7f5633e00640 1 -- 172.21.15.46:0/1068302357 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11976 (secure 0 0 0) 0x7f563c19fde0 con 0x7f563c077040 2024-09-10T07:43:10.052 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:43:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"btime":"2024-09-10T07:42:38:910267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:42:38.047718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:43:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 43 2024-09-10T07:43:10.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56200a2b80 msgr2=0x7f56200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:10.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56200a2b80 0x7f56200a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f562c031550 tx=0x7f562c005a20 comp rx=0 tx=0).stop 2024-09-10T07:43:10.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f563c077040 msgr2=0x7f563c1a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:10.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f563c077040 0x7f563c1a4c30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f562800e9f0 tx=0x7f562800edf0 comp rx=0 tx=0).stop 2024-09-10T07:43:10.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 shutdown_connections 2024-09-10T07:43:10.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563c112a30 0x7f563c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:10.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f563c077040 0x7f563c1a4c30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:10.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f56200a2b80 0x7f56200a5040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:10.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 --2- 172.21.15.46:0/1068302357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f563c076720 0x7f563c1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:10.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.054+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 >> 172.21.15.46:0/1068302357 conn(0x7f563c0ffa90 msgr2=0x7f563c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:10.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.055+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 shutdown_connections 2024-09-10T07:43:10.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:10.055+0000 7f5642400640 1 -- 172.21.15.46:0/1068302357 wait complete. 2024-09-10T07:43:10.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:10 smithi046 ceph-mon[30705]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 477 KiB/s wr, 29 op/s 2024-09-10T07:43:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:10 smithi181 ceph-mon[35397]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 477 KiB/s wr, 29 op/s 2024-09-10T07:43:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:10 smithi063 ceph-mon[35020]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 477 KiB/s wr, 29 op/s 2024-09-10T07:43:10.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 350.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:43:10.982 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id 6a5933b6 --release --await' 2024-09-10T07:43:10.983 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id 6a5933b6 --release --await 2024-09-10T07:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:11.205 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:11 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1068302357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:11.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:11 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1068302357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:11.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:11 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1068302357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 -- 172.21.15.46:0/821536713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc104010 msgr2=0x7f0bfc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 --2- 172.21.15.46:0/821536713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc104010 0x7f0bfc1043f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009900 tx=0x7f0bec02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 -- 172.21.15.46:0/821536713 shutdown_connections 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 --2- 172.21.15.46:0/821536713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bfc109140 0x7f0bfc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 --2- 172.21.15.46:0/821536713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bfc1049c0 0x7f0bfc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 --2- 172.21.15.46:0/821536713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc104010 0x7f0bfc1043f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 -- 172.21.15.46:0/821536713 >> 172.21.15.46:0/821536713 conn(0x7f0bfc0ffa50 msgr2=0x7f0bfc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 -- 172.21.15.46:0/821536713 shutdown_connections 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.646+0000 7f0c03400640 1 -- 172.21.15.46:0/821536713 wait complete. 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.647+0000 7f0c03400640 1 Processor -- start 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.647+0000 7f0c03400640 1 -- start start 2024-09-10T07:43:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.647+0000 7f0c03400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bfc104010 0x7f0bfc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.647+0000 7f0c03400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bfc1049c0 0x7f0bfc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.647+0000 7f0c03400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc109140 0x7f0bfc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.647+0000 7f0c03400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc114170 con 0x7f0bfc109140 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.647+0000 7f0c03400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc113ff0 con 0x7f0bfc104010 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.647+0000 7f0c03400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1142f0 con 0x7f0bfc1049c0 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc109140 0x7f0bfc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc109140 0x7f0bfc19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40714/0 (socket says 172.21.15.46:40714) 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 -- 172.21.15.46:0/2598890947 learned_addr learned my addr 172.21.15.46:0/2598890947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c00a00640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bfc104010 0x7f0bfc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bfc1049c0 0x7f0bfc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 -- 172.21.15.46:0/2598890947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bfc1049c0 msgr2=0x7f0bfc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bfc1049c0 0x7f0bfc19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 -- 172.21.15.46:0/2598890947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bfc104010 msgr2=0x7f0bfc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bfc104010 0x7f0bfc19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 -- 172.21.15.46:0/2598890947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc19f5e0 con 0x7f0bfc109140 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.648+0000 7f0c01400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc109140 0x7f0bfc19ef00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400eaa0 tx=0x7f0bf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.649+0000 7f0bfa000640 1 -- 172.21.15.46:0/2598890947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400cdb0 con 0x7f0bfc109140 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.649+0000 7f0bfa000640 1 -- 172.21.15.46:0/2598890947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf4005020 con 0x7f0bfc109140 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.649+0000 7f0c03400640 1 -- 172.21.15.46:0/2598890947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc19f8d0 con 0x7f0bfc109140 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.649+0000 7f0c03400640 1 -- 172.21.15.46:0/2598890947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc1a71b0 con 0x7f0bfc109140 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.649+0000 7f0bfa000640 1 -- 172.21.15.46:0/2598890947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40116b0 con 0x7f0bfc109140 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.652+0000 7f0c05fe5740 1 Processor -- start 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.652+0000 7f0bfa000640 1 -- 172.21.15.46:0/2598890947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0bf4005560 con 0x7f0bfc109140 2024-09-10T07:43:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.652+0000 7f0c05fe5740 1 -- start start 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e95830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e95d70 0x55c3b8e98160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8df13e0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8d98d80 con 0x55c3b8e95d70 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8ded3a0 con 0x55c3b8e8b4b0 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c00a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e95830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c00a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e95830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:45082/0 (socket says 172.21.15.46:45082) 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c00a00640 1 -- 172.21.15.46:0/2697339799 learned_addr learned my addr 172.21.15.46:0/2697339799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40724/0 (socket says 172.21.15.46:40724) 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e95d70 0x55c3b8e98160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0bf3e00640 1 -- 172.21.15.46:0/2697339799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 msgr2=0x55c3b8e95830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e95830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0bf3e00640 1 -- 172.21.15.46:0/2697339799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e95d70 msgr2=0x55c3b8e98160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e95d70 0x55c3b8e98160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0bf3e00640 1 -- 172.21.15.46:0/2697339799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b8db7ae0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.653+0000 7f0c00a00640 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e95830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.654+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8af70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0fff70 tx=0x7f0be40096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a7e41b2c44679 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.654+0000 7f0bfa000640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0be00a2b80 0x7f0be00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.654+0000 7f0bf1c00640 1 -- 172.21.15.46:0/2697339799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4004050 con 0x55c3b8e8ab90 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.654+0000 7f0bf1c00640 1 -- 172.21.15.46:0/2697339799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be40041f0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.654+0000 7f0bf1c00640 1 -- 172.21.15.46:0/2697339799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be40116d0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.654+0000 7f0bfa000640 1 -- 172.21.15.46:0/2598890947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0bf40cf5b0 con 0x7f0bfc109140 2024-09-10T07:43:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.655+0000 7f0c00a00640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0be00a2b80 0x7f0be00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.655+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2697339799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8ab90 msgr2=0x55c3b8e8af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.655+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8af70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0fff70 tx=0x7f0be40096d0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2697339799 shutdown_connections 2024-09-10T07:43:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e95d70 0x55c3b8e98160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e95830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2697339799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8af70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2697339799 >> 172.21.15.46:0/2697339799 conn(0x55c3b8e87120 msgr2=0x55c3b8e87500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2697339799 shutdown_connections 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2697339799 wait complete. 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c00a00640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0be00a2b80 0x7f0be00a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bec000c00 tx=0x7f0bec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.656+0000 7f0c05fe5740 1 -- start start 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0c05fe5740 1 Processor -- start 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e92c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e95d70 0x55c3b8e936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8e989e0 con 0x55c3b8e8b4b0 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8dac0f0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c3b8e98b60 con 0x55c3b8e95d70 2024-09-10T07:43:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0c05fe5740 10 client.0 _ll_register_callbacks cb 0x55c3b8d9d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.657+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c00a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e92c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e931b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40726/0 (socket says 172.21.15.46:40726) 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0bf3e00640 1 -- 172.21.15.46:0/3443799077 learned_addr learned my addr 172.21.15.46:0/3443799077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c01400640 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e95d70 0x55c3b8e936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c00a00640 1 -- 172.21.15.46:0/3443799077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e95d70 msgr2=0x55c3b8e936f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c00a00640 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e95d70 0x55c3b8e936f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c00a00640 1 -- 172.21.15.46:0/3443799077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8b4b0 msgr2=0x55c3b8e931b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c00a00640 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e931b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c00a00640 1 -- 172.21.15.46:0/3443799077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b8b623c0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c01400640 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e95d70 0x55c3b8e936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0c00a00640 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e92c70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0bec02a910 tx=0x7f0bec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d63614bb98d4d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.658+0000 7f0bea000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.659+0000 7f0bea000640 1 -- 172.21.15.46:0/3443799077 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec04e070 con 0x55c3b8e8ab90 2024-09-10T07:43:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.659+0000 7f0c05fe5740 1 -- 172.21.15.46:0/3443799077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be40041f0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.659+0000 7f0bea000640 1 -- 172.21.15.46:0/3443799077 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0bec003cc0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.659+0000 7f0bea000640 1 -- 172.21.15.46:0/3443799077 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0bec05a050 con 0x55c3b8e8ab90 2024-09-10T07:43:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.659+0000 7f0c05fe5740 10 client.26978 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.659+0000 7f0c05fe5740 1 -- 172.21.15.46:0/3443799077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c3b8ea4720 con 0x55c3b8e8ab90 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0bea000640 1 -- 172.21.15.46:0/3443799077 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be40041f0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0bea000640 1 -- 172.21.15.46:0/3443799077 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f0bec03d550 con 0x55c3b8e8ab90 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0c05fe5740 10 client.26978 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0c05fe5740 10 client.26978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0c05fe5740 20 client.26978 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c3b8ea5b80 0x55c3b8ea7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0c05fe5740 4 client.26978 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0c05fe5740 1 -- 172.21.15.46:0/3443799077 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3b8dfe950 con 0x55c3b8ea5b80 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.660+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c3b8ea5b80 0x55c3b8ea7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.661+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c3b8ea5b80 0x55c3b8ea7fe0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0eb5ba03400a61 server_cookie=9b236a1577881564 in_seq=0 out_seq=0 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.661+0000 7f0bea000640 10 client.26978 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0bea000640 1 -- 172.21.15.46:0/3443799077 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3b8dfe950 con 0x55c3b8ea5b80 2024-09-10T07:43:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0bea000640 10 client.26978 handle_command_reply: tid=0 2024-09-10T07:43:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 client.26978 shutdown 2024-09-10T07:43:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 -- 172.21.15.46:0/3443799077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 msgr2=0x55c3b8e92c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e92c70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0bec02a910 tx=0x7f0bec009260 comp rx=0 tx=0).stop 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 -- 172.21.15.46:0/3443799077 shutdown_connections 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c3b8ea5b80 0x55c3b8ea7fe0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e95d70 0x55c3b8e936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e931b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/3443799077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e92c70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0bea000640 0 client.26978 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 -- 172.21.15.46:0/3443799077 >> 172.21.15.46:0/3443799077 conn(0x55c3b8e87120 msgr2=0x55c3b8e8ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 -- 172.21.15.46:0/3443799077 shutdown_connections 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.662+0000 7f0c05fe5740 1 -- 172.21.15.46:0/3443799077 wait complete. 2024-09-10T07:43:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.663+0000 7f0c05fe5740 20 client.26978 trim_cache size 0 max 16384 2024-09-10T07:43:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.681+0000 7f0c05fe5740 1 Processor -- start 2024-09-10T07:43:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.681+0000 7f0c05fe5740 1 -- start start 2024-09-10T07:43:11.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.681+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.681+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8ead8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.681+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8eaddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.681+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1b0e60 con 0x55c3b8e95d70 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.681+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1b2a30 con 0x55c3b8e8ab90 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.681+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1b1530 con 0x55c3b8e8b4b0 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c00a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c00a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60152/0 (socket says 172.21.15.46:60152) 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c00a00640 1 -- 172.21.15.46:0/229137503 learned_addr learned my addr 172.21.15.46:0/229137503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8ead8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c01400640 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8eaddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c00a00640 1 -- 172.21.15.46:0/229137503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 msgr2=0x55c3b8ead8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c00a00640 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8ead8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c00a00640 1 -- 172.21.15.46:0/229137503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 msgr2=0x55c3b8eaddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c00a00640 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8eaddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c00a00640 1 -- 172.21.15.46:0/229137503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b8e87db0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0c01400640 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8eaddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.682+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8ead8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.683+0000 7f0c00a00640 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8d5f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0bec03d920 tx=0x7f0bec009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9b0920c67e4c8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.683+0000 7f0bf1200640 1 -- 172.21.15.46:0/229137503 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec062070 con 0x55c3b8e8ab90 2024-09-10T07:43:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.683+0000 7f0bf1200640 1 -- 172.21.15.46:0/229137503 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec03dd90 con 0x55c3b8e8ab90 2024-09-10T07:43:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.683+0000 7f0bf1200640 1 -- 172.21.15.46:0/229137503 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec04e040 con 0x55c3b8e8ab90 2024-09-10T07:43:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 -- 172.21.15.46:0/229137503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 msgr2=0x55c3b8e8d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8d5f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0bec03d920 tx=0x7f0bec009880 comp rx=0 tx=0).stop 2024-09-10T07:43:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 -- 172.21.15.46:0/229137503 shutdown_connections 2024-09-10T07:43:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8eaddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8ead8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/229137503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e8d5f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 -- 172.21.15.46:0/229137503 >> 172.21.15.46:0/229137503 conn(0x55c3b8e87120 msgr2=0x55c3b8e94cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 -- 172.21.15.46:0/229137503 shutdown_connections 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.684+0000 7f0c05fe5740 1 -- 172.21.15.46:0/229137503 wait complete. 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 1 -- start start 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 1 Processor -- start 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e99e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8f0f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1b2300 con 0x55c3b8e95d70 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1b1c00 con 0x55c3b8e8ab90 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1aee40 con 0x55c3b8e8b4b0 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.685+0000 7f0c05fe5740 10 client.0 _ll_register_callbacks cb 0x7f0bec0319f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c00a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e99e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c00a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e99e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60154/0 (socket says 172.21.15.46:60154) 2024-09-10T07:43:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c00a00640 1 -- 172.21.15.46:0/2202620443 learned_addr learned my addr 172.21.15.46:0/2202620443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c01400640 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8f0f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c00a00640 1 -- 172.21.15.46:0/2202620443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 msgr2=0x55c3b8e9a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c00a00640 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e9a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c00a00640 1 -- 172.21.15.46:0/2202620443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 msgr2=0x55c3b8f0f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c00a00640 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8f0f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.686+0000 7f0c00a00640 1 -- 172.21.15.46:0/2202620443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b8e9ac80 con 0x55c3b8e8ab90 2024-09-10T07:43:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.687+0000 7f0c00a00640 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e99e90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0bec03dd40 tx=0x7f0bec04a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f7f4b763e6be2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.687+0000 7f0bea000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.687+0000 7f0bea000640 1 -- 172.21.15.46:0/2202620443 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec06b020 con 0x55c3b8e8ab90 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.687+0000 7f0bea000640 1 -- 172.21.15.46:0/2202620443 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0bec003bc0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.687+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2202620443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bec03dd90 con 0x55c3b8e8ab90 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.687+0000 7f0bea000640 1 -- 172.21.15.46:0/2202620443 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0bec076050 con 0x55c3b8e8ab90 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.687+0000 7f0c05fe5740 10 client.26984 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.687+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2202620443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c3b8f10eb0 con 0x55c3b8e8ab90 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0bea000640 1 -- 172.21.15.46:0/2202620443 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec03dd90 con 0x55c3b8e8ab90 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0bea000640 1 -- 172.21.15.46:0/2202620443 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f0bec067570 con 0x55c3b8e8ab90 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0c05fe5740 10 client.26984 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0c05fe5740 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0c05fe5740 20 client.26984 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c3b8f11590 0x55c3b8f139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0c05fe5740 4 client.26984 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:43:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2202620443 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3b8df8e60 con 0x55c3b8f11590 2024-09-10T07:43:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.688+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c3b8f11590 0x55c3b8f139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0bf3e00640 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c3b8f11590 0x55c3b8f139b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8557649843f9e9c server_cookie=f965e8dd3da27a7c in_seq=0 out_seq=0 2024-09-10T07:43:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0bea000640 10 client.26984 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0bea000640 1 -- 172.21.15.46:0/2202620443 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c3b8df8e60 con 0x55c3b8f11590 2024-09-10T07:43:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0bea000640 10 client.26984 handle_command_reply: tid=0 2024-09-10T07:43:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 client.26984 shutdown 2024-09-10T07:43:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2202620443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 msgr2=0x55c3b8e99e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e99e90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0bec03dd40 tx=0x7f0bec04a450 comp rx=0 tx=0).stop 2024-09-10T07:43:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2202620443 shutdown_connections 2024-09-10T07:43:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c3b8f11590 0x55c3b8f139b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3b8e95d70 0x55c3b8f0f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c3b8e8b4b0 0x55c3b8e9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 --2- 172.21.15.46:0/2202620443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3b8e8ab90 0x55c3b8e99e90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0bea000640 0 client.26984 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.689+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2202620443 >> 172.21.15.46:0/2202620443 conn(0x55c3b8e87120 msgr2=0x55c3b8e859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.690+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2202620443 shutdown_connections 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.690+0000 7f0c05fe5740 1 -- 172.21.15.46:0/2202620443 wait complete. 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.690+0000 7f0c05fe5740 20 client.26984 trim_cache size 0 max 16384 2024-09-10T07:43:11.692 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 -- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0be00a2b80 msgr2=0x7f0be00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0be00a2b80 0x7f0be00a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bec000c00 tx=0x7f0bec0058d0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 -- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc109140 msgr2=0x7f0bfc19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc109140 0x7f0bfc19ef00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400eaa0 tx=0x7f0bf400ef60 comp rx=0 tx=0).stop 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 -- 172.21.15.46:0/2598890947 shutdown_connections 2024-09-10T07:43:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0be00a2b80 0x7f0be00a5040 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bfc109140 0x7f0bfc19ef00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bfc1049c0 0x7f0bfc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 --2- 172.21.15.46:0/2598890947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bfc104010 0x7f0bfc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 -- 172.21.15.46:0/2598890947 >> 172.21.15.46:0/2598890947 conn(0x7f0bfc0ffa50 msgr2=0x7f0bfc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 -- 172.21.15.46:0/2598890947 shutdown_connections 2024-09-10T07:43:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:11.691+0000 7f0c03400640 1 -- 172.21.15.46:0/2598890947 wait complete. 2024-09-10T07:43:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:12 smithi181 ceph-mon[35397]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 339 KiB/s wr, 17 op/s 2024-09-10T07:43:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:12 smithi063 ceph-mon[35020]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 339 KiB/s wr, 17 op/s 2024-09-10T07:43:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:12 smithi046 ceph-mon[30705]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 339 KiB/s wr, 17 op/s 2024-09-10T07:43:12.657 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:43:12.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:43:12.657 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 -- 172.21.15.46:0/4117707811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb430102b90 msgr2=0x7fb430102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 --2- 172.21.15.46:0/4117707811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb430102b90 0x7fb430102f70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb420009900 tx=0x7fb42002a480 comp rx=0 tx=0).stop 2024-09-10T07:43:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 -- 172.21.15.46:0/4117707811 shutdown_connections 2024-09-10T07:43:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 --2- 172.21.15.46:0/4117707811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb43010bfc0 0x7fb43010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 --2- 172.21.15.46:0/4117707811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4301034b0 0x7fb43010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 --2- 172.21.15.46:0/4117707811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb430102b90 0x7fb430102f70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 -- 172.21.15.46:0/4117707811 >> 172.21.15.46:0/4117707811 conn(0x7fb4300fc610 msgr2=0x7fb4300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 -- 172.21.15.46:0/4117707811 shutdown_connections 2024-09-10T07:43:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.286+0000 7fb42fe00640 1 -- 172.21.15.46:0/4117707811 wait complete. 2024-09-10T07:43:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.287+0000 7fb42fe00640 1 Processor -- start 2024-09-10T07:43:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42fe00640 1 -- start start 2024-09-10T07:43:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430102b90 0x7fb43019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4301034b0 0x7fb43019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb43010bfc0 0x7fb43019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb430111e10 con 0x7fb4301034b0 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb430111c90 con 0x7fb43010bfc0 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb430111f90 con 0x7fb430102b90 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430102b90 0x7fb43019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb426000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4301034b0 0x7fb43019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb43010bfc0 0x7fb43019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430102b90 0x7fb43019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:45136/0 (socket says 172.21.15.46:45136) 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.288+0000 7fb42ea00640 1 -- 172.21.15.46:0/886276875 learned_addr learned my addr 172.21.15.46:0/886276875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb42f400640 1 -- 172.21.15.46:0/886276875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430102b90 msgr2=0x7fb43019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb42f400640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430102b90 0x7fb43019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb42f400640 1 -- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4301034b0 msgr2=0x7fb43019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb42f400640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4301034b0 0x7fb43019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb42f400640 1 -- 172.21.15.46:0/886276875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43019f480 con 0x7fb43010bfc0 2024-09-10T07:43:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb42f400640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb43010bfc0 0x7fb43019eda0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb428009fd0 tx=0x7fb42800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb427e00640 1 -- 172.21.15.46:0/886276875 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42800ce20 con 0x7fb43010bfc0 2024-09-10T07:43:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb427e00640 1 -- 172.21.15.46:0/886276875 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb428005020 con 0x7fb43010bfc0 2024-09-10T07:43:13.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.289+0000 7fb427e00640 1 -- 172.21.15.46:0/886276875 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428010720 con 0x7fb43010bfc0 2024-09-10T07:43:13.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.290+0000 7fb42fe00640 1 -- 172.21.15.46:0/886276875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43019f770 con 0x7fb43010bfc0 2024-09-10T07:43:13.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.290+0000 7fb42fe00640 1 -- 172.21.15.46:0/886276875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4301a7050 con 0x7fb43010bfc0 2024-09-10T07:43:13.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb4383cb740 1 Processor -- start 2024-09-10T07:43:13.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb427e00640 1 -- 172.21.15.46:0/886276875 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb428005540 con 0x7fb43010bfc0 2024-09-10T07:43:13.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb4383cb740 1 -- start start 2024-09-10T07:43:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325520 0x5652f4325900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f43301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4332af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb4383cb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652f428b650 con 0x5652f4325520 2024-09-10T07:43:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb4383cb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5652f4232f10 con 0x5652f4330700 2024-09-10T07:43:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.293+0000 7fb4383cb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5652f4287610 con 0x5652f4325e40 2024-09-10T07:43:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f43301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f43301c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:45140/0 (socket says 172.21.15.46:45140) 2024-09-10T07:43:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb42ea00640 1 -- 172.21.15.46:0/1044812600 learned_addr learned my addr 172.21.15.46:0/1044812600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb426000640 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325520 0x5652f4325900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb426000640 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4332af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb42ea00640 1 -- 172.21.15.46:0/1044812600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 msgr2=0x5652f4332af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:13.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb42ea00640 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4332af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb42ea00640 1 -- 172.21.15.46:0/1044812600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325520 msgr2=0x5652f4325900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:13.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb42ea00640 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325520 0x5652f4325900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.294+0000 7fb42ea00640 1 -- 172.21.15.46:0/1044812600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f3f9a580 con 0x5652f4325e40 2024-09-10T07:43:13.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.295+0000 7fb42ea00640 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f43301c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb420005e80 tx=0x7fb420005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf33f0a9794a298 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:13.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.295+0000 7fb427e00640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4140a2b80 0x7fb4140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.295+0000 7fb427e00640 1 -- 172.21.15.46:0/886276875 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb4280cfaa0 con 0x7fb43010bfc0 2024-09-10T07:43:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.295+0000 7fb41fe00640 1 -- 172.21.15.46:0/1044812600 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420035040 con 0x5652f4325e40 2024-09-10T07:43:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.295+0000 7fb426000640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4140a2b80 0x7fb4140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.295+0000 7fb41fe00640 1 -- 172.21.15.46:0/1044812600 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4200044b0 con 0x5652f4325e40 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.295+0000 7fb41fe00640 1 -- 172.21.15.46:0/1044812600 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420003850 con 0x5652f4325e40 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.296+0000 7fb4383cb740 1 -- 172.21.15.46:0/1044812600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 msgr2=0x5652f43301c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.296+0000 7fb426000640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4140a2b80 0x7fb4140a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5652f4322e30 tx=0x7fb418014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.296+0000 7fb4383cb740 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f43301c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb420005e80 tx=0x7fb420005c00 comp rx=0 tx=0).stop 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.296+0000 7fb4383cb740 1 -- 172.21.15.46:0/1044812600 shutdown_connections 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.296+0000 7fb4383cb740 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4332af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.296+0000 7fb4383cb740 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f43301c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.296+0000 7fb4383cb740 1 --2- 172.21.15.46:0/1044812600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325520 0x5652f4325900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.296+0000 7fb4383cb740 1 -- 172.21.15.46:0/1044812600 >> 172.21.15.46:0/1044812600 conn(0x5652f4321ab0 msgr2=0x5652f4321e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.297+0000 7fb4383cb740 1 -- 172.21.15.46:0/1044812600 shutdown_connections 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.297+0000 7fb4383cb740 1 -- 172.21.15.46:0/1044812600 wait complete. 2024-09-10T07:43:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.297+0000 7fb4383cb740 1 -- start start 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.297+0000 7fb4383cb740 1 Processor -- start 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f432d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f432db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4330700 0x5652f432e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb4383cb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652f43332d0 con 0x5652f4325e40 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb4383cb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5652f4246350 con 0x5652f4325520 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb4383cb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5652f4333450 con 0x5652f4330700 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb4383cb740 10 client.0 _ll_register_callbacks cb 0x5652f4269290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb426000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f432db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f432d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb42f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4330700 0x5652f432e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb426000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f432db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40766/0 (socket says 172.21.15.46:40766) 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.298+0000 7fb426000640 1 -- 172.21.15.46:0/3651532129 learned_addr learned my addr 172.21.15.46:0/3651532129 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb426000640 1 -- 172.21.15.46:0/3651532129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4330700 msgr2=0x5652f432e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb426000640 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4330700 0x5652f432e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb426000640 1 -- 172.21.15.46:0/3651532129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 msgr2=0x5652f432d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb426000640 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f432d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb426000640 1 -- 172.21.15.46:0/3651532129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f4064960 con 0x5652f4325e40 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb42ea00640 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f432d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb426000640 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f432db20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb418003f60 tx=0x7fb41800b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec2f5d6482886c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb41cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb41cc00640 1 -- 172.21.15.46:0/3651532129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418005220 con 0x5652f4325e40 2024-09-10T07:43:13.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb41cc00640 1 -- 172.21.15.46:0/3651532129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb4180053c0 con 0x5652f4325e40 2024-09-10T07:43:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.299+0000 7fb4383cb740 1 -- 172.21.15.46:0/3651532129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4200044b0 con 0x5652f4325e40 2024-09-10T07:43:13.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.300+0000 7fb41cc00640 1 -- 172.21.15.46:0/3651532129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb418032050 con 0x5652f4325e40 2024-09-10T07:43:13.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.300+0000 7fb4383cb740 10 client.16587 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:13.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.300+0000 7fb4383cb740 1 -- 172.21.15.46:0/3651532129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5652f433ef60 con 0x5652f4325e40 2024-09-10T07:43:13.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.300+0000 7fb41cc00640 1 -- 172.21.15.46:0/3651532129 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4200044b0 con 0x5652f4325e40 2024-09-10T07:43:13.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.300+0000 7fb41cc00640 1 -- 172.21.15.46:0/3651532129 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7fb4180217f0 con 0x5652f4325e40 2024-09-10T07:43:13.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.301+0000 7fb4383cb740 10 client.16587 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:13.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.301+0000 7fb4383cb740 10 client.16587 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:43:13.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.301+0000 7fb4383cb740 20 client.16587 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.301+0000 7fb4383cb740 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5652f43404e0 0x5652f4342910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.301+0000 7fb4383cb740 4 client.16587 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.301+0000 7fb4383cb740 1 -- 172.21.15.46:0/3651532129 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652f4298bc0 con 0x5652f43404e0 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.301+0000 7fb42ea00640 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5652f43404e0 0x5652f4342910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.302+0000 7fb42ea00640 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5652f43404e0 0x5652f4342910 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb768c2bfa0269e9 server_cookie=b6923b8c1ac0a2ee in_seq=0 out_seq=0 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.302+0000 7fb41cc00640 10 client.16587 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.302+0000 7fb41cc00640 1 -- 172.21.15.46:0/3651532129 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652f4298bc0 con 0x5652f43404e0 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.302+0000 7fb41cc00640 10 client.16587 handle_command_reply: tid=0 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.302+0000 7fb4383cb740 1 client.16587 shutdown 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 -- 172.21.15.46:0/3651532129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 msgr2=0x5652f432db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f432db20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb418003f60 tx=0x7fb41800b2e0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 -- 172.21.15.46:0/3651532129 shutdown_connections 2024-09-10T07:43:13.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5652f43404e0 0x5652f4342910 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4330700 0x5652f432e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb41cc00640 0 client.16587 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f432db20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 --2- 172.21.15.46:0/3651532129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f432d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 -- 172.21.15.46:0/3651532129 >> 172.21.15.46:0/3651532129 conn(0x5652f4321ab0 msgr2=0x5652f4320cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 -- 172.21.15.46:0/3651532129 shutdown_connections 2024-09-10T07:43:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 1 -- 172.21.15.46:0/3651532129 wait complete. 2024-09-10T07:43:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.303+0000 7fb4383cb740 20 client.16587 trim_cache size 0 max 16384 2024-09-10T07:43:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb4383cb740 1 Processor -- start 2024-09-10T07:43:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb4383cb740 1 -- start start 2024-09-10T07:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325520 0x5652f432c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f4348220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4348760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb4383cb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301ac8a0 con 0x5652f4325e40 2024-09-10T07:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb4383cb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301ae470 con 0x5652f4330700 2024-09-10T07:43:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb4383cb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301acf70 con 0x5652f4325520 2024-09-10T07:43:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb426000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f4348220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb426000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f4348220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40780/0 (socket says 172.21.15.46:40780) 2024-09-10T07:43:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.330+0000 7fb426000640 1 -- 172.21.15.46:0/1227812329 learned_addr learned my addr 172.21.15.46:0/1227812329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb42ea00640 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325520 0x5652f432c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb42f400640 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4348760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb426000640 1 -- 172.21.15.46:0/1227812329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325520 msgr2=0x5652f432c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb426000640 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325520 0x5652f432c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb426000640 1 -- 172.21.15.46:0/1227812329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 msgr2=0x5652f4348760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb426000640 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4348760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb426000640 1 -- 172.21.15.46:0/1227812329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f431d480 con 0x5652f4325e40 2024-09-10T07:43:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb42f400640 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4348760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.331+0000 7fb426000640 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f4348220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb41801a040 tx=0x7fb418003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf3ba32b6bc5b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.332+0000 7fb41f400640 1 -- 172.21.15.46:0/1227812329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418021830 con 0x5652f4325e40 2024-09-10T07:43:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.332+0000 7fb41f400640 1 -- 172.21.15.46:0/1227812329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb418026070 con 0x5652f4325e40 2024-09-10T07:43:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.332+0000 7fb41f400640 1 -- 172.21.15.46:0/1227812329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418005020 con 0x5652f4325e40 2024-09-10T07:43:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 -- 172.21.15.46:0/1227812329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 msgr2=0x5652f4348220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f4348220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb41801a040 tx=0x7fb418003b30 comp rx=0 tx=0).stop 2024-09-10T07:43:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 -- 172.21.15.46:0/1227812329 shutdown_connections 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4330700 0x5652f4348760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4325e40 0x5652f4348220 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 --2- 172.21.15.46:0/1227812329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325520 0x5652f432c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 -- 172.21.15.46:0/1227812329 >> 172.21.15.46:0/1227812329 conn(0x5652f4321ab0 msgr2=0x5652f432bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 -- 172.21.15.46:0/1227812329 shutdown_connections 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.333+0000 7fb4383cb740 1 -- 172.21.15.46:0/1227812329 wait complete. 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 1 -- start start 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 1 Processor -- start 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f4334800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f4334d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4330700 0x5652f43aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301add40 con 0x5652f4330700 2024-09-10T07:43:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301ad640 con 0x5652f4325520 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301aa8a0 con 0x5652f4325e40 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb4383cb740 10 client.0 _ll_register_callbacks cb 0x7fb418005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.334+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f4334800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f4334800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:60216/0 (socket says 172.21.15.46:60216) 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42ea00640 1 -- 172.21.15.46:0/2490741485 learned_addr learned my addr 172.21.15.46:0/2490741485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42f400640 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4330700 0x5652f43aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42ea00640 1 -- 172.21.15.46:0/2490741485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 msgr2=0x5652f4334d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42ea00640 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f4334d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42ea00640 1 -- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4330700 msgr2=0x5652f43aa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42ea00640 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4330700 0x5652f43aa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42ea00640 1 -- 172.21.15.46:0/2490741485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f43bb740 con 0x5652f4325520 2024-09-10T07:43:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.335+0000 7fb42f400640 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4330700 0x5652f43aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.336+0000 7fb42ea00640 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f4334800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb420031550 tx=0x7fb420004c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d67ce42c68e026dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.336+0000 7fb41cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.336+0000 7fb41cc00640 1 -- 172.21.15.46:0/2490741485 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420035040 con 0x5652f4325520 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.336+0000 7fb41cc00640 1 -- 172.21.15.46:0/2490741485 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb420004930 con 0x5652f4325520 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.336+0000 7fb4383cb740 1 -- 172.21.15.46:0/2490741485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb418026070 con 0x5652f4325520 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb41cc00640 1 -- 172.21.15.46:0/2490741485 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb420060050 con 0x5652f4325520 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb4383cb740 10 client.27002 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb4383cb740 1 -- 172.21.15.46:0/2490741485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5652f43bc8f0 con 0x5652f4325520 2024-09-10T07:43:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb41cc00640 1 -- 172.21.15.46:0/2490741485 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4200044c0 con 0x5652f4325520 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb41cc00640 1 -- 172.21.15.46:0/2490741485 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7fb420031ae0 con 0x5652f4325520 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb4383cb740 10 client.27002 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb4383cb740 10 client.27002 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb4383cb740 20 client.27002 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb4383cb740 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5652f43bccf0 0x5652f43bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb4383cb740 4 client.27002 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6a5933b6", "release": true, "await": true}] 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.337+0000 7fb4383cb740 1 -- 172.21.15.46:0/2490741485 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6a5933b6", "release": true, "await": true}) -- 0x7fb418026070 con 0x5652f43bccf0 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.338+0000 7fb426000640 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5652f43bccf0 0x5652f43bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.338+0000 7fb426000640 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5652f43bccf0 0x5652f43bf0e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87b3c6f439005b0 server_cookie=9fdce5533f367429 in_seq=0 out_seq=0 2024-09-10T07:43:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.338+0000 7fb41cc00640 10 client.27002 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:13.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.407+0000 7fb41cc00640 1 -- 172.21.15.46:0/2490741485 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fb418026070 con 0x5652f43bccf0 2024-09-10T07:43:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.409+0000 7fb41cc00640 10 client.27002 handle_command_reply: tid=0 2024-09-10T07:43:13.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.409+0000 7fb4383cb740 1 client.27002 shutdown 2024-09-10T07:43:13.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb4383cb740 1 -- 172.21.15.46:0/2490741485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 msgr2=0x5652f4334800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb4383cb740 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f4334800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb420031550 tx=0x7fb420004c40 comp rx=0 tx=0).stop 2024-09-10T07:43:13.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb4383cb740 1 -- 172.21.15.46:0/2490741485 shutdown_connections 2024-09-10T07:43:13.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb4383cb740 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5652f43bccf0 0x5652f43bf0e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb4383cb740 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652f4330700 0x5652f43aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb4383cb740 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652f4325e40 0x5652f4334d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb4383cb740 1 --2- 172.21.15.46:0/2490741485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652f4325520 0x5652f4334800 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb41cc00640 0 client.27002 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:13.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.413+0000 7fb4383cb740 1 -- 172.21.15.46:0/2490741485 >> 172.21.15.46:0/2490741485 conn(0x5652f4321ab0 msgr2=0x5652f432c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:13.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.414+0000 7fb4383cb740 1 -- 172.21.15.46:0/2490741485 shutdown_connections 2024-09-10T07:43:13.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.414+0000 7fb4383cb740 1 -- 172.21.15.46:0/2490741485 wait complete. 2024-09-10T07:43:13.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.414+0000 7fb4383cb740 20 client.27002 trim_cache size 0 max 16384 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 43, 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 20, 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "6a5933b6": { 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 20, 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:43:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 140.1, 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:13.418 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:43:13.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 -- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4140a2b80 msgr2=0x7fb4140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4140a2b80 0x7fb4140a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5652f4322e30 tx=0x7fb418014040 comp rx=0 tx=0).stop 2024-09-10T07:43:13.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 -- 172.21.15.46:0/886276875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb43010bfc0 msgr2=0x7fb43019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:13.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb43010bfc0 0x7fb43019eda0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb428009fd0 tx=0x7fb42800ef10 comp rx=0 tx=0).stop 2024-09-10T07:43:13.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 -- 172.21.15.46:0/886276875 shutdown_connections 2024-09-10T07:43:13.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb43010bfc0 0x7fb43019eda0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4140a2b80 0x7fb4140a5040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4301034b0 0x7fb43019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 --2- 172.21.15.46:0/886276875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430102b90 0x7fb43019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:13.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 -- 172.21.15.46:0/886276875 >> 172.21.15.46:0/886276875 conn(0x7fb4300fc610 msgr2=0x7fb4300fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:13.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.416+0000 7fb42fe00640 1 -- 172.21.15.46:0/886276875 shutdown_connections 2024-09-10T07:43:13.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:13.418+0000 7fb42fe00640 1 -- 172.21.15.46:0/886276875 wait complete. 2024-09-10T07:43:14.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:14 smithi046 ceph-mon[30705]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 341 KiB/s wr, 17 op/s 2024-09-10T07:43:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:14 smithi181 ceph-mon[35397]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 341 KiB/s wr, 17 op/s 2024-09-10T07:43:14.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:14 smithi063 ceph-mon[35020]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 341 KiB/s wr, 17 op/s 2024-09-10T07:43:14.430 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6a5933b6', total seconds elapsed: 25.7 2024-09-10T07:43:14.430 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T07:43:14.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:43:15.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.175+0000 7f92f2a00640 1 -- 172.21.15.46:0/2342989844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 msgr2=0x7f92ec06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:15.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.175+0000 7f92f2a00640 1 --2- 172.21.15.46:0/2342989844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 0x7f92ec06ee70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009900 tx=0x7f92d802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.176+0000 7f92f2a00640 1 -- 172.21.15.46:0/2342989844 shutdown_connections 2024-09-10T07:43:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.176+0000 7f92f2a00640 1 --2- 172.21.15.46:0/2342989844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92ec11b140 0x7f92ec11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.176+0000 7f92f2a00640 1 --2- 172.21.15.46:0/2342989844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 0x7f92ec06ee70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.176+0000 7f92f2a00640 1 --2- 172.21.15.46:0/2342989844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec110190 0x7f92ec06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.176+0000 7f92f2a00640 1 -- 172.21.15.46:0/2342989844 >> 172.21.15.46:0/2342989844 conn(0x7f92ec073180 msgr2=0x7f92ec073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.178+0000 7f92f2a00640 1 -- 172.21.15.46:0/2342989844 shutdown_connections 2024-09-10T07:43:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.178+0000 7f92f2a00640 1 -- 172.21.15.46:0/2342989844 wait complete. 2024-09-10T07:43:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.180+0000 7f92f2a00640 1 Processor -- start 2024-09-10T07:43:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.181+0000 7f92f2a00640 1 -- start start 2024-09-10T07:43:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.181+0000 7f92f2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 0x7f92ec1a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.181+0000 7f92f2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92ec110190 0x7f92ec1a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.181+0000 7f92f2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec11b140 0x7f92ec1aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.182+0000 7f92f2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec120c40 con 0x7f92ec11b140 2024-09-10T07:43:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.182+0000 7f92f2a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec120ac0 con 0x7f92ec110190 2024-09-10T07:43:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.182+0000 7f92f2a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec120dc0 con 0x7f92ec06ea30 2024-09-10T07:43:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.182+0000 7f92f0c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92ec110190 0x7f92ec1a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.182+0000 7f92f1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 0x7f92ec1a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.182+0000 7f92f1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 0x7f92ec1a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:45184/0 (socket says 172.21.15.46:45184) 2024-09-10T07:43:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.182+0000 7f92f1600640 1 -- 172.21.15.46:0/3180725586 learned_addr learned my addr 172.21.15.46:0/3180725586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.182+0000 7f92f2000640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec11b140 0x7f92ec1aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.183+0000 7f92f0c00640 1 -- 172.21.15.46:0/3180725586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 msgr2=0x7f92ec1a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.183+0000 7f92f0c00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 0x7f92ec1a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.183+0000 7f92f0c00640 1 -- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec11b140 msgr2=0x7f92ec1aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.183+0000 7f92f0c00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec11b140 0x7f92ec1aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.183+0000 7f92f0c00640 1 -- 172.21.15.46:0/3180725586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec1ac110 con 0x7f92ec110190 2024-09-10T07:43:15.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.183+0000 7f92f0c00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92ec110190 0x7f92ec1a76a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92d802a9e0 tx=0x7f92d8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:15.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.183+0000 7f92ea000640 1 -- 172.21.15.46:0/3180725586 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d803f070 con 0x7f92ec110190 2024-09-10T07:43:15.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.183+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec1ac3a0 con 0x7f92ec110190 2024-09-10T07:43:15.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.184+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec1b3ce0 con 0x7f92ec110190 2024-09-10T07:43:15.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.185+0000 7f92ea000640 1 -- 172.21.15.46:0/3180725586 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d8031c90 con 0x7f92ec110190 2024-09-10T07:43:15.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.185+0000 7f92ea000640 1 -- 172.21.15.46:0/3180725586 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8035030 con 0x7f92ec110190 2024-09-10T07:43:15.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.188+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92cc005290 con 0x7f92ec110190 2024-09-10T07:43:15.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.191+0000 7f92ea000640 1 -- 172.21.15.46:0/3180725586 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f92d8042070 con 0x7f92ec110190 2024-09-10T07:43:15.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.194+0000 7f92ea000640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92d40a2cc0 0x7f92d40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.194+0000 7f92ea000640 1 -- 172.21.15.46:0/3180725586 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f92d80ef990 con 0x7f92ec110190 2024-09-10T07:43:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.195+0000 7f92f1600640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92d40a2cc0 0x7f92d40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:15.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.199+0000 7f92f1600640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92d40a2cc0 0x7f92d40a5180 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009950 tx=0x7f92e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.201+0000 7f92ea000640 1 -- 172.21.15.46:0/3180725586 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92d8049030 con 0x7f92ec110190 2024-09-10T07:43:15.286 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 355.0 s, 344.5 tps, lat 1256.252 ms stddev 4592.853 2024-09-10T07:43:15.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.626+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92cc005580 con 0x7f92ec110190 2024-09-10T07:43:15.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.627+0000 7f92ea000640 1 -- 172.21.15.46:0/3180725586 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11976 (secure 0 0 0) 0x7f92d80bc510 con 0x7f92ec110190 2024-09-10T07:43:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:43:15.630 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"btime":"2024-09-10T07:42:38:910267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:42:38.047718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:43:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 43 2024-09-10T07:43:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.631+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92d40a2cc0 msgr2=0x7f92d40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.631+0000 7f92f2a00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92d40a2cc0 0x7f92d40a5180 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009950 tx=0x7f92e0008040 comp rx=0 tx=0).stop 2024-09-10T07:43:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.631+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92ec110190 msgr2=0x7f92ec1a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.631+0000 7f92f2a00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92ec110190 0x7f92ec1a76a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92d802a9e0 tx=0x7f92d8003b90 comp rx=0 tx=0).stop 2024-09-10T07:43:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.632+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 shutdown_connections 2024-09-10T07:43:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.632+0000 7f92f2a00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f92d40a2cc0 0x7f92d40a5180 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.632+0000 7f92f2a00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec11b140 0x7f92ec1aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.632+0000 7f92f2a00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92ec110190 0x7f92ec1a76a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.632+0000 7f92f2a00640 1 --2- 172.21.15.46:0/3180725586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92ec06ea30 0x7f92ec1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.632+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 >> 172.21.15.46:0/3180725586 conn(0x7f92ec073180 msgr2=0x7f92ec119950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.633+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 shutdown_connections 2024-09-10T07:43:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:15.633+0000 7f92f2a00640 1 -- 172.21.15.46:0/3180725586 wait complete. 2024-09-10T07:43:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.241+0000 7ff540600640 1 -- 172.21.15.46:0/4098559351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538103640 msgr2=0x7ff53810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.241+0000 7ff540600640 1 --2- 172.21.15.46:0/4098559351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538103640 0x7ff53810bb30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009900 tx=0x7ff52c02a480 comp rx=0 tx=0).stop 2024-09-10T07:43:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.242+0000 7ff540600640 1 -- 172.21.15.46:0/4098559351 shutdown_connections 2024-09-10T07:43:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.242+0000 7ff540600640 1 --2- 172.21.15.46:0/4098559351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff53810c150 0x7ff53810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.242+0000 7ff540600640 1 --2- 172.21.15.46:0/4098559351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538103640 0x7ff53810bb30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.242+0000 7ff540600640 1 --2- 172.21.15.46:0/4098559351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538102d20 0x7ff538103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.242+0000 7ff540600640 1 -- 172.21.15.46:0/4098559351 >> 172.21.15.46:0/4098559351 conn(0x7ff5380fc780 msgr2=0x7ff5380feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.242+0000 7ff540600640 1 -- 172.21.15.46:0/4098559351 shutdown_connections 2024-09-10T07:43:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.242+0000 7ff540600640 1 -- 172.21.15.46:0/4098559351 wait complete. 2024-09-10T07:43:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.243+0000 7ff540600640 1 Processor -- start 2024-09-10T07:43:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.243+0000 7ff540600640 1 -- start start 2024-09-10T07:43:16.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.244+0000 7ff540600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538102d20 0x7ff5381984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:16.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.244+0000 7ff540600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538103640 0x7ff538198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.244+0000 7ff540600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff53810c150 0x7ff53819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.244+0000 7ff540600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111fe0 con 0x7ff53810c150 2024-09-10T07:43:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.244+0000 7ff540600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111e60 con 0x7ff538102d20 2024-09-10T07:43:16.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.244+0000 7ff540600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff538112160 con 0x7ff538103640 2024-09-10T07:43:16.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.244+0000 7ff53dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538102d20 0x7ff5381984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:16.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538102d20 0x7ff5381984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55922/0 (socket says 172.21.15.46:55922) 2024-09-10T07:43:16.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53dc00640 1 -- 172.21.15.46:0/4134765001 learned_addr learned my addr 172.21.15.46:0/4134765001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:16.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53e600640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff53810c150 0x7ff53819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53dc00640 1 -- 172.21.15.46:0/4134765001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538103640 msgr2=0x7ff538198a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53dc00640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538103640 0x7ff538198a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53dc00640 1 -- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff53810c150 msgr2=0x7ff53819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53dc00640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff53810c150 0x7ff53819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53dc00640 1 -- 172.21.15.46:0/4134765001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53819d4a0 con 0x7ff538102d20 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.245+0000 7ff53e600640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff53810c150 0x7ff53819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.246+0000 7ff53dc00640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538102d20 0x7ff5381984f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c890 tx=0x7ff52400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.246+0000 7ff536000640 1 -- 172.21.15.46:0/4134765001 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524004220 con 0x7ff538102d20 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.246+0000 7ff536000640 1 -- 172.21.15.46:0/4134765001 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5240043c0 con 0x7ff538102d20 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.246+0000 7ff536000640 1 -- 172.21.15.46:0/4134765001 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524004220 con 0x7ff538102d20 2024-09-10T07:43:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.246+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53819d790 con 0x7ff538102d20 2024-09-10T07:43:16.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.246+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5381008b0 con 0x7ff538102d20 2024-09-10T07:43:16.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.248+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff53806b070 con 0x7ff538102d20 2024-09-10T07:43:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.253+0000 7ff536000640 1 -- 172.21.15.46:0/4134765001 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff5240079b0 con 0x7ff538102d20 2024-09-10T07:43:16.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.257+0000 7ff536000640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5200b3e10 0x7ff5200b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:16.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.258+0000 7ff536000640 1 -- 172.21.15.46:0/4134765001 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff5240cece0 con 0x7ff538102d20 2024-09-10T07:43:16.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.261+0000 7ff53d200640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5200b3e10 0x7ff5200b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:16.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.263+0000 7ff536000640 1 -- 172.21.15.46:0/4134765001 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff524014030 con 0x7ff538102d20 2024-09-10T07:43:16.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.270+0000 7ff53d200640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5200b3e10 0x7ff5200b62d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff52c02a9a0 tx=0x7ff52c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:16.377 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:16 smithi046 ceph-mon[30705]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 369 KiB/s wr, 21 op/s 2024-09-10T07:43:16.377 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:16 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3180725586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:16 smithi181 ceph-mon[35397]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 369 KiB/s wr, 21 op/s 2024-09-10T07:43:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:16 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3180725586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:16.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:16 smithi063 ceph-mon[35020]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 369 KiB/s wr, 21 op/s 2024-09-10T07:43:16.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:16 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3180725586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.646+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff538103100 con 0x7ff538102d20 2024-09-10T07:43:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.648+0000 7ff536000640 1 -- 172.21.15.46:0/4134765001 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11976 (secure 0 0 0) 0x7ff52409b8c0 con 0x7ff538102d20 2024-09-10T07:43:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:43:16.650 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"btime":"2024-09-10T07:42:38:910267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:42:38.047718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:43:16.650 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 43 2024-09-10T07:43:16.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.654+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5200b3e10 msgr2=0x7ff5200b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.654+0000 7ff540600640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5200b3e10 0x7ff5200b62d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff52c02a9a0 tx=0x7ff52c005fd0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.654+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538102d20 msgr2=0x7ff5381984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.654+0000 7ff540600640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538102d20 0x7ff5381984f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c890 tx=0x7ff52400cd50 comp rx=0 tx=0).stop 2024-09-10T07:43:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.654+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 shutdown_connections 2024-09-10T07:43:16.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.655+0000 7ff540600640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff5200b3e10 0x7ff5200b62d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.655+0000 7ff540600640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff53810c150 0x7ff53819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.655+0000 7ff540600640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538103640 0x7ff538198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.655+0000 7ff540600640 1 --2- 172.21.15.46:0/4134765001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff538102d20 0x7ff5381984f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.655+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 >> 172.21.15.46:0/4134765001 conn(0x7ff5380fc780 msgr2=0x7ff5380fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:16.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.656+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 shutdown_connections 2024-09-10T07:43:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:16.656+0000 7ff540600640 1 -- 172.21.15.46:0/4134765001 wait complete. 2024-09-10T07:43:16.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:43:17.223 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:17 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/4134765001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:17.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:17 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/4134765001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:17.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:17 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/4134765001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:17.691 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T07:43:17.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T07:43:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:18.270 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:18 smithi046 ceph-mon[30705]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.3 MiB/s wr, 102 op/s 2024-09-10T07:43:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:18 smithi181 ceph-mon[35397]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.3 MiB/s wr, 102 op/s 2024-09-10T07:43:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:18 smithi063 ceph-mon[35020]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.3 MiB/s wr, 102 op/s 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 -- 172.21.15.46:0/1509403947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00481047e0 msgr2=0x7f0048108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 --2- 172.21.15.46:0/1509403947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00481047e0 0x7f0048108830 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0034009900 tx=0x7f003402a410 comp rx=0 tx=0).stop 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 -- 172.21.15.46:0/1509403947 shutdown_connections 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 --2- 172.21.15.46:0/1509403947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0048108f60 0x7f0048110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 --2- 172.21.15.46:0/1509403947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00481047e0 0x7f0048108830 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 --2- 172.21.15.46:0/1509403947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0048103e30 0x7f0048104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 -- 172.21.15.46:0/1509403947 >> 172.21.15.46:0/1509403947 conn(0x7f00480ff910 msgr2=0x7f0048101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 -- 172.21.15.46:0/1509403947 shutdown_connections 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.581+0000 7f004ca00640 1 -- 172.21.15.46:0/1509403947 wait complete. 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.582+0000 7f004ca00640 1 Processor -- start 2024-09-10T07:43:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.583+0000 7f004ca00640 1 -- start start 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.583+0000 7f004ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0048103e30 0x7f004819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.583+0000 7f004ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00481047e0 0x7f004819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.583+0000 7f004ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0048108f60 0x7f004819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.583+0000 7f004ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0048113fc0 con 0x7f0048103e30 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.583+0000 7f004ca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0048113e40 con 0x7f00481047e0 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.583+0000 7f004ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0048114140 con 0x7f0048108f60 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0047200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0048108f60 0x7f004819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0046800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0048103e30 0x7f004819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0045e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00481047e0 0x7f004819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0045e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00481047e0 0x7f004819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55950/0 (socket says 172.21.15.46:55950) 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0047200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0048108f60 0x7f004819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56880/0 (socket says 172.21.15.46:56880) 2024-09-10T07:43:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0045e00640 1 -- 172.21.15.46:0/824056729 learned_addr learned my addr 172.21.15.46:0/824056729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0047200640 1 -- 172.21.15.46:0/824056729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00481047e0 msgr2=0x7f004819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0047200640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00481047e0 0x7f004819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0047200640 1 -- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0048103e30 msgr2=0x7f004819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0047200640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0048103e30 0x7f004819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.584+0000 7f0047200640 1 -- 172.21.15.46:0/824056729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004819f450 con 0x7f0048108f60 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.585+0000 7f0047200640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0048108f60 0x7f004819ed70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0040009fd0 tx=0x7f004000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.585+0000 7f003f400640 1 -- 172.21.15.46:0/824056729 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000ce20 con 0x7f0048108f60 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.585+0000 7f004ca00640 1 -- 172.21.15.46:0/824056729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004819f740 con 0x7f0048108f60 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.585+0000 7f004ca00640 1 -- 172.21.15.46:0/824056729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00481a7020 con 0x7f0048108f60 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.585+0000 7f003f400640 1 -- 172.21.15.46:0/824056729 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0040010430 con 0x7f0048108f60 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.585+0000 7f003f400640 1 -- 172.21.15.46:0/824056729 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000f620 con 0x7f0048108f60 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f0050059740 1 Processor -- start 2024-09-10T07:43:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f003f400640 1 -- 172.21.15.46:0/824056729 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0040003970 con 0x7f0048108f60 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f0050059740 1 -- start start 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f0050059740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f0050059740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f0050059740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f0050059740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5638f532b3e0 con 0x5638f53d0020 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f0050059740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5638f52d2d80 con 0x5638f53c5760 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.589+0000 7f0050059740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5638f53273a0 con 0x5638f53c4e40 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55958/0 (socket says 172.21.15.46:55958) 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 -- 172.21.15.46:0/3679578468 learned_addr learned my addr 172.21.15.46:0/3679578468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0045e00640 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0045e00640 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 -- 172.21.15.46:0/3679578468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 msgr2=0x5638f53c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 -- 172.21.15.46:0/3679578468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 msgr2=0x5638f53d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 -- 172.21.15.46:0/3679578468 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f52f1ae0 con 0x5638f53c5760 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0045e00640 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f0046800640 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53cfae0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0038009cf0 tx=0x7f0038011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22067e74a896d160 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.590+0000 7f003f400640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f002c0a2b80 0x7f002c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.591+0000 7f003f400640 1 -- 172.21.15.46:0/824056729 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f00400cf520 con 0x7f0048108f60 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.591+0000 7f0033400640 1 -- 172.21.15.46:0/3679578468 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038004230 con 0x5638f53c5760 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.591+0000 7f0033400640 1 -- 172.21.15.46:0/3679578468 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00380043d0 con 0x5638f53c5760 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.591+0000 7f0045e00640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f002c0a2b80 0x7f002c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.591+0000 7f0033400640 1 -- 172.21.15.46:0/3679578468 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038019960 con 0x5638f53c5760 2024-09-10T07:43:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.591+0000 7f0050059740 1 -- 172.21.15.46:0/3679578468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 msgr2=0x5638f53cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.591+0000 7f0050059740 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53cfae0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0038009cf0 tx=0x7f0038011810 comp rx=0 tx=0).stop 2024-09-10T07:43:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 -- 172.21.15.46:0/3679578468 shutdown_connections 2024-09-10T07:43:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0045e00640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f002c0a2b80 0x7f002c0a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0034031550 tx=0x7f003402a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53cfae0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 --2- 172.21.15.46:0/3679578468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 -- 172.21.15.46:0/3679578468 >> 172.21.15.46:0/3679578468 conn(0x5638f53c13d0 msgr2=0x5638f53c17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 -- 172.21.15.46:0/3679578468 shutdown_connections 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 -- 172.21.15.46:0/3679578468 wait complete. 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 -- start start 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.592+0000 7f0050059740 1 Processor -- start 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0050059740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c4e40 0x5638f53cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0050059740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c5760 0x5638f53ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0050059740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0050059740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5638f53d2c90 con 0x5638f53d0020 2024-09-10T07:43:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0050059740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5638f52e60f0 con 0x5638f53c4e40 2024-09-10T07:43:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0050059740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5638f53d2e10 con 0x5638f53c5760 2024-09-10T07:43:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0050059740 10 client.0 _ll_register_callbacks cb 0x5638f52cdbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0045e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c5760 0x5638f53ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0045e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c5760 0x5638f53ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56902/0 (socket says 172.21.15.46:56902) 2024-09-10T07:43:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0045e00640 1 -- 172.21.15.46:0/2957562358 learned_addr learned my addr 172.21.15.46:0/2957562358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.593+0000 7f0046800640 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c4e40 0x5638f53cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0047200640 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0045e00640 1 -- 172.21.15.46:0/2957562358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c4e40 msgr2=0x5638f53cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0045e00640 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c4e40 0x5638f53cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0045e00640 1 -- 172.21.15.46:0/2957562358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 msgr2=0x5638f53cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0045e00640 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0045e00640 1 -- 172.21.15.46:0/2957562358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f509c3c0 con 0x5638f53c5760 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0047200640 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0045e00640 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c5760 0x5638f53ccd40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0034009a20 tx=0x7f0034005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73747cbb7b81bf5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0027e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0027e00640 1 -- 172.21.15.46:0/2957562358 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034005040 con 0x5638f53c5760 2024-09-10T07:43:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0050059740 1 -- 172.21.15.46:0/2957562358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00380043d0 con 0x5638f53c5760 2024-09-10T07:43:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0027e00640 1 -- 172.21.15.46:0/2957562358 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f00340051e0 con 0x5638f53c5760 2024-09-10T07:43:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0027e00640 1 -- 172.21.15.46:0/2957562358 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f00340053e0 con 0x5638f53c5760 2024-09-10T07:43:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0050059740 10 client.26980 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.594+0000 7f0050059740 1 -- 172.21.15.46:0/2957562358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5638f53debb0 con 0x5638f53c5760 2024-09-10T07:43:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0027e00640 1 -- 172.21.15.46:0/2957562358 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00380043d0 con 0x5638f53c5760 2024-09-10T07:43:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0027e00640 1 -- 172.21.15.46:0/2957562358 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f0034044030 con 0x5638f53c5760 2024-09-10T07:43:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0050059740 10 client.26980 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0050059740 10 client.26980 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0050059740 20 client.26980 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0050059740 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5638f53dffc0 0x5638f53e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0050059740 4 client.26980 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0050059740 1 -- 172.21.15.46:0/2957562358 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638f5338950 con 0x5638f53dffc0 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.595+0000 7f0046800640 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5638f53dffc0 0x5638f53e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.596+0000 7f0046800640 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5638f53dffc0 0x5638f53e2430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=442c55aae4a84f4f server_cookie=c8d73fc233dc1294 in_seq=0 out_seq=0 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.596+0000 7f0027e00640 10 client.26980 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0027e00640 1 -- 172.21.15.46:0/2957562358 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638f5338950 con 0x5638f53dffc0 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0027e00640 10 client.26980 handle_command_reply: tid=0 2024-09-10T07:43:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 client.26980 shutdown 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 -- 172.21.15.46:0/2957562358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c5760 msgr2=0x5638f53ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c5760 0x5638f53ccd40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0034009a20 tx=0x7f0034005a20 comp rx=0 tx=0).stop 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 -- 172.21.15.46:0/2957562358 shutdown_connections 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5638f53dffc0 0x5638f53e2430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53d0020 0x5638f53cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0027e00640 0 client.26980 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c5760 0x5638f53ccd40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 --2- 172.21.15.46:0/2957562358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c4e40 0x5638f53cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 -- 172.21.15.46:0/2957562358 >> 172.21.15.46:0/2957562358 conn(0x5638f53c13d0 msgr2=0x5638f53c70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.597+0000 7f0050059740 1 -- 172.21.15.46:0/2957562358 shutdown_connections 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.598+0000 7f0050059740 1 -- 172.21.15.46:0/2957562358 wait complete. 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.598+0000 7f0050059740 20 client.26980 trim_cache size 0 max 16384 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.616+0000 7f0050059740 1 Processor -- start 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.617+0000 7f0050059740 1 -- start start 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.617+0000 7f0050059740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 0x5638f53cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.617+0000 7f0050059740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.617+0000 7f0050059740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53e8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.617+0000 7f0050059740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f002800fb20 con 0x5638f53c13d0 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.617+0000 7f0050059740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00280116f0 con 0x5638f53c5760 2024-09-10T07:43:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.617+0000 7f0050059740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00280101f0 con 0x5638f53c4e40 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.618+0000 7f0047200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.618+0000 7f0047200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53e8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55972/0 (socket says 172.21.15.46:55972) 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.618+0000 7f0047200640 1 -- 172.21.15.46:0/872030659 learned_addr learned my addr 172.21.15.46:0/872030659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.618+0000 7f0045e00640 1 --2- 172.21.15.46:0/872030659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.619+0000 7f0047200640 1 -- 172.21.15.46:0/872030659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 msgr2=0x5638f53e7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.619+0000 7f0047200640 1 --2- 172.21.15.46:0/872030659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53e7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.619+0000 7f0047200640 1 -- 172.21.15.46:0/872030659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 msgr2=0x5638f53cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.619+0000 7f0047200640 1 --2- 172.21.15.46:0/872030659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 0x5638f53cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.619+0000 7f0047200640 1 -- 172.21.15.46:0/872030659 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f53c1200 con 0x5638f53c5760 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.621+0000 7f0047200640 1 --2- 172.21.15.46:0/872030659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53e8280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f004009b700 tx=0x7f004009bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f0c6d477583e0ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.621+0000 7f0032a00640 1 -- 172.21.15.46:0/872030659 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00400b4070 con 0x5638f53c5760 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.621+0000 7f0032a00640 1 -- 172.21.15.46:0/872030659 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0040099aa0 con 0x5638f53c5760 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.621+0000 7f0032a00640 1 -- 172.21.15.46:0/872030659 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004009b0b0 con 0x5638f53c5760 2024-09-10T07:43:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.622+0000 7f0050059740 1 -- 172.21.15.46:0/872030659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 msgr2=0x5638f53e8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.622+0000 7f0050059740 1 --2- 172.21.15.46:0/872030659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53e8280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f004009b700 tx=0x7f004009bac0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.627+0000 7f0050059740 1 -- 172.21.15.46:0/872030659 shutdown_connections 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.627+0000 7f0050059740 1 --2- 172.21.15.46:0/872030659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53e8280 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.627+0000 7f0050059740 1 --2- 172.21.15.46:0/872030659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.627+0000 7f0050059740 1 --2- 172.21.15.46:0/872030659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 0x5638f53cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.627+0000 7f0050059740 1 -- 172.21.15.46:0/872030659 >> 172.21.15.46:0/872030659 conn(0x5638f53bc670 msgr2=0x5638f53cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.628+0000 7f0050059740 1 -- 172.21.15.46:0/872030659 shutdown_connections 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.628+0000 7f0050059740 1 -- 172.21.15.46:0/872030659 wait complete. 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.628+0000 7f0050059740 1 -- start start 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.629+0000 7f0050059740 1 Processor -- start 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.630+0000 7f0050059740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 0x5638f53d8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.630+0000 7f0050059740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53d8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.630+0000 7f0050059740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0050059740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0028010fc0 con 0x5638f53c13d0 2024-09-10T07:43:18.858 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:43:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0050059740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00280108c0 con 0x5638f53c5760 2024-09-10T07:43:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0050059740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f002800db00 con 0x5638f53c4e40 2024-09-10T07:43:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0050059740 10 client.0 _ll_register_callbacks cb 0x7f00400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0045e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0047200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0047200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53d30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55976/0 (socket says 172.21.15.46:55976) 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0047200640 1 -- 172.21.15.46:0/3320936380 learned_addr learned my addr 172.21.15.46:0/3320936380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0046800640 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 0x5638f53d8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0047200640 1 -- 172.21.15.46:0/3320936380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 msgr2=0x5638f53d8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.631+0000 7f0047200640 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53d8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.632+0000 7f0047200640 1 -- 172.21.15.46:0/3320936380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 msgr2=0x5638f53d8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.632+0000 7f0047200640 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 0x5638f53d8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.632+0000 7f0047200640 1 -- 172.21.15.46:0/3320936380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f53d3810 con 0x5638f53c5760 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.632+0000 7f0046800640 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 0x5638f53d8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.633+0000 7f0047200640 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53d30d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0040097e60 tx=0x7f00400a47c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ec55a8ae261ae65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.633+0000 7f0027e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.634+0000 7f0027e00640 1 -- 172.21.15.46:0/3320936380 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00400b8020 con 0x5638f53c5760 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.634+0000 7f0027e00640 1 -- 172.21.15.46:0/3320936380 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f00400ba2c0 con 0x5638f53c5760 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.636+0000 7f0050059740 1 -- 172.21.15.46:0/3320936380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0040099aa0 con 0x5638f53c5760 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.636+0000 7f0027e00640 1 -- 172.21.15.46:0/3320936380 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f00400b4040 con 0x5638f53c5760 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.638+0000 7f0050059740 10 client.27032 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.638+0000 7f0050059740 1 -- 172.21.15.46:0/3320936380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5638f544b750 con 0x5638f53c5760 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.639+0000 7f0027e00640 1 -- 172.21.15.46:0/3320936380 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0040099aa0 con 0x5638f53c5760 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.639+0000 7f0027e00640 1 -- 172.21.15.46:0/3320936380 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f004001f020 con 0x5638f53c5760 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.639+0000 7f0050059740 10 client.27032 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.639+0000 7f0050059740 10 client.27032 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.639+0000 7f0050059740 20 client.27032 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.639+0000 7f0050059740 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5638f53d0020 0x5638f544dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.639+0000 7f0050059740 4 client.27032 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.639+0000 7f0050059740 1 -- 172.21.15.46:0/3320936380 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638f5332e60 con 0x5638f53d0020 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.640+0000 7f0046800640 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5638f53d0020 0x5638f544dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.640+0000 7f0046800640 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5638f53d0020 0x5638f544dd60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f4f718e4412decf6 server_cookie=ec516670f6f73434 in_seq=0 out_seq=0 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.640+0000 7f0027e00640 10 client.27032 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0027e00640 1 -- 172.21.15.46:0/3320936380 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638f5332e60 con 0x5638f53d0020 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0027e00640 10 client.27032 handle_command_reply: tid=0 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 client.27032 shutdown 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 -- 172.21.15.46:0/3320936380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 msgr2=0x5638f53d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53d30d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0040097e60 tx=0x7f00400a47c0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 -- 172.21.15.46:0/3320936380 shutdown_connections 2024-09-10T07:43:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5638f53d0020 0x5638f544dd60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638f53c5760 0x5638f53d30d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638f53c4e40 0x5638f53d8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 --2- 172.21.15.46:0/3320936380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638f53c13d0 0x5638f53d8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0050059740 1 -- 172.21.15.46:0/3320936380 >> 172.21.15.46:0/3320936380 conn(0x5638f53bc670 msgr2=0x5638f53cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.641+0000 7f0027e00640 0 client.27032 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.644+0000 7f0050059740 1 -- 172.21.15.46:0/3320936380 shutdown_connections 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.644+0000 7f0050059740 1 -- 172.21.15.46:0/3320936380 wait complete. 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.644+0000 7f0050059740 20 client.27032 trim_cache size 0 max 16384 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.646+0000 7f004ca00640 1 -- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f002c0a2b80 msgr2=0x7f002c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.646+0000 7f004ca00640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f002c0a2b80 0x7f002c0a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0034031550 tx=0x7f003402a410 comp rx=0 tx=0).stop 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.646+0000 7f004ca00640 1 -- 172.21.15.46:0/824056729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0048108f60 msgr2=0x7f004819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.646+0000 7f004ca00640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0048108f60 0x7f004819ed70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0040009fd0 tx=0x7f004000ef10 comp rx=0 tx=0).stop 2024-09-10T07:43:18.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.647+0000 7f004ca00640 1 -- 172.21.15.46:0/824056729 shutdown_connections 2024-09-10T07:43:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.647+0000 7f004ca00640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0048108f60 0x7f004819ed70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.647+0000 7f004ca00640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00481047e0 0x7f004819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.647+0000 7f004ca00640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f002c0a2b80 0x7f002c0a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.647+0000 7f004ca00640 1 --2- 172.21.15.46:0/824056729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0048103e30 0x7f004819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.647+0000 7f004ca00640 1 -- 172.21.15.46:0/824056729 >> 172.21.15.46:0/824056729 conn(0x7f00480ff910 msgr2=0x7f0048100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.647+0000 7f004ca00640 1 -- 172.21.15.46:0/824056729 shutdown_connections 2024-09-10T07:43:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:18.648+0000 7f004ca00640 1 -- 172.21.15.46:0/824056729 wait complete. 2024-09-10T07:43:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 -- 172.21.15.46:0/2713358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64102d50 msgr2=0x7f7f641031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 --2- 172.21.15.46:0/2713358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64102d50 0x7f7f641031d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009900 tx=0x7f7f5002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 -- 172.21.15.46:0/2713358 shutdown_connections 2024-09-10T07:43:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 --2- 172.21.15.46:0/2713358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64103710 0x7f7f64109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 --2- 172.21.15.46:0/2713358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64102d50 0x7f7f641031d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 --2- 172.21.15.46:0/2713358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f64101b60 0x7f7f64101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 -- 172.21.15.46:0/2713358 >> 172.21.15.46:0/2713358 conn(0x7f7f640fd2e0 msgr2=0x7f7f640ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 -- 172.21.15.46:0/2713358 shutdown_connections 2024-09-10T07:43:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.263+0000 7f7f6b400640 1 -- 172.21.15.46:0/2713358 wait complete. 2024-09-10T07:43:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.264+0000 7f7f6b400640 1 Processor -- start 2024-09-10T07:43:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.264+0000 7f7f6b400640 1 -- start start 2024-09-10T07:43:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f6b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f64101b60 0x7f7f64199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f6b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64102d50 0x7f7f6419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64103710 0x7f7f641a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f6b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6410d830 con 0x7f7f64103710 2024-09-10T07:43:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f6b400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6410d6b0 con 0x7f7f64102d50 2024-09-10T07:43:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f6b400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6410d9b0 con 0x7f7f64101b60 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f69400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64103710 0x7f7f641a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64102d50 0x7f7f6419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64102d50 0x7f7f6419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55988/0 (socket says 172.21.15.46:55988) 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f69400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64103710 0x7f7f641a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38540/0 (socket says 172.21.15.46:38540) 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.265+0000 7f7f63e00640 1 -- 172.21.15.46:0/4225805151 learned_addr learned my addr 172.21.15.46:0/4225805151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f69400640 1 -- 172.21.15.46:0/4225805151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f64101b60 msgr2=0x7f7f64199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f69400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f64101b60 0x7f7f64199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f69400640 1 -- 172.21.15.46:0/4225805151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64102d50 msgr2=0x7f7f6419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f69400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64102d50 0x7f7f6419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f69400640 1 -- 172.21.15.46:0/4225805151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f641a1c00 con 0x7f7f64103710 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f63e00640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64102d50 0x7f7f6419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f69400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64103710 0x7f7f641a1590 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009fd0 tx=0x7f7f5c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f61600640 1 -- 172.21.15.46:0/4225805151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00ce20 con 0x7f7f64103710 2024-09-10T07:43:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f61600640 1 -- 172.21.15.46:0/4225805151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c005020 con 0x7f7f64103710 2024-09-10T07:43:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f6b400640 1 -- 172.21.15.46:0/4225805151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f641a1ef0 con 0x7f7f64103710 2024-09-10T07:43:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f61600640 1 -- 172.21.15.46:0/4225805151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c010720 con 0x7f7f64103710 2024-09-10T07:43:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.266+0000 7f7f6b400640 1 -- 172.21.15.46:0/4225805151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f641a23b0 con 0x7f7f64103710 2024-09-10T07:43:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.270+0000 7f7f61600640 1 -- 172.21.15.46:0/4225805151 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f5c005540 con 0x7f7f64103710 2024-09-10T07:43:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.271+0000 7f7f6e206740 1 Processor -- start 2024-09-10T07:43:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.271+0000 7f7f6e206740 1 -- start start 2024-09-10T07:43:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.271+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e0335090 0x55d5e0335490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.271+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e03359d0 0x55d5e033a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.271+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.271+0000 7f7f6e206740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5e029a100 con 0x55d5e03359d0 2024-09-10T07:43:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.271+0000 7f7f6e206740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d5e02421b0 con 0x55d5e0335090 2024-09-10T07:43:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.271+0000 7f7f6e206740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d5e02960c0 con 0x55d5e033aa60 2024-09-10T07:43:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e0335090 0x55d5e0335490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f68a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e03359d0 0x55d5e033a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e0335090 0x55d5e0335490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55998/0 (socket says 172.21.15.46:55998) 2024-09-10T07:43:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f63e00640 1 -- 172.21.15.46:0/47556183 learned_addr learned my addr 172.21.15.46:0/47556183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f63e00640 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f61600640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f4c0a2cb0 0x7f7f4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f68a00640 1 -- 172.21.15.46:0/47556183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 msgr2=0x55d5e033cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f68a00640 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f68a00640 1 -- 172.21.15.46:0/47556183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e0335090 msgr2=0x55d5e0335490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f68a00640 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e0335090 0x55d5e0335490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f68a00640 1 -- 172.21.15.46:0/47556183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5dffad210 con 0x55d5e03359d0 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f61600640 1 -- 172.21.15.46:0/4225805151 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7f5c0cfb10 con 0x7f7f64103710 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f69400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f4c0a2cb0 0x7f7f4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f63e00640 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.272+0000 7f7f68a00640 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e03359d0 0x55d5e033a490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009b30 tx=0x7f7f5800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1677802fa55a127 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.273+0000 7f7f55c00640 1 -- 172.21.15.46:0/47556183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800ed80 con 0x55d5e03359d0 2024-09-10T07:43:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.273+0000 7f7f55c00640 1 -- 172.21.15.46:0/47556183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f58005500 con 0x55d5e03359d0 2024-09-10T07:43:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.273+0000 7f7f55c00640 1 -- 172.21.15.46:0/47556183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58010630 con 0x55d5e03359d0 2024-09-10T07:43:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 -- 172.21.15.46:0/47556183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e03359d0 msgr2=0x55d5e033a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e03359d0 0x55d5e033a490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009b30 tx=0x7f7f5800e350 comp rx=0 tx=0).stop 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f69400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f4c0a2cb0 0x7f7f4c0a5170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c01a040 tx=0x7f7f5c005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 -- 172.21.15.46:0/47556183 shutdown_connections 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e03359d0 0x55d5e033a490 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 --2- 172.21.15.46:0/47556183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e0335090 0x55d5e0335490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 -- 172.21.15.46:0/47556183 >> 172.21.15.46:0/47556183 conn(0x55d5e0330700 msgr2=0x55d5e0330b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 -- 172.21.15.46:0/47556183 shutdown_connections 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 -- 172.21.15.46:0/47556183 wait complete. 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.274+0000 7f7f6e206740 1 -- start start 2024-09-10T07:43:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f6e206740 1 Processor -- start 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e03359d0 0x55d5e033f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e03467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f6e206740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5e033da30 con 0x55d5e0335090 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f6e206740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d5e0223090 con 0x55d5e033aa60 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f6e206740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d5e033dbb0 con 0x55d5e03359d0 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f6e206740 10 client.0 _ll_register_callbacks cb 0x55d5e0277f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38554/0 (socket says 172.21.15.46:38554) 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f63e00640 1 -- 172.21.15.46:0/858148321 learned_addr learned my addr 172.21.15.46:0/858148321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.275+0000 7f7f63e00640 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e03467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f68a00640 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e03359d0 0x55d5e033f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f63e00640 1 -- 172.21.15.46:0/858148321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e03359d0 msgr2=0x55d5e033f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f63e00640 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e03359d0 0x55d5e033f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f63e00640 1 -- 172.21.15.46:0/858148321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 msgr2=0x55d5e03467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f63e00640 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e03467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f63e00640 1 -- 172.21.15.46:0/858148321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e0079620 con 0x55d5e0335090 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f68a00640 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e03359d0 0x55d5e033f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f63e00640 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009330 tx=0x7f7f50041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b87e5283f977cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f4a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f4a000640 1 -- 172.21.15.46:0/858148321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50035040 con 0x55d5e0335090 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f4a000640 1 -- 172.21.15.46:0/858148321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7f500036b0 con 0x55d5e0335090 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.276+0000 7f7f6e206740 1 -- 172.21.15.46:0/858148321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f58005500 con 0x55d5e0335090 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f4a000640 1 -- 172.21.15.46:0/858148321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7f50053050 con 0x55d5e0335090 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f6e206740 10 client.16617 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f6e206740 1 -- 172.21.15.46:0/858148321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d5e0349600 con 0x55d5e0335090 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f4a000640 1 -- 172.21.15.46:0/858148321 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f58005500 con 0x55d5e0335090 2024-09-10T07:43:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f4a000640 1 -- 172.21.15.46:0/858148321 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f7f5003d9f0 con 0x55d5e0335090 2024-09-10T07:43:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f6e206740 10 client.16617 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f6e206740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f6e206740 20 client.16617 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f6e206740 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5e034ad00 0x55d5e034d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f6e206740 4 client.16617 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.277+0000 7f7f6e206740 1 -- 172.21.15.46:0/858148321 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e02a7830 con 0x55d5e034ad00 2024-09-10T07:43:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.278+0000 7f7f68a00640 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5e034ad00 0x55d5e034d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.278+0000 7f7f68a00640 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5e034ad00 0x55d5e034d180 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=314c35813f6971cd server_cookie=3650ec6f9c6221be in_seq=0 out_seq=0 2024-09-10T07:43:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.278+0000 7f7f4a000640 10 client.16617 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f4a000640 1 -- 172.21.15.46:0/858148321 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5e02a7830 con 0x55d5e034ad00 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f4a000640 10 client.16617 handle_command_reply: tid=0 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 client.16617 shutdown 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 -- 172.21.15.46:0/858148321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 msgr2=0x55d5e033f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009330 tx=0x7f7f50041040 comp rx=0 tx=0).stop 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 -- 172.21.15.46:0/858148321 shutdown_connections 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5e034ad00 0x55d5e034d180 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e03467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e03359d0 0x55d5e033f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 --2- 172.21.15.46:0/858148321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f220 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.279+0000 7f7f6e206740 1 -- 172.21.15.46:0/858148321 >> 172.21.15.46:0/858148321 conn(0x55d5e0330700 msgr2=0x55d5e0331f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.280+0000 7f7f4a000640 0 client.16617 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.280+0000 7f7f6e206740 1 -- 172.21.15.46:0/858148321 shutdown_connections 2024-09-10T07:43:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.280+0000 7f7f6e206740 1 -- 172.21.15.46:0/858148321 wait complete. 2024-09-10T07:43:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.280+0000 7f7f6e206740 20 client.16617 trim_cache size 0 max 16384 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.297+0000 7f7f6e206740 1 Processor -- start 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.297+0000 7f7f6e206740 1 -- start start 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.297+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.297+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e03359d0 0x55d5e0346760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.297+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.297+0000 7f7f6e206740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f641ac840 con 0x55d5e0335090 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.297+0000 7f7f6e206740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f641ae410 con 0x55d5e03359d0 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.297+0000 7f7f6e206740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7f641acf10 con 0x55d5e033aa60 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f68a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e03359d0 0x55d5e0346760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f68a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e03359d0 0x55d5e0346760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56008/0 (socket says 172.21.15.46:56008) 2024-09-10T07:43:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f68a00640 1 -- 172.21.15.46:0/2295920202 learned_addr learned my addr 172.21.15.46:0/2295920202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f63e00640 1 --2- 172.21.15.46:0/2295920202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f68a00640 1 -- 172.21.15.46:0/2295920202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 msgr2=0x55d5e033a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f68a00640 1 --2- 172.21.15.46:0/2295920202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f68a00640 1 -- 172.21.15.46:0/2295920202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 msgr2=0x55d5e033f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f68a00640 1 --2- 172.21.15.46:0/2295920202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.298+0000 7f7f68a00640 1 -- 172.21.15.46:0/2295920202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e0335eb0 con 0x55d5e03359d0 2024-09-10T07:43:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.299+0000 7f7f68a00640 1 --2- 172.21.15.46:0/2295920202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e03359d0 0x55d5e0346760 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009b30 tx=0x7f7f580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af53e7c89de8543 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.299+0000 7f7f55200640 1 -- 172.21.15.46:0/2295920202 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5801d040 con 0x55d5e03359d0 2024-09-10T07:43:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.299+0000 7f7f55200640 1 -- 172.21.15.46:0/2295920202 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5801e040 con 0x55d5e03359d0 2024-09-10T07:43:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.299+0000 7f7f55200640 1 -- 172.21.15.46:0/2295920202 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f580107b0 con 0x55d5e03359d0 2024-09-10T07:43:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 -- 172.21.15.46:0/2295920202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e03359d0 msgr2=0x55d5e0346760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 --2- 172.21.15.46:0/2295920202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e03359d0 0x55d5e0346760 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009b30 tx=0x7f7f580046c0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 -- 172.21.15.46:0/2295920202 shutdown_connections 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 --2- 172.21.15.46:0/2295920202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e033aa60 0x55d5e033a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 --2- 172.21.15.46:0/2295920202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e03359d0 0x55d5e0346760 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 --2- 172.21.15.46:0/2295920202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0335090 0x55d5e033f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 -- 172.21.15.46:0/2295920202 >> 172.21.15.46:0/2295920202 conn(0x55d5e0330700 msgr2=0x55d5e033efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 -- 172.21.15.46:0/2295920202 shutdown_connections 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 -- 172.21.15.46:0/2295920202 wait complete. 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 -- start start 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.300+0000 7f7f6e206740 1 Processor -- start 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e0335090 0x55d5e0343ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e0344000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f6e206740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0345010 0x55d5e0345470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f6e206740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f641adce0 con 0x55d5e0345010 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f6e206740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f641ad5e0 con 0x55d5e033aa60 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f6e206740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7f641aa820 con 0x55d5e0335090 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f6e206740 10 client.0 _ll_register_callbacks cb 0x7f7f50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f68a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e0344000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e0335090 0x55d5e0343ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f68a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e0344000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56020/0 (socket says 172.21.15.46:56020) 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f68a00640 1 -- 172.21.15.46:0/1307729179 learned_addr learned my addr 172.21.15.46:0/1307729179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.301+0000 7f7f63e00640 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0345010 0x55d5e0345470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f68a00640 1 -- 172.21.15.46:0/1307729179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e0335090 msgr2=0x55d5e0343ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f68a00640 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e0335090 0x55d5e0343ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f68a00640 1 -- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0345010 msgr2=0x55d5e0345470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f68a00640 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0345010 0x55d5e0345470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f68a00640 1 -- 172.21.15.46:0/1307729179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e0345a00 con 0x55d5e033aa60 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f63e00640 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0345010 0x55d5e0345470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f68a00640 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e0344000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009b00 tx=0x7f7f58009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a97ef554f9ce81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f4a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f4a000640 1 -- 172.21.15.46:0/1307729179 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5803e060 con 0x55d5e033aa60 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f6e206740 1 -- 172.21.15.46:0/1307729179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5801e040 con 0x55d5e033aa60 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.302+0000 7f7f4a000640 1 -- 172.21.15.46:0/1307729179 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7f5800ed80 con 0x55d5e033aa60 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f4a000640 1 -- 172.21.15.46:0/1307729179 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7f58045050 con 0x55d5e033aa60 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f6e206740 10 client.27050 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f6e206740 1 -- 172.21.15.46:0/1307729179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d5e03b5a80 con 0x55d5e033aa60 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f4a000640 1 -- 172.21.15.46:0/1307729179 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5801e040 con 0x55d5e033aa60 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f4a000640 1 -- 172.21.15.46:0/1307729179 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f7f58010790 con 0x55d5e033aa60 2024-09-10T07:43:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f6e206740 10 client.27050 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f6e206740 10 client.27050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f6e206740 20 client.27050 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f6e206740 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5e03b6240 0x55d5e03b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f6e206740 4 client.27050 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:43:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.303+0000 7f7f6e206740 1 -- 172.21.15.46:0/1307729179 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5e032bd80 con 0x55d5e03b6240 2024-09-10T07:43:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.304+0000 7f7f63e00640 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5e03b6240 0x55d5e03b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.304+0000 7f7f63e00640 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5e03b6240 0x55d5e03b8660 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60181b891d10bb6a server_cookie=c6ea23b5cff78f7d in_seq=0 out_seq=0 2024-09-10T07:43:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.304+0000 7f7f4a000640 10 client.27050 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:20 smithi181 ceph-mon[35397]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.1 MiB/s wr, 114 op/s 2024-09-10T07:43:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:20 smithi063 ceph-mon[35020]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.1 MiB/s wr, 114 op/s 2024-09-10T07:43:20.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:20 smithi046 ceph-mon[30705]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.1 MiB/s wr, 114 op/s 2024-09-10T07:43:20.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 360.0 s, 726.8 tps, lat 128.496 ms stddev 142.751 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.682+0000 7f7f4a000640 1 -- 172.21.15.46:0/1307729179 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5e032bd80 con 0x55d5e03b6240 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.682+0000 7f7f4a000640 10 client.27050 handle_command_reply: tid=0 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.682+0000 7f7f6e206740 1 client.27050 shutdown 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.682+0000 7f7f6e206740 1 -- 172.21.15.46:0/1307729179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 msgr2=0x55d5e0344000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.682+0000 7f7f6e206740 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e0344000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009b00 tx=0x7f7f58009530 comp rx=0 tx=0).stop 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.682+0000 7f7f6e206740 1 -- 172.21.15.46:0/1307729179 shutdown_connections 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f6e206740 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5e03b6240 0x55d5e03b8660 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f6e206740 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5e0345010 0x55d5e0345470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f6e206740 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5e033aa60 0x55d5e0344000 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f6e206740 1 --2- 172.21.15.46:0/1307729179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5e0335090 0x55d5e0343ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f6e206740 1 -- 172.21.15.46:0/1307729179 >> 172.21.15.46:0/1307729179 conn(0x55d5e0330700 msgr2=0x55d5e033efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f4a000640 0 client.27050 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f6e206740 1 -- 172.21.15.46:0/1307729179 shutdown_connections 2024-09-10T07:43:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f6e206740 1 -- 172.21.15.46:0/1307729179 wait complete. 2024-09-10T07:43:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.683+0000 7f7f6e206740 20 client.27050 trim_cache size 0 max 16384 2024-09-10T07:43:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:43:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:43:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:43:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:43:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.685+0000 7f7f6b400640 1 -- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f4c0a2cb0 msgr2=0x7f7f4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.685+0000 7f7f6b400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f4c0a2cb0 0x7f7f4c0a5170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c01a040 tx=0x7f7f5c005940 comp rx=0 tx=0).stop 2024-09-10T07:43:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.685+0000 7f7f6b400640 1 -- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64103710 msgr2=0x7f7f641a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.685+0000 7f7f6b400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64103710 0x7f7f641a1590 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009fd0 tx=0x7f7f5c00ef10 comp rx=0 tx=0).stop 2024-09-10T07:43:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.686+0000 7f7f6b400640 1 -- 172.21.15.46:0/4225805151 shutdown_connections 2024-09-10T07:43:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.686+0000 7f7f6b400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7f4c0a2cb0 0x7f7f4c0a5170 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.686+0000 7f7f6b400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f64103710 0x7f7f641a1590 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.686+0000 7f7f6b400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64102d50 0x7f7f6419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.686+0000 7f7f6b400640 1 --2- 172.21.15.46:0/4225805151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f64101b60 0x7f7f64199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.686+0000 7f7f6b400640 1 -- 172.21.15.46:0/4225805151 >> 172.21.15.46:0/4225805151 conn(0x7f7f640fd2e0 msgr2=0x7f7f64074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.686+0000 7f7f6b400640 1 -- 172.21.15.46:0/4225805151 shutdown_connections 2024-09-10T07:43:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:20.686+0000 7f7f6b400640 1 -- 172.21.15.46:0/4225805151 wait complete. 2024-09-10T07:43:21.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:43:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:22.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.037+0000 7f3eeac00640 1 -- 172.21.15.46:0/2333562797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 msgr2=0x7f3ee410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.037+0000 7f3eeac00640 1 --2- 172.21.15.46:0/2333562797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 0x7f3ee410ea90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3ed001a910 tx=0x7f3ed003b430 comp rx=0 tx=0).stop 2024-09-10T07:43:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.039+0000 7f3eeac00640 1 -- 172.21.15.46:0/2333562797 shutdown_connections 2024-09-10T07:43:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.039+0000 7f3eeac00640 1 --2- 172.21.15.46:0/2333562797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 0x7f3ee410ea90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.039+0000 7f3eeac00640 1 --2- 172.21.15.46:0/2333562797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4102790 0x7f3ee41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.039+0000 7f3eeac00640 1 --2- 172.21.15.46:0/2333562797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ee4101de0 0x7f3ee41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.039+0000 7f3eeac00640 1 -- 172.21.15.46:0/2333562797 >> 172.21.15.46:0/2333562797 conn(0x7f3ee40fd8c0 msgr2=0x7f3ee40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.039+0000 7f3eeac00640 1 -- 172.21.15.46:0/2333562797 shutdown_connections 2024-09-10T07:43:22.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.039+0000 7f3eeac00640 1 -- 172.21.15.46:0/2333562797 wait complete. 2024-09-10T07:43:22.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.041+0000 7f3eeac00640 1 Processor -- start 2024-09-10T07:43:22.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.042+0000 7f3eeac00640 1 -- start start 2024-09-10T07:43:22.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.044+0000 7f3eeac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4101de0 0x7f3ee410e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.044+0000 7f3eeac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ee4102790 0x7f3ee410c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.044+0000 7f3eeac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 0x7f3ee410ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.044+0000 7f3eeac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee40729d0 con 0x7f3ee4101de0 2024-09-10T07:43:22.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.044+0000 7f3eeac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee4072850 con 0x7f3ee4102790 2024-09-10T07:43:22.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.044+0000 7f3eeac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee4072b50 con 0x7f3ee4106f10 2024-09-10T07:43:22.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.044+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ee4102790 0x7f3ee410c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 0x7f3ee410ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 0x7f3ee410ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56972/0 (socket says 172.21.15.46:56972) 2024-09-10T07:43:22.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ee4102790 0x7f3ee410c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56048/0 (socket says 172.21.15.46:56048) 2024-09-10T07:43:22.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee8c00640 1 -- 172.21.15.46:0/3769796057 learned_addr learned my addr 172.21.15.46:0/3769796057 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:22.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee8c00640 1 -- 172.21.15.46:0/3769796057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ee4102790 msgr2=0x7f3ee410c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee8c00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ee4102790 0x7f3ee410c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee8c00640 1 -- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4101de0 msgr2=0x7f3ee410e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee8c00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4101de0 0x7f3ee410e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee8c00640 1 -- 172.21.15.46:0/3769796057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee410d700 con 0x7f3ee4106f10 2024-09-10T07:43:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.045+0000 7f3ee3400640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ee4102790 0x7f3ee410c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:22.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.046+0000 7f3ee8c00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 0x7f3ee410ce90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00048f0 tx=0x7f3ed0004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:22.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.046+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3769796057 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed004f070 con 0x7f3ee4106f10 2024-09-10T07:43:22.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.046+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3769796057 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed0004c50 con 0x7f3ee4106f10 2024-09-10T07:43:22.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.046+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3769796057 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0046030 con 0x7f3ee4106f10 2024-09-10T07:43:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.047+0000 7f3eeac00640 1 -- 172.21.15.46:0/3769796057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee419e3c0 con 0x7f3ee4106f10 2024-09-10T07:43:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.048+0000 7f3eeac00640 1 -- 172.21.15.46:0/3769796057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee419e820 con 0x7f3ee4106f10 2024-09-10T07:43:22.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.057+0000 7f3eed5db740 1 Processor -- start 2024-09-10T07:43:22.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.057+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3769796057 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ed0052070 con 0x7f3ee4106f10 2024-09-10T07:43:22.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.057+0000 7f3eed5db740 1 -- start start 2024-09-10T07:43:22.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b4295790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42a0590 0x5617b42a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3eed5db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5617b41fb530 con 0x5617b42a0590 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3eed5db740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5617b41a2860 con 0x5617b42953b0 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3eed5db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5617b41f74f0 con 0x5617b4295cd0 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b4295790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b4295790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56060/0 (socket says 172.21.15.46:56060) 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.058+0000 7f3ee3400640 1 -- 172.21.15.46:0/3125906948 learned_addr learned my addr 172.21.15.46:0/3125906948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.059+0000 7f3ee3400640 1 -- 172.21.15.46:0/3125906948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 msgr2=0x5617b42a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.059+0000 7f3ee3400640 1 --2- 172.21.15.46:0/3125906948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.059+0000 7f3ee3400640 1 -- 172.21.15.46:0/3125906948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42a0590 msgr2=0x5617b42a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.059+0000 7f3ee3400640 1 --2- 172.21.15.46:0/3125906948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42a0590 0x5617b42a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.059+0000 7f3ee3400640 1 -- 172.21.15.46:0/3125906948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b3f6cb90 con 0x5617b42953b0 2024-09-10T07:43:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.059+0000 7f3ee3400640 1 --2- 172.21.15.46:0/3125906948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b4295790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009e60 tx=0x7f3ed4010620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d628249474fa38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:22.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.062+0000 7f3ee0c00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ec80a2c40 0x7f3ec80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.063+0000 7f3ed9200640 1 -- 172.21.15.46:0/3125906948 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4019ad0 con 0x5617b42953b0 2024-09-10T07:43:22.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.063+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3769796057 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ed0100320 con 0x7f3ee4106f10 2024-09-10T07:43:22.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.063+0000 7f3ed9200640 1 -- 172.21.15.46:0/3125906948 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed401d070 con 0x5617b42953b0 2024-09-10T07:43:22.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.063+0000 7f3ed9200640 1 -- 172.21.15.46:0/3125906948 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4021630 con 0x5617b42953b0 2024-09-10T07:43:22.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.063+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ec80a2c40 0x7f3ec80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3eed5db740 1 -- 172.21.15.46:0/3125906948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 msgr2=0x5617b4295790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3eed5db740 1 --2- 172.21.15.46:0/3125906948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b4295790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009e60 tx=0x7f3ed4010620 comp rx=0 tx=0).stop 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ec80a2c40 0x7f3ec80a5100 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5617b42927a0 tx=0x7f3edc0093c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3eed5db740 1 -- 172.21.15.46:0/3125906948 shutdown_connections 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3eed5db740 1 --2- 172.21.15.46:0/3125906948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42a0590 0x5617b42a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3eed5db740 1 --2- 172.21.15.46:0/3125906948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3eed5db740 1 --2- 172.21.15.46:0/3125906948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b4295790 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3eed5db740 1 -- 172.21.15.46:0/3125906948 >> 172.21.15.46:0/3125906948 conn(0x5617b4291940 msgr2=0x5617b4291d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.065+0000 7f3eed5db740 1 -- 172.21.15.46:0/3125906948 shutdown_connections 2024-09-10T07:43:22.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.066+0000 7f3eed5db740 1 -- 172.21.15.46:0/3125906948 wait complete. 2024-09-10T07:43:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.067+0000 7f3eed5db740 1 -- start start 2024-09-10T07:43:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.067+0000 7f3eed5db740 1 Processor -- start 2024-09-10T07:43:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.068+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 0x5617b429d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.068+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b429dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.068+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b429e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.068+0000 7f3eed5db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5617b42a3010 con 0x5617b42953b0 2024-09-10T07:43:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.068+0000 7f3eed5db740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5617b41b5e60 con 0x5617b42a0590 2024-09-10T07:43:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.068+0000 7f3eed5db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5617b42a3190 con 0x5617b4295cd0 2024-09-10T07:43:22.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.068+0000 7f3eed5db740 10 client.0 _ll_register_callbacks cb 0x5617b41bacc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:22.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.070+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b429dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.070+0000 7f3ee8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b429e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.070+0000 7f3ee8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b429e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56072/0 (socket says 172.21.15.46:56072) 2024-09-10T07:43:22.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.070+0000 7f3ee8c00640 1 -- 172.21.15.46:0/132287246 learned_addr learned my addr 172.21.15.46:0/132287246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.071+0000 7f3ee3400640 1 -- 172.21.15.46:0/132287246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 msgr2=0x5617b429e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.071+0000 7f3ee3400640 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b429e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.073+0000 7f3ee3400640 1 -- 172.21.15.46:0/132287246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 msgr2=0x5617b429d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.073+0000 7f3ee3400640 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 0x5617b429d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.073+0000 7f3ee3400640 1 -- 172.21.15.46:0/132287246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b41c1990 con 0x5617b4295cd0 2024-09-10T07:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.074+0000 7f3ecd600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.074+0000 7f3ee3400640 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b429dba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4004810 tx=0x7f3ed40205a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ac525e65f67457 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.074+0000 7f3ecd600640 1 -- 172.21.15.46:0/132287246 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4022670 con 0x5617b4295cd0 2024-09-10T07:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.075+0000 7f3ecd600640 1 -- 172.21.15.46:0/132287246 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ed4022810 con 0x5617b4295cd0 2024-09-10T07:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.075+0000 7f3eed5db740 1 -- 172.21.15.46:0/132287246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ed401d070 con 0x5617b4295cd0 2024-09-10T07:43:22.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.076+0000 7f3ecd600640 1 -- 172.21.15.46:0/132287246 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3ed4031050 con 0x5617b4295cd0 2024-09-10T07:43:22.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.076+0000 7f3eed5db740 10 client.27007 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:22.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.076+0000 7f3eed5db740 1 -- 172.21.15.46:0/132287246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5617b42aee60 con 0x5617b4295cd0 2024-09-10T07:43:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.077+0000 7f3ecd600640 1 -- 172.21.15.46:0/132287246 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed401d070 con 0x5617b4295cd0 2024-09-10T07:43:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.078+0000 7f3ecd600640 1 -- 172.21.15.46:0/132287246 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f3ed4022a10 con 0x5617b4295cd0 2024-09-10T07:43:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.078+0000 7f3eed5db740 10 client.27007 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.079+0000 7f3eed5db740 10 client.27007 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:43:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.079+0000 7f3eed5db740 20 client.27007 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.079+0000 7f3eed5db740 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5617b42b01b0 0x5617b42b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.079+0000 7f3eed5db740 4 client.27007 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.079+0000 7f3eed5db740 1 -- 172.21.15.46:0/132287246 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617b419e310 con 0x5617b42b01b0 2024-09-10T07:43:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.080+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5617b42b01b0 0x5617b42b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.082+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5617b42b01b0 0x5617b42b25a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b3c0de21a9aedc server_cookie=fd18213701fc94a5 in_seq=0 out_seq=0 2024-09-10T07:43:22.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.082+0000 7f3ecd600640 10 client.27007 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.086+0000 7f3ecd600640 1 -- 172.21.15.46:0/132287246 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617b419e310 con 0x5617b42b01b0 2024-09-10T07:43:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.086+0000 7f3ecd600640 10 client.27007 handle_command_reply: tid=0 2024-09-10T07:43:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.087+0000 7f3eed5db740 1 client.27007 shutdown 2024-09-10T07:43:22.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.088+0000 7f3eed5db740 1 -- 172.21.15.46:0/132287246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 msgr2=0x5617b429dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.088+0000 7f3eed5db740 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b429dba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4004810 tx=0x7f3ed40205a0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.089+0000 7f3eed5db740 1 -- 172.21.15.46:0/132287246 shutdown_connections 2024-09-10T07:43:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.089+0000 7f3eed5db740 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5617b42b01b0 0x5617b42b25a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.089+0000 7f3eed5db740 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b429e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.089+0000 7f3eed5db740 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b429dba0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.089+0000 7f3eed5db740 1 --2- 172.21.15.46:0/132287246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 0x5617b429d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.089+0000 7f3ecd600640 0 client.27007 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:22.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.089+0000 7f3eed5db740 1 -- 172.21.15.46:0/132287246 >> 172.21.15.46:0/132287246 conn(0x5617b4291940 msgr2=0x5617b4290c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:22.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.090+0000 7f3eed5db740 1 -- 172.21.15.46:0/132287246 shutdown_connections 2024-09-10T07:43:22.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.090+0000 7f3eed5db740 1 -- 172.21.15.46:0/132287246 wait complete. 2024-09-10T07:43:22.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.091+0000 7f3eed5db740 20 client.27007 trim_cache size 0 max 16384 2024-09-10T07:43:22.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.122+0000 7f3eed5db740 1 Processor -- start 2024-09-10T07:43:22.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3eed5db740 1 -- start start 2024-09-10T07:43:22.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b429cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b4295cd0 0x5617b42b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b42a0590 0x5617b42b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3eed5db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee41a4220 con 0x5617b4295cd0 2024-09-10T07:43:22.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3eed5db740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee41a5df0 con 0x5617b42953b0 2024-09-10T07:43:22.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3eed5db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee41a48f0 con 0x5617b42a0590 2024-09-10T07:43:22.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b429cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b429cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56080/0 (socket says 172.21.15.46:56080) 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.123+0000 7f3ee3e00640 1 -- 172.21.15.46:0/3345025993 learned_addr learned my addr 172.21.15.46:0/3345025993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ee8c00640 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b42a0590 0x5617b42b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ee3e00640 1 -- 172.21.15.46:0/3345025993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b42a0590 msgr2=0x5617b42b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b42a0590 0x5617b42b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ee3e00640 1 -- 172.21.15.46:0/3345025993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b4295cd0 msgr2=0x5617b42b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b4295cd0 0x5617b42b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ee3e00640 1 -- 172.21.15.46:0/3345025993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b42a2cf0 con 0x5617b42953b0 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ee8c00640 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b42a0590 0x5617b42b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:43:22.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b429cf40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3edc013040 tx=0x7f3edc005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9cd0d96235140b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.124+0000 7f3ecfe00640 1 -- 172.21.15.46:0/3345025993 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0207e0 con 0x5617b42953b0 2024-09-10T07:43:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.125+0000 7f3ecfe00640 1 -- 172.21.15.46:0/3345025993 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3edc02d070 con 0x5617b42953b0 2024-09-10T07:43:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.125+0000 7f3ecfe00640 1 -- 172.21.15.46:0/3345025993 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc028c10 con 0x5617b42953b0 2024-09-10T07:43:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.126+0000 7f3eed5db740 1 -- 172.21.15.46:0/3345025993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 msgr2=0x5617b429cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.126+0000 7f3eed5db740 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b429cf40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3edc013040 tx=0x7f3edc005bd0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.126+0000 7f3eed5db740 1 -- 172.21.15.46:0/3345025993 shutdown_connections 2024-09-10T07:43:22.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.126+0000 7f3eed5db740 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b42a0590 0x5617b42b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.127+0000 7f3eed5db740 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b4295cd0 0x5617b42b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.127+0000 7f3eed5db740 1 --2- 172.21.15.46:0/3345025993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42953b0 0x5617b429cf40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.127+0000 7f3eed5db740 1 -- 172.21.15.46:0/3345025993 >> 172.21.15.46:0/3345025993 conn(0x5617b4291940 msgr2=0x5617b429f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:22.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.128+0000 7f3eed5db740 1 -- 172.21.15.46:0/3345025993 shutdown_connections 2024-09-10T07:43:22.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.128+0000 7f3eed5db740 1 -- 172.21.15.46:0/3345025993 wait complete. 2024-09-10T07:43:22.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.129+0000 7f3eed5db740 1 -- start start 2024-09-10T07:43:22.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.130+0000 7f3eed5db740 1 Processor -- start 2024-09-10T07:43:22.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.131+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 0x5617b42a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.131+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.131+0000 7f3eed5db740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b42a5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.131+0000 7f3eed5db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee41a56c0 con 0x5617b42953b0 2024-09-10T07:43:22.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.131+0000 7f3eed5db740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee41a4fc0 con 0x5617b42a0590 2024-09-10T07:43:22.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.131+0000 7f3eed5db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee41a21c0 con 0x5617b4295cd0 2024-09-10T07:43:22.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.131+0000 7f3eed5db740 10 client.0 _ll_register_callbacks cb 0x7f3ed4009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:22.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.132+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.133+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56990/0 (socket says 172.21.15.46:56990) 2024-09-10T07:43:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.133+0000 7f3ee3400640 1 -- 172.21.15.46:0/1460966030 learned_addr learned my addr 172.21.15.46:0/1460966030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.133+0000 7f3ee8c00640 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b42a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.134+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 0x5617b42a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.135+0000 7f3ee3400640 1 -- 172.21.15.46:0/1460966030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 msgr2=0x5617b42a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.135+0000 7f3ee3400640 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b42a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.135+0000 7f3ee3400640 1 -- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 msgr2=0x5617b42a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.135+0000 7f3ee3400640 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 0x5617b42a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.135+0000 7f3ee3400640 1 -- 172.21.15.46:0/1460966030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b42a5b80 con 0x5617b4295cd0 2024-09-10T07:43:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.136+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 0x5617b42a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.137+0000 7f3ee3400640 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a8be0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40295f0 tx=0x7f3ed402b530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ad5b6b87e5c8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.138+0000 7f3ecd600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:22.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.138+0000 7f3ecd600640 1 -- 172.21.15.46:0/1460966030 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4031050 con 0x5617b4295cd0 2024-09-10T07:43:22.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.139+0000 7f3ecd600640 1 -- 172.21.15.46:0/1460966030 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ed4022470 con 0x5617b4295cd0 2024-09-10T07:43:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.139+0000 7f3eed5db740 1 -- 172.21.15.46:0/1460966030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3edc02d070 con 0x5617b4295cd0 2024-09-10T07:43:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.140+0000 7f3ecd600640 1 -- 172.21.15.46:0/1460966030 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3ed4034340 con 0x5617b4295cd0 2024-09-10T07:43:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.140+0000 7f3eed5db740 10 client.27010 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.140+0000 7f3eed5db740 1 -- 172.21.15.46:0/1460966030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5617b431b4e0 con 0x5617b4295cd0 2024-09-10T07:43:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.140+0000 7f3ecd600640 1 -- 172.21.15.46:0/1460966030 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3edc02d070 con 0x5617b4295cd0 2024-09-10T07:43:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.141+0000 7f3ecd600640 1 -- 172.21.15.46:0/1460966030 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f3ed4022a10 con 0x5617b4295cd0 2024-09-10T07:43:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.144+0000 7f3eed5db740 10 client.27010 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.144+0000 7f3eed5db740 10 client.27010 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:43:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.144+0000 7f3eed5db740 20 client.27010 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.145+0000 7f3eed5db740 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5617b42b01b0 0x5617b431dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.145+0000 7f3eed5db740 4 client.27010 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T07:43:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.145+0000 7f3eed5db740 1 -- 172.21.15.46:0/1460966030 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x5617b4208c60 con 0x5617b42b01b0 2024-09-10T07:43:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.145+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5617b42b01b0 0x5617b431dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:22.146 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:22.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.147+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5617b42b01b0 0x5617b431dca0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87282ecea1be9e5 server_cookie=f1459974dba90471 in_seq=0 out_seq=0 2024-09-10T07:43:22.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.147+0000 7f3ecd600640 10 client.27010 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:22.317 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:22 smithi046 ceph-mon[30705]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 265 op/s 2024-09-10T07:43:22.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3ecd600640 1 -- 172.21.15.46:0/1460966030 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5617b4208c60 con 0x5617b42b01b0 2024-09-10T07:43:22.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3ecd600640 10 client.27010 handle_command_reply: tid=0 2024-09-10T07:43:22.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 client.27010 shutdown 2024-09-10T07:43:22.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 -- 172.21.15.46:0/1460966030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 msgr2=0x5617b42a8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a8be0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40295f0 tx=0x7f3ed402b530 comp rx=0 tx=0).stop 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 -- 172.21.15.46:0/1460966030 shutdown_connections 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5617b42b01b0 0x5617b431dca0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617b42a0590 0x5617b42a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3ecd600640 0 client.27010 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617b4295cd0 0x5617b42a8be0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 --2- 172.21.15.46:0/1460966030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617b42953b0 0x5617b42a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.333+0000 7f3eed5db740 1 -- 172.21.15.46:0/1460966030 >> 172.21.15.46:0/1460966030 conn(0x5617b4291940 msgr2=0x5617b4296e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.334+0000 7f3eed5db740 1 -- 172.21.15.46:0/1460966030 shutdown_connections 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.334+0000 7f3eed5db740 1 -- 172.21.15.46:0/1460966030 wait complete. 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.334+0000 7f3eed5db740 20 client.27010 trim_cache size 0 max 16384 2024-09-10T07:43:22.334 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 43, 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 22, 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "409df7a9": { 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 22, 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 140.1, 2024-09-10T07:43:22.335 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:22.336 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:43:22.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 -- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ec80a2c40 msgr2=0x7f3ec80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ec80a2c40 0x7f3ec80a5100 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5617b42927a0 tx=0x7f3edc0093c0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 -- 172.21.15.46:0/3769796057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 msgr2=0x7f3ee410ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:22.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 0x7f3ee410ce90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00048f0 tx=0x7f3ed0004920 comp rx=0 tx=0).stop 2024-09-10T07:43:22.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 -- 172.21.15.46:0/3769796057 shutdown_connections 2024-09-10T07:43:22.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ee4106f10 0x7f3ee410ce90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ee4102790 0x7f3ee410c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ec80a2c40 0x7f3ec80a5100 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 --2- 172.21.15.46:0/3769796057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4101de0 0x7f3ee410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:22.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 -- 172.21.15.46:0/3769796057 >> 172.21.15.46:0/3769796057 conn(0x7f3ee40fd8c0 msgr2=0x7f3ee40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:22.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 -- 172.21.15.46:0/3769796057 shutdown_connections 2024-09-10T07:43:22.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:22.335+0000 7f3eeac00640 1 -- 172.21.15.46:0/3769796057 wait complete. 2024-09-10T07:43:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:22 smithi181 ceph-mon[35397]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 265 op/s 2024-09-10T07:43:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:22 smithi063 ceph-mon[35020]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 265 op/s 2024-09-10T07:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 -- 172.21.15.46:0/2482383837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5541049a0 msgr2=0x7fd5541089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 --2- 172.21.15.46:0/2482383837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5541049a0 0x7fd5541089f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009900 tx=0x7fd53c02a410 comp rx=0 tx=0).stop 2024-09-10T07:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 -- 172.21.15.46:0/2482383837 shutdown_connections 2024-09-10T07:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 --2- 172.21.15.46:0/2482383837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd554109120 0x7fd554110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 --2- 172.21.15.46:0/2482383837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5541049a0 0x7fd5541089f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 --2- 172.21.15.46:0/2482383837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd554103ff0 0x7fd5541043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 -- 172.21.15.46:0/2482383837 >> 172.21.15.46:0/2482383837 conn(0x7fd5540ffa90 msgr2=0x7fd554101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 -- 172.21.15.46:0/2482383837 shutdown_connections 2024-09-10T07:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.145+0000 7fd558e00640 1 -- 172.21.15.46:0/2482383837 wait complete. 2024-09-10T07:43:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.146+0000 7fd558e00640 1 Processor -- start 2024-09-10T07:43:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.146+0000 7fd558e00640 1 -- start start 2024-09-10T07:43:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.146+0000 7fd558e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd554103ff0 0x7fd55419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.146+0000 7fd558e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5541049a0 0x7fd55419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.146+0000 7fd558e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd554109120 0x7fd55419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.146+0000 7fd558e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd554114170 con 0x7fd554103ff0 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.146+0000 7fd558e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd554113ff0 con 0x7fd554109120 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.146+0000 7fd558e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541142f0 con 0x7fd5541049a0 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5541049a0 0x7fd55419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5541049a0 0x7fd55419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56998/0 (socket says 172.21.15.46:56998) 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 -- 172.21.15.46:0/1457906454 learned_addr learned my addr 172.21.15.46:0/1457906454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd552800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd554109120 0x7fd55419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd551e00640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd554103ff0 0x7fd55419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 -- 172.21.15.46:0/1457906454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd554109120 msgr2=0x7fd55419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd554109120 0x7fd55419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 -- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd554103ff0 msgr2=0x7fd55419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd554103ff0 0x7fd55419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 -- 172.21.15.46:0/1457906454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55419f600 con 0x7fd5541049a0 2024-09-10T07:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd552800640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd554109120 0x7fd55419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd551e00640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd554103ff0 0x7fd55419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.147+0000 7fd549400640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5541049a0 0x7fd55419ab90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd53c004770 tx=0x7fd53c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.148+0000 7fd54b400640 1 -- 172.21.15.46:0/1457906454 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c03f070 con 0x7fd5541049a0 2024-09-10T07:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.148+0000 7fd54b400640 1 -- 172.21.15.46:0/1457906454 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53c031ca0 con 0x7fd5541049a0 2024-09-10T07:43:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.148+0000 7fd54b400640 1 -- 172.21.15.46:0/1457906454 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c035030 con 0x7fd5541049a0 2024-09-10T07:43:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.148+0000 7fd558e00640 1 -- 172.21.15.46:0/1457906454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55419f890 con 0x7fd5541049a0 2024-09-10T07:43:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.148+0000 7fd558e00640 1 -- 172.21.15.46:0/1457906454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5541a71d0 con 0x7fd5541049a0 2024-09-10T07:43:23.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.151+0000 7fd54b400640 1 -- 172.21.15.46:0/1457906454 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd53c042070 con 0x7fd5541049a0 2024-09-10T07:43:23.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.151+0000 7fd55b861740 1 Processor -- start 2024-09-10T07:43:23.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.151+0000 7fd55b861740 1 -- start start 2024-09-10T07:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.151+0000 7fd55b861740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d22c70 0x55d777d23050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd55b861740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777d2d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd55b861740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d30240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd55b861740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d777c893e0 con 0x55d777d22c70 2024-09-10T07:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd55b861740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d777c30d80 con 0x55d777d2de50 2024-09-10T07:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd55b861740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d777c853a0 con 0x55d777d23590 2024-09-10T07:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd551e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777d2d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd551e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777d2d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57010/0 (socket says 172.21.15.46:57010) 2024-09-10T07:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd551e00640 1 -- 172.21.15.46:0/1472133716 learned_addr learned my addr 172.21.15.46:0/1472133716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd549400640 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d30240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd552800640 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d22c70 0x55d777d23050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd551e00640 1 -- 172.21.15.46:0/1472133716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 msgr2=0x55d777d30240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd551e00640 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d30240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd551e00640 1 -- 172.21.15.46:0/1472133716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d22c70 msgr2=0x55d777d23050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd551e00640 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d22c70 0x55d777d23050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.152+0000 7fd551e00640 1 -- 172.21.15.46:0/1472133716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d777c4fae0 con 0x55d777d23590 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd551e00640 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777d2d910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd5440115f0 tx=0x7fd544011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=608ea654685abb91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd543400640 1 -- 172.21.15.46:0/1472133716 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54401e070 con 0x55d777d23590 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd543400640 1 -- 172.21.15.46:0/1472133716 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54400fb20 con 0x55d777d23590 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd543400640 1 -- 172.21.15.46:0/1472133716 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544019bf0 con 0x55d777d23590 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd54b400640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5380a2b80 0x7fd5380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd54b400640 1 -- 172.21.15.46:0/1457906454 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd53c0ef670 con 0x7fd5541049a0 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd552800640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5380a2b80 0x7fd5380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd55b861740 1 -- 172.21.15.46:0/1472133716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 msgr2=0x55d777d2d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.153+0000 7fd55b861740 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777d2d910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd5440115f0 tx=0x7fd544011ab0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd55b861740 1 -- 172.21.15.46:0/1472133716 shutdown_connections 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd55b861740 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d30240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd55b861740 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777d2d910 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd55b861740 1 --2- 172.21.15.46:0/1472133716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d22c70 0x55d777d23050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd55b861740 1 -- 172.21.15.46:0/1472133716 >> 172.21.15.46:0/1472133716 conn(0x55d777d1f200 msgr2=0x55d777d1f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd55b861740 1 -- 172.21.15.46:0/1472133716 shutdown_connections 2024-09-10T07:43:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd55b861740 1 -- 172.21.15.46:0/1472133716 wait complete. 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd552800640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5380a2b80 0x7fd5380a5040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd54c005fd0 tx=0x7fd54c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.154+0000 7fd55b861740 1 -- start start 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd55b861740 1 Processor -- start 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd55b861740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d2ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd55b861740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d23590 0x55d777d2b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd55b861740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d2b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd55b861740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d777d30a20 con 0x55d777d2de50 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd55b861740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d777c440f0 con 0x55d777d23590 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd55b861740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d777d30ba0 con 0x55d777d22c70 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd55b861740 10 client.0 _ll_register_callbacks cb 0x55d777c2bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd551e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d2ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd551e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d2ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57012/0 (socket says 172.21.15.46:57012) 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd551e00640 1 -- 172.21.15.46:0/4277580501 learned_addr learned my addr 172.21.15.46:0/4277580501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd552800640 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d2b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.155+0000 7fd549400640 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d23590 0x55d777d2b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd551e00640 1 -- 172.21.15.46:0/4277580501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d23590 msgr2=0x55d777d2b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd551e00640 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d23590 0x55d777d2b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd551e00640 1 -- 172.21.15.46:0/4277580501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 msgr2=0x55d777d2b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd551e00640 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d2b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd551e00640 1 -- 172.21.15.46:0/4277580501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7779fa3c0 con 0x55d777d22c70 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd551e00640 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d2ad10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd54401c040 tx=0x7fd54400f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4f32cf9fe79d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd537e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd537e00640 1 -- 172.21.15.46:0/4277580501 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54401e040 con 0x55d777d22c70 2024-09-10T07:43:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd55b861740 1 -- 172.21.15.46:0/4277580501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54400fb20 con 0x55d777d22c70 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.156+0000 7fd537e00640 1 -- 172.21.15.46:0/4277580501 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd544004040 con 0x55d777d22c70 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd537e00640 1 -- 172.21.15.46:0/4277580501 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd54402f050 con 0x55d777d22c70 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd55b861740 10 client.27022 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd55b861740 1 -- 172.21.15.46:0/4277580501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d777d3ca30 con 0x55d777d22c70 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd537e00640 1 -- 172.21.15.46:0/4277580501 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54400fb20 con 0x55d777d22c70 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd537e00640 1 -- 172.21.15.46:0/4277580501 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7fd544003680 con 0x55d777d22c70 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd55b861740 10 client.27022 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd55b861740 10 client.27022 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd55b861740 20 client.27022 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd55b861740 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55d777d3de40 0x55d777d40230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd55b861740 4 client.27022 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.157+0000 7fd55b861740 1 -- 172.21.15.46:0/4277580501 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d777c96950 con 0x55d777d3de40 2024-09-10T07:43:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.158+0000 7fd549400640 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55d777d3de40 0x55d777d40230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.158+0000 7fd549400640 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55d777d3de40 0x55d777d40230 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b949f006653852d server_cookie=cd8e97cfe2eab577 in_seq=0 out_seq=0 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.158+0000 7fd537e00640 10 client.27022 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd537e00640 1 -- 172.21.15.46:0/4277580501 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d777c96950 con 0x55d777d3de40 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd537e00640 10 client.27022 handle_command_reply: tid=0 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 client.27022 shutdown 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 -- 172.21.15.46:0/4277580501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 msgr2=0x55d777d2ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d2ad10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd54401c040 tx=0x7fd54400f870 comp rx=0 tx=0).stop 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 -- 172.21.15.46:0/4277580501 shutdown_connections 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d2b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d23590 0x55d777d2b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55d777d3de40 0x55d777d40230 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 --2- 172.21.15.46:0/4277580501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d2ad10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 -- 172.21.15.46:0/4277580501 >> 172.21.15.46:0/4277580501 conn(0x55d777d1f200 msgr2=0x55d777d1e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:23.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd537e00640 0 client.27022 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:43:23.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 -- 172.21.15.46:0/4277580501 shutdown_connections 2024-09-10T07:43:23.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.159+0000 7fd55b861740 1 -- 172.21.15.46:0/4277580501 wait complete. 2024-09-10T07:43:23.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.160+0000 7fd55b861740 20 client.27022 trim_cache size 0 max 16384 2024-09-10T07:43:23.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.173+0000 7fd55b861740 1 Processor -- start 2024-09-10T07:43:23.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.173+0000 7fd55b861740 1 -- start start 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd55b861740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd55b861740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d23590 0x55d777d45f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd55b861740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd55b861740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541b0d70 con 0x55d777d23590 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd55b861740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541b2940 con 0x55d777d2de50 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd55b861740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541b1440 con 0x55d777d22c70 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd551e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd549400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d23590 0x55d777d45f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd551e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d45a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57020/0 (socket says 172.21.15.46:57020) 2024-09-10T07:43:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd551e00640 1 -- 172.21.15.46:0/1397744204 learned_addr learned my addr 172.21.15.46:0/1397744204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd552800640 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d2acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd549400640 1 -- 172.21.15.46:0/1397744204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 msgr2=0x55d777d45a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd549400640 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d45a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.174+0000 7fd549400640 1 -- 172.21.15.46:0/1397744204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 msgr2=0x55d777d2acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.175+0000 7fd549400640 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d2acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.175+0000 7fd549400640 1 -- 172.21.15.46:0/1397744204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d777d233c0 con 0x55d777d23590 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.175+0000 7fd551e00640 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.175+0000 7fd552800640 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.175+0000 7fd549400640 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d23590 0x55d777d45f50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55d777d20580 tx=0x7fd53c0ecc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7412344f5e098cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.175+0000 7fd542a00640 1 -- 172.21.15.46:0/1397744204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c049030 con 0x55d777d23590 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.175+0000 7fd542a00640 1 -- 172.21.15.46:0/1397744204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53c0bbc60 con 0x55d777d23590 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.175+0000 7fd542a00640 1 -- 172.21.15.46:0/1397744204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0ef360 con 0x55d777d23590 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 -- 172.21.15.46:0/1397744204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d23590 msgr2=0x55d777d45f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d23590 0x55d777d45f50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55d777d20580 tx=0x7fd53c0ecc60 comp rx=0 tx=0).stop 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 -- 172.21.15.46:0/1397744204 shutdown_connections 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d2de50 0x55d777d2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d23590 0x55d777d45f50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 --2- 172.21.15.46:0/1397744204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d22c70 0x55d777d45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 -- 172.21.15.46:0/1397744204 >> 172.21.15.46:0/1397744204 conn(0x55d777d1f200 msgr2=0x55d777d2cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 -- 172.21.15.46:0/1397744204 shutdown_connections 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 -- 172.21.15.46:0/1397744204 wait complete. 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.176+0000 7fd55b861740 1 -- start start 2024-09-10T07:43:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd55b861740 1 Processor -- start 2024-09-10T07:43:23.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd55b861740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d22c70 0x55d777da0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd55b861740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777da1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd55b861740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d31310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd55b861740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541b2210 con 0x55d777d2de50 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd55b861740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541b1b10 con 0x55d777d22c70 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd55b861740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541aed50 con 0x55d777d23590 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd55b861740 10 client.0 _ll_register_callbacks cb 0x7fd54401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd552800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d31310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd552800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d31310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38696/0 (socket says 172.21.15.46:38696) 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd552800640 1 -- 172.21.15.46:0/2310753218 learned_addr learned my addr 172.21.15.46:0/2310753218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd551e00640 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d22c70 0x55d777da0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.177+0000 7fd549400640 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777da1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd552800640 1 -- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 msgr2=0x55d777da1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd552800640 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777da1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd552800640 1 -- 172.21.15.46:0/2310753218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d22c70 msgr2=0x55d777da0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd552800640 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d22c70 0x55d777da0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd552800640 1 -- 172.21.15.46:0/2310753218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d777d31b10 con 0x55d777d2de50 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd549400640 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777da1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd552800640 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d31310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00faa0 tx=0x7fd54c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ced4f820c4a49f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd537e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd537e00640 1 -- 172.21.15.46:0/2310753218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c02e060 con 0x55d777d2de50 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.178+0000 7fd537e00640 1 -- 172.21.15.46:0/2310753218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd54c01f430 con 0x55d777d2de50 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 1 -- 172.21.15.46:0/2310753218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd53c0bbc60 con 0x55d777d2de50 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd537e00640 1 -- 172.21.15.46:0/2310753218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd54c035050 con 0x55d777d2de50 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 10 client.16644 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 1 -- 172.21.15.46:0/2310753218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d777da84f0 con 0x55d777d2de50 2024-09-10T07:43:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd537e00640 1 -- 172.21.15.46:0/2310753218 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53c0bbc60 con 0x55d777d2de50 2024-09-10T07:43:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd537e00640 1 -- 172.21.15.46:0/2310753218 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7fd54c029630 con 0x55d777d2de50 2024-09-10T07:43:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 10 client.16644 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 10 client.16644 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:43:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 20 client.16644 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55d777da8ca0 0x55d777dab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 4 client.16644 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:43:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.179+0000 7fd55b861740 1 -- 172.21.15.46:0/2310753218 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d777c90e60 con 0x55d777da8ca0 2024-09-10T07:43:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.180+0000 7fd551e00640 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55d777da8ca0 0x55d777dab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.180+0000 7fd551e00640 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55d777da8ca0 0x55d777dab0c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8957ab3d8ffa50b3 server_cookie=d076b8d882eb9e67 in_seq=0 out_seq=0 2024-09-10T07:43:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.180+0000 7fd537e00640 10 client.16644 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:43:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd537e00640 1 -- 172.21.15.46:0/2310753218 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d777c90e60 con 0x55d777da8ca0 2024-09-10T07:43:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd537e00640 10 client.16644 handle_command_reply: tid=0 2024-09-10T07:43:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 client.16644 shutdown 2024-09-10T07:43:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 -- 172.21.15.46:0/2310753218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 msgr2=0x55d777d31310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d31310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00faa0 tx=0x7fd54c004a40 comp rx=0 tx=0).stop 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 -- 172.21.15.46:0/2310753218 shutdown_connections 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d777d2de50 0x55d777d31310 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55d777da8ca0 0x55d777dab0c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d777d23590 0x55d777da1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 --2- 172.21.15.46:0/2310753218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d777d22c70 0x55d777da0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 -- 172.21.15.46:0/2310753218 >> 172.21.15.46:0/2310753218 conn(0x55d777d1f200 msgr2=0x55d777d246c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd537e00640 0 client.16644 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 -- 172.21.15.46:0/2310753218 shutdown_connections 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 1 -- 172.21.15.46:0/2310753218 wait complete. 2024-09-10T07:43:23.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.181+0000 7fd55b861740 20 client.16644 trim_cache size 0 max 16384 2024-09-10T07:43:23.185 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:43:23.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 -- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5380a2b80 msgr2=0x7fd5380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5380a2b80 0x7fd5380a5040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd54c005fd0 tx=0x7fd54c009500 comp rx=0 tx=0).stop 2024-09-10T07:43:23.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 -- 172.21.15.46:0/1457906454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5541049a0 msgr2=0x7fd55419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5541049a0 0x7fd55419ab90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd53c004770 tx=0x7fd53c003c10 comp rx=0 tx=0).stop 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 -- 172.21.15.46:0/1457906454 shutdown_connections 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd554109120 0x7fd55419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5541049a0 0x7fd55419ab90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5380a2b80 0x7fd5380a5040 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 --2- 172.21.15.46:0/1457906454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd554103ff0 0x7fd55419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 -- 172.21.15.46:0/1457906454 >> 172.21.15.46:0/1457906454 conn(0x7fd5540ffa90 msgr2=0x7fd554100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 -- 172.21.15.46:0/1457906454 shutdown_connections 2024-09-10T07:43:23.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:23.182+0000 7fd558e00640 1 -- 172.21.15.46:0/1457906454 wait complete. 2024-09-10T07:43:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:24 smithi181 ceph-mon[35397]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 331 op/s 2024-09-10T07:43:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:24 smithi063 ceph-mon[35020]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 331 op/s 2024-09-10T07:43:24.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:24 smithi046 ceph-mon[30705]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 331 op/s 2024-09-10T07:43:24.666 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '409df7a9', quiesce took 10.2 seconds. Will release after: 9.900000000000002 2024-09-10T07:43:25.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:43:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 365.0 s, 295.0 tps, lat 127.949 ms stddev 200.479 2024-09-10T07:43:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 -- 172.21.15.46:0/3090437948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820101de0 msgr2=0x7f38201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 --2- 172.21.15.46:0/3090437948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820101de0 0x7f38201021c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3810009900 tx=0x7f381002a410 comp rx=0 tx=0).stop 2024-09-10T07:43:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 -- 172.21.15.46:0/3090437948 shutdown_connections 2024-09-10T07:43:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 --2- 172.21.15.46:0/3090437948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3820106f10 0x7f382010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 --2- 172.21.15.46:0/3090437948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3820102790 0x7f38201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 --2- 172.21.15.46:0/3090437948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820101de0 0x7f38201021c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 -- 172.21.15.46:0/3090437948 >> 172.21.15.46:0/3090437948 conn(0x7f38200fd8c0 msgr2=0x7f38200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 -- 172.21.15.46:0/3090437948 shutdown_connections 2024-09-10T07:43:26.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.088+0000 7f3825c00640 1 -- 172.21.15.46:0/3090437948 wait complete. 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.089+0000 7f3825c00640 1 Processor -- start 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.089+0000 7f3825c00640 1 -- start start 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f3825c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3820101de0 0x7f382019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f3825c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3820102790 0x7f382019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f3825c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820106f10 0x7f382019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f3825c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3820111fc0 con 0x7f3820106f10 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f3825c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3820111e40 con 0x7f3820101de0 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f3825c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3820112140 con 0x7f3820102790 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820106f10 0x7f382019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820106f10 0x7f382019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55304/0 (socket says 172.21.15.46:55304) 2024-09-10T07:43:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381f400640 1 -- 172.21.15.46:0/2958061589 learned_addr learned my addr 172.21.15.46:0/2958061589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381ea00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3820101de0 0x7f382019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f3816000640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3820102790 0x7f382019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381f400640 1 -- 172.21.15.46:0/2958061589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3820102790 msgr2=0x7f382019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381f400640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3820102790 0x7f382019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381f400640 1 -- 172.21.15.46:0/2958061589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3820101de0 msgr2=0x7f382019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381f400640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3820101de0 0x7f382019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.090+0000 7f381f400640 1 -- 172.21.15.46:0/2958061589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382019f5e0 con 0x7f3820106f10 2024-09-10T07:43:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.091+0000 7f381f400640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820106f10 0x7f382019ef00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f381800eb10 tx=0x7f381800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.091+0000 7f3817e00640 1 -- 172.21.15.46:0/2958061589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818004b20 con 0x7f3820106f10 2024-09-10T07:43:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.091+0000 7f3817e00640 1 -- 172.21.15.46:0/2958061589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3818004cc0 con 0x7f3820106f10 2024-09-10T07:43:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.091+0000 7f3817e00640 1 -- 172.21.15.46:0/2958061589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818010760 con 0x7f3820106f10 2024-09-10T07:43:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.091+0000 7f3825c00640 1 -- 172.21.15.46:0/2958061589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382019f8d0 con 0x7f3820106f10 2024-09-10T07:43:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.091+0000 7f3825c00640 1 -- 172.21.15.46:0/2958061589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38201a71b0 con 0x7f3820106f10 2024-09-10T07:43:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.094+0000 7f382886c740 1 Processor -- start 2024-09-10T07:43:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.094+0000 7f3817e00640 1 -- 172.21.15.46:0/2958061589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3818010900 con 0x7f3820106f10 2024-09-10T07:43:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f382886c740 1 -- start start 2024-09-10T07:43:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f382886c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f382886c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c8590 0x563dd77d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f382886c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f382886c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563dd772e3e0 con 0x563dd77c7c70 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f382886c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563dd76d5d80 con 0x563dd77c8590 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f382886c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563dd772a3a0 con 0x563dd77d2e50 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f3816000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f3816000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55312/0 (socket says 172.21.15.46:55312) 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f3816000640 1 -- 172.21.15.46:0/1772512407 learned_addr learned my addr 172.21.15.46:0/1772512407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.095+0000 7f381ea00640 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c8590 0x563dd77d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3816000640 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3816000640 1 -- 172.21.15.46:0/1772512407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 msgr2=0x563dd77d5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3816000640 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77d5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3816000640 1 -- 172.21.15.46:0/1772512407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c8590 msgr2=0x563dd77d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:26.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3816000640 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c8590 0x563dd77d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3816000640 1 -- 172.21.15.46:0/1772512407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd76f4ae0 con 0x563dd77c7c70 2024-09-10T07:43:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3817e00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38040a2c00 0x7f38040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3817e00640 1 -- 172.21.15.46:0/2958061589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f38180cf930 con 0x7f3820106f10 2024-09-10T07:43:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f381ea00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38040a2c00 0x7f38040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.096+0000 7f3816000640 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77c8050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3808005bc0 tx=0x7f3808009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388f4736aadc7837 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.097+0000 7f380fe00640 1 -- 172.21.15.46:0/1772512407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808004050 con 0x563dd77c7c70 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.097+0000 7f380fe00640 1 -- 172.21.15.46:0/1772512407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38080041f0 con 0x563dd77c7c70 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.097+0000 7f380fe00640 1 -- 172.21.15.46:0/1772512407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380800f740 con 0x563dd77c7c70 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.097+0000 7f382886c740 1 -- 172.21.15.46:0/1772512407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 msgr2=0x563dd77c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.097+0000 7f381ea00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38040a2c00 0x7f38040a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3810000c00 tx=0x7f3810005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.097+0000 7f382886c740 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77c8050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3808005bc0 tx=0x7f3808009850 comp rx=0 tx=0).stop 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.098+0000 7f382886c740 1 -- 172.21.15.46:0/1772512407 shutdown_connections 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.098+0000 7f382886c740 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77d5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.098+0000 7f382886c740 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c8590 0x563dd77d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.098+0000 7f382886c740 1 --2- 172.21.15.46:0/1772512407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77c8050 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.098+0000 7f382886c740 1 -- 172.21.15.46:0/1772512407 >> 172.21.15.46:0/1772512407 conn(0x563dd77c4200 msgr2=0x563dd77c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.098+0000 7f382886c740 1 -- 172.21.15.46:0/1772512407 shutdown_connections 2024-09-10T07:43:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.098+0000 7f382886c740 1 -- 172.21.15.46:0/1772512407 wait complete. 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.098+0000 7f382886c740 1 -- start start 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f382886c740 1 Processor -- start 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f382886c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f382886c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 0x563dd77d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f382886c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77d2e50 0x563dd77d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f382886c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563dd77d5a20 con 0x563dd77d2e50 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f382886c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563dd76e90f0 con 0x563dd77c7c70 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f382886c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563dd77d5ba0 con 0x563dd77c8590 2024-09-10T07:43:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f382886c740 10 client.0 _ll_register_callbacks cb 0x563dd76d0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f381f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77d2e50 0x563dd77d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.099+0000 7f381f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77d2e50 0x563dd77d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55324/0 (socket says 172.21.15.46:55324) 2024-09-10T07:43:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f381f400640 1 -- 172.21.15.46:0/1782144395 learned_addr learned my addr 172.21.15.46:0/1782144395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f3816000640 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 0x563dd77d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f381ea00640 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f381f400640 1 -- 172.21.15.46:0/1782144395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 msgr2=0x563dd77d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f381f400640 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 0x563dd77d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f381f400640 1 -- 172.21.15.46:0/1782144395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 msgr2=0x563dd77cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f381f400640 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f381f400640 1 -- 172.21.15.46:0/1782144395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd749f3c0 con 0x563dd77d2e50 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f381f400640 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77d2e50 0x563dd77d0720 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3818000f80 tx=0x7f381809b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb912d3e02209b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f380cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f380cc00640 1 -- 172.21.15.46:0/1782144395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38180a4790 con 0x563dd77d2e50 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f380cc00640 1 -- 172.21.15.46:0/1782144395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f381809b200 con 0x563dd77d2e50 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.100+0000 7f382886c740 1 -- 172.21.15.46:0/1782144395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38080041f0 con 0x563dd77d2e50 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f380cc00640 1 -- 172.21.15.46:0/1782144395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f38180a0160 con 0x563dd77d2e50 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f382886c740 10 client.16656 fetch_fsmap learned FSMap version 43 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f382886c740 1 -- 172.21.15.46:0/1782144395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563dd77e1a10 con 0x563dd77d2e50 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f380cc00640 1 -- 172.21.15.46:0/1782144395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38080041f0 con 0x563dd77d2e50 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f380cc00640 1 -- 172.21.15.46:0/1782144395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f38180a12f0 con 0x563dd77d2e50 2024-09-10T07:43:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f382886c740 10 client.16656 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:43:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f382886c740 10 client.16656 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:43:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f382886c740 20 client.16656 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f382886c740 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563dd77e2ed0 0x563dd77e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f382886c740 4 client.16656 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.101+0000 7f382886c740 1 -- 172.21.15.46:0/1782144395 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dd773b950 con 0x563dd77e2ed0 2024-09-10T07:43:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.102+0000 7f381ea00640 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563dd77e2ed0 0x563dd77e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.102+0000 7f381ea00640 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563dd77e2ed0 0x563dd77e52c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53c2b279579d2023 server_cookie=fa9f833a12466ffa in_seq=0 out_seq=0 2024-09-10T07:43:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.102+0000 7f380cc00640 10 client.16656 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:43:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.103+0000 7f380cc00640 1 -- 172.21.15.46:0/1782144395 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563dd773b950 con 0x563dd77e2ed0 2024-09-10T07:43:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.103+0000 7f380cc00640 10 client.16656 handle_command_reply: tid=0 2024-09-10T07:43:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.103+0000 7f382886c740 1 client.16656 shutdown 2024-09-10T07:43:26.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.103+0000 7f382886c740 1 -- 172.21.15.46:0/1782144395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77d2e50 msgr2=0x563dd77d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.103+0000 7f382886c740 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77d2e50 0x563dd77d0720 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3818000f80 tx=0x7f381809b710 comp rx=0 tx=0).stop 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 1 -- 172.21.15.46:0/1782144395 shutdown_connections 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563dd77e2ed0 0x563dd77e52c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77d2e50 0x563dd77d0720 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 0x563dd77d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 1 --2- 172.21.15.46:0/1782144395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 1 -- 172.21.15.46:0/1782144395 >> 172.21.15.46:0/1782144395 conn(0x563dd77c4200 msgr2=0x563dd77c9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f380cc00640 0 client.16656 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 1 -- 172.21.15.46:0/1782144395 shutdown_connections 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 1 -- 172.21.15.46:0/1782144395 wait complete. 2024-09-10T07:43:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.104+0000 7f382886c740 20 client.16656 trim_cache size 0 max 16384 2024-09-10T07:43:26.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.118+0000 7f382886c740 1 Processor -- start 2024-09-10T07:43:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.118+0000 7f382886c740 1 -- start start 2024-09-10T07:43:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f382886c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f382886c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c8590 0x563dd77d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f382886c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77eab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f382886c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b0dd0 con 0x563dd77c8590 2024-09-10T07:43:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f382886c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b29a0 con 0x563dd77c7c70 2024-09-10T07:43:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f382886c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b14a0 con 0x563dd77d2e50 2024-09-10T07:43:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f3816000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c8590 0x563dd77d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f381f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f3816000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c8590 0x563dd77d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55328/0 (socket says 172.21.15.46:55328) 2024-09-10T07:43:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f3816000640 1 -- 172.21.15.46:0/437169801 learned_addr learned my addr 172.21.15.46:0/437169801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f381ea00640 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f381ea00640 1 -- 172.21.15.46:0/437169801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 msgr2=0x563dd77eab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f381ea00640 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77eab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f381ea00640 1 -- 172.21.15.46:0/437169801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c8590 msgr2=0x563dd77d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f381ea00640 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c8590 0x563dd77d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f381ea00640 1 -- 172.21.15.46:0/437169801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd77c4030 con 0x563dd77c7c70 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f381f400640 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77eab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.119+0000 7f3816000640 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c8590 0x563dd77d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.120+0000 7f381ea00640 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77ca6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f381002aa00 tx=0x7f381003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf54aecc23effe10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.120+0000 7f380f400640 1 -- 172.21.15.46:0/437169801 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810035040 con 0x563dd77c7c70 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.120+0000 7f380f400640 1 -- 172.21.15.46:0/437169801 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381005a070 con 0x563dd77c7c70 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.120+0000 7f380f400640 1 -- 172.21.15.46:0/437169801 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810032ac0 con 0x563dd77c7c70 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 -- 172.21.15.46:0/437169801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 msgr2=0x563dd77ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77ca6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f381002aa00 tx=0x7f381003f000 comp rx=0 tx=0).stop 2024-09-10T07:43:26.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 -- 172.21.15.46:0/437169801 shutdown_connections 2024-09-10T07:43:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77d2e50 0x563dd77eab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c8590 0x563dd77d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 --2- 172.21.15.46:0/437169801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77c7c70 0x563dd77ca6d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 -- 172.21.15.46:0/437169801 >> 172.21.15.46:0/437169801 conn(0x563dd77c4200 msgr2=0x563dd77d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 -- 172.21.15.46:0/437169801 shutdown_connections 2024-09-10T07:43:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.121+0000 7f382886c740 1 -- 172.21.15.46:0/437169801 wait complete. 2024-09-10T07:43:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 1 -- start start 2024-09-10T07:43:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 1 Processor -- start 2024-09-10T07:43:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77db3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 0x563dd77cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77d2e50 0x563dd77cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b2270 con 0x563dd77c7c70 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b1b70 con 0x563dd77d2e50 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38201aedb0 con 0x563dd77c8590 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f382886c740 10 client.0 _ll_register_callbacks cb 0x7f381809bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.122+0000 7f381ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77db3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77db3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55332/0 (socket says 172.21.15.46:55332) 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381ea00640 1 -- 172.21.15.46:0/3399333989 learned_addr learned my addr 172.21.15.46:0/3399333989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381f400640 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77d2e50 0x563dd77cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f3816000640 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 0x563dd77cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381ea00640 1 -- 172.21.15.46:0/3399333989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 msgr2=0x563dd77cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381ea00640 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 0x563dd77cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381ea00640 1 -- 172.21.15.46:0/3399333989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77d2e50 msgr2=0x563dd77cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381ea00640 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77d2e50 0x563dd77cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381ea00640 1 -- 172.21.15.46:0/3399333989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd77bf770 con 0x563dd77c7c70 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f381ea00640 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77db3b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3810005250 tx=0x7f3810032650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cde51a5dc4476c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.123+0000 7f380cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.124+0000 7f380cc00640 1 -- 172.21.15.46:0/3399333989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810032940 con 0x563dd77c7c70 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.124+0000 7f380cc00640 1 -- 172.21.15.46:0/3399333989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3810032ae0 con 0x563dd77c7c70 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.124+0000 7f382886c740 1 -- 172.21.15.46:0/3399333989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f381005a070 con 0x563dd77c7c70 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.124+0000 7f380cc00640 1 -- 172.21.15.46:0/3399333989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f381006f050 con 0x563dd77c7c70 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.124+0000 7f382886c740 10 client.16662 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.124+0000 7f382886c740 1 -- 172.21.15.46:0/3399333989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563dd784d7b0 con 0x563dd77c7c70 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.124+0000 7f380cc00640 1 -- 172.21.15.46:0/3399333989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381005a070 con 0x563dd77c7c70 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.124+0000 7f380cc00640 1 -- 172.21.15.46:0/3399333989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f38100035f0 con 0x563dd77c7c70 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.125+0000 7f382886c740 10 client.16662 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.125+0000 7f382886c740 10 client.16662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.125+0000 7f382886c740 20 client.16662 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.125+0000 7f382886c740 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563dd784de70 0x563dd7850290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.125+0000 7f382886c740 4 client.16662 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.125+0000 7f382886c740 1 -- 172.21.15.46:0/3399333989 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563dd7735e60 con 0x563dd784de70 2024-09-10T07:43:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.125+0000 7f3816000640 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563dd784de70 0x563dd7850290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f3816000640 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563dd784de70 0x563dd7850290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68c48bc3dc5724b8 server_cookie=bd28afbbffb2cf90 in_seq=0 out_seq=0 2024-09-10T07:43:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f380cc00640 10 client.16662 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:43:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f380cc00640 1 -- 172.21.15.46:0/3399333989 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563dd7735e60 con 0x563dd784de70 2024-09-10T07:43:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f380cc00640 10 client.16662 handle_command_reply: tid=0 2024-09-10T07:43:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 client.16662 shutdown 2024-09-10T07:43:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 -- 172.21.15.46:0/3399333989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 msgr2=0x563dd77db3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77db3b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3810005250 tx=0x7f3810032650 comp rx=0 tx=0).stop 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 -- 172.21.15.46:0/3399333989 shutdown_connections 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563dd784de70 0x563dd7850290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563dd77d2e50 0x563dd77cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563dd77c8590 0x563dd77cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 --2- 172.21.15.46:0/3399333989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd77c7c70 0x563dd77db3b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 -- 172.21.15.46:0/3399333989 >> 172.21.15.46:0/3399333989 conn(0x563dd77c4200 msgr2=0x563dd77c3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f380cc00640 0 client.16662 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 -- 172.21.15.46:0/3399333989 shutdown_connections 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.126+0000 7f382886c740 1 -- 172.21.15.46:0/3399333989 wait complete. 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f382886c740 20 client.16662 trim_cache size 0 max 16384 2024-09-10T07:43:26.129 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 -- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38040a2c00 msgr2=0x7f38040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38040a2c00 0x7f38040a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3810000c00 tx=0x7f3810005b50 comp rx=0 tx=0).stop 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 -- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820106f10 msgr2=0x7f382019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820106f10 0x7f382019ef00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f381800eb10 tx=0x7f381800c2d0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 -- 172.21.15.46:0/2958061589 shutdown_connections 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38040a2c00 0x7f38040a50c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3820106f10 0x7f382019ef00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3820102790 0x7f382019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 --2- 172.21.15.46:0/2958061589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3820101de0 0x7f382019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 -- 172.21.15.46:0/2958061589 >> 172.21.15.46:0/2958061589 conn(0x7f38200fd8c0 msgr2=0x7f38200ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 -- 172.21.15.46:0/2958061589 shutdown_connections 2024-09-10T07:43:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:26.127+0000 7f3825c00640 1 -- 172.21.15.46:0/2958061589 wait complete. 2024-09-10T07:43:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:26 smithi181 ceph-mon[35397]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 339 op/s 2024-09-10T07:43:26.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:26 smithi063 ceph-mon[35020]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 339 op/s 2024-09-10T07:43:26.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:26 smithi046 ceph-mon[30705]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 339 op/s 2024-09-10T07:43:26.983 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:43:27.372 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:27.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:27 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:43:27.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:27 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:43:27.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:27 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:43:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 -- 172.21.15.46:0/3382909890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3858104010 msgr2=0x7f38581043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 --2- 172.21.15.46:0/3382909890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3858104010 0x7f38581043f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3848009900 tx=0x7f384802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 -- 172.21.15.46:0/3382909890 shutdown_connections 2024-09-10T07:43:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 --2- 172.21.15.46:0/3382909890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3858109140 0x7f3858110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 --2- 172.21.15.46:0/3382909890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38581049c0 0x7f3858108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 --2- 172.21.15.46:0/3382909890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3858104010 0x7f38581043f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 -- 172.21.15.46:0/3382909890 >> 172.21.15.46:0/3382909890 conn(0x7f38580ffa50 msgr2=0x7f3858101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 -- 172.21.15.46:0/3382909890 shutdown_connections 2024-09-10T07:43:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.980+0000 7f385ea00640 1 -- 172.21.15.46:0/3382909890 wait complete. 2024-09-10T07:43:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.981+0000 7f385ea00640 1 Processor -- start 2024-09-10T07:43:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.981+0000 7f385ea00640 1 -- start start 2024-09-10T07:43:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f385ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3858104010 0x7f3858076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f385ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38581049c0 0x7f3858076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f385ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3858109140 0x7f3858077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f385ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3858114190 con 0x7f38581049c0 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f385ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3858114010 con 0x7f3858109140 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f385ea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3858114310 con 0x7f3858104010 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38581049c0 0x7f3858076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38581049c0 0x7f3858076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55354/0 (socket says 172.21.15.46:55354) 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857400640 1 -- 172.21.15.46:0/3986980531 learned_addr learned my addr 172.21.15.46:0/3986980531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3858104010 0x7f3858076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f385ca00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3858109140 0x7f3858077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857e00640 1 -- 172.21.15.46:0/3986980531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3858109140 msgr2=0x7f3858077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857e00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3858109140 0x7f3858077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857e00640 1 -- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38581049c0 msgr2=0x7f3858076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857e00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38581049c0 0x7f3858076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.982+0000 7f3857e00640 1 -- 172.21.15.46:0/3986980531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38581aaca0 con 0x7f3858104010 2024-09-10T07:43:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.983+0000 7f3857400640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38581049c0 0x7f3858076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.983+0000 7f3857e00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3858104010 0x7f3858076700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3848009230 tx=0x7f3848003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.983+0000 7f3854c00640 1 -- 172.21.15.46:0/3986980531 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384803f070 con 0x7f3858104010 2024-09-10T07:43:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.983+0000 7f385ea00640 1 -- 172.21.15.46:0/3986980531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38581aaf30 con 0x7f3858104010 2024-09-10T07:43:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.983+0000 7f3854c00640 1 -- 172.21.15.46:0/3986980531 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3848031c90 con 0x7f3858104010 2024-09-10T07:43:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.983+0000 7f3854c00640 1 -- 172.21.15.46:0/3986980531 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848035030 con 0x7f3858104010 2024-09-10T07:43:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.983+0000 7f385ea00640 1 -- 172.21.15.46:0/3986980531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38581ab3e0 con 0x7f3858104010 2024-09-10T07:43:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.986+0000 7f3861485740 1 Processor -- start 2024-09-10T07:43:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.986+0000 7f3861485740 1 -- start start 2024-09-10T07:43:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3854c00640 1 -- 172.21.15.46:0/3986980531 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3848042070 con 0x7f3858104010 2024-09-10T07:43:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3861485740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f7c70 0x55c43b1f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3861485740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f8590 0x55c43b202910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3861485740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b202e50 0x55c43b205240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3861485740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c43b15e3e0 con 0x55c43b202e50 2024-09-10T07:43:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3861485740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c43b105d80 con 0x55c43b1f7c70 2024-09-10T07:43:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3861485740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c43b15a3a0 con 0x55c43b1f8590 2024-09-10T07:43:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f385ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f7c70 0x55c43b1f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f385ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f7c70 0x55c43b1f8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35334/0 (socket says 172.21.15.46:35334) 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f385ca00640 1 -- 172.21.15.46:0/3197963526 learned_addr learned my addr 172.21.15.46:0/3197963526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3857e00640 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f8590 0x55c43b202910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3857400640 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b202e50 0x55c43b205240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f385ca00640 1 -- 172.21.15.46:0/3197963526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f8590 msgr2=0x55c43b202910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f385ca00640 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f8590 0x55c43b202910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f385ca00640 1 -- 172.21.15.46:0/3197963526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b202e50 msgr2=0x55c43b205240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f385ca00640 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b202e50 0x55c43b205240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f385ca00640 1 -- 172.21.15.46:0/3197963526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c43b124ae0 con 0x55c43b1f7c70 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.987+0000 7f3857400640 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b202e50 0x55c43b205240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.988+0000 7f385ca00640 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f7c70 0x55c43b1f8050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3850009f50 tx=0x7f38500118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad4edfb17558ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.988+0000 7f384d200640 1 -- 172.21.15.46:0/3197963526 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850004d20 con 0x55c43b1f7c70 2024-09-10T07:43:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.988+0000 7f384d200640 1 -- 172.21.15.46:0/3197963526 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3850004050 con 0x55c43b1f7c70 2024-09-10T07:43:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.988+0000 7f384d200640 1 -- 172.21.15.46:0/3197963526 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850005280 con 0x55c43b1f7c70 2024-09-10T07:43:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.988+0000 7f3854c00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f383c0a2b80 0x7f383c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.988+0000 7f3854c00640 1 -- 172.21.15.46:0/3986980531 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f38480ef880 con 0x7f3858104010 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3857e00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f383c0a2b80 0x7f383c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 -- 172.21.15.46:0/3197963526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f7c70 msgr2=0x55c43b1f8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f7c70 0x55c43b1f8050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3850009f50 tx=0x7f38500118d0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 -- 172.21.15.46:0/3197963526 shutdown_connections 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b202e50 0x55c43b205240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f8590 0x55c43b202910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 --2- 172.21.15.46:0/3197963526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f7c70 0x55c43b1f8050 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 -- 172.21.15.46:0/3197963526 >> 172.21.15.46:0/3197963526 conn(0x55c43b1f4200 msgr2=0x55c43b1f45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 -- 172.21.15.46:0/3197963526 shutdown_connections 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3861485740 1 -- 172.21.15.46:0/3197963526 wait complete. 2024-09-10T07:43:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.989+0000 7f3857e00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f383c0a2b80 0x7f383c0a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38480ed370 tx=0x7f38480f5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 1 -- start start 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 1 Processor -- start 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b200280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 0x55c43b2007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c43b205a20 con 0x55c43b1f7c70 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c43b1190f0 con 0x55c43b1f8590 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c43b205ba0 con 0x55c43b202e50 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.990+0000 7f3861485740 10 client.0 _ll_register_callbacks cb 0x55c43b100bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f385ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f3857400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b200280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f3857400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b200280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35336/0 (socket says 172.21.15.46:35336) 2024-09-10T07:43:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f3857400640 1 -- 172.21.15.46:0/256701553 learned_addr learned my addr 172.21.15.46:0/256701553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f385ca00640 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 0x55c43b2007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f3857400640 1 -- 172.21.15.46:0/256701553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 msgr2=0x55c43b2007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f3857400640 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 0x55c43b2007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f3857400640 1 -- 172.21.15.46:0/256701553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 msgr2=0x55c43b1ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f3857400640 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.991+0000 7f3857400640 1 -- 172.21.15.46:0/256701553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43aecf3c0 con 0x55c43b1f8590 2024-09-10T07:43:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.992+0000 7f3857400640 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b200280 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f38440129e0 tx=0x7f3844012ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7de85bb64cabd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.992+0000 7f3841600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.992+0000 7f3841600640 1 -- 172.21.15.46:0/256701553 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384400fa90 con 0x55c43b1f8590 2024-09-10T07:43:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.992+0000 7f3861485740 1 -- 172.21.15.46:0/256701553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3850004050 con 0x55c43b1f8590 2024-09-10T07:43:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.992+0000 7f3841600640 1 -- 172.21.15.46:0/256701553 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f384400fc30 con 0x55c43b1f8590 2024-09-10T07:43:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.992+0000 7f3841600640 1 -- 172.21.15.46:0/256701553 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3844024050 con 0x55c43b1f8590 2024-09-10T07:43:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.992+0000 7f3861485740 10 client.27116 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.992+0000 7f3861485740 1 -- 172.21.15.46:0/256701553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c43b211af0 con 0x55c43b1f8590 2024-09-10T07:43:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f3841600640 1 -- 172.21.15.46:0/256701553 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3850004050 con 0x55c43b1f8590 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f3841600640 1 -- 172.21.15.46:0/256701553 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f3844010430 con 0x55c43b1f8590 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f3861485740 10 client.27116 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f3861485740 10 client.27116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f3861485740 20 client.27116 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f3861485740 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55c43b213000 0x55c43b215460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f3861485740 4 client.27116 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f3861485740 1 -- 172.21.15.46:0/256701553 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c43b16b950 con 0x55c43b213000 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.993+0000 7f385ca00640 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55c43b213000 0x55c43b215460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.994+0000 7f385ca00640 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55c43b213000 0x55c43b215460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da45aef7c0b0ec12 server_cookie=c71fbf714e278667 in_seq=0 out_seq=0 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.994+0000 7f3841600640 10 client.27116 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3841600640 1 -- 172.21.15.46:0/256701553 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c43b16b950 con 0x55c43b213000 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3841600640 10 client.27116 handle_command_reply: tid=0 2024-09-10T07:43:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 client.27116 shutdown 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 -- 172.21.15.46:0/256701553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 msgr2=0x55c43b200280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b200280 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f38440129e0 tx=0x7f3844012ea0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 -- 172.21.15.46:0/256701553 shutdown_connections 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55c43b213000 0x55c43b215460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 0x55c43b2007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b200280 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 --2- 172.21.15.46:0/256701553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3841600640 0 client.27116 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 -- 172.21.15.46:0/256701553 >> 172.21.15.46:0/256701553 conn(0x55c43b1f4200 msgr2=0x55c43b1f3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 -- 172.21.15.46:0/256701553 shutdown_connections 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 1 -- 172.21.15.46:0/256701553 wait complete. 2024-09-10T07:43:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:27.995+0000 7f3861485740 20 client.27116 trim_cache size 0 max 16384 2024-09-10T07:43:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.009+0000 7f3861485740 1 Processor -- start 2024-09-10T07:43:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.009+0000 7f3861485740 1 -- start start 2024-09-10T07:43:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.009+0000 7f3861485740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.009+0000 7f3861485740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b201cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.009+0000 7f3861485740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 0x55c43b21ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.009+0000 7f3861485740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38581b52a0 con 0x55c43b1f7c70 2024-09-10T07:43:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.009+0000 7f3861485740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38581b6ed0 con 0x55c43b1f8590 2024-09-10T07:43:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.009+0000 7f3861485740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38581b5970 con 0x55c43b202e50 2024-09-10T07:43:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1fa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55394/0 (socket says 172.21.15.46:55394) 2024-09-10T07:43:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 -- 172.21.15.46:0/2472440354 learned_addr learned my addr 172.21.15.46:0/2472440354 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f3857400640 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b201cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 0x55c43b21ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 -- 172.21.15.46:0/2472440354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 msgr2=0x55c43b21ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 0x55c43b21ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 -- 172.21.15.46:0/2472440354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 msgr2=0x55c43b201cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b201cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 -- 172.21.15.46:0/2472440354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c43b1ef770 con 0x55c43b1f7c70 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.010+0000 7f385ca00640 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1fa6d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3850005c40 tx=0x7f385002a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eca0f638c4a6935 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3843e00640 1 -- 172.21.15.46:0/2472440354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850004c80 con 0x55c43b1f7c70 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3843e00640 1 -- 172.21.15.46:0/2472440354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3850004e20 con 0x55c43b1f7c70 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3843e00640 1 -- 172.21.15.46:0/2472440354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850005880 con 0x55c43b1f7c70 2024-09-10T07:43:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 -- 172.21.15.46:0/2472440354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 msgr2=0x55c43b1fa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1fa6d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3850005c40 tx=0x7f385002a040 comp rx=0 tx=0).stop 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 -- 172.21.15.46:0/2472440354 shutdown_connections 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b202e50 0x55c43b21ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b1f8590 0x55c43b201cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 --2- 172.21.15.46:0/2472440354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f7c70 0x55c43b1fa6d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 -- 172.21.15.46:0/2472440354 >> 172.21.15.46:0/2472440354 conn(0x55c43b1f4200 msgr2=0x55c43b2015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 -- 172.21.15.46:0/2472440354 shutdown_connections 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.011+0000 7f3861485740 1 -- 172.21.15.46:0/2472440354 wait complete. 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 1 -- start start 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 1 Processor -- start 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f7c70 0x55c43b1fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f8590 0x55c43b1fd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b202e50 0x55c43b1fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38581b6740 con 0x55c43b1f8590 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38581b6040 con 0x55c43b202e50 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38581b3280 con 0x55c43b1f7c70 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.012+0000 7f3861485740 10 client.0 _ll_register_callbacks cb 0x7f3844009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b202e50 0x55c43b1fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b202e50 0x55c43b1fd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35358/0 (socket says 172.21.15.46:35358) 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 -- 172.21.15.46:0/3728286713 learned_addr learned my addr 172.21.15.46:0/3728286713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f7c70 0x55c43b1fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f3857400640 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f8590 0x55c43b1fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 -- 172.21.15.46:0/3728286713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b202e50 msgr2=0x55c43b1fd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b202e50 0x55c43b1fd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 -- 172.21.15.46:0/3728286713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f8590 msgr2=0x55c43b1fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f8590 0x55c43b1fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 -- 172.21.15.46:0/3728286713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43b1f4e90 con 0x55c43b1f7c70 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f385ca00640 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f7c70 0x55c43b1fccd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3850011ed0 tx=0x7f3850005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=230cbca1a635d26f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.013+0000 7f3841600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.014+0000 7f3841600640 1 -- 172.21.15.46:0/3728286713 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850005200 con 0x55c43b1f7c70 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.014+0000 7f3861485740 1 -- 172.21.15.46:0/3728286713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3850004e20 con 0x55c43b1f7c70 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.014+0000 7f3841600640 1 -- 172.21.15.46:0/3728286713 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3850005740 con 0x55c43b1f7c70 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.014+0000 7f3841600640 1 -- 172.21.15.46:0/3728286713 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3850048050 con 0x55c43b1f7c70 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.014+0000 7f3861485740 10 client.27064 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.014+0000 7f3861485740 1 -- 172.21.15.46:0/3728286713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c43b27ddd0 con 0x55c43b1f7c70 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.014+0000 7f3841600640 1 -- 172.21.15.46:0/3728286713 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3850004e20 con 0x55c43b1f7c70 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3841600640 1 -- 172.21.15.46:0/3728286713 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f3850027740 con 0x55c43b1f7c70 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3861485740 10 client.27064 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3861485740 10 client.27064 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3861485740 20 client.27064 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3861485740 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55c43b27e2a0 0x55c43b280690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3861485740 4 client.27064 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:43:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3861485740 1 -- 172.21.15.46:0/3728286713 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c43b165e60 con 0x55c43b27e2a0 2024-09-10T07:43:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3857400640 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55c43b27e2a0 0x55c43b280690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3857400640 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55c43b27e2a0 0x55c43b280690 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39779629ce69a463 server_cookie=d25c1ee0f271784e in_seq=0 out_seq=0 2024-09-10T07:43:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.015+0000 7f3841600640 10 client.27064 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:43:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3841600640 1 -- 172.21.15.46:0/3728286713 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c43b165e60 con 0x55c43b27e2a0 2024-09-10T07:43:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3841600640 10 client.27064 handle_command_reply: tid=0 2024-09-10T07:43:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 client.27064 shutdown 2024-09-10T07:43:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 -- 172.21.15.46:0/3728286713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f7c70 msgr2=0x55c43b1fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f7c70 0x55c43b1fccd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3850011ed0 tx=0x7f3850005fd0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 -- 172.21.15.46:0/3728286713 shutdown_connections 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55c43b27e2a0 0x55c43b280690 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c43b202e50 0x55c43b1fd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3841600640 0 client.27064 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c43b1f8590 0x55c43b1fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 --2- 172.21.15.46:0/3728286713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c43b1f7c70 0x55c43b1fccd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 -- 172.21.15.46:0/3728286713 >> 172.21.15.46:0/3728286713 conn(0x55c43b1f4200 msgr2=0x55c43b1f3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 -- 172.21.15.46:0/3728286713 shutdown_connections 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 1 -- 172.21.15.46:0/3728286713 wait complete. 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.016+0000 7f3861485740 20 client.27064 trim_cache size 0 max 16384 2024-09-10T07:43:28.022 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:43:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 -- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f383c0a2b80 msgr2=0x7f383c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f383c0a2b80 0x7f383c0a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38480ed370 tx=0x7f38480f5550 comp rx=0 tx=0).stop 2024-09-10T07:43:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 -- 172.21.15.46:0/3986980531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3858104010 msgr2=0x7f3858076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3858104010 0x7f3858076700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3848009230 tx=0x7f3848003b20 comp rx=0 tx=0).stop 2024-09-10T07:43:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 -- 172.21.15.46:0/3986980531 shutdown_connections 2024-09-10T07:43:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3858109140 0x7f3858077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f383c0a2b80 0x7f383c0a5040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38581049c0 0x7f3858076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 --2- 172.21.15.46:0/3986980531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3858104010 0x7f3858076700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 -- 172.21.15.46:0/3986980531 >> 172.21.15.46:0/3986980531 conn(0x7f38580ffa50 msgr2=0x7f3858100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 -- 172.21.15.46:0/3986980531 shutdown_connections 2024-09-10T07:43:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:28.017+0000 7f385ea00640 1 -- 172.21.15.46:0/3986980531 wait complete. 2024-09-10T07:43:28.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:28 smithi181 ceph-mon[35397]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.6 MiB/s wr, 438 op/s 2024-09-10T07:43:28.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:28 smithi063 ceph-mon[35020]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.6 MiB/s wr, 438 op/s 2024-09-10T07:43:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:28 smithi046 ceph-mon[30705]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.6 MiB/s wr, 438 op/s 2024-09-10T07:43:28.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:43:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.923+0000 7f8739000640 1 -- 172.21.15.46:0/2420579874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8734112a30 msgr2=0x7f8734114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.923+0000 7f8739000640 1 --2- 172.21.15.46:0/2420579874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8734112a30 0x7f8734114e20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f872c009be0 tx=0x7f872c02a6c0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.923+0000 7f8739000640 1 -- 172.21.15.46:0/2420579874 shutdown_connections 2024-09-10T07:43:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.923+0000 7f8739000640 1 --2- 172.21.15.46:0/2420579874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8734112a30 0x7f8734114e20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.923+0000 7f8739000640 1 --2- 172.21.15.46:0/2420579874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8734077040 0x7f87341124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.923+0000 7f8739000640 1 --2- 172.21.15.46:0/2420579874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8734076720 0x7f8734076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.923+0000 7f8739000640 1 -- 172.21.15.46:0/2420579874 >> 172.21.15.46:0/2420579874 conn(0x7f87340ffa90 msgr2=0x7f8734101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:29.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.923+0000 7f8739000640 1 -- 172.21.15.46:0/2420579874 shutdown_connections 2024-09-10T07:43:29.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.924+0000 7f8739000640 1 -- 172.21.15.46:0/2420579874 wait complete. 2024-09-10T07:43:29.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.924+0000 7f8739000640 1 Processor -- start 2024-09-10T07:43:29.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8739000640 1 -- start start 2024-09-10T07:43:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8739000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8734076720 0x7f87341a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8739000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8734077040 0x7f87341a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8739000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8734112a30 0x7f873419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8739000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8734118520 con 0x7f8734112a30 2024-09-10T07:43:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8739000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f87341183a0 con 0x7f8734077040 2024-09-10T07:43:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8739000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87341186a0 con 0x7f8734076720 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8732800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8734112a30 0x7f873419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8732800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8734112a30 0x7f873419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55424/0 (socket says 172.21.15.46:55424) 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.925+0000 7f8732800640 1 -- 172.21.15.46:0/174745663 learned_addr learned my addr 172.21.15.46:0/174745663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.926+0000 7f8731400640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8734077040 0x7f87341a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.926+0000 7f8731e00640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8734076720 0x7f87341a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.926+0000 7f8732800640 1 -- 172.21.15.46:0/174745663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8734076720 msgr2=0x7f87341a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.926+0000 7f8732800640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8734076720 0x7f87341a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.926+0000 7f8732800640 1 -- 172.21.15.46:0/174745663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8734077040 msgr2=0x7f87341a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.926+0000 7f8732800640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8734077040 0x7f87341a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.926+0000 7f8732800640 1 -- 172.21.15.46:0/174745663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873419f060 con 0x7f8734112a30 2024-09-10T07:43:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.926+0000 7f8732800640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8734112a30 0x7f873419e930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f872c034040 tx=0x7f872c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.927+0000 7f872aa00640 1 -- 172.21.15.46:0/174745663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c041020 con 0x7f8734112a30 2024-09-10T07:43:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.927+0000 7f872aa00640 1 -- 172.21.15.46:0/174745663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f872c037040 con 0x7f8734112a30 2024-09-10T07:43:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.927+0000 7f872aa00640 1 -- 172.21.15.46:0/174745663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c036070 con 0x7f8734112a30 2024-09-10T07:43:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.927+0000 7f8739000640 1 -- 172.21.15.46:0/174745663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873419f2f0 con 0x7f8734112a30 2024-09-10T07:43:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.927+0000 7f8739000640 1 -- 172.21.15.46:0/174745663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87341ab640 con 0x7f8734112a30 2024-09-10T07:43:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.930+0000 7f872aa00640 1 -- 172.21.15.46:0/174745663 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f872c044020 con 0x7f8734112a30 2024-09-10T07:43:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.930+0000 7f873ba4c740 1 Processor -- start 2024-09-10T07:43:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.930+0000 7f873ba4c740 1 -- start start 2024-09-10T07:43:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc86944050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc86951240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f873ba4c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bc868aa3e0 con 0x55bc86943c70 2024-09-10T07:43:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f873ba4c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bc86851d80 con 0x55bc86944590 2024-09-10T07:43:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f873ba4c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bc868a63a0 con 0x55bc8694ee50 2024-09-10T07:43:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc86944050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35388/0 (socket says 172.21.15.46:35388) 2024-09-10T07:43:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731e00640 1 -- 172.21.15.46:0/1882364979 learned_addr learned my addr 172.21.15.46:0/1882364979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731400640 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc86951240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731e00640 1 -- 172.21.15.46:0/1882364979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 msgr2=0x55bc86951240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731e00640 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc86951240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731e00640 1 -- 172.21.15.46:0/1882364979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 msgr2=0x55bc86944050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731e00640 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc86944050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.931+0000 7f8731e00640 1 -- 172.21.15.46:0/1882364979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc86870ae0 con 0x55bc86944590 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.932+0000 7f8731400640 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc86951240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.932+0000 7f872aa00640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87180a2c00 0x7f87180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.932+0000 7f8731e00640 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694e910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8724009c90 tx=0x7f8724011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514664cf83f0ac72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.932+0000 7f872aa00640 1 -- 172.21.15.46:0/174745663 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f872c0efdb0 con 0x7f8734112a30 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.932+0000 7f871f400640 1 -- 172.21.15.46:0/1882364979 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724004230 con 0x55bc86944590 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.932+0000 7f8731400640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87180a2c00 0x7f87180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.932+0000 7f871f400640 1 -- 172.21.15.46:0/1882364979 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87240043d0 con 0x55bc86944590 2024-09-10T07:43:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.932+0000 7f871f400640 1 -- 172.21.15.46:0/1882364979 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87240197c0 con 0x55bc86944590 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 -- 172.21.15.46:0/1882364979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 msgr2=0x55bc8694e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694e910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8724009c90 tx=0x7f8724011810 comp rx=0 tx=0).stop 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f8731400640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87180a2c00 0x7f87180a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55bc86941580 tx=0x7f87200096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 -- 172.21.15.46:0/1882364979 shutdown_connections 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc86951240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694e910 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 --2- 172.21.15.46:0/1882364979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc86944050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 -- 172.21.15.46:0/1882364979 >> 172.21.15.46:0/1882364979 conn(0x55bc86940200 msgr2=0x55bc869405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 -- 172.21.15.46:0/1882364979 shutdown_connections 2024-09-10T07:43:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.933+0000 7f873ba4c740 1 -- 172.21.15.46:0/1882364979 wait complete. 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.934+0000 7f873ba4c740 1 -- start start 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.934+0000 7f873ba4c740 1 Processor -- start 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.934+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc8694bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.934+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.934+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc8694c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.934+0000 7f873ba4c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bc86951a20 con 0x55bc86943c70 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.934+0000 7f873ba4c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bc868650f0 con 0x55bc86944590 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.934+0000 7f873ba4c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bc86951ba0 con 0x55bc8694ee50 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f873ba4c740 10 client.0 _ll_register_callbacks cb 0x55bc8684cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc8694bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc8694bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55442/0 (socket says 172.21.15.46:55442) 2024-09-10T07:43:29.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731e00640 1 -- 172.21.15.46:0/288371437 learned_addr learned my addr 172.21.15.46:0/288371437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8732800640 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc8694c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731400640 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731e00640 1 -- 172.21.15.46:0/288371437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 msgr2=0x55bc8694c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731e00640 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc8694c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731e00640 1 -- 172.21.15.46:0/288371437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 msgr2=0x55bc8694c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731e00640 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8731e00640 1 -- 172.21.15.46:0/288371437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc8661b3c0 con 0x55bc86943c70 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.935+0000 7f8732800640 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc8694c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f8731e00640 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc8694bcf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87240052a0 tx=0x7f87240052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f79f61e6239fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f8713e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f8713e00640 1 -- 172.21.15.46:0/288371437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724028020 con 0x55bc86943c70 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f8713e00640 1 -- 172.21.15.46:0/288371437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f87240035f0 con 0x55bc86943c70 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f873ba4c740 1 -- 172.21.15.46:0/288371437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87240043d0 con 0x55bc86943c70 2024-09-10T07:43:29.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f8713e00640 1 -- 172.21.15.46:0/288371437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8724033050 con 0x55bc86943c70 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f873ba4c740 10 client.16692 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f873ba4c740 1 -- 172.21.15.46:0/288371437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bc8695d980 con 0x55bc86943c70 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.936+0000 7f8713e00640 1 -- 172.21.15.46:0/288371437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87240043d0 con 0x55bc86943c70 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.937+0000 7f8713e00640 1 -- 172.21.15.46:0/288371437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f8724018070 con 0x55bc86943c70 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.937+0000 7f873ba4c740 10 client.16692 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.937+0000 7f873ba4c740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.937+0000 7f873ba4c740 20 client.16692 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.937+0000 7f873ba4c740 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bc8695ee40 0x55bc86961230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.937+0000 7f873ba4c740 4 client.16692 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.937+0000 7f873ba4c740 1 -- 172.21.15.46:0/288371437 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc868b7950 con 0x55bc8695ee40 2024-09-10T07:43:29.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.937+0000 7f8731400640 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bc8695ee40 0x55bc86961230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.938+0000 7f8731400640 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bc8695ee40 0x55bc86961230 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f7fb15ad2bddf4 server_cookie=310bb1c9295bf4b5 in_seq=0 out_seq=0 2024-09-10T07:43:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.938+0000 7f8713e00640 10 client.16692 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:29.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.938+0000 7f8713e00640 1 -- 172.21.15.46:0/288371437 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc868b7950 con 0x55bc8695ee40 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.938+0000 7f8713e00640 10 client.16692 handle_command_reply: tid=0 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 client.16692 shutdown 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 -- 172.21.15.46:0/288371437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 msgr2=0x55bc8694bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc8694bcf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87240052a0 tx=0x7f87240052d0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 -- 172.21.15.46:0/288371437 shutdown_connections 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bc8695ee40 0x55bc86961230 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc8694ee50 0x55bc8694c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86944590 0x55bc8694c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 --2- 172.21.15.46:0/288371437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86943c70 0x55bc8694bcf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 -- 172.21.15.46:0/288371437 >> 172.21.15.46:0/288371437 conn(0x55bc86940200 msgr2=0x55bc86945f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f8713e00640 0 client.16692 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 -- 172.21.15.46:0/288371437 shutdown_connections 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 1 -- 172.21.15.46:0/288371437 wait complete. 2024-09-10T07:43:29.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.939+0000 7f873ba4c740 20 client.16692 trim_cache size 0 max 16384 2024-09-10T07:43:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.953+0000 7f873ba4c740 1 Processor -- start 2024-09-10T07:43:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.953+0000 7f873ba4c740 1 -- start start 2024-09-10T07:43:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.953+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86943c70 0x55bc869466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.953+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86944590 0x55bc86966ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.953+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc8694ee50 0x55bc86967020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f873ba4c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87341b52c0 con 0x55bc86944590 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f873ba4c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f87341b6ef0 con 0x55bc8694ee50 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f873ba4c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87341b5990 con 0x55bc86943c70 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86944590 0x55bc86966ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86944590 0x55bc86966ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55446/0 (socket says 172.21.15.46:55446) 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86943c70 0x55bc869466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731400640 1 -- 172.21.15.46:0/3791987611 learned_addr learned my addr 172.21.15.46:0/3791987611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8732800640 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc8694ee50 0x55bc86967020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731e00640 1 -- 172.21.15.46:0/3791987611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc8694ee50 msgr2=0x55bc86967020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731e00640 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc8694ee50 0x55bc86967020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731e00640 1 -- 172.21.15.46:0/3791987611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86944590 msgr2=0x55bc86966ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731e00640 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86944590 0x55bc86966ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731e00640 1 -- 172.21.15.46:0/3791987611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc86940030 con 0x55bc86943c70 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.954+0000 7f8731400640 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86944590 0x55bc86966ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f8731e00640 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86943c70 0x55bc869466d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8724011850 tx=0x7f8724011d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9958b9c4bab7a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f871ea00640 1 -- 172.21.15.46:0/3791987611 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724028050 con 0x55bc86943c70 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f871ea00640 1 -- 172.21.15.46:0/3791987611 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8724009b10 con 0x55bc86943c70 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f871ea00640 1 -- 172.21.15.46:0/3791987611 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872401f040 con 0x55bc86943c70 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f873ba4c740 1 -- 172.21.15.46:0/3791987611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86943c70 msgr2=0x55bc869466d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f873ba4c740 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86943c70 0x55bc869466d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8724011850 tx=0x7f8724011d50 comp rx=0 tx=0).stop 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f873ba4c740 1 -- 172.21.15.46:0/3791987611 shutdown_connections 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f873ba4c740 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc8694ee50 0x55bc86967020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f873ba4c740 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc86944590 0x55bc86966ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f873ba4c740 1 --2- 172.21.15.46:0/3791987611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86943c70 0x55bc869466d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.955+0000 7f873ba4c740 1 -- 172.21.15.46:0/3791987611 >> 172.21.15.46:0/3791987611 conn(0x55bc86940200 msgr2=0x55bc86947030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.956+0000 7f873ba4c740 1 -- 172.21.15.46:0/3791987611 shutdown_connections 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.956+0000 7f873ba4c740 1 -- 172.21.15.46:0/3791987611 wait complete. 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.956+0000 7f873ba4c740 1 -- start start 2024-09-10T07:43:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.956+0000 7f873ba4c740 1 Processor -- start 2024-09-10T07:43:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86943c70 0x55bc86951db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86944590 0x55bc869522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f873ba4c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc8694ee50 0x55bc86952830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f873ba4c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87341b6760 con 0x55bc8694ee50 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f873ba4c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f87341b6060 con 0x55bc86943c70 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f873ba4c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87341b32a0 con 0x55bc86944590 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f873ba4c740 10 client.0 _ll_register_callbacks cb 0x7f87200097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8732800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc8694ee50 0x55bc86952830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86944590 0x55bc869522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86944590 0x55bc869522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39888/0 (socket says 172.21.15.46:39888) 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86943c70 0x55bc86951db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731400640 1 -- 172.21.15.46:0/2652463335 learned_addr learned my addr 172.21.15.46:0/2652463335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8732800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc8694ee50 0x55bc86952830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55456/0 (socket says 172.21.15.46:55456) 2024-09-10T07:43:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731400640 1 -- 172.21.15.46:0/2652463335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86943c70 msgr2=0x55bc86951db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731400640 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86943c70 0x55bc86951db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731400640 1 -- 172.21.15.46:0/2652463335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc8694ee50 msgr2=0x55bc86952830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731400640 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc8694ee50 0x55bc86952830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.957+0000 7f8731400640 1 -- 172.21.15.46:0/2652463335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc869c7aa0 con 0x55bc86944590 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.958+0000 7f8731400640 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86944590 0x55bc869522f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8734079390 tx=0x7f8720004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc2701e0f97d9b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.958+0000 7f8713e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.958+0000 7f8713e00640 1 -- 172.21.15.46:0/2652463335 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872002f070 con 0x55bc86944590 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.958+0000 7f873ba4c740 1 -- 172.21.15.46:0/2652463335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8724009b10 con 0x55bc86944590 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.958+0000 7f8713e00640 1 -- 172.21.15.46:0/2652463335 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f872001edc0 con 0x55bc86944590 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f8713e00640 1 -- 172.21.15.46:0/2652463335 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f872003b050 con 0x55bc86944590 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f873ba4c740 10 client.27079 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f873ba4c740 1 -- 172.21.15.46:0/2652463335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bc869c8e10 con 0x55bc86944590 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f8713e00640 1 -- 172.21.15.46:0/2652463335 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8724009b10 con 0x55bc86944590 2024-09-10T07:43:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f8713e00640 1 -- 172.21.15.46:0/2652463335 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f87200047a0 con 0x55bc86944590 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f873ba4c740 10 client.27079 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f873ba4c740 10 client.27079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f873ba4c740 20 client.27079 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f873ba4c740 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bc869c96b0 0x55bc869cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f873ba4c740 4 client.27079 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f873ba4c740 1 -- 172.21.15.46:0/2652463335 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc868b1e60 con 0x55bc869c96b0 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.959+0000 7f8731e00640 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bc869c96b0 0x55bc869cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.960+0000 7f8731e00640 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bc869c96b0 0x55bc869cbad0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7c912ef04ecb7f server_cookie=6ad29281a1d853c0 in_seq=0 out_seq=0 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.960+0000 7f8713e00640 10 client.27079 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.960+0000 7f8713e00640 1 -- 172.21.15.46:0/2652463335 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc868b1e60 con 0x55bc869c96b0 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.960+0000 7f8713e00640 10 client.27079 handle_command_reply: tid=0 2024-09-10T07:43:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.960+0000 7f873ba4c740 1 client.27079 shutdown 2024-09-10T07:43:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.960+0000 7f873ba4c740 1 -- 172.21.15.46:0/2652463335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86944590 msgr2=0x55bc869522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.960+0000 7f873ba4c740 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86944590 0x55bc869522f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8734079390 tx=0x7f8720004730 comp rx=0 tx=0).stop 2024-09-10T07:43:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 1 -- 172.21.15.46:0/2652463335 shutdown_connections 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bc869c96b0 0x55bc869cbad0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc8694ee50 0x55bc86952830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bc86944590 0x55bc869522f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 1 --2- 172.21.15.46:0/2652463335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc86943c70 0x55bc86951db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f8713e00640 0 client.27079 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 1 -- 172.21.15.46:0/2652463335 >> 172.21.15.46:0/2652463335 conn(0x55bc86940200 msgr2=0x55bc8693eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 1 -- 172.21.15.46:0/2652463335 shutdown_connections 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 1 -- 172.21.15.46:0/2652463335 wait complete. 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.961+0000 7f873ba4c740 20 client.27079 trim_cache size 0 max 16384 2024-09-10T07:43:29.964 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:43:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 -- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87180a2c00 msgr2=0x7f87180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87180a2c00 0x7f87180a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55bc86941580 tx=0x7f87200096c0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 -- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8734112a30 msgr2=0x7f873419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8734112a30 0x7f873419e930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f872c034040 tx=0x7f872c0305d0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 -- 172.21.15.46:0/174745663 shutdown_connections 2024-09-10T07:43:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87180a2c00 0x7f87180a50c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8734112a30 0x7f873419e930 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8734077040 0x7f87341a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 --2- 172.21.15.46:0/174745663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8734076720 0x7f87341a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 -- 172.21.15.46:0/174745663 >> 172.21.15.46:0/174745663 conn(0x7f87340ffa90 msgr2=0x7f8734101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 -- 172.21.15.46:0/174745663 shutdown_connections 2024-09-10T07:43:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:29.962+0000 7f8739000640 1 -- 172.21.15.46:0/174745663 wait complete. 2024-09-10T07:43:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:30 smithi181 ceph-mon[35397]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 369 op/s 2024-09-10T07:43:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:30 smithi063 ceph-mon[35020]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 369 op/s 2024-09-10T07:43:30.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:30 smithi046 ceph-mon[30705]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 369 op/s 2024-09-10T07:43:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 370.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:43:32.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:43:32.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:43:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:32 smithi181 ceph-mon[35397]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 356 op/s 2024-09-10T07:43:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:32 smithi063 ceph-mon[35020]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 356 op/s 2024-09-10T07:43:32.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:32 smithi046 ceph-mon[30705]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 356 op/s 2024-09-10T07:43:32.452 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.069+0000 7f5348c00640 1 -- 172.21.15.46:0/528658432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344112870 msgr2=0x7f5344114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.069+0000 7f5348c00640 1 --2- 172.21.15.46:0/528658432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344112870 0x7f5344114c60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f533c009ca0 tx=0x7f533c02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.070+0000 7f5348c00640 1 -- 172.21.15.46:0/528658432 shutdown_connections 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.070+0000 7f5348c00640 1 --2- 172.21.15.46:0/528658432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344112870 0x7f5344114c60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.070+0000 7f5348c00640 1 --2- 172.21.15.46:0/528658432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5344077040 0x7f5344112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.070+0000 7f5348c00640 1 --2- 172.21.15.46:0/528658432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5344076720 0x7f5344076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.070+0000 7f5348c00640 1 -- 172.21.15.46:0/528658432 >> 172.21.15.46:0/528658432 conn(0x7f53440ff8d0 msgr2=0x7f5344101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.070+0000 7f5348c00640 1 -- 172.21.15.46:0/528658432 shutdown_connections 2024-09-10T07:43:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.070+0000 7f5348c00640 1 -- 172.21.15.46:0/528658432 wait complete. 2024-09-10T07:43:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.071+0000 7f5348c00640 1 Processor -- start 2024-09-10T07:43:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.071+0000 7f5348c00640 1 -- start start 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5348c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344076720 0x7f53441a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5348c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5344077040 0x7f53441a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5348c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5344112870 0x7f534419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5348c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5344118420 con 0x7f5344076720 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5348c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53441182a0 con 0x7f5344112870 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5348c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53441185a0 con 0x7f5344077040 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5343400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344076720 0x7f53441a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5343400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344076720 0x7f53441a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55476/0 (socket says 172.21.15.46:55476) 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5343400640 1 -- 172.21.15.46:0/615099121 learned_addr learned my addr 172.21.15.46:0/615099121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f533be00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5344077040 0x7f53441a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.072+0000 7f5343e00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5344112870 0x7f534419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5343400640 1 -- 172.21.15.46:0/615099121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5344077040 msgr2=0x7f53441a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5343400640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5344077040 0x7f53441a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5343400640 1 -- 172.21.15.46:0/615099121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5344112870 msgr2=0x7f534419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5343400640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5344112870 0x7f534419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5343400640 1 -- 172.21.15.46:0/615099121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534419ef40 con 0x7f5344076720 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5343400640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344076720 0x7f53441a4600 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f533400e9f0 tx=0x7f533400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5340c00640 1 -- 172.21.15.46:0/615099121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533400cd80 con 0x7f5344076720 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5340c00640 1 -- 172.21.15.46:0/615099121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5334010910 con 0x7f5344076720 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5348c00640 1 -- 172.21.15.46:0/615099121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534419f230 con 0x7f5344076720 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5348c00640 1 -- 172.21.15.46:0/615099121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53441ab580 con 0x7f5344076720 2024-09-10T07:43:33.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.073+0000 7f5340c00640 1 -- 172.21.15.46:0/615099121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334018930 con 0x7f5344076720 2024-09-10T07:43:33.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f534ce74740 1 Processor -- start 2024-09-10T07:43:33.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f534ce74740 1 -- start start 2024-09-10T07:43:33.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f534ce74740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db300cb0 0x5596db301090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f534ce74740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db30b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f534ce74740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db30be90 0x5596db30e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f534ce74740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596db267290 con 0x5596db300cb0 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f534ce74740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5596db20eb40 con 0x5596db30be90 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f534ce74740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5596db263250 con 0x5596db3015d0 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f5340c00640 1 -- 172.21.15.46:0/615099121 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5334010ac0 con 0x7f5344076720 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.077+0000 7f5343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db300cb0 0x5596db301090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db300cb0 0x5596db301090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55488/0 (socket says 172.21.15.46:55488) 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343e00640 1 -- 172.21.15.46:0/1804721090 learned_addr learned my addr 172.21.15.46:0/1804721090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f533be00640 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db30be90 0x5596db30e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343400640 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db30b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343e00640 1 -- 172.21.15.46:0/1804721090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 msgr2=0x5596db30b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343e00640 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db30b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343e00640 1 -- 172.21.15.46:0/1804721090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db30be90 msgr2=0x5596db30e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343e00640 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db30be90 0x5596db30e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343e00640 1 -- 172.21.15.46:0/1804721090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596dafd86a0 con 0x5596db300cb0 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.078+0000 7f5343e00640 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db300cb0 0x5596db301090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f534407aa10 tx=0x7f533c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b277b59dd88d1ded server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.079+0000 7f5333e00640 1 -- 172.21.15.46:0/1804721090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c031db0 con 0x5596db300cb0 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.079+0000 7f5333e00640 1 -- 172.21.15.46:0/1804721090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c036030 con 0x5596db300cb0 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.079+0000 7f5333e00640 1 -- 172.21.15.46:0/1804721090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c004280 con 0x5596db300cb0 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.079+0000 7f5340c00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53280a2b80 0x7f53280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.079+0000 7f5340c00640 1 -- 172.21.15.46:0/615099121 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f53340cf6e0 con 0x7f5344076720 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.079+0000 7f5343400640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53280a2b80 0x7f53280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 -- 172.21.15.46:0/1804721090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db300cb0 msgr2=0x5596db301090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db300cb0 0x5596db301090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f534407aa10 tx=0x7f533c005f80 comp rx=0 tx=0).stop 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 -- 172.21.15.46:0/1804721090 shutdown_connections 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db30be90 0x5596db30e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db30b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 --2- 172.21.15.46:0/1804721090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db300cb0 0x5596db301090 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 -- 172.21.15.46:0/1804721090 >> 172.21.15.46:0/1804721090 conn(0x5596db2fd240 msgr2=0x5596db2fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 -- 172.21.15.46:0/1804721090 shutdown_connections 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 -- 172.21.15.46:0/1804721090 wait complete. 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f5343400640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53280a2b80 0x7f53280a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f53340985e0 tx=0x7f5334018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.080+0000 7f534ce74740 1 -- start start 2024-09-10T07:43:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f534ce74740 1 Processor -- start 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f534ce74740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 0x5596db30b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f534ce74740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db308a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f534ce74740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db308fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f534ce74740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596db30e980 con 0x5596db30be90 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f534ce74740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5596db221fa0 con 0x5596db300cb0 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f534ce74740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5596db30eb00 con 0x5596db3015d0 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f534ce74740 10 client.0 _ll_register_callbacks cb 0x5596daf75ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.081+0000 7f5343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db308fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f5343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db308fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55502/0 (socket says 172.21.15.46:55502) 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f5343e00640 1 -- 172.21.15.46:0/2513438300 learned_addr learned my addr 172.21.15.46:0/2513438300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f533be00640 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db308a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f5343e00640 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 0x5596db30b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f533be00640 1 -- 172.21.15.46:0/2513438300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 msgr2=0x5596db30b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f533be00640 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 0x5596db30b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f533be00640 1 -- 172.21.15.46:0/2513438300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 msgr2=0x5596db308fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f533be00640 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db308fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f533be00640 1 -- 172.21.15.46:0/2513438300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596db3097a0 con 0x5596db3015d0 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f5343e00640 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db308fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f533be00640 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db308a60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f534419ff20 tx=0x7f532c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93602b3b4fee546a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.082+0000 7f5330c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.083+0000 7f5330c00640 1 -- 172.21.15.46:0/2513438300 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c004030 con 0x5596db3015d0 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.083+0000 7f5330c00640 1 -- 172.21.15.46:0/2513438300 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f532c0041d0 con 0x5596db3015d0 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.083+0000 7f534ce74740 1 -- 172.21.15.46:0/2513438300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f533c036030 con 0x5596db3015d0 2024-09-10T07:43:33.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.083+0000 7f5330c00640 1 -- 172.21.15.46:0/2513438300 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f532c025050 con 0x5596db3015d0 2024-09-10T07:43:33.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.083+0000 7f534ce74740 10 client.27091 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:33.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.083+0000 7f534ce74740 1 -- 172.21.15.46:0/2513438300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5596db31a8c0 con 0x5596db3015d0 2024-09-10T07:43:33.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.083+0000 7f5330c00640 1 -- 172.21.15.46:0/2513438300 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c036030 con 0x5596db3015d0 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.084+0000 7f5330c00640 1 -- 172.21.15.46:0/2513438300 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f532c003a40 con 0x5596db3015d0 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.084+0000 7f534ce74740 10 client.27091 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.084+0000 7f534ce74740 10 client.27091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.084+0000 7f534ce74740 20 client.27091 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.084+0000 7f534ce74740 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596db31bdf0 0x5596db31e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.084+0000 7f534ce74740 4 client.27091 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.084+0000 7f534ce74740 1 -- 172.21.15.46:0/2513438300 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596db274800 con 0x5596db31bdf0 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.084+0000 7f5343e00640 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596db31bdf0 0x5596db31e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.085+0000 7f5343e00640 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596db31bdf0 0x5596db31e210 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdaec69044252d9f server_cookie=540219125b71139e in_seq=0 out_seq=0 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.085+0000 7f5330c00640 10 client.27091 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.085+0000 7f5330c00640 1 -- 172.21.15.46:0/2513438300 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596db274800 con 0x5596db31bdf0 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.085+0000 7f5330c00640 10 client.27091 handle_command_reply: tid=0 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.085+0000 7f534ce74740 1 client.27091 shutdown 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 -- 172.21.15.46:0/2513438300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 msgr2=0x5596db308a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db308a60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f534419ff20 tx=0x7f532c012b60 comp rx=0 tx=0).stop 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 -- 172.21.15.46:0/2513438300 shutdown_connections 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596db31bdf0 0x5596db31e210 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db308fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db3015d0 0x5596db308a60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 --2- 172.21.15.46:0/2513438300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 0x5596db30b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f5330c00640 0 client.27091 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 -- 172.21.15.46:0/2513438300 >> 172.21.15.46:0/2513438300 conn(0x5596db2fd240 msgr2=0x5596db302f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 -- 172.21.15.46:0/2513438300 shutdown_connections 2024-09-10T07:43:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 1 -- 172.21.15.46:0/2513438300 wait complete. 2024-09-10T07:43:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.086+0000 7f534ce74740 20 client.27091 trim_cache size 0 max 16384 2024-09-10T07:43:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.104+0000 7f534ce74740 1 Processor -- start 2024-09-10T07:43:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.104+0000 7f534ce74740 1 -- start start 2024-09-10T07:43:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f534ce74740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 0x5596db304000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f534ce74740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db3015d0 0x5596db323b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f534ce74740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db30be90 0x5596db324070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f534ce74740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53441b5110 con 0x5596db3015d0 2024-09-10T07:43:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f534ce74740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53441b6d40 con 0x5596db300cb0 2024-09-10T07:43:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f534ce74740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53441b57e0 con 0x5596db30be90 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f533be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db3015d0 0x5596db323b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f533be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db3015d0 0x5596db323b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55516/0 (socket says 172.21.15.46:55516) 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f533be00640 1 -- 172.21.15.46:0/2099435718 learned_addr learned my addr 172.21.15.46:0/2099435718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f5343e00640 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 0x5596db304000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f5343e00640 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db30be90 0x5596db324070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f533be00640 1 -- 172.21.15.46:0/2099435718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db30be90 msgr2=0x5596db324070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f533be00640 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db30be90 0x5596db324070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f533be00640 1 -- 172.21.15.46:0/2099435718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 msgr2=0x5596db304000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f533be00640 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 0x5596db304000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.105+0000 7f533be00640 1 -- 172.21.15.46:0/2099435718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596db2fded0 con 0x5596db3015d0 2024-09-10T07:43:33.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.106+0000 7f533be00640 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db3015d0 0x5596db323b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f532c0059f0 tx=0x7f532c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb45d0f55c53a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.106+0000 7f5333400640 1 -- 172.21.15.46:0/2099435718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c02e070 con 0x5596db3015d0 2024-09-10T07:43:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.106+0000 7f5333400640 1 -- 172.21.15.46:0/2099435718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532c00fe00 con 0x5596db3015d0 2024-09-10T07:43:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.106+0000 7f5333400640 1 -- 172.21.15.46:0/2099435718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c022040 con 0x5596db3015d0 2024-09-10T07:43:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.106+0000 7f534ce74740 1 -- 172.21.15.46:0/2099435718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db3015d0 msgr2=0x5596db323b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db3015d0 0x5596db323b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f532c0059f0 tx=0x7f532c00f850 comp rx=0 tx=0).stop 2024-09-10T07:43:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 -- 172.21.15.46:0/2099435718 shutdown_connections 2024-09-10T07:43:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db30be90 0x5596db324070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db3015d0 0x5596db323b30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 --2- 172.21.15.46:0/2099435718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db300cb0 0x5596db304000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 -- 172.21.15.46:0/2099435718 >> 172.21.15.46:0/2099435718 conn(0x5596db2fd240 msgr2=0x5596db3038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 -- 172.21.15.46:0/2099435718 shutdown_connections 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 -- 172.21.15.46:0/2099435718 wait complete. 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 -- start start 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.107+0000 7f534ce74740 1 Processor -- start 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f534ce74740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db300cb0 0x5596db30fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f534ce74740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db3015d0 0x5596db310390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f534ce74740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db385b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f534ce74740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53441b65b0 con 0x5596db30be90 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f534ce74740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53441b5eb0 con 0x5596db3015d0 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f534ce74740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53441b30f0 con 0x5596db300cb0 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f534ce74740 10 client.0 _ll_register_callbacks cb 0x7f532c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f5343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db385b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f533be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db3015d0 0x5596db310390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f533be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db3015d0 0x5596db310390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35474/0 (socket says 172.21.15.46:35474) 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f533be00640 1 -- 172.21.15.46:0/3302313327 learned_addr learned my addr 172.21.15.46:0/3302313327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f5343e00640 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db385b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55530/0 (socket says 172.21.15.46:55530) 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.108+0000 7f5343e00640 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db300cb0 0x5596db30fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f533be00640 1 -- 172.21.15.46:0/3302313327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db300cb0 msgr2=0x5596db30fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f533be00640 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db300cb0 0x5596db30fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f533be00640 1 -- 172.21.15.46:0/3302313327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 msgr2=0x5596db385b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f533be00640 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db385b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f533be00640 1 -- 172.21.15.46:0/3302313327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596db310c90 con 0x5596db3015d0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f533be00640 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db3015d0 0x5596db310390 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f532c0126a0 tx=0x7f532c00fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b621c0d36b70bfa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f5330c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f5330c00640 1 -- 172.21.15.46:0/3302313327 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c02e030 con 0x5596db3015d0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f534ce74740 1 -- 172.21.15.46:0/3302313327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532c00fe00 con 0x5596db3015d0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.109+0000 7f5330c00640 1 -- 172.21.15.46:0/3302313327 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f532c018040 con 0x5596db3015d0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f5330c00640 1 -- 172.21.15.46:0/3302313327 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f532c03b050 con 0x5596db3015d0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f534ce74740 10 client.27158 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f534ce74740 1 -- 172.21.15.46:0/3302313327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5596db3871b0 con 0x5596db3015d0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f5330c00640 1 -- 172.21.15.46:0/3302313327 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532c00fe00 con 0x5596db3015d0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f5330c00640 1 -- 172.21.15.46:0/3302313327 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f532c0354f0 con 0x5596db3015d0 2024-09-10T07:43:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f534ce74740 10 client.27158 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f534ce74740 10 client.27158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f534ce74740 20 client.27158 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f534ce74740 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596db387930 0x5596db389d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f534ce74740 4 client.27158 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:43:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.110+0000 7f534ce74740 1 -- 172.21.15.46:0/3302313327 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5596db26ed10 con 0x5596db387930 2024-09-10T07:43:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.111+0000 7f5343e00640 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596db387930 0x5596db389d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.111+0000 7f5343e00640 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596db387930 0x5596db389d50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ff6f1469aef422 server_cookie=bbc2a79be20c14e1 in_seq=0 out_seq=0 2024-09-10T07:43:33.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.111+0000 7f5330c00640 10 client.27158 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:33.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.112+0000 7f5330c00640 1 -- 172.21.15.46:0/3302313327 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5596db26ed10 con 0x5596db387930 2024-09-10T07:43:33.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.112+0000 7f5330c00640 10 client.27158 handle_command_reply: tid=0 2024-09-10T07:43:33.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.112+0000 7f534ce74740 1 client.27158 shutdown 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.112+0000 7f534ce74740 1 -- 172.21.15.46:0/3302313327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db3015d0 msgr2=0x5596db310390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.112+0000 7f534ce74740 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db3015d0 0x5596db310390 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f532c0126a0 tx=0x7f532c00fb50 comp rx=0 tx=0).stop 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.113+0000 7f534ce74740 1 -- 172.21.15.46:0/3302313327 shutdown_connections 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.113+0000 7f534ce74740 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596db387930 0x5596db389d50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.113+0000 7f534ce74740 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596db30be90 0x5596db385b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.113+0000 7f534ce74740 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5596db3015d0 0x5596db310390 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.113+0000 7f534ce74740 1 --2- 172.21.15.46:0/3302313327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596db300cb0 0x5596db30fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.113+0000 7f534ce74740 1 -- 172.21.15.46:0/3302313327 >> 172.21.15.46:0/3302313327 conn(0x5596db2fd240 msgr2=0x5596db304090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.113+0000 7f5330c00640 0 client.27158 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.113+0000 7f534ce74740 1 -- 172.21.15.46:0/3302313327 shutdown_connections 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.114+0000 7f534ce74740 1 -- 172.21.15.46:0/3302313327 wait complete. 2024-09-10T07:43:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.114+0000 7f534ce74740 20 client.27158 trim_cache size 0 max 16384 2024-09-10T07:43:33.117 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:43:33.117 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:43:33.117 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4e9246be-275b-468f-a355-54b62c2a2f77", 2024-09-10T07:43:33.117 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:43:33.117 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 -- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53280a2b80 msgr2=0x7f53280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53280a2b80 0x7f53280a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f53340985e0 tx=0x7f5334018420 comp rx=0 tx=0).stop 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 -- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344076720 msgr2=0x7f53441a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344076720 0x7f53441a4600 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f533400e9f0 tx=0x7f533400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 -- 172.21.15.46:0/615099121 shutdown_connections 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5344112870 0x7f534419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5344077040 0x7f53441a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53280a2b80 0x7f53280a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 --2- 172.21.15.46:0/615099121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5344076720 0x7f53441a4600 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 -- 172.21.15.46:0/615099121 >> 172.21.15.46:0/615099121 conn(0x7f53440ff8d0 msgr2=0x7f5344101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 -- 172.21.15.46:0/615099121 shutdown_connections 2024-09-10T07:43:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:33.115+0000 7f5348c00640 1 -- 172.21.15.46:0/615099121 wait complete. 2024-09-10T07:43:33.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:33 smithi046 ceph-mon[30705]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.1 MiB/s wr, 206 op/s 2024-09-10T07:43:33.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:33 smithi181 ceph-mon[35397]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.1 MiB/s wr, 206 op/s 2024-09-10T07:43:33.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:33 smithi063 ceph-mon[35020]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.1 MiB/s wr, 206 op/s 2024-09-10T07:43:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 -- 172.21.15.46:0/3215670263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc0103d60 msgr2=0x7f1cc0104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 --2- 172.21.15.46:0/3215670263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc0103d60 0x7f1cc0104160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009900 tx=0x7f1cb002a410 comp rx=0 tx=0).stop 2024-09-10T07:43:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 -- 172.21.15.46:0/3215670263 shutdown_connections 2024-09-10T07:43:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 --2- 172.21.15.46:0/3215670263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc0105910 0x7f1cc010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 --2- 172.21.15.46:0/3215670263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cc0104f50 0x7f1cc01053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 --2- 172.21.15.46:0/3215670263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc0103d60 0x7f1cc0104160 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 -- 172.21.15.46:0/3215670263 >> 172.21.15.46:0/3215670263 conn(0x7f1cc00ff4e0 msgr2=0x7f1cc0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 -- 172.21.15.46:0/3215670263 shutdown_connections 2024-09-10T07:43:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.818+0000 7f1cc6800640 1 -- 172.21.15.46:0/3215670263 wait complete. 2024-09-10T07:43:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.819+0000 7f1cc6800640 1 Processor -- start 2024-09-10T07:43:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.819+0000 7f1cc6800640 1 -- start start 2024-09-10T07:43:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.819+0000 7f1cc6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc0103d60 0x7f1cc0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.819+0000 7f1cc6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc0104f50 0x7f1cc019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.819+0000 7f1cc6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cc0105910 0x7f1cc01a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.819+0000 7f1cc6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010f9e0 con 0x7f1cc0103d60 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.819+0000 7f1cc6800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010f860 con 0x7f1cc0104f50 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cc6800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010fb60 con 0x7f1cc0105910 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc0104f50 0x7f1cc019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc0103d60 0x7f1cc0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc0104f50 0x7f1cc019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35498/0 (socket says 172.21.15.46:35498) 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbea00640 1 -- 172.21.15.46:0/1794101679 learned_addr learned my addr 172.21.15.46:0/1794101679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cc0105910 0x7f1cc01a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbea00640 1 -- 172.21.15.46:0/1794101679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cc0105910 msgr2=0x7f1cc01a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbea00640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cc0105910 0x7f1cc01a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbea00640 1 -- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc0103d60 msgr2=0x7f1cc0199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbea00640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc0103d60 0x7f1cc0199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.820+0000 7f1cbea00640 1 -- 172.21.15.46:0/1794101679 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc01a1c00 con 0x7f1cc0104f50 2024-09-10T07:43:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.821+0000 7f1cbea00640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc0104f50 0x7f1cc019a520 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e9f0 tx=0x7f1cac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.821+0000 7f1cb7e00640 1 -- 172.21.15.46:0/1794101679 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00cd80 con 0x7f1cc0104f50 2024-09-10T07:43:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.821+0000 7f1cc6800640 1 -- 172.21.15.46:0/1794101679 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc01a1ef0 con 0x7f1cc0104f50 2024-09-10T07:43:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.821+0000 7f1cb7e00640 1 -- 172.21.15.46:0/1794101679 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac005500 con 0x7f1cc0104f50 2024-09-10T07:43:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.821+0000 7f1cb7e00640 1 -- 172.21.15.46:0/1794101679 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac010690 con 0x7f1cc0104f50 2024-09-10T07:43:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.821+0000 7f1cc6800640 1 -- 172.21.15.46:0/1794101679 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a2360 con 0x7f1cc0104f50 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cc9637740 1 Processor -- start 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cc9637740 1 -- start start 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cb7e00640 1 -- 172.21.15.46:0/1794101679 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1cac0046c0 con 0x7f1cc0104f50 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 0x55d9c5eee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 0x55d9c5ef3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5ef3a60 0x55d9c5ef5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cc9637740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d9c5e53100 con 0x55d9c5ef3a60 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cc9637740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d9c5dfb1b0 con 0x55d9c5eee9d0 2024-09-10T07:43:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.824+0000 7f1cc9637740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d9c5e4f0c0 con 0x55d9c5eee090 2024-09-10T07:43:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5ef3a60 0x55d9c5ef5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5ef3a60 0x55d9c5ef5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55538/0 (socket says 172.21.15.46:55538) 2024-09-10T07:43:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 -- 172.21.15.46:0/1294057237 learned_addr learned my addr 172.21.15.46:0/1294057237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 0x55d9c5eee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbf400640 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 0x55d9c5ef3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 -- 172.21.15.46:0/1294057237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 msgr2=0x55d9c5eee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 0x55d9c5eee490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 -- 172.21.15.46:0/1294057237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 msgr2=0x55d9c5ef3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 0x55d9c5ef3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 -- 172.21.15.46:0/1294057237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c5b66210 con 0x55d9c5ef3a60 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.825+0000 7f1cbea00640 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5ef3a60 0x55d9c5ef5f20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0d5a80 tx=0x7f1cac0d5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3746f0dd47eb741 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.826+0000 7f1cb4c00640 1 -- 172.21.15.46:0/1294057237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0de070 con 0x55d9c5ef3a60 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.826+0000 7f1cb4c00640 1 -- 172.21.15.46:0/1294057237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac0da440 con 0x55d9c5ef3a60 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.826+0000 7f1cb4c00640 1 -- 172.21.15.46:0/1294057237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0d12b0 con 0x55d9c5ef3a60 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.826+0000 7f1cb7e00640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ca40a2c30 0x7f1ca40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.826+0000 7f1cb7e00640 1 -- 172.21.15.46:0/1794101679 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1cac0cf790 con 0x7f1cc0104f50 2024-09-10T07:43:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.826+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ca40a2c30 0x7f1ca40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.826+0000 7f1cc9637740 1 -- 172.21.15.46:0/1294057237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5ef3a60 msgr2=0x55d9c5ef5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.826+0000 7f1cc9637740 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5ef3a60 0x55d9c5ef5f20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0d5a80 tx=0x7f1cac0d5b70 comp rx=0 tx=0).stop 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 -- 172.21.15.46:0/1294057237 shutdown_connections 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5ef3a60 0x55d9c5ef5f20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 0x55d9c5ef3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 --2- 172.21.15.46:0/1294057237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 0x55d9c5eee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 -- 172.21.15.46:0/1294057237 >> 172.21.15.46:0/1294057237 conn(0x55d9c5ee9700 msgr2=0x55d9c5ee9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ca40a2c30 0x7f1ca40a50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8005fd0 tx=0x7f1cb8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 -- 172.21.15.46:0/1294057237 shutdown_connections 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 -- 172.21.15.46:0/1294057237 wait complete. 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 -- start start 2024-09-10T07:43:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.827+0000 7f1cc9637740 1 Processor -- start 2024-09-10T07:43:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee090 0x55d9c5ef81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 0x55d9c5ef8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5eff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cc9637740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d9c5ef6a30 con 0x55d9c5eee090 2024-09-10T07:43:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cc9637740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d9c5ddc090 con 0x55d9c5eee9d0 2024-09-10T07:43:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cc9637740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d9c5ef6bb0 con 0x55d9c5ef3a60 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cc9637740 10 client.0 _ll_register_callbacks cb 0x55d9c5e30f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee090 0x55d9c5ef81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee090 0x55d9c5ef81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55548/0 (socket says 172.21.15.46:55548) 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cbf400640 1 -- 172.21.15.46:0/2508517839 learned_addr learned my addr 172.21.15.46:0/2508517839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5eff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.828+0000 7f1cbea00640 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 0x55d9c5ef8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cbf400640 1 -- 172.21.15.46:0/2508517839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 msgr2=0x55d9c5eff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cbf400640 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5eff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cbf400640 1 -- 172.21.15.46:0/2508517839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 msgr2=0x55d9c5ef8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cbf400640 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 0x55d9c5ef8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cbf400640 1 -- 172.21.15.46:0/2508517839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c5c32620 con 0x55d9c5eee090 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cbf400640 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee090 0x55d9c5ef81f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0000c00 tx=0x7f1cb0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250a492ff86d612b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1ca9600640 1 -- 172.21.15.46:0/2508517839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0035040 con 0x55d9c5eee090 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1ca9600640 1 -- 172.21.15.46:0/2508517839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1cb0004040 con 0x55d9c5eee090 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cc9637740 1 -- 172.21.15.46:0/2508517839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cac0da440 con 0x55d9c5eee090 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1ca9600640 1 -- 172.21.15.46:0/2508517839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1cb0004240 con 0x55d9c5eee090 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cc9637740 10 client.16728 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.829+0000 7f1cc9637740 1 -- 172.21.15.46:0/2508517839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d9c5f025d0 con 0x55d9c5eee090 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.830+0000 7f1ca9600640 1 -- 172.21.15.46:0/2508517839 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac0da440 con 0x55d9c5eee090 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.830+0000 7f1ca9600640 1 -- 172.21.15.46:0/2508517839 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f1cb0004940 con 0x55d9c5eee090 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.830+0000 7f1cc9637740 10 client.16728 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.830+0000 7f1cc9637740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.830+0000 7f1cc9637740 20 client.16728 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.830+0000 7f1cc9637740 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d9c5f03cd0 0x55d9c5f06170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.830+0000 7f1cc9637740 4 client.16728 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.830+0000 7f1cc9637740 1 -- 172.21.15.46:0/2508517839 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c5e60830 con 0x55d9c5f03cd0 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.831+0000 7f1cbea00640 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d9c5f03cd0 0x55d9c5f06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.831+0000 7f1ca9600640 10 client.16728 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.831+0000 7f1cbea00640 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d9c5f03cd0 0x55d9c5f06170 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00c06898ad50204 server_cookie=51df022dc89585e0 in_seq=0 out_seq=0 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1ca9600640 1 -- 172.21.15.46:0/2508517839 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9c5e60830 con 0x55d9c5f03cd0 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1ca9600640 10 client.16728 handle_command_reply: tid=0 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 client.16728 shutdown 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 -- 172.21.15.46:0/2508517839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee090 msgr2=0x55d9c5ef81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee090 0x55d9c5ef81f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0000c00 tx=0x7f1cb0005e70 comp rx=0 tx=0).stop 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 -- 172.21.15.46:0/2508517839 shutdown_connections 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d9c5f03cd0 0x55d9c5f06170 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5eff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee9d0 0x55d9c5ef8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1ca9600640 0 client.16728 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 --2- 172.21.15.46:0/2508517839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee090 0x55d9c5ef81f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 -- 172.21.15.46:0/2508517839 >> 172.21.15.46:0/2508517839 conn(0x55d9c5ee9700 msgr2=0x55d9c5eeaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 -- 172.21.15.46:0/2508517839 shutdown_connections 2024-09-10T07:43:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 1 -- 172.21.15.46:0/2508517839 wait complete. 2024-09-10T07:43:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.832+0000 7f1cc9637740 20 client.16728 trim_cache size 0 max 16384 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cc9637740 1 Processor -- start 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cc9637740 1 -- start start 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 0x55d9c5ee9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5eff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5ef3a60 0x55d9c5eea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cc9637740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ac880 con 0x55d9c5eee9d0 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cc9637740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ae4b0 con 0x55d9c5ef3a60 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cc9637740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01acf50 con 0x55d9c5eee090 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5eff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5eff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55558/0 (socket says 172.21.15.46:55558) 2024-09-10T07:43:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cbea00640 1 -- 172.21.15.46:0/839900146 learned_addr learned my addr 172.21.15.46:0/839900146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5ef3a60 0x55d9c5eea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cbf400640 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 0x55d9c5ee9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cbea00640 1 -- 172.21.15.46:0/839900146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 msgr2=0x55d9c5ee9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cbea00640 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 0x55d9c5ee9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.848+0000 7f1cbea00640 1 -- 172.21.15.46:0/839900146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5ef3a60 msgr2=0x55d9c5eea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.849+0000 7f1cbea00640 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5ef3a60 0x55d9c5eea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.849+0000 7f1cbea00640 1 -- 172.21.15.46:0/839900146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c5ee4d60 con 0x55d9c5eee9d0 2024-09-10T07:43:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.849+0000 7f1cbea00640 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5eff690 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0d7690 tx=0x7f1cac0daf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54011872d81355f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.849+0000 7f1cabe00640 1 -- 172.21.15.46:0/839900146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0de040 con 0x55d9c5eee9d0 2024-09-10T07:43:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.849+0000 7f1cabe00640 1 -- 172.21.15.46:0/839900146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac09c370 con 0x55d9c5eee9d0 2024-09-10T07:43:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.849+0000 7f1cabe00640 1 -- 172.21.15.46:0/839900146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac09bc70 con 0x55d9c5eee9d0 2024-09-10T07:43:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 -- 172.21.15.46:0/839900146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 msgr2=0x55d9c5eff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5eff690 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0d7690 tx=0x7f1cac0daf90 comp rx=0 tx=0).stop 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 -- 172.21.15.46:0/839900146 shutdown_connections 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5ef3a60 0x55d9c5eea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5eff690 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 --2- 172.21.15.46:0/839900146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5eee090 0x55d9c5ee9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 -- 172.21.15.46:0/839900146 >> 172.21.15.46:0/839900146 conn(0x55d9c5ee9700 msgr2=0x55d9c5ef85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 -- 172.21.15.46:0/839900146 shutdown_connections 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 -- 172.21.15.46:0/839900146 wait complete. 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 -- start start 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.850+0000 7f1cc9637740 1 Processor -- start 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee090 0x55d9c5efc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5efceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cc9637740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5efdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cc9637740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01add20 con 0x55d9c5eee9d0 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cc9637740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ad620 con 0x55d9c5eee090 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cc9637740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01aa860 con 0x55d9c5ef3a60 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cc9637740 10 client.0 _ll_register_callbacks cb 0x7f1cb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5efceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5efceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55564/0 (socket says 172.21.15.46:55564) 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee090 0x55d9c5efc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbea00640 1 -- 172.21.15.46:0/3976089475 learned_addr learned my addr 172.21.15.46:0/3976089475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5efdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbea00640 1 -- 172.21.15.46:0/3976089475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 msgr2=0x55d9c5efdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbea00640 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5efdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbea00640 1 -- 172.21.15.46:0/3976089475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee090 msgr2=0x55d9c5efc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbea00640 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee090 0x55d9c5efc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbea00640 1 -- 172.21.15.46:0/3976089475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c5efe4d0 con 0x55d9c5eee9d0 2024-09-10T07:43:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.851+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5efdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cbea00640 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5efceb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0a43b0 tx=0x7f1cac0cfb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f400ee2902279f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1ca9600640 1 -- 172.21.15.46:0/3976089475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0de070 con 0x55d9c5eee9d0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 1 -- 172.21.15.46:0/3976089475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cac09c370 con 0x55d9c5eee9d0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1ca9600640 1 -- 172.21.15.46:0/3976089475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1cac0cd020 con 0x55d9c5eee9d0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1ca9600640 1 -- 172.21.15.46:0/3976089475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1cac0a0020 con 0x55d9c5eee9d0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 10 client.16734 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 1 -- 172.21.15.46:0/3976089475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d9c5eff7e0 con 0x55d9c5eee9d0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1ca9600640 1 -- 172.21.15.46:0/3976089475 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac09c370 con 0x55d9c5eee9d0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1ca9600640 1 -- 172.21.15.46:0/3976089475 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f1cac0982e0 con 0x55d9c5eee9d0 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 10 client.16734 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 20 client.16734 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d9c5f6ea10 0x55d9c5f70e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 4 client.16734 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:43:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.852+0000 7f1cc9637740 1 -- 172.21.15.46:0/3976089475 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9c5e5ab80 con 0x55d9c5f6ea10 2024-09-10T07:43:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.853+0000 7f1cbf400640 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d9c5f6ea10 0x55d9c5f70e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.853+0000 7f1cbf400640 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d9c5f6ea10 0x55d9c5f70e00 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3fce303cf6d549 server_cookie=89f1c39d16fe3b6c in_seq=0 out_seq=0 2024-09-10T07:43:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.853+0000 7f1ca9600640 10 client.16734 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1ca9600640 1 -- 172.21.15.46:0/3976089475 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9c5e5ab80 con 0x55d9c5f6ea10 2024-09-10T07:43:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1ca9600640 10 client.16734 handle_command_reply: tid=0 2024-09-10T07:43:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 client.16734 shutdown 2024-09-10T07:43:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 -- 172.21.15.46:0/3976089475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 msgr2=0x55d9c5efceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5efceb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0a43b0 tx=0x7f1cac0cfb10 comp rx=0 tx=0).stop 2024-09-10T07:43:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 -- 172.21.15.46:0/3976089475 shutdown_connections 2024-09-10T07:43:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d9c5f6ea10 0x55d9c5f70e00 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d9c5ef3a60 0x55d9c5efdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9c5eee9d0 0x55d9c5efceb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1ca9600640 0 client.16734 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 --2- 172.21.15.46:0/3976089475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9c5eee090 0x55d9c5efc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 -- 172.21.15.46:0/3976089475 >> 172.21.15.46:0/3976089475 conn(0x55d9c5ee9700 msgr2=0x55d9c5ee8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 -- 172.21.15.46:0/3976089475 shutdown_connections 2024-09-10T07:43:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.863+0000 7f1cc9637740 1 -- 172.21.15.46:0/3976089475 wait complete. 2024-09-10T07:43:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc9637740 20 client.16734 trim_cache size 0 max 16384 2024-09-10T07:43:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:43:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:43:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:43:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:43:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 -- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ca40a2c30 msgr2=0x7f1ca40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ca40a2c30 0x7f1ca40a50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8005fd0 tx=0x7f1cb8009500 comp rx=0 tx=0).stop 2024-09-10T07:43:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 -- 172.21.15.46:0/1794101679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc0104f50 msgr2=0x7f1cc019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc0104f50 0x7f1cc019a520 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e9f0 tx=0x7f1cac00edf0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 -- 172.21.15.46:0/1794101679 shutdown_connections 2024-09-10T07:43:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cc0105910 0x7f1cc01a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc0104f50 0x7f1cc019a520 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ca40a2c30 0x7f1ca40a50f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 --2- 172.21.15.46:0/1794101679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc0103d60 0x7f1cc0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 -- 172.21.15.46:0/1794101679 >> 172.21.15.46:0/1794101679 conn(0x7f1cc00ff4e0 msgr2=0x7f1cc01082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 -- 172.21.15.46:0/1794101679 shutdown_connections 2024-09-10T07:43:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:33.864+0000 7f1cc6800640 1 -- 172.21.15.46:0/1794101679 wait complete. 2024-09-10T07:43:34.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:43:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:43:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:43:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:43:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:43:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:34.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:43:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:43:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:43:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:43:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:43:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:34.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:43:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 -- 172.21.15.46:0/1159403796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c102360 msgr2=0x7f4b2c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 --2- 172.21.15.46:0/1159403796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c102360 0x7f4b2c10e890 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009cd0 tx=0x7f4b2402a8e0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 -- 172.21.15.46:0/1159403796 shutdown_connections 2024-09-10T07:43:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 --2- 172.21.15.46:0/1159403796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c102360 0x7f4b2c10e890 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 --2- 172.21.15.46:0/1159403796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b2c1019c0 0x7f4b2c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 --2- 172.21.15.46:0/1159403796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b2c1079c0 0x7f4b2c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 -- 172.21.15.46:0/1159403796 >> 172.21.15.46:0/1159403796 conn(0x7f4b2c0fd6e0 msgr2=0x7f4b2c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:35.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 -- 172.21.15.46:0/1159403796 shutdown_connections 2024-09-10T07:43:35.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.145+0000 7f4b32800640 1 -- 172.21.15.46:0/1159403796 wait complete. 2024-09-10T07:43:35.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.146+0000 7f4b32800640 1 Processor -- start 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.146+0000 7f4b32800640 1 -- start start 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.146+0000 7f4b32800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c1019c0 0x7f4b2c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.146+0000 7f4b32800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b2c102360 0x7f4b2c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.146+0000 7f4b32800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b2c1079c0 0x7f4b2c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.146+0000 7f4b32800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c111dc0 con 0x7f4b2c1019c0 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.146+0000 7f4b32800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c111c40 con 0x7f4b2c1079c0 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.146+0000 7f4b32800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c111f40 con 0x7f4b2c102360 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c1019c0 0x7f4b2c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c1019c0 0x7f4b2c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55586/0 (socket says 172.21.15.46:55586) 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b2c1079c0 0x7f4b2c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31400640 1 -- 172.21.15.46:0/3176782267 learned_addr learned my addr 172.21.15.46:0/3176782267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b30a00640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b2c102360 0x7f4b2c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31400640 1 -- 172.21.15.46:0/3176782267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b2c102360 msgr2=0x7f4b2c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31400640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b2c102360 0x7f4b2c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31400640 1 -- 172.21.15.46:0/3176782267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b2c1079c0 msgr2=0x7f4b2c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31400640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b2c1079c0 0x7f4b2c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.147+0000 7f4b31400640 1 -- 172.21.15.46:0/3176782267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c19f410 con 0x7f4b2c1019c0 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.148+0000 7f4b31400640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c1019c0 0x7f4b2c19a460 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000e9f0 tx=0x7f4b2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.148+0000 7f4b2a000640 1 -- 172.21.15.46:0/3176782267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000cd80 con 0x7f4b2c1019c0 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.148+0000 7f4b2a000640 1 -- 172.21.15.46:0/3176782267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b20010910 con 0x7f4b2c1019c0 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.148+0000 7f4b2a000640 1 -- 172.21.15.46:0/3176782267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20018990 con 0x7f4b2c1019c0 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.148+0000 7f4b32800640 1 -- 172.21.15.46:0/3176782267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c19f700 con 0x7f4b2c1019c0 2024-09-10T07:43:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.148+0000 7f4b32800640 1 -- 172.21.15.46:0/3176782267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1a6fe0 con 0x7f4b2c1019c0 2024-09-10T07:43:35.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.151+0000 7f4b2a000640 1 -- 172.21.15.46:0/3176782267 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b20010ac0 con 0x7f4b2c1019c0 2024-09-10T07:43:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.151+0000 7f4b36a2e740 1 Processor -- start 2024-09-10T07:43:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.151+0000 7f4b36a2e740 1 -- start start 2024-09-10T07:43:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f1670 0x55cc4c0fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0fe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cc4c0573e0 con 0x55cc4c0f1670 2024-09-10T07:43:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cc4bffed80 con 0x55cc4c0f0d50 2024-09-10T07:43:35.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cc4c0533a0 con 0x55cc4c0fbf30 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b31400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f1670 0x55cc4c0fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35570/0 (socket says 172.21.15.46:35570) 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b31e00640 1 -- 172.21.15.46:0/2100892760 learned_addr learned my addr 172.21.15.46:0/2100892760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.152+0000 7f4b30a00640 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b31e00640 1 -- 172.21.15.46:0/2100892760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0fbf30 msgr2=0x55cc4c0fe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b31e00640 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0fe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b31e00640 1 -- 172.21.15.46:0/2100892760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f1670 msgr2=0x55cc4c0fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b31e00640 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f1670 0x55cc4c0fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b31e00640 1 -- 172.21.15.46:0/2100892760 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4c01dae0 con 0x55cc4c0f0d50 2024-09-10T07:43:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b31400640 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f1670 0x55cc4c0fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b2a000640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4b140a2c00 0x7f4b140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b31e00640 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f1130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009cd0 tx=0x7f4b24009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=addc337a70015070 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b2a000640 1 -- 172.21.15.46:0/3176782267 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4b200cf5f0 con 0x7f4b2c1019c0 2024-09-10T07:43:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b31400640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4b140a2c00 0x7f4b140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.153+0000 7f4b1e600640 1 -- 172.21.15.46:0/2100892760 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24048070 con 0x55cc4c0f0d50 2024-09-10T07:43:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b1e600640 1 -- 172.21.15.46:0/2100892760 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b24036070 con 0x55cc4c0f0d50 2024-09-10T07:43:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b1e600640 1 -- 172.21.15.46:0/2100892760 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24004440 con 0x55cc4c0f0d50 2024-09-10T07:43:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2100892760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 msgr2=0x55cc4c0f1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b31400640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4b140a2c00 0x7f4b140a50c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4b2009b820 tx=0x7f4b200d50e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f1130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009cd0 tx=0x7f4b24009a20 comp rx=0 tx=0).stop 2024-09-10T07:43:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2100892760 shutdown_connections 2024-09-10T07:43:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f1670 0x55cc4c0fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2100892760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f1130 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.154+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2100892760 >> 172.21.15.46:0/2100892760 conn(0x55cc4c0ed2e0 msgr2=0x55cc4c0ed6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.155+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2100892760 shutdown_connections 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.155+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2100892760 wait complete. 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.155+0000 7f4b36a2e740 1 -- start start 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.155+0000 7f4b36a2e740 1 Processor -- start 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0f9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cc4c0feba0 con 0x55cc4c0fbf30 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cc4c0120f0 con 0x55cc4c0f0d50 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cc4c0fed20 con 0x55cc4c0f1670 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b36a2e740 10 client.0 _ll_register_callbacks cb 0x55cc4c0039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b30a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b30a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40046/0 (socket says 172.21.15.46:40046) 2024-09-10T07:43:35.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b30a00640 1 -- 172.21.15.46:0/1133388965 learned_addr learned my addr 172.21.15.46:0/1133388965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:35.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.156+0000 7f4b31e00640 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b30a00640 1 -- 172.21.15.46:0/1133388965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 msgr2=0x55cc4c0f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:35.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b30a00640 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b30a00640 1 -- 172.21.15.46:0/1133388965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 msgr2=0x55cc4c0f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:35.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b30a00640 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b30a00640 1 -- 172.21.15.46:0/1133388965 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4bdc83c0 con 0x55cc4c0f1670 2024-09-10T07:43:35.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b30a00640 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f9410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4b180126e0 tx=0x7f4b18012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c352d39fd857801 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:35.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b12a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:35.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b12a00640 1 -- 172.21.15.46:0/1133388965 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800f920 con 0x55cc4c0f1670 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.157+0000 7f4b36a2e740 1 -- 172.21.15.46:0/1133388965 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b24036070 con 0x55cc4c0f1670 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.158+0000 7f4b12a00640 1 -- 172.21.15.46:0/1133388965 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4b1800fac0 con 0x55cc4c0f1670 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.158+0000 7f4b12a00640 1 -- 172.21.15.46:0/1133388965 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4b1800fcc0 con 0x55cc4c0f1670 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.158+0000 7f4b36a2e740 10 client.27127 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.158+0000 7f4b36a2e740 1 -- 172.21.15.46:0/1133388965 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cc4c10a9a0 con 0x55cc4c0f1670 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.158+0000 7f4b12a00640 1 -- 172.21.15.46:0/1133388965 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b24036070 con 0x55cc4c0f1670 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.158+0000 7f4b12a00640 1 -- 172.21.15.46:0/1133388965 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f4b1801c4d0 con 0x55cc4c0f1670 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.159+0000 7f4b36a2e740 10 client.27127 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.159+0000 7f4b36a2e740 10 client.27127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.159+0000 7f4b36a2e740 20 client.27127 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.159+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cc4c10bf30 0x55cc4c10e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.159+0000 7f4b36a2e740 4 client.27127 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:35.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.159+0000 7f4b36a2e740 1 -- 172.21.15.46:0/1133388965 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc4c064950 con 0x55cc4c10bf30 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.159+0000 7f4b31e00640 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cc4c10bf30 0x55cc4c10e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.160+0000 7f4b31e00640 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cc4c10bf30 0x55cc4c10e350 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea6e948a5a3f39 server_cookie=d556db191d3d923 in_seq=0 out_seq=0 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.160+0000 7f4b12a00640 10 client.27127 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.160+0000 7f4b12a00640 1 -- 172.21.15.46:0/1133388965 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc4c064950 con 0x55cc4c10bf30 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.160+0000 7f4b12a00640 10 client.27127 handle_command_reply: tid=0 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.160+0000 7f4b36a2e740 1 client.27127 shutdown 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 -- 172.21.15.46:0/1133388965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 msgr2=0x55cc4c0f9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f9410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4b180126e0 tx=0x7f4b18012ba0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 -- 172.21.15.46:0/1133388965 shutdown_connections 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cc4c10bf30 0x55cc4c10e350 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f9410 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/1133388965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b12a00640 0 client.27127 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 -- 172.21.15.46:0/1133388965 >> 172.21.15.46:0/1133388965 conn(0x55cc4c0ed2e0 msgr2=0x55cc4c0f3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 -- 172.21.15.46:0/1133388965 shutdown_connections 2024-09-10T07:43:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 1 -- 172.21.15.46:0/1133388965 wait complete. 2024-09-10T07:43:35.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.161+0000 7f4b36a2e740 20 client.27127 trim_cache size 0 max 16384 2024-09-10T07:43:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.181+0000 7f4b36a2e740 1 Processor -- start 2024-09-10T07:43:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.181+0000 7f4b36a2e740 1 -- start start 2024-09-10T07:43:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.181+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.181+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c113c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.181+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 0x55cc4c114160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.181+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b0c30 con 0x55cc4c0fbf30 2024-09-10T07:43:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.181+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b2860 con 0x55cc4c0f0d50 2024-09-10T07:43:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.181+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b1300 con 0x55cc4c0f1670 2024-09-10T07:43:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b30a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b30a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c113c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40054/0 (socket says 172.21.15.46:40054) 2024-09-10T07:43:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b30a00640 1 -- 172.21.15.46:0/3778415229 learned_addr learned my addr 172.21.15.46:0/3778415229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b31e00640 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b31e00640 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 0x55cc4c114160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b30a00640 1 -- 172.21.15.46:0/3778415229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 msgr2=0x55cc4c0f97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b30a00640 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b30a00640 1 -- 172.21.15.46:0/3778415229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 msgr2=0x55cc4c114160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b30a00640 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 0x55cc4c114160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.182+0000 7f4b30a00640 1 -- 172.21.15.46:0/3778415229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4c10e9d0 con 0x55cc4c0f1670 2024-09-10T07:43:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.183+0000 7f4b30a00640 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c113c20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b1801cb30 tx=0x7f4b180096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d72d49945fc1afa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.184+0000 7f4b1dc00640 1 -- 172.21.15.46:0/3778415229 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800f920 con 0x55cc4c0f1670 2024-09-10T07:43:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.184+0000 7f4b1dc00640 1 -- 172.21.15.46:0/3778415229 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b1800fac0 con 0x55cc4c0f1670 2024-09-10T07:43:35.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.184+0000 7f4b1dc00640 1 -- 172.21.15.46:0/3778415229 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1801c710 con 0x55cc4c0f1670 2024-09-10T07:43:35.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 -- 172.21.15.46:0/3778415229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 msgr2=0x55cc4c113c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c113c20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b1801cb30 tx=0x7f4b180096d0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 -- 172.21.15.46:0/3778415229 shutdown_connections 2024-09-10T07:43:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0fbf30 0x55cc4c114160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c113c20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/3778415229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0f0d50 0x55cc4c0f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 -- 172.21.15.46:0/3778415229 >> 172.21.15.46:0/3778415229 conn(0x55cc4c0ed2e0 msgr2=0x55cc4c0faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 -- 172.21.15.46:0/3778415229 shutdown_connections 2024-09-10T07:43:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 -- 172.21.15.46:0/3778415229 wait complete. 2024-09-10T07:43:35.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.185+0000 7f4b36a2e740 1 -- start start 2024-09-10T07:43:35.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.186+0000 7f4b36a2e740 1 Processor -- start 2024-09-10T07:43:35.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.186+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f0d50 0x55cc4c1045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.186+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.186+0000 7f4b36a2e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0f6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.186+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b20d0 con 0x55cc4c0f0d50 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.186+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b19d0 con 0x55cc4c0fbf30 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.186+0000 7f4b36a2e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1aec10 con 0x55cc4c0f1670 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b36a2e740 10 client.0 _ll_register_callbacks cb 0x7f4b18009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f0d50 0x55cc4c1045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b30a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f0d50 0x55cc4c1045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55606/0 (socket says 172.21.15.46:55606) 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 -- 172.21.15.46:0/2493411960 learned_addr learned my addr 172.21.15.46:0/2493411960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 -- 172.21.15.46:0/2493411960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 msgr2=0x55cc4c0f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 -- 172.21.15.46:0/2493411960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0fbf30 msgr2=0x55cc4c0f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b31e00640 1 -- 172.21.15.46:0/2493411960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4c0f6750 con 0x55cc4c0f0d50 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.187+0000 7f4b30a00640 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.188+0000 7f4b31e00640 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f0d50 0x55cc4c1045b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005e30 tx=0x7f4b24009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1bbbecdb326f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.188+0000 7f4b12a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.188+0000 7f4b12a00640 1 -- 172.21.15.46:0/2493411960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24048040 con 0x55cc4c0f0d50 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.188+0000 7f4b12a00640 1 -- 172.21.15.46:0/2493411960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4b24031ac0 con 0x55cc4c0f0d50 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.188+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2493411960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1800fac0 con 0x55cc4c0f0d50 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b12a00640 1 -- 172.21.15.46:0/2493411960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4b24031cc0 con 0x55cc4c0f0d50 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b36a2e740 10 client.16752 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2493411960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cc4c176b00 con 0x55cc4c0f0d50 2024-09-10T07:43:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b12a00640 1 -- 172.21.15.46:0/2493411960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b24061020 con 0x55cc4c0f0d50 2024-09-10T07:43:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b12a00640 1 -- 172.21.15.46:0/2493411960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f4b24059070 con 0x55cc4c0f0d50 2024-09-10T07:43:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b36a2e740 10 client.16752 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b36a2e740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b36a2e740 20 client.16752 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cc4c176e30 0x55cc4c179220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.189+0000 7f4b36a2e740 4 client.16752 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:43:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.190+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2493411960 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc4c05ee60 con 0x55cc4c176e30 2024-09-10T07:43:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.190+0000 7f4b30a00640 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cc4c176e30 0x55cc4c179220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:35.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b30a00640 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cc4c176e30 0x55cc4c179220 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8611282d8d8d1 server_cookie=a4f882775a9b51ce in_seq=0 out_seq=0 2024-09-10T07:43:35.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b12a00640 10 client.16752 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b12a00640 1 -- 172.21.15.46:0/2493411960 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cc4c05ee60 con 0x55cc4c176e30 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b12a00640 10 client.16752 handle_command_reply: tid=0 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 client.16752 shutdown 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2493411960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f0d50 msgr2=0x55cc4c1045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f0d50 0x55cc4c1045b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005e30 tx=0x7f4b24009a20 comp rx=0 tx=0).stop 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2493411960 shutdown_connections 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cc4c176e30 0x55cc4c179220 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cc4c0fbf30 0x55cc4c0f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cc4c0f1670 0x55cc4c0f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 --2- 172.21.15.46:0/2493411960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc4c0f0d50 0x55cc4c1045b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b12a00640 0 client.16752 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:35.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.191+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2493411960 >> 172.21.15.46:0/2493411960 conn(0x55cc4c0ed2e0 msgr2=0x55cc4c0ebb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.192+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2493411960 shutdown_connections 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.192+0000 7f4b36a2e740 1 -- 172.21.15.46:0/2493411960 wait complete. 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.192+0000 7f4b36a2e740 20 client.16752 trim_cache size 0 max 16384 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "4e9246be-275b-468f-a355-54b62c2a2f77": { 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:43:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "4e9246be-275b-468f-a355-54b62c2a2f77", 2024-09-10T07:43:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:43:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:35.195 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:43:35.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 -- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4b140a2c00 msgr2=0x7f4b140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4b140a2c00 0x7f4b140a50c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4b2009b820 tx=0x7f4b200d50e0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 -- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c1019c0 msgr2=0x7f4b2c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c1019c0 0x7f4b2c19a460 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000e9f0 tx=0x7f4b2000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 -- 172.21.15.46:0/3176782267 shutdown_connections 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b2c1079c0 0x7f4b2c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b2c102360 0x7f4b2c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4b140a2c00 0x7f4b140a50c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 --2- 172.21.15.46:0/3176782267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2c1019c0 0x7f4b2c19a460 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 -- 172.21.15.46:0/3176782267 >> 172.21.15.46:0/3176782267 conn(0x7f4b2c0fd6e0 msgr2=0x7f4b2c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 -- 172.21.15.46:0/3176782267 shutdown_connections 2024-09-10T07:43:35.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:35.193+0000 7f4b32800640 1 -- 172.21.15.46:0/3176782267 wait complete. 2024-09-10T07:43:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:35 smithi046 ceph-mon[30705]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-09-10T07:43:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 375.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:43:35.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:35 smithi181 ceph-mon[35397]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-09-10T07:43:35.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:35 smithi063 ceph-mon[35020]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-09-10T07:43:36.166 INFO:tasks.cephfs.filesystem:scrub status for tag:4e9246be-275b-468f-a355-54b62c2a2f77 - {'path': '/', 'tag': '4e9246be-275b-468f-a355-54b62c2a2f77', 'options': 'recursive,force'} 2024-09-10T07:43:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 -- 172.21.15.46:0/1809737110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e4103e30 msgr2=0x7f63e4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 --2- 172.21.15.46:0/1809737110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e4103e30 0x7f63e4104210 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009900 tx=0x7f63d402a410 comp rx=0 tx=0).stop 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 -- 172.21.15.46:0/1809737110 shutdown_connections 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 --2- 172.21.15.46:0/1809737110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63e4108f60 0x7f63e4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 --2- 172.21.15.46:0/1809737110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e41047e0 0x7f63e4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 --2- 172.21.15.46:0/1809737110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e4103e30 0x7f63e4104210 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 -- 172.21.15.46:0/1809737110 >> 172.21.15.46:0/1809737110 conn(0x7f63e40ff910 msgr2=0x7f63e4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 -- 172.21.15.46:0/1809737110 shutdown_connections 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.824+0000 7f63e9400640 1 -- 172.21.15.46:0/1809737110 wait complete. 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.825+0000 7f63e9400640 1 Processor -- start 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.825+0000 7f63e9400640 1 -- start start 2024-09-10T07:43:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e4103e30 0x7f63e419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e41047e0 0x7f63e419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63e4108f60 0x7f63e419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e9400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4114010 con 0x7f63e41047e0 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e9400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4113e90 con 0x7f63e4108f60 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e9400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4114190 con 0x7f63e4103e30 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e4103e30 0x7f63e419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e41047e0 0x7f63e419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e4103e30 0x7f63e419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:55144/0 (socket says 172.21.15.46:55144) 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e3e00640 1 -- 172.21.15.46:0/2596974805 learned_addr learned my addr 172.21.15.46:0/2596974805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e8a00640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63e4108f60 0x7f63e419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e3e00640 1 -- 172.21.15.46:0/2596974805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63e4108f60 msgr2=0x7f63e419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.826+0000 7f63e3e00640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63e4108f60 0x7f63e419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e3e00640 1 -- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e41047e0 msgr2=0x7f63e419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e3e00640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e41047e0 0x7f63e419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e3e00640 1 -- 172.21.15.46:0/2596974805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e419f4a0 con 0x7f63e4103e30 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e3e00640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e4103e30 0x7f63e419a4f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63d4004770 tx=0x7f63d4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e0c00640 1 -- 172.21.15.46:0/2596974805 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d403f070 con 0x7f63e4103e30 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e0c00640 1 -- 172.21.15.46:0/2596974805 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d4031ca0 con 0x7f63e4103e30 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e419f730 con 0x7f63e4103e30 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e0c00640 1 -- 172.21.15.46:0/2596974805 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4035030 con 0x7f63e4103e30 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.827+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e41a7070 con 0x7f63e4103e30 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.829+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e4105310 con 0x7f63e4103e30 2024-09-10T07:43:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.830+0000 7f63e0c00640 1 -- 172.21.15.46:0/2596974805 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f63d4042070 con 0x7f63e4103e30 2024-09-10T07:43:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.832+0000 7f63e0c00640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.832+0000 7f63e0c00640 1 -- 172.21.15.46:0/2596974805 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f63d40efa30 con 0x7f63e4103e30 2024-09-10T07:43:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.832+0000 7f63e3400640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:36.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.833+0000 7f63e3400640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63c80a2b80 0x7f63c80a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f63e40ffd50 tx=0x7f63d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:36.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:36.836+0000 7f63e0c00640 1 -- 172.21.15.46:0/2596974805 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63d40bc020 con 0x7f63e4103e30 2024-09-10T07:43:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.061+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e410c140 con 0x7f63e4103e30 2024-09-10T07:43:37.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.062+0000 7f63e0c00640 1 -- 172.21.15.46:0/2596974805 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11980 (secure 0 0 0) 0x7f63d40c0320 con 0x7f63e4103e30 2024-09-10T07:43:37.062 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:43:37.062 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":44,"btime":"2024-09-10T07:43:26:111291+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:43:25.508547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:43:37.062 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 44 2024-09-10T07:43:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.064+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63c80a2b80 msgr2=0x7f63c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.064+0000 7f63e9400640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63c80a2b80 0x7f63c80a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f63e40ffd50 tx=0x7f63d0005f90 comp rx=0 tx=0).stop 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.064+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e4103e30 msgr2=0x7f63e419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.064+0000 7f63e9400640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e4103e30 0x7f63e419a4f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63d4004770 tx=0x7f63d4003c10 comp rx=0 tx=0).stop 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.065+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 shutdown_connections 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.065+0000 7f63e9400640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63e4108f60 0x7f63e419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.065+0000 7f63e9400640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.065+0000 7f63e9400640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e41047e0 0x7f63e419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.065+0000 7f63e9400640 1 --2- 172.21.15.46:0/2596974805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63e4103e30 0x7f63e419a4f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.065+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 >> 172.21.15.46:0/2596974805 conn(0x7f63e40ff910 msgr2=0x7f63e4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.065+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 shutdown_connections 2024-09-10T07:43:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:37.065+0000 7f63e9400640 1 -- 172.21.15.46:0/2596974805 wait complete. 2024-09-10T07:43:37.917 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id 409df7a9 --release --await' 2024-09-10T07:43:37.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id 409df7a9 --release --await 2024-09-10T07:43:38.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:37 smithi181 ceph-mon[35397]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-09-10T07:43:38.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:37 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2596974805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:37 smithi063 ceph-mon[35020]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-09-10T07:43:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:37 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2596974805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:38.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:37 smithi046 ceph-mon[30705]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-09-10T07:43:38.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:37 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2596974805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:43:38.294 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:43:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.894+0000 7f9fe8200640 1 -- 172.21.15.46:0/1898864376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0101de0 msgr2=0x7f9fe01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.894+0000 7f9fe8200640 1 --2- 172.21.15.46:0/1898864376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0101de0 0x7f9fe01021c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009900 tx=0x7f9fd402a410 comp rx=0 tx=0).stop 2024-09-10T07:43:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.895+0000 7f9fe8200640 1 -- 172.21.15.46:0/1898864376 shutdown_connections 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.895+0000 7f9fe8200640 1 --2- 172.21.15.46:0/1898864376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe0106f10 0x7f9fe010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.895+0000 7f9fe8200640 1 --2- 172.21.15.46:0/1898864376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe0102790 0x7f9fe01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.895+0000 7f9fe8200640 1 --2- 172.21.15.46:0/1898864376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0101de0 0x7f9fe01021c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.895+0000 7f9fe8200640 1 -- 172.21.15.46:0/1898864376 >> 172.21.15.46:0/1898864376 conn(0x7f9fe00fd8c0 msgr2=0x7f9fe00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.895+0000 7f9fe8200640 1 -- 172.21.15.46:0/1898864376 shutdown_connections 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.895+0000 7f9fe8200640 1 -- 172.21.15.46:0/1898864376 wait complete. 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe8200640 1 Processor -- start 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe8200640 1 -- start start 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe8200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe0101de0 0x7f9fe00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe8200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0102790 0x7f9fe0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe8200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe0106f10 0x7f9fe0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe8200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0111f70 con 0x7f9fe0102790 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe8200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0111df0 con 0x7f9fe0106f10 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe8200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01120f0 con 0x7f9fe0101de0 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.896+0000 7f9fe4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0102790 0x7f9fe0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0102790 0x7f9fe0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42504/0 (socket says 172.21.15.46:42504) 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe5800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe0101de0 0x7f9fe00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe4e00640 1 -- 172.21.15.46:0/2932034065 learned_addr learned my addr 172.21.15.46:0/2932034065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe6200640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe0106f10 0x7f9fe0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe4e00640 1 -- 172.21.15.46:0/2932034065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe0101de0 msgr2=0x7f9fe00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe0101de0 0x7f9fe00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe4e00640 1 -- 172.21.15.46:0/2932034065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe0106f10 msgr2=0x7f9fe0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe0106f10 0x7f9fe0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe4e00640 1 -- 172.21.15.46:0/2932034065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe01a6970 con 0x7f9fe0102790 2024-09-10T07:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe6200640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe0106f10 0x7f9fe0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.897+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0102790 0x7f9fe0074a30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e9f0 tx=0x7f9fcc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.898+0000 7f9fde000640 1 -- 172.21.15.46:0/2932034065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00cd80 con 0x7f9fe0102790 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.898+0000 7f9fe8200640 1 -- 172.21.15.46:0/2932034065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe01a6c60 con 0x7f9fe0102790 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.898+0000 7f9fde000640 1 -- 172.21.15.46:0/2932034065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fcc005500 con 0x7f9fe0102790 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.898+0000 7f9fde000640 1 -- 172.21.15.46:0/2932034065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc010690 con 0x7f9fe0102790 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.898+0000 7f9fe8200640 1 -- 172.21.15.46:0/2932034065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe01a7150 con 0x7f9fe0102790 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.901+0000 7f9feae42740 1 Processor -- start 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.901+0000 7f9fde000640 1 -- 172.21.15.46:0/2932034065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9fcc0046c0 con 0x7f9fe0102790 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.901+0000 7f9feae42740 1 -- start start 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.901+0000 7f9feae42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.901+0000 7f9feae42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c1efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9feae42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9feae42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c92b7a650 con 0x557c92c14c30 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9feae42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557c92b21f10 con 0x557c92c14310 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9feae42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557c92b76610 con 0x557c92c1f4f0 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c1efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe6200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe6200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41358/0 (socket says 172.21.15.46:41358) 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe6200640 1 -- 172.21.15.46:0/2063641550 learned_addr learned my addr 172.21.15.46:0/2063641550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe5800640 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c1efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42510/0 (socket says 172.21.15.46:42510) 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe6200640 1 -- 172.21.15.46:0/2063641550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 msgr2=0x557c92c218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe6200640 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe6200640 1 -- 172.21.15.46:0/2063641550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 msgr2=0x557c92c1efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe6200640 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c1efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe6200640 1 -- 172.21.15.46:0/2063641550 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c92889580 con 0x557c92c14310 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.902+0000 7f9fe5800640 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c1efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.903+0000 7f9fe6200640 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c146f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009f30 tx=0x7f9fd8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8b208567de84a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.903+0000 7f9fde000640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fc80a2b80 0x7f9fc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.903+0000 7f9fd2a00640 1 -- 172.21.15.46:0/2063641550 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8016070 con 0x557c92c14310 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.903+0000 7f9fde000640 1 -- 172.21.15.46:0/2932034065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9fcc0cf8f0 con 0x7f9fe0102790 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.903+0000 7f9fd2a00640 1 -- 172.21.15.46:0/2063641550 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd801bb40 con 0x557c92c14310 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.903+0000 7f9fe5800640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fc80a2b80 0x7f9fc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.903+0000 7f9fd2a00640 1 -- 172.21.15.46:0/2063641550 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8012ca0 con 0x557c92c14310 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 -- 172.21.15.46:0/2063641550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 msgr2=0x557c92c146f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c146f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009f30 tx=0x7f9fd8010800 comp rx=0 tx=0).stop 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9fe5800640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fc80a2b80 0x7f9fc80a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009230 tx=0x7f9fd40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 -- 172.21.15.46:0/2063641550 shutdown_connections 2024-09-10T07:43:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c1efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 --2- 172.21.15.46:0/2063641550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c146f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 -- 172.21.15.46:0/2063641550 >> 172.21.15.46:0/2063641550 conn(0x557c92c108a0 msgr2=0x557c92c10c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 -- 172.21.15.46:0/2063641550 shutdown_connections 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 -- 172.21.15.46:0/2063641550 wait complete. 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.904+0000 7f9feae42740 1 -- start start 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9feae42740 1 Processor -- start 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9feae42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c1c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9feae42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c14c30 0x557c92c1c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9feae42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c1f4f0 0x557c92c1ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9feae42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c92c220c0 con 0x557c92c1f4f0 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9feae42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557c92b35350 con 0x557c92c14310 2024-09-10T07:43:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9feae42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557c92c22240 con 0x557c92c14c30 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9feae42740 10 client.0 _ll_register_callbacks cb 0x557c92b58290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.905+0000 7f9fe6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c1f4f0 0x557c92c1ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c1f4f0 0x557c92c1ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42516/0 (socket says 172.21.15.46:42516) 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe6200640 1 -- 172.21.15.46:0/2037377735 learned_addr learned my addr 172.21.15.46:0/2037377735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe5800640 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c1c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c14c30 0x557c92c1c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe6200640 1 -- 172.21.15.46:0/2037377735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c14c30 msgr2=0x557c92c1c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe6200640 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c14c30 0x557c92c1c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe6200640 1 -- 172.21.15.46:0/2037377735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 msgr2=0x557c92c1c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe6200640 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c1c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe6200640 1 -- 172.21.15.46:0/2037377735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c92953960 con 0x557c92c1f4f0 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fe6200640 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c1f4f0 0x557c92c1ceb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8011010 tx=0x7f9fd80124c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437fad54589b621d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.906+0000 7f9fc7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.907+0000 7f9fc7400640 1 -- 172.21.15.46:0/2037377735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8016070 con 0x557c92c1f4f0 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.907+0000 7f9fc7400640 1 -- 172.21.15.46:0/2037377735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9fd8004a40 con 0x557c92c1f4f0 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.907+0000 7f9feae42740 1 -- 172.21.15.46:0/2037377735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd801bb40 con 0x557c92c1f4f0 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.907+0000 7f9fc7400640 1 -- 172.21.15.46:0/2037377735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9fd8004c40 con 0x557c92c1f4f0 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.907+0000 7f9feae42740 10 client.16770 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.907+0000 7f9feae42740 1 -- 172.21.15.46:0/2037377735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557c92c2dd20 con 0x557c92c1f4f0 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.907+0000 7f9fc7400640 1 -- 172.21.15.46:0/2037377735 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd801bb40 con 0x557c92c1f4f0 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.907+0000 7f9fc7400640 1 -- 172.21.15.46:0/2037377735 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f9fd800f040 con 0x557c92c1f4f0 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.908+0000 7f9feae42740 10 client.16770 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.908+0000 7f9feae42740 10 client.16770 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.908+0000 7f9feae42740 20 client.16770 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.908+0000 7f9feae42740 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557c92c2f1e0 0x557c92c315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.908+0000 7f9feae42740 4 client.16770 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.908+0000 7f9feae42740 1 -- 172.21.15.46:0/2037377735 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c92b87bc0 con 0x557c92c2f1e0 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.908+0000 7f9fe5800640 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557c92c2f1e0 0x557c92c315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.909+0000 7f9fe5800640 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557c92c2f1e0 0x557c92c315d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd83a54ba5a78226 server_cookie=a8e74867acea9edc in_seq=0 out_seq=0 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.909+0000 7f9fc7400640 10 client.16770 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.909+0000 7f9fc7400640 1 -- 172.21.15.46:0/2037377735 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c92b87bc0 con 0x557c92c2f1e0 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.909+0000 7f9fc7400640 10 client.16770 handle_command_reply: tid=0 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.909+0000 7f9feae42740 1 client.16770 shutdown 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 -- 172.21.15.46:0/2037377735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c1f4f0 msgr2=0x557c92c1ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c1f4f0 0x557c92c1ceb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8011010 tx=0x7f9fd80124c0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 -- 172.21.15.46:0/2037377735 shutdown_connections 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557c92c2f1e0 0x557c92c315d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c1f4f0 0x557c92c1ceb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c14c30 0x557c92c1c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 --2- 172.21.15.46:0/2037377735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c1c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 -- 172.21.15.46:0/2037377735 >> 172.21.15.46:0/2037377735 conn(0x557c92c108a0 msgr2=0x557c92c165d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9fc7400640 0 client.16770 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 -- 172.21.15.46:0/2037377735 shutdown_connections 2024-09-10T07:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 1 -- 172.21.15.46:0/2037377735 wait complete. 2024-09-10T07:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.910+0000 7f9feae42740 20 client.16770 trim_cache size 0 max 16384 2024-09-10T07:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.928+0000 7f9feae42740 1 Processor -- start 2024-09-10T07:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9feae42740 1 -- start start 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9feae42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14310 0x557c92c1ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9feae42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14c30 0x557c92c36de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9feae42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c37320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9feae42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b0ec0 con 0x557c92c14310 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9feae42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b2af0 con 0x557c92c14c30 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9feae42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b1590 con 0x557c92c1f4f0 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9fe5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14310 0x557c92c1ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9fe5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14310 0x557c92c1ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42522/0 (socket says 172.21.15.46:42522) 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9fe5800640 1 -- 172.21.15.46:0/806333114 learned_addr learned my addr 172.21.15.46:0/806333114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14c30 0x557c92c36de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.929+0000 7f9fe6200640 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c37320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fe5800640 1 -- 172.21.15.46:0/806333114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 msgr2=0x557c92c37320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fe5800640 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c37320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fe5800640 1 -- 172.21.15.46:0/806333114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14c30 msgr2=0x557c92c36de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fe5800640 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14c30 0x557c92c36de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fe5800640 1 -- 172.21.15.46:0/806333114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c92c318d0 con 0x557c92c14310 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fe5800640 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14310 0x557c92c1ad00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4054fd0 tx=0x7f9fd403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0bee4b0c6705ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fd2000640 1 -- 172.21.15.46:0/806333114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4035040 con 0x557c92c14310 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fd2000640 1 -- 172.21.15.46:0/806333114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd405a070 con 0x557c92c14310 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.930+0000 7f9fd2000640 1 -- 172.21.15.46:0/806333114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4054940 con 0x557c92c14310 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 -- 172.21.15.46:0/806333114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14310 msgr2=0x557c92c1ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14310 0x557c92c1ad00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4054fd0 tx=0x7f9fd403f000 comp rx=0 tx=0).stop 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 -- 172.21.15.46:0/806333114 shutdown_connections 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c37320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14c30 0x557c92c36de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 --2- 172.21.15.46:0/806333114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14310 0x557c92c1ad00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 -- 172.21.15.46:0/806333114 >> 172.21.15.46:0/806333114 conn(0x557c92c108a0 msgr2=0x557c92c0faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 -- 172.21.15.46:0/806333114 shutdown_connections 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.931+0000 7f9feae42740 1 -- 172.21.15.46:0/806333114 wait complete. 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.932+0000 7f9feae42740 1 -- start start 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.932+0000 7f9feae42740 1 Processor -- start 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.932+0000 7f9feae42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c23180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.932+0000 7f9feae42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.932+0000 7f9feae42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c23c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.932+0000 7f9feae42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b2360 con 0x557c92c14c30 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.932+0000 7f9feae42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b1c60 con 0x557c92c14310 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.932+0000 7f9feae42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01aeea0 con 0x557c92c1f4f0 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9feae42740 10 client.0 _ll_register_callbacks cb 0x7f9fd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe5800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c23180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c23c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe5800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c23180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41388/0 (socket says 172.21.15.46:41388) 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe5800640 1 -- 172.21.15.46:0/822761741 learned_addr learned my addr 172.21.15.46:0/822761741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c23c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:55192/0 (socket says 172.21.15.46:55192) 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe4e00640 1 -- 172.21.15.46:0/822761741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 msgr2=0x557c92c23c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c23c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe4e00640 1 -- 172.21.15.46:0/822761741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 msgr2=0x557c92c23180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c23180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe4e00640 1 -- 172.21.15.46:0/822761741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c92c1b3e0 con 0x557c92c14c30 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.933+0000 7f9fe5800640 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c23180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9fe4e00640 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c236c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x557c92c11c20 tx=0x7f9fcc09a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79bdd5f75593958 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9fc7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9fc7400640 1 -- 172.21.15.46:0/822761741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0d2e60 con 0x557c92c14c30 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9fc7400640 1 -- 172.21.15.46:0/822761741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9fcc0a4750 con 0x557c92c14c30 2024-09-10T07:43:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9feae42740 1 -- 172.21.15.46:0/822761741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd405a070 con 0x557c92c14c30 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9fc7400640 1 -- 172.21.15.46:0/822761741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9fcc09c470 con 0x557c92c14c30 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9feae42740 10 client.16776 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9feae42740 1 -- 172.21.15.46:0/822761741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557c92c99f30 con 0x557c92c14c30 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9fc7400640 1 -- 172.21.15.46:0/822761741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd405a070 con 0x557c92c14c30 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.934+0000 7f9fc7400640 1 -- 172.21.15.46:0/822761741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f9fcc0cfce0 con 0x557c92c14c30 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.935+0000 7f9feae42740 10 client.16776 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.935+0000 7f9feae42740 10 client.16776 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.935+0000 7f9feae42740 20 client.16776 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.935+0000 7f9feae42740 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557c92c9a770 0x557c92c9cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.935+0000 7f9feae42740 4 client.16776 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "409df7a9", "release": true, "await": true}] 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.935+0000 7f9feae42740 1 -- 172.21.15.46:0/822761741 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "409df7a9", "release": true, "await": true}) -- 0x557c92b820d0 con 0x557c92c9a770 2024-09-10T07:43:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.935+0000 7f9fe5800640 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557c92c9a770 0x557c92c9cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.936+0000 7f9fe5800640 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557c92c9a770 0x557c92c9cb90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a19cee4f85524472 server_cookie=252cf22ebeec765f in_seq=0 out_seq=0 2024-09-10T07:43:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.936+0000 7f9fc7400640 10 client.16776 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:38.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.967+0000 7f9fc7400640 1 -- 172.21.15.46:0/822761741 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557c92b820d0 con 0x557c92c9a770 2024-09-10T07:43:38.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.967+0000 7f9fc7400640 10 client.16776 handle_command_reply: tid=0 2024-09-10T07:43:38.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.967+0000 7f9feae42740 1 client.16776 shutdown 2024-09-10T07:43:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.967+0000 7f9feae42740 1 -- 172.21.15.46:0/822761741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 msgr2=0x557c92c236c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.967+0000 7f9feae42740 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c236c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x557c92c11c20 tx=0x7f9fcc09a280 comp rx=0 tx=0).stop 2024-09-10T07:43:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.972+0000 7f9feae42740 1 -- 172.21.15.46:0/822761741 shutdown_connections 2024-09-10T07:43:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.972+0000 7f9feae42740 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557c92c9a770 0x557c92c9cb90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.972+0000 7f9feae42740 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557c92c1f4f0 0x557c92c23c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.972+0000 7f9feae42740 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c92c14c30 0x557c92c236c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.972+0000 7f9feae42740 1 --2- 172.21.15.46:0/822761741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c92c14310 0x557c92c23180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.972+0000 7f9feae42740 1 -- 172.21.15.46:0/822761741 >> 172.21.15.46:0/822761741 conn(0x557c92c108a0 msgr2=0x557c92c0faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.972+0000 7f9fc7400640 0 client.16776 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.974+0000 7f9feae42740 1 -- 172.21.15.46:0/822761741 shutdown_connections 2024-09-10T07:43:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.974+0000 7f9feae42740 1 -- 172.21.15.46:0/822761741 wait complete. 2024-09-10T07:43:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.975+0000 7f9feae42740 20 client.16776 trim_cache size 0 max 16384 2024-09-10T07:43:38.975 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:43:38.975 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 44, 2024-09-10T07:43:38.975 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:43:38.975 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 24, 2024-09-10T07:43:38.975 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:43:38.975 INFO:teuthology.orchestra.run.smithi046.stdout: "409df7a9": { 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 24, 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 140.1, 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:43:38.976 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:43:38.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:38.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:38.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:38.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:38.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:43:38.977 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:43:38.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 -- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fc80a2b80 msgr2=0x7f9fc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fc80a2b80 0x7f9fc80a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009230 tx=0x7f9fd40058d0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 -- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0102790 msgr2=0x7f9fe0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:38.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0102790 0x7f9fe0074a30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e9f0 tx=0x7f9fcc00edf0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 -- 172.21.15.46:0/2932034065 shutdown_connections 2024-09-10T07:43:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe0106f10 0x7f9fe0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9fc80a2b80 0x7f9fc80a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe0102790 0x7f9fe0074a30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 --2- 172.21.15.46:0/2932034065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe0101de0 0x7f9fe00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 -- 172.21.15.46:0/2932034065 >> 172.21.15.46:0/2932034065 conn(0x7f9fe00fd8c0 msgr2=0x7f9fe00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 -- 172.21.15.46:0/2932034065 shutdown_connections 2024-09-10T07:43:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:43:38.979+0000 7f9fe8200640 1 -- 172.21.15.46:0/2932034065 wait complete. 2024-09-10T07:43:39.844 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '409df7a9', total seconds elapsed: 25.4 2024-09-10T07:43:40.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:39 smithi181 ceph-mon[35397]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 205 KiB/s wr, 112 op/s 2024-09-10T07:43:40.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:39 smithi063 ceph-mon[35020]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 205 KiB/s wr, 112 op/s 2024-09-10T07:43:40.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:39 smithi046 ceph-mon[30705]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 205 KiB/s wr, 112 op/s 2024-09-10T07:43:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 380.0 s, 279.9 tps, lat 1321.437 ms stddev 4384.285 2024-09-10T07:43:42.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:41 smithi181 ceph-mon[35397]: pgmap v611: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 307 KiB/s wr, 142 op/s 2024-09-10T07:43:42.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:41 smithi063 ceph-mon[35020]: pgmap v611: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 307 KiB/s wr, 142 op/s 2024-09-10T07:43:42.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:41 smithi046 ceph-mon[30705]: pgmap v611: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 307 KiB/s wr, 142 op/s 2024-09-10T07:43:44.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:43 smithi181 ceph-mon[35397]: pgmap v612: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T07:43:44.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:43 smithi063 ceph-mon[35020]: pgmap v612: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T07:43:44.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:43 smithi046 ceph-mon[30705]: pgmap v612: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T07:43:45.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 385.0 s, 904.2 tps, lat 112.213 ms stddev 131.212 2024-09-10T07:43:46.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:45 smithi181 ceph-mon[35397]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-09-10T07:43:46.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:45 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:43:46.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:45 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:43:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:45 smithi063 ceph-mon[35020]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-09-10T07:43:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:45 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:43:46.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:45 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:43:46.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:45 smithi046 ceph-mon[30705]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-09-10T07:43:46.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:45 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:43:46.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:45 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:43:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.069+0000 7f8f9d800640 1 -- 172.21.15.46:0/2287589996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98072d20 msgr2=0x7f8f98073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.069+0000 7f8f9d800640 1 --2- 172.21.15.46:0/2287589996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98072d20 0x7f8f98073180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009900 tx=0x7f8f8402a440 comp rx=0 tx=0).stop 2024-09-10T07:43:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.071+0000 7f8f9d800640 1 -- 172.21.15.46:0/2287589996 shutdown_connections 2024-09-10T07:43:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.071+0000 7f8f9d800640 1 --2- 172.21.15.46:0/2287589996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f980737d0 0x7f8f98112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.071+0000 7f8f9d800640 1 --2- 172.21.15.46:0/2287589996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98072d20 0x7f8f98073180 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.071+0000 7f8f9d800640 1 --2- 172.21.15.46:0/2287589996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f98109930 0x7f8f98109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.071+0000 7f8f9d800640 1 -- 172.21.15.46:0/2287589996 >> 172.21.15.46:0/2287589996 conn(0x7f8f9806ece0 msgr2=0x7f8f98071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.071+0000 7f8f9d800640 1 -- 172.21.15.46:0/2287589996 shutdown_connections 2024-09-10T07:43:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.072+0000 7f8f9d800640 1 -- 172.21.15.46:0/2287589996 wait complete. 2024-09-10T07:43:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.073+0000 7f8f9d800640 1 Processor -- start 2024-09-10T07:43:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.073+0000 7f8f9d800640 1 -- start start 2024-09-10T07:43:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f9d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f98072d20 0x7f8f981a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f9d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f980737d0 0x7f8f981a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f9d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98109930 0x7f8f981a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f9d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98116080 con 0x7f8f98072d20 2024-09-10T07:43:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f9d800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98115f00 con 0x7f8f98109930 2024-09-10T07:43:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f9d800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98116200 con 0x7f8f980737d0 2024-09-10T07:43:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f97400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98109930 0x7f8f981a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f97400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98109930 0x7f8f981a9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36508/0 (socket says 172.21.15.46:36508) 2024-09-10T07:43:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.074+0000 7f8f97400640 1 -- 172.21.15.46:0/3577828505 learned_addr learned my addr 172.21.15.46:0/3577828505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.075+0000 7f8f96000640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f980737d0 0x7f8f981a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.075+0000 7f8f97400640 1 -- 172.21.15.46:0/3577828505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f980737d0 msgr2=0x7f8f981a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.075+0000 7f8f97400640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f980737d0 0x7f8f981a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.075+0000 7f8f97400640 1 -- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f98072d20 msgr2=0x7f8f981a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.075+0000 7f8f97400640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f98072d20 0x7f8f981a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.075+0000 7f8f97400640 1 -- 172.21.15.46:0/3577828505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f981aa3d0 con 0x7f8f98109930 2024-09-10T07:43:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.076+0000 7f8f97400640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98109930 0x7f8f981a9d60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000cce0 tx=0x7f8f9000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.076+0000 7f8f8f400640 1 -- 172.21.15.46:0/3577828505 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9000f450 con 0x7f8f98109930 2024-09-10T07:43:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.076+0000 7f8f8f400640 1 -- 172.21.15.46:0/3577828505 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f9000fad0 con 0x7f8f98109930 2024-09-10T07:43:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.076+0000 7f8f8f400640 1 -- 172.21.15.46:0/3577828505 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90018c30 con 0x7f8f98109930 2024-09-10T07:43:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.076+0000 7f8f9d800640 1 -- 172.21.15.46:0/3577828505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f981aa6c0 con 0x7f8f98109930 2024-09-10T07:43:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.076+0000 7f8f9d800640 1 -- 172.21.15.46:0/3577828505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9810c7b0 con 0x7f8f98109930 2024-09-10T07:43:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.082+0000 7f8fa06bb740 1 Processor -- start 2024-09-10T07:43:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.084+0000 7f8f8f400640 1 -- 172.21.15.46:0/3577828505 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f9000f5f0 con 0x7f8f98109930 2024-09-10T07:43:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.087+0000 7f8f8f400640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f7c0a2d00 0x7f8f7c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.087+0000 7f8f8f400640 1 -- 172.21.15.46:0/3577828505 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8f900cfab0 con 0x7f8f98109930 2024-09-10T07:43:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.087+0000 7f8fa06bb740 1 -- start start 2024-09-10T07:43:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8f96000640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f7c0a2d00 0x7f8f7c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d34490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 0x55e8c0d39490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0d3bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e8c0c99100 con 0x55e8c0d349d0 2024-09-10T07:43:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e8c0c411b0 con 0x55e8c0d34090 2024-09-10T07:43:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e8c0c950c0 con 0x55e8c0d39a60 2024-09-10T07:43:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8f96a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d34490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8f96a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d34490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36518/0 (socket says 172.21.15.46:36518) 2024-09-10T07:43:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8f96a00640 1 -- 172.21.15.46:0/2720079672 learned_addr learned my addr 172.21.15.46:0/2720079672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.088+0000 7f8f97400640 1 --2- 172.21.15.46:0/2720079672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0d3bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.089+0000 7f8f96a00640 1 -- 172.21.15.46:0/2720079672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 msgr2=0x55e8c0d3bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.089+0000 7f8f96a00640 1 --2- 172.21.15.46:0/2720079672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0d3bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.089+0000 7f8f96a00640 1 -- 172.21.15.46:0/2720079672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 msgr2=0x55e8c0d39490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:43:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.089+0000 7f8f96a00640 1 --2- 172.21.15.46:0/2720079672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 0x55e8c0d39490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.089+0000 7f8f96a00640 1 -- 172.21.15.46:0/2720079672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c09ac210 con 0x55e8c0d34090 2024-09-10T07:43:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.089+0000 7f8f96a00640 1 --2- 172.21.15.46:0/2720079672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d34490 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8f880097c0 tx=0x7f8f88009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59eaf1c7b7a49ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.089+0000 7f8f83e00640 1 -- 172.21.15.46:0/2720079672 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88015020 con 0x55e8c0d34090 2024-09-10T07:43:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.090+0000 7f8f83e00640 1 -- 172.21.15.46:0/2720079672 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f88011420 con 0x55e8c0d34090 2024-09-10T07:43:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.090+0000 7f8f83e00640 1 -- 172.21.15.46:0/2720079672 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f880105c0 con 0x55e8c0d34090 2024-09-10T07:43:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2720079672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 msgr2=0x55e8c0d34490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/2720079672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d34490 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8f880097c0 tx=0x7f8f88009c80 comp rx=0 tx=0).stop 2024-09-10T07:43:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8f96000640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f7c0a2d00 0x7f8f7c0a51c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8f84006010 tx=0x7f8f840094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2720079672 shutdown_connections 2024-09-10T07:43:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/2720079672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0d3bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/2720079672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 0x55e8c0d39490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/2720079672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d34490 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2720079672 >> 172.21.15.46:0/2720079672 conn(0x55e8c0d2f700 msgr2=0x55e8c0d2fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2720079672 shutdown_connections 2024-09-10T07:43:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.091+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2720079672 wait complete. 2024-09-10T07:43:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.092+0000 7f8fa06bb740 1 -- start start 2024-09-10T07:43:47.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.092+0000 7f8fa06bb740 1 Processor -- start 2024-09-10T07:43:47.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.093+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.093+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d3e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.093+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d45740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.093+0000 7f8f96a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.093+0000 7f8f96a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36534/0 (socket says 172.21.15.46:36534) 2024-09-10T07:43:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.094+0000 7f8f96000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d3e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.094+0000 7f8f96000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d3e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38034/0 (socket says 172.21.15.46:38034) 2024-09-10T07:43:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.096+0000 7f8f96a00640 1 -- 172.21.15.46:0/4011574939 learned_addr learned my addr 172.21.15.46:0/4011574939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.096+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e8c0d3ca30 con 0x55e8c0d39a60 2024-09-10T07:43:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.096+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e8c0c22090 con 0x55e8c0d34090 2024-09-10T07:43:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.096+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e8c0d3cbb0 con 0x55e8c0d349d0 2024-09-10T07:43:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.096+0000 7f8fa06bb740 10 client.0 _ll_register_callbacks cb 0x55e8c0c76f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.096+0000 7f8f97400640 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d45740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.097+0000 7f8f96a00640 1 -- 172.21.15.46:0/4011574939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 msgr2=0x55e8c0d3e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.097+0000 7f8f96a00640 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d3e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.097+0000 7f8f96a00640 1 -- 172.21.15.46:0/4011574939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 msgr2=0x55e8c0d45740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.097+0000 7f8f96a00640 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d45740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.097+0000 7f8f96a00640 1 -- 172.21.15.46:0/4011574939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c0a78620 con 0x55e8c0d34090 2024-09-10T07:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.098+0000 7f8f96a00640 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e1c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006c70 tx=0x7f8f88011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc58c0329c051a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.098+0000 7f8f80c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:43:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.098+0000 7f8f80c00640 1 -- 172.21.15.46:0/4011574939 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88020070 con 0x55e8c0d34090 2024-09-10T07:43:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.098+0000 7f8f80c00640 1 -- 172.21.15.46:0/4011574939 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8f8800bce0 con 0x55e8c0d34090 2024-09-10T07:43:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.098+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f88011420 con 0x55e8c0d34090 2024-09-10T07:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.100+0000 7f8f80c00640 1 -- 172.21.15.46:0/4011574939 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8f88004020 con 0x55e8c0d34090 2024-09-10T07:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.100+0000 7f8fa06bb740 10 client.27227 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.100+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e8c0d48530 con 0x55e8c0d34090 2024-09-10T07:43:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.100+0000 7f8f80c00640 1 -- 172.21.15.46:0/4011574939 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f88011420 con 0x55e8c0d34090 2024-09-10T07:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.101+0000 7f8f80c00640 1 -- 172.21.15.46:0/4011574939 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f8f8801a420 con 0x55e8c0d34090 2024-09-10T07:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.101+0000 7f8fa06bb740 10 client.27227 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.101+0000 7f8fa06bb740 10 client.27227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.101+0000 7f8fa06bb740 20 client.27227 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.101+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e8c0d49a80 0x55e8c0d4bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.101+0000 7f8fa06bb740 4 client.27227 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:43:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.102+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8c0ca6830 con 0x55e8c0d49a80 2024-09-10T07:43:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.103+0000 7f8f96000640 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e8c0d49a80 0x55e8c0d4bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.105+0000 7f8f96000640 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e8c0d49a80 0x55e8c0d4bec0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d3e261d019fd9c server_cookie=81dcc2588145c7d5 in_seq=0 out_seq=0 2024-09-10T07:43:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.105+0000 7f8f80c00640 10 client.27227 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.106+0000 7f8f80c00640 1 -- 172.21.15.46:0/4011574939 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8c0ca6830 con 0x55e8c0d49a80 2024-09-10T07:43:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.106+0000 7f8f80c00640 10 client.27227 handle_command_reply: tid=0 2024-09-10T07:43:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.107+0000 7f8fa06bb740 1 client.27227 shutdown 2024-09-10T07:43:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.108+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 msgr2=0x55e8c0d3e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.108+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e1c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006c70 tx=0x7f8f88011880 comp rx=0 tx=0).stop 2024-09-10T07:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.108+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 shutdown_connections 2024-09-10T07:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.109+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e8c0d49a80 0x55e8c0d4bec0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.109+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d45740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.109+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.109+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4011574939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e1c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.109+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 >> 172.21.15.46:0/4011574939 conn(0x55e8c0d2f700 msgr2=0x55e8c0d30f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.109+0000 7f8f80c00640 0 client.27227 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.109+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 shutdown_connections 2024-09-10T07:43:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.110+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4011574939 wait complete. 2024-09-10T07:43:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.112+0000 7f8fa06bb740 20 client.27227 trim_cache size 0 max 16384 2024-09-10T07:43:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.140+0000 7f8fa06bb740 1 Processor -- start 2024-09-10T07:43:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.140+0000 7f8fa06bb740 1 -- start start 2024-09-10T07:43:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d2fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981b94f0 con 0x55e8c0d39a60 2024-09-10T07:43:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981bb120 con 0x55e8c0d34090 2024-09-10T07:43:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981b9bc0 con 0x55e8c0d349d0 2024-09-10T07:43:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8f96000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8f96000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d391a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38046/0 (socket says 172.21.15.46:38046) 2024-09-10T07:43:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8f96000640 1 -- 172.21.15.46:0/2631423165 learned_addr learned my addr 172.21.15.46:0/2631423165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8f96a00640 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.141+0000 7f8f97400640 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d2fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.142+0000 7f8f96000640 1 -- 172.21.15.46:0/2631423165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 msgr2=0x55e8c0d3e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.142+0000 7f8f96000640 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.142+0000 7f8f96000640 1 -- 172.21.15.46:0/2631423165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 msgr2=0x55e8c0d2fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.142+0000 7f8f96000640 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d2fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.142+0000 7f8f96000640 1 -- 172.21.15.46:0/2631423165 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c0d38e20 con 0x55e8c0d349d0 2024-09-10T07:43:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.142+0000 7f8f96a00640 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.142+0000 7f8f97400640 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d2fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:43:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.142+0000 7f8f96000640 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d391a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f84000c00 tx=0x7f8f84040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f2c93c952b8cef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.143+0000 7f8f83400640 1 -- 172.21.15.46:0/2631423165 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f840046c0 con 0x55e8c0d349d0 2024-09-10T07:43:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.143+0000 7f8f83400640 1 -- 172.21.15.46:0/2631423165 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f84056070 con 0x55e8c0d349d0 2024-09-10T07:43:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.143+0000 7f8f83400640 1 -- 172.21.15.46:0/2631423165 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84031b60 con 0x55e8c0d349d0 2024-09-10T07:43:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2631423165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 msgr2=0x55e8c0d391a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d391a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f84000c00 tx=0x7f8f84040000 comp rx=0 tx=0).stop 2024-09-10T07:43:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2631423165 shutdown_connections 2024-09-10T07:43:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d39a60 0x55e8c0d2fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d349d0 0x55e8c0d391a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/2631423165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d3e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2631423165 >> 172.21.15.46:0/2631423165 conn(0x55e8c0d2f700 msgr2=0x55e8c0d3df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2631423165 shutdown_connections 2024-09-10T07:43:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.144+0000 7f8fa06bb740 1 -- 172.21.15.46:0/2631423165 wait complete. 2024-09-10T07:43:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.145+0000 7f8fa06bb740 1 -- start start 2024-09-10T07:43:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.145+0000 7f8fa06bb740 1 Processor -- start 2024-09-10T07:43:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d50b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 0x55e8c0d510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8fa06bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0db26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981ba990 con 0x55e8c0d349d0 2024-09-10T07:43:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981ba290 con 0x55e8c0d34090 2024-09-10T07:43:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8fa06bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981b74d0 con 0x55e8c0d39a60 2024-09-10T07:43:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8fa06bb740 10 client.0 _ll_register_callbacks cb 0x7f8f900077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:43:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8f97400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0db26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8f97400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0db26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38050/0 (socket says 172.21.15.46:38050) 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8f97400640 1 -- 172.21.15.46:0/4101883690 learned_addr learned my addr 172.21.15.46:0/4101883690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.147+0000 7f8f96a00640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d50b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f96000640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 0x55e8c0d510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f97400640 1 -- 172.21.15.46:0/4101883690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 msgr2=0x55e8c0d50b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f97400640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d50b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f97400640 1 -- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 msgr2=0x55e8c0d510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f97400640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 0x55e8c0d510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f97400640 1 -- 172.21.15.46:0/4101883690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c0db2c00 con 0x55e8c0d39a60 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f96a00640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d50b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f96000640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 0x55e8c0d510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:43:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f97400640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0db26c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000ab20 tx=0x7f8f90018690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab741c1bf9f53127 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:43:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f80c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:43:47.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:46 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:43:47.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:46 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:43:47.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:46 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:43:47.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:46 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:43:47.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:46 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:43:47.151 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:46 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.148+0000 7f8f80c00640 1 -- 172.21.15.46:0/4101883690 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f900badd0 con 0x55e8c0d39a60 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.149+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4101883690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f84056070 con 0x55e8c0d39a60 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.149+0000 7f8f80c00640 1 -- 172.21.15.46:0/4101883690 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8f900a4770 con 0x55e8c0d39a60 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.149+0000 7f8f80c00640 1 -- 172.21.15.46:0/4101883690 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8f9009c690 con 0x55e8c0d39a60 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.149+0000 7f8fa06bb740 10 client.27172 fetch_fsmap learned FSMap version 44 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.149+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4101883690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e8c0db3ef0 con 0x55e8c0d39a60 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.149+0000 7f8f80c00640 1 -- 172.21.15.46:0/4101883690 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f84056070 con 0x55e8c0d39a60 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.150+0000 7f8f80c00640 1 -- 172.21.15.46:0/4101883690 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f8f900850d0 con 0x55e8c0d39a60 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.150+0000 7f8fa06bb740 10 client.27172 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:43:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.150+0000 7f8fa06bb740 10 client.27172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:43:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.150+0000 7f8fa06bb740 20 client.27172 populate_metadata read hostname 'smithi046' 2024-09-10T07:43:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.150+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e8c0db48b0 0x55e8c0db6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:43:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.150+0000 7f8fa06bb740 4 client.27172 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:43:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.150+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4101883690 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8c0ca0b80 con 0x55e8c0db48b0 2024-09-10T07:43:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.150+0000 7f8f96a00640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e8c0db48b0 0x55e8c0db6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:43:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.151+0000 7f8f80c00640 10 client.27172 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.151+0000 7f8f96a00640 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e8c0db48b0 0x55e8c0db6cd0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e7500f3cd6d531 server_cookie=789a7ab8e975e86e in_seq=0 out_seq=0 2024-09-10T07:43:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:46 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:43:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:46 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:43:47.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:46 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:43:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.162+0000 7f8f80c00640 1 -- 172.21.15.46:0/4101883690 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8c0ca0b80 con 0x55e8c0db48b0 2024-09-10T07:43:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.162+0000 7f8f80c00640 10 client.27172 handle_command_reply: tid=0 2024-09-10T07:43:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.162+0000 7f8fa06bb740 1 client.27172 shutdown 2024-09-10T07:43:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.162+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4101883690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 msgr2=0x55e8c0db26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.162+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0db26c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000ab20 tx=0x7f8f90018690 comp rx=0 tx=0).stop 2024-09-10T07:43:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4101883690 shutdown_connections 2024-09-10T07:43:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e8c0db48b0 0x55e8c0db6cd0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8c0d39a60 0x55e8c0db26c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8c0d349d0 0x55e8c0d510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 1 --2- 172.21.15.46:0/4101883690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8c0d34090 0x55e8c0d50b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4101883690 >> 172.21.15.46:0/4101883690 conn(0x55e8c0d2f700 msgr2=0x55e8c0d3df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8f80c00640 0 client.27172 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4101883690 shutdown_connections 2024-09-10T07:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 1 -- 172.21.15.46:0/4101883690 wait complete. 2024-09-10T07:43:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.163+0000 7f8fa06bb740 20 client.27172 trim_cache size 0 max 16384 2024-09-10T07:43:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:43:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:43:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:43:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:43:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.165+0000 7f8f9d800640 1 -- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f7c0a2d00 msgr2=0x7f8f7c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.165+0000 7f8f9d800640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f7c0a2d00 0x7f8f7c0a51c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8f84006010 tx=0x7f8f840094f0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.165+0000 7f8f9d800640 1 -- 172.21.15.46:0/3577828505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98109930 msgr2=0x7f8f981a9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:43:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.165+0000 7f8f9d800640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98109930 0x7f8f981a9d60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000cce0 tx=0x7f8f9000a620 comp rx=0 tx=0).stop 2024-09-10T07:43:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.166+0000 7f8f9d800640 1 -- 172.21.15.46:0/3577828505 shutdown_connections 2024-09-10T07:43:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.166+0000 7f8f9d800640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f98109930 0x7f8f981a9d60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.166+0000 7f8f9d800640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f980737d0 0x7f8f981a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.166+0000 7f8f9d800640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8f7c0a2d00 0x7f8f7c0a51c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.166+0000 7f8f9d800640 1 --2- 172.21.15.46:0/3577828505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f98072d20 0x7f8f981a27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:43:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.166+0000 7f8f9d800640 1 -- 172.21.15.46:0/3577828505 >> 172.21.15.46:0/3577828505 conn(0x7f8f9806ece0 msgr2=0x7f8f9810a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:43:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.166+0000 7f8f9d800640 1 -- 172.21.15.46:0/3577828505 shutdown_connections 2024-09-10T07:43:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:43:47.166+0000 7f8f9d800640 1 -- 172.21.15.46:0/3577828505 wait complete. 2024-09-10T07:43:48.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:47 smithi181 ceph-mon[35397]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 4.9 MiB/s wr, 376 op/s 2024-09-10T07:43:48.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:47 smithi063 ceph-mon[35020]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 4.9 MiB/s wr, 376 op/s 2024-09-10T07:43:48.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:47 smithi046 ceph-mon[30705]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 4.9 MiB/s wr, 376 op/s 2024-09-10T07:43:50.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:49 smithi181 ceph-mon[35397]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-09-10T07:43:50.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:49 smithi063 ceph-mon[35020]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-09-10T07:43:50.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:49 smithi046 ceph-mon[30705]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-09-10T07:43:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 390.0 s, 805.4 tps, lat 121.466 ms stddev 147.179 2024-09-10T07:43:52.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:51 smithi181 ceph-mon[35397]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.6 MiB/s wr, 483 op/s 2024-09-10T07:43:52.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:51 smithi063 ceph-mon[35020]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.6 MiB/s wr, 483 op/s 2024-09-10T07:43:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:51 smithi046 ceph-mon[30705]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.6 MiB/s wr, 483 op/s 2024-09-10T07:43:54.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:53 smithi181 ceph-mon[35397]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.9 MiB/s wr, 521 op/s 2024-09-10T07:43:54.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:53 smithi063 ceph-mon[35020]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.9 MiB/s wr, 521 op/s 2024-09-10T07:43:54.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:53 smithi046 ceph-mon[30705]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.9 MiB/s wr, 521 op/s 2024-09-10T07:43:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 395.0 s, 1012.8 tps, lat 100.102 ms stddev 121.961 2024-09-10T07:43:56.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:55 smithi181 ceph-mon[35397]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.9 MiB/s wr, 494 op/s 2024-09-10T07:43:56.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:55 smithi063 ceph-mon[35020]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.9 MiB/s wr, 494 op/s 2024-09-10T07:43:56.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:55 smithi046 ceph-mon[30705]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.9 MiB/s wr, 494 op/s 2024-09-10T07:43:58.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:57 smithi181 ceph-mon[35397]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 683 op/s 2024-09-10T07:43:58.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:57 smithi063 ceph-mon[35020]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 683 op/s 2024-09-10T07:43:58.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:57 smithi046 ceph-mon[30705]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 683 op/s 2024-09-10T07:44:00.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:43:59 smithi181 ceph-mon[35397]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 539 op/s 2024-09-10T07:44:00.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:43:59 smithi063 ceph-mon[35020]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 539 op/s 2024-09-10T07:44:00.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:43:59 smithi046 ceph-mon[30705]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 539 op/s 2024-09-10T07:44:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.354+0000 7f26ede00640 1 -- 172.21.15.46:0/800237133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81003c0 msgr2=0x7f26e81027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.354+0000 7f26ede00640 1 --2- 172.21.15.46:0/800237133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81003c0 0x7f26e81027d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009900 tx=0x7f26d402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.354+0000 7f26ede00640 1 -- 172.21.15.46:0/800237133 shutdown_connections 2024-09-10T07:44:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.354+0000 7f26ede00640 1 --2- 172.21.15.46:0/800237133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e81056c0 0x7f26e8105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.354+0000 7f26ede00640 1 --2- 172.21.15.46:0/800237133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e8102d10 0x7f26e8105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.354+0000 7f26ede00640 1 --2- 172.21.15.46:0/800237133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81003c0 0x7f26e81027d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.355+0000 7f26ede00640 1 -- 172.21.15.46:0/800237133 >> 172.21.15.46:0/800237133 conn(0x7f26e80fc1b0 msgr2=0x7f26e80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.355+0000 7f26ede00640 1 -- 172.21.15.46:0/800237133 shutdown_connections 2024-09-10T07:44:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.355+0000 7f26ede00640 1 -- 172.21.15.46:0/800237133 wait complete. 2024-09-10T07:44:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.356+0000 7f26ede00640 1 Processor -- start 2024-09-10T07:44:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.356+0000 7f26ede00640 1 -- start start 2024-09-10T07:44:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.356+0000 7f26ede00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e81003c0 0x7f26e810b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e8102d10 0x7f26e810be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26ede00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81056c0 0x7f26e81ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26ede00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8113e50 con 0x7f26e8102d10 2024-09-10T07:44:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26ede00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8113cd0 con 0x7f26e81056c0 2024-09-10T07:44:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26ede00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8113fd0 con 0x7f26e81003c0 2024-09-10T07:44:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26e7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81056c0 0x7f26e81ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e81003c0 0x7f26e810b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26e7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81056c0 0x7f26e81ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34158/0 (socket says 172.21.15.46:34158) 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26e7400640 1 -- 172.21.15.46:0/2532447428 learned_addr learned my addr 172.21.15.46:0/2532447428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.357+0000 7f26e6000640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e8102d10 0x7f26e810be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.358+0000 7f26e7400640 1 -- 172.21.15.46:0/2532447428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e81003c0 msgr2=0x7f26e810b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.358+0000 7f26e7400640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e81003c0 0x7f26e810b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.358+0000 7f26e7400640 1 -- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e8102d10 msgr2=0x7f26e810be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.358+0000 7f26e7400640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e8102d10 0x7f26e810be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.358+0000 7f26e7400640 1 -- 172.21.15.46:0/2532447428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e810ef90 con 0x7f26e81056c0 2024-09-10T07:44:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.358+0000 7f26e7400640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81056c0 0x7f26e81ae500 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f26e000eaa0 tx=0x7f26e000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.359+0000 7f26eca00640 1 -- 172.21.15.46:0/2532447428 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e000cdb0 con 0x7f26e81056c0 2024-09-10T07:44:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.359+0000 7f26eca00640 1 -- 172.21.15.46:0/2532447428 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e0005020 con 0x7f26e81056c0 2024-09-10T07:44:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.359+0000 7f26eca00640 1 -- 172.21.15.46:0/2532447428 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00116b0 con 0x7f26e81056c0 2024-09-10T07:44:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.359+0000 7f26ede00640 1 -- 172.21.15.46:0/2532447428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81aeb00 con 0x7f26e81056c0 2024-09-10T07:44:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.359+0000 7f26ede00640 1 -- 172.21.15.46:0/2532447428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81aefd0 con 0x7f26e81056c0 2024-09-10T07:44:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.363+0000 7f26f0bd9740 1 Processor -- start 2024-09-10T07:44:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.365+0000 7f26eca00640 1 -- 172.21.15.46:0/2532447428 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26e0005560 con 0x7f26e81056c0 2024-09-10T07:44:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.366+0000 7f26f0bd9740 1 -- start start 2024-09-10T07:44:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.366+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bf5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.366+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bfa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.366+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bfcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.366+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557793b5a100 con 0x557793bf5090 2024-09-10T07:44:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.366+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557793b021b0 con 0x557793bf59d0 2024-09-10T07:44:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.366+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557793b560c0 con 0x557793bfaa60 2024-09-10T07:44:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.367+0000 7f26e6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bf5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.368+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bfa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.368+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bfa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34170/0 (socket says 172.21.15.46:34170) 2024-09-10T07:44:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.368+0000 7f26e6a00640 1 -- 172.21.15.46:0/2018065167 learned_addr learned my addr 172.21.15.46:0/2018065167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.368+0000 7f26e6a00640 1 -- 172.21.15.46:0/2018065167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 msgr2=0x557793bfcf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:44:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.368+0000 7f26e6a00640 1 --2- 172.21.15.46:0/2018065167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bfcf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.369+0000 7f26eca00640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26cc0a2cb0 0x7f26cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.369+0000 7f26e6a00640 1 -- 172.21.15.46:0/2018065167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 msgr2=0x557793bf5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.369+0000 7f26e6a00640 1 --2- 172.21.15.46:0/2018065167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bf5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.369+0000 7f26eca00640 1 -- 172.21.15.46:0/2532447428 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f26e00cf530 con 0x7f26e81056c0 2024-09-10T07:44:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.369+0000 7f26e6a00640 1 -- 172.21.15.46:0/2018065167 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55779386d210 con 0x557793bf59d0 2024-09-10T07:44:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.370+0000 7f26e6000640 1 --2- 172.21.15.46:0/2018065167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bf5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.370+0000 7f26e7400640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26cc0a2cb0 0x7f26cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.371+0000 7f26e6a00640 1 --2- 172.21.15.46:0/2018065167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bfa490 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f26d4000c00 tx=0x7f26d4004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a28e5d07d87ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.371+0000 7f26e7400640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26cc0a2cb0 0x7f26cc0a5170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f26e000efd0 tx=0x7f26e00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.372+0000 7f26d3e00640 1 -- 172.21.15.46:0/2018065167 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4035040 con 0x557793bf59d0 2024-09-10T07:44:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.372+0000 7f26d3e00640 1 -- 172.21.15.46:0/2018065167 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4005520 con 0x557793bf59d0 2024-09-10T07:44:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.372+0000 7f26d3e00640 1 -- 172.21.15.46:0/2018065167 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4031910 con 0x557793bf59d0 2024-09-10T07:44:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.373+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2018065167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 msgr2=0x557793bfa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.373+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2018065167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bfa490 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f26d4000c00 tx=0x7f26d4004170 comp rx=0 tx=0).stop 2024-09-10T07:44:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.374+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2018065167 shutdown_connections 2024-09-10T07:44:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.374+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2018065167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bfcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.374+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2018065167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bfa490 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.374+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2018065167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bf5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.374+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2018065167 >> 172.21.15.46:0/2018065167 conn(0x557793bf0700 msgr2=0x557793bf0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.374+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2018065167 shutdown_connections 2024-09-10T07:44:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.374+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2018065167 wait complete. 2024-09-10T07:44:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.375+0000 7f26f0bd9740 1 -- start start 2024-09-10T07:44:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.376+0000 7f26f0bd9740 1 Processor -- start 2024-09-10T07:44:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.378+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.378+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.378+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793c06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.378+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557793bfd9d0 con 0x557793bf5090 2024-09-10T07:44:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.378+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557793ae3090 con 0x557793bf59d0 2024-09-10T07:44:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.378+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557793bfdb50 con 0x557793bfaa60 2024-09-10T07:44:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.378+0000 7f26f0bd9740 10 client.0 _ll_register_callbacks cb 0x557793b37f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34182/0 (socket says 172.21.15.46:34182) 2024-09-10T07:44:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6a00640 1 -- 172.21.15.46:0/2454861632 learned_addr learned my addr 172.21.15.46:0/2454861632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6000640 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793c06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6a00640 1 -- 172.21.15.46:0/2454861632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 msgr2=0x557793c06860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6a00640 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793c06860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6a00640 1 -- 172.21.15.46:0/2454861632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 msgr2=0x557793bff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6a00640 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.379+0000 7f26e6a00640 1 -- 172.21.15.46:0/2454861632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557793939620 con 0x557793bf59d0 2024-09-10T07:44:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.380+0000 7f26e6a00640 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bff820 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f26d40314e0 tx=0x7f26d4009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155b32995741846e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.382+0000 7f26d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.382+0000 7f26d0c00640 1 -- 172.21.15.46:0/2454861632 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4053060 con 0x557793bf59d0 2024-09-10T07:44:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.382+0000 7f26d0c00640 1 -- 172.21.15.46:0/2454861632 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f26d4004980 con 0x557793bf59d0 2024-09-10T07:44:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.382+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2454861632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d4005520 con 0x557793bf59d0 2024-09-10T07:44:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.382+0000 7f26d0c00640 1 -- 172.21.15.46:0/2454861632 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f26d4035050 con 0x557793bf59d0 2024-09-10T07:44:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.382+0000 7f26f0bd9740 10 client.27242 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.382+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2454861632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557793c095c0 con 0x557793bf59d0 2024-09-10T07:44:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.383+0000 7f26d0c00640 1 -- 172.21.15.46:0/2454861632 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4005520 con 0x557793bf59d0 2024-09-10T07:44:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.385+0000 7f26d0c00640 1 -- 172.21.15.46:0/2454861632 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f26d40035f0 con 0x557793bf59d0 2024-09-10T07:44:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.387+0000 7f26f0bd9740 10 client.27242 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.388+0000 7f26f0bd9740 10 client.27242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.388+0000 7f26f0bd9740 20 client.27242 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.388+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557793c0a910 0x557793c0cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.388+0000 7f26f0bd9740 4 client.27242 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.388+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2454861632 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557793b67830 con 0x557793c0a910 2024-09-10T07:44:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.389+0000 7f26e6000640 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557793c0a910 0x557793c0cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.390+0000 7f26d0c00640 10 client.27242 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.390+0000 7f26e6000640 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557793c0a910 0x557793c0cd00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52cc6d910b2a3a71 server_cookie=ec916ead4db049cd in_seq=0 out_seq=0 2024-09-10T07:44:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.391+0000 7f26d0c00640 1 -- 172.21.15.46:0/2454861632 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557793b67830 con 0x557793c0a910 2024-09-10T07:44:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.391+0000 7f26d0c00640 10 client.27242 handle_command_reply: tid=0 2024-09-10T07:44:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.391+0000 7f26f0bd9740 1 client.27242 shutdown 2024-09-10T07:44:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.393+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2454861632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 msgr2=0x557793bff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.393+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bff820 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f26d40314e0 tx=0x7f26d4009660 comp rx=0 tx=0).stop 2024-09-10T07:44:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.394+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2454861632 shutdown_connections 2024-09-10T07:44:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.394+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557793c0a910 0x557793c0cd00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.394+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793c06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.394+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793bff820 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.394+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/2454861632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.394+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2454861632 >> 172.21.15.46:0/2454861632 conn(0x557793bf0700 msgr2=0x557793bf1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.395+0000 7f26d0c00640 0 client.27242 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.396+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2454861632 shutdown_connections 2024-09-10T07:44:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.396+0000 7f26f0bd9740 1 -- 172.21.15.46:0/2454861632 wait complete. 2024-09-10T07:44:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.396+0000 7f26f0bd9740 20 client.27242 trim_cache size 0 max 16384 2024-09-10T07:44:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.439+0000 7f26f0bd9740 1 Processor -- start 2024-09-10T07:44:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.439+0000 7f26f0bd9740 1 -- start start 2024-09-10T07:44:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bf1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793c06820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bf2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81b9540 con 0x557793bf5090 2024-09-10T07:44:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81bb170 con 0x557793bf59d0 2024-09-10T07:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81b9c10 con 0x557793bfaa60 2024-09-10T07:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26e6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bf2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26e6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bf2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:52160/0 (socket says 172.21.15.46:52160) 2024-09-10T07:44:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.440+0000 7f26e6000640 1 -- 172.21.15.46:0/1531511829 learned_addr learned my addr 172.21.15.46:0/1531511829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.441+0000 7f26e6000640 1 -- 172.21.15.46:0/1531511829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 msgr2=0x557793c06820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.441+0000 7f26e6000640 1 --2- 172.21.15.46:0/1531511829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793c06820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.441+0000 7f26e6000640 1 -- 172.21.15.46:0/1531511829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 msgr2=0x557793bf1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.441+0000 7f26e6000640 1 --2- 172.21.15.46:0/1531511829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bf1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.441+0000 7f26e6000640 1 -- 172.21.15.46:0/1531511829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557793bebd60 con 0x557793bfaa60 2024-09-10T07:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.442+0000 7f26e6000640 1 --2- 172.21.15.46:0/1531511829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bf2030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x557793bfe8b0 tx=0x7f26d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc97efe543b0977b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.442+0000 7f26d3400640 1 -- 172.21.15.46:0/1531511829 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8004110 con 0x557793bfaa60 2024-09-10T07:44:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.444+0000 7f26d3400640 1 -- 172.21.15.46:0/1531511829 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d8005500 con 0x557793bfaa60 2024-09-10T07:44:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.444+0000 7f26d3400640 1 -- 172.21.15.46:0/1531511829 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8004df0 con 0x557793bfaa60 2024-09-10T07:44:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.445+0000 7f26f0bd9740 1 -- 172.21.15.46:0/1531511829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 msgr2=0x557793bf2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.445+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/1531511829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bf2030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x557793bfe8b0 tx=0x7f26d80118d0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.446+0000 7f26f0bd9740 1 -- 172.21.15.46:0/1531511829 shutdown_connections 2024-09-10T07:44:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.446+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/1531511829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bfaa60 0x557793bf2030 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.446+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/1531511829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793c06820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.446+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/1531511829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bf5090 0x557793bf1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.446+0000 7f26f0bd9740 1 -- 172.21.15.46:0/1531511829 >> 172.21.15.46:0/1531511829 conn(0x557793bf0700 msgr2=0x557793bef880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.446+0000 7f26f0bd9740 1 -- 172.21.15.46:0/1531511829 shutdown_connections 2024-09-10T07:44:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.446+0000 7f26f0bd9740 1 -- 172.21.15.46:0/1531511829 wait complete. 2024-09-10T07:44:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.446+0000 7f26f0bd9740 1 -- start start 2024-09-10T07:44:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.447+0000 7f26f0bd9740 1 Processor -- start 2024-09-10T07:44:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.449+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bf5090 0x557793c01b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.450+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793c02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.450+0000 7f26e6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bf5090 0x557793c01b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.451+0000 7f26e6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bf5090 0x557793c01b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:52168/0 (socket says 172.21.15.46:52168) 2024-09-10T07:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.451+0000 7f26f0bd9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bfaa60 0x557793c05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.451+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81ba9e0 con 0x557793bfaa60 2024-09-10T07:44:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.451+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81ba2e0 con 0x557793bf59d0 2024-09-10T07:44:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.451+0000 7f26f0bd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81b7520 con 0x557793bf5090 2024-09-10T07:44:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.451+0000 7f26f0bd9740 10 client.0 _ll_register_callbacks cb 0x7f26d4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.451+0000 7f26e6000640 1 -- 172.21.15.46:0/3619349808 learned_addr learned my addr 172.21.15.46:0/3619349808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.453+0000 7f26e6a00640 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793c02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.453+0000 7f26e6000640 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bfaa60 0x557793c05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.454+0000 7f26e6000640 1 -- 172.21.15.46:0/3619349808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 msgr2=0x557793c02070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.454+0000 7f26e6000640 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793c02070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.454+0000 7f26e6000640 1 -- 172.21.15.46:0/3619349808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bfaa60 msgr2=0x557793c05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.454+0000 7f26e6000640 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bfaa60 0x557793c05090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.454+0000 7f26e6000640 1 -- 172.21.15.46:0/3619349808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557793bfcf00 con 0x557793bf5090 2024-09-10T07:44:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.454+0000 7f26e6a00640 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793c02070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.455+0000 7f26e6000640 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bf5090 0x557793c01b30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005780 tx=0x7f26d80056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43164ae8ccc7ec1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.455+0000 7f26d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.455+0000 7f26d0c00640 1 -- 172.21.15.46:0/3619349808 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8004d00 con 0x557793bf5090 2024-09-10T07:44:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.455+0000 7f26d0c00640 1 -- 172.21.15.46:0/3619349808 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f26d8004ea0 con 0x557793bf5090 2024-09-10T07:44:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.455+0000 7f26f0bd9740 1 -- 172.21.15.46:0/3619349808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d8005500 con 0x557793bf5090 2024-09-10T07:44:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.456+0000 7f26d0c00640 1 -- 172.21.15.46:0/3619349808 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f26d803a050 con 0x557793bf5090 2024-09-10T07:44:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.457+0000 7f26f0bd9740 10 client.27187 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.457+0000 7f26f0bd9740 1 -- 172.21.15.46:0/3619349808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557793c067e0 con 0x557793bf5090 2024-09-10T07:44:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.457+0000 7f26d0c00640 1 -- 172.21.15.46:0/3619349808 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d8005500 con 0x557793bf5090 2024-09-10T07:44:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.457+0000 7f26d0c00640 1 -- 172.21.15.46:0/3619349808 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f26d801e390 con 0x557793bf5090 2024-09-10T07:44:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.457+0000 7f26f0bd9740 10 client.27187 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.457+0000 7f26f0bd9740 10 client.27187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.457+0000 7f26f0bd9740 20 client.27187 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.457+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557793c86900 0x557793c88d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.458+0000 7f26f0bd9740 4 client.27187 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:44:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.458+0000 7f26f0bd9740 1 -- 172.21.15.46:0/3619349808 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557793b61b80 con 0x557793c86900 2024-09-10T07:44:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.458+0000 7f26e6a00640 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557793c86900 0x557793c88d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.460+0000 7f26e6a00640 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557793c86900 0x557793c88d20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd6c6e6a7e1383a server_cookie=302f75c836fe1889 in_seq=0 out_seq=0 2024-09-10T07:44:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.460+0000 7f26d0c00640 10 client.27187 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.492+0000 7f26d0c00640 1 -- 172.21.15.46:0/3619349808 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557793b61b80 con 0x557793c86900 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.492+0000 7f26d0c00640 10 client.27187 handle_command_reply: tid=0 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 client.27187 shutdown 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 -- 172.21.15.46:0/3619349808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bf5090 msgr2=0x557793c01b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bf5090 0x557793c01b30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005780 tx=0x7f26d80056b0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 -- 172.21.15.46:0/3619349808 shutdown_connections 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557793c86900 0x557793c88d20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557793bfaa60 0x557793c05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557793bf59d0 0x557793c02070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26d0c00640 0 client.27187 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 --2- 172.21.15.46:0/3619349808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557793bf5090 0x557793c01b30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.493+0000 7f26f0bd9740 1 -- 172.21.15.46:0/3619349808 >> 172.21.15.46:0/3619349808 conn(0x557793bf0700 msgr2=0x557793bf0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.494+0000 7f26f0bd9740 1 -- 172.21.15.46:0/3619349808 shutdown_connections 2024-09-10T07:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.495+0000 7f26f0bd9740 1 -- 172.21.15.46:0/3619349808 wait complete. 2024-09-10T07:44:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.495+0000 7f26f0bd9740 20 client.27187 trim_cache size 0 max 16384 2024-09-10T07:44:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:44:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:44:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:44:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.496+0000 7f26ede00640 1 -- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26cc0a2cb0 msgr2=0x7f26cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.496+0000 7f26ede00640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26cc0a2cb0 0x7f26cc0a5170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f26e000efd0 tx=0x7f26e00058d0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.496+0000 7f26ede00640 1 -- 172.21.15.46:0/2532447428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81056c0 msgr2=0x7f26e81ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.496+0000 7f26ede00640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81056c0 0x7f26e81ae500 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f26e000eaa0 tx=0x7f26e000ef60 comp rx=0 tx=0).stop 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.497+0000 7f26ede00640 1 -- 172.21.15.46:0/2532447428 shutdown_connections 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.497+0000 7f26ede00640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81056c0 0x7f26e81ae500 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.497+0000 7f26ede00640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26cc0a2cb0 0x7f26cc0a5170 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.497+0000 7f26ede00640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e8102d10 0x7f26e810be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.497+0000 7f26ede00640 1 --2- 172.21.15.46:0/2532447428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26e81003c0 0x7f26e810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.497+0000 7f26ede00640 1 -- 172.21.15.46:0/2532447428 >> 172.21.15.46:0/2532447428 conn(0x7f26e80fc1b0 msgr2=0x7f26e8103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.497+0000 7f26ede00640 1 -- 172.21.15.46:0/2532447428 shutdown_connections 2024-09-10T07:44:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:00.497+0000 7f26ede00640 1 -- 172.21.15.46:0/2532447428 wait complete. 2024-09-10T07:44:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 400.0 s, 759.8 tps, lat 130.135 ms stddev 140.674 2024-09-10T07:44:02.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:01 smithi181 ceph-mon[35397]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 666 op/s 2024-09-10T07:44:02.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:01 smithi063 ceph-mon[35020]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 666 op/s 2024-09-10T07:44:02.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:01 smithi046 ceph-mon[30705]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 666 op/s 2024-09-10T07:44:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:03 smithi181 ceph-mon[35397]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 587 op/s 2024-09-10T07:44:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:03 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:44:04.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:03 smithi063 ceph-mon[35020]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 587 op/s 2024-09-10T07:44:04.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:03 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:44:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:03 smithi046 ceph-mon[30705]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 587 op/s 2024-09-10T07:44:04.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:03 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:44:05.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:44:05.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:44:05.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:04 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:44:05.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:44:05.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:44:05.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:04 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:44:05.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:44:05.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:44:05.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:04 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:44:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 405.0 s, 650.2 tps, lat 155.886 ms stddev 207.627 2024-09-10T07:44:06.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:05 smithi181 ceph-mon[35397]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 524 op/s 2024-09-10T07:44:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:05 smithi063 ceph-mon[35020]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 524 op/s 2024-09-10T07:44:06.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:05 smithi046 ceph-mon[30705]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 524 op/s 2024-09-10T07:44:06.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:44:06.944 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.740+0000 7f38e7000640 1 -- 172.21.15.46:0/2491604704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0100500 msgr2=0x7f38e010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.740+0000 7f38e7000640 1 --2- 172.21.15.46:0/2491604704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0100500 0x7f38e010ea20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009c10 tx=0x7f38d802a700 comp rx=0 tx=0).stop 2024-09-10T07:44:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.740+0000 7f38e7000640 1 -- 172.21.15.46:0/2491604704 shutdown_connections 2024-09-10T07:44:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.740+0000 7f38e7000640 1 --2- 172.21.15.46:0/2491604704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0100500 0x7f38e010ea20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.740+0000 7f38e7000640 1 --2- 172.21.15.46:0/2491604704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e00ffb60 0x7f38e00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.740+0000 7f38e7000640 1 --2- 172.21.15.46:0/2491604704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e0106b80 0x7f38e0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.740+0000 7f38e7000640 1 -- 172.21.15.46:0/2491604704 >> 172.21.15.46:0/2491604704 conn(0x7f38e00fba90 msgr2=0x7f38e00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.741+0000 7f38e7000640 1 -- 172.21.15.46:0/2491604704 shutdown_connections 2024-09-10T07:44:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.741+0000 7f38e7000640 1 -- 172.21.15.46:0/2491604704 wait complete. 2024-09-10T07:44:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.742+0000 7f38e7000640 1 Processor -- start 2024-09-10T07:44:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.743+0000 7f38e7000640 1 -- start start 2024-09-10T07:44:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.743+0000 7f38e7000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e00ffb60 0x7f38e0071440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.743+0000 7f38e7000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0106b80 0x7f38e0071980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.743+0000 7f38e7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e006c700 0x7f38e006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.743+0000 7f38e7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38e010fe00 con 0x7f38e006c700 2024-09-10T07:44:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.743+0000 7f38e7000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38e010fc80 con 0x7f38e00ffb60 2024-09-10T07:44:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.743+0000 7f38e7000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38e010ff80 con 0x7f38e0106b80 2024-09-10T07:44:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e00ffb60 0x7f38e0071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e00ffb60 0x7f38e0071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40466/0 (socket says 172.21.15.46:40466) 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38dfe00640 1 -- 172.21.15.46:0/1749126650 learned_addr learned my addr 172.21.15.46:0/1749126650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38df400640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0106b80 0x7f38e0071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38dfe00640 1 -- 172.21.15.46:0/1749126650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0106b80 msgr2=0x7f38e0071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0106b80 0x7f38e0071980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38dfe00640 1 -- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e006c700 msgr2=0x7f38e006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e006c700 0x7f38e006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.744+0000 7f38dfe00640 1 -- 172.21.15.46:0/1749126650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e006d2b0 con 0x7f38e00ffb60 2024-09-10T07:44:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.745+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e00ffb60 0x7f38e0071440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f38d000b590 tx=0x7f38d000ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:07.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.747+0000 7f38dcc00640 1 -- 172.21.15.46:0/1749126650 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0008500 con 0x7f38e00ffb60 2024-09-10T07:44:07.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.747+0000 7f38dcc00640 1 -- 172.21.15.46:0/1749126650 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d0008b80 con 0x7f38e00ffb60 2024-09-10T07:44:07.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.747+0000 7f38dcc00640 1 -- 172.21.15.46:0/1749126650 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0008500 con 0x7f38e00ffb60 2024-09-10T07:44:07.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.747+0000 7f38e7000640 1 -- 172.21.15.46:0/1749126650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e00701b0 con 0x7f38e00ffb60 2024-09-10T07:44:07.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.747+0000 7f38e7000640 1 -- 172.21.15.46:0/1749126650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e00705e0 con 0x7f38e00ffb60 2024-09-10T07:44:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.757+0000 7f38e9bc1740 1 Processor -- start 2024-09-10T07:44:07.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.760+0000 7f38dcc00640 1 -- 172.21.15.46:0/1749126650 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f38d001eb40 con 0x7f38e00ffb60 2024-09-10T07:44:07.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.763+0000 7f38dcc00640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38c40a2e40 0x7f38c40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.763+0000 7f38dcc00640 1 -- 172.21.15.46:0/1749126650 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f38d0009040 con 0x7f38e00ffb60 2024-09-10T07:44:07.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.764+0000 7f38e5000640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38c40a2e40 0x7f38c40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.764+0000 7f38e9bc1740 1 -- start start 2024-09-10T07:44:07.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a7ecf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55af4a7533e0 con 0x55af4a7f7d70 2024-09-10T07:44:07.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af4a6fad80 con 0x55af4a7ed4b0 2024-09-10T07:44:07.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55af4a74f3a0 con 0x55af4a7ecb90 2024-09-10T07:44:07.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a7ecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38df400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a7ecf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48914/0 (socket says 172.21.15.46:48914) 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.765+0000 7f38dfe00640 1 -- 172.21.15.46:0/1328726741 learned_addr learned my addr 172.21.15.46:0/1328726741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38dfe00640 1 -- 172.21.15.46:0/1328726741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 msgr2=0x55af4a7f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1328726741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38dfe00640 1 -- 172.21.15.46:0/1328726741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 msgr2=0x55af4a7fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1328726741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38dfe00640 1 -- 172.21.15.46:0/1328726741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af4a719ae0 con 0x55af4a7ecb90 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1328726741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a7ecf70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38d00a36a0 tx=0x7f38d00a35f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fadb2204a60390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38d5600640 1 -- 172.21.15.46:0/1328726741 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d00ac380 con 0x55af4a7ecb90 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38d5600640 1 -- 172.21.15.46:0/1328726741 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d00a1a10 con 0x55af4a7ecb90 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.766+0000 7f38d5600640 1 -- 172.21.15.46:0/1328726741 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0008e50 con 0x55af4a7ecb90 2024-09-10T07:44:07.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.767+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1328726741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 msgr2=0x55af4a7ecf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.767+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1328726741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a7ecf70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38d00a36a0 tx=0x7f38d00a35f0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.768+0000 7f38e5000640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38c40a2e40 0x7f38c40a5300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f38d8004af0 tx=0x7f38d8009320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.768+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1328726741 shutdown_connections 2024-09-10T07:44:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.768+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1328726741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.768+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1328726741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.768+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1328726741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a7ecf70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.768+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1328726741 >> 172.21.15.46:0/1328726741 conn(0x55af4a7e9120 msgr2=0x55af4a7e9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.768+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1328726741 shutdown_connections 2024-09-10T07:44:07.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.769+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1328726741 wait complete. 2024-09-10T07:44:07.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.769+0000 7f38e9bc1740 1 -- start start 2024-09-10T07:44:07.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.770+0000 7f38e9bc1740 1 Processor -- start 2024-09-10T07:44:07.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.770+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.770+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.770+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7f7d70 0x55af4a7f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.771+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.771+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40474/0 (socket says 172.21.15.46:40474) 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.771+0000 7f38df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.771+0000 7f38df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46484/0 (socket says 172.21.15.46:46484) 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.772+0000 7f38dfe00640 1 -- 172.21.15.46:0/1998884064 learned_addr learned my addr 172.21.15.46:0/1998884064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.772+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55af4a7fa9e0 con 0x55af4a7ed4b0 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.772+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af4a70e0f0 con 0x55af4a7ecb90 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.772+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55af4a7fab60 con 0x55af4a7f7d70 2024-09-10T07:44:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.772+0000 7f38e5000640 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7f7d70 0x55af4a7f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.772+0000 7f38dfe00640 1 -- 172.21.15.46:0/1998884064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7f7d70 msgr2=0x55af4a7f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:07.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.772+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7f7d70 0x55af4a7f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.772+0000 7f38e9bc1740 10 client.0 _ll_register_callbacks cb 0x55af4a6ff9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.774+0000 7f38dfe00640 1 -- 172.21.15.46:0/1998884064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7ed4b0 msgr2=0x55af4a7f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.774+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.774+0000 7f38dfe00640 1 -- 172.21.15.46:0/1998884064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4a4c43c0 con 0x55af4a7ecb90 2024-09-10T07:44:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.774+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d00a1a10 con 0x55af4a7ecb90 2024-09-10T07:44:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.774+0000 7f38e5000640 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7f7d70 0x55af4a7f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.774+0000 7f38df400640 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:07.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.775+0000 7f38dfe00640 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7f4d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f38d00d7350 tx=0x7f38d00a3890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6429cd0ed80560ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:07.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.775+0000 7f38ce000640 10 client.27257 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:07.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.776+0000 7f38ce000640 1 -- 172.21.15.46:0/1998884064 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d00bb030 con 0x55af4a7ecb90 2024-09-10T07:44:07.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.776+0000 7f38ce000640 1 -- 172.21.15.46:0/1998884064 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f38d00a17b0 con 0x55af4a7ecb90 2024-09-10T07:44:07.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.776+0000 7f38ce000640 1 -- 172.21.15.46:0/1998884064 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f38d00ac380 con 0x55af4a7ecb90 2024-09-10T07:44:07.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.776+0000 7f38e9bc1740 10 client.27257 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:07.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.776+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=44}) -- 0x55af4a806620 con 0x55af4a7ecb90 2024-09-10T07:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.778+0000 7f38ce000640 1 -- 172.21.15.46:0/1998884064 <== mon.2 v2:172.21.15.63:3300/0 4 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f38d00a2b60 con 0x55af4a7ecb90 2024-09-10T07:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.778+0000 7f38e9bc1740 10 client.27257 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.779+0000 7f38e9bc1740 10 client.27257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.779+0000 7f38e9bc1740 20 client.27257 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:07.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.779+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af4a807970 0x55af4a809d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.779+0000 7f38e9bc1740 4 client.27257 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:07.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.779+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af4a760950 con 0x55af4a807970 2024-09-10T07:44:07.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.779+0000 7f38df400640 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af4a807970 0x55af4a809d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.781+0000 7f38df400640 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af4a807970 0x55af4a809d60 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8dfd688db7f9df6 server_cookie=f275a09c47344d94 in_seq=0 out_seq=0 2024-09-10T07:44:07.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.781+0000 7f38ce000640 10 client.27257 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:07.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.783+0000 7f38ce000640 1 -- 172.21.15.46:0/1998884064 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af4a760950 con 0x55af4a807970 2024-09-10T07:44:07.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.783+0000 7f38ce000640 10 client.27257 handle_command_reply: tid=0 2024-09-10T07:44:07.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.783+0000 7f38e9bc1740 1 client.27257 shutdown 2024-09-10T07:44:07.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 msgr2=0x55af4a7f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7f4d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f38d00d7350 tx=0x7f38d00a3890 comp rx=0 tx=0).stop 2024-09-10T07:44:07.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 shutdown_connections 2024-09-10T07:44:07.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af4a807970 0x55af4a809d60 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7f7d70 0x55af4a7f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7ed4b0 0x55af4a7f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/1998884064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7f4d30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38ce000640 0 client.27257 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:07.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.784+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 >> 172.21.15.46:0/1998884064 conn(0x55af4a7e9120 msgr2=0x55af4a7eee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:07.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.785+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 shutdown_connections 2024-09-10T07:44:07.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.785+0000 7f38e9bc1740 1 -- 172.21.15.46:0/1998884064 wait complete. 2024-09-10T07:44:07.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.785+0000 7f38e9bc1740 20 client.27257 trim_cache size 0 max 16384 2024-09-10T07:44:07.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.822+0000 7f38e9bc1740 1 Processor -- start 2024-09-10T07:44:07.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.822+0000 7f38e9bc1740 1 -- start start 2024-09-10T07:44:07.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ed4b0 0x55af4a80f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a80f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01b54b0 con 0x55af4a7f7d70 2024-09-10T07:44:07.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01b70e0 con 0x55af4a7ecb90 2024-09-10T07:44:07.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01b5b80 con 0x55af4a7ed4b0 2024-09-10T07:44:07.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40478/0 (socket says 172.21.15.46:40478) 2024-09-10T07:44:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38dfe00640 1 -- 172.21.15.46:0/3550186033 learned_addr learned my addr 172.21.15.46:0/3550186033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.823+0000 7f38df400640 1 --2- 172.21.15.46:0/3550186033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ed4b0 0x55af4a80f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.824+0000 7f38df400640 1 -- 172.21.15.46:0/3550186033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 msgr2=0x55af4a7eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.824+0000 7f38df400640 1 --2- 172.21.15.46:0/3550186033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.824+0000 7f38df400640 1 -- 172.21.15.46:0/3550186033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 msgr2=0x55af4a80f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.824+0000 7f38df400640 1 --2- 172.21.15.46:0/3550186033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a80f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.824+0000 7f38df400640 1 -- 172.21.15.46:0/3550186033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af4a7ed2e0 con 0x55af4a7ed4b0 2024-09-10T07:44:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.825+0000 7f38df400640 1 --2- 172.21.15.46:0/3550186033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ed4b0 0x55af4a80f330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55af4a7f6370 tx=0x7f38c800b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5fa66339ce364 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.825+0000 7f38d4c00640 1 -- 172.21.15.46:0/3550186033 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8005020 con 0x55af4a7ed4b0 2024-09-10T07:44:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.826+0000 7f38d4c00640 1 -- 172.21.15.46:0/3550186033 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c80047a0 con 0x55af4a7ed4b0 2024-09-10T07:44:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.826+0000 7f38d4c00640 1 -- 172.21.15.46:0/3550186033 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003920 con 0x55af4a7ed4b0 2024-09-10T07:44:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.827+0000 7f38e9bc1740 1 -- 172.21.15.46:0/3550186033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ed4b0 msgr2=0x55af4a80f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.827+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/3550186033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ed4b0 0x55af4a80f330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55af4a7f6370 tx=0x7f38c800b390 comp rx=0 tx=0).stop 2024-09-10T07:44:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.828+0000 7f38e9bc1740 1 -- 172.21.15.46:0/3550186033 shutdown_connections 2024-09-10T07:44:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.830+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/3550186033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a80f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.830+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/3550186033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ed4b0 0x55af4a80f330 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.830+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/3550186033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ecb90 0x55af4a7eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.830+0000 7f38e9bc1740 1 -- 172.21.15.46:0/3550186033 >> 172.21.15.46:0/3550186033 conn(0x55af4a7e9120 msgr2=0x55af4a7ee5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.833+0000 7f38e9bc1740 1 -- 172.21.15.46:0/3550186033 shutdown_connections 2024-09-10T07:44:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.833+0000 7f38e9bc1740 1 -- 172.21.15.46:0/3550186033 wait complete. 2024-09-10T07:44:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.836+0000 7f38e9bc1740 1 -- start start 2024-09-10T07:44:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.836+0000 7f38e9bc1740 1 Processor -- start 2024-09-10T07:44:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.837+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a800410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.837+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7fb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.837+0000 7f38e9bc1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.837+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01b6950 con 0x55af4a7f7d70 2024-09-10T07:44:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.837+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01b6250 con 0x55af4a7ed4b0 2024-09-10T07:44:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.837+0000 7f38e9bc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01b3490 con 0x55af4a7ecb90 2024-09-10T07:44:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.837+0000 7f38e9bc1740 10 client.0 _ll_register_callbacks cb 0x7f38d00a3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.838+0000 7f38e5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.838+0000 7f38e5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46500/0 (socket says 172.21.15.46:46500) 2024-09-10T07:44:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.838+0000 7f38e5000640 1 -- 172.21.15.46:0/60203867 learned_addr learned my addr 172.21.15.46:0/60203867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.838+0000 7f38e5000640 1 -- 172.21.15.46:0/60203867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 msgr2=0x55af4a800410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:44:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.838+0000 7f38e5000640 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a800410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.838+0000 7f38e5000640 1 -- 172.21.15.46:0/60203867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 msgr2=0x55af4a7fb740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:44:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.839+0000 7f38df400640 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.839+0000 7f38e5000640 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7fb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.839+0000 7f38e5000640 1 -- 172.21.15.46:0/60203867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4a7fc370 con 0x55af4a7f7d70 2024-09-10T07:44:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.840+0000 7f38df400640 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.841+0000 7f38e5000640 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fbc80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f38e01077c0 tx=0x7f38d8030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5f68b2f02ba25d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.841+0000 7f38ce000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:44:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.842+0000 7f38ce000640 1 -- 172.21.15.46:0/60203867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8045050 con 0x55af4a7f7d70 2024-09-10T07:44:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.842+0000 7f38ce000640 1 -- 172.21.15.46:0/60203867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f38d8031850 con 0x55af4a7f7d70 2024-09-10T07:44:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.842+0000 7f38e9bc1740 1 -- 172.21.15.46:0/60203867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c80047a0 con 0x55af4a7f7d70 2024-09-10T07:44:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.842+0000 7f38ce000640 1 -- 172.21.15.46:0/60203867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f38d8058050 con 0x55af4a7f7d70 2024-09-10T07:44:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.842+0000 7f38e9bc1740 10 client.16800 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.842+0000 7f38e9bc1740 1 -- 172.21.15.46:0/60203867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55af4a8721d0 con 0x55af4a7f7d70 2024-09-10T07:44:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.843+0000 7f38ce000640 1 -- 172.21.15.46:0/60203867 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c80047a0 con 0x55af4a7f7d70 2024-09-10T07:44:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.844+0000 7f38ce000640 1 -- 172.21.15.46:0/60203867 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f38d803c070 con 0x55af4a7f7d70 2024-09-10T07:44:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.844+0000 7f38e9bc1740 10 client.16800 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.844+0000 7f38e9bc1740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.844+0000 7f38e9bc1740 20 client.16800 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.844+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af4a872a30 0x55af4a874e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.844+0000 7f38e9bc1740 4 client.16800 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:44:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.844+0000 7f38e9bc1740 1 -- 172.21.15.46:0/60203867 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af4a75ae60 con 0x55af4a872a30 2024-09-10T07:44:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.845+0000 7f38dfe00640 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af4a872a30 0x55af4a874e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.845+0000 7f38dfe00640 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af4a872a30 0x55af4a874e50 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e3603bf9fad82c server_cookie=d8a4255a9e93cc04 in_seq=0 out_seq=0 2024-09-10T07:44:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.845+0000 7f38ce000640 10 client.16800 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.846+0000 7f38ce000640 1 -- 172.21.15.46:0/60203867 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55af4a75ae60 con 0x55af4a872a30 2024-09-10T07:44:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.846+0000 7f38ce000640 10 client.16800 handle_command_reply: tid=0 2024-09-10T07:44:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.846+0000 7f38e9bc1740 1 client.16800 shutdown 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.847+0000 7f38e9bc1740 1 -- 172.21.15.46:0/60203867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 msgr2=0x55af4a7fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.847+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fbc80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f38e01077c0 tx=0x7f38d8030cc0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.847+0000 7f38e9bc1740 1 -- 172.21.15.46:0/60203867 shutdown_connections 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55af4a872a30 0x55af4a874e50 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4a7f7d70 0x55af4a7fbc80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af4a7ed4b0 0x55af4a7fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38e9bc1740 1 --2- 172.21.15.46:0/60203867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af4a7ecb90 0x55af4a800410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38ce000640 0 client.16800 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:07.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38e9bc1740 1 -- 172.21.15.46:0/60203867 >> 172.21.15.46:0/60203867 conn(0x55af4a7e9120 msgr2=0x55af4a7eee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:07.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38e9bc1740 1 -- 172.21.15.46:0/60203867 shutdown_connections 2024-09-10T07:44:07.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38e9bc1740 1 -- 172.21.15.46:0/60203867 wait complete. 2024-09-10T07:44:07.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.848+0000 7f38e9bc1740 20 client.16800 trim_cache size 0 max 16384 2024-09-10T07:44:07.849 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:44:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:44:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:44:07.850 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:44:07.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 -- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38c40a2e40 msgr2=0x7f38c40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38c40a2e40 0x7f38c40a5300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f38d8004af0 tx=0x7f38d8009320 comp rx=0 tx=0).stop 2024-09-10T07:44:07.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 -- 172.21.15.46:0/1749126650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e00ffb60 msgr2=0x7f38e0071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:07.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e00ffb60 0x7f38e0071440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f38d000b590 tx=0x7f38d000ba50 comp rx=0 tx=0).stop 2024-09-10T07:44:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 -- 172.21.15.46:0/1749126650 shutdown_connections 2024-09-10T07:44:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f38c40a2e40 0x7f38c40a5300 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e006c700 0x7f38e006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e0106b80 0x7f38e0071980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 --2- 172.21.15.46:0/1749126650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e00ffb60 0x7f38e0071440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.850+0000 7f38e7000640 1 -- 172.21.15.46:0/1749126650 >> 172.21.15.46:0/1749126650 conn(0x7f38e00fba90 msgr2=0x7f38e00fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.855+0000 7f38e7000640 1 -- 172.21.15.46:0/1749126650 shutdown_connections 2024-09-10T07:44:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:07.855+0000 7f38e7000640 1 -- 172.21.15.46:0/1749126650 wait complete. 2024-09-10T07:44:08.320 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:07 smithi046 ceph-mon[30705]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 694 op/s 2024-09-10T07:44:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:07 smithi181 ceph-mon[35397]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 694 op/s 2024-09-10T07:44:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:07 smithi063 ceph-mon[35020]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 694 op/s 2024-09-10T07:44:09.768 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:44:09.769 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:44:10.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:09 smithi046 ceph-mon[30705]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.6 MiB/s wr, 530 op/s 2024-09-10T07:44:10.200 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:09 smithi181 ceph-mon[35397]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.6 MiB/s wr, 530 op/s 2024-09-10T07:44:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:09 smithi063 ceph-mon[35020]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.6 MiB/s wr, 530 op/s 2024-09-10T07:44:10.462 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:10.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 410.0 s, 297.4 tps, lat 322.775 ms stddev 498.322 2024-09-10T07:44:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.300+0000 7fcaaec00640 1 -- 172.21.15.46:0/4100990047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 msgr2=0x7fcaa810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.300+0000 7fcaaec00640 1 --2- 172.21.15.46:0/4100990047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 0x7fcaa810e240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fca94009860 tx=0x7fca9402a340 comp rx=0 tx=0).stop 2024-09-10T07:44:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.300+0000 7fcaaec00640 1 -- 172.21.15.46:0/4100990047 shutdown_connections 2024-09-10T07:44:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.300+0000 7fcaaec00640 1 --2- 172.21.15.46:0/4100990047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa810e970 0x7fcaa8115310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.300+0000 7fcaaec00640 1 --2- 172.21.15.46:0/4100990047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 0x7fcaa810e240 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.300+0000 7fcaaec00640 1 --2- 172.21.15.46:0/4100990047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa8109450 0x7fcaa8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.300+0000 7fcaaec00640 1 -- 172.21.15.46:0/4100990047 >> 172.21.15.46:0/4100990047 conn(0x7fcaa8073180 msgr2=0x7fcaa8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.301+0000 7fcaaec00640 1 -- 172.21.15.46:0/4100990047 shutdown_connections 2024-09-10T07:44:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.301+0000 7fcaaec00640 1 -- 172.21.15.46:0/4100990047 wait complete. 2024-09-10T07:44:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.302+0000 7fcaaec00640 1 Processor -- start 2024-09-10T07:44:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.302+0000 7fcaaec00640 1 -- start start 2024-09-10T07:44:11.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.303+0000 7fcaaec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa8109450 0x7fcaa81a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:11.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.303+0000 7fcaaec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 0x7fcaa81a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:11.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.303+0000 7fcaaec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa810e970 0x7fcaa81aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:11.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.303+0000 7fcaad800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa8109450 0x7fcaa81a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:11.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.304+0000 7fcaad800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa8109450 0x7fcaa81a7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48954/0 (socket says 172.21.15.46:48954) 2024-09-10T07:44:11.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.304+0000 7fcaace00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 0x7fcaa81a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:11.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.304+0000 7fcaace00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 0x7fcaa81a76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40504/0 (socket says 172.21.15.46:40504) 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.304+0000 7fcaad800640 1 -- 172.21.15.46:0/3634346142 learned_addr learned my addr 172.21.15.46:0/3634346142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.304+0000 7fcaaec00640 1 -- 172.21.15.46:0/3634346142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8118360 con 0x7fcaa810e970 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.304+0000 7fcaaec00640 1 -- 172.21.15.46:0/3634346142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81181e0 con 0x7fcaa8109e00 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.304+0000 7fcaaec00640 1 -- 172.21.15.46:0/3634346142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81184e0 con 0x7fcaa8109450 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.304+0000 7fcaae200640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa810e970 0x7fcaa81aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.305+0000 7fcaace00640 1 -- 172.21.15.46:0/3634346142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa8109450 msgr2=0x7fcaa81a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.305+0000 7fcaace00640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa8109450 0x7fcaa81a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.305+0000 7fcaace00640 1 -- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa810e970 msgr2=0x7fcaa81aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.305+0000 7fcaace00640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa810e970 0x7fcaa81aba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.305+0000 7fcaace00640 1 -- 172.21.15.46:0/3634346142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa81ac120 con 0x7fcaa8109e00 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.305+0000 7fcaae200640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa810e970 0x7fcaa81aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.305+0000 7fcaad800640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa8109450 0x7fcaa81a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.305+0000 7fcaace00640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 0x7fcaa81a76b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fca9402a860 tx=0x7fca94033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.306+0000 7fcaa6000640 1 -- 172.21.15.46:0/3634346142 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94037040 con 0x7fcaa8109e00 2024-09-10T07:44:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.306+0000 7fcaaec00640 1 -- 172.21.15.46:0/3634346142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa81ac3b0 con 0x7fcaa8109e00 2024-09-10T07:44:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.306+0000 7fcaaec00640 1 -- 172.21.15.46:0/3634346142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81b3cf0 con 0x7fcaa8109e00 2024-09-10T07:44:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.306+0000 7fcaa6000640 1 -- 172.21.15.46:0/3634346142 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca94030550 con 0x7fcaa8109e00 2024-09-10T07:44:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.306+0000 7fcaa6000640 1 -- 172.21.15.46:0/3634346142 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94003930 con 0x7fcaa8109e00 2024-09-10T07:44:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.308+0000 7fcaaec00640 1 -- 172.21.15.46:0/3634346142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca88005290 con 0x7fcaa8109e00 2024-09-10T07:44:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.312+0000 7fcaa6000640 1 -- 172.21.15.46:0/3634346142 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca94042030 con 0x7fcaa8109e00 2024-09-10T07:44:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.316+0000 7fcaa6000640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fca900a2c80 0x7fca900a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.317+0000 7fcaa6000640 1 -- 172.21.15.46:0/3634346142 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fca940ee8f0 con 0x7fcaa8109e00 2024-09-10T07:44:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.317+0000 7fcaa6000640 1 -- 172.21.15.46:0/3634346142 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca9411d1b0 con 0x7fcaa8109e00 2024-09-10T07:44:11.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.322+0000 7fcaad800640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fca900a2c80 0x7fca900a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.324+0000 7fcaad800640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fca900a2c80 0x7fca900a5140 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0092f0 tx=0x7fca9c00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.747+0000 7fcaaec00640 1 -- 172.21.15.46:0/3634346142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca88005580 con 0x7fcaa8109e00 2024-09-10T07:44:11.753 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:44:11.753 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":44,"btime":"2024-09-10T07:43:26:111291+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:43:25.508547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:44:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.749+0000 7fcaa6000640 1 -- 172.21.15.46:0/3634346142 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11980 (secure 0 0 0) 0x7fca940bb500 con 0x7fcaa8109e00 2024-09-10T07:44:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 44 2024-09-10T07:44:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.754+0000 7fca9b400640 1 -- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fca900a2c80 msgr2=0x7fca900a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.754+0000 7fca9b400640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fca900a2c80 0x7fca900a5140 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0092f0 tx=0x7fca9c00c040 comp rx=0 tx=0).stop 2024-09-10T07:44:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.754+0000 7fca9b400640 1 -- 172.21.15.46:0/3634346142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 msgr2=0x7fcaa81a76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.754+0000 7fca9b400640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 0x7fcaa81a76b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fca9402a860 tx=0x7fca94033d90 comp rx=0 tx=0).stop 2024-09-10T07:44:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.755+0000 7fca9b400640 1 -- 172.21.15.46:0/3634346142 shutdown_connections 2024-09-10T07:44:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.755+0000 7fca9b400640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fca900a2c80 0x7fca900a5140 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.755+0000 7fca9b400640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa810e970 0x7fcaa81aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.755+0000 7fca9b400640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaa8109e00 0x7fcaa81a76b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.755+0000 7fca9b400640 1 --2- 172.21.15.46:0/3634346142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa8109450 0x7fcaa81a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.755+0000 7fca9b400640 1 -- 172.21.15.46:0/3634346142 >> 172.21.15.46:0/3634346142 conn(0x7fcaa8073180 msgr2=0x7fcaa810c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.756+0000 7fca9b400640 1 -- 172.21.15.46:0/3634346142 shutdown_connections 2024-09-10T07:44:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:11.756+0000 7fca9b400640 1 -- 172.21.15.46:0/3634346142 wait complete. 2024-09-10T07:44:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:11 smithi046 ceph-mon[30705]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-09-10T07:44:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:11 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3634346142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:44:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:11 smithi181 ceph-mon[35397]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-09-10T07:44:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:11 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3634346142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:44:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:11 smithi063 ceph-mon[35020]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-09-10T07:44:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:11 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3634346142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:44:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.709+0000 7f31ca800640 1 -- 172.21.15.46:0/2190612450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 msgr2=0x7f31c4075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.709+0000 7f31ca800640 1 --2- 172.21.15.46:0/2190612450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 0x7f31c4075050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009900 tx=0x7f31b002a4a0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.710+0000 7f31ca800640 1 -- 172.21.15.46:0/2190612450 shutdown_connections 2024-09-10T07:44:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.710+0000 7f31ca800640 1 --2- 172.21.15.46:0/2190612450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31c41099f0 0x7f31c410beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.710+0000 7f31ca800640 1 --2- 172.21.15.46:0/2190612450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 0x7f31c4075050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.710+0000 7f31ca800640 1 --2- 172.21.15.46:0/2190612450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31c40761d0 0x7f31c4074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.710+0000 7f31ca800640 1 -- 172.21.15.46:0/2190612450 >> 172.21.15.46:0/2190612450 conn(0x7f31c40fd2e0 msgr2=0x7f31c40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.711+0000 7f31ca800640 1 -- 172.21.15.46:0/2190612450 shutdown_connections 2024-09-10T07:44:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.711+0000 7f31ca800640 1 -- 172.21.15.46:0/2190612450 wait complete. 2024-09-10T07:44:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.712+0000 7f31ca800640 1 Processor -- start 2024-09-10T07:44:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.713+0000 7f31ca800640 1 -- start start 2024-09-10T07:44:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.714+0000 7f31ca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 0x7f31c4104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.714+0000 7f31ca800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31c40761d0 0x7f31c4105420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.714+0000 7f31c3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 0x7f31c4104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.714+0000 7f31c3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 0x7f31c4104ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48976/0 (socket says 172.21.15.46:48976) 2024-09-10T07:44:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.715+0000 7f31ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31c41099f0 0x7f31c4101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.715+0000 7f31ca800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31c410fac0 con 0x7f31c41099f0 2024-09-10T07:44:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.715+0000 7f31ca800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31c410f940 con 0x7f31c40761d0 2024-09-10T07:44:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.715+0000 7f31ca800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f31c410fc40 con 0x7f31c4074bf0 2024-09-10T07:44:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.715+0000 7f31c2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31c40761d0 0x7f31c4105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.715+0000 7f31c2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31c40761d0 0x7f31c4105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40514/0 (socket says 172.21.15.46:40514) 2024-09-10T07:44:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.715+0000 7f31c2a00640 1 -- 172.21.15.46:0/1712540601 learned_addr learned my addr 172.21.15.46:0/1712540601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.716+0000 7f31c3e00640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31c41099f0 0x7f31c4101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.716+0000 7f31c2a00640 1 -- 172.21.15.46:0/1712540601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 msgr2=0x7f31c4104ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.716+0000 7f31c2a00640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 0x7f31c4104ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.716+0000 7f31c2a00640 1 -- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31c41099f0 msgr2=0x7f31c4101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.716+0000 7f31c2a00640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31c41099f0 0x7f31c4101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.716+0000 7f31c2a00640 1 -- 172.21.15.46:0/1712540601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c4102310 con 0x7f31c40761d0 2024-09-10T07:44:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.718+0000 7f31c2a00640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31c40761d0 0x7f31c4105420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f31b0031440 tx=0x7f31b0032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.718+0000 7f31bbe00640 1 -- 172.21.15.46:0/1712540601 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b003f030 con 0x7f31c40761d0 2024-09-10T07:44:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.718+0000 7f31bbe00640 1 -- 172.21.15.46:0/1712540601 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b0003ce0 con 0x7f31c40761d0 2024-09-10T07:44:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.718+0000 7f31bbe00640 1 -- 172.21.15.46:0/1712540601 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0035070 con 0x7f31c40761d0 2024-09-10T07:44:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.718+0000 7f31ca800640 1 -- 172.21.15.46:0/1712540601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c41a6510 con 0x7f31c40761d0 2024-09-10T07:44:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.719+0000 7f31ca800640 1 -- 172.21.15.46:0/1712540601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c41a68c0 con 0x7f31c40761d0 2024-09-10T07:44:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.724+0000 7f31bbe00640 1 -- 172.21.15.46:0/1712540601 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f31b0042030 con 0x7f31c40761d0 2024-09-10T07:44:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.724+0000 7f31cd5e0740 1 Processor -- start 2024-09-10T07:44:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.725+0000 7f31cd5e0740 1 -- start start 2024-09-10T07:44:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.727+0000 7f31bbe00640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31a80a2d30 0x7f31a80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.727+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d069490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.727+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d0699d0 0x560f4d06e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.727+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d06ea60 0x560f4d070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.727+0000 7f31cd5e0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560f4cfce100 con 0x560f4d06ea60 2024-09-10T07:44:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.727+0000 7f31cd5e0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560f4cf761b0 con 0x560f4d069090 2024-09-10T07:44:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.727+0000 7f31cd5e0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560f4cfca0c0 con 0x560f4d0699d0 2024-09-10T07:44:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.727+0000 7f31bbe00640 1 -- 172.21.15.46:0/1712540601 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f31b00efa70 con 0x7f31c40761d0 2024-09-10T07:44:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.728+0000 7f31c3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.728+0000 7f31c3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d069490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40522/0 (socket says 172.21.15.46:40522) 2024-09-10T07:44:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.728+0000 7f31c3400640 1 -- 172.21.15.46:0/3838736786 learned_addr learned my addr 172.21.15.46:0/3838736786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.729+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d0699d0 0x560f4d06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.729+0000 7f31c3e00640 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d06ea60 0x560f4d070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.729+0000 7f31c3e00640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31a80a2d30 0x7f31a80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.729+0000 7f31c2a00640 1 -- 172.21.15.46:0/3838736786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 msgr2=0x560f4d069490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.729+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d069490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.729+0000 7f31c2a00640 1 -- 172.21.15.46:0/3838736786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d06ea60 msgr2=0x560f4d070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.729+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d06ea60 0x560f4d070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.729+0000 7f31c2a00640 1 -- 172.21.15.46:0/3838736786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4cce1210 con 0x560f4d0699d0 2024-09-10T07:44:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.730+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d0699d0 0x560f4d06e490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f31b0000c00 tx=0x7f31b002a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=187a97f34f31e1a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.730+0000 7f31b8c00640 1 -- 172.21.15.46:0/3838736786 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00c02a0 con 0x560f4d0699d0 2024-09-10T07:44:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.730+0000 7f31b8c00640 1 -- 172.21.15.46:0/3838736786 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b00c4b60 con 0x560f4d0699d0 2024-09-10T07:44:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.730+0000 7f31b8c00640 1 -- 172.21.15.46:0/3838736786 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00c4910 con 0x560f4d0699d0 2024-09-10T07:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.731+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3838736786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d0699d0 msgr2=0x560f4d06e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.731+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d0699d0 0x560f4d06e490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f31b0000c00 tx=0x7f31b002a4a0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.731+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3838736786 shutdown_connections 2024-09-10T07:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.731+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d06ea60 0x560f4d070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.731+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d0699d0 0x560f4d06e490 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.731+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3838736786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d069490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.731+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3838736786 >> 172.21.15.46:0/3838736786 conn(0x560f4d064700 msgr2=0x560f4d064b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.732+0000 7f31c3e00640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31a80a2d30 0x7f31a80a51f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f31bc005fd0 tx=0x7f31bc009440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.732+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3838736786 shutdown_connections 2024-09-10T07:44:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.734+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3838736786 wait complete. 2024-09-10T07:44:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.735+0000 7f31cd5e0740 1 -- start start 2024-09-10T07:44:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.739+0000 7f31cd5e0740 1 Processor -- start 2024-09-10T07:44:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.740+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.740+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d0699d0 0x560f4d073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.740+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d07a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.741+0000 7f31c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d07a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.741+0000 7f31c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d07a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48998/0 (socket says 172.21.15.46:48998) 2024-09-10T07:44:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.741+0000 7f31c3e00640 1 -- 172.21.15.46:0/4118382218 learned_addr learned my addr 172.21.15.46:0/4118382218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.741+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560f4d0717f0 con 0x560f4d0699d0 2024-09-10T07:44:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.741+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560f4cf57090 con 0x560f4d069090 2024-09-10T07:44:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.741+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560f4d071970 con 0x560f4d06ea60 2024-09-10T07:44:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.741+0000 7f31cd5e0740 10 client.0 _ll_register_callbacks cb 0x560f4cfabf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.741+0000 7f31c3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.742+0000 7f31c3400640 1 -- 172.21.15.46:0/4118382218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 msgr2=0x560f4d07a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.742+0000 7f31c3400640 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d07a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.742+0000 7f31c3400640 1 -- 172.21.15.46:0/4118382218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d0699d0 msgr2=0x560f4d073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.742+0000 7f31c3400640 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d0699d0 0x560f4d073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.742+0000 7f31c3400640 1 -- 172.21.15.46:0/4118382218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4cdad620 con 0x560f4d069090 2024-09-10T07:44:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.743+0000 7f31c3400640 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d072f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006fd0 tx=0x7f31b40119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=875f5ca4edfb8d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.743+0000 7f31ad600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.747+0000 7f31ad600640 1 -- 172.21.15.46:0/4118382218 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4013670 con 0x560f4d069090 2024-09-10T07:44:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.747+0000 7f31ad600640 1 -- 172.21.15.46:0/4118382218 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f31b4013810 con 0x560f4d069090 2024-09-10T07:44:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.747+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31b00c4b60 con 0x560f4d069090 2024-09-10T07:44:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.747+0000 7f31ad600640 1 -- 172.21.15.46:0/4118382218 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f31b4026050 con 0x560f4d069090 2024-09-10T07:44:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.747+0000 7f31cd5e0740 10 client.27275 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.747+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560f4d07d370 con 0x560f4d069090 2024-09-10T07:44:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.751+0000 7f31ad600640 1 -- 172.21.15.46:0/4118382218 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b00c4b60 con 0x560f4d069090 2024-09-10T07:44:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.751+0000 7f31ad600640 1 -- 172.21.15.46:0/4118382218 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f31b401d400 con 0x560f4d069090 2024-09-10T07:44:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.751+0000 7f31cd5e0740 10 client.27275 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.751+0000 7f31cd5e0740 10 client.27275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.751+0000 7f31cd5e0740 20 client.27275 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.751+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560f4d07ebf0 0x560f4d081070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.752+0000 7f31cd5e0740 4 client.27275 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.752+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f4cfdb830 con 0x560f4d07ebf0 2024-09-10T07:44:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.752+0000 7f31c2a00640 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560f4d07ebf0 0x560f4d081070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.755+0000 7f31c2a00640 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560f4d07ebf0 0x560f4d081070 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fedd019cc3dd4b2 server_cookie=74ddf4055c091a36 in_seq=0 out_seq=0 2024-09-10T07:44:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.755+0000 7f31ad600640 10 client.27275 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31ad600640 1 -- 172.21.15.46:0/4118382218 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f4cfdb830 con 0x560f4d07ebf0 2024-09-10T07:44:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31ad600640 10 client.27275 handle_command_reply: tid=0 2024-09-10T07:44:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 client.27275 shutdown 2024-09-10T07:44:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 msgr2=0x560f4d072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d072f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006fd0 tx=0x7f31b40119d0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 shutdown_connections 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560f4d07ebf0 0x560f4d081070 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d07a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d0699d0 0x560f4d073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/4118382218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d069090 0x560f4d072f30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 >> 172.21.15.46:0/4118382218 conn(0x560f4d064700 msgr2=0x560f4d065f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.758+0000 7f31ad600640 0 client.27275 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.759+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 shutdown_connections 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.759+0000 7f31cd5e0740 1 -- 172.21.15.46:0/4118382218 wait complete. 2024-09-10T07:44:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.759+0000 7f31cd5e0740 20 client.27275 trim_cache size 0 max 16384 2024-09-10T07:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.800+0000 7f31cd5e0740 1 Processor -- start 2024-09-10T07:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.801+0000 7f31cd5e0740 1 -- start start 2024-09-10T07:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 0x560f4d073500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d07a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d063770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31cd5e0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b0d20 con 0x560f4d069090 2024-09-10T07:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31cd5e0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b2950 con 0x560f4d0699d0 2024-09-10T07:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31cd5e0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b13f0 con 0x560f4d06ea60 2024-09-10T07:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31c2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d07a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31c2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d07a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:40544/0 (socket says 172.21.15.46:40544) 2024-09-10T07:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.802+0000 7f31c2a00640 1 -- 172.21.15.46:0/3622636786 learned_addr learned my addr 172.21.15.46:0/3622636786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.803+0000 7f31c3e00640 1 --2- 172.21.15.46:0/3622636786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d063770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.803+0000 7f31c2a00640 1 -- 172.21.15.46:0/3622636786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 msgr2=0x560f4d063770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.803+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3622636786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d063770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.803+0000 7f31c2a00640 1 -- 172.21.15.46:0/3622636786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 msgr2=0x560f4d073500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.803+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3622636786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 0x560f4d073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.803+0000 7f31c2a00640 1 -- 172.21.15.46:0/3622636786 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4d06a660 con 0x560f4d0699d0 2024-09-10T07:44:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.803+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3622636786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d07a470 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f31b0005c20 tx=0x7f31b00bbae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3870b707fff43a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.804+0000 7f31afe00640 1 -- 172.21.15.46:0/3622636786 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b009f070 con 0x560f4d0699d0 2024-09-10T07:44:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.804+0000 7f31afe00640 1 -- 172.21.15.46:0/3622636786 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b003be10 con 0x560f4d0699d0 2024-09-10T07:44:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.804+0000 7f31afe00640 1 -- 172.21.15.46:0/3622636786 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00c46d0 con 0x560f4d0699d0 2024-09-10T07:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.809+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3622636786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 msgr2=0x560f4d07a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.809+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3622636786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d07a470 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f31b0005c20 tx=0x7f31b00bbae0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.810+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3622636786 shutdown_connections 2024-09-10T07:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.810+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3622636786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d063770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.810+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3622636786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d07a470 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.810+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3622636786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 0x560f4d073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.810+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3622636786 >> 172.21.15.46:0/3622636786 conn(0x560f4d064700 msgr2=0x560f4d063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.810+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3622636786 shutdown_connections 2024-09-10T07:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.810+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3622636786 wait complete. 2024-09-10T07:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.811+0000 7f31cd5e0740 1 -- start start 2024-09-10T07:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.811+0000 7f31cd5e0740 1 Processor -- start 2024-09-10T07:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.812+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 0x560f4d077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.813+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d078000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.813+0000 7f31cd5e0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d078fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.813+0000 7f31c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d078fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.814+0000 7f31c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d078fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49012/0 (socket says 172.21.15.46:49012) 2024-09-10T07:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.814+0000 7f31c3e00640 1 -- 172.21.15.46:0/3895204134 learned_addr learned my addr 172.21.15.46:0/3895204134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.814+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b21c0 con 0x560f4d069090 2024-09-10T07:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.814+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d078000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.814+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b1ac0 con 0x560f4d0699d0 2024-09-10T07:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.814+0000 7f31c3400640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 0x560f4d077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.814+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41aed00 con 0x560f4d06ea60 2024-09-10T07:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.815+0000 7f31cd5e0740 10 client.0 _ll_register_callbacks cb 0x7f31b40095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.816+0000 7f31c3400640 1 -- 172.21.15.46:0/3895204134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 msgr2=0x560f4d078fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.816+0000 7f31c3400640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d078fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.816+0000 7f31c3400640 1 -- 172.21.15.46:0/3895204134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 msgr2=0x560f4d078000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.816+0000 7f31c3400640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d078000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.816+0000 7f31c3400640 1 -- 172.21.15.46:0/3895204134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4d0795d0 con 0x560f4d069090 2024-09-10T07:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.816+0000 7f31c3e00640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d078fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.816+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d078000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.817+0000 7f31c3400640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 0x560f4d077ac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f31b400b010 tx=0x7f31b401d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefefcbc110d09a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.817+0000 7f31ad600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:44:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.818+0000 7f31ad600640 1 -- 172.21.15.46:0/3895204134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b401dc50 con 0x560f4d069090 2024-09-10T07:44:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.818+0000 7f31ad600640 1 -- 172.21.15.46:0/3895204134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f31b4004040 con 0x560f4d069090 2024-09-10T07:44:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.818+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31b003be10 con 0x560f4d069090 2024-09-10T07:44:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.819+0000 7f31ad600640 1 -- 172.21.15.46:0/3895204134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f31b4009150 con 0x560f4d069090 2024-09-10T07:44:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.819+0000 7f31cd5e0740 10 client.16809 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.819+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560f4d0e9910 con 0x560f4d069090 2024-09-10T07:44:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.821+0000 7f31ad600640 1 -- 172.21.15.46:0/3895204134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b003be10 con 0x560f4d069090 2024-09-10T07:44:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.821+0000 7f31ad600640 1 -- 172.21.15.46:0/3895204134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f31b4013670 con 0x560f4d069090 2024-09-10T07:44:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.821+0000 7f31cd5e0740 10 client.16809 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.821+0000 7f31cd5e0740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.821+0000 7f31cd5e0740 20 client.16809 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.821+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560f4d0ea220 0x560f4d0ec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.822+0000 7f31cd5e0740 4 client.16809 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:44:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.822+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f4cfd5b80 con 0x560f4d0ea220 2024-09-10T07:44:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.823+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560f4d0ea220 0x560f4d0ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.828+0000 7f31ad600640 10 client.16809 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.828+0000 7f31c2a00640 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560f4d0ea220 0x560f4d0ec640 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac9554c84dca0fe server_cookie=ba1f64ec5dbafdf8 in_seq=0 out_seq=0 2024-09-10T07:44:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31ad600640 1 -- 172.21.15.46:0/3895204134 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f4cfd5b80 con 0x560f4d0ea220 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31ad600640 10 client.16809 handle_command_reply: tid=0 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 client.16809 shutdown 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 msgr2=0x560f4d077ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 0x560f4d077ac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f31b400b010 tx=0x7f31b401d610 comp rx=0 tx=0).stop 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 shutdown_connections 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560f4d0ea220 0x560f4d0ec640 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f4d06ea60 0x560f4d078fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f4d0699d0 0x560f4d078000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 --2- 172.21.15.46:0/3895204134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f4d069090 0x560f4d077ac0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 >> 172.21.15.46:0/3895204134 conn(0x560f4d064700 msgr2=0x560f4d063880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.928+0000 7f31ad600640 0 client.16809 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.929+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 shutdown_connections 2024-09-10T07:44:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.929+0000 7f31cd5e0740 1 -- 172.21.15.46:0/3895204134 wait complete. 2024-09-10T07:44:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.929+0000 7f31cd5e0740 20 client.16809 trim_cache size 0 max 16384 2024-09-10T07:44:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:44:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:44:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:44:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:44:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 -- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31a80a2d30 msgr2=0x7f31a80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31a80a2d30 0x7f31a80a51f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f31bc005fd0 tx=0x7f31bc009440 comp rx=0 tx=0).stop 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 -- 172.21.15.46:0/1712540601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31c40761d0 msgr2=0x7f31c4105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31c40761d0 0x7f31c4105420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f31b0031440 tx=0x7f31b0032e20 comp rx=0 tx=0).stop 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 -- 172.21.15.46:0/1712540601 shutdown_connections 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f31a80a2d30 0x7f31a80a51f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31c41099f0 0x7f31c4101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31c40761d0 0x7f31c4105420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 --2- 172.21.15.46:0/1712540601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c4074bf0 0x7f31c4104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 -- 172.21.15.46:0/1712540601 >> 172.21.15.46:0/1712540601 conn(0x7f31c40fd2e0 msgr2=0x7f31c4105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.931+0000 7f31ca800640 1 -- 172.21.15.46:0/1712540601 shutdown_connections 2024-09-10T07:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:13.932+0000 7f31ca800640 1 -- 172.21.15.46:0/1712540601 wait complete. 2024-09-10T07:44:14.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:13 smithi046 ceph-mon[30705]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-10T07:44:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:13 smithi181 ceph-mon[35397]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-10T07:44:14.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:13 smithi063 ceph-mon[35020]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-10T07:44:14.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:44:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 415.0 s, 669.8 tps, lat 151.849 ms stddev 217.346 2024-09-10T07:44:16.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 -- 172.21.15.46:0/1768799673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf60101150 msgr2=0x7fdf6010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 --2- 172.21.15.46:0/1768799673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf60101150 0x7fdf6010de10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009900 tx=0x7fdf5002a440 comp rx=0 tx=0).stop 2024-09-10T07:44:16.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:15 smithi046 ceph-mon[30705]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 1.43k op/s 2024-09-10T07:44:16.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 -- 172.21.15.46:0/1768799673 shutdown_connections 2024-09-10T07:44:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 --2- 172.21.15.46:0/1768799673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6010e350 0x7fdf60110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 --2- 172.21.15.46:0/1768799673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf60101150 0x7fdf6010de10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 --2- 172.21.15.46:0/1768799673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf601007a0 0x7fdf60100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 -- 172.21.15.46:0/1768799673 >> 172.21.15.46:0/1768799673 conn(0x7fdf600fc610 msgr2=0x7fdf600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 -- 172.21.15.46:0/1768799673 shutdown_connections 2024-09-10T07:44:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.277+0000 7fdf67200640 1 -- 172.21.15.46:0/1768799673 wait complete. 2024-09-10T07:44:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.279+0000 7fdf67200640 1 Processor -- start 2024-09-10T07:44:16.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.279+0000 7fdf67200640 1 -- start start 2024-09-10T07:44:16.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.279+0000 7fdf67200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf601007a0 0x7fdf6010db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.280+0000 7fdf67200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf60101150 0x7fdf60104ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.280+0000 7fdf65e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf601007a0 0x7fdf6010db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.280+0000 7fdf65e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf601007a0 0x7fdf6010db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53954/0 (socket says 172.21.15.46:53954) 2024-09-10T07:44:16.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.280+0000 7fdf65e00640 1 -- 172.21.15.46:0/3696799848 learned_addr learned my addr 172.21.15.46:0/3696799848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.283+0000 7fdf67200640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6010e350 0x7fdf60105430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.283+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf60104730 con 0x7fdf6010e350 2024-09-10T07:44:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.283+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601045b0 con 0x7fdf601007a0 2024-09-10T07:44:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.283+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601048b0 con 0x7fdf60101150 2024-09-10T07:44:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.283+0000 7fdf65400640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf60101150 0x7fdf60104ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.284+0000 7fdf66800640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6010e350 0x7fdf60105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.285+0000 7fdf65e00640 1 -- 172.21.15.46:0/3696799848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf60101150 msgr2=0x7fdf60104ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.285+0000 7fdf65e00640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf60101150 0x7fdf60104ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.285+0000 7fdf65e00640 1 -- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6010e350 msgr2=0x7fdf60105430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.285+0000 7fdf65e00640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6010e350 0x7fdf60105430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.285+0000 7fdf65e00640 1 -- 172.21.15.46:0/3696799848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf60105c00 con 0x7fdf601007a0 2024-09-10T07:44:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.286+0000 7fdf65e00640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf601007a0 0x7fdf6010db20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400e9f0 tx=0x7fdf5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.286+0000 7fdf5aa00640 1 -- 172.21.15.46:0/3696799848 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400cd80 con 0x7fdf601007a0 2024-09-10T07:44:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.286+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf60109570 con 0x7fdf601007a0 2024-09-10T07:44:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.287+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf60109ad0 con 0x7fdf601007a0 2024-09-10T07:44:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.288+0000 7fdf5aa00640 1 -- 172.21.15.46:0/3696799848 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54010910 con 0x7fdf601007a0 2024-09-10T07:44:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.288+0000 7fdf5aa00640 1 -- 172.21.15.46:0/3696799848 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540188f0 con 0x7fdf601007a0 2024-09-10T07:44:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.297+0000 7fdf6b3f5740 1 Processor -- start 2024-09-10T07:44:16.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.297+0000 7fdf6b3f5740 1 -- start start 2024-09-10T07:44:16.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.298+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.298+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cc760 0x55e1e74d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.298+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e74d9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.298+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e1e74323e0 con 0x55e1e74cbe40 2024-09-10T07:44:16.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.298+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e1e73d9d80 con 0x55e1e74d7020 2024-09-10T07:44:16.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.298+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e1e742e3a0 con 0x55e1e74cc760 2024-09-10T07:44:16.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.299+0000 7fdf66800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.299+0000 7fdf65400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e74d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.300+0000 7fdf65400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e74d9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53964/0 (socket says 172.21.15.46:53964) 2024-09-10T07:44:16.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.300+0000 7fdf65400640 1 -- 172.21.15.46:0/1177332412 learned_addr learned my addr 172.21.15.46:0/1177332412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:16.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.300+0000 7fdf5aa00640 1 -- 172.21.15.46:0/3696799848 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf54018bd0 con 0x7fdf601007a0 2024-09-10T07:44:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.300+0000 7fdf65e00640 1 --2- 172.21.15.46:0/1177332412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cc760 0x55e1e74d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.301+0000 7fdf65e00640 1 -- 172.21.15.46:0/1177332412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 msgr2=0x55e1e74d9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.301+0000 7fdf65e00640 1 --2- 172.21.15.46:0/1177332412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e74d9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.301+0000 7fdf65e00640 1 -- 172.21.15.46:0/1177332412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 msgr2=0x55e1e74cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.301+0000 7fdf65e00640 1 --2- 172.21.15.46:0/1177332412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.301+0000 7fdf65e00640 1 -- 172.21.15.46:0/1177332412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e73f8ae0 con 0x55e1e74cc760 2024-09-10T07:44:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.301+0000 7fdf65e00640 1 --2- 172.21.15.46:0/1177332412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cc760 0x55e1e74d6ae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdf540ccd50 tx=0x7fdf540d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e091940ee8dbf02d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.302+0000 7fdf4f000640 1 -- 172.21.15.46:0/1177332412 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540de070 con 0x55e1e74cc760 2024-09-10T07:44:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.302+0000 7fdf5aa00640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf480a2c00 0x7fdf480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.302+0000 7fdf4f000640 1 -- 172.21.15.46:0/1177332412 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5409c210 con 0x55e1e74cc760 2024-09-10T07:44:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.302+0000 7fdf4f000640 1 -- 172.21.15.46:0/1177332412 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5409bb10 con 0x55e1e74cc760 2024-09-10T07:44:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.302+0000 7fdf5aa00640 1 -- 172.21.15.46:0/3696799848 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdf540cf690 con 0x7fdf601007a0 2024-09-10T07:44:16.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.303+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1177332412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cc760 msgr2=0x55e1e74d6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.303+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1177332412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cc760 0x55e1e74d6ae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdf540ccd50 tx=0x7fdf540d7670 comp rx=0 tx=0).stop 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.304+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1177332412 shutdown_connections 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.304+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1177332412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e74d9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.304+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1177332412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cc760 0x55e1e74d6ae0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.304+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1177332412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.304+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1177332412 >> 172.21.15.46:0/1177332412 conn(0x55e1e74c83d0 msgr2=0x55e1e74c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.304+0000 7fdf66800640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf480a2c00 0x7fdf480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.304+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1177332412 shutdown_connections 2024-09-10T07:44:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.305+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1177332412 wait complete. 2024-09-10T07:44:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.307+0000 7fdf6b3f5740 1 -- start start 2024-09-10T07:44:16.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.308+0000 7fdf6b3f5740 1 Processor -- start 2024-09-10T07:44:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.308+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74cc760 0x55e1e74d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74d7020 0x55e1e74d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e1e74eac00 con 0x55e1e74cbe40 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e1e73ed0f0 con 0x55e1e74cc760 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e1e74ead80 con 0x55e1e74d7020 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf6b3f5740 10 client.0 _ll_register_callbacks cb 0x55e1e73d4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf65400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74cc760 0x55e1e74d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf65400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74cc760 0x55e1e74d4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53968/0 (socket says 172.21.15.46:53968) 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.309+0000 7fdf65400640 1 -- 172.21.15.46:0/1598394160 learned_addr learned my addr 172.21.15.46:0/1598394160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.310+0000 7fdf65400640 1 -- 172.21.15.46:0/1598394160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74d7020 msgr2=0x55e1e74d4880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:44:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.310+0000 7fdf65400640 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74d7020 0x55e1e74d4880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.310+0000 7fdf65400640 1 -- 172.21.15.46:0/1598394160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 msgr2=0x55e1e74d3e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:44:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.310+0000 7fdf65400640 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74d3e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.310+0000 7fdf65400640 1 -- 172.21.15.46:0/1598394160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e71a33c0 con 0x55e1e74cc760 2024-09-10T07:44:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.310+0000 7fdf65400640 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74cc760 0x55e1e74d4340 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009230 tx=0x7fdf500048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=572bfd43b55261ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.310+0000 7fdf43400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.312+0000 7fdf43400640 1 -- 172.21.15.46:0/1598394160 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50005410 con 0x55e1e74cc760 2024-09-10T07:44:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.312+0000 7fdf43400640 1 -- 172.21.15.46:0/1598394160 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdf500055b0 con 0x55e1e74cc760 2024-09-10T07:44:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.312+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1598394160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5409c210 con 0x55e1e74cc760 2024-09-10T07:44:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.313+0000 7fdf66800640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf480a2c00 0x7fdf480a50c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf60069750 tx=0x7fdf5c00e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.316+0000 7fdf43400640 1 -- 172.21.15.46:0/1598394160 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdf50047440 con 0x55e1e74cc760 2024-09-10T07:44:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.316+0000 7fdf6b3f5740 10 client.27293 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.316+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1598394160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e1e74f6c00 con 0x55e1e74cc760 2024-09-10T07:44:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.319+0000 7fdf43400640 1 -- 172.21.15.46:0/1598394160 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5409c210 con 0x55e1e74cc760 2024-09-10T07:44:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.319+0000 7fdf43400640 1 -- 172.21.15.46:0/1598394160 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7fdf50045070 con 0x55e1e74cc760 2024-09-10T07:44:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.319+0000 7fdf6b3f5740 10 client.27293 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.319+0000 7fdf6b3f5740 10 client.27293 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:44:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.319+0000 7fdf6b3f5740 20 client.27293 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.319+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e1e74f85a0 0x55e1e74fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.319+0000 7fdf6b3f5740 4 client.27293 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.319+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1598394160 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1e743f950 con 0x55e1e74f85a0 2024-09-10T07:44:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.320+0000 7fdf65e00640 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e1e74f85a0 0x55e1e74fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.322+0000 7fdf65e00640 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e1e74f85a0 0x55e1e74fa9e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2edf6364dc701d93 server_cookie=631d20c80e337fb9 in_seq=0 out_seq=0 2024-09-10T07:44:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.322+0000 7fdf43400640 10 client.27293 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:44:16.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.326+0000 7fdf43400640 1 -- 172.21.15.46:0/1598394160 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1e743f950 con 0x55e1e74f85a0 2024-09-10T07:44:16.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.326+0000 7fdf43400640 10 client.27293 handle_command_reply: tid=0 2024-09-10T07:44:16.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.327+0000 7fdf6b3f5740 1 client.27293 shutdown 2024-09-10T07:44:16.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1598394160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74cc760 msgr2=0x55e1e74d4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74cc760 0x55e1e74d4340 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009230 tx=0x7fdf500048f0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1598394160 shutdown_connections 2024-09-10T07:44:16.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e1e74f85a0 0x55e1e74fa9e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf43400640 0 client.27293 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:44:16.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74d7020 0x55e1e74d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74cc760 0x55e1e74d4340 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/1598394160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.328+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1598394160 >> 172.21.15.46:0/1598394160 conn(0x55e1e74c83d0 msgr2=0x55e1e74ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:16.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.329+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1598394160 shutdown_connections 2024-09-10T07:44:16.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.329+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/1598394160 wait complete. 2024-09-10T07:44:16.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.329+0000 7fdf6b3f5740 20 client.27293 trim_cache size 0 max 16384 2024-09-10T07:44:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.358+0000 7fdf6b3f5740 1 Processor -- start 2024-09-10T07:44:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.358+0000 7fdf6b3f5740 1 -- start start 2024-09-10T07:44:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e75002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e7500830 0x55e1e74f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4400fb60 con 0x55e1e74cbe40 2024-09-10T07:44:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44011790 con 0x55e1e74d7020 2024-09-10T07:44:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44010230 con 0x55e1e7500830 2024-09-10T07:44:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf65400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e75002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf65400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e75002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53972/0 (socket says 172.21.15.46:53972) 2024-09-10T07:44:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.359+0000 7fdf65400640 1 -- 172.21.15.46:0/3758630785 learned_addr learned my addr 172.21.15.46:0/3758630785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.360+0000 7fdf66800640 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e7500830 0x55e1e74f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.360+0000 7fdf65e00640 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.360+0000 7fdf66800640 1 -- 172.21.15.46:0/3758630785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 msgr2=0x55e1e75002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.360+0000 7fdf66800640 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e75002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.360+0000 7fdf66800640 1 -- 172.21.15.46:0/3758630785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 msgr2=0x55e1e74d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.360+0000 7fdf66800640 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.360+0000 7fdf66800640 1 -- 172.21.15.46:0/3758630785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e74c8200 con 0x55e1e7500830 2024-09-10T07:44:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.361+0000 7fdf66800640 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e7500830 0x55e1e74f0050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf60102980 tx=0x7fdf5c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f29da1e64b6d8b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.362+0000 7fdf4e600640 1 -- 172.21.15.46:0/3758630785 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c017b10 con 0x55e1e7500830 2024-09-10T07:44:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.362+0000 7fdf4e600640 1 -- 172.21.15.46:0/3758630785 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c026070 con 0x55e1e7500830 2024-09-10T07:44:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.362+0000 7fdf4e600640 1 -- 172.21.15.46:0/3758630785 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c016390 con 0x55e1e7500830 2024-09-10T07:44:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.363+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/3758630785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e7500830 msgr2=0x55e1e74f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.363+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e7500830 0x55e1e74f0050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf60102980 tx=0x7fdf5c012000 comp rx=0 tx=0).stop 2024-09-10T07:44:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.363+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/3758630785 shutdown_connections 2024-09-10T07:44:16.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.364+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e7500830 0x55e1e74f0050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.364+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e74d7020 0x55e1e75002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.364+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/3758630785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74cbe40 0x55e1e74d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.364+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/3758630785 >> 172.21.15.46:0/3758630785 conn(0x55e1e74c83d0 msgr2=0x55e1e74d5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:16.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.364+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/3758630785 shutdown_connections 2024-09-10T07:44:16.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.364+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/3758630785 wait complete. 2024-09-10T07:44:16.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.365+0000 7fdf6b3f5740 1 -- start start 2024-09-10T07:44:16.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.365+0000 7fdf6b3f5740 1 Processor -- start 2024-09-10T07:44:16.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.367+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cbe40 0x55e1e74edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.368+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74d7020 0x55e1e74ee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.368+0000 7fdf65e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cbe40 0x55e1e74edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.368+0000 7fdf65e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cbe40 0x55e1e74edfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37504/0 (socket says 172.21.15.46:37504) 2024-09-10T07:44:16.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.368+0000 7fdf6b3f5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e7500830 0x55e1e74eea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.369+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44011000 con 0x55e1e74d7020 2024-09-10T07:44:16.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.369+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44010900 con 0x55e1e7500830 2024-09-10T07:44:16.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.369+0000 7fdf6b3f5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4400db00 con 0x55e1e74cbe40 2024-09-10T07:44:16.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.369+0000 7fdf6b3f5740 10 client.0 _ll_register_callbacks cb 0x7fdf50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:16.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.369+0000 7fdf66800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e7500830 0x55e1e74eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.369+0000 7fdf66800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e7500830 0x55e1e74eea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53980/0 (socket says 172.21.15.46:53980) 2024-09-10T07:44:16.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.369+0000 7fdf66800640 1 -- 172.21.15.46:0/2721405290 learned_addr learned my addr 172.21.15.46:0/2721405290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:16.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.370+0000 7fdf66800640 1 -- 172.21.15.46:0/2721405290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cbe40 msgr2=0x55e1e74edfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:16.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.370+0000 7fdf66800640 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cbe40 0x55e1e74edfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.370+0000 7fdf66800640 1 -- 172.21.15.46:0/2721405290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74d7020 msgr2=0x55e1e74ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:16.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.370+0000 7fdf66800640 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74d7020 0x55e1e74ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.370+0000 7fdf66800640 1 -- 172.21.15.46:0/2721405290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e74cecb0 con 0x55e1e7500830 2024-09-10T07:44:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.371+0000 7fdf43400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:16.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.371+0000 7fdf66800640 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e7500830 0x55e1e74eea40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c021ba0 tx=0x7fdf5c017b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b0f1dc64eb1fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:16.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.372+0000 7fdf43400640 1 -- 172.21.15.46:0/2721405290 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c034610 con 0x55e1e7500830 2024-09-10T07:44:16.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.373+0000 7fdf43400640 1 -- 172.21.15.46:0/2721405290 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdf5c0347b0 con 0x55e1e7500830 2024-09-10T07:44:16.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.373+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/2721405290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5c026070 con 0x55e1e7500830 2024-09-10T07:44:16.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.374+0000 7fdf43400640 1 -- 172.21.15.46:0/2721405290 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdf5c03d050 con 0x55e1e7500830 2024-09-10T07:44:16.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.374+0000 7fdf6b3f5740 10 client.27299 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:16.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.374+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/2721405290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e1e75635c0 con 0x55e1e7500830 2024-09-10T07:44:16.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.375+0000 7fdf43400640 1 -- 172.21.15.46:0/2721405290 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c026070 con 0x55e1e7500830 2024-09-10T07:44:16.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.375+0000 7fdf43400640 1 -- 172.21.15.46:0/2721405290 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7fdf5c02a510 con 0x55e1e7500830 2024-09-10T07:44:16.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.375+0000 7fdf6b3f5740 10 client.27299 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:16.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.375+0000 7fdf6b3f5740 10 client.27299 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:44:16.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.375+0000 7fdf6b3f5740 20 client.27299 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:16.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.375+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e1e7563c30 0x55e1e7566050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:16.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.375+0000 7fdf6b3f5740 4 client.27299 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:44:16.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.375+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/2721405290 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1e7439e60 con 0x55e1e7563c30 2024-09-10T07:44:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.376+0000 7fdf65e00640 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e1e7563c30 0x55e1e7566050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.378+0000 7fdf65e00640 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e1e7563c30 0x55e1e7566050 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a332b49e1f31692 server_cookie=15e6bbab66bd677 in_seq=0 out_seq=0 2024-09-10T07:44:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.378+0000 7fdf43400640 10 client.27299 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:44:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.378+0000 7fdf43400640 1 -- 172.21.15.46:0/2721405290 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1e7439e60 con 0x55e1e7563c30 2024-09-10T07:44:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.379+0000 7fdf43400640 10 client.27299 handle_command_reply: tid=0 2024-09-10T07:44:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.379+0000 7fdf6b3f5740 1 client.27299 shutdown 2024-09-10T07:44:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.383+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/2721405290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e7500830 msgr2=0x55e1e74eea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.383+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e7500830 0x55e1e74eea40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c021ba0 tx=0x7fdf5c017b40 comp rx=0 tx=0).stop 2024-09-10T07:44:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.383+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/2721405290 shutdown_connections 2024-09-10T07:44:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.383+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55e1e7563c30 0x55e1e7566050 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.383+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e1e7500830 0x55e1e74eea40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.383+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1e74d7020 0x55e1e74ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.383+0000 7fdf6b3f5740 1 --2- 172.21.15.46:0/2721405290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e1e74cbe40 0x55e1e74edfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.383+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/2721405290 >> 172.21.15.46:0/2721405290 conn(0x55e1e74c83d0 msgr2=0x55e1e74cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.384+0000 7fdf43400640 0 client.27299 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:44:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.384+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/2721405290 shutdown_connections 2024-09-10T07:44:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.385+0000 7fdf6b3f5740 1 -- 172.21.15.46:0/2721405290 wait complete. 2024-09-10T07:44:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.385+0000 7fdf6b3f5740 20 client.27299 trim_cache size 0 max 16384 2024-09-10T07:44:16.388 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:44:16.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.388+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf480a2c00 msgr2=0x7fdf480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.388+0000 7fdf67200640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf480a2c00 0x7fdf480a50c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf60069750 tx=0x7fdf5c00e040 comp rx=0 tx=0).stop 2024-09-10T07:44:16.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.388+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf601007a0 msgr2=0x7fdf6010db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:16.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.388+0000 7fdf67200640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf601007a0 0x7fdf6010db20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400e9f0 tx=0x7fdf5400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.391+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 shutdown_connections 2024-09-10T07:44:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.391+0000 7fdf67200640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdf480a2c00 0x7fdf480a50c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.391+0000 7fdf67200640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6010e350 0x7fdf60105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.391+0000 7fdf67200640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf60101150 0x7fdf60104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.391+0000 7fdf67200640 1 --2- 172.21.15.46:0/3696799848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf601007a0 0x7fdf6010db20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.391+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 >> 172.21.15.46:0/3696799848 conn(0x7fdf600fc610 msgr2=0x7fdf600fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.391+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 shutdown_connections 2024-09-10T07:44:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:16.391+0000 7fdf67200640 1 -- 172.21.15.46:0/3696799848 wait complete. 2024-09-10T07:44:16.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:15 smithi181 ceph-mon[35397]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 1.43k op/s 2024-09-10T07:44:16.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:15 smithi063 ceph-mon[35020]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 1.43k op/s 2024-09-10T07:44:17.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:44:18.164 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:17 smithi046 ceph-mon[30705]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 24 MiB/s wr, 1.59k op/s 2024-09-10T07:44:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:17 smithi181 ceph-mon[35397]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 24 MiB/s wr, 1.59k op/s 2024-09-10T07:44:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:17 smithi063 ceph-mon[35020]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 24 MiB/s wr, 1.59k op/s 2024-09-10T07:44:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.359+0000 7f1f0fe00640 1 -- 172.21.15.46:0/145581264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 msgr2=0x7f1f08076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.359+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/145581264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 0x7f1f08076b00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009860 tx=0x7f1efc02a340 comp rx=0 tx=0).stop 2024-09-10T07:44:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.361+0000 7f1f0fe00640 1 -- 172.21.15.46:0/145581264 shutdown_connections 2024-09-10T07:44:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.361+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/145581264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f08112a30 0x7f1f08114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.361+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/145581264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08077040 0x7f1f081124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.361+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/145581264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 0x7f1f08076b00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.361+0000 7f1f0fe00640 1 -- 172.21.15.46:0/145581264 >> 172.21.15.46:0/145581264 conn(0x7f1f080ffa90 msgr2=0x7f1f08101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.361+0000 7f1f0fe00640 1 -- 172.21.15.46:0/145581264 shutdown_connections 2024-09-10T07:44:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.363+0000 7f1f0fe00640 1 -- 172.21.15.46:0/145581264 wait complete. 2024-09-10T07:44:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.365+0000 7f1f0fe00640 1 Processor -- start 2024-09-10T07:44:19.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.368+0000 7f1f0fe00640 1 -- start start 2024-09-10T07:44:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.368+0000 7f1f0fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 0x7f1f081121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.368+0000 7f1f0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f08077040 0x7f1f0810c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.368+0000 7f1f0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08112a30 0x7f1f0810c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.368+0000 7f1f0fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081046f0 con 0x7f1f08077040 2024-09-10T07:44:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.368+0000 7f1f0fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08104570 con 0x7f1f08112a30 2024-09-10T07:44:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.368+0000 7f1f0fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08104870 con 0x7f1f08076720 2024-09-10T07:44:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.369+0000 7f1f0ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f08077040 0x7f1f0810c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.370+0000 7f1f0ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f08077040 0x7f1f0810c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37752/0 (socket says 172.21.15.46:37752) 2024-09-10T07:44:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.370+0000 7f1f0ca00640 1 -- 172.21.15.46:0/765896305 learned_addr learned my addr 172.21.15.46:0/765896305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.370+0000 7f1f0d400640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 0x7f1f081121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.370+0000 7f1f0de00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08112a30 0x7f1f0810c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.370+0000 7f1f0d400640 1 -- 172.21.15.46:0/765896305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08112a30 msgr2=0x7f1f0810c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.371+0000 7f1f0d400640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08112a30 0x7f1f0810c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.371+0000 7f1f0d400640 1 -- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f08077040 msgr2=0x7f1f0810c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.371+0000 7f1f0d400640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f08077040 0x7f1f0810c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.371+0000 7f1f0d400640 1 -- 172.21.15.46:0/765896305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0810cef0 con 0x7f1f08076720 2024-09-10T07:44:19.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.371+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f08077040 0x7f1f0810c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:19.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.371+0000 7f1f0de00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08112a30 0x7f1f0810c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:44:19.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.371+0000 7f1f0d400640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 0x7f1f081121b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1efc035040 tx=0x7f1efc033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:19.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.372+0000 7f1f06000640 1 -- 172.21.15.46:0/765896305 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc037040 con 0x7f1f08076720 2024-09-10T07:44:19.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.372+0000 7f1f06000640 1 -- 172.21.15.46:0/765896305 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1efc030550 con 0x7f1f08076720 2024-09-10T07:44:19.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.372+0000 7f1f06000640 1 -- 172.21.15.46:0/765896305 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc037040 con 0x7f1f08076720 2024-09-10T07:44:19.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.373+0000 7f1f0fe00640 1 -- 172.21.15.46:0/765896305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0806bc10 con 0x7f1f08076720 2024-09-10T07:44:19.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.373+0000 7f1f0fe00640 1 -- 172.21.15.46:0/765896305 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0806bfe0 con 0x7f1f08076720 2024-09-10T07:44:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.382+0000 7f1f127f9740 1 Processor -- start 2024-09-10T07:44:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.382+0000 7f1f06000640 1 -- 172.21.15.46:0/765896305 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1efc042020 con 0x7f1f08076720 2024-09-10T07:44:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.383+0000 7f1f127f9740 1 -- start start 2024-09-10T07:44:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.383+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af54c70 0x55842af55050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.384+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.384+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af5fe50 0x55842af62240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.384+0000 7f1f127f9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55842aebb3e0 con 0x55842af54c70 2024-09-10T07:44:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.384+0000 7f1f127f9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55842ae62d80 con 0x55842af55590 2024-09-10T07:44:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.384+0000 7f1f127f9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55842aeb73a0 con 0x55842af5fe50 2024-09-10T07:44:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.384+0000 7f1f0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af54c70 0x55842af55050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af54c70 0x55842af55050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37758/0 (socket says 172.21.15.46:37758) 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0de00640 1 -- 172.21.15.46:0/119593108 learned_addr learned my addr 172.21.15.46:0/119593108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af5fe50 0x55842af62240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0de00640 1 -- 172.21.15.46:0/119593108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af5fe50 msgr2=0x55842af62240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0de00640 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af5fe50 0x55842af62240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0de00640 1 -- 172.21.15.46:0/119593108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 msgr2=0x55842af5f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0de00640 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.385+0000 7f1f0de00640 1 -- 172.21.15.46:0/119593108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842ae81ae0 con 0x55842af54c70 2024-09-10T07:44:19.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.386+0000 7f1f0d400640 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:19.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.387+0000 7f1f0de00640 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af54c70 0x55842af55050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009ec0 tx=0x7f1f0000f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eeb7b3a10456a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:19.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.387+0000 7f1f06000640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ef00a2e50 0x7f1ef00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.388+0000 7f1f06000640 1 -- 172.21.15.46:0/765896305 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1efc0ee330 con 0x7f1f08076720 2024-09-10T07:44:19.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.388+0000 7f1efa600640 1 -- 172.21.15.46:0/119593108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0001d070 con 0x55842af54c70 2024-09-10T07:44:19.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.388+0000 7f1efa600640 1 -- 172.21.15.46:0/119593108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f0000dad0 con 0x55842af54c70 2024-09-10T07:44:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.388+0000 7f1efa600640 1 -- 172.21.15.46:0/119593108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00018c70 con 0x55842af54c70 2024-09-10T07:44:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.389+0000 7f1f0d400640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ef00a2e50 0x7f1ef00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.390+0000 7f1f127f9740 1 -- 172.21.15.46:0/119593108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af54c70 msgr2=0x55842af55050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.390+0000 7f1f127f9740 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af54c70 0x55842af55050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009ec0 tx=0x7f1f0000f7d0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.390+0000 7f1f127f9740 1 -- 172.21.15.46:0/119593108 shutdown_connections 2024-09-10T07:44:19.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.390+0000 7f1f127f9740 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af5fe50 0x55842af62240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.390+0000 7f1f127f9740 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.390+0000 7f1f127f9740 1 --2- 172.21.15.46:0/119593108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af54c70 0x55842af55050 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.390+0000 7f1f127f9740 1 -- 172.21.15.46:0/119593108 >> 172.21.15.46:0/119593108 conn(0x55842af51200 msgr2=0x55842af515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:19.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.391+0000 7f1f0d400640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ef00a2e50 0x7f1ef00a5310 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009780 tx=0x7f1efc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:19.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.392+0000 7f1f127f9740 1 -- 172.21.15.46:0/119593108 shutdown_connections 2024-09-10T07:44:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.393+0000 7f1f127f9740 1 -- 172.21.15.46:0/119593108 wait complete. 2024-09-10T07:44:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.395+0000 7f1f127f9740 1 -- start start 2024-09-10T07:44:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.395+0000 7f1f127f9740 1 Processor -- start 2024-09-10T07:44:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.396+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af5cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.396+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.396+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af5fe50 0x55842af5d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.396+0000 7f1f127f9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55842af62a20 con 0x55842af5fe50 2024-09-10T07:44:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.396+0000 7f1f127f9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55842ae760f0 con 0x55842af55590 2024-09-10T07:44:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.396+0000 7f1f127f9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55842af62ba0 con 0x55842af54c70 2024-09-10T07:44:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.397+0000 7f1f127f9740 10 client.0 _ll_register_callbacks cb 0x55842ae5dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.397+0000 7f1f0ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.397+0000 7f1f0ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:54012/0 (socket says 172.21.15.46:54012) 2024-09-10T07:44:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.397+0000 7f1f0ca00640 1 -- 172.21.15.46:0/1497141253 learned_addr learned my addr 172.21.15.46:0/1497141253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.397+0000 7f1f0de00640 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af5cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.397+0000 7f1f0ca00640 1 -- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 msgr2=0x55842af5cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.398+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af5cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.398+0000 7f1f0ca00640 1 -- 172.21.15.46:0/1497141253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af5fe50 msgr2=0x55842af5d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.398+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af5fe50 0x55842af5d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.398+0000 7f1f0ca00640 1 -- 172.21.15.46:0/1497141253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842ac2c3c0 con 0x55842af55590 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.398+0000 7f1f0de00640 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af5cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.399+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5d310 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4004540 tx=0x7f1ef40039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5b7ddcb7daea16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.399+0000 7f1eeea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.400+0000 7f1eeea00640 1 -- 172.21.15.46:0/1497141253 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011830 con 0x55842af55590 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.400+0000 7f1eeea00640 1 -- 172.21.15.46:0/1497141253 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1ef40119d0 con 0x55842af55590 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.400+0000 7f1f127f9740 1 -- 172.21.15.46:0/1497141253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0000dad0 con 0x55842af55590 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.400+0000 7f1eeea00640 1 -- 172.21.15.46:0/1497141253 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1ef4011bd0 con 0x55842af55590 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.400+0000 7f1f127f9740 10 client.27308 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.400+0000 7f1f127f9740 1 -- 172.21.15.46:0/1497141253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55842af6eab0 con 0x55842af55590 2024-09-10T07:44:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.401+0000 7f1eeea00640 1 -- 172.21.15.46:0/1497141253 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f0000dad0 con 0x55842af55590 2024-09-10T07:44:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.402+0000 7f1eeea00640 1 -- 172.21.15.46:0/1497141253 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f1ef4025420 con 0x55842af55590 2024-09-10T07:44:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.402+0000 7f1f127f9740 10 client.27308 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.402+0000 7f1f127f9740 10 client.27308 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:44:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.402+0000 7f1f127f9740 20 client.27308 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.402+0000 7f1f127f9740 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55842af6ff70 0x55842af72380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.402+0000 7f1f127f9740 4 client.27308 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.402+0000 7f1f127f9740 1 -- 172.21.15.46:0/1497141253 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55842aec8950 con 0x55842af6ff70 2024-09-10T07:44:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.403+0000 7f1f0de00640 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55842af6ff70 0x55842af72380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.404+0000 7f1f0de00640 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55842af6ff70 0x55842af72380 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6dbff89af5ee8c6 server_cookie=8cb233a15bf0ed0c in_seq=0 out_seq=0 2024-09-10T07:44:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.404+0000 7f1eeea00640 10 client.27308 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:44:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.405+0000 7f1eeea00640 1 -- 172.21.15.46:0/1497141253 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55842aec8950 con 0x55842af6ff70 2024-09-10T07:44:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.405+0000 7f1eeea00640 10 client.27308 handle_command_reply: tid=0 2024-09-10T07:44:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.405+0000 7f1f127f9740 1 client.27308 shutdown 2024-09-10T07:44:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.405+0000 7f1f127f9740 1 -- 172.21.15.46:0/1497141253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 msgr2=0x55842af5d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.405+0000 7f1f127f9740 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5d310 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4004540 tx=0x7f1ef40039c0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.406+0000 7f1f127f9740 1 -- 172.21.15.46:0/1497141253 shutdown_connections 2024-09-10T07:44:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.406+0000 7f1f127f9740 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af5fe50 0x55842af5d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.406+0000 7f1f127f9740 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af5d310 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.406+0000 7f1f127f9740 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55842af6ff70 0x55842af72380 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.406+0000 7f1f127f9740 1 --2- 172.21.15.46:0/1497141253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af5cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.406+0000 7f1f127f9740 1 -- 172.21.15.46:0/1497141253 >> 172.21.15.46:0/1497141253 conn(0x55842af51200 msgr2=0x55842af50380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.406+0000 7f1eeea00640 0 client.27308 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:44:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.407+0000 7f1f127f9740 1 -- 172.21.15.46:0/1497141253 shutdown_connections 2024-09-10T07:44:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.407+0000 7f1f127f9740 1 -- 172.21.15.46:0/1497141253 wait complete. 2024-09-10T07:44:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.407+0000 7f1f127f9740 20 client.27308 trim_cache size 0 max 16384 2024-09-10T07:44:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.433+0000 7f1f127f9740 1 Processor -- start 2024-09-10T07:44:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.433+0000 7f1f127f9740 1 -- start start 2024-09-10T07:44:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.434+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af57720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.434+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af55590 0x55842af5ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.434+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af5fe50 0x55842af77c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.434+0000 7f1f127f9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081c6130 con 0x55842af55590 2024-09-10T07:44:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.434+0000 7f1f127f9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081c7d60 con 0x55842af5fe50 2024-09-10T07:44:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.434+0000 7f1f127f9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081c6800 con 0x55842af54c70 2024-09-10T07:44:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.436+0000 7f1f0de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af57720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.436+0000 7f1f0de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af57720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37554/0 (socket says 172.21.15.46:37554) 2024-09-10T07:44:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.436+0000 7f1f0de00640 1 -- 172.21.15.46:0/94688701 learned_addr learned my addr 172.21.15.46:0/94688701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.436+0000 7f1f0de00640 1 --2- 172.21.15.46:0/94688701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af5fe50 0x55842af77c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.437+0000 7f1f0de00640 1 -- 172.21.15.46:0/94688701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af5fe50 msgr2=0x55842af77c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.437+0000 7f1f0de00640 1 --2- 172.21.15.46:0/94688701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af5fe50 0x55842af77c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.437+0000 7f1f0de00640 1 -- 172.21.15.46:0/94688701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af55590 msgr2=0x55842af5ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.437+0000 7f1f0de00640 1 --2- 172.21.15.46:0/94688701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af55590 0x55842af5ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.437+0000 7f1f0de00640 1 -- 172.21.15.46:0/94688701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842af54aa0 con 0x55842af54c70 2024-09-10T07:44:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.438+0000 7f1f0de00640 1 --2- 172.21.15.46:0/94688701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af57720 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000fcd0 tx=0x7f1f000187b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae76b31dabfa9f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.439+0000 7f1ef9c00640 1 -- 172.21.15.46:0/94688701 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0001d030 con 0x55842af54c70 2024-09-10T07:44:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.439+0000 7f1ef9c00640 1 -- 172.21.15.46:0/94688701 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f00028040 con 0x55842af54c70 2024-09-10T07:44:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.439+0000 7f1ef9c00640 1 -- 172.21.15.46:0/94688701 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000d5f0 con 0x55842af54c70 2024-09-10T07:44:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.441+0000 7f1f127f9740 1 -- 172.21.15.46:0/94688701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 msgr2=0x55842af57720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.441+0000 7f1f127f9740 1 --2- 172.21.15.46:0/94688701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af57720 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000fcd0 tx=0x7f1f000187b0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.442+0000 7f1f127f9740 1 -- 172.21.15.46:0/94688701 shutdown_connections 2024-09-10T07:44:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.442+0000 7f1f127f9740 1 --2- 172.21.15.46:0/94688701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af5fe50 0x55842af77c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.442+0000 7f1f127f9740 1 --2- 172.21.15.46:0/94688701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af55590 0x55842af5ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.442+0000 7f1f127f9740 1 --2- 172.21.15.46:0/94688701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af57720 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.442+0000 7f1f127f9740 1 -- 172.21.15.46:0/94688701 >> 172.21.15.46:0/94688701 conn(0x55842af51200 msgr2=0x55842af5e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.442+0000 7f1f127f9740 1 -- 172.21.15.46:0/94688701 shutdown_connections 2024-09-10T07:44:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.442+0000 7f1f127f9740 1 -- 172.21.15.46:0/94688701 wait complete. 2024-09-10T07:44:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.443+0000 7f1f127f9740 1 -- start start 2024-09-10T07:44:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.443+0000 7f1f127f9740 1 Processor -- start 2024-09-10T07:44:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.444+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af68280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.445+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af63600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.445+0000 7f1f127f9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af5fe50 0x55842af63b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.445+0000 7f1f127f9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081c75d0 con 0x55842af5fe50 2024-09-10T07:44:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.445+0000 7f1f127f9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081c6ed0 con 0x55842af55590 2024-09-10T07:44:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.446+0000 7f1f127f9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08076b00 con 0x55842af54c70 2024-09-10T07:44:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.446+0000 7f1f0de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af68280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.446+0000 7f1f0de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af68280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37568/0 (socket says 172.21.15.46:37568) 2024-09-10T07:44:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.446+0000 7f1f0de00640 1 -- 172.21.15.46:0/3445562310 learned_addr learned my addr 172.21.15.46:0/3445562310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.447+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af63600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.447+0000 7f1f127f9740 10 client.0 _ll_register_callbacks cb 0x7f1ef400be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.448+0000 7f1f0de00640 1 -- 172.21.15.46:0/3445562310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 msgr2=0x55842af63600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.448+0000 7f1f0de00640 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af63600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.448+0000 7f1f0de00640 1 -- 172.21.15.46:0/3445562310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af5fe50 msgr2=0x55842af63b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.448+0000 7f1f0de00640 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af5fe50 0x55842af63b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.449+0000 7f1f0de00640 1 -- 172.21.15.46:0/3445562310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842af4c770 con 0x55842af54c70 2024-09-10T07:44:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.449+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af63600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:44:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.451+0000 7f1eeea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.451+0000 7f1f0de00640 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af68280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000a010 tx=0x7f1f0000fd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba79570dfea22083 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.452+0000 7f1eeea00640 1 -- 172.21.15.46:0/3445562310 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0001d040 con 0x55842af54c70 2024-09-10T07:44:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.452+0000 7f1eeea00640 1 -- 172.21.15.46:0/3445562310 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1f00002030 con 0x55842af54c70 2024-09-10T07:44:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.452+0000 7f1f127f9740 1 -- 172.21.15.46:0/3445562310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f00028040 con 0x55842af54c70 2024-09-10T07:44:19.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.453+0000 7f1eeea00640 1 -- 172.21.15.46:0/3445562310 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1f00018a30 con 0x55842af54c70 2024-09-10T07:44:19.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.454+0000 7f1f127f9740 10 client.27256 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:19.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.454+0000 7f1f127f9740 1 -- 172.21.15.46:0/3445562310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55842afda780 con 0x55842af54c70 2024-09-10T07:44:19.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.454+0000 7f1eeea00640 1 -- 172.21.15.46:0/3445562310 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f00028040 con 0x55842af54c70 2024-09-10T07:44:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.458+0000 7f1eeea00640 1 -- 172.21.15.46:0/3445562310 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f1f0002d020 con 0x55842af54c70 2024-09-10T07:44:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.459+0000 7f1f127f9740 10 client.27256 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.459+0000 7f1f127f9740 10 client.27256 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:44:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.459+0000 7f1f127f9740 20 client.27256 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.459+0000 7f1f127f9740 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55842afdac00 0x55842afdcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.459+0000 7f1f127f9740 4 client.27256 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:44:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.459+0000 7f1f127f9740 1 -- 172.21.15.46:0/3445562310 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55842aec2e60 con 0x55842afdac00 2024-09-10T07:44:19.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.459+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55842afdac00 0x55842afdcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:19.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.460+0000 7f1f0ca00640 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55842afdac00 0x55842afdcff0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7875d4d51e370f1 server_cookie=b3032e57dfa53240 in_seq=0 out_seq=0 2024-09-10T07:44:19.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.460+0000 7f1eeea00640 10 client.27256 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:44:19.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1eeea00640 1 -- 172.21.15.46:0/3445562310 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55842aec2e60 con 0x55842afdac00 2024-09-10T07:44:19.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1eeea00640 10 client.27256 handle_command_reply: tid=0 2024-09-10T07:44:19.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1f127f9740 1 client.27256 shutdown 2024-09-10T07:44:19.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1f127f9740 1 -- 172.21.15.46:0/3445562310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 msgr2=0x55842af68280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1f127f9740 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af68280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000a010 tx=0x7f1f0000fd00 comp rx=0 tx=0).stop 2024-09-10T07:44:19.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1f127f9740 1 -- 172.21.15.46:0/3445562310 shutdown_connections 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1f127f9740 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842af5fe50 0x55842af63b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1f127f9740 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55842af55590 0x55842af63600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.461+0000 7f1f127f9740 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55842afdac00 0x55842afdcff0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.462+0000 7f1f127f9740 1 --2- 172.21.15.46:0/3445562310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55842af54c70 0x55842af68280 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.462+0000 7f1f127f9740 1 -- 172.21.15.46:0/3445562310 >> 172.21.15.46:0/3445562310 conn(0x55842af51200 msgr2=0x55842af5ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.462+0000 7f1eeea00640 0 client.27256 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.462+0000 7f1f127f9740 1 -- 172.21.15.46:0/3445562310 shutdown_connections 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.462+0000 7f1f127f9740 1 -- 172.21.15.46:0/3445562310 wait complete. 2024-09-10T07:44:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.462+0000 7f1f127f9740 20 client.27256 trim_cache size 0 max 16384 2024-09-10T07:44:19.464 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:44:19.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.463+0000 7f1f0fe00640 1 -- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ef00a2e50 msgr2=0x7f1ef00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.463+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ef00a2e50 0x7f1ef00a5310 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009780 tx=0x7f1efc005f90 comp rx=0 tx=0).stop 2024-09-10T07:44:19.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.464+0000 7f1f0fe00640 1 -- 172.21.15.46:0/765896305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 msgr2=0x7f1f081121b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:19.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.464+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 0x7f1f081121b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1efc035040 tx=0x7f1efc033d10 comp rx=0 tx=0).stop 2024-09-10T07:44:19.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.466+0000 7f1f0fe00640 1 -- 172.21.15.46:0/765896305 shutdown_connections 2024-09-10T07:44:19.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.466+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08112a30 0x7f1f0810c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.466+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ef00a2e50 0x7f1ef00a5310 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.466+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f08077040 0x7f1f0810c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.466+0000 7f1f0fe00640 1 --2- 172.21.15.46:0/765896305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f08076720 0x7f1f081121b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.466+0000 7f1f0fe00640 1 -- 172.21.15.46:0/765896305 >> 172.21.15.46:0/765896305 conn(0x7f1f080ffa90 msgr2=0x7f1f08101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.467+0000 7f1f0fe00640 1 -- 172.21.15.46:0/765896305 shutdown_connections 2024-09-10T07:44:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:19.467+0000 7f1f0fe00640 1 -- 172.21.15.46:0/765896305 wait complete. 2024-09-10T07:44:20.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:19 smithi046 ceph-mon[30705]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-09-10T07:44:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:19 smithi181 ceph-mon[35397]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-09-10T07:44:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:19 smithi063 ceph-mon[35020]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-09-10T07:44:20.405 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:44:20.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 420.0 s, 956.4 tps, lat 106.536 ms stddev 134.102 2024-09-10T07:44:21.101 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.004+0000 7f0a57200640 1 -- 172.21.15.46:0/1656175031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5810bfc0 msgr2=0x7f0a5810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.004+0000 7f0a57200640 1 --2- 172.21.15.46:0/1656175031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5810bfc0 0x7f0a5810e460 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009860 tx=0x7f0a5002a3d0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.006+0000 7f0a57200640 1 -- 172.21.15.46:0/1656175031 shutdown_connections 2024-09-10T07:44:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.006+0000 7f0a57200640 1 --2- 172.21.15.46:0/1656175031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5810bfc0 0x7f0a5810e460 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.006+0000 7f0a57200640 1 --2- 172.21.15.46:0/1656175031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a581034b0 0x7f0a5810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.006+0000 7f0a57200640 1 --2- 172.21.15.46:0/1656175031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a58102b90 0x7f0a58102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.006+0000 7f0a57200640 1 -- 172.21.15.46:0/1656175031 >> 172.21.15.46:0/1656175031 conn(0x7f0a580fc610 msgr2=0x7f0a580fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.008+0000 7f0a57200640 1 -- 172.21.15.46:0/1656175031 shutdown_connections 2024-09-10T07:44:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.009+0000 7f0a57200640 1 -- 172.21.15.46:0/1656175031 wait complete. 2024-09-10T07:44:22.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.013+0000 7f0a57200640 1 Processor -- start 2024-09-10T07:44:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.013+0000 7f0a57200640 1 -- start start 2024-09-10T07:44:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.013+0000 7f0a57200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a58102b90 0x7f0a5819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.013+0000 7f0a57200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a581034b0 0x7f0a5819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.013+0000 7f0a57200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5810bfc0 0x7f0a5819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.013+0000 7f0a57200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58111d90 con 0x7f0a5810bfc0 2024-09-10T07:44:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.013+0000 7f0a57200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58111c10 con 0x7f0a58102b90 2024-09-10T07:44:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.013+0000 7f0a57200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58111f10 con 0x7f0a581034b0 2024-09-10T07:44:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.014+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a58102b90 0x7f0a5819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.014+0000 7f0a55400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a581034b0 0x7f0a5819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.014+0000 7f0a55400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a581034b0 0x7f0a5819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37584/0 (socket says 172.21.15.46:37584) 2024-09-10T07:44:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.014+0000 7f0a55400640 1 -- 172.21.15.46:0/235527953 learned_addr learned my addr 172.21.15.46:0/235527953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:22.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.016+0000 7f0a56800640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5810bfc0 0x7f0a5819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.017+0000 7f0a55e00640 1 -- 172.21.15.46:0/235527953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a581034b0 msgr2=0x7f0a5819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.017+0000 7f0a55e00640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a581034b0 0x7f0a5819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.017+0000 7f0a55e00640 1 -- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5810bfc0 msgr2=0x7f0a5819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.018+0000 7f0a55e00640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5810bfc0 0x7f0a5819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.018+0000 7f0a55e00640 1 -- 172.21.15.46:0/235527953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5819f3f0 con 0x7f0a58102b90 2024-09-10T07:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.018+0000 7f0a55400640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a581034b0 0x7f0a5819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.019+0000 7f0a55e00640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a58102b90 0x7f0a5819a420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800ed50 tx=0x7f0a4800c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.020+0000 7f0a4ea00640 1 -- 172.21.15.46:0/235527953 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48003b70 con 0x7f0a58102b90 2024-09-10T07:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.020+0000 7f0a4ea00640 1 -- 172.21.15.46:0/235527953 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a48003d10 con 0x7f0a58102b90 2024-09-10T07:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.020+0000 7f0a4ea00640 1 -- 172.21.15.46:0/235527953 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48017ab0 con 0x7f0a58102b90 2024-09-10T07:44:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.020+0000 7f0a57200640 1 -- 172.21.15.46:0/235527953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5819f6e0 con 0x7f0a58102b90 2024-09-10T07:44:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.020+0000 7f0a57200640 1 -- 172.21.15.46:0/235527953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5806bc10 con 0x7f0a58102b90 2024-09-10T07:44:22.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.030+0000 7f0a5fcd1740 1 Processor -- start 2024-09-10T07:44:22.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.031+0000 7f0a5fcd1740 1 -- start start 2024-09-10T07:44:22.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.031+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.032+0000 7f0a4ea00640 1 -- 172.21.15.46:0/235527953 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a4801e020 con 0x7f0a58102b90 2024-09-10T07:44:22.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.032+0000 7f0a56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.033+0000 7f0a56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37810/0 (socket says 172.21.15.46:37810) 2024-09-10T07:44:22.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.033+0000 7f0a56800640 1 -- 172.21.15.46:0/3297847926 learned_addr learned my addr 172.21.15.46:0/3297847926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:22.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.034+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca900d72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.034+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900d7830 0x55ca900d9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.034+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/3297847926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ca9002a3e0 con 0x55ca900cc650 2024-09-10T07:44:22.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.034+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/3297847926 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ca8ffd1d80 con 0x55ca900d7830 2024-09-10T07:44:22.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.034+0000 7f0a4ea00640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a3c0a2d10 0x7f0a3c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.035+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/3297847926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ca900263a0 con 0x55ca900ccf70 2024-09-10T07:44:22.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.035+0000 7f0a55e00640 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca900d72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.035+0000 7f0a55400640 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900d7830 0x55ca900d9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.035+0000 7f0a4ea00640 1 -- 172.21.15.46:0/235527953 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0a480cf310 con 0x7f0a58102b90 2024-09-10T07:44:22.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.036+0000 7f0a55400640 1 -- 172.21.15.46:0/3297847926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 msgr2=0x55ca900d72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:22.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.036+0000 7f0a55400640 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca900d72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.036+0000 7f0a55400640 1 -- 172.21.15.46:0/3297847926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 msgr2=0x55ca900cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:22.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.036+0000 7f0a55400640 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.036+0000 7f0a55400640 1 -- 172.21.15.46:0/3297847926 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8fff0ae0 con 0x55ca900d7830 2024-09-10T07:44:22.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.036+0000 7f0a56800640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a3c0a2d10 0x7f0a3c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.036+0000 7f0a55400640 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900d7830 0x55ca900d9c20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0a44005fd0 tx=0x7f0a440108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e7ba3af348c1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:22.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.037+0000 7f0a43400640 1 -- 172.21.15.46:0/3297847926 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44011820 con 0x55ca900d7830 2024-09-10T07:44:22.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.037+0000 7f0a43400640 1 -- 172.21.15.46:0/3297847926 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4401b430 con 0x55ca900d7830 2024-09-10T07:44:22.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.037+0000 7f0a43400640 1 -- 172.21.15.46:0/3297847926 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400d5a0 con 0x55ca900d7830 2024-09-10T07:44:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.038+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/3297847926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900d7830 msgr2=0x55ca900d9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.038+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900d7830 0x55ca900d9c20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0a44005fd0 tx=0x7f0a440108e0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.038+0000 7f0a56800640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a3c0a2d10 0x7f0a3c0a51d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0a50034000 tx=0x7f0a5002a3d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.039+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/3297847926 shutdown_connections 2024-09-10T07:44:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.039+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900d7830 0x55ca900d9c20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.039+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca900d72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.039+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/3297847926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.039+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/3297847926 >> 172.21.15.46:0/3297847926 conn(0x55ca900c8be0 msgr2=0x55ca900c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.040+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/3297847926 shutdown_connections 2024-09-10T07:44:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.040+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/3297847926 wait complete. 2024-09-10T07:44:22.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.041+0000 7f0a5fcd1740 1 -- start start 2024-09-10T07:44:22.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.041+0000 7f0a5fcd1740 1 Processor -- start 2024-09-10T07:44:22.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.042+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.042+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900ccf70 0x55ca900d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.042+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900d7830 0x55ca900d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.042+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ca900da450 con 0x55ca900cc650 2024-09-10T07:44:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.042+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ca8ffe50f0 con 0x55ca900ccf70 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.042+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ca900da5d0 con 0x55ca900d7830 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.042+0000 7f0a5fcd1740 10 client.0 _ll_register_callbacks cb 0x55ca8ffccbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.043+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.043+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37820/0 (socket says 172.21.15.46:37820) 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.043+0000 7f0a55e00640 1 -- 172.21.15.46:0/2880575013 learned_addr learned my addr 172.21.15.46:0/2880575013 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.044+0000 7f0a55e00640 1 -- 172.21.15.46:0/2880575013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900d7830 msgr2=0x55ca900d50e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.044+0000 7f0a55e00640 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900d7830 0x55ca900d50e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.044+0000 7f0a55e00640 1 -- 172.21.15.46:0/2880575013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900ccf70 msgr2=0x55ca900d4ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.044+0000 7f0a55e00640 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900ccf70 0x55ca900d4ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.044+0000 7f0a55e00640 1 -- 172.21.15.46:0/2880575013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8fd9b3c0 con 0x55ca900cc650 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.046+0000 7f0a55e00640 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900d4660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a480ccae0 tx=0x7f0a4806d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c38cb30388ae02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.046+0000 7f0a37e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.047+0000 7f0a37e00640 1 -- 172.21.15.46:0/2880575013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4809fba0 con 0x55ca900cc650 2024-09-10T07:44:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.047+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2880575013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a4401b430 con 0x55ca900cc650 2024-09-10T07:44:22.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.047+0000 7f0a37e00640 1 -- 172.21.15.46:0/2880575013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0a4809ba50 con 0x55ca900cc650 2024-09-10T07:44:22.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.047+0000 7f0a37e00640 1 -- 172.21.15.46:0/2880575013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0a4809bc50 con 0x55ca900cc650 2024-09-10T07:44:22.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.047+0000 7f0a5fcd1740 10 client.16839 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.047+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2880575013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ca900e64e0 con 0x55ca900cc650 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.048+0000 7f0a37e00640 1 -- 172.21.15.46:0/2880575013 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4401b430 con 0x55ca900cc650 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.048+0000 7f0a37e00640 1 -- 172.21.15.46:0/2880575013 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f0a4800b030 con 0x55ca900cc650 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.048+0000 7f0a5fcd1740 10 client.16839 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.048+0000 7f0a5fcd1740 10 client.16839 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.048+0000 7f0a5fcd1740 20 client.16839 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.048+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ca900e7940 0x55ca900e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.048+0000 7f0a5fcd1740 4 client.16839 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.048+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2880575013 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca90037950 con 0x55ca900e7940 2024-09-10T07:44:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.049+0000 7f0a55400640 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ca900e7940 0x55ca900e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.050+0000 7f0a55400640 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ca900e7940 0x55ca900e9d90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85edc1f5e0fd0662 server_cookie=5ebfed2899d7f0a4 in_seq=0 out_seq=0 2024-09-10T07:44:22.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.051+0000 7f0a37e00640 10 client.16839 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:44:22.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.051+0000 7f0a37e00640 1 -- 172.21.15.46:0/2880575013 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca90037950 con 0x55ca900e7940 2024-09-10T07:44:22.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.051+0000 7f0a37e00640 10 client.16839 handle_command_reply: tid=0 2024-09-10T07:44:22.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.051+0000 7f0a5fcd1740 1 client.16839 shutdown 2024-09-10T07:44:22.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.052+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2880575013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 msgr2=0x55ca900d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.052+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900d4660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a480ccae0 tx=0x7f0a4806d880 comp rx=0 tx=0).stop 2024-09-10T07:44:22.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2880575013 shutdown_connections 2024-09-10T07:44:22.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ca900e7940 0x55ca900e9d90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900d7830 0x55ca900d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900ccf70 0x55ca900d4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2880575013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900cc650 0x55ca900d4660 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2880575013 >> 172.21.15.46:0/2880575013 conn(0x55ca900c8be0 msgr2=0x55ca900ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:22.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a37e00640 0 client.16839 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:44:22.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2880575013 shutdown_connections 2024-09-10T07:44:22.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.053+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2880575013 wait complete. 2024-09-10T07:44:22.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.054+0000 7f0a5fcd1740 20 client.16839 trim_cache size 0 max 16384 2024-09-10T07:44:22.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.078+0000 7f0a5fcd1740 1 Processor -- start 2024-09-10T07:44:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.078+0000 7f0a5fcd1740 1 -- start start 2024-09-10T07:44:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.078+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900cc650 0x55ca900ef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.078+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900ccf70 0x55ca900efba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.078+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900d7830 0x55ca900bec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.078+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58071560 con 0x55ca900ccf70 2024-09-10T07:44:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.078+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58073190 con 0x55ca900cc650 2024-09-10T07:44:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.078+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58071c30 con 0x55ca900d7830 2024-09-10T07:44:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.079+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900cc650 0x55ca900ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.079+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900cc650 0x55ca900ef660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:54086/0 (socket says 172.21.15.46:54086) 2024-09-10T07:44:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.079+0000 7f0a55e00640 1 -- 172.21.15.46:0/2633571981 learned_addr learned my addr 172.21.15.46:0/2633571981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.079+0000 7f0a55400640 1 --2- 172.21.15.46:0/2633571981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900ccf70 0x55ca900efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.080+0000 7f0a55e00640 1 -- 172.21.15.46:0/2633571981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900d7830 msgr2=0x55ca900bec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.080+0000 7f0a55e00640 1 --2- 172.21.15.46:0/2633571981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900d7830 0x55ca900bec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.080+0000 7f0a55e00640 1 -- 172.21.15.46:0/2633571981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900ccf70 msgr2=0x55ca900efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.080+0000 7f0a55e00640 1 --2- 172.21.15.46:0/2633571981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900ccf70 0x55ca900efba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.080+0000 7f0a55e00640 1 -- 172.21.15.46:0/2633571981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca900e3700 con 0x55ca900cc650 2024-09-10T07:44:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.081+0000 7f0a55e00640 1 --2- 172.21.15.46:0/2633571981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900cc650 0x55ca900ef660 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0a480174f0 tx=0x7f0a48099d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a8b5a535b83304b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.081+0000 7f0a42a00640 1 -- 172.21.15.46:0/2633571981 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a480b4070 con 0x55ca900cc650 2024-09-10T07:44:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.081+0000 7f0a42a00640 1 -- 172.21.15.46:0/2633571981 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4809fba0 con 0x55ca900cc650 2024-09-10T07:44:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.081+0000 7f0a42a00640 1 -- 172.21.15.46:0/2633571981 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4806c070 con 0x55ca900cc650 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2633571981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900cc650 msgr2=0x55ca900ef660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2633571981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900cc650 0x55ca900ef660 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0a480174f0 tx=0x7f0a48099d00 comp rx=0 tx=0).stop 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2633571981 shutdown_connections 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2633571981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900d7830 0x55ca900bec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2633571981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900ccf70 0x55ca900efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/2633571981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca900cc650 0x55ca900ef660 secure :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0a480174f0 tx=0x7f0a48099d00 comp rx=0 tx=0).stop 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2633571981 >> 172.21.15.46:0/2633571981 conn(0x55ca900c8be0 msgr2=0x55ca900d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2633571981 shutdown_connections 2024-09-10T07:44:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.083+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/2633571981 wait complete. 2024-09-10T07:44:22.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.084+0000 7f0a5fcd1740 1 -- start start 2024-09-10T07:44:22.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.084+0000 7f0a5fcd1740 1 Processor -- start 2024-09-10T07:44:22.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca9014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900d7830 0x55ca9014bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a5fcd1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca9014c460 0x55ca9014c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58072a00 con 0x55ca900d7830 2024-09-10T07:44:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58072300 con 0x55ca9014c460 2024-09-10T07:44:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a5fcd1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5806f560 con 0x55ca900ccf70 2024-09-10T07:44:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a5fcd1740 10 client.0 _ll_register_callbacks cb 0x7f0a4809b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a55400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900d7830 0x55ca9014bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca9014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca9014dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:37612/0 (socket says 172.21.15.46:37612) 2024-09-10T07:44:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a55400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900d7830 0x55ca9014bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37846/0 (socket says 172.21.15.46:37846) 2024-09-10T07:44:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a55400640 1 -- 172.21.15.46:0/4161994407 learned_addr learned my addr 172.21.15.46:0/4161994407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.085+0000 7f0a56800640 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca9014c460 0x55ca9014c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.086+0000 7f0a55e00640 1 -- 172.21.15.46:0/4161994407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca9014c460 msgr2=0x55ca9014c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.086+0000 7f0a55e00640 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca9014c460 0x55ca9014c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.086+0000 7f0a55e00640 1 -- 172.21.15.46:0/4161994407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900d7830 msgr2=0x55ca9014bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:22.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.086+0000 7f0a55e00640 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900d7830 0x55ca9014bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.086+0000 7f0a55e00640 1 -- 172.21.15.46:0/4161994407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9014e430 con 0x55ca900ccf70 2024-09-10T07:44:22.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.087+0000 7f0a55e00640 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca9014dc10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55ca900d2ec0 tx=0x7f0a4809b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82c9109eb1dcb2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:22.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.087+0000 7f0a37e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:22.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.087+0000 7f0a37e00640 1 -- 172.21.15.46:0/4161994407 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4806c050 con 0x55ca900ccf70 2024-09-10T07:44:22.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.088+0000 7f0a37e00640 1 -- 172.21.15.46:0/4161994407 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0a48004720 con 0x55ca900ccf70 2024-09-10T07:44:22.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.088+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/4161994407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca90031e60 con 0x55ca900ccf70 2024-09-10T07:44:22.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.088+0000 7f0a37e00640 1 -- 172.21.15.46:0/4161994407 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0a480b4070 con 0x55ca900ccf70 2024-09-10T07:44:22.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.089+0000 7f0a5fcd1740 10 client.27268 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.089+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/4161994407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ca900df9a0 con 0x55ca900ccf70 2024-09-10T07:44:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.089+0000 7f0a37e00640 1 -- 172.21.15.46:0/4161994407 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a480ba020 con 0x55ca900ccf70 2024-09-10T07:44:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.090+0000 7f0a37e00640 1 -- 172.21.15.46:0/4161994407 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f0a48097540 con 0x55ca900ccf70 2024-09-10T07:44:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.090+0000 7f0a5fcd1740 10 client.27268 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.090+0000 7f0a5fcd1740 10 client.27268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:44:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.090+0000 7f0a5fcd1740 20 client.27268 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.090+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ca900e0070 0x55ca900c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:22.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.090+0000 7f0a5fcd1740 4 client.27268 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:44:22.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.090+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/4161994407 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca8ffae010 con 0x55ca900e0070 2024-09-10T07:44:22.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.091+0000 7f0a55400640 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ca900e0070 0x55ca900c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:22.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.091+0000 7f0a37e00640 10 client.27268 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:44:22.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.091+0000 7f0a55400640 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ca900e0070 0x55ca900c6cb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b668dcf06361b751 server_cookie=f55dd5a3f1767b3a in_seq=0 out_seq=0 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a37e00640 1 -- 172.21.15.46:0/4161994407 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca8ffae010 con 0x55ca900e0070 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a37e00640 10 client.27268 handle_command_reply: tid=0 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 client.27268 shutdown 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/4161994407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 msgr2=0x55ca9014dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca9014dc10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55ca900d2ec0 tx=0x7f0a4809b6f0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/4161994407 shutdown_connections 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ca900e0070 0x55ca900c6cb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca9014c460 0x55ca9014c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca900d7830 0x55ca9014bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 --2- 172.21.15.46:0/4161994407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca900ccf70 0x55ca9014dc10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.092+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/4161994407 >> 172.21.15.46:0/4161994407 conn(0x55ca900c8be0 msgr2=0x55ca900e26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.093+0000 7f0a37e00640 0 client.27268 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:44:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.093+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/4161994407 shutdown_connections 2024-09-10T07:44:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.093+0000 7f0a5fcd1740 1 -- 172.21.15.46:0/4161994407 wait complete. 2024-09-10T07:44:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.093+0000 7f0a5fcd1740 20 client.27268 trim_cache size 0 max 16384 2024-09-10T07:44:22.094 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:44:22.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 -- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a3c0a2d10 msgr2=0x7f0a3c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a3c0a2d10 0x7f0a3c0a51d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0a50034000 tx=0x7f0a5002a3d0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 -- 172.21.15.46:0/235527953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a58102b90 msgr2=0x7f0a5819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a58102b90 0x7f0a5819a420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800ed50 tx=0x7f0a4800c6e0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 -- 172.21.15.46:0/235527953 shutdown_connections 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0a3c0a2d10 0x7f0a3c0a51d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a5810bfc0 0x7f0a5819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a581034b0 0x7f0a5819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 --2- 172.21.15.46:0/235527953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a58102b90 0x7f0a5819a420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.094+0000 7f0a57200640 1 -- 172.21.15.46:0/235527953 >> 172.21.15.46:0/235527953 conn(0x7f0a580fc610 msgr2=0x7f0a5810d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.095+0000 7f0a57200640 1 -- 172.21.15.46:0/235527953 shutdown_connections 2024-09-10T07:44:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:22.095+0000 7f0a57200640 1 -- 172.21.15.46:0/235527953 wait complete. 2024-09-10T07:44:22.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:21 smithi046 ceph-mon[30705]: pgmap v631: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-09-10T07:44:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:21 smithi181 ceph-mon[35397]: pgmap v631: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-09-10T07:44:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:21 smithi063 ceph-mon[35020]: pgmap v631: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-09-10T07:44:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:24 smithi181 ceph-mon[35397]: pgmap v632: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1k op/s 2024-09-10T07:44:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:24 smithi063 ceph-mon[35020]: pgmap v632: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1k op/s 2024-09-10T07:44:24.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:24 smithi046 ceph-mon[30705]: pgmap v632: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1k op/s 2024-09-10T07:44:25.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:44:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 425.0 s, 758.2 tps, lat 131.520 ms stddev 160.474 2024-09-10T07:44:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:26.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:26 smithi046 ceph-mon[30705]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-09-10T07:44:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:26 smithi181 ceph-mon[35397]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-09-10T07:44:26.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:26 smithi063 ceph-mon[35020]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-09-10T07:44:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 -- 172.21.15.46:0/3230628117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec0114ab0 msgr2=0x7f1ec0116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 --2- 172.21.15.46:0/3230628117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec0114ab0 0x7f1ec0116ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009ca0 tx=0x7f1eb802a950 comp rx=0 tx=0).stop 2024-09-10T07:44:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 -- 172.21.15.46:0/3230628117 shutdown_connections 2024-09-10T07:44:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 --2- 172.21.15.46:0/3230628117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec0114ab0 0x7f1ec0116ea0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 --2- 172.21.15.46:0/3230628117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ec0109b10 0x7f1ec0114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 --2- 172.21.15.46:0/3230628117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ec01091f0 0x7f1ec01095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 -- 172.21.15.46:0/3230628117 >> 172.21.15.46:0/3230628117 conn(0x7f1ec006e140 msgr2=0x7f1ec006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:26.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 -- 172.21.15.46:0/3230628117 shutdown_connections 2024-09-10T07:44:26.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.552+0000 7f1ec8200640 1 -- 172.21.15.46:0/3230628117 wait complete. 2024-09-10T07:44:26.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.553+0000 7f1ec8200640 1 Processor -- start 2024-09-10T07:44:26.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.556+0000 7f1ec8200640 1 -- start start 2024-09-10T07:44:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.556+0000 7f1ec8200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec01091f0 0x7f1ec0113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.556+0000 7f1ec8200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ec0109b10 0x7f1ec0114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.556+0000 7f1ec8200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ec0114ab0 0x7f1ec010b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.556+0000 7f1ec8200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec011a7e0 con 0x7f1ec0109b10 2024-09-10T07:44:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.556+0000 7f1ec8200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec011a660 con 0x7f1ec0114ab0 2024-09-10T07:44:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.556+0000 7f1ec8200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec011a960 con 0x7f1ec01091f0 2024-09-10T07:44:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.557+0000 7f1ec6200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ec0114ab0 0x7f1ec010b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.557+0000 7f1ec6200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ec0114ab0 0x7f1ec010b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45284/0 (socket says 172.21.15.46:45284) 2024-09-10T07:44:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.557+0000 7f1ec6200640 1 -- 172.21.15.46:0/4233339961 learned_addr learned my addr 172.21.15.46:0/4233339961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.557+0000 7f1ec5800640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec01091f0 0x7f1ec0113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.557+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ec0109b10 0x7f1ec0114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.558+0000 7f1ec5800640 1 -- 172.21.15.46:0/4233339961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ec0114ab0 msgr2=0x7f1ec010b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.558+0000 7f1ec5800640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ec0114ab0 0x7f1ec010b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.558+0000 7f1ec5800640 1 -- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ec0109b10 msgr2=0x7f1ec0114480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.558+0000 7f1ec5800640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ec0109b10 0x7f1ec0114480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.558+0000 7f1ec5800640 1 -- 172.21.15.46:0/4233339961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec010c250 con 0x7f1ec01091f0 2024-09-10T07:44:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.558+0000 7f1ec6200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ec0114ab0 0x7f1ec010b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:44:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.558+0000 7f1ec5800640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec01091f0 0x7f1ec0113f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400b3f0 tx=0x7f1eb400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.559+0000 7f1ebe000640 1 -- 172.21.15.46:0/4233339961 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4015020 con 0x7f1ec01091f0 2024-09-10T07:44:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.559+0000 7f1ebe000640 1 -- 172.21.15.46:0/4233339961 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb4011d10 con 0x7f1ec01091f0 2024-09-10T07:44:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.559+0000 7f1ebe000640 1 -- 172.21.15.46:0/4233339961 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400f2e0 con 0x7f1ec01091f0 2024-09-10T07:44:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.560+0000 7f1ec8200640 1 -- 172.21.15.46:0/4233339961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec010c540 con 0x7f1ec01091f0 2024-09-10T07:44:26.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.560+0000 7f1ec8200640 1 -- 172.21.15.46:0/4233339961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec010fbd0 con 0x7f1ec01091f0 2024-09-10T07:44:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.571+0000 7f1ebe000640 1 -- 172.21.15.46:0/4233339961 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1eb4018020 con 0x7f1ec01091f0 2024-09-10T07:44:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.571+0000 7f1ecae46740 1 Processor -- start 2024-09-10T07:44:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.574+0000 7f1ebe000640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ea80a2c40 0x7f1ea80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.575+0000 7f1ecae46740 1 -- start start 2024-09-10T07:44:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.575+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f778050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.575+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f778590 0x55b50f782910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.575+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f785240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.575+0000 7f1ecae46740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b50f6de3e0 con 0x55b50f778590 2024-09-10T07:44:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.575+0000 7f1ecae46740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b50f685d80 con 0x55b50f777c70 2024-09-10T07:44:26.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.575+0000 7f1ecae46740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b50f6da3a0 con 0x55b50f782e50 2024-09-10T07:44:26.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.576+0000 7f1ec4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f778590 0x55b50f782910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.576+0000 7f1ec4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f778590 0x55b50f782910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44572/0 (socket says 172.21.15.46:44572) 2024-09-10T07:44:26.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.576+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1454945446 learned_addr learned my addr 172.21.15.46:0/1454945446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:26.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.577+0000 7f1ec6200640 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f785240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.577+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1454945446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 msgr2=0x55b50f785240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.577+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f785240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.577+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1454945446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 msgr2=0x55b50f778050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:26.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.577+0000 7f1ec5800640 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f778050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.577+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f778050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.577+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1454945446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50f6a4ae0 con 0x55b50f778590 2024-09-10T07:44:26.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.577+0000 7f1ec5800640 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f778050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:26.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.578+0000 7f1ec6200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ea80a2c40 0x7f1ea80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.579+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f778590 0x55b50f782910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00bae0 tx=0x7f1eac00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed86f24cb70c30c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:26.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.579+0000 7f1ebe000640 1 -- 172.21.15.46:0/4233339961 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1eb40cea40 con 0x7f1ec01091f0 2024-09-10T07:44:26.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.580+0000 7f1eb2a00640 1 -- 172.21.15.46:0/1454945446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac017420 con 0x55b50f778590 2024-09-10T07:44:26.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.580+0000 7f1eb2a00640 1 -- 172.21.15.46:0/1454945446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac017aa0 con 0x55b50f778590 2024-09-10T07:44:26.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.580+0000 7f1eb2a00640 1 -- 172.21.15.46:0/1454945446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac00fb20 con 0x55b50f778590 2024-09-10T07:44:26.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.580+0000 7f1ec6200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ea80a2c40 0x7f1ea80a5100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8005fa0 tx=0x7f1eb8005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.581+0000 7f1ecae46740 1 -- 172.21.15.46:0/1454945446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f778590 msgr2=0x55b50f782910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.581+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f778590 0x55b50f782910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00bae0 tx=0x7f1eac00d340 comp rx=0 tx=0).stop 2024-09-10T07:44:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.582+0000 7f1ecae46740 1 -- 172.21.15.46:0/1454945446 shutdown_connections 2024-09-10T07:44:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.582+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f785240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.582+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f778590 0x55b50f782910 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.582+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1454945446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f778050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.582+0000 7f1ecae46740 1 -- 172.21.15.46:0/1454945446 >> 172.21.15.46:0/1454945446 conn(0x55b50f774200 msgr2=0x55b50f7745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.582+0000 7f1ecae46740 1 -- 172.21.15.46:0/1454945446 shutdown_connections 2024-09-10T07:44:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.583+0000 7f1ecae46740 1 -- 172.21.15.46:0/1454945446 wait complete. 2024-09-10T07:44:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.583+0000 7f1ecae46740 1 -- start start 2024-09-10T07:44:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.584+0000 7f1ecae46740 1 Processor -- start 2024-09-10T07:44:26.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.585+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f77fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.585+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f780220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f780760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ecae46740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b50f785a20 con 0x55b50f777c70 2024-09-10T07:44:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ecae46740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b50f6990f0 con 0x55b50f778590 2024-09-10T07:44:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ec4e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f780220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ecae46740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b50f785ba0 con 0x55b50f782e50 2024-09-10T07:44:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ec4e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f780220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45290/0 (socket says 172.21.15.46:45290) 2024-09-10T07:44:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ec4e00640 1 -- 172.21.15.46:0/4273096631 learned_addr learned my addr 172.21.15.46:0/4273096631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ec6200640 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f780760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.586+0000 7f1ecae46740 10 client.0 _ll_register_callbacks cb 0x55b50f680bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ec5800640 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f77fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ec6200640 1 -- 172.21.15.46:0/4273096631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 msgr2=0x55b50f780220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ec6200640 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f780220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ec6200640 1 -- 172.21.15.46:0/4273096631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 msgr2=0x55b50f77fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ec6200640 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f77fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ec6200640 1 -- 172.21.15.46:0/4273096631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50f44f3c0 con 0x55b50f782e50 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ec5800640 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f77fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ec6200640 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f780760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55b50f775580 tx=0x7f1eb80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40452a9778cadd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:26.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.587+0000 7f1ea7400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.588+0000 7f1ea7400640 1 -- 172.21.15.46:0/4273096631 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8053060 con 0x55b50f782e50 2024-09-10T07:44:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.588+0000 7f1ea7400640 1 -- 172.21.15.46:0/4273096631 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1eb8040030 con 0x55b50f782e50 2024-09-10T07:44:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.588+0000 7f1ecae46740 1 -- 172.21.15.46:0/4273096631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eac017aa0 con 0x55b50f782e50 2024-09-10T07:44:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.589+0000 7f1ea7400640 1 -- 172.21.15.46:0/4273096631 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1eb8034040 con 0x55b50f782e50 2024-09-10T07:44:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.589+0000 7f1ecae46740 10 client.27277 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:26.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.589+0000 7f1ecae46740 1 -- 172.21.15.46:0/4273096631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b50f791940 con 0x55b50f782e50 2024-09-10T07:44:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.589+0000 7f1ea7400640 1 -- 172.21.15.46:0/4273096631 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac017aa0 con 0x55b50f782e50 2024-09-10T07:44:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.591+0000 7f1ea7400640 1 -- 172.21.15.46:0/4273096631 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f1eb8030a70 con 0x55b50f782e50 2024-09-10T07:44:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.591+0000 7f1ecae46740 10 client.27277 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.591+0000 7f1ecae46740 10 client.27277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:44:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.592+0000 7f1ecae46740 20 client.27277 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:26.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.592+0000 7f1ecae46740 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b50f792e00 0x55b50f795290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.592+0000 7f1ecae46740 4 client.27277 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:26.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.592+0000 7f1ecae46740 1 -- 172.21.15.46:0/4273096631 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b50f6eb950 con 0x55b50f792e00 2024-09-10T07:44:26.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.592+0000 7f1ec5800640 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b50f792e00 0x55b50f795290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.593+0000 7f1ec5800640 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b50f792e00 0x55b50f795290 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a27647937ce9ab2 server_cookie=def6c009379bd471 in_seq=0 out_seq=0 2024-09-10T07:44:26.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.593+0000 7f1ea7400640 10 client.27277 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:44:26.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.594+0000 7f1ea7400640 1 -- 172.21.15.46:0/4273096631 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b50f6eb950 con 0x55b50f792e00 2024-09-10T07:44:26.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.594+0000 7f1ea7400640 10 client.27277 handle_command_reply: tid=0 2024-09-10T07:44:26.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.594+0000 7f1ecae46740 1 client.27277 shutdown 2024-09-10T07:44:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 -- 172.21.15.46:0/4273096631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 msgr2=0x55b50f780760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f780760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55b50f775580 tx=0x7f1eb80095c0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 -- 172.21.15.46:0/4273096631 shutdown_connections 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b50f792e00 0x55b50f795290 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f780760 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f780220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 --2- 172.21.15.46:0/4273096631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f77fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 -- 172.21.15.46:0/4273096631 >> 172.21.15.46:0/4273096631 conn(0x55b50f774200 msgr2=0x55b50f773380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ea7400640 0 client.27277 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.595+0000 7f1ecae46740 1 -- 172.21.15.46:0/4273096631 shutdown_connections 2024-09-10T07:44:26.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.596+0000 7f1ecae46740 1 -- 172.21.15.46:0/4273096631 wait complete. 2024-09-10T07:44:26.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.596+0000 7f1ecae46740 20 client.27277 trim_cache size 0 max 16384 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.627+0000 7f1ecae46740 1 Processor -- start 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.627+0000 7f1ecae46740 1 -- start start 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.628+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f79ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.628+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f79b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.628+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f781540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.628+0000 7f1ecae46740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01c6350 con 0x55b50f777c70 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.628+0000 7f1ecae46740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01c7f80 con 0x55b50f778590 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.628+0000 7f1ecae46740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01c6a20 con 0x55b50f782e50 2024-09-10T07:44:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.628+0000 7f1ec4e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f79b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.629+0000 7f1ec4e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f79b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45306/0 (socket says 172.21.15.46:45306) 2024-09-10T07:44:26.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.629+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1224070289 learned_addr learned my addr 172.21.15.46:0/1224070289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:26.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.629+0000 7f1ec6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f781540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.629+0000 7f1ec5800640 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f79ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.630+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1224070289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 msgr2=0x55b50f781540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:26.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.630+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f781540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.630+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1224070289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 msgr2=0x55b50f79ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.630+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f79ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.630+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1224070289 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50f76f770 con 0x55b50f778590 2024-09-10T07:44:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.630+0000 7f1ec5800640 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f79ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.631+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f79b0a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005d20 tx=0x7f1eac00ddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a771db3baa1b8aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.631+0000 7f1eb2000640 1 -- 172.21.15.46:0/1224070289 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac027020 con 0x55b50f778590 2024-09-10T07:44:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.632+0000 7f1eb2000640 1 -- 172.21.15.46:0/1224070289 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac023b50 con 0x55b50f778590 2024-09-10T07:44:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.632+0000 7f1eb2000640 1 -- 172.21.15.46:0/1224070289 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac017ab0 con 0x55b50f778590 2024-09-10T07:44:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.633+0000 7f1ecae46740 1 -- 172.21.15.46:0/1224070289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 msgr2=0x55b50f79b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.633+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f79b0a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005d20 tx=0x7f1eac00ddf0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.633+0000 7f1ecae46740 1 -- 172.21.15.46:0/1224070289 shutdown_connections 2024-09-10T07:44:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.633+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f782e50 0x55b50f781540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.633+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f778590 0x55b50f79b0a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.633+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1224070289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f777c70 0x55b50f79ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.633+0000 7f1ecae46740 1 -- 172.21.15.46:0/1224070289 >> 172.21.15.46:0/1224070289 conn(0x55b50f774200 msgr2=0x55b50f77af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.635+0000 7f1ecae46740 1 -- 172.21.15.46:0/1224070289 shutdown_connections 2024-09-10T07:44:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.635+0000 7f1ecae46740 1 -- 172.21.15.46:0/1224070289 wait complete. 2024-09-10T07:44:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.636+0000 7f1ecae46740 1 -- start start 2024-09-10T07:44:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.636+0000 7f1ecae46740 1 Processor -- start 2024-09-10T07:44:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.637+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f78b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.637+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f778590 0x55b50f7866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.637+0000 7f1ecae46740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f782e50 0x55b50f786be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.638+0000 7f1ecae46740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01c77f0 con 0x55b50f782e50 2024-09-10T07:44:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.638+0000 7f1ecae46740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01c70f0 con 0x55b50f777c70 2024-09-10T07:44:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.638+0000 7f1ecae46740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01c4330 con 0x55b50f778590 2024-09-10T07:44:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.638+0000 7f1ecae46740 10 client.0 _ll_register_callbacks cb 0x7f1eb8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.638+0000 7f1ec4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f778590 0x55b50f7866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.638+0000 7f1ec4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f778590 0x55b50f7866a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60062/0 (socket says 172.21.15.46:60062) 2024-09-10T07:44:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.638+0000 7f1ec4e00640 1 -- 172.21.15.46:0/1712733943 learned_addr learned my addr 172.21.15.46:0/1712733943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.638+0000 7f1ec5800640 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f78b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.639+0000 7f1ec5800640 1 -- 172.21.15.46:0/1712733943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f778590 msgr2=0x55b50f7866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:26.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.639+0000 7f1ec5800640 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f778590 0x55b50f7866a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.639+0000 7f1ec5800640 1 -- 172.21.15.46:0/1712733943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f782e50 msgr2=0x55b50f786be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:26.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.639+0000 7f1ec5800640 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f782e50 0x55b50f786be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.639+0000 7f1ec5800640 1 -- 172.21.15.46:0/1712733943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50f787220 con 0x55b50f777c70 2024-09-10T07:44:26.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.640+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f778590 0x55b50f7866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:26.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.640+0000 7f1ec5800640 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f78b370 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40cc190 tx=0x7f1eb40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3a3860f1b1b690 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:26.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.640+0000 7f1ea7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:26.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.642+0000 7f1ea7400640 1 -- 172.21.15.46:0/1712733943 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40a3b90 con 0x55b50f777c70 2024-09-10T07:44:26.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.642+0000 7f1ea7400640 1 -- 172.21.15.46:0/1712733943 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1eb4010380 con 0x55b50f777c70 2024-09-10T07:44:26.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.642+0000 7f1ecae46740 1 -- 172.21.15.46:0/1712733943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eac023b50 con 0x55b50f777c70 2024-09-10T07:44:26.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ea7400640 1 -- 172.21.15.46:0/1712733943 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1eb40a3b90 con 0x55b50f777c70 2024-09-10T07:44:26.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ecae46740 10 client.27341 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:26.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ecae46740 1 -- 172.21.15.46:0/1712733943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b50f7fd960 con 0x55b50f777c70 2024-09-10T07:44:26.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ea7400640 1 -- 172.21.15.46:0/1712733943 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb4099220 con 0x55b50f777c70 2024-09-10T07:44:26.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ea7400640 1 -- 172.21.15.46:0/1712733943 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f1eb40b9b60 con 0x55b50f777c70 2024-09-10T07:44:26.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ecae46740 10 client.27341 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:26.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ecae46740 10 client.27341 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:44:26.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ecae46740 20 client.27341 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:26.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b50f7fe030 0x55b50f800450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:26.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ecae46740 4 client.27341 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:44:26.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.643+0000 7f1ecae46740 1 -- 172.21.15.46:0/1712733943 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1eac023b50 con 0x55b50f7fe030 2024-09-10T07:44:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.644+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b50f7fe030 0x55b50f800450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ec4e00640 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b50f7fe030 0x55b50f800450 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfce3c167a5fb3f0 server_cookie=ab090c2d91db8a48 in_seq=0 out_seq=0 2024-09-10T07:44:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ea7400640 10 client.27341 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:44:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ea7400640 1 -- 172.21.15.46:0/1712733943 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1eac023b50 con 0x55b50f7fe030 2024-09-10T07:44:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ea7400640 10 client.27341 handle_command_reply: tid=0 2024-09-10T07:44:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 client.27341 shutdown 2024-09-10T07:44:26.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 -- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 msgr2=0x55b50f78b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f78b370 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40cc190 tx=0x7f1eb40046c0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 -- 172.21.15.46:0/1712733943 shutdown_connections 2024-09-10T07:44:26.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55b50f7fe030 0x55b50f800450 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b50f782e50 0x55b50f786be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b50f778590 0x55b50f7866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ea7400640 0 client.27341 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:44:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 --2- 172.21.15.46:0/1712733943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b50f777c70 0x55b50f78b370 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 -- 172.21.15.46:0/1712733943 >> 172.21.15.46:0/1712733943 conn(0x55b50f774200 msgr2=0x55b50f77fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 -- 172.21.15.46:0/1712733943 shutdown_connections 2024-09-10T07:44:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.645+0000 7f1ecae46740 1 -- 172.21.15.46:0/1712733943 wait complete. 2024-09-10T07:44:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.646+0000 7f1ecae46740 20 client.27341 trim_cache size 0 max 16384 2024-09-10T07:44:26.646 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:44:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 -- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ea80a2c40 msgr2=0x7f1ea80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ea80a2c40 0x7f1ea80a5100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8005fa0 tx=0x7f1eb8005ef0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 -- 172.21.15.46:0/4233339961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec01091f0 msgr2=0x7f1ec0113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec01091f0 0x7f1ec0113f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400b3f0 tx=0x7f1eb400b8b0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 -- 172.21.15.46:0/4233339961 shutdown_connections 2024-09-10T07:44:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ec0114ab0 0x7f1ec010b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1ea80a2c40 0x7f1ea80a5100 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ec0109b10 0x7f1ec0114480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 --2- 172.21.15.46:0/4233339961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ec01091f0 0x7f1ec0113f40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:26.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.647+0000 7f1ec8200640 1 -- 172.21.15.46:0/4233339961 >> 172.21.15.46:0/4233339961 conn(0x7f1ec006e140 msgr2=0x7f1ec010a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:26.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.648+0000 7f1ec8200640 1 -- 172.21.15.46:0/4233339961 shutdown_connections 2024-09-10T07:44:26.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:26.648+0000 7f1ec8200640 1 -- 172.21.15.46:0/4233339961 wait complete. 2024-09-10T07:44:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.142+0000 7f2d50c00640 1 -- 172.21.15.46:0/1770633731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c078790 msgr2=0x7f2d4c079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.142+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1770633731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c078790 0x7f2d4c079030 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009ca0 tx=0x7f2d4402a8e0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.143+0000 7f2d50c00640 1 -- 172.21.15.46:0/1770633731 shutdown_connections 2024-09-10T07:44:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.143+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1770633731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c078790 0x7f2d4c079030 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.143+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1770633731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d4c077df0 0x7f2d4c078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.143+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1770633731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4c076bb0 0x7f2d4c076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.143+0000 7f2d50c00640 1 -- 172.21.15.46:0/1770633731 >> 172.21.15.46:0/1770633731 conn(0x7f2d4c0ff4e0 msgr2=0x7f2d4c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.143+0000 7f2d50c00640 1 -- 172.21.15.46:0/1770633731 shutdown_connections 2024-09-10T07:44:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.144+0000 7f2d50c00640 1 -- 172.21.15.46:0/1770633731 wait complete. 2024-09-10T07:44:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.145+0000 7f2d50c00640 1 Processor -- start 2024-09-10T07:44:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.146+0000 7f2d50c00640 1 -- start start 2024-09-10T07:44:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.146+0000 7f2d50c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d4c076bb0 0x7f2d4c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.147+0000 7f2d50c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c077df0 0x7f2d4c071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.147+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d4c076bb0 0x7f2d4c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.147+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d4c076bb0 0x7f2d4c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45348/0 (socket says 172.21.15.46:45348) 2024-09-10T07:44:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.147+0000 7f2d49e00640 1 -- 172.21.15.46:0/1004096700 learned_addr learned my addr 172.21.15.46:0/1004096700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.147+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4c078790 0x7f2d4c1ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c113f20 con 0x7f2d4c078790 2024-09-10T07:44:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d49400640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c077df0 0x7f2d4c071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c113da0 con 0x7f2d4c076bb0 2024-09-10T07:44:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1140a0 con 0x7f2d4c077df0 2024-09-10T07:44:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d4a800640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4c078790 0x7f2d4c1ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d49400640 1 -- 172.21.15.46:0/1004096700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d4c076bb0 msgr2=0x7f2d4c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d49400640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d4c076bb0 0x7f2d4c070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d49400640 1 -- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4c078790 msgr2=0x7f2d4c1ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d49400640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4c078790 0x7f2d4c1ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.148+0000 7f2d49400640 1 -- 172.21.15.46:0/1004096700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c071730 con 0x7f2d4c077df0 2024-09-10T07:44:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.149+0000 7f2d4a800640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4c078790 0x7f2d4c1ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:44:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.149+0000 7f2d49400640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c077df0 0x7f2d4c071090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009fd0 tx=0x7f2d3800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.149+0000 7f2d42a00640 1 -- 172.21.15.46:0/1004096700 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38015020 con 0x7f2d4c077df0 2024-09-10T07:44:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.149+0000 7f2d42a00640 1 -- 172.21.15.46:0/1004096700 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d38011d10 con 0x7f2d4c077df0 2024-09-10T07:44:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.149+0000 7f2d42a00640 1 -- 172.21.15.46:0/1004096700 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800f2e0 con 0x7f2d4c077df0 2024-09-10T07:44:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.149+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4c1aec90 con 0x7f2d4c077df0 2024-09-10T07:44:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.150+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4c1af160 con 0x7f2d4c077df0 2024-09-10T07:44:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.163+0000 7f2d53ac7740 1 Processor -- start 2024-09-10T07:44:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.163+0000 7f2d53ac7740 1 -- start start 2024-09-10T07:44:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.163+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c2090 0x55f3d01c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.163+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c29d0 0x55f3d01c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.163+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c7a60 0x55f3d01c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.163+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3d0127100 con 0x55f3d01c7a60 2024-09-10T07:44:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.163+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3d00cf1b0 con 0x55f3d01c2090 2024-09-10T07:44:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.163+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f3d01230c0 con 0x55f3d01c29d0 2024-09-10T07:44:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.164+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c29d0 0x55f3d01c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.164+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c29d0 0x55f3d01c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60110/0 (socket says 172.21.15.46:60110) 2024-09-10T07:44:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.164+0000 7f2d49e00640 1 -- 172.21.15.46:0/2500470427 learned_addr learned my addr 172.21.15.46:0/2500470427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.165+0000 7f2d4a800640 1 --2- 172.21.15.46:0/2500470427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c2090 0x55f3d01c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.165+0000 7f2d42a00640 1 -- 172.21.15.46:0/1004096700 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2d38018020 con 0x7f2d4c077df0 2024-09-10T07:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.165+0000 7f2d4a800640 1 -- 172.21.15.46:0/2500470427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c29d0 msgr2=0x55f3d01c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.165+0000 7f2d4a800640 1 --2- 172.21.15.46:0/2500470427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c29d0 0x55f3d01c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.165+0000 7f2d4a800640 1 -- 172.21.15.46:0/2500470427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c7a60 msgr2=0x55f3d01c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.165+0000 7f2d4a800640 1 --2- 172.21.15.46:0/2500470427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c7a60 0x55f3d01c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.165+0000 7f2d4a800640 1 -- 172.21.15.46:0/2500470427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3cfe3a210 con 0x55f3d01c2090 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.166+0000 7f2d4a800640 1 --2- 172.21.15.46:0/2500470427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c2090 0x55f3d01c2490 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2d4402ae00 tx=0x7f2d440315a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed84ad4f8d239c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.168+0000 7f2d37000640 1 -- 172.21.15.46:0/2500470427 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44031d00 con 0x55f3d01c2090 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.168+0000 7f2d37000640 1 -- 172.21.15.46:0/2500470427 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d44036040 con 0x55f3d01c2090 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.168+0000 7f2d37000640 1 -- 172.21.15.46:0/2500470427 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d440305e0 con 0x55f3d01c2090 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 -- 172.21.15.46:0/2500470427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c2090 msgr2=0x55f3d01c2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/2500470427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c2090 0x55f3d01c2490 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2d4402ae00 tx=0x7f2d440315a0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 -- 172.21.15.46:0/2500470427 shutdown_connections 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/2500470427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c7a60 0x55f3d01c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/2500470427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c29d0 0x55f3d01c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/2500470427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c2090 0x55f3d01c2490 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 -- 172.21.15.46:0/2500470427 >> 172.21.15.46:0/2500470427 conn(0x55f3d01bd700 msgr2=0x55f3d01bdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 -- 172.21.15.46:0/2500470427 shutdown_connections 2024-09-10T07:44:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.169+0000 7f2d53ac7740 1 -- 172.21.15.46:0/2500470427 wait complete. 2024-09-10T07:44:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.170+0000 7f2d53ac7740 1 -- start start 2024-09-10T07:44:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.170+0000 7f2d53ac7740 1 Processor -- start 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.170+0000 7f2d42a00640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d300a2c40 0x7f2d300a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 0x55f3d01cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3d01caa20 con 0x55f3d01c29d0 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3d00b0090 con 0x55f3d01c7a60 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f3d01caba0 con 0x55f3d01c2090 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d53ac7740 10 client.0 _ll_register_callbacks cb 0x55f3d0104f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60112/0 (socket says 172.21.15.46:60112) 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.171+0000 7f2d49e00640 1 -- 172.21.15.46:0/737123896 learned_addr learned my addr 172.21.15.46:0/737123896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.172+0000 7f2d49e00640 1 -- 172.21.15.46:0/737123896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 msgr2=0x55f3d01d3840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:44:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.172+0000 7f2d49e00640 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d3840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.172+0000 7f2d49e00640 1 -- 172.21.15.46:0/737123896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 msgr2=0x55f3d01cc800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.172+0000 7f2d49e00640 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 0x55f3d01cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.172+0000 7f2d49e00640 1 -- 172.21.15.46:0/737123896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3cff06620 con 0x55f3d01c2090 2024-09-10T07:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.172+0000 7f2d4a800640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d300a2c40 0x7f2d300a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.172+0000 7f2d49e00640 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01cc2c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c011260 tx=0x7f2d3c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831b9d8e079d8713 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.172+0000 7f2d2b400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.173+0000 7f2d42a00640 1 -- 172.21.15.46:0/1004096700 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2d380cebc0 con 0x7f2d4c077df0 2024-09-10T07:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.173+0000 7f2d2b400640 1 -- 172.21.15.46:0/737123896 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c019070 con 0x55f3d01c2090 2024-09-10T07:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.173+0000 7f2d2b400640 1 -- 172.21.15.46:0/737123896 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2d3c015870 con 0x55f3d01c2090 2024-09-10T07:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.173+0000 7f2d53ac7740 1 -- 172.21.15.46:0/737123896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d44036040 con 0x55f3d01c2090 2024-09-10T07:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.174+0000 7f2d4a800640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d300a2c40 0x7f2d300a5100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009980 tx=0x7f2d4403d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.174+0000 7f2d2b400640 1 -- 172.21.15.46:0/737123896 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d3c023050 con 0x55f3d01c2090 2024-09-10T07:44:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d53ac7740 10 client.27295 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d53ac7740 1 -- 172.21.15.46:0/737123896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f3d01d6750 con 0x55f3d01c2090 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d2b400640 1 -- 172.21.15.46:0/737123896 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d44036040 con 0x55f3d01c2090 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d2b400640 1 -- 172.21.15.46:0/737123896 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f2d3c01ec80 con 0x55f3d01c2090 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d53ac7740 10 client.27295 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d53ac7740 10 client.27295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d53ac7740 20 client.27295 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3d01d7f50 0x55f3d01da390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d53ac7740 4 client.27295 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.177+0000 7f2d53ac7740 1 -- 172.21.15.46:0/737123896 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3d0134830 con 0x55f3d01d7f50 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.181+0000 7f2d49400640 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3d01d7f50 0x55f3d01da390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.181+0000 7f2d49400640 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3d01d7f50 0x55f3d01da390 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a492181454ab99d server_cookie=40fc45c7e0a28b6c in_seq=0 out_seq=0 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.181+0000 7f2d2b400640 10 client.27295 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.182+0000 7f2d2b400640 1 -- 172.21.15.46:0/737123896 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3d0134830 con 0x55f3d01d7f50 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.182+0000 7f2d2b400640 10 client.27295 handle_command_reply: tid=0 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.183+0000 7f2d53ac7740 1 client.27295 shutdown 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.183+0000 7f2d53ac7740 1 -- 172.21.15.46:0/737123896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 msgr2=0x55f3d01cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.183+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01cc2c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c011260 tx=0x7f2d3c011720 comp rx=0 tx=0).stop 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.187+0000 7f2d53ac7740 1 -- 172.21.15.46:0/737123896 shutdown_connections 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.188+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3d01d7f50 0x55f3d01da390 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.188+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.188+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 0x55f3d01cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.188+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/737123896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01cc2c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.188+0000 7f2d53ac7740 1 -- 172.21.15.46:0/737123896 >> 172.21.15.46:0/737123896 conn(0x55f3d01bd700 msgr2=0x55f3d01bef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.188+0000 7f2d2b400640 0 client.27295 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.189+0000 7f2d53ac7740 1 -- 172.21.15.46:0/737123896 shutdown_connections 2024-09-10T07:44:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.190+0000 7f2d53ac7740 1 -- 172.21.15.46:0/737123896 wait complete. 2024-09-10T07:44:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.190+0000 7f2d53ac7740 20 client.27295 trim_cache size 0 max 16384 2024-09-10T07:44:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.239+0000 7f2d53ac7740 1 Processor -- start 2024-09-10T07:44:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.239+0000 7f2d53ac7740 1 -- start start 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c2090 0x55f3d01cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c29d0 0x55f3d01d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c7a60 0x55f3d01bdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1b9580 con 0x55f3d01c2090 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1bb1b0 con 0x55f3d01c29d0 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d49400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c29d0 0x55f3d01d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d49400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c29d0 0x55f3d01d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45370/0 (socket says 172.21.15.46:45370) 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d49400640 1 -- 172.21.15.46:0/1289903950 learned_addr learned my addr 172.21.15.46:0/1289903950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d4a800640 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c7a60 0x55f3d01bdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.240+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1b9c50 con 0x55f3d01c7a60 2024-09-10T07:44:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.241+0000 7f2d49e00640 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c2090 0x55f3d01cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.241+0000 7f2d49400640 1 -- 172.21.15.46:0/1289903950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c7a60 msgr2=0x55f3d01bdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.241+0000 7f2d49400640 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c7a60 0x55f3d01bdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.241+0000 7f2d49400640 1 -- 172.21.15.46:0/1289903950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c2090 msgr2=0x55f3d01cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.241+0000 7f2d49400640 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c2090 0x55f3d01cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.241+0000 7f2d49400640 1 -- 172.21.15.46:0/1289903950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d01c9e50 con 0x55f3d01c29d0 2024-09-10T07:44:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.241+0000 7f2d49400640 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c29d0 0x55f3d01d3800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800bd70 tx=0x7f2d3800e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a38b22e0f913ecfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.242+0000 7f2d36600640 1 -- 172.21.15.46:0/1289903950 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3809b740 con 0x55f3d01c29d0 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.242+0000 7f2d36600640 1 -- 172.21.15.46:0/1289903950 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3809b040 con 0x55f3d01c29d0 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.242+0000 7f2d36600640 1 -- 172.21.15.46:0/1289903950 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38099340 con 0x55f3d01c29d0 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.243+0000 7f2d53ac7740 1 -- 172.21.15.46:0/1289903950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c29d0 msgr2=0x55f3d01d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.243+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c29d0 0x55f3d01d3800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800bd70 tx=0x7f2d3800e000 comp rx=0 tx=0).stop 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.244+0000 7f2d53ac7740 1 -- 172.21.15.46:0/1289903950 shutdown_connections 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.244+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c7a60 0x55f3d01bdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.244+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c29d0 0x55f3d01d3800 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.244+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/1289903950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c2090 0x55f3d01cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.244+0000 7f2d53ac7740 1 -- 172.21.15.46:0/1289903950 >> 172.21.15.46:0/1289903950 conn(0x55f3d01bd700 msgr2=0x55f3d01cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.244+0000 7f2d53ac7740 1 -- 172.21.15.46:0/1289903950 shutdown_connections 2024-09-10T07:44:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.244+0000 7f2d53ac7740 1 -- 172.21.15.46:0/1289903950 wait complete. 2024-09-10T07:44:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.244+0000 7f2d53ac7740 1 -- start start 2024-09-10T07:44:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.245+0000 7f2d53ac7740 1 Processor -- start 2024-09-10T07:44:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.245+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.245+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 0x55f3d01d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.245+0000 7f2d53ac7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.245+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1baa20 con 0x55f3d01c29d0 2024-09-10T07:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.245+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1ba320 con 0x55f3d01c7a60 2024-09-10T07:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.245+0000 7f2d53ac7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1b7560 con 0x55f3d01c2090 2024-09-10T07:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.246+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.246+0000 7f2d4a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.246+0000 7f2d4a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45378/0 (socket says 172.21.15.46:45378) 2024-09-10T07:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.246+0000 7f2d4a800640 1 -- 172.21.15.46:0/444022527 learned_addr learned my addr 172.21.15.46:0/444022527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.246+0000 7f2d53ac7740 10 client.0 _ll_register_callbacks cb 0x7f2d3c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.247+0000 7f2d49e00640 1 -- 172.21.15.46:0/444022527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 msgr2=0x55f3d01d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.247+0000 7f2d49400640 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 0x55f3d01d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.248+0000 7f2d49e00640 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.248+0000 7f2d49e00640 1 -- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 msgr2=0x55f3d01d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.248+0000 7f2d49e00640 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 0x55f3d01d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.248+0000 7f2d49e00640 1 -- 172.21.15.46:0/444022527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d01d2920 con 0x55f3d01c2090 2024-09-10T07:44:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.248+0000 7f2d49e00640 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01d0dc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c014500 tx=0x7f2d3c009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9df10823ff00703 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.249+0000 7f2d2b400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.249+0000 7f2d4a800640 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.249+0000 7f2d2b400640 1 -- 172.21.15.46:0/444022527 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c019070 con 0x55f3d01c2090 2024-09-10T07:44:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.249+0000 7f2d2b400640 1 -- 172.21.15.46:0/444022527 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2d3c0059c0 con 0x55f3d01c2090 2024-09-10T07:44:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.249+0000 7f2d53ac7740 1 -- 172.21.15.46:0/444022527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3809b040 con 0x55f3d01c2090 2024-09-10T07:44:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.249+0000 7f2d49400640 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 0x55f3d01d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.251+0000 7f2d2b400640 1 -- 172.21.15.46:0/444022527 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d3c005be0 con 0x55f3d01c2090 2024-09-10T07:44:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.251+0000 7f2d53ac7740 10 client.27301 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.251+0000 7f2d53ac7740 1 -- 172.21.15.46:0/444022527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f3d0253c80 con 0x55f3d01c2090 2024-09-10T07:44:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.251+0000 7f2d2b400640 1 -- 172.21.15.46:0/444022527 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3809b040 con 0x55f3d01c2090 2024-09-10T07:44:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.252+0000 7f2d2b400640 1 -- 172.21.15.46:0/444022527 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f2d3c0062a0 con 0x55f3d01c2090 2024-09-10T07:44:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.252+0000 7f2d53ac7740 10 client.27301 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.252+0000 7f2d53ac7740 10 client.27301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.252+0000 7f2d53ac7740 20 client.27301 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.252+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3d0254530 0x55f3d0256950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.252+0000 7f2d53ac7740 4 client.27301 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:44:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.252+0000 7f2d53ac7740 1 -- 172.21.15.46:0/444022527 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3d012eb80 con 0x55f3d0254530 2024-09-10T07:44:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.254+0000 7f2d49400640 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3d0254530 0x55f3d0256950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.256+0000 7f2d49400640 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3d0254530 0x55f3d0256950 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a8780ac021c728 server_cookie=38df7e904e5f44e8 in_seq=0 out_seq=0 2024-09-10T07:44:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.257+0000 7f2d2b400640 10 client.27301 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.273+0000 7f2d2b400640 1 -- 172.21.15.46:0/444022527 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3d012eb80 con 0x55f3d0254530 2024-09-10T07:44:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.273+0000 7f2d2b400640 10 client.27301 handle_command_reply: tid=0 2024-09-10T07:44:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 client.27301 shutdown 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 -- 172.21.15.46:0/444022527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 msgr2=0x55f3d01d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01d0dc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c014500 tx=0x7f2d3c009730 comp rx=0 tx=0).stop 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 -- 172.21.15.46:0/444022527 shutdown_connections 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f3d0254530 0x55f3d0256950 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3d01c7a60 0x55f3d01d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3d01c29d0 0x55f3d01d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 --2- 172.21.15.46:0/444022527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f3d01c2090 0x55f3d01d0dc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d53ac7740 1 -- 172.21.15.46:0/444022527 >> 172.21.15.46:0/444022527 conn(0x55f3d01bd700 msgr2=0x55f3d01bc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.274+0000 7f2d2b400640 0 client.27301 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.275+0000 7f2d53ac7740 1 -- 172.21.15.46:0/444022527 shutdown_connections 2024-09-10T07:44:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.275+0000 7f2d53ac7740 1 -- 172.21.15.46:0/444022527 wait complete. 2024-09-10T07:44:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.275+0000 7f2d53ac7740 20 client.27301 trim_cache size 0 max 16384 2024-09-10T07:44:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:44:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:44:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:44:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:44:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.276+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d300a2c40 msgr2=0x7f2d300a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.276+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d300a2c40 0x7f2d300a5100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009980 tx=0x7f2d4403d040 comp rx=0 tx=0).stop 2024-09-10T07:44:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.276+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c077df0 msgr2=0x7f2d4c071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.276+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c077df0 0x7f2d4c071090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009fd0 tx=0x7f2d3800b870 comp rx=0 tx=0).stop 2024-09-10T07:44:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.277+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 shutdown_connections 2024-09-10T07:44:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.277+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d300a2c40 0x7f2d300a5100 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.277+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4c078790 0x7f2d4c1ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.277+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c077df0 0x7f2d4c071090 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.277+0000 7f2d50c00640 1 --2- 172.21.15.46:0/1004096700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d4c076bb0 0x7f2d4c070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.277+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 >> 172.21.15.46:0/1004096700 conn(0x7f2d4c0ff4e0 msgr2=0x7f2d4c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.278+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 shutdown_connections 2024-09-10T07:44:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:27.278+0000 7f2d50c00640 1 -- 172.21.15.46:0/1004096700 wait complete. 2024-09-10T07:44:28.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:28 smithi046 ceph-mon[30705]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-09-10T07:44:28.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:28 smithi181 ceph-mon[35397]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-09-10T07:44:28.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:28 smithi063 ceph-mon[35020]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-09-10T07:44:28.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:44:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:30 smithi181 ceph-mon[35397]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 491 op/s 2024-09-10T07:44:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:30 smithi063 ceph-mon[35020]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 491 op/s 2024-09-10T07:44:30.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:30 smithi046 ceph-mon[30705]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 491 op/s 2024-09-10T07:44:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 430.0 s, 734.4 tps, lat 132.909 ms stddev 148.412 2024-09-10T07:44:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.623+0000 7f6aa5000640 1 -- 172.21.15.46:0/254408937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 msgr2=0x7f6aa0114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.623+0000 7f6aa5000640 1 --2- 172.21.15.46:0/254408937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 0x7f6aa0114e20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009860 tx=0x7f6a9802a340 comp rx=0 tx=0).stop 2024-09-10T07:44:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.624+0000 7f6aa5000640 1 -- 172.21.15.46:0/254408937 shutdown_connections 2024-09-10T07:44:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.624+0000 7f6aa5000640 1 --2- 172.21.15.46:0/254408937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 0x7f6aa0114e20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.624+0000 7f6aa5000640 1 --2- 172.21.15.46:0/254408937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa0077040 0x7f6aa01124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.624+0000 7f6aa5000640 1 --2- 172.21.15.46:0/254408937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa0076720 0x7f6aa0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.624+0000 7f6aa5000640 1 -- 172.21.15.46:0/254408937 >> 172.21.15.46:0/254408937 conn(0x7f6aa00ffa90 msgr2=0x7f6aa0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.624+0000 7f6aa5000640 1 -- 172.21.15.46:0/254408937 shutdown_connections 2024-09-10T07:44:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.624+0000 7f6aa5000640 1 -- 172.21.15.46:0/254408937 wait complete. 2024-09-10T07:44:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.625+0000 7f6aa5000640 1 Processor -- start 2024-09-10T07:44:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.626+0000 7f6aa5000640 1 -- start start 2024-09-10T07:44:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.626+0000 7f6aa5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa0076720 0x7f6aa01a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.626+0000 7f6aa5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa0077040 0x7f6aa01a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.626+0000 7f6aa5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 0x7f6aa019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.626+0000 7f6aa5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01185d0 con 0x7f6aa0077040 2024-09-10T07:44:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.626+0000 7f6aa5000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa0118450 con 0x7f6aa0076720 2024-09-10T07:44:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.626+0000 7f6aa5000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa0118750 con 0x7f6aa0112a30 2024-09-10T07:44:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 0x7f6aa019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 0x7f6aa019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60170/0 (socket says 172.21.15.46:60170) 2024-09-10T07:44:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9e800640 1 -- 172.21.15.46:0/4269866923 learned_addr learned my addr 172.21.15.46:0/4269866923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9d400640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa0077040 0x7f6aa01a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9de00640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa0076720 0x7f6aa01a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9e800640 1 -- 172.21.15.46:0/4269866923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa0076720 msgr2=0x7f6aa01a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9e800640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa0076720 0x7f6aa01a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9e800640 1 -- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa0077040 msgr2=0x7f6aa01a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9e800640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa0077040 0x7f6aa01a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.627+0000 7f6a9e800640 1 -- 172.21.15.46:0/4269866923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa019f050 con 0x7f6aa0112a30 2024-09-10T07:44:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.628+0000 7f6a9e800640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 0x7f6aa019e920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6a9802a930 tx=0x7f6a98033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.628+0000 7f6a96a00640 1 -- 172.21.15.46:0/4269866923 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98037040 con 0x7f6aa0112a30 2024-09-10T07:44:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.628+0000 7f6aa5000640 1 -- 172.21.15.46:0/4269866923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa019f2e0 con 0x7f6aa0112a30 2024-09-10T07:44:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.628+0000 7f6a96a00640 1 -- 172.21.15.46:0/4269866923 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a98040070 con 0x7f6aa0112a30 2024-09-10T07:44:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.628+0000 7f6a96a00640 1 -- 172.21.15.46:0/4269866923 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98003950 con 0x7f6aa0112a30 2024-09-10T07:44:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.629+0000 7f6aa5000640 1 -- 172.21.15.46:0/4269866923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa01ab630 con 0x7f6aa0112a30 2024-09-10T07:44:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.634+0000 7f6aa79fd740 1 Processor -- start 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.636+0000 7f6a96a00640 1 -- 172.21.15.46:0/4269866923 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a98003af0 con 0x7f6aa0112a30 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.636+0000 7f6aa79fd740 1 -- start start 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.636+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.636+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627347910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.636+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 0x55962734a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.636+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596272a33e0 con 0x55962733d590 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55962724ad80 con 0x559627347e50 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55962729f3a0 con 0x55962733cc70 2024-09-10T07:44:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60182/0 (socket says 172.21.15.46:60182) 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 -- 172.21.15.46:0/2969601006 learned_addr learned my addr 172.21.15.46:0/2969601006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 0x55962734a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9de00640 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627347910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 -- 172.21.15.46:0/2969601006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 msgr2=0x55962734a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 0x55962734a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 -- 172.21.15.46:0/2969601006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 msgr2=0x559627347910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627347910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.637+0000 7f6a9d400640 1 -- 172.21.15.46:0/2969601006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559627269ae0 con 0x55962733cc70 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.638+0000 7f6a9de00640 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627347910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:44:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.638+0000 7f6a96a00640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a840a2bc0 0x7f6a840a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.638+0000 7f6a96a00640 1 -- 172.21.15.46:0/4269866923 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6a980ef3a0 con 0x7f6aa0112a30 2024-09-10T07:44:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.639+0000 7f6a9d400640 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733d050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009fd0 tx=0x7f6a8c0094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a6f28bf85fc229f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.639+0000 7f6a9de00640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a840a2bc0 0x7f6a840a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.640+0000 7f6a8b400640 1 -- 172.21.15.46:0/2969601006 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c015630 con 0x55962733cc70 2024-09-10T07:44:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.640+0000 7f6a8b400640 1 -- 172.21.15.46:0/2969601006 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c03f070 con 0x55962733cc70 2024-09-10T07:44:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.640+0000 7f6a8b400640 1 -- 172.21.15.46:0/2969601006 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c03a9c0 con 0x55962733cc70 2024-09-10T07:44:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.641+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2969601006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 msgr2=0x55962733d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.641+0000 7f6a9de00640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a840a2bc0 0x7f6a840a5080 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55962733a060 tx=0x7f6a900095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.641+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733d050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009fd0 tx=0x7f6a8c0094d0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.642+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2969601006 shutdown_connections 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.642+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 0x55962734a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.642+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627347910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.642+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/2969601006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733d050 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.642+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2969601006 >> 172.21.15.46:0/2969601006 conn(0x559627339200 msgr2=0x5596273395e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.642+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2969601006 shutdown_connections 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.642+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2969601006 wait complete. 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.642+0000 7f6aa79fd740 1 -- start start 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.643+0000 7f6aa79fd740 1 Processor -- start 2024-09-10T07:44:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.643+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627344d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.643+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x5596273452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.643+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559627347e50 0x559627345810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.643+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55962734a9c0 con 0x55962733d590 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.643+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55962725e0f0 con 0x55962733cc70 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.643+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55962734ab40 con 0x559627347e50 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.643+0000 7f6aa79fd740 10 client.0 _ll_register_callbacks cb 0x559627245bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627344d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627344d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45420/0 (socket says 172.21.15.46:45420) 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9de00640 1 -- 172.21.15.46:0/1947560472 learned_addr learned my addr 172.21.15.46:0/1947560472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9e800640 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559627347e50 0x559627345810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9d400640 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x5596273452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9de00640 1 -- 172.21.15.46:0/1947560472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559627347e50 msgr2=0x559627345810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9de00640 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559627347e50 0x559627345810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9de00640 1 -- 172.21.15.46:0/1947560472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 msgr2=0x5596273452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9de00640 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x5596273452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.644+0000 7f6a9de00640 1 -- 172.21.15.46:0/1947560472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596270143c0 con 0x55962733cc70 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.645+0000 7f6a9de00640 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627344d90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000ec90 tx=0x7f6a90004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a08ede1f864393 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.645+0000 7f6a7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.645+0000 7f6a7fe00640 1 -- 172.21.15.46:0/1947560472 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9002c060 con 0x55962733cc70 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.645+0000 7f6aa79fd740 1 -- 172.21.15.46:0/1947560472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c03f070 con 0x55962733cc70 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.645+0000 7f6a7fe00640 1 -- 172.21.15.46:0/1947560472 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6a90016030 con 0x55962733cc70 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.645+0000 7f6a7fe00640 1 -- 172.21.15.46:0/1947560472 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6a90033050 con 0x55962733cc70 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.645+0000 7f6aa79fd740 10 client.27362 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.645+0000 7f6aa79fd740 1 -- 172.21.15.46:0/1947560472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559627356aa0 con 0x55962733cc70 2024-09-10T07:44:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.647+0000 7f6a7fe00640 1 -- 172.21.15.46:0/1947560472 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c03f070 con 0x55962733cc70 2024-09-10T07:44:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.648+0000 7f6a7fe00640 1 -- 172.21.15.46:0/1947560472 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f6a9000b040 con 0x55962733cc70 2024-09-10T07:44:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.648+0000 7f6aa79fd740 10 client.27362 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.648+0000 7f6aa79fd740 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.648+0000 7f6aa79fd740 20 client.27362 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.648+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559627358000 0x55962735a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.648+0000 7f6aa79fd740 4 client.27362 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.648+0000 7f6aa79fd740 1 -- 172.21.15.46:0/1947560472 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596272b0950 con 0x559627358000 2024-09-10T07:44:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.649+0000 7f6a9d400640 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559627358000 0x55962735a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.650+0000 7f6a7fe00640 10 client.27362 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.651+0000 7f6a9d400640 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559627358000 0x55962735a480 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd3d7ea1cbb83a3 server_cookie=9377c691eb3e7f03 in_seq=0 out_seq=0 2024-09-10T07:44:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.655+0000 7f6a7fe00640 1 -- 172.21.15.46:0/1947560472 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596272b0950 con 0x559627358000 2024-09-10T07:44:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.655+0000 7f6a7fe00640 10 client.27362 handle_command_reply: tid=0 2024-09-10T07:44:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.655+0000 7f6aa79fd740 1 client.27362 shutdown 2024-09-10T07:44:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.657+0000 7f6aa79fd740 1 -- 172.21.15.46:0/1947560472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 msgr2=0x559627344d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.657+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627344d90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000ec90 tx=0x7f6a90004ab0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6aa79fd740 1 -- 172.21.15.46:0/1947560472 shutdown_connections 2024-09-10T07:44:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559627358000 0x55962735a480 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6a7fe00640 0 client.27362 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559627347e50 0x559627345810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x5596273452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/1947560472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627344d90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6aa79fd740 1 -- 172.21.15.46:0/1947560472 >> 172.21.15.46:0/1947560472 conn(0x559627339200 msgr2=0x55962733ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6aa79fd740 1 -- 172.21.15.46:0/1947560472 shutdown_connections 2024-09-10T07:44:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.659+0000 7f6aa79fd740 1 -- 172.21.15.46:0/1947560472 wait complete. 2024-09-10T07:44:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.660+0000 7f6aa79fd740 20 client.27362 trim_cache size 0 max 16384 2024-09-10T07:44:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.689+0000 7f6aa79fd740 1 Processor -- start 2024-09-10T07:44:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.690+0000 7f6aa79fd740 1 -- start start 2024-09-10T07:44:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.691+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.691+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627345660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.691+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 0x559627360820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.691+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01acb70 con 0x55962733d590 2024-09-10T07:44:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.691+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01ae7a0 con 0x559627347e50 2024-09-10T07:44:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.691+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01ad240 con 0x55962733cc70 2024-09-10T07:44:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.691+0000 7f6a9de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.691+0000 7f6a9de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60202/0 (socket says 172.21.15.46:60202) 2024-09-10T07:44:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.692+0000 7f6a9de00640 1 -- 172.21.15.46:0/2398743100 learned_addr learned my addr 172.21.15.46:0/2398743100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.692+0000 7f6a9e800640 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 0x559627360820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.692+0000 7f6a9de00640 1 -- 172.21.15.46:0/2398743100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 msgr2=0x559627360820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.692+0000 7f6a9de00640 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 0x559627360820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.692+0000 7f6a9de00640 1 -- 172.21.15.46:0/2398743100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 msgr2=0x559627345660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:44:30.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.693+0000 7f6a9d400640 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627345660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.693+0000 7f6a9de00640 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627345660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.693+0000 7f6a9de00640 1 -- 172.21.15.46:0/2398743100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962733a2b0 con 0x55962733cc70 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.694+0000 7f6a9de00640 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733f6d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6a90006010 tx=0x7f6a9001e050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2066cfe9fe71e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.694+0000 7f6a8aa00640 1 -- 172.21.15.46:0/2398743100 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9003c070 con 0x55962733cc70 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.694+0000 7f6a8aa00640 1 -- 172.21.15.46:0/2398743100 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9000edf0 con 0x55962733cc70 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.696+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2398743100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 msgr2=0x55962733f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.696+0000 7f6a8aa00640 1 -- 172.21.15.46:0/2398743100 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9001ca50 con 0x55962733cc70 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.696+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733f6d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6a90006010 tx=0x7f6a9001e050 comp rx=0 tx=0).stop 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.697+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2398743100 shutdown_connections 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.697+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559627347e50 0x559627360820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.697+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962733d590 0x559627345660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.697+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/2398743100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733cc70 0x55962733f6d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.697+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2398743100 >> 172.21.15.46:0/2398743100 conn(0x559627339200 msgr2=0x559627346e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:30.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.697+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2398743100 shutdown_connections 2024-09-10T07:44:30.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.698+0000 7f6aa79fd740 1 -- 172.21.15.46:0/2398743100 wait complete. 2024-09-10T07:44:30.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.699+0000 7f6aa79fd740 1 -- start start 2024-09-10T07:44:30.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.699+0000 7f6aa79fd740 1 Processor -- start 2024-09-10T07:44:30.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.702+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627350110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.702+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733d590 0x559627350650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.702+0000 7f6aa79fd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559627347e50 0x55962734b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.703+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01ae010 con 0x559627347e50 2024-09-10T07:44:30.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.703+0000 7f6a9de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627350110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.703+0000 7f6a9de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627350110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45442/0 (socket says 172.21.15.46:45442) 2024-09-10T07:44:30.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.703+0000 7f6a9de00640 1 -- 172.21.15.46:0/3065322807 learned_addr learned my addr 172.21.15.46:0/3065322807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.703+0000 7f6aa79fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01ad910 con 0x55962733cc70 2024-09-10T07:44:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.703+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa0073710 con 0x55962733d590 2024-09-10T07:44:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.703+0000 7f6aa79fd740 10 client.0 _ll_register_callbacks cb 0x7f6a90009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.705+0000 7f6a9d400640 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733d590 0x559627350650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.705+0000 7f6a9d400640 1 -- 172.21.15.46:0/3065322807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 msgr2=0x559627350110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.706+0000 7f6a9d400640 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627350110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.706+0000 7f6a9d400640 1 -- 172.21.15.46:0/3065322807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559627347e50 msgr2=0x55962734b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.706+0000 7f6a9d400640 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559627347e50 0x55962734b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.706+0000 7f6a9d400640 1 -- 172.21.15.46:0/3065322807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559627334770 con 0x55962733d590 2024-09-10T07:44:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.706+0000 7f6a9d400640 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733d590 0x559627350650 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x559627344620 tx=0x7f6a8c03a460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcd56a324b0b58fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.706+0000 7f6a7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:30.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.706+0000 7f6a7fe00640 1 -- 172.21.15.46:0/3065322807 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c051060 con 0x55962733d590 2024-09-10T07:44:30.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.707+0000 7f6a7fe00640 1 -- 172.21.15.46:0/3065322807 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6a8c03a820 con 0x55962733d590 2024-09-10T07:44:30.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.707+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9000edf0 con 0x55962733d590 2024-09-10T07:44:30.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.707+0000 7f6a7fe00640 1 -- 172.21.15.46:0/3065322807 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c058050 con 0x55962733d590 2024-09-10T07:44:30.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.707+0000 7f6aa79fd740 10 client.27319 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:30.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.708+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5596273c3270 con 0x55962733d590 2024-09-10T07:44:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.708+0000 7f6a7fe00640 1 -- 172.21.15.46:0/3065322807 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9000edf0 con 0x55962733d590 2024-09-10T07:44:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.708+0000 7f6a7fe00640 1 -- 172.21.15.46:0/3065322807 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f6a8c015ab0 con 0x55962733d590 2024-09-10T07:44:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.709+0000 7f6aa79fd740 10 client.27319 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.709+0000 7f6aa79fd740 10 client.27319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.709+0000 7f6aa79fd740 20 client.27319 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.709+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596273c35a0 0x5596273c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.710+0000 7f6aa79fd740 4 client.27319 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:44:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.710+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596272aae60 con 0x5596273c35a0 2024-09-10T07:44:30.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.711+0000 7f6a9de00640 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596273c35a0 0x5596273c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.712+0000 7f6a7fe00640 10 client.27319 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.712+0000 7f6a9de00640 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596273c35a0 0x5596273c5990 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b40dd7512154d9d server_cookie=ae3fb805f99903a9 in_seq=0 out_seq=0 2024-09-10T07:44:30.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.713+0000 7f6a7fe00640 1 -- 172.21.15.46:0/3065322807 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596272aae60 con 0x5596273c35a0 2024-09-10T07:44:30.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.713+0000 7f6a7fe00640 10 client.27319 handle_command_reply: tid=0 2024-09-10T07:44:30.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.714+0000 7f6aa79fd740 1 client.27319 shutdown 2024-09-10T07:44:30.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.714+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733d590 msgr2=0x559627350650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.714+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733d590 0x559627350650 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x559627344620 tx=0x7f6a8c03a460 comp rx=0 tx=0).stop 2024-09-10T07:44:30.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.714+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 shutdown_connections 2024-09-10T07:44:30.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.714+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5596273c35a0 0x5596273c5990 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.715+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559627347e50 0x55962734b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.715+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55962733d590 0x559627350650 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.715+0000 7f6a7fe00640 0 client.27319 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.715+0000 7f6aa79fd740 1 --2- 172.21.15.46:0/3065322807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55962733cc70 0x559627350110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.715+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 >> 172.21.15.46:0/3065322807 conn(0x559627339200 msgr2=0x5596273465f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.715+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 shutdown_connections 2024-09-10T07:44:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.715+0000 7f6aa79fd740 1 -- 172.21.15.46:0/3065322807 wait complete. 2024-09-10T07:44:30.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.715+0000 7f6aa79fd740 20 client.27319 trim_cache size 0 max 16384 2024-09-10T07:44:30.716 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:44:30.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 -- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a840a2bc0 msgr2=0x7f6a840a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a840a2bc0 0x7f6a840a5080 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55962733a060 tx=0x7f6a900095c0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 -- 172.21.15.46:0/4269866923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 msgr2=0x7f6aa019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:30.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 0x7f6aa019e920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6a9802a930 tx=0x7f6a98033d10 comp rx=0 tx=0).stop 2024-09-10T07:44:30.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 -- 172.21.15.46:0/4269866923 shutdown_connections 2024-09-10T07:44:30.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0112a30 0x7f6aa019e920 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6a840a2bc0 0x7f6a840a5080 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa0077040 0x7f6aa01a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 --2- 172.21.15.46:0/4269866923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa0076720 0x7f6aa01a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:30.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.718+0000 7f6aa5000640 1 -- 172.21.15.46:0/4269866923 >> 172.21.15.46:0/4269866923 conn(0x7f6aa00ffa90 msgr2=0x7f6aa0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:30.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.719+0000 7f6aa5000640 1 -- 172.21.15.46:0/4269866923 shutdown_connections 2024-09-10T07:44:30.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:30.719+0000 7f6aa5000640 1 -- 172.21.15.46:0/4269866923 wait complete. 2024-09-10T07:44:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:32 smithi181 ceph-mon[35397]: pgmap v636: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 593 op/s 2024-09-10T07:44:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:32 smithi063 ceph-mon[35020]: pgmap v636: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 593 op/s 2024-09-10T07:44:32.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:32 smithi046 ceph-mon[30705]: pgmap v636: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 593 op/s 2024-09-10T07:44:32.734 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:44:32.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:44:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:34 smithi181 ceph-mon[35397]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-09-10T07:44:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:34 smithi063 ceph-mon[35020]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-09-10T07:44:34.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:34 smithi046 ceph-mon[30705]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-09-10T07:44:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.377+0000 7f28cb800640 1 -- 172.21.15.46:0/676556572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c410ef30 msgr2=0x7f28c410d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.378+0000 7f28cb800640 1 --2- 172.21.15.46:0/676556572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c410ef30 0x7f28c410d2a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f28b00073b0 tx=0x7f28b002aff0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.379+0000 7f28cb800640 1 -- 172.21.15.46:0/676556572 shutdown_connections 2024-09-10T07:44:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.379+0000 7f28cb800640 1 --2- 172.21.15.46:0/676556572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c410d9b0 0x7f28c4115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.379+0000 7f28cb800640 1 --2- 172.21.15.46:0/676556572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c410ef30 0x7f28c410d2a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.379+0000 7f28cb800640 1 --2- 172.21.15.46:0/676556572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c410e580 0x7f28c410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.379+0000 7f28cb800640 1 -- 172.21.15.46:0/676556572 >> 172.21.15.46:0/676556572 conn(0x7f28c406e140 msgr2=0x7f28c406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.380+0000 7f28cb800640 1 -- 172.21.15.46:0/676556572 shutdown_connections 2024-09-10T07:44:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.380+0000 7f28cb800640 1 -- 172.21.15.46:0/676556572 wait complete. 2024-09-10T07:44:35.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.383+0000 7f28cb800640 1 Processor -- start 2024-09-10T07:44:35.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.383+0000 7f28cb800640 1 -- start start 2024-09-10T07:44:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.384+0000 7f28cb800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c410d9b0 0x7f28c41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.384+0000 7f28cb800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c410e580 0x7f28c41a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.384+0000 7f28cb800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c410ef30 0x7f28c419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.384+0000 7f28cb800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4118560 con 0x7f28c410d9b0 2024-09-10T07:44:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.384+0000 7f28cb800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41183e0 con 0x7f28c410ef30 2024-09-10T07:44:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.384+0000 7f28cb800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41186e0 con 0x7f28c410e580 2024-09-10T07:44:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c410e580 0x7f28c41a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c410e580 0x7f28c41a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60234/0 (socket says 172.21.15.46:60234) 2024-09-10T07:44:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c3e00640 1 -- 172.21.15.46:0/1699090327 learned_addr learned my addr 172.21.15.46:0/1699090327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:35.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c9800640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c410ef30 0x7f28c419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c3e00640 1 -- 172.21.15.46:0/1699090327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c410ef30 msgr2=0x7f28c419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c3e00640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c410ef30 0x7f28c419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c3e00640 1 -- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c410d9b0 msgr2=0x7f28c41a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c3e00640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c410d9b0 0x7f28c41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.385+0000 7f28c3e00640 1 -- 172.21.15.46:0/1699090327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c419efd0 con 0x7f28c410e580 2024-09-10T07:44:35.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.386+0000 7f28c3e00640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c410e580 0x7f28c41a4bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f28b0035040 tx=0x7f28b0033910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:35.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.387+0000 7f28c1600640 1 -- 172.21.15.46:0/1699090327 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0007a50 con 0x7f28c410e580 2024-09-10T07:44:35.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.387+0000 7f28c1600640 1 -- 172.21.15.46:0/1699090327 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b002bdd0 con 0x7f28c410e580 2024-09-10T07:44:35.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.387+0000 7f28cb800640 1 -- 172.21.15.46:0/1699090327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c419f260 con 0x7f28c410e580 2024-09-10T07:44:35.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.388+0000 7f28c1600640 1 -- 172.21.15.46:0/1699090327 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b000a930 con 0x7f28c410e580 2024-09-10T07:44:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.390+0000 7f28cb800640 1 -- 172.21.15.46:0/1699090327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c41ab510 con 0x7f28c410e580 2024-09-10T07:44:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.401+0000 7f28c1600640 1 -- 172.21.15.46:0/1699090327 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f28b0043030 con 0x7f28c410e580 2024-09-10T07:44:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.404+0000 7f28c1600640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a80a2c40 0x7f28a80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.404+0000 7f28c1600640 1 -- 172.21.15.46:0/1699090327 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28b00f5170 con 0x7f28c410e580 2024-09-10T07:44:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.404+0000 7f28ce202740 1 Processor -- start 2024-09-10T07:44:35.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.405+0000 7f28ce202740 1 -- start start 2024-09-10T07:44:35.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.405+0000 7f28ce202740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e8f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.405+0000 7f28ce202740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e99f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.405+0000 7f28ce202740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405e9c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.405+0000 7f28ce202740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f405ded290 con 0x55f405e8f2d0 2024-09-10T07:44:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.405+0000 7f28ce202740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f405d94b40 con 0x55f405e9a4b0 2024-09-10T07:44:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.405+0000 7f28ce202740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f405de9250 con 0x55f405e8fbf0 2024-09-10T07:44:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.407+0000 7f28c9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405e9c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.407+0000 7f28c9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405e9c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45478/0 (socket says 172.21.15.46:45478) 2024-09-10T07:44:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.407+0000 7f28c9800640 1 -- 172.21.15.46:0/3250841287 learned_addr learned my addr 172.21.15.46:0/3250841287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.407+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e99f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.407+0000 7f28c8e00640 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e8f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.408+0000 7f28c8e00640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a80a2c40 0x7f28a80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.408+0000 7f28c9800640 1 -- 172.21.15.46:0/3250841287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 msgr2=0x55f405e99f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.408+0000 7f28c9800640 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e99f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.408+0000 7f28c9800640 1 -- 172.21.15.46:0/3250841287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 msgr2=0x55f405e8f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.408+0000 7f28c9800640 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e8f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.408+0000 7f28c9800640 1 -- 172.21.15.46:0/3250841287 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f405b5e6a0 con 0x55f405e9a4b0 2024-09-10T07:44:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.408+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e99f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.409+0000 7f28c9800640 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405e9c8a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009fd0 tx=0x7f28bc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86bd8c49f196c125 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.409+0000 7f28c8e00640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a80a2c40 0x7f28a80a5100 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005ab0 tx=0x7f28b400d5a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.409+0000 7f28ba000640 1 -- 172.21.15.46:0/3250841287 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc004d20 con 0x55f405e9a4b0 2024-09-10T07:44:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.409+0000 7f28ba000640 1 -- 172.21.15.46:0/3250841287 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28bc004050 con 0x55f405e9a4b0 2024-09-10T07:44:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.409+0000 7f28ba000640 1 -- 172.21.15.46:0/3250841287 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc005280 con 0x55f405e9a4b0 2024-09-10T07:44:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.410+0000 7f28ce202740 1 -- 172.21.15.46:0/3250841287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 msgr2=0x55f405e9c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.410+0000 7f28ce202740 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405e9c8a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009fd0 tx=0x7f28bc0118d0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.411+0000 7f28ce202740 1 -- 172.21.15.46:0/3250841287 shutdown_connections 2024-09-10T07:44:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.411+0000 7f28ce202740 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405e9c8a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.411+0000 7f28ce202740 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e99f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.411+0000 7f28ce202740 1 --2- 172.21.15.46:0/3250841287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e8f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.411+0000 7f28ce202740 1 -- 172.21.15.46:0/3250841287 >> 172.21.15.46:0/3250841287 conn(0x55f405e8b860 msgr2=0x55f405e8bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.411+0000 7f28ce202740 1 -- 172.21.15.46:0/3250841287 shutdown_connections 2024-09-10T07:44:35.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.412+0000 7f28ce202740 1 -- 172.21.15.46:0/3250841287 wait complete. 2024-09-10T07:44:35.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.412+0000 7f28ce202740 1 -- start start 2024-09-10T07:44:35.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.413+0000 7f28ce202740 1 Processor -- start 2024-09-10T07:44:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.414+0000 7f28ce202740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.415+0000 7f28ce202740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8fbf0 0x55f405e88930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.415+0000 7f28ce202740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e9a4b0 0x55f405e88e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.415+0000 7f28ce202740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f405e82d00 con 0x55f405e8f2d0 2024-09-10T07:44:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.415+0000 7f28ce202740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f405da7fa0 con 0x55f405e8fbf0 2024-09-10T07:44:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28ce202740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f405e82e80 con 0x55f405e9a4b0 2024-09-10T07:44:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28c9800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e9a4b0 0x55f405e88e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28c9800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e9a4b0 0x55f405e88e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60258/0 (socket says 172.21.15.46:60258) 2024-09-10T07:44:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28c9800640 1 -- 172.21.15.46:0/3772898855 learned_addr learned my addr 172.21.15.46:0/3772898855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28ce202740 10 client.0 _ll_register_callbacks cb 0x55f405afbba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28c9800640 1 -- 172.21.15.46:0/3772898855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8fbf0 msgr2=0x55f405e88930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:44:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28c9800640 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8fbf0 0x55f405e88930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28c9800640 1 -- 172.21.15.46:0/3772898855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 msgr2=0x55f405e883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28c9800640 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.416+0000 7f28c9800640 1 -- 172.21.15.46:0/3772898855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f405e991d0 con 0x55f405e9a4b0 2024-09-10T07:44:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.417+0000 7f28c9800640 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e9a4b0 0x55f405e88e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f28bc000f80 tx=0x7f28bc004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34b491e3a30315c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.417+0000 7f28aea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.417+0000 7f28ce202740 1 -- 172.21.15.46:0/3772898855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28bc004050 con 0x55f405e9a4b0 2024-09-10T07:44:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.417+0000 7f28aea00640 1 -- 172.21.15.46:0/3772898855 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc028020 con 0x55f405e9a4b0 2024-09-10T07:44:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.417+0000 7f28aea00640 1 -- 172.21.15.46:0/3772898855 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28bc003c40 con 0x55f405e9a4b0 2024-09-10T07:44:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.418+0000 7f28aea00640 1 -- 172.21.15.46:0/3772898855 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f28bc00f4e0 con 0x55f405e9a4b0 2024-09-10T07:44:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.418+0000 7f28ce202740 10 client.27331 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.418+0000 7f28ce202740 1 -- 172.21.15.46:0/3772898855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=44}) -- 0x55f405eb1790 con 0x55f405e9a4b0 2024-09-10T07:44:35.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.419+0000 7f28aea00640 1 -- 172.21.15.46:0/3772898855 <== mon.1 v2:172.21.15.181:3300/0 4 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f28bc00f680 con 0x55f405e9a4b0 2024-09-10T07:44:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.421+0000 7f28ce202740 10 client.27331 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.421+0000 7f28ce202740 10 client.27331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.421+0000 7f28ce202740 20 client.27331 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.421+0000 7f28ce202740 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f405eb2ae0 0x55f405eb4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.421+0000 7f28ce202740 4 client.27331 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.421+0000 7f28ce202740 1 -- 172.21.15.46:0/3772898855 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f405dfa800 con 0x55f405eb2ae0 2024-09-10T07:44:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.423+0000 7f28c8e00640 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f405eb2ae0 0x55f405eb4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.426+0000 7f28c8e00640 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f405eb2ae0 0x55f405eb4ed0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333e80a1cbe4ccb1 server_cookie=16798ce7b23dcbc5 in_seq=0 out_seq=0 2024-09-10T07:44:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.427+0000 7f28aea00640 10 client.27331 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.430+0000 7f28aea00640 1 -- 172.21.15.46:0/3772898855 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f405dfa800 con 0x55f405eb2ae0 2024-09-10T07:44:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.430+0000 7f28aea00640 10 client.27331 handle_command_reply: tid=0 2024-09-10T07:44:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.430+0000 7f28ce202740 1 client.27331 shutdown 2024-09-10T07:44:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28ce202740 1 -- 172.21.15.46:0/3772898855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e9a4b0 msgr2=0x55f405e88e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28ce202740 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e9a4b0 0x55f405e88e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f28bc000f80 tx=0x7f28bc004c70 comp rx=0 tx=0).stop 2024-09-10T07:44:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28ce202740 1 -- 172.21.15.46:0/3772898855 shutdown_connections 2024-09-10T07:44:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28ce202740 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f405eb2ae0 0x55f405eb4ed0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28ce202740 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e9a4b0 0x55f405e88e70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28ce202740 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8fbf0 0x55f405e88930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28ce202740 1 --2- 172.21.15.46:0/3772898855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28ce202740 1 -- 172.21.15.46:0/3772898855 >> 172.21.15.46:0/3772898855 conn(0x55f405e8b860 msgr2=0x55f405e821c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.431+0000 7f28aea00640 0 client.27331 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.432+0000 7f28ce202740 1 -- 172.21.15.46:0/3772898855 shutdown_connections 2024-09-10T07:44:35.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.432+0000 7f28ce202740 1 -- 172.21.15.46:0/3772898855 wait complete. 2024-09-10T07:44:35.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.432+0000 7f28ce202740 20 client.27331 trim_cache size 0 max 16384 2024-09-10T07:44:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.468+0000 7f28ce202740 1 Processor -- start 2024-09-10T07:44:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.468+0000 7f28ce202740 1 -- start start 2024-09-10T07:44:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.469+0000 7f28ce202740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8f2d0 0x55f405e837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.469+0000 7f28ce202740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405eba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.469+0000 7f28ce202740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e9a4b0 0x55f405ebab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.469+0000 7f28ce202740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b5300 con 0x55f405e9a4b0 2024-09-10T07:44:35.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.469+0000 7f28ce202740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b6f30 con 0x55f405e8f2d0 2024-09-10T07:44:35.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.469+0000 7f28ce202740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b59d0 con 0x55f405e8fbf0 2024-09-10T07:44:35.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.470+0000 7f28c9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e9a4b0 0x55f405ebab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.470+0000 7f28c9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e9a4b0 0x55f405ebab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44770/0 (socket says 172.21.15.46:44770) 2024-09-10T07:44:35.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.471+0000 7f28c9800640 1 -- 172.21.15.46:0/3596457401 learned_addr learned my addr 172.21.15.46:0/3596457401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:35.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.471+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405eba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.471+0000 7f28c8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8f2d0 0x55f405e837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.472+0000 7f28c3e00640 1 -- 172.21.15.46:0/3596457401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8f2d0 msgr2=0x55f405e837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.472+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3596457401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8f2d0 0x55f405e837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.472+0000 7f28c3e00640 1 -- 172.21.15.46:0/3596457401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e9a4b0 msgr2=0x55f405ebab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.472+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3596457401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e9a4b0 0x55f405ebab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.472+0000 7f28c3e00640 1 -- 172.21.15.46:0/3596457401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f405e9a2a0 con 0x55f405e8fbf0 2024-09-10T07:44:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.472+0000 7f28c9800640 1 --2- 172.21.15.46:0/3596457401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e9a4b0 0x55f405ebab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.472+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3596457401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405eba640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f28c419efa0 tx=0x7f28b000a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=408237a2c81d853d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.473+0000 7f28b9600640 1 -- 172.21.15.46:0/3596457401 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00f6060 con 0x55f405e8fbf0 2024-09-10T07:44:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.473+0000 7f28b9600640 1 -- 172.21.15.46:0/3596457401 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b0007e10 con 0x55f405e8fbf0 2024-09-10T07:44:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.473+0000 7f28b9600640 1 -- 172.21.15.46:0/3596457401 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00ba9d0 con 0x55f405e8fbf0 2024-09-10T07:44:35.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.474+0000 7f28ce202740 1 -- 172.21.15.46:0/3596457401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 msgr2=0x55f405eba640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.474+0000 7f28ce202740 1 --2- 172.21.15.46:0/3596457401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405eba640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f28c419efa0 tx=0x7f28b000a040 comp rx=0 tx=0).stop 2024-09-10T07:44:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.474+0000 7f28ce202740 1 -- 172.21.15.46:0/3596457401 shutdown_connections 2024-09-10T07:44:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.474+0000 7f28ce202740 1 --2- 172.21.15.46:0/3596457401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e9a4b0 0x55f405ebab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.474+0000 7f28ce202740 1 --2- 172.21.15.46:0/3596457401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405eba640 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.474+0000 7f28ce202740 1 --2- 172.21.15.46:0/3596457401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e8f2d0 0x55f405e837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.474+0000 7f28ce202740 1 -- 172.21.15.46:0/3596457401 >> 172.21.15.46:0/3596457401 conn(0x55f405e8b860 msgr2=0x55f405e830e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.475+0000 7f28ce202740 1 -- 172.21.15.46:0/3596457401 shutdown_connections 2024-09-10T07:44:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.478+0000 7f28ce202740 1 -- 172.21.15.46:0/3596457401 wait complete. 2024-09-10T07:44:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.478+0000 7f28ce202740 1 -- start start 2024-09-10T07:44:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.479+0000 7f28ce202740 1 Processor -- start 2024-09-10T07:44:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.479+0000 7f28ce202740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.481+0000 7f28ce202740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e94c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.481+0000 7f28ce202740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405f16650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.481+0000 7f28c8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.481+0000 7f28c8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e946d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44776/0 (socket says 172.21.15.46:44776) 2024-09-10T07:44:35.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.481+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e94c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.481+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e94c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60282/0 (socket says 172.21.15.46:60282) 2024-09-10T07:44:35.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.481+0000 7f28c9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405f16650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.481+0000 7f28c9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405f16650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45498/0 (socket says 172.21.15.46:45498) 2024-09-10T07:44:35.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.482+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b67a0 con 0x55f405e8f2d0 2024-09-10T07:44:35.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.482+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b60a0 con 0x55f405e9a4b0 2024-09-10T07:44:35.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.482+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b32e0 con 0x55f405e8fbf0 2024-09-10T07:44:35.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.482+0000 7f28ce202740 10 client.0 _ll_register_callbacks cb 0x7f28b400da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:35.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.482+0000 7f28c8e00640 1 -- 172.21.15.46:0/2009584291 learned_addr learned my addr 172.21.15.46:0/2009584291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:35.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.483+0000 7f28c3e00640 1 -- 172.21.15.46:0/2009584291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 msgr2=0x55f405f16650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:35.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.483+0000 7f28c3e00640 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405f16650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.483+0000 7f28c3e00640 1 -- 172.21.15.46:0/2009584291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 msgr2=0x55f405e946d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:35.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.483+0000 7f28c3e00640 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e946d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.483+0000 7f28c3e00640 1 -- 172.21.15.46:0/2009584291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f405f16c90 con 0x55f405e8fbf0 2024-09-10T07:44:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.483+0000 7f28c3e00640 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e94c10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f28b00c2820 tx=0x7f28b00bc870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63187e0a2d953ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.483+0000 7f28aea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.484+0000 7f28aea00640 1 -- 172.21.15.46:0/2009584291 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00d72b0 con 0x55f405e8fbf0 2024-09-10T07:44:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.484+0000 7f28aea00640 1 -- 172.21.15.46:0/2009584291 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28b00c6a90 con 0x55f405e8fbf0 2024-09-10T07:44:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.485+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f405df4d10 con 0x55f405e8fbf0 2024-09-10T07:44:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.485+0000 7f28aea00640 1 -- 172.21.15.46:0/2009584291 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f28b00c2660 con 0x55f405e8fbf0 2024-09-10T07:44:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.485+0000 7f28ce202740 10 client.27337 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.485+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f405f1e040 con 0x55f405e8fbf0 2024-09-10T07:44:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.486+0000 7f28aea00640 1 -- 172.21.15.46:0/2009584291 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f405df4d10 con 0x55f405e8fbf0 2024-09-10T07:44:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.486+0000 7f28aea00640 1 -- 172.21.15.46:0/2009584291 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7f28b00bd7b0 con 0x55f405e8fbf0 2024-09-10T07:44:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.486+0000 7f28ce202740 10 client.27337 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:35.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.486+0000 7f28ce202740 10 client.27337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:35.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.486+0000 7f28ce202740 20 client.27337 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:35.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.486+0000 7f28ce202740 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f405f1e7e0 0x55f405f20c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:35.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.486+0000 7f28ce202740 4 client.27337 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:44:35.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.486+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f405d70fb0 con 0x55f405f1e7e0 2024-09-10T07:44:35.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.491+0000 7f28c8e00640 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f405f1e7e0 0x55f405f20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.495+0000 7f28c8e00640 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f405f1e7e0 0x55f405f20c00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f05d47ed36dc15e server_cookie=32e390e2a8ec8078 in_seq=0 out_seq=0 2024-09-10T07:44:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.495+0000 7f28aea00640 10 client.27337 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:35.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.502+0000 7f28aea00640 1 -- 172.21.15.46:0/2009584291 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f405d70fb0 con 0x55f405f1e7e0 2024-09-10T07:44:35.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.502+0000 7f28aea00640 10 client.27337 handle_command_reply: tid=0 2024-09-10T07:44:35.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.503+0000 7f28ce202740 1 client.27337 shutdown 2024-09-10T07:44:35.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 msgr2=0x55f405e94c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28ce202740 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e94c10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f28b00c2820 tx=0x7f28b00bc870 comp rx=0 tx=0).stop 2024-09-10T07:44:35.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 shutdown_connections 2024-09-10T07:44:35.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28ce202740 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f405f1e7e0 0x55f405f20c00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28ce202740 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f405e9a4b0 0x55f405f16650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28ce202740 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f405e8fbf0 0x55f405e94c10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28ce202740 1 --2- 172.21.15.46:0/2009584291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f405e8f2d0 0x55f405e946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 >> 172.21.15.46:0/2009584291 conn(0x55f405e8b860 msgr2=0x55f405e91540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:35.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.505+0000 7f28aea00640 0 client.27337 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:35.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.507+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 shutdown_connections 2024-09-10T07:44:35.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.507+0000 7f28ce202740 1 -- 172.21.15.46:0/2009584291 wait complete. 2024-09-10T07:44:35.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.507+0000 7f28ce202740 20 client.27337 trim_cache size 0 max 16384 2024-09-10T07:44:35.508 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:44:35.508 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:44:35.508 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "cda57228-d1ec-4799-828c-0798baccefdc", 2024-09-10T07:44:35.508 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:44:35.508 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:44:35.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.508+0000 7f28cb800640 1 -- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a80a2c40 msgr2=0x7f28a80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.508+0000 7f28cb800640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a80a2c40 0x7f28a80a5100 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005ab0 tx=0x7f28b400d5a0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.508+0000 7f28cb800640 1 -- 172.21.15.46:0/1699090327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c410e580 msgr2=0x7f28c41a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.508+0000 7f28cb800640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c410e580 0x7f28c41a4bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f28b0035040 tx=0x7f28b0033910 comp rx=0 tx=0).stop 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.509+0000 7f28cb800640 1 -- 172.21.15.46:0/1699090327 shutdown_connections 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.509+0000 7f28cb800640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c410ef30 0x7f28c419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.509+0000 7f28cb800640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c410e580 0x7f28c41a4bf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.509+0000 7f28cb800640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a80a2c40 0x7f28a80a5100 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005ab0 tx=0x7f28b400d5a0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.509+0000 7f28cb800640 1 --2- 172.21.15.46:0/1699090327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c410d9b0 0x7f28c41a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.509+0000 7f28cb800640 1 -- 172.21.15.46:0/1699090327 >> 172.21.15.46:0/1699090327 conn(0x7f28c406e140 msgr2=0x7f28c410b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:35.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.509+0000 7f28cb800640 1 -- 172.21.15.46:0/1699090327 shutdown_connections 2024-09-10T07:44:35.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:35.510+0000 7f28cb800640 1 -- 172.21.15.46:0/1699090327 wait complete. 2024-09-10T07:44:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 435.0 s, 538.6 tps, lat 184.761 ms stddev 204.663 2024-09-10T07:44:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:36 smithi181 ceph-mon[35397]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.4 MiB/s wr, 522 op/s 2024-09-10T07:44:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:36 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:44:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:36 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:36 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:44:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:36 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:44:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:36 smithi063 ceph-mon[35020]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.4 MiB/s wr, 522 op/s 2024-09-10T07:44:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:36 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:44:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:36 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:36 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:44:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:36 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:44:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:36 smithi046 ceph-mon[30705]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.4 MiB/s wr, 522 op/s 2024-09-10T07:44:36.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:36 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:44:36.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:36.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:44:36.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:44:36.611 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:44:37.280 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:44:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:44:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:44:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:44:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:44:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:44:37.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:44:37.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:37.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:44:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.071+0000 7fcbaae00640 1 -- 172.21.15.46:0/1626717154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba4100780 msgr2=0x7fcba4109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.071+0000 7fcbaae00640 1 --2- 172.21.15.46:0/1626717154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba4100780 0x7fcba4109be0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009890 tx=0x7fcb9402a440 comp rx=0 tx=0).stop 2024-09-10T07:44:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.071+0000 7fcbaae00640 1 -- 172.21.15.46:0/1626717154 shutdown_connections 2024-09-10T07:44:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.071+0000 7fcbaae00640 1 --2- 172.21.15.46:0/1626717154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba410a2f0 0x7fcba4110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.071+0000 7fcbaae00640 1 --2- 172.21.15.46:0/1626717154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba4100780 0x7fcba4109be0 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009890 tx=0x7fcb9402a440 comp rx=0 tx=0).stop 2024-09-10T07:44:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.071+0000 7fcbaae00640 1 --2- 172.21.15.46:0/1626717154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba40ffe60 0x7fcba4100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.071+0000 7fcbaae00640 1 -- 172.21.15.46:0/1626717154 >> 172.21.15.46:0/1626717154 conn(0x7fcba40fba90 msgr2=0x7fcba40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.072+0000 7fcbaae00640 1 -- 172.21.15.46:0/1626717154 shutdown_connections 2024-09-10T07:44:38.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.072+0000 7fcbaae00640 1 -- 172.21.15.46:0/1626717154 wait complete. 2024-09-10T07:44:38.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.073+0000 7fcbaae00640 1 Processor -- start 2024-09-10T07:44:38.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.076+0000 7fcbaae00640 1 -- start start 2024-09-10T07:44:38.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.076+0000 7fcbaae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba40ffe60 0x7fcba4110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.076+0000 7fcbaae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba410a2f0 0x7fcba4073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.077+0000 7fcbaae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba410ec20 0x7fcba410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.077+0000 7fcba3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba40ffe60 0x7fcba4110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.077+0000 7fcba3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba40ffe60 0x7fcba4110b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56704/0 (socket says 172.21.15.46:56704) 2024-09-10T07:44:38.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.077+0000 7fcba3e00640 1 -- 172.21.15.46:0/2128076451 learned_addr learned my addr 172.21.15.46:0/2128076451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:38.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.078+0000 7fcba8e00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba410ec20 0x7fcba410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.078+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4103dc0 con 0x7fcba410ec20 2024-09-10T07:44:38.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.078+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4103c40 con 0x7fcba40ffe60 2024-09-10T07:44:38.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.078+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4103f40 con 0x7fcba410a2f0 2024-09-10T07:44:38.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.079+0000 7fcba3400640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba410a2f0 0x7fcba4073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.079+0000 7fcba3400640 1 -- 172.21.15.46:0/2128076451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba40ffe60 msgr2=0x7fcba4110b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.079+0000 7fcba3400640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba40ffe60 0x7fcba4110b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.079+0000 7fcba3400640 1 -- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba410ec20 msgr2=0x7fcba410f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.079+0000 7fcba3400640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba410ec20 0x7fcba410f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.079+0000 7fcba3400640 1 -- 172.21.15.46:0/2128076451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba410f6f0 con 0x7fcba410a2f0 2024-09-10T07:44:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.080+0000 7fcba3400640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba410a2f0 0x7fcba4073c70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb94008000 tx=0x7fcb94030940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.082+0000 7fcba0c00640 1 -- 172.21.15.46:0/2128076451 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94040030 con 0x7fcba410a2f0 2024-09-10T07:44:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.082+0000 7fcba0c00640 1 -- 172.21.15.46:0/2128076451 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb94005550 con 0x7fcba410a2f0 2024-09-10T07:44:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.082+0000 7fcba0c00640 1 -- 172.21.15.46:0/2128076451 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940319c0 con 0x7fcba410a2f0 2024-09-10T07:44:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.082+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba410f980 con 0x7fcba410a2f0 2024-09-10T07:44:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.082+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba41b4250 con 0x7fcba410a2f0 2024-09-10T07:44:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.088+0000 7fcba0c00640 1 -- 172.21.15.46:0/2128076451 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb94043070 con 0x7fcba410a2f0 2024-09-10T07:44:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.089+0000 7fcbad856740 1 Processor -- start 2024-09-10T07:44:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.090+0000 7fcba0c00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb880a2d90 0x7fcb880a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.090+0000 7fcba0c00640 1 -- 172.21.15.46:0/2128076451 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcb940f0b60 con 0x7fcba410a2f0 2024-09-10T07:44:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.090+0000 7fcbad856740 1 -- start start 2024-09-10T07:44:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcbad856740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcbad856740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f45830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcbad856740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 0x560fb6f48160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcbad856740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560fb6ea13e0 con 0x560fb6f3b4b0 2024-09-10T07:44:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcbad856740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560fb6e48d80 con 0x560fb6f3ab90 2024-09-10T07:44:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcbad856740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560fb6e9d3a0 con 0x560fb6f45d70 2024-09-10T07:44:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcba3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcba3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f45830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcba3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f3af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56716/0 (socket says 172.21.15.46:56716) 2024-09-10T07:44:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.091+0000 7fcba3e00640 1 -- 172.21.15.46:0/4195985340 learned_addr learned my addr 172.21.15.46:0/4195985340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.092+0000 7fcba8e00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb880a2d90 0x7fcb880a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.093+0000 7fcba8e00640 1 --2- 172.21.15.46:0/4195985340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 0x560fb6f48160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.093+0000 7fcba3400640 1 -- 172.21.15.46:0/4195985340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 msgr2=0x560fb6f48160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.093+0000 7fcba3400640 1 --2- 172.21.15.46:0/4195985340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 0x560fb6f48160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.093+0000 7fcba3400640 1 -- 172.21.15.46:0/4195985340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 msgr2=0x560fb6f3af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.093+0000 7fcba3400640 1 --2- 172.21.15.46:0/4195985340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f3af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.093+0000 7fcba3400640 1 -- 172.21.15.46:0/4195985340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb6e67ae0 con 0x560fb6f3b4b0 2024-09-10T07:44:38.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.094+0000 7fcba3400640 1 --2- 172.21.15.46:0/4195985340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f45830 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcb940b99b0 tx=0x7fcb9402a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798e1f1fdd04242e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:38.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.094+0000 7fcb99600640 1 -- 172.21.15.46:0/4195985340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940d5020 con 0x560fb6f3b4b0 2024-09-10T07:44:38.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.094+0000 7fcb99600640 1 -- 172.21.15.46:0/4195985340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb940c1390 con 0x560fb6f3b4b0 2024-09-10T07:44:38.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.095+0000 7fcb99600640 1 -- 172.21.15.46:0/4195985340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940b9650 con 0x560fb6f3b4b0 2024-09-10T07:44:38.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.095+0000 7fcbad856740 1 -- 172.21.15.46:0/4195985340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 msgr2=0x560fb6f45830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.095+0000 7fcbad856740 1 --2- 172.21.15.46:0/4195985340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f45830 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcb940b99b0 tx=0x7fcb9402a440 comp rx=0 tx=0).stop 2024-09-10T07:44:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.096+0000 7fcbad856740 1 -- 172.21.15.46:0/4195985340 shutdown_connections 2024-09-10T07:44:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.096+0000 7fcbad856740 1 --2- 172.21.15.46:0/4195985340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 0x560fb6f48160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.096+0000 7fcbad856740 1 --2- 172.21.15.46:0/4195985340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f45830 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.096+0000 7fcbad856740 1 --2- 172.21.15.46:0/4195985340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f3af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.096+0000 7fcbad856740 1 -- 172.21.15.46:0/4195985340 >> 172.21.15.46:0/4195985340 conn(0x560fb6f37120 msgr2=0x560fb6f37500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.096+0000 7fcbad856740 1 -- 172.21.15.46:0/4195985340 shutdown_connections 2024-09-10T07:44:38.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.096+0000 7fcbad856740 1 -- 172.21.15.46:0/4195985340 wait complete. 2024-09-10T07:44:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.097+0000 7fcbad856740 1 -- start start 2024-09-10T07:44:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.097+0000 7fcbad856740 1 Processor -- start 2024-09-10T07:44:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.099+0000 7fcbad856740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3ab90 0x560fb6f42b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.099+0000 7fcbad856740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.099+0000 7fcbad856740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f43600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.099+0000 7fcbad856740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560fb6f48700 con 0x560fb6f3b4b0 2024-09-10T07:44:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.099+0000 7fcbad856740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560fb6e5c0f0 con 0x560fb6f45d70 2024-09-10T07:44:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.099+0000 7fcbad856740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560fb6f48880 con 0x560fb6f3ab90 2024-09-10T07:44:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.099+0000 7fcbad856740 10 client.0 _ll_register_callbacks cb 0x560fb6e4d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58930/0 (socket says 172.21.15.46:58930) 2024-09-10T07:44:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba8e00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb880a2d90 0x7fcb880a5250 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x560fb6f384a0 tx=0x7fcb8c009860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba3400640 1 -- 172.21.15.46:0/1527920586 learned_addr learned my addr 172.21.15.46:0/1527920586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba3e00640 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3ab90 0x560fb6f42b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba8e00640 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba8e00640 1 -- 172.21.15.46:0/1527920586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3ab90 msgr2=0x560fb6f42b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba8e00640 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3ab90 0x560fb6f42b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba8e00640 1 -- 172.21.15.46:0/1527920586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 msgr2=0x560fb6f430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba8e00640 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.100+0000 7fcba8e00640 1 -- 172.21.15.46:0/1527920586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb6c123c0 con 0x560fb6f45d70 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.101+0000 7fcba3e00640 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3ab90 0x560fb6f42b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.101+0000 7fcba8e00640 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f43600 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005fd0 tx=0x7fcb8c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c278711ed8b6a648 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.101+0000 7fcb92000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.101+0000 7fcb92000640 1 -- 172.21.15.46:0/1527920586 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c025070 con 0x560fb6f45d70 2024-09-10T07:44:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.102+0000 7fcb92000640 1 -- 172.21.15.46:0/1527920586 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcb8c012c40 con 0x560fb6f45d70 2024-09-10T07:44:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.103+0000 7fcbad856740 1 -- 172.21.15.46:0/1527920586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb940c1390 con 0x560fb6f45d70 2024-09-10T07:44:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.103+0000 7fcb92000640 1 -- 172.21.15.46:0/1527920586 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcb8c021470 con 0x560fb6f45d70 2024-09-10T07:44:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.103+0000 7fcbad856740 10 client.27395 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.103+0000 7fcbad856740 1 -- 172.21.15.46:0/1527920586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560fb6f54290 con 0x560fb6f45d70 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.105+0000 7fcb92000640 1 -- 172.21.15.46:0/1527920586 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb940c1390 con 0x560fb6f45d70 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.105+0000 7fcb92000640 1 -- 172.21.15.46:0/1527920586 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7fcb8c01f040 con 0x560fb6f45d70 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.106+0000 7fcbad856740 10 client.27395 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.106+0000 7fcbad856740 10 client.27395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.106+0000 7fcbad856740 20 client.27395 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.106+0000 7fcbad856740 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560fb6f55760 0x560fb6f57bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.106+0000 7fcbad856740 4 client.27395 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.106+0000 7fcbad856740 1 -- 172.21.15.46:0/1527920586 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fb6eae950 con 0x560fb6f55760 2024-09-10T07:44:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.107+0000 7fcba3e00640 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560fb6f55760 0x560fb6f57bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.108+0000 7fcb92000640 10 client.27395 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.108+0000 7fcba3e00640 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560fb6f55760 0x560fb6f57bc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ce914a2a128ef1 server_cookie=5152dd90e95698ee in_seq=0 out_seq=0 2024-09-10T07:44:38.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.110+0000 7fcb92000640 1 -- 172.21.15.46:0/1527920586 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fb6eae950 con 0x560fb6f55760 2024-09-10T07:44:38.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.110+0000 7fcb92000640 10 client.27395 handle_command_reply: tid=0 2024-09-10T07:44:38.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.111+0000 7fcbad856740 1 client.27395 shutdown 2024-09-10T07:44:38.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.111+0000 7fcbad856740 1 -- 172.21.15.46:0/1527920586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 msgr2=0x560fb6f43600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.111+0000 7fcbad856740 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f43600 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005fd0 tx=0x7fcb8c004730 comp rx=0 tx=0).stop 2024-09-10T07:44:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.112+0000 7fcbad856740 1 -- 172.21.15.46:0/1527920586 shutdown_connections 2024-09-10T07:44:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.112+0000 7fcbad856740 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560fb6f55760 0x560fb6f57bc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.112+0000 7fcbad856740 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f43600 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.112+0000 7fcbad856740 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.112+0000 7fcbad856740 1 --2- 172.21.15.46:0/1527920586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3ab90 0x560fb6f42b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.112+0000 7fcbad856740 1 -- 172.21.15.46:0/1527920586 >> 172.21.15.46:0/1527920586 conn(0x560fb6f37120 msgr2=0x560fb6f3ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.113+0000 7fcb92000640 0 client.27395 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:38.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.113+0000 7fcbad856740 1 -- 172.21.15.46:0/1527920586 shutdown_connections 2024-09-10T07:44:38.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.113+0000 7fcbad856740 1 -- 172.21.15.46:0/1527920586 wait complete. 2024-09-10T07:44:38.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.113+0000 7fcbad856740 20 client.27395 trim_cache size 0 max 16384 2024-09-10T07:44:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.148+0000 7fcbad856740 1 Processor -- start 2024-09-10T07:44:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.149+0000 7fcbad856740 1 -- start start 2024-09-10T07:44:38.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.149+0000 7fcbad856740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f5d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.149+0000 7fcbad856740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f5da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.150+0000 7fcbad856740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 0x560fb6f4dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.150+0000 7fcbad856740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8000fba0 con 0x560fb6f3b4b0 2024-09-10T07:44:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.150+0000 7fcbad856740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb800117d0 con 0x560fb6f3ab90 2024-09-10T07:44:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.150+0000 7fcbad856740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb80010270 con 0x560fb6f45d70 2024-09-10T07:44:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.150+0000 7fcba3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f5da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.150+0000 7fcba3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f5d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.150+0000 7fcba3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f5d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56732/0 (socket says 172.21.15.46:56732) 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.150+0000 7fcba3e00640 1 -- 172.21.15.46:0/2446675456 learned_addr learned my addr 172.21.15.46:0/2446675456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.151+0000 7fcba3e00640 1 -- 172.21.15.46:0/2446675456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 msgr2=0x560fb6f4dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.151+0000 7fcba3e00640 1 --2- 172.21.15.46:0/2446675456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 0x560fb6f4dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.151+0000 7fcba3e00640 1 -- 172.21.15.46:0/2446675456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 msgr2=0x560fb6f5da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.151+0000 7fcba3e00640 1 --2- 172.21.15.46:0/2446675456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f5da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.151+0000 7fcba3e00640 1 -- 172.21.15.46:0/2446675456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb6f58240 con 0x560fb6f3ab90 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.151+0000 7fcba3400640 1 --2- 172.21.15.46:0/2446675456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f5da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.152+0000 7fcba3e00640 1 --2- 172.21.15.46:0/2446675456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f5d4d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x560fb6f58470 tx=0x7fcb9c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daad12d5c6ae64a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.152+0000 7fcb98c00640 1 -- 172.21.15.46:0/2446675456 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c004460 con 0x560fb6f3ab90 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.152+0000 7fcb98c00640 1 -- 172.21.15.46:0/2446675456 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c004600 con 0x560fb6f3ab90 2024-09-10T07:44:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.152+0000 7fcb98c00640 1 -- 172.21.15.46:0/2446675456 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c01c5e0 con 0x560fb6f3ab90 2024-09-10T07:44:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.154+0000 7fcbad856740 1 -- 172.21.15.46:0/2446675456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 msgr2=0x560fb6f5d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.154+0000 7fcbad856740 1 --2- 172.21.15.46:0/2446675456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f5d4d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x560fb6f58470 tx=0x7fcb9c009500 comp rx=0 tx=0).stop 2024-09-10T07:44:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.154+0000 7fcbad856740 1 -- 172.21.15.46:0/2446675456 shutdown_connections 2024-09-10T07:44:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.155+0000 7fcbad856740 1 --2- 172.21.15.46:0/2446675456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f45d70 0x560fb6f4dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.155+0000 7fcbad856740 1 --2- 172.21.15.46:0/2446675456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3b4b0 0x560fb6f5da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.155+0000 7fcbad856740 1 --2- 172.21.15.46:0/2446675456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f3ab90 0x560fb6f5d4d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.155+0000 7fcbad856740 1 -- 172.21.15.46:0/2446675456 >> 172.21.15.46:0/2446675456 conn(0x560fb6f37120 msgr2=0x560fb6f3ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.155+0000 7fcbad856740 1 -- 172.21.15.46:0/2446675456 shutdown_connections 2024-09-10T07:44:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.155+0000 7fcbad856740 1 -- 172.21.15.46:0/2446675456 wait complete. 2024-09-10T07:44:38.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.156+0000 7fcbad856740 1 -- start start 2024-09-10T07:44:38.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.156+0000 7fcbad856740 1 Processor -- start 2024-09-10T07:44:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.159+0000 7fcbad856740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3ab90 0x560fb6f4b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.160+0000 7fcbad856740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3b4b0 0x560fb6f4be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.160+0000 7fcbad856740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f4c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.161+0000 7fcbad856740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb80011040 con 0x560fb6f3ab90 2024-09-10T07:44:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.161+0000 7fcbad856740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb80010940 con 0x560fb6f45d70 2024-09-10T07:44:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.161+0000 7fcbad856740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8000db40 con 0x560fb6f3b4b0 2024-09-10T07:44:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.161+0000 7fcbad856740 10 client.0 _ll_register_callbacks cb 0x7fcb8c0128f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.161+0000 7fcba8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f4c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.161+0000 7fcba8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f4c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56748/0 (socket says 172.21.15.46:56748) 2024-09-10T07:44:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.161+0000 7fcba8e00640 1 -- 172.21.15.46:0/4232140256 learned_addr learned my addr 172.21.15.46:0/4232140256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.161+0000 7fcba3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3b4b0 0x560fb6f4be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.162+0000 7fcba8e00640 1 -- 172.21.15.46:0/4232140256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3b4b0 msgr2=0x560fb6f4be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.162+0000 7fcba8e00640 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3b4b0 0x560fb6f4be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.162+0000 7fcba8e00640 1 -- 172.21.15.46:0/4232140256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3ab90 msgr2=0x560fb6f4b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.162+0000 7fcba8e00640 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3ab90 0x560fb6f4b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.162+0000 7fcba8e00640 1 -- 172.21.15.46:0/4232140256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb6f51960 con 0x560fb6f45d70 2024-09-10T07:44:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.163+0000 7fcba8e00640 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f4c3d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0208b0 tx=0x7fcb8c020cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71da61832907f5e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.163+0000 7fcb92000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.164+0000 7fcb92000640 1 -- 172.21.15.46:0/4232140256 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c021470 con 0x560fb6f45d70 2024-09-10T07:44:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.164+0000 7fcb92000640 1 -- 172.21.15.46:0/4232140256 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcb8c0205e0 con 0x560fb6f45d70 2024-09-10T07:44:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.164+0000 7fcbad856740 1 -- 172.21.15.46:0/4232140256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9c004600 con 0x560fb6f45d70 2024-09-10T07:44:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.164+0000 7fcb92000640 1 -- 172.21.15.46:0/4232140256 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcb8c011e80 con 0x560fb6f45d70 2024-09-10T07:44:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.164+0000 7fcbad856740 10 client.27401 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.164+0000 7fcbad856740 1 -- 172.21.15.46:0/4232140256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560fb6fc06b0 con 0x560fb6f45d70 2024-09-10T07:44:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.165+0000 7fcb92000640 1 -- 172.21.15.46:0/4232140256 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c004600 con 0x560fb6f45d70 2024-09-10T07:44:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.165+0000 7fcb92000640 1 -- 172.21.15.46:0/4232140256 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7fcb8c021980 con 0x560fb6f45d70 2024-09-10T07:44:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.166+0000 7fcbad856740 10 client.27401 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.166+0000 7fcbad856740 10 client.27401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.166+0000 7fcbad856740 20 client.27401 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.166+0000 7fcbad856740 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560fb6fc0e60 0x560fb6fc32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.166+0000 7fcbad856740 4 client.27401 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:44:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.166+0000 7fcbad856740 1 -- 172.21.15.46:0/4232140256 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560fb6ea8e60 con 0x560fb6fc0e60 2024-09-10T07:44:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.166+0000 7fcba3e00640 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560fb6fc0e60 0x560fb6fc32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.168+0000 7fcba3e00640 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560fb6fc0e60 0x560fb6fc32c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a528b87f943149f5 server_cookie=1da46e12b5dbea8d in_seq=0 out_seq=0 2024-09-10T07:44:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.168+0000 7fcb92000640 10 client.27401 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.170+0000 7fcb92000640 1 -- 172.21.15.46:0/4232140256 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560fb6ea8e60 con 0x560fb6fc0e60 2024-09-10T07:44:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.170+0000 7fcb92000640 10 client.27401 handle_command_reply: tid=0 2024-09-10T07:44:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.170+0000 7fcbad856740 1 client.27401 shutdown 2024-09-10T07:44:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.171+0000 7fcbad856740 1 -- 172.21.15.46:0/4232140256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 msgr2=0x560fb6f4c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.171+0000 7fcbad856740 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f4c3d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0208b0 tx=0x7fcb8c020cd0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.172+0000 7fcbad856740 1 -- 172.21.15.46:0/4232140256 shutdown_connections 2024-09-10T07:44:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.172+0000 7fcbad856740 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560fb6fc0e60 0x560fb6fc32c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.172+0000 7fcbad856740 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560fb6f45d70 0x560fb6f4c3d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.172+0000 7fcbad856740 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560fb6f3b4b0 0x560fb6f4be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.172+0000 7fcbad856740 1 --2- 172.21.15.46:0/4232140256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fb6f3ab90 0x560fb6f4b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.172+0000 7fcbad856740 1 -- 172.21.15.46:0/4232140256 >> 172.21.15.46:0/4232140256 conn(0x560fb6f37120 msgr2=0x560fb6f359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.172+0000 7fcb92000640 0 client.27401 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.174+0000 7fcbad856740 1 -- 172.21.15.46:0/4232140256 shutdown_connections 2024-09-10T07:44:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.174+0000 7fcbad856740 1 -- 172.21.15.46:0/4232140256 wait complete. 2024-09-10T07:44:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.175+0000 7fcbad856740 20 client.27401 trim_cache size 0 max 16384 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout: "cda57228-d1ec-4799-828c-0798baccefdc": { 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "cda57228-d1ec-4799-828c-0798baccefdc", 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:44:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:44:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.176+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb880a2d90 msgr2=0x7fcb880a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.176+0000 7fcbaae00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb880a2d90 0x7fcb880a5250 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x560fb6f384a0 tx=0x7fcb8c009860 comp rx=0 tx=0).stop 2024-09-10T07:44:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.176+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba410a2f0 msgr2=0x7fcba4073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.176+0000 7fcbaae00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba410a2f0 0x7fcba4073c70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb94008000 tx=0x7fcb94030940 comp rx=0 tx=0).stop 2024-09-10T07:44:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.177+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 shutdown_connections 2024-09-10T07:44:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.177+0000 7fcbaae00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb880a2d90 0x7fcb880a5250 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.177+0000 7fcbaae00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba410ec20 0x7fcba410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.177+0000 7fcbaae00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba410a2f0 0x7fcba4073c70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.177+0000 7fcbaae00640 1 --2- 172.21.15.46:0/2128076451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba40ffe60 0x7fcba4110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.177+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 >> 172.21.15.46:0/2128076451 conn(0x7fcba40fba90 msgr2=0x7fcba40fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.177+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 shutdown_connections 2024-09-10T07:44:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:38.177+0000 7fcbaae00640 1 -- 172.21.15.46:0/2128076451 wait complete. 2024-09-10T07:44:38.288 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:38 smithi046 ceph-mon[30705]: pgmap v639: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T07:44:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:38 smithi181 ceph-mon[35397]: pgmap v639: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T07:44:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:38 smithi063 ceph-mon[35020]: pgmap v639: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T07:44:39.214 INFO:tasks.cephfs.filesystem:scrub status for tag:cda57228-d1ec-4799-828c-0798baccefdc - {'path': '/', 'tag': 'cda57228-d1ec-4799-828c-0798baccefdc', 'options': 'recursive,force'} 2024-09-10T07:44:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:40 smithi181 ceph-mon[35397]: pgmap v640: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-09-10T07:44:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:40 smithi063 ceph-mon[35020]: pgmap v640: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-09-10T07:44:40.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:40 smithi046 ceph-mon[30705]: pgmap v640: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-09-10T07:44:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.471+0000 7fa916c00640 1 -- 172.21.15.46:0/61967371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa910103050 msgr2=0x7fa910103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.471+0000 7fa916c00640 1 --2- 172.21.15.46:0/61967371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa910103050 0x7fa910103450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa904009be0 tx=0x7fa90402a6d0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.471+0000 7fa916c00640 1 -- 172.21.15.46:0/61967371 shutdown_connections 2024-09-10T07:44:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.471+0000 7fa916c00640 1 --2- 172.21.15.46:0/61967371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa910101f40 0x7fa910109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.471+0000 7fa916c00640 1 --2- 172.21.15.46:0/61967371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa910101580 0x7fa910101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.471+0000 7fa916c00640 1 --2- 172.21.15.46:0/61967371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa910103050 0x7fa910103450 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.471+0000 7fa916c00640 1 -- 172.21.15.46:0/61967371 >> 172.21.15.46:0/61967371 conn(0x7fa9100fd0d0 msgr2=0x7fa9100ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.472+0000 7fa916c00640 1 -- 172.21.15.46:0/61967371 shutdown_connections 2024-09-10T07:44:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.472+0000 7fa916c00640 1 -- 172.21.15.46:0/61967371 wait complete. 2024-09-10T07:44:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.473+0000 7fa916c00640 1 Processor -- start 2024-09-10T07:44:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.474+0000 7fa916c00640 1 -- start start 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa916c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa910101580 0x7fa910199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa916c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa910101f40 0x7fa91019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa916c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa910103050 0x7fa9101a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa916c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010d6d0 con 0x7fa910101580 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa916c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010d550 con 0x7fa910101f40 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa916c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010d850 con 0x7fa910103050 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa914e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa910101f40 0x7fa91019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa914e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa910101f40 0x7fa91019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56768/0 (socket says 172.21.15.46:56768) 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa914e00640 1 -- 172.21.15.46:0/1662111415 learned_addr learned my addr 172.21.15.46:0/1662111415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa916200640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa910103050 0x7fa9101a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.475+0000 7fa915800640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa910101580 0x7fa910199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa914e00640 1 -- 172.21.15.46:0/1662111415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa910103050 msgr2=0x7fa9101a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa914e00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa910103050 0x7fa9101a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa914e00640 1 -- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa910101580 msgr2=0x7fa910199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa914e00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa910101580 0x7fa910199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa914e00640 1 -- 172.21.15.46:0/1662111415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9101a1bf0 con 0x7fa910101f40 2024-09-10T07:44:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa914e00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa910101f40 0x7fa91019a510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00d950 tx=0x7fa8fc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa90e000640 1 -- 172.21.15.46:0/1662111415 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc017070 con 0x7fa910101f40 2024-09-10T07:44:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa90e000640 1 -- 172.21.15.46:0/1662111415 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc009c30 con 0x7fa910101f40 2024-09-10T07:44:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa90e000640 1 -- 172.21.15.46:0/1662111415 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00f6b0 con 0x7fa910101f40 2024-09-10T07:44:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.476+0000 7fa916c00640 1 -- 172.21.15.46:0/1662111415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9101a1ee0 con 0x7fa910101f40 2024-09-10T07:44:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.477+0000 7fa916c00640 1 -- 172.21.15.46:0/1662111415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9101a2350 con 0x7fa910101f40 2024-09-10T07:44:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa90e000640 1 -- 172.21.15.46:0/1662111415 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa8fc0109d0 con 0x7fa910101f40 2024-09-10T07:44:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa91b25c740 1 Processor -- start 2024-09-10T07:44:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa91b25c740 1 -- start start 2024-09-10T07:44:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e98b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e990660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e9930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa91b25c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55626e8f0100 con 0x55626e990c30 2024-09-10T07:44:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa91b25c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55626e8981b0 con 0x55626e98bba0 2024-09-10T07:44:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.480+0000 7fa91b25c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55626e8ec0c0 con 0x55626e98b260 2024-09-10T07:44:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.481+0000 7fa915800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e990660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.481+0000 7fa915800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e990660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56776/0 (socket says 172.21.15.46:56776) 2024-09-10T07:44:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.482+0000 7fa915800640 1 -- 172.21.15.46:0/4091805129 learned_addr learned my addr 172.21.15.46:0/4091805129 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.481+0000 7fa916200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e98b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.482+0000 7fa914e00640 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e9930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.482+0000 7fa90e000640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8f80a2c00 0x7fa8f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.483+0000 7fa915800640 1 -- 172.21.15.46:0/4091805129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 msgr2=0x55626e98b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.483+0000 7fa915800640 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e98b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.483+0000 7fa915800640 1 -- 172.21.15.46:0/4091805129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 msgr2=0x55626e9930f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.483+0000 7fa90e000640 1 -- 172.21.15.46:0/1662111415 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa8fc0cef30 con 0x7fa910101f40 2024-09-10T07:44:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.483+0000 7fa915800640 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e9930f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.483+0000 7fa915800640 1 -- 172.21.15.46:0/4091805129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626e603210 con 0x55626e98bba0 2024-09-10T07:44:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.483+0000 7fa914e00640 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e9930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.484+0000 7fa916200640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8f80a2c00 0x7fa8f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.484+0000 7fa915800640 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e990660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa904000f80 tx=0x7fa9040092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae7aa20730991fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.485+0000 7fa902a00640 1 -- 172.21.15.46:0/4091805129 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904040030 con 0x55626e98bba0 2024-09-10T07:44:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.485+0000 7fa902a00640 1 -- 172.21.15.46:0/4091805129 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa904030cb0 con 0x55626e98bba0 2024-09-10T07:44:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.485+0000 7fa902a00640 1 -- 172.21.15.46:0/4091805129 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904036040 con 0x55626e98bba0 2024-09-10T07:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa916200640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8f80a2c00 0x7fa8f80a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa90800bd60 tx=0x7fa908004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa91b25c740 1 -- 172.21.15.46:0/4091805129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 msgr2=0x55626e990660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa91b25c740 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e990660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa904000f80 tx=0x7fa9040092f0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa91b25c740 1 -- 172.21.15.46:0/4091805129 shutdown_connections 2024-09-10T07:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa91b25c740 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e9930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa91b25c740 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e990660 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa91b25c740 1 --2- 172.21.15.46:0/4091805129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e98b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa91b25c740 1 -- 172.21.15.46:0/4091805129 >> 172.21.15.46:0/4091805129 conn(0x55626e9868d0 msgr2=0x55626e986cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.486+0000 7fa91b25c740 1 -- 172.21.15.46:0/4091805129 shutdown_connections 2024-09-10T07:44:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.487+0000 7fa91b25c740 1 -- 172.21.15.46:0/4091805129 wait complete. 2024-09-10T07:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.487+0000 7fa91b25c740 1 -- start start 2024-09-10T07:44:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.488+0000 7fa91b25c740 1 Processor -- start 2024-09-10T07:44:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.489+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e9953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.489+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e9958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.489+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e99c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.490+0000 7fa91b25c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55626e993ba0 con 0x55626e990c30 2024-09-10T07:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.490+0000 7fa91b25c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55626e879090 con 0x55626e98bba0 2024-09-10T07:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.490+0000 7fa91b25c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55626e993d20 con 0x55626e98b260 2024-09-10T07:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.490+0000 7fa91b25c740 10 client.0 _ll_register_callbacks cb 0x55626e8cdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.491+0000 7fa916200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e99c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.491+0000 7fa916200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e99c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58982/0 (socket says 172.21.15.46:58982) 2024-09-10T07:44:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.491+0000 7fa916200640 1 -- 172.21.15.46:0/2605043395 learned_addr learned my addr 172.21.15.46:0/2605043395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.492+0000 7fa915800640 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e9953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.492+0000 7fa915800640 1 -- 172.21.15.46:0/2605043395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 msgr2=0x55626e9958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.492+0000 7fa915800640 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e9958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.492+0000 7fa915800640 1 -- 172.21.15.46:0/2605043395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 msgr2=0x55626e99c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.492+0000 7fa915800640 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e99c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.492+0000 7fa915800640 1 -- 172.21.15.46:0/2605043395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626e6cf620 con 0x55626e98b260 2024-09-10T07:44:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.493+0000 7fa915800640 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e9953b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa904009be0 tx=0x7fa904030ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24348b455a753961 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.493+0000 7fa8f7400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.493+0000 7fa8f7400640 1 -- 172.21.15.46:0/2605043395 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904049070 con 0x55626e98b260 2024-09-10T07:44:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.494+0000 7fa8f7400640 1 -- 172.21.15.46:0/2605043395 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa9040303a0 con 0x55626e98b260 2024-09-10T07:44:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.494+0000 7fa91b25c740 1 -- 172.21.15.46:0/2605043395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa904030cb0 con 0x55626e98b260 2024-09-10T07:44:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.494+0000 7fa8f7400640 1 -- 172.21.15.46:0/2605043395 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa904053050 con 0x55626e98b260 2024-09-10T07:44:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.494+0000 7fa91b25c740 10 client.27364 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.494+0000 7fa91b25c740 1 -- 172.21.15.46:0/2605043395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55626e993ba0 con 0x55626e98b260 2024-09-10T07:44:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.496+0000 7fa8f7400640 1 -- 172.21.15.46:0/2605043395 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa904030cb0 con 0x55626e98b260 2024-09-10T07:44:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.496+0000 7fa8f7400640 1 -- 172.21.15.46:0/2605043395 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7fa9040315e0 con 0x55626e98b260 2024-09-10T07:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.498+0000 7fa91b25c740 10 client.27364 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.498+0000 7fa91b25c740 10 client.27364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.498+0000 7fa91b25c740 20 client.27364 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.498+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55626e9a0920 0x55626e9a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.498+0000 7fa91b25c740 4 client.27364 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.498+0000 7fa91b25c740 1 -- 172.21.15.46:0/2605043395 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55626e8fd830 con 0x55626e9a0920 2024-09-10T07:44:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.500+0000 7fa914e00640 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55626e9a0920 0x55626e9a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.502+0000 7fa8f7400640 10 client.27364 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.502+0000 7fa914e00640 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55626e9a0920 0x55626e9a2d40 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b50cf719f8c93f server_cookie=f0acc42b7a56c015 in_seq=0 out_seq=0 2024-09-10T07:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.509+0000 7fa8f7400640 1 -- 172.21.15.46:0/2605043395 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55626e8fd830 con 0x55626e9a0920 2024-09-10T07:44:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.509+0000 7fa8f7400640 10 client.27364 handle_command_reply: tid=0 2024-09-10T07:44:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.509+0000 7fa91b25c740 1 client.27364 shutdown 2024-09-10T07:44:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa91b25c740 1 -- 172.21.15.46:0/2605043395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 msgr2=0x55626e9953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e9953b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa904009be0 tx=0x7fa904030ef0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa91b25c740 1 -- 172.21.15.46:0/2605043395 shutdown_connections 2024-09-10T07:44:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55626e9a0920 0x55626e9a2d40 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e990c30 0x55626e99c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e9958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2605043395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98b260 0x55626e9953b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa91b25c740 1 -- 172.21.15.46:0/2605043395 >> 172.21.15.46:0/2605043395 conn(0x55626e9868d0 msgr2=0x55626e9880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.510+0000 7fa8f7400640 0 client.27364 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.511+0000 7fa91b25c740 1 -- 172.21.15.46:0/2605043395 shutdown_connections 2024-09-10T07:44:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.513+0000 7fa91b25c740 1 -- 172.21.15.46:0/2605043395 wait complete. 2024-09-10T07:44:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.514+0000 7fa91b25c740 20 client.27364 trim_cache size 0 max 16384 2024-09-10T07:44:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.556+0000 7fa91b25c740 1 Processor -- start 2024-09-10T07:44:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.556+0000 7fa91b25c740 1 -- start start 2024-09-10T07:44:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.556+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 0x55626e985900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.556+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e985e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.556+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e990c30 0x55626e99c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.556+0000 7fa91b25c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ac850 con 0x55626e98b260 2024-09-10T07:44:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.556+0000 7fa91b25c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ae480 con 0x55626e98bba0 2024-09-10T07:44:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.556+0000 7fa91b25c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101acf20 con 0x55626e990c30 2024-09-10T07:44:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.557+0000 7fa914e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e985e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.557+0000 7fa916200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e990c30 0x55626e99c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.557+0000 7fa916200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e990c30 0x55626e99c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:46904/0 (socket says 172.21.15.46:46904) 2024-09-10T07:44:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.557+0000 7fa916200640 1 -- 172.21.15.46:0/2646867455 learned_addr learned my addr 172.21.15.46:0/2646867455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.557+0000 7fa915800640 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 0x55626e985900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.558+0000 7fa916200640 1 -- 172.21.15.46:0/2646867455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 msgr2=0x55626e985e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.558+0000 7fa916200640 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e985e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.558+0000 7fa916200640 1 -- 172.21.15.46:0/2646867455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 msgr2=0x55626e985900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.558+0000 7fa916200640 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 0x55626e985900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.558+0000 7fa916200640 1 -- 172.21.15.46:0/2646867455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626e9866f0 con 0x55626e990c30 2024-09-10T07:44:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.558+0000 7fa914e00640 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e985e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.558+0000 7fa916200640 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e990c30 0x55626e99c850 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b960 tx=0x7fa9080042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c14e3024633fad2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.562+0000 7fa902000640 1 -- 172.21.15.46:0/2646867455 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90801a040 con 0x55626e990c30 2024-09-10T07:44:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.562+0000 7fa902000640 1 -- 172.21.15.46:0/2646867455 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa908027030 con 0x55626e990c30 2024-09-10T07:44:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.562+0000 7fa902000640 1 -- 172.21.15.46:0/2646867455 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908023900 con 0x55626e990c30 2024-09-10T07:44:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.563+0000 7fa91b25c740 1 -- 172.21.15.46:0/2646867455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e990c30 msgr2=0x55626e99c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.563+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e990c30 0x55626e99c850 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b960 tx=0x7fa9080042c0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.563+0000 7fa91b25c740 1 -- 172.21.15.46:0/2646867455 shutdown_connections 2024-09-10T07:44:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.563+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e990c30 0x55626e99c850 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.563+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e98bba0 0x55626e985e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.563+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2646867455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 0x55626e985900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.563+0000 7fa91b25c740 1 -- 172.21.15.46:0/2646867455 >> 172.21.15.46:0/2646867455 conn(0x55626e9868d0 msgr2=0x55626e9851f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.564+0000 7fa91b25c740 1 -- 172.21.15.46:0/2646867455 shutdown_connections 2024-09-10T07:44:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.564+0000 7fa91b25c740 1 -- 172.21.15.46:0/2646867455 wait complete. 2024-09-10T07:44:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.564+0000 7fa91b25c740 1 -- start start 2024-09-10T07:44:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.565+0000 7fa91b25c740 1 Processor -- start 2024-09-10T07:44:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 0x55626e99a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98bba0 0x55626e99a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa91b25c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e990c30 0x55626e99b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa91b25c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101adcf0 con 0x55626e98b260 2024-09-10T07:44:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa916200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e990c30 0x55626e99b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa916200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e990c30 0x55626e99b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56808/0 (socket says 172.21.15.46:56808) 2024-09-10T07:44:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa916200640 1 -- 172.21.15.46:0/2261595476 learned_addr learned my addr 172.21.15.46:0/2261595476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa915800640 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 0x55626e99a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.566+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ad5f0 con 0x55626e990c30 2024-09-10T07:44:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.567+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101aa830 con 0x55626e98bba0 2024-09-10T07:44:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.567+0000 7fa91b25c740 10 client.0 _ll_register_callbacks cb 0x7fa908004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.567+0000 7fa914e00640 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98bba0 0x55626e99a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.567+0000 7fa916200640 1 -- 172.21.15.46:0/2261595476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98bba0 msgr2=0x55626e99a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.567+0000 7fa916200640 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98bba0 0x55626e99a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.567+0000 7fa916200640 1 -- 172.21.15.46:0/2261595476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 msgr2=0x55626e99a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.567+0000 7fa916200640 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 0x55626e99a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.567+0000 7fa916200640 1 -- 172.21.15.46:0/2261595476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626e99bb50 con 0x55626e990c30 2024-09-10T07:44:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.568+0000 7fa916200640 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e990c30 0x55626e99b540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b320 tx=0x7fa90800b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6161fe9fcba0e4bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.568+0000 7fa8f7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:44:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.568+0000 7fa8f7400640 1 -- 172.21.15.46:0/2261595476 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90801a040 con 0x55626e990c30 2024-09-10T07:44:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.569+0000 7fa8f7400640 1 -- 172.21.15.46:0/2261595476 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa908023ad0 con 0x55626e990c30 2024-09-10T07:44:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.569+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa908027030 con 0x55626e990c30 2024-09-10T07:44:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.569+0000 7fa8f7400640 1 -- 172.21.15.46:0/2261595476 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa90803b050 con 0x55626e990c30 2024-09-10T07:44:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.570+0000 7fa91b25c740 10 client.27416 fetch_fsmap learned FSMap version 44 2024-09-10T07:44:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.570+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55626ea0b620 con 0x55626e990c30 2024-09-10T07:44:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.571+0000 7fa8f7400640 1 -- 172.21.15.46:0/2261595476 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa908027030 con 0x55626e990c30 2024-09-10T07:44:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.571+0000 7fa8f7400640 1 -- 172.21.15.46:0/2261595476 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 44) ==== 8390+0+0 (secure 0 0 0) 0x7fa908034070 con 0x55626e990c30 2024-09-10T07:44:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.571+0000 7fa91b25c740 10 client.27416 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T07:44:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.571+0000 7fa91b25c740 10 client.27416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.571+0000 7fa91b25c740 20 client.27416 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.571+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55626ea0c010 0x55626ea0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.571+0000 7fa91b25c740 4 client.27416 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:44:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.571+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55626e8f7b80 con 0x55626ea0c010 2024-09-10T07:44:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.573+0000 7fa915800640 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55626ea0c010 0x55626ea0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.576+0000 7fa915800640 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55626ea0c010 0x55626ea0e430 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b37d97e32715a1c server_cookie=eb2528a87c8fbd8b in_seq=0 out_seq=0 2024-09-10T07:44:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.576+0000 7fa8f7400640 10 client.27416 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:40.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 440.0 s, 899.2 tps, lat 113.037 ms stddev 139.028 2024-09-10T07:44:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.640+0000 7fa8f7400640 1 -- 172.21.15.46:0/2261595476 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55626e8f7b80 con 0x55626ea0c010 2024-09-10T07:44:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.640+0000 7fa8f7400640 10 client.27416 handle_command_reply: tid=0 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.640+0000 7fa91b25c740 1 client.27416 shutdown 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.641+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e990c30 msgr2=0x55626e99b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.641+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e990c30 0x55626e99b540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b320 tx=0x7fa90800b6b0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 shutdown_connections 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55626ea0c010 0x55626ea0e430 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55626e990c30 0x55626e99b540 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55626e98bba0 0x55626e99a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 1 --2- 172.21.15.46:0/2261595476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55626e98b260 0x55626e99a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa8f7400640 0 client.27416 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 >> 172.21.15.46:0/2261595476 conn(0x55626e9868d0 msgr2=0x55626e995620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 shutdown_connections 2024-09-10T07:44:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 1 -- 172.21.15.46:0/2261595476 wait complete. 2024-09-10T07:44:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.642+0000 7fa91b25c740 20 client.27416 trim_cache size 0 max 16384 2024-09-10T07:44:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:44:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:44:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:44:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:44:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 -- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8f80a2c00 msgr2=0x7fa8f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8f80a2c00 0x7fa8f80a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa90800bd60 tx=0x7fa908004730 comp rx=0 tx=0).stop 2024-09-10T07:44:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 -- 172.21.15.46:0/1662111415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa910101f40 msgr2=0x7fa91019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa910101f40 0x7fa91019a510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00d950 tx=0x7fa8fc00de10 comp rx=0 tx=0).stop 2024-09-10T07:44:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 -- 172.21.15.46:0/1662111415 shutdown_connections 2024-09-10T07:44:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa910103050 0x7fa9101a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa910101f40 0x7fa91019a510 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa8f80a2c00 0x7fa8f80a50c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 --2- 172.21.15.46:0/1662111415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa910101580 0x7fa910199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.643+0000 7fa916c00640 1 -- 172.21.15.46:0/1662111415 >> 172.21.15.46:0/1662111415 conn(0x7fa9100fd0d0 msgr2=0x7fa9101083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.646+0000 7fa916c00640 1 -- 172.21.15.46:0/1662111415 shutdown_connections 2024-09-10T07:44:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:40.646+0000 7fa916c00640 1 -- 172.21.15.46:0/1662111415 wait complete. 2024-09-10T07:44:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:42 smithi181 ceph-mon[35397]: pgmap v641: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 6.5 MiB/s wr, 677 op/s 2024-09-10T07:44:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:42 smithi063 ceph-mon[35020]: pgmap v641: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 6.5 MiB/s wr, 677 op/s 2024-09-10T07:44:42.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:42 smithi046 ceph-mon[30705]: pgmap v641: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 6.5 MiB/s wr, 677 op/s 2024-09-10T07:44:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:44 smithi181 ceph-mon[35397]: pgmap v642: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.6 MiB/s wr, 657 op/s 2024-09-10T07:44:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:44 smithi063 ceph-mon[35020]: pgmap v642: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.6 MiB/s wr, 657 op/s 2024-09-10T07:44:44.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:44 smithi046 ceph-mon[30705]: pgmap v642: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.6 MiB/s wr, 657 op/s 2024-09-10T07:44:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 445.0 s, 1010.2 tps, lat 100.233 ms stddev 127.088 2024-09-10T07:44:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:46 smithi181 ceph-mon[35397]: pgmap v643: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-09-10T07:44:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:46 smithi063 ceph-mon[35020]: pgmap v643: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-09-10T07:44:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:46 smithi046 ceph-mon[30705]: pgmap v643: 129 pgs: 129 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-09-10T07:44:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:48 smithi181 ceph-mon[35397]: pgmap v644: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 7.0 MiB/s wr, 751 op/s 2024-09-10T07:44:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:48 smithi063 ceph-mon[35020]: pgmap v644: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 7.0 MiB/s wr, 751 op/s 2024-09-10T07:44:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:48 smithi046 ceph-mon[30705]: pgmap v644: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 7.0 MiB/s wr, 751 op/s 2024-09-10T07:44:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:50 smithi181 ceph-mon[35397]: pgmap v645: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T07:44:50.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:50 smithi063 ceph-mon[35020]: pgmap v645: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T07:44:50.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:50 smithi046 ceph-mon[30705]: pgmap v645: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T07:44:50.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 450.0 s, 695.0 tps, lat 140.755 ms stddev 157.769 2024-09-10T07:44:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:51 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:44:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:51 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:44:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:51 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:44:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:51 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:44:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:51 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:44:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:51 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:44:51.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:51 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:44:51.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:51 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:44:51.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:51 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:44:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:52 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:44:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:52 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:44:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:52 smithi181 ceph-mon[35397]: pgmap v646: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 626 op/s 2024-09-10T07:44:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:52 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:44:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:52 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:44:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:52 smithi063 ceph-mon[35020]: pgmap v646: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 626 op/s 2024-09-10T07:44:52.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:52 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:44:52.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:52 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:44:52.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:52 smithi046 ceph-mon[30705]: pgmap v646: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 626 op/s 2024-09-10T07:44:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.831+0000 7fd997400640 1 -- 172.21.15.46:0/3670649529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 msgr2=0x7fd990074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.831+0000 7fd997400640 1 --2- 172.21.15.46:0/3670649529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 0x7fd990074620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd984009900 tx=0x7fd98402a480 comp rx=0 tx=0).stop 2024-09-10T07:44:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.832+0000 7fd997400640 1 -- 172.21.15.46:0/3670649529 shutdown_connections 2024-09-10T07:44:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.832+0000 7fd997400640 1 --2- 172.21.15.46:0/3670649529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9901099f0 0x7fd99010beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.832+0000 7fd997400640 1 --2- 172.21.15.46:0/3670649529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd990074bf0 0x7fd990075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.832+0000 7fd997400640 1 --2- 172.21.15.46:0/3670649529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 0x7fd990074620 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.832+0000 7fd997400640 1 -- 172.21.15.46:0/3670649529 >> 172.21.15.46:0/3670649529 conn(0x7fd9900fd2e0 msgr2=0x7fd9900ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.832+0000 7fd997400640 1 -- 172.21.15.46:0/3670649529 shutdown_connections 2024-09-10T07:44:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.832+0000 7fd997400640 1 -- 172.21.15.46:0/3670649529 wait complete. 2024-09-10T07:44:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.833+0000 7fd997400640 1 Processor -- start 2024-09-10T07:44:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd997400640 1 -- start start 2024-09-10T07:44:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd997400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd990074bf0 0x7fd990199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd997400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 0x7fd99019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd997400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9901099f0 0x7fd9901a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd997400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010f9d0 con 0x7fd9901099f0 2024-09-10T07:44:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd997400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010f850 con 0x7fd990074bf0 2024-09-10T07:44:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd997400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010fb50 con 0x7fd9900761d0 2024-09-10T07:44:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd994a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd990074bf0 0x7fd990199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd994a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd990074bf0 0x7fd990199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59638/0 (socket says 172.21.15.46:59638) 2024-09-10T07:44:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd994a00640 1 -- 172.21.15.46:0/3631420908 learned_addr learned my addr 172.21.15.46:0/3631420908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.834+0000 7fd98fe00640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 0x7fd99019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.835+0000 7fd995400640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9901099f0 0x7fd9901a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.835+0000 7fd98fe00640 1 -- 172.21.15.46:0/3631420908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd990074bf0 msgr2=0x7fd990199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.835+0000 7fd98fe00640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd990074bf0 0x7fd990199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.835+0000 7fd98fe00640 1 -- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9901099f0 msgr2=0x7fd9901a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.835+0000 7fd98fe00640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9901099f0 0x7fd9901a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.835+0000 7fd98fe00640 1 -- 172.21.15.46:0/3631420908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9901a1bf0 con 0x7fd9900761d0 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.836+0000 7fd98fe00640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 0x7fd99019a510 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00e9f0 tx=0x7fd97c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.837+0000 7fd98d600640 1 -- 172.21.15.46:0/3631420908 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00cd80 con 0x7fd9900761d0 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.837+0000 7fd98d600640 1 -- 172.21.15.46:0/3631420908 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97c0035f0 con 0x7fd9900761d0 2024-09-10T07:44:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.837+0000 7fd98d600640 1 -- 172.21.15.46:0/3631420908 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c010690 con 0x7fd9900761d0 2024-09-10T07:44:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.837+0000 7fd997400640 1 -- 172.21.15.46:0/3631420908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99006bc10 con 0x7fd9900761d0 2024-09-10T07:44:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.837+0000 7fd997400640 1 -- 172.21.15.46:0/3631420908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99006c110 con 0x7fd9900761d0 2024-09-10T07:44:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.843+0000 7fd98d600640 1 -- 172.21.15.46:0/3631420908 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd97c003b30 con 0x7fd9900761d0 2024-09-10T07:44:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.846+0000 7fd98d600640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9780a2e20 0x7fd9780a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.847+0000 7fd994a00640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9780a2e20 0x7fd9780a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.849+0000 7fd98d600640 1 -- 172.21.15.46:0/3631420908 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd97c0cf6e0 con 0x7fd9900761d0 2024-09-10T07:44:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.849+0000 7fd99a249740 1 Processor -- start 2024-09-10T07:44:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.851+0000 7fd994a00640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9780a2e20 0x7fd9780a52e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd984031440 tx=0x7fd9840315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.852+0000 7fd99a249740 1 -- start start 2024-09-10T07:44:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.852+0000 7fd99a249740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 0x55dff2872ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.852+0000 7fd99a249740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff2877ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.852+0000 7fd99a249740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff2878270 0x55dff287a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.852+0000 7fd99a249740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dff27cf100 con 0x55dff28728a0 2024-09-10T07:44:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.852+0000 7fd99a249740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dff27771b0 con 0x55dff2878270 2024-09-10T07:44:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.852+0000 7fd99a249740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dff27cb0c0 con 0x55dff28731e0 2024-09-10T07:44:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.853+0000 7fd994a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 0x55dff2872ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.853+0000 7fd994a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 0x55dff2872ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45976/0 (socket says 172.21.15.46:45976) 2024-09-10T07:44:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.853+0000 7fd994a00640 1 -- 172.21.15.46:0/811536682 learned_addr learned my addr 172.21.15.46:0/811536682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.853+0000 7fd98fe00640 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff2877ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.853+0000 7fd995400640 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff2878270 0x55dff287a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.854+0000 7fd995400640 1 -- 172.21.15.46:0/811536682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 msgr2=0x55dff2877ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.854+0000 7fd995400640 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff2877ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.854+0000 7fd995400640 1 -- 172.21.15.46:0/811536682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 msgr2=0x55dff2872ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.854+0000 7fd995400640 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 0x55dff2872ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.854+0000 7fd995400640 1 -- 172.21.15.46:0/811536682 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff24e2210 con 0x55dff2878270 2024-09-10T07:44:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.858+0000 7fd995400640 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff2878270 0x55dff287a730 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd988009f30 tx=0x7fd988010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509471a3c1111ea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.858+0000 7fd981c00640 1 -- 172.21.15.46:0/811536682 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988016070 con 0x55dff2878270 2024-09-10T07:44:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.859+0000 7fd981c00640 1 -- 172.21.15.46:0/811536682 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd988004d20 con 0x55dff2878270 2024-09-10T07:44:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.859+0000 7fd981c00640 1 -- 172.21.15.46:0/811536682 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988012870 con 0x55dff2878270 2024-09-10T07:44:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.860+0000 7fd99a249740 1 -- 172.21.15.46:0/811536682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff2878270 msgr2=0x55dff287a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.860+0000 7fd99a249740 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff2878270 0x55dff287a730 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd988009f30 tx=0x7fd988010800 comp rx=0 tx=0).stop 2024-09-10T07:44:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.860+0000 7fd99a249740 1 -- 172.21.15.46:0/811536682 shutdown_connections 2024-09-10T07:44:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.860+0000 7fd99a249740 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff2878270 0x55dff287a730 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.860+0000 7fd99a249740 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff2877ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.860+0000 7fd99a249740 1 --2- 172.21.15.46:0/811536682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 0x55dff2872ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.860+0000 7fd99a249740 1 -- 172.21.15.46:0/811536682 >> 172.21.15.46:0/811536682 conn(0x55dff286df10 msgr2=0x55dff286e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.863+0000 7fd99a249740 1 -- 172.21.15.46:0/811536682 shutdown_connections 2024-09-10T07:44:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.863+0000 7fd99a249740 1 -- 172.21.15.46:0/811536682 wait complete. 2024-09-10T07:44:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.864+0000 7fd99a249740 1 -- start start 2024-09-10T07:44:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.864+0000 7fd99a249740 1 Processor -- start 2024-09-10T07:44:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.865+0000 7fd99a249740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28728a0 0x55dff287cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.865+0000 7fd99a249740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff287cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.865+0000 7fd99a249740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2884030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.865+0000 7fd98fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff287cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.865+0000 7fd98fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff287cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57528/0 (socket says 172.21.15.46:57528) 2024-09-10T07:44:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.865+0000 7fd994a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28728a0 0x55dff287cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.866+0000 7fd995400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2884030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.866+0000 7fd994a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28728a0 0x55dff287cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59660/0 (socket says 172.21.15.46:59660) 2024-09-10T07:44:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.866+0000 7fd995400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2884030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45978/0 (socket says 172.21.15.46:45978) 2024-09-10T07:44:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.866+0000 7fd98fe00640 1 -- 172.21.15.46:0/591095572 learned_addr learned my addr 172.21.15.46:0/591095572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.866+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dff287b240 con 0x55dff2878270 2024-09-10T07:44:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.868+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dff2758090 con 0x55dff28728a0 2024-09-10T07:44:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.868+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dff287b3c0 con 0x55dff28731e0 2024-09-10T07:44:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.868+0000 7fd99a249740 10 client.0 _ll_register_callbacks cb 0x55dff27acf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd98fe00640 1 -- 172.21.15.46:0/591095572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28728a0 msgr2=0x55dff287cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd98fe00640 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28728a0 0x55dff287cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd98fe00640 1 -- 172.21.15.46:0/591095572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 msgr2=0x55dff2884030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd98fe00640 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2884030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd98fe00640 1 -- 172.21.15.46:0/591095572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff25ae620 con 0x55dff28731e0 2024-09-10T07:44:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd995400640 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2884030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:53.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd994a00640 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28728a0 0x55dff287cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:44:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd98fe00640 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff287cff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd97c01f000 tx=0x7fd97c09c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1de0442e65efc8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd976000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.869+0000 7fd976000640 1 -- 172.21.15.46:0/591095572 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b9060 con 0x55dff28731e0 2024-09-10T07:44:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.870+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd988004d20 con 0x55dff28731e0 2024-09-10T07:44:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.870+0000 7fd976000640 1 -- 172.21.15.46:0/591095572 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd97c0037a0 con 0x55dff28731e0 2024-09-10T07:44:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.870+0000 7fd976000640 1 -- 172.21.15.46:0/591095572 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd97c0981d0 con 0x55dff28731e0 2024-09-10T07:44:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.870+0000 7fd99a249740 10 client.27382 fetch_fsmap learned FSMap version 45 2024-09-10T07:44:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.870+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55dff2886f40 con 0x55dff28731e0 2024-09-10T07:44:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.876+0000 7fd976000640 1 -- 172.21.15.46:0/591095572 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd988004d20 con 0x55dff28731e0 2024-09-10T07:44:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.877+0000 7fd976000640 1 -- 172.21.15.46:0/591095572 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7fd97c09afb0 con 0x55dff28731e0 2024-09-10T07:44:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.877+0000 7fd99a249740 10 client.27382 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T07:44:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.877+0000 7fd99a249740 10 client.27382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.877+0000 7fd99a249740 20 client.27382 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.877+0000 7fd99a249740 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dff28885c0 0x55dff288aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.877+0000 7fd99a249740 4 client.27382 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:44:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.877+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff27dc830 con 0x55dff28885c0 2024-09-10T07:44:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.878+0000 7fd994a00640 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dff28885c0 0x55dff288aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.881+0000 7fd994a00640 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dff28885c0 0x55dff288aa60 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a68d3c9e6e318d server_cookie=fe3aeaa1a31cd17 in_seq=0 out_seq=0 2024-09-10T07:44:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.881+0000 7fd976000640 10 client.27382 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.884+0000 7fd976000640 1 -- 172.21.15.46:0/591095572 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dff27dc830 con 0x55dff28885c0 2024-09-10T07:44:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.884+0000 7fd976000640 10 client.27382 handle_command_reply: tid=0 2024-09-10T07:44:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.884+0000 7fd99a249740 1 client.27382 shutdown 2024-09-10T07:44:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.884+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 msgr2=0x55dff287cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.884+0000 7fd99a249740 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff287cff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd97c01f000 tx=0x7fd97c09c3a0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.885+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 shutdown_connections 2024-09-10T07:44:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.885+0000 7fd99a249740 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dff28885c0 0x55dff288aa60 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.885+0000 7fd99a249740 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2884030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.885+0000 7fd99a249740 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28731e0 0x55dff287cff0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.885+0000 7fd99a249740 1 --2- 172.21.15.46:0/591095572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28728a0 0x55dff287cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.885+0000 7fd976000640 0 client.27382 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.885+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 >> 172.21.15.46:0/591095572 conn(0x55dff286df10 msgr2=0x55dff286f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.886+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 shutdown_connections 2024-09-10T07:44:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.886+0000 7fd99a249740 1 -- 172.21.15.46:0/591095572 wait complete. 2024-09-10T07:44:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.886+0000 7fd99a249740 20 client.27382 trim_cache size 0 max 16384 2024-09-10T07:44:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.921+0000 7fd99a249740 1 Processor -- start 2024-09-10T07:44:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.921+0000 7fd99a249740 1 -- start start 2024-09-10T07:44:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd99a249740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 0x55dff2870dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd99a249740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2883ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd99a249740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff2878270 0x55dff2871310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd99a249740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9900722c0 con 0x55dff28728a0 2024-09-10T07:44:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd99a249740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd990073ec0 con 0x55dff28731e0 2024-09-10T07:44:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd99a249740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd990072990 con 0x55dff2878270 2024-09-10T07:44:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd995400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff2878270 0x55dff2871310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd995400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff2878270 0x55dff2871310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57542/0 (socket says 172.21.15.46:57542) 2024-09-10T07:44:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd995400640 1 -- 172.21.15.46:0/1278795440 learned_addr learned my addr 172.21.15.46:0/1278795440 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.922+0000 7fd98fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2883ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.923+0000 7fd995400640 1 -- 172.21.15.46:0/1278795440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 msgr2=0x55dff2883ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.923+0000 7fd995400640 1 --2- 172.21.15.46:0/1278795440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2883ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.923+0000 7fd995400640 1 -- 172.21.15.46:0/1278795440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 msgr2=0x55dff2870dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.923+0000 7fd995400640 1 --2- 172.21.15.46:0/1278795440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 0x55dff2870dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.923+0000 7fd995400640 1 -- 172.21.15.46:0/1278795440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff2860d60 con 0x55dff2878270 2024-09-10T07:44:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.923+0000 7fd98fe00640 1 --2- 172.21.15.46:0/1278795440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2883ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.928+0000 7fd995400640 1 --2- 172.21.15.46:0/1278795440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff2878270 0x55dff2871310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd988011010 tx=0x7fd988005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ef3c67811d49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.928+0000 7fd981200640 1 -- 172.21.15.46:0/1278795440 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988016050 con 0x55dff2878270 2024-09-10T07:44:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.928+0000 7fd981200640 1 -- 172.21.15.46:0/1278795440 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd988003e50 con 0x55dff2878270 2024-09-10T07:44:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.929+0000 7fd981200640 1 -- 172.21.15.46:0/1278795440 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988012720 con 0x55dff2878270 2024-09-10T07:44:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.930+0000 7fd99a249740 1 -- 172.21.15.46:0/1278795440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff2878270 msgr2=0x55dff2871310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.930+0000 7fd99a249740 1 --2- 172.21.15.46:0/1278795440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff2878270 0x55dff2871310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd988011010 tx=0x7fd988005e30 comp rx=0 tx=0).stop 2024-09-10T07:44:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.930+0000 7fd99a249740 1 -- 172.21.15.46:0/1278795440 shutdown_connections 2024-09-10T07:44:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.930+0000 7fd99a249740 1 --2- 172.21.15.46:0/1278795440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff2878270 0x55dff2871310 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.930+0000 7fd99a249740 1 --2- 172.21.15.46:0/1278795440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2883ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.930+0000 7fd99a249740 1 --2- 172.21.15.46:0/1278795440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff28728a0 0x55dff2870dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.930+0000 7fd99a249740 1 -- 172.21.15.46:0/1278795440 >> 172.21.15.46:0/1278795440 conn(0x55dff286df10 msgr2=0x55dff2874430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.931+0000 7fd99a249740 1 -- 172.21.15.46:0/1278795440 shutdown_connections 2024-09-10T07:44:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.931+0000 7fd99a249740 1 -- 172.21.15.46:0/1278795440 wait complete. 2024-09-10T07:44:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.931+0000 7fd99a249740 1 -- start start 2024-09-10T07:44:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.932+0000 7fd99a249740 1 Processor -- start 2024-09-10T07:44:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.933+0000 7fd99a249740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28728a0 0x55dff2881ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.933+0000 7fd99a249740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2882020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.933+0000 7fd99a249740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2883030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.933+0000 7fd994a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28728a0 0x55dff2881ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.934+0000 7fd994a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28728a0 0x55dff2881ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57556/0 (socket says 172.21.15.46:57556) 2024-09-10T07:44:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.934+0000 7fd99a249740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd990073730 con 0x55dff2878270 2024-09-10T07:44:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.934+0000 7fd99a249740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd990073060 con 0x55dff28731e0 2024-09-10T07:44:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.934+0000 7fd99a249740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd9900702a0 con 0x55dff28728a0 2024-09-10T07:44:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.934+0000 7fd99a249740 10 client.0 _ll_register_callbacks cb 0x7fd97c097ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:44:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.934+0000 7fd995400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2883030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.934+0000 7fd98fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2882020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.934+0000 7fd994a00640 1 -- 172.21.15.46:0/372592774 learned_addr learned my addr 172.21.15.46:0/372592774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.935+0000 7fd994a00640 1 -- 172.21.15.46:0/372592774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 msgr2=0x55dff2882020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.935+0000 7fd994a00640 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2882020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.935+0000 7fd994a00640 1 -- 172.21.15.46:0/372592774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 msgr2=0x55dff2883030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:44:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.935+0000 7fd994a00640 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2883030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.935+0000 7fd994a00640 1 -- 172.21.15.46:0/372592774 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff2883640 con 0x55dff28728a0 2024-09-10T07:44:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.935+0000 7fd98fe00640 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2882020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:44:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.936+0000 7fd994a00640 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28728a0 0x55dff2881ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd984000c00 tx=0x7fd984009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce10d8e7afe08d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.936+0000 7fd976000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:44:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.936+0000 7fd976000640 1 -- 172.21.15.46:0/372592774 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984035050 con 0x55dff28728a0 2024-09-10T07:44:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.937+0000 7fd976000640 1 -- 172.21.15.46:0/372592774 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd9840035f0 con 0x55dff28728a0 2024-09-10T07:44:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.937+0000 7fd99a249740 1 -- 172.21.15.46:0/372592774 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd988003e50 con 0x55dff28728a0 2024-09-10T07:44:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.937+0000 7fd976000640 1 -- 172.21.15.46:0/372592774 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd98405e050 con 0x55dff28728a0 2024-09-10T07:44:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.937+0000 7fd99a249740 10 client.27388 fetch_fsmap learned FSMap version 45 2024-09-10T07:44:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.937+0000 7fd99a249740 1 -- 172.21.15.46:0/372592774 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55dff28f36c0 con 0x55dff28728a0 2024-09-10T07:44:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.937+0000 7fd976000640 1 -- 172.21.15.46:0/372592774 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd988003e50 con 0x55dff28728a0 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.938+0000 7fd976000640 1 -- 172.21.15.46:0/372592774 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7fd9840498f0 con 0x55dff28728a0 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.938+0000 7fd99a249740 10 client.27388 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.938+0000 7fd99a249740 10 client.27388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.938+0000 7fd99a249740 20 client.27388 populate_metadata read hostname 'smithi046' 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.938+0000 7fd99a249740 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dff28f4160 0x55dff28f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.938+0000 7fd99a249740 4 client.27388 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.938+0000 7fd99a249740 1 -- 172.21.15.46:0/372592774 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dff27d6b80 con 0x55dff28f4160 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.938+0000 7fd98fe00640 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dff28f4160 0x55dff28f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.939+0000 7fd98fe00640 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dff28f4160 0x55dff28f6580 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39aaa099fb81f8f7 server_cookie=d49434c949ddc94a in_seq=0 out_seq=0 2024-09-10T07:44:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.939+0000 7fd976000640 10 client.27388 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.998+0000 7fd976000640 1 -- 172.21.15.46:0/372592774 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dff27d6b80 con 0x55dff28f4160 2024-09-10T07:44:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.998+0000 7fd976000640 10 client.27388 handle_command_reply: tid=0 2024-09-10T07:44:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.998+0000 7fd99a249740 1 client.27388 shutdown 2024-09-10T07:44:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 -- 172.21.15.46:0/372592774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28728a0 msgr2=0x55dff2881ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28728a0 0x55dff2881ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd984000c00 tx=0x7fd984009880 comp rx=0 tx=0).stop 2024-09-10T07:44:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 -- 172.21.15.46:0/372592774 shutdown_connections 2024-09-10T07:44:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55dff28f4160 0x55dff28f6580 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dff2878270 0x55dff2883030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff28731e0 0x55dff2882020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd976000640 0 client.27388 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:44:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 --2- 172.21.15.46:0/372592774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dff28728a0 0x55dff2881ae0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 -- 172.21.15.46:0/372592774 >> 172.21.15.46:0/372592774 conn(0x55dff286df10 msgr2=0x55dff2864060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 -- 172.21.15.46:0/372592774 shutdown_connections 2024-09-10T07:44:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 1 -- 172.21.15.46:0/372592774 wait complete. 2024-09-10T07:44:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:53.999+0000 7fd99a249740 20 client.27388 trim_cache size 0 max 16384 2024-09-10T07:44:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:44:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:44:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:44:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:44:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.000+0000 7fd997400640 1 -- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9780a2e20 msgr2=0x7fd9780a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.000+0000 7fd997400640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9780a2e20 0x7fd9780a52e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd984031440 tx=0x7fd9840315b0 comp rx=0 tx=0).stop 2024-09-10T07:44:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 -- 172.21.15.46:0/3631420908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 msgr2=0x7fd99019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 0x7fd99019a510 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00e9f0 tx=0x7fd97c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 -- 172.21.15.46:0/3631420908 shutdown_connections 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9780a2e20 0x7fd9780a52e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9901099f0 0x7fd9901a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9900761d0 0x7fd99019a510 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 --2- 172.21.15.46:0/3631420908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd990074bf0 0x7fd990199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 -- 172.21.15.46:0/3631420908 >> 172.21.15.46:0/3631420908 conn(0x7fd9900fd2e0 msgr2=0x7fd990105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 -- 172.21.15.46:0/3631420908 shutdown_connections 2024-09-10T07:44:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:44:54.001+0000 7fd997400640 1 -- 172.21.15.46:0/3631420908 wait complete. 2024-09-10T07:44:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:54 smithi046 ceph-mon[30705]: pgmap v647: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-09-10T07:44:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:54 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:44:54.503 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-10T07:44:54.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:44:54.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:54 smithi181 ceph-mon[35397]: pgmap v647: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-09-10T07:44:54.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:54 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:44:54.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:54 smithi063 ceph-mon[35020]: pgmap v647: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-09-10T07:44:54.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:54 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:44:55.170 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 455.0 s, 675.8 tps, lat 140.755 ms stddev 153.366 2024-09-10T07:44:56.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:56 smithi181 ceph-mon[35397]: pgmap v648: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-09-10T07:44:56.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:56 smithi063 ceph-mon[35020]: pgmap v648: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-09-10T07:44:56.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:56 smithi046 ceph-mon[30705]: pgmap v648: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-09-10T07:44:57.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:57 smithi046 ceph-mon[30705]: pgmap v649: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 MiB/s wr, 645 op/s 2024-09-10T07:44:57.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:57 smithi181 ceph-mon[35397]: pgmap v649: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 MiB/s wr, 645 op/s 2024-09-10T07:44:57.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:57 smithi063 ceph-mon[35020]: pgmap v649: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 MiB/s wr, 645 op/s 2024-09-10T07:44:58.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.098+0000 7f4e27e00640 1 -- 172.21.15.46:0/642946974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 msgr2=0x7f4e2811ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:58.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.098+0000 7f4e27e00640 1 --2- 172.21.15.46:0/642946974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 0x7f4e2811ab20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009900 tx=0x7f4e1002a410 comp rx=0 tx=0).stop 2024-09-10T07:44:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.098+0000 7f4e27e00640 1 -- 172.21.15.46:0/642946974 shutdown_connections 2024-09-10T07:44:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.098+0000 7f4e27e00640 1 --2- 172.21.15.46:0/642946974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e2811b060 0x7f4e2811d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.098+0000 7f4e27e00640 1 --2- 172.21.15.46:0/642946974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 0x7f4e2811ab20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.098+0000 7f4e27e00640 1 --2- 172.21.15.46:0/642946974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2806dba0 0x7f4e2806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.098+0000 7f4e27e00640 1 -- 172.21.15.46:0/642946974 >> 172.21.15.46:0/642946974 conn(0x7f4e2806cbe0 msgr2=0x7f4e2806cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.099+0000 7f4e27e00640 1 -- 172.21.15.46:0/642946974 shutdown_connections 2024-09-10T07:44:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.099+0000 7f4e27e00640 1 -- 172.21.15.46:0/642946974 wait complete. 2024-09-10T07:44:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.100+0000 7f4e27e00640 1 Processor -- start 2024-09-10T07:44:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.101+0000 7f4e27e00640 1 -- start start 2024-09-10T07:44:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2806dba0 0x7f4e281a7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 0x7f4e281a7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e2811b060 0x7f4e281ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e27e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28120be0 con 0x7f4e2806dba0 2024-09-10T07:44:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e27e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28120a60 con 0x7f4e2811b060 2024-09-10T07:44:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e27e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28120d60 con 0x7f4e2806e4c0 2024-09-10T07:44:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e26000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 0x7f4e281a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e27400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e2811b060 0x7f4e281ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e26000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 0x7f4e281a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33412/0 (socket says 172.21.15.46:33412) 2024-09-10T07:44:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e26000640 1 -- 172.21.15.46:0/4149756619 learned_addr learned my addr 172.21.15.46:0/4149756619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:44:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e26000640 1 -- 172.21.15.46:0/4149756619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e2811b060 msgr2=0x7f4e281ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e26000640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e2811b060 0x7f4e281ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e26000640 1 -- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2806dba0 msgr2=0x7f4e281a7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e26000640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2806dba0 0x7f4e281a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.102+0000 7f4e26000640 1 -- 172.21.15.46:0/4149756619 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e281ac0b0 con 0x7f4e2806e4c0 2024-09-10T07:44:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.103+0000 7f4e26000640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 0x7f4e281a7640 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e10004770 tx=0x7f4e10003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.103+0000 7f4e1f400640 1 -- 172.21.15.46:0/4149756619 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1003f070 con 0x7f4e2806e4c0 2024-09-10T07:44:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.104+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e281ac340 con 0x7f4e2806e4c0 2024-09-10T07:44:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.104+0000 7f4e1f400640 1 -- 172.21.15.46:0/4149756619 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e10031ca0 con 0x7f4e2806e4c0 2024-09-10T07:44:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.104+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e281b3c80 con 0x7f4e2806e4c0 2024-09-10T07:44:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.104+0000 7f4e1f400640 1 -- 172.21.15.46:0/4149756619 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10035030 con 0x7f4e2806e4c0 2024-09-10T07:44:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.106+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2806df80 con 0x7f4e2806e4c0 2024-09-10T07:44:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.108+0000 7f4e1f400640 1 -- 172.21.15.46:0/4149756619 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e10042070 con 0x7f4e2806e4c0 2024-09-10T07:44:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.109+0000 7f4e1f400640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e0c0a2b80 0x7f4e0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:44:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.110+0000 7f4e1f400640 1 -- 172.21.15.46:0/4149756619 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4e100ef7c0 con 0x7f4e2806e4c0 2024-09-10T07:44:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.110+0000 7f4e26a00640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e0c0a2b80 0x7f4e0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:44:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.111+0000 7f4e26a00640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e0c0a2b80 0x7f4e0c0a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e20005fd0 tx=0x7f4e20009710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:44:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.114+0000 7f4e1f400640 1 -- 172.21.15.46:0/4149756619 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e100bc340 con 0x7f4e2806e4c0 2024-09-10T07:44:58.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.510+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e281b3e50 con 0x7f4e2806e4c0 2024-09-10T07:44:58.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.511+0000 7f4e1f400640 1 -- 172.21.15.46:0/4149756619 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11979 (secure 0 0 0) 0x7f4e100bbc40 con 0x7f4e2806e4c0 2024-09-10T07:44:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:44:58.512 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"btime":"2024-09-10T07:44:53:148560+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:44:53.049151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:44:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 45 2024-09-10T07:44:58.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e0c0a2b80 msgr2=0x7f4e0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e0c0a2b80 0x7f4e0c0a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e20005fd0 tx=0x7f4e20009710 comp rx=0 tx=0).stop 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 msgr2=0x7f4e281a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 0x7f4e281a7640 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e10004770 tx=0x7f4e10003c10 comp rx=0 tx=0).stop 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 shutdown_connections 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e2811b060 0x7f4e281ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e2806e4c0 0x7f4e281a7640 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e0c0a2b80 0x7f4e0c0a5040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 --2- 172.21.15.46:0/4149756619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2806dba0 0x7f4e281a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.516+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 >> 172.21.15.46:0/4149756619 conn(0x7f4e2806cbe0 msgr2=0x7f4e2811b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.517+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 shutdown_connections 2024-09-10T07:44:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:44:58.517+0000 7f4e27e00640 1 -- 172.21.15.46:0/4149756619 wait complete. 2024-09-10T07:44:58.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:58 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/4149756619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:44:58.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:58 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/4149756619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:44:58.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:58 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/4149756619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:44:59.414 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-10T07:44:59.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-10T07:44:59.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:44:59 smithi046 ceph-mon[30705]: pgmap v650: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.8 MiB/s wr, 477 op/s 2024-09-10T07:44:59.899 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:44:59 smithi181 ceph-mon[35397]: pgmap v650: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.8 MiB/s wr, 477 op/s 2024-09-10T07:44:59.900 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:44:59 smithi063 ceph-mon[35020]: pgmap v650: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.8 MiB/s wr, 477 op/s 2024-09-10T07:45:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 460.0 s, 519.8 tps, lat 205.223 ms stddev 296.543 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.998+0000 7f90c5a00640 1 -- 172.21.15.46:0/2075907325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0102360 msgr2=0x7f90c010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.998+0000 7f90c5a00640 1 --2- 172.21.15.46:0/2075907325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0102360 0x7f90c010e890 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009ca0 tx=0x7f90b802a8b0 comp rx=0 tx=0).stop 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.998+0000 7f90c5a00640 1 -- 172.21.15.46:0/2075907325 shutdown_connections 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.998+0000 7f90c5a00640 1 --2- 172.21.15.46:0/2075907325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c0102360 0x7f90c010e890 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.998+0000 7f90c5a00640 1 --2- 172.21.15.46:0/2075907325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c01019c0 0x7f90c0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.998+0000 7f90c5a00640 1 --2- 172.21.15.46:0/2075907325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c01079c0 0x7f90c0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.998+0000 7f90c5a00640 1 -- 172.21.15.46:0/2075907325 >> 172.21.15.46:0/2075907325 conn(0x7f90c00fd740 msgr2=0x7f90c00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.999+0000 7f90c5a00640 1 -- 172.21.15.46:0/2075907325 shutdown_connections 2024-09-10T07:45:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:00.999+0000 7f90c5a00640 1 -- 172.21.15.46:0/2075907325 wait complete. 2024-09-10T07:45:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.001+0000 7f90c5a00640 1 Processor -- start 2024-09-10T07:45:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.001+0000 7f90c5a00640 1 -- start start 2024-09-10T07:45:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.001+0000 7f90c5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c01019c0 0x7f90c01106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.001+0000 7f90c5a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0102360 0x7f90c0110bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.001+0000 7f90c5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c01079c0 0x7f90c0111130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.002+0000 7f90c5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0075c00 con 0x7f90c01079c0 2024-09-10T07:45:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.002+0000 7f90c5a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0075a80 con 0x7f90c01019c0 2024-09-10T07:45:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.002+0000 7f90c5a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0075d80 con 0x7f90c0102360 2024-09-10T07:45:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.002+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c01019c0 0x7f90c01106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.002+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c01019c0 0x7f90c01106b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41400/0 (socket says 172.21.15.46:41400) 2024-09-10T07:45:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.002+0000 7f90bfe00640 1 -- 172.21.15.46:0/3382673389 learned_addr learned my addr 172.21.15.46:0/3382673389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.002+0000 7f90bf400640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0102360 0x7f90c0110bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.003+0000 7f90bfe00640 1 -- 172.21.15.46:0/3382673389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0102360 msgr2=0x7f90c0110bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.003+0000 7f90bfe00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0102360 0x7f90c0110bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.003+0000 7f90bfe00640 1 -- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c01079c0 msgr2=0x7f90c0111130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.003+0000 7f90bfe00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c01079c0 0x7f90c0111130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.003+0000 7f90bfe00640 1 -- 172.21.15.46:0/3382673389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c01aab00 con 0x7f90c01019c0 2024-09-10T07:45:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.003+0000 7f90bf400640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0102360 0x7f90c0110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.004+0000 7f90bfe00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c01019c0 0x7f90c01106b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00c8f0 tx=0x7f90ac00cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.004+0000 7f90bcc00640 1 -- 172.21.15.46:0/3382673389 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac004220 con 0x7f90c01019c0 2024-09-10T07:45:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.004+0000 7f90bcc00640 1 -- 172.21.15.46:0/3382673389 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90ac007e10 con 0x7f90c01019c0 2024-09-10T07:45:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.004+0000 7f90bcc00640 1 -- 172.21.15.46:0/3382673389 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac004a00 con 0x7f90c01019c0 2024-09-10T07:45:01.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.004+0000 7f90c5a00640 1 -- 172.21.15.46:0/3382673389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c01aadf0 con 0x7f90c01019c0 2024-09-10T07:45:01.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.005+0000 7f90c5a00640 1 -- 172.21.15.46:0/3382673389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01ab320 con 0x7f90c01019c0 2024-09-10T07:45:01.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.009+0000 7f90c9aa3740 1 Processor -- start 2024-09-10T07:45:01.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.011+0000 7f90c9aa3740 1 -- start start 2024-09-10T07:45:01.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.011+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04560 0x55bd09f04940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04e80 0x55bd09f0f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f11b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bd09e6a530 con 0x55bd09f04560 2024-09-10T07:45:01.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bd09e11860 con 0x55bd09f0f740 2024-09-10T07:45:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd09e664f0 con 0x55bd09f04e80 2024-09-10T07:45:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90bf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f11b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90bf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f11b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41416/0 (socket says 172.21.15.46:41416) 2024-09-10T07:45:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90bf400640 1 -- 172.21.15.46:0/799811669 learned_addr learned my addr 172.21.15.46:0/799811669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:01.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.012+0000 7f90bcc00640 1 -- 172.21.15.46:0/3382673389 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90ac007930 con 0x7f90c01019c0 2024-09-10T07:45:01.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.013+0000 7f90bf400640 1 -- 172.21.15.46:0/799811669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04e80 msgr2=0x55bd09f0f200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:45:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.013+0000 7f90bf400640 1 --2- 172.21.15.46:0/799811669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04e80 0x55bd09f0f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.013+0000 7f90bf400640 1 -- 172.21.15.46:0/799811669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04560 msgr2=0x55bd09f04940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.013+0000 7f90bf400640 1 --2- 172.21.15.46:0/799811669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04560 0x55bd09f04940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.013+0000 7f90bf400640 1 -- 172.21.15.46:0/799811669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd09bdbb90 con 0x55bd09f0f740 2024-09-10T07:45:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.013+0000 7f90bf400640 1 --2- 172.21.15.46:0/799811669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f11b30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009ca0 tx=0x7f90b0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=255af5e9d8223b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.013+0000 7f90b5600640 1 -- 172.21.15.46:0/799811669 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0004030 con 0x55bd09f0f740 2024-09-10T07:45:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.015+0000 7f90b5600640 1 -- 172.21.15.46:0/799811669 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b00041d0 con 0x55bd09f0f740 2024-09-10T07:45:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.015+0000 7f90b5600640 1 -- 172.21.15.46:0/799811669 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0012650 con 0x55bd09f0f740 2024-09-10T07:45:01.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.016+0000 7f90c9aa3740 1 -- 172.21.15.46:0/799811669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 msgr2=0x55bd09f11b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:01.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.016+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/799811669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f11b30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009ca0 tx=0x7f90b0010740 comp rx=0 tx=0).stop 2024-09-10T07:45:01.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.016+0000 7f90bcc00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f90a40a2cc0 0x7f90a40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.017+0000 7f90bcc00640 1 -- 172.21.15.46:0/3382673389 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f90ac0cf0a0 con 0x7f90c01019c0 2024-09-10T07:45:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.017+0000 7f90c9aa3740 1 -- 172.21.15.46:0/799811669 shutdown_connections 2024-09-10T07:45:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.017+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/799811669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f11b30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.017+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/799811669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04e80 0x55bd09f0f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.017+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/799811669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04560 0x55bd09f04940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.017+0000 7f90c9aa3740 1 -- 172.21.15.46:0/799811669 >> 172.21.15.46:0/799811669 conn(0x55bd09f00af0 msgr2=0x55bd09f00ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.017+0000 7f90c5000640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f90a40a2cc0 0x7f90a40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.018+0000 7f90c9aa3740 1 -- 172.21.15.46:0/799811669 shutdown_connections 2024-09-10T07:45:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.018+0000 7f90c9aa3740 1 -- 172.21.15.46:0/799811669 wait complete. 2024-09-10T07:45:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.019+0000 7f90c9aa3740 1 -- start start 2024-09-10T07:45:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.019+0000 7f90c9aa3740 1 Processor -- start 2024-09-10T07:45:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.020+0000 7f90c5000640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f90a40a2cc0 0x7f90a40a5180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009a60 tx=0x7f90b8005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.021+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.022+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.022+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04e80 0x55bd09f0cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.022+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.022+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bd09f12310 con 0x55bd09f04e80 2024-09-10T07:45:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.022+0000 7f90c5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.022+0000 7f90c5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33456/0 (socket says 172.21.15.46:33456) 2024-09-10T07:45:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.022+0000 7f90c5000640 1 -- 172.21.15.46:0/3303761274 learned_addr learned my addr 172.21.15.46:0/3303761274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.022+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41428/0 (socket says 172.21.15.46:41428) 2024-09-10T07:45:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.023+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bd09e24e60 con 0x55bd09f04560 2024-09-10T07:45:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.023+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd09f12490 con 0x55bd09f0f740 2024-09-10T07:45:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.023+0000 7f90bf400640 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04e80 0x55bd09f0cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.023+0000 7f90c9aa3740 10 client.0 _ll_register_callbacks cb 0x55bd09e29cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.024+0000 7f90bf400640 1 -- 172.21.15.46:0/3303761274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 msgr2=0x55bd09f0d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.024+0000 7f90bf400640 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.024+0000 7f90bf400640 1 -- 172.21.15.46:0/3303761274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 msgr2=0x55bd09f0c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.024+0000 7f90bf400640 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.024+0000 7f90bf400640 1 -- 172.21.15.46:0/3303761274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd09e30990 con 0x55bd09f04e80 2024-09-10T07:45:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.025+0000 7f90bf400640 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04e80 0x55bd09f0cbf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55bd09f01e70 tx=0x7f90b00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d626b61536610edf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.025+0000 7f90aa000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:45:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.026+0000 7f90aa000640 1 -- 172.21.15.46:0/3303761274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b002a060 con 0x55bd09f04e80 2024-09-10T07:45:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.026+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b00041d0 con 0x55bd09f04e80 2024-09-10T07:45:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.027+0000 7f90aa000640 1 -- 172.21.15.46:0/3303761274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f90b0004380 con 0x55bd09f04e80 2024-09-10T07:45:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.029+0000 7f90aa000640 1 -- 172.21.15.46:0/3303761274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f90b0009870 con 0x55bd09f04e80 2024-09-10T07:45:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.029+0000 7f90c9aa3740 10 client.16926 fetch_fsmap learned FSMap version 45 2024-09-10T07:45:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.029+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bd09f1dfa0 con 0x55bd09f04e80 2024-09-10T07:45:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.030+0000 7f90aa000640 1 -- 172.21.15.46:0/3303761274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b00041d0 con 0x55bd09f04e80 2024-09-10T07:45:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.030+0000 7f90aa000640 1 -- 172.21.15.46:0/3303761274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f90b000f040 con 0x55bd09f04e80 2024-09-10T07:45:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.031+0000 7f90c9aa3740 10 client.16926 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T07:45:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.031+0000 7f90c9aa3740 10 client.16926 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:45:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.031+0000 7f90c9aa3740 20 client.16926 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.031+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bd09f1f3b0 0x55bd09f217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.031+0000 7f90c9aa3740 4 client.16926 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.031+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd09e0d310 con 0x55bd09f1f3b0 2024-09-10T07:45:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.032+0000 7f90bfe00640 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bd09f1f3b0 0x55bd09f217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.036+0000 7f90bfe00640 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bd09f1f3b0 0x55bd09f217a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a8147d6b123e11e server_cookie=95593394d1308ccf in_seq=0 out_seq=0 2024-09-10T07:45:01.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.036+0000 7f90aa000640 10 client.16926 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90aa000640 1 -- 172.21.15.46:0/3303761274 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd09e0d310 con 0x55bd09f1f3b0 2024-09-10T07:45:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90aa000640 10 client.16926 handle_command_reply: tid=0 2024-09-10T07:45:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90c9aa3740 1 client.16926 shutdown 2024-09-10T07:45:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04e80 msgr2=0x55bd09f0cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04e80 0x55bd09f0cbf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55bd09f01e70 tx=0x7f90b00058b0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 shutdown_connections 2024-09-10T07:45:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bd09f1f3b0 0x55bd09f217a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90aa000640 0 client.16926 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f04e80 0x55bd09f0cbf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.040+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3303761274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.041+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 >> 172.21.15.46:0/3303761274 conn(0x55bd09f00af0 msgr2=0x55bd09f06820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.041+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 shutdown_connections 2024-09-10T07:45:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.043+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3303761274 wait complete. 2024-09-10T07:45:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.043+0000 7f90c9aa3740 20 client.16926 trim_cache size 0 max 16384 2024-09-10T07:45:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.086+0000 7f90c9aa3740 1 Processor -- start 2024-09-10T07:45:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.086+0000 7f90c9aa3740 1 -- start start 2024-09-10T07:45:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 0x55bd09f09dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01b51e0 con 0x55bd09f09990 2024-09-10T07:45:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01b6de0 con 0x55bd09f04560 2024-09-10T07:45:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01b58b0 con 0x55bd09f0f740 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90bf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90bf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33464/0 (socket says 172.21.15.46:33464) 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90bf400640 1 -- 172.21.15.46:0/1828369456 learned_addr learned my addr 172.21.15.46:0/1828369456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90bfe00640 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.087+0000 7f90c5000640 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 0x55bd09f09dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.088+0000 7f90bf400640 1 -- 172.21.15.46:0/1828369456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 msgr2=0x55bd09f0cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.088+0000 7f90bf400640 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.088+0000 7f90bf400640 1 -- 172.21.15.46:0/1828369456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 msgr2=0x55bd09f09dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.088+0000 7f90bf400640 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 0x55bd09f09dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.088+0000 7f90bf400640 1 -- 172.21.15.46:0/1828369456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd09f042e0 con 0x55bd09f0f740 2024-09-10T07:45:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.088+0000 7f90bfe00640 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.088+0000 7f90bf400640 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0e510 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009a30 tx=0x7f90b0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed3c5624738d73f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.089+0000 7f90b4c00640 1 -- 172.21.15.46:0/1828369456 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0034080 con 0x55bd09f0f740 2024-09-10T07:45:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.089+0000 7f90b4c00640 1 -- 172.21.15.46:0/1828369456 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b0003a20 con 0x55bd09f0f740 2024-09-10T07:45:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.089+0000 7f90b4c00640 1 -- 172.21.15.46:0/1828369456 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0018320 con 0x55bd09f0f740 2024-09-10T07:45:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.090+0000 7f90c9aa3740 1 -- 172.21.15.46:0/1828369456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 msgr2=0x55bd09f0e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.090+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0e510 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009a30 tx=0x7f90b0010740 comp rx=0 tx=0).stop 2024-09-10T07:45:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.091+0000 7f90c9aa3740 1 -- 172.21.15.46:0/1828369456 shutdown_connections 2024-09-10T07:45:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.091+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 0x55bd09f09dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.091+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f0f740 0x55bd09f0e510 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.091+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/1828369456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f04560 0x55bd09f0cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.091+0000 7f90c9aa3740 1 -- 172.21.15.46:0/1828369456 >> 172.21.15.46:0/1828369456 conn(0x55bd09f00af0 msgr2=0x55bd09f0de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:01.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.092+0000 7f90c9aa3740 1 -- 172.21.15.46:0/1828369456 shutdown_connections 2024-09-10T07:45:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.094+0000 7f90c9aa3740 1 -- 172.21.15.46:0/1828369456 wait complete. 2024-09-10T07:45:01.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.095+0000 7f90c9aa3740 1 -- start start 2024-09-10T07:45:01.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.095+0000 7f90c9aa3740 1 Processor -- start 2024-09-10T07:45:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.096+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04560 0x55bd09f22e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.096+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 0x55bd09f23370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.096+0000 7f90c9aa3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f82cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.096+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01b6650 con 0x55bd09f09990 2024-09-10T07:45:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.096+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01b5f80 con 0x55bd09f0f740 2024-09-10T07:45:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.096+0000 7f90c9aa3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01b3160 con 0x55bd09f04560 2024-09-10T07:45:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.096+0000 7f90c9aa3740 10 client.0 _ll_register_callbacks cb 0x7f90b0009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04560 0x55bd09f22e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04560 0x55bd09f22e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33476/0 (socket says 172.21.15.46:33476) 2024-09-10T07:45:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90bfe00640 1 -- 172.21.15.46:0/3100126674 learned_addr learned my addr 172.21.15.46:0/3100126674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90bf400640 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 0x55bd09f23370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90c5000640 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f82cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90c5000640 1 -- 172.21.15.46:0/3100126674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04560 msgr2=0x55bd09f22e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90c5000640 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04560 0x55bd09f22e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90c5000640 1 -- 172.21.15.46:0/3100126674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 msgr2=0x55bd09f23370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90c5000640 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 0x55bd09f23370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90c5000640 1 -- 172.21.15.46:0/3100126674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd09f83300 con 0x55bd09f0f740 2024-09-10T07:45:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90c5000640 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f82cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009ca0 tx=0x7f90b80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f8473a1a39fe40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.097+0000 7f90aa000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:01.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.098+0000 7f90aa000640 1 -- 172.21.15.46:0/3100126674 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b80043d0 con 0x55bd09f0f740 2024-09-10T07:45:01.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.098+0000 7f90aa000640 1 -- 172.21.15.46:0/3100126674 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f90b8004570 con 0x55bd09f0f740 2024-09-10T07:45:01.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.099+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3100126674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bd09e77c60 con 0x55bd09f0f740 2024-09-10T07:45:01.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.099+0000 7f90aa000640 1 -- 172.21.15.46:0/3100126674 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f90b8004770 con 0x55bd09f0f740 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.099+0000 7f90c9aa3740 10 client.27455 fetch_fsmap learned FSMap version 45 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.099+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3100126674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bd09f8a670 con 0x55bd09f0f740 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.099+0000 7f90aa000640 1 -- 172.21.15.46:0/3100126674 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bd09e77c60 con 0x55bd09f0f740 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.100+0000 7f90aa000640 1 -- 172.21.15.46:0/3100126674 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f90b805d5b0 con 0x55bd09f0f740 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.100+0000 7f90c9aa3740 10 client.27455 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.100+0000 7f90c9aa3740 10 client.27455 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.100+0000 7f90c9aa3740 20 client.27455 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.100+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bd09f8af70 0x55bd09f8d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.100+0000 7f90c9aa3740 4 client.27455 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-10T07:45:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.100+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3100126674 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x55bd09e71fb0 con 0x55bd09f8af70 2024-09-10T07:45:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.100+0000 7f90bfe00640 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bd09f8af70 0x55bd09f8d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.101+0000 7f90bfe00640 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bd09f8af70 0x55bd09f8d390 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95a15854d10a678 server_cookie=a23e82c4e2fecc5 in_seq=0 out_seq=0 2024-09-10T07:45:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.101+0000 7f90aa000640 10 client.27455 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.284+0000 7f90aa000640 1 -- 172.21.15.46:0/3100126674 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bd09e71fb0 con 0x55bd09f8af70 2024-09-10T07:45:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.284+0000 7f90aa000640 10 client.27455 handle_command_reply: tid=0 2024-09-10T07:45:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.285+0000 7f90c9aa3740 1 client.27455 shutdown 2024-09-10T07:45:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.285+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3100126674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 msgr2=0x55bd09f82cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:01.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.285+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f82cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009ca0 tx=0x7f90b80095c0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.289+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3100126674 shutdown_connections 2024-09-10T07:45:01.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.289+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bd09f8af70 0x55bd09f8d390 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.289+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd09f0f740 0x55bd09f82cc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.289+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd09f09990 0x55bd09f23370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.289+0000 7f90c9aa3740 1 --2- 172.21.15.46:0/3100126674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd09f04560 0x55bd09f22e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.289+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3100126674 >> 172.21.15.46:0/3100126674 conn(0x55bd09f00af0 msgr2=0x55bd09f08980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:01.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.289+0000 7f90aa000640 0 client.27455 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:01.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.289+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3100126674 shutdown_connections 2024-09-10T07:45:01.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.290+0000 7f90c9aa3740 1 -- 172.21.15.46:0/3100126674 wait complete. 2024-09-10T07:45:01.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.290+0000 7f90c9aa3740 20 client.27455 trim_cache size 0 max 16384 2024-09-10T07:45:01.290 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:45:01.290 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 45, 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 26, 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "26d3f25d": { 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 26, 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:45:01.291 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 133.4, 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:01.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:01.293 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:45:01.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 -- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f90a40a2cc0 msgr2=0x7f90a40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:01.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f90a40a2cc0 0x7f90a40a5180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009a60 tx=0x7f90b8005f80 comp rx=0 tx=0).stop 2024-09-10T07:45:01.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 -- 172.21.15.46:0/3382673389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c01019c0 msgr2=0x7f90c01106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:01.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c01019c0 0x7f90c01106b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00c8f0 tx=0x7f90ac00cdb0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 -- 172.21.15.46:0/3382673389 shutdown_connections 2024-09-10T07:45:01.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f90a40a2cc0 0x7f90a40a5180 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c01079c0 0x7f90c0111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0102360 0x7f90c0110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 --2- 172.21.15.46:0/3382673389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c01019c0 0x7f90c01106b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:01.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 -- 172.21.15.46:0/3382673389 >> 172.21.15.46:0/3382673389 conn(0x7f90c00fd740 msgr2=0x7f90c00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:01.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 -- 172.21.15.46:0/3382673389 shutdown_connections 2024-09-10T07:45:01.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:01.291+0000 7f90c5a00640 1 -- 172.21.15.46:0/3382673389 wait complete. 2024-09-10T07:45:01.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:01 smithi046 ceph-mon[30705]: pgmap v651: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 546 op/s 2024-09-10T07:45:02.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:01 smithi181 ceph-mon[35397]: pgmap v651: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 546 op/s 2024-09-10T07:45:02.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:01 smithi063 ceph-mon[35020]: pgmap v651: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 546 op/s 2024-09-10T07:45:02.932 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '26d3f25d', quiesce took 8.4 seconds. Will release after: 5.0 2024-09-10T07:45:04.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:03 smithi181 ceph-mon[35397]: pgmap v652: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 MiB/s wr, 559 op/s 2024-09-10T07:45:04.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:03 smithi063 ceph-mon[35020]: pgmap v652: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 MiB/s wr, 559 op/s 2024-09-10T07:45:04.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:03 smithi046 ceph-mon[30705]: pgmap v652: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 MiB/s wr, 559 op/s 2024-09-10T07:45:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 465.0 s, 67.9 tps, lat 158.367 ms stddev 142.103 2024-09-10T07:45:06.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:05 smithi181 ceph-mon[35397]: pgmap v653: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 MiB/s wr, 461 op/s 2024-09-10T07:45:06.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:45:06.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:45:06.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:45:06.150 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:05 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:45:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:05 smithi063 ceph-mon[35020]: pgmap v653: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 MiB/s wr, 461 op/s 2024-09-10T07:45:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:45:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:45:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:45:06.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:05 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:45:06.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:05 smithi046 ceph-mon[30705]: pgmap v653: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 MiB/s wr, 461 op/s 2024-09-10T07:45:06.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:45:06.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:45:06.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:45:06.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:05 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 -- 172.21.15.46:0/4108835698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 msgr2=0x7f28bc1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 --2- 172.21.15.46:0/4108835698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 0x7f28bc1053d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009900 tx=0x7f28a802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 -- 172.21.15.46:0/4108835698 shutdown_connections 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 --2- 172.21.15.46:0/4108835698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28bc105910 0x7f28bc10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 --2- 172.21.15.46:0/4108835698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 0x7f28bc1053d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 --2- 172.21.15.46:0/4108835698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28bc103d60 0x7f28bc104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 -- 172.21.15.46:0/4108835698 >> 172.21.15.46:0/4108835698 conn(0x7f28bc0ff4e0 msgr2=0x7f28bc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 -- 172.21.15.46:0/4108835698 shutdown_connections 2024-09-10T07:45:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.144+0000 7f28c3e00640 1 -- 172.21.15.46:0/4108835698 wait complete. 2024-09-10T07:45:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.145+0000 7f28c3e00640 1 Processor -- start 2024-09-10T07:45:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.145+0000 7f28c3e00640 1 -- start start 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.145+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28bc103d60 0x7f28bc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.145+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 0x7f28bc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.145+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28bc105910 0x7f28bc1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.145+0000 7f28c3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc10f9e0 con 0x7f28bc104f50 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.145+0000 7f28c3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc10f860 con 0x7f28bc105910 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.145+0000 7f28c3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc10fb60 con 0x7f28bc103d60 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 0x7f28bc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28bc105910 0x7f28bc1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28bc105910 0x7f28bc1a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47860/0 (socket says 172.21.15.46:47860) 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 0x7f28bc19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43092/0 (socket says 172.21.15.46:43092) 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1e00640 1 -- 172.21.15.46:0/3892214207 learned_addr learned my addr 172.21.15.46:0/3892214207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1400640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28bc103d60 0x7f28bc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1e00640 1 -- 172.21.15.46:0/3892214207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28bc103d60 msgr2=0x7f28bc199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28bc103d60 0x7f28bc199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1e00640 1 -- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 msgr2=0x7f28bc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 0x7f28bc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.146+0000 7f28c1e00640 1 -- 172.21.15.46:0/3892214207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc1a1bd0 con 0x7f28bc105910 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.147+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28bc105910 0x7f28bc1a1560 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f28b400eb10 tx=0x7f28b400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.147+0000 7f28ba000640 1 -- 172.21.15.46:0/3892214207 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4004b20 con 0x7f28bc105910 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.147+0000 7f28ba000640 1 -- 172.21.15.46:0/3892214207 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b4004cc0 con 0x7f28bc105910 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.147+0000 7f28c3e00640 1 -- 172.21.15.46:0/3892214207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28bc1a1ec0 con 0x7f28bc105910 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.147+0000 7f28ba000640 1 -- 172.21.15.46:0/3892214207 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4010760 con 0x7f28bc105910 2024-09-10T07:45:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.147+0000 7f28c3e00640 1 -- 172.21.15.46:0/3892214207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28bc1a2380 con 0x7f28bc105910 2024-09-10T07:45:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.150+0000 7f28c6e39740 1 Processor -- start 2024-09-10T07:45:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.150+0000 7f28ba000640 1 -- 172.21.15.46:0/3892214207 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f28b4005020 con 0x7f28bc105910 2024-09-10T07:45:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.150+0000 7f28c6e39740 1 -- start start 2024-09-10T07:45:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 0x55f9e9adf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ae1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c6e39740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f9e9a3f100 con 0x55f9e9ada090 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c6e39740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f9e99e71b0 con 0x55f9e9adfa60 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c6e39740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f9e9a3b0c0 con 0x55f9e9ada9d0 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43108/0 (socket says 172.21.15.46:43108) 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c0a00640 1 -- 172.21.15.46:0/2407083482 learned_addr learned my addr 172.21.15.46:0/2407083482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c1400640 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 0x55f9e9adf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.151+0000 7f28c0a00640 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ae1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28c0a00640 1 -- 172.21.15.46:0/2407083482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 msgr2=0x55f9e9adf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28c0a00640 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 0x55f9e9adf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28c0a00640 1 -- 172.21.15.46:0/2407083482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 msgr2=0x55f9e9ae1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28c0a00640 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ae1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28c0a00640 1 -- 172.21.15.46:0/2407083482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e9752210 con 0x55f9e9ada090 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28ba000640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a40a2cb0 0x7f28a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28c0a00640 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada490 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f28a8005ec0 tx=0x7f28a8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b4c9ae1d7712bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28ba000640 1 -- 172.21.15.46:0/3892214207 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28b40cf950 con 0x7f28bc105910 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28c1400640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a40a2cb0 0x7f28a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28ae600640 1 -- 172.21.15.46:0/2407083482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8035040 con 0x55f9e9ada090 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28ae600640 1 -- 172.21.15.46:0/2407083482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a8005510 con 0x55f9e9ada090 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.152+0000 7f28ae600640 1 -- 172.21.15.46:0/2407083482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8004a30 con 0x55f9e9ada090 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 -- 172.21.15.46:0/2407083482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 msgr2=0x55f9e9ada490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c1400640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a40a2cb0 0x7f28a40a5170 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55f9e9adb400 tx=0x7f28b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada490 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f28a8005ec0 tx=0x7f28a8005fd0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 -- 172.21.15.46:0/2407083482 shutdown_connections 2024-09-10T07:45:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ae1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 0x55f9e9adf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 --2- 172.21.15.46:0/2407083482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada490 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 -- 172.21.15.46:0/2407083482 >> 172.21.15.46:0/2407083482 conn(0x55f9e9ad5700 msgr2=0x55f9e9ad5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 -- 172.21.15.46:0/2407083482 shutdown_connections 2024-09-10T07:45:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.153+0000 7f28c6e39740 1 -- 172.21.15.46:0/2407083482 wait complete. 2024-09-10T07:45:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 1 -- start start 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 1 Processor -- start 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada090 0x55f9e9ae4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 0x55f9e9ae4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9aeb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f9e9ae29d0 con 0x55f9e9ada9d0 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f9e99c8090 con 0x55f9e9adfa60 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f9e9ae2b50 con 0x55f9e9ada090 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.154+0000 7f28c6e39740 10 client.0 _ll_register_callbacks cb 0x55f9e9a1cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 0x55f9e9ae4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9aeb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada090 0x55f9e9ae4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9aeb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47878/0 (socket says 172.21.15.46:47878) 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 -- 172.21.15.46:0/3078032415 learned_addr learned my addr 172.21.15.46:0/3078032415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 -- 172.21.15.46:0/3078032415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada090 msgr2=0x55f9e9ae4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada090 0x55f9e9ae4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 -- 172.21.15.46:0/3078032415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 msgr2=0x55f9e9ae4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 0x55f9e9ae4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 -- 172.21.15.46:0/3078032415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e981e620 con 0x55f9e9adfa60 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9aeb7d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f28b40a4510 tx=0x7f28b409b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a06b7c1d5fb7b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.155+0000 7f28a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28a2a00640 1 -- 172.21.15.46:0/3078032415 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b40a47b0 con 0x55f9e9adfa60 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28a2a00640 1 -- 172.21.15.46:0/3078032415 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28b4005310 con 0x55f9e9adfa60 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 1 -- 172.21.15.46:0/3078032415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28a8005510 con 0x55f9e9adfa60 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28a2a00640 1 -- 172.21.15.46:0/3078032415 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f28b40a0180 con 0x55f9e9adfa60 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 10 client.27467 fetch_fsmap learned FSMap version 45 2024-09-10T07:45:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 1 -- 172.21.15.46:0/3078032415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f9e9aee690 con 0x55f9e9adfa60 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28a2a00640 1 -- 172.21.15.46:0/3078032415 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a8005510 con 0x55f9e9adfa60 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28a2a00640 1 -- 172.21.15.46:0/3078032415 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f28b40b1070 con 0x55f9e9adfa60 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 10 client.27467 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 10 client.27467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 20 client.27467 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f9e9aefdb0 0x55f9e9af21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 4 client.27467 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.156+0000 7f28c6e39740 1 -- 172.21.15.46:0/3078032415 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9e9a4c830 con 0x55f9e9aefdb0 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.157+0000 7f28c1400640 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f9e9aefdb0 0x55f9e9af21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.157+0000 7f28c1400640 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f9e9aefdb0 0x55f9e9af21f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6139fc57c6e563c4 server_cookie=67e6aede142bd8fe in_seq=0 out_seq=0 2024-09-10T07:45:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.157+0000 7f28a2a00640 10 client.27467 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28a2a00640 1 -- 172.21.15.46:0/3078032415 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9e9a4c830 con 0x55f9e9aefdb0 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28a2a00640 10 client.27467 handle_command_reply: tid=0 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 client.27467 shutdown 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 -- 172.21.15.46:0/3078032415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 msgr2=0x55f9e9aeb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9aeb7d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f28b40a4510 tx=0x7f28b409b730 comp rx=0 tx=0).stop 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 -- 172.21.15.46:0/3078032415 shutdown_connections 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f9e9aefdb0 0x55f9e9af21f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9aeb7d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 0x55f9e9ae4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28a2a00640 0 client.27467 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3078032415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada090 0x55f9e9ae4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.158+0000 7f28c6e39740 1 -- 172.21.15.46:0/3078032415 >> 172.21.15.46:0/3078032415 conn(0x55f9e9ad5700 msgr2=0x55f9e9ad6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.159+0000 7f28c6e39740 1 -- 172.21.15.46:0/3078032415 shutdown_connections 2024-09-10T07:45:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.159+0000 7f28c6e39740 1 -- 172.21.15.46:0/3078032415 wait complete. 2024-09-10T07:45:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.159+0000 7f28c6e39740 20 client.27467 trim_cache size 0 max 16384 2024-09-10T07:45:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.174+0000 7f28c6e39740 1 Processor -- start 2024-09-10T07:45:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.174+0000 7f28c6e39740 1 -- start start 2024-09-10T07:45:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.174+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.174+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 0x55f9e9aeb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.174+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ad65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.174+0000 7f28c6e39740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1ac820 con 0x55f9e9ada090 2024-09-10T07:45:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.174+0000 7f28c6e39740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1ae420 con 0x55f9e9adfa60 2024-09-10T07:45:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.174+0000 7f28c6e39740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1acef0 con 0x55f9e9ada9d0 2024-09-10T07:45:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ad65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43126/0 (socket says 172.21.15.46:43126) 2024-09-10T07:45:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1400640 1 -- 172.21.15.46:0/3034714416 learned_addr learned my addr 172.21.15.46:0/3034714416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c0a00640 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 0x55f9e9aeb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1e00640 1 -- 172.21.15.46:0/3034714416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 msgr2=0x55f9e9aeb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 0x55f9e9aeb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1e00640 1 -- 172.21.15.46:0/3034714416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 msgr2=0x55f9e9ada470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1e00640 1 -- 172.21.15.46:0/3034714416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e9adf800 con 0x55f9e9adfa60 2024-09-10T07:45:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1400640 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ad65b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f28b40049d0 tx=0x7f28b409bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0cc0b3364f02a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.175+0000 7f28adc00640 1 -- 172.21.15.46:0/3034714416 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4098070 con 0x55f9e9adfa60 2024-09-10T07:45:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28adc00640 1 -- 172.21.15.46:0/3034714416 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b40a4a40 con 0x55f9e9adfa60 2024-09-10T07:45:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28adc00640 1 -- 172.21.15.46:0/3034714416 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4005310 con 0x55f9e9adfa60 2024-09-10T07:45:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 -- 172.21.15.46:0/3034714416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 msgr2=0x55f9e9ad65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ad65b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f28b40049d0 tx=0x7f28b409bf50 comp rx=0 tx=0).stop 2024-09-10T07:45:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 -- 172.21.15.46:0/3034714416 shutdown_connections 2024-09-10T07:45:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9adfa60 0x55f9e9ad65b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9ada9d0 0x55f9e9aeb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3034714416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada090 0x55f9e9ada470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 -- 172.21.15.46:0/3034714416 >> 172.21.15.46:0/3034714416 conn(0x55f9e9ad5700 msgr2=0x55f9e9ae4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 -- 172.21.15.46:0/3034714416 shutdown_connections 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.176+0000 7f28c6e39740 1 -- 172.21.15.46:0/3034714416 wait complete. 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 1 -- start start 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 1 Processor -- start 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9ada090 0x55f9e9ae8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 0x55f9e9ae91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9adfa60 0x55f9e9aea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1adc90 con 0x55f9e9ada9d0 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1ad5c0 con 0x55f9e9ada090 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1aa800 con 0x55f9e9adfa60 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.177+0000 7f28c6e39740 10 client.0 _ll_register_callbacks cb 0x7f28b409bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9ada090 0x55f9e9ae8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 0x55f9e9ae91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9ada090 0x55f9e9ae8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47892/0 (socket says 172.21.15.46:47892) 2024-09-10T07:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 -- 172.21.15.46:0/3731279179 learned_addr learned my addr 172.21.15.46:0/3731279179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1e00640 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9adfa60 0x55f9e9aea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 -- 172.21.15.46:0/3731279179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9adfa60 msgr2=0x55f9e9aea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9adfa60 0x55f9e9aea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 -- 172.21.15.46:0/3731279179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 msgr2=0x55f9e9ae91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 0x55f9e9ae91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 -- 172.21.15.46:0/3731279179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e9aea730 con 0x55f9e9ada090 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28c1400640 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9ada090 0x55f9e9ae8c70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f28b0004470 tx=0x7f28b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3748a1e8a5d9c45c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.178+0000 7f28a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28a2a00640 1 -- 172.21.15.46:0/3731279179 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0003bb0 con 0x55f9e9ada090 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 1 -- 172.21.15.46:0/3731279179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b40a4a40 con 0x55f9e9ada090 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28a2a00640 1 -- 172.21.15.46:0/3731279179 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28b0003d50 con 0x55f9e9ada090 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28a2a00640 1 -- 172.21.15.46:0/3731279179 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f28b0005020 con 0x55f9e9ada090 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 10 client.27473 fetch_fsmap learned FSMap version 45 2024-09-10T07:45:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 1 -- 172.21.15.46:0/3731279179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f9e9b5aa90 con 0x55f9e9ada090 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28a2a00640 1 -- 172.21.15.46:0/3731279179 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b40a4a40 con 0x55f9e9ada090 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28a2a00640 1 -- 172.21.15.46:0/3731279179 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f28b002b380 con 0x55f9e9ada090 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 10 client.27473 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 10 client.27473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 20 client.27473 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f9e9b5b380 0x55f9e9b5d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 4 client.27473 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.179+0000 7f28c6e39740 1 -- 172.21.15.46:0/3731279179 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9e9ad0d80 con 0x55f9e9b5b380 2024-09-10T07:45:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.180+0000 7f28c0a00640 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f9e9b5b380 0x55f9e9b5d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.180+0000 7f28c0a00640 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f9e9b5b380 0x55f9e9b5d7a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f61d79acb9cc8199 server_cookie=5106ad8ed6bfb20c in_seq=0 out_seq=0 2024-09-10T07:45:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.180+0000 7f28a2a00640 10 client.27473 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28a2a00640 1 -- 172.21.15.46:0/3731279179 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9e9ad0d80 con 0x55f9e9b5b380 2024-09-10T07:45:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28a2a00640 10 client.27473 handle_command_reply: tid=0 2024-09-10T07:45:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 client.27473 shutdown 2024-09-10T07:45:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 -- 172.21.15.46:0/3731279179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9ada090 msgr2=0x55f9e9ae8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9ada090 0x55f9e9ae8c70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f28b0004470 tx=0x7f28b00046c0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 -- 172.21.15.46:0/3731279179 shutdown_connections 2024-09-10T07:45:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f9e9b5b380 0x55f9e9b5d7a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f9e9adfa60 0x55f9e9aea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9e9ada9d0 0x55f9e9ae91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 --2- 172.21.15.46:0/3731279179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e9ada090 0x55f9e9ae8c70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28a2a00640 0 client.27473 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.188+0000 7f28c6e39740 1 -- 172.21.15.46:0/3731279179 >> 172.21.15.46:0/3731279179 conn(0x55f9e9ad5700 msgr2=0x55f9e9ad4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.189+0000 7f28c6e39740 1 -- 172.21.15.46:0/3731279179 shutdown_connections 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.189+0000 7f28c6e39740 1 -- 172.21.15.46:0/3731279179 wait complete. 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.189+0000 7f28c6e39740 20 client.27473 trim_cache size 0 max 16384 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:45:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:45:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.189+0000 7f28c3e00640 1 -- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a40a2cb0 msgr2=0x7f28a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.189+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a40a2cb0 0x7f28a40a5170 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55f9e9adb400 tx=0x7f28b0005f90 comp rx=0 tx=0).stop 2024-09-10T07:45:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.189+0000 7f28c3e00640 1 -- 172.21.15.46:0/3892214207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28bc105910 msgr2=0x7f28bc1a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.189+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28bc105910 0x7f28bc1a1560 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f28b400eb10 tx=0x7f28b400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.190+0000 7f28c3e00640 1 -- 172.21.15.46:0/3892214207 shutdown_connections 2024-09-10T07:45:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.190+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28bc105910 0x7f28bc1a1560 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.190+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28a40a2cb0 0x7f28a40a5170 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.190+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28bc104f50 0x7f28bc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.190+0000 7f28c3e00640 1 --2- 172.21.15.46:0/3892214207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28bc103d60 0x7f28bc199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.190+0000 7f28c3e00640 1 -- 172.21.15.46:0/3892214207 >> 172.21.15.46:0/3892214207 conn(0x7f28bc0ff4e0 msgr2=0x7f28bc108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.190+0000 7f28c3e00640 1 -- 172.21.15.46:0/3892214207 shutdown_connections 2024-09-10T07:45:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:07.190+0000 7f28c3e00640 1 -- 172.21.15.46:0/3892214207 wait complete. 2024-09-10T07:45:07.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:45:08.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:07 smithi181 ceph-mon[35397]: pgmap v654: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-10T07:45:08.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:07 smithi063 ceph-mon[35020]: pgmap v654: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-10T07:45:08.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:07 smithi046 ceph-mon[30705]: pgmap v654: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-10T07:45:08.329 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 -- 172.21.15.46:0/1173042272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60112a50 msgr2=0x7ffb60114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 --2- 172.21.15.46:0/1173042272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60112a50 0x7ffb60114e40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009ca0 tx=0x7ffb5802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 -- 172.21.15.46:0/1173042272 shutdown_connections 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 --2- 172.21.15.46:0/1173042272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60112a50 0x7ffb60114e40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 --2- 172.21.15.46:0/1173042272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb60077040 0x7ffb60112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 --2- 172.21.15.46:0/1173042272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60076720 0x7ffb60076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 -- 172.21.15.46:0/1173042272 >> 172.21.15.46:0/1173042272 conn(0x7ffb600ffa50 msgr2=0x7ffb60101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 -- 172.21.15.46:0/1173042272 shutdown_connections 2024-09-10T07:45:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.929+0000 7ffb68400640 1 -- 172.21.15.46:0/1173042272 wait complete. 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.930+0000 7ffb68400640 1 Processor -- start 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.930+0000 7ffb68400640 1 -- start start 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb68400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60076720 0x7ffb6019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb68400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb60077040 0x7ffb6019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb68400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60112a50 0x7ffb601a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb68400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601185f0 con 0x7ffb60076720 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb68400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60118470 con 0x7ffb60077040 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb68400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60118770 con 0x7ffb60112a50 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb65a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60076720 0x7ffb6019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb65a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60076720 0x7ffb6019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43150/0 (socket says 172.21.15.46:43150) 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb65a00640 1 -- 172.21.15.46:0/1323145945 learned_addr learned my addr 172.21.15.46:0/1323145945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb66400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60112a50 0x7ffb601a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.931+0000 7ffb65000640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb60077040 0x7ffb6019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb65a00640 1 -- 172.21.15.46:0/1323145945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60112a50 msgr2=0x7ffb601a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb65a00640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60112a50 0x7ffb601a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb65a00640 1 -- 172.21.15.46:0/1323145945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb60077040 msgr2=0x7ffb6019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb65a00640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb60077040 0x7ffb6019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb65a00640 1 -- 172.21.15.46:0/1323145945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb601a3bc0 con 0x7ffb60076720 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb65a00640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60076720 0x7ffb6019eb90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9f0 tx=0x7ffb5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb5e000640 1 -- 172.21.15.46:0/1323145945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400cd80 con 0x7ffb60076720 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb601a3eb0 con 0x7ffb60076720 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb5e000640 1 -- 172.21.15.46:0/1323145945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb54005500 con 0x7ffb60076720 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb5e000640 1 -- 172.21.15.46:0/1323145945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54010690 con 0x7ffb60076720 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.932+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb601ab740 con 0x7ffb60076720 2024-09-10T07:45:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.933+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb60076b00 con 0x7ffb60076720 2024-09-10T07:45:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.935+0000 7ffb5e000640 1 -- 172.21.15.46:0/1323145945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb540046c0 con 0x7ffb60076720 2024-09-10T07:45:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.937+0000 7ffb5e000640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffb480a2c00 0x7ffb480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.937+0000 7ffb5e000640 1 -- 172.21.15.46:0/1323145945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ffb540cf7a0 con 0x7ffb60076720 2024-09-10T07:45:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.937+0000 7ffb65000640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffb480a2c00 0x7ffb480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.938+0000 7ffb65000640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffb480a2c00 0x7ffb480a50c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffb60079360 tx=0x7ffb4c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:08.941+0000 7ffb5e000640 1 -- 172.21.15.46:0/1323145945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb5409c320 con 0x7ffb60076720 2024-09-10T07:45:09.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.163+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb6010dbe0 con 0x7ffb60076720 2024-09-10T07:45:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.164+0000 7ffb5e000640 1 -- 172.21.15.46:0/1323145945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11979 (secure 0 0 0) 0x7ffb54086020 con 0x7ffb60076720 2024-09-10T07:45:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:45:09.164 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"btime":"2024-09-10T07:44:53:148560+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:44:53.049151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:45:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 45 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffb480a2c00 msgr2=0x7ffb480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffb480a2c00 0x7ffb480a50c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffb60079360 tx=0x7ffb4c00b410 comp rx=0 tx=0).stop 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60076720 msgr2=0x7ffb6019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60076720 0x7ffb6019eb90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9f0 tx=0x7ffb5400edf0 comp rx=0 tx=0).stop 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 shutdown_connections 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60112a50 0x7ffb601a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb60077040 0x7ffb6019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffb480a2c00 0x7ffb480a50c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 --2- 172.21.15.46:0/1323145945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb60076720 0x7ffb6019eb90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 >> 172.21.15.46:0/1323145945 conn(0x7ffb600ffa50 msgr2=0x7ffb60101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 shutdown_connections 2024-09-10T07:45:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:09.166+0000 7ffb68400640 1 -- 172.21.15.46:0/1323145945 wait complete. 2024-09-10T07:45:09.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:45:10.140 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:09 smithi046 ceph-mon[30705]: pgmap v655: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-09-10T07:45:10.140 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:09 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1323145945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:10.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:09 smithi181 ceph-mon[35397]: pgmap v655: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-09-10T07:45:10.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:09 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1323145945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:10.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:09 smithi063 ceph-mon[35020]: pgmap v655: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2024-09-10T07:45:10.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:09 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1323145945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:10.179 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id 26d3f25d --release --await' 2024-09-10T07:45:10.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id 26d3f25d --release --await 2024-09-10T07:45:10.262 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:10.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 470.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:45:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 -- 172.21.15.46:0/588942947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0109120 msgr2=0x7fd5a0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/588942947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0109120 0x7fd5a0110ca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ca0 tx=0x7fd59802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 -- 172.21.15.46:0/588942947 shutdown_connections 2024-09-10T07:45:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/588942947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0109120 0x7fd5a0110ca0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/588942947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a01049a0 0x7fd5a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/588942947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a0103ff0 0x7fd5a01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 -- 172.21.15.46:0/588942947 >> 172.21.15.46:0/588942947 conn(0x7fd5a00ffa90 msgr2=0x7fd5a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 -- 172.21.15.46:0/588942947 shutdown_connections 2024-09-10T07:45:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.884+0000 7fd5a5e00640 1 -- 172.21.15.46:0/588942947 wait complete. 2024-09-10T07:45:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.885+0000 7fd5a5e00640 1 Processor -- start 2024-09-10T07:45:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd5a5e00640 1 -- start start 2024-09-10T07:45:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd5a5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a0103ff0 0x7fd5a019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd5a5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a01049a0 0x7fd5a019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd5a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0109120 0x7fd5a019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd5a5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0114150 con 0x7fd5a0109120 2024-09-10T07:45:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd5a5e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0113fd0 con 0x7fd5a0103ff0 2024-09-10T07:45:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd5a5e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01142d0 con 0x7fd5a01049a0 2024-09-10T07:45:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd59f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0109120 0x7fd5a019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a0103ff0 0x7fd5a019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a0103ff0 0x7fd5a019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47928/0 (socket says 172.21.15.46:47928) 2024-09-10T07:45:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd59ea00640 1 -- 172.21.15.46:0/3023292499 learned_addr learned my addr 172.21.15.46:0/3023292499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.886+0000 7fd59e000640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a01049a0 0x7fd5a019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.887+0000 7fd59ea00640 1 -- 172.21.15.46:0/3023292499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a01049a0 msgr2=0x7fd5a019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.887+0000 7fd59ea00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a01049a0 0x7fd5a019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.887+0000 7fd59ea00640 1 -- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0109120 msgr2=0x7fd5a019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.887+0000 7fd59ea00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0109120 0x7fd5a019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.887+0000 7fd59ea00640 1 -- 172.21.15.46:0/3023292499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a019f5e0 con 0x7fd5a0103ff0 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.887+0000 7fd59ea00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a0103ff0 0x7fd5a019a630 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e9f0 tx=0x7fd59000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.887+0000 7fd5a4a00640 1 -- 172.21.15.46:0/3023292499 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59000cd80 con 0x7fd5a0103ff0 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.888+0000 7fd5a5e00640 1 -- 172.21.15.46:0/3023292499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a019f8d0 con 0x7fd5a0103ff0 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.888+0000 7fd5a4a00640 1 -- 172.21.15.46:0/3023292499 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd590005500 con 0x7fd5a0103ff0 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.888+0000 7fd5a4a00640 1 -- 172.21.15.46:0/3023292499 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590010690 con 0x7fd5a0103ff0 2024-09-10T07:45:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.888+0000 7fd5a5e00640 1 -- 172.21.15.46:0/3023292499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01a71b0 con 0x7fd5a0103ff0 2024-09-10T07:45:10.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.891+0000 7fd5a8a15740 1 Processor -- start 2024-09-10T07:45:10.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd5a8a15740 1 -- start start 2024-09-10T07:45:10.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c551b90 0x55c26c551f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c55c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c55cd70 0x55c26c55f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c26c4b83e0 con 0x55c26c551b90 2024-09-10T07:45:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c26c45fd80 con 0x55c26c5524b0 2024-09-10T07:45:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c26c4b43a0 con 0x55c26c55cd70 2024-09-10T07:45:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd5a4a00640 1 -- 172.21.15.46:0/3023292499 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5900046c0 con 0x7fd5a0103ff0 2024-09-10T07:45:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd59e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c55cd70 0x55c26c55f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd59f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c551b90 0x55c26c551f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd59e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c55cd70 0x55c26c55f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:41642/0 (socket says 172.21.15.46:41642) 2024-09-10T07:45:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.892+0000 7fd59e000640 1 -- 172.21.15.46:0/2154059890 learned_addr learned my addr 172.21.15.46:0/2154059890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.893+0000 7fd59ea00640 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c55c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.893+0000 7fd59e000640 1 -- 172.21.15.46:0/2154059890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 msgr2=0x55c26c55c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.893+0000 7fd59e000640 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c55c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.893+0000 7fd59e000640 1 -- 172.21.15.46:0/2154059890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c551b90 msgr2=0x55c26c551f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.893+0000 7fd59e000640 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c551b90 0x55c26c551f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.893+0000 7fd59e000640 1 -- 172.21.15.46:0/2154059890 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c26c47eae0 con 0x55c26c55cd70 2024-09-10T07:45:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.893+0000 7fd59ea00640 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c55c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.894+0000 7fd59e000640 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c55cd70 0x55c26c55f160 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009c10 tx=0x7fd58c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f84f40785850fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.894+0000 7fd58be00640 1 -- 172.21.15.46:0/2154059890 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004230 con 0x55c26c55cd70 2024-09-10T07:45:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.894+0000 7fd58be00640 1 -- 172.21.15.46:0/2154059890 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c0043d0 con 0x55c26c55cd70 2024-09-10T07:45:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.894+0000 7fd58be00640 1 -- 172.21.15.46:0/2154059890 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004e00 con 0x55c26c55cd70 2024-09-10T07:45:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.894+0000 7fd5a4a00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5840a2b80 0x7fd5840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.894+0000 7fd5a4a00640 1 -- 172.21.15.46:0/3023292499 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd5900cf7a0 con 0x7fd5a0103ff0 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.895+0000 7fd5a8a15740 1 -- 172.21.15.46:0/2154059890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c55cd70 msgr2=0x55c26c55f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.895+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c55cd70 0x55c26c55f160 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009c10 tx=0x7fd58c011810 comp rx=0 tx=0).stop 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.895+0000 7fd59f400640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5840a2b80 0x7fd5840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.895+0000 7fd5a8a15740 1 -- 172.21.15.46:0/2154059890 shutdown_connections 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.895+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c55cd70 0x55c26c55f160 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.895+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c55c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.895+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/2154059890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c551b90 0x55c26c551f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.895+0000 7fd5a8a15740 1 -- 172.21.15.46:0/2154059890 >> 172.21.15.46:0/2154059890 conn(0x55c26c54e120 msgr2=0x55c26c54e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.896+0000 7fd5a8a15740 1 -- 172.21.15.46:0/2154059890 shutdown_connections 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.896+0000 7fd59f400640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5840a2b80 0x7fd5840a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd598004af0 tx=0x7fd598004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.896+0000 7fd5a8a15740 1 -- 172.21.15.46:0/2154059890 wait complete. 2024-09-10T07:45:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 1 -- start start 2024-09-10T07:45:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 1 Processor -- start 2024-09-10T07:45:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c559b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c5524b0 0x55c26c55a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c55cd70 0x55c26c55a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c26c55f9e0 con 0x55c26c5524b0 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c26c4730f0 con 0x55c26c55cd70 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c26c55fb60 con 0x55c26c551b90 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.897+0000 7fd5a8a15740 10 client.0 _ll_register_callbacks cb 0x55c26c4649e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c55cd70 0x55c26c55a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c559b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c559b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:41648/0 (socket says 172.21.15.46:41648) 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59ea00640 1 -- 172.21.15.46:0/1979143645 learned_addr learned my addr 172.21.15.46:0/1979143645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c55cd70 0x55c26c55a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47944/0 (socket says 172.21.15.46:47944) 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59e000640 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c5524b0 0x55c26c55a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59ea00640 1 -- 172.21.15.46:0/1979143645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c55cd70 msgr2=0x55c26c55a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59ea00640 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c55cd70 0x55c26c55a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59ea00640 1 -- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c5524b0 msgr2=0x55c26c55a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59ea00640 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c5524b0 0x55c26c55a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59ea00640 1 -- 172.21.15.46:0/1979143645 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c26c2293c0 con 0x55c26c551b90 2024-09-10T07:45:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.898+0000 7fd59e000640 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c5524b0 0x55c26c55a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd59ea00640 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c559b60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c6a0 tx=0x7fd59009b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495d9284970779a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd588c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd588c00640 1 -- 172.21.15.46:0/1979143645 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900a4660 con 0x55c26c551b90 2024-09-10T07:45:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd588c00640 1 -- 172.21.15.46:0/1979143645 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd59009b0d0 con 0x55c26c551b90 2024-09-10T07:45:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1979143645 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58c0043d0 con 0x55c26c551b90 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd588c00640 1 -- 172.21.15.46:0/1979143645 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd5900a0030 con 0x55c26c551b90 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd5a8a15740 10 client.27442 fetch_fsmap learned FSMap version 46 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1979143645 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c26c56b500 con 0x55c26c551b90 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.899+0000 7fd588c00640 1 -- 172.21.15.46:0/1979143645 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c0043d0 con 0x55c26c551b90 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.900+0000 7fd588c00640 1 -- 172.21.15.46:0/1979143645 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 46) ==== 8378+0+0 (secure 0 0 0) 0x7fd5900b4020 con 0x55c26c551b90 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.900+0000 7fd5a8a15740 10 client.27442 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.900+0000 7fd5a8a15740 10 client.27442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.900+0000 7fd5a8a15740 20 client.27442 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.900+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c26c56c9c0 0x55c26c56ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.900+0000 7fd5a8a15740 4 client.27442 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.900+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1979143645 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c26c4c5950 con 0x55c26c56c9c0 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.900+0000 7fd59e000640 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c26c56c9c0 0x55c26c56ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.901+0000 7fd59e000640 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c26c56c9c0 0x55c26c56ee30 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7065848af02d2cb7 server_cookie=5362cf8d2ac681e3 in_seq=0 out_seq=0 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.901+0000 7fd588c00640 10 client.27442 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd588c00640 1 -- 172.21.15.46:0/1979143645 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c26c4c5950 con 0x55c26c56c9c0 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd588c00640 10 client.27442 handle_command_reply: tid=0 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 client.27442 shutdown 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1979143645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 msgr2=0x55c26c559b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c559b60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c6a0 tx=0x7fd59009b5e0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1979143645 shutdown_connections 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c26c56c9c0 0x55c26c56ee30 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c55cd70 0x55c26c55a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c5524b0 0x55c26c55a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1979143645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c559b60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd588c00640 0 client.27442 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1979143645 >> 172.21.15.46:0/1979143645 conn(0x55c26c54e120 msgr2=0x55c26c553e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1979143645 shutdown_connections 2024-09-10T07:45:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1979143645 wait complete. 2024-09-10T07:45:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.902+0000 7fd5a8a15740 20 client.27442 trim_cache size 0 max 16384 2024-09-10T07:45:10.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.923+0000 7fd5a8a15740 1 Processor -- start 2024-09-10T07:45:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.923+0000 7fd5a8a15740 1 -- start start 2024-09-10T07:45:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.923+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c551b90 0x55c26c55be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.923+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c5524b0 0x55c26c559440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.923+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 0x55c26c574700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.923+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b0dc0 con 0x55c26c55cd70 2024-09-10T07:45:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.923+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b2960 con 0x55c26c551b90 2024-09-10T07:45:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.923+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b1490 con 0x55c26c5524b0 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c551b90 0x55c26c55be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c5524b0 0x55c26c559440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c551b90 0x55c26c55be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47948/0 (socket says 172.21.15.46:47948) 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 -- 172.21.15.46:0/1267443232 learned_addr learned my addr 172.21.15.46:0/1267443232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59f400640 1 --2- 172.21.15.46:0/1267443232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 0x55c26c574700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 -- 172.21.15.46:0/1267443232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c5524b0 msgr2=0x55c26c559440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 --2- 172.21.15.46:0/1267443232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c5524b0 0x55c26c559440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 -- 172.21.15.46:0/1267443232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 msgr2=0x55c26c574700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 --2- 172.21.15.46:0/1267443232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 0x55c26c574700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 -- 172.21.15.46:0/1267443232 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c26c54df50 con 0x55c26c551b90 2024-09-10T07:45:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.924+0000 7fd59ea00640 1 --2- 172.21.15.46:0/1267443232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c551b90 0x55c26c55be90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd590099f30 tx=0x7fd59009be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751867e6d63afe21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.925+0000 7fd58b400640 1 -- 172.21.15.46:0/1267443232 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59009aec0 con 0x55c26c551b90 2024-09-10T07:45:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.925+0000 7fd58b400640 1 -- 172.21.15.46:0/1267443232 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5900a48f0 con 0x55c26c551b90 2024-09-10T07:45:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.925+0000 7fd58b400640 1 -- 172.21.15.46:0/1267443232 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59009b330 con 0x55c26c551b90 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.925+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1267443232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c551b90 msgr2=0x55c26c55be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.925+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1267443232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c551b90 0x55c26c55be90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd590099f30 tx=0x7fd59009be00 comp rx=0 tx=0).stop 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.926+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1267443232 shutdown_connections 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.926+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1267443232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 0x55c26c574700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.926+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1267443232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c5524b0 0x55c26c559440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.926+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/1267443232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c551b90 0x55c26c55be90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.926+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1267443232 >> 172.21.15.46:0/1267443232 conn(0x55c26c54e120 msgr2=0x55c26c54d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.926+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1267443232 shutdown_connections 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.926+0000 7fd5a8a15740 1 -- 172.21.15.46:0/1267443232 wait complete. 2024-09-10T07:45:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.927+0000 7fd5a8a15740 1 -- start start 2024-09-10T07:45:10.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.927+0000 7fd5a8a15740 1 Processor -- start 2024-09-10T07:45:10.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.927+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c5652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c5565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd5a8a15740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 0x55c26c556b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b2230 con 0x55c26c55cd70 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b1b60 con 0x55c26c5524b0 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd5a8a15740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01aeda0 con 0x55c26c551b90 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd5a8a15740 10 client.0 _ll_register_callbacks cb 0x7fd59009ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c5652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c5565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c5565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47960/0 (socket says 172.21.15.46:47960) 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59e000640 1 -- 172.21.15.46:0/706519269 learned_addr learned my addr 172.21.15.46:0/706519269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:10.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59f400640 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 0x55c26c556b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59ea00640 1 -- 172.21.15.46:0/706519269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 msgr2=0x55c26c5565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59ea00640 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c5565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59ea00640 1 -- 172.21.15.46:0/706519269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 msgr2=0x55c26c556b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59ea00640 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 0x55c26c556b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.928+0000 7fd59ea00640 1 -- 172.21.15.46:0/706519269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c26c5570c0 con 0x55c26c551b90 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.929+0000 7fd59ea00640 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c5652b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ccf70 tx=0x7fd59009bb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56aaa16dddef534b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.929+0000 7fd588c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.929+0000 7fd588c00640 1 -- 172.21.15.46:0/706519269 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59009b2d0 con 0x55c26c551b90 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.929+0000 7fd588c00640 1 -- 172.21.15.46:0/706519269 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd5900979d0 con 0x55c26c551b90 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.929+0000 7fd5a8a15740 1 -- 172.21.15.46:0/706519269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5900a48f0 con 0x55c26c551b90 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.929+0000 7fd588c00640 1 -- 172.21.15.46:0/706519269 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd5900a4460 con 0x55c26c551b90 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.929+0000 7fd5a8a15740 10 client.27448 fetch_fsmap learned FSMap version 46 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.929+0000 7fd5a8a15740 1 -- 172.21.15.46:0/706519269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c26c5570c0 con 0x55c26c551b90 2024-09-10T07:45:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.930+0000 7fd588c00640 1 -- 172.21.15.46:0/706519269 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5900a48f0 con 0x55c26c551b90 2024-09-10T07:45:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.930+0000 7fd588c00640 1 -- 172.21.15.46:0/706519269 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 46) ==== 8378+0+0 (secure 0 0 0) 0x7fd5900b4070 con 0x55c26c551b90 2024-09-10T07:45:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.930+0000 7fd5a8a15740 10 client.27448 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T07:45:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.930+0000 7fd5a8a15740 10 client.27448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.930+0000 7fd5a8a15740 20 client.27448 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.930+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c26c5dfff0 0x55c26c5e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.930+0000 7fd5a8a15740 4 client.27448 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:45:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.930+0000 7fd5a8a15740 1 -- 172.21.15.46:0/706519269 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c26c4bfe60 con 0x55c26c5dfff0 2024-09-10T07:45:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.931+0000 7fd59e000640 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c26c5dfff0 0x55c26c5e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.931+0000 7fd59e000640 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c26c5dfff0 0x55c26c5e23e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701d47666401d260 server_cookie=9070c7bcf6d1d356 in_seq=0 out_seq=0 2024-09-10T07:45:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.931+0000 7fd588c00640 10 client.27448 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.931+0000 7fd588c00640 1 -- 172.21.15.46:0/706519269 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c26c4bfe60 con 0x55c26c5dfff0 2024-09-10T07:45:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd588c00640 10 client.27448 handle_command_reply: tid=0 2024-09-10T07:45:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 client.27448 shutdown 2024-09-10T07:45:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 -- 172.21.15.46:0/706519269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 msgr2=0x55c26c5652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c5652b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ccf70 tx=0x7fd59009bb80 comp rx=0 tx=0).stop 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 -- 172.21.15.46:0/706519269 shutdown_connections 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c26c5dfff0 0x55c26c5e23e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c26c55cd70 0x55c26c556b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd588c00640 0 client.27448 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c26c5524b0 0x55c26c5565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 --2- 172.21.15.46:0/706519269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c26c551b90 0x55c26c5652b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 -- 172.21.15.46:0/706519269 >> 172.21.15.46:0/706519269 conn(0x55c26c54e120 msgr2=0x55c26c55a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 -- 172.21.15.46:0/706519269 shutdown_connections 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.932+0000 7fd5a8a15740 1 -- 172.21.15.46:0/706519269 wait complete. 2024-09-10T07:45:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.933+0000 7fd5a8a15740 20 client.27448 trim_cache size 0 max 16384 2024-09-10T07:45:10.934 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:45:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:45:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:45:10.934 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:45:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.933+0000 7fd5a5e00640 1 -- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5840a2b80 msgr2=0x7fd5840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.933+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5840a2b80 0x7fd5840a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd598004af0 tx=0x7fd598004a80 comp rx=0 tx=0).stop 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 -- 172.21.15.46:0/3023292499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a0103ff0 msgr2=0x7fd5a019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a0103ff0 0x7fd5a019a630 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e9f0 tx=0x7fd59000edf0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 -- 172.21.15.46:0/3023292499 shutdown_connections 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5840a2b80 0x7fd5840a5040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0109120 0x7fd5a019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a01049a0 0x7fd5a019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 --2- 172.21.15.46:0/3023292499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a0103ff0 0x7fd5a019a630 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 -- 172.21.15.46:0/3023292499 >> 172.21.15.46:0/3023292499 conn(0x7fd5a00ffa90 msgr2=0x7fd5a01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 -- 172.21.15.46:0/3023292499 shutdown_connections 2024-09-10T07:45:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:10.934+0000 7fd5a5e00640 1 -- 172.21.15.46:0/3023292499 wait complete. 2024-09-10T07:45:10.939 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.799+0000 7f3b2be00640 1 -- 172.21.15.46:0/3095867184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c103e30 msgr2=0x7f3b2c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.799+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3095867184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c103e30 0x7f3b2c104210 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009900 tx=0x7f3b1c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.799+0000 7f3b2be00640 1 -- 172.21.15.46:0/3095867184 shutdown_connections 2024-09-10T07:45:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.799+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3095867184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b2c108f60 0x7f3b2c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.799+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3095867184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c1047e0 0x7f3b2c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.799+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3095867184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c103e30 0x7f3b2c104210 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.799+0000 7f3b2be00640 1 -- 172.21.15.46:0/3095867184 >> 172.21.15.46:0/3095867184 conn(0x7f3b2c0ff8d0 msgr2=0x7f3b2c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.799+0000 7f3b2be00640 1 -- 172.21.15.46:0/3095867184 shutdown_connections 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.800+0000 7f3b2be00640 1 -- 172.21.15.46:0/3095867184 wait complete. 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.800+0000 7f3b2be00640 1 Processor -- start 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2be00640 1 -- start start 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b2c103e30 0x7f3b2c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1047e0 0x7f3b2c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c108f60 0x7f3b2c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c114010 con 0x7f3b2c1047e0 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c113e90 con 0x7f3b2c103e30 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c114190 con 0x7f3b2c108f60 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1047e0 0x7f3b2c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b2c103e30 0x7f3b2c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1047e0 0x7f3b2c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43228/0 (socket says 172.21.15.46:43228) 2024-09-10T07:45:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.801+0000 7f3b2a000640 1 -- 172.21.15.46:0/3281654743 learned_addr learned my addr 172.21.15.46:0/3281654743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b2b400640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c108f60 0x7f3b2c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b2a000640 1 -- 172.21.15.46:0/3281654743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c108f60 msgr2=0x7f3b2c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b2a000640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c108f60 0x7f3b2c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b2a000640 1 -- 172.21.15.46:0/3281654743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b2c103e30 msgr2=0x7f3b2c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b2a000640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b2c103e30 0x7f3b2c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b2a000640 1 -- 172.21.15.46:0/3281654743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c19f460 con 0x7f3b2c1047e0 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b2a000640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1047e0 0x7f3b2c19a9f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800e9f0 tx=0x7f3b1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b23400640 1 -- 172.21.15.46:0/3281654743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800cd80 con 0x7f3b2c1047e0 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b23400640 1 -- 172.21.15.46:0/3281654743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b18005500 con 0x7f3b2c1047e0 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b23400640 1 -- 172.21.15.46:0/3281654743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18010690 con 0x7f3b2c1047e0 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.802+0000 7f3b2be00640 1 -- 172.21.15.46:0/3281654743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c19f750 con 0x7f3b2c1047e0 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.803+0000 7f3b2be00640 1 -- 172.21.15.46:0/3281654743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c1a7030 con 0x7f3b2c1047e0 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.806+0000 7f3b345f3740 1 Processor -- start 2024-09-10T07:45:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.806+0000 7f3b23400640 1 -- 172.21.15.46:0/3281654743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b180046c0 con 0x7f3b2c1047e0 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.806+0000 7f3b345f3740 1 -- start start 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7520 0x55c27cfe7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 0x55c27cff21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27cff4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b345f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c27cf4d650 con 0x55c27cfe7520 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b345f3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c27cef4f10 con 0x55c27cff2700 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b345f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c27cf49610 con 0x55c27cfe7e40 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7520 0x55c27cfe7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27cff4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7520 0x55c27cfe7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43230/0 (socket says 172.21.15.46:43230) 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b2b400640 1 -- 172.21.15.46:0/3672951875 learned_addr learned my addr 172.21.15.46:0/3672951875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.807+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 0x55c27cff21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b2b400640 1 -- 172.21.15.46:0/3672951875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 msgr2=0x55c27cff21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b2b400640 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 0x55c27cff21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b2b400640 1 -- 172.21.15.46:0/3672951875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 msgr2=0x55c27cff4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b2b400640 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27cff4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b2b400640 1 -- 172.21.15.46:0/3672951875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c27cc5c580 con 0x55c27cfe7520 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b2a000640 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27cff4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b2b400640 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7520 0x55c27cfe7900 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009f20 tx=0x7f3b2400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e311ed91d26c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b23400640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b100a2c00 0x7f3b100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b23400640 1 -- 172.21.15.46:0/3281654743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b180cf850 con 0x7f3b2c1047e0 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b17e00640 1 -- 172.21.15.46:0/3672951875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24015070 con 0x55c27cfe7520 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b17e00640 1 -- 172.21.15.46:0/3672951875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2401ab40 con 0x55c27cfe7520 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b17e00640 1 -- 172.21.15.46:0/3672951875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24011ca0 con 0x55c27cfe7520 2024-09-10T07:45:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.808+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b100a2c00 0x7f3b100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 -- 172.21.15.46:0/3672951875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7520 msgr2=0x55c27cfe7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7520 0x55c27cfe7900 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009f20 tx=0x7f3b2400f7f0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b100a2c00 0x7f3b100a50c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c000c00 tx=0x7f3b1c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 -- 172.21.15.46:0/3672951875 shutdown_connections 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27cff4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 0x55c27cff21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 --2- 172.21.15.46:0/3672951875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7520 0x55c27cfe7900 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 -- 172.21.15.46:0/3672951875 >> 172.21.15.46:0/3672951875 conn(0x55c27cfe3ab0 msgr2=0x55c27cfe3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 -- 172.21.15.46:0/3672951875 shutdown_connections 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.809+0000 7f3b345f3740 1 -- 172.21.15.46:0/3672951875 wait complete. 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 1 -- start start 2024-09-10T07:45:11.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 1 Processor -- start 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cfef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 0x55c27cfefba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cff2700 0x55c27cff00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c27cff52d0 con 0x55c27cff2700 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c27cf08350 con 0x55c27cfe7520 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c27cff5450 con 0x55c27cfe7e40 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.810+0000 7f3b345f3740 10 client.0 _ll_register_callbacks cb 0x55c27cf2b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cfef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cfef660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47992/0 (socket says 172.21.15.46:47992) 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2aa00640 1 -- 172.21.15.46:0/4066915531 learned_addr learned my addr 172.21.15.46:0/4066915531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2b400640 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cff2700 0x55c27cff00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2a000640 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 0x55c27cfefba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2aa00640 1 -- 172.21.15.46:0/4066915531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 msgr2=0x55c27cfefba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 0x55c27cfefba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2aa00640 1 -- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cff2700 msgr2=0x55c27cff00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cff2700 0x55c27cff00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2aa00640 1 -- 172.21.15.46:0/4066915531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c27cd26960 con 0x55c27cfe7520 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.811+0000 7f3b2b400640 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cff2700 0x55c27cff00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.812+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cfef660 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009900 tx=0x7f3b1c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d856b7f0e98f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.812+0000 7f3b14c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.812+0000 7f3b14c00640 1 -- 172.21.15.46:0/4066915531 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c04e070 con 0x55c27cfe7520 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.812+0000 7f3b345f3740 1 -- 172.21.15.46:0/4066915531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2401ab40 con 0x55c27cfe7520 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.812+0000 7f3b14c00640 1 -- 172.21.15.46:0/4066915531 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b1c004d10 con 0x55c27cfe7520 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.812+0000 7f3b14c00640 1 -- 172.21.15.46:0/4066915531 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3b1c05a050 con 0x55c27cfe7520 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.812+0000 7f3b345f3740 10 client.27506 fetch_fsmap learned FSMap version 46 2024-09-10T07:45:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.812+0000 7f3b345f3740 1 -- 172.21.15.46:0/4066915531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c27d000fa0 con 0x55c27cfe7520 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.813+0000 7f3b14c00640 1 -- 172.21.15.46:0/4066915531 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2401ab40 con 0x55c27cfe7520 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.813+0000 7f3b14c00640 1 -- 172.21.15.46:0/4066915531 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 46) ==== 8378+0+0 (secure 0 0 0) 0x7f3b1c0038d0 con 0x55c27cfe7520 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.813+0000 7f3b345f3740 10 client.27506 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.813+0000 7f3b345f3740 10 client.27506 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.813+0000 7f3b345f3740 20 client.27506 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.813+0000 7f3b345f3740 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c27d0023b0 0x55c27d004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.813+0000 7f3b345f3740 4 client.27506 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.813+0000 7f3b345f3740 1 -- 172.21.15.46:0/4066915531 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c27cf5abc0 con 0x55c27d0023b0 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.814+0000 7f3b2a000640 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c27d0023b0 0x55c27d004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.814+0000 7f3b2a000640 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c27d0023b0 0x55c27d004810 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61530f93a3340b5 server_cookie=9d8e0242b6375d86 in_seq=0 out_seq=0 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.814+0000 7f3b14c00640 10 client.27506 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b14c00640 1 -- 172.21.15.46:0/4066915531 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c27cf5abc0 con 0x55c27d0023b0 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b14c00640 10 client.27506 handle_command_reply: tid=0 2024-09-10T07:45:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 client.27506 shutdown 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 -- 172.21.15.46:0/4066915531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 msgr2=0x55c27cfef660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cfef660 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009900 tx=0x7f3b1c009260 comp rx=0 tx=0).stop 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 -- 172.21.15.46:0/4066915531 shutdown_connections 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c27d0023b0 0x55c27d004810 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cff2700 0x55c27cff00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7e40 0x55c27cfefba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b14c00640 0 client.27506 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 --2- 172.21.15.46:0/4066915531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cfef660 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.815+0000 7f3b345f3740 1 -- 172.21.15.46:0/4066915531 >> 172.21.15.46:0/4066915531 conn(0x55c27cfe3ab0 msgr2=0x55c27cfe2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:11.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.816+0000 7f3b345f3740 1 -- 172.21.15.46:0/4066915531 shutdown_connections 2024-09-10T07:45:11.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.816+0000 7f3b345f3740 1 -- 172.21.15.46:0/4066915531 wait complete. 2024-09-10T07:45:11.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.816+0000 7f3b345f3740 20 client.27506 trim_cache size 0 max 16384 2024-09-10T07:45:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.834+0000 7f3b345f3740 1 Processor -- start 2024-09-10T07:45:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.834+0000 7f3b345f3740 1 -- start start 2024-09-10T07:45:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.834+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7520 0x55c27cfe2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.834+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cfedf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.834+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27d00a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.834+0000 7f3b345f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b0c00 con 0x55c27cfe7e40 2024-09-10T07:45:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.834+0000 7f3b345f3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b27a0 con 0x55c27cff2700 2024-09-10T07:45:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.834+0000 7f3b345f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b12d0 con 0x55c27cfe7520 2024-09-10T07:45:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7520 0x55c27cfe2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7520 0x55c27cfe2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:41702/0 (socket says 172.21.15.46:41702) 2024-09-10T07:45:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2aa00640 1 -- 172.21.15.46:0/2354494774 learned_addr learned my addr 172.21.15.46:0/2354494774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2a000640 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cfedf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2b400640 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27d00a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2aa00640 1 -- 172.21.15.46:0/2354494774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 msgr2=0x55c27d00a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27d00a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2aa00640 1 -- 172.21.15.46:0/2354494774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 msgr2=0x55c27cfedf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cfedf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2aa00640 1 -- 172.21.15.46:0/2354494774 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c27cfe3550 con 0x55c27cfe7520 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.835+0000 7f3b2b400640 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27d00a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.836+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7520 0x55c27cfe2b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c005c40 tx=0x7f3b1c005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2735e4696976ae84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.836+0000 7f3b17400640 1 -- 172.21.15.46:0/2354494774 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c062070 con 0x55c27cfe7520 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.836+0000 7f3b17400640 1 -- 172.21.15.46:0/2354494774 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c0035f0 con 0x55c27cfe7520 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.836+0000 7f3b17400640 1 -- 172.21.15.46:0/2354494774 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c04e050 con 0x55c27cfe7520 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 -- 172.21.15.46:0/2354494774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7520 msgr2=0x55c27cfe2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7520 0x55c27cfe2b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c005c40 tx=0x7f3b1c005760 comp rx=0 tx=0).stop 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 -- 172.21.15.46:0/2354494774 shutdown_connections 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cff2700 0x55c27d00a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cfedf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 --2- 172.21.15.46:0/2354494774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cfe7520 0x55c27cfe2b40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 -- 172.21.15.46:0/2354494774 >> 172.21.15.46:0/2354494774 conn(0x55c27cfe3ab0 msgr2=0x55c27cff15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 -- 172.21.15.46:0/2354494774 shutdown_connections 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 -- 172.21.15.46:0/2354494774 wait complete. 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.837+0000 7f3b345f3740 1 -- start start 2024-09-10T07:45:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b345f3740 1 Processor -- start 2024-09-10T07:45:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cffabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cff7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b345f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cff2700 0x55c27cff7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b345f3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b2070 con 0x55c27cfe7e40 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b345f3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b19a0 con 0x55c27cfe7520 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b345f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1aebe0 con 0x55c27cff2700 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b345f3740 10 client.0 _ll_register_callbacks cb 0x7f3b1c0042f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cff7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cff7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43268/0 (socket says 172.21.15.46:43268) 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b2a000640 1 -- 172.21.15.46:0/1677528351 learned_addr learned my addr 172.21.15.46:0/1677528351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cffabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.838+0000 7f3b2b400640 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cff2700 0x55c27cff7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b2a000640 1 -- 172.21.15.46:0/1677528351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cff2700 msgr2=0x55c27cff7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b2a000640 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cff2700 0x55c27cff7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b2a000640 1 -- 172.21.15.46:0/1677528351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 msgr2=0x55c27cffabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b2a000640 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cffabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b2a000640 1 -- 172.21.15.46:0/1677528351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c27cff81a0 con 0x55c27cfe7e40 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cffabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b2a000640 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cff7670 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b180986a0 tx=0x7f3b180d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8b044ced6367af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b14c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b14c00640 1 -- 172.21.15.46:0/1677528351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1809c3d0 con 0x55c27cfe7e40 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b14c00640 1 -- 172.21.15.46:0/1677528351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b18098340 con 0x55c27cfe7e40 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.839+0000 7f3b345f3740 1 -- 172.21.15.46:0/1677528351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b1c0035f0 con 0x55c27cfe7e40 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b14c00640 1 -- 172.21.15.46:0/1677528351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3b1809bcd0 con 0x55c27cfe7e40 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b345f3740 10 client.16992 fetch_fsmap learned FSMap version 46 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b345f3740 1 -- 172.21.15.46:0/1677528351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c27d06df40 con 0x55c27cfe7e40 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b14c00640 1 -- 172.21.15.46:0/1677528351 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c0035f0 con 0x55c27cfe7e40 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b14c00640 1 -- 172.21.15.46:0/1677528351 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) ==== 8378+0+0 (secure 0 0 0) 0x7f3b180a4540 con 0x55c27cfe7e40 2024-09-10T07:45:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b345f3740 10 client.16992 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T07:45:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b345f3740 10 client.16992 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:45:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b345f3740 20 client.16992 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b345f3740 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c27d06e7a0 0x55c27d070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b345f3740 4 client.16992 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "26d3f25d", "release": true, "await": true}] 2024-09-10T07:45:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.840+0000 7f3b345f3740 1 -- 172.21.15.46:0/1677528351 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "26d3f25d", "release": true, "await": true}) -- 0x55c27cf550d0 con 0x55c27d06e7a0 2024-09-10T07:45:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.841+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c27d06e7a0 0x55c27d070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.841+0000 7f3b2aa00640 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c27d06e7a0 0x55c27d070bc0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1633f16affc16dc server_cookie=be59633bc3a133c9 in_seq=0 out_seq=0 2024-09-10T07:45:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.841+0000 7f3b14c00640 10 client.16992 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.882+0000 7f3b14c00640 1 -- 172.21.15.46:0/1677528351 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c27cf550d0 con 0x55c27d06e7a0 2024-09-10T07:45:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.882+0000 7f3b14c00640 10 client.16992 handle_command_reply: tid=0 2024-09-10T07:45:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.886+0000 7f3b345f3740 1 client.16992 shutdown 2024-09-10T07:45:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b345f3740 1 -- 172.21.15.46:0/1677528351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 msgr2=0x55c27cff7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b345f3740 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cff7670 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b180986a0 tx=0x7f3b180d5530 comp rx=0 tx=0).stop 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b345f3740 1 -- 172.21.15.46:0/1677528351 shutdown_connections 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b345f3740 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c27d06e7a0 0x55c27d070bc0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b345f3740 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c27cff2700 0x55c27cff7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b345f3740 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c27cfe7e40 0x55c27cff7670 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b345f3740 1 --2- 172.21.15.46:0/1677528351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c27cfe7520 0x55c27cffabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b345f3740 1 -- 172.21.15.46:0/1677528351 >> 172.21.15.46:0/1677528351 conn(0x55c27cfe3ab0 msgr2=0x55c27cffd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.887+0000 7f3b14c00640 0 client.16992 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.888+0000 7f3b345f3740 1 -- 172.21.15.46:0/1677528351 shutdown_connections 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.888+0000 7f3b345f3740 1 -- 172.21.15.46:0/1677528351 wait complete. 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.889+0000 7f3b345f3740 20 client.16992 trim_cache size 0 max 16384 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 46, 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 28, 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "26d3f25d": { 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 28, 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 133.4, 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.890+0000 7f3b2be00640 1 -- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b100a2c00 msgr2=0x7f3b100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.890+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b100a2c00 0x7f3b100a50c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c000c00 tx=0x7f3b1c005b50 comp rx=0 tx=0).stop 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.890+0000 7f3b2be00640 1 -- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1047e0 msgr2=0x7f3b2c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.890+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1047e0 0x7f3b2c19a9f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800e9f0 tx=0x7f3b1800edf0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.891+0000 7f3b2be00640 1 -- 172.21.15.46:0/3281654743 shutdown_connections 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.891+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c108f60 0x7f3b2c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.891+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b100a2c00 0x7f3b100a50c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.891+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1047e0 0x7f3b2c19a9f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.891+0000 7f3b2be00640 1 --2- 172.21.15.46:0/3281654743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b2c103e30 0x7f3b2c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.891+0000 7f3b2be00640 1 -- 172.21.15.46:0/3281654743 >> 172.21.15.46:0/3281654743 conn(0x7f3b2c0ff8d0 msgr2=0x7f3b2c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.891+0000 7f3b2be00640 1 -- 172.21.15.46:0/3281654743 shutdown_connections 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:11.892+0000 7f3b2be00640 1 -- 172.21.15.46:0/3281654743 wait complete. 2024-09-10T07:45:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:11 smithi181 ceph-mon[35397]: pgmap v656: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-09-10T07:45:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:11 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:11 smithi063 ceph-mon[35020]: pgmap v656: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-09-10T07:45:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:11 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:12.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:11 smithi046 ceph-mon[30705]: pgmap v656: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-09-10T07:45:12.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:11 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:12.804 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:45:12.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:45:13.395 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '26d3f25d', total seconds elapsed: 18.9 2024-09-10T07:45:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:14.001 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:13 smithi046 ceph-mon[30705]: pgmap v657: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-09-10T07:45:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:13 smithi181 ceph-mon[35397]: pgmap v657: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-09-10T07:45:14.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:13 smithi063 ceph-mon[35020]: pgmap v657: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-09-10T07:45:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.351+0000 7f00c3000640 1 -- 172.21.15.46:0/823149822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1068d0 msgr2=0x7f00bc106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.351+0000 7f00c3000640 1 --2- 172.21.15.46:0/823149822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1068d0 0x7f00bc106cb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f00a800fb40 tx=0x7f00a80329a0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.351+0000 7f00c3000640 1 -- 172.21.15.46:0/823149822 shutdown_connections 2024-09-10T07:45:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.351+0000 7f00c3000640 1 --2- 172.21.15.46:0/823149822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00bc101270 0x7f00bc10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.351+0000 7f00c3000640 1 --2- 172.21.15.46:0/823149822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00bc1008d0 0x7f00bc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.351+0000 7f00c3000640 1 --2- 172.21.15.46:0/823149822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1068d0 0x7f00bc106cb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.351+0000 7f00c3000640 1 -- 172.21.15.46:0/823149822 >> 172.21.15.46:0/823149822 conn(0x7f00bc0fc610 msgr2=0x7f00bc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.352+0000 7f00c3000640 1 -- 172.21.15.46:0/823149822 shutdown_connections 2024-09-10T07:45:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.352+0000 7f00c3000640 1 -- 172.21.15.46:0/823149822 wait complete. 2024-09-10T07:45:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.354+0000 7f00c3000640 1 Processor -- start 2024-09-10T07:45:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.355+0000 7f00c3000640 1 -- start start 2024-09-10T07:45:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.357+0000 7f00c3000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00bc1008d0 0x7f00bc1a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.357+0000 7f00c3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00bc101270 0x7f00bc1a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.357+0000 7f00c3000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1abb40 0x7f00bc1abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.357+0000 7f00c3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10fc20 con 0x7f00bc101270 2024-09-10T07:45:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.357+0000 7f00c3000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10faa0 con 0x7f00bc1008d0 2024-09-10T07:45:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.357+0000 7f00c3000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10fda0 con 0x7f00bc1abb40 2024-09-10T07:45:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.360+0000 7f00c1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00bc1008d0 0x7f00bc1a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.360+0000 7f00c1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00bc1008d0 0x7f00bc1a7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48036/0 (socket says 172.21.15.46:48036) 2024-09-10T07:45:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.360+0000 7f00c1c00640 1 -- 172.21.15.46:0/4152078723 learned_addr learned my addr 172.21.15.46:0/4152078723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.360+0000 7f00c2600640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1abb40 0x7f00bc1abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.360+0000 7f00c1200640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00bc101270 0x7f00bc1a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.362+0000 7f00c2600640 1 -- 172.21.15.46:0/4152078723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00bc1008d0 msgr2=0x7f00bc1a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.362+0000 7f00c2600640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00bc1008d0 0x7f00bc1a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.363+0000 7f00c2600640 1 -- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00bc101270 msgr2=0x7f00bc1a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.363+0000 7f00c2600640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00bc101270 0x7f00bc1a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.363+0000 7f00c2600640 1 -- 172.21.15.46:0/4152078723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc1ac600 con 0x7f00bc1abb40 2024-09-10T07:45:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.364+0000 7f00c1c00640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00bc1008d0 0x7f00bc1a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.364+0000 7f00c1200640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00bc101270 0x7f00bc1a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.365+0000 7f00c2600640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1abb40 0x7f00bc1abff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f00b4007b20 tx=0x7f00b400bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.365+0000 7f00ba000640 1 -- 172.21.15.46:0/4152078723 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4019070 con 0x7f00bc1abb40 2024-09-10T07:45:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.367+0000 7f00ba000640 1 -- 172.21.15.46:0/4152078723 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b400b230 con 0x7f00bc1abb40 2024-09-10T07:45:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.367+0000 7f00ba000640 1 -- 172.21.15.46:0/4152078723 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004020 con 0x7f00bc1abb40 2024-09-10T07:45:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.367+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1ac8f0 con 0x7f00bc1abb40 2024-09-10T07:45:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.367+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1b4080 con 0x7f00bc1abb40 2024-09-10T07:45:14.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.369+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc106cb0 con 0x7f00bc1abb40 2024-09-10T07:45:14.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.373+0000 7f00ba000640 1 -- 172.21.15.46:0/4152078723 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00b4004a40 con 0x7f00bc1abb40 2024-09-10T07:45:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.376+0000 7f00ba000640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00a40a2cc0 0x7f00a40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.377+0000 7f00c1c00640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00a40a2cc0 0x7f00a40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.377+0000 7f00ba000640 1 -- 172.21.15.46:0/4152078723 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f00b40cf680 con 0x7f00bc1abb40 2024-09-10T07:45:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.378+0000 7f00ba000640 1 -- 172.21.15.46:0/4152078723 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00b409c200 con 0x7f00bc1abb40 2024-09-10T07:45:14.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.379+0000 7f00c1c00640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00a40a2cc0 0x7f00a40a5180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f00a800fb10 tx=0x7f00a8011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.805+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00bc1a85d0 con 0x7f00bc1abb40 2024-09-10T07:45:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:45:14.806 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"btime":"2024-09-10T07:45:10:836143+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:45:10.518302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:45:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.806+0000 7f00ba000640 1 -- 172.21.15.46:0/4152078723 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11975 (secure 0 0 0) 0x7f00bc1a85d0 con 0x7f00bc1abb40 2024-09-10T07:45:14.807 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 46 2024-09-10T07:45:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.809+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00a40a2cc0 msgr2=0x7f00a40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.809+0000 7f00c3000640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00a40a2cc0 0x7f00a40a5180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f00a800fb10 tx=0x7f00a8011040 comp rx=0 tx=0).stop 2024-09-10T07:45:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.809+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1abb40 msgr2=0x7f00bc1abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.809+0000 7f00c3000640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1abb40 0x7f00bc1abff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f00b4007b20 tx=0x7f00b400bf60 comp rx=0 tx=0).stop 2024-09-10T07:45:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.810+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 shutdown_connections 2024-09-10T07:45:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.810+0000 7f00c3000640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00bc1abb40 0x7f00bc1abff0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.810+0000 7f00c3000640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f00a40a2cc0 0x7f00a40a5180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.810+0000 7f00c3000640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00bc101270 0x7f00bc1a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.810+0000 7f00c3000640 1 --2- 172.21.15.46:0/4152078723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00bc1008d0 0x7f00bc1a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.810+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 >> 172.21.15.46:0/4152078723 conn(0x7f00bc0fc610 msgr2=0x7f00bc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.810+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 shutdown_connections 2024-09-10T07:45:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:14.810+0000 7f00c3000640 1 -- 172.21.15.46:0/4152078723 wait complete. 2024-09-10T07:45:15.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:14 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/4152078723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:15.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:14 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/4152078723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:15.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:14 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/4152078723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 475.0 s, 543.3 tps, lat 534.171 ms stddev 2049.478 2024-09-10T07:45:15.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:45:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:15 smithi181 ceph-mon[35397]: pgmap v658: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 452 KiB/s wr, 65 op/s 2024-09-10T07:45:16.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:15 smithi063 ceph-mon[35020]: pgmap v658: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 452 KiB/s wr, 65 op/s 2024-09-10T07:45:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:15 smithi046 ceph-mon[30705]: pgmap v658: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 452 KiB/s wr, 65 op/s 2024-09-10T07:45:16.576 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:17.125 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:16 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.320+0000 7f832c200640 1 -- 172.21.15.46:0/1175134367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 msgr2=0x7f8324106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.320+0000 7f832c200640 1 --2- 172.21.15.46:0/1175134367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 0x7f8324106e70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8318009860 tx=0x7f831802a340 comp rx=0 tx=0).stop 2024-09-10T07:45:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.321+0000 7f832c200640 1 -- 172.21.15.46:0/1175134367 shutdown_connections 2024-09-10T07:45:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.321+0000 7f832c200640 1 --2- 172.21.15.46:0/1175134367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8324100860 0x7f83241045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.321+0000 7f832c200640 1 --2- 172.21.15.46:0/1175134367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8324107440 0x7f8324100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.321+0000 7f832c200640 1 --2- 172.21.15.46:0/1175134367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 0x7f8324106e70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.321+0000 7f832c200640 1 -- 172.21.15.46:0/1175134367 >> 172.21.15.46:0/1175134367 conn(0x7f8324077230 msgr2=0x7f83240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.321+0000 7f832c200640 1 -- 172.21.15.46:0/1175134367 shutdown_connections 2024-09-10T07:45:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.322+0000 7f832c200640 1 -- 172.21.15.46:0/1175134367 wait complete. 2024-09-10T07:45:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.323+0000 7f832c200640 1 Processor -- start 2024-09-10T07:45:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.323+0000 7f832c200640 1 -- start start 2024-09-10T07:45:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f832c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8324100860 0x7f83241a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f832c200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 0x7f83241a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f832c200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832419e890 0x7f832419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f832c200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83241185e0 con 0x7f8324100860 2024-09-10T07:45:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f832c200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8324118460 con 0x7f832419e890 2024-09-10T07:45:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f832c200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8324118760 con 0x7f8324106a90 2024-09-10T07:45:17.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f8328e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 0x7f83241a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f8329800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8324100860 0x7f83241a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f8328e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 0x7f83241a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50182/0 (socket says 172.21.15.46:50182) 2024-09-10T07:45:17.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f8328e00640 1 -- 172.21.15.46:0/2671225068 learned_addr learned my addr 172.21.15.46:0/2671225068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:17.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.324+0000 7f832a200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832419e890 0x7f832419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.325+0000 7f832a200640 1 -- 172.21.15.46:0/2671225068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 msgr2=0x7f83241a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.325+0000 7f832a200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 0x7f83241a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.325+0000 7f832a200640 1 -- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8324100860 msgr2=0x7f83241a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.325+0000 7f832a200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8324100860 0x7f83241a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.325+0000 7f832a200640 1 -- 172.21.15.46:0/2671225068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832419f3b0 con 0x7f832419e890 2024-09-10T07:45:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.325+0000 7f8328e00640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 0x7f83241a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.325+0000 7f832a200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832419e890 0x7f832419ed40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f831c009fd0 tx=0x7f831c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.326+0000 7f8322000640 1 -- 172.21.15.46:0/2671225068 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c00cd60 con 0x7f832419e890 2024-09-10T07:45:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.326+0000 7f8322000640 1 -- 172.21.15.46:0/2671225068 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831c005020 con 0x7f832419e890 2024-09-10T07:45:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.326+0000 7f8322000640 1 -- 172.21.15.46:0/2671225068 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c010690 con 0x7f832419e890 2024-09-10T07:45:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.326+0000 7f832c200640 1 -- 172.21.15.46:0/2671225068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832419f6a0 con 0x7f832419e890 2024-09-10T07:45:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.326+0000 7f832c200640 1 -- 172.21.15.46:0/2671225068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241ab890 con 0x7f832419e890 2024-09-10T07:45:17.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.329+0000 7f8322000640 1 -- 172.21.15.46:0/2671225068 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f831c005540 con 0x7f832419e890 2024-09-10T07:45:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.330+0000 7f832ee3f740 1 Processor -- start 2024-09-10T07:45:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.330+0000 7f8322000640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f830c0a2d00 0x7f830c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f832ee3f740 1 -- start start 2024-09-10T07:45:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f8322000640 1 -- 172.21.15.46:0/2671225068 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f831c0cfa40 con 0x7f832419e890 2024-09-10T07:45:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f8328e00640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f830c0a2d00 0x7f830c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188dc70 0x561d3188e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188e590 0x561d31898910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f832ee3f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561d317f43e0 con 0x561d3188dc70 2024-09-10T07:45:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f832ee3f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561d3179bd80 con 0x561d3188e590 2024-09-10T07:45:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.331+0000 7f832ee3f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561d317f03a0 con 0x561d31898e50 2024-09-10T07:45:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.332+0000 7f8329800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188dc70 0x561d3188e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.332+0000 7f8329800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188dc70 0x561d3188e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53632/0 (socket says 172.21.15.46:53632) 2024-09-10T07:45:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.332+0000 7f8329800640 1 -- 172.21.15.46:0/1806609754 learned_addr learned my addr 172.21.15.46:0/1806609754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.332+0000 7f8328e00640 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188e590 0x561d31898910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.332+0000 7f832a200640 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.333+0000 7f8329800640 1 -- 172.21.15.46:0/1806609754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 msgr2=0x561d3189b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:17.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.333+0000 7f8329800640 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.333+0000 7f8329800640 1 -- 172.21.15.46:0/1806609754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188e590 msgr2=0x561d31898910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:17.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.333+0000 7f8329800640 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188e590 0x561d31898910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.333+0000 7f8329800640 1 -- 172.21.15.46:0/1806609754 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d317baae0 con 0x561d3188dc70 2024-09-10T07:45:17.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.333+0000 7f832a200640 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:17.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.334+0000 7f8328e00640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f830c0a2d00 0x7f830c0a51c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8324104030 tx=0x7f8310011890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:17.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.335+0000 7f8329800640 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188dc70 0x561d3188e050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f831802a930 tx=0x7f83180303e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f56d295bd02dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:17.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.336+0000 7f8316a00640 1 -- 172.21.15.46:0/1806609754 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318041020 con 0x561d3188dc70 2024-09-10T07:45:17.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.336+0000 7f8316a00640 1 -- 172.21.15.46:0/1806609754 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318030550 con 0x561d3188dc70 2024-09-10T07:45:17.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.336+0000 7f8316a00640 1 -- 172.21.15.46:0/1806609754 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318037040 con 0x561d3188dc70 2024-09-10T07:45:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.337+0000 7f832ee3f740 1 -- 172.21.15.46:0/1806609754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188dc70 msgr2=0x561d3188e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.337+0000 7f832ee3f740 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188dc70 0x561d3188e050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f831802a930 tx=0x7f83180303e0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.338+0000 7f832ee3f740 1 -- 172.21.15.46:0/1806609754 shutdown_connections 2024-09-10T07:45:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.338+0000 7f832ee3f740 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.338+0000 7f832ee3f740 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188e590 0x561d31898910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.338+0000 7f832ee3f740 1 --2- 172.21.15.46:0/1806609754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188dc70 0x561d3188e050 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.338+0000 7f832ee3f740 1 -- 172.21.15.46:0/1806609754 >> 172.21.15.46:0/1806609754 conn(0x561d3188a200 msgr2=0x561d3188a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.340+0000 7f832ee3f740 1 -- 172.21.15.46:0/1806609754 shutdown_connections 2024-09-10T07:45:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.340+0000 7f832ee3f740 1 -- 172.21.15.46:0/1806609754 wait complete. 2024-09-10T07:45:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.340+0000 7f832ee3f740 1 -- start start 2024-09-10T07:45:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.341+0000 7f832ee3f740 1 Processor -- start 2024-09-10T07:45:17.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.341+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d31895d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.341+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d31896270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.341+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 0x561d318967b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f832ee3f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561d3189ba20 con 0x561d3188e590 2024-09-10T07:45:17.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f832ee3f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561d317af0f0 con 0x561d31898e50 2024-09-10T07:45:17.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f832ee3f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561d3189bba0 con 0x561d3188dc70 2024-09-10T07:45:17.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f832ee3f740 10 client.0 _ll_register_callbacks cb 0x561d31796bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8329800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d31895d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8329800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d31895d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50192/0 (socket says 172.21.15.46:50192) 2024-09-10T07:45:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8329800640 1 -- 172.21.15.46:0/3272454146 learned_addr learned my addr 172.21.15.46:0/3272454146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f832a200640 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 0x561d318967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8328e00640 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d31896270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8329800640 1 -- 172.21.15.46:0/3272454146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 msgr2=0x561d318967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8329800640 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 0x561d318967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8329800640 1 -- 172.21.15.46:0/3272454146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 msgr2=0x561d31896270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8329800640 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d31896270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8329800640 1 -- 172.21.15.46:0/3272454146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d315653c0 con 0x561d3188dc70 2024-09-10T07:45:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.342+0000 7f8328e00640 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d31896270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.343+0000 7f8329800640 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d31895d30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8318035040 tx=0x7f83180095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0b8d9c70d174e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.343+0000 7f830b400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.343+0000 7f830b400640 1 -- 172.21.15.46:0/3272454146 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831804c070 con 0x561d3188dc70 2024-09-10T07:45:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.343+0000 7f830b400640 1 -- 172.21.15.46:0/3272454146 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f831803d480 con 0x561d3188dc70 2024-09-10T07:45:17.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.343+0000 7f832ee3f740 1 -- 172.21.15.46:0/3272454146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8318030550 con 0x561d3188dc70 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.344+0000 7f830b400640 1 -- 172.21.15.46:0/3272454146 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8318058050 con 0x561d3188dc70 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.344+0000 7f832ee3f740 10 client.27481 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.344+0000 7f832ee3f740 1 -- 172.21.15.46:0/3272454146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561d3189ba20 con 0x561d3188dc70 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.344+0000 7f830b400640 1 -- 172.21.15.46:0/3272454146 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318030550 con 0x561d3188dc70 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.345+0000 7f830b400640 1 -- 172.21.15.46:0/3272454146 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f8318046870 con 0x561d3188dc70 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.345+0000 7f832ee3f740 10 client.27481 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.345+0000 7f832ee3f740 10 client.27481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.345+0000 7f832ee3f740 20 client.27481 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.345+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x561d318a8b50 0x561d318aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.345+0000 7f832ee3f740 4 client.27481 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.345+0000 7f832ee3f740 1 -- 172.21.15.46:0/3272454146 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d31801950 con 0x561d318a8b50 2024-09-10T07:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.346+0000 7f8328e00640 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x561d318a8b50 0x561d318aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.347+0000 7f8328e00640 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x561d318a8b50 0x561d318aaf40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c06af9355c757e2e server_cookie=fd20bd1eec415dbe in_seq=0 out_seq=0 2024-09-10T07:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.347+0000 7f830b400640 10 client.27481 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:45:17.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.348+0000 7f830b400640 1 -- 172.21.15.46:0/3272454146 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d31801950 con 0x561d318a8b50 2024-09-10T07:45:17.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.348+0000 7f830b400640 10 client.27481 handle_command_reply: tid=0 2024-09-10T07:45:17.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.348+0000 7f832ee3f740 1 client.27481 shutdown 2024-09-10T07:45:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.352+0000 7f832ee3f740 1 -- 172.21.15.46:0/3272454146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 msgr2=0x561d31895d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.352+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d31895d30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8318035040 tx=0x7f83180095b0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.353+0000 7f832ee3f740 1 -- 172.21.15.46:0/3272454146 shutdown_connections 2024-09-10T07:45:17.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.353+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x561d318a8b50 0x561d318aaf40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.353+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 0x561d318967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.353+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d31896270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.353+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3272454146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d31895d30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.353+0000 7f832ee3f740 1 -- 172.21.15.46:0/3272454146 >> 172.21.15.46:0/3272454146 conn(0x561d3188a200 msgr2=0x561d31889380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.353+0000 7f830b400640 0 client.27481 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:45:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.354+0000 7f832ee3f740 1 -- 172.21.15.46:0/3272454146 shutdown_connections 2024-09-10T07:45:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.356+0000 7f832ee3f740 1 -- 172.21.15.46:0/3272454146 wait complete. 2024-09-10T07:45:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.356+0000 7f832ee3f740 20 client.27481 trim_cache size 0 max 16384 2024-09-10T07:45:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.382+0000 7f832ee3f740 1 Processor -- start 2024-09-10T07:45:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.382+0000 7f832ee3f740 1 -- start start 2024-09-10T07:45:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.383+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d318b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.383+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d318b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.383+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 0x561d31888b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.383+0000 7f832ee3f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b52a0 con 0x561d3188e590 2024-09-10T07:45:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.383+0000 7f832ee3f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b6e10 con 0x561d31898e50 2024-09-10T07:45:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.383+0000 7f832ee3f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b5970 con 0x561d3188dc70 2024-09-10T07:45:17.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.384+0000 7f8329800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d318b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.384+0000 7f8329800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d318b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50194/0 (socket says 172.21.15.46:50194) 2024-09-10T07:45:17.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.384+0000 7f8329800640 1 -- 172.21.15.46:0/145094848 learned_addr learned my addr 172.21.15.46:0/145094848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.384+0000 7f8329800640 1 -- 172.21.15.46:0/145094848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 msgr2=0x561d31888b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.384+0000 7f8329800640 1 --2- 172.21.15.46:0/145094848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 0x561d31888b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.384+0000 7f8329800640 1 -- 172.21.15.46:0/145094848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 msgr2=0x561d318b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.385+0000 7f8329800640 1 --2- 172.21.15.46:0/145094848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d318b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.385+0000 7f8329800640 1 -- 172.21.15.46:0/145094848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d31885770 con 0x561d3188dc70 2024-09-10T07:45:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.385+0000 7f8329800640 1 --2- 172.21.15.46:0/145094848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d318b0720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f831802a930 tx=0x7f8318001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e07fdcce530fd5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.386+0000 7f8316000640 1 -- 172.21.15.46:0/145094848 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831804c050 con 0x561d3188dc70 2024-09-10T07:45:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.386+0000 7f8316000640 1 -- 172.21.15.46:0/145094848 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318030550 con 0x561d3188dc70 2024-09-10T07:45:17.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.386+0000 7f8316000640 1 -- 172.21.15.46:0/145094848 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318046e60 con 0x561d3188dc70 2024-09-10T07:45:17.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 -- 172.21.15.46:0/145094848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 msgr2=0x561d318b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 --2- 172.21.15.46:0/145094848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d318b0720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f831802a930 tx=0x7f8318001e50 comp rx=0 tx=0).stop 2024-09-10T07:45:17.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 -- 172.21.15.46:0/145094848 shutdown_connections 2024-09-10T07:45:17.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 --2- 172.21.15.46:0/145094848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d31898e50 0x561d31888b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 --2- 172.21.15.46:0/145094848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d318b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 --2- 172.21.15.46:0/145094848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d3188dc70 0x561d318b0720 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 -- 172.21.15.46:0/145094848 >> 172.21.15.46:0/145094848 conn(0x561d3188a200 msgr2=0x561d318aab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 -- 172.21.15.46:0/145094848 shutdown_connections 2024-09-10T07:45:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.388+0000 7f832ee3f740 1 -- 172.21.15.46:0/145094848 wait complete. 2024-09-10T07:45:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.389+0000 7f832ee3f740 1 -- start start 2024-09-10T07:45:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.389+0000 7f832ee3f740 1 Processor -- start 2024-09-10T07:45:17.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188dc70 0x561d318a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d3189c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f832ee3f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f832ee3f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b6710 con 0x561d3188e590 2024-09-10T07:45:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f832ee3f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b6040 con 0x561d3188dc70 2024-09-10T07:45:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f832ee3f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b3280 con 0x561d31898e50 2024-09-10T07:45:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f8329800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188dc70 0x561d318a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f832ee3f740 10 client.0 _ll_register_callbacks cb 0x7f8310011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f8329800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188dc70 0x561d318a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41694/0 (socket says 172.21.15.46:41694) 2024-09-10T07:45:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f8329800640 1 -- 172.21.15.46:0/3374623393 learned_addr learned my addr 172.21.15.46:0/3374623393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:17.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.392+0000 7f832a200640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.393+0000 7f8329800640 1 -- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 msgr2=0x561d3189ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:17.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.393+0000 7f8329800640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.393+0000 7f8328e00640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d3189c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.393+0000 7f8329800640 1 -- 172.21.15.46:0/3374623393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 msgr2=0x561d3189c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:17.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.393+0000 7f8329800640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d3189c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.393+0000 7f8329800640 1 -- 172.21.15.46:0/3374623393 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d3189d680 con 0x561d3188dc70 2024-09-10T07:45:17.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.394+0000 7f8329800640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188dc70 0x561d318a15c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8318009830 tx=0x7f8318030b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155a112b42d49de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:17.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.394+0000 7f830b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:17.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.394+0000 7f832a200640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:17.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.394+0000 7f8328e00640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d3189c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:17.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.394+0000 7f830b400640 1 -- 172.21.15.46:0/3374623393 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318060070 con 0x561d3188dc70 2024-09-10T07:45:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.394+0000 7f830b400640 1 -- 172.21.15.46:0/3374623393 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8318046650 con 0x561d3188dc70 2024-09-10T07:45:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.394+0000 7f832ee3f740 1 -- 172.21.15.46:0/3374623393 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8318030550 con 0x561d3188dc70 2024-09-10T07:45:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.395+0000 7f830b400640 1 -- 172.21.15.46:0/3374623393 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f831806c050 con 0x561d3188dc70 2024-09-10T07:45:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.395+0000 7f832ee3f740 10 client.27533 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.395+0000 7f832ee3f740 1 -- 172.21.15.46:0/3374623393 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561d31913780 con 0x561d3188dc70 2024-09-10T07:45:17.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.395+0000 7f830b400640 1 -- 172.21.15.46:0/3374623393 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318030550 con 0x561d3188dc70 2024-09-10T07:45:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.396+0000 7f830b400640 1 -- 172.21.15.46:0/3374623393 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f83180325c0 con 0x561d3188dc70 2024-09-10T07:45:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.396+0000 7f832ee3f740 10 client.27533 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.396+0000 7f832ee3f740 10 client.27533 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:45:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.396+0000 7f832ee3f740 20 client.27533 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.396+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x561d31913d90 0x561d31916180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.396+0000 7f832ee3f740 4 client.27533 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:45:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.396+0000 7f832ee3f740 1 -- 172.21.15.46:0/3374623393 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d317fbe60 con 0x561d31913d90 2024-09-10T07:45:17.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.397+0000 7f8328e00640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x561d31913d90 0x561d31916180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:17.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:16 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:17.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.399+0000 7f8328e00640 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x561d31913d90 0x561d31916180 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3fa91d53f12349d server_cookie=1ead59f29323ba5d in_seq=0 out_seq=0 2024-09-10T07:45:17.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.399+0000 7f830b400640 10 client.27533 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:45:17.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.399+0000 7f830b400640 1 -- 172.21.15.46:0/3374623393 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d317fbe60 con 0x561d31913d90 2024-09-10T07:45:17.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.399+0000 7f830b400640 10 client.27533 handle_command_reply: tid=0 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.399+0000 7f832ee3f740 1 client.27533 shutdown 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 -- 172.21.15.46:0/3374623393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188dc70 msgr2=0x561d318a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188dc70 0x561d318a15c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8318009830 tx=0x7f8318030b40 comp rx=0 tx=0).stop 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 -- 172.21.15.46:0/3374623393 shutdown_connections 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x561d31913d90 0x561d31916180 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d31898e50 0x561d3189ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d3188e590 0x561d3189c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f830b400640 0 client.27533 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 --2- 172.21.15.46:0/3374623393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561d3188dc70 0x561d318a15c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 -- 172.21.15.46:0/3374623393 >> 172.21.15.46:0/3374623393 conn(0x561d3188a200 msgr2=0x561d31897590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:17.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:16 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:17.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 -- 172.21.15.46:0/3374623393 shutdown_connections 2024-09-10T07:45:17.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.400+0000 7f832ee3f740 1 -- 172.21.15.46:0/3374623393 wait complete. 2024-09-10T07:45:17.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.401+0000 7f832ee3f740 20 client.27533 trim_cache size 0 max 16384 2024-09-10T07:45:17.402 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:45:17.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 -- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f830c0a2d00 msgr2=0x7f830c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f830c0a2d00 0x7f830c0a51c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8324104030 tx=0x7f8310011890 comp rx=0 tx=0).stop 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 -- 172.21.15.46:0/2671225068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832419e890 msgr2=0x7f832419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832419e890 0x7f832419ed40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f831c009fd0 tx=0x7f831c00ee50 comp rx=0 tx=0).stop 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 -- 172.21.15.46:0/2671225068 shutdown_connections 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f832419e890 0x7f832419ed40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324106a90 0x7f83241a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f830c0a2d00 0x7f830c0a51c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 --2- 172.21.15.46:0/2671225068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8324100860 0x7f83241a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.402+0000 7f832c200640 1 -- 172.21.15.46:0/2671225068 >> 172.21.15.46:0/2671225068 conn(0x7f8324077230 msgr2=0x7f8324102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.403+0000 7f832c200640 1 -- 172.21.15.46:0/2671225068 shutdown_connections 2024-09-10T07:45:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:17.403+0000 7f832c200640 1 -- 172.21.15.46:0/2671225068 wait complete. 2024-09-10T07:45:18.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:17 smithi046 ceph-mon[30705]: pgmap v659: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-10T07:45:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:17 smithi181 ceph-mon[35397]: pgmap v659: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-10T07:45:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:17 smithi063 ceph-mon[35020]: pgmap v659: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-10T07:45:18.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:45:19.048 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.883+0000 7fe89f200640 1 -- 172.21.15.46:0/1524773768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0100980 msgr2=0x7fe8a0100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.883+0000 7fe89f200640 1 --2- 172.21.15.46:0/1524773768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0100980 0x7fe8a0100d60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe890009900 tx=0x7fe89002a480 comp rx=0 tx=0).stop 2024-09-10T07:45:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.884+0000 7fe89f200640 1 -- 172.21.15.46:0/1524773768 shutdown_connections 2024-09-10T07:45:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.884+0000 7fe89f200640 1 --2- 172.21.15.46:0/1524773768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a0105d90 0x7fe8a0106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.884+0000 7fe89f200640 1 --2- 172.21.15.46:0/1524773768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a01012a0 0x7fe8a0105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.884+0000 7fe89f200640 1 --2- 172.21.15.46:0/1524773768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0100980 0x7fe8a0100d60 secure :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe890009900 tx=0x7fe89002a480 comp rx=0 tx=0).stop 2024-09-10T07:45:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.884+0000 7fe89f200640 1 -- 172.21.15.46:0/1524773768 >> 172.21.15.46:0/1524773768 conn(0x7fe8a00fc610 msgr2=0x7fe8a00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.885+0000 7fe89f200640 1 -- 172.21.15.46:0/1524773768 shutdown_connections 2024-09-10T07:45:19.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.885+0000 7fe89f200640 1 -- 172.21.15.46:0/1524773768 wait complete. 2024-09-10T07:45:19.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.886+0000 7fe89f200640 1 Processor -- start 2024-09-10T07:45:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.886+0000 7fe89f200640 1 -- start start 2024-09-10T07:45:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a01012a0 0x7fe8a01a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a0105d90 0x7fe8a01a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a019ea40 0x7fe8a019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0118430 con 0x7fe8a01012a0 2024-09-10T07:45:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01182b0 con 0x7fe8a019ea40 2024-09-10T07:45:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01185b0 con 0x7fe8a0105d90 2024-09-10T07:45:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a01012a0 0x7fe8a01a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a01012a0 0x7fe8a01a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53694/0 (socket says 172.21.15.46:53694) 2024-09-10T07:45:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.887+0000 7fe89de00640 1 -- 172.21.15.46:0/3577064231 learned_addr learned my addr 172.21.15.46:0/3577064231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.888+0000 7fe89d400640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a0105d90 0x7fe8a01a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.888+0000 7fe89e800640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a019ea40 0x7fe8a019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.889+0000 7fe89de00640 1 -- 172.21.15.46:0/3577064231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a0105d90 msgr2=0x7fe8a01a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.889+0000 7fe89de00640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a0105d90 0x7fe8a01a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.889+0000 7fe89de00640 1 -- 172.21.15.46:0/3577064231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a019ea40 msgr2=0x7fe8a019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.889+0000 7fe89de00640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a019ea40 0x7fe8a019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.889+0000 7fe89de00640 1 -- 172.21.15.46:0/3577064231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a019f560 con 0x7fe8a01012a0 2024-09-10T07:45:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.891+0000 7fe89de00640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a01012a0 0x7fe8a01a4880 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe890005ab0 tx=0x7fe8900318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.891+0000 7fe896a00640 1 -- 172.21.15.46:0/3577064231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89003f030 con 0x7fe8a01012a0 2024-09-10T07:45:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.892+0000 7fe896a00640 1 -- 172.21.15.46:0/3577064231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe890003ce0 con 0x7fe8a01012a0 2024-09-10T07:45:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.892+0000 7fe89f200640 1 -- 172.21.15.46:0/3577064231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a019f7f0 con 0x7fe8a01012a0 2024-09-10T07:45:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.892+0000 7fe89f200640 1 -- 172.21.15.46:0/3577064231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a01ab980 con 0x7fe8a01012a0 2024-09-10T07:45:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.893+0000 7fe896a00640 1 -- 172.21.15.46:0/3577064231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890035070 con 0x7fe8a01012a0 2024-09-10T07:45:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.899+0000 7fe8a785a740 1 Processor -- start 2024-09-10T07:45:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.900+0000 7fe8a785a740 1 -- start start 2024-09-10T07:45:19.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.900+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb7e40 0x55fd78bb8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.900+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb8760 0x55fd78bc2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.900+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 0x55fd78bc5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.900+0000 7fe8a785a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fd78b1e3e0 con 0x55fd78bc3020 2024-09-10T07:45:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.900+0000 7fe8a785a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fd78ac5d80 con 0x55fd78bb7e40 2024-09-10T07:45:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.900+0000 7fe8a785a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fd78b1a3a0 con 0x55fd78bb8760 2024-09-10T07:45:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.901+0000 7fe89e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb7e40 0x55fd78bb8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.901+0000 7fe89e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb7e40 0x55fd78bb8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41724/0 (socket says 172.21.15.46:41724) 2024-09-10T07:45:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.901+0000 7fe89e800640 1 -- 172.21.15.46:0/2532763922 learned_addr learned my addr 172.21.15.46:0/2532763922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:19.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.902+0000 7fe89e800640 1 -- 172.21.15.46:0/2532763922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb8760 msgr2=0x55fd78bc2ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:45:19.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.902+0000 7fe89e800640 1 --2- 172.21.15.46:0/2532763922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb8760 0x55fd78bc2ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.902+0000 7fe89e800640 1 -- 172.21.15.46:0/2532763922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 msgr2=0x55fd78bc5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.902+0000 7fe89e800640 1 --2- 172.21.15.46:0/2532763922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 0x55fd78bc5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.902+0000 7fe89e800640 1 -- 172.21.15.46:0/2532763922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd78ae4ae0 con 0x55fd78bb7e40 2024-09-10T07:45:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.902+0000 7fe89e800640 1 --2- 172.21.15.46:0/2532763922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb7e40 0x55fd78bb8220 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe89800a430 tx=0x7fe898009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce91cc8251e515c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.903+0000 7fe88b000640 1 -- 172.21.15.46:0/2532763922 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898013790 con 0x55fd78bb7e40 2024-09-10T07:45:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.903+0000 7fe88b000640 1 -- 172.21.15.46:0/2532763922 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe898019070 con 0x55fd78bb7e40 2024-09-10T07:45:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.903+0000 7fe88b000640 1 -- 172.21.15.46:0/2532763922 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8980122e0 con 0x55fd78bb7e40 2024-09-10T07:45:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.904+0000 7fe896a00640 1 -- 172.21.15.46:0/3577064231 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe890042030 con 0x7fe8a01012a0 2024-09-10T07:45:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.904+0000 7fe8a785a740 1 -- 172.21.15.46:0/2532763922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb7e40 msgr2=0x55fd78bb8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.904+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2532763922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb7e40 0x55fd78bb8220 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe89800a430 tx=0x7fe898009650 comp rx=0 tx=0).stop 2024-09-10T07:45:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.906+0000 7fe8a785a740 1 -- 172.21.15.46:0/2532763922 shutdown_connections 2024-09-10T07:45:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.907+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2532763922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 0x55fd78bc5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.907+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2532763922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb8760 0x55fd78bc2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.907+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2532763922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb7e40 0x55fd78bb8220 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.907+0000 7fe8a785a740 1 -- 172.21.15.46:0/2532763922 >> 172.21.15.46:0/2532763922 conn(0x55fd78bb43d0 msgr2=0x55fd78bb47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.907+0000 7fe8a785a740 1 -- 172.21.15.46:0/2532763922 shutdown_connections 2024-09-10T07:45:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.908+0000 7fe8a785a740 1 -- 172.21.15.46:0/2532763922 wait complete. 2024-09-10T07:45:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.908+0000 7fe896a00640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8840a2c10 0x7fe8840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.908+0000 7fe896a00640 1 -- 172.21.15.46:0/3577064231 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe8900ef590 con 0x7fe8a01012a0 2024-09-10T07:45:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.911+0000 7fe89d400640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8840a2c10 0x7fe8840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.913+0000 7fe89d400640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8840a2c10 0x7fe8840a50d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55fd78bb5750 tx=0x7fe88c0094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.913+0000 7fe8a785a740 1 -- start start 2024-09-10T07:45:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.914+0000 7fe8a785a740 1 Processor -- start 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb7e40 0x55fd78bbfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bc3020 0x55fd78bc08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe8a785a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fd78bc5bf0 con 0x55fd78bb7e40 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe8a785a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fd78ad90f0 con 0x55fd78bb8760 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe8a785a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fd78bc5d70 con 0x55fd78bc3020 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe8a785a740 10 client.0 _ll_register_callbacks cb 0x55fd78ac0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe89de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb7e40 0x55fd78bbfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe89de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb7e40 0x55fd78bbfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53708/0 (socket says 172.21.15.46:53708) 2024-09-10T07:45:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.917+0000 7fe89de00640 1 -- 172.21.15.46:0/1703242453 learned_addr learned my addr 172.21.15.46:0/1703242453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.918+0000 7fe89d400640 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.918+0000 7fe89e800640 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bc3020 0x55fd78bc08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.919+0000 7fe89d400640 1 -- 172.21.15.46:0/1703242453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bc3020 msgr2=0x55fd78bc08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.919+0000 7fe89d400640 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bc3020 0x55fd78bc08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.919+0000 7fe89d400640 1 -- 172.21.15.46:0/1703242453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb7e40 msgr2=0x55fd78bbfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.919+0000 7fe89d400640 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb7e40 0x55fd78bbfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.919+0000 7fe89d400640 1 -- 172.21.15.46:0/1703242453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd7888f3c0 con 0x55fd78bb8760 2024-09-10T07:45:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.920+0000 7fe89d400640 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc0380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe8a01a0230 tx=0x7fe88c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1dc4b8a24593e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.920+0000 7fe87f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.921+0000 7fe87f400640 1 -- 172.21.15.46:0/1703242453 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c023020 con 0x55fd78bb8760 2024-09-10T07:45:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.921+0000 7fe87f400640 1 -- 172.21.15.46:0/1703242453 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe88c013450 con 0x55fd78bb8760 2024-09-10T07:45:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.921+0000 7fe8a785a740 1 -- 172.21.15.46:0/1703242453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe898019070 con 0x55fd78bb8760 2024-09-10T07:45:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.921+0000 7fe87f400640 1 -- 172.21.15.46:0/1703242453 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe88c01b040 con 0x55fd78bb8760 2024-09-10T07:45:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.921+0000 7fe8a785a740 10 client.27545 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.921+0000 7fe8a785a740 1 -- 172.21.15.46:0/1703242453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55fd78bd1cc0 con 0x55fd78bb8760 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe87f400640 1 -- 172.21.15.46:0/1703242453 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe898019070 con 0x55fd78bb8760 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe87f400640 1 -- 172.21.15.46:0/1703242453 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7fe88c030020 con 0x55fd78bb8760 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe8a785a740 10 client.27545 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe8a785a740 10 client.27545 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe8a785a740 20 client.27545 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe8a785a740 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd78bd3600 0x55fd78bd5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe8a785a740 4 client.27545 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe8a785a740 1 -- 172.21.15.46:0/1703242453 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd78b2b950 con 0x55fd78bd3600 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.922+0000 7fe89de00640 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd78bd3600 0x55fd78bd5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.923+0000 7fe87f400640 10 client.27545 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:45:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.924+0000 7fe89de00640 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd78bd3600 0x55fd78bd5a60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=255cac8239110d33 server_cookie=4bdf5f07bc33a317 in_seq=0 out_seq=0 2024-09-10T07:45:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.925+0000 7fe87f400640 1 -- 172.21.15.46:0/1703242453 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd78b2b950 con 0x55fd78bd3600 2024-09-10T07:45:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.925+0000 7fe87f400640 10 client.27545 handle_command_reply: tid=0 2024-09-10T07:45:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.925+0000 7fe8a785a740 1 client.27545 shutdown 2024-09-10T07:45:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.926+0000 7fe8a785a740 1 -- 172.21.15.46:0/1703242453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 msgr2=0x55fd78bc0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.926+0000 7fe8a785a740 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc0380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe8a01a0230 tx=0x7fe88c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 1 -- 172.21.15.46:0/1703242453 shutdown_connections 2024-09-10T07:45:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd78bd3600 0x55fd78bd5a60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe87f400640 0 client.27545 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:45:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bc3020 0x55fd78bc08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc0380 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 1 --2- 172.21.15.46:0/1703242453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb7e40 0x55fd78bbfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 1 -- 172.21.15.46:0/1703242453 >> 172.21.15.46:0/1703242453 conn(0x55fd78bb43d0 msgr2=0x55fd78bba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 1 -- 172.21.15.46:0/1703242453 shutdown_connections 2024-09-10T07:45:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 1 -- 172.21.15.46:0/1703242453 wait complete. 2024-09-10T07:45:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.927+0000 7fe8a785a740 20 client.27545 trim_cache size 0 max 16384 2024-09-10T07:45:19.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.956+0000 7fe8a785a740 1 Processor -- start 2024-09-10T07:45:19.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.956+0000 7fe8a785a740 1 -- start start 2024-09-10T07:45:19.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bbf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb8760 0x55fd78bc16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bc3020 0x55fd78bdb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe8a785a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe88000fb60 con 0x55fd78bb8760 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe8a785a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8800116d0 con 0x55fd78bc3020 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe8a785a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe880010230 con 0x55fd78bb7e40 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe89de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bbf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe89de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bbf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50238/0 (socket says 172.21.15.46:50238) 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe89de00640 1 -- 172.21.15.46:0/2595250688 learned_addr learned my addr 172.21.15.46:0/2595250688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.957+0000 7fe89d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb8760 0x55fd78bc16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.958+0000 7fe89e800640 1 --2- 172.21.15.46:0/2595250688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bc3020 0x55fd78bdb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.958+0000 7fe89de00640 1 -- 172.21.15.46:0/2595250688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bc3020 msgr2=0x55fd78bdb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.958+0000 7fe89de00640 1 --2- 172.21.15.46:0/2595250688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bc3020 0x55fd78bdb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.958+0000 7fe89de00640 1 -- 172.21.15.46:0/2595250688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb8760 msgr2=0x55fd78bc16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.958+0000 7fe89de00640 1 --2- 172.21.15.46:0/2595250688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb8760 0x55fd78bc16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.958+0000 7fe89de00640 1 -- 172.21.15.46:0/2595250688 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd78bc2550 con 0x55fd78bb7e40 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.958+0000 7fe89de00640 1 --2- 172.21.15.46:0/2595250688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bbf720 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe890005ec0 tx=0x7fe890032fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=828e9a44b62bf6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.959+0000 7fe88a600640 1 -- 172.21.15.46:0/2595250688 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8900d9070 con 0x55fd78bb7e40 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.959+0000 7fe88a600640 1 -- 172.21.15.46:0/2595250688 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8900bba10 con 0x55fd78bb7e40 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.959+0000 7fe88a600640 1 -- 172.21.15.46:0/2595250688 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8900ee040 con 0x55fd78bb7e40 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.960+0000 7fe8a785a740 1 -- 172.21.15.46:0/2595250688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 msgr2=0x55fd78bbf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.960+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2595250688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bbf720 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe890005ec0 tx=0x7fe890032fc0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.961+0000 7fe8a785a740 1 -- 172.21.15.46:0/2595250688 shutdown_connections 2024-09-10T07:45:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.961+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2595250688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bc3020 0x55fd78bdb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.961+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2595250688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bb8760 0x55fd78bc16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.961+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2595250688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bbf720 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.961+0000 7fe8a785a740 1 -- 172.21.15.46:0/2595250688 >> 172.21.15.46:0/2595250688 conn(0x55fd78bb43d0 msgr2=0x55fd78bc0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.961+0000 7fe8a785a740 1 -- 172.21.15.46:0/2595250688 shutdown_connections 2024-09-10T07:45:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.961+0000 7fe8a785a740 1 -- 172.21.15.46:0/2595250688 wait complete. 2024-09-10T07:45:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.962+0000 7fe8a785a740 1 -- start start 2024-09-10T07:45:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.963+0000 7fe8a785a740 1 Processor -- start 2024-09-10T07:45:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bcb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe8a785a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 0x55fd78bc4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe8a785a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe880010fd0 con 0x55fd78bc3020 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe8a785a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe880010900 con 0x55fd78bb8760 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe8a785a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe88000db00 con 0x55fd78bb7e40 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe8a785a740 10 client.0 _ll_register_callbacks cb 0x7fe88c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe89d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe89d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41760/0 (socket says 172.21.15.46:41760) 2024-09-10T07:45:19.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.964+0000 7fe89d400640 1 -- 172.21.15.46:0/2086201151 learned_addr learned my addr 172.21.15.46:0/2086201151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.965+0000 7fe89de00640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bcb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.965+0000 7fe89d400640 1 -- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 msgr2=0x55fd78bcb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.965+0000 7fe89d400640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bcb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.965+0000 7fe89d400640 1 -- 172.21.15.46:0/2086201151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 msgr2=0x55fd78bc4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.965+0000 7fe89e800640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 0x55fd78bc4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.965+0000 7fe89d400640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 0x55fd78bc4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.965+0000 7fe89d400640 1 -- 172.21.15.46:0/2086201151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd78bc4c90 con 0x55fd78bb8760 2024-09-10T07:45:19.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.965+0000 7fe89de00640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bcb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.966+0000 7fe89e800640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 0x55fd78bc4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.966+0000 7fe89d400640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc3f00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe88c026360 tx=0x7fe88c009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d109235299bad441 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.966+0000 7fe87f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.967+0000 7fe87f400640 1 -- 172.21.15.46:0/2086201151 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c038070 con 0x55fd78bb8760 2024-09-10T07:45:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.967+0000 7fe87f400640 1 -- 172.21.15.46:0/2086201151 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe88c0307c0 con 0x55fd78bb8760 2024-09-10T07:45:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.967+0000 7fe8a785a740 1 -- 172.21.15.46:0/2086201151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8900bba10 con 0x55fd78bb8760 2024-09-10T07:45:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.968+0000 7fe87f400640 1 -- 172.21.15.46:0/2086201151 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe88c041050 con 0x55fd78bb8760 2024-09-10T07:45:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.968+0000 7fe8a785a740 10 client.27551 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.968+0000 7fe8a785a740 1 -- 172.21.15.46:0/2086201151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55fd78c4f410 con 0x55fd78bb8760 2024-09-10T07:45:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.969+0000 7fe87f400640 1 -- 172.21.15.46:0/2086201151 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8900bba10 con 0x55fd78bb8760 2024-09-10T07:45:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.969+0000 7fe87f400640 1 -- 172.21.15.46:0/2086201151 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7fe88c030020 con 0x55fd78bb8760 2024-09-10T07:45:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.969+0000 7fe8a785a740 10 client.27551 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.969+0000 7fe8a785a740 10 client.27551 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:45:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.969+0000 7fe8a785a740 20 client.27551 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.969+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd78c4fb00 0x55fd78c51f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.969+0000 7fe8a785a740 4 client.27551 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:45:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.969+0000 7fe8a785a740 1 -- 172.21.15.46:0/2086201151 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd78b25e60 con 0x55fd78c4fb00 2024-09-10T07:45:19.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.970+0000 7fe89de00640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd78c4fb00 0x55fd78c51f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:19.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.971+0000 7fe89de00640 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd78c4fb00 0x55fd78c51f20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26a772eb5c3623e0 server_cookie=9d50d81ed7f8c2df in_seq=0 out_seq=0 2024-09-10T07:45:19.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.971+0000 7fe87f400640 10 client.27551 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:45:19.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.971+0000 7fe87f400640 1 -- 172.21.15.46:0/2086201151 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd78b25e60 con 0x55fd78c4fb00 2024-09-10T07:45:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.971+0000 7fe87f400640 10 client.27551 handle_command_reply: tid=0 2024-09-10T07:45:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.971+0000 7fe8a785a740 1 client.27551 shutdown 2024-09-10T07:45:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.974+0000 7fe8a785a740 1 -- 172.21.15.46:0/2086201151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 msgr2=0x55fd78bc3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.974+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc3f00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe88c026360 tx=0x7fe88c009390 comp rx=0 tx=0).stop 2024-09-10T07:45:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe8a785a740 1 -- 172.21.15.46:0/2086201151 shutdown_connections 2024-09-10T07:45:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd78bc3020 0x55fd78bc4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fd78bb8760 0x55fd78bc3f00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55fd78c4fb00 0x55fd78c51f20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe8a785a740 1 --2- 172.21.15.46:0/2086201151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd78bb7e40 0x55fd78bcb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe87f400640 0 client.27551 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:45:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe8a785a740 1 -- 172.21.15.46:0/2086201151 >> 172.21.15.46:0/2086201151 conn(0x55fd78bb43d0 msgr2=0x55fd78bb2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe8a785a740 1 -- 172.21.15.46:0/2086201151 shutdown_connections 2024-09-10T07:45:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.975+0000 7fe8a785a740 1 -- 172.21.15.46:0/2086201151 wait complete. 2024-09-10T07:45:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.976+0000 7fe8a785a740 20 client.27551 trim_cache size 0 max 16384 2024-09-10T07:45:19.976 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:45:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 -- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8840a2c10 msgr2=0x7fe8840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8840a2c10 0x7fe8840a50d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55fd78bb5750 tx=0x7fe88c0094b0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 -- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a01012a0 msgr2=0x7fe8a01a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a01012a0 0x7fe8a01a4880 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe890005ab0 tx=0x7fe8900318d0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 -- 172.21.15.46:0/3577064231 shutdown_connections 2024-09-10T07:45:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a019ea40 0x7fe8a019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a0105d90 0x7fe8a01a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8840a2c10 0x7fe8840a50d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 --2- 172.21.15.46:0/3577064231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a01012a0 0x7fe8a01a4880 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.980+0000 7fe89f200640 1 -- 172.21.15.46:0/3577064231 >> 172.21.15.46:0/3577064231 conn(0x7fe8a00fc610 msgr2=0x7fe8a010f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.981+0000 7fe89f200640 1 -- 172.21.15.46:0/3577064231 shutdown_connections 2024-09-10T07:45:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:19.981+0000 7fe89f200640 1 -- 172.21.15.46:0/3577064231 wait complete. 2024-09-10T07:45:20.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:19 smithi046 ceph-mon[30705]: pgmap v660: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-10T07:45:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.365+0000 7f827f400640 1 -- 172.21.15.46:0/1433372044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82780737d0 msgr2=0x7f8278112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.365+0000 7f827f400640 1 --2- 172.21.15.46:0/1433372044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82780737d0 0x7f8278112540 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8270009be0 tx=0x7f827002a6c0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.366+0000 7f827f400640 1 -- 172.21.15.46:0/1433372044 shutdown_connections 2024-09-10T07:45:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.366+0000 7f827f400640 1 --2- 172.21.15.46:0/1433372044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82780737d0 0x7f8278112540 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.366+0000 7f827f400640 1 --2- 172.21.15.46:0/1433372044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8278072d20 0x7f8278073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.366+0000 7f827f400640 1 --2- 172.21.15.46:0/1433372044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278109930 0x7f8278109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.366+0000 7f827f400640 1 -- 172.21.15.46:0/1433372044 >> 172.21.15.46:0/1433372044 conn(0x7f827806ece0 msgr2=0x7f8278071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.366+0000 7f827f400640 1 -- 172.21.15.46:0/1433372044 shutdown_connections 2024-09-10T07:45:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.366+0000 7f827f400640 1 -- 172.21.15.46:0/1433372044 wait complete. 2024-09-10T07:45:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.368+0000 7f827f400640 1 Processor -- start 2024-09-10T07:45:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.368+0000 7f827f400640 1 -- start start 2024-09-10T07:45:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.368+0000 7f827f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278072d20 0x7f82781a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.368+0000 7f827f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82780737d0 0x7f82781a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.368+0000 7f827f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8278109930 0x7f82781a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.368+0000 7f827f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f827810a7b0 con 0x7f82780737d0 2024-09-10T07:45:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.368+0000 7f827f400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f827810a630 con 0x7f8278072d20 2024-09-10T07:45:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.368+0000 7f827f400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f827810a930 con 0x7f8278109930 2024-09-10T07:45:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.369+0000 7f827d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8278109930 0x7f82781a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.369+0000 7f827d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8278109930 0x7f82781a9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50278/0 (socket says 172.21.15.46:50278) 2024-09-10T07:45:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.369+0000 7f827d400640 1 -- 172.21.15.46:0/1333077751 learned_addr learned my addr 172.21.15.46:0/1333077751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.369+0000 7f827ca00640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278072d20 0x7f82781a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.369+0000 7f827ca00640 1 -- 172.21.15.46:0/1333077751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8278109930 msgr2=0x7f82781a9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.369+0000 7f827ca00640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8278109930 0x7f82781a9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.370+0000 7f827ca00640 1 -- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82780737d0 msgr2=0x7f82781a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.370+0000 7f827ca00640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82780737d0 0x7f82781a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.370+0000 7f827ca00640 1 -- 172.21.15.46:0/1333077751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82781aa350 con 0x7f8278072d20 2024-09-10T07:45:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.370+0000 7f827d400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8278109930 0x7f82781a9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.370+0000 7f827ca00640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278072d20 0x7f82781a2730 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c970 tx=0x7f826c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.371+0000 7f8275600640 1 -- 172.21.15.46:0/1333077751 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c007bb0 con 0x7f8278072d20 2024-09-10T07:45:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.371+0000 7f8275600640 1 -- 172.21.15.46:0/1333077751 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826c007d50 con 0x7f8278072d20 2024-09-10T07:45:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.371+0000 7f827f400640 1 -- 172.21.15.46:0/1333077751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82781aa640 con 0x7f8278072d20 2024-09-10T07:45:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.371+0000 7f8275600640 1 -- 172.21.15.46:0/1333077751 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c00f450 con 0x7f8278072d20 2024-09-10T07:45:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.371+0000 7f827f400640 1 -- 172.21.15.46:0/1333077751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781aabe0 con 0x7f8278072d20 2024-09-10T07:45:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.382+0000 7f828225c740 1 Processor -- start 2024-09-10T07:45:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.383+0000 7f8275600640 1 -- 172.21.15.46:0/1333077751 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f826c004ed0 con 0x7f8278072d20 2024-09-10T07:45:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.383+0000 7f828225c740 1 -- start start 2024-09-10T07:45:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.383+0000 7f828225c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a25c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.384+0000 7f828225c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c9f0 0x55e83a2614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.384+0000 7f828225c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a263f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.384+0000 7f828225c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e83a1c1100 con 0x55e83a25c9f0 2024-09-10T07:45:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.384+0000 7f828225c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e83a1691b0 con 0x55e83a25c0b0 2024-09-10T07:45:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.384+0000 7f828225c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e83a1bd0c0 con 0x55e83a261a80 2024-09-10T07:45:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.384+0000 7f827d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a25c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.384+0000 7f827d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a25c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41784/0 (socket says 172.21.15.46:41784) 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.384+0000 7f827d400640 1 -- 172.21.15.46:0/2419388208 learned_addr learned my addr 172.21.15.46:0/2419388208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.385+0000 7f8275600640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82600a2cb0 0x7f82600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.386+0000 7f8275600640 1 -- 172.21.15.46:0/1333077751 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f826c0cefb0 con 0x7f8278072d20 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.386+0000 7f827d400640 1 -- 172.21.15.46:0/2419388208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 msgr2=0x55e83a263f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.386+0000 7f827d400640 1 --2- 172.21.15.46:0/2419388208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a263f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.386+0000 7f827d400640 1 -- 172.21.15.46:0/2419388208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c9f0 msgr2=0x55e83a2614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.386+0000 7f827d400640 1 --2- 172.21.15.46:0/2419388208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c9f0 0x55e83a2614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.386+0000 7f827d400640 1 -- 172.21.15.46:0/2419388208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e839ed4210 con 0x55e83a25c0b0 2024-09-10T07:45:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.386+0000 7f827d400640 1 --2- 172.21.15.46:0/2419388208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a25c4b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8270030580 tx=0x7f82700305b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a21b7a1ceb96479 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.387+0000 7f8269c00640 1 -- 172.21.15.46:0/2419388208 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270041030 con 0x55e83a25c0b0 2024-09-10T07:45:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.387+0000 7f8269c00640 1 -- 172.21.15.46:0/2419388208 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8270003e50 con 0x55e83a25c0b0 2024-09-10T07:45:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.387+0000 7f8269c00640 1 -- 172.21.15.46:0/2419388208 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270036040 con 0x55e83a25c0b0 2024-09-10T07:45:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.389+0000 7f828225c740 1 -- 172.21.15.46:0/2419388208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 msgr2=0x55e83a25c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.389+0000 7f828225c740 1 --2- 172.21.15.46:0/2419388208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a25c4b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8270030580 tx=0x7f82700305b0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.389+0000 7f828225c740 1 -- 172.21.15.46:0/2419388208 shutdown_connections 2024-09-10T07:45:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.389+0000 7f828225c740 1 --2- 172.21.15.46:0/2419388208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a263f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.389+0000 7f828225c740 1 --2- 172.21.15.46:0/2419388208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c9f0 0x55e83a2614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.389+0000 7f828225c740 1 --2- 172.21.15.46:0/2419388208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a25c4b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.389+0000 7f828225c740 1 -- 172.21.15.46:0/2419388208 >> 172.21.15.46:0/2419388208 conn(0x55e83a257720 msgr2=0x55e83a257b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.389+0000 7f828225c740 1 -- 172.21.15.46:0/2419388208 shutdown_connections 2024-09-10T07:45:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.390+0000 7f827d400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82600a2cb0 0x7f82600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.390+0000 7f828225c740 1 -- 172.21.15.46:0/2419388208 wait complete. 2024-09-10T07:45:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.390+0000 7f828225c740 1 -- start start 2024-09-10T07:45:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.391+0000 7f828225c740 1 Processor -- start 2024-09-10T07:45:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.393+0000 7f827d400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82600a2cb0 0x7f82600a5170 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8270006010 tx=0x7f8270030430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.394+0000 7f828225c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c0b0 0x55e83a26ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.394+0000 7f828225c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c9f0 0x55e83a26ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.394+0000 7f828225c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a275fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.394+0000 7f828225c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e83a26d200 con 0x55e83a25c0b0 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.394+0000 7f828225c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e83a14a090 con 0x55e83a25c9f0 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.394+0000 7f828225c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e83a26d380 con 0x55e83a261a80 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.394+0000 7f828225c740 10 client.0 _ll_register_callbacks cb 0x55e83a19ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.395+0000 7f827d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a275fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.395+0000 7f827d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a275fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50302/0 (socket says 172.21.15.46:50302) 2024-09-10T07:45:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.395+0000 7f827d400640 1 -- 172.21.15.46:0/2025719258 learned_addr learned my addr 172.21.15.46:0/2025719258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.395+0000 7f827d400640 1 -- 172.21.15.46:0/2025719258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c9f0 msgr2=0x55e83a26ef70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:45:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.395+0000 7f827d400640 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c9f0 0x55e83a26ef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.395+0000 7f827d400640 1 -- 172.21.15.46:0/2025719258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c0b0 msgr2=0x55e83a26ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.395+0000 7f827d400640 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c0b0 0x55e83a26ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.395+0000 7f827d400640 1 -- 172.21.15.46:0/2025719258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e839fa0620 con 0x55e83a261a80 2024-09-10T07:45:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.398+0000 7f827d400640 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a275fb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8270030580 tx=0x7f82700305b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a5b097c542ce72e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.398+0000 7f825e000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.398+0000 7f825e000640 1 -- 172.21.15.46:0/2025719258 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827004c070 con 0x55e83a261a80 2024-09-10T07:45:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.399+0000 7f825e000640 1 -- 172.21.15.46:0/2025719258 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f827006c670 con 0x55e83a261a80 2024-09-10T07:45:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.399+0000 7f828225c740 1 -- 172.21.15.46:0/2025719258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8270003e50 con 0x55e83a261a80 2024-09-10T07:45:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:19 smithi181 ceph-mon[35397]: pgmap v660: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-10T07:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.399+0000 7f825e000640 1 -- 172.21.15.46:0/2025719258 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8270041040 con 0x55e83a261a80 2024-09-10T07:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.399+0000 7f828225c740 10 client.27505 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.399+0000 7f828225c740 1 -- 172.21.15.46:0/2025719258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e83a26d200 con 0x55e83a261a80 2024-09-10T07:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.400+0000 7f825e000640 1 -- 172.21.15.46:0/2025719258 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8270003e50 con 0x55e83a261a80 2024-09-10T07:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.400+0000 7f825e000640 1 -- 172.21.15.46:0/2025719258 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f827003d9b0 con 0x55e83a261a80 2024-09-10T07:45:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:19 smithi063 ceph-mon[35020]: pgmap v660: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-10T07:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.401+0000 7f828225c740 10 client.27505 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.401+0000 7f828225c740 10 client.27505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.401+0000 7f828225c740 20 client.27505 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.401+0000 7f828225c740 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e83a265ac0 0x55e83a267f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.401+0000 7f828225c740 4 client.27505 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.401+0000 7f828225c740 1 -- 172.21.15.46:0/2025719258 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e83a1ce830 con 0x55e83a265ac0 2024-09-10T07:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.401+0000 7f827ca00640 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e83a265ac0 0x55e83a267f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.402+0000 7f827ca00640 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e83a265ac0 0x55e83a267f10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e66dbb7dc2ad2946 server_cookie=2b085086b19cc9d1 in_seq=0 out_seq=0 2024-09-10T07:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.402+0000 7f825e000640 10 client.27505 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.403+0000 7f825e000640 1 -- 172.21.15.46:0/2025719258 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e83a1ce830 con 0x55e83a265ac0 2024-09-10T07:45:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.403+0000 7f825e000640 10 client.27505 handle_command_reply: tid=0 2024-09-10T07:45:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.404+0000 7f828225c740 1 client.27505 shutdown 2024-09-10T07:45:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 -- 172.21.15.46:0/2025719258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 msgr2=0x55e83a275fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a275fb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8270030580 tx=0x7f82700305b0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 -- 172.21.15.46:0/2025719258 shutdown_connections 2024-09-10T07:45:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e83a265ac0 0x55e83a267f10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a261a80 0x55e83a275fb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c9f0 0x55e83a26ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 --2- 172.21.15.46:0/2025719258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c0b0 0x55e83a26ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 -- 172.21.15.46:0/2025719258 >> 172.21.15.46:0/2025719258 conn(0x55e83a257720 msgr2=0x55e83a258f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f825e000640 0 client.27505 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.405+0000 7f828225c740 1 -- 172.21.15.46:0/2025719258 shutdown_connections 2024-09-10T07:45:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.406+0000 7f828225c740 1 -- 172.21.15.46:0/2025719258 wait complete. 2024-09-10T07:45:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.406+0000 7f828225c740 20 client.27505 trim_cache size 0 max 16384 2024-09-10T07:45:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.440+0000 7f828225c740 1 Processor -- start 2024-09-10T07:45:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.440+0000 7f828225c740 1 -- start start 2024-09-10T07:45:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.440+0000 7f828225c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a26c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.440+0000 7f828225c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c9f0 0x55e83a26ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.440+0000 7f828225c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a261a80 0x55e83a26ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.440+0000 7f828225c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82781b0c30 con 0x55e83a261a80 2024-09-10T07:45:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.440+0000 7f828225c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82781b27a0 con 0x55e83a25c0b0 2024-09-10T07:45:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.440+0000 7f828225c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82781b1300 con 0x55e83a25c9f0 2024-09-10T07:45:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.441+0000 7f827ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a26c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.441+0000 7f8277e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c9f0 0x55e83a26ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.441+0000 7f8277e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c9f0 0x55e83a26ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50308/0 (socket says 172.21.15.46:50308) 2024-09-10T07:45:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.441+0000 7f827ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a26c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41794/0 (socket says 172.21.15.46:41794) 2024-09-10T07:45:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.443+0000 7f8277e00640 1 -- 172.21.15.46:0/556171563 learned_addr learned my addr 172.21.15.46:0/556171563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.441+0000 7f827d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a261a80 0x55e83a26ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.443+0000 7f827ca00640 1 -- 172.21.15.46:0/556171563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c9f0 msgr2=0x55e83a26ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.443+0000 7f827ca00640 1 --2- 172.21.15.46:0/556171563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c9f0 0x55e83a26ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.443+0000 7f827ca00640 1 -- 172.21.15.46:0/556171563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a261a80 msgr2=0x55e83a26ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.443+0000 7f827ca00640 1 --2- 172.21.15.46:0/556171563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a261a80 0x55e83a26ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.443+0000 7f827ca00640 1 -- 172.21.15.46:0/556171563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e83a257540 con 0x55e83a25c0b0 2024-09-10T07:45:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.443+0000 7f827d400640 1 --2- 172.21.15.46:0/556171563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a261a80 0x55e83a26ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.444+0000 7f827ca00640 1 --2- 172.21.15.46:0/556171563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a26c7d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55e83a25d420 tx=0x7f826c0d1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ee645796447c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.444+0000 7f8269200640 1 -- 172.21.15.46:0/556171563 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c01d040 con 0x55e83a25c0b0 2024-09-10T07:45:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.445+0000 7f8269200640 1 -- 172.21.15.46:0/556171563 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826c09f840 con 0x55e83a25c0b0 2024-09-10T07:45:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.445+0000 7f8269200640 1 -- 172.21.15.46:0/556171563 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c097b00 con 0x55e83a25c0b0 2024-09-10T07:45:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.446+0000 7f828225c740 1 -- 172.21.15.46:0/556171563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 msgr2=0x55e83a26c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.446+0000 7f828225c740 1 --2- 172.21.15.46:0/556171563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a26c7d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55e83a25d420 tx=0x7f826c0d1040 comp rx=0 tx=0).stop 2024-09-10T07:45:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.447+0000 7f828225c740 1 -- 172.21.15.46:0/556171563 shutdown_connections 2024-09-10T07:45:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.447+0000 7f828225c740 1 --2- 172.21.15.46:0/556171563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a261a80 0x55e83a26ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.447+0000 7f828225c740 1 --2- 172.21.15.46:0/556171563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c9f0 0x55e83a26ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.447+0000 7f828225c740 1 --2- 172.21.15.46:0/556171563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a25c0b0 0x55e83a26c7d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.447+0000 7f828225c740 1 -- 172.21.15.46:0/556171563 >> 172.21.15.46:0/556171563 conn(0x55e83a257720 msgr2=0x55e83a25dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.447+0000 7f828225c740 1 -- 172.21.15.46:0/556171563 shutdown_connections 2024-09-10T07:45:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.447+0000 7f828225c740 1 -- 172.21.15.46:0/556171563 wait complete. 2024-09-10T07:45:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.448+0000 7f828225c740 1 -- start start 2024-09-10T07:45:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.448+0000 7f828225c740 1 Processor -- start 2024-09-10T07:45:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f828225c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c0b0 0x55e83a27d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f828225c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c9f0 0x55e83a27d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f828225c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a261a80 0x55e83a27db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f828225c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82781b20a0 con 0x55e83a25c9f0 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f828225c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82781b19d0 con 0x55e83a261a80 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f828225c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82781aec10 con 0x55e83a25c0b0 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f828225c740 10 client.0 _ll_register_callbacks cb 0x7f82700304a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f827ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c0b0 0x55e83a27d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f827ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c0b0 0x55e83a27d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50318/0 (socket says 172.21.15.46:50318) 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f827ca00640 1 -- 172.21.15.46:0/2107684029 learned_addr learned my addr 172.21.15.46:0/2107684029 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.449+0000 7f827d400640 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a261a80 0x55e83a27db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f827ca00640 1 -- 172.21.15.46:0/2107684029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a261a80 msgr2=0x55e83a27db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f827ca00640 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a261a80 0x55e83a27db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f827ca00640 1 -- 172.21.15.46:0/2107684029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c9f0 msgr2=0x55e83a27d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f827ca00640 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c9f0 0x55e83a27d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f827ca00640 1 -- 172.21.15.46:0/2107684029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e83a2dafc0 con 0x55e83a25c0b0 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f827d400640 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a261a80 0x55e83a27db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f827ca00640 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c0b0 0x55e83a27d110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f826c097e60 tx=0x7f826c09b550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b863219006d93b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f825e000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.450+0000 7f825e000640 1 -- 172.21.15.46:0/2107684029 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c01d040 con 0x55e83a25c0b0 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.451+0000 7f825e000640 1 -- 172.21.15.46:0/2107684029 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f826c099790 con 0x55e83a25c0b0 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.451+0000 7f828225c740 1 -- 172.21.15.46:0/2107684029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f826c09f840 con 0x55e83a25c0b0 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.451+0000 7f825e000640 1 -- 172.21.15.46:0/2107684029 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f826c0a4100 con 0x55e83a25c0b0 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.451+0000 7f828225c740 10 client.27511 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.451+0000 7f828225c740 1 -- 172.21.15.46:0/2107684029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e83a2dc2b0 con 0x55e83a25c0b0 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.451+0000 7f825e000640 1 -- 172.21.15.46:0/2107684029 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826c09f840 con 0x55e83a25c0b0 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.451+0000 7f825e000640 1 -- 172.21.15.46:0/2107684029 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f826c0b41b0 con 0x55e83a25c0b0 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.452+0000 7f828225c740 10 client.27511 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.452+0000 7f828225c740 10 client.27511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.452+0000 7f828225c740 20 client.27511 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.452+0000 7f828225c740 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e83a2dcc40 0x55e83a2df060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.452+0000 7f828225c740 4 client.27511 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:45:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.452+0000 7f828225c740 1 -- 172.21.15.46:0/2107684029 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e83a1c8b80 con 0x55e83a2dcc40 2024-09-10T07:45:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.452+0000 7f8277e00640 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e83a2dcc40 0x55e83a2df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.453+0000 7f8277e00640 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e83a2dcc40 0x55e83a2df060 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56d0db59f8d946f server_cookie=caa662f138bd4d05 in_seq=0 out_seq=0 2024-09-10T07:45:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.453+0000 7f825e000640 10 client.27511 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.542+0000 7f825e000640 1 -- 172.21.15.46:0/2107684029 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e83a1c8b80 con 0x55e83a2dcc40 2024-09-10T07:45:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.542+0000 7f825e000640 10 client.27511 handle_command_reply: tid=0 2024-09-10T07:45:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.542+0000 7f828225c740 1 client.27511 shutdown 2024-09-10T07:45:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 -- 172.21.15.46:0/2107684029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c0b0 msgr2=0x55e83a27d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c0b0 0x55e83a27d110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f826c097e60 tx=0x7f826c09b550 comp rx=0 tx=0).stop 2024-09-10T07:45:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 -- 172.21.15.46:0/2107684029 shutdown_connections 2024-09-10T07:45:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e83a2dcc40 0x55e83a2df060 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e83a261a80 0x55e83a27db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e83a25c9f0 0x55e83a27d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 --2- 172.21.15.46:0/2107684029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e83a25c0b0 0x55e83a27d110 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 -- 172.21.15.46:0/2107684029 >> 172.21.15.46:0/2107684029 conn(0x55e83a257720 msgr2=0x55e83a256060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f825e000640 0 client.27511 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.543+0000 7f828225c740 1 -- 172.21.15.46:0/2107684029 shutdown_connections 2024-09-10T07:45:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.544+0000 7f828225c740 1 -- 172.21.15.46:0/2107684029 wait complete. 2024-09-10T07:45:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.544+0000 7f828225c740 20 client.27511 trim_cache size 0 max 16384 2024-09-10T07:45:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:45:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:45:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:45:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:45:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 -- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82600a2cb0 msgr2=0x7f82600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82600a2cb0 0x7f82600a5170 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8270006010 tx=0x7f8270030430 comp rx=0 tx=0).stop 2024-09-10T07:45:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 -- 172.21.15.46:0/1333077751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278072d20 msgr2=0x7f82781a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278072d20 0x7f82781a2730 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c970 tx=0x7f826c00ce30 comp rx=0 tx=0).stop 2024-09-10T07:45:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 -- 172.21.15.46:0/1333077751 shutdown_connections 2024-09-10T07:45:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8278109930 0x7f82781a9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82600a2cb0 0x7f82600a5170 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82780737d0 0x7f82781a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 --2- 172.21.15.46:0/1333077751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278072d20 0x7f82781a2730 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 -- 172.21.15.46:0/1333077751 >> 172.21.15.46:0/1333077751 conn(0x7f827806ece0 msgr2=0x7f827810d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 -- 172.21.15.46:0/1333077751 shutdown_connections 2024-09-10T07:45:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:20.545+0000 7f827f400640 1 -- 172.21.15.46:0/1333077751 wait complete. 2024-09-10T07:45:20.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 480.0 s, 545.8 tps, lat 174.816 ms stddev 186.494 2024-09-10T07:45:21.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:45:21.725 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:22.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:21 smithi046 ceph-mon[30705]: pgmap v661: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-10T07:45:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:21 smithi181 ceph-mon[35397]: pgmap v661: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-10T07:45:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:21 smithi063 ceph-mon[35020]: pgmap v661: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-10T07:45:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 -- 172.21.15.46:0/633436593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d8102360 msgr2=0x7fa6d810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 --2- 172.21.15.46:0/633436593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d8102360 0x7fa6d810e890 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009be0 tx=0x7fa6d002a6c0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 -- 172.21.15.46:0/633436593 shutdown_connections 2024-09-10T07:45:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 --2- 172.21.15.46:0/633436593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d8102360 0x7fa6d810e890 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 --2- 172.21.15.46:0/633436593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d81019c0 0x7fa6d8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 --2- 172.21.15.46:0/633436593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6d81079c0 0x7fa6d8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 -- 172.21.15.46:0/633436593 >> 172.21.15.46:0/633436593 conn(0x7fa6d80fd740 msgr2=0x7fa6d80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 -- 172.21.15.46:0/633436593 shutdown_connections 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.514+0000 7fa6dea00640 1 -- 172.21.15.46:0/633436593 wait complete. 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.515+0000 7fa6dea00640 1 Processor -- start 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.515+0000 7fa6dea00640 1 -- start start 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6d81019c0 0x7fa6d8108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d8102360 0x7fa6d8108e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d81079c0 0x7fa6d8109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8111dc0 con 0x7fa6d81019c0 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8111c40 con 0x7fa6d81079c0 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8111f40 con 0x7fa6d8102360 2024-09-10T07:45:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d8102360 0x7fa6d8108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6d81019c0 0x7fa6d8108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.516+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d8102360 0x7fa6d8108e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50336/0 (socket says 172.21.15.46:50336) 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.517+0000 7fa6dcc00640 1 -- 172.21.15.46:0/3028504277 learned_addr learned my addr 172.21.15.46:0/3028504277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.517+0000 7fa6de000640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d81079c0 0x7fa6d8109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.517+0000 7fa6de000640 1 -- 172.21.15.46:0/3028504277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d8102360 msgr2=0x7fa6d8108e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.517+0000 7fa6de000640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d8102360 0x7fa6d8108e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.517+0000 7fa6de000640 1 -- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6d81019c0 msgr2=0x7fa6d8108910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.517+0000 7fa6de000640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6d81019c0 0x7fa6d8108910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.518+0000 7fa6de000640 1 -- 172.21.15.46:0/3028504277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d81aab20 con 0x7fa6d81079c0 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.518+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d8102360 0x7fa6d8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.518+0000 7fa6dd600640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6d81019c0 0x7fa6d8108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.520+0000 7fa6de000640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d81079c0 0x7fa6d8109390 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009b30 tx=0x7fa6d0030ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.520+0000 7fa6d6000640 1 -- 172.21.15.46:0/3028504277 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0041020 con 0x7fa6d81079c0 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.520+0000 7fa6d6000640 1 -- 172.21.15.46:0/3028504277 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d0003e50 con 0x7fa6d81079c0 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.520+0000 7fa6dea00640 1 -- 172.21.15.46:0/3028504277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d81aadb0 con 0x7fa6d81079c0 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.520+0000 7fa6dea00640 1 -- 172.21.15.46:0/3028504277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d81ab260 con 0x7fa6d81079c0 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.521+0000 7fa6d6000640 1 -- 172.21.15.46:0/3028504277 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0041020 con 0x7fa6d81079c0 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.525+0000 7fa6d6000640 1 -- 172.21.15.46:0/3028504277 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa6d0044020 con 0x7fa6d81079c0 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.526+0000 7fa6e2aad740 1 Processor -- start 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.526+0000 7fa6e2aad740 1 -- start start 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.526+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4e2760 0x564ebb4ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4ed020 0x564ebb4ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ebb4483e0 con 0x564ebb4e2760 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564ebb3efd80 con 0x564ebb4ed020 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564ebb4443a0 con 0x564ebb4e1e40 2024-09-10T07:45:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4e2760 0x564ebb4ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50338/0 (socket says 172.21.15.46:50338) 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6dcc00640 1 -- 172.21.15.46:0/3257805950 learned_addr learned my addr 172.21.15.46:0/3257805950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4ed020 0x564ebb4ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.527+0000 7fa6d6000640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa6c00a2d80 0x7fa6c00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6d6000640 1 -- 172.21.15.46:0/3028504277 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa6d00ef540 con 0x7fa6d81079c0 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6dcc00640 1 -- 172.21.15.46:0/3257805950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4ed020 msgr2=0x564ebb4ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4ed020 0x564ebb4ef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6dcc00640 1 -- 172.21.15.46:0/3257805950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4e2760 msgr2=0x564ebb4ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4e2760 0x564ebb4ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6dcc00640 1 -- 172.21.15.46:0/3257805950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ebb40eae0 con 0x564ebb4e1e40 2024-09-10T07:45:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6dd600640 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4e2760 0x564ebb4ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e2220 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc013cb0 tx=0x7fa6cc009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a23871b1c8b4cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6de000640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa6c00a2d80 0x7fa6c00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6c6600640 1 -- 172.21.15.46:0/3257805950 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc013da0 con 0x564ebb4e1e40 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6c6600640 1 -- 172.21.15.46:0/3257805950 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc013170 con 0x564ebb4e1e40 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.528+0000 7fa6c6600640 1 -- 172.21.15.46:0/3257805950 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00e3b0 con 0x564ebb4e1e40 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.529+0000 7fa6e2aad740 1 -- 172.21.15.46:0/3257805950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 msgr2=0x564ebb4e2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.529+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e2220 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc013cb0 tx=0x7fa6cc009bd0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.529+0000 7fa6de000640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa6c00a2d80 0x7fa6c00a5240 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009be0 tx=0x7fa6d002a6c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.529+0000 7fa6e2aad740 1 -- 172.21.15.46:0/3257805950 shutdown_connections 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.529+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4ed020 0x564ebb4ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.529+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4e2760 0x564ebb4ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.529+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/3257805950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e2220 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.529+0000 7fa6e2aad740 1 -- 172.21.15.46:0/3257805950 >> 172.21.15.46:0/3257805950 conn(0x564ebb4de3d0 msgr2=0x564ebb4de7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.530+0000 7fa6e2aad740 1 -- 172.21.15.46:0/3257805950 shutdown_connections 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.530+0000 7fa6e2aad740 1 -- 172.21.15.46:0/3257805950 wait complete. 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.530+0000 7fa6e2aad740 1 -- start start 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.530+0000 7fa6e2aad740 1 Processor -- start 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.531+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.531+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb4ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.531+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb4eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.531+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ebb500c00 con 0x564ebb4ed020 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.531+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564ebb4030f0 con 0x564ebb4e2760 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.531+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564ebb500d80 con 0x564ebb4e1e40 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.531+0000 7fa6e2aad740 10 client.0 _ll_register_callbacks cb 0x564ebb3eabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb4ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb4ea500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:41836/0 (socket says 172.21.15.46:41836) 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dd600640 1 -- 172.21.15.46:0/1022685443 learned_addr learned my addr 172.21.15.46:0/1022685443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dd600640 1 -- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 msgr2=0x564ebb4e9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dd600640 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dd600640 1 -- 172.21.15.46:0/1022685443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 msgr2=0x564ebb4eaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dd600640 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb4eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dd600640 1 -- 172.21.15.46:0/1022685443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ebb1b93c0 con 0x564ebb4e2760 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.532+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.533+0000 7fa6dd600640 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb4ea500 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8011780 tx=0x7fa6c8011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd2c30534b8381df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.534+0000 7fa6baa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.534+0000 7fa6baa00640 1 -- 172.21.15.46:0/1022685443 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c801f060 con 0x564ebb4e2760 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.534+0000 7fa6baa00640 1 -- 172.21.15.46:0/1022685443 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa6c8010ba0 con 0x564ebb4e2760 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.534+0000 7fa6e2aad740 1 -- 172.21.15.46:0/1022685443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc013170 con 0x564ebb4e2760 2024-09-10T07:45:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.534+0000 7fa6baa00640 1 -- 172.21.15.46:0/1022685443 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa6c8010da0 con 0x564ebb4e2760 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.534+0000 7fa6e2aad740 10 client.27578 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.535+0000 7fa6e2aad740 1 -- 172.21.15.46:0/1022685443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564ebb50cd90 con 0x564ebb4e2760 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.535+0000 7fa6baa00640 1 -- 172.21.15.46:0/1022685443 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc013170 con 0x564ebb4e2760 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.536+0000 7fa6baa00640 1 -- 172.21.15.46:0/1022685443 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7fa6c800f4e0 con 0x564ebb4e2760 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.536+0000 7fa6e2aad740 10 client.27578 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.536+0000 7fa6e2aad740 10 client.27578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.536+0000 7fa6e2aad740 20 client.27578 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.536+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x564ebb50e250 0x564ebb510640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.536+0000 7fa6e2aad740 4 client.27578 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.536+0000 7fa6e2aad740 1 -- 172.21.15.46:0/1022685443 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ebb455950 con 0x564ebb50e250 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.536+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x564ebb50e250 0x564ebb510640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.537+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x564ebb50e250 0x564ebb510640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e445b71f28df883f server_cookie=4da8f668e8f04b9c in_seq=0 out_seq=0 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.537+0000 7fa6baa00640 10 client.27578 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:45:22.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.538+0000 7fa6baa00640 1 -- 172.21.15.46:0/1022685443 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ebb455950 con 0x564ebb50e250 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.538+0000 7fa6baa00640 10 client.27578 handle_command_reply: tid=0 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.538+0000 7fa6e2aad740 1 client.27578 shutdown 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.538+0000 7fa6e2aad740 1 -- 172.21.15.46:0/1022685443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 msgr2=0x564ebb4ea500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.538+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb4ea500 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8011780 tx=0x7fa6c8011c40 comp rx=0 tx=0).stop 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 1 -- 172.21.15.46:0/1022685443 shutdown_connections 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x564ebb50e250 0x564ebb510640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb4eaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb4ea500 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/1022685443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb4e9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 1 -- 172.21.15.46:0/1022685443 >> 172.21.15.46:0/1022685443 conn(0x564ebb4de3d0 msgr2=0x564ebb4e4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6baa00640 0 client.27578 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 1 -- 172.21.15.46:0/1022685443 shutdown_connections 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 1 -- 172.21.15.46:0/1022685443 wait complete. 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.539+0000 7fa6e2aad740 20 client.27578 trim_cache size 0 max 16384 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.572+0000 7fa6e2aad740 1 Processor -- start 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.576+0000 7fa6e2aad740 1 -- start start 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.576+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e1e40 0x564ebb4ea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.576+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e2760 0x564ebb515ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.576+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb5163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.576+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ac920 con 0x564ebb4ed020 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.576+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ae490 con 0x564ebb4e1e40 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.576+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81acff0 con 0x564ebb4e2760 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e2760 0x564ebb515ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e2760 0x564ebb515ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50358/0 (socket says 172.21.15.46:50358) 2024-09-10T07:45:22.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dd600640 1 -- 172.21.15.46:0/307553980 learned_addr learned my addr 172.21.15.46:0/307553980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/307553980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e1e40 0x564ebb4ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dd600640 1 -- 172.21.15.46:0/307553980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e1e40 msgr2=0x564ebb4ea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dd600640 1 --2- 172.21.15.46:0/307553980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e1e40 0x564ebb4ea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dd600640 1 -- 172.21.15.46:0/307553980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 msgr2=0x564ebb5163e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dd600640 1 --2- 172.21.15.46:0/307553980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb5163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.577+0000 7fa6dd600640 1 -- 172.21.15.46:0/307553980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ebb4e2590 con 0x564ebb4e2760 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.578+0000 7fa6dd600640 1 --2- 172.21.15.46:0/307553980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e2760 0x564ebb515ea0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800f900 tx=0x7fa6c800bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92913582ac5ea7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.578+0000 7fa6c5c00640 1 -- 172.21.15.46:0/307553980 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c802e020 con 0x564ebb4e2760 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.578+0000 7fa6c5c00640 1 -- 172.21.15.46:0/307553980 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c8005510 con 0x564ebb4e2760 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.579+0000 7fa6c5c00640 1 -- 172.21.15.46:0/307553980 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8004660 con 0x564ebb4e2760 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.580+0000 7fa6e2aad740 1 -- 172.21.15.46:0/307553980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e2760 msgr2=0x564ebb515ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.580+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/307553980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e2760 0x564ebb515ea0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800f900 tx=0x7fa6c800bf30 comp rx=0 tx=0).stop 2024-09-10T07:45:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.580+0000 7fa6e2aad740 1 -- 172.21.15.46:0/307553980 shutdown_connections 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.580+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/307553980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb5163e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.580+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/307553980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e2760 0x564ebb515ea0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.580+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/307553980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e1e40 0x564ebb4ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.580+0000 7fa6e2aad740 1 -- 172.21.15.46:0/307553980 >> 172.21.15.46:0/307553980 conn(0x564ebb4de3d0 msgr2=0x564ebb4ec040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.580+0000 7fa6e2aad740 1 -- 172.21.15.46:0/307553980 shutdown_connections 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.581+0000 7fa6e2aad740 1 -- 172.21.15.46:0/307553980 wait complete. 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.581+0000 7fa6e2aad740 1 -- start start 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.582+0000 7fa6e2aad740 1 Processor -- start 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.583+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb501f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.584+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb5024b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.584+0000 7fa6e2aad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb5029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.584+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81add90 con 0x564ebb4ed020 2024-09-10T07:45:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.584+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ad6c0 con 0x564ebb4e2760 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.584+0000 7fa6e2aad740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8109e40 con 0x564ebb4e1e40 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.584+0000 7fa6e2aad740 10 client.0 _ll_register_callbacks cb 0x7fa6c800bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.585+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb501f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.585+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb501f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50366/0 (socket says 172.21.15.46:50366) 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.585+0000 7fa6dcc00640 1 -- 172.21.15.46:0/2898709733 learned_addr learned my addr 172.21.15.46:0/2898709733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.585+0000 7fa6dd600640 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb5024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.586+0000 7fa6dd600640 1 -- 172.21.15.46:0/2898709733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 msgr2=0x564ebb501f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.586+0000 7fa6dd600640 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb501f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.587+0000 7fa6dd600640 1 -- 172.21.15.46:0/2898709733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 msgr2=0x564ebb5029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.587+0000 7fa6dd600640 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb5029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.587+0000 7fa6dd600640 1 -- 172.21.15.46:0/2898709733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ebb578070 con 0x564ebb4e2760 2024-09-10T07:45:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.587+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb501f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.588+0000 7fa6dd600640 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb5024b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800f9a0 tx=0x7fa6c8005d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=766a208709543dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.588+0000 7fa6baa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.589+0000 7fa6baa00640 1 -- 172.21.15.46:0/2898709733 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c801f020 con 0x564ebb4e2760 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.589+0000 7fa6e2aad740 1 -- 172.21.15.46:0/2898709733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c8005510 con 0x564ebb4e2760 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.589+0000 7fa6baa00640 1 -- 172.21.15.46:0/2898709733 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa6c8017070 con 0x564ebb4e2760 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.590+0000 7fa6baa00640 1 -- 172.21.15.46:0/2898709733 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa6c802e040 con 0x564ebb4e2760 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.590+0000 7fa6e2aad740 10 client.27584 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.590+0000 7fa6e2aad740 1 -- 172.21.15.46:0/2898709733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564ebb579330 con 0x564ebb4e2760 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.591+0000 7fa6baa00640 1 -- 172.21.15.46:0/2898709733 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c8005510 con 0x564ebb4e2760 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.591+0000 7fa6baa00640 1 -- 172.21.15.46:0/2898709733 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7fa6c8033480 con 0x564ebb4e2760 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.591+0000 7fa6e2aad740 10 client.27584 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.592+0000 7fa6e2aad740 10 client.27584 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.592+0000 7fa6e2aad740 20 client.27584 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.592+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x564ebb579990 0x564ebb57bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.592+0000 7fa6e2aad740 4 client.27584 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.592+0000 7fa6e2aad740 1 -- 172.21.15.46:0/2898709733 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ebb44fe60 con 0x564ebb579990 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.592+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x564ebb579990 0x564ebb57bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.593+0000 7fa6baa00640 10 client.27584 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.593+0000 7fa6dcc00640 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x564ebb579990 0x564ebb57bdb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2013f93d7e50e6d5 server_cookie=9af3eded9d45fb49 in_seq=0 out_seq=0 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.593+0000 7fa6baa00640 1 -- 172.21.15.46:0/2898709733 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ebb44fe60 con 0x564ebb579990 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.593+0000 7fa6baa00640 10 client.27584 handle_command_reply: tid=0 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.593+0000 7fa6e2aad740 1 client.27584 shutdown 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.594+0000 7fa6e2aad740 1 -- 172.21.15.46:0/2898709733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 msgr2=0x564ebb5024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.594+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb5024b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800f9a0 tx=0x7fa6c8005d00 comp rx=0 tx=0).stop 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6e2aad740 1 -- 172.21.15.46:0/2898709733 shutdown_connections 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x564ebb579990 0x564ebb57bdb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ebb4ed020 0x564ebb5029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ebb4e2760 0x564ebb5024b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6e2aad740 1 --2- 172.21.15.46:0/2898709733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564ebb4e1e40 0x564ebb501f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6e2aad740 1 -- 172.21.15.46:0/2898709733 >> 172.21.15.46:0/2898709733 conn(0x564ebb4de3d0 msgr2=0x564ebb4e38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6baa00640 0 client.27584 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:45:22.848 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:45:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6e2aad740 1 -- 172.21.15.46:0/2898709733 shutdown_connections 2024-09-10T07:45:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.595+0000 7fa6e2aad740 1 -- 172.21.15.46:0/2898709733 wait complete. 2024-09-10T07:45:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.596+0000 7fa6e2aad740 20 client.27584 trim_cache size 0 max 16384 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 -- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa6c00a2d80 msgr2=0x7fa6c00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa6c00a2d80 0x7fa6c00a5240 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009be0 tx=0x7fa6d002a6c0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 -- 172.21.15.46:0/3028504277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d81079c0 msgr2=0x7fa6d8109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d81079c0 0x7fa6d8109390 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009b30 tx=0x7fa6d0030ca0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 -- 172.21.15.46:0/3028504277 shutdown_connections 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6d81079c0 0x7fa6d8109390 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d8102360 0x7fa6d8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa6c00a2d80 0x7fa6c00a5240 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 --2- 172.21.15.46:0/3028504277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6d81019c0 0x7fa6d8108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.597+0000 7fa6dea00640 1 -- 172.21.15.46:0/3028504277 >> 172.21.15.46:0/3028504277 conn(0x7fa6d80fd740 msgr2=0x7fa6d810cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.598+0000 7fa6dea00640 1 -- 172.21.15.46:0/3028504277 shutdown_connections 2024-09-10T07:45:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:22.598+0000 7fa6dea00640 1 -- 172.21.15.46:0/3028504277 wait complete. 2024-09-10T07:45:24.289 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:23 smithi046 ceph-mon[30705]: pgmap v662: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-09-10T07:45:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:23 smithi181 ceph-mon[35397]: pgmap v662: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-09-10T07:45:24.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:23 smithi063 ceph-mon[35020]: pgmap v662: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-09-10T07:45:24.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:45:25.549 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 485.0 s, 645.0 tps, lat 159.919 ms stddev 212.714 2024-09-10T07:45:26.002 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:25 smithi046 ceph-mon[30705]: pgmap v663: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 MiB/s wr, 433 op/s 2024-09-10T07:45:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.359+0000 7f4aa3e00640 1 -- 172.21.15.46:0/753253468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 msgr2=0x7f4aa41041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.359+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/753253468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 0x7f4aa41041f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009ca0 tx=0x7f4a9c02a8e0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.359+0000 7f4aa3e00640 1 -- 172.21.15.46:0/753253468 shutdown_connections 2024-09-10T07:45:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.359+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/753253468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa4108f40 0x7f4aa4110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.359+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/753253468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa41047c0 0x7f4aa4108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.359+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/753253468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 0x7f4aa41041f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.359+0000 7f4aa3e00640 1 -- 172.21.15.46:0/753253468 >> 172.21.15.46:0/753253468 conn(0x7f4aa40ff870 msgr2=0x7f4aa4101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.360+0000 7f4aa3e00640 1 -- 172.21.15.46:0/753253468 shutdown_connections 2024-09-10T07:45:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.360+0000 7f4aa3e00640 1 -- 172.21.15.46:0/753253468 wait complete. 2024-09-10T07:45:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.362+0000 7f4aa3e00640 1 Processor -- start 2024-09-10T07:45:26.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.362+0000 7f4aa3e00640 1 -- start start 2024-09-10T07:45:26.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.362+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 0x7f4aa4076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.362+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa41047c0 0x7f4aa4076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.363+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4108f40 0x7f4aa4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.363+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 0x7f4aa4076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.363+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 0x7f4aa4076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44242/0 (socket says 172.21.15.46:44242) 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.363+0000 7f4aa2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa41047c0 0x7f4aa4076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.363+0000 7f4aa2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa41047c0 0x7f4aa4076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49650/0 (socket says 172.21.15.46:49650) 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.364+0000 7f4aa2a00640 1 -- 172.21.15.46:0/1871436460 learned_addr learned my addr 172.21.15.46:0/1871436460 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.364+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4114030 con 0x7f4aa4108f40 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.364+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4113eb0 con 0x7f4aa4103e10 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.364+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41141b0 con 0x7f4aa41047c0 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.364+0000 7f4aa3400640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4108f40 0x7f4aa4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.365+0000 7f4aa2a00640 1 -- 172.21.15.46:0/1871436460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa41047c0 msgr2=0x7f4aa4076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.365+0000 7f4aa2a00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa41047c0 0x7f4aa4076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.365+0000 7f4aa2a00640 1 -- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4108f40 msgr2=0x7f4aa4077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.365+0000 7f4aa2a00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4108f40 0x7f4aa4077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.365+0000 7f4aa2a00640 1 -- 172.21.15.46:0/1871436460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa4072aa0 con 0x7f4aa4103e10 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.365+0000 7f4aa2000640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa41047c0 0x7f4aa4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.365+0000 7f4aa3400640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4108f40 0x7f4aa4077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.366+0000 7f4aa2a00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 0x7f4aa4076700 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c005e30 tx=0x7f4a9c0039f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.366+0000 7f4a9b400640 1 -- 172.21.15.46:0/1871436460 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c003de0 con 0x7f4aa4103e10 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.366+0000 7f4a9b400640 1 -- 172.21.15.46:0/1871436460 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9c004520 con 0x7f4aa4103e10 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.366+0000 7f4a9b400640 1 -- 172.21.15.46:0/1871436460 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c030dd0 con 0x7f4aa4103e10 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.367+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa4072d30 con 0x7f4aa4103e10 2024-09-10T07:45:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.368+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa4073220 con 0x7f4aa4103e10 2024-09-10T07:45:26.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.375+0000 7f4aac6b3740 1 Processor -- start 2024-09-10T07:45:26.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.378+0000 7f4aac6b3740 1 -- start start 2024-09-10T07:45:26.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.378+0000 7f4a9b400640 1 -- 172.21.15.46:0/1871436460 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a9c004040 con 0x7f4aa4103e10 2024-09-10T07:45:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.383+0000 7f4a9b400640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a880a2c80 0x7f4a880a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.383+0000 7f4a9b400640 1 -- 172.21.15.46:0/1871436460 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4a9c0f0990 con 0x7f4aa4103e10 2024-09-10T07:45:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.384+0000 7f4aa3400640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a880a2c80 0x7f4a880a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.384+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a310220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.384+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.384+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a31d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.384+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55858a2763e0 con 0x55858a310760 2024-09-10T07:45:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.384+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55858a21dd80 con 0x55858a31b020 2024-09-10T07:45:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.384+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55858a2723a0 con 0x55858a30fe40 2024-09-10T07:45:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.385+0000 7f4aa2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.385+0000 7f4aa2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43116/0 (socket says 172.21.15.46:43116) 2024-09-10T07:45:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.385+0000 7f4aa2000640 1 -- 172.21.15.46:0/240875190 learned_addr learned my addr 172.21.15.46:0/240875190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.386+0000 7f4aa2000640 1 -- 172.21.15.46:0/240875190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 msgr2=0x55858a310220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:45:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.386+0000 7f4aa2000640 1 --2- 172.21.15.46:0/240875190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a310220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.386+0000 7f4aa2000640 1 -- 172.21.15.46:0/240875190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 msgr2=0x55858a31d410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:45:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.386+0000 7f4aa2000640 1 --2- 172.21.15.46:0/240875190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a31d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.386+0000 7f4aa2000640 1 -- 172.21.15.46:0/240875190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858a23cae0 con 0x55858a310760 2024-09-10T07:45:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.387+0000 7f4aa2000640 1 --2- 172.21.15.46:0/240875190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31aae0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4a9401a840 tx=0x7f4a9401ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d22efefece1dcf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.387+0000 7f4aa3400640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a880a2c80 0x7f4a880a5140 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000bbe0 tx=0x7f4a9000b5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.388+0000 7f4a8fe00640 1 -- 172.21.15.46:0/240875190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9401e950 con 0x55858a310760 2024-09-10T07:45:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.388+0000 7f4a8fe00640 1 -- 172.21.15.46:0/240875190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9402a070 con 0x55858a310760 2024-09-10T07:45:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.388+0000 7f4a8fe00640 1 -- 172.21.15.46:0/240875190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a940233e0 con 0x55858a310760 2024-09-10T07:45:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.390+0000 7f4aac6b3740 1 -- 172.21.15.46:0/240875190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 msgr2=0x55858a31aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.390+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/240875190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31aae0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4a9401a840 tx=0x7f4a9401ad00 comp rx=0 tx=0).stop 2024-09-10T07:45:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.391+0000 7f4aac6b3740 1 -- 172.21.15.46:0/240875190 shutdown_connections 2024-09-10T07:45:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.391+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/240875190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a31d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.391+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/240875190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31aae0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.391+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/240875190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a310220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.391+0000 7f4aac6b3740 1 -- 172.21.15.46:0/240875190 >> 172.21.15.46:0/240875190 conn(0x55858a30c3d0 msgr2=0x55858a30c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.391+0000 7f4aac6b3740 1 -- 172.21.15.46:0/240875190 shutdown_connections 2024-09-10T07:45:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.392+0000 7f4aac6b3740 1 -- 172.21.15.46:0/240875190 wait complete. 2024-09-10T07:45:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.392+0000 7f4aac6b3740 1 -- start start 2024-09-10T07:45:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.393+0000 7f4aac6b3740 1 Processor -- start 2024-09-10T07:45:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.396+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a317f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.396+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a3184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.396+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a318a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.396+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55858a31dbf0 con 0x55858a310760 2024-09-10T07:45:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.396+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55858a2310f0 con 0x55858a31b020 2024-09-10T07:45:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.396+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55858a31dd70 con 0x55858a30fe40 2024-09-10T07:45:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.396+0000 7f4aac6b3740 10 client.0 _ll_register_callbacks cb 0x55858a218bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.397+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a318a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.397+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a317f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.397+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a318a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44252/0 (socket says 172.21.15.46:44252) 2024-09-10T07:45:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.397+0000 7f4aa3400640 1 -- 172.21.15.46:0/2228076996 learned_addr learned my addr 172.21.15.46:0/2228076996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.398+0000 7f4aa2000640 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a3184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:25 smithi181 ceph-mon[35397]: pgmap v663: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 MiB/s wr, 433 op/s 2024-09-10T07:45:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.398+0000 7f4aa3400640 1 -- 172.21.15.46:0/2228076996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 msgr2=0x55858a317f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.398+0000 7f4aa3400640 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a317f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.398+0000 7f4aa3400640 1 -- 172.21.15.46:0/2228076996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 msgr2=0x55858a3184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.398+0000 7f4aa3400640 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a3184d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.398+0000 7f4aa3400640 1 -- 172.21.15.46:0/2228076996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558589fe73c0 con 0x55858a31b020 2024-09-10T07:45:26.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:25 smithi063 ceph-mon[35020]: pgmap v663: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 MiB/s wr, 433 op/s 2024-09-10T07:45:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.400+0000 7f4aa2000640 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a3184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.400+0000 7f4aa3400640 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a318a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a900114a0 tx=0x7f4a900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cde0e3ad979a380 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.400+0000 7f4a8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.401+0000 7f4a8cc00640 1 -- 172.21.15.46:0/2228076996 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90022070 con 0x55858a31b020 2024-09-10T07:45:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.401+0000 7f4a8cc00640 1 -- 172.21.15.46:0/2228076996 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4a9001e860 con 0x55858a31b020 2024-09-10T07:45:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.401+0000 7f4aac6b3740 1 -- 172.21.15.46:0/2228076996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a9402a070 con 0x55858a31b020 2024-09-10T07:45:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.402+0000 7f4a8cc00640 1 -- 172.21.15.46:0/2228076996 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4a9002c050 con 0x55858a31b020 2024-09-10T07:45:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.402+0000 7f4aac6b3740 10 client.27593 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.402+0000 7f4aac6b3740 1 -- 172.21.15.46:0/2228076996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55858a329d10 con 0x55858a31b020 2024-09-10T07:45:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.403+0000 7f4a8cc00640 1 -- 172.21.15.46:0/2228076996 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9402a070 con 0x55858a31b020 2024-09-10T07:45:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.403+0000 7f4a8cc00640 1 -- 172.21.15.46:0/2228076996 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f4a900103d0 con 0x55858a31b020 2024-09-10T07:45:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.403+0000 7f4aac6b3740 10 client.27593 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.403+0000 7f4aac6b3740 10 client.27593 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:45:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.403+0000 7f4aac6b3740 20 client.27593 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.403+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55858a32b220 0x55858a32d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.403+0000 7f4aac6b3740 4 client.27593 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.403+0000 7f4aac6b3740 1 -- 172.21.15.46:0/2228076996 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55858a283950 con 0x55858a32b220 2024-09-10T07:45:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.405+0000 7f4aa2a00640 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55858a32b220 0x55858a32d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.406+0000 7f4aa2a00640 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55858a32b220 0x55858a32d660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4830b02c483ac15d server_cookie=d77b4efa6312716e in_seq=0 out_seq=0 2024-09-10T07:45:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.406+0000 7f4a8cc00640 10 client.27593 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:45:26.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.407+0000 7f4a8cc00640 1 -- 172.21.15.46:0/2228076996 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55858a283950 con 0x55858a32b220 2024-09-10T07:45:26.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.407+0000 7f4a8cc00640 10 client.27593 handle_command_reply: tid=0 2024-09-10T07:45:26.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.407+0000 7f4aac6b3740 1 client.27593 shutdown 2024-09-10T07:45:26.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.408+0000 7f4aac6b3740 1 -- 172.21.15.46:0/2228076996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 msgr2=0x55858a318a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.408+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a318a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a900114a0 tx=0x7f4a900046c0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.409+0000 7f4aac6b3740 1 -- 172.21.15.46:0/2228076996 shutdown_connections 2024-09-10T07:45:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.409+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55858a32b220 0x55858a32d660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.409+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a31b020 0x55858a318a10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.409+0000 7f4a8cc00640 0 client.27593 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:45:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.409+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a3184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.409+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/2228076996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a317f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.410+0000 7f4aac6b3740 1 -- 172.21.15.46:0/2228076996 >> 172.21.15.46:0/2228076996 conn(0x55858a30c3d0 msgr2=0x55858a3120b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.410+0000 7f4aac6b3740 1 -- 172.21.15.46:0/2228076996 shutdown_connections 2024-09-10T07:45:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.410+0000 7f4aac6b3740 1 -- 172.21.15.46:0/2228076996 wait complete. 2024-09-10T07:45:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.410+0000 7f4aac6b3740 20 client.27593 trim_cache size 0 max 16384 2024-09-10T07:45:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.438+0000 7f4aac6b3740 1 Processor -- start 2024-09-10T07:45:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.438+0000 7f4aac6b3740 1 -- start start 2024-09-10T07:45:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.438+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a318860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.438+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a310760 0x55858a311fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.438+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a31b020 0x55858a332f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.438+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41b5190 con 0x55858a31b020 2024-09-10T07:45:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.438+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41b6d00 con 0x55858a310760 2024-09-10T07:45:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.438+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41b5860 con 0x55858a30fe40 2024-09-10T07:45:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.439+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a318860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.439+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a318860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49666/0 (socket says 172.21.15.46:49666) 2024-09-10T07:45:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.439+0000 7f4aa2a00640 1 -- 172.21.15.46:0/367446900 learned_addr learned my addr 172.21.15.46:0/367446900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.439+0000 7f4aa3400640 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a31b020 0x55858a332f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.439+0000 7f4aa2000640 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a310760 0x55858a311fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.440+0000 7f4aa2000640 1 -- 172.21.15.46:0/367446900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 msgr2=0x55858a318860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:26.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.440+0000 7f4aa2000640 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a318860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.440+0000 7f4aa2000640 1 -- 172.21.15.46:0/367446900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a31b020 msgr2=0x55858a332f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:26.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.440+0000 7f4aa2000640 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a31b020 0x55858a332f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.440+0000 7f4aa2000640 1 -- 172.21.15.46:0/367446900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858a31a900 con 0x55858a310760 2024-09-10T07:45:26.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.440+0000 7f4aa2000640 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a310760 0x55858a311fa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55858a3195d0 tx=0x7f4a9401ea00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c926bd8778327e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:26.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.441+0000 7f4a8f400640 1 -- 172.21.15.46:0/367446900 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94034020 con 0x55858a310760 2024-09-10T07:45:26.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.441+0000 7f4a8f400640 1 -- 172.21.15.46:0/367446900 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9402a050 con 0x55858a310760 2024-09-10T07:45:26.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.441+0000 7f4a8f400640 1 -- 172.21.15.46:0/367446900 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a940259d0 con 0x55858a310760 2024-09-10T07:45:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.443+0000 7f4aac6b3740 1 -- 172.21.15.46:0/367446900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a310760 msgr2=0x55858a311fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.443+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a310760 0x55858a311fa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55858a3195d0 tx=0x7f4a9401ea00 comp rx=0 tx=0).stop 2024-09-10T07:45:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.444+0000 7f4aac6b3740 1 -- 172.21.15.46:0/367446900 shutdown_connections 2024-09-10T07:45:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.444+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a31b020 0x55858a332f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.444+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a310760 0x55858a311fa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.444+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/367446900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a30fe40 0x55858a318860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.444+0000 7f4aac6b3740 1 -- 172.21.15.46:0/367446900 >> 172.21.15.46:0/367446900 conn(0x55858a30c3d0 msgr2=0x55858a311890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.444+0000 7f4aac6b3740 1 -- 172.21.15.46:0/367446900 shutdown_connections 2024-09-10T07:45:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.444+0000 7f4aac6b3740 1 -- 172.21.15.46:0/367446900 wait complete. 2024-09-10T07:45:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.445+0000 7f4aac6b3740 1 -- start start 2024-09-10T07:45:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.445+0000 7f4aac6b3740 1 Processor -- start 2024-09-10T07:45:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a30fe40 0x55858a31f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aac6b3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a31b020 0x55858a394f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41b6600 con 0x55858a310760 2024-09-10T07:45:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41b5f30 con 0x55858a30fe40 2024-09-10T07:45:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aac6b3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41b10f0 con 0x55858a31b020 2024-09-10T07:45:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aac6b3740 10 client.0 _ll_register_callbacks cb 0x7f4a90015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a30fe40 0x55858a31f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.446+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a31b020 0x55858a394f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.447+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a31b020 0x55858a394f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49672/0 (socket says 172.21.15.46:49672) 2024-09-10T07:45:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.447+0000 7f4aa3400640 1 -- 172.21.15.46:0/995612630 learned_addr learned my addr 172.21.15.46:0/995612630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.447+0000 7f4aa2000640 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.448+0000 7f4aa3400640 1 -- 172.21.15.46:0/995612630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a30fe40 msgr2=0x55858a31f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.448+0000 7f4aa3400640 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a30fe40 0x55858a31f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.448+0000 7f4aa3400640 1 -- 172.21.15.46:0/995612630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 msgr2=0x55858a31fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.448+0000 7f4aa3400640 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.448+0000 7f4aa3400640 1 -- 172.21.15.46:0/995612630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858a3954f0 con 0x55858a31b020 2024-09-10T07:45:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.448+0000 7f4a8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.448+0000 7f4aa3400640 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a31b020 0x55858a394f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a900107a0 tx=0x7f4a9001e660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d928784e7bb779 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.449+0000 7f4a8cc00640 1 -- 172.21.15.46:0/995612630 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90022040 con 0x55858a31b020 2024-09-10T07:45:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.449+0000 7f4a8cc00640 1 -- 172.21.15.46:0/995612630 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4a9001e860 con 0x55858a31b020 2024-09-10T07:45:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.449+0000 7f4aac6b3740 1 -- 172.21.15.46:0/995612630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a9402a050 con 0x55858a31b020 2024-09-10T07:45:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.449+0000 7f4a8cc00640 1 -- 172.21.15.46:0/995612630 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4a9001ea60 con 0x55858a31b020 2024-09-10T07:45:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.449+0000 7f4aac6b3740 10 client.27544 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.449+0000 7f4aac6b3740 1 -- 172.21.15.46:0/995612630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55858a396800 con 0x55858a31b020 2024-09-10T07:45:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.450+0000 7f4a8cc00640 1 -- 172.21.15.46:0/995612630 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9402a050 con 0x55858a31b020 2024-09-10T07:45:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.450+0000 7f4a8cc00640 1 -- 172.21.15.46:0/995612630 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f4a90027850 con 0x55858a31b020 2024-09-10T07:45:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.450+0000 7f4aac6b3740 10 client.27544 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.450+0000 7f4aac6b3740 10 client.27544 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:45:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.450+0000 7f4aac6b3740 20 client.27544 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.450+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55858a32b220 0x55858a399010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.450+0000 7f4aac6b3740 4 client.27544 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:45:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.450+0000 7f4aac6b3740 1 -- 172.21.15.46:0/995612630 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55858a27de60 con 0x55858a32b220 2024-09-10T07:45:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.451+0000 7f4aa2a00640 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55858a32b220 0x55858a399010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.451+0000 7f4aa2a00640 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55858a32b220 0x55858a399010 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf125a323f790d58 server_cookie=ec0fd112d492ecc5 in_seq=0 out_seq=0 2024-09-10T07:45:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.451+0000 7f4a8cc00640 10 client.27544 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:45:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4a8cc00640 1 -- 172.21.15.46:0/995612630 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55858a27de60 con 0x55858a32b220 2024-09-10T07:45:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4a8cc00640 10 client.27544 handle_command_reply: tid=0 2024-09-10T07:45:26.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 client.27544 shutdown 2024-09-10T07:45:26.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 -- 172.21.15.46:0/995612630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a31b020 msgr2=0x55858a394f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a31b020 0x55858a394f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a900107a0 tx=0x7f4a9001e660 comp rx=0 tx=0).stop 2024-09-10T07:45:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 -- 172.21.15.46:0/995612630 shutdown_connections 2024-09-10T07:45:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55858a32b220 0x55858a399010 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55858a31b020 0x55858a394f00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55858a310760 0x55858a31fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 --2- 172.21.15.46:0/995612630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858a30fe40 0x55858a31f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4a8cc00640 0 client.27544 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:45:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.452+0000 7f4aac6b3740 1 -- 172.21.15.46:0/995612630 >> 172.21.15.46:0/995612630 conn(0x55858a30c3d0 msgr2=0x55858a30b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:26.455 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:45:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.453+0000 7f4aac6b3740 1 -- 172.21.15.46:0/995612630 shutdown_connections 2024-09-10T07:45:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.453+0000 7f4aac6b3740 1 -- 172.21.15.46:0/995612630 wait complete. 2024-09-10T07:45:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.453+0000 7f4aac6b3740 20 client.27544 trim_cache size 0 max 16384 2024-09-10T07:45:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.454+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a880a2c80 msgr2=0x7f4a880a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.454+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a880a2c80 0x7f4a880a5140 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000bbe0 tx=0x7f4a9000b5c0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.454+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 msgr2=0x7f4aa4076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.454+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 0x7f4aa4076700 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c005e30 tx=0x7f4a9c0039f0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.456+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 shutdown_connections 2024-09-10T07:45:26.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.456+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a880a2c80 0x7f4a880a5140 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.456+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4108f40 0x7f4aa4077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.456+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa41047c0 0x7f4aa4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.456+0000 7f4aa3e00640 1 --2- 172.21.15.46:0/1871436460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa4103e10 0x7f4aa4076700 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:26.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.456+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 >> 172.21.15.46:0/1871436460 conn(0x7f4aa40ff870 msgr2=0x7f4aa4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:26.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.457+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 shutdown_connections 2024-09-10T07:45:26.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:26.457+0000 7f4aa3e00640 1 -- 172.21.15.46:0/1871436460 wait complete. 2024-09-10T07:45:28.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:45:28.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:27 smithi181 ceph-mon[35397]: pgmap v664: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.6 MiB/s wr, 588 op/s 2024-09-10T07:45:28.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:27 smithi063 ceph-mon[35020]: pgmap v664: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.6 MiB/s wr, 588 op/s 2024-09-10T07:45:28.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:27 smithi046 ceph-mon[30705]: pgmap v664: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.6 MiB/s wr, 588 op/s 2024-09-10T07:45:28.725 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.695+0000 7f6800800640 1 -- 172.21.15.46:0/246601897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f8109b10 msgr2=0x7f67f8114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.696+0000 7f6800800640 1 --2- 172.21.15.46:0/246601897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f8109b10 0x7f67f8114570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009900 tx=0x7f67e802a4d0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.697+0000 7f6800800640 1 -- 172.21.15.46:0/246601897 shutdown_connections 2024-09-10T07:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.697+0000 7f6800800640 1 --2- 172.21.15.46:0/246601897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f8114ab0 0x7f67f8116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.697+0000 7f6800800640 1 --2- 172.21.15.46:0/246601897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f8109b10 0x7f67f8114570 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.697+0000 7f6800800640 1 --2- 172.21.15.46:0/246601897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67f81091f0 0x7f67f81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.698+0000 7f6800800640 1 -- 172.21.15.46:0/246601897 >> 172.21.15.46:0/246601897 conn(0x7f67f806e140 msgr2=0x7f67f806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.698+0000 7f6800800640 1 -- 172.21.15.46:0/246601897 shutdown_connections 2024-09-10T07:45:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.698+0000 7f6800800640 1 -- 172.21.15.46:0/246601897 wait complete. 2024-09-10T07:45:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.703+0000 7f6800800640 1 Processor -- start 2024-09-10T07:45:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.704+0000 7f6800800640 1 -- start start 2024-09-10T07:45:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.704+0000 7f6800800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f81091f0 0x7f67f810a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.704+0000 7f6800800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67f8109b10 0x7f67f810a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.704+0000 7f6800800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f8114ab0 0x7f67f810ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.704+0000 7f6800800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67f811a840 con 0x7f67f8114ab0 2024-09-10T07:45:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.704+0000 7f6800800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67f811a6c0 con 0x7f67f8109b10 2024-09-10T07:45:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.704+0000 7f6800800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67f811a9c0 con 0x7f67f81091f0 2024-09-10T07:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.705+0000 7f67fd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67f8109b10 0x7f67f810a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.705+0000 7f67fd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67f8109b10 0x7f67f810a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44286/0 (socket says 172.21.15.46:44286) 2024-09-10T07:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.705+0000 7f67fd400640 1 -- 172.21.15.46:0/77617572 learned_addr learned my addr 172.21.15.46:0/77617572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.705+0000 7f67fe800640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f8114ab0 0x7f67f810ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.706+0000 7f67fd400640 1 -- 172.21.15.46:0/77617572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f81091f0 msgr2=0x7f67f810a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.706+0000 7f67fd400640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f81091f0 0x7f67f810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.706+0000 7f67fd400640 1 -- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f8114ab0 msgr2=0x7f67f810ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.706+0000 7f67fd400640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f8114ab0 0x7f67f810ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.706+0000 7f67fd400640 1 -- 172.21.15.46:0/77617572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f81af240 con 0x7f67f8109b10 2024-09-10T07:45:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.707+0000 7f67fd400640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67f8109b10 0x7f67f810a910 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f67e8004bd0 tx=0x7f67e8005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.707+0000 7f67f2a00640 1 -- 172.21.15.46:0/77617572 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8040030 con 0x7f67f8109b10 2024-09-10T07:45:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.707+0000 7f6800800640 1 -- 172.21.15.46:0/77617572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f81af4d0 con 0x7f67f8109b10 2024-09-10T07:45:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.707+0000 7f67f2a00640 1 -- 172.21.15.46:0/77617572 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e803c920 con 0x7f67f8109b10 2024-09-10T07:45:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.707+0000 7f6800800640 1 -- 172.21.15.46:0/77617572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f81afa00 con 0x7f67f8109b10 2024-09-10T07:45:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.708+0000 7f67fe800640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f8114ab0 0x7f67f810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:29.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.711+0000 7f67f2a00640 1 -- 172.21.15.46:0/77617572 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e803c640 con 0x7f67f8109b10 2024-09-10T07:45:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.712+0000 7f680344e740 1 Processor -- start 2024-09-10T07:45:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.713+0000 7f680344e740 1 -- start start 2024-09-10T07:45:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.713+0000 7f680344e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479cc70 0x555d8479d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.713+0000 7f680344e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.713+0000 7f680344e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d847a7e50 0x555d847aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.713+0000 7f680344e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d847033e0 con 0x555d8479d590 2024-09-10T07:45:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.713+0000 7f680344e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555d846aad80 con 0x555d847a7e50 2024-09-10T07:45:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.713+0000 7f680344e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555d846ff3a0 con 0x555d8479cc70 2024-09-10T07:45:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.714+0000 7f67fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.715+0000 7f67fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43164/0 (socket says 172.21.15.46:43164) 2024-09-10T07:45:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.715+0000 7f67fde00640 1 -- 172.21.15.46:0/3279136610 learned_addr learned my addr 172.21.15.46:0/3279136610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.717+0000 7f67fde00640 1 -- 172.21.15.46:0/3279136610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479cc70 msgr2=0x555d8479d050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:45:29.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.717+0000 7f67fde00640 1 --2- 172.21.15.46:0/3279136610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479cc70 0x555d8479d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.717+0000 7f67fde00640 1 -- 172.21.15.46:0/3279136610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d847a7e50 msgr2=0x555d847aa240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:45:29.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.717+0000 7f67fde00640 1 --2- 172.21.15.46:0/3279136610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d847a7e50 0x555d847aa240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.717+0000 7f67fde00640 1 -- 172.21.15.46:0/3279136610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d846c9ae0 con 0x555d8479d590 2024-09-10T07:45:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.718+0000 7f67fde00640 1 --2- 172.21.15.46:0/3279136610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847a7910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00bb10 tx=0x7f67ec00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2adc5e1f2b8f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.719+0000 7f67e7000640 1 -- 172.21.15.46:0/3279136610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec016070 con 0x555d8479d590 2024-09-10T07:45:29.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.719+0000 7f67e7000640 1 -- 172.21.15.46:0/3279136610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec005500 con 0x555d8479d590 2024-09-10T07:45:29.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.719+0000 7f67e7000640 1 -- 172.21.15.46:0/3279136610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec016070 con 0x555d8479d590 2024-09-10T07:45:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.721+0000 7f680344e740 1 -- 172.21.15.46:0/3279136610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 msgr2=0x555d847a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.721+0000 7f680344e740 1 --2- 172.21.15.46:0/3279136610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847a7910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00bb10 tx=0x7f67ec00d2e0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.722+0000 7f680344e740 1 -- 172.21.15.46:0/3279136610 shutdown_connections 2024-09-10T07:45:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.722+0000 7f680344e740 1 --2- 172.21.15.46:0/3279136610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d847a7e50 0x555d847aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.722+0000 7f680344e740 1 --2- 172.21.15.46:0/3279136610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847a7910 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.722+0000 7f680344e740 1 --2- 172.21.15.46:0/3279136610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479cc70 0x555d8479d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.722+0000 7f680344e740 1 -- 172.21.15.46:0/3279136610 >> 172.21.15.46:0/3279136610 conn(0x555d84799200 msgr2=0x555d847995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.723+0000 7f67f2a00640 1 -- 172.21.15.46:0/77617572 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67e8043030 con 0x7f67f8109b10 2024-09-10T07:45:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.725+0000 7f67f2a00640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67e00a2d80 0x7f67e00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.726+0000 7f67fde00640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67e00a2d80 0x7f67e00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.727+0000 7f67f2a00640 1 -- 172.21.15.46:0/77617572 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f67e803c440 con 0x7f67f8109b10 2024-09-10T07:45:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.728+0000 7f67fde00640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67e00a2d80 0x7f67e00a5240 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00bb10 tx=0x7f67ec004bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.730+0000 7f680344e740 1 -- 172.21.15.46:0/3279136610 shutdown_connections 2024-09-10T07:45:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.730+0000 7f680344e740 1 -- 172.21.15.46:0/3279136610 wait complete. 2024-09-10T07:45:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.731+0000 7f680344e740 1 -- start start 2024-09-10T07:45:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.731+0000 7f680344e740 1 Processor -- start 2024-09-10T07:45:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.733+0000 7f680344e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.733+0000 7f680344e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.733+0000 7f680344e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d847a57b0 0x555d847b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.733+0000 7f680344e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d847aaa20 con 0x555d847a57b0 2024-09-10T07:45:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.733+0000 7f680344e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555d846be0f0 con 0x555d8479cc70 2024-09-10T07:45:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.733+0000 7f680344e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555d847aaba0 con 0x555d8479d590 2024-09-10T07:45:29.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.733+0000 7f680344e740 10 client.0 _ll_register_callbacks cb 0x555d846a5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:29.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.733+0000 7f67fd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.734+0000 7f67fd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847a5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49704/0 (socket says 172.21.15.46:49704) 2024-09-10T07:45:29.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.734+0000 7f67fd400640 1 -- 172.21.15.46:0/1451132956 learned_addr learned my addr 172.21.15.46:0/1451132956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:29.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.734+0000 7f67fde00640 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.735+0000 7f67fd400640 1 -- 172.21.15.46:0/1451132956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 msgr2=0x555d847a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.737+0000 7f67fd400640 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.737+0000 7f67fd400640 1 -- 172.21.15.46:0/1451132956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d847a57b0 msgr2=0x555d847b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.737+0000 7f67fd400640 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d847a57b0 0x555d847b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.737+0000 7f67fd400640 1 -- 172.21.15.46:0/1451132956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d844743c0 con 0x555d8479d590 2024-09-10T07:45:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.737+0000 7f67fde00640 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.739+0000 7f67fd400640 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847a5270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f67e80b8b10 tx=0x7f67e80b6390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db71eedf3c2edcb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.739+0000 7f67db400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.741+0000 7f67db400640 1 -- 172.21.15.46:0/1451132956 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e80b82c0 con 0x555d8479d590 2024-09-10T07:45:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.741+0000 7f67db400640 1 -- 172.21.15.46:0/1451132956 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f67e80b75e0 con 0x555d8479d590 2024-09-10T07:45:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.741+0000 7f680344e740 1 -- 172.21.15.46:0/1451132956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ec005500 con 0x555d8479d590 2024-09-10T07:45:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.742+0000 7f67db400640 1 -- 172.21.15.46:0/1451132956 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f67e80b8460 con 0x555d8479d590 2024-09-10T07:45:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.742+0000 7f680344e740 10 client.27550 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.742+0000 7f680344e740 1 -- 172.21.15.46:0/1451132956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555d847aaa20 con 0x555d8479d590 2024-09-10T07:45:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.743+0000 7f67db400640 1 -- 172.21.15.46:0/1451132956 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec005500 con 0x555d8479d590 2024-09-10T07:45:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.744+0000 7f67db400640 1 -- 172.21.15.46:0/1451132956 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f67e80b7960 con 0x555d8479d590 2024-09-10T07:45:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.744+0000 7f680344e740 10 client.27550 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.744+0000 7f680344e740 10 client.27550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.745+0000 7f680344e740 20 client.27550 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.745+0000 7f680344e740 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555d847b8260 0x555d847ba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.745+0000 7f680344e740 4 client.27550 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.745+0000 7f680344e740 1 -- 172.21.15.46:0/1451132956 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d84710950 con 0x555d847b8260 2024-09-10T07:45:29.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.746+0000 7f67fde00640 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555d847b8260 0x555d847ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.748+0000 7f67fde00640 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555d847b8260 0x555d847ba680 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c85ed2417103a974 server_cookie=e880c041179c966a in_seq=0 out_seq=0 2024-09-10T07:45:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.749+0000 7f67db400640 10 client.27550 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.749+0000 7f67db400640 1 -- 172.21.15.46:0/1451132956 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d84710950 con 0x555d847b8260 2024-09-10T07:45:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.749+0000 7f67db400640 10 client.27550 handle_command_reply: tid=0 2024-09-10T07:45:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.749+0000 7f680344e740 1 client.27550 shutdown 2024-09-10T07:45:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.751+0000 7f680344e740 1 -- 172.21.15.46:0/1451132956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 msgr2=0x555d847a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.751+0000 7f680344e740 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847a5270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f67e80b8b10 tx=0x7f67e80b6390 comp rx=0 tx=0).stop 2024-09-10T07:45:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 1 -- 172.21.15.46:0/1451132956 shutdown_connections 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555d847b8260 0x555d847ba680 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d847a57b0 0x555d847b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847a5270 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 1 --2- 172.21.15.46:0/1451132956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 1 -- 172.21.15.46:0/1451132956 >> 172.21.15.46:0/1451132956 conn(0x555d84799200 msgr2=0x555d8479eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f67db400640 0 client.27550 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 1 -- 172.21.15.46:0/1451132956 shutdown_connections 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 1 -- 172.21.15.46:0/1451132956 wait complete. 2024-09-10T07:45:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.752+0000 7f680344e740 20 client.27550 trim_cache size 0 max 16384 2024-09-10T07:45:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.780+0000 7f680344e740 1 Processor -- start 2024-09-10T07:45:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.782+0000 7f680344e740 1 -- start start 2024-09-10T07:45:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.782+0000 7f680344e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d84799200 0x555d847b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.782+0000 7f680344e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479cc70 0x555d847a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.782+0000 7f680344e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847bff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.782+0000 7f680344e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67f81bdb30 con 0x555d8479cc70 2024-09-10T07:45:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.782+0000 7f680344e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67f81bf6a0 con 0x555d84799200 2024-09-10T07:45:29.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.782+0000 7f680344e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67f81be200 con 0x555d8479d590 2024-09-10T07:45:29.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.784+0000 7f67fde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d84799200 0x555d847b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.784+0000 7f67fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479cc70 0x555d847a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.784+0000 7f67fde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d84799200 0x555d847b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44308/0 (socket says 172.21.15.46:44308) 2024-09-10T07:45:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.784+0000 7f67fde00640 1 -- 172.21.15.46:0/2646943345 learned_addr learned my addr 172.21.15.46:0/2646943345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.784+0000 7f67fe800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.785+0000 7f67fde00640 1 -- 172.21.15.46:0/2646943345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 msgr2=0x555d847bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.785+0000 7f67fde00640 1 --2- 172.21.15.46:0/2646943345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.785+0000 7f67fde00640 1 -- 172.21.15.46:0/2646943345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479cc70 msgr2=0x555d847a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.786+0000 7f67fde00640 1 --2- 172.21.15.46:0/2646943345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479cc70 0x555d847a6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.786+0000 7f67fde00640 1 -- 172.21.15.46:0/2646943345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d847b4b70 con 0x555d84799200 2024-09-10T07:45:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.786+0000 7f67fe800640 1 --2- 172.21.15.46:0/2646943345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.787+0000 7f67fde00640 1 --2- 172.21.15.46:0/2646943345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d84799200 0x555d847b3cf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f67ec004c60 tx=0x7f67ec00d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8aac8f98639335b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.788+0000 7f67e6600640 1 -- 172.21.15.46:0/2646943345 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec014070 con 0x555d84799200 2024-09-10T07:45:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.788+0000 7f67e6600640 1 -- 172.21.15.46:0/2646943345 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec030b30 con 0x555d84799200 2024-09-10T07:45:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.788+0000 7f67e6600640 1 -- 172.21.15.46:0/2646943345 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec016050 con 0x555d84799200 2024-09-10T07:45:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.790+0000 7f680344e740 1 -- 172.21.15.46:0/2646943345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d84799200 msgr2=0x555d847b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.790+0000 7f680344e740 1 --2- 172.21.15.46:0/2646943345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d84799200 0x555d847b3cf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f67ec004c60 tx=0x7f67ec00d2e0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.790+0000 7f680344e740 1 -- 172.21.15.46:0/2646943345 shutdown_connections 2024-09-10T07:45:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.790+0000 7f680344e740 1 --2- 172.21.15.46:0/2646943345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d8479d590 0x555d847bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.790+0000 7f680344e740 1 --2- 172.21.15.46:0/2646943345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479cc70 0x555d847a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.790+0000 7f680344e740 1 --2- 172.21.15.46:0/2646943345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d84799200 0x555d847b3cf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.791+0000 7f680344e740 1 -- 172.21.15.46:0/2646943345 >> 172.21.15.46:0/2646943345 conn(0x555d847a53d0 msgr2=0x555d847a6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.791+0000 7f680344e740 1 -- 172.21.15.46:0/2646943345 shutdown_connections 2024-09-10T07:45:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.791+0000 7f680344e740 1 -- 172.21.15.46:0/2646943345 wait complete. 2024-09-10T07:45:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.792+0000 7f680344e740 1 -- start start 2024-09-10T07:45:29.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.792+0000 7f680344e740 1 Processor -- start 2024-09-10T07:45:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.793+0000 7f680344e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84799200 0x555d847abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.794+0000 7f680344e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.794+0000 7f680344e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847ac820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.794+0000 7f680344e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67f81befa0 con 0x555d8479d590 2024-09-10T07:45:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.794+0000 7f680344e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67f81be8d0 con 0x555d8479cc70 2024-09-10T07:45:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.794+0000 7f680344e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67f81bbb50 con 0x555d84799200 2024-09-10T07:45:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.795+0000 7f680344e740 10 client.0 _ll_register_callbacks cb 0x7f67e8039f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.795+0000 7f67fde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84799200 0x555d847abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.795+0000 7f67fde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84799200 0x555d847abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49720/0 (socket says 172.21.15.46:49720) 2024-09-10T07:45:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.795+0000 7f67fde00640 1 -- 172.21.15.46:0/4264647871 learned_addr learned my addr 172.21.15.46:0/4264647871 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.795+0000 7f67fd400640 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.795+0000 7f67fe800640 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.796+0000 7f67fd400640 1 -- 172.21.15.46:0/4264647871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84799200 msgr2=0x555d847abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.796+0000 7f67fd400640 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84799200 0x555d847abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.796+0000 7f67fd400640 1 -- 172.21.15.46:0/4264647871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 msgr2=0x555d847ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.796+0000 7f67fd400640 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.796+0000 7f67fd400640 1 -- 172.21.15.46:0/4264647871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d84822100 con 0x555d8479cc70 2024-09-10T07:45:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.797+0000 7f67fde00640 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84799200 0x555d847abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.798+0000 7f67db400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.798+0000 7f67fd400640 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847ac2e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f67e8004fa0 tx=0x7f67e8005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d0c6aa4e05c790 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.798+0000 7f67db400640 1 -- 172.21.15.46:0/4264647871 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e80b8600 con 0x555d8479cc70 2024-09-10T07:45:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.798+0000 7f67db400640 1 -- 172.21.15.46:0/4264647871 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f67e80c43c0 con 0x555d8479cc70 2024-09-10T07:45:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.799+0000 7f680344e740 1 -- 172.21.15.46:0/4264647871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ec030b30 con 0x555d8479cc70 2024-09-10T07:45:29.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.800+0000 7f67db400640 1 -- 172.21.15.46:0/4264647871 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f67e80b87d0 con 0x555d8479cc70 2024-09-10T07:45:29.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.800+0000 7f680344e740 10 client.27614 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:29.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.800+0000 7f680344e740 1 -- 172.21.15.46:0/4264647871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555d84822100 con 0x555d8479cc70 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.801+0000 7f67db400640 1 -- 172.21.15.46:0/4264647871 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec030b30 con 0x555d8479cc70 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.801+0000 7f67db400640 1 -- 172.21.15.46:0/4264647871 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f67e80b94f0 con 0x555d8479cc70 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.801+0000 7f680344e740 10 client.27614 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.801+0000 7f680344e740 10 client.27614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.801+0000 7f680344e740 20 client.27614 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.801+0000 7f680344e740 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555d847a57b0 0x555d84825770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.801+0000 7f680344e740 4 client.27614 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.801+0000 7f680344e740 1 -- 172.21.15.46:0/4264647871 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d8470ae60 con 0x555d847a57b0 2024-09-10T07:45:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.802+0000 7f67fde00640 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555d847a57b0 0x555d84825770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.804+0000 7f67fde00640 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555d847a57b0 0x555d84825770 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f190d77c7226c969 server_cookie=68660bff65a4e9b0 in_seq=0 out_seq=0 2024-09-10T07:45:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.804+0000 7f67db400640 10 client.27614 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f67db400640 1 -- 172.21.15.46:0/4264647871 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d8470ae60 con 0x555d847a57b0 2024-09-10T07:45:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f67db400640 10 client.27614 handle_command_reply: tid=0 2024-09-10T07:45:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 client.27614 shutdown 2024-09-10T07:45:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 -- 172.21.15.46:0/4264647871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 msgr2=0x555d847ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847ac2e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f67e8004fa0 tx=0x7f67e8005b70 comp rx=0 tx=0).stop 2024-09-10T07:45:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 -- 172.21.15.46:0/4264647871 shutdown_connections 2024-09-10T07:45:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x555d847a57b0 0x555d84825770 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d8479d590 0x555d847ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d8479cc70 0x555d847ac2e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 --2- 172.21.15.46:0/4264647871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84799200 0x555d847abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.805+0000 7f680344e740 1 -- 172.21.15.46:0/4264647871 >> 172.21.15.46:0/4264647871 conn(0x555d847a53d0 msgr2=0x555d847a6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:29.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.807+0000 7f67db400640 0 client.27614 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:29.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.807+0000 7f680344e740 1 -- 172.21.15.46:0/4264647871 shutdown_connections 2024-09-10T07:45:29.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.807+0000 7f680344e740 1 -- 172.21.15.46:0/4264647871 wait complete. 2024-09-10T07:45:29.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.807+0000 7f680344e740 20 client.27614 trim_cache size 0 max 16384 2024-09-10T07:45:29.809 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.810+0000 7f6800800640 1 -- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67e00a2d80 msgr2=0x7f67e00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.810+0000 7f6800800640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67e00a2d80 0x7f67e00a5240 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00bb10 tx=0x7f67ec004bf0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.810+0000 7f6800800640 1 -- 172.21.15.46:0/77617572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67f8109b10 msgr2=0x7f67f810a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.810+0000 7f6800800640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67f8109b10 0x7f67f810a910 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f67e8004bd0 tx=0x7f67e8005320 comp rx=0 tx=0).stop 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.811+0000 7f6800800640 1 -- 172.21.15.46:0/77617572 shutdown_connections 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.811+0000 7f6800800640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67e00a2d80 0x7f67e00a5240 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.811+0000 7f6800800640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f8114ab0 0x7f67f810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.811+0000 7f6800800640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67f8109b10 0x7f67f810a910 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.811+0000 7f6800800640 1 --2- 172.21.15.46:0/77617572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67f81091f0 0x7f67f810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:29.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.811+0000 7f6800800640 1 -- 172.21.15.46:0/77617572 >> 172.21.15.46:0/77617572 conn(0x7f67f806e140 msgr2=0x7f67f8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:29.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.811+0000 7f6800800640 1 -- 172.21.15.46:0/77617572 shutdown_connections 2024-09-10T07:45:29.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:29.812+0000 7f6800800640 1 -- 172.21.15.46:0/77617572 wait complete. 2024-09-10T07:45:30.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:29 smithi046 ceph-mon[30705]: pgmap v665: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-09-10T07:45:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:29 smithi181 ceph-mon[35397]: pgmap v665: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-09-10T07:45:30.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:29 smithi063 ceph-mon[35020]: pgmap v665: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-09-10T07:45:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 490.0 s, 678.4 tps, lat 148.120 ms stddev 170.910 2024-09-10T07:45:31.877 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:45:31.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:45:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:31 smithi181 ceph-mon[35397]: pgmap v666: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2024-09-10T07:45:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:31 smithi063 ceph-mon[35020]: pgmap v666: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2024-09-10T07:45:32.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:31 smithi046 ceph-mon[30705]: pgmap v666: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2024-09-10T07:45:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 -- 172.21.15.46:0/2565555594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c105910 msgr2=0x7fa21c10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 --2- 172.21.15.46:0/2565555594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c105910 0x7fa21c10c1a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa214009ca0 tx=0x7fa21402a8f0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 -- 172.21.15.46:0/2565555594 shutdown_connections 2024-09-10T07:45:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 --2- 172.21.15.46:0/2565555594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c105910 0x7fa21c10c1a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 --2- 172.21.15.46:0/2565555594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c104f50 0x7fa21c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 --2- 172.21.15.46:0/2565555594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa21c103d60 0x7fa21c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 -- 172.21.15.46:0/2565555594 >> 172.21.15.46:0/2565555594 conn(0x7fa21c0ff4e0 msgr2=0x7fa21c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 -- 172.21.15.46:0/2565555594 shutdown_connections 2024-09-10T07:45:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.752+0000 7fa222800640 1 -- 172.21.15.46:0/2565555594 wait complete. 2024-09-10T07:45:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.756+0000 7fa222800640 1 Processor -- start 2024-09-10T07:45:33.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa222800640 1 -- start start 2024-09-10T07:45:33.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa222800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa21c103d60 0x7fa21c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa222800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c104f50 0x7fa21c073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa222800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c105910 0x7fa21c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa222800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c06c660 con 0x7fa21c105910 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa222800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c06c4e0 con 0x7fa21c103d60 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa222800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c06c7e0 con 0x7fa21c104f50 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa212a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c104f50 0x7fa21c073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa21be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c105910 0x7fa21c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa212a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c104f50 0x7fa21c073c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49740/0 (socket says 172.21.15.46:49740) 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa21be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c105910 0x7fa21c071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43198/0 (socket says 172.21.15.46:43198) 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.757+0000 7fa212a00640 1 -- 172.21.15.46:0/4180772471 learned_addr learned my addr 172.21.15.46:0/4180772471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.758+0000 7fa21be00640 1 -- 172.21.15.46:0/4180772471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c104f50 msgr2=0x7fa21c073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.758+0000 7fa21be00640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c104f50 0x7fa21c073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.758+0000 7fa21be00640 1 -- 172.21.15.46:0/4180772471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa21c103d60 msgr2=0x7fa21c0736c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:45:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.758+0000 7fa21b400640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa21c103d60 0x7fa21c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.758+0000 7fa21be00640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa21c103d60 0x7fa21c0736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.758+0000 7fa21be00640 1 -- 172.21.15.46:0/4180772471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c072400 con 0x7fa21c105910 2024-09-10T07:45:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.759+0000 7fa21b400640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa21c103d60 0x7fa21c0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.759+0000 7fa21be00640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c105910 0x7fa21c071d00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa214031af0 tx=0x7fa214031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.760+0000 7fa218c00640 1 -- 172.21.15.46:0/4180772471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214004040 con 0x7fa21c105910 2024-09-10T07:45:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.760+0000 7fa222800640 1 -- 172.21.15.46:0/4180772471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21c072690 con 0x7fa21c105910 2024-09-10T07:45:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.760+0000 7fa222800640 1 -- 172.21.15.46:0/4180772471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21c10f100 con 0x7fa21c105910 2024-09-10T07:45:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.760+0000 7fa218c00640 1 -- 172.21.15.46:0/4180772471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa21402d070 con 0x7fa21c105910 2024-09-10T07:45:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.760+0000 7fa218c00640 1 -- 172.21.15.46:0/4180772471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21402fb00 con 0x7fa21c105910 2024-09-10T07:45:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.767+0000 7fa218c00640 1 -- 172.21.15.46:0/4180772471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa21402fca0 con 0x7fa21c105910 2024-09-10T07:45:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.769+0000 7fa225662740 1 Processor -- start 2024-09-10T07:45:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.770+0000 7fa218c00640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2000a2d30 0x7fa2000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.770+0000 7fa218c00640 1 -- 172.21.15.46:0/4180772471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa214113be0 con 0x7fa21c105910 2024-09-10T07:45:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.770+0000 7fa225662740 1 -- start start 2024-09-10T07:45:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa225662740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04a8a0 0x563d8d04aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa225662740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04b1e0 0x563d8d04fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa225662740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d052730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa225662740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d8cfa7100 con 0x563d8d04a8a0 2024-09-10T07:45:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa225662740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563d8cf4f1b0 con 0x563d8d04b1e0 2024-09-10T07:45:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa225662740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563d8cfa30c0 con 0x563d8d050270 2024-09-10T07:45:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa21be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d052730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa21be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d052730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49742/0 (socket says 172.21.15.46:49742) 2024-09-10T07:45:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.771+0000 7fa21be00640 1 -- 172.21.15.46:0/2876617228 learned_addr learned my addr 172.21.15.46:0/2876617228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.772+0000 7fa21be00640 1 -- 172.21.15.46:0/2876617228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04b1e0 msgr2=0x563d8d04fca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:45:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.772+0000 7fa21be00640 1 --2- 172.21.15.46:0/2876617228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04b1e0 0x563d8d04fca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.772+0000 7fa21be00640 1 -- 172.21.15.46:0/2876617228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04a8a0 msgr2=0x563d8d04aca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:45:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.772+0000 7fa21be00640 1 --2- 172.21.15.46:0/2876617228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04a8a0 0x563d8d04aca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.772+0000 7fa21be00640 1 -- 172.21.15.46:0/2876617228 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8ccba210 con 0x563d8d050270 2024-09-10T07:45:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.772+0000 7fa21be00640 1 --2- 172.21.15.46:0/2876617228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d052730 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa2140e87a0 tx=0x7fa2140dc3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ffffa4b3d1672a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.773+0000 7fa212a00640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2000a2d30 0x7fa2000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.773+0000 7fa210c00640 1 -- 172.21.15.46:0/2876617228 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21402ee50 con 0x563d8d050270 2024-09-10T07:45:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.773+0000 7fa210c00640 1 -- 172.21.15.46:0/2876617228 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa214036070 con 0x563d8d050270 2024-09-10T07:45:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.773+0000 7fa210c00640 1 -- 172.21.15.46:0/2876617228 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21402e2a0 con 0x563d8d050270 2024-09-10T07:45:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.775+0000 7fa225662740 1 -- 172.21.15.46:0/2876617228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 msgr2=0x563d8d052730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.775+0000 7fa225662740 1 --2- 172.21.15.46:0/2876617228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d052730 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa2140e87a0 tx=0x7fa2140dc3c0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.775+0000 7fa225662740 1 -- 172.21.15.46:0/2876617228 shutdown_connections 2024-09-10T07:45:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.775+0000 7fa225662740 1 --2- 172.21.15.46:0/2876617228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d052730 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.775+0000 7fa225662740 1 --2- 172.21.15.46:0/2876617228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04b1e0 0x563d8d04fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.775+0000 7fa225662740 1 --2- 172.21.15.46:0/2876617228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04a8a0 0x563d8d04aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.775+0000 7fa225662740 1 -- 172.21.15.46:0/2876617228 >> 172.21.15.46:0/2876617228 conn(0x563d8d045f10 msgr2=0x563d8d046310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.776+0000 7fa225662740 1 -- 172.21.15.46:0/2876617228 shutdown_connections 2024-09-10T07:45:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.776+0000 7fa212a00640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2000a2d30 0x7fa2000a51f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa204006a50 tx=0x7fa204008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.776+0000 7fa225662740 1 -- 172.21.15.46:0/2876617228 wait complete. 2024-09-10T07:45:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.779+0000 7fa225662740 1 -- start start 2024-09-10T07:45:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.780+0000 7fa225662740 1 Processor -- start 2024-09-10T07:45:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.780+0000 7fa225662740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04a8a0 0x563d8d0546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.780+0000 7fa225662740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d054c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.780+0000 7fa225662740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d05bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.780+0000 7fa225662740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d8d053000 con 0x563d8d04b1e0 2024-09-10T07:45:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.780+0000 7fa225662740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563d8cf30090 con 0x563d8d04a8a0 2024-09-10T07:45:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.780+0000 7fa225662740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563d8d053180 con 0x563d8d050270 2024-09-10T07:45:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.780+0000 7fa225662740 10 client.0 _ll_register_callbacks cb 0x563d8cf84f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.781+0000 7fa21be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d05bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.781+0000 7fa21b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04a8a0 0x563d8d0546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.782+0000 7fa21b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04a8a0 0x563d8d0546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44360/0 (socket says 172.21.15.46:44360) 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.782+0000 7fa21b400640 1 -- 172.21.15.46:0/226529689 learned_addr learned my addr 172.21.15.46:0/226529689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.782+0000 7fa212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d054c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.783+0000 7fa21be00640 1 -- 172.21.15.46:0/226529689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04a8a0 msgr2=0x563d8d0546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.783+0000 7fa21be00640 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04a8a0 0x563d8d0546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.783+0000 7fa21be00640 1 -- 172.21.15.46:0/226529689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 msgr2=0x563d8d054c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.783+0000 7fa21be00640 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d054c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.783+0000 7fa21be00640 1 -- 172.21.15.46:0/226529689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8cd86620 con 0x563d8d050270 2024-09-10T07:45:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.784+0000 7fa21be00640 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d05bc70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa21402f040 tx=0x7fa21402e370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44cda7dda05a2bb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.784+0000 7fa209600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.784+0000 7fa209600640 1 -- 172.21.15.46:0/226529689 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214037430 con 0x563d8d050270 2024-09-10T07:45:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.785+0000 7fa209600640 1 -- 172.21.15.46:0/226529689 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa2140e40d0 con 0x563d8d050270 2024-09-10T07:45:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.785+0000 7fa225662740 1 -- 172.21.15.46:0/226529689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa214036070 con 0x563d8d050270 2024-09-10T07:45:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.785+0000 7fa209600640 1 -- 172.21.15.46:0/226529689 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa2140098d0 con 0x563d8d050270 2024-09-10T07:45:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.785+0000 7fa225662740 10 client.27568 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.785+0000 7fa225662740 1 -- 172.21.15.46:0/226529689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563d8d053000 con 0x563d8d050270 2024-09-10T07:45:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.786+0000 7fa209600640 1 -- 172.21.15.46:0/226529689 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa214036070 con 0x563d8d050270 2024-09-10T07:45:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.786+0000 7fa209600640 1 -- 172.21.15.46:0/226529689 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7fa2140dc6d0 con 0x563d8d050270 2024-09-10T07:45:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.786+0000 7fa225662740 10 client.27568 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.786+0000 7fa225662740 10 client.27568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.786+0000 7fa225662740 20 client.27568 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.786+0000 7fa225662740 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563d8d05fe60 0x563d8d062280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.786+0000 7fa225662740 4 client.27568 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.786+0000 7fa225662740 1 -- 172.21.15.46:0/226529689 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d8cfb4830 con 0x563d8d05fe60 2024-09-10T07:45:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.788+0000 7fa21b400640 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563d8d05fe60 0x563d8d062280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.791+0000 7fa21b400640 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563d8d05fe60 0x563d8d062280 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80be9c44c14f0df server_cookie=b2e1dd8dc9ac79ba in_seq=0 out_seq=0 2024-09-10T07:45:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.791+0000 7fa209600640 10 client.27568 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.794+0000 7fa209600640 1 -- 172.21.15.46:0/226529689 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d8cfb4830 con 0x563d8d05fe60 2024-09-10T07:45:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.794+0000 7fa209600640 10 client.27568 handle_command_reply: tid=0 2024-09-10T07:45:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.794+0000 7fa225662740 1 client.27568 shutdown 2024-09-10T07:45:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa225662740 1 -- 172.21.15.46:0/226529689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 msgr2=0x563d8d05bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa225662740 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d05bc70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa21402f040 tx=0x7fa21402e370 comp rx=0 tx=0).stop 2024-09-10T07:45:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa225662740 1 -- 172.21.15.46:0/226529689 shutdown_connections 2024-09-10T07:45:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa225662740 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563d8d05fe60 0x563d8d062280 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa225662740 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d050270 0x563d8d05bc70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa225662740 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d054c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa225662740 1 --2- 172.21.15.46:0/226529689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d04a8a0 0x563d8d0546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa225662740 1 -- 172.21.15.46:0/226529689 >> 172.21.15.46:0/226529689 conn(0x563d8d045f10 msgr2=0x563d8d047720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.795+0000 7fa209600640 0 client.27568 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.796+0000 7fa225662740 1 -- 172.21.15.46:0/226529689 shutdown_connections 2024-09-10T07:45:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.796+0000 7fa225662740 1 -- 172.21.15.46:0/226529689 wait complete. 2024-09-10T07:45:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.796+0000 7fa225662740 20 client.27568 trim_cache size 0 max 16384 2024-09-10T07:45:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.824+0000 7fa225662740 1 Processor -- start 2024-09-10T07:45:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.824+0000 7fa225662740 1 -- start start 2024-09-10T07:45:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa225662740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04a8a0 0x563d8d05c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa225662740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d05bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa225662740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa225662740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f800faf0 con 0x563d8d04b1e0 2024-09-10T07:45:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa225662740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8011660 con 0x563d8d050270 2024-09-10T07:45:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa225662740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f80101c0 con 0x563d8d04a8a0 2024-09-10T07:45:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa21b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04a8a0 0x563d8d05c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa21be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d05bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d05bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43210/0 (socket says 172.21.15.46:43210) 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa212a00640 1 -- 172.21.15.46:0/187974520 learned_addr learned my addr 172.21.15.46:0/187974520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.825+0000 7fa21be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44362/0 (socket says 172.21.15.46:44362) 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa212a00640 1 -- 172.21.15.46:0/187974520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04a8a0 msgr2=0x563d8d05c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa212a00640 1 --2- 172.21.15.46:0/187974520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04a8a0 0x563d8d05c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa212a00640 1 -- 172.21.15.46:0/187974520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 msgr2=0x563d8d05c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa212a00640 1 --2- 172.21.15.46:0/187974520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa212a00640 1 -- 172.21.15.46:0/187974520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8d0628e0 con 0x563d8d04b1e0 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa212a00640 1 --2- 172.21.15.46:0/187974520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d05bc30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa20400d950 tx=0x7fa204013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a423bf83bd63e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa20be00640 1 -- 172.21.15.46:0/187974520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204014bc0 con 0x563d8d04b1e0 2024-09-10T07:45:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa20be00640 1 -- 172.21.15.46:0/187974520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa204025070 con 0x563d8d04b1e0 2024-09-10T07:45:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.826+0000 7fa20be00640 1 -- 172.21.15.46:0/187974520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2040123e0 con 0x563d8d04b1e0 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.827+0000 7fa225662740 1 -- 172.21.15.46:0/187974520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 msgr2=0x563d8d05bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.827+0000 7fa225662740 1 --2- 172.21.15.46:0/187974520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d05bc30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa20400d950 tx=0x7fa204013000 comp rx=0 tx=0).stop 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.828+0000 7fa225662740 1 -- 172.21.15.46:0/187974520 shutdown_connections 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.828+0000 7fa225662740 1 --2- 172.21.15.46:0/187974520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.828+0000 7fa225662740 1 --2- 172.21.15.46:0/187974520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04b1e0 0x563d8d05bc30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.828+0000 7fa225662740 1 --2- 172.21.15.46:0/187974520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04a8a0 0x563d8d05c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.828+0000 7fa225662740 1 -- 172.21.15.46:0/187974520 >> 172.21.15.46:0/187974520 conn(0x563d8d045f10 msgr2=0x563d8d0546f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.828+0000 7fa225662740 1 -- 172.21.15.46:0/187974520 shutdown_connections 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.828+0000 7fa225662740 1 -- 172.21.15.46:0/187974520 wait complete. 2024-09-10T07:45:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.829+0000 7fa225662740 1 -- start start 2024-09-10T07:45:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.829+0000 7fa225662740 1 Processor -- start 2024-09-10T07:45:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.829+0000 7fa225662740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04a8a0 0x563d8d058d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.829+0000 7fa225662740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04b1e0 0x563d8d0592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa225662740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa225662740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8010f60 con 0x563d8d04a8a0 2024-09-10T07:45:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa225662740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8010890 con 0x563d8d050270 2024-09-10T07:45:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa225662740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f800dad0 con 0x563d8d04b1e0 2024-09-10T07:45:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa225662740 10 client.0 _ll_register_callbacks cb 0x7fa2140dbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa212a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04b1e0 0x563d8d0592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44370/0 (socket says 172.21.15.46:44370) 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 -- 172.21.15.46:0/2561955507 learned_addr learned my addr 172.21.15.46:0/2561955507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 -- 172.21.15.46:0/2561955507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04b1e0 msgr2=0x563d8d0592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04b1e0 0x563d8d0592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 -- 172.21.15.46:0/2561955507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04a8a0 msgr2=0x563d8d058d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04a8a0 0x563d8d058d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 -- 172.21.15.46:0/2561955507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8d05a870 con 0x563d8d050270 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa21be00640 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05a260 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2140dfd60 tx=0x7fa214005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e0405429bee187 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.830+0000 7fa209600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa209600640 1 -- 172.21.15.46:0/2561955507 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140e0760 con 0x563d8d050270 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa209600640 1 -- 172.21.15.46:0/2561955507 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa2140e0360 con 0x563d8d050270 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa225662740 1 -- 172.21.15.46:0/2561955507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa204025070 con 0x563d8d050270 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa209600640 1 -- 172.21.15.46:0/2561955507 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa2140e8cd0 con 0x563d8d050270 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa225662740 10 client.27623 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa225662740 1 -- 172.21.15.46:0/2561955507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563d8d05a870 con 0x563d8d050270 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa209600640 1 -- 172.21.15.46:0/2561955507 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa204025070 con 0x563d8d050270 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa209600640 1 -- 172.21.15.46:0/2561955507 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7fa2140c9120 con 0x563d8d050270 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.831+0000 7fa225662740 10 client.27623 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.832+0000 7fa225662740 10 client.27623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.832+0000 7fa225662740 20 client.27623 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.832+0000 7fa225662740 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563d8d0cab40 0x563d8d0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.832+0000 7fa225662740 4 client.27623 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:45:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.832+0000 7fa225662740 1 -- 172.21.15.46:0/2561955507 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d8cfaeb80 con 0x563d8d0cab40 2024-09-10T07:45:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.832+0000 7fa21b400640 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563d8d0cab40 0x563d8d0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.833+0000 7fa21b400640 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563d8d0cab40 0x563d8d0ccf30 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7708d761eef0f76 server_cookie=cdaff69c8011fd5 in_seq=0 out_seq=0 2024-09-10T07:45:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.833+0000 7fa209600640 10 client.27623 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa209600640 1 -- 172.21.15.46:0/2561955507 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d8cfaeb80 con 0x563d8d0cab40 2024-09-10T07:45:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa209600640 10 client.27623 handle_command_reply: tid=0 2024-09-10T07:45:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 client.27623 shutdown 2024-09-10T07:45:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 -- 172.21.15.46:0/2561955507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 msgr2=0x563d8d05a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05a260 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2140dfd60 tx=0x7fa214005fb0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 -- 172.21.15.46:0/2561955507 shutdown_connections 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563d8d0cab40 0x563d8d0ccf30 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563d8d050270 0x563d8d05a260 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d8d04b1e0 0x563d8d0592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 --2- 172.21.15.46:0/2561955507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d8d04a8a0 0x563d8d058d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.845+0000 7fa225662740 1 -- 172.21.15.46:0/2561955507 >> 172.21.15.46:0/2561955507 conn(0x563d8d045f10 msgr2=0x563d8d05c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.846+0000 7fa209600640 0 client.27623 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.846+0000 7fa225662740 1 -- 172.21.15.46:0/2561955507 shutdown_connections 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.846+0000 7fa225662740 1 -- 172.21.15.46:0/2561955507 wait complete. 2024-09-10T07:45:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.846+0000 7fa225662740 20 client.27623 trim_cache size 0 max 16384 2024-09-10T07:45:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:45:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:45:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:45:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:45:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 -- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2000a2d30 msgr2=0x7fa2000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2000a2d30 0x7fa2000a51f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa204006a50 tx=0x7fa204008040 comp rx=0 tx=0).stop 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 -- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c105910 msgr2=0x7fa21c071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c105910 0x7fa21c071d00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa214031af0 tx=0x7fa214031a00 comp rx=0 tx=0).stop 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 -- 172.21.15.46:0/4180772471 shutdown_connections 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2000a2d30 0x7fa2000a51f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c105910 0x7fa21c071d00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21c104f50 0x7fa21c073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 --2- 172.21.15.46:0/4180772471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa21c103d60 0x7fa21c0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 -- 172.21.15.46:0/4180772471 >> 172.21.15.46:0/4180772471 conn(0x7fa21c0ff4e0 msgr2=0x7fa21c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 -- 172.21.15.46:0/4180772471 shutdown_connections 2024-09-10T07:45:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:33.847+0000 7fa222800640 1 -- 172.21.15.46:0/4180772471 wait complete. 2024-09-10T07:45:34.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.001+0000 7f850aa00640 1 -- 172.21.15.46:0/1179795586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8504103ff0 msgr2=0x7f85041043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.001+0000 7f850aa00640 1 --2- 172.21.15.46:0/1179795586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8504103ff0 0x7f85041043d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009900 tx=0x7f84f402a400 comp rx=0 tx=0).stop 2024-09-10T07:45:34.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.001+0000 7f850aa00640 1 -- 172.21.15.46:0/1179795586 shutdown_connections 2024-09-10T07:45:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.001+0000 7f850aa00640 1 --2- 172.21.15.46:0/1179795586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504109120 0x7f8504110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.001+0000 7f850aa00640 1 --2- 172.21.15.46:0/1179795586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85041049a0 0x7f85041089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.001+0000 7f850aa00640 1 --2- 172.21.15.46:0/1179795586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8504103ff0 0x7f85041043d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.001+0000 7f850aa00640 1 -- 172.21.15.46:0/1179795586 >> 172.21.15.46:0/1179795586 conn(0x7f85040ffab0 msgr2=0x7f8504101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.002+0000 7f850aa00640 1 -- 172.21.15.46:0/1179795586 shutdown_connections 2024-09-10T07:45:34.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.002+0000 7f850aa00640 1 -- 172.21.15.46:0/1179795586 wait complete. 2024-09-10T07:45:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.003+0000 7f850aa00640 1 Processor -- start 2024-09-10T07:45:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f850aa00640 1 -- start start 2024-09-10T07:45:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f850aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504103ff0 0x7f850419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f850aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85041049a0 0x7f850419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f850aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8504109120 0x7f850419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f850aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8504114170 con 0x7f85041049a0 2024-09-10T07:45:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f850aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8504113ff0 con 0x7f8504103ff0 2024-09-10T07:45:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f850aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85041142f0 con 0x7f8504109120 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f8503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504103ff0 0x7f850419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f8503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85041049a0 0x7f850419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f8503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504103ff0 0x7f850419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44382/0 (socket says 172.21.15.46:44382) 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.004+0000 7f8503e00640 1 -- 172.21.15.46:0/4292297624 learned_addr learned my addr 172.21.15.46:0/4292297624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.005+0000 7f8508a00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8504109120 0x7f850419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.005+0000 7f8503e00640 1 -- 172.21.15.46:0/4292297624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8504109120 msgr2=0x7f850419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.005+0000 7f8503e00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8504109120 0x7f850419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.005+0000 7f8503e00640 1 -- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85041049a0 msgr2=0x7f850419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.005+0000 7f8503e00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85041049a0 0x7f850419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.005+0000 7f8503e00640 1 -- 172.21.15.46:0/4292297624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850419f5c0 con 0x7f8504103ff0 2024-09-10T07:45:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.005+0000 7f8503e00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504103ff0 0x7f850419a610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f84f4004770 tx=0x7f84f4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.006+0000 7f8500c00640 1 -- 172.21.15.46:0/4292297624 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f403f070 con 0x7f8504103ff0 2024-09-10T07:45:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.006+0000 7f8500c00640 1 -- 172.21.15.46:0/4292297624 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f4031ca0 con 0x7f8504103ff0 2024-09-10T07:45:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.006+0000 7f8500c00640 1 -- 172.21.15.46:0/4292297624 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4035030 con 0x7f8504103ff0 2024-09-10T07:45:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.006+0000 7f850aa00640 1 -- 172.21.15.46:0/4292297624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850419f850 con 0x7f8504103ff0 2024-09-10T07:45:34.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.006+0000 7f850aa00640 1 -- 172.21.15.46:0/4292297624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850419fce0 con 0x7f8504103ff0 2024-09-10T07:45:34.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.009+0000 7f850d619740 1 Processor -- start 2024-09-10T07:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.009+0000 7f850d619740 1 -- start start 2024-09-10T07:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.009+0000 7f850d619740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f56aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.009+0000 7f850d619740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f575760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.009+0000 7f850d619740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f578090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.009+0000 7f850d619740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c11f4d1290 con 0x55c11f575ca0 2024-09-10T07:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.009+0000 7f850d619740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c11f478b40 con 0x55c11f56b3e0 2024-09-10T07:45:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.009+0000 7f850d619740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c11f4cd250 con 0x55c11f56aac0 2024-09-10T07:45:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8508a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f56aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8508a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f56aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49786/0 (socket says 172.21.15.46:49786) 2024-09-10T07:45:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8508a00640 1 -- 172.21.15.46:0/2103535585 learned_addr learned my addr 172.21.15.46:0/2103535585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8503400640 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f578090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8500c00640 1 -- 172.21.15.46:0/4292297624 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f84f4042070 con 0x7f8504103ff0 2024-09-10T07:45:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8508a00640 1 -- 172.21.15.46:0/2103535585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 msgr2=0x55c11f575760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8508a00640 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f575760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8508a00640 1 -- 172.21.15.46:0/2103535585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 msgr2=0x55c11f578090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8508a00640 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f578090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8508a00640 1 -- 172.21.15.46:0/2103535585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11f2426a0 con 0x55c11f56aac0 2024-09-10T07:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.010+0000 7f8503400640 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f578090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.011+0000 7f8508a00640 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f56aea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009f50 tx=0x7f84fc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26034aaeac2b8c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.012+0000 7f84f9200640 1 -- 172.21.15.46:0/2103535585 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc004d20 con 0x55c11f56aac0 2024-09-10T07:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.012+0000 7f84f9200640 1 -- 172.21.15.46:0/2103535585 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84fc004050 con 0x55c11f56aac0 2024-09-10T07:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.012+0000 7f84f9200640 1 -- 172.21.15.46:0/2103535585 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc005280 con 0x55c11f56aac0 2024-09-10T07:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.012+0000 7f8500c00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84e80a2d50 0x7f84e80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.012+0000 7f8500c00640 1 -- 172.21.15.46:0/4292297624 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f84f40efb40 con 0x7f8504103ff0 2024-09-10T07:45:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.013+0000 7f8503e00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84e80a2d50 0x7f84e80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.013+0000 7f850d619740 1 -- 172.21.15.46:0/2103535585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 msgr2=0x55c11f56aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.013+0000 7f850d619740 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f56aea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009f50 tx=0x7f84fc0118d0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.017+0000 7f8503e00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84e80a2d50 0x7f84e80a5210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84f40bbd80 tx=0x7f84f402a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:34.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.017+0000 7f850d619740 1 -- 172.21.15.46:0/2103535585 shutdown_connections 2024-09-10T07:45:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.017+0000 7f850d619740 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f578090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.017+0000 7f850d619740 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f575760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.017+0000 7f850d619740 1 --2- 172.21.15.46:0/2103535585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f56aea0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.017+0000 7f850d619740 1 -- 172.21.15.46:0/2103535585 >> 172.21.15.46:0/2103535585 conn(0x55c11f567050 msgr2=0x55c11f567430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.018+0000 7f850d619740 1 -- 172.21.15.46:0/2103535585 shutdown_connections 2024-09-10T07:45:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.018+0000 7f850d619740 1 -- 172.21.15.46:0/2103535585 wait complete. 2024-09-10T07:45:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.019+0000 7f850d619740 1 -- start start 2024-09-10T07:45:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.019+0000 7f850d619740 1 Processor -- start 2024-09-10T07:45:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.023+0000 7f850d619740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56aac0 0x55c11f575640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.023+0000 7f850d619740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56b3e0 0x55c11f5728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.023+0000 7f850d619740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f572e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.023+0000 7f850d619740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c11f578910 con 0x55c11f575ca0 2024-09-10T07:45:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.023+0000 7f850d619740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c11f48bfa0 con 0x55c11f56aac0 2024-09-10T07:45:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.023+0000 7f850d619740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c11f578a90 con 0x55c11f56b3e0 2024-09-10T07:45:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.023+0000 7f850d619740 10 client.0 _ll_register_callbacks cb 0x55c11f1dfba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.024+0000 7f8508a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56aac0 0x55c11f575640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.024+0000 7f8508a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56aac0 0x55c11f575640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44386/0 (socket says 172.21.15.46:44386) 2024-09-10T07:45:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.024+0000 7f8508a00640 1 -- 172.21.15.46:0/4121044252 learned_addr learned my addr 172.21.15.46:0/4121044252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.025+0000 7f8503400640 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56b3e0 0x55c11f5728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.025+0000 7f8503400640 1 -- 172.21.15.46:0/4121044252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56aac0 msgr2=0x55c11f575640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.026+0000 7f8503400640 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56aac0 0x55c11f575640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.026+0000 7f8503400640 1 -- 172.21.15.46:0/4121044252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 msgr2=0x55c11f572e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.026+0000 7f8503400640 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f572e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.026+0000 7f8503400640 1 -- 172.21.15.46:0/4121044252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11f5736c0 con 0x55c11f56b3e0 2024-09-10T07:45:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.026+0000 7f8508a00640 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56aac0 0x55c11f575640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.027+0000 7f8503400640 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56b3e0 0x55c11f5728f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85040fff60 tx=0x7f84f0013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e475779c286fed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.027+0000 7f84ed600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:34.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.027+0000 7f84ed600640 1 -- 172.21.15.46:0/4121044252 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f00119c0 con 0x55c11f56b3e0 2024-09-10T07:45:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.028+0000 7f84ed600640 1 -- 172.21.15.46:0/4121044252 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f84f0011b60 con 0x55c11f56b3e0 2024-09-10T07:45:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.030+0000 7f850d619740 1 -- 172.21.15.46:0/4121044252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84fc004050 con 0x55c11f56b3e0 2024-09-10T07:45:34.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.030+0000 7f84ed600640 1 -- 172.21.15.46:0/4121044252 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f84f0011d60 con 0x55c11f56b3e0 2024-09-10T07:45:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.030+0000 7f850d619740 10 client.27586 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.030+0000 7f850d619740 1 -- 172.21.15.46:0/4121044252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c11f58d0f0 con 0x55c11f56b3e0 2024-09-10T07:45:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.031+0000 7f84ed600640 1 -- 172.21.15.46:0/4121044252 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84fc004050 con 0x55c11f56b3e0 2024-09-10T07:45:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.031+0000 7f84ed600640 1 -- 172.21.15.46:0/4121044252 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f84f001a070 con 0x55c11f56b3e0 2024-09-10T07:45:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.033+0000 7f850d619740 10 client.27586 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.033+0000 7f850d619740 10 client.27586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.033+0000 7f850d619740 20 client.27586 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.033+0000 7f850d619740 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c11f58e500 0x55c11f590930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.033+0000 7f850d619740 4 client.27586 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.033+0000 7f850d619740 1 -- 172.21.15.46:0/4121044252 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c11f4de800 con 0x55c11f58e500 2024-09-10T07:45:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.038+0000 7f8508a00640 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c11f58e500 0x55c11f590930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.040+0000 7f8508a00640 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c11f58e500 0x55c11f590930 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9df9b6fc73227e server_cookie=a7ba6ad9e092e89b in_seq=0 out_seq=0 2024-09-10T07:45:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.040+0000 7f84ed600640 10 client.27586 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:34.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.041+0000 7f84ed600640 1 -- 172.21.15.46:0/4121044252 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c11f4de800 con 0x55c11f58e500 2024-09-10T07:45:34.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.041+0000 7f84ed600640 10 client.27586 handle_command_reply: tid=0 2024-09-10T07:45:34.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.041+0000 7f850d619740 1 client.27586 shutdown 2024-09-10T07:45:34.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.042+0000 7f850d619740 1 -- 172.21.15.46:0/4121044252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56b3e0 msgr2=0x55c11f5728f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.042+0000 7f850d619740 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56b3e0 0x55c11f5728f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85040fff60 tx=0x7f84f0013b70 comp rx=0 tx=0).stop 2024-09-10T07:45:34.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f850d619740 1 -- 172.21.15.46:0/4121044252 shutdown_connections 2024-09-10T07:45:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f850d619740 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c11f58e500 0x55c11f590930 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f84ed600640 0 client.27586 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f850d619740 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f572e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f850d619740 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56b3e0 0x55c11f5728f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f850d619740 1 --2- 172.21.15.46:0/4121044252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56aac0 0x55c11f575640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f850d619740 1 -- 172.21.15.46:0/4121044252 >> 172.21.15.46:0/4121044252 conn(0x55c11f567050 msgr2=0x55c11f5661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f850d619740 1 -- 172.21.15.46:0/4121044252 shutdown_connections 2024-09-10T07:45:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.043+0000 7f850d619740 1 -- 172.21.15.46:0/4121044252 wait complete. 2024-09-10T07:45:34.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.044+0000 7f850d619740 20 client.27586 trim_cache size 0 max 16384 2024-09-10T07:45:34.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.088+0000 7f850d619740 1 Processor -- start 2024-09-10T07:45:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.088+0000 7f850d619740 1 -- start start 2024-09-10T07:45:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f850d619740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f596200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f850d619740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f596740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f850d619740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f57dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f850d619740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85041acb10 con 0x55c11f575ca0 2024-09-10T07:45:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f850d619740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85041ae680 con 0x55c11f56b3e0 2024-09-10T07:45:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f850d619740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85041ad1e0 con 0x55c11f56aac0 2024-09-10T07:45:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f8508a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f596200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f8508a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f596200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49800/0 (socket says 172.21.15.46:49800) 2024-09-10T07:45:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.089+0000 7f8508a00640 1 -- 172.21.15.46:0/3844056752 learned_addr learned my addr 172.21.15.46:0/3844056752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.090+0000 7f8503400640 1 --2- 172.21.15.46:0/3844056752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f596740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.091+0000 7f8503400640 1 -- 172.21.15.46:0/3844056752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 msgr2=0x55c11f596200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.091+0000 7f8503400640 1 --2- 172.21.15.46:0/3844056752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f596200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.091+0000 7f8503400640 1 -- 172.21.15.46:0/3844056752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 msgr2=0x55c11f57dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.091+0000 7f8503400640 1 --2- 172.21.15.46:0/3844056752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f57dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.091+0000 7f8503400640 1 -- 172.21.15.46:0/3844056752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11f573890 con 0x55c11f56b3e0 2024-09-10T07:45:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.091+0000 7f8503400640 1 --2- 172.21.15.46:0/3844056752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f596740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f84f0026a40 tx=0x7f84f0011f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4b9f722473a8bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.092+0000 7f84efe00640 1 -- 172.21.15.46:0/3844056752 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0005480 con 0x55c11f56b3e0 2024-09-10T07:45:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.092+0000 7f84efe00640 1 -- 172.21.15.46:0/3844056752 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f0005620 con 0x55c11f56b3e0 2024-09-10T07:45:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.092+0000 7f84efe00640 1 -- 172.21.15.46:0/3844056752 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0005480 con 0x55c11f56b3e0 2024-09-10T07:45:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.094+0000 7f850d619740 1 -- 172.21.15.46:0/3844056752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 msgr2=0x55c11f596740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.094+0000 7f850d619740 1 --2- 172.21.15.46:0/3844056752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f596740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f84f0026a40 tx=0x7f84f0011f80 comp rx=0 tx=0).stop 2024-09-10T07:45:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.094+0000 7f850d619740 1 -- 172.21.15.46:0/3844056752 shutdown_connections 2024-09-10T07:45:34.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.094+0000 7f850d619740 1 --2- 172.21.15.46:0/3844056752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f575ca0 0x55c11f57dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.094+0000 7f850d619740 1 --2- 172.21.15.46:0/3844056752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f596740 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.094+0000 7f850d619740 1 --2- 172.21.15.46:0/3844056752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f56aac0 0x55c11f596200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.094+0000 7f850d619740 1 -- 172.21.15.46:0/3844056752 >> 172.21.15.46:0/3844056752 conn(0x55c11f567050 msgr2=0x55c11f574970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.095+0000 7f850d619740 1 -- 172.21.15.46:0/3844056752 shutdown_connections 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.095+0000 7f850d619740 1 -- 172.21.15.46:0/3844056752 wait complete. 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.095+0000 7f850d619740 1 -- start start 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.096+0000 7f850d619740 1 Processor -- start 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.096+0000 7f850d619740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f56aac0 0x55c11f57bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.096+0000 7f850d619740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f57c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.096+0000 7f850d619740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f575ca0 0x55c11f57c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.096+0000 7f850d619740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85041adf80 con 0x55c11f56aac0 2024-09-10T07:45:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.096+0000 7f850d619740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85041ad8b0 con 0x55c11f56b3e0 2024-09-10T07:45:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.096+0000 7f850d619740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85041aaaf0 con 0x55c11f575ca0 2024-09-10T07:45:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.097+0000 7f850d619740 10 client.0 _ll_register_callbacks cb 0x7f84f00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:34.099 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:33 smithi046 ceph-mon[30705]: pgmap v667: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-09-10T07:45:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.099+0000 7f8508a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f575ca0 0x55c11f57c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.099+0000 7f8508a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f575ca0 0x55c11f57c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49802/0 (socket says 172.21.15.46:49802) 2024-09-10T07:45:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.099+0000 7f8508a00640 1 -- 172.21.15.46:0/1332157959 learned_addr learned my addr 172.21.15.46:0/1332157959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.099+0000 7f8503400640 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f57c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.100+0000 7f8508a00640 1 -- 172.21.15.46:0/1332157959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 msgr2=0x55c11f57c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:34.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.100+0000 7f8508a00640 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f57c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.101+0000 7f8508a00640 1 -- 172.21.15.46:0/1332157959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f56aac0 msgr2=0x55c11f57bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:34.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.101+0000 7f8508a00640 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f56aac0 0x55c11f57bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.101+0000 7f8508a00640 1 -- 172.21.15.46:0/1332157959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11f609240 con 0x55c11f575ca0 2024-09-10T07:45:34.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.102+0000 7f84ed600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:34.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.102+0000 7f8508a00640 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f575ca0 0x55c11f57c560 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0096e0 tx=0x7f84fc004050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d12195ee237187cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:34.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.103+0000 7f84ed600640 1 -- 172.21.15.46:0/1332157959 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc003c30 con 0x55c11f575ca0 2024-09-10T07:45:34.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.103+0000 7f850d619740 1 -- 172.21.15.46:0/1332157959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84f0005620 con 0x55c11f575ca0 2024-09-10T07:45:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.103+0000 7f84ed600640 1 -- 172.21.15.46:0/1332157959 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f84fc003a30 con 0x55c11f575ca0 2024-09-10T07:45:34.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.105+0000 7f84ed600640 1 -- 172.21.15.46:0/1332157959 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f84fc03b050 con 0x55c11f575ca0 2024-09-10T07:45:34.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.105+0000 7f850d619740 10 client.27592 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:34.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.105+0000 7f850d619740 1 -- 172.21.15.46:0/1332157959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c11f5ef410 con 0x55c11f575ca0 2024-09-10T07:45:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.106+0000 7f84ed600640 1 -- 172.21.15.46:0/1332157959 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f0005620 con 0x55c11f575ca0 2024-09-10T07:45:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.106+0000 7f84ed600640 1 -- 172.21.15.46:0/1332157959 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f84fc019430 con 0x55c11f575ca0 2024-09-10T07:45:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.106+0000 7f850d619740 10 client.27592 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.106+0000 7f850d619740 10 client.27592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.106+0000 7f850d619740 20 client.27592 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.106+0000 7f850d619740 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c11f5efb90 0x55c11f5f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.106+0000 7f850d619740 4 client.27592 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:45:34.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.106+0000 7f850d619740 1 -- 172.21.15.46:0/1332157959 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c11f4d8d10 con 0x55c11f5efb90 2024-09-10T07:45:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.107+0000 7f8503400640 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c11f5efb90 0x55c11f5f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.108+0000 7f84ed600640 10 client.27592 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:34.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.108+0000 7f8503400640 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c11f5efb90 0x55c11f5f1fb0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b4f20334272d69 server_cookie=9fe466e718b0a4f9 in_seq=0 out_seq=0 2024-09-10T07:45:34.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.111+0000 7f84ed600640 1 -- 172.21.15.46:0/1332157959 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c11f4d8d10 con 0x55c11f5efb90 2024-09-10T07:45:34.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.111+0000 7f84ed600640 10 client.27592 handle_command_reply: tid=0 2024-09-10T07:45:34.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.111+0000 7f850d619740 1 client.27592 shutdown 2024-09-10T07:45:34.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f850d619740 1 -- 172.21.15.46:0/1332157959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f575ca0 msgr2=0x55c11f57c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f850d619740 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f575ca0 0x55c11f57c560 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0096e0 tx=0x7f84fc004050 comp rx=0 tx=0).stop 2024-09-10T07:45:34.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f850d619740 1 -- 172.21.15.46:0/1332157959 shutdown_connections 2024-09-10T07:45:34.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f850d619740 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c11f5efb90 0x55c11f5f1fb0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f850d619740 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c11f575ca0 0x55c11f57c560 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f850d619740 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c11f56b3e0 0x55c11f57c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f850d619740 1 --2- 172.21.15.46:0/1332157959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c11f56aac0 0x55c11f57bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f84ed600640 0 client.27592 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:34.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.114+0000 7f850d619740 1 -- 172.21.15.46:0/1332157959 >> 172.21.15.46:0/1332157959 conn(0x55c11f567050 msgr2=0x55c11f571090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:34.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.115+0000 7f850d619740 1 -- 172.21.15.46:0/1332157959 shutdown_connections 2024-09-10T07:45:34.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.115+0000 7f850d619740 1 -- 172.21.15.46:0/1332157959 wait complete. 2024-09-10T07:45:34.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.117+0000 7f850d619740 20 client.27592 trim_cache size 0 max 16384 2024-09-10T07:45:34.119 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:45:34.119 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:45:34.119 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "16750b3d-056f-44ef-822f-c5f1782530c6", 2024-09-10T07:45:34.119 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:45:34.120 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:45:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.119+0000 7f850aa00640 1 -- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84e80a2d50 msgr2=0x7f84e80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.119+0000 7f850aa00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84e80a2d50 0x7f84e80a5210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84f40bbd80 tx=0x7f84f402a400 comp rx=0 tx=0).stop 2024-09-10T07:45:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.119+0000 7f850aa00640 1 -- 172.21.15.46:0/4292297624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504103ff0 msgr2=0x7f850419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.119+0000 7f850aa00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504103ff0 0x7f850419a610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f84f4004770 tx=0x7f84f4003c10 comp rx=0 tx=0).stop 2024-09-10T07:45:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.120+0000 7f850aa00640 1 -- 172.21.15.46:0/4292297624 shutdown_connections 2024-09-10T07:45:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.120+0000 7f850aa00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8504109120 0x7f850419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.120+0000 7f850aa00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84e80a2d50 0x7f84e80a5210 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.120+0000 7f850aa00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85041049a0 0x7f850419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.120+0000 7f850aa00640 1 --2- 172.21.15.46:0/4292297624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504103ff0 0x7f850419a610 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.120+0000 7f850aa00640 1 -- 172.21.15.46:0/4292297624 >> 172.21.15.46:0/4292297624 conn(0x7f85040ffab0 msgr2=0x7f8504100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.120+0000 7f850aa00640 1 -- 172.21.15.46:0/4292297624 shutdown_connections 2024-09-10T07:45:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:34.120+0000 7f850aa00640 1 -- 172.21.15.46:0/4292297624 wait complete. 2024-09-10T07:45:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:33 smithi181 ceph-mon[35397]: pgmap v667: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-09-10T07:45:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:33 smithi063 ceph-mon[35020]: pgmap v667: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-09-10T07:45:35.301 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.302 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.303 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:34 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:45:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:45:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:45:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:45:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:45:35.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:45:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:45:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:45:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:34 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:45:35.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:45:35.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:45:35.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:34 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 495.0 s, 539.8 tps, lat 181.830 ms stddev 261.957 2024-09-10T07:45:35.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:45:36.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:35 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:36.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:35 smithi046 ceph-mon[30705]: pgmap v668: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 516 op/s 2024-09-10T07:45:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:35 smithi181 ceph-mon[35397]: pgmap v668: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 516 op/s 2024-09-10T07:45:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:45:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:35 smithi063 ceph-mon[35020]: pgmap v668: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 516 op/s 2024-09-10T07:45:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:38.282 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.6 2024-09-10T07:45:38.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:45:38.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:37 smithi046 ceph-mon[30705]: pgmap v669: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-09-10T07:45:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:37 smithi181 ceph-mon[35397]: pgmap v669: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-09-10T07:45:38.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:37 smithi063 ceph-mon[35020]: pgmap v669: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-09-10T07:45:38.776 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:45:38.776 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:45:38.776 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:45:38.776 INFO:teuthology.orchestra.run.smithi046.stdout: "16750b3d-056f-44ef-822f-c5f1782530c6": { 2024-09-10T07:45:38.776 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:45:38.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "16750b3d-056f-44ef-822f-c5f1782530c6", 2024-09-10T07:45:38.776 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:45:38.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:38.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:38.777 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:45:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.396+0000 7fe8b8600640 1 -- 172.21.15.46:0/1415539645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8b010d520 msgr2=0x7fe8b01150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.396+0000 7fe8b8600640 1 --2- 172.21.15.46:0/1415539645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8b010d520 0x7fe8b01150a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009860 tx=0x7fe89c02a340 comp rx=0 tx=0).stop 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.397+0000 7fe8b8600640 1 -- 172.21.15.46:0/1415539645 shutdown_connections 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.397+0000 7fe8b8600640 1 --2- 172.21.15.46:0/1415539645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8b010d520 0x7fe8b01150a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.397+0000 7fe8b8600640 1 --2- 172.21.15.46:0/1415539645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b0108da0 0x7fe8b010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.397+0000 7fe8b8600640 1 --2- 172.21.15.46:0/1415539645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b01083f0 0x7fe8b01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.397+0000 7fe8b8600640 1 -- 172.21.15.46:0/1415539645 >> 172.21.15.46:0/1415539645 conn(0x7fe8b0077af0 msgr2=0x7fe8b0079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.398+0000 7fe8b8600640 1 -- 172.21.15.46:0/1415539645 shutdown_connections 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.398+0000 7fe8b8600640 1 -- 172.21.15.46:0/1415539645 wait complete. 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.401+0000 7fe8b8600640 1 Processor -- start 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.402+0000 7fe8b8600640 1 -- start start 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.403+0000 7fe8b8600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8b01083f0 0x7fe8b00826b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.403+0000 7fe8b8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b0108da0 0x7fe8b0082bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.404+0000 7fe8b8600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b010d520 0x7fe8b007b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.404+0000 7fe8b8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01185a0 con 0x7fe8b0108da0 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.404+0000 7fe8b8600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0118420 con 0x7fe8b01083f0 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.404+0000 7fe8b8600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0118720 con 0x7fe8b010d520 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.405+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b010d520 0x7fe8b007b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.405+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b0108da0 0x7fe8b0082bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.405+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8b01083f0 0x7fe8b00826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.408+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b010d520 0x7fe8b007b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43738/0 (socket says 172.21.15.46:43738) 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.408+0000 7fe8b6600640 1 -- 172.21.15.46:0/61970608 learned_addr learned my addr 172.21.15.46:0/61970608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.408+0000 7fe8b6600640 1 -- 172.21.15.46:0/61970608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8b01083f0 msgr2=0x7fe8b00826b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.408+0000 7fe8b6600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8b01083f0 0x7fe8b00826b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.408+0000 7fe8b6600640 1 -- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b0108da0 msgr2=0x7fe8b0082bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.408+0000 7fe8b6600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b0108da0 0x7fe8b0082bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.408+0000 7fe8b6600640 1 -- 172.21.15.46:0/61970608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b007c170 con 0x7fe8b010d520 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.409+0000 7fe8b6600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b010d520 0x7fe8b007b860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe89c02a930 tx=0x7fe89c033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.409+0000 7fe8ae000640 1 -- 172.21.15.46:0/61970608 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c037040 con 0x7fe8b010d520 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.410+0000 7fe8b8600640 1 -- 172.21.15.46:0/61970608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b007c3a0 con 0x7fe8b010d520 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.410+0000 7fe8b8600640 1 -- 172.21.15.46:0/61970608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b00803b0 con 0x7fe8b010d520 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.411+0000 7fe8ae000640 1 -- 172.21.15.46:0/61970608 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe89c004250 con 0x7fe8b010d520 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.411+0000 7fe8ae000640 1 -- 172.21.15.46:0/61970608 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0046c0 con 0x7fe8b010d520 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.416+0000 7fe8bb090740 1 Processor -- start 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.416+0000 7fe8ae000640 1 -- 172.21.15.46:0/61970608 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe89c030550 con 0x7fe8b010d520 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.417+0000 7fe8bb090740 1 -- start start 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.418+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b1e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.418+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e44b0 0x55846b1ee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.418+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 0x55846b1f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.418+0000 7fe8bb090740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55846b14a3e0 con 0x55846b1e3b90 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.418+0000 7fe8bb090740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55846b0f1d80 con 0x55846b1eed70 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.418+0000 7fe8bb090740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55846b1463a0 con 0x55846b1e44b0 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.421+0000 7fe8ae000640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8980a2c00 0x7fe8980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.421+0000 7fe8ae000640 1 -- 172.21.15.46:0/61970608 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe89c0ef1c0 con 0x7fe8b010d520 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.422+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e44b0 0x55846b1ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.422+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e44b0 0x55846b1ee830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43748/0 (socket says 172.21.15.46:43748) 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.422+0000 7fe8b5c00640 1 -- 172.21.15.46:0/2113196680 learned_addr learned my addr 172.21.15.46:0/2113196680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.422+0000 7fe8b5200640 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b1e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.422+0000 7fe8b6600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8980a2c00 0x7fe8980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.422+0000 7fe8b5200640 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 0x55846b1f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.423+0000 7fe8b5c00640 1 -- 172.21.15.46:0/2113196680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 msgr2=0x55846b1f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.423+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 0x55846b1f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.423+0000 7fe8b5c00640 1 -- 172.21.15.46:0/2113196680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 msgr2=0x55846b1e3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.423+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b1e3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.423+0000 7fe8b5c00640 1 -- 172.21.15.46:0/2113196680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846b110ae0 con 0x55846b1e44b0 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.423+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e44b0 0x55846b1ee830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8007ed0 tx=0x7fe8a800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8d9137b23c0121 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.425+0000 7fe8a2a00640 1 -- 172.21.15.46:0/2113196680 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8014070 con 0x55846b1e44b0 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.426+0000 7fe8a2a00640 1 -- 172.21.15.46:0/2113196680 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a8004d20 con 0x55846b1e44b0 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.426+0000 7fe8a2a00640 1 -- 172.21.15.46:0/2113196680 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8005550 con 0x55846b1e44b0 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.428+0000 7fe8b6600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8980a2c00 0x7fe8980a50c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0bb630 tx=0x7fe89c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.430+0000 7fe8bb090740 1 -- 172.21.15.46:0/2113196680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e44b0 msgr2=0x55846b1ee830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.430+0000 7fe8bb090740 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e44b0 0x55846b1ee830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8007ed0 tx=0x7fe8a800e720 comp rx=0 tx=0).stop 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.431+0000 7fe8bb090740 1 -- 172.21.15.46:0/2113196680 shutdown_connections 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.431+0000 7fe8bb090740 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 0x55846b1f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.431+0000 7fe8bb090740 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e44b0 0x55846b1ee830 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.431+0000 7fe8bb090740 1 --2- 172.21.15.46:0/2113196680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b1e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.431+0000 7fe8bb090740 1 -- 172.21.15.46:0/2113196680 >> 172.21.15.46:0/2113196680 conn(0x55846b1e0120 msgr2=0x55846b1e0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.432+0000 7fe8bb090740 1 -- 172.21.15.46:0/2113196680 shutdown_connections 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.432+0000 7fe8bb090740 1 -- 172.21.15.46:0/2113196680 wait complete. 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.434+0000 7fe8bb090740 1 -- start start 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.434+0000 7fe8bb090740 1 Processor -- start 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.436+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e3b90 0x55846b1ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.436+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.436+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 0x55846b1ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.436+0000 7fe8bb090740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55846b1f19e0 con 0x55846b1e44b0 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.436+0000 7fe8bb090740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55846b1050f0 con 0x55846b1eed70 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.436+0000 7fe8bb090740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55846b1f1b60 con 0x55846b1e3b90 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.436+0000 7fe8bb090740 10 client.0 _ll_register_callbacks cb 0x55846b0f69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.437+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e3b90 0x55846b1ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.437+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e3b90 0x55846b1ebc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43756/0 (socket says 172.21.15.46:43756) 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.437+0000 7fe8b5200640 1 -- 172.21.15.46:0/3643896022 learned_addr learned my addr 172.21.15.46:0/3643896022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.437+0000 7fe8b5200640 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 0x55846b1ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.437+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.438+0000 7fe8b5200640 1 -- 172.21.15.46:0/3643896022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 msgr2=0x55846b1ec6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.438+0000 7fe8b5200640 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 0x55846b1ec6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.439+0000 7fe8b5200640 1 -- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 msgr2=0x55846b1ec1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.439+0000 7fe8b5200640 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1ec1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.439+0000 7fe8b5200640 1 -- 172.21.15.46:0/3643896022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846aebb3c0 con 0x55846b1e3b90 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.439+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1ec1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.440+0000 7fe8b5200640 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e3b90 0x55846b1ebc60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005fd0 tx=0x7fe8a4014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ed5ca1410479ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.440+0000 7fe897400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:45:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.441+0000 7fe897400640 1 -- 172.21.15.46:0/3643896022 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4010c60 con 0x55846b1e3b90 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.441+0000 7fe897400640 1 -- 172.21.15.46:0/3643896022 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe8a4009040 con 0x55846b1e3b90 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.441+0000 7fe8bb090740 1 -- 172.21.15.46:0/3643896022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a8004d20 con 0x55846b1e3b90 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.442+0000 7fe897400640 1 -- 172.21.15.46:0/3643896022 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe8a4010e00 con 0x55846b1e3b90 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.443+0000 7fe8bb090740 10 client.27604 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.443+0000 7fe8bb090740 1 -- 172.21.15.46:0/3643896022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55846b1fd7b0 con 0x55846b1e3b90 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.443+0000 7fe897400640 1 -- 172.21.15.46:0/3643896022 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a8004d20 con 0x55846b1e3b90 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.445+0000 7fe897400640 1 -- 172.21.15.46:0/3643896022 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7fe8a4021680 con 0x55846b1e3b90 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.445+0000 7fe8bb090740 10 client.27604 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.445+0000 7fe8bb090740 10 client.27604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.445+0000 7fe8bb090740 20 client.27604 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.445+0000 7fe8bb090740 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55846b1fecc0 0x55846b2010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.445+0000 7fe8bb090740 4 client.27604 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.445+0000 7fe8bb090740 1 -- 172.21.15.46:0/3643896022 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846b157950 con 0x55846b1fecc0 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.446+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55846b1fecc0 0x55846b2010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.447+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55846b1fecc0 0x55846b2010d0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a688b4be180430ef server_cookie=688eded655559f5a in_seq=0 out_seq=0 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.447+0000 7fe897400640 10 client.27604 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.450+0000 7fe897400640 1 -- 172.21.15.46:0/3643896022 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55846b157950 con 0x55846b1fecc0 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.450+0000 7fe897400640 10 client.27604 handle_command_reply: tid=0 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.450+0000 7fe8bb090740 1 client.27604 shutdown 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 -- 172.21.15.46:0/3643896022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e3b90 msgr2=0x55846b1ebc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e3b90 0x55846b1ebc60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005fd0 tx=0x7fe8a4014040 comp rx=0 tx=0).stop 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 -- 172.21.15.46:0/3643896022 shutdown_connections 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55846b1fecc0 0x55846b2010d0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1eed70 0x55846b1ec6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1ec1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 --2- 172.21.15.46:0/3643896022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1e3b90 0x55846b1ebc60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 -- 172.21.15.46:0/3643896022 >> 172.21.15.46:0/3643896022 conn(0x55846b1e0120 msgr2=0x55846b1e5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe897400640 0 client.27604 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.451+0000 7fe8bb090740 1 -- 172.21.15.46:0/3643896022 shutdown_connections 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.452+0000 7fe8bb090740 1 -- 172.21.15.46:0/3643896022 wait complete. 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.452+0000 7fe8bb090740 20 client.27604 trim_cache size 0 max 16384 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.486+0000 7fe8bb090740 1 Processor -- start 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.486+0000 7fe8bb090740 1 -- start start 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.486+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b2069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.486+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e44b0 0x55846b206f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.487+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b1e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.487+0000 7fe8bb090740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe89000fb10 con 0x55846b1e3b90 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.487+0000 7fe8bb090740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe890011680 con 0x55846b1e44b0 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.487+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b2069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.487+0000 7fe8bb090740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900101e0 con 0x55846b1eed70 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.487+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b2069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43290/0 (socket says 172.21.15.46:43290) 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.487+0000 7fe8b5200640 1 -- 172.21.15.46:0/889981765 learned_addr learned my addr 172.21.15.46:0/889981765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.487+0000 7fe8b5200640 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b1e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.488+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e44b0 0x55846b206f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.488+0000 7fe8b5200640 1 -- 172.21.15.46:0/889981765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e44b0 msgr2=0x55846b206f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.488+0000 7fe8b5200640 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e44b0 0x55846b206f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.488+0000 7fe8b5200640 1 -- 172.21.15.46:0/889981765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 msgr2=0x55846b2069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.488+0000 7fe8b5200640 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b2069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.488+0000 7fe8b5200640 1 -- 172.21.15.46:0/889981765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846b1e0db0 con 0x55846b1eed70 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.488+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e44b0 0x55846b206f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.488+0000 7fe8b5200640 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b1e8ec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40096a0 tx=0x7fe8a4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b64d134e234b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.489+0000 7fe8a2000640 1 -- 172.21.15.46:0/889981765 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4021420 con 0x55846b1eed70 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.489+0000 7fe8a2000640 1 -- 172.21.15.46:0/889981765 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a40215c0 con 0x55846b1eed70 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.489+0000 7fe8a2000640 1 -- 172.21.15.46:0/889981765 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4003a10 con 0x55846b1eed70 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.490+0000 7fe8bb090740 1 -- 172.21.15.46:0/889981765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 msgr2=0x55846b1e8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.490+0000 7fe8bb090740 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b1e8ec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40096a0 tx=0x7fe8a4004730 comp rx=0 tx=0).stop 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.491+0000 7fe8bb090740 1 -- 172.21.15.46:0/889981765 shutdown_connections 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.491+0000 7fe8bb090740 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b1e8ec0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.491+0000 7fe8bb090740 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e44b0 0x55846b206f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.491+0000 7fe8bb090740 1 --2- 172.21.15.46:0/889981765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e3b90 0x55846b2069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.491+0000 7fe8bb090740 1 -- 172.21.15.46:0/889981765 >> 172.21.15.46:0/889981765 conn(0x55846b1e0120 msgr2=0x55846b1e5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.492+0000 7fe8bb090740 1 -- 172.21.15.46:0/889981765 shutdown_connections 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.492+0000 7fe8bb090740 1 -- 172.21.15.46:0/889981765 wait complete. 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.493+0000 7fe8bb090740 1 -- start start 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.493+0000 7fe8bb090740 1 Processor -- start 2024-09-10T07:45:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.495+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e3b90 0x55846b1f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.495+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.495+0000 7fe8bb090740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b262980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.495+0000 7fe8bb090740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe890010f80 con 0x55846b1e44b0 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.495+0000 7fe8bb090740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900108b0 con 0x55846b1e3b90 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.495+0000 7fe8bb090740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe89000daf0 con 0x55846b1eed70 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.497+0000 7fe8bb090740 10 client.0 _ll_register_callbacks cb 0x7fe8a4012b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.497+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.498+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e3b90 0x55846b1f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.498+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43300/0 (socket says 172.21.15.46:43300) 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.498+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e3b90 0x55846b1f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36078/0 (socket says 172.21.15.46:36078) 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.498+0000 7fe8b5c00640 1 -- 172.21.15.46:0/1625424830 learned_addr learned my addr 172.21.15.46:0/1625424830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.499+0000 7fe8b5200640 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b262980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe8b5200640 1 -- 172.21.15.46:0/1625424830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 msgr2=0x55846b262980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe8b5200640 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b262980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe8b5200640 1 -- 172.21.15.46:0/1625424830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 msgr2=0x55846b1f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe8b5200640 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe8b5200640 1 -- 172.21.15.46:0/1625424830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846b262f70 con 0x55846b1e3b90 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe8b5200640 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e3b90 0x55846b1f33e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005fd0 tx=0x7fe8a40035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4a49adf12f1e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe897400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.500+0000 7fe897400640 1 -- 172.21.15.46:0/1625424830 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4021420 con 0x55846b1e3b90 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.501+0000 7fe897400640 1 -- 172.21.15.46:0/1625424830 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe8a4003a50 con 0x55846b1e3b90 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.501+0000 7fe8bb090740 1 -- 172.21.15.46:0/1625424830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a40215c0 con 0x55846b1e3b90 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.501+0000 7fe897400640 1 -- 172.21.15.46:0/1625424830 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe8a4003c50 con 0x55846b1e3b90 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.501+0000 7fe8bb090740 10 client.27656 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.501+0000 7fe8bb090740 1 -- 172.21.15.46:0/1625424830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55846b26a300 con 0x55846b1e3b90 2024-09-10T07:45:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe897400640 1 -- 172.21.15.46:0/1625424830 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a40215c0 con 0x55846b1e3b90 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe897400640 1 -- 172.21.15.46:0/1625424830 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7fe8a402b540 con 0x55846b1e3b90 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe8bb090740 10 client.27656 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe8bb090740 10 client.27656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe8bb090740 20 client.27656 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe8bb090740 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55846b26a7e0 0x55846b26cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe8bb090740 4 client.27656 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe8bb090740 1 -- 172.21.15.46:0/1625424830 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55846b151e60 con 0x55846b26a7e0 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.502+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55846b26a7e0 0x55846b26cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.503+0000 7fe8b5c00640 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55846b26a7e0 0x55846b26cbd0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=603d2b3a9ecf195f server_cookie=ea990c14216f7c8e in_seq=0 out_seq=0 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.503+0000 7fe897400640 10 client.27656 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.503+0000 7fe897400640 1 -- 172.21.15.46:0/1625424830 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55846b151e60 con 0x55846b26a7e0 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.503+0000 7fe897400640 10 client.27656 handle_command_reply: tid=0 2024-09-10T07:45:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.503+0000 7fe8bb090740 1 client.27656 shutdown 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.503+0000 7fe8bb090740 1 -- 172.21.15.46:0/1625424830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e3b90 msgr2=0x55846b1f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.503+0000 7fe8bb090740 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e3b90 0x55846b1f33e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005fd0 tx=0x7fe8a40035f0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe8bb090740 1 -- 172.21.15.46:0/1625424830 shutdown_connections 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe8bb090740 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55846b26a7e0 0x55846b26cbd0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe8bb090740 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55846b1eed70 0x55846b262980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe8bb090740 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55846b1e44b0 0x55846b1f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe8bb090740 1 --2- 172.21.15.46:0/1625424830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55846b1e3b90 0x55846b1f33e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe897400640 0 client.27656 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe8bb090740 1 -- 172.21.15.46:0/1625424830 >> 172.21.15.46:0/1625424830 conn(0x55846b1e0120 msgr2=0x55846b1de9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe8bb090740 1 -- 172.21.15.46:0/1625424830 shutdown_connections 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.504+0000 7fe8bb090740 1 -- 172.21.15.46:0/1625424830 wait complete. 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8bb090740 20 client.27656 trim_cache size 0 max 16384 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 -- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8980a2c00 msgr2=0x7fe8980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8980a2c00 0x7fe8980a50c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0bb630 tx=0x7fe89c005f90 comp rx=0 tx=0).stop 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 -- 172.21.15.46:0/61970608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b010d520 msgr2=0x7fe8b007b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b010d520 0x7fe8b007b860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe89c02a930 tx=0x7fe89c033d10 comp rx=0 tx=0).stop 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 -- 172.21.15.46:0/61970608 shutdown_connections 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b010d520 0x7fe8b007b860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8980a2c00 0x7fe8980a50c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b0108da0 0x7fe8b0082bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 --2- 172.21.15.46:0/61970608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8b01083f0 0x7fe8b00826b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.505+0000 7fe8b8600640 1 -- 172.21.15.46:0/61970608 >> 172.21.15.46:0/61970608 conn(0x7fe8b0077af0 msgr2=0x7fe8b0078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.506+0000 7fe8b8600640 1 -- 172.21.15.46:0/61970608 shutdown_connections 2024-09-10T07:45:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:37.506+0000 7fe8b8600640 1 -- 172.21.15.46:0/61970608 wait complete. 2024-09-10T07:45:40.000 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:40 smithi046 ceph-mon[30705]: pgmap v670: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 4.8 MiB/s wr, 590 op/s 2024-09-10T07:45:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:40 smithi181 ceph-mon[35397]: pgmap v670: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 4.8 MiB/s wr, 590 op/s 2024-09-10T07:45:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:40 smithi063 ceph-mon[35020]: pgmap v670: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 4.8 MiB/s wr, 590 op/s 2024-09-10T07:45:40.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 500.0 s, 537.2 tps, lat 185.223 ms stddev 223.737 2024-09-10T07:45:40.825 INFO:tasks.cephfs.filesystem:scrub status for tag:16750b3d-056f-44ef-822f-c5f1782530c6 - {'path': '/', 'tag': '16750b3d-056f-44ef-822f-c5f1782530c6', 'options': 'recursive,force'} 2024-09-10T07:45:40.862 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 -- 172.21.15.46:0/3510480960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 msgr2=0x7f592c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 --2- 172.21.15.46:0/3510480960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 0x7f592c100b60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f591c009900 tx=0x7f591c02a400 comp rx=0 tx=0).stop 2024-09-10T07:45:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 -- 172.21.15.46:0/3510480960 shutdown_connections 2024-09-10T07:45:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 --2- 172.21.15.46:0/3510480960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c10e330 0x7f592c110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 --2- 172.21.15.46:0/3510480960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f592c101130 0x7f592c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 --2- 172.21.15.46:0/3510480960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 0x7f592c100b60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 -- 172.21.15.46:0/3510480960 >> 172.21.15.46:0/3510480960 conn(0x7f592c0fc610 msgr2=0x7f592c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 -- 172.21.15.46:0/3510480960 shutdown_connections 2024-09-10T07:45:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.643+0000 7f5931c00640 1 -- 172.21.15.46:0/3510480960 wait complete. 2024-09-10T07:45:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.644+0000 7f5931c00640 1 Processor -- start 2024-09-10T07:45:41.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f5931c00640 1 -- start start 2024-09-10T07:45:41.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f5931c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 0x7f592c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:41.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f5931c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f592c101130 0x7f592c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:41.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f5931c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c10e330 0x7f592c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:41.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f5931c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f592c114030 con 0x7f592c100780 2024-09-10T07:45:41.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f5931c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f592c113eb0 con 0x7f592c10e330 2024-09-10T07:45:41.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f5931c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f592c1141b0 con 0x7f592c101130 2024-09-10T07:45:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f592be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 0x7f592c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f592be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 0x7f592c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43332/0 (socket says 172.21.15.46:43332) 2024-09-10T07:45:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.645+0000 7f592be00640 1 -- 172.21.15.46:0/3379094693 learned_addr learned my addr 172.21.15.46:0/3379094693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.646+0000 7f592b400640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f592c101130 0x7f592c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.646+0000 7f5931200640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c10e330 0x7f592c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.646+0000 7f592b400640 1 -- 172.21.15.46:0/3379094693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c10e330 msgr2=0x7f592c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.646+0000 7f592b400640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c10e330 0x7f592c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.646+0000 7f592b400640 1 -- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 msgr2=0x7f592c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.646+0000 7f592b400640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 0x7f592c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.646+0000 7f592b400640 1 -- 172.21.15.46:0/3379094693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c19f460 con 0x7f592c101130 2024-09-10T07:45:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.646+0000 7f592be00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 0x7f592c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.647+0000 7f592b400640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f592c101130 0x7f592c19a9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5918005fd0 tx=0x7f591800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.647+0000 7f5928c00640 1 -- 172.21.15.46:0/3379094693 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918009c30 con 0x7f592c101130 2024-09-10T07:45:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.648+0000 7f5928c00640 1 -- 172.21.15.46:0/3379094693 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5918009dd0 con 0x7f592c101130 2024-09-10T07:45:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.648+0000 7f5928c00640 1 -- 172.21.15.46:0/3379094693 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918010460 con 0x7f592c101130 2024-09-10T07:45:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.648+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592c19f750 con 0x7f592c101130 2024-09-10T07:45:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.648+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592c1a7030 con 0x7f592c101130 2024-09-10T07:45:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.649+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592c100b60 con 0x7f592c101130 2024-09-10T07:45:41.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.652+0000 7f5928c00640 1 -- 172.21.15.46:0/3379094693 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5918003ef0 con 0x7f592c101130 2024-09-10T07:45:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.654+0000 7f5928c00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59100a2c00 0x7f59100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.654+0000 7f5928c00640 1 -- 172.21.15.46:0/3379094693 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f59180cef00 con 0x7f592c101130 2024-09-10T07:45:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.654+0000 7f592be00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59100a2c00 0x7f59100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.655+0000 7f592be00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59100a2c00 0x7f59100a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f591c0096c0 tx=0x7f591c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:41.655+0000 7f5928c00640 1 -- 172.21.15.46:0/3379094693 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5918017020 con 0x7f592c101130 2024-09-10T07:45:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.078+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f592c19b810 con 0x7f592c101130 2024-09-10T07:45:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.080+0000 7f5928c00640 1 -- 172.21.15.46:0/3379094693 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11973 (secure 0 0 0) 0x7f592c19b810 con 0x7f592c101130 2024-09-10T07:45:42.081 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:45:42.081 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":47,"btime":"2024-09-10T07:45:15:869438+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:45:15.518390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:45:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 47 2024-09-10T07:45:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59100a2c00 msgr2=0x7f59100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59100a2c00 0x7f59100a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f591c0096c0 tx=0x7f591c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:45:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f592c101130 msgr2=0x7f592c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f592c101130 0x7f592c19a9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5918005fd0 tx=0x7f591800dde0 comp rx=0 tx=0).stop 2024-09-10T07:45:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 shutdown_connections 2024-09-10T07:45:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c10e330 0x7f592c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f592c101130 0x7f592c19a9f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59100a2c00 0x7f59100a50c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 --2- 172.21.15.46:0/3379094693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592c100780 0x7f592c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:42.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.085+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 >> 172.21.15.46:0/3379094693 conn(0x7f592c0fc610 msgr2=0x7f592c0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:42.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.086+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 shutdown_connections 2024-09-10T07:45:42.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:42.086+0000 7f5931c00640 1 -- 172.21.15.46:0/3379094693 wait complete. 2024-09-10T07:45:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:42 smithi181 ceph-mon[35397]: pgmap v671: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-09-10T07:45:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:42 smithi063 ceph-mon[35020]: pgmap v671: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-09-10T07:45:42.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:42 smithi046 ceph-mon[30705]: pgmap v671: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-09-10T07:45:43.151 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0' 2024-09-10T07:45:43.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0 2024-09-10T07:45:43.294 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:43 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3379094693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:43.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:43 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3379094693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:43.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:43 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3379094693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:45:43.850 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:45:44.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:44 smithi046 ceph-mon[30705]: pgmap v672: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.1 MiB/s wr, 646 op/s 2024-09-10T07:45:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:44 smithi181 ceph-mon[35397]: pgmap v672: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.1 MiB/s wr, 646 op/s 2024-09-10T07:45:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:44 smithi063 ceph-mon[35020]: pgmap v672: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.1 MiB/s wr, 646 op/s 2024-09-10T07:45:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 -- 172.21.15.46:0/468818424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 msgr2=0x7f6b08076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 --2- 172.21.15.46:0/468818424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 0x7f6b08076b00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009900 tx=0x7f6af802a400 comp rx=0 tx=0).stop 2024-09-10T07:45:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 -- 172.21.15.46:0/468818424 shutdown_connections 2024-09-10T07:45:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 --2- 172.21.15.46:0/468818424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b08112a30 0x7f6b08114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 --2- 172.21.15.46:0/468818424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b08077040 0x7f6b081124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 --2- 172.21.15.46:0/468818424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 0x7f6b08076b00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 -- 172.21.15.46:0/468818424 >> 172.21.15.46:0/468818424 conn(0x7f6b080ffa90 msgr2=0x7f6b08101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 -- 172.21.15.46:0/468818424 shutdown_connections 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.561+0000 7f6b0e400640 1 -- 172.21.15.46:0/468818424 wait complete. 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.562+0000 7f6b0e400640 1 Processor -- start 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.562+0000 7f6b0e400640 1 -- start start 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.562+0000 7f6b0e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 0x7f6b081a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b0e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b08077040 0x7f6b081a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b0e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b08112a30 0x7f6b0819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b0e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081185a0 con 0x7f6b08076720 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b0e400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08118420 con 0x7f6b08112a30 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b0e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08118720 con 0x7f6b08077040 2024-09-10T07:45:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 0x7f6b081a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 0x7f6b081a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43358/0 (socket says 172.21.15.46:43358) 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07400640 1 -- 172.21.15.46:0/3354116877 learned_addr learned my addr 172.21.15.46:0/3354116877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07e00640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b08112a30 0x7f6b0819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b06a00640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b08077040 0x7f6b081a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07400640 1 -- 172.21.15.46:0/3354116877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b08077040 msgr2=0x7f6b081a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b08077040 0x7f6b081a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07400640 1 -- 172.21.15.46:0/3354116877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b08112a30 msgr2=0x7f6b0819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b08112a30 0x7f6b0819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.563+0000 7f6b07400640 1 -- 172.21.15.46:0/3354116877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0819efe0 con 0x7f6b08076720 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.564+0000 7f6b07400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 0x7f6b081a46f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6af8006010 tx=0x7f6af8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.564+0000 7f6affe00640 1 -- 172.21.15.46:0/3354116877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af803f030 con 0x7f6b08076720 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.564+0000 7f6b0e400640 1 -- 172.21.15.46:0/3354116877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0819f270 con 0x7f6b08076720 2024-09-10T07:45:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.564+0000 7f6affe00640 1 -- 172.21.15.46:0/3354116877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af8004440 con 0x7f6b08076720 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.564+0000 7f6affe00640 1 -- 172.21.15.46:0/3354116877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8035070 con 0x7f6b08076720 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.564+0000 7f6b0e400640 1 -- 172.21.15.46:0/3354116877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b081ab5c0 con 0x7f6b08076720 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6b10e78740 1 Processor -- start 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6affe00640 1 -- 172.21.15.46:0/3354116877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6af8042030 con 0x7f6b08076720 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6b10e78740 1 -- start start 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd4a3b0 0x55d1edd4a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4acd0 0x55d1edd55050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd55590 0x55d1edd57980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6b10e78740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1edcb0530 con 0x55d1edd4acd0 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6b10e78740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d1edc57860 con 0x55d1edd55590 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.567+0000 7f6b10e78740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d1edcac4f0 con 0x55d1edd4a3b0 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd4a3b0 0x55d1edd4a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4acd0 0x55d1edd55050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd4a3b0 0x55d1edd4a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43814/0 (socket says 172.21.15.46:43814) 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07e00640 1 -- 172.21.15.46:0/1175437189 learned_addr learned my addr 172.21.15.46:0/1175437189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b06a00640 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd55590 0x55d1edd57980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07e00640 1 -- 172.21.15.46:0/1175437189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd55590 msgr2=0x55d1edd57980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07e00640 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd55590 0x55d1edd57980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07e00640 1 -- 172.21.15.46:0/1175437189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4acd0 msgr2=0x55d1edd55050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07e00640 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4acd0 0x55d1edd55050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07e00640 1 -- 172.21.15.46:0/1175437189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1eda21b90 con 0x55d1edd4a3b0 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.568+0000 7f6b07400640 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4acd0 0x55d1edd55050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6b07e00640 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd4a3b0 0x55d1edd4a790 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009f30 tx=0x7f6b000118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1556e730a36cfd0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6afcc00640 1 -- 172.21.15.46:0/1175437189 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00004d20 con 0x55d1edd4a3b0 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6afcc00640 1 -- 172.21.15.46:0/1175437189 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b00004050 con 0x55d1edd4a3b0 2024-09-10T07:45:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6affe00640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6aec0a2c00 0x7f6aec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6afcc00640 1 -- 172.21.15.46:0/1175437189 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00005220 con 0x55d1edd4a3b0 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6affe00640 1 -- 172.21.15.46:0/3354116877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6af80ef790 con 0x7f6b08076720 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6b07400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6aec0a2c00 0x7f6aec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6b10e78740 1 -- 172.21.15.46:0/1175437189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd4a3b0 msgr2=0x55d1edd4a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.569+0000 7f6b10e78740 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd4a3b0 0x55d1edd4a790 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009f30 tx=0x7f6b000118d0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 -- 172.21.15.46:0/1175437189 shutdown_connections 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd55590 0x55d1edd57980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4acd0 0x55d1edd55050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 --2- 172.21.15.46:0/1175437189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd4a3b0 0x55d1edd4a790 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 -- 172.21.15.46:0/1175437189 >> 172.21.15.46:0/1175437189 conn(0x55d1edd46940 msgr2=0x55d1edd46d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 -- 172.21.15.46:0/1175437189 shutdown_connections 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 -- 172.21.15.46:0/1175437189 wait complete. 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 -- start start 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.570+0000 7f6b10e78740 1 Processor -- start 2024-09-10T07:45:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b07400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6aec0a2c00 0x7f6aec0a50c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6af8032c00 tx=0x7f6af802a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd52630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd52b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b10e78740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1edd57fb0 con 0x55d1edd4a3b0 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b10e78740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d1edc6ae60 con 0x55d1edd4acd0 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b10e78740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d1edd58130 con 0x55d1edd55590 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b10e78740 10 client.0 _ll_register_callbacks cb 0x55d1edc6fcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd52630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b06a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd52b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd52630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43382/0 (socket says 172.21.15.46:43382) 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b07e00640 1 -- 172.21.15.46:0/2965110644 learned_addr learned my addr 172.21.15.46:0/2965110644 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.571+0000 7f6b07e00640 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6b06a00640 1 -- 172.21.15.46:0/2965110644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 msgr2=0x55d1edd530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6b06a00640 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6b06a00640 1 -- 172.21.15.46:0/2965110644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 msgr2=0x55d1edd52630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6b06a00640 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd52630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6b06a00640 1 -- 172.21.15.46:0/2965110644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1edc76990 con 0x55d1edd4acd0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6b07e00640 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6b06a00640 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd52b70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6b081a00a0 tx=0x7f6af4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc4ebffefb066f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6af1600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6af1600640 1 -- 172.21.15.46:0/2965110644 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af400f920 con 0x55d1edd4acd0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6b10e78740 1 -- 172.21.15.46:0/2965110644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b00004050 con 0x55d1edd4acd0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.572+0000 7f6af1600640 1 -- 172.21.15.46:0/2965110644 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6af400fac0 con 0x55d1edd4acd0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.573+0000 7f6af1600640 1 -- 172.21.15.46:0/2965110644 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6af400fcc0 con 0x55d1edd4acd0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.573+0000 7f6b10e78740 10 client.27674 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.573+0000 7f6b10e78740 1 -- 172.21.15.46:0/2965110644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d1edd63de0 con 0x55d1edd4acd0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.573+0000 7f6af1600640 1 -- 172.21.15.46:0/2965110644 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b00004050 con 0x55d1edd4acd0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.574+0000 7f6af1600640 1 -- 172.21.15.46:0/2965110644 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f6af4010430 con 0x55d1edd4acd0 2024-09-10T07:45:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.574+0000 7f6b10e78740 10 client.27674 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.574+0000 7f6b10e78740 10 client.27674 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.574+0000 7f6b10e78740 20 client.27674 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.574+0000 7f6b10e78740 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1edd651f0 0x55d1edd67620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.574+0000 7f6b10e78740 4 client.27674 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.574+0000 7f6b10e78740 1 -- 172.21.15.46:0/2965110644 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1edc53310 con 0x55d1edd651f0 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.574+0000 7f6b07e00640 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1edd651f0 0x55d1edd67620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.575+0000 7f6af1600640 10 client.27674 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.575+0000 7f6b07e00640 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1edd651f0 0x55d1edd67620 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42326f01f163256e server_cookie=975a5047f732739b in_seq=0 out_seq=0 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.575+0000 7f6af1600640 1 -- 172.21.15.46:0/2965110644 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1edc53310 con 0x55d1edd651f0 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.575+0000 7f6af1600640 10 client.27674 handle_command_reply: tid=0 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.575+0000 7f6b10e78740 1 client.27674 shutdown 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 -- 172.21.15.46:0/2965110644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 msgr2=0x55d1edd52b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd52b70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6b081a00a0 tx=0x7f6af4012b60 comp rx=0 tx=0).stop 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 -- 172.21.15.46:0/2965110644 shutdown_connections 2024-09-10T07:45:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1edd651f0 0x55d1edd67620 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd52b70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6af1600640 0 client.27674 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 --2- 172.21.15.46:0/2965110644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd52630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 -- 172.21.15.46:0/2965110644 >> 172.21.15.46:0/2965110644 conn(0x55d1edd46940 msgr2=0x55d1edd45c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 -- 172.21.15.46:0/2965110644 shutdown_connections 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 1 -- 172.21.15.46:0/2965110644 wait complete. 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.576+0000 7f6b10e78740 20 client.27674 trim_cache size 0 max 16384 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.594+0000 7f6b10e78740 1 Processor -- start 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.594+0000 7f6b10e78740 1 -- start start 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.594+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd6cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.594+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.594+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd4f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.594+0000 7f6b10e78740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b5310 con 0x55d1edd4a3b0 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.594+0000 7f6b10e78740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b6e80 con 0x55d1edd4acd0 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.594+0000 7f6b10e78740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b59e0 con 0x55d1edd55590 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd6cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd6cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43394/0 (socket says 172.21.15.46:43394) 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b07e00640 1 -- 172.21.15.46:0/363918238 learned_addr learned my addr 172.21.15.46:0/363918238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b06a00640 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b06a00640 1 -- 172.21.15.46:0/363918238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 msgr2=0x55d1edd4f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b06a00640 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd4f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b06a00640 1 -- 172.21.15.46:0/363918238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 msgr2=0x55d1edd6cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b06a00640 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd6cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b06a00640 1 -- 172.21.15.46:0/363918238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1edd479f0 con 0x55d1edd4acd0 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b07e00640 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.595+0000 7f6b06a00640 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd6d430 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55d1edd53be0 tx=0x7f6af4009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5e715a31513529 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6af3e00640 1 -- 172.21.15.46:0/363918238 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4000e60 con 0x55d1edd4acd0 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6af3e00640 1 -- 172.21.15.46:0/363918238 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af400fe60 con 0x55d1edd4acd0 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6af3e00640 1 -- 172.21.15.46:0/363918238 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af40035f0 con 0x55d1edd4acd0 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 -- 172.21.15.46:0/363918238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 msgr2=0x55d1edd6d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd6d430 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55d1edd53be0 tx=0x7f6af4009850 comp rx=0 tx=0).stop 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 -- 172.21.15.46:0/363918238 shutdown_connections 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd4f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd6d430 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 --2- 172.21.15.46:0/363918238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 -- 172.21.15.46:0/363918238 >> 172.21.15.46:0/363918238 conn(0x55d1edd46940 msgr2=0x55d1edd45c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 -- 172.21.15.46:0/363918238 shutdown_connections 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.596+0000 7f6b10e78740 1 -- 172.21.15.46:0/363918238 wait complete. 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 1 -- start start 2024-09-10T07:45:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 1 Processor -- start 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd5d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd5dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b6780 con 0x55d1edd4a3b0 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b60b0 con 0x55d1edd4acd0 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b32f0 con 0x55d1edd55590 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.597+0000 7f6b10e78740 10 client.0 _ll_register_callbacks cb 0x7f6af4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd5d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd5dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd5dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:36154/0 (socket says 172.21.15.46:36154) 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 -- 172.21.15.46:0/3647741872 learned_addr learned my addr 172.21.15.46:0/3647741872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b07e00640 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 -- 172.21.15.46:0/3647741872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 msgr2=0x55d1edd5a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd5a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 -- 172.21.15.46:0/3647741872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 msgr2=0x55d1edd5d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd5d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 -- 172.21.15.46:0/3647741872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1edd5ac70 con 0x55d1edd4acd0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b07e00640 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd5a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6b06a00640 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd5dba0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009900 tx=0x7f6af4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d78520716c18e2de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.598+0000 7f6af1600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6af1600640 1 -- 172.21.15.46:0/3647741872 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4000e40 con 0x55d1edd4acd0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6af1600640 1 -- 172.21.15.46:0/3647741872 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6af4018070 con 0x55d1edd4acd0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 1 -- 172.21.15.46:0/3647741872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af400fe60 con 0x55d1edd4acd0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6af1600640 1 -- 172.21.15.46:0/3647741872 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6af400f920 con 0x55d1edd4acd0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 10 client.27680 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 1 -- 172.21.15.46:0/3647741872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d1eddd0a90 con 0x55d1edd4acd0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6af1600640 1 -- 172.21.15.46:0/3647741872 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af400fe60 con 0x55d1edd4acd0 2024-09-10T07:45:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6af1600640 1 -- 172.21.15.46:0/3647741872 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7f6af402a4c0 con 0x55d1edd4acd0 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 10 client.27680 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 10 client.27680 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 20 client.27680 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1eddd1170 0x55d1eddd3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 4 client.27680 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}] 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.599+0000 7f6b10e78740 1 -- 172.21.15.46:0/3647741872 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}) -- 0x55d1edcbdc60 con 0x55d1eddd1170 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.600+0000 7f6b07e00640 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1eddd1170 0x55d1eddd3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.600+0000 7f6af1600640 10 client.27680 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.600+0000 7f6b07e00640 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1eddd1170 0x55d1eddd3590 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0bfc227b7fee57 server_cookie=a1d635b4e56996de in_seq=0 out_seq=0 2024-09-10T07:45:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6af1600640 1 -- 172.21.15.46:0/3647741872 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d1edcbdc60 con 0x55d1eddd1170 2024-09-10T07:45:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6af1600640 10 client.27680 handle_command_reply: tid=0 2024-09-10T07:45:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 client.27680 shutdown 2024-09-10T07:45:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 -- 172.21.15.46:0/3647741872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 msgr2=0x55d1edd5dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd5dba0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009900 tx=0x7f6af4005270 comp rx=0 tx=0).stop 2024-09-10T07:45:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 -- 172.21.15.46:0/3647741872 shutdown_connections 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1eddd1170 0x55d1eddd3590 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1edd55590 0x55d1edd5a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6af1600640 0 client.27680 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1edd4acd0 0x55d1edd5dba0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 --2- 172.21.15.46:0/3647741872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1edd4a3b0 0x55d1edd5d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.883+0000 7f6b10e78740 1 -- 172.21.15.46:0/3647741872 >> 172.21.15.46:0/3647741872 conn(0x55d1edd46940 msgr2=0x55d1edd4be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.884+0000 7f6b10e78740 1 -- 172.21.15.46:0/3647741872 shutdown_connections 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.884+0000 7f6b10e78740 1 -- 172.21.15.46:0/3647741872 wait complete. 2024-09-10T07:45:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.884+0000 7f6b10e78740 20 client.27680 trim_cache size 0 max 16384 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 47, 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 30, 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "d5411a2f": { 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 30, 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:45:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 144.6, 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:45:44.887 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 -- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6aec0a2c00 msgr2=0x7f6aec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6aec0a2c00 0x7f6aec0a50c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6af8032c00 tx=0x7f6af802a400 comp rx=0 tx=0).stop 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 -- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 msgr2=0x7f6b081a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 0x7f6b081a46f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6af8006010 tx=0x7f6af8004040 comp rx=0 tx=0).stop 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 -- 172.21.15.46:0/3354116877 shutdown_connections 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b08112a30 0x7f6b0819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b08077040 0x7f6b081a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6aec0a2c00 0x7f6aec0a50c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 --2- 172.21.15.46:0/3354116877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b08076720 0x7f6b081a46f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 -- 172.21.15.46:0/3354116877 >> 172.21.15.46:0/3354116877 conn(0x7f6b080ffa90 msgr2=0x7f6b08101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 -- 172.21.15.46:0/3354116877 shutdown_connections 2024-09-10T07:45:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:45:44.885+0000 7f6b0e400640 1 -- 172.21.15.46:0/3354116877 wait complete. 2024-09-10T07:45:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 505.0 s, 418.8 tps, lat 194.982 ms stddev 207.291 2024-09-10T07:45:45.734 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd5411a2f', quiesce took 7.5 seconds. Will release after: 17.1 2024-09-10T07:45:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:46 smithi181 ceph-mon[35397]: pgmap v673: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.5 MiB/s wr, 571 op/s 2024-09-10T07:45:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:46 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:45:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:46 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:45:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:46 smithi063 ceph-mon[35020]: pgmap v673: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.5 MiB/s wr, 571 op/s 2024-09-10T07:45:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:46 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:45:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:46 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:45:46.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:46 smithi046 ceph-mon[30705]: pgmap v673: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.5 MiB/s wr, 571 op/s 2024-09-10T07:45:46.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:46 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:45:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:46 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:45:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 -- 172.21.15.46:0/552723524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10104f50 msgr2=0x7faf101053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 --2- 172.21.15.46:0/552723524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10104f50 0x7faf101053d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faefc009900 tx=0x7faefc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:45:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 -- 172.21.15.46:0/552723524 shutdown_connections 2024-09-10T07:45:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 --2- 172.21.15.46:0/552723524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf10105910 0x7faf1010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 --2- 172.21.15.46:0/552723524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10104f50 0x7faf101053d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 --2- 172.21.15.46:0/552723524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf10103d60 0x7faf10104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 -- 172.21.15.46:0/552723524 >> 172.21.15.46:0/552723524 conn(0x7faf100ff4e0 msgr2=0x7faf10101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 -- 172.21.15.46:0/552723524 shutdown_connections 2024-09-10T07:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.985+0000 7faf17e00640 1 -- 172.21.15.46:0/552723524 wait complete. 2024-09-10T07:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.986+0000 7faf17e00640 1 Processor -- start 2024-09-10T07:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.986+0000 7faf17e00640 1 -- start start 2024-09-10T07:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.986+0000 7faf17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10103d60 0x7faf10199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.986+0000 7faf17e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf10104f50 0x7faf1019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.986+0000 7faf17e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf10105910 0x7faf101a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf17e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010f9c0 con 0x7faf10103d60 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf17e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010f840 con 0x7faf10104f50 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf17e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010fb40 con 0x7faf10105910 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10103d60 0x7faf10199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf14a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf10104f50 0x7faf1019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf14a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf10104f50 0x7faf1019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56752/0 (socket says 172.21.15.46:56752) 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf14a00640 1 -- 172.21.15.46:0/1021037196 learned_addr learned my addr 172.21.15.46:0/1021037196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf15e00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf10105910 0x7faf101a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf15400640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10103d60 0x7faf10199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37106/0 (socket says 172.21.15.46:37106) 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf14a00640 1 -- 172.21.15.46:0/1021037196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf10105910 msgr2=0x7faf101a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf14a00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf10105910 0x7faf101a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf14a00640 1 -- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10103d60 msgr2=0x7faf10199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf14a00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10103d60 0x7faf10199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf14a00640 1 -- 172.21.15.46:0/1021037196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf101a1be0 con 0x7faf10104f50 2024-09-10T07:45:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.987+0000 7faf15400640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10103d60 0x7faf10199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.988+0000 7faf14a00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf10104f50 0x7faf1019a500 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faefc009230 tx=0x7faefc031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.988+0000 7faf0e000640 1 -- 172.21.15.46:0/1021037196 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc03f030 con 0x7faf10104f50 2024-09-10T07:45:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.988+0000 7faf0e000640 1 -- 172.21.15.46:0/1021037196 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faefc003ad0 con 0x7faf10104f50 2024-09-10T07:45:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.988+0000 7faf17e00640 1 -- 172.21.15.46:0/1021037196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf101a1e70 con 0x7faf10104f50 2024-09-10T07:45:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.988+0000 7faf0e000640 1 -- 172.21.15.46:0/1021037196 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc035070 con 0x7faf10104f50 2024-09-10T07:45:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.988+0000 7faf17e00640 1 -- 172.21.15.46:0/1021037196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf101a22e0 con 0x7faf10104f50 2024-09-10T07:45:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.991+0000 7faf1ae27740 1 Processor -- start 2024-09-10T07:45:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.991+0000 7faf1ae27740 1 -- start start 2024-09-10T07:45:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.991+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f644490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.991+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f649490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.991+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f649a60 0x56190f64bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.991+0000 7faf1ae27740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56190f5a9100 con 0x56190f6449d0 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.991+0000 7faf1ae27740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56190f5511b0 con 0x56190f644090 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.991+0000 7faf1ae27740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56190f5a50c0 con 0x56190f649a60 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f649490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f644490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f644490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56762/0 (socket says 172.21.15.46:56762) 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f649490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37110/0 (socket says 172.21.15.46:37110) 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15e00640 1 -- 172.21.15.46:0/3336080493 learned_addr learned my addr 172.21.15.46:0/3336080493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf0e000640 1 -- 172.21.15.46:0/1021037196 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faefc042030 con 0x7faf10104f50 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15e00640 1 -- 172.21.15.46:0/3336080493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f649a60 msgr2=0x56190f64bf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:45:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15e00640 1 --2- 172.21.15.46:0/3336080493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f649a60 0x56190f64bf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15e00640 1 -- 172.21.15.46:0/3336080493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 msgr2=0x56190f649490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15e00640 1 --2- 172.21.15.46:0/3336080493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f649490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15e00640 1 -- 172.21.15.46:0/3336080493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190f2bc210 con 0x56190f644090 2024-09-10T07:45:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.992+0000 7faf15400640 1 --2- 172.21.15.46:0/3336080493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f649490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:45:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf15e00640 1 --2- 172.21.15.46:0/3336080493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f644490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faf08010370 tx=0x7faf08010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b59de4e44bc79d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf02600640 1 -- 172.21.15.46:0/3336080493 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08016070 con 0x56190f644090 2024-09-10T07:45:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf02600640 1 -- 172.21.15.46:0/3336080493 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08004d20 con 0x56190f644090 2024-09-10T07:45:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf02600640 1 -- 172.21.15.46:0/3336080493 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08005480 con 0x56190f644090 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf1ae27740 1 -- 172.21.15.46:0/3336080493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 msgr2=0x56190f644490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf1ae27740 1 --2- 172.21.15.46:0/3336080493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f644490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faf08010370 tx=0x7faf08010830 comp rx=0 tx=0).stop 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf1ae27740 1 -- 172.21.15.46:0/3336080493 shutdown_connections 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf1ae27740 1 --2- 172.21.15.46:0/3336080493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f649a60 0x56190f64bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf1ae27740 1 --2- 172.21.15.46:0/3336080493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f649490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf1ae27740 1 --2- 172.21.15.46:0/3336080493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f644490 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf1ae27740 1 -- 172.21.15.46:0/3336080493 >> 172.21.15.46:0/3336080493 conn(0x56190f63f700 msgr2=0x56190f63fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.993+0000 7faf1ae27740 1 -- 172.21.15.46:0/3336080493 shutdown_connections 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.994+0000 7faf1ae27740 1 -- 172.21.15.46:0/3336080493 wait complete. 2024-09-10T07:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.994+0000 7faf0e000640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faef80a2bc0 0x7faef80a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.994+0000 7faf0e000640 1 -- 172.21.15.46:0/1021037196 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faefc0ef7e0 con 0x7faf10104f50 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.994+0000 7faf1ae27740 1 -- start start 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.994+0000 7faf15400640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faef80a2bc0 0x7faef80a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.994+0000 7faf1ae27740 1 Processor -- start 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f644090 0x56190f64e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf15400640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faef80a2bc0 0x7faef80a5080 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x56190f640580 tx=0x7faf04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f64e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f655890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf1ae27740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56190f64ca30 con 0x56190f6449d0 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf1ae27740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56190f532090 con 0x56190f649a60 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf1ae27740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56190f64cbb0 con 0x56190f644090 2024-09-10T07:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf1ae27740 10 client.0 _ll_register_callbacks cb 0x56190f586f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf14a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f64e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf14a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f64e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37120/0 (socket says 172.21.15.46:37120) 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf14a00640 1 -- 172.21.15.46:0/961536800 learned_addr learned my addr 172.21.15.46:0/961536800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf15400640 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f644090 0x56190f64e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.995+0000 7faf15e00640 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f655890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faf14a00640 1 -- 172.21.15.46:0/961536800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f644090 msgr2=0x56190f64e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faf14a00640 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f644090 0x56190f64e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faf14a00640 1 -- 172.21.15.46:0/961536800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 msgr2=0x56190f655890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faf14a00640 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f655890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faf14a00640 1 -- 172.21.15.46:0/961536800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190f388620 con 0x56190f6449d0 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faf14a00640 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f64e850 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faefc0314e0 tx=0x7faefc0bb790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedd4a8e69225041 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faef6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faef6a00640 1 -- 172.21.15.46:0/961536800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc049040 con 0x56190f6449d0 2024-09-10T07:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faef6a00640 1 -- 172.21.15.46:0/961536800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faefc0b83e0 con 0x56190f6449d0 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faf1ae27740 1 -- 172.21.15.46:0/961536800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf08004d20 con 0x56190f6449d0 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faef6a00640 1 -- 172.21.15.46:0/961536800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7faefc0c4930 con 0x56190f6449d0 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.996+0000 7faf1ae27740 10 client.17100 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faf1ae27740 1 -- 172.21.15.46:0/961536800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56190f658680 con 0x56190f6449d0 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faef6a00640 1 -- 172.21.15.46:0/961536800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08004d20 con 0x56190f6449d0 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faef6a00640 1 -- 172.21.15.46:0/961536800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7faefc09abe0 con 0x56190f6449d0 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faf1ae27740 10 client.17100 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faf1ae27740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faf1ae27740 20 client.17100 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faf1ae27740 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56190f659e30 0x56190f65c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faf1ae27740 4 client.17100 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faf1ae27740 1 -- 172.21.15.46:0/961536800 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56190f5b6830 con 0x56190f659e30 2024-09-10T07:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.997+0000 7faf15400640 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56190f659e30 0x56190f65c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.998+0000 7faf15400640 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56190f659e30 0x56190f65c250 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b99f49562a954219 server_cookie=ebd7e273a54c53f0 in_seq=0 out_seq=0 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.998+0000 7faef6a00640 10 client.17100 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faef6a00640 1 -- 172.21.15.46:0/961536800 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56190f5b6830 con 0x56190f659e30 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faef6a00640 10 client.17100 handle_command_reply: tid=0 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 client.17100 shutdown 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 -- 172.21.15.46:0/961536800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 msgr2=0x56190f64e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f64e850 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faefc0314e0 tx=0x7faefc0bb790 comp rx=0 tx=0).stop 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 -- 172.21.15.46:0/961536800 shutdown_connections 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56190f659e30 0x56190f65c250 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f655890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f6449d0 0x56190f64e850 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 --2- 172.21.15.46:0/961536800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f644090 0x56190f64e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faef6a00640 0 client.17100 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 -- 172.21.15.46:0/961536800 >> 172.21.15.46:0/961536800 conn(0x56190f63f700 msgr2=0x56190f640f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 -- 172.21.15.46:0/961536800 shutdown_connections 2024-09-10T07:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 1 -- 172.21.15.46:0/961536800 wait complete. 2024-09-10T07:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:46.999+0000 7faf1ae27740 20 client.17100 trim_cache size 0 max 16384 2024-09-10T07:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf1ae27740 1 Processor -- start 2024-09-10T07:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf1ae27740 1 -- start start 2024-09-10T07:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f6405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f655850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f649a60 0x56190f640af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf1ae27740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf101ac740 con 0x56190f649a60 2024-09-10T07:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf1ae27740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf101ae2b0 con 0x56190f644090 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf1ae27740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf101ace10 con 0x56190f6449d0 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f649a60 0x56190f640af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f649a60 0x56190f640af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37126/0 (socket says 172.21.15.46:37126) 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf15e00640 1 -- 172.21.15.46:0/2121513516 learned_addr learned my addr 172.21.15.46:0/2121513516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf14a00640 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f655850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.015+0000 7faf15400640 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f6405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf15e00640 1 -- 172.21.15.46:0/2121513516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 msgr2=0x56190f655850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf15e00640 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f655850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf15e00640 1 -- 172.21.15.46:0/2121513516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 msgr2=0x56190f6405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf15e00640 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f6405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf15e00640 1 -- 172.21.15.46:0/2121513516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190f64dec0 con 0x56190f649a60 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf15400640 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f6405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf15e00640 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f649a60 0x56190f640af0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faf080098b0 tx=0x7faf08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38f0a2df99236da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf01c00640 1 -- 172.21.15.46:0/2121513516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08016070 con 0x56190f649a60 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf01c00640 1 -- 172.21.15.46:0/2121513516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08009c20 con 0x56190f649a60 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.016+0000 7faf01c00640 1 -- 172.21.15.46:0/2121513516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08012af0 con 0x56190f649a60 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 -- 172.21.15.46:0/2121513516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f649a60 msgr2=0x56190f640af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f649a60 0x56190f640af0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faf080098b0 tx=0x7faf08009bb0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 -- 172.21.15.46:0/2121513516 shutdown_connections 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f649a60 0x56190f640af0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f655850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 --2- 172.21.15.46:0/2121513516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f644090 0x56190f6405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 -- 172.21.15.46:0/2121513516 >> 172.21.15.46:0/2121513516 conn(0x56190f63f700 msgr2=0x56190f645c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 -- 172.21.15.46:0/2121513516 shutdown_connections 2024-09-10T07:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 -- 172.21.15.46:0/2121513516 wait complete. 2024-09-10T07:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 -- start start 2024-09-10T07:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.017+0000 7faf1ae27740 1 Processor -- start 2024-09-10T07:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f644090 0x56190f652f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f653450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf1ae27740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f654460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf1ae27740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf101adbb0 con 0x56190f644090 2024-09-10T07:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf1ae27740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf101ad4e0 con 0x56190f649a60 2024-09-10T07:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf1ae27740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf101aa720 con 0x56190f6449d0 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf1ae27740 10 client.0 _ll_register_callbacks cb 0x7faefc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f644090 0x56190f652f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f644090 0x56190f652f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37136/0 (socket says 172.21.15.46:37136) 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15400640 1 -- 172.21.15.46:0/616514929 learned_addr learned my addr 172.21.15.46:0/616514929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf14a00640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f653450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15e00640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f654460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15400640 1 -- 172.21.15.46:0/616514929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 msgr2=0x56190f653450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15400640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f653450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15400640 1 -- 172.21.15.46:0/616514929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 msgr2=0x56190f654460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15400640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f654460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15400640 1 -- 172.21.15.46:0/616514929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190f654a70 con 0x56190f644090 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.018+0000 7faf15e00640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f654460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:45:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faf14a00640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f653450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faf15400640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f644090 0x56190f652f10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faf04009e90 tx=0x7faf040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fac2879607b9a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faef6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faef6a00640 1 -- 172.21.15.46:0/616514929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04003d80 con 0x56190f644090 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faef6a00640 1 -- 172.21.15.46:0/616514929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7faf0402b3b0 con 0x56190f644090 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faf1ae27740 1 -- 172.21.15.46:0/616514929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56190f5b0b80 con 0x56190f644090 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faef6a00640 1 -- 172.21.15.46:0/616514929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7faf0403b050 con 0x56190f644090 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faf1ae27740 10 client.17106 fetch_fsmap learned FSMap version 47 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.019+0000 7faf1ae27740 1 -- 172.21.15.46:0/616514929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56190f6c4b30 con 0x56190f644090 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faef6a00640 1 -- 172.21.15.46:0/616514929 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56190f5b0b80 con 0x56190f644090 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faef6a00640 1 -- 172.21.15.46:0/616514929 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) ==== 8374+0+0 (secure 0 0 0) 0x7faf0400f8f0 con 0x56190f644090 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faf1ae27740 10 client.17106 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faf1ae27740 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faf1ae27740 20 client.17106 populate_metadata read hostname 'smithi046' 2024-09-10T07:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faf1ae27740 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56190f6c5510 0x56190f6c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faf1ae27740 4 client.17106 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faf1ae27740 1 -- 172.21.15.46:0/616514929 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56190f52d7c0 con 0x56190f6c5510 2024-09-10T07:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faf14a00640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56190f6c5510 0x56190f6c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faef6a00640 10 client.17106 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.020+0000 7faf14a00640 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56190f6c5510 0x56190f6c7930 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297ecc0fe11771a server_cookie=ecdb9b8558140a2f in_seq=0 out_seq=0 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faef6a00640 1 -- 172.21.15.46:0/616514929 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56190f52d7c0 con 0x56190f6c5510 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faef6a00640 10 client.17106 handle_command_reply: tid=0 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 client.17106 shutdown 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 -- 172.21.15.46:0/616514929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f644090 msgr2=0x56190f652f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f644090 0x56190f652f10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faf04009e90 tx=0x7faf040046c0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 -- 172.21.15.46:0/616514929 shutdown_connections 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56190f6c5510 0x56190f6c7930 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56190f649a60 0x56190f654460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56190f6449d0 0x56190f653450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 --2- 172.21.15.46:0/616514929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56190f644090 0x56190f652f10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faef6a00640 0 client.17106 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 -- 172.21.15.46:0/616514929 >> 172.21.15.46:0/616514929 conn(0x56190f63f700 msgr2=0x56190f63e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 -- 172.21.15.46:0/616514929 shutdown_connections 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 1 -- 172.21.15.46:0/616514929 wait complete. 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.029+0000 7faf1ae27740 20 client.17106 trim_cache size 0 max 16384 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:45:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:45:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 -- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faef80a2bc0 msgr2=0x7faef80a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faef80a2bc0 0x7faef80a5080 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x56190f640580 tx=0x7faf04005f90 comp rx=0 tx=0).stop 2024-09-10T07:45:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 -- 172.21.15.46:0/1021037196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf10104f50 msgr2=0x7faf1019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:45:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf10104f50 0x7faf1019a500 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faefc009230 tx=0x7faefc031b60 comp rx=0 tx=0).stop 2024-09-10T07:45:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 -- 172.21.15.46:0/1021037196 shutdown_connections 2024-09-10T07:45:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf10105910 0x7faf101a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf10104f50 0x7faf1019a500 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7faef80a2bc0 0x7faef80a5080 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 --2- 172.21.15.46:0/1021037196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf10103d60 0x7faf10199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:45:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 -- 172.21.15.46:0/1021037196 >> 172.21.15.46:0/1021037196 conn(0x7faf100ff4e0 msgr2=0x7faf10108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:45:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 -- 172.21.15.46:0/1021037196 shutdown_connections 2024-09-10T07:45:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:45:47.030+0000 7faf17e00640 1 -- 172.21.15.46:0/1021037196 wait complete. 2024-09-10T07:45:47.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:47 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:45:47.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:47 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:45:47.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:47 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:45:47.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:47 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:45:47.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:47 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:45:47.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:47 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:45:47.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:47 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:45:47.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:47 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:45:47.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:47 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:45:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:48 smithi181 ceph-mon[35397]: pgmap v674: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 5.3 MiB/s wr, 662 op/s 2024-09-10T07:45:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:48 smithi063 ceph-mon[35020]: pgmap v674: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 5.3 MiB/s wr, 662 op/s 2024-09-10T07:45:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:48 smithi046 ceph-mon[30705]: pgmap v674: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 5.3 MiB/s wr, 662 op/s 2024-09-10T07:45:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:50 smithi181 ceph-mon[35397]: pgmap v675: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.6 MiB/s wr, 425 op/s 2024-09-10T07:45:50.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:50 smithi063 ceph-mon[35020]: pgmap v675: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.6 MiB/s wr, 425 op/s 2024-09-10T07:45:50.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:50 smithi046 ceph-mon[30705]: pgmap v675: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.6 MiB/s wr, 425 op/s 2024-09-10T07:45:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 510.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:45:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:52 smithi181 ceph-mon[35397]: pgmap v676: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-09-10T07:45:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:52 smithi063 ceph-mon[35020]: pgmap v676: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-09-10T07:45:52.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:52 smithi046 ceph-mon[30705]: pgmap v676: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-09-10T07:45:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:54 smithi181 ceph-mon[35397]: pgmap v677: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-09-10T07:45:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:54 smithi063 ceph-mon[35020]: pgmap v677: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-09-10T07:45:54.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:54 smithi046 ceph-mon[30705]: pgmap v677: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-09-10T07:45:55.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 515.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:45:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:56 smithi181 ceph-mon[35397]: pgmap v678: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-10T07:45:56.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:56 smithi063 ceph-mon[35020]: pgmap v678: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-10T07:45:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:56 smithi046 ceph-mon[30705]: pgmap v678: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-10T07:45:57.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:57 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:57.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:57 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:57.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:57 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:45:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:45:58 smithi181 ceph-mon[35397]: pgmap v679: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-10T07:45:58.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:45:58 smithi063 ceph-mon[35020]: pgmap v679: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-10T07:45:58.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:45:58 smithi046 ceph-mon[30705]: pgmap v679: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-10T07:46:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 -- 172.21.15.46:0/4105259101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 msgr2=0x7febb8101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 --2- 172.21.15.46:0/4105259101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 0x7febb8101a00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feba0009900 tx=0x7feba002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 -- 172.21.15.46:0/4105259101 shutdown_connections 2024-09-10T07:46:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 --2- 172.21.15.46:0/4105259101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febb8101f40 0x7febb8109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 --2- 172.21.15.46:0/4105259101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 0x7febb8101a00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 --2- 172.21.15.46:0/4105259101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb8103050 0x7febb8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 -- 172.21.15.46:0/4105259101 >> 172.21.15.46:0/4105259101 conn(0x7febb80fd0d0 msgr2=0x7febb80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 -- 172.21.15.46:0/4105259101 shutdown_connections 2024-09-10T07:46:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.161+0000 7febbd600640 1 -- 172.21.15.46:0/4105259101 wait complete. 2024-09-10T07:46:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.162+0000 7febbd600640 1 Processor -- start 2024-09-10T07:46:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.162+0000 7febbd600640 1 -- start start 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.162+0000 7febbd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 0x7febb8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.162+0000 7febbd600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febb8101f40 0x7febb819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febbd600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb8103050 0x7febb81a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febbd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febb810d740 con 0x7febb8101580 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febbd600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7febb810d5c0 con 0x7febb8103050 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febbd600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7febb810d8c0 con 0x7febb8101f40 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 0x7febb8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 0x7febb8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45750/0 (socket says 172.21.15.46:45750) 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 -- 172.21.15.46:0/3948787526 learned_addr learned my addr 172.21.15.46:0/3948787526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febbcc00640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb8103050 0x7febb81a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febafe00640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febb8101f40 0x7febb819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 -- 172.21.15.46:0/3948787526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febb8101f40 msgr2=0x7febb819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febb8101f40 0x7febb819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 -- 172.21.15.46:0/3948787526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb8103050 msgr2=0x7febb81a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb8103050 0x7febb81a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 -- 172.21.15.46:0/3948787526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb81a1c10 con 0x7febb8101580 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.163+0000 7febb7e00640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 0x7febb8199ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feba800e9c0 tx=0x7feba800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.164+0000 7febb5600640 1 -- 172.21.15.46:0/3948787526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba800cd80 con 0x7febb8101580 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.164+0000 7febb5600640 1 -- 172.21.15.46:0/3948787526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba80035f0 con 0x7febb8101580 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.164+0000 7febbd600640 1 -- 172.21.15.46:0/3948787526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb81a1f00 con 0x7febb8101580 2024-09-10T07:46:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.164+0000 7febb5600640 1 -- 172.21.15.46:0/3948787526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8010690 con 0x7febb8101580 2024-09-10T07:46:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.164+0000 7febbd600640 1 -- 172.21.15.46:0/3948787526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb81a2360 con 0x7febb8101580 2024-09-10T07:46:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febc1aa9740 1 Processor -- start 2024-09-10T07:46:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febb5600640 1 -- 172.21.15.46:0/3948787526 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feba80037a0 con 0x7febb8101580 2024-09-10T07:46:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febc1aa9740 1 -- start start 2024-09-10T07:46:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febc1aa9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556868436100 con 0x5568684d1260 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febc1aa9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5568683de1b0 con 0x5568684d6c30 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.167+0000 7febc1aa9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5568684320c0 con 0x5568684d1ba0 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febbcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febbcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684d1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45764/0 (socket says 172.21.15.46:45764) 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febbcc00640 1 -- 172.21.15.46:0/1474386278 learned_addr learned my addr 172.21.15.46:0/1474386278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febb7e00640 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febafe00640 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febbcc00640 1 -- 172.21.15.46:0/1474386278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 msgr2=0x5568684d6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febbcc00640 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684d6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febbcc00640 1 -- 172.21.15.46:0/1474386278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 msgr2=0x5568684d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febbcc00640 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febbcc00640 1 -- 172.21.15.46:0/1474386278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556868149210 con 0x5568684d1260 2024-09-10T07:46:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.168+0000 7febafe00640 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.169+0000 7febbcc00640 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684d1660 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7febb0009ee0 tx=0x7febb0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e08d5e254016d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.169+0000 7febb5600640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb9c0a2b80 0x7feb9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.169+0000 7febad200640 1 -- 172.21.15.46:0/1474386278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0016070 con 0x5568684d1260 2024-09-10T07:46:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.169+0000 7febad200640 1 -- 172.21.15.46:0/1474386278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb0004d20 con 0x5568684d1260 2024-09-10T07:46:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.169+0000 7febad200640 1 -- 172.21.15.46:0/1474386278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0012830 con 0x5568684d1260 2024-09-10T07:46:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.169+0000 7febb5600640 1 -- 172.21.15.46:0/3948787526 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feba80cf960 con 0x7febb8101580 2024-09-10T07:46:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.169+0000 7febb7e00640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb9c0a2b80 0x7feb9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 -- 172.21.15.46:0/1474386278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 msgr2=0x5568684d1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684d1660 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7febb0009ee0 tx=0x7febb0010740 comp rx=0 tx=0).stop 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 -- 172.21.15.46:0/1474386278 shutdown_connections 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1474386278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684d1660 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 -- 172.21.15.46:0/1474386278 >> 172.21.15.46:0/1474386278 conn(0x5568684cc8d0 msgr2=0x5568684cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 -- 172.21.15.46:0/1474386278 shutdown_connections 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febb7e00640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb9c0a2b80 0x7feb9c0a5040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feba80987b0 tx=0x7feba80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 -- 172.21.15.46:0/1474386278 wait complete. 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.170+0000 7febc1aa9740 1 -- start start 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febc1aa9740 1 Processor -- start 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1ba0 0x5568684db930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d6c30 0x5568684e2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febc1aa9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5568684d9c00 con 0x5568684d1260 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febc1aa9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5568683bf090 con 0x5568684d1ba0 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febc1aa9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5568684d9d80 con 0x5568684d6c30 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febc1aa9740 10 client.0 _ll_register_callbacks cb 0x556868413f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febafe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1ba0 0x5568684db930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febafe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1ba0 0x5568684db930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:55928/0 (socket says 172.21.15.46:55928) 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.171+0000 7febbcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7febafe00640 1 -- 172.21.15.46:0/1729275479 learned_addr learned my addr 172.21.15.46:0/1729275479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7febbcc00640 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d6c30 0x5568684e2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7febafe00640 1 -- 172.21.15.46:0/1729275479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d6c30 msgr2=0x5568684e2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7febafe00640 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d6c30 0x5568684e2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7febafe00640 1 -- 172.21.15.46:0/1729275479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 msgr2=0x5568684db3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7febafe00640 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684db3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7febafe00640 1 -- 172.21.15.46:0/1729275479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556868215620 con 0x5568684d1ba0 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7febafe00640 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1ba0 0x5568684db930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7feba0031440 tx=0x7feba0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf138dbd7d6155 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.172+0000 7feba5600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.173+0000 7feba5600640 1 -- 172.21.15.46:0/1729275479 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0035040 con 0x5568684d1ba0 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.173+0000 7febc1aa9740 1 -- 172.21.15.46:0/1729275479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb0004d20 con 0x5568684d1ba0 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.173+0000 7feba5600640 1 -- 172.21.15.46:0/1729275479 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feba00318b0 con 0x5568684d1ba0 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.173+0000 7feba5600640 1 -- 172.21.15.46:0/1729275479 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feba004e050 con 0x5568684d1ba0 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.173+0000 7febc1aa9740 10 client.27707 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.173+0000 7febc1aa9740 1 -- 172.21.15.46:0/1729275479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5568684e5820 con 0x5568684d1ba0 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.173+0000 7feba5600640 1 -- 172.21.15.46:0/1729275479 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb0004d20 con 0x5568684d1ba0 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.173+0000 7feba5600640 1 -- 172.21.15.46:0/1729275479 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7feba00035f0 con 0x5568684d1ba0 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7febc1aa9740 10 client.27707 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7febc1aa9740 10 client.27707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7febc1aa9740 20 client.27707 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5568684e6e50 0x5568684e9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7febc1aa9740 4 client.27707 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7febc1aa9740 1 -- 172.21.15.46:0/1729275479 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556868443830 con 0x5568684e6e50 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7febbcc00640 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5568684e6e50 0x5568684e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7febbcc00640 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5568684e6e50 0x5568684e9240 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec743a9753983af9 server_cookie=55fdd34abe7fd37c in_seq=0 out_seq=0 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.174+0000 7feba5600640 10 client.27707 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7feba5600640 1 -- 172.21.15.46:0/1729275479 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556868443830 con 0x5568684e6e50 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7feba5600640 10 client.27707 handle_command_reply: tid=0 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 client.27707 shutdown 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 -- 172.21.15.46:0/1729275479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1ba0 msgr2=0x5568684db930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1ba0 0x5568684db930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7feba0031440 tx=0x7feba0005270 comp rx=0 tx=0).stop 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 -- 172.21.15.46:0/1729275479 shutdown_connections 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5568684e6e50 0x5568684e9240 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d6c30 0x5568684e2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1ba0 0x5568684db930 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1729275479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1260 0x5568684db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7febc1aa9740 1 -- 172.21.15.46:0/1729275479 >> 172.21.15.46:0/1729275479 conn(0x5568684cc8d0 msgr2=0x5568684ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.175+0000 7feba5600640 0 client.27707 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.176+0000 7febc1aa9740 1 -- 172.21.15.46:0/1729275479 shutdown_connections 2024-09-10T07:46:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.176+0000 7febc1aa9740 1 -- 172.21.15.46:0/1729275479 wait complete. 2024-09-10T07:46:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.176+0000 7febc1aa9740 20 client.27707 trim_cache size 0 max 16384 2024-09-10T07:46:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.191+0000 7febc1aa9740 1 Processor -- start 2024-09-10T07:46:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.191+0000 7febc1aa9740 1 -- start start 2024-09-10T07:46:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.191+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1260 0x5568684cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.191+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684e2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.191+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d6c30 0x5568684cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.191+0000 7febc1aa9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febb81ac7c0 con 0x5568684d6c30 2024-09-10T07:46:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.191+0000 7febc1aa9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7febb81ae300 con 0x5568684d1260 2024-09-10T07:46:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.191+0000 7febc1aa9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7febb81ace90 con 0x5568684d1ba0 2024-09-10T07:46:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684e2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684e2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34164/0 (socket says 172.21.15.46:34164) 2024-09-10T07:46:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 -- 172.21.15.46:0/4118255578 learned_addr learned my addr 172.21.15.46:0/4118255578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febbcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d6c30 0x5568684cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febbcc00640 1 --2- 172.21.15.46:0/4118255578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1260 0x5568684cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 -- 172.21.15.46:0/4118255578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1260 msgr2=0x5568684cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 --2- 172.21.15.46:0/4118255578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1260 0x5568684cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 -- 172.21.15.46:0/4118255578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d6c30 msgr2=0x5568684cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 --2- 172.21.15.46:0/4118255578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d6c30 0x5568684cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 -- 172.21.15.46:0/4118255578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568684d9f00 con 0x5568684d1ba0 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.192+0000 7febafe00640 1 --2- 172.21.15.46:0/4118255578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684e2930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feba0009850 tx=0x7feba0005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d1ac5b05cfbfe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.193+0000 7feba7e00640 1 -- 172.21.15.46:0/4118255578 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0035030 con 0x5568684d1ba0 2024-09-10T07:46:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.193+0000 7feba7e00640 1 -- 172.21.15.46:0/4118255578 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba0040070 con 0x5568684d1ba0 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.193+0000 7feba7e00640 1 -- 172.21.15.46:0/4118255578 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba00037b0 con 0x5568684d1ba0 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.193+0000 7febc1aa9740 1 -- 172.21.15.46:0/4118255578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 msgr2=0x5568684e2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.193+0000 7febc1aa9740 1 --2- 172.21.15.46:0/4118255578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684e2930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feba0009850 tx=0x7feba0005730 comp rx=0 tx=0).stop 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 -- 172.21.15.46:0/4118255578 shutdown_connections 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 --2- 172.21.15.46:0/4118255578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d6c30 0x5568684cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 --2- 172.21.15.46:0/4118255578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1ba0 0x5568684e2930 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 --2- 172.21.15.46:0/4118255578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d1260 0x5568684cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 -- 172.21.15.46:0/4118255578 >> 172.21.15.46:0/4118255578 conn(0x5568684cc8d0 msgr2=0x5568684d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 -- 172.21.15.46:0/4118255578 shutdown_connections 2024-09-10T07:46:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 -- 172.21.15.46:0/4118255578 wait complete. 2024-09-10T07:46:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 -- start start 2024-09-10T07:46:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.194+0000 7febc1aa9740 1 Processor -- start 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1260 0x5568684dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1ba0 0x5568684e0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febc1aa9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684e1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febc1aa9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febb81adc30 con 0x5568684d1ba0 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febc1aa9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7febb81ad560 con 0x5568684d6c30 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febc1aa9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7febb81aa7a0 con 0x5568684d1260 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febc1aa9740 10 client.0 _ll_register_callbacks cb 0x7feba0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1ba0 0x5568684e0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febbcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1260 0x5568684dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1ba0 0x5568684e0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45792/0 (socket says 172.21.15.46:45792) 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febafe00640 1 -- 172.21.15.46:0/1351554980 learned_addr learned my addr 172.21.15.46:0/1351554980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febbcc00640 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684e1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febafe00640 1 -- 172.21.15.46:0/1351554980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1260 msgr2=0x5568684dfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febafe00640 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1260 0x5568684dfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febafe00640 1 -- 172.21.15.46:0/1351554980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 msgr2=0x5568684e1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febafe00640 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684e1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febafe00640 1 -- 172.21.15.46:0/1351554980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568684e1660 con 0x5568684d1ba0 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.195+0000 7febbcc00640 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7febafe00640 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1ba0 0x5568684e0040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7feba0005ab0 tx=0x7feba002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511e3571e81622c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7feba5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7feba5600640 1 -- 172.21.15.46:0/1351554980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0035070 con 0x5568684d1ba0 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7febc1aa9740 1 -- 172.21.15.46:0/1351554980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba0040070 con 0x5568684d1ba0 2024-09-10T07:46:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7feba5600640 1 -- 172.21.15.46:0/1351554980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feba00048d0 con 0x5568684d1ba0 2024-09-10T07:46:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7feba5600640 1 -- 172.21.15.46:0/1351554980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feba0009520 con 0x5568684d1ba0 2024-09-10T07:46:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7febc1aa9740 10 client.17124 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7febc1aa9740 1 -- 172.21.15.46:0/1351554980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5568684e2970 con 0x5568684d1ba0 2024-09-10T07:46:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.196+0000 7feba5600640 1 -- 172.21.15.46:0/1351554980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba0040070 con 0x5568684d1ba0 2024-09-10T07:46:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7feba5600640 1 -- 172.21.15.46:0/1351554980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7feba0045470 con 0x5568684d1ba0 2024-09-10T07:46:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7febc1aa9740 10 client.17124 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7febc1aa9740 10 client.17124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7febc1aa9740 20 client.17124 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556868551ff0 0x5568685543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7febc1aa9740 4 client.17124 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:46:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7febc1aa9740 1 -- 172.21.15.46:0/1351554980 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55686843db80 con 0x556868551ff0 2024-09-10T07:46:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7febbcc00640 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556868551ff0 0x5568685543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7febbcc00640 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556868551ff0 0x5568685543e0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58fc4182ce725abe server_cookie=307e96cccbc3334d in_seq=0 out_seq=0 2024-09-10T07:46:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.197+0000 7feba5600640 10 client.17124 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.206+0000 7feba5600640 1 -- 172.21.15.46:0/1351554980 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55686843db80 con 0x556868551ff0 2024-09-10T07:46:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.206+0000 7feba5600640 10 client.17124 handle_command_reply: tid=0 2024-09-10T07:46:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.206+0000 7febc1aa9740 1 client.17124 shutdown 2024-09-10T07:46:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 -- 172.21.15.46:0/1351554980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1ba0 msgr2=0x5568684e0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1ba0 0x5568684e0040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7feba0005ab0 tx=0x7feba002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 -- 172.21.15.46:0/1351554980 shutdown_connections 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556868551ff0 0x5568685543e0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568684d6c30 0x5568684e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7feba5600640 0 client.17124 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568684d1ba0 0x5568684e0040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 --2- 172.21.15.46:0/1351554980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568684d1260 0x5568684dfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 -- 172.21.15.46:0/1351554980 >> 172.21.15.46:0/1351554980 conn(0x5568684cc8d0 msgr2=0x5568684cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 -- 172.21.15.46:0/1351554980 shutdown_connections 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 1 -- 172.21.15.46:0/1351554980 wait complete. 2024-09-10T07:46:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.207+0000 7febc1aa9740 20 client.17124 trim_cache size 0 max 16384 2024-09-10T07:46:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:46:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:46:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:46:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:46:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 -- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb9c0a2b80 msgr2=0x7feb9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb9c0a2b80 0x7feb9c0a5040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feba80987b0 tx=0x7feba80d5530 comp rx=0 tx=0).stop 2024-09-10T07:46:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 -- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 msgr2=0x7febb8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 0x7febb8199ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feba800e9c0 tx=0x7feba800ee80 comp rx=0 tx=0).stop 2024-09-10T07:46:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 -- 172.21.15.46:0/3948787526 shutdown_connections 2024-09-10T07:46:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febb8103050 0x7febb81a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febb8101f40 0x7febb819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb9c0a2b80 0x7feb9c0a5040 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 --2- 172.21.15.46:0/3948787526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb8101580 0x7febb8199ff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 -- 172.21.15.46:0/3948787526 >> 172.21.15.46:0/3948787526 conn(0x7febb80fd0d0 msgr2=0x7febb8108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 -- 172.21.15.46:0/3948787526 shutdown_connections 2024-09-10T07:46:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:00.208+0000 7febbd600640 1 -- 172.21.15.46:0/3948787526 wait complete. 2024-09-10T07:46:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:00 smithi181 ceph-mon[35397]: pgmap v680: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 98 KiB/s wr, 25 op/s 2024-09-10T07:46:00.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:00 smithi063 ceph-mon[35020]: pgmap v680: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 98 KiB/s wr, 25 op/s 2024-09-10T07:46:00.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:00 smithi046 ceph-mon[30705]: pgmap v680: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 98 KiB/s wr, 25 op/s 2024-09-10T07:46:00.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 520.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:46:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:02 smithi181 ceph-mon[35397]: pgmap v681: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 122 KiB/s wr, 27 op/s 2024-09-10T07:46:02.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:02 smithi063 ceph-mon[35020]: pgmap v681: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 122 KiB/s wr, 27 op/s 2024-09-10T07:46:02.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:02 smithi046 ceph-mon[30705]: pgmap v681: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 122 KiB/s wr, 27 op/s 2024-09-10T07:46:02.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:46:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 -- 172.21.15.46:0/351039764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c101df0 msgr2=0x7f4e4c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 --2- 172.21.15.46:0/351039764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c101df0 0x7f4e4c1021d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009ca0 tx=0x7f4e3c02a8b0 comp rx=0 tx=0).stop 2024-09-10T07:46:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 -- 172.21.15.46:0/351039764 shutdown_connections 2024-09-10T07:46:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 --2- 172.21.15.46:0/351039764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e4c106f20 0x7f4e4c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 --2- 172.21.15.46:0/351039764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c1027a0 0x7f4e4c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 --2- 172.21.15.46:0/351039764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c101df0 0x7f4e4c1021d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 -- 172.21.15.46:0/351039764 >> 172.21.15.46:0/351039764 conn(0x7f4e4c0fd8b0 msgr2=0x7f4e4c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 -- 172.21.15.46:0/351039764 shutdown_connections 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.780+0000 7f4e53200640 1 -- 172.21.15.46:0/351039764 wait complete. 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.781+0000 7f4e53200640 1 Processor -- start 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.781+0000 7f4e53200640 1 -- start start 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.781+0000 7f4e53200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c101df0 0x7f4e4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.781+0000 7f4e53200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c1027a0 0x7f4e4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.781+0000 7f4e53200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e4c106f20 0x7f4e4c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.781+0000 7f4e53200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c111f40 con 0x7f4e4c1027a0 2024-09-10T07:46:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.781+0000 7f4e53200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c111dc0 con 0x7f4e4c106f20 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.781+0000 7f4e53200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1120c0 con 0x7f4e4c101df0 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c1027a0 0x7f4e4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c1027a0 0x7f4e4c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45812/0 (socket says 172.21.15.46:45812) 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 -- 172.21.15.46:0/482932799 learned_addr learned my addr 172.21.15.46:0/482932799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e51200640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e4c106f20 0x7f4e4c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e4be00640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c101df0 0x7f4e4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 -- 172.21.15.46:0/482932799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c101df0 msgr2=0x7f4e4c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c101df0 0x7f4e4c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 -- 172.21.15.46:0/482932799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e4c106f20 msgr2=0x7f4e4c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e4c106f20 0x7f4e4c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 -- 172.21.15.46:0/482932799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c1a68e0 con 0x7f4e4c1027a0 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.782+0000 7f4e43400640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c1027a0 0x7f4e4c074a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400d950 tx=0x7f4e3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.783+0000 7f4e49600640 1 -- 172.21.15.46:0/482932799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34017070 con 0x7f4e4c1027a0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.783+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c1a6bd0 con 0x7f4e4c1027a0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.783+0000 7f4e49600640 1 -- 172.21.15.46:0/482932799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e34009c30 con 0x7f4e4c1027a0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.783+0000 7f4e49600640 1 -- 172.21.15.46:0/482932799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400f6b0 con 0x7f4e4c1027a0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.783+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1a7110 con 0x7f4e4c1027a0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.784+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c1021d0 con 0x7f4e4c1027a0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.786+0000 7f4e49600640 1 -- 172.21.15.46:0/482932799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e340109d0 con 0x7f4e4c1027a0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.787+0000 7f4e49600640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e300a2b80 0x7f4e300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.787+0000 7f4e49600640 1 -- 172.21.15.46:0/482932799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4e340cef80 con 0x7f4e4c1027a0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.787+0000 7f4e4be00640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e300a2b80 0x7f4e300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.788+0000 7f4e4be00640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e300a2b80 0x7f4e300a5040 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c004af0 tx=0x7f4e3c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:03.791+0000 7f4e49600640 1 -- 172.21.15.46:0/482932799 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e34009de0 con 0x7f4e4c1027a0 2024-09-10T07:46:04.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.013+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e4c10a100 con 0x7f4e4c1027a0 2024-09-10T07:46:04.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.014+0000 7f4e49600640 1 -- 172.21.15.46:0/482932799 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11972 (secure 0 0 0) 0x7f4e3409b4b0 con 0x7f4e4c1027a0 2024-09-10T07:46:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:46:04.014 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"btime":"2024-09-10T07:45:56:057049+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:45:55.519456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:46:04.014 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 48 2024-09-10T07:46:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e300a2b80 msgr2=0x7f4e300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e300a2b80 0x7f4e300a5040 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c004af0 tx=0x7f4e3c004a40 comp rx=0 tx=0).stop 2024-09-10T07:46:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c1027a0 msgr2=0x7f4e4c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c1027a0 0x7f4e4c074a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400d950 tx=0x7f4e3400de10 comp rx=0 tx=0).stop 2024-09-10T07:46:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 shutdown_connections 2024-09-10T07:46:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e4c106f20 0x7f4e4c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e300a2b80 0x7f4e300a5040 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e4c1027a0 0x7f4e4c074a30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 --2- 172.21.15.46:0/482932799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c101df0 0x7f4e4c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 >> 172.21.15.46:0/482932799 conn(0x7f4e4c0fd8b0 msgr2=0x7f4e4c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 shutdown_connections 2024-09-10T07:46:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:04.017+0000 7f4e53200640 1 -- 172.21.15.46:0/482932799 wait complete. 2024-09-10T07:46:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:04 smithi181 ceph-mon[35397]: pgmap v682: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 345 KiB/s wr, 50 op/s 2024-09-10T07:46:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:04 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/482932799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:04 smithi063 ceph-mon[35020]: pgmap v682: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 345 KiB/s wr, 50 op/s 2024-09-10T07:46:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:04 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/482932799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:04.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:04 smithi046 ceph-mon[30705]: pgmap v682: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 345 KiB/s wr, 50 op/s 2024-09-10T07:46:04.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:04 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/482932799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:05.417 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id d5411a2f --release --await' 2024-09-10T07:46:05.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id d5411a2f --release --await 2024-09-10T07:46:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 525.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:46:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:06 smithi181 ceph-mon[35397]: pgmap v683: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 612 KiB/s wr, 51 op/s 2024-09-10T07:46:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:06 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:46:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:06 smithi063 ceph-mon[35020]: pgmap v683: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 612 KiB/s wr, 51 op/s 2024-09-10T07:46:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:06 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:46:06.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:06 smithi046 ceph-mon[30705]: pgmap v683: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 612 KiB/s wr, 51 op/s 2024-09-10T07:46:06.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:06 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:46:07.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.979+0000 7f81e2200640 1 -- 172.21.15.46:0/2249668294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc1047e0 msgr2=0x7f81dc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.979+0000 7f81e2200640 1 --2- 172.21.15.46:0/2249668294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc1047e0 0x7f81dc108830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009900 tx=0x7f81c802a400 comp rx=0 tx=0).stop 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.980+0000 7f81e2200640 1 -- 172.21.15.46:0/2249668294 shutdown_connections 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.980+0000 7f81e2200640 1 --2- 172.21.15.46:0/2249668294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81dc108f60 0x7f81dc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.980+0000 7f81e2200640 1 --2- 172.21.15.46:0/2249668294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc1047e0 0x7f81dc108830 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.980+0000 7f81e2200640 1 --2- 172.21.15.46:0/2249668294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81dc103e30 0x7f81dc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.980+0000 7f81e2200640 1 -- 172.21.15.46:0/2249668294 >> 172.21.15.46:0/2249668294 conn(0x7f81dc0ff8d0 msgr2=0x7f81dc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.980+0000 7f81e2200640 1 -- 172.21.15.46:0/2249668294 shutdown_connections 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.980+0000 7f81e2200640 1 -- 172.21.15.46:0/2249668294 wait complete. 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e2200640 1 Processor -- start 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e2200640 1 -- start start 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc103e30 0x7f81dc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e2200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81dc1047e0 0x7f81dc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e2200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81dc108f60 0x7f81dc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc113fc0 con 0x7f81dc103e30 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e2200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc113e40 con 0x7f81dc108f60 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e2200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc114140 con 0x7f81dc1047e0 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc103e30 0x7f81dc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc103e30 0x7f81dc19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34532/0 (socket says 172.21.15.46:34532) 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81e0e00640 1 -- 172.21.15.46:0/3310015818 learned_addr learned my addr 172.21.15.46:0/3310015818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.981+0000 7f81dbe00640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81dc1047e0 0x7f81dc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e1800640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81dc108f60 0x7f81dc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e0e00640 1 -- 172.21.15.46:0/3310015818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81dc1047e0 msgr2=0x7f81dc19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e0e00640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81dc1047e0 0x7f81dc19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e0e00640 1 -- 172.21.15.46:0/3310015818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81dc108f60 msgr2=0x7f81dc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e0e00640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81dc108f60 0x7f81dc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e0e00640 1 -- 172.21.15.46:0/3310015818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc19f400 con 0x7f81dc103e30 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e1800640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81dc108f60 0x7f81dc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e0e00640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc103e30 0x7f81dc19a450 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f81d000e9f0 tx=0x7f81d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81d9600640 1 -- 172.21.15.46:0/3310015818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d000cd80 con 0x7f81dc103e30 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81d9600640 1 -- 172.21.15.46:0/3310015818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d0010910 con 0x7f81dc103e30 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81d9600640 1 -- 172.21.15.46:0/3310015818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0018990 con 0x7f81dc103e30 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.982+0000 7f81e2200640 1 -- 172.21.15.46:0/3310015818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc19f6f0 con 0x7f81dc103e30 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.983+0000 7f81e2200640 1 -- 172.21.15.46:0/3310015818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc1a6fd0 con 0x7f81dc103e30 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e6403740 1 Processor -- start 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81d9600640 1 -- 172.21.15.46:0/3310015818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f81d0010ac0 con 0x7f81dc103e30 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e6403740 1 -- start start 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e6403740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad86900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e6403740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e6403740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad91700 0x55cb3ad93af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e6403740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cb3acec650 con 0x55cb3ad91700 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e6403740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb3ac93f10 con 0x55cb3ad86520 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e6403740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cb3ace8610 con 0x55cb3ad86e40 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad86900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad86900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50580/0 (socket says 172.21.15.46:50580) 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e1800640 1 -- 172.21.15.46:0/2814083302 learned_addr learned my addr 172.21.15.46:0/2814083302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81e0e00640 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.986+0000 7f81dbe00640 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad91700 0x55cb3ad93af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.987+0000 7f81e1800640 1 -- 172.21.15.46:0/2814083302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 msgr2=0x55cb3ad911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.987+0000 7f81e1800640 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.987+0000 7f81e1800640 1 -- 172.21.15.46:0/2814083302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad91700 msgr2=0x55cb3ad93af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.987+0000 7f81e1800640 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad91700 0x55cb3ad93af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.987+0000 7f81e1800640 1 -- 172.21.15.46:0/2814083302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3a9fb580 con 0x55cb3ad86520 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.987+0000 7f81e1800640 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad86900 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f81d4011790 tx=0x7f81d4011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abe4b889bab5ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.987+0000 7f81d9600640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81c40a2c00 0x7f81c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.987+0000 7f81ce000640 1 -- 172.21.15.46:0/2814083302 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4003c30 con 0x55cb3ad86520 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81ce000640 1 -- 172.21.15.46:0/2814083302 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d400fe80 con 0x55cb3ad86520 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81ce000640 1 -- 172.21.15.46:0/2814083302 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40193e0 con 0x55cb3ad86520 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81d9600640 1 -- 172.21.15.46:0/3310015818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f81d00cf690 con 0x7f81dc103e30 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e0e00640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81c40a2c00 0x7f81c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e6403740 1 -- 172.21.15.46:0/2814083302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 msgr2=0x55cb3ad86900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e6403740 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad86900 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f81d4011790 tx=0x7f81d4011c50 comp rx=0 tx=0).stop 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e6403740 1 -- 172.21.15.46:0/2814083302 shutdown_connections 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e6403740 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad91700 0x55cb3ad93af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e6403740 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e6403740 1 --2- 172.21.15.46:0/2814083302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad86900 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e0e00640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81c40a2c00 0x7f81c40a50c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f81d0098590 tx=0x7f81d00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.988+0000 7f81e6403740 1 -- 172.21.15.46:0/2814083302 >> 172.21.15.46:0/2814083302 conn(0x55cb3ad82ab0 msgr2=0x55cb3ad82e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.989+0000 7f81e6403740 1 -- 172.21.15.46:0/2814083302 shutdown_connections 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.989+0000 7f81e6403740 1 -- 172.21.15.46:0/2814083302 wait complete. 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.989+0000 7f81e6403740 1 -- start start 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.989+0000 7f81e6403740 1 Processor -- start 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e6403740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad8e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e6403740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86e40 0x55cb3ad8eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e6403740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad91700 0x55cb3ad8f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e6403740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cb3ad942d0 con 0x55cb3ad86e40 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e6403740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb3aca7350 con 0x55cb3ad86520 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e6403740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cb3ad94450 con 0x55cb3ad91700 2024-09-10T07:46:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e6403740 10 client.0 _ll_register_callbacks cb 0x55cb3acca290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86e40 0x55cb3ad8eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad8e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86e40 0x55cb3ad8eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34558/0 (socket says 172.21.15.46:34558) 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81dbe00640 1 -- 172.21.15.46:0/1101441111 learned_addr learned my addr 172.21.15.46:0/1101441111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e1800640 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad91700 0x55cb3ad8f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81dbe00640 1 -- 172.21.15.46:0/1101441111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad91700 msgr2=0x55cb3ad8f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81dbe00640 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad91700 0x55cb3ad8f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81dbe00640 1 -- 172.21.15.46:0/1101441111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 msgr2=0x55cb3ad8e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81dbe00640 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad8e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81dbe00640 1 -- 172.21.15.46:0/1101441111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3aac5960 con 0x55cb3ad86e40 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.990+0000 7f81e1800640 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad91700 0x55cb3ad8f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.991+0000 7f81dbe00640 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86e40 0x55cb3ad8eb60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009900 tx=0x7f81c80317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe052eb9946c8677 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.991+0000 7f81c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.991+0000 7f81c2a00640 1 -- 172.21.15.46:0/1101441111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8035040 con 0x55cb3ad86e40 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.991+0000 7f81c2a00640 1 -- 172.21.15.46:0/1101441111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f81c8004460 con 0x55cb3ad86e40 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.991+0000 7f81e6403740 1 -- 172.21.15.46:0/1101441111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cb3acf9bc0 con 0x55cb3ad86e40 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.991+0000 7f81c2a00640 1 -- 172.21.15.46:0/1101441111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f81c80035f0 con 0x55cb3ad86e40 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.991+0000 7f81e6403740 10 client.17142 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.991+0000 7f81e6403740 1 -- 172.21.15.46:0/1101441111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cb3ad9ff10 con 0x55cb3ad86e40 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81c2a00640 1 -- 172.21.15.46:0/1101441111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cb3acf9bc0 con 0x55cb3ad86e40 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81c2a00640 1 -- 172.21.15.46:0/1101441111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f81c8046880 con 0x55cb3ad86e40 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81e6403740 10 client.17142 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81e6403740 10 client.17142 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81e6403740 20 client.17142 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81e6403740 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb3ada13d0 0x55cb3ada3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81e6403740 4 client.17142 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81e6403740 1 -- 172.21.15.46:0/1101441111 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb3acf40d0 con 0x55cb3ada13d0 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.992+0000 7f81e1800640 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb3ada13d0 0x55cb3ada3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.993+0000 7f81e1800640 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb3ada13d0 0x55cb3ada3800 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea32252a5b874026 server_cookie=1a86c42fa159a2bd in_seq=0 out_seq=0 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.993+0000 7f81c2a00640 10 client.17142 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81c2a00640 1 -- 172.21.15.46:0/1101441111 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb3acf40d0 con 0x55cb3ada13d0 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81c2a00640 10 client.17142 handle_command_reply: tid=0 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 client.17142 shutdown 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 -- 172.21.15.46:0/1101441111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86e40 msgr2=0x55cb3ad8eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86e40 0x55cb3ad8eb60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009900 tx=0x7f81c80317e0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 -- 172.21.15.46:0/1101441111 shutdown_connections 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb3ada13d0 0x55cb3ada3800 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad91700 0x55cb3ad8f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86e40 0x55cb3ad8eb60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 --2- 172.21.15.46:0/1101441111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad86520 0x55cb3ad8e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81c2a00640 0 client.17142 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 -- 172.21.15.46:0/1101441111 >> 172.21.15.46:0/1101441111 conn(0x55cb3ad82ab0 msgr2=0x55cb3ad81cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 -- 172.21.15.46:0/1101441111 shutdown_connections 2024-09-10T07:46:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 1 -- 172.21.15.46:0/1101441111 wait complete. 2024-09-10T07:46:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:06.994+0000 7f81e6403740 20 client.17142 trim_cache size 0 max 16384 2024-09-10T07:46:07.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e6403740 1 Processor -- start 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e6403740 1 -- start start 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e6403740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ada9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e6403740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ada9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e6403740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 0x55cb3ad8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e6403740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1b0cd0 con 0x55cb3ad86520 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e6403740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1b2810 con 0x55cb3ad91700 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e6403740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1b13a0 con 0x55cb3ad86e40 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.013+0000 7f81e1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ada9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ada9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34562/0 (socket says 172.21.15.46:34562) 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 -- 172.21.15.46:0/2147391597 learned_addr learned my addr 172.21.15.46:0/2147391597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81dbe00640 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ada9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 0x55cb3ad8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 -- 172.21.15.46:0/2147391597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 msgr2=0x55cb3ada9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ada9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 -- 172.21.15.46:0/2147391597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 msgr2=0x55cb3ad8b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 0x55cb3ad8b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 -- 172.21.15.46:0/2147391597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3ad828e0 con 0x55cb3ad86520 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81e1800640 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ada9110 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f81d4004570 tx=0x7f81d40045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9a2e34c038936a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.014+0000 7f81cd600640 1 -- 172.21.15.46:0/2147391597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4004260 con 0x55cb3ad86520 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81cd600640 1 -- 172.21.15.46:0/2147391597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d4003a30 con 0x55cb3ad86520 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81cd600640 1 -- 172.21.15.46:0/2147391597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4005050 con 0x55cb3ad86520 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 -- 172.21.15.46:0/2147391597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 msgr2=0x55cb3ada9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ada9110 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f81d4004570 tx=0x7f81d40045a0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 -- 172.21.15.46:0/2147391597 shutdown_connections 2024-09-10T07:46:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 0x55cb3ad8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ada9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 --2- 172.21.15.46:0/2147391597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ada9110 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 -- 172.21.15.46:0/2147391597 >> 172.21.15.46:0/2147391597 conn(0x55cb3ad82ab0 msgr2=0x55cb3ad81cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 -- 172.21.15.46:0/2147391597 shutdown_connections 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.015+0000 7f81e6403740 1 -- 172.21.15.46:0/2147391597 wait complete. 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.016+0000 7f81e6403740 1 -- start start 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.016+0000 7f81e6403740 1 Processor -- start 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.016+0000 7f81e6403740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ad996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.016+0000 7f81e6403740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad99be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.016+0000 7f81e6403740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 0x55cb3ad96670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.016+0000 7f81e6403740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1b2140 con 0x55cb3ad86520 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.016+0000 7f81e6403740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1b1a70 con 0x55cb3ad91700 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.016+0000 7f81e6403740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1aecb0 con 0x55cb3ad86e40 2024-09-10T07:46:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81e6403740 10 client.0 _ll_register_callbacks cb 0x7f81c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81e1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ad996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81e1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ad996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34570/0 (socket says 172.21.15.46:34570) 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81e1800640 1 -- 172.21.15.46:0/2712864525 learned_addr learned my addr 172.21.15.46:0/2712864525 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81dbe00640 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad99be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81e1800640 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 0x55cb3ad96670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81dbe00640 1 -- 172.21.15.46:0/2712864525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 msgr2=0x55cb3ad96670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81dbe00640 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 0x55cb3ad96670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81dbe00640 1 -- 172.21.15.46:0/2712864525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 msgr2=0x55cb3ad996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.017+0000 7f81dbe00640 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ad996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.018+0000 7f81dbe00640 1 -- 172.21.15.46:0/2712864525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3ad83740 con 0x55cb3ad86e40 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.018+0000 7f81dbe00640 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad99be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f81c802a920 tx=0x7f81c8005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b435494c5bec1ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.018+0000 7f81c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.018+0000 7f81c2a00640 1 -- 172.21.15.46:0/2712864525 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8035050 con 0x55cb3ad86e40 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.018+0000 7f81c2a00640 1 -- 172.21.15.46:0/2712864525 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f81c8046470 con 0x55cb3ad86e40 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.018+0000 7f81e6403740 1 -- 172.21.15.46:0/2712864525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d4003a30 con 0x55cb3ad86e40 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81c2a00640 1 -- 172.21.15.46:0/2712864525 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f81c8005d50 con 0x55cb3ad86e40 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81e6403740 10 client.27688 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81e6403740 1 -- 172.21.15.46:0/2712864525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cb3ad83b60 con 0x55cb3ad86e40 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81c2a00640 1 -- 172.21.15.46:0/2712864525 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d4003a30 con 0x55cb3ad86e40 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81c2a00640 1 -- 172.21.15.46:0/2712864525 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f81c80038b0 con 0x55cb3ad86e40 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81e6403740 10 client.27688 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81e6403740 10 client.27688 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81e6403740 20 client.27688 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81e6403740 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb3ae156d0 0x55cb3ae17af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81e6403740 4 client.27688 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d5411a2f", "release": true, "await": true}] 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.019+0000 7f81e6403740 1 -- 172.21.15.46:0/2712864525 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d5411a2f", "release": true, "await": true}) -- 0x55cb3ac70430 con 0x55cb3ae156d0 2024-09-10T07:46:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.020+0000 7f81e1800640 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb3ae156d0 0x55cb3ae17af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.020+0000 7f81e1800640 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb3ae156d0 0x55cb3ae17af0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15565bdddbf97fb6 server_cookie=4ac386bd2a78fceb in_seq=0 out_seq=0 2024-09-10T07:46:07.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.020+0000 7f81c2a00640 10 client.27688 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.054+0000 7f81c2a00640 1 -- 172.21.15.46:0/2712864525 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cb3ac70430 con 0x55cb3ae156d0 2024-09-10T07:46:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.054+0000 7f81c2a00640 10 client.27688 handle_command_reply: tid=0 2024-09-10T07:46:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.054+0000 7f81e6403740 1 client.27688 shutdown 2024-09-10T07:46:07.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.058+0000 7f81e6403740 1 -- 172.21.15.46:0/2712864525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 msgr2=0x55cb3ad99be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.058+0000 7f81e6403740 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad99be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f81c802a920 tx=0x7f81c8005270 comp rx=0 tx=0).stop 2024-09-10T07:46:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 1 -- 172.21.15.46:0/2712864525 shutdown_connections 2024-09-10T07:46:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb3ae156d0 0x55cb3ae17af0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb3ad91700 0x55cb3ad96670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb3ad86e40 0x55cb3ad99be0 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f81c802a920 tx=0x7f81c8005270 comp rx=0 tx=0).stop 2024-09-10T07:46:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 1 --2- 172.21.15.46:0/2712864525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb3ad86520 0x55cb3ad996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 1 -- 172.21.15.46:0/2712864525 >> 172.21.15.46:0/2712864525 conn(0x55cb3ad82ab0 msgr2=0x55cb3ad87f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81c2a00640 0 client.27688 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:07.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 1 -- 172.21.15.46:0/2712864525 shutdown_connections 2024-09-10T07:46:07.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 1 -- 172.21.15.46:0/2712864525 wait complete. 2024-09-10T07:46:07.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.059+0000 7f81e6403740 20 client.27688 trim_cache size 0 max 16384 2024-09-10T07:46:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:46:07.063 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 48, 2024-09-10T07:46:07.063 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 32, 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "d5411a2f": { 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 32, 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 144.6, 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:46:07.064 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.065+0000 7f81e2200640 1 -- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81c40a2c00 msgr2=0x7f81c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.065+0000 7f81e2200640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81c40a2c00 0x7f81c40a50c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f81d0098590 tx=0x7f81d00d5530 comp rx=0 tx=0).stop 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.065+0000 7f81e2200640 1 -- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc103e30 msgr2=0x7f81dc19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.065+0000 7f81e2200640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc103e30 0x7f81dc19a450 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f81d000e9f0 tx=0x7f81d000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.066+0000 7f81e2200640 1 -- 172.21.15.46:0/3310015818 shutdown_connections 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.066+0000 7f81e2200640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81dc108f60 0x7f81dc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.066+0000 7f81e2200640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81dc1047e0 0x7f81dc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.066+0000 7f81e2200640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81c40a2c00 0x7f81c40a50c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.066+0000 7f81e2200640 1 --2- 172.21.15.46:0/3310015818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81dc103e30 0x7f81dc19a450 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.066+0000 7f81e2200640 1 -- 172.21.15.46:0/3310015818 >> 172.21.15.46:0/3310015818 conn(0x7f81dc0ff8d0 msgr2=0x7f81dc100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.066+0000 7f81e2200640 1 -- 172.21.15.46:0/3310015818 shutdown_connections 2024-09-10T07:46:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:07.066+0000 7f81e2200640 1 -- 172.21.15.46:0/3310015818 wait complete. 2024-09-10T07:46:08.219 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd5411a2f', total seconds elapsed: 29.9 2024-09-10T07:46:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:08 smithi181 ceph-mon[35397]: pgmap v684: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 613 KiB/s wr, 52 op/s 2024-09-10T07:46:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:08 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:08 smithi063 ceph-mon[35020]: pgmap v684: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 613 KiB/s wr, 52 op/s 2024-09-10T07:46:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:08 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:08 smithi046 ceph-mon[30705]: pgmap v684: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 613 KiB/s wr, 52 op/s 2024-09-10T07:46:08.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:08.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:08 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:10 smithi181 ceph-mon[35397]: pgmap v685: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 657 KiB/s wr, 54 op/s 2024-09-10T07:46:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:10 smithi063 ceph-mon[35020]: pgmap v685: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 657 KiB/s wr, 54 op/s 2024-09-10T07:46:10.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:10 smithi046 ceph-mon[30705]: pgmap v685: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 657 KiB/s wr, 54 op/s 2024-09-10T07:46:10.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 530.0 s, 338.9 tps, lat 1527.847 ms stddev 5330.379 2024-09-10T07:46:10.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:46:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:11 smithi181 ceph-mon[35397]: pgmap v686: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 998 KiB/s wr, 99 op/s 2024-09-10T07:46:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:12.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:11 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:11 smithi063 ceph-mon[35020]: pgmap v686: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 998 KiB/s wr, 99 op/s 2024-09-10T07:46:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:12.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:11 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:11 smithi046 ceph-mon[30705]: pgmap v686: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 998 KiB/s wr, 99 op/s 2024-09-10T07:46:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:12.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:11 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.905+0000 7f782a800640 1 -- 172.21.15.46:0/1617333369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824100430 msgr2=0x7f7824100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.905+0000 7f782a800640 1 --2- 172.21.15.46:0/1617333369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824100430 0x7f7824100810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7810009860 tx=0x7f781002a340 comp rx=0 tx=0).stop 2024-09-10T07:46:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.907+0000 7f782a800640 1 -- 172.21.15.46:0/1617333369 shutdown_connections 2024-09-10T07:46:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.907+0000 7f782a800640 1 --2- 172.21.15.46:0/1617333369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7824105560 0x7f782410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.907+0000 7f782a800640 1 --2- 172.21.15.46:0/1617333369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7824100de0 0x7f7824104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.907+0000 7f782a800640 1 --2- 172.21.15.46:0/1617333369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824100430 0x7f7824100810 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.907+0000 7f782a800640 1 -- 172.21.15.46:0/1617333369 >> 172.21.15.46:0/1617333369 conn(0x7f7824077230 msgr2=0x7f78240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.908+0000 7f782a800640 1 -- 172.21.15.46:0/1617333369 shutdown_connections 2024-09-10T07:46:12.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.908+0000 7f782a800640 1 -- 172.21.15.46:0/1617333369 wait complete. 2024-09-10T07:46:12.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.909+0000 7f782a800640 1 Processor -- start 2024-09-10T07:46:12.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.910+0000 7f782a800640 1 -- start start 2024-09-10T07:46:12.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.910+0000 7f782a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7824100430 0x7f782410d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.910+0000 7f782a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824100de0 0x7f782410a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.910+0000 7f782a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7824105560 0x7f782410a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.910+0000 7f782a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7824114120 con 0x7f7824100de0 2024-09-10T07:46:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.910+0000 7f782a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7824113fa0 con 0x7f7824105560 2024-09-10T07:46:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.910+0000 7f782a800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78241142a0 con 0x7f7824100430 2024-09-10T07:46:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.911+0000 7f7823e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7824105560 0x7f782410a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.911+0000 7f7823e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7824105560 0x7f782410a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50638/0 (socket says 172.21.15.46:50638) 2024-09-10T07:46:12.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.911+0000 7f7823e00640 1 -- 172.21.15.46:0/1769462842 learned_addr learned my addr 172.21.15.46:0/1769462842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:12.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.911+0000 7f7822a00640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824100de0 0x7f782410a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.912+0000 7f7823e00640 1 -- 172.21.15.46:0/1769462842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7824100430 msgr2=0x7f782410d030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T07:46:12.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.912+0000 7f7823e00640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7824100430 0x7f782410d030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.912+0000 7f7823e00640 1 -- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824100de0 msgr2=0x7f782410a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:12.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.912+0000 7f7823e00640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824100de0 0x7f782410a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.912+0000 7f7823e00640 1 -- 172.21.15.46:0/1769462842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78241aae40 con 0x7f7824105560 2024-09-10T07:46:12.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.914+0000 7f7823e00640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7824105560 0x7f782410a6d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f781c00afa0 tx=0x7f781c008700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:12.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.915+0000 7f781be00640 1 -- 172.21.15.46:0/1769462842 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c004b20 con 0x7f7824105560 2024-09-10T07:46:12.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.915+0000 7f781be00640 1 -- 172.21.15.46:0/1769462842 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f781c003e50 con 0x7f7824105560 2024-09-10T07:46:12.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.915+0000 7f781be00640 1 -- 172.21.15.46:0/1769462842 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c004b20 con 0x7f7824105560 2024-09-10T07:46:12.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.915+0000 7f782a800640 1 -- 172.21.15.46:0/1769462842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78241ab010 con 0x7f7824105560 2024-09-10T07:46:12.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.915+0000 7f782a800640 1 -- 172.21.15.46:0/1769462842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78241ab410 con 0x7f7824105560 2024-09-10T07:46:12.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.921+0000 7f781be00640 1 -- 172.21.15.46:0/1769462842 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f781c01e020 con 0x7f7824105560 2024-09-10T07:46:12.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.923+0000 7f781be00640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78080a2e00 0x7f78080a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.923+0000 7f781be00640 1 -- 172.21.15.46:0/1769462842 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f781c0ceb20 con 0x7f7824105560 2024-09-10T07:46:12.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.923+0000 7f782d1df740 1 Processor -- start 2024-09-10T07:46:12.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.924+0000 7f782d1df740 1 -- start start 2024-09-10T07:46:12.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.924+0000 7f7823400640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78080a2e00 0x7f78080a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.924+0000 7f782d1df740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f122b90 0x557e6f122f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.924+0000 7f782d1df740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1234b0 0x557e6f12d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.924+0000 7f782d1df740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f12dd70 0x557e6f130160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.924+0000 7f7822a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1234b0 0x557e6f12d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.924+0000 7f7822a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1234b0 0x557e6f12d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39728/0 (socket says 172.21.15.46:39728) 2024-09-10T07:46:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.925+0000 7f7823400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f122b90 0x557e6f122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.925+0000 7f7823400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f122b90 0x557e6f122f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50640/0 (socket says 172.21.15.46:50640) 2024-09-10T07:46:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.925+0000 7f782d1df740 1 -- 172.21.15.46:0/2079926698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557e6f0893e0 con 0x557e6f12dd70 2024-09-10T07:46:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.925+0000 7f7822a00640 1 -- 172.21.15.46:0/2079926698 learned_addr learned my addr 172.21.15.46:0/2079926698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.925+0000 7f782d1df740 1 -- 172.21.15.46:0/2079926698 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557e6f030d80 con 0x557e6f122b90 2024-09-10T07:46:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.925+0000 7f782d1df740 1 -- 172.21.15.46:0/2079926698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557e6f0853a0 con 0x557e6f1234b0 2024-09-10T07:46:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.925+0000 7f7823e00640 1 --2- 172.21.15.46:0/2079926698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f12dd70 0x557e6f130160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.926+0000 7f7823e00640 1 -- 172.21.15.46:0/2079926698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1234b0 msgr2=0x557e6f12d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.926+0000 7f7823e00640 1 --2- 172.21.15.46:0/2079926698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1234b0 0x557e6f12d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.926+0000 7f7823e00640 1 -- 172.21.15.46:0/2079926698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f122b90 msgr2=0x557e6f122f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.926+0000 7f7823e00640 1 --2- 172.21.15.46:0/2079926698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f122b90 0x557e6f122f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.926+0000 7f7823e00640 1 -- 172.21.15.46:0/2079926698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6f04fae0 con 0x557e6f12dd70 2024-09-10T07:46:12.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.926+0000 7f7823e00640 1 --2- 172.21.15.46:0/2079926698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f12dd70 0x557e6f130160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f781c00af70 tx=0x7f781c09aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aff84968eb7c189 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:12.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.926+0000 7f7823400640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78080a2e00 0x7f78080a52c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78100095f0 tx=0x7f7810030d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:12.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.927+0000 7f7818c00640 1 -- 172.21.15.46:0/2079926698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c01b030 con 0x557e6f12dd70 2024-09-10T07:46:12.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.927+0000 7f7818c00640 1 -- 172.21.15.46:0/2079926698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f781c0a3c70 con 0x557e6f12dd70 2024-09-10T07:46:12.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.927+0000 7f7818c00640 1 -- 172.21.15.46:0/2079926698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c097670 con 0x557e6f12dd70 2024-09-10T07:46:12.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.928+0000 7f782d1df740 1 -- 172.21.15.46:0/2079926698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f12dd70 msgr2=0x557e6f130160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:12.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.928+0000 7f782d1df740 1 --2- 172.21.15.46:0/2079926698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f12dd70 0x557e6f130160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f781c00af70 tx=0x7f781c09aa20 comp rx=0 tx=0).stop 2024-09-10T07:46:12.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.928+0000 7f782d1df740 1 -- 172.21.15.46:0/2079926698 shutdown_connections 2024-09-10T07:46:12.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.928+0000 7f782d1df740 1 --2- 172.21.15.46:0/2079926698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f12dd70 0x557e6f130160 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.928+0000 7f782d1df740 1 --2- 172.21.15.46:0/2079926698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1234b0 0x557e6f12d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.928+0000 7f782d1df740 1 --2- 172.21.15.46:0/2079926698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f122b90 0x557e6f122f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.928+0000 7f782d1df740 1 -- 172.21.15.46:0/2079926698 >> 172.21.15.46:0/2079926698 conn(0x557e6f11f120 msgr2=0x557e6f11f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.930+0000 7f782d1df740 1 -- 172.21.15.46:0/2079926698 shutdown_connections 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.930+0000 7f782d1df740 1 -- 172.21.15.46:0/2079926698 wait complete. 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.930+0000 7f782d1df740 1 -- start start 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.931+0000 7f782d1df740 1 Processor -- start 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.932+0000 7f782d1df740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f122b90 0x557e6f12acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.932+0000 7f782d1df740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f12b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.932+0000 7f782d1df740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f12b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.932+0000 7f782d1df740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557e6f1309e0 con 0x557e6f1234b0 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.932+0000 7f782d1df740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557e6f0440f0 con 0x557e6f12dd70 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.932+0000 7f782d1df740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557e6f130b60 con 0x557e6f122b90 2024-09-10T07:46:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.932+0000 7f782d1df740 10 client.0 _ll_register_callbacks cb 0x557e6f0359e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.933+0000 7f7823e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f12b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.933+0000 7f7823e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f12b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50656/0 (socket says 172.21.15.46:50656) 2024-09-10T07:46:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.933+0000 7f7823e00640 1 -- 172.21.15.46:0/2637729161 learned_addr learned my addr 172.21.15.46:0/2637729161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.934+0000 7f7823400640 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f122b90 0x557e6f12acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.933+0000 7f7822a00640 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f12b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.934+0000 7f7823400640 1 -- 172.21.15.46:0/2637729161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 msgr2=0x557e6f12b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.934+0000 7f7823400640 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f12b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.934+0000 7f7823400640 1 -- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 msgr2=0x557e6f12b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.934+0000 7f7823400640 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f12b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.935+0000 7f7823400640 1 -- 172.21.15.46:0/2637729161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6edfa3c0 con 0x557e6f122b90 2024-09-10T07:46:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.935+0000 7f7822a00640 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f12b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.935+0000 7f7823400640 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f122b90 0x557e6f12acd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7810009830 tx=0x7f7810005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e657a52b6ee35d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.935+0000 7f780d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.936+0000 7f780d600640 1 -- 172.21.15.46:0/2637729161 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781004f060 con 0x557e6f122b90 2024-09-10T07:46:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.936+0000 7f780d600640 1 -- 172.21.15.46:0/2637729161 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f78100037f0 con 0x557e6f122b90 2024-09-10T07:46:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.936+0000 7f782d1df740 1 -- 172.21.15.46:0/2637729161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f781c0a3c70 con 0x557e6f122b90 2024-09-10T07:46:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.937+0000 7f780d600640 1 -- 172.21.15.46:0/2637729161 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7810037040 con 0x557e6f122b90 2024-09-10T07:46:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.937+0000 7f782d1df740 10 client.27697 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.937+0000 7f782d1df740 1 -- 172.21.15.46:0/2637729161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557e6f13c760 con 0x557e6f122b90 2024-09-10T07:46:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.937+0000 7f780d600640 1 -- 172.21.15.46:0/2637729161 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f781c0a3c70 con 0x557e6f122b90 2024-09-10T07:46:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.937+0000 7f780d600640 1 -- 172.21.15.46:0/2637729161 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f7810032af0 con 0x557e6f122b90 2024-09-10T07:46:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.938+0000 7f782d1df740 10 client.27697 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.938+0000 7f782d1df740 10 client.27697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.938+0000 7f782d1df740 20 client.27697 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.938+0000 7f782d1df740 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557e6f13db70 0x557e6f13ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.938+0000 7f782d1df740 4 client.27697 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.938+0000 7f782d1df740 1 -- 172.21.15.46:0/2637729161 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e6f096950 con 0x557e6f13db70 2024-09-10T07:46:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.939+0000 7f7822a00640 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557e6f13db70 0x557e6f13ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.941+0000 7f780d600640 10 client.27697 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.941+0000 7f7822a00640 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557e6f13db70 0x557e6f13ffc0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295af52d7fc29a6 server_cookie=294c79759242ea20 in_seq=0 out_seq=0 2024-09-10T07:46:12.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.945+0000 7f780d600640 1 -- 172.21.15.46:0/2637729161 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e6f096950 con 0x557e6f13db70 2024-09-10T07:46:12.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.945+0000 7f780d600640 10 client.27697 handle_command_reply: tid=0 2024-09-10T07:46:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.945+0000 7f782d1df740 1 client.27697 shutdown 2024-09-10T07:46:12.945 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:46:12.945 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:46:12.945 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:12 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f782d1df740 1 -- 172.21.15.46:0/2637729161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f122b90 msgr2=0x557e6f12acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f782d1df740 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f122b90 0x557e6f12acd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7810009830 tx=0x7f7810005f90 comp rx=0 tx=0).stop 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f782d1df740 1 -- 172.21.15.46:0/2637729161 shutdown_connections 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f782d1df740 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557e6f13db70 0x557e6f13ffc0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f782d1df740 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f12b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f782d1df740 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f12b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f782d1df740 1 --2- 172.21.15.46:0/2637729161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f122b90 0x557e6f12acd0 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7810009830 tx=0x7f7810005f90 comp rx=0 tx=0).stop 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f782d1df740 1 -- 172.21.15.46:0/2637729161 >> 172.21.15.46:0/2637729161 conn(0x557e6f11f120 msgr2=0x557e6f124e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.946+0000 7f780d600640 0 client.27697 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:12.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.949+0000 7f782d1df740 1 -- 172.21.15.46:0/2637729161 shutdown_connections 2024-09-10T07:46:12.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.949+0000 7f782d1df740 1 -- 172.21.15.46:0/2637729161 wait complete. 2024-09-10T07:46:12.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.949+0000 7f782d1df740 20 client.27697 trim_cache size 0 max 16384 2024-09-10T07:46:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.984+0000 7f782d1df740 1 Processor -- start 2024-09-10T07:46:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.984+0000 7f782d1df740 1 -- start start 2024-09-10T07:46:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.986+0000 7f782d1df740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f145890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.986+0000 7f782d1df740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.986+0000 7f782d1df740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1245e0 0x557e6f124a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.986+0000 7f782d1df740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78241b53d0 con 0x557e6f1234b0 2024-09-10T07:46:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.986+0000 7f782d1df740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78241b6f10 con 0x557e6f12dd70 2024-09-10T07:46:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.986+0000 7f782d1df740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78241b5aa0 con 0x557e6f1245e0 2024-09-10T07:46:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.987+0000 7f7822a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.987+0000 7f7822a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f145dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50666/0 (socket says 172.21.15.46:50666) 2024-09-10T07:46:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.987+0000 7f7822a00640 1 -- 172.21.15.46:0/2634017204 learned_addr learned my addr 172.21.15.46:0/2634017204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.987+0000 7f7823400640 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.987+0000 7f7823e00640 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1245e0 0x557e6f124a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.989+0000 7f7822a00640 1 -- 172.21.15.46:0/2634017204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1245e0 msgr2=0x557e6f124a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.989+0000 7f7822a00640 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1245e0 0x557e6f124a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.989+0000 7f7822a00640 1 -- 172.21.15.46:0/2634017204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 msgr2=0x557e6f145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.989+0000 7f7822a00640 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f145890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.989+0000 7f7822a00640 1 -- 172.21.15.46:0/2634017204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6f1229c0 con 0x557e6f12dd70 2024-09-10T07:46:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.989+0000 7f7822a00640 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f145dd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7814000c00 tx=0x7f781400f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc08899f5920f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.990+0000 7f780fe00640 1 -- 172.21.15.46:0/2634017204 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781400de60 con 0x557e6f12dd70 2024-09-10T07:46:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.990+0000 7f780fe00640 1 -- 172.21.15.46:0/2634017204 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f781401ad30 con 0x557e6f12dd70 2024-09-10T07:46:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.990+0000 7f780fe00640 1 -- 172.21.15.46:0/2634017204 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78140047a0 con 0x557e6f12dd70 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 -- 172.21.15.46:0/2634017204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 msgr2=0x557e6f145dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f145dd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7814000c00 tx=0x7f781400f040 comp rx=0 tx=0).stop 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 -- 172.21.15.46:0/2634017204 shutdown_connections 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f1245e0 0x557e6f124a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f12dd70 0x557e6f145dd0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 --2- 172.21.15.46:0/2634017204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 -- 172.21.15.46:0/2634017204 >> 172.21.15.46:0/2634017204 conn(0x557e6f11f120 msgr2=0x557e6f129dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 -- 172.21.15.46:0/2634017204 shutdown_connections 2024-09-10T07:46:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.991+0000 7f782d1df740 1 -- 172.21.15.46:0/2634017204 wait complete. 2024-09-10T07:46:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.992+0000 7f782d1df740 1 -- start start 2024-09-10T07:46:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.992+0000 7f782d1df740 1 Processor -- start 2024-09-10T07:46:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f782d1df740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f131db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f782d1df740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f1245e0 0x557e6f1322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f782d1df740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f12dd70 0x557e6f132830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f782d1df740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78241b6840 con 0x557e6f1234b0 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f782d1df740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78241b6170 con 0x557e6f1245e0 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f782d1df740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78241b3370 con 0x557e6f12dd70 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f782d1df740 10 client.0 _ll_register_callbacks cb 0x7f7814005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f7822a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f1245e0 0x557e6f1322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f7822a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f1245e0 0x557e6f1322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50678/0 (socket says 172.21.15.46:50678) 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f7822a00640 1 -- 172.21.15.46:0/3354473901 learned_addr learned my addr 172.21.15.46:0/3354473901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f7823e00640 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f12dd70 0x557e6f132830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.993+0000 7f7823400640 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f131db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.994+0000 7f7823e00640 1 -- 172.21.15.46:0/3354473901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f1245e0 msgr2=0x557e6f1322f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.994+0000 7f7823e00640 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f1245e0 0x557e6f1322f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.994+0000 7f7823e00640 1 -- 172.21.15.46:0/3354473901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 msgr2=0x557e6f131db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.994+0000 7f7823e00640 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f131db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.994+0000 7f7823e00640 1 -- 172.21.15.46:0/3354473901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6f1a7b00 con 0x557e6f12dd70 2024-09-10T07:46:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.995+0000 7f7823e00640 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f12dd70 0x557e6f132830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f781c097460 tx=0x7f781c0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c1894a4732be5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.995+0000 7f780d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.995+0000 7f780d600640 1 -- 172.21.15.46:0/3354473901 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c01b040 con 0x557e6f12dd70 2024-09-10T07:46:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.995+0000 7f782d1df740 1 -- 172.21.15.46:0/3354473901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f781401ad30 con 0x557e6f12dd70 2024-09-10T07:46:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.995+0000 7f780d600640 1 -- 172.21.15.46:0/3354473901 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f781c097670 con 0x557e6f12dd70 2024-09-10T07:46:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.996+0000 7f780d600640 1 -- 172.21.15.46:0/3354473901 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f781c082050 con 0x557e6f12dd70 2024-09-10T07:46:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.996+0000 7f782d1df740 10 client.27703 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.996+0000 7f782d1df740 1 -- 172.21.15.46:0/3354473901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557e6f1a8e40 con 0x557e6f12dd70 2024-09-10T07:46:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.996+0000 7f780d600640 1 -- 172.21.15.46:0/3354473901 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f781401ad30 con 0x557e6f12dd70 2024-09-10T07:46:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.996+0000 7f780d600640 1 -- 172.21.15.46:0/3354473901 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f781c0a37e0 con 0x557e6f12dd70 2024-09-10T07:46:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.997+0000 7f782d1df740 10 client.27703 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.997+0000 7f782d1df740 10 client.27703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.997+0000 7f782d1df740 20 client.27703 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.997+0000 7f782d1df740 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557e6f1a90d0 0x557e6f1ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.997+0000 7f782d1df740 4 client.27703 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:46:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.997+0000 7f782d1df740 1 -- 172.21.15.46:0/3354473901 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e6f090e60 con 0x557e6f1a90d0 2024-09-10T07:46:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.998+0000 7f7823400640 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557e6f1a90d0 0x557e6f1ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.998+0000 7f7823400640 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557e6f1a90d0 0x557e6f1ab4c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00f7faed2d4375f server_cookie=c658b19fb88b528c in_seq=0 out_seq=0 2024-09-10T07:46:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.998+0000 7f780d600640 10 client.27703 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f780d600640 1 -- 172.21.15.46:0/3354473901 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e6f090e60 con 0x557e6f1a90d0 2024-09-10T07:46:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f780d600640 10 client.27703 handle_command_reply: tid=0 2024-09-10T07:46:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 client.27703 shutdown 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 -- 172.21.15.46:0/3354473901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f12dd70 msgr2=0x557e6f132830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f12dd70 0x557e6f132830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f781c097460 tx=0x7f781c0048f0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 -- 172.21.15.46:0/3354473901 shutdown_connections 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557e6f1a90d0 0x557e6f1ab4c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e6f12dd70 0x557e6f132830 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e6f1245e0 0x557e6f1322f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 --2- 172.21.15.46:0/3354473901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6f1234b0 0x557e6f131db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 -- 172.21.15.46:0/3354473901 >> 172.21.15.46:0/3354473901 conn(0x557e6f11f120 msgr2=0x557e6f129dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f780d600640 0 client.27703 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:12.999+0000 7f782d1df740 1 -- 172.21.15.46:0/3354473901 shutdown_connections 2024-09-10T07:46:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.000+0000 7f782d1df740 1 -- 172.21.15.46:0/3354473901 wait complete. 2024-09-10T07:46:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.000+0000 7f782d1df740 20 client.27703 trim_cache size 0 max 16384 2024-09-10T07:46:13.000 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:46:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 -- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78080a2e00 msgr2=0x7f78080a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78080a2e00 0x7f78080a52c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78100095f0 tx=0x7f7810030d60 comp rx=0 tx=0).stop 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 -- 172.21.15.46:0/1769462842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7824105560 msgr2=0x7f782410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7824105560 0x7f782410a6d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f781c00afa0 tx=0x7f781c008700 comp rx=0 tx=0).stop 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 -- 172.21.15.46:0/1769462842 shutdown_connections 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7824105560 0x7f782410a6d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78080a2e00 0x7f78080a52c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7824100de0 0x7f782410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 --2- 172.21.15.46:0/1769462842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7824100430 0x7f782410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.001+0000 7f782a800640 1 -- 172.21.15.46:0/1769462842 >> 172.21.15.46:0/1769462842 conn(0x7f7824077230 msgr2=0x7f78240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.002+0000 7f782a800640 1 -- 172.21.15.46:0/1769462842 shutdown_connections 2024-09-10T07:46:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:13.002+0000 7f782a800640 1 -- 172.21.15.46:0/1769462842 wait complete. 2024-09-10T07:46:13.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:46:13.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:46:13.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:12 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:46:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:46:13.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:12 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:46:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.402+0000 7f962ea00640 1 -- 172.21.15.46:0/3451041684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96280ffe10 msgr2=0x7f9628100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.402+0000 7f962ea00640 1 --2- 172.21.15.46:0/3451041684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96280ffe10 0x7f9628100210 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9618009900 tx=0x7f961802a400 comp rx=0 tx=0).stop 2024-09-10T07:46:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.420+0000 7f962ea00640 1 -- 172.21.15.46:0/3451041684 shutdown_connections 2024-09-10T07:46:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.420+0000 7f962ea00640 1 --2- 172.21.15.46:0/3451041684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96281037a0 0x7f962810a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.420+0000 7f962ea00640 1 --2- 172.21.15.46:0/3451041684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9628102de0 0x7f9628103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.420+0000 7f962ea00640 1 --2- 172.21.15.46:0/3451041684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96280ffe10 0x7f9628100210 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.420+0000 7f962ea00640 1 -- 172.21.15.46:0/3451041684 >> 172.21.15.46:0/3451041684 conn(0x7f96280fb930 msgr2=0x7f96280fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.421+0000 7f962ea00640 1 -- 172.21.15.46:0/3451041684 shutdown_connections 2024-09-10T07:46:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.421+0000 7f962ea00640 1 -- 172.21.15.46:0/3451041684 wait complete. 2024-09-10T07:46:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.422+0000 7f962ea00640 1 Processor -- start 2024-09-10T07:46:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.424+0000 7f962ea00640 1 -- start start 2024-09-10T07:46:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.424+0000 7f962ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96280ffe10 0x7f962806bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.424+0000 7f962ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9628102de0 0x7f962806c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.424+0000 7f962ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96281037a0 0x7f9628073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.425+0000 7f9627e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96280ffe10 0x7f962806bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.425+0000 7f9627e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96280ffe10 0x7f962806bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34626/0 (socket says 172.21.15.46:34626) 2024-09-10T07:46:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.425+0000 7f962ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96281037a0 0x7f9628073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.425+0000 7f962ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96281037a0 0x7f9628073540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50698/0 (socket says 172.21.15.46:50698) 2024-09-10T07:46:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.426+0000 7f962ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9628101c80 con 0x7f96280ffe10 2024-09-10T07:46:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.426+0000 7f962ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9628101b00 con 0x7f96281037a0 2024-09-10T07:46:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.426+0000 7f9627400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9628102de0 0x7f962806c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.426+0000 7f9627400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9628102de0 0x7f962806c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39784/0 (socket says 172.21.15.46:39784) 2024-09-10T07:46:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.426+0000 7f9627400640 1 -- 172.21.15.46:0/602356448 learned_addr learned my addr 172.21.15.46:0/602356448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.427+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9628101e00 con 0x7f9628102de0 2024-09-10T07:46:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.427+0000 7f9627400640 1 -- 172.21.15.46:0/602356448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96281037a0 msgr2=0x7f9628073540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.427+0000 7f9627400640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96281037a0 0x7f9628073540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.427+0000 7f9627400640 1 -- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96280ffe10 msgr2=0x7f962806bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.427+0000 7f9627400640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96280ffe10 0x7f962806bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.427+0000 7f9627400640 1 -- 172.21.15.46:0/602356448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9628073bb0 con 0x7f9628102de0 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.428+0000 7f9627400640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9628102de0 0x7f962806c4d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9f0 tx=0x7f961400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.429+0000 7f9624c00640 1 -- 172.21.15.46:0/602356448 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400cd80 con 0x7f9628102de0 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.429+0000 7f9624c00640 1 -- 172.21.15.46:0/602356448 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9614010910 con 0x7f9628102de0 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.429+0000 7f9624c00640 1 -- 172.21.15.46:0/602356448 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614018990 con 0x7f9628102de0 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.429+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9628073ea0 con 0x7f9628102de0 2024-09-10T07:46:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.429+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281af120 con 0x7f9628102de0 2024-09-10T07:46:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.435+0000 7f96317f1740 1 Processor -- start 2024-09-10T07:46:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.437+0000 7f9624c00640 1 -- 172.21.15.46:0/602356448 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9614018b30 con 0x7f9628102de0 2024-09-10T07:46:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.438+0000 7f96317f1740 1 -- start start 2024-09-10T07:46:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f96317f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee70a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f96317f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee70f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f96317f1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee711f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f96317f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee66f100 con 0x55d5ee70a9d0 2024-09-10T07:46:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f96317f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee6171b0 con 0x55d5ee70a090 2024-09-10T07:46:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f96317f1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee66b0c0 con 0x55d5ee70fa60 2024-09-10T07:46:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f962ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee70a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f962ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee70a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50702/0 (socket says 172.21.15.46:50702) 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f962ca00640 1 -- 172.21.15.46:0/292184540 learned_addr learned my addr 172.21.15.46:0/292184540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f9627400640 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee711f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f9627e00640 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee70f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.439+0000 7f9624c00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f960c0a2cb0 0x7f960c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.440+0000 7f962ca00640 1 -- 172.21.15.46:0/292184540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 msgr2=0x55d5ee711f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.440+0000 7f962ca00640 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee711f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.440+0000 7f962ca00640 1 -- 172.21.15.46:0/292184540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 msgr2=0x55d5ee70f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.440+0000 7f962ca00640 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee70f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.440+0000 7f962ca00640 1 -- 172.21.15.46:0/292184540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ee382210 con 0x55d5ee70a090 2024-09-10T07:46:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.440+0000 7f9627e00640 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee70f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.442+0000 7f962ca00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f960c0a2cb0 0x7f960c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.442+0000 7f962ca00640 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee70a490 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f962000a5d0 tx=0x7f962000aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c5df0eb45729283 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.442+0000 7f9624c00640 1 -- 172.21.15.46:0/602356448 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f96140cf7a0 con 0x7f9628102de0 2024-09-10T07:46:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.443+0000 7f961d200640 1 -- 172.21.15.46:0/292184540 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620016a10 con 0x55d5ee70a090 2024-09-10T07:46:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.443+0000 7f961d200640 1 -- 172.21.15.46:0/292184540 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9620016bb0 con 0x55d5ee70a090 2024-09-10T07:46:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.443+0000 7f961d200640 1 -- 172.21.15.46:0/292184540 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620015660 con 0x55d5ee70a090 2024-09-10T07:46:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.444+0000 7f96317f1740 1 -- 172.21.15.46:0/292184540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 msgr2=0x55d5ee70a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.444+0000 7f96317f1740 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee70a490 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f962000a5d0 tx=0x7f962000aa90 comp rx=0 tx=0).stop 2024-09-10T07:46:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.444+0000 7f962ca00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f960c0a2cb0 0x7f960c0a5170 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f962001e000 tx=0x7f96200158e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.447+0000 7f96317f1740 1 -- 172.21.15.46:0/292184540 shutdown_connections 2024-09-10T07:46:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.447+0000 7f96317f1740 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee711f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.447+0000 7f96317f1740 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee70f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.447+0000 7f96317f1740 1 --2- 172.21.15.46:0/292184540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee70a490 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.447+0000 7f96317f1740 1 -- 172.21.15.46:0/292184540 >> 172.21.15.46:0/292184540 conn(0x55d5ee705700 msgr2=0x55d5ee705b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.448+0000 7f96317f1740 1 -- 172.21.15.46:0/292184540 shutdown_connections 2024-09-10T07:46:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.448+0000 7f96317f1740 1 -- 172.21.15.46:0/292184540 wait complete. 2024-09-10T07:46:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.449+0000 7f96317f1740 1 -- start start 2024-09-10T07:46:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.449+0000 7f96317f1740 1 Processor -- start 2024-09-10T07:46:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.450+0000 7f96317f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee714250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.450+0000 7f96317f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee714790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f96317f1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee71b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f96317f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee712a30 con 0x55d5ee70a9d0 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f96317f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee5f8090 con 0x55d5ee70a090 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f96317f1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d5ee712bb0 con 0x55d5ee70fa60 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f9627e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee714250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f9627e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee714250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50718/0 (socket says 172.21.15.46:50718) 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f9627e00640 1 -- 172.21.15.46:0/2047573352 learned_addr learned my addr 172.21.15.46:0/2047573352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f96317f1740 10 client.0 _ll_register_callbacks cb 0x55d5ee64cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f962ca00640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee71b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.451+0000 7f9627400640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee714790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f962ca00640 1 -- 172.21.15.46:0/2047573352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 msgr2=0x55d5ee714250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f962ca00640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee714250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f962ca00640 1 -- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 msgr2=0x55d5ee714790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f962ca00640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee714790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f962ca00640 1 -- 172.21.15.46:0/2047573352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ee44e620 con 0x55d5ee70fa60 2024-09-10T07:46:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f9627400640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee714790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f9627e00640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee714250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f962ca00640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee71b7d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f962000a5a0 tx=0x7f9620017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac9f5e43f350c9ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.452+0000 7f9611600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.453+0000 7f9611600640 1 -- 172.21.15.46:0/2047573352 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620015eb0 con 0x55d5ee70fa60 2024-09-10T07:46:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.453+0000 7f9611600640 1 -- 172.21.15.46:0/2047573352 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9620016810 con 0x55d5ee70fa60 2024-09-10T07:46:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.454+0000 7f96317f1740 1 -- 172.21.15.46:0/2047573352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9620016bb0 con 0x55d5ee70fa60 2024-09-10T07:46:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.454+0000 7f9611600640 1 -- 172.21.15.46:0/2047573352 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9620016a10 con 0x55d5ee70fa60 2024-09-10T07:46:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.454+0000 7f96317f1740 10 client.27715 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.454+0000 7f96317f1740 1 -- 172.21.15.46:0/2047573352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d5ee71e5a0 con 0x55d5ee70fa60 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.454+0000 7f9611600640 1 -- 172.21.15.46:0/2047573352 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9620016bb0 con 0x55d5ee70fa60 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.455+0000 7f9611600640 1 -- 172.21.15.46:0/2047573352 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f9620015380 con 0x55d5ee70fa60 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.455+0000 7f96317f1740 10 client.27715 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.455+0000 7f96317f1740 10 client.27715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.455+0000 7f96317f1740 20 client.27715 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.455+0000 7f96317f1740 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5ee71fb30 0x55d5ee721fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.455+0000 7f96317f1740 4 client.27715 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.455+0000 7f96317f1740 1 -- 172.21.15.46:0/2047573352 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5ee67c830 con 0x55d5ee71fb30 2024-09-10T07:46:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.456+0000 7f9627e00640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5ee71fb30 0x55d5ee721fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.460+0000 7f9627e00640 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5ee71fb30 0x55d5ee721fd0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520b9ba3a8010284 server_cookie=fc2f75b1aef5f045 in_seq=0 out_seq=0 2024-09-10T07:46:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.460+0000 7f9611600640 10 client.27715 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f9611600640 1 -- 172.21.15.46:0/2047573352 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5ee67c830 con 0x55d5ee71fb30 2024-09-10T07:46:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f9611600640 10 client.27715 handle_command_reply: tid=0 2024-09-10T07:46:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 client.27715 shutdown 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 -- 172.21.15.46:0/2047573352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 msgr2=0x55d5ee71b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee71b7d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f962000a5a0 tx=0x7f9620017000 comp rx=0 tx=0).stop 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 -- 172.21.15.46:0/2047573352 shutdown_connections 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5ee71fb30 0x55d5ee721fd0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee71b7d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a9d0 0x55d5ee714790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 --2- 172.21.15.46:0/2047573352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a090 0x55d5ee714250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f9611600640 0 client.27715 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.464+0000 7f96317f1740 1 -- 172.21.15.46:0/2047573352 >> 172.21.15.46:0/2047573352 conn(0x55d5ee705700 msgr2=0x55d5ee706f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.465+0000 7f96317f1740 1 -- 172.21.15.46:0/2047573352 shutdown_connections 2024-09-10T07:46:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.465+0000 7f96317f1740 1 -- 172.21.15.46:0/2047573352 wait complete. 2024-09-10T07:46:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.465+0000 7f96317f1740 20 client.27715 trim_cache size 0 max 16384 2024-09-10T07:46:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.507+0000 7f96317f1740 1 Processor -- start 2024-09-10T07:46:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.507+0000 7f96317f1740 1 -- start start 2024-09-10T07:46:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.508+0000 7f96317f1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70a090 0x55d5ee70a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.508+0000 7f96317f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee71b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.508+0000 7f9627e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70a090 0x55d5ee70a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.508+0000 7f9627e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70a090 0x55d5ee70a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39796/0 (socket says 172.21.15.46:39796) 2024-09-10T07:46:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.508+0000 7f96317f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70fa60 0x55d5ee704770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.508+0000 7f96317f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96281b9620 con 0x55d5ee70fa60 2024-09-10T07:46:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.508+0000 7f96317f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96281bb160 con 0x55d5ee70a9d0 2024-09-10T07:46:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.508+0000 7f96317f1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96281b9cf0 con 0x55d5ee70a090 2024-09-10T07:46:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.509+0000 7f9627e00640 1 -- 172.21.15.46:0/3913342809 learned_addr learned my addr 172.21.15.46:0/3913342809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.509+0000 7f962ca00640 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70fa60 0x55d5ee704770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.509+0000 7f9627e00640 1 -- 172.21.15.46:0/3913342809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 msgr2=0x55d5ee71b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.509+0000 7f9627e00640 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee71b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.509+0000 7f9627e00640 1 -- 172.21.15.46:0/3913342809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70fa60 msgr2=0x55d5ee704770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.509+0000 7f9627e00640 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70fa60 0x55d5ee704770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.509+0000 7f9627e00640 1 -- 172.21.15.46:0/3913342809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ee705520 con 0x55d5ee70a090 2024-09-10T07:46:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.509+0000 7f962ca00640 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70fa60 0x55d5ee704770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.510+0000 7f9627e00640 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70a090 0x55d5ee70a470 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f961802a920 tx=0x7f96180315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d74771afc9f9b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.510+0000 7f9613e00640 1 -- 172.21.15.46:0/3913342809 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618035040 con 0x55d5ee70a090 2024-09-10T07:46:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.510+0000 7f9613e00640 1 -- 172.21.15.46:0/3913342809 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9618031b90 con 0x55d5ee70a090 2024-09-10T07:46:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.510+0000 7f9613e00640 1 -- 172.21.15.46:0/3913342809 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618004d30 con 0x55d5ee70a090 2024-09-10T07:46:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.511+0000 7f96317f1740 1 -- 172.21.15.46:0/3913342809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70a090 msgr2=0x55d5ee70a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.511+0000 7f96317f1740 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70a090 0x55d5ee70a470 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f961802a920 tx=0x7f96180315b0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.512+0000 7f96317f1740 1 -- 172.21.15.46:0/3913342809 shutdown_connections 2024-09-10T07:46:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.512+0000 7f96317f1740 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70fa60 0x55d5ee704770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.512+0000 7f96317f1740 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee71b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.512+0000 7f96317f1740 1 --2- 172.21.15.46:0/3913342809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70a090 0x55d5ee70a470 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.512+0000 7f96317f1740 1 -- 172.21.15.46:0/3913342809 >> 172.21.15.46:0/3913342809 conn(0x55d5ee705700 msgr2=0x55d5ee704060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.512+0000 7f96317f1740 1 -- 172.21.15.46:0/3913342809 shutdown_connections 2024-09-10T07:46:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.512+0000 7f96317f1740 1 -- 172.21.15.46:0/3913342809 wait complete. 2024-09-10T07:46:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.513+0000 7f96317f1740 1 -- start start 2024-09-10T07:46:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.514+0000 7f96317f1740 1 Processor -- start 2024-09-10T07:46:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.514+0000 7f96317f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a090 0x55d5ee71b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f96317f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee717f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f96317f1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee718480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f9627400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee717f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f9627400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee717f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50722/0 (socket says 172.21.15.46:50722) 2024-09-10T07:46:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f96317f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96281baa90 con 0x55d5ee70a090 2024-09-10T07:46:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f96317f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96281ba3c0 con 0x55d5ee70a9d0 2024-09-10T07:46:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f96317f1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96281b7600 con 0x55d5ee70fa60 2024-09-10T07:46:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f9627400640 1 -- 172.21.15.46:0/1690596651 learned_addr learned my addr 172.21.15.46:0/1690596651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.515+0000 7f96317f1740 10 client.0 _ll_register_callbacks cb 0x7f9614097cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.516+0000 7f962ca00640 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee718480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.516+0000 7f9627400640 1 -- 172.21.15.46:0/1690596651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 msgr2=0x55d5ee718480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.516+0000 7f9627400640 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee718480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.516+0000 7f9627400640 1 -- 172.21.15.46:0/1690596651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a090 msgr2=0x55d5ee71b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.516+0000 7f9627400640 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a090 0x55d5ee71b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.516+0000 7f9627400640 1 -- 172.21.15.46:0/1690596651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ee718b20 con 0x55d5ee70a9d0 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.517+0000 7f9611600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.517+0000 7f9627400640 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee717f40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f96140ccec0 tx=0x7f9614005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=838b2fb13fa5aea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.517+0000 7f9611600640 1 -- 172.21.15.46:0/1690596651 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140d5020 con 0x55d5ee70a9d0 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.517+0000 7f96317f1740 1 -- 172.21.15.46:0/1690596651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9618031b90 con 0x55d5ee70a9d0 2024-09-10T07:46:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.517+0000 7f9611600640 1 -- 172.21.15.46:0/1690596651 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9614097970 con 0x55d5ee70a9d0 2024-09-10T07:46:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.518+0000 7f9611600640 1 -- 172.21.15.46:0/1690596651 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f96140d5020 con 0x55d5ee70a9d0 2024-09-10T07:46:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.518+0000 7f96317f1740 10 client.27767 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.518+0000 7f96317f1740 1 -- 172.21.15.46:0/1690596651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d5ee789a70 con 0x55d5ee70a9d0 2024-09-10T07:46:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.518+0000 7f9611600640 1 -- 172.21.15.46:0/1690596651 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9618031b90 con 0x55d5ee70a9d0 2024-09-10T07:46:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.520+0000 7f9611600640 1 -- 172.21.15.46:0/1690596651 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f96140cfb90 con 0x55d5ee70a9d0 2024-09-10T07:46:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.520+0000 7f96317f1740 10 client.27767 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.520+0000 7f96317f1740 10 client.27767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.520+0000 7f96317f1740 20 client.27767 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.520+0000 7f96317f1740 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5ee78a380 0x55d5ee78c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.520+0000 7f96317f1740 4 client.27767 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:46:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.520+0000 7f96317f1740 1 -- 172.21.15.46:0/1690596651 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5ee676b80 con 0x55d5ee78a380 2024-09-10T07:46:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.524+0000 7f9627e00640 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5ee78a380 0x55d5ee78c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.525+0000 7f9627e00640 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5ee78a380 0x55d5ee78c7a0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9714bb3747f7762 server_cookie=dc0c0b35453b102b in_seq=0 out_seq=0 2024-09-10T07:46:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.525+0000 7f9611600640 10 client.27767 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.565+0000 7f9611600640 1 -- 172.21.15.46:0/1690596651 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5ee676b80 con 0x55d5ee78a380 2024-09-10T07:46:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.565+0000 7f9611600640 10 client.27767 handle_command_reply: tid=0 2024-09-10T07:46:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 client.27767 shutdown 2024-09-10T07:46:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 -- 172.21.15.46:0/1690596651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 msgr2=0x55d5ee717f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee717f40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f96140ccec0 tx=0x7f9614005f10 comp rx=0 tx=0).stop 2024-09-10T07:46:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 -- 172.21.15.46:0/1690596651 shutdown_connections 2024-09-10T07:46:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d5ee78a380 0x55d5ee78c7a0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5ee70fa60 0x55d5ee718480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5ee70a9d0 0x55d5ee717f40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 --2- 172.21.15.46:0/1690596651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5ee70a090 0x55d5ee71b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f96317f1740 1 -- 172.21.15.46:0/1690596651 >> 172.21.15.46:0/1690596651 conn(0x55d5ee705700 msgr2=0x55d5ee704060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.568+0000 7f9611600640 0 client.27767 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.569+0000 7f96317f1740 1 -- 172.21.15.46:0/1690596651 shutdown_connections 2024-09-10T07:46:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.569+0000 7f96317f1740 1 -- 172.21.15.46:0/1690596651 wait complete. 2024-09-10T07:46:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.569+0000 7f96317f1740 20 client.27767 trim_cache size 0 max 16384 2024-09-10T07:46:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:46:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:46:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:46:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:46:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.571+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f960c0a2cb0 msgr2=0x7f960c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.571+0000 7f962ea00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f960c0a2cb0 0x7f960c0a5170 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f962001e000 tx=0x7f96200158e0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.571+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9628102de0 msgr2=0x7f962806c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.571+0000 7f962ea00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9628102de0 0x7f962806c4d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9f0 tx=0x7f961400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.573+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 shutdown_connections 2024-09-10T07:46:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.573+0000 7f962ea00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96281037a0 0x7f9628073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.573+0000 7f962ea00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9628102de0 0x7f962806c4d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.573+0000 7f962ea00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f960c0a2cb0 0x7f960c0a5170 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.573+0000 7f962ea00640 1 --2- 172.21.15.46:0/602356448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96280ffe10 0x7f962806bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.573+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 >> 172.21.15.46:0/602356448 conn(0x7f96280fb930 msgr2=0x7f96280fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.573+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 shutdown_connections 2024-09-10T07:46:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:13.575+0000 7f962ea00640 1 -- 172.21.15.46:0/602356448 wait complete. 2024-09-10T07:46:14.006 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:46:14.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:46:14.122 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:13 smithi046 ceph-mon[30705]: pgmap v687: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2024-09-10T07:46:14.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:13 smithi181 ceph-mon[35397]: pgmap v687: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2024-09-10T07:46:14.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:13 smithi063 ceph-mon[35020]: pgmap v687: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2024-09-10T07:46:14.739 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 535.0 s, 625.2 tps, lat 166.512 ms stddev 241.179 2024-09-10T07:46:15.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.771+0000 7fec3b200640 1 -- 172.21.15.46:0/3070913757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec34103ff0 msgr2=0x7fec341043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:15.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.771+0000 7fec3b200640 1 --2- 172.21.15.46:0/3070913757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec34103ff0 0x7fec341043d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009be0 tx=0x7fec2c02a710 comp rx=0 tx=0).stop 2024-09-10T07:46:15.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.771+0000 7fec3b200640 1 -- 172.21.15.46:0/3070913757 shutdown_connections 2024-09-10T07:46:15.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.771+0000 7fec3b200640 1 --2- 172.21.15.46:0/3070913757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34109120 0x7fec34110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:15.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.771+0000 7fec3b200640 1 --2- 172.21.15.46:0/3070913757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec341049a0 0x7fec341089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:15.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.771+0000 7fec3b200640 1 --2- 172.21.15.46:0/3070913757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec34103ff0 0x7fec341043d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:15.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.771+0000 7fec3b200640 1 -- 172.21.15.46:0/3070913757 >> 172.21.15.46:0/3070913757 conn(0x7fec340ffab0 msgr2=0x7fec34101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:15.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.772+0000 7fec3b200640 1 -- 172.21.15.46:0/3070913757 shutdown_connections 2024-09-10T07:46:15.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.772+0000 7fec3b200640 1 -- 172.21.15.46:0/3070913757 wait complete. 2024-09-10T07:46:15.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.774+0000 7fec3b200640 1 Processor -- start 2024-09-10T07:46:15.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.775+0000 7fec3b200640 1 -- start start 2024-09-10T07:46:15.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.779+0000 7fec3b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34103ff0 0x7fec3419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:15.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.779+0000 7fec3b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec341049a0 0x7fec3419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:15.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.780+0000 7fec33e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34103ff0 0x7fec3419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:15.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.780+0000 7fec33e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34103ff0 0x7fec3419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:39816/0 (socket says 172.21.15.46:39816) 2024-09-10T07:46:15.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.780+0000 7fec33400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec341049a0 0x7fec3419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:15.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.780+0000 7fec33400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec341049a0 0x7fec3419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:50730/0 (socket says 172.21.15.46:50730) 2024-09-10T07:46:15.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.780+0000 7fec33e00640 1 -- 172.21.15.46:0/3398137462 learned_addr learned my addr 172.21.15.46:0/3398137462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:15.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.781+0000 7fec3b200640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec34109120 0x7fec3419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:15.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.781+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fec34114170 con 0x7fec34109120 2024-09-10T07:46:15.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.781+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec34113ff0 con 0x7fec341049a0 2024-09-10T07:46:15.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.781+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fec341142f0 con 0x7fec34103ff0 2024-09-10T07:46:15.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.782+0000 7fec39200640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec34109120 0x7fec3419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:15.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.783+0000 7fec33400640 1 -- 172.21.15.46:0/3398137462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34103ff0 msgr2=0x7fec3419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:15.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.783+0000 7fec33400640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34103ff0 0x7fec3419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:15.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.783+0000 7fec33400640 1 -- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec34109120 msgr2=0x7fec3419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:15.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.783+0000 7fec33400640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec34109120 0x7fec3419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:15.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.783+0000 7fec33400640 1 -- 172.21.15.46:0/3398137462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3419f6e0 con 0x7fec341049a0 2024-09-10T07:46:15.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.785+0000 7fec33e00640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34103ff0 0x7fec3419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:15.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.785+0000 7fec33400640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec341049a0 0x7fec3419ac70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fec20005fd0 tx=0x7fec2000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:15.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.785+0000 7fec30c00640 1 -- 172.21.15.46:0/3398137462 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2000cd00 con 0x7fec341049a0 2024-09-10T07:46:15.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.786+0000 7fec30c00640 1 -- 172.21.15.46:0/3398137462 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec200035f0 con 0x7fec341049a0 2024-09-10T07:46:15.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.786+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3419f9d0 con 0x7fec341049a0 2024-09-10T07:46:15.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.786+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec341a72b0 con 0x7fec341049a0 2024-09-10T07:46:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.787+0000 7fec30c00640 1 -- 172.21.15.46:0/3398137462 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20010670 con 0x7fec341049a0 2024-09-10T07:46:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.788+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec14005290 con 0x7fec341049a0 2024-09-10T07:46:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.792+0000 7fec30c00640 1 -- 172.21.15.46:0/3398137462 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec20043ae0 con 0x7fec341049a0 2024-09-10T07:46:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.795+0000 7fec30c00640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec180a2d90 0x7fec180a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.796+0000 7fec30c00640 1 -- 172.21.15.46:0/3398137462 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fec200cf3d0 con 0x7fec341049a0 2024-09-10T07:46:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.796+0000 7fec30c00640 1 -- 172.21.15.46:0/3398137462 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec20097f40 con 0x7fec341049a0 2024-09-10T07:46:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.796+0000 7fec33e00640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec180a2d90 0x7fec180a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:15.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:15.800+0000 7fec33e00640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec180a2d90 0x7fec180a5250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fec2c006010 tx=0x7fec2c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:16.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:15 smithi181 ceph-mon[35397]: pgmap v688: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-09-10T07:46:16.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:15 smithi063 ceph-mon[35020]: pgmap v688: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-09-10T07:46:16.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:15 smithi046 ceph-mon[30705]: pgmap v688: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-09-10T07:46:16.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.219+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec14005580 con 0x7fec341049a0 2024-09-10T07:46:16.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.219+0000 7fec30c00640 1 -- 172.21.15.46:0/3398137462 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11972 (secure 0 0 0) 0x7fec2009bfd0 con 0x7fec341049a0 2024-09-10T07:46:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:46:16.220 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"btime":"2024-09-10T07:45:56:057049+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:45:55.519456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:46:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 48 2024-09-10T07:46:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec180a2d90 msgr2=0x7fec180a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec180a2d90 0x7fec180a5250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fec2c006010 tx=0x7fec2c000f40 comp rx=0 tx=0).stop 2024-09-10T07:46:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec341049a0 msgr2=0x7fec3419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec341049a0 0x7fec3419ac70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fec20005fd0 tx=0x7fec2000ee50 comp rx=0 tx=0).stop 2024-09-10T07:46:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 shutdown_connections 2024-09-10T07:46:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec180a2d90 0x7fec180a5250 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec34109120 0x7fec3419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec341049a0 0x7fec3419ac70 secure :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fec20005fd0 tx=0x7fec2000ee50 comp rx=0 tx=0).stop 2024-09-10T07:46:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 --2- 172.21.15.46:0/3398137462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34103ff0 0x7fec3419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.223+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 >> 172.21.15.46:0/3398137462 conn(0x7fec340ffab0 msgr2=0x7fec34100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.224+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 shutdown_connections 2024-09-10T07:46:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:16.226+0000 7fec3b200640 1 -- 172.21.15.46:0/3398137462 wait complete. 2024-09-10T07:46:17.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:16 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3398137462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:17.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:16 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3398137462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:17.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:16 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3398137462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:17.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:46:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:18.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:17 smithi181 ceph-mon[35397]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-09-10T07:46:18.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:17 smithi063 ceph-mon[35020]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-09-10T07:46:18.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:17 smithi046 ceph-mon[30705]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-09-10T07:46:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.787+0000 7f2c0d600640 1 -- 172.21.15.46:0/399786768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c081058d0 msgr2=0x7f2c08114a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.787+0000 7f2c0d600640 1 --2- 172.21.15.46:0/399786768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c081058d0 0x7f2c08114a60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009be0 tx=0x7f2c0002a750 comp rx=0 tx=0).stop 2024-09-10T07:46:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.789+0000 7f2c0d600640 1 -- 172.21.15.46:0/399786768 shutdown_connections 2024-09-10T07:46:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.789+0000 7f2c0d600640 1 --2- 172.21.15.46:0/399786768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c081058d0 0x7f2c08114a60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.789+0000 7f2c0d600640 1 --2- 172.21.15.46:0/399786768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c08074ee0 0x7f2c08112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.789+0000 7f2c0d600640 1 --2- 172.21.15.46:0/399786768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c080745c0 0x7f2c080749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.789+0000 7f2c0d600640 1 -- 172.21.15.46:0/399786768 >> 172.21.15.46:0/399786768 conn(0x7f2c080fd740 msgr2=0x7f2c080ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.790+0000 7f2c0d600640 1 -- 172.21.15.46:0/399786768 shutdown_connections 2024-09-10T07:46:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.790+0000 7f2c0d600640 1 -- 172.21.15.46:0/399786768 wait complete. 2024-09-10T07:46:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.792+0000 7f2c0d600640 1 Processor -- start 2024-09-10T07:46:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.793+0000 7f2c0d600640 1 -- start start 2024-09-10T07:46:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.795+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c080745c0 0x7f2c08073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.796+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c08074ee0 0x7f2c08073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.796+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c0806cb00 0x7f2c0806cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c07400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c08074ee0 0x7f2c08073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c07400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c08074ee0 0x7f2c08073ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:36756/0 (socket says 172.21.15.46:36756) 2024-09-10T07:46:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c080745c0 0x7f2c08073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c080745c0 0x7f2c08073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58038/0 (socket says 172.21.15.46:58038) 2024-09-10T07:46:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c07e00640 1 -- 172.21.15.46:0/1859558471 learned_addr learned my addr 172.21.15.46:0/1859558471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081183a0 con 0x7f2c080745c0 2024-09-10T07:46:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08118220 con 0x7f2c0806cb00 2024-09-10T07:46:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08118520 con 0x7f2c08074ee0 2024-09-10T07:46:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.797+0000 7f2c0cc00640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c0806cb00 0x7f2c0806cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.798+0000 7f2c07400640 1 -- 172.21.15.46:0/1859558471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c0806cb00 msgr2=0x7f2c0806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.798+0000 7f2c07400640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c0806cb00 0x7f2c0806cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.798+0000 7f2c07400640 1 -- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c080745c0 msgr2=0x7f2c08073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.798+0000 7f2c07400640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c080745c0 0x7f2c08073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.798+0000 7f2c07400640 1 -- 172.21.15.46:0/1859558471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c080714b0 con 0x7f2c08074ee0 2024-09-10T07:46:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.799+0000 7f2c07400640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c08074ee0 0x7f2c08073ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400e790 tx=0x7f2bf400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.800+0000 7f2c04c00640 1 -- 172.21.15.46:0/1859558471 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf400eea0 con 0x7f2c08074ee0 2024-09-10T07:46:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.800+0000 7f2c04c00640 1 -- 172.21.15.46:0/1859558471 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf400ccc0 con 0x7f2c08074ee0 2024-09-10T07:46:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.800+0000 7f2c04c00640 1 -- 172.21.15.46:0/1859558471 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4018750 con 0x7f2c08074ee0 2024-09-10T07:46:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.800+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c080717a0 con 0x7f2c08074ee0 2024-09-10T07:46:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.800+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c08071c90 con 0x7f2c08074ee0 2024-09-10T07:46:18.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.806+0000 7f2c04c00640 1 -- 172.21.15.46:0/1859558471 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2bf40188f0 con 0x7f2c08074ee0 2024-09-10T07:46:18.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.809+0000 7f2c11695740 1 Processor -- start 2024-09-10T07:46:18.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.813+0000 7f2c04c00640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2bec0a2d10 0x7f2bec0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.814+0000 7f2c04c00640 1 -- 172.21.15.46:0/1859558471 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2bf40cf350 con 0x7f2c08074ee0 2024-09-10T07:46:18.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.816+0000 7f2c0cc00640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2bec0a2d10 0x7f2bec0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.817+0000 7f2c11695740 1 -- start start 2024-09-10T07:46:18.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.817+0000 7f2c0cc00640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2bec0a2d10 0x7f2bec0a51d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c00006010 tx=0x7f2c00009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:18.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.817+0000 7f2c11695740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b222d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.817+0000 7f2c11695740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2237ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.817+0000 7f2c11695740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b223a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.818+0000 7f2c07e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b222d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.819+0000 7f2c07e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b222d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:36766/0 (socket says 172.21.15.46:36766) 2024-09-10T07:46:18.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.819+0000 7f2c07400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2237ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.819+0000 7f2c07400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2237ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38858/0 (socket says 172.21.15.46:38858) 2024-09-10T07:46:18.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.820+0000 7f2c11695740 1 -- 172.21.15.46:0/1715379097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5625b21933e0 con 0x5625b2238020 2024-09-10T07:46:18.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.820+0000 7f2c11695740 1 -- 172.21.15.46:0/1715379097 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625b213ad80 con 0x5625b222d760 2024-09-10T07:46:18.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.820+0000 7f2c11695740 1 -- 172.21.15.46:0/1715379097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5625b218f3a0 con 0x5625b222ce40 2024-09-10T07:46:18.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.820+0000 7f2c0cc00640 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b223a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.820+0000 7f2c07e00640 1 -- 172.21.15.46:0/1715379097 learned_addr learned my addr 172.21.15.46:0/1715379097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:18.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.821+0000 7f2c07400640 1 -- 172.21.15.46:0/1715379097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 msgr2=0x5625b222d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.821+0000 7f2c07400640 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b222d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.821+0000 7f2c07400640 1 -- 172.21.15.46:0/1715379097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 msgr2=0x5625b223a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.821+0000 7f2c07400640 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b223a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.821+0000 7f2c07400640 1 -- 172.21.15.46:0/1715379097 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b2159ae0 con 0x5625b222d760 2024-09-10T07:46:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.821+0000 7f2c07400640 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2237ae0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40ccbc0 tx=0x7f2bf409b3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c28e0996ebfc6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.821+0000 7f2c07e00640 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b222d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.822+0000 7f2bfd200640 1 -- 172.21.15.46:0/1715379097 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40b4070 con 0x5625b222d760 2024-09-10T07:46:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.822+0000 7f2bfd200640 1 -- 172.21.15.46:0/1715379097 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf4099c80 con 0x5625b222d760 2024-09-10T07:46:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.822+0000 7f2bfd200640 1 -- 172.21.15.46:0/1715379097 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf409add0 con 0x5625b222d760 2024-09-10T07:46:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.823+0000 7f2c11695740 1 -- 172.21.15.46:0/1715379097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 msgr2=0x5625b2237ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.823+0000 7f2c11695740 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2237ae0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40ccbc0 tx=0x7f2bf409b3a0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.824+0000 7f2c11695740 1 -- 172.21.15.46:0/1715379097 shutdown_connections 2024-09-10T07:46:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.824+0000 7f2c11695740 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b223a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.824+0000 7f2c11695740 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2237ae0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.824+0000 7f2c11695740 1 --2- 172.21.15.46:0/1715379097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b222d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.824+0000 7f2c11695740 1 -- 172.21.15.46:0/1715379097 >> 172.21.15.46:0/1715379097 conn(0x5625b22293d0 msgr2=0x5625b22297b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.825+0000 7f2c11695740 1 -- 172.21.15.46:0/1715379097 shutdown_connections 2024-09-10T07:46:18.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.825+0000 7f2c11695740 1 -- 172.21.15.46:0/1715379097 wait complete. 2024-09-10T07:46:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.826+0000 7f2c11695740 1 -- start start 2024-09-10T07:46:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.826+0000 7f2c11695740 1 Processor -- start 2024-09-10T07:46:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c11695740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b2234fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c11695740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2235510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c11695740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b2235a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c11695740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5625b223abf0 con 0x5625b2238020 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c11695740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625b214e0f0 con 0x5625b222d760 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c11695740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5625b223ad70 con 0x5625b222ce40 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c11695740 10 client.0 _ll_register_callbacks cb 0x5625b2135bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c07400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2235510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c07400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2235510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38874/0 (socket says 172.21.15.46:38874) 2024-09-10T07:46:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.828+0000 7f2c07400640 1 -- 172.21.15.46:0/3740225508 learned_addr learned my addr 172.21.15.46:0/3740225508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.829+0000 7f2c07400640 1 -- 172.21.15.46:0/3740225508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 msgr2=0x5625b2234fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.829+0000 7f2c07400640 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b2234fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.829+0000 7f2c07400640 1 -- 172.21.15.46:0/3740225508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 msgr2=0x5625b2235a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.829+0000 7f2c07400640 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b2235a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.829+0000 7f2c07400640 1 -- 172.21.15.46:0/3740225508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b1f043c0 con 0x5625b222d760 2024-09-10T07:46:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.829+0000 7f2c07400640 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2235510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4097660 tx=0x7f2bf4099e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d783421a76deeb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.829+0000 7f2bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:18.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.830+0000 7f2bf1600640 1 -- 172.21.15.46:0/3740225508 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40b9060 con 0x5625b222d760 2024-09-10T07:46:18.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.830+0000 7f2bf1600640 1 -- 172.21.15.46:0/3740225508 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2bf40976d0 con 0x5625b222d760 2024-09-10T07:46:18.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.830+0000 7f2c11695740 1 -- 172.21.15.46:0/3740225508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf4099c80 con 0x5625b222d760 2024-09-10T07:46:18.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.830+0000 7f2bf1600640 1 -- 172.21.15.46:0/3740225508 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2bf40b4070 con 0x5625b222d760 2024-09-10T07:46:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.832+0000 7f2c11695740 10 client.27785 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.832+0000 7f2c11695740 1 -- 172.21.15.46:0/3740225508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5625b2246cf0 con 0x5625b222d760 2024-09-10T07:46:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.833+0000 7f2bf1600640 1 -- 172.21.15.46:0/3740225508 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf4099c80 con 0x5625b222d760 2024-09-10T07:46:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.833+0000 7f2bf1600640 1 -- 172.21.15.46:0/3740225508 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f2bf4018ce0 con 0x5625b222d760 2024-09-10T07:46:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.833+0000 7f2c11695740 10 client.27785 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.833+0000 7f2c11695740 10 client.27785 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:46:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.833+0000 7f2c11695740 20 client.27785 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:18.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.833+0000 7f2c11695740 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5625b2248040 0x5625b224a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.834+0000 7f2c11695740 4 client.27785 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:18.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.834+0000 7f2c11695740 1 -- 172.21.15.46:0/3740225508 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625b21a0950 con 0x5625b2248040 2024-09-10T07:46:18.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.834+0000 7f2c07e00640 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5625b2248040 0x5625b224a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.835+0000 7f2c07e00640 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5625b2248040 0x5625b224a430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5aeb6b89c01617c0 server_cookie=6d9f621ebf49b101 in_seq=0 out_seq=0 2024-09-10T07:46:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.835+0000 7f2bf1600640 10 client.27785 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:46:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.836+0000 7f2bf1600640 1 -- 172.21.15.46:0/3740225508 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625b21a0950 con 0x5625b2248040 2024-09-10T07:46:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.837+0000 7f2bf1600640 10 client.27785 handle_command_reply: tid=0 2024-09-10T07:46:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.838+0000 7f2c11695740 1 client.27785 shutdown 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2c11695740 1 -- 172.21.15.46:0/3740225508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 msgr2=0x5625b2235510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2c11695740 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2235510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4097660 tx=0x7f2bf4099e30 comp rx=0 tx=0).stop 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2c11695740 1 -- 172.21.15.46:0/3740225508 shutdown_connections 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2c11695740 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5625b2248040 0x5625b224a430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2c11695740 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b2235a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2c11695740 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b2235510 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2c11695740 1 --2- 172.21.15.46:0/3740225508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222ce40 0x5625b2234fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2c11695740 1 -- 172.21.15.46:0/3740225508 >> 172.21.15.46:0/3740225508 conn(0x5625b22293d0 msgr2=0x5625b222f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.839+0000 7f2bf1600640 0 client.27785 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:46:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.840+0000 7f2c11695740 1 -- 172.21.15.46:0/3740225508 shutdown_connections 2024-09-10T07:46:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.840+0000 7f2c11695740 1 -- 172.21.15.46:0/3740225508 wait complete. 2024-09-10T07:46:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.840+0000 7f2c11695740 20 client.27785 trim_cache size 0 max 16384 2024-09-10T07:46:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.864+0000 7f2c11695740 1 Processor -- start 2024-09-10T07:46:18.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.864+0000 7f2c11695740 1 -- start start 2024-09-10T07:46:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.864+0000 7f2c11695740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222ce40 0x5625b22358a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.864+0000 7f2c11695740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222d760 0x5625b224d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.864+0000 7f2c11695740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b224d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.864+0000 7f2c11695740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081bd8c0 con 0x5625b2238020 2024-09-10T07:46:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.864+0000 7f2c11695740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081bf400 con 0x5625b222ce40 2024-09-10T07:46:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.864+0000 7f2c11695740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081bdf90 con 0x5625b222d760 2024-09-10T07:46:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.865+0000 7f2c07e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222ce40 0x5625b22358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.865+0000 7f2c07400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222d760 0x5625b224d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.865+0000 7f2c07400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222d760 0x5625b224d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:36780/0 (socket says 172.21.15.46:36780) 2024-09-10T07:46:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.865+0000 7f2c07400640 1 -- 172.21.15.46:0/20571292 learned_addr learned my addr 172.21.15.46:0/20571292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.866+0000 7f2c07400640 1 -- 172.21.15.46:0/20571292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222ce40 msgr2=0x5625b22358a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.866+0000 7f2c07400640 1 --2- 172.21.15.46:0/20571292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222ce40 0x5625b22358a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.866+0000 7f2c07400640 1 -- 172.21.15.46:0/20571292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 msgr2=0x5625b224d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.866+0000 7f2c07400640 1 --2- 172.21.15.46:0/20571292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b224d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.866+0000 7f2c07400640 1 -- 172.21.15.46:0/20571292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b2228c80 con 0x5625b222d760 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.866+0000 7f2c07400640 1 --2- 172.21.15.46:0/20571292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222d760 0x5625b224d070 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4005fd0 tx=0x7f2bf4010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231f8c454be68245 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.867+0000 7f2bf3e00640 1 -- 172.21.15.46:0/20571292 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40b9070 con 0x5625b222d760 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.867+0000 7f2bf3e00640 1 -- 172.21.15.46:0/20571292 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf4097ff0 con 0x5625b222d760 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.867+0000 7f2bf3e00640 1 -- 172.21.15.46:0/20571292 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4018be0 con 0x5625b222d760 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.868+0000 7f2c11695740 1 -- 172.21.15.46:0/20571292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222d760 msgr2=0x5625b224d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.868+0000 7f2c11695740 1 --2- 172.21.15.46:0/20571292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222d760 0x5625b224d070 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4005fd0 tx=0x7f2bf4010d80 comp rx=0 tx=0).stop 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.868+0000 7f2c11695740 1 -- 172.21.15.46:0/20571292 shutdown_connections 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.868+0000 7f2c11695740 1 --2- 172.21.15.46:0/20571292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b2238020 0x5625b224d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.868+0000 7f2c11695740 1 --2- 172.21.15.46:0/20571292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b222d760 0x5625b224d070 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.868+0000 7f2c11695740 1 --2- 172.21.15.46:0/20571292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222ce40 0x5625b22358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.868+0000 7f2c11695740 1 -- 172.21.15.46:0/20571292 >> 172.21.15.46:0/20571292 conn(0x5625b22293d0 msgr2=0x5625b222f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.869+0000 7f2c11695740 1 -- 172.21.15.46:0/20571292 shutdown_connections 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.869+0000 7f2c11695740 1 -- 172.21.15.46:0/20571292 wait complete. 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.869+0000 7f2c11695740 1 -- start start 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.869+0000 7f2c11695740 1 Processor -- start 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.870+0000 7f2c11695740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b222ce40 0x5625b224a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.870+0000 7f2c11695740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b224ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.870+0000 7f2c11695740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b2238020 0x5625b22a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.870+0000 7f2c11695740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081bed30 con 0x5625b222ce40 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.870+0000 7f2c11695740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081be660 con 0x5625b222d760 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.870+0000 7f2c11695740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081bb8a0 con 0x5625b2238020 2024-09-10T07:46:18.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.870+0000 7f2c11695740 10 client.0 _ll_register_callbacks cb 0x7f2bf40b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.870+0000 7f2c0cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b2238020 0x5625b22a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c0cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b2238020 0x5625b22a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:36790/0 (socket says 172.21.15.46:36790) 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c0cc00640 1 -- 172.21.15.46:0/2498758054 learned_addr learned my addr 172.21.15.46:0/2498758054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c07400640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b224ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c07e00640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b222ce40 0x5625b224a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c0cc00640 1 -- 172.21.15.46:0/2498758054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 msgr2=0x5625b224ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c0cc00640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b224ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c0cc00640 1 -- 172.21.15.46:0/2498758054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b222ce40 msgr2=0x5625b224a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c0cc00640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b222ce40 0x5625b224a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c0cc00640 1 -- 172.21.15.46:0/2498758054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b22a9fd0 con 0x5625b2238020 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c07e00640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b222ce40 0x5625b224a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c07400640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b224ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:18.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2c0cc00640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b2238020 0x5625b22a9930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009be0 tx=0x7f2c00030d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7101d7181b9b3119 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.871+0000 7f2bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.872+0000 7f2bf1600640 1 -- 172.21.15.46:0/2498758054 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0004f020 con 0x5625b2238020 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.872+0000 7f2bf1600640 1 -- 172.21.15.46:0/2498758054 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2c0003d660 con 0x5625b2238020 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.872+0000 7f2c11695740 1 -- 172.21.15.46:0/2498758054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf4097ff0 con 0x5625b2238020 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.872+0000 7f2bf1600640 1 -- 172.21.15.46:0/2498758054 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2c0005a050 con 0x5625b2238020 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.872+0000 7f2c11695740 10 client.27739 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.872+0000 7f2c11695740 1 -- 172.21.15.46:0/2498758054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5625b22b2560 con 0x5625b2238020 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.873+0000 7f2bf1600640 1 -- 172.21.15.46:0/2498758054 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf4097ff0 con 0x5625b2238020 2024-09-10T07:46:18.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.873+0000 7f2bf1600640 1 -- 172.21.15.46:0/2498758054 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f2c0004b9b0 con 0x5625b2238020 2024-09-10T07:46:18.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.875+0000 7f2c11695740 10 client.27739 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:18.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.875+0000 7f2c11695740 10 client.27739 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:46:18.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.875+0000 7f2c11695740 20 client.27739 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.875+0000 7f2c11695740 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5625b22b2cb0 0x5625b22b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.875+0000 7f2c11695740 4 client.27739 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.875+0000 7f2c11695740 1 -- 172.21.15.46:0/2498758054 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625b219ae60 con 0x5625b22b2cb0 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.876+0000 7f2c07e00640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5625b22b2cb0 0x5625b22b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.876+0000 7f2c07e00640 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5625b22b2cb0 0x5625b22b50d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eab19b4fb9d4565a server_cookie=20cdac0fba804fed in_seq=0 out_seq=0 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.876+0000 7f2bf1600640 10 client.27739 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2bf1600640 1 -- 172.21.15.46:0/2498758054 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625b219ae60 con 0x5625b22b2cb0 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2bf1600640 10 client.27739 handle_command_reply: tid=0 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 client.27739 shutdown 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 -- 172.21.15.46:0/2498758054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b2238020 msgr2=0x5625b22a9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b2238020 0x5625b22a9930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009be0 tx=0x7f2c00030d50 comp rx=0 tx=0).stop 2024-09-10T07:46:18.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 -- 172.21.15.46:0/2498758054 shutdown_connections 2024-09-10T07:46:18.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5625b22b2cb0 0x5625b22b50d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b2238020 0x5625b22a9930 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625b222d760 0x5625b224ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 --2- 172.21.15.46:0/2498758054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625b222ce40 0x5625b224a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2bf1600640 0 client.27739 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:46:18.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 -- 172.21.15.46:0/2498758054 >> 172.21.15.46:0/2498758054 conn(0x5625b22293d0 msgr2=0x5625b2236830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:18.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.877+0000 7f2c11695740 1 -- 172.21.15.46:0/2498758054 shutdown_connections 2024-09-10T07:46:18.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.878+0000 7f2c11695740 1 -- 172.21.15.46:0/2498758054 wait complete. 2024-09-10T07:46:18.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.878+0000 7f2c11695740 20 client.27739 trim_cache size 0 max 16384 2024-09-10T07:46:18.879 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2bec0a2d10 msgr2=0x7f2bec0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2bec0a2d10 0x7f2bec0a51d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c00006010 tx=0x7f2c00009840 comp rx=0 tx=0).stop 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c08074ee0 msgr2=0x7f2c08073ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c08074ee0 0x7f2c08073ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400e790 tx=0x7f2bf400ec50 comp rx=0 tx=0).stop 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 shutdown_connections 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c0806cb00 0x7f2c0806cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c08074ee0 0x7f2c08073ec0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2bec0a2d10 0x7f2bec0a51d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 --2- 172.21.15.46:0/1859558471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c080745c0 0x7f2c08073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 >> 172.21.15.46:0/1859558471 conn(0x7f2c080fd740 msgr2=0x7f2c08113600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 shutdown_connections 2024-09-10T07:46:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:18.879+0000 7f2c0d600640 1 -- 172.21.15.46:0/1859558471 wait complete. 2024-09-10T07:46:19.903 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:19 smithi046 ceph-mon[30705]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 MiB/s wr, 371 op/s 2024-09-10T07:46:20.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:46:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:19 smithi181 ceph-mon[35397]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 MiB/s wr, 371 op/s 2024-09-10T07:46:20.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:19 smithi063 ceph-mon[35020]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 MiB/s wr, 371 op/s 2024-09-10T07:46:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 540.0 s, 704.2 tps, lat 139.467 ms stddev 157.562 2024-09-10T07:46:20.835 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.587+0000 7fe8a7a00640 1 -- 172.21.15.46:0/3146784791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a010e410 msgr2=0x7fe8a0110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.587+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/3146784791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a010e410 0x7fe8a0110800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe898009be0 tx=0x7fe89802a750 comp rx=0 tx=0).stop 2024-09-10T07:46:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.588+0000 7fe8a7a00640 1 -- 172.21.15.46:0/3146784791 shutdown_connections 2024-09-10T07:46:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.588+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/3146784791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a010e410 0x7fe8a0110800 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.588+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/3146784791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a0074ee0 0x7fe8a010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.588+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/3146784791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a00745c0 0x7fe8a00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.588+0000 7fe8a7a00640 1 -- 172.21.15.46:0/3146784791 >> 172.21.15.46:0/3146784791 conn(0x7fe8a00fd890 msgr2=0x7fe8a00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.589+0000 7fe8a7a00640 1 -- 172.21.15.46:0/3146784791 shutdown_connections 2024-09-10T07:46:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.589+0000 7fe8a7a00640 1 -- 172.21.15.46:0/3146784791 wait complete. 2024-09-10T07:46:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.590+0000 7fe8a7a00640 1 Processor -- start 2024-09-10T07:46:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.591+0000 7fe8a7a00640 1 -- start start 2024-09-10T07:46:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.592+0000 7fe8a7a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a00745c0 0x7fe8a0073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.593+0000 7fe8a5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a00745c0 0x7fe8a0073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.593+0000 7fe8a7a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0074ee0 0x7fe8a006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.593+0000 7fe8a7a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a010e410 0x7fe8a006d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.593+0000 7fe8a5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a00745c0 0x7fe8a0073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58080/0 (socket says 172.21.15.46:58080) 2024-09-10T07:46:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.593+0000 7fe8a5000640 1 -- 172.21.15.46:0/2847417455 learned_addr learned my addr 172.21.15.46:0/2847417455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.593+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0114140 con 0x7fe8a00745c0 2024-09-10T07:46:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.593+0000 7fe89fe00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0074ee0 0x7fe8a006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.594+0000 7fe8a5a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a010e410 0x7fe8a006d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.594+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0113fc0 con 0x7fe8a0074ee0 2024-09-10T07:46:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.594+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01142c0 con 0x7fe8a010e410 2024-09-10T07:46:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.594+0000 7fe8a5000640 1 -- 172.21.15.46:0/2847417455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a010e410 msgr2=0x7fe8a006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.594+0000 7fe8a5000640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a010e410 0x7fe8a006d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.594+0000 7fe8a5000640 1 -- 172.21.15.46:0/2847417455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0074ee0 msgr2=0x7fe8a006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.594+0000 7fe8a5000640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0074ee0 0x7fe8a006cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.594+0000 7fe8a5000640 1 -- 172.21.15.46:0/2847417455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a006d650 con 0x7fe8a00745c0 2024-09-10T07:46:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.595+0000 7fe89fe00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0074ee0 0x7fe8a006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.595+0000 7fe8a5a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a010e410 0x7fe8a006d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.595+0000 7fe8a5000640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a00745c0 0x7fe8a0073a40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e9f0 tx=0x7fe89400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.595+0000 7fe89d600640 1 -- 172.21.15.46:0/2847417455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89400cd80 con 0x7fe8a00745c0 2024-09-10T07:46:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.595+0000 7fe89d600640 1 -- 172.21.15.46:0/2847417455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe894010910 con 0x7fe8a00745c0 2024-09-10T07:46:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.595+0000 7fe89d600640 1 -- 172.21.15.46:0/2847417455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894018980 con 0x7fe8a00745c0 2024-09-10T07:46:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.595+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a0071570 con 0x7fe8a00745c0 2024-09-10T07:46:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.595+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a0071a60 con 0x7fe8a00745c0 2024-09-10T07:46:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.600+0000 7fe8aa66d740 1 Processor -- start 2024-09-10T07:46:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe8aa66d740 1 -- start start 2024-09-10T07:46:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8e480 0x560e42d8e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d99120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d99660 0x560e42d9ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560e42cec3e0 con 0x560e42d8e480 2024-09-10T07:46:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560e42c93d80 con 0x560e42d99660 2024-09-10T07:46:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560e42ce83a0 con 0x560e42d8eda0 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe89d600640 1 -- 172.21.15.46:0/2847417455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe894010ac0 con 0x7fe8a00745c0 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe89fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d99660 0x560e42d9ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe8a5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8e480 0x560e42d8e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.601+0000 7fe89fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d99660 0x560e42d9ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38916/0 (socket says 172.21.15.46:38916) 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe89fe00640 1 -- 172.21.15.46:0/2824456630 learned_addr learned my addr 172.21.15.46:0/2824456630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe8a5000640 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d99120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe89fe00640 1 -- 172.21.15.46:0/2824456630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 msgr2=0x560e42d99120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe89fe00640 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d99120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe89fe00640 1 -- 172.21.15.46:0/2824456630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8e480 msgr2=0x560e42d8e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe89fe00640 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8e480 0x560e42d8e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe89fe00640 1 -- 172.21.15.46:0/2824456630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e42cb2ae0 con 0x560e42d99660 2024-09-10T07:46:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe8a5a00640 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8e480 0x560e42d8e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.602+0000 7fe89fe00640 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d99660 0x560e42d9ba50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009bb0 tx=0x7fe88c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfda839c6686866f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.603+0000 7fe892000640 1 -- 172.21.15.46:0/2824456630 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c005020 con 0x560e42d99660 2024-09-10T07:46:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.603+0000 7fe892000640 1 -- 172.21.15.46:0/2824456630 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88c0047a0 con 0x560e42d99660 2024-09-10T07:46:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.603+0000 7fe892000640 1 -- 172.21.15.46:0/2824456630 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c011680 con 0x560e42d99660 2024-09-10T07:46:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.603+0000 7fe89d600640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8880a2cc0 0x7fe8880a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.603+0000 7fe89d600640 1 -- 172.21.15.46:0/2847417455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe8940cf810 con 0x7fe8a00745c0 2024-09-10T07:46:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.603+0000 7fe8a5a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8880a2cc0 0x7fe8880a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.604+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2824456630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d99660 msgr2=0x560e42d9ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.604+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d99660 0x560e42d9ba50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009bb0 tx=0x7fe88c00f730 comp rx=0 tx=0).stop 2024-09-10T07:46:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.605+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2824456630 shutdown_connections 2024-09-10T07:46:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.605+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d99660 0x560e42d9ba50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.605+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d99120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.605+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/2824456630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8e480 0x560e42d8e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.605+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2824456630 >> 172.21.15.46:0/2824456630 conn(0x560e42d8aa10 msgr2=0x560e42d8adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.605+0000 7fe8a5a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8880a2cc0 0x7fe8880a5180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe898009b30 tx=0x7fe898030c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.605+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2824456630 shutdown_connections 2024-09-10T07:46:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.605+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2824456630 wait complete. 2024-09-10T07:46:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.606+0000 7fe8aa66d740 1 -- start start 2024-09-10T07:46:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.606+0000 7fe8aa66d740 1 Processor -- start 2024-09-10T07:46:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.607+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.607+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8eda0 0x560e42d96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.608+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d99660 0x560e42d96f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.608+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560e42d9c230 con 0x560e42d8eda0 2024-09-10T07:46:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.608+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560e42ca70f0 con 0x560e42d8e480 2024-09-10T07:46:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.608+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560e42d9c3b0 con 0x560e42d99660 2024-09-10T07:46:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.608+0000 7fe8aa66d740 10 client.0 _ll_register_callbacks cb 0x560e42c8ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.608+0000 7fe8a5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.608+0000 7fe8a5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38922/0 (socket says 172.21.15.46:38922) 2024-09-10T07:46:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.608+0000 7fe8a5000640 1 -- 172.21.15.46:0/3538646109 learned_addr learned my addr 172.21.15.46:0/3538646109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe89fe00640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8eda0 0x560e42d96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe8a5a00640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d99660 0x560e42d96f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe8a5000640 1 -- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d99660 msgr2=0x560e42d96f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe8a5000640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d99660 0x560e42d96f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe8a5000640 1 -- 172.21.15.46:0/3538646109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8eda0 msgr2=0x560e42d96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe8a5000640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8eda0 0x560e42d96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe8a5000640 1 -- 172.21.15.46:0/3538646109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e42a5d3c0 con 0x560e42d8e480 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe89fe00640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8eda0 0x560e42d96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.609+0000 7fe8a5a00640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d99660 0x560e42d96f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.610+0000 7fe8a5000640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d964d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe89409b990 tx=0x7fe89409c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622046756163e9b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.611+0000 7fe886a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.611+0000 7fe886a00640 1 -- 172.21.15.46:0/3538646109 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8940a4900 con 0x560e42d8e480 2024-09-10T07:46:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.611+0000 7fe886a00640 1 -- 172.21.15.46:0/3538646109 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe89409b0e0 con 0x560e42d8e480 2024-09-10T07:46:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.611+0000 7fe8aa66d740 1 -- 172.21.15.46:0/3538646109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88c0047a0 con 0x560e42d8e480 2024-09-10T07:46:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.612+0000 7fe886a00640 1 -- 172.21.15.46:0/3538646109 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe89409b300 con 0x560e42d8e480 2024-09-10T07:46:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.612+0000 7fe8aa66d740 10 client.27800 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.612+0000 7fe8aa66d740 1 -- 172.21.15.46:0/3538646109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560e42d847a0 con 0x560e42d8e480 2024-09-10T07:46:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.612+0000 7fe886a00640 1 -- 172.21.15.46:0/3538646109 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88c0047a0 con 0x560e42d8e480 2024-09-10T07:46:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.613+0000 7fe886a00640 1 -- 172.21.15.46:0/3538646109 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fe894097ae0 con 0x560e42d8e480 2024-09-10T07:46:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.613+0000 7fe8aa66d740 10 client.27800 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.613+0000 7fe8aa66d740 10 client.27800 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:46:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.613+0000 7fe8aa66d740 20 client.27800 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.613+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x560e42d85c60 0x560e42d88120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.613+0000 7fe8aa66d740 4 client.27800 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.613+0000 7fe8aa66d740 1 -- 172.21.15.46:0/3538646109 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e42cf9950 con 0x560e42d85c60 2024-09-10T07:46:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.614+0000 7fe89fe00640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x560e42d85c60 0x560e42d88120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.616+0000 7fe89fe00640 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x560e42d85c60 0x560e42d88120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c62abbafa6686afd server_cookie=5246435abcc8644c in_seq=0 out_seq=0 2024-09-10T07:46:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.616+0000 7fe886a00640 10 client.27800 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:46:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.617+0000 7fe886a00640 1 -- 172.21.15.46:0/3538646109 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e42cf9950 con 0x560e42d85c60 2024-09-10T07:46:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.617+0000 7fe886a00640 10 client.27800 handle_command_reply: tid=0 2024-09-10T07:46:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.617+0000 7fe8aa66d740 1 client.27800 shutdown 2024-09-10T07:46:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.617+0000 7fe8aa66d740 1 -- 172.21.15.46:0/3538646109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 msgr2=0x560e42d964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.617+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d964d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe89409b990 tx=0x7fe89409c390 comp rx=0 tx=0).stop 2024-09-10T07:46:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.618+0000 7fe8aa66d740 1 -- 172.21.15.46:0/3538646109 shutdown_connections 2024-09-10T07:46:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.618+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x560e42d85c60 0x560e42d88120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.618+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d99660 0x560e42d96f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.618+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d8eda0 0x560e42d96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.618+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/3538646109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d964d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.618+0000 7fe8aa66d740 1 -- 172.21.15.46:0/3538646109 >> 172.21.15.46:0/3538646109 conn(0x560e42d8aa10 msgr2=0x560e42d81380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.618+0000 7fe886a00640 0 client.27800 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:46:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.618+0000 7fe8aa66d740 1 -- 172.21.15.46:0/3538646109 shutdown_connections 2024-09-10T07:46:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.619+0000 7fe8aa66d740 1 -- 172.21.15.46:0/3538646109 wait complete. 2024-09-10T07:46:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.619+0000 7fe8aa66d740 20 client.27800 trim_cache size 0 max 16384 2024-09-10T07:46:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.649+0000 7fe8aa66d740 1 Processor -- start 2024-09-10T07:46:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.649+0000 7fe8aa66d740 1 -- start start 2024-09-10T07:46:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d90ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d89980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 0x560e42d89ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b5480 con 0x560e42d99660 2024-09-10T07:46:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b6fc0 con 0x560e42d8e480 2024-09-10T07:46:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b5b50 con 0x560e42d8eda0 2024-09-10T07:46:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe89fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d89980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe89fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d89980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:36836/0 (socket says 172.21.15.46:36836) 2024-09-10T07:46:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.650+0000 7fe89fe00640 1 -- 172.21.15.46:0/2128297677 learned_addr learned my addr 172.21.15.46:0/2128297677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.651+0000 7fe89fe00640 1 -- 172.21.15.46:0/2128297677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 msgr2=0x560e42d90ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.651+0000 7fe8a5000640 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d90ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.652+0000 7fe89fe00640 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d90ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.652+0000 7fe89fe00640 1 -- 172.21.15.46:0/2128297677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 msgr2=0x560e42d89ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.652+0000 7fe89fe00640 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 0x560e42d89ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.652+0000 7fe89fe00640 1 -- 172.21.15.46:0/2128297677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e42d8a840 con 0x560e42d8eda0 2024-09-10T07:46:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.652+0000 7fe89fe00640 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d89980 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009b00 tx=0x7fe88c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70fd639dd33a61d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.653+0000 7fe891600640 1 -- 172.21.15.46:0/2128297677 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c004020 con 0x560e42d8eda0 2024-09-10T07:46:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.653+0000 7fe891600640 1 -- 172.21.15.46:0/2128297677 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88c0041c0 con 0x560e42d8eda0 2024-09-10T07:46:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.653+0000 7fe891600640 1 -- 172.21.15.46:0/2128297677 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0266a0 con 0x560e42d8eda0 2024-09-10T07:46:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.653+0000 7fe8a5000640 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d90ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.654+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2128297677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 msgr2=0x560e42d89980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.654+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d89980 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009b00 tx=0x7fe88c000bc0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.655+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2128297677 shutdown_connections 2024-09-10T07:46:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.655+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 0x560e42d89ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.655+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8eda0 0x560e42d89980 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.655+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/2128297677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8e480 0x560e42d90ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.655+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2128297677 >> 172.21.15.46:0/2128297677 conn(0x560e42d8aa10 msgr2=0x560e42d81380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.655+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2128297677 shutdown_connections 2024-09-10T07:46:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.656+0000 7fe8aa66d740 1 -- 172.21.15.46:0/2128297677 wait complete. 2024-09-10T07:46:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.656+0000 7fe8aa66d740 1 -- start start 2024-09-10T07:46:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.657+0000 7fe8aa66d740 1 Processor -- start 2024-09-10T07:46:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8e480 0x560e42da1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8eda0 0x560e42da1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe8aa66d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 0x560e42d92720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b68f0 con 0x560e42d99660 2024-09-10T07:46:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b6220 con 0x560e42d8eda0 2024-09-10T07:46:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe8aa66d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b3480 con 0x560e42d8e480 2024-09-10T07:46:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe89fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8eda0 0x560e42da1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe8aa66d740 10 client.0 _ll_register_callbacks cb 0x7fe89409b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe89fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8eda0 0x560e42da1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38942/0 (socket says 172.21.15.46:38942) 2024-09-10T07:46:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe89fe00640 1 -- 172.21.15.46:0/956306935 learned_addr learned my addr 172.21.15.46:0/956306935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.658+0000 7fe8a5000640 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8e480 0x560e42da1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.659+0000 7fe8a5a00640 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 0x560e42d92720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.659+0000 7fe8a5000640 1 -- 172.21.15.46:0/956306935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8eda0 msgr2=0x560e42da1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.659+0000 7fe8a5000640 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8eda0 0x560e42da1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.659+0000 7fe8a5000640 1 -- 172.21.15.46:0/956306935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 msgr2=0x560e42d92720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.659+0000 7fe8a5000640 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 0x560e42d92720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.659+0000 7fe8a5000640 1 -- 172.21.15.46:0/956306935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e42d7d770 con 0x560e42d8e480 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.659+0000 7fe8a5000640 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8e480 0x560e42da1630 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe894009620 tx=0x7fe894098040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=738220fe7a8183cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.659+0000 7fe886a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.660+0000 7fe886a00640 1 -- 172.21.15.46:0/956306935 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8940b8020 con 0x560e42d8e480 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.660+0000 7fe886a00640 1 -- 172.21.15.46:0/956306935 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe894010430 con 0x560e42d8e480 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.660+0000 7fe8aa66d740 1 -- 172.21.15.46:0/956306935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88c0041c0 con 0x560e42d8e480 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.660+0000 7fe886a00640 1 -- 172.21.15.46:0/956306935 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe8940a4900 con 0x560e42d8e480 2024-09-10T07:46:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.660+0000 7fe8aa66d740 10 client.27754 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.660+0000 7fe8aa66d740 1 -- 172.21.15.46:0/956306935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560e42e1cd00 con 0x560e42d8e480 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.661+0000 7fe886a00640 1 -- 172.21.15.46:0/956306935 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88c0041c0 con 0x560e42d8e480 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.661+0000 7fe886a00640 1 -- 172.21.15.46:0/956306935 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fe8940ba2b0 con 0x560e42d8e480 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.661+0000 7fe8aa66d740 10 client.27754 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.661+0000 7fe8aa66d740 10 client.27754 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.661+0000 7fe8aa66d740 20 client.27754 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.661+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x560e42e1d280 0x560e42e1f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.661+0000 7fe8aa66d740 4 client.27754 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.661+0000 7fe8aa66d740 1 -- 172.21.15.46:0/956306935 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e42cf3e60 con 0x560e42e1d280 2024-09-10T07:46:21.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.662+0000 7fe89fe00640 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x560e42e1d280 0x560e42e1f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe89fe00640 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x560e42e1d280 0x560e42e1f670 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c94cfd2995d0c978 server_cookie=387757e7e47bde48 in_seq=0 out_seq=0 2024-09-10T07:46:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe886a00640 10 client.27754 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:46:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe886a00640 1 -- 172.21.15.46:0/956306935 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e42cf3e60 con 0x560e42e1d280 2024-09-10T07:46:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe886a00640 10 client.27754 handle_command_reply: tid=0 2024-09-10T07:46:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 client.27754 shutdown 2024-09-10T07:46:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 -- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8e480 msgr2=0x560e42da1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8e480 0x560e42da1630 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe894009620 tx=0x7fe894098040 comp rx=0 tx=0).stop 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 -- 172.21.15.46:0/956306935 shutdown_connections 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e42d99660 0x560e42d92720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e42d8eda0 0x560e42da1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x560e42e1d280 0x560e42e1f670 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 --2- 172.21.15.46:0/956306935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e42d8e480 0x560e42da1630 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe8aa66d740 1 -- 172.21.15.46:0/956306935 >> 172.21.15.46:0/956306935 conn(0x560e42d8aa10 msgr2=0x560e42d81380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.663+0000 7fe886a00640 0 client.27754 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:46:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.664+0000 7fe8aa66d740 1 -- 172.21.15.46:0/956306935 shutdown_connections 2024-09-10T07:46:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.664+0000 7fe8aa66d740 1 -- 172.21.15.46:0/956306935 wait complete. 2024-09-10T07:46:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.664+0000 7fe8aa66d740 20 client.27754 trim_cache size 0 max 16384 2024-09-10T07:46:21.667 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:46:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.666+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8880a2cc0 msgr2=0x7fe8880a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.666+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8880a2cc0 0x7fe8880a5180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe898009b30 tx=0x7fe898030c30 comp rx=0 tx=0).stop 2024-09-10T07:46:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.666+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a00745c0 msgr2=0x7fe8a0073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.666+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a00745c0 0x7fe8a0073a40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e9f0 tx=0x7fe89400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.667+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 shutdown_connections 2024-09-10T07:46:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.667+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a010e410 0x7fe8a006d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.667+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8a0074ee0 0x7fe8a006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.667+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8880a2cc0 0x7fe8880a5180 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.667+0000 7fe8a7a00640 1 --2- 172.21.15.46:0/2847417455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a00745c0 0x7fe8a0073a40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.667+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 >> 172.21.15.46:0/2847417455 conn(0x7fe8a00fd890 msgr2=0x7fe8a00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.668+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 shutdown_connections 2024-09-10T07:46:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:21.668+0000 7fe8a7a00640 1 -- 172.21.15.46:0/2847417455 wait complete. 2024-09-10T07:46:22.165 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:21 smithi046 ceph-mon[30705]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-09-10T07:46:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:21 smithi181 ceph-mon[35397]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-09-10T07:46:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:21 smithi063 ceph-mon[35020]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-09-10T07:46:22.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:46:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:24.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:23 smithi046 ceph-mon[30705]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 517 op/s 2024-09-10T07:46:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:23 smithi181 ceph-mon[35397]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 517 op/s 2024-09-10T07:46:24.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:23 smithi063 ceph-mon[35020]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 517 op/s 2024-09-10T07:46:24.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.475+0000 7f8442000640 1 -- 172.21.15.46:0/2564724065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 msgr2=0x7f843c110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.475+0000 7f8442000640 1 --2- 172.21.15.46:0/2564724065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 0x7f843c110800 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8434009900 tx=0x7f843402a4d0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.476+0000 7f8442000640 1 -- 172.21.15.46:0/2564724065 shutdown_connections 2024-09-10T07:46:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.476+0000 7f8442000640 1 --2- 172.21.15.46:0/2564724065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 0x7f843c110800 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.476+0000 7f8442000640 1 --2- 172.21.15.46:0/2564724065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c074ee0 0x7f843c10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.476+0000 7f8442000640 1 --2- 172.21.15.46:0/2564724065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0745c0 0x7f843c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.476+0000 7f8442000640 1 -- 172.21.15.46:0/2564724065 >> 172.21.15.46:0/2564724065 conn(0x7f843c0fd8b0 msgr2=0x7f843c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.476+0000 7f8442000640 1 -- 172.21.15.46:0/2564724065 shutdown_connections 2024-09-10T07:46:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.476+0000 7f8442000640 1 -- 172.21.15.46:0/2564724065 wait complete. 2024-09-10T07:46:24.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.478+0000 7f8442000640 1 Processor -- start 2024-09-10T07:46:24.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.478+0000 7f8442000640 1 -- start start 2024-09-10T07:46:24.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f8442000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0745c0 0x7f843c1075c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f8442000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c074ee0 0x7f843c107b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f8442000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 0x7f843c1026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f843be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 0x7f843c1026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f843be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 0x7f843c1026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38964/0 (socket says 172.21.15.46:38964) 2024-09-10T07:46:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f843be00640 1 -- 172.21.15.46:0/4218666565 learned_addr learned my addr 172.21.15.46:0/4218666565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f8432a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c074ee0 0x7f843c107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1141f0 con 0x7f843c0745c0 2024-09-10T07:46:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f843c114070 con 0x7f843c10e410 2024-09-10T07:46:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.479+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f843c114370 con 0x7f843c074ee0 2024-09-10T07:46:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.480+0000 7f843b400640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0745c0 0x7f843c1075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.480+0000 7f843be00640 1 -- 172.21.15.46:0/4218666565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c074ee0 msgr2=0x7f843c107b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.481+0000 7f843be00640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c074ee0 0x7f843c107b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.481+0000 7f843be00640 1 -- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0745c0 msgr2=0x7f843c1075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.481+0000 7f843be00640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0745c0 0x7f843c1075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.481+0000 7f843be00640 1 -- 172.21.15.46:0/4218666565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c102fa0 con 0x7f843c10e410 2024-09-10T07:46:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.481+0000 7f8432a00640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c074ee0 0x7f843c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.482+0000 7f843b400640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0745c0 0x7f843c1075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:24.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.482+0000 7f843be00640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 0x7f843c1026e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8434005ab0 tx=0x7f8434005370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:24.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.483+0000 7f8438c00640 1 -- 172.21.15.46:0/4218666565 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434040030 con 0x7f843c10e410 2024-09-10T07:46:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.483+0000 7f8438c00640 1 -- 172.21.15.46:0/4218666565 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843403c920 con 0x7f843c10e410 2024-09-10T07:46:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.483+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c103230 con 0x7f843c10e410 2024-09-10T07:46:24.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.485+0000 7f8438c00640 1 -- 172.21.15.46:0/4218666565 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434031940 con 0x7f843c10e410 2024-09-10T07:46:24.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.485+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c10d4b0 con 0x7f843c10e410 2024-09-10T07:46:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.495+0000 7f8438c00640 1 -- 172.21.15.46:0/4218666565 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8434043030 con 0x7f843c10e410 2024-09-10T07:46:24.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.498+0000 7f8438c00640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84200a2c00 0x7f84200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.500+0000 7f8438c00640 1 -- 172.21.15.46:0/4218666565 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f84340ef650 con 0x7f843c10e410 2024-09-10T07:46:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.500+0000 7f843b400640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84200a2c00 0x7f84200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.501+0000 7f84449cd740 1 Processor -- start 2024-09-10T07:46:24.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.504+0000 7f84449cd740 1 -- start start 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.504+0000 7f84449cd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1ec70 0x560e32b1f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.504+0000 7f84449cd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1f590 0x560e32b29910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.504+0000 7f84449cd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b2c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.504+0000 7f84449cd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560e32a853e0 con 0x560e32b1ec70 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.504+0000 7f84449cd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560e32a2cd80 con 0x560e32b29e50 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.504+0000 7f84449cd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560e32a813a0 con 0x560e32b1f590 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.505+0000 7f843be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b2c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.505+0000 7f843be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b2c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38976/0 (socket says 172.21.15.46:38976) 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.505+0000 7f843be00640 1 -- 172.21.15.46:0/2469809375 learned_addr learned my addr 172.21.15.46:0/2469809375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.505+0000 7f843be00640 1 -- 172.21.15.46:0/2469809375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1f590 msgr2=0x560e32b29910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.505+0000 7f843be00640 1 --2- 172.21.15.46:0/2469809375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1f590 0x560e32b29910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.505+0000 7f843be00640 1 -- 172.21.15.46:0/2469809375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1ec70 msgr2=0x560e32b1f050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.505+0000 7f843b400640 1 --2- 172.21.15.46:0/2469809375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1ec70 0x560e32b1f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.506+0000 7f843be00640 1 --2- 172.21.15.46:0/2469809375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1ec70 0x560e32b1f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.506+0000 7f843be00640 1 -- 172.21.15.46:0/2469809375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e32a4bae0 con 0x560e32b29e50 2024-09-10T07:46:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.507+0000 7f843be00640 1 --2- 172.21.15.46:0/2469809375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b2c240 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f84340bbac0 tx=0x7f8434009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1af85902e180f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.507+0000 7f843b400640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84200a2c00 0x7f84200a50c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f842c009fd0 tx=0x7f842c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.507+0000 7f842be00640 1 -- 172.21.15.46:0/2469809375 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84340bce60 con 0x560e32b29e50 2024-09-10T07:46:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.507+0000 7f842be00640 1 -- 172.21.15.46:0/2469809375 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84340bfee0 con 0x560e32b29e50 2024-09-10T07:46:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.507+0000 7f842be00640 1 -- 172.21.15.46:0/2469809375 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843404a030 con 0x560e32b29e50 2024-09-10T07:46:24.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.508+0000 7f84449cd740 1 -- 172.21.15.46:0/2469809375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 msgr2=0x560e32b2c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.508+0000 7f84449cd740 1 --2- 172.21.15.46:0/2469809375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b2c240 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f84340bbac0 tx=0x7f8434009260 comp rx=0 tx=0).stop 2024-09-10T07:46:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.511+0000 7f84449cd740 1 -- 172.21.15.46:0/2469809375 shutdown_connections 2024-09-10T07:46:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.511+0000 7f84449cd740 1 --2- 172.21.15.46:0/2469809375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b2c240 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.511+0000 7f84449cd740 1 --2- 172.21.15.46:0/2469809375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1f590 0x560e32b29910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.511+0000 7f84449cd740 1 --2- 172.21.15.46:0/2469809375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1ec70 0x560e32b1f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.511+0000 7f84449cd740 1 -- 172.21.15.46:0/2469809375 >> 172.21.15.46:0/2469809375 conn(0x560e32b1b200 msgr2=0x560e32b1b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.511+0000 7f84449cd740 1 -- 172.21.15.46:0/2469809375 shutdown_connections 2024-09-10T07:46:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.513+0000 7f84449cd740 1 -- 172.21.15.46:0/2469809375 wait complete. 2024-09-10T07:46:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.514+0000 7f84449cd740 1 -- start start 2024-09-10T07:46:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.514+0000 7f84449cd740 1 Processor -- start 2024-09-10T07:46:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.515+0000 7f84449cd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1ec70 0x560e32b26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.515+0000 7f84449cd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.515+0000 7f84449cd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b29e50 0x560e32b27820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.515+0000 7f84449cd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560e32b2c7e0 con 0x560e32b1f590 2024-09-10T07:46:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.515+0000 7f84449cd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560e32a400f0 con 0x560e32b1ec70 2024-09-10T07:46:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.515+0000 7f84449cd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560e32b2c960 con 0x560e32b29e50 2024-09-10T07:46:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f84449cd740 10 client.0 _ll_register_callbacks cb 0x560e32a27bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f843b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1ec70 0x560e32b26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f843b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1ec70 0x560e32b26da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38984/0 (socket says 172.21.15.46:38984) 2024-09-10T07:46:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f843b400640 1 -- 172.21.15.46:0/2246220661 learned_addr learned my addr 172.21.15.46:0/2246220661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f8432a00640 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f843b400640 1 -- 172.21.15.46:0/2246220661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b29e50 msgr2=0x560e32b27820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:46:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f843b400640 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b29e50 0x560e32b27820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f843b400640 1 -- 172.21.15.46:0/2246220661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 msgr2=0x560e32b272e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f843b400640 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b272e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.516+0000 7f843b400640 1 -- 172.21.15.46:0/2246220661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e327f63c0 con 0x560e32b1ec70 2024-09-10T07:46:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.517+0000 7f843b400640 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1ec70 0x560e32b26da0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f842c010f90 tx=0x7f842c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=925263be74ddb233 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.517+0000 7f8428c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.517+0000 7f8428c00640 1 -- 172.21.15.46:0/2246220661 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c025020 con 0x560e32b1ec70 2024-09-10T07:46:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.518+0000 7f8428c00640 1 -- 172.21.15.46:0/2246220661 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f842c021680 con 0x560e32b1ec70 2024-09-10T07:46:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.518+0000 7f84449cd740 1 -- 172.21.15.46:0/2246220661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84340bfee0 con 0x560e32b1ec70 2024-09-10T07:46:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.518+0000 7f8428c00640 1 -- 172.21.15.46:0/2246220661 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f842c030050 con 0x560e32b1ec70 2024-09-10T07:46:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.518+0000 7f84449cd740 10 client.27815 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.518+0000 7f84449cd740 1 -- 172.21.15.46:0/2246220661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560e32b38850 con 0x560e32b1ec70 2024-09-10T07:46:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.518+0000 7f8428c00640 1 -- 172.21.15.46:0/2246220661 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84340bfee0 con 0x560e32b1ec70 2024-09-10T07:46:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.519+0000 7f8428c00640 1 -- 172.21.15.46:0/2246220661 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f842c00f040 con 0x560e32b1ec70 2024-09-10T07:46:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.519+0000 7f84449cd740 10 client.27815 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.519+0000 7f84449cd740 10 client.27815 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:46:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.519+0000 7f84449cd740 20 client.27815 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.519+0000 7f84449cd740 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560e32b39db0 0x560e32b3c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.519+0000 7f84449cd740 4 client.27815 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:24.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.519+0000 7f84449cd740 1 -- 172.21.15.46:0/2246220661 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e32a92950 con 0x560e32b39db0 2024-09-10T07:46:24.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.519+0000 7f8432a00640 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560e32b39db0 0x560e32b3c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.520+0000 7f8432a00640 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560e32b39db0 0x560e32b3c230 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e63baa5c95705289 server_cookie=52bdbd10829973bd in_seq=0 out_seq=0 2024-09-10T07:46:24.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.520+0000 7f8428c00640 10 client.27815 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:46:24.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.521+0000 7f8428c00640 1 -- 172.21.15.46:0/2246220661 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e32a92950 con 0x560e32b39db0 2024-09-10T07:46:24.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f8428c00640 10 client.27815 handle_command_reply: tid=0 2024-09-10T07:46:24.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 client.27815 shutdown 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 -- 172.21.15.46:0/2246220661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1ec70 msgr2=0x560e32b26da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1ec70 0x560e32b26da0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f842c010f90 tx=0x7f842c004a40 comp rx=0 tx=0).stop 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 -- 172.21.15.46:0/2246220661 shutdown_connections 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560e32b39db0 0x560e32b3c230 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b29e50 0x560e32b27820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f8428c00640 0 client.27815 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 --2- 172.21.15.46:0/2246220661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1ec70 0x560e32b26da0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.522+0000 7f84449cd740 1 -- 172.21.15.46:0/2246220661 >> 172.21.15.46:0/2246220661 conn(0x560e32b1b200 msgr2=0x560e32b20f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:24.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.523+0000 7f84449cd740 1 -- 172.21.15.46:0/2246220661 shutdown_connections 2024-09-10T07:46:24.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.523+0000 7f84449cd740 1 -- 172.21.15.46:0/2246220661 wait complete. 2024-09-10T07:46:24.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.523+0000 7f84449cd740 20 client.27815 trim_cache size 0 max 16384 2024-09-10T07:46:24.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.550+0000 7f84449cd740 1 Processor -- start 2024-09-10T07:46:24.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.550+0000 7f84449cd740 1 -- start start 2024-09-10T07:46:24.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.550+0000 7f84449cd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b41b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.550+0000 7f84449cd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b42040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f84449cd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b24020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f84449cd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1bdb60 con 0x560e32b1f590 2024-09-10T07:46:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f84449cd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1bf6a0 con 0x560e32b29e50 2024-09-10T07:46:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f84449cd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1be230 con 0x560e32b1ec70 2024-09-10T07:46:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f843be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b24020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f843be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b24020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38986/0 (socket says 172.21.15.46:38986) 2024-09-10T07:46:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f843be00640 1 -- 172.21.15.46:0/1498022311 learned_addr learned my addr 172.21.15.46:0/1498022311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f8432a00640 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b42040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.551+0000 7f843b400640 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b41b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.552+0000 7f843b400640 1 -- 172.21.15.46:0/1498022311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 msgr2=0x560e32b24020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.552+0000 7f843b400640 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b24020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.552+0000 7f843b400640 1 -- 172.21.15.46:0/1498022311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 msgr2=0x560e32b42040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.552+0000 7f843b400640 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b42040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.552+0000 7f843b400640 1 -- 172.21.15.46:0/1498022311 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e32b16770 con 0x560e32b1ec70 2024-09-10T07:46:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.552+0000 7f8432a00640 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b42040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.552+0000 7f843b400640 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b41b00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f842c020be0 tx=0x7f842c009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e6ff533d700068 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.553+0000 7f842b400640 1 -- 172.21.15.46:0/1498022311 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c038070 con 0x560e32b1ec70 2024-09-10T07:46:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.553+0000 7f842b400640 1 -- 172.21.15.46:0/1498022311 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f842c016030 con 0x560e32b1ec70 2024-09-10T07:46:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.553+0000 7f842b400640 1 -- 172.21.15.46:0/1498022311 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c025050 con 0x560e32b1ec70 2024-09-10T07:46:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.554+0000 7f84449cd740 1 -- 172.21.15.46:0/1498022311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 msgr2=0x560e32b41b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.554+0000 7f84449cd740 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b41b00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f842c020be0 tx=0x7f842c009f10 comp rx=0 tx=0).stop 2024-09-10T07:46:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.554+0000 7f84449cd740 1 -- 172.21.15.46:0/1498022311 shutdown_connections 2024-09-10T07:46:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.554+0000 7f84449cd740 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b29e50 0x560e32b24020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.554+0000 7f84449cd740 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b1f590 0x560e32b42040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.554+0000 7f84449cd740 1 --2- 172.21.15.46:0/1498022311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b41b00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.554+0000 7f84449cd740 1 -- 172.21.15.46:0/1498022311 >> 172.21.15.46:0/1498022311 conn(0x560e32b1b200 msgr2=0x560e32b20f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.555+0000 7f84449cd740 1 -- 172.21.15.46:0/1498022311 shutdown_connections 2024-09-10T07:46:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.555+0000 7f84449cd740 1 -- 172.21.15.46:0/1498022311 wait complete. 2024-09-10T07:46:24.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.555+0000 7f84449cd740 1 -- start start 2024-09-10T07:46:24.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.556+0000 7f84449cd740 1 Processor -- start 2024-09-10T07:46:24.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.556+0000 7f84449cd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b23f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.556+0000 7f84449cd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1f590 0x560e32b24460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.556+0000 7f84449cd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b29e50 0x560e32b249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.556+0000 7f84449cd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1befd0 con 0x560e32b29e50 2024-09-10T07:46:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.556+0000 7f84449cd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1be900 con 0x560e32b1f590 2024-09-10T07:46:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.556+0000 7f84449cd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1bbb40 con 0x560e32b1ec70 2024-09-10T07:46:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.556+0000 7f84449cd740 10 client.0 _ll_register_callbacks cb 0x7f842c010670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b23f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b23f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:36904/0 (socket says 172.21.15.46:36904) 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843b400640 1 -- 172.21.15.46:0/2535054330 learned_addr learned my addr 172.21.15.46:0/2535054330 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f8432a00640 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1f590 0x560e32b24460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843be00640 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b29e50 0x560e32b249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843b400640 1 -- 172.21.15.46:0/2535054330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1f590 msgr2=0x560e32b24460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843b400640 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1f590 0x560e32b24460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843b400640 1 -- 172.21.15.46:0/2535054330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b29e50 msgr2=0x560e32b249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843b400640 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b29e50 0x560e32b249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.557+0000 7f843b400640 1 -- 172.21.15.46:0/2535054330 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e32b1be90 con 0x560e32b1ec70 2024-09-10T07:46:24.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.558+0000 7f843be00640 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b29e50 0x560e32b249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:24.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.558+0000 7f843b400640 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b23f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f842c020a70 tx=0x7f842c00f1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8afa0b3d2009d0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:24.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.558+0000 7f8428c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:24.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.558+0000 7f8428c00640 1 -- 172.21.15.46:0/2535054330 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c041020 con 0x560e32b1ec70 2024-09-10T07:46:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.558+0000 7f8428c00640 1 -- 172.21.15.46:0/2535054330 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f842c0203d0 con 0x560e32b1ec70 2024-09-10T07:46:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.558+0000 7f84449cd740 1 -- 172.21.15.46:0/2535054330 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842c016030 con 0x560e32b1ec70 2024-09-10T07:46:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.559+0000 7f8428c00640 1 -- 172.21.15.46:0/2535054330 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f842c04c050 con 0x560e32b1ec70 2024-09-10T07:46:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.559+0000 7f84449cd740 10 client.27766 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.559+0000 7f84449cd740 1 -- 172.21.15.46:0/2535054330 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560e32ba4d60 con 0x560e32b1ec70 2024-09-10T07:46:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.559+0000 7f8428c00640 1 -- 172.21.15.46:0/2535054330 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f842c016030 con 0x560e32b1ec70 2024-09-10T07:46:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.560+0000 7f8428c00640 1 -- 172.21.15.46:0/2535054330 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f842c03d600 con 0x560e32b1ec70 2024-09-10T07:46:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.560+0000 7f84449cd740 10 client.27766 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.560+0000 7f84449cd740 10 client.27766 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:46:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.560+0000 7f84449cd740 20 client.27766 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.560+0000 7f84449cd740 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560e32ba5360 0x560e32ba7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.560+0000 7f84449cd740 4 client.27766 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:46:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.560+0000 7f84449cd740 1 -- 172.21.15.46:0/2535054330 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e32a8ce60 con 0x560e32ba5360 2024-09-10T07:46:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.560+0000 7f8432a00640 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560e32ba5360 0x560e32ba7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.561+0000 7f8432a00640 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560e32ba5360 0x560e32ba7750 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d386e3622b5c49ca server_cookie=b49c3b772799dbd9 in_seq=0 out_seq=0 2024-09-10T07:46:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.562+0000 7f8428c00640 10 client.27766 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:46:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.562+0000 7f8428c00640 1 -- 172.21.15.46:0/2535054330 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e32a8ce60 con 0x560e32ba5360 2024-09-10T07:46:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.562+0000 7f8428c00640 10 client.27766 handle_command_reply: tid=0 2024-09-10T07:46:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.562+0000 7f84449cd740 1 client.27766 shutdown 2024-09-10T07:46:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.562+0000 7f84449cd740 1 -- 172.21.15.46:0/2535054330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 msgr2=0x560e32b23f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.562+0000 7f84449cd740 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b23f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f842c020a70 tx=0x7f842c00f1a0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f84449cd740 1 -- 172.21.15.46:0/2535054330 shutdown_connections 2024-09-10T07:46:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f84449cd740 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x560e32ba5360 0x560e32ba7750 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f8428c00640 0 client.27766 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:46:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f84449cd740 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e32b29e50 0x560e32b249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f84449cd740 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560e32b1f590 0x560e32b24460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f84449cd740 1 --2- 172.21.15.46:0/2535054330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e32b1ec70 0x560e32b23f20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f84449cd740 1 -- 172.21.15.46:0/2535054330 >> 172.21.15.46:0/2535054330 conn(0x560e32b1b200 msgr2=0x560e32b20710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f84449cd740 1 -- 172.21.15.46:0/2535054330 shutdown_connections 2024-09-10T07:46:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.563+0000 7f84449cd740 1 -- 172.21.15.46:0/2535054330 wait complete. 2024-09-10T07:46:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.564+0000 7f84449cd740 20 client.27766 trim_cache size 0 max 16384 2024-09-10T07:46:24.564 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:46:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84200a2c00 msgr2=0x7f84200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84200a2c00 0x7f84200a50c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f842c009fd0 tx=0x7f842c009680 comp rx=0 tx=0).stop 2024-09-10T07:46:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 msgr2=0x7f843c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 0x7f843c1026e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8434005ab0 tx=0x7f8434005370 comp rx=0 tx=0).stop 2024-09-10T07:46:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 shutdown_connections 2024-09-10T07:46:24.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f843c10e410 0x7f843c1026e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c074ee0 0x7f843c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f84200a2c00 0x7f84200a50c0 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f842c009fd0 tx=0x7f842c009680 comp rx=0 tx=0).stop 2024-09-10T07:46:24.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 --2- 172.21.15.46:0/4218666565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0745c0 0x7f843c1075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.565+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 >> 172.21.15.46:0/4218666565 conn(0x7f843c0fd8b0 msgr2=0x7f843c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.568+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 shutdown_connections 2024-09-10T07:46:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:24.568+0000 7f8442000640 1 -- 172.21.15.46:0/4218666565 wait complete. 2024-09-10T07:46:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 545.0 s, 714.2 tps, lat 139.303 ms stddev 164.867 2024-09-10T07:46:25.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:46:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:25 smithi181 ceph-mon[35397]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 454 op/s 2024-09-10T07:46:26.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:25 smithi063 ceph-mon[35020]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 454 op/s 2024-09-10T07:46:26.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:25 smithi046 ceph-mon[30705]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 454 op/s 2024-09-10T07:46:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.777+0000 7fb5ece00640 1 -- 172.21.15.46:0/4221427718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 msgr2=0x7fb5e8105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.777+0000 7fb5ece00640 1 --2- 172.21.15.46:0/4221427718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 0x7fb5e8105b70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb5e001a910 tx=0x7fb5e003b430 comp rx=0 tx=0).stop 2024-09-10T07:46:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.777+0000 7fb5ece00640 1 -- 172.21.15.46:0/4221427718 shutdown_connections 2024-09-10T07:46:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.777+0000 7fb5ece00640 1 --2- 172.21.15.46:0/4221427718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 0x7fb5e8105b70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.777+0000 7fb5ece00640 1 --2- 172.21.15.46:0/4221427718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5e8102d10 0x7fb5e8105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.777+0000 7fb5ece00640 1 --2- 172.21.15.46:0/4221427718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5e81003c0 0x7fb5e81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.777+0000 7fb5ece00640 1 -- 172.21.15.46:0/4221427718 >> 172.21.15.46:0/4221427718 conn(0x7fb5e80fc1b0 msgr2=0x7fb5e80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.778+0000 7fb5ece00640 1 -- 172.21.15.46:0/4221427718 shutdown_connections 2024-09-10T07:46:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.778+0000 7fb5ece00640 1 -- 172.21.15.46:0/4221427718 wait complete. 2024-09-10T07:46:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.780+0000 7fb5ece00640 1 Processor -- start 2024-09-10T07:46:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.783+0000 7fb5ece00640 1 -- start start 2024-09-10T07:46:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.783+0000 7fb5ece00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5e81003c0 0x7fb5e806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.783+0000 7fb5ece00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5e8102d10 0x7fb5e806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.784+0000 7fb5ece00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 0x7fb5e8072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.784+0000 7fb5e5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5e81003c0 0x7fb5e806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.784+0000 7fb5ece00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8109650 con 0x7fb5e8102d10 2024-09-10T07:46:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.784+0000 7fb5ece00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81094d0 con 0x7fb5e81003c0 2024-09-10T07:46:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.784+0000 7fb5ece00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81097d0 con 0x7fb5e81056c0 2024-09-10T07:46:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.784+0000 7fb5e6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 0x7fb5e8072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.785+0000 7fb5e6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 0x7fb5e8072e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43382/0 (socket says 172.21.15.46:43382) 2024-09-10T07:46:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.785+0000 7fb5e6800640 1 -- 172.21.15.46:0/3636112038 learned_addr learned my addr 172.21.15.46:0/3636112038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.785+0000 7fb5dd400640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5e8102d10 0x7fb5e806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.785+0000 7fb5e5e00640 1 -- 172.21.15.46:0/3636112038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 msgr2=0x7fb5e8072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.785+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 0x7fb5e8072e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.785+0000 7fb5e5e00640 1 -- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5e8102d10 msgr2=0x7fb5e806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.785+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5e8102d10 0x7fb5e806c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.785+0000 7fb5e5e00640 1 -- 172.21.15.46:0/3636112038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e80734e0 con 0x7fb5e81003c0 2024-09-10T07:46:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.786+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5e81003c0 0x7fb5e806bcb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800bd70 tx=0x7fb5d800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.786+0000 7fb5dd400640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5e8102d10 0x7fb5e806c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.787+0000 7fb5df400640 1 -- 172.21.15.46:0/3636112038 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8019070 con 0x7fb5e81003c0 2024-09-10T07:46:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.787+0000 7fb5df400640 1 -- 172.21.15.46:0/3636112038 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d8014920 con 0x7fb5e81003c0 2024-09-10T07:46:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.787+0000 7fb5df400640 1 -- 172.21.15.46:0/3636112038 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8012980 con 0x7fb5e81003c0 2024-09-10T07:46:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.787+0000 7fb5ece00640 1 -- 172.21.15.46:0/3636112038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e80737d0 con 0x7fb5e81003c0 2024-09-10T07:46:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.787+0000 7fb5ece00640 1 -- 172.21.15.46:0/3636112038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e8073cd0 con 0x7fb5e81003c0 2024-09-10T07:46:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.792+0000 7fb5efcc0740 1 Processor -- start 2024-09-10T07:46:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.793+0000 7fb5efcc0740 1 -- start start 2024-09-10T07:46:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.793+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c08090 0x556a43c08490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c0da60 0x556a43c0ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556a43b6d100 con 0x556a43c08090 2024-09-10T07:46:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556a43b151b0 con 0x556a43c0da60 2024-09-10T07:46:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556a43b690c0 con 0x556a43c089d0 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5df400640 1 -- 172.21.15.46:0/3636112038 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb5d801c070 con 0x7fb5e81003c0 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5dd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c0da60 0x556a43c0ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5dd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c0da60 0x556a43c0ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35474/0 (socket says 172.21.15.46:35474) 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.794+0000 7fb5dd400640 1 -- 172.21.15.46:0/723226351 learned_addr learned my addr 172.21.15.46:0/723226351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.795+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/723226351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.795+0000 7fb5e5e00640 1 -- 172.21.15.46:0/723226351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c0da60 msgr2=0x556a43c0ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.795+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/723226351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c0da60 0x556a43c0ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.795+0000 7fb5e5e00640 1 -- 172.21.15.46:0/723226351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c08090 msgr2=0x556a43c08490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.795+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/723226351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c08090 0x556a43c08490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.795+0000 7fb5e5e00640 1 -- 172.21.15.46:0/723226351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a43880210 con 0x556a43c089d0 2024-09-10T07:46:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.796+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/723226351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c0d490 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80cc1c0 tx=0x7fb5d80d2b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cddc926d60b7989 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.796+0000 7fb5d7400640 1 -- 172.21.15.46:0/723226351 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80d7930 con 0x556a43c089d0 2024-09-10T07:46:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.796+0000 7fb5d7400640 1 -- 172.21.15.46:0/723226351 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d80d8040 con 0x556a43c089d0 2024-09-10T07:46:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.796+0000 7fb5d7400640 1 -- 172.21.15.46:0/723226351 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80db3a0 con 0x556a43c089d0 2024-09-10T07:46:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.797+0000 7fb5efcc0740 1 -- 172.21.15.46:0/723226351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 msgr2=0x556a43c0d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.797+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/723226351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c0d490 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80cc1c0 tx=0x7fb5d80d2b30 comp rx=0 tx=0).stop 2024-09-10T07:46:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.798+0000 7fb5efcc0740 1 -- 172.21.15.46:0/723226351 shutdown_connections 2024-09-10T07:46:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.798+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/723226351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c0da60 0x556a43c0ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.798+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/723226351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c0d490 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.798+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/723226351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c08090 0x556a43c08490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.798+0000 7fb5efcc0740 1 -- 172.21.15.46:0/723226351 >> 172.21.15.46:0/723226351 conn(0x556a43c03700 msgr2=0x556a43c03b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.799+0000 7fb5efcc0740 1 -- 172.21.15.46:0/723226351 shutdown_connections 2024-09-10T07:46:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.799+0000 7fb5efcc0740 1 -- 172.21.15.46:0/723226351 wait complete. 2024-09-10T07:46:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.802+0000 7fb5efcc0740 1 -- start start 2024-09-10T07:46:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.802+0000 7fb5efcc0740 1 Processor -- start 2024-09-10T07:46:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5df400640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5cc0a2cb0 0x7fb5cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5df400640 1 -- 172.21.15.46:0/3636112038 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb5d80ceb00 con 0x7fb5e81003c0 2024-09-10T07:46:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c12330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c12870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 0x556a43c198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556a43c10a30 con 0x556a43c0da60 2024-09-10T07:46:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556a43af6090 con 0x556a43c08090 2024-09-10T07:46:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556a43c10bb0 con 0x556a43c089d0 2024-09-10T07:46:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.803+0000 7fb5efcc0740 10 client.0 _ll_register_callbacks cb 0x556a43b4af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.806+0000 7fb5dd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c12870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.806+0000 7fb5dd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c12870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43408/0 (socket says 172.21.15.46:43408) 2024-09-10T07:46:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.806+0000 7fb5dd400640 1 -- 172.21.15.46:0/679857175 learned_addr learned my addr 172.21.15.46:0/679857175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.807+0000 7fb5dd400640 1 -- 172.21.15.46:0/679857175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 msgr2=0x556a43c12330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.807+0000 7fb5dd400640 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c12330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.807+0000 7fb5dd400640 1 -- 172.21.15.46:0/679857175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 msgr2=0x556a43c198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.807+0000 7fb5dd400640 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 0x556a43c198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.807+0000 7fb5dd400640 1 -- 172.21.15.46:0/679857175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4394c620 con 0x556a43c089d0 2024-09-10T07:46:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.807+0000 7fb5e6800640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5cc0a2cb0 0x7fb5cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.808+0000 7fb5dd400640 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c12870 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00106a0 tx=0x7fb5d0010b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e85929e8caa78c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.808+0000 7fb5cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.810+0000 7fb5e6800640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5cc0a2cb0 0x7fb5cc0a5170 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005ec0 tx=0x7fb5e0005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.811+0000 7fb5cbe00640 1 -- 172.21.15.46:0/679857175 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0010ea0 con 0x556a43c089d0 2024-09-10T07:46:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.811+0000 7fb5cbe00640 1 -- 172.21.15.46:0/679857175 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb5d0004020 con 0x556a43c089d0 2024-09-10T07:46:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.811+0000 7fb5efcc0740 1 -- 172.21.15.46:0/679857175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d80d8040 con 0x556a43c089d0 2024-09-10T07:46:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.812+0000 7fb5cbe00640 1 -- 172.21.15.46:0/679857175 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb5d0010ea0 con 0x556a43c089d0 2024-09-10T07:46:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.812+0000 7fb5efcc0740 10 client.27778 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.813+0000 7fb5efcc0740 1 -- 172.21.15.46:0/679857175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556a43c1c7c0 con 0x556a43c089d0 2024-09-10T07:46:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.814+0000 7fb5cbe00640 1 -- 172.21.15.46:0/679857175 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d000edb0 con 0x556a43c089d0 2024-09-10T07:46:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.815+0000 7fb5cbe00640 1 -- 172.21.15.46:0/679857175 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fb5d000f040 con 0x556a43c089d0 2024-09-10T07:46:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.816+0000 7fb5efcc0740 10 client.27778 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.816+0000 7fb5efcc0740 10 client.27778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.816+0000 7fb5efcc0740 20 client.27778 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.816+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556a43c1df00 0x556a43c202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.816+0000 7fb5efcc0740 4 client.27778 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.817+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556a43c1df00 0x556a43c202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.818+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556a43c1df00 0x556a43c202f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c324f66852e1e5b server_cookie=3ad71d5f16371bc3 in_seq=0 out_seq=0 2024-09-10T07:46:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.818+0000 7fb5efcc0740 1 -- 172.21.15.46:0/679857175 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb5d80d8040 con 0x556a43c1df00 2024-09-10T07:46:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.818+0000 7fb5cbe00640 10 client.27778 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.822+0000 7fb5cbe00640 1 -- 172.21.15.46:0/679857175 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb5d80d8040 con 0x556a43c1df00 2024-09-10T07:46:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.822+0000 7fb5cbe00640 10 client.27778 handle_command_reply: tid=0 2024-09-10T07:46:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.822+0000 7fb5efcc0740 1 client.27778 shutdown 2024-09-10T07:46:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5efcc0740 1 -- 172.21.15.46:0/679857175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 msgr2=0x556a43c12870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c12870 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00106a0 tx=0x7fb5d0010b60 comp rx=0 tx=0).stop 2024-09-10T07:46:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5efcc0740 1 -- 172.21.15.46:0/679857175 shutdown_connections 2024-09-10T07:46:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556a43c1df00 0x556a43c202f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 0x556a43c198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c12870 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/679857175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c12330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5cbe00640 0 client.27778 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.823+0000 7fb5efcc0740 1 -- 172.21.15.46:0/679857175 >> 172.21.15.46:0/679857175 conn(0x556a43c03700 msgr2=0x556a43c04f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.825+0000 7fb5efcc0740 1 -- 172.21.15.46:0/679857175 shutdown_connections 2024-09-10T07:46:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.826+0000 7fb5efcc0740 1 -- 172.21.15.46:0/679857175 wait complete. 2024-09-10T07:46:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.826+0000 7fb5efcc0740 20 client.27778 trim_cache size 0 max 16384 2024-09-10T07:46:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.867+0000 7fb5efcc0740 1 Processor -- start 2024-09-10T07:46:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.867+0000 7fb5efcc0740 1 -- start start 2024-09-10T07:46:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.868+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c02770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.868+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c19870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.868+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 0x556a43c24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.868+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81b0aa0 con 0x556a43c0da60 2024-09-10T07:46:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.868+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81b25e0 con 0x556a43c08090 2024-09-10T07:46:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.868+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81b1170 con 0x556a43c089d0 2024-09-10T07:46:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.869+0000 7fb5dd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c19870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.869+0000 7fb5e5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c02770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.869+0000 7fb5e5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c02770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35500/0 (socket says 172.21.15.46:35500) 2024-09-10T07:46:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.869+0000 7fb5dd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c19870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43416/0 (socket says 172.21.15.46:43416) 2024-09-10T07:46:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.869+0000 7fb5e5e00640 1 -- 172.21.15.46:0/1919973236 learned_addr learned my addr 172.21.15.46:0/1919973236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:26.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.869+0000 7fb5e6800640 1 --2- 172.21.15.46:0/1919973236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 0x556a43c24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.871+0000 7fb5e6800640 1 -- 172.21.15.46:0/1919973236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 msgr2=0x556a43c19870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.871+0000 7fb5e6800640 1 --2- 172.21.15.46:0/1919973236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c19870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.871+0000 7fb5e6800640 1 -- 172.21.15.46:0/1919973236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 msgr2=0x556a43c02770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.871+0000 7fb5e6800640 1 --2- 172.21.15.46:0/1919973236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c02770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.871+0000 7fb5e6800640 1 -- 172.21.15.46:0/1919973236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a43c20660 con 0x556a43c0da60 2024-09-10T07:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.872+0000 7fb5e6800640 1 --2- 172.21.15.46:0/1919973236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 0x556a43c24bb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006010 tx=0x7fb5e004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa4f2aecde0855b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.873+0000 7fb5d6a00640 1 -- 172.21.15.46:0/1919973236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0052070 con 0x556a43c0da60 2024-09-10T07:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.873+0000 7fb5d6a00640 1 -- 172.21.15.46:0/1919973236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e005f070 con 0x556a43c0da60 2024-09-10T07:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.873+0000 7fb5d6a00640 1 -- 172.21.15.46:0/1919973236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0046040 con 0x556a43c0da60 2024-09-10T07:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.874+0000 7fb5efcc0740 1 -- 172.21.15.46:0/1919973236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 msgr2=0x556a43c24bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.874+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/1919973236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 0x556a43c24bb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006010 tx=0x7fb5e004e000 comp rx=0 tx=0).stop 2024-09-10T07:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.875+0000 7fb5efcc0740 1 -- 172.21.15.46:0/1919973236 shutdown_connections 2024-09-10T07:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.875+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/1919973236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c0da60 0x556a43c24bb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.875+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/1919973236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c089d0 0x556a43c19870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.875+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/1919973236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c02770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.875+0000 7fb5efcc0740 1 -- 172.21.15.46:0/1919973236 >> 172.21.15.46:0/1919973236 conn(0x556a43c03700 msgr2=0x556a43c02060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.875+0000 7fb5efcc0740 1 -- 172.21.15.46:0/1919973236 shutdown_connections 2024-09-10T07:46:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.877+0000 7fb5efcc0740 1 -- 172.21.15.46:0/1919973236 wait complete. 2024-09-10T07:46:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.877+0000 7fb5efcc0740 1 -- start start 2024-09-10T07:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.879+0000 7fb5efcc0740 1 Processor -- start 2024-09-10T07:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.879+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c16db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.879+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c089d0 0x556a43c172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.879+0000 7fb5efcc0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c0da60 0x556a43c18300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.879+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81b1f10 con 0x556a43c089d0 2024-09-10T07:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.879+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81b1840 con 0x556a43c08090 2024-09-10T07:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.879+0000 7fb5efcc0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81aeb30 con 0x556a43c0da60 2024-09-10T07:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.879+0000 7fb5efcc0740 10 client.0 _ll_register_callbacks cb 0x7fb5d00093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.880+0000 7fb5dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c089d0 0x556a43c172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.880+0000 7fb5e5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c16db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.880+0000 7fb5dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c089d0 0x556a43c172f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42664/0 (socket says 172.21.15.46:42664) 2024-09-10T07:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.880+0000 7fb5dd400640 1 -- 172.21.15.46:0/3988981468 learned_addr learned my addr 172.21.15.46:0/3988981468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.880+0000 7fb5e6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c0da60 0x556a43c18300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.881+0000 7fb5dd400640 1 -- 172.21.15.46:0/3988981468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c0da60 msgr2=0x556a43c18300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.881+0000 7fb5dd400640 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c0da60 0x556a43c18300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.881+0000 7fb5dd400640 1 -- 172.21.15.46:0/3988981468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 msgr2=0x556a43c16db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.881+0000 7fb5dd400640 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c16db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.881+0000 7fb5dd400640 1 -- 172.21.15.46:0/3988981468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a43c18910 con 0x556a43c089d0 2024-09-10T07:46:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.883+0000 7fb5dd400640 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c089d0 0x556a43c172f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0003d10 tx=0x7fb5d0003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17abee87d2c2e53f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.883+0000 7fb5cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:46:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.883+0000 7fb5cbe00640 1 -- 172.21.15.46:0/3988981468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d002b050 con 0x556a43c089d0 2024-09-10T07:46:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.884+0000 7fb5cbe00640 1 -- 172.21.15.46:0/3988981468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb5d0005020 con 0x556a43c089d0 2024-09-10T07:46:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.884+0000 7fb5efcc0740 1 -- 172.21.15.46:0/3988981468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e005f070 con 0x556a43c089d0 2024-09-10T07:46:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.885+0000 7fb5cbe00640 1 -- 172.21.15.46:0/3988981468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb5d0005220 con 0x556a43c089d0 2024-09-10T07:46:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.885+0000 7fb5efcc0740 10 client.17211 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.885+0000 7fb5efcc0740 1 -- 172.21.15.46:0/3988981468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556a43c88b90 con 0x556a43c089d0 2024-09-10T07:46:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.886+0000 7fb5cbe00640 1 -- 172.21.15.46:0/3988981468 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e005f070 con 0x556a43c089d0 2024-09-10T07:46:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.886+0000 7fb5cbe00640 1 -- 172.21.15.46:0/3988981468 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fb5d0004200 con 0x556a43c089d0 2024-09-10T07:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.886+0000 7fb5efcc0740 10 client.17211 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.886+0000 7fb5efcc0740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.886+0000 7fb5efcc0740 20 client.17211 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.886+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556a43c893d0 0x556a43c8b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.886+0000 7fb5efcc0740 4 client.17211 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.886+0000 7fb5efcc0740 1 -- 172.21.15.46:0/3988981468 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a43b7a830 con 0x556a43c893d0 2024-09-10T07:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.887+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556a43c893d0 0x556a43c8b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.889+0000 7fb5e5e00640 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556a43c893d0 0x556a43c8b7f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f937989f7f73a0 server_cookie=d07373d6d5a9e775 in_seq=0 out_seq=0 2024-09-10T07:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.890+0000 7fb5cbe00640 10 client.17211 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.944+0000 7fb5cbe00640 1 -- 172.21.15.46:0/3988981468 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a43b7a830 con 0x556a43c893d0 2024-09-10T07:46:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.944+0000 7fb5cbe00640 10 client.17211 handle_command_reply: tid=0 2024-09-10T07:46:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.944+0000 7fb5efcc0740 1 client.17211 shutdown 2024-09-10T07:46:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 -- 172.21.15.46:0/3988981468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c089d0 msgr2=0x556a43c172f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c089d0 0x556a43c172f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0003d10 tx=0x7fb5d0003670 comp rx=0 tx=0).stop 2024-09-10T07:46:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 -- 172.21.15.46:0/3988981468 shutdown_connections 2024-09-10T07:46:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556a43c893d0 0x556a43c8b7f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a43c0da60 0x556a43c18300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a43c089d0 0x556a43c172f0 secure :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0003d10 tx=0x7fb5d0003670 comp rx=0 tx=0).stop 2024-09-10T07:46:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 --2- 172.21.15.46:0/3988981468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a43c08090 0x556a43c16db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 -- 172.21.15.46:0/3988981468 >> 172.21.15.46:0/3988981468 conn(0x556a43c03700 msgr2=0x556a43c02880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5cbe00640 0 client.17211 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 -- 172.21.15.46:0/3988981468 shutdown_connections 2024-09-10T07:46:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.945+0000 7fb5efcc0740 1 -- 172.21.15.46:0/3988981468 wait complete. 2024-09-10T07:46:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.946+0000 7fb5efcc0740 20 client.17211 trim_cache size 0 max 16384 2024-09-10T07:46:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:46:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:46:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:46:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:46:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 -- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5cc0a2cb0 msgr2=0x7fb5cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5cc0a2cb0 0x7fb5cc0a5170 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005ec0 tx=0x7fb5e0005df0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 -- 172.21.15.46:0/3636112038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5e81003c0 msgr2=0x7fb5e806bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5e81003c0 0x7fb5e806bcb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800bd70 tx=0x7fb5d800be60 comp rx=0 tx=0).stop 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 -- 172.21.15.46:0/3636112038 shutdown_connections 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e81056c0 0x7fb5e8072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5cc0a2cb0 0x7fb5cc0a5170 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5e8102d10 0x7fb5e806c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 --2- 172.21.15.46:0/3636112038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5e81003c0 0x7fb5e806bcb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.947+0000 7fb5ece00640 1 -- 172.21.15.46:0/3636112038 >> 172.21.15.46:0/3636112038 conn(0x7fb5e80fc1b0 msgr2=0x7fb5e80fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.948+0000 7fb5ece00640 1 -- 172.21.15.46:0/3636112038 shutdown_connections 2024-09-10T07:46:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:26.948+0000 7fb5ece00640 1 -- 172.21.15.46:0/3636112038 wait complete. 2024-09-10T07:46:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.204+0000 7fe83c800640 1 -- 172.21.15.46:0/261071813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 msgr2=0x7fe834074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.204+0000 7fe83c800640 1 --2- 172.21.15.46:0/261071813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 0x7fe834074a00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe828009900 tx=0x7fe82802a480 comp rx=0 tx=0).stop 2024-09-10T07:46:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.205+0000 7fe83c800640 1 -- 172.21.15.46:0/261071813 shutdown_connections 2024-09-10T07:46:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.205+0000 7fe83c800640 1 --2- 172.21.15.46:0/261071813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83410e3c0 0x7fe834110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.205+0000 7fe83c800640 1 --2- 172.21.15.46:0/261071813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe834074f40 0x7fe83410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.205+0000 7fe83c800640 1 --2- 172.21.15.46:0/261071813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 0x7fe834074a00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.205+0000 7fe83c800640 1 -- 172.21.15.46:0/261071813 >> 172.21.15.46:0/261071813 conn(0x7fe8340fd8c0 msgr2=0x7fe8340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.206+0000 7fe83c800640 1 -- 172.21.15.46:0/261071813 shutdown_connections 2024-09-10T07:46:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.206+0000 7fe83c800640 1 -- 172.21.15.46:0/261071813 wait complete. 2024-09-10T07:46:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.208+0000 7fe83c800640 1 Processor -- start 2024-09-10T07:46:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.208+0000 7fe83c800640 1 -- start start 2024-09-10T07:46:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.208+0000 7fe83c800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 0x7fe83419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe83c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe834074f40 0x7fe83419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe83c800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83410e3c0 0x7fe83419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe83c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe834114150 con 0x7fe834074f40 2024-09-10T07:46:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe83c800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe834113fd0 con 0x7fe834074620 2024-09-10T07:46:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe83c800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8341142d0 con 0x7fe83410e3c0 2024-09-10T07:46:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe839e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 0x7fe83419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe839e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 0x7fe83419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35532/0 (socket says 172.21.15.46:35532) 2024-09-10T07:46:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe839e00640 1 -- 172.21.15.46:0/1240082827 learned_addr learned my addr 172.21.15.46:0/1240082827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe839400640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe834074f40 0x7fe83419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.209+0000 7fe83a800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83410e3c0 0x7fe83419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.210+0000 7fe839400640 1 -- 172.21.15.46:0/1240082827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83410e3c0 msgr2=0x7fe83419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.210+0000 7fe839400640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83410e3c0 0x7fe83419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.210+0000 7fe839400640 1 -- 172.21.15.46:0/1240082827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 msgr2=0x7fe83419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.210+0000 7fe839400640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 0x7fe83419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.210+0000 7fe839400640 1 -- 172.21.15.46:0/1240082827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83419f600 con 0x7fe834074f40 2024-09-10T07:46:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.210+0000 7fe839e00640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 0x7fe83419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.211+0000 7fe839400640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe834074f40 0x7fe83419ab90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe82400e9f0 tx=0x7fe82400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.212+0000 7fe82ea00640 1 -- 172.21.15.46:0/1240082827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82400ccc0 con 0x7fe834074f40 2024-09-10T07:46:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.212+0000 7fe82ea00640 1 -- 172.21.15.46:0/1240082827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe824010910 con 0x7fe834074f40 2024-09-10T07:46:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.212+0000 7fe82ea00640 1 -- 172.21.15.46:0/1240082827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824018960 con 0x7fe834074f40 2024-09-10T07:46:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.212+0000 7fe83c800640 1 -- 172.21.15.46:0/1240082827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83419f8f0 con 0x7fe834074f40 2024-09-10T07:46:27.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.212+0000 7fe83c800640 1 -- 172.21.15.46:0/1240082827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe834101a10 con 0x7fe834074f40 2024-09-10T07:46:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.218+0000 7fe83f431740 1 Processor -- start 2024-09-10T07:46:27.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.218+0000 7fe82ea00640 1 -- 172.21.15.46:0/1240082827 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe824010ac0 con 0x7fe834074f40 2024-09-10T07:46:27.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.220+0000 7fe83f431740 1 -- start start 2024-09-10T07:46:27.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.226+0000 7fe83f431740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e529c70 0x560b2e52a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.226+0000 7fe83f431740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e534910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.226+0000 7fe83f431740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e534e50 0x560b2e537240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.226+0000 7fe83f431740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b2e4903e0 con 0x560b2e534e50 2024-09-10T07:46:27.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.226+0000 7fe83f431740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560b2e437d80 con 0x560b2e529c70 2024-09-10T07:46:27.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.226+0000 7fe83f431740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b2e48c3a0 con 0x560b2e52a590 2024-09-10T07:46:27.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.227+0000 7fe82ea00640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe81c0a2c00 0x7fe81c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.227+0000 7fe82ea00640 1 -- 172.21.15.46:0/1240082827 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe8240cf6b0 con 0x7fe834074f40 2024-09-10T07:46:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.228+0000 7fe839e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e534910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.228+0000 7fe83a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e529c70 0x560b2e52a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.228+0000 7fe83a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e529c70 0x560b2e52a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35538/0 (socket says 172.21.15.46:35538) 2024-09-10T07:46:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.228+0000 7fe83a800640 1 -- 172.21.15.46:0/3928937686 learned_addr learned my addr 172.21.15.46:0/3928937686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.229+0000 7fe83a800640 1 -- 172.21.15.46:0/3928937686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 msgr2=0x560b2e534910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.229+0000 7fe83a800640 1 --2- 172.21.15.46:0/3928937686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e534910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.229+0000 7fe83a800640 1 -- 172.21.15.46:0/3928937686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e534e50 msgr2=0x560b2e537240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.229+0000 7fe83a800640 1 --2- 172.21.15.46:0/3928937686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e534e50 0x560b2e537240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.229+0000 7fe83a800640 1 -- 172.21.15.46:0/3928937686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2e456ae0 con 0x560b2e529c70 2024-09-10T07:46:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.229+0000 7fe83a800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe81c0a2c00 0x7fe81c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.230+0000 7fe83a800640 1 --2- 172.21.15.46:0/3928937686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e529c70 0x560b2e52a050 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe8300099e0 tx=0x7fe830009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd91b7917d11e7e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.231+0000 7fe823000640 1 -- 172.21.15.46:0/3928937686 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830025040 con 0x560b2e529c70 2024-09-10T07:46:27.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.232+0000 7fe823000640 1 -- 172.21.15.46:0/3928937686 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe83000be30 con 0x560b2e529c70 2024-09-10T07:46:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.234+0000 7fe83f431740 1 -- 172.21.15.46:0/3928937686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e529c70 msgr2=0x560b2e52a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.234+0000 7fe83f431740 1 --2- 172.21.15.46:0/3928937686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e529c70 0x560b2e52a050 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe8300099e0 tx=0x7fe830009ea0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.234+0000 7fe83a800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe81c0a2c00 0x7fe81c0a50c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe83002b000 tx=0x7fe83002b780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.234+0000 7fe83f431740 1 -- 172.21.15.46:0/3928937686 shutdown_connections 2024-09-10T07:46:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.234+0000 7fe83f431740 1 --2- 172.21.15.46:0/3928937686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e534e50 0x560b2e537240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.234+0000 7fe83f431740 1 --2- 172.21.15.46:0/3928937686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e534910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.234+0000 7fe83f431740 1 --2- 172.21.15.46:0/3928937686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e529c70 0x560b2e52a050 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.234+0000 7fe83f431740 1 -- 172.21.15.46:0/3928937686 >> 172.21.15.46:0/3928937686 conn(0x560b2e526200 msgr2=0x560b2e5265e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.236+0000 7fe83f431740 1 -- 172.21.15.46:0/3928937686 shutdown_connections 2024-09-10T07:46:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.236+0000 7fe83f431740 1 -- 172.21.15.46:0/3928937686 wait complete. 2024-09-10T07:46:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.236+0000 7fe83f431740 1 -- start start 2024-09-10T07:46:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.237+0000 7fe83f431740 1 Processor -- start 2024-09-10T07:46:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.237+0000 7fe83f431740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a590 0x560b2e5347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.237+0000 7fe83f431740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e534e50 0x560b2e531a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.237+0000 7fe83f431740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e5323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.237+0000 7fe83f431740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b2e537ac0 con 0x560b2e52a590 2024-09-10T07:46:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.237+0000 7fe83f431740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560b2e44b0f0 con 0x560b2e531f90 2024-09-10T07:46:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.237+0000 7fe83f431740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560b2e537c40 con 0x560b2e534e50 2024-09-10T07:46:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.238+0000 7fe83f431740 10 client.0 _ll_register_callbacks cb 0x560b2e432bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.238+0000 7fe839e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a590 0x560b2e5347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.238+0000 7fe839400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e534e50 0x560b2e531a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.239+0000 7fe839e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a590 0x560b2e5347a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42694/0 (socket says 172.21.15.46:42694) 2024-09-10T07:46:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.239+0000 7fe839400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e534e50 0x560b2e531a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43456/0 (socket says 172.21.15.46:43456) 2024-09-10T07:46:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.239+0000 7fe839e00640 1 -- 172.21.15.46:0/100876817 learned_addr learned my addr 172.21.15.46:0/100876817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.240+0000 7fe839400640 1 -- 172.21.15.46:0/100876817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 msgr2=0x560b2e5323f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.240+0000 7fe839400640 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e5323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.240+0000 7fe839400640 1 -- 172.21.15.46:0/100876817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a590 msgr2=0x560b2e5347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.240+0000 7fe839400640 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a590 0x560b2e5347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.240+0000 7fe839400640 1 -- 172.21.15.46:0/100876817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2e2013c0 con 0x560b2e534e50 2024-09-10T07:46:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.241+0000 7fe839e00640 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a590 0x560b2e5347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.241+0000 7fe839400640 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e534e50 0x560b2e531a50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x560b2e527580 tx=0x7fe82409b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523e855cc4527ed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.241+0000 7fe817400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.243+0000 7fe817400640 1 -- 172.21.15.46:0/100876817 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8240cf430 con 0x560b2e534e50 2024-09-10T07:46:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.243+0000 7fe83f431740 1 -- 172.21.15.46:0/100876817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560b2e49d950 con 0x560b2e534e50 2024-09-10T07:46:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.244+0000 7fe817400640 1 -- 172.21.15.46:0/100876817 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe8240cfb60 con 0x560b2e534e50 2024-09-10T07:46:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.244+0000 7fe817400640 1 -- 172.21.15.46:0/100876817 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe8240cfd60 con 0x560b2e534e50 2024-09-10T07:46:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.244+0000 7fe83f431740 10 client.27796 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:27.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.248+0000 7fe83f431740 1 -- 172.21.15.46:0/100876817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560b2e554d40 con 0x560b2e534e50 2024-09-10T07:46:27.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.249+0000 7fe817400640 1 -- 172.21.15.46:0/100876817 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560b2e49d950 con 0x560b2e534e50 2024-09-10T07:46:27.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.249+0000 7fe817400640 1 -- 172.21.15.46:0/100876817 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fe8240ce070 con 0x560b2e534e50 2024-09-10T07:46:27.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.250+0000 7fe83f431740 10 client.27796 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:27.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.250+0000 7fe83f431740 10 client.27796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:46:27.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.250+0000 7fe83f431740 20 client.27796 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:27.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.250+0000 7fe83f431740 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x560b2e556150 0x560b2e558540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.250+0000 7fe83f431740 4 client.27796 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:27.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.250+0000 7fe83f431740 1 -- 172.21.15.46:0/100876817 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b2e497e60 con 0x560b2e556150 2024-09-10T07:46:27.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.252+0000 7fe839e00640 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x560b2e556150 0x560b2e558540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.253+0000 7fe839e00640 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x560b2e556150 0x560b2e558540 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1021376528f30464 server_cookie=14c6cdf9bcdecf3b in_seq=0 out_seq=0 2024-09-10T07:46:27.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.253+0000 7fe817400640 10 client.27796 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:46:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.254+0000 7fe817400640 1 -- 172.21.15.46:0/100876817 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b2e497e60 con 0x560b2e556150 2024-09-10T07:46:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.254+0000 7fe817400640 10 client.27796 handle_command_reply: tid=0 2024-09-10T07:46:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.254+0000 7fe83f431740 1 client.27796 shutdown 2024-09-10T07:46:27.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe83f431740 1 -- 172.21.15.46:0/100876817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e534e50 msgr2=0x560b2e531a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe83f431740 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e534e50 0x560b2e531a50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x560b2e527580 tx=0x7fe82409b660 comp rx=0 tx=0).stop 2024-09-10T07:46:27.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe83f431740 1 -- 172.21.15.46:0/100876817 shutdown_connections 2024-09-10T07:46:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe83f431740 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x560b2e556150 0x560b2e558540 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe83f431740 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e5323f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe83f431740 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e534e50 0x560b2e531a50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe83f431740 1 --2- 172.21.15.46:0/100876817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a590 0x560b2e5347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe817400640 0 client.27796 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:46:27.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.256+0000 7fe83f431740 1 -- 172.21.15.46:0/100876817 >> 172.21.15.46:0/100876817 conn(0x560b2e526200 msgr2=0x560b2e52b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:27.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.257+0000 7fe83f431740 1 -- 172.21.15.46:0/100876817 shutdown_connections 2024-09-10T07:46:27.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.258+0000 7fe83f431740 1 -- 172.21.15.46:0/100876817 wait complete. 2024-09-10T07:46:27.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.258+0000 7fe83f431740 20 client.27796 trim_cache size 0 max 16384 2024-09-10T07:46:27.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.292+0000 7fe83f431740 1 Processor -- start 2024-09-10T07:46:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.293+0000 7fe83f431740 1 -- start start 2024-09-10T07:46:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.293+0000 7fe83f431740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e531310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.293+0000 7fe83f431740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e531850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.293+0000 7fe83f431740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a100 0x560b2e52b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.293+0000 7fe83f431740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8341acc10 con 0x560b2e52a100 2024-09-10T07:46:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.293+0000 7fe83f431740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8341ae750 con 0x560b2e531f90 2024-09-10T07:46:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.293+0000 7fe83f431740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8341ad2e0 con 0x560b2e52a590 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e531850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e531850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35546/0 (socket says 172.21.15.46:35546) 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839400640 1 -- 172.21.15.46:0/2647460577 learned_addr learned my addr 172.21.15.46:0/2647460577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839e00640 1 --2- 172.21.15.46:0/2647460577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e531310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839400640 1 -- 172.21.15.46:0/2647460577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 msgr2=0x560b2e531310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839400640 1 --2- 172.21.15.46:0/2647460577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e531310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839400640 1 -- 172.21.15.46:0/2647460577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a100 msgr2=0x560b2e52b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839400640 1 --2- 172.21.15.46:0/2647460577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a100 0x560b2e52b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.294+0000 7fe839400640 1 -- 172.21.15.46:0/2647460577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2e5375b0 con 0x560b2e531f90 2024-09-10T07:46:27.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.295+0000 7fe839400640 1 --2- 172.21.15.46:0/2647460577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e531850 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe82409bbd0 tx=0x7fe8240cfa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a72ce1844cc1b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.296+0000 7fe822600640 1 -- 172.21.15.46:0/2647460577 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8240cfd60 con 0x560b2e531f90 2024-09-10T07:46:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.296+0000 7fe822600640 1 -- 172.21.15.46:0/2647460577 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe824018b00 con 0x560b2e531f90 2024-09-10T07:46:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.296+0000 7fe822600640 1 -- 172.21.15.46:0/2647460577 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8240ba2c0 con 0x560b2e531f90 2024-09-10T07:46:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.298+0000 7fe83f431740 1 -- 172.21.15.46:0/2647460577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 msgr2=0x560b2e531850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.298+0000 7fe83f431740 1 --2- 172.21.15.46:0/2647460577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e531850 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe82409bbd0 tx=0x7fe8240cfa20 comp rx=0 tx=0).stop 2024-09-10T07:46:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.299+0000 7fe83f431740 1 -- 172.21.15.46:0/2647460577 shutdown_connections 2024-09-10T07:46:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.299+0000 7fe83f431740 1 --2- 172.21.15.46:0/2647460577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e52a100 0x560b2e52b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.299+0000 7fe83f431740 1 --2- 172.21.15.46:0/2647460577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e531f90 0x560b2e531850 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.299+0000 7fe83f431740 1 --2- 172.21.15.46:0/2647460577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e531310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.299+0000 7fe83f431740 1 -- 172.21.15.46:0/2647460577 >> 172.21.15.46:0/2647460577 conn(0x560b2e526200 msgr2=0x560b2e5299f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.299+0000 7fe83f431740 1 -- 172.21.15.46:0/2647460577 shutdown_connections 2024-09-10T07:46:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.299+0000 7fe83f431740 1 -- 172.21.15.46:0/2647460577 wait complete. 2024-09-10T07:46:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.300+0000 7fe83f431740 1 -- start start 2024-09-10T07:46:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.301+0000 7fe83f431740 1 Processor -- start 2024-09-10T07:46:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.303+0000 7fe83f431740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e52a100 0x560b2e53cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.303+0000 7fe83f431740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e53d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.303+0000 7fe83f431740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e531f90 0x560b2e5ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.303+0000 7fe83f431740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8341ae080 con 0x560b2e531f90 2024-09-10T07:46:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.303+0000 7fe83f431740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8341ad9b0 con 0x560b2e52a100 2024-09-10T07:46:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.303+0000 7fe83f431740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8341aabb0 con 0x560b2e52a590 2024-09-10T07:46:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.303+0000 7fe83f431740 10 client.0 _ll_register_callbacks cb 0x7fe82400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e52a100 0x560b2e53cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e52a100 0x560b2e53cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35562/0 (socket says 172.21.15.46:35562) 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839e00640 1 -- 172.21.15.46:0/2807139502 learned_addr learned my addr 172.21.15.46:0/2807139502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe83a800640 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e531f90 0x560b2e5ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839400640 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e53d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839e00640 1 -- 172.21.15.46:0/2807139502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 msgr2=0x560b2e53d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839e00640 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e53d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839e00640 1 -- 172.21.15.46:0/2807139502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e531f90 msgr2=0x560b2e5ba0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839e00640 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e531f90 0x560b2e5ba0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.304+0000 7fe839e00640 1 -- 172.21.15.46:0/2807139502 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2e5ba680 con 0x560b2e52a100 2024-09-10T07:46:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.305+0000 7fe817400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.305+0000 7fe839e00640 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e52a100 0x560b2e53cee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe828009230 tx=0x7fe828004130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08d78a4ada3ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.308+0000 7fe817400640 1 -- 172.21.15.46:0/2807139502 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828035040 con 0x560b2e52a100 2024-09-10T07:46:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.308+0000 7fe83f431740 1 -- 172.21.15.46:0/2807139502 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe824018b00 con 0x560b2e52a100 2024-09-10T07:46:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.308+0000 7fe817400640 1 -- 172.21.15.46:0/2807139502 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe828005040 con 0x560b2e52a100 2024-09-10T07:46:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.309+0000 7fe817400640 1 -- 172.21.15.46:0/2807139502 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe828035040 con 0x560b2e52a100 2024-09-10T07:46:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.310+0000 7fe83f431740 10 client.27851 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.310+0000 7fe83f431740 1 -- 172.21.15.46:0/2807139502 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560b2e5d26b0 con 0x560b2e52a100 2024-09-10T07:46:27.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.310+0000 7fe817400640 1 -- 172.21.15.46:0/2807139502 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe824018b00 con 0x560b2e52a100 2024-09-10T07:46:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.313+0000 7fe817400640 1 -- 172.21.15.46:0/2807139502 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fe828004760 con 0x560b2e52a100 2024-09-10T07:46:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.313+0000 7fe83f431740 10 client.27851 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.313+0000 7fe83f431740 10 client.27851 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:46:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.313+0000 7fe83f431740 20 client.27851 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.313+0000 7fe83f431740 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x560b2e5d2d30 0x560b2e5d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.313+0000 7fe83f431740 4 client.27851 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:46:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.313+0000 7fe83f431740 1 -- 172.21.15.46:0/2807139502 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b2e414010 con 0x560b2e5d2d30 2024-09-10T07:46:27.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.313+0000 7fe839400640 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x560b2e5d2d30 0x560b2e5d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.314+0000 7fe839400640 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x560b2e5d2d30 0x560b2e5d5120 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da35e90aa9d96e80 server_cookie=ec36d9fed96c4525 in_seq=0 out_seq=0 2024-09-10T07:46:27.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.314+0000 7fe817400640 10 client.27851 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:46:27.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.314+0000 7fe817400640 1 -- 172.21.15.46:0/2807139502 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b2e414010 con 0x560b2e5d2d30 2024-09-10T07:46:27.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.314+0000 7fe817400640 10 client.27851 handle_command_reply: tid=0 2024-09-10T07:46:27.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.314+0000 7fe83f431740 1 client.27851 shutdown 2024-09-10T07:46:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.314+0000 7fe83f431740 1 -- 172.21.15.46:0/2807139502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e52a100 msgr2=0x560b2e53cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.314+0000 7fe83f431740 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e52a100 0x560b2e53cee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe828009230 tx=0x7fe828004130 comp rx=0 tx=0).stop 2024-09-10T07:46:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 1 -- 172.21.15.46:0/2807139502 shutdown_connections 2024-09-10T07:46:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x560b2e5d2d30 0x560b2e5d5120 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b2e531f90 0x560b2e5ba0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560b2e52a590 0x560b2e53d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 1 --2- 172.21.15.46:0/2807139502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560b2e52a100 0x560b2e53cee0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 1 -- 172.21.15.46:0/2807139502 >> 172.21.15.46:0/2807139502 conn(0x560b2e526200 msgr2=0x560b2e52b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe817400640 0 client.27851 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:46:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 1 -- 172.21.15.46:0/2807139502 shutdown_connections 2024-09-10T07:46:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 1 -- 172.21.15.46:0/2807139502 wait complete. 2024-09-10T07:46:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.315+0000 7fe83f431740 20 client.27851 trim_cache size 0 max 16384 2024-09-10T07:46:27.315 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 -- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe81c0a2c00 msgr2=0x7fe81c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe81c0a2c00 0x7fe81c0a50c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe83002b000 tx=0x7fe83002b780 comp rx=0 tx=0).stop 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 -- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe834074f40 msgr2=0x7fe83419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe834074f40 0x7fe83419ab90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe82400e9f0 tx=0x7fe82400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 -- 172.21.15.46:0/1240082827 shutdown_connections 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83410e3c0 0x7fe83419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe81c0a2c00 0x7fe81c0a50c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe834074f40 0x7fe83419ab90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 --2- 172.21.15.46:0/1240082827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834074620 0x7fe83419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.319+0000 7fe83c800640 1 -- 172.21.15.46:0/1240082827 >> 172.21.15.46:0/1240082827 conn(0x7fe8340fd8c0 msgr2=0x7fe8340fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:27.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.320+0000 7fe83c800640 1 -- 172.21.15.46:0/1240082827 shutdown_connections 2024-09-10T07:46:27.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:27.320+0000 7fe83c800640 1 -- 172.21.15.46:0/1240082827 wait complete. 2024-09-10T07:46:28.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:27 smithi181 ceph-mon[35397]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 613 op/s 2024-09-10T07:46:28.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:27 smithi063 ceph-mon[35020]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 613 op/s 2024-09-10T07:46:28.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:27 smithi046 ceph-mon[30705]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 613 op/s 2024-09-10T07:46:29.203 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:46:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:29 smithi181 ceph-mon[35397]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-09-10T07:46:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:29 smithi063 ceph-mon[35020]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-09-10T07:46:30.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:29 smithi046 ceph-mon[30705]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-09-10T07:46:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 550.0 s, 522.2 tps, lat 189.956 ms stddev 248.286 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 -- 172.21.15.46:0/2767043299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd964102d40 msgr2=0x7fd964103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 --2- 172.21.15.46:0/2767043299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd964102d40 0x7fd964103120 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd954009900 tx=0x7fd95402a480 comp rx=0 tx=0).stop 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 -- 172.21.15.46:0/2767043299 shutdown_connections 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 --2- 172.21.15.46:0/2767043299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd96410c170 0x7fd96410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 --2- 172.21.15.46:0/2767043299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd964103660 0x7fd96410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 --2- 172.21.15.46:0/2767043299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd964102d40 0x7fd964103120 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 -- 172.21.15.46:0/2767043299 >> 172.21.15.46:0/2767043299 conn(0x7fd9640fc7a0 msgr2=0x7fd9640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 -- 172.21.15.46:0/2767043299 shutdown_connections 2024-09-10T07:46:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.819+0000 7fd969c00640 1 -- 172.21.15.46:0/2767043299 wait complete. 2024-09-10T07:46:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.820+0000 7fd969c00640 1 Processor -- start 2024-09-10T07:46:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.820+0000 7fd969c00640 1 -- start start 2024-09-10T07:46:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd969c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd964102d40 0x7fd9641a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd969c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd964103660 0x7fd9641a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd969c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd96410c170 0x7fd96419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd969c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd964102590 con 0x7fd96410c170 2024-09-10T07:46:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd969c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd964102410 con 0x7fd964102d40 2024-09-10T07:46:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd969c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd964102710 con 0x7fd964103660 2024-09-10T07:46:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd962000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd964103660 0x7fd9641a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd962a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd964102d40 0x7fd9641a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd962a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd964102d40 0x7fd9641a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35594/0 (socket says 172.21.15.46:35594) 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd962a00640 1 -- 172.21.15.46:0/813504404 learned_addr learned my addr 172.21.15.46:0/813504404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd962000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd964103660 0x7fd9641a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43496/0 (socket says 172.21.15.46:43496) 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.821+0000 7fd963400640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd96410c170 0x7fd96419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.822+0000 7fd962000640 1 -- 172.21.15.46:0/813504404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd964102d40 msgr2=0x7fd9641a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.822+0000 7fd962000640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd964102d40 0x7fd9641a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.822+0000 7fd962000640 1 -- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd96410c170 msgr2=0x7fd96419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.822+0000 7fd962000640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd96410c170 0x7fd96419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.822+0000 7fd962000640 1 -- 172.21.15.46:0/813504404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96419ae90 con 0x7fd964103660 2024-09-10T07:46:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.822+0000 7fd962000640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd964103660 0x7fd9641a0a40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd95000e7a0 tx=0x7fd95000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.822+0000 7fd95b400640 1 -- 172.21.15.46:0/813504404 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95000ccc0 con 0x7fd964103660 2024-09-10T07:46:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.823+0000 7fd969c00640 1 -- 172.21.15.46:0/813504404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96419b180 con 0x7fd964103660 2024-09-10T07:46:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.823+0000 7fd95b400640 1 -- 172.21.15.46:0/813504404 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd950009530 con 0x7fd964103660 2024-09-10T07:46:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.823+0000 7fd95b400640 1 -- 172.21.15.46:0/813504404 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950018880 con 0x7fd964103660 2024-09-10T07:46:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.823+0000 7fd969c00640 1 -- 172.21.15.46:0/813504404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9641a74f0 con 0x7fd964103660 2024-09-10T07:46:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.826+0000 7fd96c85d740 1 Processor -- start 2024-09-10T07:46:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.826+0000 7fd96c85d740 1 -- start start 2024-09-10T07:46:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b81f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd95b400640 1 -- 172.21.15.46:0/813504404 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd950018a20 con 0x7fd964103660 2024-09-10T07:46:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b95120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b97a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd96c85d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556fc7ae83e0 con 0x556fc7b81b90 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd96c85d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556fc7a8fd80 con 0x556fc7b824b0 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd96c85d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556fc7ae43a0 con 0x556fc7b95660 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd962a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b95120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd962000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b97a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd962000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b97a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43512/0 (socket says 172.21.15.46:43512) 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd962000640 1 -- 172.21.15.46:0/3715217647 learned_addr learned my addr 172.21.15.46:0/3715217647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.827+0000 7fd963400640 1 --2- 172.21.15.46:0/3715217647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b81f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.828+0000 7fd962000640 1 -- 172.21.15.46:0/3715217647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 msgr2=0x556fc7b95120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.828+0000 7fd962000640 1 --2- 172.21.15.46:0/3715217647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b95120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.828+0000 7fd962000640 1 -- 172.21.15.46:0/3715217647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 msgr2=0x556fc7b81f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.828+0000 7fd962000640 1 --2- 172.21.15.46:0/3715217647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b81f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.828+0000 7fd962000640 1 -- 172.21.15.46:0/3715217647 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fc7aaeae0 con 0x556fc7b95660 2024-09-10T07:46:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.828+0000 7fd962000640 1 --2- 172.21.15.46:0/3715217647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b97a50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd9500d5700 tx=0x7fd9500d5bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b9d1143fb4a4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.829+0000 7fd94fe00640 1 -- 172.21.15.46:0/3715217647 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9500e2070 con 0x556fc7b95660 2024-09-10T07:46:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.829+0000 7fd94fe00640 1 -- 172.21.15.46:0/3715217647 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9500d5e10 con 0x556fc7b95660 2024-09-10T07:46:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.829+0000 7fd94fe00640 1 -- 172.21.15.46:0/3715217647 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9500d15f0 con 0x556fc7b95660 2024-09-10T07:46:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.829+0000 7fd95b400640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9480a2c00 0x7fd9480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.829+0000 7fd95b400640 1 -- 172.21.15.46:0/813504404 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd9500cf600 con 0x7fd964103660 2024-09-10T07:46:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.829+0000 7fd963400640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9480a2c00 0x7fd9480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.830+0000 7fd96c85d740 1 -- 172.21.15.46:0/3715217647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 msgr2=0x556fc7b97a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.830+0000 7fd96c85d740 1 --2- 172.21.15.46:0/3715217647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b97a50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd9500d5700 tx=0x7fd9500d5bc0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.830+0000 7fd96c85d740 1 -- 172.21.15.46:0/3715217647 shutdown_connections 2024-09-10T07:46:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.830+0000 7fd96c85d740 1 --2- 172.21.15.46:0/3715217647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b97a50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.831+0000 7fd96c85d740 1 --2- 172.21.15.46:0/3715217647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b95120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.831+0000 7fd96c85d740 1 --2- 172.21.15.46:0/3715217647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b81f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.831+0000 7fd96c85d740 1 -- 172.21.15.46:0/3715217647 >> 172.21.15.46:0/3715217647 conn(0x556fc7b7e200 msgr2=0x556fc7b7e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.832+0000 7fd963400640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9480a2c00 0x7fd9480a50c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd95c005fd0 tx=0x7fd95c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.832+0000 7fd96c85d740 1 -- 172.21.15.46:0/3715217647 shutdown_connections 2024-09-10T07:46:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.833+0000 7fd96c85d740 1 -- 172.21.15.46:0/3715217647 wait complete. 2024-09-10T07:46:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.833+0000 7fd96c85d740 1 -- start start 2024-09-10T07:46:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.834+0000 7fd96c85d740 1 Processor -- start 2024-09-10T07:46:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.834+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.834+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b92af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.834+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b93030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.834+0000 7fd96c85d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556fc7b981d0 con 0x556fc7b81b90 2024-09-10T07:46:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.834+0000 7fd96c85d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556fc7aa30f0 con 0x556fc7b824b0 2024-09-10T07:46:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.834+0000 7fd96c85d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556fc7b98350 con 0x556fc7b95660 2024-09-10T07:46:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.834+0000 7fd96c85d740 10 client.0 _ll_register_callbacks cb 0x556fc7a8abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.835+0000 7fd963400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b93030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.835+0000 7fd963400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b93030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43524/0 (socket says 172.21.15.46:43524) 2024-09-10T07:46:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.835+0000 7fd963400640 1 -- 172.21.15.46:0/4025350037 learned_addr learned my addr 172.21.15.46:0/4025350037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.835+0000 7fd962000640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b92af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.835+0000 7fd962a00640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.835+0000 7fd963400640 1 -- 172.21.15.46:0/4025350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 msgr2=0x556fc7b92af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.835+0000 7fd963400640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b92af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.835+0000 7fd963400640 1 -- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 msgr2=0x556fc7b925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd963400640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b925b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd963400640 1 -- 172.21.15.46:0/4025350037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fc78593c0 con 0x556fc7b95660 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd962000640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b92af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd962a00640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd963400640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b93030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd95c012930 tx=0x7fd95c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfcd0610fb61d98e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd94cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd94cc00640 1 -- 172.21.15.46:0/4025350037 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00fce0 con 0x556fc7b95660 2024-09-10T07:46:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd96c85d740 1 -- 172.21.15.46:0/4025350037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9500d5e10 con 0x556fc7b95660 2024-09-10T07:46:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.836+0000 7fd94cc00640 1 -- 172.21.15.46:0/4025350037 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd95c01f420 con 0x556fc7b95660 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.837+0000 7fd94cc00640 1 -- 172.21.15.46:0/4025350037 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd95c030050 con 0x556fc7b95660 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.837+0000 7fd96c85d740 10 client.27814 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.837+0000 7fd96c85d740 1 -- 172.21.15.46:0/4025350037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556fc7ba4350 con 0x556fc7b95660 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.837+0000 7fd94cc00640 1 -- 172.21.15.46:0/4025350037 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9500d5e10 con 0x556fc7b95660 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.837+0000 7fd94cc00640 1 -- 172.21.15.46:0/4025350037 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fd95c0276c0 con 0x556fc7b95660 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.838+0000 7fd96c85d740 10 client.27814 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.838+0000 7fd96c85d740 10 client.27814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.838+0000 7fd96c85d740 20 client.27814 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.838+0000 7fd96c85d740 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556fc7ba5760 0x556fc7ba7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.838+0000 7fd96c85d740 4 client.27814 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.838+0000 7fd96c85d740 1 -- 172.21.15.46:0/4025350037 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fc7af5950 con 0x556fc7ba5760 2024-09-10T07:46:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.838+0000 7fd962a00640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556fc7ba5760 0x556fc7ba7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.839+0000 7fd962a00640 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556fc7ba5760 0x556fc7ba7b50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e38371cb073d8130 server_cookie=b37abae92f2207a4 in_seq=0 out_seq=0 2024-09-10T07:46:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.839+0000 7fd94cc00640 10 client.27814 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.840+0000 7fd94cc00640 1 -- 172.21.15.46:0/4025350037 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fc7af5950 con 0x556fc7ba5760 2024-09-10T07:46:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.840+0000 7fd94cc00640 10 client.27814 handle_command_reply: tid=0 2024-09-10T07:46:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.840+0000 7fd96c85d740 1 client.27814 shutdown 2024-09-10T07:46:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 -- 172.21.15.46:0/4025350037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 msgr2=0x556fc7b93030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b93030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd95c012930 tx=0x7fd95c004a40 comp rx=0 tx=0).stop 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 -- 172.21.15.46:0/4025350037 shutdown_connections 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556fc7ba5760 0x556fc7ba7b50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b93030 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b92af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 --2- 172.21.15.46:0/4025350037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 -- 172.21.15.46:0/4025350037 >> 172.21.15.46:0/4025350037 conn(0x556fc7b7e200 msgr2=0x556fc7b7d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd94cc00640 0 client.27814 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 -- 172.21.15.46:0/4025350037 shutdown_connections 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 1 -- 172.21.15.46:0/4025350037 wait complete. 2024-09-10T07:46:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.841+0000 7fd96c85d740 20 client.27814 trim_cache size 0 max 16384 2024-09-10T07:46:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.858+0000 7fd96c85d740 1 Processor -- start 2024-09-10T07:46:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.859+0000 7fd96c85d740 1 -- start start 2024-09-10T07:46:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.859+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b91e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.859+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b93e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.859+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7bad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.859+0000 7fd96c85d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641acc30 con 0x556fc7b81b90 2024-09-10T07:46:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.859+0000 7fd96c85d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641ae770 con 0x556fc7b824b0 2024-09-10T07:46:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.859+0000 7fd96c85d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641ad300 con 0x556fc7b95660 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7bad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd962000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b93e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7bad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43536/0 (socket says 172.21.15.46:43536) 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd962000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b93e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35622/0 (socket says 172.21.15.46:35622) 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 -- 172.21.15.46:0/1310676584 learned_addr learned my addr 172.21.15.46:0/1310676584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd962a00640 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b91e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 -- 172.21.15.46:0/1310676584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 msgr2=0x556fc7b93e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b93e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 -- 172.21.15.46:0/1310676584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 msgr2=0x556fc7b91e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b91e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 -- 172.21.15.46:0/1310676584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fc7b94cc0 con 0x556fc7b95660 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd962a00640 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b91e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.860+0000 7fd963400640 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7bad3f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd95c027a90 tx=0x7fd95c00f7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295f774e4fa61448 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.861+0000 7fd94f400640 1 -- 172.21.15.46:0/1310676584 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c02d040 con 0x556fc7b95660 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.861+0000 7fd94f400640 1 -- 172.21.15.46:0/1310676584 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd95c004d70 con 0x556fc7b95660 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.861+0000 7fd94f400640 1 -- 172.21.15.46:0/1310676584 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0347b0 con 0x556fc7b95660 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 -- 172.21.15.46:0/1310676584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 msgr2=0x556fc7bad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7bad3f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd95c027a90 tx=0x7fd95c00f7a0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 -- 172.21.15.46:0/1310676584 shutdown_connections 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7bad3f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b824b0 0x556fc7b93e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 --2- 172.21.15.46:0/1310676584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b81b90 0x556fc7b91e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 -- 172.21.15.46:0/1310676584 >> 172.21.15.46:0/1310676584 conn(0x556fc7b7e200 msgr2=0x556fc7b7d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 -- 172.21.15.46:0/1310676584 shutdown_connections 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 -- 172.21.15.46:0/1310676584 wait complete. 2024-09-10T07:46:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.862+0000 7fd96c85d740 1 -- start start 2024-09-10T07:46:31.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd96c85d740 1 Processor -- start 2024-09-10T07:46:31.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b81b90 0x556fc7b8f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b824b0 0x556fc7b8fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd96c85d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b8ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd96c85d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641ae0a0 con 0x556fc7b824b0 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd96c85d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641ad9d0 con 0x556fc7b81b90 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd96c85d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641aac10 con 0x556fc7b95660 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd96c85d740 10 client.0 _ll_register_callbacks cb 0x7fd95c009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd962a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b81b90 0x556fc7b8f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd962a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b81b90 0x556fc7b8f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35626/0 (socket says 172.21.15.46:35626) 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd962a00640 1 -- 172.21.15.46:0/76439448 learned_addr learned my addr 172.21.15.46:0/76439448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.863+0000 7fd963400640 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b8ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.864+0000 7fd962000640 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b824b0 0x556fc7b8fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.864+0000 7fd962a00640 1 -- 172.21.15.46:0/76439448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 msgr2=0x556fc7b8ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.864+0000 7fd962a00640 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b8ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.864+0000 7fd962a00640 1 -- 172.21.15.46:0/76439448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b824b0 msgr2=0x556fc7b8fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.864+0000 7fd962a00640 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b824b0 0x556fc7b8fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.864+0000 7fd962a00640 1 -- 172.21.15.46:0/76439448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fc7c0f3c0 con 0x556fc7b81b90 2024-09-10T07:46:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.864+0000 7fd962a00640 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b81b90 0x556fc7b8f4f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd954005c40 tx=0x7fd954004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bc221135d0beedb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.864+0000 7fd94cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.865+0000 7fd94cc00640 1 -- 172.21.15.46:0/76439448 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954035040 con 0x556fc7b81b90 2024-09-10T07:46:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.865+0000 7fd94cc00640 1 -- 172.21.15.46:0/76439448 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd954005450 con 0x556fc7b81b90 2024-09-10T07:46:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.865+0000 7fd96c85d740 1 -- 172.21.15.46:0/76439448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd95c004d70 con 0x556fc7b81b90 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.865+0000 7fd94cc00640 1 -- 172.21.15.46:0/76439448 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd954054050 con 0x556fc7b81b90 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.865+0000 7fd96c85d740 10 client.27869 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.865+0000 7fd96c85d740 1 -- 172.21.15.46:0/76439448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556fc7c10640 con 0x556fc7b81b90 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.865+0000 7fd94cc00640 1 -- 172.21.15.46:0/76439448 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd95c004d70 con 0x556fc7b81b90 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.866+0000 7fd94cc00640 1 -- 172.21.15.46:0/76439448 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7fd954003710 con 0x556fc7b81b90 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.866+0000 7fd96c85d740 10 client.27869 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.866+0000 7fd96c85d740 10 client.27869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.866+0000 7fd96c85d740 20 client.27869 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.866+0000 7fd96c85d740 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556fc7c10de0 0x556fc7c13200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.866+0000 7fd96c85d740 4 client.27869 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.866+0000 7fd96c85d740 1 -- 172.21.15.46:0/76439448 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fc7aefe60 con 0x556fc7c10de0 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.866+0000 7fd962000640 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556fc7c10de0 0x556fc7c13200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd962000640 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556fc7c10de0 0x556fc7c13200 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=907681d966b63519 server_cookie=dfa846e542ef64bf in_seq=0 out_seq=0 2024-09-10T07:46:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd94cc00640 10 client.27869 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd94cc00640 1 -- 172.21.15.46:0/76439448 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fc7aefe60 con 0x556fc7c10de0 2024-09-10T07:46:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd94cc00640 10 client.27869 handle_command_reply: tid=0 2024-09-10T07:46:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 client.27869 shutdown 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 -- 172.21.15.46:0/76439448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b81b90 msgr2=0x556fc7b8f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b81b90 0x556fc7b8f4f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd954005c40 tx=0x7fd954004270 comp rx=0 tx=0).stop 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 -- 172.21.15.46:0/76439448 shutdown_connections 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556fc7c10de0 0x556fc7c13200 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556fc7b95660 0x556fc7b8ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fc7b824b0 0x556fc7b8fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 --2- 172.21.15.46:0/76439448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556fc7b81b90 0x556fc7b8f4f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.867+0000 7fd96c85d740 1 -- 172.21.15.46:0/76439448 >> 172.21.15.46:0/76439448 conn(0x556fc7b7e200 msgr2=0x556fc7b92570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.868+0000 7fd94cc00640 0 client.27869 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.868+0000 7fd96c85d740 1 -- 172.21.15.46:0/76439448 shutdown_connections 2024-09-10T07:46:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.868+0000 7fd96c85d740 1 -- 172.21.15.46:0/76439448 wait complete. 2024-09-10T07:46:31.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.868+0000 7fd96c85d740 20 client.27869 trim_cache size 0 max 16384 2024-09-10T07:46:31.870 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:46:31.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 -- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9480a2c00 msgr2=0x7fd9480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9480a2c00 0x7fd9480a50c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd95c005fd0 tx=0x7fd95c009560 comp rx=0 tx=0).stop 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 -- 172.21.15.46:0/813504404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd964103660 msgr2=0x7fd9641a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd964103660 0x7fd9641a0a40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd95000e7a0 tx=0x7fd95000ec60 comp rx=0 tx=0).stop 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 -- 172.21.15.46:0/813504404 shutdown_connections 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd9480a2c00 0x7fd9480a50c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd96410c170 0x7fd96419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd964103660 0x7fd9641a0a40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 --2- 172.21.15.46:0/813504404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd964102d40 0x7fd9641a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.869+0000 7fd969c00640 1 -- 172.21.15.46:0/813504404 >> 172.21.15.46:0/813504404 conn(0x7fd9640fc7a0 msgr2=0x7fd9640fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.870+0000 7fd969c00640 1 -- 172.21.15.46:0/813504404 shutdown_connections 2024-09-10T07:46:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:31.870+0000 7fd969c00640 1 -- 172.21.15.46:0/813504404 wait complete. 2024-09-10T07:46:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:31 smithi181 ceph-mon[35397]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 MiB/s wr, 549 op/s 2024-09-10T07:46:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:31 smithi063 ceph-mon[35020]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 MiB/s wr, 549 op/s 2024-09-10T07:46:32.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:31 smithi046 ceph-mon[30705]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 MiB/s wr, 549 op/s 2024-09-10T07:46:34.022 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:46:34.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:46:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:33 smithi181 ceph-mon[35397]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 MiB/s wr, 582 op/s 2024-09-10T07:46:34.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:33 smithi063 ceph-mon[35020]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 MiB/s wr, 582 op/s 2024-09-10T07:46:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:33 smithi046 ceph-mon[30705]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 MiB/s wr, 582 op/s 2024-09-10T07:46:34.781 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 555.0 s, 735.4 tps, lat 140.343 ms stddev 164.538 2024-09-10T07:46:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 -- 172.21.15.46:0/1648910422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4109140 msgr2=0x7f5bc4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 --2- 172.21.15.46:0/1648910422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4109140 0x7f5bc4110cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009be0 tx=0x7f5bbc02a710 comp rx=0 tx=0).stop 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 -- 172.21.15.46:0/1648910422 shutdown_connections 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 --2- 172.21.15.46:0/1648910422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4109140 0x7f5bc4110cc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 --2- 172.21.15.46:0/1648910422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc41049c0 0x7f5bc4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 --2- 172.21.15.46:0/1648910422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bc4104010 0x7f5bc41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 -- 172.21.15.46:0/1648910422 >> 172.21.15.46:0/1648910422 conn(0x7f5bc40ffa50 msgr2=0x7f5bc4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 -- 172.21.15.46:0/1648910422 shutdown_connections 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.600+0000 7f5bcc400640 1 -- 172.21.15.46:0/1648910422 wait complete. 2024-09-10T07:46:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.601+0000 7f5bcc400640 1 Processor -- start 2024-09-10T07:46:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.602+0000 7f5bcc400640 1 -- start start 2024-09-10T07:46:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.602+0000 7f5bcc400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4104010 0x7f5bc419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.602+0000 7f5bcc400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bc41049c0 0x7f5bc419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4104010 0x7f5bc419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4104010 0x7f5bc419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43572/0 (socket says 172.21.15.46:43572) 2024-09-10T07:46:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9a00640 1 -- 172.21.15.46:0/279183577 learned_addr learned my addr 172.21.15.46:0/279183577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bcc400640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4109140 0x7f5bc419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4114150 con 0x7f5bc4109140 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4113fd0 con 0x7f5bc41049c0 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc41142d0 con 0x7f5bc4104010 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9000640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bc41049c0 0x7f5bc419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9a00640 1 -- 172.21.15.46:0/279183577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bc41049c0 msgr2=0x7f5bc419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bc41049c0 0x7f5bc419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9a00640 1 -- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4109140 msgr2=0x7f5bc419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bca400640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4109140 0x7f5bc419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4109140 0x7f5bc419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9a00640 1 -- 172.21.15.46:0/279183577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc419f5e0 con 0x7f5bc4104010 2024-09-10T07:46:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.603+0000 7f5bc9000640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bc41049c0 0x7f5bc419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.604+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4104010 0x7f5bc419a630 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800e9f0 tx=0x7f5bb800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.604+0000 7f5bc2000640 1 -- 172.21.15.46:0/279183577 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800cd80 con 0x7f5bc4104010 2024-09-10T07:46:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.604+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc419f8d0 con 0x7f5bc4104010 2024-09-10T07:46:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.605+0000 7f5bc2000640 1 -- 172.21.15.46:0/279183577 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb8010910 con 0x7f5bc4104010 2024-09-10T07:46:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.605+0000 7f5bc2000640 1 -- 172.21.15.46:0/279183577 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80188f0 con 0x7f5bc4104010 2024-09-10T07:46:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.605+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc41a71b0 con 0x7f5bc4104010 2024-09-10T07:46:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.614+0000 7f5bc2000640 1 -- 172.21.15.46:0/279183577 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5bb8018bd0 con 0x7f5bc4104010 2024-09-10T07:46:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.616+0000 7f5bc2000640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5bac0a2c80 0x7f5bac0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.617+0000 7f5bc2000640 1 -- 172.21.15.46:0/279183577 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5bb80cfe80 con 0x7f5bc4104010 2024-09-10T07:46:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bcefd4740 1 Processor -- start 2024-09-10T07:46:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bcefd4740 1 -- start start 2024-09-10T07:46:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bc9000640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5bac0a2c80 0x7f5bac0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fc9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fdcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fdf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x565015f30290 con 0x565015fc99e0 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x565015ed7b40 con 0x565015fca300 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.618+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x565015f2c250 con 0x565015fdd4b0 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.619+0000 7f5bc9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fdcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.619+0000 7f5bc9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fdcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35654/0 (socket says 172.21.15.46:35654) 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.619+0000 7f5bc9000640 1 -- 172.21.15.46:0/474615434 learned_addr learned my addr 172.21.15.46:0/474615434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.620+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fc9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.620+0000 7f5bca400640 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fdf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.621+0000 7f5bca400640 1 -- 172.21.15.46:0/474615434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 msgr2=0x565015fdcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.621+0000 7f5bca400640 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fdcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.621+0000 7f5bca400640 1 -- 172.21.15.46:0/474615434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 msgr2=0x565015fc9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.621+0000 7f5bca400640 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fc9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.621+0000 7f5bca400640 1 -- 172.21.15.46:0/474615434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565015ca16a0 con 0x565015fdd4b0 2024-09-10T07:46:35.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.621+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fc9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:35.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.622+0000 7f5bc9000640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5bac0a2c80 0x7f5bac0a5140 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00129c0 tx=0x7f5bb0009cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.623+0000 7f5bca400640 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fdf8a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009b30 tx=0x7f5bbc0304a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc71f79f0aab6392 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.624+0000 7f5bb6a00640 1 -- 172.21.15.46:0/474615434 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc041020 con 0x565015fdd4b0 2024-09-10T07:46:35.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.624+0000 7f5bb6a00640 1 -- 172.21.15.46:0/474615434 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bbc037040 con 0x565015fdd4b0 2024-09-10T07:46:35.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.624+0000 7f5bb6a00640 1 -- 172.21.15.46:0/474615434 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc036070 con 0x565015fdd4b0 2024-09-10T07:46:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.625+0000 7f5bcefd4740 1 -- 172.21.15.46:0/474615434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 msgr2=0x565015fdf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.625+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fdf8a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009b30 tx=0x7f5bbc0304a0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.626+0000 7f5bcefd4740 1 -- 172.21.15.46:0/474615434 shutdown_connections 2024-09-10T07:46:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.626+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fdf8a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.626+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fdcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.626+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/474615434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fc9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.626+0000 7f5bcefd4740 1 -- 172.21.15.46:0/474615434 >> 172.21.15.46:0/474615434 conn(0x565015fc6050 msgr2=0x565015fc6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.626+0000 7f5bcefd4740 1 -- 172.21.15.46:0/474615434 shutdown_connections 2024-09-10T07:46:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.626+0000 7f5bcefd4740 1 -- 172.21.15.46:0/474615434 wait complete. 2024-09-10T07:46:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.627+0000 7f5bcefd4740 1 -- start start 2024-09-10T07:46:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.627+0000 7f5bcefd4740 1 Processor -- start 2024-09-10T07:46:35.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fdcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bca400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bca400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fda5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43594/0 (socket says 172.21.15.46:43594) 2024-09-10T07:46:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bca400640 1 -- 172.21.15.46:0/366575914 learned_addr learned my addr 172.21.15.46:0/366575914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x565015fe0060 con 0x565015fc99e0 2024-09-10T07:46:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x565015eeafa0 con 0x565015fca300 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x565015fe01e0 con 0x565015fdd4b0 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.632+0000 7f5bcefd4740 10 client.0 _ll_register_callbacks cb 0x565015c3eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.633+0000 7f5bc9000640 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.633+0000 7f5bc9000640 1 -- 172.21.15.46:0/366575914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 msgr2=0x565015fda5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.633+0000 7f5bc9000640 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fda5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.633+0000 7f5bc9000640 1 -- 172.21.15.46:0/366575914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 msgr2=0x565015fdcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.633+0000 7f5bc9000640 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fdcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.633+0000 7f5bc9000640 1 -- 172.21.15.46:0/366575914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565015fdae00 con 0x565015fca300 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.633+0000 7f5bca400640 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.634+0000 7f5bc9000640 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fda090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005fd0 tx=0x7f5bb0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a52d05fdca7328 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.635+0000 7f5bab400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.635+0000 7f5bab400640 1 -- 172.21.15.46:0/366575914 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0003800 con 0x565015fca300 2024-09-10T07:46:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.635+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bbc037040 con 0x565015fca300 2024-09-10T07:46:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.635+0000 7f5bab400640 1 -- 172.21.15.46:0/366575914 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5bb001b040 con 0x565015fca300 2024-09-10T07:46:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.637+0000 7f5bab400640 1 -- 172.21.15.46:0/366575914 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5bb0003800 con 0x565015fca300 2024-09-10T07:46:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.637+0000 7f5bcefd4740 10 client.27881 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.637+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x565015fec090 con 0x565015fca300 2024-09-10T07:46:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.637+0000 7f5bab400640 1 -- 172.21.15.46:0/366575914 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bbc037040 con 0x565015fca300 2024-09-10T07:46:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.638+0000 7f5bab400640 1 -- 172.21.15.46:0/366575914 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f5bb001f050 con 0x565015fca300 2024-09-10T07:46:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.638+0000 7f5bcefd4740 10 client.27881 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.638+0000 7f5bcefd4740 10 client.27881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.638+0000 7f5bcefd4740 20 client.27881 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.638+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x565015fed560 0x565015fefa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.638+0000 7f5bcefd4740 4 client.27881 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.638+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565015f3d800 con 0x565015fed560 2024-09-10T07:46:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.639+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x565015fed560 0x565015fefa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.644+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x565015fed560 0x565015fefa00 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96f2cc87e1056db server_cookie=32306951c7e41e7c in_seq=0 out_seq=0 2024-09-10T07:46:35.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.644+0000 7f5bab400640 10 client.27881 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:35.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.650+0000 7f5bab400640 1 -- 172.21.15.46:0/366575914 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565015f3d800 con 0x565015fed560 2024-09-10T07:46:35.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.650+0000 7f5bab400640 10 client.27881 handle_command_reply: tid=0 2024-09-10T07:46:35.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.650+0000 7f5bcefd4740 1 client.27881 shutdown 2024-09-10T07:46:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.650+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 msgr2=0x565015fda090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.650+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fda090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005fd0 tx=0x7f5bb0004730 comp rx=0 tx=0).stop 2024-09-10T07:46:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 shutdown_connections 2024-09-10T07:46:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x565015fed560 0x565015fefa00 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fdd4b0 0x565015fda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fca300 0x565015fda090 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/366575914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fdcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 >> 172.21.15.46:0/366575914 conn(0x565015fc6050 msgr2=0x565015fc51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bab400640 0 client.27881 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 shutdown_connections 2024-09-10T07:46:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 1 -- 172.21.15.46:0/366575914 wait complete. 2024-09-10T07:46:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.652+0000 7f5bcefd4740 20 client.27881 trim_cache size 0 max 16384 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.684+0000 7f5bcefd4740 1 Processor -- start 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.684+0000 7f5bcefd4740 1 -- start start 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fc99e0 0x565015fda4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fca300 0x565015fd9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015ff5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba400fb90 con 0x565015fca300 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bc9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fc99e0 0x565015fda4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40116d0 con 0x565015fdd4b0 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4010260 con 0x565015fc99e0 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bc9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fc99e0 0x565015fda4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43596/0 (socket says 172.21.15.46:43596) 2024-09-10T07:46:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.685+0000 7f5bc9a00640 1 -- 172.21.15.46:0/2208545807 learned_addr learned my addr 172.21.15.46:0/2208545807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bca400640 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015ff5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bc9000640 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fca300 0x565015fd9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bc9a00640 1 -- 172.21.15.46:0/2208545807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 msgr2=0x565015ff5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015ff5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bc9a00640 1 -- 172.21.15.46:0/2208545807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fca300 msgr2=0x565015fd9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fca300 0x565015fd9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bc9a00640 1 -- 172.21.15.46:0/2208545807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565015fd85e0 con 0x565015fc99e0 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bc9000640 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fca300 0x565015fd9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fc99e0 0x565015fda4b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80997f0 tx=0x7f5bb80bd050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de6664b32f246b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bb6000640 1 -- 172.21.15.46:0/2208545807 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb809b120 con 0x565015fc99e0 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bb6000640 1 -- 172.21.15.46:0/2208545807 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x565015f3d800 con 0x565015fc99e0 2024-09-10T07:46:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.686+0000 7f5bb6000640 1 -- 172.21.15.46:0/2208545807 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb809c560 con 0x565015fc99e0 2024-09-10T07:46:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.688+0000 7f5bcefd4740 1 -- 172.21.15.46:0/2208545807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fc99e0 msgr2=0x565015fda4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.688+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fc99e0 0x565015fda4b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80997f0 tx=0x7f5bb80bd050 comp rx=0 tx=0).stop 2024-09-10T07:46:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.689+0000 7f5bcefd4740 1 -- 172.21.15.46:0/2208545807 shutdown_connections 2024-09-10T07:46:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.689+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015ff5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.689+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fca300 0x565015fd9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.689+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/2208545807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fc99e0 0x565015fda4b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.689+0000 7f5bcefd4740 1 -- 172.21.15.46:0/2208545807 >> 172.21.15.46:0/2208545807 conn(0x565015fc6050 msgr2=0x565015fd8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.691+0000 7f5bcefd4740 1 -- 172.21.15.46:0/2208545807 shutdown_connections 2024-09-10T07:46:35.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.691+0000 7f5bcefd4740 1 -- 172.21.15.46:0/2208545807 wait complete. 2024-09-10T07:46:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.691+0000 7f5bcefd4740 1 -- start start 2024-09-10T07:46:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.692+0000 7f5bcefd4740 1 Processor -- start 2024-09-10T07:46:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.692+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fe5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.692+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fca300 0x565015fd6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.692+0000 7f5bcefd4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015fd7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.692+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4011000 con 0x565015fc99e0 2024-09-10T07:46:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.692+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4010930 con 0x565015fdd4b0 2024-09-10T07:46:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.692+0000 7f5bcefd4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba400db70 con 0x565015fca300 2024-09-10T07:46:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.692+0000 7f5bcefd4740 10 client.0 _ll_register_callbacks cb 0x7f5bb00126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bc9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fca300 0x565015fd6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bc9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fca300 0x565015fd6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43604/0 (socket says 172.21.15.46:43604) 2024-09-10T07:46:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bc9000640 1 -- 172.21.15.46:0/1699706283 learned_addr learned my addr 172.21.15.46:0/1699706283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bca400640 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015fd7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fe5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bca400640 1 -- 172.21.15.46:0/1699706283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fca300 msgr2=0x565015fd6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bca400640 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fca300 0x565015fd6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bca400640 1 -- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 msgr2=0x565015fe5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bca400640 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fe5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.693+0000 7f5bca400640 1 -- 172.21.15.46:0/1699706283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565015fd7730 con 0x565015fdd4b0 2024-09-10T07:46:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.694+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fe5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.694+0000 7f5bca400640 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015fd7140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc030ef0 tx=0x7f5bbc030f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56f4906e48f4a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.694+0000 7f5bab400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.694+0000 7f5bab400640 1 -- 172.21.15.46:0/1699706283 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc036040 con 0x565015fdd4b0 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.694+0000 7f5bab400640 1 -- 172.21.15.46:0/1699706283 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5bbc003970 con 0x565015fdd4b0 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.694+0000 7f5bcefd4740 1 -- 172.21.15.46:0/1699706283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565015f3d800 con 0x565015fdd4b0 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bab400640 1 -- 172.21.15.46:0/1699706283 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5bbc041040 con 0x565015fdd4b0 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bcefd4740 10 client.27887 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bcefd4740 1 -- 172.21.15.46:0/1699706283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x565016058090 con 0x565015fdd4b0 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bab400640 1 -- 172.21.15.46:0/1699706283 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x565015f3d800 con 0x565015fdd4b0 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bab400640 1 -- 172.21.15.46:0/1699706283 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f5bbc03d480 con 0x565015fdd4b0 2024-09-10T07:46:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bcefd4740 10 client.27887 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:35.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bcefd4740 10 client.27887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:35.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bcefd4740 20 client.27887 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:35.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x565016058370 0x56501605a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bcefd4740 4 client.27887 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:46:35.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.695+0000 7f5bcefd4740 1 -- 172.21.15.46:0/1699706283 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565015f37d10 con 0x565016058370 2024-09-10T07:46:35.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.696+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x565016058370 0x56501605a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.697+0000 7f5bc9a00640 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x565016058370 0x56501605a760 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e866c49fc4563520 server_cookie=a78007916ee45d90 in_seq=0 out_seq=0 2024-09-10T07:46:35.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.697+0000 7f5bab400640 10 client.27887 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:35.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.699+0000 7f5bab400640 1 -- 172.21.15.46:0/1699706283 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565015f37d10 con 0x565016058370 2024-09-10T07:46:35.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.699+0000 7f5bab400640 10 client.27887 handle_command_reply: tid=0 2024-09-10T07:46:35.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.699+0000 7f5bcefd4740 1 client.27887 shutdown 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bcefd4740 1 -- 172.21.15.46:0/1699706283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 msgr2=0x565015fd7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015fd7140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc030ef0 tx=0x7f5bbc030f20 comp rx=0 tx=0).stop 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bcefd4740 1 -- 172.21.15.46:0/1699706283 shutdown_connections 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x565016058370 0x56501605a760 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565015fdd4b0 0x565015fd7140 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x565015fca300 0x565015fd6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bcefd4740 1 --2- 172.21.15.46:0/1699706283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565015fc99e0 0x565015fe5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bcefd4740 1 -- 172.21.15.46:0/1699706283 >> 172.21.15.46:0/1699706283 conn(0x565015fc6050 msgr2=0x565015fd9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.700+0000 7f5bab400640 0 client.27887 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:35.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.701+0000 7f5bcefd4740 1 -- 172.21.15.46:0/1699706283 shutdown_connections 2024-09-10T07:46:35.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.701+0000 7f5bcefd4740 1 -- 172.21.15.46:0/1699706283 wait complete. 2024-09-10T07:46:35.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.701+0000 7f5bcefd4740 20 client.27887 trim_cache size 0 max 16384 2024-09-10T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9b35bafd-e5df-4468-8c08-0e2a76325dd9", 2024-09-10T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5bac0a2c80 msgr2=0x7f5bac0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5bac0a2c80 0x7f5bac0a5140 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00129c0 tx=0x7f5bb0009cc0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4104010 msgr2=0x7f5bc419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4104010 0x7f5bc419a630 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800e9f0 tx=0x7f5bb800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 shutdown_connections 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5bac0a2c80 0x7f5bac0a5140 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4109140 0x7f5bc419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bc41049c0 0x7f5bc419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 --2- 172.21.15.46:0/279183577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc4104010 0x7f5bc419a630 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 >> 172.21.15.46:0/279183577 conn(0x7f5bc40ffa50 msgr2=0x7f5bc4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 shutdown_connections 2024-09-10T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:35.702+0000 7f5bcc400640 1 -- 172.21.15.46:0/279183577 wait complete. 2024-09-10T07:46:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:35 smithi181 ceph-mon[35397]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-09-10T07:46:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:35 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:46:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:35 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:35 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:46:36.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:35 smithi063 ceph-mon[35020]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-09-10T07:46:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:35 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:46:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:35 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:35 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:46:36.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:35 smithi046 ceph-mon[30705]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-09-10T07:46:36.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:35 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:46:36.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:35 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:36.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:35 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:36 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:46:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:46:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:46:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:46:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:46:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:46:37.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:36 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:46:37.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:46:37.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:37.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:36 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:37 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:37 smithi063 ceph-mon[35020]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-10T07:46:38.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:37 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:37 smithi046 ceph-mon[30705]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-10T07:46:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:37 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:46:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:37 smithi181 ceph-mon[35397]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-10T07:46:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.910+0000 7f32cbe00640 1 -- 172.21.15.46:0/3118889317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc110190 msgr2=0x7f32cc06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.910+0000 7f32cbe00640 1 --2- 172.21.15.46:0/3118889317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc110190 0x7f32cc06e4f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009900 tx=0x7f32bc02a400 comp rx=0 tx=0).stop 2024-09-10T07:46:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.910+0000 7f32cbe00640 1 -- 172.21.15.46:0/3118889317 shutdown_connections 2024-09-10T07:46:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.910+0000 7f32cbe00640 1 --2- 172.21.15.46:0/3118889317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32cc11b140 0x7f32cc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.910+0000 7f32cbe00640 1 --2- 172.21.15.46:0/3118889317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32cc06ea30 0x7f32cc06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.910+0000 7f32cbe00640 1 --2- 172.21.15.46:0/3118889317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc110190 0x7f32cc06e4f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.910+0000 7f32cbe00640 1 -- 172.21.15.46:0/3118889317 >> 172.21.15.46:0/3118889317 conn(0x7f32cc073180 msgr2=0x7f32cc073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.911+0000 7f32cbe00640 1 -- 172.21.15.46:0/3118889317 shutdown_connections 2024-09-10T07:46:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.913+0000 7f32cbe00640 1 -- 172.21.15.46:0/3118889317 wait complete. 2024-09-10T07:46:38.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.915+0000 7f32cbe00640 1 Processor -- start 2024-09-10T07:46:38.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.915+0000 7f32cbe00640 1 -- start start 2024-09-10T07:46:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc06ea30 0x7f32cc1a7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32cc110190 0x7f32cc1a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32cc11b140 0x7f32cc1ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32cbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc120be0 con 0x7f32cc110190 2024-09-10T07:46:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32cbe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc120a60 con 0x7f32cc11b140 2024-09-10T07:46:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc06ea30 0x7f32cc1a7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32cbe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc120d60 con 0x7f32cc06ea30 2024-09-10T07:46:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc06ea30 0x7f32cc1a7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49846/0 (socket says 172.21.15.46:49846) 2024-09-10T07:46:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32caa00640 1 -- 172.21.15.46:0/2138069416 learned_addr learned my addr 172.21.15.46:0/2138069416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.916+0000 7f32cb400640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32cc11b140 0x7f32cc1ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.917+0000 7f32ca000640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32cc110190 0x7f32cc1a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.917+0000 7f32cb400640 1 -- 172.21.15.46:0/2138069416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc06ea30 msgr2=0x7f32cc1a7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.917+0000 7f32cb400640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc06ea30 0x7f32cc1a7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.917+0000 7f32cb400640 1 -- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32cc110190 msgr2=0x7f32cc1a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.917+0000 7f32cb400640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32cc110190 0x7f32cc1a7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.917+0000 7f32cb400640 1 -- 172.21.15.46:0/2138069416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc1ac0d0 con 0x7f32cc11b140 2024-09-10T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.920+0000 7f32cb400640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32cc11b140 0x7f32cc1ab9f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f32c4005fd0 tx=0x7f32c400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.921+0000 7f32c3400640 1 -- 172.21.15.46:0/2138069416 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4003b70 con 0x7f32cc11b140 2024-09-10T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.922+0000 7f32cbe00640 1 -- 172.21.15.46:0/2138069416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc1ac3c0 con 0x7f32cc11b140 2024-09-10T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.922+0000 7f32c3400640 1 -- 172.21.15.46:0/2138069416 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c400ee90 con 0x7f32cc11b140 2024-09-10T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.922+0000 7f32c3400640 1 -- 172.21.15.46:0/2138069416 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4003b70 con 0x7f32cc11b140 2024-09-10T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.922+0000 7f32cbe00640 1 -- 172.21.15.46:0/2138069416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1b3ca0 con 0x7f32cc11b140 2024-09-10T07:46:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.926+0000 7f32c3400640 1 -- 172.21.15.46:0/2138069416 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32c4004020 con 0x7f32cc11b140 2024-09-10T07:46:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.927+0000 7f32d4030740 1 Processor -- start 2024-09-10T07:46:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.927+0000 7f32d4030740 1 -- start start 2024-09-10T07:46:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.928+0000 7f32d4030740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 0x55d1aacde130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.928+0000 7f32d4030740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aace89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.928+0000 7f32d4030740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aace8f30 0x55d1aaceb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.928+0000 7f32d4030740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1aac443e0 con 0x55d1aacddd50 2024-09-10T07:46:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.928+0000 7f32d4030740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d1aabebd80 con 0x55d1aace8f30 2024-09-10T07:46:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.928+0000 7f32d4030740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d1aac403a0 con 0x55d1aacde670 2024-09-10T07:46:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.928+0000 7f32caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aace89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.929+0000 7f32ca000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aace8f30 0x55d1aaceb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.929+0000 7f32ca000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aace8f30 0x55d1aaceb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42442/0 (socket says 172.21.15.46:42442) 2024-09-10T07:46:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.929+0000 7f32ca000640 1 -- 172.21.15.46:0/3105631159 learned_addr learned my addr 172.21.15.46:0/3105631159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.931+0000 7f32c3400640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32b00a2e50 0x7f32b00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.931+0000 7f32ca000640 1 -- 172.21.15.46:0/3105631159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 msgr2=0x55d1aace89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.931+0000 7f32ca000640 1 --2- 172.21.15.46:0/3105631159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aace89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.931+0000 7f32ca000640 1 -- 172.21.15.46:0/3105631159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 msgr2=0x55d1aacde130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.932+0000 7f32ca000640 1 --2- 172.21.15.46:0/3105631159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 0x55d1aacde130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.932+0000 7f32ca000640 1 -- 172.21.15.46:0/3105631159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1aac0aae0 con 0x55d1aace8f30 2024-09-10T07:46:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.932+0000 7f32cb400640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32b00a2e50 0x7f32b00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.932+0000 7f32ca000640 1 --2- 172.21.15.46:0/3105631159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aace8f30 0x55d1aaceb320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f32b800ec10 tx=0x7f32b8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b9109db101052d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.933+0000 7f32b7400640 1 -- 172.21.15.46:0/3105631159 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8012450 con 0x55d1aace8f30 2024-09-10T07:46:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.933+0000 7f32b7400640 1 -- 172.21.15.46:0/3105631159 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b8012ad0 con 0x55d1aace8f30 2024-09-10T07:46:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.933+0000 7f32b7400640 1 -- 172.21.15.46:0/3105631159 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8011ae0 con 0x55d1aace8f30 2024-09-10T07:46:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.934+0000 7f32c3400640 1 -- 172.21.15.46:0/2138069416 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f32c40cf370 con 0x7f32cc11b140 2024-09-10T07:46:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.934+0000 7f32cb400640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32b00a2e50 0x7f32b00a5310 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f32c4003f50 tx=0x7f32c4005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.935+0000 7f32d4030740 1 -- 172.21.15.46:0/3105631159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aace8f30 msgr2=0x55d1aaceb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.935+0000 7f32d4030740 1 --2- 172.21.15.46:0/3105631159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aace8f30 0x55d1aaceb320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f32b800ec10 tx=0x7f32b8010740 comp rx=0 tx=0).stop 2024-09-10T07:46:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.935+0000 7f32d4030740 1 -- 172.21.15.46:0/3105631159 shutdown_connections 2024-09-10T07:46:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.936+0000 7f32d4030740 1 --2- 172.21.15.46:0/3105631159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aace8f30 0x55d1aaceb320 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.936+0000 7f32d4030740 1 --2- 172.21.15.46:0/3105631159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aace89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.936+0000 7f32d4030740 1 --2- 172.21.15.46:0/3105631159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 0x55d1aacde130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.936+0000 7f32d4030740 1 -- 172.21.15.46:0/3105631159 >> 172.21.15.46:0/3105631159 conn(0x55d1aacda2e0 msgr2=0x55d1aacda6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.936+0000 7f32d4030740 1 -- 172.21.15.46:0/3105631159 shutdown_connections 2024-09-10T07:46:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.936+0000 7f32d4030740 1 -- 172.21.15.46:0/3105631159 wait complete. 2024-09-10T07:46:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.937+0000 7f32d4030740 1 -- start start 2024-09-10T07:46:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.937+0000 7f32d4030740 1 Processor -- start 2024-09-10T07:46:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.944+0000 7f32d4030740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aace5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.944+0000 7f32d4030740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacde670 0x55d1aace6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.944+0000 7f32d4030740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 0x55d1aace6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.944+0000 7f32d4030740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1aacfcbb0 con 0x55d1aacde670 2024-09-10T07:46:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.944+0000 7f32d4030740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d1aabff0f0 con 0x55d1aacddd50 2024-09-10T07:46:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.944+0000 7f32d4030740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d1aacfcd30 con 0x55d1aace8f30 2024-09-10T07:46:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.944+0000 7f32d4030740 10 client.0 _ll_register_callbacks cb 0x55d1aabf09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.945+0000 7f32ca000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aace5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.945+0000 7f32ca000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aace5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42452/0 (socket says 172.21.15.46:42452) 2024-09-10T07:46:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.945+0000 7f32ca000640 1 -- 172.21.15.46:0/2633224879 learned_addr learned my addr 172.21.15.46:0/2633224879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.945+0000 7f32caa00640 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacde670 0x55d1aace6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.945+0000 7f32ca000640 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 0x55d1aace6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.946+0000 7f32ca000640 1 -- 172.21.15.46:0/2633224879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 msgr2=0x55d1aace6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.946+0000 7f32ca000640 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 0x55d1aace6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.946+0000 7f32ca000640 1 -- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacde670 msgr2=0x55d1aace6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.946+0000 7f32ca000640 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacde670 0x55d1aace6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.946+0000 7f32ca000640 1 -- 172.21.15.46:0/2633224879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1aa9b53c0 con 0x55d1aacddd50 2024-09-10T07:46:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.946+0000 7f32caa00640 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacde670 0x55d1aace6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.947+0000 7f32ca000640 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aace5ed0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f32b8010fd0 tx=0x7f32b8011740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ea07a2cb921c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.947+0000 7f32abe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.947+0000 7f32abe00640 1 -- 172.21.15.46:0/2633224879 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8012450 con 0x55d1aacddd50 2024-09-10T07:46:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.947+0000 7f32abe00640 1 -- 172.21.15.46:0/2633224879 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f32b80125f0 con 0x55d1aacddd50 2024-09-10T07:46:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.948+0000 7f32d4030740 1 -- 172.21.15.46:0/2633224879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b8012ad0 con 0x55d1aacddd50 2024-09-10T07:46:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.948+0000 7f32abe00640 1 -- 172.21.15.46:0/2633224879 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f32b80127f0 con 0x55d1aacddd50 2024-09-10T07:46:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32d4030740 10 client.27899 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32d4030740 1 -- 172.21.15.46:0/2633224879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d1aad08900 con 0x55d1aacddd50 2024-09-10T07:46:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32abe00640 1 -- 172.21.15.46:0/2633224879 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b8012ad0 con 0x55d1aacddd50 2024-09-10T07:46:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32abe00640 1 -- 172.21.15.46:0/2633224879 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f32b800f040 con 0x55d1aacddd50 2024-09-10T07:46:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32d4030740 10 client.27899 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32d4030740 10 client.27899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32d4030740 20 client.27899 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32d4030740 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1aad09cb0 0x55d1aad0c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32d4030740 4 client.27899 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.949+0000 7f32d4030740 1 -- 172.21.15.46:0/2633224879 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1aac51950 con 0x55d1aad09cb0 2024-09-10T07:46:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.950+0000 7f32caa00640 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1aad09cb0 0x55d1aad0c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.951+0000 7f32caa00640 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1aad09cb0 0x55d1aad0c0a0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be707a449b17777b server_cookie=174a4b9840293d20 in_seq=0 out_seq=0 2024-09-10T07:46:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.951+0000 7f32abe00640 10 client.27899 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.952+0000 7f32abe00640 1 -- 172.21.15.46:0/2633224879 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1aac51950 con 0x55d1aad09cb0 2024-09-10T07:46:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.952+0000 7f32abe00640 10 client.27899 handle_command_reply: tid=0 2024-09-10T07:46:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.952+0000 7f32d4030740 1 client.27899 shutdown 2024-09-10T07:46:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 -- 172.21.15.46:0/2633224879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 msgr2=0x55d1aace5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aace5ed0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f32b8010fd0 tx=0x7f32b8011740 comp rx=0 tx=0).stop 2024-09-10T07:46:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 -- 172.21.15.46:0/2633224879 shutdown_connections 2024-09-10T07:46:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1aad09cb0 0x55d1aad0c0a0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 0x55d1aace6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacde670 0x55d1aace6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 --2- 172.21.15.46:0/2633224879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aace5ed0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 -- 172.21.15.46:0/2633224879 >> 172.21.15.46:0/2633224879 conn(0x55d1aacda2e0 msgr2=0x55d1aace0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32abe00640 0 client.27899 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 -- 172.21.15.46:0/2633224879 shutdown_connections 2024-09-10T07:46:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.953+0000 7f32d4030740 1 -- 172.21.15.46:0/2633224879 wait complete. 2024-09-10T07:46:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.955+0000 7f32d4030740 20 client.27899 trim_cache size 0 max 16384 2024-09-10T07:46:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32d4030740 1 Processor -- start 2024-09-10T07:46:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32d4030740 1 -- start start 2024-09-10T07:46:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32d4030740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aad11830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32d4030740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aad11d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32d4030740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aace8f30 0x55d1aad01f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32d4030740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b5120 con 0x55d1aace8f30 2024-09-10T07:46:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32d4030740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b6c60 con 0x55d1aacddd50 2024-09-10T07:46:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32d4030740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b57f0 con 0x55d1aacde670 2024-09-10T07:46:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aad11d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aad11d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49878/0 (socket says 172.21.15.46:49878) 2024-09-10T07:46:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.989+0000 7f32caa00640 1 -- 172.21.15.46:0/2745160578 learned_addr learned my addr 172.21.15.46:0/2745160578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.990+0000 7f32ca000640 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aad11830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.990+0000 7f32ca000640 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aace8f30 0x55d1aad01f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.990+0000 7f32caa00640 1 -- 172.21.15.46:0/2745160578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 msgr2=0x55d1aad11830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.990+0000 7f32caa00640 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aad11830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.990+0000 7f32caa00640 1 -- 172.21.15.46:0/2745160578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aace8f30 msgr2=0x55d1aad01f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.990+0000 7f32caa00640 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aace8f30 0x55d1aad01f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.990+0000 7f32caa00640 1 -- 172.21.15.46:0/2745160578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1aacdeef0 con 0x55d1aacde670 2024-09-10T07:46:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.990+0000 7f32ca000640 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aace8f30 0x55d1aad01f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.991+0000 7f32caa00640 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aad11d70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f32bc004220 tx=0x7f32bc02a900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=536a808c9c926a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.991+0000 7f32b6a00640 1 -- 172.21.15.46:0/2745160578 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc035040 con 0x55d1aacde670 2024-09-10T07:46:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.992+0000 7f32b6a00640 1 -- 172.21.15.46:0/2745160578 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc04a070 con 0x55d1aacde670 2024-09-10T07:46:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.992+0000 7f32b6a00640 1 -- 172.21.15.46:0/2745160578 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc031930 con 0x55d1aacde670 2024-09-10T07:46:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.993+0000 7f32d4030740 1 -- 172.21.15.46:0/2745160578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 msgr2=0x55d1aad11d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.993+0000 7f32d4030740 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aad11d70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f32bc004220 tx=0x7f32bc02a900 comp rx=0 tx=0).stop 2024-09-10T07:46:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.993+0000 7f32d4030740 1 -- 172.21.15.46:0/2745160578 shutdown_connections 2024-09-10T07:46:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.993+0000 7f32d4030740 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aace8f30 0x55d1aad01f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.993+0000 7f32d4030740 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aacde670 0x55d1aad11d70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.993+0000 7f32d4030740 1 --2- 172.21.15.46:0/2745160578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacddd50 0x55d1aad11830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.993+0000 7f32d4030740 1 -- 172.21.15.46:0/2745160578 >> 172.21.15.46:0/2745160578 conn(0x55d1aacda2e0 msgr2=0x55d1aace7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.994+0000 7f32d4030740 1 -- 172.21.15.46:0/2745160578 shutdown_connections 2024-09-10T07:46:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.994+0000 7f32d4030740 1 -- 172.21.15.46:0/2745160578 wait complete. 2024-09-10T07:46:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.994+0000 7f32d4030740 1 -- start start 2024-09-10T07:46:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.995+0000 7f32d4030740 1 Processor -- start 2024-09-10T07:46:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.995+0000 7f32d4030740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 0x55d1aad02620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.995+0000 7f32d4030740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacde670 0x55d1aacff970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.995+0000 7f32d4030740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 0x55d1aacffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.995+0000 7f32d4030740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b6590 con 0x55d1aacddd50 2024-09-10T07:46:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.995+0000 7f32d4030740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b5ec0 con 0x55d1aacde670 2024-09-10T07:46:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.996+0000 7f32d4030740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc118300 con 0x55d1aace8f30 2024-09-10T07:46:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.996+0000 7f32d4030740 10 client.0 _ll_register_callbacks cb 0x7f32b8011e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32caa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacde670 0x55d1aacff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32caa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacde670 0x55d1aacff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42476/0 (socket says 172.21.15.46:42476) 2024-09-10T07:46:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32caa00640 1 -- 172.21.15.46:0/1610564802 learned_addr learned my addr 172.21.15.46:0/1610564802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32ca000640 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 0x55d1aacffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32ca000640 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 0x55d1aad02620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32caa00640 1 -- 172.21.15.46:0/1610564802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 msgr2=0x55d1aacffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32caa00640 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 0x55d1aacffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32caa00640 1 -- 172.21.15.46:0/1610564802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 msgr2=0x55d1aad02620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32caa00640 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 0x55d1aad02620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.997+0000 7f32caa00640 1 -- 172.21.15.46:0/1610564802 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1aad006e0 con 0x55d1aacde670 2024-09-10T07:46:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.998+0000 7f32caa00640 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacde670 0x55d1aacff970 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f32bc004650 tx=0x7f32bc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88488c00849f10fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.998+0000 7f32abe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.999+0000 7f32abe00640 1 -- 172.21.15.46:0/1610564802 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc052070 con 0x55d1aacde670 2024-09-10T07:46:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.999+0000 7f32abe00640 1 -- 172.21.15.46:0/1610564802 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f32bc005100 con 0x55d1aacde670 2024-09-10T07:46:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.999+0000 7f32d4030740 1 -- 172.21.15.46:0/1610564802 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32bc04a070 con 0x55d1aacde670 2024-09-10T07:46:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.999+0000 7f32abe00640 1 -- 172.21.15.46:0/1610564802 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f32bc035050 con 0x55d1aacde670 2024-09-10T07:46:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.999+0000 7f32d4030740 10 client.27905 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:38.999+0000 7f32d4030740 1 -- 172.21.15.46:0/1610564802 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d1aad74790 con 0x55d1aacde670 2024-09-10T07:46:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.000+0000 7f32abe00640 1 -- 172.21.15.46:0/1610564802 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc04a070 con 0x55d1aacde670 2024-09-10T07:46:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.000+0000 7f32abe00640 1 -- 172.21.15.46:0/1610564802 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f32bc0035f0 con 0x55d1aacde670 2024-09-10T07:46:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.000+0000 7f32d4030740 10 client.27905 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.000+0000 7f32d4030740 10 client.27905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.000+0000 7f32d4030740 20 client.27905 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.000+0000 7f32d4030740 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1aad74f10 0x55d1aad77330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.000+0000 7f32d4030740 4 client.27905 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:46:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.000+0000 7f32d4030740 1 -- 172.21.15.46:0/1610564802 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1aac4be60 con 0x55d1aad74f10 2024-09-10T07:46:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.001+0000 7f32ca000640 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1aad74f10 0x55d1aad77330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.001+0000 7f32ca000640 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1aad74f10 0x55d1aad77330 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e0d360ed281001 server_cookie=24bd050e583a4907 in_seq=0 out_seq=0 2024-09-10T07:46:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.001+0000 7f32abe00640 10 client.27905 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32abe00640 1 -- 172.21.15.46:0/1610564802 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d1aac4be60 con 0x55d1aad74f10 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32abe00640 10 client.27905 handle_command_reply: tid=0 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 client.27905 shutdown 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 -- 172.21.15.46:0/1610564802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacde670 msgr2=0x55d1aacff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacde670 0x55d1aacff970 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f32bc004650 tx=0x7f32bc000bc0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 -- 172.21.15.46:0/1610564802 shutdown_connections 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d1aad74f10 0x55d1aad77330 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1aace8f30 0x55d1aacffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1aacde670 0x55d1aacff970 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 --2- 172.21.15.46:0/1610564802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1aacddd50 0x55d1aad02620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32d4030740 1 -- 172.21.15.46:0/1610564802 >> 172.21.15.46:0/1610564802 conn(0x55d1aacda2e0 msgr2=0x55d1aace7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.002+0000 7f32abe00640 0 client.27905 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.003+0000 7f32d4030740 1 -- 172.21.15.46:0/1610564802 shutdown_connections 2024-09-10T07:46:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.003+0000 7f32d4030740 1 -- 172.21.15.46:0/1610564802 wait complete. 2024-09-10T07:46:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.003+0000 7f32d4030740 20 client.27905 trim_cache size 0 max 16384 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout: "9b35bafd-e5df-4468-8c08-0e2a76325dd9": { 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "9b35bafd-e5df-4468-8c08-0e2a76325dd9", 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:39.004 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:46:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 -- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32b00a2e50 msgr2=0x7f32b00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32b00a2e50 0x7f32b00a5310 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f32c4003f50 tx=0x7f32c4005bd0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 -- 172.21.15.46:0/2138069416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32cc11b140 msgr2=0x7f32cc1ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32cc11b140 0x7f32cc1ab9f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f32c4005fd0 tx=0x7f32c400c6a0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 -- 172.21.15.46:0/2138069416 shutdown_connections 2024-09-10T07:46:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32cc11b140 0x7f32cc1ab9f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f32b00a2e50 0x7f32b00a5310 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32cc110190 0x7f32cc1a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 --2- 172.21.15.46:0/2138069416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32cc06ea30 0x7f32cc1a7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.004+0000 7f32cbe00640 1 -- 172.21.15.46:0/2138069416 >> 172.21.15.46:0/2138069416 conn(0x7f32cc073180 msgr2=0x7f32cc119310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.005+0000 7f32cbe00640 1 -- 172.21.15.46:0/2138069416 shutdown_connections 2024-09-10T07:46:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:39.005+0000 7f32cbe00640 1 -- 172.21.15.46:0/2138069416 wait complete. 2024-09-10T07:46:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.117+0000 7f0efd200640 1 -- 172.21.15.46:0/2762431141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef8108470 msgr2=0x7f0ef8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.117+0000 7f0efd200640 1 --2- 172.21.15.46:0/2762431141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef8108470 0x7f0ef8108870 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009900 tx=0x7f0ee002a410 comp rx=0 tx=0).stop 2024-09-10T07:46:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.117+0000 7f0efd200640 1 -- 172.21.15.46:0/2762431141 shutdown_connections 2024-09-10T07:46:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.117+0000 7f0efd200640 1 --2- 172.21.15.46:0/2762431141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef8109d40 0x7f0ef81105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.117+0000 7f0efd200640 1 --2- 172.21.15.46:0/2762431141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef8109380 0x7f0ef8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.117+0000 7f0efd200640 1 --2- 172.21.15.46:0/2762431141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef8108470 0x7f0ef8108870 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.117+0000 7f0efd200640 1 -- 172.21.15.46:0/2762431141 >> 172.21.15.46:0/2762431141 conn(0x7f0ef8077470 msgr2=0x7f0ef80798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.118+0000 7f0efd200640 1 -- 172.21.15.46:0/2762431141 shutdown_connections 2024-09-10T07:46:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.118+0000 7f0efd200640 1 -- 172.21.15.46:0/2762431141 wait complete. 2024-09-10T07:46:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.119+0000 7f0efd200640 1 Processor -- start 2024-09-10T07:46:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.123+0000 7f0efd200640 1 -- start start 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0efd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef8108470 0x7f0ef807fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0efd200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef8109380 0x7f0ef807ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0efd200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef8109d40 0x7f0ef81ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0efd200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8113ef0 con 0x7f0ef8108470 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0efd200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8113d70 con 0x7f0ef8109380 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0efd200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8114070 con 0x7f0ef8109d40 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0ef5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef8109380 0x7f0ef807ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0ef6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef8109d40 0x7f0ef81ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0ef6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef8109d40 0x7f0ef81ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49910/0 (socket says 172.21.15.46:49910) 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.124+0000 7f0ef6800640 1 -- 172.21.15.46:0/1773150658 learned_addr learned my addr 172.21.15.46:0/1773150658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.125+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef8108470 0x7f0ef807fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.125+0000 7f0ef6800640 1 -- 172.21.15.46:0/1773150658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef8109380 msgr2=0x7f0ef807ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.125+0000 7f0ef6800640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef8109380 0x7f0ef807ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.125+0000 7f0ef6800640 1 -- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef8108470 msgr2=0x7f0ef807fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.127+0000 7f0ef6800640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef8108470 0x7f0ef807fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.127+0000 7f0ef6800640 1 -- 172.21.15.46:0/1773150658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef81aeb40 con 0x7f0ef8109d40 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.127+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef8108470 0x7f0ef807fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.127+0000 7f0ef6800640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef8109d40 0x7f0ef81ae600 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000b970 tx=0x7f0ef000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.128+0000 7f0eeea00640 1 -- 172.21.15.46:0/1773150658 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0019070 con 0x7f0ef8109d40 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.128+0000 7f0efd200640 1 -- 172.21.15.46:0/1773150658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef81aed10 con 0x7f0ef8109d40 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.128+0000 7f0efd200640 1 -- 172.21.15.46:0/1773150658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef81af1b0 con 0x7f0ef8109d40 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.128+0000 7f0eeea00640 1 -- 172.21.15.46:0/1773150658 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef0014440 con 0x7f0ef8109d40 2024-09-10T07:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.128+0000 7f0eeea00640 1 -- 172.21.15.46:0/1773150658 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00075d0 con 0x7f0ef8109d40 2024-09-10T07:46:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.135+0000 7f0f000da740 1 Processor -- start 2024-09-10T07:46:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.140+0000 7f0eeea00640 1 -- 172.21.15.46:0/1773150658 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ef001c070 con 0x7f0ef8109d40 2024-09-10T07:46:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.141+0000 7f0f000da740 1 -- start start 2024-09-10T07:46:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.141+0000 7f0f000da740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c8664490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.141+0000 7f0f000da740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 0x5629c8669490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.141+0000 7f0f000da740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c866bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.141+0000 7f0f000da740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629c85c9100 con 0x5629c86649d0 2024-09-10T07:46:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.141+0000 7f0f000da740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5629c85711b0 con 0x5629c8669a60 2024-09-10T07:46:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.141+0000 7f0f000da740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5629c85c50c0 con 0x5629c8664090 2024-09-10T07:46:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.142+0000 7f0ef5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c866bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.142+0000 7f0ef5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c866bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42486/0 (socket says 172.21.15.46:42486) 2024-09-10T07:46:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.142+0000 7f0ef5400640 1 -- 172.21.15.46:0/3636118260 learned_addr learned my addr 172.21.15.46:0/3636118260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.143+0000 7f0ef5400640 1 -- 172.21.15.46:0/3636118260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 msgr2=0x5629c8664490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.143+0000 7f0ef5400640 1 --2- 172.21.15.46:0/3636118260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c8664490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.143+0000 7f0ef5400640 1 -- 172.21.15.46:0/3636118260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 msgr2=0x5629c8669490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.143+0000 7f0ef5400640 1 --2- 172.21.15.46:0/3636118260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 0x5629c8669490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.143+0000 7f0ef5400640 1 -- 172.21.15.46:0/3636118260 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c82dc210 con 0x5629c8669a60 2024-09-10T07:46:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.144+0000 7f0ef5400640 1 --2- 172.21.15.46:0/3636118260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c866bf20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009a80 tx=0x7f0ee8009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2009b3acb9bfd986 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.144+0000 7f0ee7400640 1 -- 172.21.15.46:0/3636118260 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8018020 con 0x5629c8669a60 2024-09-10T07:46:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.144+0000 7f0ee7400640 1 -- 172.21.15.46:0/3636118260 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee8010c20 con 0x5629c8669a60 2024-09-10T07:46:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.144+0000 7f0ee7400640 1 -- 172.21.15.46:0/3636118260 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8013610 con 0x5629c8669a60 2024-09-10T07:46:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.145+0000 7f0f000da740 1 -- 172.21.15.46:0/3636118260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 msgr2=0x5629c866bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.145+0000 7f0f000da740 1 --2- 172.21.15.46:0/3636118260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c866bf20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009a80 tx=0x7f0ee8009f40 comp rx=0 tx=0).stop 2024-09-10T07:46:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.147+0000 7f0eeea00640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0edc0a2c30 0x7f0edc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.147+0000 7f0eeea00640 1 -- 172.21.15.46:0/1773150658 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0ef00ce950 con 0x7f0ef8109d40 2024-09-10T07:46:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.148+0000 7f0ef5400640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0edc0a2c30 0x7f0edc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.149+0000 7f0f000da740 1 -- 172.21.15.46:0/3636118260 shutdown_connections 2024-09-10T07:46:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.149+0000 7f0f000da740 1 --2- 172.21.15.46:0/3636118260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c866bf20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.149+0000 7f0f000da740 1 --2- 172.21.15.46:0/3636118260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 0x5629c8669490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.149+0000 7f0f000da740 1 --2- 172.21.15.46:0/3636118260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c8664490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.149+0000 7f0f000da740 1 -- 172.21.15.46:0/3636118260 >> 172.21.15.46:0/3636118260 conn(0x5629c865f700 msgr2=0x5629c865fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.149+0000 7f0ef5400640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0edc0a2c30 0x7f0edc0a50f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009890 tx=0x7f0ee80137b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.151+0000 7f0f000da740 1 -- 172.21.15.46:0/3636118260 shutdown_connections 2024-09-10T07:46:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.152+0000 7f0f000da740 1 -- 172.21.15.46:0/3636118260 wait complete. 2024-09-10T07:46:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.153+0000 7f0f000da740 1 -- start start 2024-09-10T07:46:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.154+0000 7f0f000da740 1 Processor -- start 2024-09-10T07:46:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.154+0000 7f0f000da740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c866e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.154+0000 7f0f000da740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 0x5629c866e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.154+0000 7f0f000da740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c86757e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.155+0000 7f0f000da740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629c866ca80 con 0x5629c86649d0 2024-09-10T07:46:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.155+0000 7f0ef6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c86757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.155+0000 7f0ef6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c86757e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42498/0 (socket says 172.21.15.46:42498) 2024-09-10T07:46:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.155+0000 7f0ef6800640 1 -- 172.21.15.46:0/733797357 learned_addr learned my addr 172.21.15.46:0/733797357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.155+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5629c8552090 con 0x5629c8669a60 2024-09-10T07:46:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.155+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5629c866cc00 con 0x5629c8664090 2024-09-10T07:46:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.156+0000 7f0f000da740 10 client.0 _ll_register_callbacks cb 0x5629c85a6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.156+0000 7f0ef5400640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 0x5629c866e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.157+0000 7f0ef6800640 1 -- 172.21.15.46:0/733797357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 msgr2=0x5629c866e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.157+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c866e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.158+0000 7f0ef6800640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c866e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.158+0000 7f0ef6800640 1 -- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 msgr2=0x5629c866e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.158+0000 7f0ef6800640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 0x5629c866e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.158+0000 7f0ef6800640 1 -- 172.21.15.46:0/733797357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c83a8620 con 0x5629c8669a60 2024-09-10T07:46:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.158+0000 7f0ef5400640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 0x5629c866e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.159+0000 7f0ef6800640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c86757e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0ef009adc0 tx=0x7f0ef009f2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c63f9c1ddb4d6da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.159+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c866e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.159+0000 7f0edbe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.159+0000 7f0edbe00640 1 -- 172.21.15.46:0/733797357 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00991f0 con 0x5629c8669a60 2024-09-10T07:46:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.159+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee8010c20 con 0x5629c8669a60 2024-09-10T07:46:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.159+0000 7f0edbe00640 1 -- 172.21.15.46:0/733797357 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0ef009a280 con 0x5629c8669a60 2024-09-10T07:46:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.160+0000 7f0edbe00640 1 -- 172.21.15.46:0/733797357 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0ef009a4c0 con 0x5629c8669a60 2024-09-10T07:46:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.160+0000 7f0f000da740 10 client.27917 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.160+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5629c86786a0 con 0x5629c8669a60 2024-09-10T07:46:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.161+0000 7f0edbe00640 1 -- 172.21.15.46:0/733797357 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee8010c20 con 0x5629c8669a60 2024-09-10T07:46:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.162+0000 7f0edbe00640 1 -- 172.21.15.46:0/733797357 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f0ef00a0370 con 0x5629c8669a60 2024-09-10T07:46:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.162+0000 7f0f000da740 10 client.27917 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.162+0000 7f0f000da740 10 client.27917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.162+0000 7f0f000da740 20 client.27917 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.162+0000 7f0f000da740 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629c8679d20 0x5629c867c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.162+0000 7f0f000da740 4 client.27917 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.162+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c85d6830 con 0x5629c8679d20 2024-09-10T07:46:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.164+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629c8679d20 0x5629c867c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.166+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629c8679d20 0x5629c867c1b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c81b13f5b317c38 server_cookie=7d1ad95866fce5a7 in_seq=0 out_seq=0 2024-09-10T07:46:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.166+0000 7f0edbe00640 10 client.27917 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.168+0000 7f0edbe00640 1 -- 172.21.15.46:0/733797357 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629c85d6830 con 0x5629c8679d20 2024-09-10T07:46:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.168+0000 7f0edbe00640 10 client.27917 handle_command_reply: tid=0 2024-09-10T07:46:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.168+0000 7f0f000da740 1 client.27917 shutdown 2024-09-10T07:46:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.169+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 msgr2=0x5629c86757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.170+0000 7f0f000da740 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c86757e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0ef009adc0 tx=0x7f0ef009f2a0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.170+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 shutdown_connections 2024-09-10T07:46:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.170+0000 7f0f000da740 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629c8679d20 0x5629c867c1b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.170+0000 7f0f000da740 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8669a60 0x5629c86757e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.170+0000 7f0f000da740 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c86649d0 0x5629c866e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.170+0000 7f0edbe00640 0 client.27917 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.170+0000 7f0f000da740 1 --2- 172.21.15.46:0/733797357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c866e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.170+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 >> 172.21.15.46:0/733797357 conn(0x5629c865f700 msgr2=0x5629c8660f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.174+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 shutdown_connections 2024-09-10T07:46:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.174+0000 7f0f000da740 1 -- 172.21.15.46:0/733797357 wait complete. 2024-09-10T07:46:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.175+0000 7f0f000da740 20 client.27917 trim_cache size 0 max 16384 2024-09-10T07:46:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.221+0000 7f0f000da740 1 Processor -- start 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.221+0000 7f0f000da740 1 -- start start 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.221+0000 7f0f000da740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8664090 0x5629c86625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.221+0000 7f0f000da740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c86649d0 0x5629c86757a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.221+0000 7f0f000da740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8662b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.221+0000 7f0f000da740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed400fc10 con 0x5629c8669a60 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.221+0000 7f0f000da740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4011750 con 0x5629c8664090 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.221+0000 7f0f000da740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed40102e0 con 0x5629c86649d0 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c86649d0 0x5629c86757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c86649d0 0x5629c86757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49926/0 (socket says 172.21.15.46:49926) 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5400640 1 -- 172.21.15.46:0/1382349880 learned_addr learned my addr 172.21.15.46:0/1382349880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8664090 0x5629c86625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef6800640 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8662b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5400640 1 -- 172.21.15.46:0/1382349880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8664090 msgr2=0x5629c86625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5400640 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8664090 0x5629c86625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5400640 1 -- 172.21.15.46:0/1382349880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 msgr2=0x5629c8662b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5400640 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8662b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.222+0000 7f0ef5400640 1 -- 172.21.15.46:0/1382349880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c865f520 con 0x5629c86649d0 2024-09-10T07:46:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.223+0000 7f0ef6800640 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8662b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.223+0000 7f0ef5400640 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c86649d0 0x5629c86757a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009a20 tx=0x7f0ee8025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6951d3b9d100d465 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.223+0000 7f0ee6a00640 1 -- 172.21.15.46:0/1382349880 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8018040 con 0x5629c86649d0 2024-09-10T07:46:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.223+0000 7f0ee6a00640 1 -- 172.21.15.46:0/1382349880 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee8013880 con 0x5629c86649d0 2024-09-10T07:46:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.223+0000 7f0ee6a00640 1 -- 172.21.15.46:0/1382349880 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8025070 con 0x5629c86649d0 2024-09-10T07:46:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.224+0000 7f0f000da740 1 -- 172.21.15.46:0/1382349880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c86649d0 msgr2=0x5629c86757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.224+0000 7f0f000da740 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c86649d0 0x5629c86757a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009a20 tx=0x7f0ee8025000 comp rx=0 tx=0).stop 2024-09-10T07:46:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.228+0000 7f0f000da740 1 -- 172.21.15.46:0/1382349880 shutdown_connections 2024-09-10T07:46:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.228+0000 7f0f000da740 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8662b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.228+0000 7f0f000da740 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c86649d0 0x5629c86757a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.228+0000 7f0f000da740 1 --2- 172.21.15.46:0/1382349880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c8664090 0x5629c86625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.228+0000 7f0f000da740 1 -- 172.21.15.46:0/1382349880 >> 172.21.15.46:0/1382349880 conn(0x5629c865f700 msgr2=0x5629c866e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.228+0000 7f0f000da740 1 -- 172.21.15.46:0/1382349880 shutdown_connections 2024-09-10T07:46:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.228+0000 7f0f000da740 1 -- 172.21.15.46:0/1382349880 wait complete. 2024-09-10T07:46:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.229+0000 7f0f000da740 1 -- start start 2024-09-10T07:46:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.229+0000 7f0f000da740 1 Processor -- start 2024-09-10T07:46:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.231+0000 7f0f000da740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c8672c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.231+0000 7f0f000da740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c86649d0 0x5629c86731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.231+0000 7f0f000da740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8674190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.232+0000 7f0f000da740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4011080 con 0x5629c8669a60 2024-09-10T07:46:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.232+0000 7f0f000da740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed40109b0 con 0x5629c86649d0 2024-09-10T07:46:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.232+0000 7f0f000da740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed400dbf0 con 0x5629c8664090 2024-09-10T07:46:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.232+0000 7f0f000da740 10 client.0 _ll_register_callbacks cb 0x7f0ef009ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.234+0000 7f0ef5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c86649d0 0x5629c86731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.234+0000 7f0ef5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c86649d0 0x5629c86731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42524/0 (socket says 172.21.15.46:42524) 2024-09-10T07:46:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.234+0000 7f0ef5400640 1 -- 172.21.15.46:0/1376099653 learned_addr learned my addr 172.21.15.46:0/1376099653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.234+0000 7f0ef6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8674190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.235+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c8672c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.236+0000 7f0ef5400640 1 -- 172.21.15.46:0/1376099653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 msgr2=0x5629c8672c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.236+0000 7f0ef5400640 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c8672c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.236+0000 7f0ef5400640 1 -- 172.21.15.46:0/1376099653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 msgr2=0x5629c8674190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.236+0000 7f0ef5400640 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8674190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.236+0000 7f0ef5400640 1 -- 172.21.15.46:0/1376099653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c86747a0 con 0x5629c86649d0 2024-09-10T07:46:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.236+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c8672c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.236+0000 7f0ef6800640 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8674190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.237+0000 7f0ef5400640 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c86649d0 0x5629c86731d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80095c0 tx=0x7f0ee8014bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=764d1f64a95d83ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.237+0000 7f0edbe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.238+0000 7f0edbe00640 1 -- 172.21.15.46:0/1376099653 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8037020 con 0x5629c86649d0 2024-09-10T07:46:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.238+0000 7f0edbe00640 1 -- 172.21.15.46:0/1376099653 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0ee802b8d0 con 0x5629c86649d0 2024-09-10T07:46:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.238+0000 7f0f000da740 1 -- 172.21.15.46:0/1376099653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee8013880 con 0x5629c86649d0 2024-09-10T07:46:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.238+0000 7f0edbe00640 1 -- 172.21.15.46:0/1376099653 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0ee8042050 con 0x5629c86649d0 2024-09-10T07:46:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.238+0000 7f0f000da740 10 client.27920 fetch_fsmap learned FSMap version 48 2024-09-10T07:46:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.239+0000 7f0f000da740 1 -- 172.21.15.46:0/1376099653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5629c8675ab0 con 0x5629c86649d0 2024-09-10T07:46:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.239+0000 7f0edbe00640 1 -- 172.21.15.46:0/1376099653 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee8013880 con 0x5629c86649d0 2024-09-10T07:46:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.240+0000 7f0edbe00640 1 -- 172.21.15.46:0/1376099653 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f0ee80333f0 con 0x5629c86649d0 2024-09-10T07:46:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.240+0000 7f0f000da740 10 client.27920 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T07:46:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.240+0000 7f0f000da740 10 client.27920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.240+0000 7f0f000da740 20 client.27920 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.240+0000 7f0f000da740 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629c86e4ea0 0x5629c86e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.240+0000 7f0f000da740 4 client.27920 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:46:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.240+0000 7f0f000da740 1 -- 172.21.15.46:0/1376099653 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629c85d0b80 con 0x5629c86e4ea0 2024-09-10T07:46:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.242+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629c86e4ea0 0x5629c86e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.243+0000 7f0ef5e00640 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629c86e4ea0 0x5629c86e72c0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90603a48f9d0607 server_cookie=74600a77491feca9 in_seq=0 out_seq=0 2024-09-10T07:46:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.243+0000 7f0edbe00640 10 client.27920 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.351+0000 7f0edbe00640 1 -- 172.21.15.46:0/1376099653 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629c85d0b80 con 0x5629c86e4ea0 2024-09-10T07:46:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.351+0000 7f0edbe00640 10 client.27920 handle_command_reply: tid=0 2024-09-10T07:46:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.351+0000 7f0f000da740 1 client.27920 shutdown 2024-09-10T07:46:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.351+0000 7f0f000da740 1 -- 172.21.15.46:0/1376099653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c86649d0 msgr2=0x5629c86731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.351+0000 7f0f000da740 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c86649d0 0x5629c86731d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80095c0 tx=0x7f0ee8014bf0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0f000da740 1 -- 172.21.15.46:0/1376099653 shutdown_connections 2024-09-10T07:46:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0f000da740 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629c86e4ea0 0x5629c86e72c0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0f000da740 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629c8669a60 0x5629c8674190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0f000da740 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629c86649d0 0x5629c86731d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0f000da740 1 --2- 172.21.15.46:0/1376099653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629c8664090 0x5629c8672c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0f000da740 1 -- 172.21.15.46:0/1376099653 >> 172.21.15.46:0/1376099653 conn(0x5629c865f700 msgr2=0x5629c865e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0edbe00640 0 client.27920 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0f000da740 1 -- 172.21.15.46:0/1376099653 shutdown_connections 2024-09-10T07:46:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.352+0000 7f0f000da740 1 -- 172.21.15.46:0/1376099653 wait complete. 2024-09-10T07:46:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.353+0000 7f0f000da740 20 client.27920 trim_cache size 0 max 16384 2024-09-10T07:46:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:46:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:46:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:46:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 -- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0edc0a2c30 msgr2=0x7f0edc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0edc0a2c30 0x7f0edc0a50f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009890 tx=0x7f0ee80137b0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 -- 172.21.15.46:0/1773150658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef8109d40 msgr2=0x7f0ef81ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef8109d40 0x7f0ef81ae600 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000b970 tx=0x7f0ef000bd70 comp rx=0 tx=0).stop 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 -- 172.21.15.46:0/1773150658 shutdown_connections 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ef8109d40 0x7f0ef81ae600 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef8109380 0x7f0ef807ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0edc0a2c30 0x7f0edc0a50f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 --2- 172.21.15.46:0/1773150658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef8108470 0x7f0ef807fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.354+0000 7f0efd200640 1 -- 172.21.15.46:0/1773150658 >> 172.21.15.46:0/1773150658 conn(0x7f0ef8077470 msgr2=0x7f0ef810c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.355+0000 7f0efd200640 1 -- 172.21.15.46:0/1773150658 shutdown_connections 2024-09-10T07:46:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:40.355+0000 7f0efd200640 1 -- 172.21.15.46:0/1773150658 wait complete. 2024-09-10T07:46:40.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:39 smithi181 ceph-mon[35397]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.6 MiB/s wr, 561 op/s 2024-09-10T07:46:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:39 smithi063 ceph-mon[35020]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.6 MiB/s wr, 561 op/s 2024-09-10T07:46:40.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:39 smithi046 ceph-mon[30705]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.6 MiB/s wr, 561 op/s 2024-09-10T07:46:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 560.0 s, 707.0 tps, lat 138.352 ms stddev 151.876 2024-09-10T07:46:40.939 INFO:tasks.cephfs.filesystem:scrub status for tag:9b35bafd-e5df-4468-8c08-0e2a76325dd9 - {'path': '/', 'tag': '9b35bafd-e5df-4468-8c08-0e2a76325dd9', 'options': 'recursive,force'} 2024-09-10T07:46:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:42 smithi181 ceph-mon[35397]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-10T07:46:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:42 smithi063 ceph-mon[35020]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-10T07:46:42.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:42 smithi046 ceph-mon[30705]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.2 MiB/s wr, 647 op/s 2024-09-10T07:46:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:44 smithi181 ceph-mon[35397]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 5.3 MiB/s wr, 673 op/s 2024-09-10T07:46:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:44 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:46:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:44 smithi063 ceph-mon[35020]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 5.3 MiB/s wr, 673 op/s 2024-09-10T07:46:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:44 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:46:44.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:44 smithi046 ceph-mon[30705]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 5.3 MiB/s wr, 673 op/s 2024-09-10T07:46:44.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:44 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:46:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 565.0 s, 607.8 tps, lat 163.652 ms stddev 196.562 2024-09-10T07:46:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:46 smithi181 ceph-mon[35397]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.1 MiB/s wr, 559 op/s 2024-09-10T07:46:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:46 smithi063 ceph-mon[35020]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.1 MiB/s wr, 559 op/s 2024-09-10T07:46:46.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:46 smithi046 ceph-mon[30705]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.1 MiB/s wr, 559 op/s 2024-09-10T07:46:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:48 smithi181 ceph-mon[35397]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 5.7 MiB/s wr, 727 op/s 2024-09-10T07:46:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:48 smithi063 ceph-mon[35020]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 5.7 MiB/s wr, 727 op/s 2024-09-10T07:46:48.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:48 smithi046 ceph-mon[30705]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 5.7 MiB/s wr, 727 op/s 2024-09-10T07:46:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:50 smithi181 ceph-mon[35397]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 4.5 MiB/s wr, 591 op/s 2024-09-10T07:46:50.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:50 smithi063 ceph-mon[35020]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 4.5 MiB/s wr, 591 op/s 2024-09-10T07:46:50.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:50 smithi046 ceph-mon[30705]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 4.5 MiB/s wr, 591 op/s 2024-09-10T07:46:50.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 570.0 s, 653.6 tps, lat 156.049 ms stddev 177.688 2024-09-10T07:46:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:51 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:46:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:51 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:46:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:51 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:46:51.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:51 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:46:51.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:51 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:46:51.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:51 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:46:52.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:52 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:46:52.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:52 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:46:52.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:52 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:46:52.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:52 smithi063 ceph-mon[35020]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.8 MiB/s wr, 612 op/s 2024-09-10T07:46:52.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:52 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:46:52.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:52 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:46:52.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:52 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:46:52.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:52 smithi181 ceph-mon[35397]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.8 MiB/s wr, 612 op/s 2024-09-10T07:46:52.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:52 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:46:52.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:52 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:46:52.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:52 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:46:52.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:52 smithi046 ceph-mon[30705]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.8 MiB/s wr, 612 op/s 2024-09-10T07:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 -- 172.21.15.46:0/689995210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 msgr2=0x7f4a5c109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 --2- 172.21.15.46:0/689995210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 0x7f4a5c109fa0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009be0 tx=0x7f4a5402a6d0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 -- 172.21.15.46:0/689995210 shutdown_connections 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 --2- 172.21.15.46:0/689995210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 0x7f4a5c109fa0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 --2- 172.21.15.46:0/689995210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a5c102d50 0x7f4a5c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 --2- 172.21.15.46:0/689995210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a5c101b60 0x7f4a5c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 -- 172.21.15.46:0/689995210 >> 172.21.15.46:0/689995210 conn(0x7f4a5c0fd2e0 msgr2=0x7f4a5c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 -- 172.21.15.46:0/689995210 shutdown_connections 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.559+0000 7f4a62200640 1 -- 172.21.15.46:0/689995210 wait complete. 2024-09-10T07:46:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.560+0000 7f4a62200640 1 Processor -- start 2024-09-10T07:46:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.561+0000 7f4a62200640 1 -- start start 2024-09-10T07:46:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.561+0000 7f4a62200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a5c101b60 0x7f4a5c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.561+0000 7f4a62200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a5c102d50 0x7f4a5c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.562+0000 7f4a62200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 0x7f4a5c1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.562+0000 7f4a62200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c10d7c0 con 0x7f4a5c101b60 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.562+0000 7f4a62200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c10d640 con 0x7f4a5c103710 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.562+0000 7f4a62200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c10d940 con 0x7f4a5c102d50 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.562+0000 7f4a5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a5c101b60 0x7f4a5c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.562+0000 7f4a5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a5c101b60 0x7f4a5c199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51542/0 (socket says 172.21.15.46:51542) 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.562+0000 7f4a5b400640 1 -- 172.21.15.46:0/3550694722 learned_addr learned my addr 172.21.15.46:0/3550694722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.562+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a5c102d50 0x7f4a5c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.563+0000 7f4a5be00640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 0x7f4a5c1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.563+0000 7f4a5b400640 1 -- 172.21.15.46:0/3550694722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a5c102d50 msgr2=0x7f4a5c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.563+0000 7f4a5b400640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a5c102d50 0x7f4a5c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.563+0000 7f4a5b400640 1 -- 172.21.15.46:0/3550694722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 msgr2=0x7f4a5c1a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.563+0000 7f4a5b400640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 0x7f4a5c1a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.563+0000 7f4a5b400640 1 -- 172.21.15.46:0/3550694722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c1a1ba0 con 0x7f4a5c101b60 2024-09-10T07:46:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.563+0000 7f4a5be00640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 0x7f4a5c1a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.564+0000 7f4a5b400640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a5c101b60 0x7f4a5c199f80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00e9f0 tx=0x7f4a4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.564+0000 7f4a53e00640 1 -- 172.21.15.46:0/3550694722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00ccc0 con 0x7f4a5c101b60 2024-09-10T07:46:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.564+0000 7f4a62200640 1 -- 172.21.15.46:0/3550694722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c1a1e90 con 0x7f4a5c101b60 2024-09-10T07:46:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.564+0000 7f4a53e00640 1 -- 172.21.15.46:0/3550694722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4c0035f0 con 0x7f4a5c101b60 2024-09-10T07:46:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.564+0000 7f4a53e00640 1 -- 172.21.15.46:0/3550694722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00ce60 con 0x7f4a5c101b60 2024-09-10T07:46:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.565+0000 7f4a62200640 1 -- 172.21.15.46:0/3550694722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c1a2350 con 0x7f4a5c101b60 2024-09-10T07:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.569+0000 7f4a53e00640 1 -- 172.21.15.46:0/3550694722 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a4c0037a0 con 0x7f4a5c101b60 2024-09-10T07:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.570+0000 7f4a6509a740 1 Processor -- start 2024-09-10T07:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.570+0000 7f4a6509a740 1 -- start start 2024-09-10T07:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e1948a0 0x56403e194ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a53e00640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a400a2d30 0x7f4a400a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a53e00640 1 -- 172.21.15.46:0/3550694722 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4a4c0cf690 con 0x7f4a5c101b60 2024-09-10T07:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1951e0 0x56403e199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 0x56403e19c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a6509a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56403e0f1100 con 0x56403e19a270 2024-09-10T07:46:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a6509a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56403e0991b0 con 0x56403e1951e0 2024-09-10T07:46:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a6509a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56403e0ed0c0 con 0x56403e1948a0 2024-09-10T07:46:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.572+0000 7f4a5be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e1948a0 0x56403e194ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.573+0000 7f4a5be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e1948a0 0x56403e194ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38342/0 (socket says 172.21.15.46:38342) 2024-09-10T07:46:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.573+0000 7f4a5be00640 1 -- 172.21.15.46:0/1976843182 learned_addr learned my addr 172.21.15.46:0/1976843182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.575+0000 7f4a5b400640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a400a2d30 0x7f4a400a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.575+0000 7f4a5be00640 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 0x56403e19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.575+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1951e0 0x56403e199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.575+0000 7f4a5be00640 1 -- 172.21.15.46:0/1976843182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e1948a0 msgr2=0x56403e194ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.575+0000 7f4a5be00640 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e1948a0 0x56403e194ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.575+0000 7f4a5be00640 1 -- 172.21.15.46:0/1976843182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1951e0 msgr2=0x56403e199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.575+0000 7f4a5be00640 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1951e0 0x56403e199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.575+0000 7f4a5be00640 1 -- 172.21.15.46:0/1976843182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403de04210 con 0x56403e19a270 2024-09-10T07:46:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.576+0000 7f4a5be00640 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 0x56403e19c730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a54005ee0 tx=0x7f4a5402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bc53d7e95a36e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.577+0000 7f4a47e00640 1 -- 172.21.15.46:0/1976843182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54036040 con 0x56403e19a270 2024-09-10T07:46:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.577+0000 7f4a5b400640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a400a2d30 0x7f4a400a51f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0a4250 tx=0x7f4a4c0045d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.577+0000 7f4a47e00640 1 -- 172.21.15.46:0/1976843182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a54037040 con 0x56403e19a270 2024-09-10T07:46:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.577+0000 7f4a47e00640 1 -- 172.21.15.46:0/1976843182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54004580 con 0x56403e19a270 2024-09-10T07:46:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.578+0000 7f4a6509a740 1 -- 172.21.15.46:0/1976843182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 msgr2=0x56403e19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.578+0000 7f4a6509a740 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 0x56403e19c730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a54005ee0 tx=0x7f4a5402a6d0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.579+0000 7f4a6509a740 1 -- 172.21.15.46:0/1976843182 shutdown_connections 2024-09-10T07:46:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.579+0000 7f4a6509a740 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 0x56403e19c730 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.579+0000 7f4a6509a740 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1951e0 0x56403e199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.579+0000 7f4a6509a740 1 --2- 172.21.15.46:0/1976843182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e1948a0 0x56403e194ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.579+0000 7f4a6509a740 1 -- 172.21.15.46:0/1976843182 >> 172.21.15.46:0/1976843182 conn(0x56403e18ff10 msgr2=0x56403e190310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.579+0000 7f4a6509a740 1 -- 172.21.15.46:0/1976843182 shutdown_connections 2024-09-10T07:46:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.579+0000 7f4a6509a740 1 -- 172.21.15.46:0/1976843182 wait complete. 2024-09-10T07:46:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.580+0000 7f4a6509a740 1 -- start start 2024-09-10T07:46:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.580+0000 7f4a6509a740 1 Processor -- start 2024-09-10T07:46:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.581+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 0x56403e19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.581+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1951e0 0x56403e19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.581+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.581+0000 7f4a6509a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56403e19d1e0 con 0x56403e1951e0 2024-09-10T07:46:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.581+0000 7f4a6509a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56403e07a090 con 0x56403e1948a0 2024-09-10T07:46:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.581+0000 7f4a6509a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56403e19d360 con 0x56403e19a270 2024-09-10T07:46:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.581+0000 7f4a6509a740 10 client.0 _ll_register_callbacks cb 0x56403e0cef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.582+0000 7f4a5aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1951e0 0x56403e19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.582+0000 7f4a5aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1951e0 0x56403e19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51556/0 (socket says 172.21.15.46:51556) 2024-09-10T07:46:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.582+0000 7f4a5aa00640 1 -- 172.21.15.46:0/279355150 learned_addr learned my addr 172.21.15.46:0/279355150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.583+0000 7f4a5aa00640 1 -- 172.21.15.46:0/279355150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 msgr2=0x56403e1a6010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:46:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.583+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a6010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.583+0000 7f4a5aa00640 1 -- 172.21.15.46:0/279355150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 msgr2=0x56403e19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.583+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 0x56403e19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.583+0000 7f4a5aa00640 1 -- 172.21.15.46:0/279355150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403ded0620 con 0x56403e1951e0 2024-09-10T07:46:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.586+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1951e0 0x56403e19efd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a48012700 tx=0x7f4a48012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1d3971f2da5fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.588+0000 7f4a44c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:46:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.588+0000 7f4a44c00640 1 -- 172.21.15.46:0/279355150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48004050 con 0x56403e1951e0 2024-09-10T07:46:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.588+0000 7f4a44c00640 1 -- 172.21.15.46:0/279355150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4a480041f0 con 0x56403e1951e0 2024-09-10T07:46:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.588+0000 7f4a6509a740 1 -- 172.21.15.46:0/279355150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a54037040 con 0x56403e1951e0 2024-09-10T07:46:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.588+0000 7f4a44c00640 1 -- 172.21.15.46:0/279355150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f4a480043f0 con 0x56403e1951e0 2024-09-10T07:46:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.588+0000 7f4a6509a740 10 client.17277 fetch_fsmap learned FSMap version 50 2024-09-10T07:46:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.588+0000 7f4a6509a740 1 -- 172.21.15.46:0/279355150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56403e1a8ed0 con 0x56403e1951e0 2024-09-10T07:46:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.589+0000 7f4a44c00640 1 -- 172.21.15.46:0/279355150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a54037040 con 0x56403e1951e0 2024-09-10T07:46:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.590+0000 7f4a44c00640 1 -- 172.21.15.46:0/279355150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f4a48003c50 con 0x56403e1951e0 2024-09-10T07:46:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.590+0000 7f4a6509a740 10 client.17277 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:46:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.590+0000 7f4a6509a740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.590+0000 7f4a6509a740 20 client.17277 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.591+0000 7f4a6509a740 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56403e1aa550 0x56403e1ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.591+0000 7f4a6509a740 4 client.17277 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.591+0000 7f4a6509a740 1 -- 172.21.15.46:0/279355150 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56403e0fe830 con 0x56403e1aa550 2024-09-10T07:46:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.594+0000 7f4a5be00640 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56403e1aa550 0x56403e1ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.599+0000 7f4a44c00640 10 client.17277 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.599+0000 7f4a5be00640 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56403e1aa550 0x56403e1ac9c0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d9d2f451ead3bd server_cookie=197dfbb2d63a057 in_seq=0 out_seq=0 2024-09-10T07:46:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.601+0000 7f4a44c00640 1 -- 172.21.15.46:0/279355150 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56403e0fe830 con 0x56403e1aa550 2024-09-10T07:46:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.601+0000 7f4a44c00640 10 client.17277 handle_command_reply: tid=0 2024-09-10T07:46:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.601+0000 7f4a6509a740 1 client.17277 shutdown 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a6509a740 1 -- 172.21.15.46:0/279355150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1951e0 msgr2=0x56403e19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a6509a740 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1951e0 0x56403e19efd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a48012700 tx=0x7f4a48012bc0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a6509a740 1 -- 172.21.15.46:0/279355150 shutdown_connections 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a6509a740 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56403e1aa550 0x56403e1ac9c0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a6509a740 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a6509a740 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1951e0 0x56403e19efd0 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a48012700 tx=0x7f4a48012bc0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a6509a740 1 --2- 172.21.15.46:0/279355150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 0x56403e19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a6509a740 1 -- 172.21.15.46:0/279355150 >> 172.21.15.46:0/279355150 conn(0x56403e18ff10 msgr2=0x56403e191720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.602+0000 7f4a44c00640 0 client.17277 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.604+0000 7f4a6509a740 1 -- 172.21.15.46:0/279355150 shutdown_connections 2024-09-10T07:46:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.605+0000 7f4a6509a740 1 -- 172.21.15.46:0/279355150 wait complete. 2024-09-10T07:46:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.605+0000 7f4a6509a740 20 client.17277 trim_cache size 0 max 16384 2024-09-10T07:46:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.649+0000 7f4a6509a740 1 Processor -- start 2024-09-10T07:46:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.650+0000 7f4a6509a740 1 -- start start 2024-09-10T07:46:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.650+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1948a0 0x56403e191300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.650+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.650+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e182b00 0x56403e191840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.650+0000 7f4a6509a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1a85c0 con 0x56403e1948a0 2024-09-10T07:46:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.650+0000 7f4a6509a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1aa100 con 0x56403e182b00 2024-09-10T07:46:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.650+0000 7f4a6509a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1a8c90 con 0x56403e19a270 2024-09-10T07:46:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.651+0000 7f4a5aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.651+0000 7f4a5aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38370/0 (socket says 172.21.15.46:38370) 2024-09-10T07:46:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.651+0000 7f4a5aa00640 1 -- 172.21.15.46:0/739948198 learned_addr learned my addr 172.21.15.46:0/739948198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.651+0000 7f4a5be00640 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e182b00 0x56403e191840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.652+0000 7f4a5be00640 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1948a0 0x56403e191300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.652+0000 7f4a5aa00640 1 -- 172.21.15.46:0/739948198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e182b00 msgr2=0x56403e191840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.652+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e182b00 0x56403e191840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.652+0000 7f4a5aa00640 1 -- 172.21.15.46:0/739948198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1948a0 msgr2=0x56403e191300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.652+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1948a0 0x56403e191300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.652+0000 7f4a5aa00640 1 -- 172.21.15.46:0/739948198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403e18fd30 con 0x56403e19a270 2024-09-10T07:46:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.652+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a5fd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4a48000c00 tx=0x7f4a480096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b123fd601f1220a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.653+0000 7f4a47400640 1 -- 172.21.15.46:0/739948198 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4802c040 con 0x56403e19a270 2024-09-10T07:46:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.653+0000 7f4a47400640 1 -- 172.21.15.46:0/739948198 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48010670 con 0x56403e19a270 2024-09-10T07:46:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.653+0000 7f4a47400640 1 -- 172.21.15.46:0/739948198 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480052b0 con 0x56403e19a270 2024-09-10T07:46:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.654+0000 7f4a6509a740 1 -- 172.21.15.46:0/739948198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 msgr2=0x56403e1a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.654+0000 7f4a6509a740 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a5fd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4a48000c00 tx=0x7f4a480096c0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.656+0000 7f4a6509a740 1 -- 172.21.15.46:0/739948198 shutdown_connections 2024-09-10T07:46:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.656+0000 7f4a6509a740 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e182b00 0x56403e191840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.656+0000 7f4a6509a740 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e19a270 0x56403e1a5fd0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.656+0000 7f4a6509a740 1 --2- 172.21.15.46:0/739948198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e1948a0 0x56403e191300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.656+0000 7f4a6509a740 1 -- 172.21.15.46:0/739948198 >> 172.21.15.46:0/739948198 conn(0x56403e18ff10 msgr2=0x56403e186880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.657+0000 7f4a6509a740 1 -- 172.21.15.46:0/739948198 shutdown_connections 2024-09-10T07:46:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.657+0000 7f4a6509a740 1 -- 172.21.15.46:0/739948198 wait complete. 2024-09-10T07:46:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.658+0000 7f4a6509a740 1 -- start start 2024-09-10T07:46:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.658+0000 7f4a6509a740 1 Processor -- start 2024-09-10T07:46:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.659+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e182b00 0x56403e1a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.659+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 0x56403e1a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.659+0000 7f4a6509a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 0x56403e1a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.659+0000 7f4a6509a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1a9a30 con 0x56403e19a270 2024-09-10T07:46:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.659+0000 7f4a6509a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1a9360 con 0x56403e1948a0 2024-09-10T07:46:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.659+0000 7f4a6509a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1a65c0 con 0x56403e182b00 2024-09-10T07:46:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.659+0000 7f4a6509a740 10 client.0 _ll_register_callbacks cb 0x7f4a480097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.660+0000 7f4a5be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e182b00 0x56403e1a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.660+0000 7f4a5be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e182b00 0x56403e1a3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38374/0 (socket says 172.21.15.46:38374) 2024-09-10T07:46:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.660+0000 7f4a5be00640 1 -- 172.21.15.46:0/2675261469 learned_addr learned my addr 172.21.15.46:0/2675261469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.661+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 0x56403e1a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.661+0000 7f4a5be00640 1 -- 172.21.15.46:0/2675261469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 msgr2=0x56403e1a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.661+0000 7f4a5be00640 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 0x56403e1a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.661+0000 7f4a5be00640 1 -- 172.21.15.46:0/2675261469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 msgr2=0x56403e1a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.661+0000 7f4a5be00640 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 0x56403e1a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.661+0000 7f4a5be00640 1 -- 172.21.15.46:0/2675261469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403e1a5540 con 0x56403e182b00 2024-09-10T07:46:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.662+0000 7f4a5be00640 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e182b00 0x56403e1a3a30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009be0 tx=0x7f4a5402f300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad1f028c694760e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.662+0000 7f4a44c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.663+0000 7f4a44c00640 1 -- 172.21.15.46:0/2675261469 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54036070 con 0x56403e182b00 2024-09-10T07:46:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.663+0000 7f4a6509a740 1 -- 172.21.15.46:0/2675261469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a48010670 con 0x56403e182b00 2024-09-10T07:46:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.663+0000 7f4a44c00640 1 -- 172.21.15.46:0/2675261469 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4a54003970 con 0x56403e182b00 2024-09-10T07:46:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.663+0000 7f4a44c00640 1 -- 172.21.15.46:0/2675261469 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f4a5405d050 con 0x56403e182b00 2024-09-10T07:46:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.664+0000 7f4a6509a740 10 client.27880 fetch_fsmap learned FSMap version 50 2024-09-10T07:46:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.664+0000 7f4a6509a740 1 -- 172.21.15.46:0/2675261469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56403e187330 con 0x56403e182b00 2024-09-10T07:46:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.665+0000 7f4a44c00640 1 -- 172.21.15.46:0/2675261469 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48010670 con 0x56403e182b00 2024-09-10T07:46:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.666+0000 7f4a44c00640 1 -- 172.21.15.46:0/2675261469 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f4a5403a070 con 0x56403e182b00 2024-09-10T07:46:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.666+0000 7f4a6509a740 10 client.27880 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:46:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.666+0000 7f4a6509a740 10 client.27880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:46:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.666+0000 7f4a6509a740 20 client.27880 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.666+0000 7f4a6509a740 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56403e187760 0x56403e189b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.666+0000 7f4a6509a740 4 client.27880 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:46:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.666+0000 7f4a6509a740 1 -- 172.21.15.46:0/2675261469 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56403e0f8b80 con 0x56403e187760 2024-09-10T07:46:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.667+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56403e187760 0x56403e189b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.668+0000 7f4a5aa00640 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56403e187760 0x56403e189b50 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e162875491d9c0b server_cookie=e44b50fd883ade16 in_seq=0 out_seq=0 2024-09-10T07:46:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.670+0000 7f4a44c00640 10 client.27880 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.687+0000 7f4a44c00640 1 -- 172.21.15.46:0/2675261469 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56403e0f8b80 con 0x56403e187760 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.687+0000 7f4a44c00640 10 client.27880 handle_command_reply: tid=0 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.687+0000 7f4a6509a740 1 client.27880 shutdown 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.687+0000 7f4a6509a740 1 -- 172.21.15.46:0/2675261469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e182b00 msgr2=0x56403e1a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.687+0000 7f4a6509a740 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e182b00 0x56403e1a3a30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009be0 tx=0x7f4a5402f300 comp rx=0 tx=0).stop 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 1 -- 172.21.15.46:0/2675261469 shutdown_connections 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56403e187760 0x56403e189b50 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403e19a270 0x56403e1a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a44c00640 0 client.27880 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56403e1948a0 0x56403e1a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 1 --2- 172.21.15.46:0/2675261469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56403e182b00 0x56403e1a3a30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 1 -- 172.21.15.46:0/2675261469 >> 172.21.15.46:0/2675261469 conn(0x56403e18ff10 msgr2=0x56403e1902f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 1 -- 172.21.15.46:0/2675261469 shutdown_connections 2024-09-10T07:46:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 1 -- 172.21.15.46:0/2675261469 wait complete. 2024-09-10T07:46:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.688+0000 7f4a6509a740 20 client.27880 trim_cache size 0 max 16384 2024-09-10T07:46:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:46:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:46:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:46:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 -- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a400a2d30 msgr2=0x7f4a400a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a400a2d30 0x7f4a400a51f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0a4250 tx=0x7f4a4c0045d0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 -- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a5c101b60 msgr2=0x7f4a5c199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a5c101b60 0x7f4a5c199f80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00e9f0 tx=0x7f4a4c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 -- 172.21.15.46:0/3550694722 shutdown_connections 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a5c103710 0x7f4a5c1a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a5c102d50 0x7f4a5c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a400a2d30 0x7f4a400a51f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 --2- 172.21.15.46:0/3550694722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a5c101b60 0x7f4a5c199f80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.689+0000 7f4a62200640 1 -- 172.21.15.46:0/3550694722 >> 172.21.15.46:0/3550694722 conn(0x7f4a5c0fd2e0 msgr2=0x7f4a5c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.690+0000 7f4a62200640 1 -- 172.21.15.46:0/3550694722 shutdown_connections 2024-09-10T07:46:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:46:53.690+0000 7f4a62200640 1 -- 172.21.15.46:0/3550694722 wait complete. 2024-09-10T07:46:53.906 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-10T07:46:53.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:46:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:54 smithi181 ceph-mon[35397]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.8 MiB/s wr, 568 op/s 2024-09-10T07:46:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:54 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:46:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:54 smithi063 ceph-mon[35020]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.8 MiB/s wr, 568 op/s 2024-09-10T07:46:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:54 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:46:54.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:54 smithi046 ceph-mon[30705]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.8 MiB/s wr, 568 op/s 2024-09-10T07:46:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:54 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:46:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 575.0 s, 788.2 tps, lat 125.332 ms stddev 136.714 2024-09-10T07:46:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.713+0000 7fc2b1e00640 1 -- 172.21.15.46:0/3518099991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac109450 msgr2=0x7fc2ac110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.713+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/3518099991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac109450 0x7fc2ac110250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc294009900 tx=0x7fc29402a420 comp rx=0 tx=0).stop 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.714+0000 7fc2b1e00640 1 -- 172.21.15.46:0/3518099991 shutdown_connections 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.714+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/3518099991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2ac11b140 0x7fc2ac11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.714+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/3518099991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac109830 0x7fc2ac109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.714+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/3518099991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac109450 0x7fc2ac110250 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.714+0000 7fc2b1e00640 1 -- 172.21.15.46:0/3518099991 >> 172.21.15.46:0/3518099991 conn(0x7fc2ac06e140 msgr2=0x7fc2ac06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.714+0000 7fc2b1e00640 1 -- 172.21.15.46:0/3518099991 shutdown_connections 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.714+0000 7fc2b1e00640 1 -- 172.21.15.46:0/3518099991 wait complete. 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.716+0000 7fc2b1e00640 1 Processor -- start 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.717+0000 7fc2b1e00640 1 -- start start 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.717+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2ac109450 0x7fc2ac1a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.717+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac109830 0x7fc2ac1a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.718+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac11b140 0x7fc2ac1abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.718+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac120df0 con 0x7fc2ac109830 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.718+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac120c70 con 0x7fc2ac109450 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.718+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac120f70 con 0x7fc2ac11b140 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.718+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2ac109450 0x7fc2ac1a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.718+0000 7fc2ab400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac11b140 0x7fc2ac1abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.718+0000 7fc2ab400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac11b140 0x7fc2ac1abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38384/0 (socket says 172.21.15.46:38384) 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.718+0000 7fc2ab400640 1 -- 172.21.15.46:0/2177072647 learned_addr learned my addr 172.21.15.46:0/2177072647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.719+0000 7fc2a2000640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac109830 0x7fc2ac1a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.720+0000 7fc2a2000640 1 -- 172.21.15.46:0/2177072647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac11b140 msgr2=0x7fc2ac1abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.720+0000 7fc2a2000640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac11b140 0x7fc2ac1abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.720+0000 7fc2a2000640 1 -- 172.21.15.46:0/2177072647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2ac109450 msgr2=0x7fc2ac1a7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.720+0000 7fc2a2000640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2ac109450 0x7fc2ac1a7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.720+0000 7fc2a2000640 1 -- 172.21.15.46:0/2177072647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac1ac240 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.720+0000 7fc2aaa00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2ac109450 0x7fc2ac1a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.721+0000 7fc2a2000640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac109830 0x7fc2ac1a77d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00e9f0 tx=0x7fc29c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.722+0000 7fc2b0a00640 1 -- 172.21.15.46:0/2177072647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00ccc0 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.722+0000 7fc2b0a00640 1 -- 172.21.15.46:0/2177072647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c010910 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.722+0000 7fc2b0a00640 1 -- 172.21.15.46:0/2177072647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c018960 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.722+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac1ac530 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.722+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1b3e10 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.725+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac1a85f0 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.728+0000 7fc2b0a00640 1 -- 172.21.15.46:0/2177072647 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc29c010ac0 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.730+0000 7fc2b0a00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc28c0a2c00 0x7fc28c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.731+0000 7fc2b0a00640 1 -- 172.21.15.46:0/2177072647 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc29c0cf780 con 0x7fc2ac109830 2024-09-10T07:46:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.733+0000 7fc2aaa00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc28c0a2c00 0x7fc28c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:56.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.740+0000 7fc2aaa00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc28c0a2c00 0x7fc28c0a50c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc29402aa10 tx=0x7fc294005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:56.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:55.750+0000 7fc2b0a00640 1 -- 172.21.15.46:0/2177072647 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc29c09c360 con 0x7fc2ac109830 2024-09-10T07:46:56.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.159+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2ac02d7c0 con 0x7fc2ac109830 2024-09-10T07:46:56.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.166+0000 7fc2b0a00640 1 -- 172.21.15.46:0/2177072647 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7fc29c09bc60 con 0x7fc2ac109830 2024-09-10T07:46:56.170 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:46:56.170 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"btime":"2024-09-10T07:46:53:072255+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:46:52.230107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc28c0a2c00 msgr2=0x7fc28c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc28c0a2c00 0x7fc28c0a50c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc29402aa10 tx=0x7fc294005fd0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac109830 msgr2=0x7fc2ac1a77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac109830 0x7fc2ac1a77d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00e9f0 tx=0x7fc29c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 shutdown_connections 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2ac11b140 0x7fc2ac1abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc28c0a2c00 0x7fc28c0a50c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac109830 0x7fc2ac1a77d0 secure :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00e9f0 tx=0x7fc29c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 --2- 172.21.15.46:0/2177072647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2ac109450 0x7fc2ac1a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:56.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.171+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 >> 172.21.15.46:0/2177072647 conn(0x7fc2ac06e140 msgr2=0x7fc2ac11bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:56.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.172+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 shutdown_connections 2024-09-10T07:46:56.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:56.172+0000 7fc2b1e00640 1 -- 172.21.15.46:0/2177072647 wait complete. 2024-09-10T07:46:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:56 smithi181 ceph-mon[35397]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-09-10T07:46:56.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:56 smithi063 ceph-mon[35020]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-09-10T07:46:56.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:56 smithi046 ceph-mon[30705]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-09-10T07:46:57.075 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-10T07:46:57.075 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-10T07:46:57.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:57 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2177072647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:57.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:57 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2177072647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:57.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:57 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2177072647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:46:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:46:58.334 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:46:58 smithi046 ceph-mon[30705]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-10T07:46:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:46:58 smithi181 ceph-mon[35397]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-10T07:46:58.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:46:58 smithi063 ceph-mon[35020]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-10T07:46:58.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.546+0000 7fc77aa00640 1 -- 172.21.15.46:0/3708166923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7741008c0 msgr2=0x7fc774100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.546+0000 7fc77aa00640 1 --2- 172.21.15.46:0/3708166923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7741008c0 0x7fc774100ca0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc764009930 tx=0x7fc76402a420 comp rx=0 tx=0).stop 2024-09-10T07:46:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.547+0000 7fc77aa00640 1 -- 172.21.15.46:0/3708166923 shutdown_connections 2024-09-10T07:46:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.547+0000 7fc77aa00640 1 --2- 172.21.15.46:0/3708166923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774105ec0 0x7fc774106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.547+0000 7fc77aa00640 1 --2- 172.21.15.46:0/3708166923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7741011e0 0x7fc774105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.547+0000 7fc77aa00640 1 --2- 172.21.15.46:0/3708166923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7741008c0 0x7fc774100ca0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.547+0000 7fc77aa00640 1 -- 172.21.15.46:0/3708166923 >> 172.21.15.46:0/3708166923 conn(0x7fc7740fc7d0 msgr2=0x7fc7740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.547+0000 7fc77aa00640 1 -- 172.21.15.46:0/3708166923 shutdown_connections 2024-09-10T07:46:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.547+0000 7fc77aa00640 1 -- 172.21.15.46:0/3708166923 wait complete. 2024-09-10T07:46:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.548+0000 7fc77aa00640 1 Processor -- start 2024-09-10T07:46:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.548+0000 7fc77aa00640 1 -- start start 2024-09-10T07:46:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.548+0000 7fc77aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7741008c0 0x7fc7741a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.548+0000 7fc77aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7741011e0 0x7fc7741a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.548+0000 7fc77aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc774105ec0 0x7fc77419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.548+0000 7fc77aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc774118660 con 0x7fc7741008c0 2024-09-10T07:46:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.548+0000 7fc77aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741184e0 con 0x7fc774105ec0 2024-09-10T07:46:58.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.548+0000 7fc77aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741187e0 con 0x7fc7741011e0 2024-09-10T07:46:58.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc778a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc774105ec0 0x7fc77419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc773e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7741008c0 0x7fc7741a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc778a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc774105ec0 0x7fc77419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53168/0 (socket says 172.21.15.46:53168) 2024-09-10T07:46:58.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc778a00640 1 -- 172.21.15.46:0/632909451 learned_addr learned my addr 172.21.15.46:0/632909451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:58.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc773400640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7741011e0 0x7fc7741a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc773e00640 1 -- 172.21.15.46:0/632909451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7741011e0 msgr2=0x7fc7741a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc773e00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7741011e0 0x7fc7741a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc773e00640 1 -- 172.21.15.46:0/632909451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc774105ec0 msgr2=0x7fc77419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc773e00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc774105ec0 0x7fc77419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.549+0000 7fc773e00640 1 -- 172.21.15.46:0/632909451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77419f060 con 0x7fc7741008c0 2024-09-10T07:46:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.550+0000 7fc773e00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7741008c0 0x7fc7741a4770 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc76402a940 tx=0x7fc764003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.550+0000 7fc770c00640 1 -- 172.21.15.46:0/632909451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76403f070 con 0x7fc7741008c0 2024-09-10T07:46:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.550+0000 7fc77aa00640 1 -- 172.21.15.46:0/632909451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77419f2f0 con 0x7fc7741008c0 2024-09-10T07:46:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.551+0000 7fc770c00640 1 -- 172.21.15.46:0/632909451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc764031c90 con 0x7fc7741008c0 2024-09-10T07:46:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.551+0000 7fc770c00640 1 -- 172.21.15.46:0/632909451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764035030 con 0x7fc7741008c0 2024-09-10T07:46:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.551+0000 7fc77aa00640 1 -- 172.21.15.46:0/632909451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7741ab640 con 0x7fc7741008c0 2024-09-10T07:46:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.555+0000 7fc77d67e740 1 Processor -- start 2024-09-10T07:46:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.555+0000 7fc770c00640 1 -- 172.21.15.46:0/632909451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc764042070 con 0x7fc7741008c0 2024-09-10T07:46:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.555+0000 7fc77d67e740 1 -- start start 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cb3b0 0x562b561cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cbcd0 0x562b561d6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561d8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc77d67e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562b56131530 con 0x562b561cb3b0 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc77d67e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b560d8860 con 0x562b561cbcd0 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc77d67e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562b5612d4f0 con 0x562b561d6590 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc773e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cbcd0 0x562b561d6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc778a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cb3b0 0x562b561cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.556+0000 7fc773e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cbcd0 0x562b561d6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53182/0 (socket says 172.21.15.46:53182) 2024-09-10T07:46:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773e00640 1 -- 172.21.15.46:0/2645331820 learned_addr learned my addr 172.21.15.46:0/2645331820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773400640 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561d8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773e00640 1 -- 172.21.15.46:0/2645331820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 msgr2=0x562b561d8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773e00640 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561d8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773e00640 1 -- 172.21.15.46:0/2645331820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cb3b0 msgr2=0x562b561cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773e00640 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cb3b0 0x562b561cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773e00640 1 -- 172.21.15.46:0/2645331820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b55ea2b90 con 0x562b561cbcd0 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc778a00640 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cb3b0 0x562b561cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773400640 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc773e00640 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cbcd0 0x562b561d6050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc7640f5c20 tx=0x7fc7640f5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3390dd0508fa2aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.557+0000 7fc770c00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc7580a2c00 0x7fc7580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.558+0000 7fc769200640 1 -- 172.21.15.46:0/2645331820 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640fe070 con 0x562b561cbcd0 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.558+0000 7fc770c00640 1 -- 172.21.15.46:0/632909451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc7640ef8a0 con 0x7fc7741008c0 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.558+0000 7fc778a00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc7580a2c00 0x7fc7580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.558+0000 7fc769200640 1 -- 172.21.15.46:0/2645331820 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7640bc420 con 0x562b561cbcd0 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.558+0000 7fc769200640 1 -- 172.21.15.46:0/2645331820 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640bb680 con 0x562b561cbcd0 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 -- 172.21.15.46:0/2645331820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cbcd0 msgr2=0x562b561d6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cbcd0 0x562b561d6050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc7640f5c20 tx=0x7fc7640f5d10 comp rx=0 tx=0).stop 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc778a00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc7580a2c00 0x7fc7580a50c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009270 tx=0x7fc76c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 -- 172.21.15.46:0/2645331820 shutdown_connections 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cbcd0 0x562b561d6050 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 --2- 172.21.15.46:0/2645331820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cb3b0 0x562b561cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 -- 172.21.15.46:0/2645331820 >> 172.21.15.46:0/2645331820 conn(0x562b561c7940 msgr2=0x562b561c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 -- 172.21.15.46:0/2645331820 shutdown_connections 2024-09-10T07:46:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.559+0000 7fc77d67e740 1 -- 172.21.15.46:0/2645331820 wait complete. 2024-09-10T07:46:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.560+0000 7fc77d67e740 1 -- start start 2024-09-10T07:46:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.560+0000 7fc77d67e740 1 Processor -- start 2024-09-10T07:46:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cb3b0 0x562b561d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561d38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561d6590 0x562b561d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc77d67e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562b561e9f30 con 0x562b561cbcd0 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc77d67e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b560ebe60 con 0x562b561d6590 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc77d67e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562b561ea0b0 con 0x562b561cb3b0 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc77d67e740 10 client.0 _ll_register_callbacks cb 0x562b560f0cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc773e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cb3b0 0x562b561d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc773e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cb3b0 0x562b561d33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57508/0 (socket says 172.21.15.46:57508) 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc773e00640 1 -- 172.21.15.46:0/3452768062 learned_addr learned my addr 172.21.15.46:0/3452768062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc773400640 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561d38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.561+0000 7fc778a00640 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561d6590 0x562b561d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.562+0000 7fc773e00640 1 -- 172.21.15.46:0/3452768062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561d6590 msgr2=0x562b561d3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.562+0000 7fc773e00640 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561d6590 0x562b561d3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.562+0000 7fc773e00640 1 -- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 msgr2=0x562b561d38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.563+0000 7fc773e00640 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561d38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.563+0000 7fc773e00640 1 -- 172.21.15.46:0/3452768062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b560f7990 con 0x562b561cb3b0 2024-09-10T07:46:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.563+0000 7fc773400640 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561d38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.564+0000 7fc773e00640 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cb3b0 0x562b561d33b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc7640f7670 tx=0x7fc7640092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee01ba31aee0fae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.564+0000 7fc75d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:46:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.564+0000 7fc75d600640 1 -- 172.21.15.46:0/3452768062 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640fe070 con 0x562b561cb3b0 2024-09-10T07:46:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.564+0000 7fc77d67e740 1 -- 172.21.15.46:0/3452768062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7640bc420 con 0x562b561cb3b0 2024-09-10T07:46:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.564+0000 7fc75d600640 1 -- 172.21.15.46:0/3452768062 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc7640b8390 con 0x562b561cb3b0 2024-09-10T07:46:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.564+0000 7fc75d600640 1 -- 172.21.15.46:0/3452768062 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc764049030 con 0x562b561cb3b0 2024-09-10T07:46:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.564+0000 7fc77d67e740 10 client.27898 fetch_fsmap learned FSMap version 50 2024-09-10T07:46:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.564+0000 7fc77d67e740 1 -- 172.21.15.46:0/3452768062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562b561e9f30 con 0x562b561cb3b0 2024-09-10T07:46:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.565+0000 7fc75d600640 1 -- 172.21.15.46:0/3452768062 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7640bc420 con 0x562b561cb3b0 2024-09-10T07:46:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.566+0000 7fc75d600640 1 -- 172.21.15.46:0/3452768062 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fc7640bbf80 con 0x562b561cb3b0 2024-09-10T07:46:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.567+0000 7fc77d67e740 10 client.27898 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:46:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.567+0000 7fc77d67e740 10 client.27898 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:46:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.567+0000 7fc77d67e740 20 client.27898 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.567+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b561f6d40 0x562b561f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.567+0000 7fc77d67e740 4 client.27898 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.567+0000 7fc77d67e740 1 -- 172.21.15.46:0/3452768062 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b560d4310 con 0x562b561f6d40 2024-09-10T07:46:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.568+0000 7fc773400640 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b561f6d40 0x562b561f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.569+0000 7fc773400640 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b561f6d40 0x562b561f91c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=125f66cc212e6f77 server_cookie=13eac443e44bddbd in_seq=0 out_seq=0 2024-09-10T07:46:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.569+0000 7fc75d600640 10 client.27898 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.570+0000 7fc75d600640 1 -- 172.21.15.46:0/3452768062 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b560d4310 con 0x562b561f6d40 2024-09-10T07:46:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.570+0000 7fc75d600640 10 client.27898 handle_command_reply: tid=0 2024-09-10T07:46:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.570+0000 7fc77d67e740 1 client.27898 shutdown 2024-09-10T07:46:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 -- 172.21.15.46:0/3452768062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cb3b0 msgr2=0x562b561d33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cb3b0 0x562b561d33b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc7640f7670 tx=0x7fc7640092d0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 -- 172.21.15.46:0/3452768062 shutdown_connections 2024-09-10T07:46:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b561f6d40 0x562b561f91c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561d6590 0x562b561d3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561d38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3452768062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cb3b0 0x562b561d33b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 -- 172.21.15.46:0/3452768062 >> 172.21.15.46:0/3452768062 conn(0x562b561c7940 msgr2=0x562b561c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc75d600640 0 client.27898 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 -- 172.21.15.46:0/3452768062 shutdown_connections 2024-09-10T07:46:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.571+0000 7fc77d67e740 1 -- 172.21.15.46:0/3452768062 wait complete. 2024-09-10T07:46:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.572+0000 7fc77d67e740 20 client.27898 trim_cache size 0 max 16384 2024-09-10T07:46:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.599+0000 7fc77d67e740 1 Processor -- start 2024-09-10T07:46:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.599+0000 7fc77d67e740 1 -- start start 2024-09-10T07:46:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cb3b0 0x562b561c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cbcd0 0x562b561fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561d6590 0x562b561fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc77d67e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc75400fb50 con 0x562b561d6590 2024-09-10T07:46:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc77d67e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc754011690 con 0x562b561cb3b0 2024-09-10T07:46:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc77d67e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc754010220 con 0x562b561cbcd0 2024-09-10T07:46:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc773e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cb3b0 0x562b561c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc773e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cb3b0 0x562b561c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53202/0 (socket says 172.21.15.46:53202) 2024-09-10T07:46:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc773e00640 1 -- 172.21.15.46:0/3619400075 learned_addr learned my addr 172.21.15.46:0/3619400075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.600+0000 7fc773400640 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cbcd0 0x562b561fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.601+0000 7fc778a00640 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561d6590 0x562b561fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.601+0000 7fc773e00640 1 -- 172.21.15.46:0/3619400075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cbcd0 msgr2=0x562b561fea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.601+0000 7fc773e00640 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cbcd0 0x562b561fea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.601+0000 7fc773e00640 1 -- 172.21.15.46:0/3619400075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561d6590 msgr2=0x562b561fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.601+0000 7fc773e00640 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561d6590 0x562b561fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.601+0000 7fc773e00640 1 -- 172.21.15.46:0/3619400075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b561cbb00 con 0x562b561cb3b0 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.602+0000 7fc773400640 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cbcd0 0x562b561fea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.602+0000 7fc773e00640 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cb3b0 0x562b561c6a90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc7640bbcb0 tx=0x7fc7640f5530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b47a300eb1ff51b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.602+0000 7fc75fe00640 1 -- 172.21.15.46:0/3619400075 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640fe070 con 0x562b561cb3b0 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.602+0000 7fc75fe00640 1 -- 172.21.15.46:0/3619400075 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7640056a0 con 0x562b561cb3b0 2024-09-10T07:46:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.602+0000 7fc75fe00640 1 -- 172.21.15.46:0/3619400075 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640c4990 con 0x562b561cb3b0 2024-09-10T07:46:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.603+0000 7fc77d67e740 1 -- 172.21.15.46:0/3619400075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cb3b0 msgr2=0x562b561c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.603+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cb3b0 0x562b561c6a90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc7640bbcb0 tx=0x7fc7640f5530 comp rx=0 tx=0).stop 2024-09-10T07:46:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.604+0000 7fc77d67e740 1 -- 172.21.15.46:0/3619400075 shutdown_connections 2024-09-10T07:46:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.604+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561d6590 0x562b561fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.604+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561cbcd0 0x562b561fea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.604+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3619400075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561cb3b0 0x562b561c6a90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.604+0000 7fc77d67e740 1 -- 172.21.15.46:0/3619400075 >> 172.21.15.46:0/3619400075 conn(0x562b561c7940 msgr2=0x562b561d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.604+0000 7fc77d67e740 1 -- 172.21.15.46:0/3619400075 shutdown_connections 2024-09-10T07:46:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.604+0000 7fc77d67e740 1 -- 172.21.15.46:0/3619400075 wait complete. 2024-09-10T07:46:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.605+0000 7fc77d67e740 1 -- start start 2024-09-10T07:46:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.605+0000 7fc77d67e740 1 Processor -- start 2024-09-10T07:46:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561ef300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc77d67e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561efd80 0x562b561ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc77d67e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc754010fc0 con 0x562b561cbcd0 2024-09-10T07:46:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc77d67e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7540108f0 con 0x562b561efd80 2024-09-10T07:46:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc77d67e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc75400db30 con 0x562b561d6590 2024-09-10T07:46:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc77d67e740 10 client.0 _ll_register_callbacks cb 0x7fc7640f5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc778a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561efd80 0x562b561ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc778a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561efd80 0x562b561ec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53214/0 (socket says 172.21.15.46:53214) 2024-09-10T07:46:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc778a00640 1 -- 172.21.15.46:0/3818783962 learned_addr learned my addr 172.21.15.46:0/3818783962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:46:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.606+0000 7fc773e00640 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc773400640 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc778a00640 1 -- 172.21.15.46:0/3818783962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 msgr2=0x562b561ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc778a00640 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561ef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc778a00640 1 -- 172.21.15.46:0/3818783962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 msgr2=0x562b561ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc778a00640 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561ef300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc778a00640 1 -- 172.21.15.46:0/3818783962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b561ecb80 con 0x562b561efd80 2024-09-10T07:46:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc773400640 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc778a00640 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561efd80 0x562b561ec350 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc76c011950 tx=0x7fc76c015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3428d06bf69cc08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.607+0000 7fc75d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:46:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.608+0000 7fc75d600640 1 -- 172.21.15.46:0/3818783962 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c004d20 con 0x562b561efd80 2024-09-10T07:46:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.608+0000 7fc75d600640 1 -- 172.21.15.46:0/3818783962 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc76c01e6e0 con 0x562b561efd80 2024-09-10T07:46:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.608+0000 7fc77d67e740 1 -- 172.21.15.46:0/3818783962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7640056a0 con 0x562b561efd80 2024-09-10T07:46:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.608+0000 7fc75d600640 1 -- 172.21.15.46:0/3818783962 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc76c035050 con 0x562b561efd80 2024-09-10T07:46:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.608+0000 7fc77d67e740 10 client.27956 fetch_fsmap learned FSMap version 50 2024-09-10T07:46:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.608+0000 7fc77d67e740 1 -- 172.21.15.46:0/3818783962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562b56262650 con 0x562b561efd80 2024-09-10T07:46:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.608+0000 7fc75d600640 1 -- 172.21.15.46:0/3818783962 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7640056a0 con 0x562b561efd80 2024-09-10T07:46:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.609+0000 7fc75d600640 1 -- 172.21.15.46:0/3818783962 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fc76c00f040 con 0x562b561efd80 2024-09-10T07:46:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.609+0000 7fc77d67e740 10 client.27956 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:46:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.609+0000 7fc77d67e740 10 client.27956 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:46:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.609+0000 7fc77d67e740 20 client.27956 populate_metadata read hostname 'smithi046' 2024-09-10T07:46:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.609+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b56262e70 0x562b56265290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.609+0000 7fc77d67e740 4 client.27956 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-10T07:46:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.609+0000 7fc77d67e740 1 -- 172.21.15.46:0/3818783962 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x562b5613ec60 con 0x562b56262e70 2024-09-10T07:46:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.609+0000 7fc773e00640 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b56262e70 0x562b56265290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.611+0000 7fc773e00640 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b56262e70 0x562b56265290 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5145f1ddc8fdde23 server_cookie=29e28bcc8906bce0 in_seq=0 out_seq=0 2024-09-10T07:46:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.611+0000 7fc75d600640 10 client.27956 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:58.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc75d600640 1 -- 172.21.15.46:0/3818783962 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562b5613ec60 con 0x562b56262e70 2024-09-10T07:46:58.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc75d600640 10 client.27956 handle_command_reply: tid=0 2024-09-10T07:46:58.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 client.27956 shutdown 2024-09-10T07:46:58.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 -- 172.21.15.46:0/3818783962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561efd80 msgr2=0x562b561ec350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561efd80 0x562b561ec350 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc76c011950 tx=0x7fc76c015040 comp rx=0 tx=0).stop 2024-09-10T07:46:58.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 -- 172.21.15.46:0/3818783962 shutdown_connections 2024-09-10T07:46:58.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562b56262e70 0x562b56265290 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b561efd80 0x562b561ec350 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562b561d6590 0x562b561ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 --2- 172.21.15.46:0/3818783962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b561cbcd0 0x562b561ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc75d600640 0 client.27956 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:46:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 -- 172.21.15.46:0/3818783962 >> 172.21.15.46:0/3818783962 conn(0x562b561c7940 msgr2=0x562b561cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 -- 172.21.15.46:0/3818783962 shutdown_connections 2024-09-10T07:46:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 1 -- 172.21.15.46:0/3818783962 wait complete. 2024-09-10T07:46:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.912+0000 7fc77d67e740 20 client.27956 trim_cache size 0 max 16384 2024-09-10T07:46:58.913 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 50, 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 34, 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "f491a352": { 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 34, 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 130.0, 2024-09-10T07:46:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:46:58.915 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 -- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc7580a2c00 msgr2=0x7fc7580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc7580a2c00 0x7fc7580a50c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009270 tx=0x7fc76c009680 comp rx=0 tx=0).stop 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 -- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7741008c0 msgr2=0x7fc7741a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7741008c0 0x7fc7741a4770 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc76402a940 tx=0x7fc764003b90 comp rx=0 tx=0).stop 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 -- 172.21.15.46:0/632909451 shutdown_connections 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc774105ec0 0x7fc77419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7741011e0 0x7fc7741a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc7580a2c00 0x7fc7580a50c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 --2- 172.21.15.46:0/632909451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7741008c0 0x7fc7741a4770 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:58.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 -- 172.21.15.46:0/632909451 >> 172.21.15.46:0/632909451 conn(0x7fc7740fc7d0 msgr2=0x7fc774103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:58.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.913+0000 7fc77aa00640 1 -- 172.21.15.46:0/632909451 shutdown_connections 2024-09-10T07:46:58.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:46:58.914+0000 7fc77aa00640 1 -- 172.21.15.46:0/632909451 wait complete. 2024-09-10T07:46:59.816 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f491a352', quiesce took 5.9 seconds. Will release after: 4.1 2024-09-10T07:47:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:00 smithi181 ceph-mon[35397]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T07:47:00.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:00 smithi063 ceph-mon[35020]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T07:47:00.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:00 smithi046 ceph-mon[30705]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T07:47:00.601 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 580.0 s, 365.5 tps, lat 166.608 ms stddev 199.101 2024-09-10T07:47:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:02 smithi181 ceph-mon[35397]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-10T07:47:02.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:02 smithi063 ceph-mon[35020]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-10T07:47:02.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:02 smithi046 ceph-mon[30705]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-10T07:47:03.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:47:04.309 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:04 smithi181 ceph-mon[35397]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 517 op/s 2024-09-10T07:47:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:04 smithi063 ceph-mon[35020]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 517 op/s 2024-09-10T07:47:04.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:04 smithi046 ceph-mon[30705]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 517 op/s 2024-09-10T07:47:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 -- 172.21.15.46:0/1006779574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4112a30 msgr2=0x7f33e4114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 --2- 172.21.15.46:0/1006779574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4112a30 0x7f33e4114e20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009ca0 tx=0x7f33dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 -- 172.21.15.46:0/1006779574 shutdown_connections 2024-09-10T07:47:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 --2- 172.21.15.46:0/1006779574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4112a30 0x7f33e4114e20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 --2- 172.21.15.46:0/1006779574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4077040 0x7f33e41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 --2- 172.21.15.46:0/1006779574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4076720 0x7f33e4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 -- 172.21.15.46:0/1006779574 >> 172.21.15.46:0/1006779574 conn(0x7f33e40ffa90 msgr2=0x7f33e4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 -- 172.21.15.46:0/1006779574 shutdown_connections 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.876+0000 7f33ea400640 1 -- 172.21.15.46:0/1006779574 wait complete. 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.877+0000 7f33ea400640 1 Processor -- start 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.877+0000 7f33ea400640 1 -- start start 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33ea400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4076720 0x7f33e41a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33ea400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4077040 0x7f33e41a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33ea400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4112a30 0x7f33e419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33ea400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41185e0 con 0x7f33e4077040 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33ea400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4118460 con 0x7f33e4112a30 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33ea400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4118760 con 0x7f33e4076720 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4077040 0x7f33e41a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4077040 0x7f33e41a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48564/0 (socket says 172.21.15.46:48564) 2024-09-10T07:47:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e2a00640 1 -- 172.21.15.46:0/552861333 learned_addr learned my addr 172.21.15.46:0/552861333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e3400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4076720 0x7f33e41a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e3e00640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4112a30 0x7f33e419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e2a00640 1 -- 172.21.15.46:0/552861333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4076720 msgr2=0x7f33e41a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e2a00640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4076720 0x7f33e41a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e2a00640 1 -- 172.21.15.46:0/552861333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4112a30 msgr2=0x7f33e419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e2a00640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4112a30 0x7f33e419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.878+0000 7f33e2a00640 1 -- 172.21.15.46:0/552861333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e419f040 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.879+0000 7f33e2a00640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4077040 0x7f33e41a4c90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f33d000d770 tx=0x7f33d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.879+0000 7f33dbe00640 1 -- 172.21.15.46:0/552861333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0017070 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.879+0000 7f33dbe00640 1 -- 172.21.15.46:0/552861333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33d0005500 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.879+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e419f330 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.879+0000 7f33dbe00640 1 -- 172.21.15.46:0/552861333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0004e00 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.879+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41ab620 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.880+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e4076b00 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.883+0000 7f33dbe00640 1 -- 172.21.15.46:0/552861333 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f33d0004fa0 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.885+0000 7f33dbe00640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33c80a2c00 0x7f33c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.885+0000 7f33dbe00640 1 -- 172.21.15.46:0/552861333 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f33d00ced10 con 0x7f33e4077040 2024-09-10T07:47:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.885+0000 7f33e3400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33c80a2c00 0x7f33c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.886+0000 7f33e3400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33c80a2c00 0x7f33c80a50c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f33e419fa50 tx=0x7f33d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:04.890+0000 7f33dbe00640 1 -- 172.21.15.46:0/552861333 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33d00d4020 con 0x7f33e4077040 2024-09-10T07:47:05.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.113+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33e410dbc0 con 0x7f33e4077040 2024-09-10T07:47:05.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.114+0000 7f33dbe00640 1 -- 172.21.15.46:0/552861333 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7f33d009b240 con 0x7f33e4077040 2024-09-10T07:47:05.115 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:47:05.115 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"btime":"2024-09-10T07:46:53:072255+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:46:52.230107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:47:05.115 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-09-10T07:47:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33c80a2c00 msgr2=0x7f33c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33c80a2c00 0x7f33c80a50c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f33e419fa50 tx=0x7f33d400b410 comp rx=0 tx=0).stop 2024-09-10T07:47:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4077040 msgr2=0x7f33e41a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4077040 0x7f33e41a4c90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f33d000d770 tx=0x7f33d000dc30 comp rx=0 tx=0).stop 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 shutdown_connections 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33e4112a30 0x7f33e419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33c80a2c00 0x7f33c80a50c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4077040 0x7f33e41a4c90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 --2- 172.21.15.46:0/552861333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e4076720 0x7f33e41a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 >> 172.21.15.46:0/552861333 conn(0x7f33e40ffa90 msgr2=0x7f33e4113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 shutdown_connections 2024-09-10T07:47:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:05.117+0000 7f33ea400640 1 -- 172.21.15.46:0/552861333 wait complete. 2024-09-10T07:47:05.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:05 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/552861333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:05.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:05 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/552861333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:05.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:05 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/552861333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 585.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:47:05.876 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id f491a352 --release --await' 2024-09-10T07:47:05.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id f491a352 --release --await 2024-09-10T07:47:06.253 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:06.393 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:06 smithi046 ceph-mon[30705]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-09-10T07:47:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:06 smithi181 ceph-mon[35397]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-09-10T07:47:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:06 smithi063 ceph-mon[35020]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-09-10T07:47:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 -- 172.21.15.46:0/3882454863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c103050 msgr2=0x7f379c103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 --2- 172.21.15.46:0/3882454863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c103050 0x7f379c103450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f378c009900 tx=0x7f378c02a410 comp rx=0 tx=0).stop 2024-09-10T07:47:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 -- 172.21.15.46:0/3882454863 shutdown_connections 2024-09-10T07:47:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 --2- 172.21.15.46:0/3882454863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f379c101f40 0x7f379c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 --2- 172.21.15.46:0/3882454863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c101580 0x7f379c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 --2- 172.21.15.46:0/3882454863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c103050 0x7f379c103450 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 -- 172.21.15.46:0/3882454863 >> 172.21.15.46:0/3882454863 conn(0x7f379c0fd0d0 msgr2=0x7f379c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 -- 172.21.15.46:0/3882454863 shutdown_connections 2024-09-10T07:47:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.811+0000 7f37a0e00640 1 -- 172.21.15.46:0/3882454863 wait complete. 2024-09-10T07:47:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.812+0000 7f37a0e00640 1 Processor -- start 2024-09-10T07:47:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.812+0000 7f37a0e00640 1 -- start start 2024-09-10T07:47:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f37a0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c101580 0x7f379c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f37a0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c101f40 0x7f379c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f37a0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f379c103050 0x7f379c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f37a0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f379c10d720 con 0x7f379c101f40 2024-09-10T07:47:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f37a0e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f379c10d5a0 con 0x7f379c103050 2024-09-10T07:47:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f37a0e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f379c10d8a0 con 0x7f379c101580 2024-09-10T07:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c101f40 0x7f379c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c101f40 0x7f379c19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48358/0 (socket says 172.21.15.46:48358) 2024-09-10T07:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379aa00640 1 -- 172.21.15.46:0/806490346 learned_addr learned my addr 172.21.15.46:0/806490346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379b400640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c101580 0x7f379c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379be00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f379c103050 0x7f379c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379aa00640 1 -- 172.21.15.46:0/806490346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c101580 msgr2=0x7f379c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379aa00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c101580 0x7f379c19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379aa00640 1 -- 172.21.15.46:0/806490346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f379c103050 msgr2=0x7f379c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379aa00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f379c103050 0x7f379c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379aa00640 1 -- 172.21.15.46:0/806490346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c1a1c30 con 0x7f379c101f40 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379be00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f379c103050 0x7f379c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.813+0000 7f379b400640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c101580 0x7f379c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.814+0000 7f379aa00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c101f40 0x7f379c19a550 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f378800d950 tx=0x7f378800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.814+0000 7f3793e00640 1 -- 172.21.15.46:0/806490346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788017070 con 0x7f379c101f40 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.814+0000 7f3793e00640 1 -- 172.21.15.46:0/806490346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3788009c30 con 0x7f379c101f40 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.814+0000 7f37a0e00640 1 -- 172.21.15.46:0/806490346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c1a1f20 con 0x7f379c101f40 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.814+0000 7f3793e00640 1 -- 172.21.15.46:0/806490346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378800f6b0 con 0x7f379c101f40 2024-09-10T07:47:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.814+0000 7f37a0e00640 1 -- 172.21.15.46:0/806490346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c1a23e0 con 0x7f379c101f40 2024-09-10T07:47:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.818+0000 7f37a5456740 1 Processor -- start 2024-09-10T07:47:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.818+0000 7f37a5456740 1 -- start start 2024-09-10T07:47:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.818+0000 7f37a5456740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c9251660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.818+0000 7f37a5456740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c9256660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f37a5456740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 0x55c0c92590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f37a5456740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c0c91b6100 con 0x55c0c9251ba0 2024-09-10T07:47:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f37a5456740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c0c915e1b0 con 0x55c0c9251260 2024-09-10T07:47:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f37a5456740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0c91b20c0 con 0x55c0c9256c30 2024-09-10T07:47:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c9256660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f3793e00640 1 -- 172.21.15.46:0/806490346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37880109d0 con 0x7f379c101f40 2024-09-10T07:47:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c9251660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c9256660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48362/0 (socket says 172.21.15.46:48362) 2024-09-10T07:47:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379b400640 1 -- 172.21.15.46:0/1974851573 learned_addr learned my addr 172.21.15.46:0/1974851573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379be00640 1 -- 172.21.15.46:0/1974851573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 msgr2=0x55c0c92590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379be00640 1 --2- 172.21.15.46:0/1974851573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 0x55c0c92590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379be00640 1 -- 172.21.15.46:0/1974851573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 msgr2=0x55c0c9256660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379be00640 1 --2- 172.21.15.46:0/1974851573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c9256660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379be00640 1 -- 172.21.15.46:0/1974851573 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c8ec9210 con 0x55c0c9251260 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.819+0000 7f379b400640 1 --2- 172.21.15.46:0/1974851573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c9256660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.820+0000 7f379be00640 1 --2- 172.21.15.46:0/1974851573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c9251660 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3794009f90 tx=0x7f3794010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52f0de9abb8b057b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.820+0000 7f3790c00640 1 -- 172.21.15.46:0/1974851573 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794004d20 con 0x55c0c9251260 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.820+0000 7f3790c00640 1 -- 172.21.15.46:0/1974851573 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37940119e0 con 0x55c0c9251260 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.820+0000 7f3790c00640 1 -- 172.21.15.46:0/1974851573 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794005650 con 0x55c0c9251260 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f3793e00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37800a2b80 0x7f37800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f3793e00640 1 -- 172.21.15.46:0/806490346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f37880cf000 con 0x7f379c101f40 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f37a5456740 1 -- 172.21.15.46:0/1974851573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 msgr2=0x55c0c9251660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f37a5456740 1 --2- 172.21.15.46:0/1974851573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c9251660 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3794009f90 tx=0x7f3794010860 comp rx=0 tx=0).stop 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f379b400640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37800a2b80 0x7f37800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f37a5456740 1 -- 172.21.15.46:0/1974851573 shutdown_connections 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f37a5456740 1 --2- 172.21.15.46:0/1974851573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 0x55c0c92590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f37a5456740 1 --2- 172.21.15.46:0/1974851573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c9256660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f37a5456740 1 --2- 172.21.15.46:0/1974851573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c9251660 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f37a5456740 1 -- 172.21.15.46:0/1974851573 >> 172.21.15.46:0/1974851573 conn(0x55c0c924c8d0 msgr2=0x55c0c924ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.821+0000 7f37a5456740 1 -- 172.21.15.46:0/1974851573 shutdown_connections 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.822+0000 7f37a5456740 1 -- 172.21.15.46:0/1974851573 wait complete. 2024-09-10T07:47:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.822+0000 7f379b400640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37800a2b80 0x7f37800a5040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f378c000c00 tx=0x7f378c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.822+0000 7f37a5456740 1 -- start start 2024-09-10T07:47:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.822+0000 7f37a5456740 1 Processor -- start 2024-09-10T07:47:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f37a5456740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251260 0x55c0c925b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f37a5456740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251ba0 0x55c0c925b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f37a5456740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9256c30 0x55c0c9262a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f37a5456740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9259c00 con 0x55c0c9256c30 2024-09-10T07:47:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f37a5456740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c0c913f090 con 0x55c0c9251ba0 2024-09-10T07:47:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f37a5456740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9259d80 con 0x55c0c9251260 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f37a5456740 10 client.0 _ll_register_callbacks cb 0x55c0c9193f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251260 0x55c0c925b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9256c30 0x55c0c9262a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251260 0x55c0c925b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49252/0 (socket says 172.21.15.46:49252) 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379b400640 1 -- 172.21.15.46:0/1273076706 learned_addr learned my addr 172.21.15.46:0/1273076706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379aa00640 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251ba0 0x55c0c925b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379b400640 1 -- 172.21.15.46:0/1273076706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251ba0 msgr2=0x55c0c925b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379b400640 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251ba0 0x55c0c925b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379b400640 1 -- 172.21.15.46:0/1273076706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9256c30 msgr2=0x55c0c9262a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379b400640 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9256c30 0x55c0c9262a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379b400640 1 -- 172.21.15.46:0/1273076706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c8f95620 con 0x55c0c9251260 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.823+0000 7f379be00640 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9256c30 0x55c0c9262a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.824+0000 7f379b400640 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251260 0x55c0c925b4a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f378c02aa00 tx=0x7f378c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb72ac2dbd4e8ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.824+0000 7f3785600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.824+0000 7f3785600640 1 -- 172.21.15.46:0/1273076706 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c035040 con 0x55c0c9251260 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.824+0000 7f37a5456740 1 -- 172.21.15.46:0/1273076706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37940119e0 con 0x55c0c9251260 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.824+0000 7f3785600640 1 -- 172.21.15.46:0/1273076706 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f378c004c20 con 0x55c0c9251260 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f3785600640 1 -- 172.21.15.46:0/1273076706 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f378c05a050 con 0x55c0c9251260 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f37a5456740 10 client.27916 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f37a5456740 1 -- 172.21.15.46:0/1273076706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55c0c92658d0 con 0x55c0c9251260 2024-09-10T07:47:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f3785600640 1 -- 172.21.15.46:0/1273076706 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37940119e0 con 0x55c0c9251260 2024-09-10T07:47:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f3785600640 1 -- 172.21.15.46:0/1273076706 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f378c044030 con 0x55c0c9251260 2024-09-10T07:47:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f37a5456740 10 client.27916 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f37a5456740 10 client.27916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f37a5456740 20 client.27916 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f37a5456740 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0c9266f50 0x55c0c9269360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f37a5456740 4 client.27916 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.825+0000 7f37a5456740 1 -- 172.21.15.46:0/1273076706 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0c91c3830 con 0x55c0c9266f50 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.826+0000 7f379aa00640 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0c9266f50 0x55c0c9269360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.826+0000 7f3785600640 10 client.27916 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.826+0000 7f379aa00640 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0c9266f50 0x55c0c9269360 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43620eeea7fd669 server_cookie=ceade282b92fa5b6 in_seq=0 out_seq=0 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f3785600640 1 -- 172.21.15.46:0/1273076706 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0c91c3830 con 0x55c0c9266f50 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f3785600640 10 client.27916 handle_command_reply: tid=0 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 client.27916 shutdown 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 -- 172.21.15.46:0/1273076706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251260 msgr2=0x55c0c925b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251260 0x55c0c925b4a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f378c02aa00 tx=0x7f378c009260 comp rx=0 tx=0).stop 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 -- 172.21.15.46:0/1273076706 shutdown_connections 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0c9266f50 0x55c0c9269360 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9256c30 0x55c0c9262a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f3785600640 0 client.27916 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251ba0 0x55c0c925b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 --2- 172.21.15.46:0/1273076706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251260 0x55c0c925b4a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 -- 172.21.15.46:0/1273076706 >> 172.21.15.46:0/1273076706 conn(0x55c0c924c8d0 msgr2=0x55c0c924e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 -- 172.21.15.46:0/1273076706 shutdown_connections 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.827+0000 7f37a5456740 1 -- 172.21.15.46:0/1273076706 wait complete. 2024-09-10T07:47:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.828+0000 7f37a5456740 20 client.27916 trim_cache size 0 max 16384 2024-09-10T07:47:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.843+0000 7f37a5456740 1 Processor -- start 2024-09-10T07:47:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.843+0000 7f37a5456740 1 -- start start 2024-09-10T07:47:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.843+0000 7f37a5456740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c924dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.843+0000 7f37a5456740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c92629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.843+0000 7f37a5456740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 0x55c0c924e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.843+0000 7f37a5456740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1ac7e0 con 0x55c0c9251ba0 2024-09-10T07:47:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.843+0000 7f37a5456740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1ae320 con 0x55c0c9251260 2024-09-10T07:47:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.843+0000 7f37a5456740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1aceb0 con 0x55c0c9256c30 2024-09-10T07:47:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c92629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c92629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48374/0 (socket says 172.21.15.46:48374) 2024-09-10T07:47:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 -- 172.21.15.46:0/225294551 learned_addr learned my addr 172.21.15.46:0/225294551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379b400640 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c924dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379be00640 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 0x55c0c924e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 -- 172.21.15.46:0/225294551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 msgr2=0x55c0c924e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 0x55c0c924e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 -- 172.21.15.46:0/225294551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 msgr2=0x55c0c924dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c924dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 -- 172.21.15.46:0/225294551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c924c6f0 con 0x55c0c9251ba0 2024-09-10T07:47:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.844+0000 7f379aa00640 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c92629e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3788009610 tx=0x7f378800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdb51a3f22ee16f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.845+0000 7f3787e00640 1 -- 172.21.15.46:0/225294551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788020040 con 0x55c0c9251ba0 2024-09-10T07:47:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.845+0000 7f3787e00640 1 -- 172.21.15.46:0/225294551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f378809f890 con 0x55c0c9251ba0 2024-09-10T07:47:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.845+0000 7f3787e00640 1 -- 172.21.15.46:0/225294551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37880a3cc0 con 0x55c0c9251ba0 2024-09-10T07:47:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.845+0000 7f37a5456740 1 -- 172.21.15.46:0/225294551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 msgr2=0x55c0c92629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.845+0000 7f37a5456740 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c92629e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3788009610 tx=0x7f378800bf60 comp rx=0 tx=0).stop 2024-09-10T07:47:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 -- 172.21.15.46:0/225294551 shutdown_connections 2024-09-10T07:47:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9256c30 0x55c0c924e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251ba0 0x55c0c92629e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 --2- 172.21.15.46:0/225294551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9251260 0x55c0c924dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 -- 172.21.15.46:0/225294551 >> 172.21.15.46:0/225294551 conn(0x55c0c924c8d0 msgr2=0x55c0c924ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 -- 172.21.15.46:0/225294551 shutdown_connections 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 -- 172.21.15.46:0/225294551 wait complete. 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 -- start start 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.846+0000 7f37a5456740 1 Processor -- start 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f37a5456740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251260 0x55c0c9260150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f37a5456740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251ba0 0x55c0c9260690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f37a5456740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9256c30 0x55c0c9261630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f37a5456740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1adc50 con 0x55c0c9251260 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f37a5456740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1ad580 con 0x55c0c9256c30 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f37a5456740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1aa7c0 con 0x55c0c9251ba0 2024-09-10T07:47:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f37a5456740 10 client.0 _ll_register_callbacks cb 0x7f378c005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251260 0x55c0c9260150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251260 0x55c0c9260150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48390/0 (socket says 172.21.15.46:48390) 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 -- 172.21.15.46:0/2604065934 learned_addr learned my addr 172.21.15.46:0/2604065934 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379aa00640 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251ba0 0x55c0c9260690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379be00640 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9256c30 0x55c0c9261630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 -- 172.21.15.46:0/2604065934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251ba0 msgr2=0x55c0c9260690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251ba0 0x55c0c9260690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 -- 172.21.15.46:0/2604065934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9256c30 msgr2=0x55c0c9261630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9256c30 0x55c0c9261630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 -- 172.21.15.46:0/2604065934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c9261c40 con 0x55c0c9251260 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f379b400640 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251260 0x55c0c9260150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f378c042040 tx=0x7f378c005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57064e456219363 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.847+0000 7f3785600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.848+0000 7f3785600640 1 -- 172.21.15.46:0/2604065934 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c063060 con 0x55c0c9251260 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.848+0000 7f37a5456740 1 -- 172.21.15.46:0/2604065934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f378809f890 con 0x55c0c9251260 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.848+0000 7f3785600640 1 -- 172.21.15.46:0/2604065934 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f378c004c20 con 0x55c0c9251260 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.848+0000 7f3785600640 1 -- 172.21.15.46:0/2604065934 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f378c06a050 con 0x55c0c9251260 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.848+0000 7f37a5456740 10 client.17322 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.848+0000 7f37a5456740 1 -- 172.21.15.46:0/2604065934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55c0c92d1d10 con 0x55c0c9251260 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.848+0000 7f3785600640 1 -- 172.21.15.46:0/2604065934 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f378809f890 con 0x55c0c9251260 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.849+0000 7f3785600640 1 -- 172.21.15.46:0/2604065934 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f378c044040 con 0x55c0c9251260 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.849+0000 7f37a5456740 10 client.17322 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.849+0000 7f37a5456740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.849+0000 7f37a5456740 20 client.17322 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.849+0000 7f37a5456740 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0c92d2660 0x55c0c92d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.849+0000 7f37a5456740 4 client.17322 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:47:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.849+0000 7f37a5456740 1 -- 172.21.15.46:0/2604065934 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0c91bdb80 con 0x55c0c92d2660 2024-09-10T07:47:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.849+0000 7f379aa00640 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0c92d2660 0x55c0c92d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.850+0000 7f379aa00640 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0c92d2660 0x55c0c92d4a80 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26eecdebf467f574 server_cookie=e09f8c6d12fe170d in_seq=0 out_seq=0 2024-09-10T07:47:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.850+0000 7f3785600640 10 client.17322 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f3785600640 1 -- 172.21.15.46:0/2604065934 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0c91bdb80 con 0x55c0c92d2660 2024-09-10T07:47:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f3785600640 10 client.17322 handle_command_reply: tid=0 2024-09-10T07:47:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 client.17322 shutdown 2024-09-10T07:47:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 -- 172.21.15.46:0/2604065934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251260 msgr2=0x55c0c9260150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251260 0x55c0c9260150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f378c042040 tx=0x7f378c005990 comp rx=0 tx=0).stop 2024-09-10T07:47:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 -- 172.21.15.46:0/2604065934 shutdown_connections 2024-09-10T07:47:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0c92d2660 0x55c0c92d4a80 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0c9256c30 0x55c0c9261630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0c9251ba0 0x55c0c9260690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 --2- 172.21.15.46:0/2604065934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c9251260 0x55c0c9260150 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f3785600640 0 client.17322 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 -- 172.21.15.46:0/2604065934 >> 172.21.15.46:0/2604065934 conn(0x55c0c924c8d0 msgr2=0x55c0c924ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 -- 172.21.15.46:0/2604065934 shutdown_connections 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.855+0000 7f37a5456740 1 -- 172.21.15.46:0/2604065934 wait complete. 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.856+0000 7f37a5456740 20 client.17322 trim_cache size 0 max 16384 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:47:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:47:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:47:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.856+0000 7f37a0e00640 1 -- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37800a2b80 msgr2=0x7f37800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.856+0000 7f37a0e00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37800a2b80 0x7f37800a5040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f378c000c00 tx=0x7f378c0315b0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.856+0000 7f37a0e00640 1 -- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c101f40 msgr2=0x7f379c19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.856+0000 7f37a0e00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c101f40 0x7f379c19a550 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f378800d950 tx=0x7f378800de10 comp rx=0 tx=0).stop 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.856+0000 7f37a0e00640 1 -- 172.21.15.46:0/806490346 shutdown_connections 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.857+0000 7f37a0e00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f379c103050 0x7f379c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.857+0000 7f37a0e00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37800a2b80 0x7f37800a5040 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.857+0000 7f37a0e00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f379c101f40 0x7f379c19a550 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.857+0000 7f37a0e00640 1 --2- 172.21.15.46:0/806490346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c101580 0x7f379c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.857+0000 7f37a0e00640 1 -- 172.21.15.46:0/806490346 >> 172.21.15.46:0/806490346 conn(0x7f379c0fd0d0 msgr2=0x7f379c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.857+0000 7f37a0e00640 1 -- 172.21.15.46:0/806490346 shutdown_connections 2024-09-10T07:47:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:06.857+0000 7f37a0e00640 1 -- 172.21.15.46:0/806490346 wait complete. 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 -- 172.21.15.46:0/209980959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 msgr2=0x7f83001043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 --2- 172.21.15.46:0/209980959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 0x7f83001043f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009900 tx=0x7f82f402a410 comp rx=0 tx=0).stop 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 -- 172.21.15.46:0/209980959 shutdown_connections 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 --2- 172.21.15.46:0/209980959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8300109140 0x7f8300110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 --2- 172.21.15.46:0/209980959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83001049c0 0x7f8300108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 --2- 172.21.15.46:0/209980959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 0x7f83001043f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 -- 172.21.15.46:0/209980959 >> 172.21.15.46:0/209980959 conn(0x7f83000ffa50 msgr2=0x7f8300101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 -- 172.21.15.46:0/209980959 shutdown_connections 2024-09-10T07:47:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.866+0000 7f8308400640 1 -- 172.21.15.46:0/209980959 wait complete. 2024-09-10T07:47:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.868+0000 7f8308400640 1 Processor -- start 2024-09-10T07:47:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.868+0000 7f8308400640 1 -- start start 2024-09-10T07:47:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.868+0000 7f8308400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 0x7f830019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.868+0000 7f8308400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83001049c0 0x7f830019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.868+0000 7f8308400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8300109140 0x7f830019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.868+0000 7f8308400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8300114170 con 0x7f8300104010 2024-09-10T07:47:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.868+0000 7f8308400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8300113ff0 con 0x7f8300109140 2024-09-10T07:47:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.868+0000 7f8308400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83001142f0 con 0x7f83001049c0 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 0x7f830019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 0x7f830019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48402/0 (socket says 172.21.15.46:48402) 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305a00640 1 -- 172.21.15.46:0/4245245856 learned_addr learned my addr 172.21.15.46:0/4245245856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8306400640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8300109140 0x7f830019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305000640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83001049c0 0x7f830019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305000640 1 -- 172.21.15.46:0/4245245856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8300109140 msgr2=0x7f830019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305000640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8300109140 0x7f830019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305000640 1 -- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 msgr2=0x7f830019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305000640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 0x7f830019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305000640 1 -- 172.21.15.46:0/4245245856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830019f600 con 0x7f83001049c0 2024-09-10T07:47:06.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.869+0000 7f8305a00640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 0x7f830019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:06.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.870+0000 7f8305000640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83001049c0 0x7f830019ab90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00e9f0 tx=0x7f82ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.870+0000 7f82fe000640 1 -- 172.21.15.46:0/4245245856 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00ccc0 con 0x7f83001049c0 2024-09-10T07:47:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.870+0000 7f82fe000640 1 -- 172.21.15.46:0/4245245856 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec010910 con 0x7f83001049c0 2024-09-10T07:47:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.870+0000 7f82fe000640 1 -- 172.21.15.46:0/4245245856 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec018960 con 0x7f83001049c0 2024-09-10T07:47:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.870+0000 7f8308400640 1 -- 172.21.15.46:0/4245245856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830019f8f0 con 0x7f83001049c0 2024-09-10T07:47:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.870+0000 7f8308400640 1 -- 172.21.15.46:0/4245245856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83001a71d0 con 0x7f83001049c0 2024-09-10T07:47:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.873+0000 7f830ae21740 1 Processor -- start 2024-09-10T07:47:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.873+0000 7f82fe000640 1 -- 172.21.15.46:0/4245245856 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f82ec010ac0 con 0x7f83001049c0 2024-09-10T07:47:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f830ae21740 1 -- start start 2024-09-10T07:47:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f830ae21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9310 0x556c677e96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f830ae21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9c30 0x556c677f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f830ae21740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f830ae21740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556c6774f650 con 0x556c677f44f0 2024-09-10T07:47:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f830ae21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556c676f6f10 con 0x556c677e9310 2024-09-10T07:47:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f830ae21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556c6774b610 con 0x556c677e9c30 2024-09-10T07:47:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f8306400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9310 0x556c677e96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f8305000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f8306400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9310 0x556c677e96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:39522/0 (socket says 172.21.15.46:39522) 2024-09-10T07:47:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f8306400640 1 -- 172.21.15.46:0/571599202 learned_addr learned my addr 172.21.15.46:0/571599202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.874+0000 7f8305a00640 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9c30 0x556c677f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f8306400640 1 -- 172.21.15.46:0/571599202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9c30 msgr2=0x556c677f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f8306400640 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9c30 0x556c677f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f8306400640 1 -- 172.21.15.46:0/571599202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 msgr2=0x556c677f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f8306400640 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f8306400640 1 -- 172.21.15.46:0/571599202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c6745e580 con 0x556c677e9310 2024-09-10T07:47:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f8305000640 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f8306400640 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9310 0x556c677e96f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009f50 tx=0x7f82f80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c297f5d202badccc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f82fe000640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82e80a2b80 0x7f82e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f82fe000640 1 -- 172.21.15.46:0/4245245856 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f82ec0cf7b0 con 0x7f83001049c0 2024-09-10T07:47:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f82f2a00640 1 -- 172.21.15.46:0/571599202 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8004d20 con 0x556c677e9310 2024-09-10T07:47:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.875+0000 7f82f2a00640 1 -- 172.21.15.46:0/571599202 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f8004050 con 0x556c677e9310 2024-09-10T07:47:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.876+0000 7f8305a00640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82e80a2b80 0x7f82e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.876+0000 7f82f2a00640 1 -- 172.21.15.46:0/571599202 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8005220 con 0x556c677e9310 2024-09-10T07:47:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 -- 172.21.15.46:0/571599202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9310 msgr2=0x556c677e96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9310 0x556c677e96f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009f50 tx=0x7f82f80118d0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f8305a00640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82e80a2b80 0x7f82e80a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x556c677e6700 tx=0x7f82f4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 -- 172.21.15.46:0/571599202 shutdown_connections 2024-09-10T07:47:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9c30 0x556c677f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 --2- 172.21.15.46:0/571599202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9310 0x556c677e96f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 -- 172.21.15.46:0/571599202 >> 172.21.15.46:0/571599202 conn(0x556c677e58a0 msgr2=0x556c677e5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 -- 172.21.15.46:0/571599202 shutdown_connections 2024-09-10T07:47:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 -- 172.21.15.46:0/571599202 wait complete. 2024-09-10T07:47:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.877+0000 7f830ae21740 1 -- start start 2024-09-10T07:47:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.878+0000 7f830ae21740 1 Processor -- start 2024-09-10T07:47:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.878+0000 7f830ae21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 0x556c677f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.878+0000 7f830ae21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9c30 0x556c677f1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.878+0000 7f830ae21740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.878+0000 7f830ae21740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556c677f70c0 con 0x556c677f44f0 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.878+0000 7f830ae21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556c6770a350 con 0x556c677e9c30 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.878+0000 7f830ae21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556c677f7240 con 0x556c677e9310 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.878+0000 7f830ae21740 10 client.0 _ll_register_callbacks cb 0x556c6772d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8306400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8306400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48420/0 (socket says 172.21.15.46:48420) 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8306400640 1 -- 172.21.15.46:0/3349939303 learned_addr learned my addr 172.21.15.46:0/3349939303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8305000640 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9c30 0x556c677f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8305a00640 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 0x556c677f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8305000640 1 -- 172.21.15.46:0/3349939303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 msgr2=0x556c677f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8305000640 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 0x556c677f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8305000640 1 -- 172.21.15.46:0/3349939303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 msgr2=0x556c677f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8305000640 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8305000640 1 -- 172.21.15.46:0/3349939303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c67528960 con 0x556c677e9c30 2024-09-10T07:47:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f8305000640 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9c30 0x556c677f1300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x556c677e6c20 tx=0x7f82ec09a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eb529a7b4b07ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.879+0000 7f82e7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f82e7400640 1 -- 172.21.15.46:0/3349939303 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0cfda0 con 0x556c677e9c30 2024-09-10T07:47:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f82e7400640 1 -- 172.21.15.46:0/3349939303 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f82ec09b130 con 0x556c677e9c30 2024-09-10T07:47:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 1 -- 172.21.15.46:0/3349939303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f8004050 con 0x556c677e9c30 2024-09-10T07:47:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f82e7400640 1 -- 172.21.15.46:0/3349939303 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f82ec09ffe0 con 0x556c677e9c30 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 10 client.27992 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 1 -- 172.21.15.46:0/3349939303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556c67802c50 con 0x556c677e9c30 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f82e7400640 1 -- 172.21.15.46:0/3349939303 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f8004050 con 0x556c677e9c30 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f82e7400640 1 -- 172.21.15.46:0/3349939303 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f82ec097a30 con 0x556c677e9c30 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 10 client.27992 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 10 client.27992 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 20 client.27992 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556c67804060 0x556c678064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 4 client.27992 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.880+0000 7f830ae21740 1 -- 172.21.15.46:0/3349939303 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c6775cbc0 con 0x556c67804060 2024-09-10T07:47:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.881+0000 7f8305a00640 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556c67804060 0x556c678064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.881+0000 7f8305a00640 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556c67804060 0x556c678064a0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5e0f0d48e69ffb server_cookie=192e77d30ebff779 in_seq=0 out_seq=0 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.881+0000 7f82e7400640 10 client.27992 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f82e7400640 1 -- 172.21.15.46:0/3349939303 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c6775cbc0 con 0x556c67804060 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f82e7400640 10 client.27992 handle_command_reply: tid=0 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 client.27992 shutdown 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 -- 172.21.15.46:0/3349939303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9c30 msgr2=0x556c677f1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9c30 0x556c677f1300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x556c677e6c20 tx=0x7f82ec09a1f0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 -- 172.21.15.46:0/3349939303 shutdown_connections 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556c67804060 0x556c678064a0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677f44f0 0x556c677f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e9c30 0x556c677f1300 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 --2- 172.21.15.46:0/3349939303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 0x556c677f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f82e7400640 0 client.27992 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.882+0000 7f830ae21740 1 -- 172.21.15.46:0/3349939303 >> 172.21.15.46:0/3349939303 conn(0x556c677e58a0 msgr2=0x556c677e4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.883+0000 7f830ae21740 1 -- 172.21.15.46:0/3349939303 shutdown_connections 2024-09-10T07:47:06.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.883+0000 7f830ae21740 1 -- 172.21.15.46:0/3349939303 wait complete. 2024-09-10T07:47:06.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.883+0000 7f830ae21740 20 client.27992 trim_cache size 0 max 16384 2024-09-10T07:47:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f830ae21740 1 Processor -- start 2024-09-10T07:47:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f830ae21740 1 -- start start 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f830ae21740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e9310 0x556c677ebd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f830ae21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677f44f0 0x556c6780bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f830ae21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e0a10 0x556c6780c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f830ae21740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83001b0e90 con 0x556c677e9310 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f830ae21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83001b29d0 con 0x556c677e0a10 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f830ae21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83001b1560 con 0x556c677f44f0 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f8305a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e9310 0x556c677ebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f8305a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e9310 0x556c677ebd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48426/0 (socket says 172.21.15.46:48426) 2024-09-10T07:47:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.901+0000 7f8305a00640 1 -- 172.21.15.46:0/2909193427 learned_addr learned my addr 172.21.15.46:0/2909193427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f8306400640 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e0a10 0x556c6780c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f8305000640 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677f44f0 0x556c6780bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f8305a00640 1 -- 172.21.15.46:0/2909193427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677f44f0 msgr2=0x556c6780bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f8305a00640 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677f44f0 0x556c6780bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f8305a00640 1 -- 172.21.15.46:0/2909193427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e0a10 msgr2=0x556c6780c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f8305a00640 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e0a10 0x556c6780c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f8305a00640 1 -- 172.21.15.46:0/2909193427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c677e56d0 con 0x556c677e9310 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f8305a00640 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e9310 0x556c677ebd70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f82f40096c0 tx=0x7f82f403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b57d61539f85e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f82f2000640 1 -- 172.21.15.46:0/2909193427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4035040 con 0x556c677e9310 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f82f2000640 1 -- 172.21.15.46:0/2909193427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f4059070 con 0x556c677e9310 2024-09-10T07:47:06.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.902+0000 7f82f2000640 1 -- 172.21.15.46:0/2909193427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4003850 con 0x556c677e9310 2024-09-10T07:47:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 -- 172.21.15.46:0/2909193427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e9310 msgr2=0x556c677ebd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e9310 0x556c677ebd70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f82f40096c0 tx=0x7f82f403f000 comp rx=0 tx=0).stop 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 -- 172.21.15.46:0/2909193427 shutdown_connections 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677e0a10 0x556c6780c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677f44f0 0x556c6780bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 --2- 172.21.15.46:0/2909193427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e9310 0x556c677ebd70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 -- 172.21.15.46:0/2909193427 >> 172.21.15.46:0/2909193427 conn(0x556c677e58a0 msgr2=0x556c677e4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 -- 172.21.15.46:0/2909193427 shutdown_connections 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.903+0000 7f830ae21740 1 -- 172.21.15.46:0/2909193427 wait complete. 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 1 -- start start 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 1 Processor -- start 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e0a10 0x556c677f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 0x556c677f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677f44f0 0x556c677f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83001b2300 con 0x556c677e0a10 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83001b1c30 con 0x556c677f44f0 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83001aee70 con 0x556c677e9310 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.904+0000 7f830ae21740 10 client.0 _ll_register_callbacks cb 0x7f82ec00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e0a10 0x556c677f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e0a10 0x556c677f9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48440/0 (socket says 172.21.15.46:48440) 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 -- 172.21.15.46:0/2485783980 learned_addr learned my addr 172.21.15.46:0/2485783980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8306400640 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677f44f0 0x556c677f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305000640 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 0x556c677f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 -- 172.21.15.46:0/2485783980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 msgr2=0x556c677f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 0x556c677f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 -- 172.21.15.46:0/2485783980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677f44f0 msgr2=0x556c677f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677f44f0 0x556c677f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 -- 172.21.15.46:0/2485783980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c677e6950 con 0x556c677e0a10 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f8305a00640 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e0a10 0x556c677f9320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005ab0 tx=0x7f82f4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca04dc60e230b6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.905+0000 7f82e7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.906+0000 7f82e7400640 1 -- 172.21.15.46:0/2485783980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4061070 con 0x556c677e0a10 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.906+0000 7f830ae21740 1 -- 172.21.15.46:0/2485783980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f4059070 con 0x556c677e0a10 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.906+0000 7f82e7400640 1 -- 172.21.15.46:0/2485783980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f82f4004960 con 0x556c677e0a10 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.906+0000 7f82e7400640 1 -- 172.21.15.46:0/2485783980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f82f4004020 con 0x556c677e0a10 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.906+0000 7f830ae21740 10 client.17340 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.906+0000 7f830ae21740 1 -- 172.21.15.46:0/2485783980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556c6786ff70 con 0x556c677e0a10 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.906+0000 7f82e7400640 1 -- 172.21.15.46:0/2485783980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f4059070 con 0x556c677e0a10 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.906+0000 7f82e7400640 1 -- 172.21.15.46:0/2485783980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f82f40533f0 con 0x556c677e0a10 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f830ae21740 10 client.17340 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f830ae21740 10 client.17340 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f830ae21740 20 client.17340 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f830ae21740 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556c67870630 0x556c67872a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f830ae21740 4 client.17340 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f491a352", "release": true, "await": true}] 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f830ae21740 1 -- 172.21.15.46:0/2485783980 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f491a352", "release": true, "await": true}) -- 0x556c677570d0 con 0x556c67870630 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f8305000640 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556c67870630 0x556c67872a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f8305000640 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556c67870630 0x556c67872a50 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3cdfa1ba0a017a server_cookie=9c1d28920c32dbe1 in_seq=0 out_seq=0 2024-09-10T07:47:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.907+0000 7f82e7400640 10 client.17340 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.940+0000 7f82e7400640 1 -- 172.21.15.46:0/2485783980 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556c677570d0 con 0x556c67870630 2024-09-10T07:47:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.940+0000 7f82e7400640 10 client.17340 handle_command_reply: tid=0 2024-09-10T07:47:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.946+0000 7f830ae21740 1 client.17340 shutdown 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f830ae21740 1 -- 172.21.15.46:0/2485783980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e0a10 msgr2=0x556c677f9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f830ae21740 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e0a10 0x556c677f9320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005ab0 tx=0x7f82f4004d30 comp rx=0 tx=0).stop 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f830ae21740 1 -- 172.21.15.46:0/2485783980 shutdown_connections 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f830ae21740 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556c67870630 0x556c67872a50 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f830ae21740 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c677f44f0 0x556c677f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f830ae21740 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c677e9310 0x556c677f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f830ae21740 1 --2- 172.21.15.46:0/2485783980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c677e0a10 0x556c677f9320 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005ab0 tx=0x7f82f4004d30 comp rx=0 tx=0).stop 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f830ae21740 1 -- 172.21.15.46:0/2485783980 >> 172.21.15.46:0/2485783980 conn(0x556c677e58a0 msgr2=0x556c677ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.947+0000 7f82e7400640 0 client.17340 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.948+0000 7f830ae21740 1 -- 172.21.15.46:0/2485783980 shutdown_connections 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.948+0000 7f830ae21740 1 -- 172.21.15.46:0/2485783980 wait complete. 2024-09-10T07:47:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.948+0000 7f830ae21740 20 client.17340 trim_cache size 0 max 16384 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 50, 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 36, 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "f491a352": { 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 36, 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:47:06.951 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 130.0, 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:06.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:06.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:06.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:06.953 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:47:06.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 -- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82e80a2b80 msgr2=0x7f82e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82e80a2b80 0x7f82e80a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x556c677e6700 tx=0x7f82f4005a20 comp rx=0 tx=0).stop 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 -- 172.21.15.46:0/4245245856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83001049c0 msgr2=0x7f830019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83001049c0 0x7f830019ab90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00e9f0 tx=0x7f82ec00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 -- 172.21.15.46:0/4245245856 shutdown_connections 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8300109140 0x7f830019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83001049c0 0x7f830019ab90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f82e80a2b80 0x7f82e80a5040 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 --2- 172.21.15.46:0/4245245856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8300104010 0x7f830019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 -- 172.21.15.46:0/4245245856 >> 172.21.15.46:0/4245245856 conn(0x7f83000ffa50 msgr2=0x7f8300100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 -- 172.21.15.46:0/4245245856 shutdown_connections 2024-09-10T07:47:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:06.952+0000 7f8308400640 1 -- 172.21.15.46:0/4245245856 wait complete. 2024-09-10T07:47:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:08 smithi181 ceph-mon[35397]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.3 MiB/s wr, 376 op/s 2024-09-10T07:47:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:08 smithi063 ceph-mon[35020]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.3 MiB/s wr, 376 op/s 2024-09-10T07:47:08.404 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:08 smithi046 ceph-mon[30705]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.3 MiB/s wr, 376 op/s 2024-09-10T07:47:08.407 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f491a352', total seconds elapsed: 14.5 2024-09-10T07:47:10.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:10 smithi046 ceph-mon[30705]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-10T07:47:10.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 590.0 s, 391.3 tps, lat 608.158 ms stddev 1869.509 2024-09-10T07:47:10.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:10 smithi181 ceph-mon[35397]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-10T07:47:10.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:10 smithi063 ceph-mon[35020]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-10T07:47:10.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:47:12.006 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:12.449 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:12 smithi181 ceph-mon[35397]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-10T07:47:12.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:12 smithi063 ceph-mon[35020]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-10T07:47:12.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:12 smithi046 ceph-mon[30705]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-10T07:47:13.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:47:13.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:13 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:47:13.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:13 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:47:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:13.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:13 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:14 smithi181 ceph-mon[35397]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-09-10T07:47:14.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:14 smithi063 ceph-mon[35020]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-09-10T07:47:14.929 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:14 smithi046 ceph-mon[30705]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-09-10T07:47:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.946+0000 7fa11f200640 1 -- 172.21.15.46:0/2165299628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 msgr2=0x7fa12006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.946+0000 7fa11f200640 1 --2- 172.21.15.46:0/2165299628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 0x7fa12006ee70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa110009860 tx=0x7fa11002a360 comp rx=0 tx=0).stop 2024-09-10T07:47:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.947+0000 7fa11f200640 1 -- 172.21.15.46:0/2165299628 shutdown_connections 2024-09-10T07:47:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.947+0000 7fa11f200640 1 --2- 172.21.15.46:0/2165299628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12011b140 0x7fa12011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.947+0000 7fa11f200640 1 --2- 172.21.15.46:0/2165299628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 0x7fa12006ee70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.947+0000 7fa11f200640 1 --2- 172.21.15.46:0/2165299628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120110190 0x7fa12006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.947+0000 7fa11f200640 1 -- 172.21.15.46:0/2165299628 >> 172.21.15.46:0/2165299628 conn(0x7fa120073180 msgr2=0x7fa120073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.947+0000 7fa11f200640 1 -- 172.21.15.46:0/2165299628 shutdown_connections 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.947+0000 7fa11f200640 1 -- 172.21.15.46:0/2165299628 wait complete. 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.949+0000 7fa11f200640 1 Processor -- start 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.949+0000 7fa11f200640 1 -- start start 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.950+0000 7fa11f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 0x7fa1201a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.950+0000 7fa11f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa120110190 0x7fa1201a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.950+0000 7fa11f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa12011b140 0x7fa1201aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.950+0000 7fa11de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 0x7fa1201a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 0x7fa1201a71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49356/0 (socket says 172.21.15.46:49356) 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa120120be0 con 0x7fa120110190 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa120120a60 con 0x7fa12011b140 2024-09-10T07:47:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa120120d60 con 0x7fa12006ea30 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11de00640 1 -- 172.21.15.46:0/1117585659 learned_addr learned my addr 172.21.15.46:0/1117585659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11e800640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa12011b140 0x7fa1201aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11d400640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa120110190 0x7fa1201a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11de00640 1 -- 172.21.15.46:0/1117585659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa12011b140 msgr2=0x7fa1201aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11de00640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa12011b140 0x7fa1201aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11de00640 1 -- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa120110190 msgr2=0x7fa1201a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11de00640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa120110190 0x7fa1201a7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.951+0000 7fa11de00640 1 -- 172.21.15.46:0/1117585659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1201ac170 con 0x7fa12006ea30 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.952+0000 7fa11de00640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 0x7fa1201a71c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00e9c0 tx=0x7fa10c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.952+0000 7fa116a00640 1 -- 172.21.15.46:0/1117585659 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00ccc0 con 0x7fa12006ea30 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.952+0000 7fa11f200640 1 -- 172.21.15.46:0/1117585659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1201ac460 con 0x7fa12006ea30 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.953+0000 7fa11f200640 1 -- 172.21.15.46:0/1117585659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1201b3d40 con 0x7fa12006ea30 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.953+0000 7fa116a00640 1 -- 172.21.15.46:0/1117585659 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa10c005500 con 0x7fa12006ea30 2024-09-10T07:47:15.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.953+0000 7fa116a00640 1 -- 172.21.15.46:0/1117585659 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00ccc0 con 0x7fa12006ea30 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.961+0000 7fa116a00640 1 -- 172.21.15.46:0/1117585659 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa10c0046c0 con 0x7fa12006ea30 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.961+0000 7fa127a4f740 1 Processor -- start 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.962+0000 7fa127a4f740 1 -- start start 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.963+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.963+0000 7fa11e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.963+0000 7fa116a00640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1040a2ed0 0x7fa1040a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.964+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d3670 0x558b947dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.964+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947ddf30 0x558b947e0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.964+0000 7fa127a4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558b947393e0 con 0x558b947ddf30 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.964+0000 7fa127a4f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558b946e0d80 con 0x558b947d2d50 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.964+0000 7fa127a4f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b947353a0 con 0x558b947d3670 2024-09-10T07:47:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.964+0000 7fa116a00640 1 -- 172.21.15.46:0/1117585659 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa10c0cf460 con 0x7fa12006ea30 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.964+0000 7fa11e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947d3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:39590/0 (socket says 172.21.15.46:39590) 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.964+0000 7fa11e800640 1 -- 172.21.15.46:0/1414529242 learned_addr learned my addr 172.21.15.46:0/1414529242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.965+0000 7fa11d400640 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d3670 0x558b947dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.965+0000 7fa11e800640 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947ddf30 0x558b947e0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.965+0000 7fa11de00640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1040a2ed0 0x7fa1040a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.966+0000 7fa11e800640 1 -- 172.21.15.46:0/1414529242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d3670 msgr2=0x558b947dd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.966+0000 7fa11e800640 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d3670 0x558b947dd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.966+0000 7fa11e800640 1 -- 172.21.15.46:0/1414529242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947ddf30 msgr2=0x558b947e0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.966+0000 7fa11e800640 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947ddf30 0x558b947e0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.966+0000 7fa11e800640 1 -- 172.21.15.46:0/1414529242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b946ffae0 con 0x558b947d2d50 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.966+0000 7fa11e800640 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947d3130 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa118009fd0 tx=0x7fa118009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ac9d0758430236 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.967+0000 7fa11de00640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1040a2ed0 0x7fa1040a5390 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa10c018000 tx=0x7fa10c0ce040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.967+0000 7fa10b400640 1 -- 172.21.15.46:0/1414529242 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118003970 con 0x558b947d2d50 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.967+0000 7fa10b400640 1 -- 172.21.15.46:0/1414529242 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11800f860 con 0x558b947d2d50 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.968+0000 7fa10b400640 1 -- 172.21.15.46:0/1414529242 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118004370 con 0x558b947d2d50 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.971+0000 7fa127a4f740 1 -- 172.21.15.46:0/1414529242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 msgr2=0x558b947d3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.972+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947d3130 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa118009fd0 tx=0x7fa118009a90 comp rx=0 tx=0).stop 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.972+0000 7fa127a4f740 1 -- 172.21.15.46:0/1414529242 shutdown_connections 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.972+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947ddf30 0x558b947e0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.972+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d3670 0x558b947dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.972+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1414529242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947d3130 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.972+0000 7fa127a4f740 1 -- 172.21.15.46:0/1414529242 >> 172.21.15.46:0/1414529242 conn(0x558b947cf2e0 msgr2=0x558b947cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.973+0000 7fa127a4f740 1 -- 172.21.15.46:0/1414529242 shutdown_connections 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.973+0000 7fa127a4f740 1 -- 172.21.15.46:0/1414529242 wait complete. 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.974+0000 7fa127a4f740 1 -- start start 2024-09-10T07:47:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.974+0000 7fa127a4f740 1 Processor -- start 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.975+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d2d50 0x558b947dae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.975+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d3670 0x558b947db380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.975+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 0x558b947db8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.975+0000 7fa127a4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558b947e0ba0 con 0x558b947d2d50 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.975+0000 7fa127a4f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558b946f40f0 con 0x558b947d3670 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.975+0000 7fa127a4f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b947e0d20 con 0x558b947ddf30 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.975+0000 7fa127a4f740 10 client.0 _ll_register_callbacks cb 0x558b946e59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.977+0000 7fa11d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d3670 0x558b947db380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.977+0000 7fa11d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d3670 0x558b947db380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:39592/0 (socket says 172.21.15.46:39592) 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.977+0000 7fa11d400640 1 -- 172.21.15.46:0/2886320211 learned_addr learned my addr 172.21.15.46:0/2886320211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.978+0000 7fa11e800640 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 0x558b947db8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.978+0000 7fa11e800640 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d2d50 0x558b947dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.978+0000 7fa11d400640 1 -- 172.21.15.46:0/2886320211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 msgr2=0x558b947db8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.978+0000 7fa11d400640 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 0x558b947db8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.978+0000 7fa11d400640 1 -- 172.21.15.46:0/2886320211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d2d50 msgr2=0x558b947dae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.978+0000 7fa11d400640 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d2d50 0x558b947dae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.979+0000 7fa11d400640 1 -- 172.21.15.46:0/2886320211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b944aa3c0 con 0x558b947d3670 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.980+0000 7fa11d400640 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d3670 0x558b947db380 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa110009860 tx=0x7fa1100095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44a34f1968aff53a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.980+0000 7fa0ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.981+0000 7fa0ffe00640 1 -- 172.21.15.46:0/2886320211 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110037040 con 0x558b947d3670 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.982+0000 7fa0ffe00640 1 -- 172.21.15.46:0/2886320211 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa110030520 con 0x558b947d3670 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.982+0000 7fa127a4f740 1 -- 172.21.15.46:0/2886320211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11800f860 con 0x558b947d3670 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.982+0000 7fa0ffe00640 1 -- 172.21.15.46:0/2886320211 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa110030720 con 0x558b947d3670 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.982+0000 7fa127a4f740 10 client.28010 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:15.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.982+0000 7fa127a4f740 1 -- 172.21.15.46:0/2886320211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558b947fd920 con 0x558b947d3670 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.983+0000 7fa0ffe00640 1 -- 172.21.15.46:0/2886320211 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11800f860 con 0x558b947d3670 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.983+0000 7fa0ffe00640 1 -- 172.21.15.46:0/2886320211 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fa110004a20 con 0x558b947d3670 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.984+0000 7fa127a4f740 10 client.28010 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.984+0000 7fa127a4f740 10 client.28010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.984+0000 7fa127a4f740 20 client.28010 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.984+0000 7fa127a4f740 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558b947fee30 0x558b94801270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.984+0000 7fa127a4f740 4 client.28010 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.984+0000 7fa127a4f740 1 -- 172.21.15.46:0/2886320211 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b94746950 con 0x558b947fee30 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.984+0000 7fa11e800640 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558b947fee30 0x558b94801270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.985+0000 7fa0ffe00640 10 client.28010 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.985+0000 7fa11e800640 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558b947fee30 0x558b94801270 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e667c766350c93f5 server_cookie=f4d03ba2d2b9735f in_seq=0 out_seq=0 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.990+0000 7fa0ffe00640 1 -- 172.21.15.46:0/2886320211 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b94746950 con 0x558b947fee30 2024-09-10T07:47:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.990+0000 7fa0ffe00640 10 client.28010 handle_command_reply: tid=0 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.990+0000 7fa127a4f740 1 client.28010 shutdown 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.993+0000 7fa127a4f740 1 -- 172.21.15.46:0/2886320211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d3670 msgr2=0x558b947db380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.993+0000 7fa127a4f740 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d3670 0x558b947db380 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa110009860 tx=0x7fa1100095a0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.994+0000 7fa127a4f740 1 -- 172.21.15.46:0/2886320211 shutdown_connections 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.994+0000 7fa127a4f740 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558b947fee30 0x558b94801270 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.994+0000 7fa127a4f740 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 0x558b947db8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.994+0000 7fa127a4f740 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d3670 0x558b947db380 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.994+0000 7fa127a4f740 1 --2- 172.21.15.46:0/2886320211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d2d50 0x558b947dae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.994+0000 7fa127a4f740 1 -- 172.21.15.46:0/2886320211 >> 172.21.15.46:0/2886320211 conn(0x558b947cf2e0 msgr2=0x558b947d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.994+0000 7fa0ffe00640 0 client.28010 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.994+0000 7fa127a4f740 1 -- 172.21.15.46:0/2886320211 shutdown_connections 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.995+0000 7fa127a4f740 1 -- 172.21.15.46:0/2886320211 wait complete. 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:13.995+0000 7fa127a4f740 20 client.28010 trim_cache size 0 max 16384 2024-09-10T07:47:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.033+0000 7fa127a4f740 1 Processor -- start 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.033+0000 7fa127a4f740 1 -- start start 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.034+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d2d50 0x558b94806b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.036+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d3670 0x558b94807080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.036+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947ddf30 0x558b947d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.036+0000 7fa127a4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa10000fb60 con 0x558b947d3670 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.036+0000 7fa127a4f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1000116a0 con 0x558b947ddf30 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.036+0000 7fa127a4f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa100010230 con 0x558b947d2d50 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.037+0000 7fa11e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d2d50 0x558b94806b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.037+0000 7fa11e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d2d50 0x558b94806b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:49380/0 (socket says 172.21.15.46:49380) 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.037+0000 7fa11e800640 1 -- 172.21.15.46:0/981527494 learned_addr learned my addr 172.21.15.46:0/981527494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.037+0000 7fa11e800640 1 --2- 172.21.15.46:0/981527494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947ddf30 0x558b947d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.038+0000 7fa11e800640 1 -- 172.21.15.46:0/981527494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947ddf30 msgr2=0x558b947d80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.038+0000 7fa11e800640 1 --2- 172.21.15.46:0/981527494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947ddf30 0x558b947d80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.039+0000 7fa11e800640 1 -- 172.21.15.46:0/981527494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d3670 msgr2=0x558b94807080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.039+0000 7fa11e800640 1 --2- 172.21.15.46:0/981527494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d3670 0x558b94807080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.040+0000 7fa11e800640 1 -- 172.21.15.46:0/981527494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b947d9390 con 0x558b947d2d50 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.042+0000 7fa11e800640 1 --2- 172.21.15.46:0/981527494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d2d50 0x558b94806b40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x558b947dc0b0 tx=0x7fa118025040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814997b587ea0b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.046+0000 7fa10aa00640 1 -- 172.21.15.46:0/981527494 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118003ba0 con 0x558b947d2d50 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.046+0000 7fa10aa00640 1 -- 172.21.15.46:0/981527494 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11800fd40 con 0x558b947d2d50 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.046+0000 7fa10aa00640 1 -- 172.21.15.46:0/981527494 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118005550 con 0x558b947d2d50 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.047+0000 7fa127a4f740 1 -- 172.21.15.46:0/981527494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d2d50 msgr2=0x558b94806b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.047+0000 7fa127a4f740 1 --2- 172.21.15.46:0/981527494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d2d50 0x558b94806b40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x558b947dc0b0 tx=0x7fa118025040 comp rx=0 tx=0).stop 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.053+0000 7fa127a4f740 1 -- 172.21.15.46:0/981527494 shutdown_connections 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.053+0000 7fa127a4f740 1 --2- 172.21.15.46:0/981527494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947ddf30 0x558b947d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.053+0000 7fa127a4f740 1 --2- 172.21.15.46:0/981527494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d3670 0x558b94807080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.053+0000 7fa127a4f740 1 --2- 172.21.15.46:0/981527494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947d2d50 0x558b94806b40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.053+0000 7fa127a4f740 1 -- 172.21.15.46:0/981527494 >> 172.21.15.46:0/981527494 conn(0x558b947cf2e0 msgr2=0x558b947d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.060+0000 7fa127a4f740 1 -- 172.21.15.46:0/981527494 shutdown_connections 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.060+0000 7fa127a4f740 1 -- 172.21.15.46:0/981527494 wait complete. 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.060+0000 7fa127a4f740 1 -- start start 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.062+0000 7fa127a4f740 1 Processor -- start 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.066+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.068+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d3670 0x558b947e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.068+0000 7fa127a4f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 0x558b94862ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.068+0000 7fa127a4f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa100010fd0 con 0x558b947d3670 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.068+0000 7fa127a4f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa100010900 con 0x558b947d2d50 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.068+0000 7fa127a4f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa10000db00 con 0x558b947ddf30 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.068+0000 7fa127a4f740 10 client.0 _ll_register_callbacks cb 0x7fa1100096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.071+0000 7fa11e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.071+0000 7fa11e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947e25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:39612/0 (socket says 172.21.15.46:39612) 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.071+0000 7fa11e800640 1 -- 172.21.15.46:0/1222190785 learned_addr learned my addr 172.21.15.46:0/1222190785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.072+0000 7fa11e800640 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 0x558b94862ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.072+0000 7fa11e800640 1 -- 172.21.15.46:0/1222190785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 msgr2=0x558b94862ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.072+0000 7fa11e800640 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 0x558b94862ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.072+0000 7fa11e800640 1 -- 172.21.15.46:0/1222190785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d3670 msgr2=0x558b947e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.072+0000 7fa11e800640 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d3670 0x558b947e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.072+0000 7fa11e800640 1 -- 172.21.15.46:0/1222190785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b94863120 con 0x558b947d2d50 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.075+0000 7fa11e800640 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947e25b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa118013680 tx=0x7fa1180039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3dd9c50e50f681a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.075+0000 7fa0ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.082+0000 7fa0ffe00640 1 -- 172.21.15.46:0/1222190785 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118003da0 con 0x558b947d2d50 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.083+0000 7fa0ffe00640 1 -- 172.21.15.46:0/1222190785 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa11800f860 con 0x558b947d2d50 2024-09-10T07:47:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.083+0000 7fa127a4f740 1 -- 172.21.15.46:0/1222190785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11800fd40 con 0x558b947d2d50 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.087+0000 7fa0ffe00640 1 -- 172.21.15.46:0/1222190785 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa118005070 con 0x558b947d2d50 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.087+0000 7fa127a4f740 10 client.28016 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.087+0000 7fa127a4f740 1 -- 172.21.15.46:0/1222190785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558b9486a460 con 0x558b947d2d50 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.087+0000 7fa0ffe00640 1 -- 172.21.15.46:0/1222190785 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11800fd40 con 0x558b947d2d50 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.087+0000 7fa0ffe00640 1 -- 172.21.15.46:0/1222190785 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fa11803b030 con 0x558b947d2d50 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.087+0000 7fa127a4f740 10 client.28016 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.087+0000 7fa127a4f740 10 client.28016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.087+0000 7fa127a4f740 20 client.28016 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.088+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558b9486a830 0x558b9486cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.088+0000 7fa127a4f740 4 client.28016 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.088+0000 7fa127a4f740 1 -- 172.21.15.46:0/1222190785 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b94740e60 con 0x558b9486a830 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.088+0000 7fa11d400640 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558b9486a830 0x558b9486cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.089+0000 7fa11d400640 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558b9486a830 0x558b9486cc20 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d36e3a5685cd424 server_cookie=fb06c0d904a69d5a in_seq=0 out_seq=0 2024-09-10T07:47:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.089+0000 7fa0ffe00640 10 client.28016 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.091+0000 7fa0ffe00640 1 -- 172.21.15.46:0/1222190785 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558b94740e60 con 0x558b9486a830 2024-09-10T07:47:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.091+0000 7fa0ffe00640 10 client.28016 handle_command_reply: tid=0 2024-09-10T07:47:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.091+0000 7fa127a4f740 1 client.28016 shutdown 2024-09-10T07:47:15.268 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:47:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:47:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:47:15.268 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.091+0000 7fa127a4f740 1 -- 172.21.15.46:0/1222190785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 msgr2=0x558b947e25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.091+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947e25b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa118013680 tx=0x7fa1180039d0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.092+0000 7fa127a4f740 1 -- 172.21.15.46:0/1222190785 shutdown_connections 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.092+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558b9486a830 0x558b9486cc20 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.092+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b947ddf30 0x558b94862ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.092+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558b947d3670 0x558b947e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.092+0000 7fa127a4f740 1 --2- 172.21.15.46:0/1222190785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558b947d2d50 0x558b947e25b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.092+0000 7fa127a4f740 1 -- 172.21.15.46:0/1222190785 >> 172.21.15.46:0/1222190785 conn(0x558b947cf2e0 msgr2=0x558b947cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.092+0000 7fa0ffe00640 0 client.28016 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.093+0000 7fa127a4f740 1 -- 172.21.15.46:0/1222190785 shutdown_connections 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.093+0000 7fa127a4f740 1 -- 172.21.15.46:0/1222190785 wait complete. 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.093+0000 7fa127a4f740 20 client.28016 trim_cache size 0 max 16384 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.094+0000 7fa11f200640 1 -- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1040a2ed0 msgr2=0x7fa1040a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.094+0000 7fa11f200640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1040a2ed0 0x7fa1040a5390 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa10c018000 tx=0x7fa10c0ce040 comp rx=0 tx=0).stop 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.094+0000 7fa11f200640 1 -- 172.21.15.46:0/1117585659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 msgr2=0x7fa1201a71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.094+0000 7fa11f200640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 0x7fa1201a71c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00e9c0 tx=0x7fa10c00ee80 comp rx=0 tx=0).stop 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.095+0000 7fa11f200640 1 -- 172.21.15.46:0/1117585659 shutdown_connections 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.095+0000 7fa11f200640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa12011b140 0x7fa1201aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.095+0000 7fa11f200640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa1040a2ed0 0x7fa1040a5390 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.095+0000 7fa11f200640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa120110190 0x7fa1201a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.095+0000 7fa11f200640 1 --2- 172.21.15.46:0/1117585659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa12006ea30 0x7fa1201a71c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.095+0000 7fa11f200640 1 -- 172.21.15.46:0/1117585659 >> 172.21.15.46:0/1117585659 conn(0x7fa120073180 msgr2=0x7fa120119370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.095+0000 7fa11f200640 1 -- 172.21.15.46:0/1117585659 shutdown_connections 2024-09-10T07:47:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:14.096+0000 7fa11f200640 1 -- 172.21.15.46:0/1117585659 wait complete. 2024-09-10T07:47:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 595.0 s, 351.8 tps, lat 278.462 ms stddev 455.808 2024-09-10T07:47:16.399 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:47:16.399 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:47:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:16 smithi046 ceph-mon[30705]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-09-10T07:47:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:16.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:16 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:16 smithi181 ceph-mon[35397]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-09-10T07:47:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:16 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:16.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:16 smithi063 ceph-mon[35020]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.6 MiB/s wr, 280 op/s 2024-09-10T07:47:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:16 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:18 smithi181 ceph-mon[35397]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 MiB/s wr, 743 op/s 2024-09-10T07:47:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:47:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:47:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:18 smithi063 ceph-mon[35020]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 MiB/s wr, 743 op/s 2024-09-10T07:47:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:47:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:47:18.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:18 smithi046 ceph-mon[30705]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 MiB/s wr, 743 op/s 2024-09-10T07:47:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:47:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:47:18.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:47:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.171+0000 7f7bff200640 1 -- 172.21.15.46:0/2035176026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c00102ae0 msgr2=0x7f7c00104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.171+0000 7f7bff200640 1 --2- 172.21.15.46:0/2035176026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c00102ae0 0x7f7c00104f50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009860 tx=0x7f7bec02a340 comp rx=0 tx=0).stop 2024-09-10T07:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.172+0000 7f7bff200640 1 -- 172.21.15.46:0/2035176026 shutdown_connections 2024-09-10T07:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.172+0000 7f7bff200640 1 --2- 172.21.15.46:0/2035176026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c00105490 0x7f7c00105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.172+0000 7f7bff200640 1 --2- 172.21.15.46:0/2035176026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c00102ae0 0x7f7c00104f50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.172+0000 7f7bff200640 1 --2- 172.21.15.46:0/2035176026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c00100190 0x7f7c001025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.172+0000 7f7bff200640 1 -- 172.21.15.46:0/2035176026 >> 172.21.15.46:0/2035176026 conn(0x7f7c000fbfa0 msgr2=0x7f7c000fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.173+0000 7f7bff200640 1 -- 172.21.15.46:0/2035176026 shutdown_connections 2024-09-10T07:47:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.173+0000 7f7bff200640 1 -- 172.21.15.46:0/2035176026 wait complete. 2024-09-10T07:47:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.181+0000 7f7bff200640 1 Processor -- start 2024-09-10T07:47:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.181+0000 7f7bff200640 1 -- start start 2024-09-10T07:47:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.182+0000 7f7bff200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c00100190 0x7f7c0019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.182+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c00100190 0x7f7c0019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.182+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c00100190 0x7f7c0019e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48498/0 (socket says 172.21.15.46:48498) 2024-09-10T07:47:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.184+0000 7f7bff200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c00102ae0 0x7f7c0019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.184+0000 7f7bfde00640 1 -- 172.21.15.46:0/153138583 learned_addr learned my addr 172.21.15.46:0/153138583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.184+0000 7f7bff200640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c00105490 0x7f7c001a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.184+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00113b90 con 0x7f7c00105490 2024-09-10T07:47:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.184+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00113a10 con 0x7f7c00102ae0 2024-09-10T07:47:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.184+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00113d10 con 0x7f7c00100190 2024-09-10T07:47:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.185+0000 7f7bfd400640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c00102ae0 0x7f7c0019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.187+0000 7f7bfd400640 1 -- 172.21.15.46:0/153138583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c00100190 msgr2=0x7f7c0019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.187+0000 7f7bfd400640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c00100190 0x7f7c0019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.187+0000 7f7bfd400640 1 -- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c00105490 msgr2=0x7f7c001a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.187+0000 7f7bfd400640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c00105490 0x7f7c001a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.187+0000 7f7bfd400640 1 -- 172.21.15.46:0/153138583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c001a5ec0 con 0x7f7c00102ae0 2024-09-10T07:47:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.189+0000 7f7bfd400640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c00102ae0 0x7f7c0019e7e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7bec035040 tx=0x7f7bec033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.195+0000 7f7bf6a00640 1 -- 172.21.15.46:0/153138583 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec037040 con 0x7f7c00102ae0 2024-09-10T07:47:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.196+0000 7f7bf6a00640 1 -- 172.21.15.46:0/153138583 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bec030850 con 0x7f7c00102ae0 2024-09-10T07:47:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.196+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c001a6150 con 0x7f7c00102ae0 2024-09-10T07:47:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.196+0000 7f7bf6a00640 1 -- 172.21.15.46:0/153138583 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec037040 con 0x7f7c00102ae0 2024-09-10T07:47:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.196+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c001a65c0 con 0x7f7c00102ae0 2024-09-10T07:47:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.202+0000 7f7c07a68740 1 Processor -- start 2024-09-10T07:47:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.202+0000 7f7bf6a00640 1 -- 172.21.15.46:0/153138583 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7bec0046c0 con 0x7f7c00102ae0 2024-09-10T07:47:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.204+0000 7f7c07a68740 1 -- start start 2024-09-10T07:47:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.204+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 0x562c2e889660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.204+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 0x562c2e88e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.204+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.204+0000 7f7c07a68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562c2e7ee100 con 0x562c2e889260 2024-09-10T07:47:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.204+0000 7f7c07a68740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562c2e7961b0 con 0x562c2e88ec30 2024-09-10T07:47:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.204+0000 7f7c07a68740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562c2e7ea0c0 con 0x562c2e889ba0 2024-09-10T07:47:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.204+0000 7f7bf6a00640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7be40a2d80 0x7f7be40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.205+0000 7f7bfe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 0x562c2e889660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.205+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 0x562c2e88e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.205+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 0x562c2e88e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48514/0 (socket says 172.21.15.46:48514) 2024-09-10T07:47:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.205+0000 7f7bfde00640 1 -- 172.21.15.46:0/3142839849 learned_addr learned my addr 172.21.15.46:0/3142839849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.206+0000 7f7bf6a00640 1 -- 172.21.15.46:0/153138583 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7bec0eeb00 con 0x7f7c00102ae0 2024-09-10T07:47:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.207+0000 7f7bfd400640 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.207+0000 7f7bfe800640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7be40a2d80 0x7f7be40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.208+0000 7f7bfd400640 1 -- 172.21.15.46:0/3142839849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 msgr2=0x562c2e88e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.208+0000 7f7bfd400640 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 0x562c2e88e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.208+0000 7f7bfd400640 1 -- 172.21.15.46:0/3142839849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 msgr2=0x562c2e889660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.208+0000 7f7bfd400640 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 0x562c2e889660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.208+0000 7f7bfd400640 1 -- 172.21.15.46:0/3142839849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2e501210 con 0x562c2e88ec30 2024-09-10T07:47:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.208+0000 7f7bfe800640 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 0x562c2e889660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.209+0000 7f7bfd400640 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8910f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7bec02a930 tx=0x7f7bec0b7440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3c47a33615d0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.210+0000 7f7beb000640 1 -- 172.21.15.46:0/3142839849 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0d41e0 con 0x562c2e88ec30 2024-09-10T07:47:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.210+0000 7f7beb000640 1 -- 172.21.15.46:0/3142839849 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bec0db0b0 con 0x562c2e88ec30 2024-09-10T07:47:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.210+0000 7f7beb000640 1 -- 172.21.15.46:0/3142839849 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0d3510 con 0x562c2e88ec30 2024-09-10T07:47:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.211+0000 7f7c07a68740 1 -- 172.21.15.46:0/3142839849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 msgr2=0x562c2e8910f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.211+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8910f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7bec02a930 tx=0x7f7bec0b7440 comp rx=0 tx=0).stop 2024-09-10T07:47:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.212+0000 7f7bfe800640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7be40a2d80 0x7f7be40a5240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009eb0 tx=0x7f7bf0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.212+0000 7f7c07a68740 1 -- 172.21.15.46:0/3142839849 shutdown_connections 2024-09-10T07:47:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.212+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8910f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.212+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 0x562c2e88e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.212+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3142839849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 0x562c2e889660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.212+0000 7f7c07a68740 1 -- 172.21.15.46:0/3142839849 >> 172.21.15.46:0/3142839849 conn(0x562c2e8848d0 msgr2=0x562c2e884cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.212+0000 7f7c07a68740 1 -- 172.21.15.46:0/3142839849 shutdown_connections 2024-09-10T07:47:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.214+0000 7f7c07a68740 1 -- 172.21.15.46:0/3142839849 wait complete. 2024-09-10T07:47:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.214+0000 7f7c07a68740 1 -- start start 2024-09-10T07:47:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.215+0000 7f7c07a68740 1 Processor -- start 2024-09-10T07:47:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.219+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 0x562c2e8a4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.219+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 0x562c2e8a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.220+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.220+0000 7f7bfe800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.220+0000 7f7bfe800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35652/0 (socket says 172.21.15.46:35652) 2024-09-10T07:47:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.220+0000 7f7bfe800640 1 -- 172.21.15.46:0/3578120078 learned_addr learned my addr 172.21.15.46:0/3578120078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.220+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562c2e8a29d0 con 0x562c2e889260 2024-09-10T07:47:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.220+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562c2e777090 con 0x562c2e88ec30 2024-09-10T07:47:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.220+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562c2e8a2b50 con 0x562c2e889ba0 2024-09-10T07:47:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.221+0000 7f7bfe800640 1 -- 172.21.15.46:0/3578120078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 msgr2=0x562c2e8a4690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:47:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.221+0000 7f7bfe800640 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 0x562c2e8a4690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.221+0000 7f7bfe800640 1 -- 172.21.15.46:0/3578120078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 msgr2=0x562c2e8a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.221+0000 7f7bfe800640 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 0x562c2e8a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.221+0000 7f7bfe800640 1 -- 172.21.15.46:0/3578120078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2e5cd620 con 0x562c2e88ec30 2024-09-10T07:47:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.221+0000 7f7c07a68740 10 client.0 _ll_register_callbacks cb 0x562c2e7cbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.222+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bec0db0b0 con 0x562c2e88ec30 2024-09-10T07:47:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.223+0000 7f7bfe800640 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8ab6d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000fc30 tx=0x7f7bf00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1323d7b2e2511a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.223+0000 7f7bdf400640 10 client.28028 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.223+0000 7f7bdf400640 1 -- 172.21.15.46:0/3578120078 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0023020 con 0x562c2e88ec30 2024-09-10T07:47:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.224+0000 7f7bdf400640 1 -- 172.21.15.46:0/3578120078 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7bf0004230 con 0x562c2e88ec30 2024-09-10T07:47:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.224+0000 7f7bdf400640 1 -- 172.21.15.46:0/3578120078 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7bf0004430 con 0x562c2e88ec30 2024-09-10T07:47:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.224+0000 7f7c07a68740 10 client.28028 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.224+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=50}) -- 0x562c2e8ae540 con 0x562c2e88ec30 2024-09-10T07:47:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.225+0000 7f7bdf400640 1 -- 172.21.15.46:0/3578120078 <== mon.2 v2:172.21.15.63:3300/0 4 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f7bf001c050 con 0x562c2e88ec30 2024-09-10T07:47:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.225+0000 7f7c07a68740 10 client.28028 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.225+0000 7f7c07a68740 10 client.28028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.225+0000 7f7c07a68740 20 client.28028 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.225+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562c2e8afba0 0x562c2e8b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.226+0000 7f7c07a68740 4 client.28028 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.226+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c2e7fb830 con 0x562c2e8afba0 2024-09-10T07:47:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.231+0000 7f7bfde00640 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562c2e8afba0 0x562c2e8b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.236+0000 7f7bdf400640 10 client.28028 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.236+0000 7f7bfde00640 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562c2e8afba0 0x562c2e8b2010 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de56ed21c07c6662 server_cookie=9f09c1e0770a5b12 in_seq=0 out_seq=0 2024-09-10T07:47:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.238+0000 7f7bdf400640 1 -- 172.21.15.46:0/3578120078 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c2e7fb830 con 0x562c2e8afba0 2024-09-10T07:47:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.238+0000 7f7bdf400640 10 client.28028 handle_command_reply: tid=0 2024-09-10T07:47:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.238+0000 7f7c07a68740 1 client.28028 shutdown 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 msgr2=0x562c2e8ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8ab6d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000fc30 tx=0x7f7bf00046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 shutdown_connections 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562c2e8afba0 0x562c2e8b2010 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e8ab6d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889ba0 0x562c2e8a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 --2- 172.21.15.46:0/3578120078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889260 0x562c2e8a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7bdf400640 0 client.28028 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 >> 172.21.15.46:0/3578120078 conn(0x562c2e8848d0 msgr2=0x562c2e8860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 shutdown_connections 2024-09-10T07:47:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.239+0000 7f7c07a68740 1 -- 172.21.15.46:0/3578120078 wait complete. 2024-09-10T07:47:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.240+0000 7f7c07a68740 20 client.28028 trim_cache size 0 max 16384 2024-09-10T07:47:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.288+0000 7f7c07a68740 1 Processor -- start 2024-09-10T07:47:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.289+0000 7f7c07a68740 1 -- start start 2024-09-10T07:47:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.289+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e885780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.289+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889ba0 0x562c2e8ab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.289+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e885cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.289+0000 7f7c07a68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a83b0 con 0x562c2e889ba0 2024-09-10T07:47:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.289+0000 7f7c07a68740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a9ef0 con 0x562c2e88ec30 2024-09-10T07:47:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.289+0000 7f7c07a68740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a8a80 con 0x562c2e889260 2024-09-10T07:47:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.290+0000 7f7bfe800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e885cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.290+0000 7f7bfe800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e885cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35656/0 (socket says 172.21.15.46:35656) 2024-09-10T07:47:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.290+0000 7f7bfe800640 1 -- 172.21.15.46:0/740591627 learned_addr learned my addr 172.21.15.46:0/740591627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.292+0000 7f7bfde00640 1 --2- 172.21.15.46:0/740591627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e885780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.292+0000 7f7bfe800640 1 -- 172.21.15.46:0/740591627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 msgr2=0x562c2e885780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.292+0000 7f7bfe800640 1 --2- 172.21.15.46:0/740591627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e885780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.292+0000 7f7bfe800640 1 -- 172.21.15.46:0/740591627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889ba0 msgr2=0x562c2e8ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.292+0000 7f7bfe800640 1 --2- 172.21.15.46:0/740591627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889ba0 0x562c2e8ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.292+0000 7f7bfe800640 1 -- 172.21.15.46:0/740591627 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2e8b2660 con 0x562c2e88ec30 2024-09-10T07:47:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.292+0000 7f7bfe800640 1 --2- 172.21.15.46:0/740591627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e885cc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00107c0 tx=0x7f7bf001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb7e2205c0f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.293+0000 7f7bea600640 1 -- 172.21.15.46:0/740591627 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0036070 con 0x562c2e88ec30 2024-09-10T07:47:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.293+0000 7f7bea600640 1 -- 172.21.15.46:0/740591627 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bec0db0b0 con 0x562c2e88ec30 2024-09-10T07:47:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.293+0000 7f7bea600640 1 -- 172.21.15.46:0/740591627 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0023040 con 0x562c2e88ec30 2024-09-10T07:47:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.295+0000 7f7c07a68740 1 -- 172.21.15.46:0/740591627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 msgr2=0x562c2e885cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.295+0000 7f7c07a68740 1 --2- 172.21.15.46:0/740591627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e885cc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00107c0 tx=0x7f7bf001f000 comp rx=0 tx=0).stop 2024-09-10T07:47:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.296+0000 7f7c07a68740 1 -- 172.21.15.46:0/740591627 shutdown_connections 2024-09-10T07:47:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.296+0000 7f7c07a68740 1 --2- 172.21.15.46:0/740591627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e88ec30 0x562c2e885cc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.296+0000 7f7c07a68740 1 --2- 172.21.15.46:0/740591627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e889ba0 0x562c2e8ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.296+0000 7f7c07a68740 1 --2- 172.21.15.46:0/740591627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e885780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.296+0000 7f7c07a68740 1 -- 172.21.15.46:0/740591627 >> 172.21.15.46:0/740591627 conn(0x562c2e8848d0 msgr2=0x562c2e8a4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.298+0000 7f7c07a68740 1 -- 172.21.15.46:0/740591627 shutdown_connections 2024-09-10T07:47:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.298+0000 7f7c07a68740 1 -- 172.21.15.46:0/740591627 wait complete. 2024-09-10T07:47:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.298+0000 7f7c07a68740 1 -- start start 2024-09-10T07:47:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.299+0000 7f7c07a68740 1 Processor -- start 2024-09-10T07:47:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.300+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e8a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.300+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e889ba0 0x562c2e8a8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.301+0000 7f7c07a68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e88ec30 0x562c2e8a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.301+0000 7f7c07a68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a9820 con 0x562c2e88ec30 2024-09-10T07:47:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.301+0000 7f7c07a68740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001a9150 con 0x562c2e889ba0 2024-09-10T07:47:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.301+0000 7f7c07a68740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0010ea60 con 0x562c2e889260 2024-09-10T07:47:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.301+0000 7f7c07a68740 10 client.0 _ll_register_callbacks cb 0x7f7bf0005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.301+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e8a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.301+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e8a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48536/0 (socket says 172.21.15.46:48536) 2024-09-10T07:47:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.302+0000 7f7bfde00640 1 -- 172.21.15.46:0/518401674 learned_addr learned my addr 172.21.15.46:0/518401674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.302+0000 7f7bfd400640 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e889ba0 0x562c2e8a8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.303+0000 7f7bfde00640 1 -- 172.21.15.46:0/518401674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e889ba0 msgr2=0x562c2e8a8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.303+0000 7f7bfde00640 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e889ba0 0x562c2e8a8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.303+0000 7f7bfde00640 1 -- 172.21.15.46:0/518401674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e88ec30 msgr2=0x562c2e8a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.303+0000 7f7bfde00640 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e88ec30 0x562c2e8a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.303+0000 7f7bfde00640 1 -- 172.21.15.46:0/518401674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2e8aa350 con 0x562c2e889260 2024-09-10T07:47:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.303+0000 7f7bfd400640 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e889ba0 0x562c2e8a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.304+0000 7f7bfde00640 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e8a87f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x562c2e885750 tx=0x7f7bf800edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0a7deced7b0ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.304+0000 7f7bdf400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:47:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.305+0000 7f7bdf400640 1 -- 172.21.15.46:0/518401674 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf801adf0 con 0x562c2e889260 2024-09-10T07:47:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.305+0000 7f7bdf400640 1 -- 172.21.15.46:0/518401674 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7bf8021420 con 0x562c2e889260 2024-09-10T07:47:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.308+0000 7f7c07a68740 1 -- 172.21.15.46:0/518401674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bec0db0b0 con 0x562c2e889260 2024-09-10T07:47:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.309+0000 7f7bdf400640 1 -- 172.21.15.46:0/518401674 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7bf8021620 con 0x562c2e889260 2024-09-10T07:47:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.309+0000 7f7c07a68740 10 client.27973 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.309+0000 7f7c07a68740 1 -- 172.21.15.46:0/518401674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562c2e8ab730 con 0x562c2e889260 2024-09-10T07:47:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.310+0000 7f7bdf400640 1 -- 172.21.15.46:0/518401674 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bec0db0b0 con 0x562c2e889260 2024-09-10T07:47:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.310+0000 7f7bdf400640 1 -- 172.21.15.46:0/518401674 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f7bf8017040 con 0x562c2e889260 2024-09-10T07:47:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.310+0000 7f7c07a68740 10 client.27973 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.310+0000 7f7c07a68740 10 client.27973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.310+0000 7f7c07a68740 20 client.27973 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.310+0000 7f7c07a68740 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562c2e91add0 0x562c2e91d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.310+0000 7f7c07a68740 4 client.27973 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:47:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.310+0000 7f7c07a68740 1 -- 172.21.15.46:0/518401674 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c2e7f5b80 con 0x562c2e91add0 2024-09-10T07:47:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.311+0000 7f7bfd400640 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562c2e91add0 0x562c2e91d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.312+0000 7f7bfd400640 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562c2e91add0 0x562c2e91d1f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc11499f75af58d4 server_cookie=6fb2126268e372ca in_seq=0 out_seq=0 2024-09-10T07:47:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.312+0000 7f7bdf400640 10 client.27973 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.331+0000 7f7bdf400640 1 -- 172.21.15.46:0/518401674 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c2e7f5b80 con 0x562c2e91add0 2024-09-10T07:47:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.331+0000 7f7bdf400640 10 client.27973 handle_command_reply: tid=0 2024-09-10T07:47:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.331+0000 7f7c07a68740 1 client.27973 shutdown 2024-09-10T07:47:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7c07a68740 1 -- 172.21.15.46:0/518401674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 msgr2=0x562c2e8a87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7c07a68740 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e8a87f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x562c2e885750 tx=0x7f7bf800edb0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7c07a68740 1 -- 172.21.15.46:0/518401674 shutdown_connections 2024-09-10T07:47:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7c07a68740 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562c2e91add0 0x562c2e91d1f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7c07a68740 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c2e88ec30 0x562c2e8a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7c07a68740 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c2e889ba0 0x562c2e8a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7c07a68740 1 --2- 172.21.15.46:0/518401674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c2e889260 0x562c2e8a87f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7c07a68740 1 -- 172.21.15.46:0/518401674 >> 172.21.15.46:0/518401674 conn(0x562c2e8848d0 msgr2=0x562c2e8831f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.332+0000 7f7bdf400640 0 client.27973 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.333+0000 7f7c07a68740 1 -- 172.21.15.46:0/518401674 shutdown_connections 2024-09-10T07:47:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.334+0000 7f7c07a68740 1 -- 172.21.15.46:0/518401674 wait complete. 2024-09-10T07:47:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.334+0000 7f7c07a68740 20 client.27973 trim_cache size 0 max 16384 2024-09-10T07:47:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:47:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:47:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:47:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:47:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7be40a2d80 msgr2=0x7f7be40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7be40a2d80 0x7f7be40a5240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009eb0 tx=0x7f7bf0005f90 comp rx=0 tx=0).stop 2024-09-10T07:47:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c00102ae0 msgr2=0x7f7c0019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c00102ae0 0x7f7c0019e7e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7bec035040 tx=0x7f7bec033d90 comp rx=0 tx=0).stop 2024-09-10T07:47:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 shutdown_connections 2024-09-10T07:47:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7be40a2d80 0x7f7be40a5240 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c00105490 0x7f7c001a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c00102ae0 0x7f7c0019e7e0 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7bec035040 tx=0x7f7bec033d90 comp rx=0 tx=0).stop 2024-09-10T07:47:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 --2- 172.21.15.46:0/153138583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c00100190 0x7f7c0019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.338+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 >> 172.21.15.46:0/153138583 conn(0x7f7c000fbfa0 msgr2=0x7f7c00100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.340+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 shutdown_connections 2024-09-10T07:47:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:20.340+0000 7f7bff200640 1 -- 172.21.15.46:0/153138583 wait complete. 2024-09-10T07:47:20.504 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:20 smithi046 ceph-mon[30705]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.5 MiB/s wr, 740 op/s 2024-09-10T07:47:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 600.0 s, 378.6 tps, lat 269.082 ms stddev 363.380 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 -- 172.21.15.46:0/4284046752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409c6d0 msgr2=0x7fcb2409cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 --2- 172.21.15.46:0/4284046752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409c6d0 0x7fcb2409cab0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009930 tx=0x7fcb1802a420 comp rx=0 tx=0).stop 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 -- 172.21.15.46:0/4284046752 shutdown_connections 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 --2- 172.21.15.46:0/4284046752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb240a0f50 0x7fcb240a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 --2- 172.21.15.46:0/4284046752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb2409d080 0x7fcb240a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 --2- 172.21.15.46:0/4284046752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409c6d0 0x7fcb2409cab0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 -- 172.21.15.46:0/4284046752 >> 172.21.15.46:0/4284046752 conn(0x7fcb24009f40 msgr2=0x7fcb2400a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 -- 172.21.15.46:0/4284046752 shutdown_connections 2024-09-10T07:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.639+0000 7fcb2b200640 1 -- 172.21.15.46:0/4284046752 wait complete. 2024-09-10T07:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.641+0000 7fcb2b200640 1 Processor -- start 2024-09-10T07:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.641+0000 7fcb2b200640 1 -- start start 2024-09-10T07:47:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.642+0000 7fcb2b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409d080 0x7fcb241326b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.642+0000 7fcb2b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb240a0f50 0x7fcb24132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.642+0000 7fcb2b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24136f80 0x7fcb24137430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.642+0000 7fcb2b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb240ac020 con 0x7fcb2409d080 2024-09-10T07:47:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.642+0000 7fcb2b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb240abea0 con 0x7fcb24136f80 2024-09-10T07:47:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.642+0000 7fcb2b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb240ac1a0 con 0x7fcb240a0f50 2024-09-10T07:47:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.643+0000 7fcb29e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409d080 0x7fcb241326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.643+0000 7fcb29e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409d080 0x7fcb241326b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47494/0 (socket says 172.21.15.46:47494) 2024-09-10T07:47:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.643+0000 7fcb29e00640 1 -- 172.21.15.46:0/1882371392 learned_addr learned my addr 172.21.15.46:0/1882371392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.643+0000 7fcb2a800640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24136f80 0x7fcb24137430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.644+0000 7fcb29400640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb240a0f50 0x7fcb24132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.645+0000 7fcb29e00640 1 -- 172.21.15.46:0/1882371392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb240a0f50 msgr2=0x7fcb24132bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:20 smithi181 ceph-mon[35397]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.5 MiB/s wr, 740 op/s 2024-09-10T07:47:20.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:20 smithi063 ceph-mon[35020]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.5 MiB/s wr, 740 op/s 2024-09-10T07:47:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.645+0000 7fcb29e00640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb240a0f50 0x7fcb24132bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.645+0000 7fcb29e00640 1 -- 172.21.15.46:0/1882371392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24136f80 msgr2=0x7fcb24137430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.645+0000 7fcb29e00640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24136f80 0x7fcb24137430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.645+0000 7fcb29e00640 1 -- 172.21.15.46:0/1882371392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb24137a40 con 0x7fcb2409d080 2024-09-10T07:47:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.645+0000 7fcb2a800640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24136f80 0x7fcb24137430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.649+0000 7fcb29e00640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409d080 0x7fcb241326b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcb18006010 tx=0x7fcb18003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.650+0000 7fcb22a00640 1 -- 172.21.15.46:0/1882371392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1803f020 con 0x7fcb2409d080 2024-09-10T07:47:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.650+0000 7fcb22a00640 1 -- 172.21.15.46:0/1882371392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb18031ca0 con 0x7fcb2409d080 2024-09-10T07:47:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.650+0000 7fcb22a00640 1 -- 172.21.15.46:0/1882371392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18035070 con 0x7fcb2409d080 2024-09-10T07:47:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.650+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb24137cd0 con 0x7fcb2409d080 2024-09-10T07:47:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.650+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2413f400 con 0x7fcb2409d080 2024-09-10T07:47:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.652+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2409cab0 con 0x7fcb2409d080 2024-09-10T07:47:20.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.667+0000 7fcb22a00640 1 -- 172.21.15.46:0/1882371392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb18042020 con 0x7fcb2409d080 2024-09-10T07:47:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.671+0000 7fcb22a00640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb100a2ce0 0x7fcb100a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.671+0000 7fcb22a00640 1 -- 172.21.15.46:0/1882371392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcb180efd20 con 0x7fcb2409d080 2024-09-10T07:47:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.672+0000 7fcb22a00640 1 -- 172.21.15.46:0/1882371392 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb18049070 con 0x7fcb2409d080 2024-09-10T07:47:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.672+0000 7fcb29400640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb100a2ce0 0x7fcb100a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:20.676+0000 7fcb29400640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb100a2ce0 0x7fcb100a51a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c067c70 tx=0x7fcb2c0506b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.093+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb2413f720 con 0x7fcb2409d080 2024-09-10T07:47:21.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.094+0000 7fcb22a00640 1 -- 172.21.15.46:0/1882371392 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7fcb180bc980 con 0x7fcb2409d080 2024-09-10T07:47:21.096 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:47:21.096 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"btime":"2024-09-10T07:46:53:072255+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:46:52.230107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:47:21.096 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-09-10T07:47:21.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb100a2ce0 msgr2=0x7fcb100a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:21.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb100a2ce0 0x7fcb100a51a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c067c70 tx=0x7fcb2c0506b0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409d080 msgr2=0x7fcb241326b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:21.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409d080 0x7fcb241326b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcb18006010 tx=0x7fcb18003c10 comp rx=0 tx=0).stop 2024-09-10T07:47:21.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 shutdown_connections 2024-09-10T07:47:21.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb24136f80 0x7fcb24137430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb240a0f50 0x7fcb24132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcb100a2ce0 0x7fcb100a51a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 --2- 172.21.15.46:0/1882371392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb2409d080 0x7fcb241326b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.099+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 >> 172.21.15.46:0/1882371392 conn(0x7fcb24009f40 msgr2=0x7fcb2409a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:21.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.100+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 shutdown_connections 2024-09-10T07:47:21.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:21.100+0000 7fcb2b200640 1 -- 172.21.15.46:0/1882371392 wait complete. 2024-09-10T07:47:21.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:21 smithi046 ceph-mon[30705]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 907 op/s 2024-09-10T07:47:21.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:21 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1882371392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:21.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:21 smithi181 ceph-mon[35397]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 907 op/s 2024-09-10T07:47:21.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:21 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1882371392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:21.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:21 smithi063 ceph-mon[35020]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 907 op/s 2024-09-10T07:47:21.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:21 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1882371392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:22.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:47:23.659 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:24.129 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:23 smithi046 ceph-mon[30705]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 901 op/s 2024-09-10T07:47:24.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:23 smithi181 ceph-mon[35397]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 901 op/s 2024-09-10T07:47:24.149 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:23 smithi063 ceph-mon[35020]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 901 op/s 2024-09-10T07:47:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.836+0000 7fded0a00640 1 -- 172.21.15.46:0/3355142763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc11b140 msgr2=0x7fdecc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.836+0000 7fded0a00640 1 --2- 172.21.15.46:0/3355142763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc11b140 0x7fdecc11d530 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009900 tx=0x7fdeb802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.836+0000 7fded0a00640 1 -- 172.21.15.46:0/3355142763 shutdown_connections 2024-09-10T07:47:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.836+0000 7fded0a00640 1 --2- 172.21.15.46:0/3355142763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc11b140 0x7fdecc11d530 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.836+0000 7fded0a00640 1 --2- 172.21.15.46:0/3355142763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdecc06ea30 0x7fdecc06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.836+0000 7fded0a00640 1 --2- 172.21.15.46:0/3355142763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc110190 0x7fdecc06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.836+0000 7fded0a00640 1 -- 172.21.15.46:0/3355142763 >> 172.21.15.46:0/3355142763 conn(0x7fdecc073180 msgr2=0x7fdecc073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.837+0000 7fded0a00640 1 -- 172.21.15.46:0/3355142763 shutdown_connections 2024-09-10T07:47:24.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.837+0000 7fded0a00640 1 -- 172.21.15.46:0/3355142763 wait complete. 2024-09-10T07:47:24.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.838+0000 7fded0a00640 1 Processor -- start 2024-09-10T07:47:24.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.839+0000 7fded0a00640 1 -- start start 2024-09-10T07:47:24.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.839+0000 7fded0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc06ea30 0x7fdecc1a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.839+0000 7fded0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdecc110190 0x7fdecc1a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.839+0000 7fded0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc11b140 0x7fdecc1ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fdecb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc06ea30 0x7fdecc1a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fdecb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc06ea30 0x7fdecc1a6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48556/0 (socket says 172.21.15.46:48556) 2024-09-10T07:47:24.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fdecaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdecc110190 0x7fdecc1a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fdecaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdecc110190 0x7fdecc1a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35702/0 (socket says 172.21.15.46:35702) 2024-09-10T07:47:24.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fded0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc120be0 con 0x7fdecc11b140 2024-09-10T07:47:24.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fded0a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc120a60 con 0x7fdecc110190 2024-09-10T07:47:24.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fded0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc120d60 con 0x7fdecc06ea30 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc11b140 0x7fdecc1ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.840+0000 7fdecb400640 1 -- 172.21.15.46:0/386832772 learned_addr learned my addr 172.21.15.46:0/386832772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.841+0000 7fdecb400640 1 -- 172.21.15.46:0/386832772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdecc110190 msgr2=0x7fdecc1a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.841+0000 7fdecb400640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdecc110190 0x7fdecc1a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.841+0000 7fdecb400640 1 -- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc11b140 msgr2=0x7fdecc1ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.841+0000 7fdecb400640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc11b140 0x7fdecc1ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.841+0000 7fdecb400640 1 -- 172.21.15.46:0/386832772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc1abee0 con 0x7fdecc06ea30 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.841+0000 7fdecbe00640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc11b140 0x7fdecc1ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.842+0000 7fdecb400640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc06ea30 0x7fdecc1a6f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00c920 tx=0x7fdebc00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.842+0000 7fdec3e00640 1 -- 172.21.15.46:0/386832772 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc007c70 con 0x7fdecc06ea30 2024-09-10T07:47:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.842+0000 7fded0a00640 1 -- 172.21.15.46:0/386832772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc1ac1d0 con 0x7fdecc06ea30 2024-09-10T07:47:24.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.843+0000 7fdec3e00640 1 -- 172.21.15.46:0/386832772 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdebc007e10 con 0x7fdecc06ea30 2024-09-10T07:47:24.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.843+0000 7fded0a00640 1 -- 172.21.15.46:0/386832772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc1b3ab0 con 0x7fdecc06ea30 2024-09-10T07:47:24.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.843+0000 7fdec3e00640 1 -- 172.21.15.46:0/386832772 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc018890 con 0x7fdecc06ea30 2024-09-10T07:47:24.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.847+0000 7fded4c45740 1 Processor -- start 2024-09-10T07:47:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.847+0000 7fded4c45740 1 -- start start 2024-09-10T07:47:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fded4c45740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94ce40 0x56173a94d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fded4c45740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94d760 0x56173a957ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fded4c45740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a958020 0x56173a95a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fded4c45740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56173a8b33e0 con 0x56173a94ce40 2024-09-10T07:47:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fded4c45740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56173a85ad80 con 0x56173a958020 2024-09-10T07:47:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fded4c45740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56173a8af3a0 con 0x56173a94d760 2024-09-10T07:47:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fdec3e00640 1 -- 172.21.15.46:0/386832772 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdebc018b30 con 0x7fdecc06ea30 2024-09-10T07:47:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fdecaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a958020 0x56173a95a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fdecaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a958020 0x56173a95a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35710/0 (socket says 172.21.15.46:35710) 2024-09-10T07:47:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fdecaa00640 1 -- 172.21.15.46:0/127133777 learned_addr learned my addr 172.21.15.46:0/127133777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.848+0000 7fdecb400640 1 --2- 172.21.15.46:0/127133777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94d760 0x56173a957ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.849+0000 7fdecaa00640 1 -- 172.21.15.46:0/127133777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94d760 msgr2=0x56173a957ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.849+0000 7fdecaa00640 1 --2- 172.21.15.46:0/127133777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94d760 0x56173a957ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.849+0000 7fdecaa00640 1 -- 172.21.15.46:0/127133777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94ce40 msgr2=0x56173a94d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.850+0000 7fdec3e00640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdeb00a2c80 0x7fdeb00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.851+0000 7fdec3e00640 1 -- 172.21.15.46:0/386832772 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdebc0cf380 con 0x7fdecc06ea30 2024-09-10T07:47:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.851+0000 7fdecaa00640 1 --2- 172.21.15.46:0/127133777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94ce40 0x56173a94d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.851+0000 7fdecaa00640 1 -- 172.21.15.46:0/127133777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173a879ae0 con 0x56173a958020 2024-09-10T07:47:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.853+0000 7fdecaa00640 1 --2- 172.21.15.46:0/127133777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a958020 0x56173a95a410 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdec4007f60 tx=0x7fdec4007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a84314befaee1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.853+0000 7fdeb7e00640 1 -- 172.21.15.46:0/127133777 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4004d20 con 0x56173a958020 2024-09-10T07:47:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.853+0000 7fdeb7e00640 1 -- 172.21.15.46:0/127133777 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec4010be0 con 0x56173a958020 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.853+0000 7fdeb7e00640 1 -- 172.21.15.46:0/127133777 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4005790 con 0x56173a958020 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.854+0000 7fdecb400640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdeb00a2c80 0x7fdeb00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.854+0000 7fded4c45740 1 -- 172.21.15.46:0/127133777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a958020 msgr2=0x56173a95a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.854+0000 7fded4c45740 1 --2- 172.21.15.46:0/127133777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a958020 0x56173a95a410 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdec4007f60 tx=0x7fdec4007f90 comp rx=0 tx=0).stop 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.855+0000 7fded4c45740 1 -- 172.21.15.46:0/127133777 shutdown_connections 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.855+0000 7fded4c45740 1 --2- 172.21.15.46:0/127133777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a958020 0x56173a95a410 secure :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdec4007f60 tx=0x7fdec4007f90 comp rx=0 tx=0).stop 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.855+0000 7fded4c45740 1 --2- 172.21.15.46:0/127133777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94d760 0x56173a957ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.855+0000 7fded4c45740 1 --2- 172.21.15.46:0/127133777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94ce40 0x56173a94d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.855+0000 7fded4c45740 1 -- 172.21.15.46:0/127133777 >> 172.21.15.46:0/127133777 conn(0x56173a9493d0 msgr2=0x56173a9497b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.857+0000 7fdecb400640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdeb00a2c80 0x7fdeb00a5140 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdebc09b790 tx=0x7fdebc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.857+0000 7fded4c45740 1 -- 172.21.15.46:0/127133777 shutdown_connections 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.857+0000 7fded4c45740 1 -- 172.21.15.46:0/127133777 wait complete. 2024-09-10T07:47:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.858+0000 7fded4c45740 1 -- start start 2024-09-10T07:47:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.858+0000 7fded4c45740 1 Processor -- start 2024-09-10T07:47:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.859+0000 7fded4c45740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a94ce40 0x56173a955140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fded4c45740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94d760 0x56173a955680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a94ce40 0x56173a955140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a94ce40 0x56173a955140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:35712/0 (socket says 172.21.15.46:35712) 2024-09-10T07:47:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fdecbe00640 1 -- 172.21.15.46:0/1619052132 learned_addr learned my addr 172.21.15.46:0/1619052132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fded4c45740 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a955bc0 0x56173a9640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fdecaa00640 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94d760 0x56173a955680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56173a95abf0 con 0x56173a94d760 2024-09-10T07:47:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56173a86e0f0 con 0x56173a94ce40 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.860+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56173a95ad70 con 0x56173a955bc0 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.861+0000 7fded4c45740 10 client.0 _ll_register_callbacks cb 0x56173a855bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.861+0000 7fdecaa00640 1 -- 172.21.15.46:0/1619052132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a955bc0 msgr2=0x56173a9640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.861+0000 7fdecaa00640 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a955bc0 0x56173a9640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.861+0000 7fdecaa00640 1 -- 172.21.15.46:0/1619052132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a94ce40 msgr2=0x56173a955140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.861+0000 7fdecaa00640 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a94ce40 0x56173a955140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.861+0000 7fdecaa00640 1 -- 172.21.15.46:0/1619052132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173a6243c0 con 0x56173a94d760 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.862+0000 7fdecaa00640 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94d760 0x56173a955680 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x56173a94a480 tx=0x7fdec40051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a59cb0270d94fae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.862+0000 7fdeb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.862+0000 7fdeb4c00640 1 -- 172.21.15.46:0/1619052132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4026020 con 0x56173a94d760 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.862+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdec4010be0 con 0x56173a94d760 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.862+0000 7fdeb4c00640 1 -- 172.21.15.46:0/1619052132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdec4005710 con 0x56173a94d760 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.862+0000 7fdeb4c00640 1 -- 172.21.15.46:0/1619052132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdec4031050 con 0x56173a94d760 2024-09-10T07:47:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fded4c45740 10 client.17367 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56173a967280 con 0x56173a94d760 2024-09-10T07:47:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fdeb4c00640 1 -- 172.21.15.46:0/1619052132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec4010be0 con 0x56173a94d760 2024-09-10T07:47:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fdeb4c00640 1 -- 172.21.15.46:0/1619052132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fdec4022670 con 0x56173a94d760 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fded4c45740 10 client.17367 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fded4c45740 10 client.17367 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fded4c45740 20 client.17367 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fded4c45740 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56173a968690 0x56173a96aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fded4c45740 4 client.17367 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.863+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56173a8c0950 con 0x56173a968690 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.864+0000 7fdecbe00640 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56173a968690 0x56173a96aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.864+0000 7fdecbe00640 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56173a968690 0x56173a96aa80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aaa45836eca86120 server_cookie=8686c35cd8ef23c5 in_seq=0 out_seq=0 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.864+0000 7fdeb4c00640 10 client.17367 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:47:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.865+0000 7fdeb4c00640 1 -- 172.21.15.46:0/1619052132 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56173a8c0950 con 0x56173a968690 2024-09-10T07:47:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.865+0000 7fdeb4c00640 10 client.17367 handle_command_reply: tid=0 2024-09-10T07:47:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.865+0000 7fded4c45740 1 client.17367 shutdown 2024-09-10T07:47:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94d760 msgr2=0x56173a955680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fded4c45740 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94d760 0x56173a955680 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x56173a94a480 tx=0x7fdec40051a0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 shutdown_connections 2024-09-10T07:47:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fded4c45740 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56173a968690 0x56173a96aa80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fdeb4c00640 0 client.17367 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:47:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fded4c45740 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a955bc0 0x56173a9640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fded4c45740 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a94d760 0x56173a955680 secure :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0x56173a94a480 tx=0x7fdec40051a0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fded4c45740 1 --2- 172.21.15.46:0/1619052132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a94ce40 0x56173a955140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.866+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 >> 172.21.15.46:0/1619052132 conn(0x56173a9493d0 msgr2=0x56173a94f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.867+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 shutdown_connections 2024-09-10T07:47:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.867+0000 7fded4c45740 1 -- 172.21.15.46:0/1619052132 wait complete. 2024-09-10T07:47:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.868+0000 7fded4c45740 20 client.17367 trim_cache size 0 max 16384 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.896+0000 7fded4c45740 1 Processor -- start 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.896+0000 7fded4c45740 1 -- start start 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.896+0000 7fded4c45740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a963ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.896+0000 7fded4c45740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a9483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.896+0000 7fded4c45740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 0x56173a970130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.896+0000 7fded4c45740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac00fb30 con 0x56173a955bc0 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.896+0000 7fded4c45740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac011670 con 0x56173a954230 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.896+0000 7fded4c45740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac010200 con 0x56173a94ce40 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a963ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a963ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48592/0 (socket says 172.21.15.46:48592) 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a9483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 -- 172.21.15.46:0/583988760 learned_addr learned my addr 172.21.15.46:0/583988760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 0x56173a970130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 -- 172.21.15.46:0/583988760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 msgr2=0x56173a970130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 0x56173a970130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 -- 172.21.15.46:0/583988760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 msgr2=0x56173a9483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a9483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecbe00640 1 -- 172.21.15.46:0/583988760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173a94a060 con 0x56173a94ce40 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.897+0000 7fdecaa00640 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a9483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.898+0000 7fdecbe00640 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a963ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdeb802a910 tx=0x7fdeb8005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4642c25479867d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.899+0000 7fdeb7400640 1 -- 172.21.15.46:0/583988760 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb804a070 con 0x56173a94ce40 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.899+0000 7fdeb7400640 1 -- 172.21.15.46:0/583988760 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb80053c0 con 0x56173a94ce40 2024-09-10T07:47:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.899+0000 7fdeb7400640 1 -- 172.21.15.46:0/583988760 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8035040 con 0x56173a94ce40 2024-09-10T07:47:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.900+0000 7fded4c45740 1 -- 172.21.15.46:0/583988760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 msgr2=0x56173a963ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.900+0000 7fded4c45740 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a963ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdeb802a910 tx=0x7fdeb8005e50 comp rx=0 tx=0).stop 2024-09-10T07:47:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.900+0000 7fded4c45740 1 -- 172.21.15.46:0/583988760 shutdown_connections 2024-09-10T07:47:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.900+0000 7fded4c45740 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 0x56173a970130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.900+0000 7fded4c45740 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a9483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.900+0000 7fded4c45740 1 --2- 172.21.15.46:0/583988760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a963ee0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.900+0000 7fded4c45740 1 -- 172.21.15.46:0/583988760 >> 172.21.15.46:0/583988760 conn(0x56173a9493d0 msgr2=0x56173a947c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.901+0000 7fded4c45740 1 -- 172.21.15.46:0/583988760 shutdown_connections 2024-09-10T07:47:24.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.901+0000 7fded4c45740 1 -- 172.21.15.46:0/583988760 wait complete. 2024-09-10T07:47:24.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.902+0000 7fded4c45740 1 -- start start 2024-09-10T07:47:24.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.902+0000 7fded4c45740 1 Processor -- start 2024-09-10T07:47:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fded4c45740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a95bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fded4c45740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 0x56173a95c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fded4c45740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a95c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fded4c45740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac010fa0 con 0x56173a955bc0 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fded4c45740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac0108d0 con 0x56173a954230 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fded4c45740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac00db10 con 0x56173a94ce40 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fded4c45740 10 client.0 _ll_register_callbacks cb 0x7fdebc0d2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a95bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.903+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a95c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.904+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a95bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48598/0 (socket says 172.21.15.46:48598) 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.904+0000 7fdecbe00640 1 -- 172.21.15.46:0/816154000 learned_addr learned my addr 172.21.15.46:0/816154000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.904+0000 7fdecbe00640 1 -- 172.21.15.46:0/816154000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 msgr2=0x56173a95c4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.904+0000 7fdecbe00640 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 0x56173a95c4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.904+0000 7fdecbe00640 1 -- 172.21.15.46:0/816154000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 msgr2=0x56173a95c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:24.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.904+0000 7fdecbe00640 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a95c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.904+0000 7fdecbe00640 1 -- 172.21.15.46:0/816154000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173a9d2270 con 0x56173a94ce40 2024-09-10T07:47:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.904+0000 7fdecbe00640 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a95bf60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80052b0 tx=0x7fdeb803e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38757cb870c9340b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.905+0000 7fdeb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:47:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.905+0000 7fdeb4c00640 1 -- 172.21.15.46:0/816154000 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8035070 con 0x56173a94ce40 2024-09-10T07:47:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.905+0000 7fded4c45740 1 -- 172.21.15.46:0/816154000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb80053c0 con 0x56173a94ce40 2024-09-10T07:47:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.905+0000 7fdeb4c00640 1 -- 172.21.15.46:0/816154000 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdeb80048c0 con 0x56173a94ce40 2024-09-10T07:47:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.905+0000 7fdeb4c00640 1 -- 172.21.15.46:0/816154000 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdeb804a070 con 0x56173a94ce40 2024-09-10T07:47:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.905+0000 7fded4c45740 10 client.27994 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.905+0000 7fded4c45740 1 -- 172.21.15.46:0/816154000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56173a9d3530 con 0x56173a94ce40 2024-09-10T07:47:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.907+0000 7fdeb4c00640 1 -- 172.21.15.46:0/816154000 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb80053c0 con 0x56173a94ce40 2024-09-10T07:47:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.908+0000 7fdeb4c00640 1 -- 172.21.15.46:0/816154000 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fdeb80035f0 con 0x56173a94ce40 2024-09-10T07:47:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.908+0000 7fded4c45740 10 client.27994 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.908+0000 7fded4c45740 10 client.27994 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:47:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.908+0000 7fded4c45740 20 client.27994 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.908+0000 7fded4c45740 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56173a9d3b50 0x56173a9d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.909+0000 7fded4c45740 4 client.27994 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:47:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.909+0000 7fded4c45740 1 -- 172.21.15.46:0/816154000 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56173a8bae60 con 0x56173a9d3b50 2024-09-10T07:47:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.909+0000 7fdecaa00640 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56173a9d3b50 0x56173a9d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.910+0000 7fdecaa00640 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56173a9d3b50 0x56173a9d5f40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f95d39cec9a3d16 server_cookie=62c63939fc96e654 in_seq=0 out_seq=0 2024-09-10T07:47:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.910+0000 7fdeb4c00640 10 client.27994 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:47:24.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fdeb4c00640 1 -- 172.21.15.46:0/816154000 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56173a8bae60 con 0x56173a9d3b50 2024-09-10T07:47:24.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fdeb4c00640 10 client.27994 handle_command_reply: tid=0 2024-09-10T07:47:24.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 client.27994 shutdown 2024-09-10T07:47:24.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 -- 172.21.15.46:0/816154000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 msgr2=0x56173a95bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a95bf60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80052b0 tx=0x7fdeb803e040 comp rx=0 tx=0).stop 2024-09-10T07:47:24.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 -- 172.21.15.46:0/816154000 shutdown_connections 2024-09-10T07:47:24.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56173a9d3b50 0x56173a9d5f40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56173a955bc0 0x56173a95c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fdeb4c00640 0 client.27994 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:47:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56173a954230 0x56173a95c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 --2- 172.21.15.46:0/816154000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56173a94ce40 0x56173a95bf60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 -- 172.21.15.46:0/816154000 >> 172.21.15.46:0/816154000 conn(0x56173a9493d0 msgr2=0x56173a9484b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 -- 172.21.15.46:0/816154000 shutdown_connections 2024-09-10T07:47:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.911+0000 7fded4c45740 1 -- 172.21.15.46:0/816154000 wait complete. 2024-09-10T07:47:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded4c45740 20 client.27994 trim_cache size 0 max 16384 2024-09-10T07:47:24.914 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:47:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 -- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdeb00a2c80 msgr2=0x7fdeb00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdeb00a2c80 0x7fdeb00a5140 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdebc09b790 tx=0x7fdebc000bc0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 -- 172.21.15.46:0/386832772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc06ea30 msgr2=0x7fdecc1a6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc06ea30 0x7fdecc1a6f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00c920 tx=0x7fdebc00cde0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 -- 172.21.15.46:0/386832772 shutdown_connections 2024-09-10T07:47:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdeb00a2c80 0x7fdeb00a5140 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc11b140 0x7fdecc1ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdecc110190 0x7fdecc1a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 --2- 172.21.15.46:0/386832772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdecc06ea30 0x7fdecc1a6f30 secure :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00c920 tx=0x7fdebc00cde0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.912+0000 7fded0a00640 1 -- 172.21.15.46:0/386832772 >> 172.21.15.46:0/386832772 conn(0x7fdecc073180 msgr2=0x7fdecc1191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.913+0000 7fded0a00640 1 -- 172.21.15.46:0/386832772 shutdown_connections 2024-09-10T07:47:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:24.913+0000 7fded0a00640 1 -- 172.21.15.46:0/386832772 wait complete. 2024-09-10T07:47:25.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 605.0 s, 519.8 tps, lat 194.214 ms stddev 234.903 2024-09-10T07:47:26.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:47:26.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:25 smithi046 ceph-mon[30705]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T07:47:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:25 smithi181 ceph-mon[35397]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T07:47:26.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:25 smithi063 ceph-mon[35020]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T07:47:26.979 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-09-10T07:47:26.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:47:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:28.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:27 smithi046 ceph-mon[30705]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-09-10T07:47:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.376+0000 7f1430c00640 1 -- 172.21.15.46:0/1494399676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f142c103e30 msgr2=0x7f142c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.376+0000 7f1430c00640 1 --2- 172.21.15.46:0/1494399676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f142c103e30 0x7f142c104210 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f141c009900 tx=0x7f141c02a410 comp rx=0 tx=0).stop 2024-09-10T07:47:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.376+0000 7f1430c00640 1 -- 172.21.15.46:0/1494399676 shutdown_connections 2024-09-10T07:47:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.376+0000 7f1430c00640 1 --2- 172.21.15.46:0/1494399676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c108f60 0x7f142c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.376+0000 7f1430c00640 1 --2- 172.21.15.46:0/1494399676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f142c1047e0 0x7f142c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.376+0000 7f1430c00640 1 --2- 172.21.15.46:0/1494399676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f142c103e30 0x7f142c104210 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.376+0000 7f1430c00640 1 -- 172.21.15.46:0/1494399676 >> 172.21.15.46:0/1494399676 conn(0x7f142c0ff910 msgr2=0x7f142c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.377+0000 7f1430c00640 1 -- 172.21.15.46:0/1494399676 shutdown_connections 2024-09-10T07:47:28.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.378+0000 7f1430c00640 1 -- 172.21.15.46:0/1494399676 wait complete. 2024-09-10T07:47:28.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.379+0000 7f1430c00640 1 Processor -- start 2024-09-10T07:47:28.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.382+0000 7f1430c00640 1 -- start start 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.382+0000 7f1430c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f142c103e30 0x7f142c110970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.382+0000 7f1430c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c1047e0 0x7f142c10da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.382+0000 7f1430c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f142c108f60 0x7f142c10dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.382+0000 7f1430c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f142c113fd0 con 0x7f142c103e30 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.382+0000 7f1430c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f142c113e50 con 0x7f142c108f60 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.382+0000 7f1430c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f142c114150 con 0x7f142c1047e0 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.383+0000 7f142be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f142c108f60 0x7f142c10dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.383+0000 7f142be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f142c108f60 0x7f142c10dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48908/0 (socket says 172.21.15.46:48908) 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.383+0000 7f142be00640 1 -- 172.21.15.46:0/2921597228 learned_addr learned my addr 172.21.15.46:0/2921597228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:28.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.383+0000 7f142be00640 1 -- 172.21.15.46:0/2921597228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c1047e0 msgr2=0x7f142c10da70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.383+0000 7f142be00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c1047e0 0x7f142c10da70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.383+0000 7f142be00640 1 -- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f142c103e30 msgr2=0x7f142c110970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.383+0000 7f142be00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f142c103e30 0x7f142c110970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.383+0000 7f142be00640 1 -- 172.21.15.46:0/2921597228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c1a24e0 con 0x7f142c108f60 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.384+0000 7f142be00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f142c108f60 0x7f142c10dfb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f142400eb10 tx=0x7f142400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.384+0000 7f1423e00640 1 -- 172.21.15.46:0/2921597228 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424004b20 con 0x7f142c108f60 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.384+0000 7f1423e00640 1 -- 172.21.15.46:0/2921597228 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1424004cc0 con 0x7f142c108f60 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.384+0000 7f1423e00640 1 -- 172.21.15.46:0/2921597228 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14240107b0 con 0x7f142c108f60 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.384+0000 7f1430c00640 1 -- 172.21.15.46:0/2921597228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142c1a2770 con 0x7f142c108f60 2024-09-10T07:47:28.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.384+0000 7f1430c00640 1 -- 172.21.15.46:0/2921597228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142c1a2c60 con 0x7f142c108f60 2024-09-10T07:47:28.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.393+0000 7f1434e3b740 1 Processor -- start 2024-09-10T07:47:28.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.393+0000 7f1423e00640 1 -- 172.21.15.46:0/2921597228 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1424005020 con 0x7f142c108f60 2024-09-10T07:47:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.395+0000 7f1423e00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f14100a2cd0 0x7f14100a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.396+0000 7f1434e3b740 1 -- start start 2024-09-10T07:47:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.396+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.396+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 0x556e6e1abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.396+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.396+0000 7f1423e00640 1 -- 172.21.15.46:0/2921597228 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1424099f70 con 0x7f142c108f60 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.397+0000 7f142aa00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f14100a2cd0 0x7f14100a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.397+0000 7f1434e3b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556e6e1073e0 con 0x556e6e1ac020 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.397+0000 7f1434e3b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e6e0aed80 con 0x556e6e1a1760 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.397+0000 7f1434e3b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e6e1033a0 con 0x556e6e1a0e40 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.397+0000 7f142be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.397+0000 7f142be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55322/0 (socket says 172.21.15.46:55322) 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.397+0000 7f142be00640 1 -- 172.21.15.46:0/298195604 learned_addr learned my addr 172.21.15.46:0/298195604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.398+0000 7f142b400640 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.398+0000 7f142be00640 1 -- 172.21.15.46:0/298195604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 msgr2=0x556e6e1a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.398+0000 7f142be00640 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.398+0000 7f142be00640 1 -- 172.21.15.46:0/298195604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 msgr2=0x556e6e1abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.398+0000 7f142aa00640 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 0x556e6e1abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.398+0000 7f142be00640 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 0x556e6e1abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.398+0000 7f142be00640 1 -- 172.21.15.46:0/298195604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6e0cdae0 con 0x556e6e1ac020 2024-09-10T07:47:28.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:27 smithi063 ceph-mon[35020]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-09-10T07:47:28.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:27 smithi181 ceph-mon[35397]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-09-10T07:47:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.400+0000 7f142be00640 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1ae410 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f14240a4610 tx=0x7f14240a3590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c353254e0247758 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.400+0000 7f1417e00640 1 -- 172.21.15.46:0/298195604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424098900 con 0x556e6e1ac020 2024-09-10T07:47:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.401+0000 7f1417e00640 1 -- 172.21.15.46:0/298195604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142409c5d0 con 0x556e6e1ac020 2024-09-10T07:47:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.401+0000 7f1417e00640 1 -- 172.21.15.46:0/298195604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424098aa0 con 0x556e6e1ac020 2024-09-10T07:47:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.402+0000 7f1434e3b740 1 -- 172.21.15.46:0/298195604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 msgr2=0x556e6e1ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.402+0000 7f1434e3b740 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1ae410 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f14240a4610 tx=0x7f14240a3590 comp rx=0 tx=0).stop 2024-09-10T07:47:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.403+0000 7f1434e3b740 1 -- 172.21.15.46:0/298195604 shutdown_connections 2024-09-10T07:47:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.403+0000 7f1434e3b740 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1ae410 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.403+0000 7f1434e3b740 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 0x556e6e1abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.403+0000 7f1434e3b740 1 --2- 172.21.15.46:0/298195604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.403+0000 7f1434e3b740 1 -- 172.21.15.46:0/298195604 >> 172.21.15.46:0/298195604 conn(0x556e6e19d3d0 msgr2=0x556e6e19d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.404+0000 7f142aa00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f14100a2cd0 0x7f14100a5190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f142c10f1e0 tx=0x7f1418009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.404+0000 7f1434e3b740 1 -- 172.21.15.46:0/298195604 shutdown_connections 2024-09-10T07:47:28.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.405+0000 7f1434e3b740 1 -- 172.21.15.46:0/298195604 wait complete. 2024-09-10T07:47:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.406+0000 7f1434e3b740 1 -- start start 2024-09-10T07:47:28.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.408+0000 7f1434e3b740 1 Processor -- start 2024-09-10T07:47:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.410+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1a0e40 0x556e6e1a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.412+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.412+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1ac020 0x556e6e1a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.412+0000 7f142aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.412+0000 7f142aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34870/0 (socket says 172.21.15.46:34870) 2024-09-10T07:47:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.412+0000 7f142b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1a0e40 0x556e6e1a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.413+0000 7f142b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1a0e40 0x556e6e1a8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55330/0 (socket says 172.21.15.46:55330) 2024-09-10T07:47:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.413+0000 7f142be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1ac020 0x556e6e1a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.414+0000 7f1434e3b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556e6e1aeb60 con 0x556e6e1a0e40 2024-09-10T07:47:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.414+0000 7f1434e3b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e6e0c20f0 con 0x556e6e1ac020 2024-09-10T07:47:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.414+0000 7f1434e3b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e6e1aece0 con 0x556e6e1a1760 2024-09-10T07:47:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.414+0000 7f142be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1ac020 0x556e6e1a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48928/0 (socket says 172.21.15.46:48928) 2024-09-10T07:47:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.414+0000 7f142be00640 1 -- 172.21.15.46:0/1893026854 learned_addr learned my addr 172.21.15.46:0/1893026854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:28.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.414+0000 7f1434e3b740 10 client.0 _ll_register_callbacks cb 0x556e6e0a9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.418+0000 7f142b400640 1 -- 172.21.15.46:0/1893026854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 msgr2=0x556e6e1a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.418+0000 7f142b400640 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.418+0000 7f142b400640 1 -- 172.21.15.46:0/1893026854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1ac020 msgr2=0x556e6e1a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.418+0000 7f142b400640 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1ac020 0x556e6e1a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.418+0000 7f142b400640 1 -- 172.21.15.46:0/1893026854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6de783c0 con 0x556e6e1a0e40 2024-09-10T07:47:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.418+0000 7f142aa00640 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.419+0000 7f142b400640 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1a0e40 0x556e6e1a8f30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f141c02a930 tx=0x7f141c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cdc1aad59692a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.419+0000 7f1414c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:28.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.420+0000 7f1414c00640 1 -- 172.21.15.46:0/1893026854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c050030 con 0x556e6e1a0e40 2024-09-10T07:47:28.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.420+0000 7f1414c00640 1 -- 172.21.15.46:0/1893026854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f141c005570 con 0x556e6e1a0e40 2024-09-10T07:47:28.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.421+0000 7f1434e3b740 1 -- 172.21.15.46:0/1893026854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f142409c5d0 con 0x556e6e1a0e40 2024-09-10T07:47:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.421+0000 7f1414c00640 1 -- 172.21.15.46:0/1893026854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f141c035070 con 0x556e6e1a0e40 2024-09-10T07:47:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.421+0000 7f1434e3b740 10 client.17382 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.421+0000 7f1434e3b740 1 -- 172.21.15.46:0/1893026854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556e6e1babb0 con 0x556e6e1a0e40 2024-09-10T07:47:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.422+0000 7f1414c00640 1 -- 172.21.15.46:0/1893026854 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142409c5d0 con 0x556e6e1a0e40 2024-09-10T07:47:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.422+0000 7f1414c00640 1 -- 172.21.15.46:0/1893026854 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f141c04a040 con 0x556e6e1a0e40 2024-09-10T07:47:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.422+0000 7f1434e3b740 10 client.17382 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.422+0000 7f1434e3b740 10 client.17382 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:47:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.422+0000 7f1434e3b740 20 client.17382 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.422+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x556e6e1bc140 0x556e6e1be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.422+0000 7f1434e3b740 4 client.17382 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.422+0000 7f1434e3b740 1 -- 172.21.15.46:0/1893026854 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e6e114950 con 0x556e6e1bc140 2024-09-10T07:47:28.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.424+0000 7f142aa00640 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x556e6e1bc140 0x556e6e1be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.425+0000 7f142aa00640 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x556e6e1bc140 0x556e6e1be5e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62f86ff4bd1c27f0 server_cookie=2e73994e94de2f23 in_seq=0 out_seq=0 2024-09-10T07:47:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.425+0000 7f1414c00640 10 client.17382 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:47:28.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.426+0000 7f1414c00640 1 -- 172.21.15.46:0/1893026854 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e6e114950 con 0x556e6e1bc140 2024-09-10T07:47:28.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.426+0000 7f1414c00640 10 client.17382 handle_command_reply: tid=0 2024-09-10T07:47:28.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.426+0000 7f1434e3b740 1 client.17382 shutdown 2024-09-10T07:47:28.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.426+0000 7f1434e3b740 1 -- 172.21.15.46:0/1893026854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1a0e40 msgr2=0x556e6e1a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.426+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1a0e40 0x556e6e1a8f30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f141c02a930 tx=0x7f141c005270 comp rx=0 tx=0).stop 2024-09-10T07:47:28.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.428+0000 7f1434e3b740 1 -- 172.21.15.46:0/1893026854 shutdown_connections 2024-09-10T07:47:28.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.428+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1ac020 0x556e6e1a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.428+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x556e6e1bc140 0x556e6e1be5e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.428+0000 7f1414c00640 0 client.17382 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:47:28.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.428+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.428+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1893026854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1a0e40 0x556e6e1a8f30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.428+0000 7f1434e3b740 1 -- 172.21.15.46:0/1893026854 >> 172.21.15.46:0/1893026854 conn(0x556e6e19d3d0 msgr2=0x556e6e1a3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.429+0000 7f1434e3b740 1 -- 172.21.15.46:0/1893026854 shutdown_connections 2024-09-10T07:47:28.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.429+0000 7f1434e3b740 1 -- 172.21.15.46:0/1893026854 wait complete. 2024-09-10T07:47:28.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.429+0000 7f1434e3b740 20 client.17382 trim_cache size 0 max 16384 2024-09-10T07:47:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.466+0000 7f1434e3b740 1 Processor -- start 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.466+0000 7f1434e3b740 1 -- start start 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.466+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a0e40 0x556e6e198300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.466+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.466+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.466+0000 7f1434e3b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f142c1a41c0 con 0x556e6e1ac020 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.466+0000 7f1434e3b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f142c1a5d00 con 0x556e6e1a0e40 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.466+0000 7f1434e3b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f142c1a4890 con 0x556e6e1a1760 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.467+0000 7f142aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.467+0000 7f142aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34872/0 (socket says 172.21.15.46:34872) 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.467+0000 7f142aa00640 1 -- 172.21.15.46:0/1350469961 learned_addr learned my addr 172.21.15.46:0/1350469961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.467+0000 7f142b400640 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a0e40 0x556e6e198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.467+0000 7f142b400640 1 -- 172.21.15.46:0/1350469961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 msgr2=0x556e6e1c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.467+0000 7f142b400640 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.467+0000 7f142be00640 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.467+0000 7f142b400640 1 -- 172.21.15.46:0/1350469961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 msgr2=0x556e6e1c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.468+0000 7f142b400640 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.468+0000 7f142b400640 1 -- 172.21.15.46:0/1350469961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6e1a7ee0 con 0x556e6e1a0e40 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.468+0000 7f142b400640 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a0e40 0x556e6e198300 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f141c001eb0 tx=0x7f141c0097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87b5d1c344e7c21d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.469+0000 7f1417400640 1 -- 172.21.15.46:0/1350469961 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c050070 con 0x556e6e1a0e40 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.469+0000 7f1417400640 1 -- 172.21.15.46:0/1350469961 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c063020 con 0x556e6e1a0e40 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.469+0000 7f1417400640 1 -- 172.21.15.46:0/1350469961 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c004ba0 con 0x556e6e1a0e40 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.470+0000 7f1434e3b740 1 -- 172.21.15.46:0/1350469961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a0e40 msgr2=0x556e6e198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.470+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a0e40 0x556e6e198300 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f141c001eb0 tx=0x7f141c0097f0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.470+0000 7f1434e3b740 1 -- 172.21.15.46:0/1350469961 shutdown_connections 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.470+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e1c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.470+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a1760 0x556e6e1c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.470+0000 7f1434e3b740 1 --2- 172.21.15.46:0/1350469961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a0e40 0x556e6e198300 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.470+0000 7f1434e3b740 1 -- 172.21.15.46:0/1350469961 >> 172.21.15.46:0/1350469961 conn(0x556e6e19d3d0 msgr2=0x556e6e1a3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.471+0000 7f1434e3b740 1 -- 172.21.15.46:0/1350469961 shutdown_connections 2024-09-10T07:47:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.471+0000 7f1434e3b740 1 -- 172.21.15.46:0/1350469961 wait complete. 2024-09-10T07:47:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.471+0000 7f1434e3b740 1 -- start start 2024-09-10T07:47:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.472+0000 7f1434e3b740 1 Processor -- start 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.474+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.475+0000 7f1434e3b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 0x556e6e1b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.475+0000 7f142b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.475+0000 7f142b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34888/0 (socket says 172.21.15.46:34888) 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.475+0000 7f142b400640 1 -- 172.21.15.46:0/516499111 learned_addr learned my addr 172.21.15.46:0/516499111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.475+0000 7f1434e3b740 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e225ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.476+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f142c1a5630 con 0x556e6e1ac020 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.476+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f142c1a4f60 con 0x556e6e1a1760 2024-09-10T07:47:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.476+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f142c0736a0 con 0x556e6e1a0e40 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.476+0000 7f1434e3b740 10 client.0 _ll_register_callbacks cb 0x7f1418005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.477+0000 7f142be00640 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e225ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.477+0000 7f142b400640 1 -- 172.21.15.46:0/516499111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 msgr2=0x556e6e1b0540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.477+0000 7f142b400640 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 0x556e6e1b0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.477+0000 7f142b400640 1 -- 172.21.15.46:0/516499111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 msgr2=0x556e6e225ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.477+0000 7f142b400640 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e225ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.477+0000 7f142b400640 1 -- 172.21.15.46:0/516499111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6e1b0e20 con 0x556e6e1a0e40 2024-09-10T07:47:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.479+0000 7f142b400640 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1b0000 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f141c006010 tx=0x7f141c005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=190cf6725fdc2ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.479+0000 7f1414c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:47:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.479+0000 7f1414c00640 1 -- 172.21.15.46:0/516499111 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c06d060 con 0x556e6e1a0e40 2024-09-10T07:47:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.480+0000 7f1414c00640 1 -- 172.21.15.46:0/516499111 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f141c0048d0 con 0x556e6e1a0e40 2024-09-10T07:47:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.480+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141c063020 con 0x556e6e1a0e40 2024-09-10T07:47:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.480+0000 7f1414c00640 1 -- 172.21.15.46:0/516499111 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f141c050040 con 0x556e6e1a0e40 2024-09-10T07:47:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.480+0000 7f1434e3b740 10 client.28006 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.480+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556e6e2274f0 con 0x556e6e1a0e40 2024-09-10T07:47:28.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.481+0000 7f1414c00640 1 -- 172.21.15.46:0/516499111 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c063020 con 0x556e6e1a0e40 2024-09-10T07:47:28.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.481+0000 7f1414c00640 1 -- 172.21.15.46:0/516499111 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f141c04a070 con 0x556e6e1a0e40 2024-09-10T07:47:28.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.482+0000 7f1434e3b740 10 client.28006 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:28.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.482+0000 7f1434e3b740 10 client.28006 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:47:28.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.482+0000 7f1434e3b740 20 client.28006 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:28.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.482+0000 7f1434e3b740 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x556e6e227be0 0x556e6e22a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.482+0000 7f1434e3b740 4 client.28006 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:47:28.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.482+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e6e10ee60 con 0x556e6e227be0 2024-09-10T07:47:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.483+0000 7f142aa00640 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x556e6e227be0 0x556e6e22a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.486+0000 7f142aa00640 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x556e6e227be0 0x556e6e22a060 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42a352e4d6a2f38 server_cookie=55ec395bb7a40754 in_seq=0 out_seq=0 2024-09-10T07:47:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.486+0000 7f1414c00640 10 client.28006 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.486+0000 7f1414c00640 1 -- 172.21.15.46:0/516499111 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e6e10ee60 con 0x556e6e227be0 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.486+0000 7f1414c00640 10 client.28006 handle_command_reply: tid=0 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.486+0000 7f1434e3b740 1 client.28006 shutdown 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 msgr2=0x556e6e1b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1b0000 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f141c006010 tx=0x7f141c005b50 comp rx=0 tx=0).stop 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 shutdown_connections 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e6e1ac020 0x556e6e225ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6e1a1760 0x556e6e1b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x556e6e227be0 0x556e6e22a060 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 --2- 172.21.15.46:0/516499111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e6e1a0e40 0x556e6e1b0000 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 >> 172.21.15.46:0/516499111 conn(0x556e6e19d3d0 msgr2=0x556e6e19bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1414c00640 0 client.28006 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:47:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.487+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 shutdown_connections 2024-09-10T07:47:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.488+0000 7f1434e3b740 1 -- 172.21.15.46:0/516499111 wait complete. 2024-09-10T07:47:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.488+0000 7f1434e3b740 20 client.28006 trim_cache size 0 max 16384 2024-09-10T07:47:28.490 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:47:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 -- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f14100a2cd0 msgr2=0x7f14100a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f14100a2cd0 0x7f14100a5190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f142c10f1e0 tx=0x7f1418009650 comp rx=0 tx=0).stop 2024-09-10T07:47:28.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 -- 172.21.15.46:0/2921597228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f142c108f60 msgr2=0x7f142c10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f142c108f60 0x7f142c10dfb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f142400eb10 tx=0x7f142400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 -- 172.21.15.46:0/2921597228 shutdown_connections 2024-09-10T07:47:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f142c108f60 0x7f142c10dfb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c1047e0 0x7f142c10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f14100a2cd0 0x7f14100a5190 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 --2- 172.21.15.46:0/2921597228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f142c103e30 0x7f142c110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.490+0000 7f1430c00640 1 -- 172.21.15.46:0/2921597228 >> 172.21.15.46:0/2921597228 conn(0x7f142c0ff910 msgr2=0x7f142c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.493+0000 7f1430c00640 1 -- 172.21.15.46:0/2921597228 shutdown_connections 2024-09-10T07:47:28.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:28.493+0000 7f1430c00640 1 -- 172.21.15.46:0/2921597228 wait complete. 2024-09-10T07:47:30.198 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:29 smithi046 ceph-mon[30705]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 459 op/s 2024-09-10T07:47:30.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:47:30.305 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:29 smithi181 ceph-mon[35397]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 459 op/s 2024-09-10T07:47:30.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:29 smithi063 ceph-mon[35020]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 459 op/s 2024-09-10T07:47:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 610.0 s, 520.6 tps, lat 191.480 ms stddev 240.902 2024-09-10T07:47:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:47:31.602 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"btime":"2024-09-10T07:46:53:072255+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:46:52.230107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:47:31.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 -- 172.21.15.46:0/634396961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db810d530 msgr2=0x7f9db81150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 --2- 172.21.15.46:0/634396961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db810d530 0x7f9db81150b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009ca0 tx=0x7f9db002a8d0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 -- 172.21.15.46:0/634396961 shutdown_connections 2024-09-10T07:47:31.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 --2- 172.21.15.46:0/634396961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db810d530 0x7f9db81150b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 --2- 172.21.15.46:0/634396961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db8108e00 0x7f9db810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 --2- 172.21.15.46:0/634396961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db8108450 0x7f9db8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 -- 172.21.15.46:0/634396961 >> 172.21.15.46:0/634396961 conn(0x7f9db8077ba0 msgr2=0x7f9db8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 -- 172.21.15.46:0/634396961 shutdown_connections 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.123+0000 7f9dc0400640 1 -- 172.21.15.46:0/634396961 wait complete. 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.124+0000 7f9dc0400640 1 Processor -- start 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.124+0000 7f9dc0400640 1 -- start start 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dc0400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db8108450 0x7f9db8082730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dc0400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db8108e00 0x7f9db8082c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dc0400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db810d530 0x7f9db807b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dc0400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8118620 con 0x7f9db8108e00 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dc0400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9db81184a0 con 0x7f9db810d530 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dc0400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9db81187a0 con 0x7f9db8108450 2024-09-10T07:47:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dbe400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db810d530 0x7f9db807b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dbe400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db810d530 0x7f9db807b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48966/0 (socket says 172.21.15.46:48966) 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dbe400640 1 -- 172.21.15.46:0/3874696986 learned_addr learned my addr 172.21.15.46:0/3874696986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dbe400640 1 -- 172.21.15.46:0/3874696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db8108450 msgr2=0x7f9db8082730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dbd000640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db8108e00 0x7f9db8082c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.125+0000 7f9dbda00640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db8108450 0x7f9db8082730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dbe400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db8108450 0x7f9db8082730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dbe400640 1 -- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db8108e00 msgr2=0x7f9db8082c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dbe400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db8108e00 0x7f9db8082c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dbe400640 1 -- 172.21.15.46:0/3874696986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db807c1c0 con 0x7f9db810d530 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dbd000640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db8108e00 0x7f9db8082c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dbda00640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db8108450 0x7f9db8082730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dbe400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db810d530 0x7f9db807b900 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009430 tx=0x7f9db0003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9db6000640 1 -- 172.21.15.46:0/3874696986 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0031720 con 0x7f9db810d530 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9db6000640 1 -- 172.21.15.46:0/3874696986 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db00318c0 con 0x7f9db810d530 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9db6000640 1 -- 172.21.15.46:0/3874696986 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db002fb50 con 0x7f9db810d530 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db807c450 con 0x7f9db810d530 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.126+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db8080480 con 0x7f9db810d530 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.131+0000 7f9db6000640 1 -- 172.21.15.46:0/3874696986 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9db002fcf0 con 0x7f9db810d530 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.132+0000 7f9db6000640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9da00a2c80 0x7f9da00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.132+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db81098e0 con 0x7f9db810d530 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.133+0000 7f9db6000640 1 -- 172.21.15.46:0/3874696986 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9db00f0770 con 0x7f9db810d530 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.133+0000 7f9dbda00640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9da00a2c80 0x7f9da00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.134+0000 7f9dbda00640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9da00a2c80 0x7f9da00a5140 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00a8b0 tx=0x7f9dac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.138+0000 7f9db6000640 1 -- 172.21.15.46:0/3874696986 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9db00bd2f0 con 0x7f9db810d530 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.560+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9db8080ef0 con 0x7f9db810d530 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.562+0000 7f9db6000640 1 -- 172.21.15.46:0/3874696986 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7f9db00bcbf0 con 0x7f9db810d530 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.566+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9da00a2c80 msgr2=0x7f9da00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.566+0000 7f9dc0400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9da00a2c80 0x7f9da00a5140 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00a8b0 tx=0x7f9dac008040 comp rx=0 tx=0).stop 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.567+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db810d530 msgr2=0x7f9db807b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.567+0000 7f9dc0400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db810d530 0x7f9db807b900 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009430 tx=0x7f9db0003f10 comp rx=0 tx=0).stop 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.568+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 shutdown_connections 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.568+0000 7f9dc0400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db810d530 0x7f9db807b900 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.568+0000 7f9dc0400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9da00a2c80 0x7f9da00a5140 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.568+0000 7f9dc0400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9db8108e00 0x7f9db8082c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.568+0000 7f9dc0400640 1 --2- 172.21.15.46:0/3874696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9db8108450 0x7f9db8082730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.568+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 >> 172.21.15.46:0/3874696986 conn(0x7f9db8077ba0 msgr2=0x7f9db8078e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.569+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 shutdown_connections 2024-09-10T07:47:31.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:31.569+0000 7f9dc0400640 1 -- 172.21.15.46:0/3874696986 wait complete. 2024-09-10T07:47:32.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:31 smithi181 ceph-mon[35397]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 571 op/s 2024-09-10T07:47:32.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:31 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3874696986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:32.406 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:31 smithi063 ceph-mon[35020]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 571 op/s 2024-09-10T07:47:32.406 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:31 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3874696986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:32.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:31 smithi046 ceph-mon[30705]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 571 op/s 2024-09-10T07:47:32.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:31 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3874696986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:32.635 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-09-10T07:47:32.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-09-10T07:47:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:33.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.579+0000 7fc45fe00640 1 -- 172.21.15.46:0/776008208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601079c0 msgr2=0x7fc460107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.579+0000 7fc45fe00640 1 --2- 172.21.15.46:0/776008208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601079c0 0x7fc460107da0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc458009ca0 tx=0x7fc45802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.581+0000 7fc45fe00640 1 -- 172.21.15.46:0/776008208 shutdown_connections 2024-09-10T07:47:33.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.581+0000 7fc45fe00640 1 --2- 172.21.15.46:0/776008208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460102360 0x7fc46010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.581+0000 7fc45fe00640 1 --2- 172.21.15.46:0/776008208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4601019c0 0x7fc460101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.581+0000 7fc45fe00640 1 --2- 172.21.15.46:0/776008208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601079c0 0x7fc460107da0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.581+0000 7fc45fe00640 1 -- 172.21.15.46:0/776008208 >> 172.21.15.46:0/776008208 conn(0x7fc4600fd740 msgr2=0x7fc4600ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.581+0000 7fc45fe00640 1 -- 172.21.15.46:0/776008208 shutdown_connections 2024-09-10T07:47:33.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.583+0000 7fc45fe00640 1 -- 172.21.15.46:0/776008208 wait complete. 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.584+0000 7fc45fe00640 1 Processor -- start 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.585+0000 7fc45fe00640 1 -- start start 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.585+0000 7fc45fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601019c0 0x7fc460117290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.585+0000 7fc45fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460102360 0x7fc460111310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.585+0000 7fc45fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601079c0 0x7fc460111850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.586+0000 7fc45e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460102360 0x7fc460111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.586+0000 7fc45e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460102360 0x7fc460111310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34920/0 (socket says 172.21.15.46:34920) 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.586+0000 7fc45e000640 1 -- 172.21.15.46:0/1118022339 learned_addr learned my addr 172.21.15.46:0/1118022339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.586+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc46010fce0 con 0x7fc4601079c0 2024-09-10T07:47:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.586+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc46010fb60 con 0x7fc4601019c0 2024-09-10T07:47:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.586+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc46010fe60 con 0x7fc460102360 2024-09-10T07:47:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.586+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601019c0 0x7fc460117290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.587+0000 7fc45ea00640 1 -- 172.21.15.46:0/1118022339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460102360 msgr2=0x7fc460111310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.587+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460102360 0x7fc460111310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.587+0000 7fc45ea00640 1 -- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601079c0 msgr2=0x7fc460111850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.587+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601079c0 0x7fc460111850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.588+0000 7fc45ea00640 1 -- 172.21.15.46:0/1118022339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc460117d80 con 0x7fc4601019c0 2024-09-10T07:47:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.588+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601019c0 0x7fc460117290 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc45802aeb0 tx=0x7fc458031680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.588+0000 7fc457400640 1 -- 172.21.15.46:0/1118022339 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458031970 con 0x7fc4601019c0 2024-09-10T07:47:33.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.588+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc460118010 con 0x7fc4601019c0 2024-09-10T07:47:33.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.589+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601185a0 con 0x7fc4601019c0 2024-09-10T07:47:33.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.589+0000 7fc457400640 1 -- 172.21.15.46:0/1118022339 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc458004520 con 0x7fc4601019c0 2024-09-10T07:47:33.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.591+0000 7fc457400640 1 -- 172.21.15.46:0/1118022339 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458031970 con 0x7fc4601019c0 2024-09-10T07:47:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.594+0000 7fc4687be740 1 Processor -- start 2024-09-10T07:47:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.596+0000 7fc4687be740 1 -- start start 2024-09-10T07:47:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.596+0000 7fc4687be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.597+0000 7fc4687be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4cb760 0x55ab2a4d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.597+0000 7fc45f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.597+0000 7fc45f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34936/0 (socket says 172.21.15.46:34936) 2024-09-10T07:47:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.597+0000 7fc4687be740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.597+0000 7fc4687be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a4313e0 con 0x55ab2a4cb760 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.597+0000 7fc4687be740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a3d8d80 con 0x55ab2a4d6020 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.597+0000 7fc4687be740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a42d3a0 con 0x55ab2a4cae40 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.598+0000 7fc45f400640 1 -- 172.21.15.46:0/1549531542 learned_addr learned my addr 172.21.15.46:0/1549531542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.598+0000 7fc45e000640 1 --2- 172.21.15.46:0/1549531542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.599+0000 7fc45f400640 1 -- 172.21.15.46:0/1549531542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 msgr2=0x55ab2a4d8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.599+0000 7fc45f400640 1 --2- 172.21.15.46:0/1549531542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.599+0000 7fc45f400640 1 -- 172.21.15.46:0/1549531542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4cb760 msgr2=0x55ab2a4d5ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.599+0000 7fc45f400640 1 --2- 172.21.15.46:0/1549531542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4cb760 0x55ab2a4d5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.599+0000 7fc45f400640 1 -- 172.21.15.46:0/1549531542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2a3f7ae0 con 0x55ab2a4cae40 2024-09-10T07:47:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.599+0000 7fc45f400640 1 --2- 172.21.15.46:0/1549531542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4cb220 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc450009790 tx=0x7fc450009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e360fca15b5324e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.607+0000 7fc44be00640 1 -- 172.21.15.46:0/1549531542 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450015020 con 0x55ab2a4cae40 2024-09-10T07:47:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.608+0000 7fc44be00640 1 -- 172.21.15.46:0/1549531542 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc450011420 con 0x55ab2a4cae40 2024-09-10T07:47:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.608+0000 7fc44be00640 1 -- 172.21.15.46:0/1549531542 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4500105c0 con 0x55ab2a4cae40 2024-09-10T07:47:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.609+0000 7fc4687be740 1 -- 172.21.15.46:0/1549531542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 msgr2=0x55ab2a4cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.609+0000 7fc4687be740 1 --2- 172.21.15.46:0/1549531542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4cb220 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc450009790 tx=0x7fc450009c50 comp rx=0 tx=0).stop 2024-09-10T07:47:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.610+0000 7fc4687be740 1 -- 172.21.15.46:0/1549531542 shutdown_connections 2024-09-10T07:47:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.610+0000 7fc4687be740 1 --2- 172.21.15.46:0/1549531542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.610+0000 7fc4687be740 1 --2- 172.21.15.46:0/1549531542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4cb760 0x55ab2a4d5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.610+0000 7fc4687be740 1 --2- 172.21.15.46:0/1549531542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4cb220 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.610+0000 7fc4687be740 1 -- 172.21.15.46:0/1549531542 >> 172.21.15.46:0/1549531542 conn(0x55ab2a4c73d0 msgr2=0x55ab2a4c77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.626+0000 7fc457400640 1 -- 172.21.15.46:0/1118022339 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc4580046d0 con 0x7fc4601019c0 2024-09-10T07:47:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.626+0000 7fc4687be740 1 -- 172.21.15.46:0/1549531542 shutdown_connections 2024-09-10T07:47:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.627+0000 7fc4687be740 1 -- 172.21.15.46:0/1549531542 wait complete. 2024-09-10T07:47:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.627+0000 7fc4687be740 1 -- start start 2024-09-10T07:47:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.628+0000 7fc4687be740 1 Processor -- start 2024-09-10T07:47:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.628+0000 7fc457400640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2f50 0x7fc4440a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.628+0000 7fc4687be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.628+0000 7fc4687be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.628+0000 7fc4687be740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.628+0000 7fc4687be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a4d8bf0 con 0x55ab2a4d6020 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.628+0000 7fc4687be740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a3ec0f0 con 0x55ab2a4d3b10 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.628+0000 7fc4687be740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ab2a4d8d70 con 0x55ab2a4cae40 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.629+0000 7fc4687be740 10 client.0 _ll_register_callbacks cb 0x55ab2a3d3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.629+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.630+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34938/0 (socket says 172.21.15.46:34938) 2024-09-10T07:47:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.630+0000 7fc45ea00640 1 -- 172.21.15.46:0/1573285433 learned_addr learned my addr 172.21.15.46:0/1573285433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.631+0000 7fc45e000640 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.631+0000 7fc457400640 1 -- 172.21.15.46:0/1118022339 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc458031770 con 0x7fc4601019c0 2024-09-10T07:47:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.631+0000 7fc45f400640 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.633+0000 7fc45ea00640 1 -- 172.21.15.46:0/1573285433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d3b10 msgr2=0x55ab2a4e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.633+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.633+0000 7fc45ea00640 1 -- 172.21.15.46:0/1573285433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 msgr2=0x55ab2a4d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.633+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.633+0000 7fc45ea00640 1 -- 172.21.15.46:0/1573285433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2a1a23c0 con 0x55ab2a4cae40 2024-09-10T07:47:33.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.633+0000 7fc45f400640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2f50 0x7fc4440a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.635+0000 7fc448c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:47:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.635+0000 7fc45f400640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2f50 0x7fc4440a5410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc450009520 tx=0x7fc450008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.635+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d3090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc4580049d0 tx=0x7fc4580bbf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ed11091c4254f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.636+0000 7fc448c00640 1 -- 172.21.15.46:0/1573285433 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580098b0 con 0x55ab2a4cae40 2024-09-10T07:47:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.637+0000 7fc448c00640 1 -- 172.21.15.46:0/1573285433 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc4580b8970 con 0x55ab2a4cae40 2024-09-10T07:47:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.637+0000 7fc4687be740 1 -- 172.21.15.46:0/1573285433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc450011420 con 0x55ab2a4cae40 2024-09-10T07:47:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.637+0000 7fc448c00640 1 -- 172.21.15.46:0/1573285433 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc4580bc300 con 0x55ab2a4cae40 2024-09-10T07:47:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.638+0000 7fc4687be740 10 client.28018 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.638+0000 7fc4687be740 1 -- 172.21.15.46:0/1573285433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ab2a4d8bf0 con 0x55ab2a4cae40 2024-09-10T07:47:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.638+0000 7fc448c00640 1 -- 172.21.15.46:0/1573285433 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc450011420 con 0x55ab2a4cae40 2024-09-10T07:47:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.638+0000 7fc448c00640 1 -- 172.21.15.46:0/1573285433 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fc4580dc070 con 0x55ab2a4cae40 2024-09-10T07:47:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.638+0000 7fc4687be740 10 client.28018 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.638+0000 7fc4687be740 10 client.28018 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:47:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.639+0000 7fc4687be740 20 client.28018 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.639+0000 7fc4687be740 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ab2a4e6530 0x55ab2a4e8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.639+0000 7fc4687be740 4 client.28018 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.639+0000 7fc4687be740 1 -- 172.21.15.46:0/1573285433 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab2a43e950 con 0x55ab2a4e6530 2024-09-10T07:47:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.640+0000 7fc45e000640 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ab2a4e6530 0x55ab2a4e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.642+0000 7fc45e000640 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ab2a4e6530 0x55ab2a4e8970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4be13a102fd24d0e server_cookie=fdca6719ef47f1fa in_seq=0 out_seq=0 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.642+0000 7fc448c00640 10 client.28018 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.643+0000 7fc448c00640 1 -- 172.21.15.46:0/1573285433 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab2a43e950 con 0x55ab2a4e6530 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.643+0000 7fc448c00640 10 client.28018 handle_command_reply: tid=0 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.643+0000 7fc4687be740 1 client.28018 shutdown 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.644+0000 7fc4687be740 1 -- 172.21.15.46:0/1573285433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 msgr2=0x55ab2a4d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.644+0000 7fc4687be740 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d3090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc4580049d0 tx=0x7fc4580bbf00 comp rx=0 tx=0).stop 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.644+0000 7fc4687be740 1 -- 172.21.15.46:0/1573285433 shutdown_connections 2024-09-10T07:47:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.644+0000 7fc4687be740 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ab2a4e6530 0x55ab2a4e8970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.644+0000 7fc4687be740 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.644+0000 7fc4687be740 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.644+0000 7fc4687be740 1 --2- 172.21.15.46:0/1573285433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d3090 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.644+0000 7fc4687be740 1 -- 172.21.15.46:0/1573285433 >> 172.21.15.46:0/1573285433 conn(0x55ab2a4c73d0 msgr2=0x55ab2a4cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.645+0000 7fc448c00640 0 client.28018 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:47:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.653+0000 7fc4687be740 1 -- 172.21.15.46:0/1573285433 shutdown_connections 2024-09-10T07:47:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.653+0000 7fc4687be740 1 -- 172.21.15.46:0/1573285433 wait complete. 2024-09-10T07:47:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.653+0000 7fc4687be740 20 client.28018 trim_cache size 0 max 16384 2024-09-10T07:47:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.684+0000 7fc4687be740 1 Processor -- start 2024-09-10T07:47:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.684+0000 7fc4687be740 1 -- start start 2024-09-10T07:47:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.685+0000 7fc4687be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.685+0000 7fc4687be740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.685+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.686+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34954/0 (socket says 172.21.15.46:34954) 2024-09-10T07:47:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.685+0000 7fc4687be740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4e1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.686+0000 7fc4687be740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc460121c30 con 0x55ab2a4d3b10 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.686+0000 7fc45ea00640 1 -- 172.21.15.46:0/2299628119 learned_addr learned my addr 172.21.15.46:0/2299628119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.686+0000 7fc4687be740 1 -- 172.21.15.46:0/2299628119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc460123770 con 0x55ab2a4d6020 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.686+0000 7fc4687be740 1 -- 172.21.15.46:0/2299628119 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc460122300 con 0x55ab2a4cae40 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.686+0000 7fc45f400640 1 --2- 172.21.15.46:0/2299628119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4e1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.687+0000 7fc45f400640 1 -- 172.21.15.46:0/2299628119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 msgr2=0x55ab2a4d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.687+0000 7fc45f400640 1 --2- 172.21.15.46:0/2299628119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.687+0000 7fc45f400640 1 -- 172.21.15.46:0/2299628119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d3b10 msgr2=0x55ab2a4cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.687+0000 7fc45f400640 1 --2- 172.21.15.46:0/2299628119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.687+0000 7fc45f400640 1 -- 172.21.15.46:0/2299628119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2a4d4030 con 0x55ab2a4d6020 2024-09-10T07:47:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.687+0000 7fc45f400640 1 --2- 172.21.15.46:0/2299628119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4e1e80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc450011dc0 tx=0x7fc450009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e9b8f84536aa7a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.688+0000 7fc44b400640 1 -- 172.21.15.46:0/2299628119 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450015040 con 0x55ab2a4d6020 2024-09-10T07:47:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.689+0000 7fc44b400640 1 -- 172.21.15.46:0/2299628119 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45002c070 con 0x55ab2a4d6020 2024-09-10T07:47:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.689+0000 7fc44b400640 1 -- 172.21.15.46:0/2299628119 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450026660 con 0x55ab2a4d6020 2024-09-10T07:47:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.690+0000 7fc4687be740 1 -- 172.21.15.46:0/2299628119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 msgr2=0x55ab2a4e1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.690+0000 7fc4687be740 1 --2- 172.21.15.46:0/2299628119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4e1e80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc450011dc0 tx=0x7fc450009c50 comp rx=0 tx=0).stop 2024-09-10T07:47:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.691+0000 7fc4687be740 1 -- 172.21.15.46:0/2299628119 shutdown_connections 2024-09-10T07:47:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.694+0000 7fc4687be740 1 --2- 172.21.15.46:0/2299628119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4d6020 0x55ab2a4e1e80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.694+0000 7fc4687be740 1 --2- 172.21.15.46:0/2299628119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.694+0000 7fc4687be740 1 --2- 172.21.15.46:0/2299628119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4cae40 0x55ab2a4d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.694+0000 7fc4687be740 1 -- 172.21.15.46:0/2299628119 >> 172.21.15.46:0/2299628119 conn(0x55ab2a4c73d0 msgr2=0x55ab2a4c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.694+0000 7fc4687be740 1 -- 172.21.15.46:0/2299628119 shutdown_connections 2024-09-10T07:47:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.694+0000 7fc4687be740 1 -- 172.21.15.46:0/2299628119 wait complete. 2024-09-10T07:47:33.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.696+0000 7fc4687be740 1 -- start start 2024-09-10T07:47:33.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.697+0000 7fc4687be740 1 Processor -- start 2024-09-10T07:47:33.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.701+0000 7fc4687be740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4cae40 0x55ab2a4de470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.701+0000 7fc4687be740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4d97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.702+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4cae40 0x55ab2a4de470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.702+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4cae40 0x55ab2a4de470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49006/0 (socket says 172.21.15.46:49006) 2024-09-10T07:47:33.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.702+0000 7fc45ea00640 1 -- 172.21.15.46:0/589575674 learned_addr learned my addr 172.21.15.46:0/589575674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:33.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.702+0000 7fc4687be740 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.703+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601230a0 con 0x55ab2a4d6020 2024-09-10T07:47:33.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.703+0000 7fc45e000640 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4d97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.704+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601229d0 con 0x55ab2a4cae40 2024-09-10T07:47:33.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.704+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc460103140 con 0x55ab2a4d3b10 2024-09-10T07:47:33.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.704+0000 7fc45f400640 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.705+0000 7fc4687be740 10 client.0 _ll_register_callbacks cb 0x7fc458003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.706+0000 7fc45ea00640 1 -- 172.21.15.46:0/589575674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4d3b10 msgr2=0x55ab2a4d97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.707+0000 7fc45ea00640 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4d97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.707+0000 7fc45ea00640 1 -- 172.21.15.46:0/589575674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 msgr2=0x55ab2a4d9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.707+0000 7fc45ea00640 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.707+0000 7fc45ea00640 1 -- 172.21.15.46:0/589575674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2a4c8480 con 0x55ab2a4cae40 2024-09-10T07:47:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.709+0000 7fc45f400640 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.709+0000 7fc45ea00640 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4cae40 0x55ab2a4de470 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc4580dc4e0 tx=0x7fc4580c4c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8fb9d919a923fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.709+0000 7fc448c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.710+0000 7fc448c00640 1 -- 172.21.15.46:0/589575674 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580bc0d0 con 0x55ab2a4cae40 2024-09-10T07:47:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.710+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45002c070 con 0x55ab2a4cae40 2024-09-10T07:47:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.710+0000 7fc448c00640 1 -- 172.21.15.46:0/589575674 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc458031970 con 0x55ab2a4cae40 2024-09-10T07:47:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.711+0000 7fc448c00640 1 -- 172.21.15.46:0/589575674 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc4580098b0 con 0x55ab2a4cae40 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.711+0000 7fc4687be740 10 client.28082 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.711+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ab2a550db0 con 0x55ab2a4cae40 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.712+0000 7fc448c00640 1 -- 172.21.15.46:0/589575674 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45002c070 con 0x55ab2a4cae40 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.712+0000 7fc448c00640 1 -- 172.21.15.46:0/589575674 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fc4580bb750 con 0x55ab2a4cae40 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.712+0000 7fc4687be740 10 client.28082 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.712+0000 7fc4687be740 10 client.28082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.712+0000 7fc4687be740 20 client.28082 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.712+0000 7fc4687be740 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ab2a5512e0 0x55ab2a5536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.712+0000 7fc4687be740 4 client.28082 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:47:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.712+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab2a438e60 con 0x55ab2a5512e0 2024-09-10T07:47:33.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.713+0000 7fc45e000640 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ab2a5512e0 0x55ab2a5536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.714+0000 7fc45e000640 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ab2a5512e0 0x55ab2a5536d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1221f4109df04db server_cookie=65230f1f2ce0ebf2 in_seq=0 out_seq=0 2024-09-10T07:47:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.714+0000 7fc448c00640 10 client.28082 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:47:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc448c00640 1 -- 172.21.15.46:0/589575674 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab2a438e60 con 0x55ab2a5512e0 2024-09-10T07:47:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc448c00640 10 client.28082 handle_command_reply: tid=0 2024-09-10T07:47:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 client.28082 shutdown 2024-09-10T07:47:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4cae40 msgr2=0x55ab2a4de470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4cae40 0x55ab2a4de470 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc4580dc4e0 tx=0x7fc4580c4c80 comp rx=0 tx=0).stop 2024-09-10T07:47:33.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 shutdown_connections 2024-09-10T07:47:33.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55ab2a5512e0 0x55ab2a5536d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab2a4d6020 0x55ab2a4d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab2a4d3b10 0x55ab2a4d97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 --2- 172.21.15.46:0/589575674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab2a4cae40 0x55ab2a4de470 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc448c00640 0 client.28082 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:47:33.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.715+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 >> 172.21.15.46:0/589575674 conn(0x55ab2a4c73d0 msgr2=0x55ab2a4c5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.716+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 shutdown_connections 2024-09-10T07:47:33.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.716+0000 7fc4687be740 1 -- 172.21.15.46:0/589575674 wait complete. 2024-09-10T07:47:33.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.716+0000 7fc4687be740 20 client.28082 trim_cache size 0 max 16384 2024-09-10T07:47:33.718 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:47:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.718+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2f50 msgr2=0x7fc4440a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.718+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2f50 0x7fc4440a5410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc450009520 tx=0x7fc450008040 comp rx=0 tx=0).stop 2024-09-10T07:47:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.718+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601019c0 msgr2=0x7fc460117290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.718+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601019c0 0x7fc460117290 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc45802aeb0 tx=0x7fc458031680 comp rx=0 tx=0).stop 2024-09-10T07:47:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.719+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 shutdown_connections 2024-09-10T07:47:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.719+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4440a2f50 0x7fc4440a5410 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.719+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601079c0 0x7fc460111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.719+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc460102360 0x7fc460111310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.719+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1118022339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4601019c0 0x7fc460117290 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.719+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 >> 172.21.15.46:0/1118022339 conn(0x7fc4600fd740 msgr2=0x7fc4600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.719+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 shutdown_connections 2024-09-10T07:47:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:33.719+0000 7fc45fe00640 1 -- 172.21.15.46:0/1118022339 wait complete. 2024-09-10T07:47:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.090+0000 7fd821e00640 1 -- 172.21.15.46:0/4128639189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 msgr2=0x7fd81c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.090+0000 7fd821e00640 1 --2- 172.21.15.46:0/4128639189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 0x7fd81c109ca0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ca0 tx=0x7fd81402a8d0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.090+0000 7fd821e00640 1 -- 172.21.15.46:0/4128639189 shutdown_connections 2024-09-10T07:47:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.090+0000 7fd821e00640 1 --2- 172.21.15.46:0/4128639189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 0x7fd81c109ca0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.090+0000 7fd821e00640 1 --2- 172.21.15.46:0/4128639189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c104f00 0x7fd81c1072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.091+0000 7fd821e00640 1 --2- 172.21.15.46:0/4128639189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd81c1025d0 0x7fd81c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.091+0000 7fd821e00640 1 -- 172.21.15.46:0/4128639189 >> 172.21.15.46:0/4128639189 conn(0x7fd81c0fc1b0 msgr2=0x7fd81c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.091+0000 7fd821e00640 1 -- 172.21.15.46:0/4128639189 shutdown_connections 2024-09-10T07:47:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.091+0000 7fd821e00640 1 -- 172.21.15.46:0/4128639189 wait complete. 2024-09-10T07:47:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.092+0000 7fd821e00640 1 Processor -- start 2024-09-10T07:47:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.092+0000 7fd821e00640 1 -- start start 2024-09-10T07:47:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.093+0000 7fd821e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c1025d0 0x7fd81c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.093+0000 7fd821e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd81c104f00 0x7fd81c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.093+0000 7fd821e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 0x7fd81c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.093+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c1025d0 0x7fd81c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.093+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c1025d0 0x7fd81c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34978/0 (socket says 172.21.15.46:34978) 2024-09-10T07:47:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.093+0000 7fd81b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 0x7fd81c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd81c104f00 0x7fd81c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd81c104f00 0x7fd81c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49024/0 (socket says 172.21.15.46:49024) 2024-09-10T07:47:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 0x7fd81c1a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55398/0 (socket says 172.21.15.46:55398) 2024-09-10T07:47:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd821e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c10d890 con 0x7fd81c107830 2024-09-10T07:47:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd821e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c10d710 con 0x7fd81c104f00 2024-09-10T07:47:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd821e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c10da10 con 0x7fd81c1025d0 2024-09-10T07:47:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81aa00640 1 -- 172.21.15.46:0/3407075384 learned_addr learned my addr 172.21.15.46:0/3407075384 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81a000640 1 -- 172.21.15.46:0/3407075384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c1025d0 msgr2=0x7fd81c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81a000640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c1025d0 0x7fd81c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81a000640 1 -- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 msgr2=0x7fd81c1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81a000640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 0x7fd81c1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.094+0000 7fd81a000640 1 -- 172.21.15.46:0/3407075384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c1a1cb0 con 0x7fd81c104f00 2024-09-10T07:47:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.095+0000 7fd81a000640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd81c104f00 0x7fd81c19a5d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd80800e9c0 tx=0x7fd80800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.095+0000 7fd820a00640 1 -- 172.21.15.46:0/3407075384 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80800ccc0 con 0x7fd81c104f00 2024-09-10T07:47:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.095+0000 7fd821e00640 1 -- 172.21.15.46:0/3407075384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c1a1fa0 con 0x7fd81c104f00 2024-09-10T07:47:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.096+0000 7fd821e00640 1 -- 172.21.15.46:0/3407075384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1a2530 con 0x7fd81c104f00 2024-09-10T07:47:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.096+0000 7fd820a00640 1 -- 172.21.15.46:0/3407075384 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd808010910 con 0x7fd81c104f00 2024-09-10T07:47:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.096+0000 7fd820a00640 1 -- 172.21.15.46:0/3407075384 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8080187e0 con 0x7fd81c104f00 2024-09-10T07:47:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.101+0000 7fd820a00640 1 -- 172.21.15.46:0/3407075384 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd808010ac0 con 0x7fd81c104f00 2024-09-10T07:47:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.103+0000 7fd820a00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8000a2d00 0x7fd8000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.103+0000 7fd820a00640 1 -- 172.21.15.46:0/3407075384 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd8080cf560 con 0x7fd81c104f00 2024-09-10T07:47:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.103+0000 7fd81aa00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8000a2d00 0x7fd8000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.104+0000 7fd81aa00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8000a2d00 0x7fd8000a51c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009a30 tx=0x7fd80c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.370+0000 7fd824e31740 1 Processor -- start 2024-09-10T07:47:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.371+0000 7fd824e31740 1 -- start start 2024-09-10T07:47:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.371+0000 7fd824e31740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb1090 0x562bfafb1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.371+0000 7fd824e31740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafb6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.371+0000 7fd824e31740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafb8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.371+0000 7fd824e31740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562bfaf16100 con 0x562bfafb6a60 2024-09-10T07:47:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.371+0000 7fd824e31740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562bfaebe1b0 con 0x562bfafb1090 2024-09-10T07:47:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.371+0000 7fd824e31740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562bfaf120c0 con 0x562bfafb19d0 2024-09-10T07:47:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.371+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb1090 0x562bfafb1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb1090 0x562bfafb1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49036/0 (socket says 172.21.15.46:49036) 2024-09-10T07:47:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81aa00640 1 -- 172.21.15.46:0/3859939625 learned_addr learned my addr 172.21.15.46:0/3859939625 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81a000640 1 --2- 172.21.15.46:0/3859939625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafb6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81aa00640 1 -- 172.21.15.46:0/3859939625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 msgr2=0x562bfafb6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81aa00640 1 --2- 172.21.15.46:0/3859939625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafb6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81aa00640 1 -- 172.21.15.46:0/3859939625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 msgr2=0x562bfafb8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81aa00640 1 --2- 172.21.15.46:0/3859939625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafb8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81aa00640 1 -- 172.21.15.46:0/3859939625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfac29210 con 0x562bfafb1090 2024-09-10T07:47:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.372+0000 7fd81aa00640 1 --2- 172.21.15.46:0/3859939625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb1090 0x562bfafb1490 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009fd0 tx=0x7fd80c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a18842036b33fba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.373+0000 7fd807e00640 1 -- 172.21.15.46:0/3859939625 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c01b040 con 0x562bfafb1090 2024-09-10T07:47:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.373+0000 7fd807e00640 1 -- 172.21.15.46:0/3859939625 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80c024070 con 0x562bfafb1090 2024-09-10T07:47:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.373+0000 7fd807e00640 1 -- 172.21.15.46:0/3859939625 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c01f3f0 con 0x562bfafb1090 2024-09-10T07:47:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.374+0000 7fd824e31740 1 -- 172.21.15.46:0/3859939625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb1090 msgr2=0x562bfafb1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.374+0000 7fd824e31740 1 --2- 172.21.15.46:0/3859939625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb1090 0x562bfafb1490 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009fd0 tx=0x7fd80c011000 comp rx=0 tx=0).stop 2024-09-10T07:47:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.375+0000 7fd824e31740 1 -- 172.21.15.46:0/3859939625 shutdown_connections 2024-09-10T07:47:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.375+0000 7fd824e31740 1 --2- 172.21.15.46:0/3859939625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafb8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.375+0000 7fd824e31740 1 --2- 172.21.15.46:0/3859939625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafb6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.375+0000 7fd824e31740 1 --2- 172.21.15.46:0/3859939625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb1090 0x562bfafb1490 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.375+0000 7fd824e31740 1 -- 172.21.15.46:0/3859939625 >> 172.21.15.46:0/3859939625 conn(0x562bfafac700 msgr2=0x562bfafacb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.376+0000 7fd824e31740 1 -- 172.21.15.46:0/3859939625 shutdown_connections 2024-09-10T07:47:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.376+0000 7fd824e31740 1 -- 172.21.15.46:0/3859939625 wait complete. 2024-09-10T07:47:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.376+0000 7fd824e31740 1 -- start start 2024-09-10T07:47:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.376+0000 7fd824e31740 1 Processor -- start 2024-09-10T07:47:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.378+0000 7fd824e31740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb1090 0x562bfafbb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.379+0000 7fd824e31740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb19d0 0x562bfafbb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.379+0000 7fd824e31740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb6a60 0x562bfafc27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.379+0000 7fd824e31740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562bfafb9a30 con 0x562bfafb1090 2024-09-10T07:47:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.379+0000 7fd824e31740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562bfae9f090 con 0x562bfafb19d0 2024-09-10T07:47:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.379+0000 7fd824e31740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562bfafb9bb0 con 0x562bfafb6a60 2024-09-10T07:47:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.379+0000 7fd824e31740 10 client.0 _ll_register_callbacks cb 0x562bfaef3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb1090 0x562bfafbb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb1090 0x562bfafbb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55424/0 (socket says 172.21.15.46:55424) 2024-09-10T07:47:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81aa00640 1 -- 172.21.15.46:0/1529574171 learned_addr learned my addr 172.21.15.46:0/1529574171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81b400640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb6a60 0x562bfafc27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81a000640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb19d0 0x562bfafbb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81aa00640 1 -- 172.21.15.46:0/1529574171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb6a60 msgr2=0x562bfafc27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81aa00640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb6a60 0x562bfafc27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81aa00640 1 -- 172.21.15.46:0/1529574171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb19d0 msgr2=0x562bfafbb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81aa00640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb19d0 0x562bfafbb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.380+0000 7fd81aa00640 1 -- 172.21.15.46:0/1529574171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfacf5620 con 0x562bfafb1090 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.381+0000 7fd81a000640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb19d0 0x562bfafbb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.381+0000 7fd81b400640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb6a60 0x562bfafc27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.381+0000 7fd81aa00640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb1090 0x562bfafbb240 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd80c015910 tx=0x7fd80c013fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeecfd91d645c2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.381+0000 7fd804c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.381+0000 7fd804c00640 1 -- 172.21.15.46:0/1529574171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c02d070 con 0x562bfafb1090 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.381+0000 7fd804c00640 1 -- 172.21.15.46:0/1529574171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd80c032460 con 0x562bfafb1090 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.382+0000 7fd824e31740 1 -- 172.21.15.46:0/1529574171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80c024070 con 0x562bfafb1090 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.382+0000 7fd804c00640 1 -- 172.21.15.46:0/1529574171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd80c0326a0 con 0x562bfafb1090 2024-09-10T07:47:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.382+0000 7fd824e31740 10 client.17400 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.382+0000 7fd824e31740 1 -- 172.21.15.46:0/1529574171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562bfafcddc0 con 0x562bfafb1090 2024-09-10T07:47:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.382+0000 7fd804c00640 1 -- 172.21.15.46:0/1529574171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80c032d20 con 0x562bfafb1090 2024-09-10T07:47:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.383+0000 7fd804c00640 1 -- 172.21.15.46:0/1529574171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fd80c01e030 con 0x562bfafb1090 2024-09-10T07:47:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.383+0000 7fd824e31740 10 client.17400 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.383+0000 7fd824e31740 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.383+0000 7fd824e31740 20 client.17400 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.383+0000 7fd824e31740 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562bfafcf230 0x562bfafd1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.383+0000 7fd824e31740 4 client.17400 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.383+0000 7fd824e31740 1 -- 172.21.15.46:0/1529574171 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd80c024070 con 0x562bfafcf230 2024-09-10T07:47:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.384+0000 7fd81a000640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562bfafcf230 0x562bfafd1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.385+0000 7fd81a000640 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562bfafcf230 0x562bfafd1620 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11af4543cd1c2573 server_cookie=186a865c918c86ef in_seq=0 out_seq=0 2024-09-10T07:47:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.385+0000 7fd804c00640 10 client.17400 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.385+0000 7fd804c00640 1 -- 172.21.15.46:0/1529574171 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd80c024070 con 0x562bfafcf230 2024-09-10T07:47:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.385+0000 7fd804c00640 10 client.17400 handle_command_reply: tid=0 2024-09-10T07:47:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 client.17400 shutdown 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 -- 172.21.15.46:0/1529574171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb1090 msgr2=0x562bfafbb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb1090 0x562bfafbb240 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd80c015910 tx=0x7fd80c013fe0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 -- 172.21.15.46:0/1529574171 shutdown_connections 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562bfafcf230 0x562bfafd1620 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb6a60 0x562bfafc27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb19d0 0x562bfafbb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 --2- 172.21.15.46:0/1529574171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb1090 0x562bfafbb240 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd824e31740 1 -- 172.21.15.46:0/1529574171 >> 172.21.15.46:0/1529574171 conn(0x562bfafac700 msgr2=0x562bfafadf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.386+0000 7fd804c00640 0 client.17400 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.387+0000 7fd824e31740 1 -- 172.21.15.46:0/1529574171 shutdown_connections 2024-09-10T07:47:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.387+0000 7fd824e31740 1 -- 172.21.15.46:0/1529574171 wait complete. 2024-09-10T07:47:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.387+0000 7fd824e31740 20 client.17400 trim_cache size 0 max 16384 2024-09-10T07:47:34.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:33 smithi181 ceph-mon[35397]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 475 op/s 2024-09-10T07:47:34.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:33 smithi063 ceph-mon[35020]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 475 op/s 2024-09-10T07:47:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:33 smithi046 ceph-mon[30705]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 475 op/s 2024-09-10T07:47:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.421+0000 7fd824e31740 1 Processor -- start 2024-09-10T07:47:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.422+0000 7fd824e31740 1 -- start start 2024-09-10T07:47:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.422+0000 7fd824e31740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafbb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.422+0000 7fd824e31740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.422+0000 7fd824e31740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.423+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafbb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.424+0000 7fd824e31740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc00fb50 con 0x562bfafb6a60 2024-09-10T07:47:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.424+0000 7fd824e31740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc011690 con 0x562bfafb3220 2024-09-10T07:47:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.424+0000 7fd824e31740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc010220 con 0x562bfafb19d0 2024-09-10T07:47:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.424+0000 7fd81b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.424+0000 7fd81b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49050/0 (socket says 172.21.15.46:49050) 2024-09-10T07:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.424+0000 7fd81b400640 1 -- 172.21.15.46:0/2218006768 learned_addr learned my addr 172.21.15.46:0/2218006768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.424+0000 7fd81a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.425+0000 7fd81b400640 1 -- 172.21.15.46:0/2218006768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 msgr2=0x562bfafbb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.425+0000 7fd81b400640 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafbb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.425+0000 7fd81b400640 1 -- 172.21.15.46:0/2218006768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 msgr2=0x562bfafc26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.425+0000 7fd81b400640 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.425+0000 7fd81b400640 1 -- 172.21.15.46:0/2218006768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfafbc3c0 con 0x562bfafb3220 2024-09-10T07:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.425+0000 7fd81aa00640 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafbb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.426+0000 7fd81a000640 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.426+0000 7fd81b400640 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafab770 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ca0 tx=0x7fd81402f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e72748e06ba9b1a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.427+0000 7fd807400640 1 -- 172.21.15.46:0/2218006768 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814031660 con 0x562bfafb3220 2024-09-10T07:47:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.429+0000 7fd807400640 1 -- 172.21.15.46:0/2218006768 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd814036040 con 0x562bfafb3220 2024-09-10T07:47:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.429+0000 7fd807400640 1 -- 172.21.15.46:0/2218006768 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8140047b0 con 0x562bfafb3220 2024-09-10T07:47:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.430+0000 7fd824e31740 1 -- 172.21.15.46:0/2218006768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 msgr2=0x562bfafab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.430+0000 7fd824e31740 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafab770 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ca0 tx=0x7fd81402f440 comp rx=0 tx=0).stop 2024-09-10T07:47:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.430+0000 7fd824e31740 1 -- 172.21.15.46:0/2218006768 shutdown_connections 2024-09-10T07:47:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.430+0000 7fd824e31740 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafab770 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.431+0000 7fd824e31740 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.431+0000 7fd824e31740 1 --2- 172.21.15.46:0/2218006768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafbb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.431+0000 7fd824e31740 1 -- 172.21.15.46:0/2218006768 >> 172.21.15.46:0/2218006768 conn(0x562bfafac700 msgr2=0x562bfafab060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.433+0000 7fd824e31740 1 -- 172.21.15.46:0/2218006768 shutdown_connections 2024-09-10T07:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.433+0000 7fd824e31740 1 -- 172.21.15.46:0/2218006768 wait complete. 2024-09-10T07:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.434+0000 7fd824e31740 1 -- start start 2024-09-10T07:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.434+0000 7fd824e31740 1 Processor -- start 2024-09-10T07:47:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.435+0000 7fd824e31740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafc07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.436+0000 7fd824e31740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafc0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.436+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafc07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.436+0000 7fd81aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafc07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35024/0 (socket says 172.21.15.46:35024) 2024-09-10T07:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.436+0000 7fd824e31740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.437+0000 7fd81aa00640 1 -- 172.21.15.46:0/2692398540 learned_addr learned my addr 172.21.15.46:0/2692398540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.437+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc010fc0 con 0x562bfafb6a60 2024-09-10T07:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.437+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc0108f0 con 0x562bfafb3220 2024-09-10T07:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.437+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc00db30 con 0x562bfafb19d0 2024-09-10T07:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.437+0000 7fd824e31740 10 client.0 _ll_register_callbacks cb 0x7fd808097e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.437+0000 7fd81b400640 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.438+0000 7fd81a000640 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafc0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.439+0000 7fd81b400640 1 -- 172.21.15.46:0/2692398540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 msgr2=0x562bfafc07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.439+0000 7fd81b400640 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafc07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.439+0000 7fd81b400640 1 -- 172.21.15.46:0/2692398540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 msgr2=0x562bfafc0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.439+0000 7fd81b400640 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafc0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.439+0000 7fd81b400640 1 -- 172.21.15.46:0/2692398540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfafc2310 con 0x562bfafb6a60 2024-09-10T07:47:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.440+0000 7fd81aa00640 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafc07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.441+0000 7fd81b400640 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc1d00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd814031a10 tx=0x7fd814031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1346d7ccd320437 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.441+0000 7fd804c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.443+0000 7fd804c00640 1 -- 172.21.15.46:0/2692398540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814003d80 con 0x562bfafb6a60 2024-09-10T07:47:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.443+0000 7fd804c00640 1 -- 172.21.15.46:0/2692398540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd814003f20 con 0x562bfafb6a60 2024-09-10T07:47:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.444+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd814036040 con 0x562bfafb6a60 2024-09-10T07:47:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.446+0000 7fd804c00640 1 -- 172.21.15.46:0/2692398540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd814057050 con 0x562bfafb6a60 2024-09-10T07:47:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.446+0000 7fd824e31740 10 client.17406 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.446+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562bfb03a620 con 0x562bfafb6a60 2024-09-10T07:47:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.448+0000 7fd804c00640 1 -- 172.21.15.46:0/2692398540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd814036040 con 0x562bfafb6a60 2024-09-10T07:47:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.449+0000 7fd804c00640 1 -- 172.21.15.46:0/2692398540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fd8140305d0 con 0x562bfafb6a60 2024-09-10T07:47:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.449+0000 7fd824e31740 10 client.17406 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.449+0000 7fd824e31740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.449+0000 7fd824e31740 20 client.17406 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.449+0000 7fd824e31740 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562bfb03af80 0x562bfb03d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.449+0000 7fd824e31740 4 client.17406 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:47:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.449+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562bfaf23830 con 0x562bfb03af80 2024-09-10T07:47:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.450+0000 7fd81aa00640 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562bfb03af80 0x562bfb03d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.451+0000 7fd804c00640 10 client.17406 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.451+0000 7fd81aa00640 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562bfb03af80 0x562bfb03d3a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15fc1cda5da47edd server_cookie=3804edf2b732eb4 in_seq=0 out_seq=0 2024-09-10T07:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.493+0000 7fd804c00640 1 -- 172.21.15.46:0/2692398540 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562bfaf23830 con 0x562bfb03af80 2024-09-10T07:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.493+0000 7fd804c00640 10 client.17406 handle_command_reply: tid=0 2024-09-10T07:47:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.493+0000 7fd824e31740 1 client.17406 shutdown 2024-09-10T07:47:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 msgr2=0x562bfafc1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc1d00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd814031a10 tx=0x7fd814031690 comp rx=0 tx=0).stop 2024-09-10T07:47:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 shutdown_connections 2024-09-10T07:47:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562bfb03af80 0x562bfb03d3a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562bfafb6a60 0x562bfafc1d00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bfafb3220 0x562bfafc0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd804c00640 0 client.17406 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 --2- 172.21.15.46:0/2692398540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562bfafb19d0 0x562bfafc07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 >> 172.21.15.46:0/2692398540 conn(0x562bfafac700 msgr2=0x562bfafafe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.494+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 shutdown_connections 2024-09-10T07:47:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.495+0000 7fd824e31740 1 -- 172.21.15.46:0/2692398540 wait complete. 2024-09-10T07:47:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.495+0000 7fd824e31740 20 client.17406 trim_cache size 0 max 16384 2024-09-10T07:47:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:47:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:47:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:47:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.496+0000 7fd821e00640 1 -- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8000a2d00 msgr2=0x7fd8000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.496+0000 7fd821e00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8000a2d00 0x7fd8000a51c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009a30 tx=0x7fd80c009190 comp rx=0 tx=0).stop 2024-09-10T07:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.496+0000 7fd821e00640 1 -- 172.21.15.46:0/3407075384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd81c104f00 msgr2=0x7fd81c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.496+0000 7fd821e00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd81c104f00 0x7fd81c19a5d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd80800e9c0 tx=0x7fd80800ee80 comp rx=0 tx=0).stop 2024-09-10T07:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.497+0000 7fd821e00640 1 -- 172.21.15.46:0/3407075384 shutdown_connections 2024-09-10T07:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.497+0000 7fd821e00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd8000a2d00 0x7fd8000a51c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.497+0000 7fd821e00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd81c107830 0x7fd81c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.497+0000 7fd821e00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd81c104f00 0x7fd81c19a5d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.497+0000 7fd821e00640 1 --2- 172.21.15.46:0/3407075384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c1025d0 0x7fd81c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.497+0000 7fd821e00640 1 -- 172.21.15.46:0/3407075384 >> 172.21.15.46:0/3407075384 conn(0x7fd81c0fc1b0 msgr2=0x7fd81c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.497+0000 7fd821e00640 1 -- 172.21.15.46:0/3407075384 shutdown_connections 2024-09-10T07:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:34.497+0000 7fd821e00640 1 -- 172.21.15.46:0/3407075384 wait complete. 2024-09-10T07:47:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 615.0 s, 572.0 tps, lat 175.165 ms stddev 193.718 2024-09-10T07:47:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:35 smithi181 ceph-mon[35397]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.2 MiB/s wr, 462 op/s 2024-09-10T07:47:36.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:35 smithi063 ceph-mon[35020]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.2 MiB/s wr, 462 op/s 2024-09-10T07:47:36.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:35 smithi046 ceph-mon[30705]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.2 MiB/s wr, 462 op/s 2024-09-10T07:47:36.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:47:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.787+0000 7f0b02200640 1 -- 172.21.15.46:0/1064174456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af40a4a50 msgr2=0x7f0af40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.787+0000 7f0b02200640 1 --2- 172.21.15.46:0/1064174456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af40a4a50 0x7f0af40a4e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0afc065a40 tx=0x7f0afc08e420 comp rx=0 tx=0).stop 2024-09-10T07:47:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.787+0000 7f0b02200640 1 -- 172.21.15.46:0/1064174456 shutdown_connections 2024-09-10T07:47:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.787+0000 7f0b02200640 1 --2- 172.21.15.46:0/1064174456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af40a5370 0x7f0af400c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.787+0000 7f0b02200640 1 --2- 172.21.15.46:0/1064174456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0af40a8ff0 0x7f0af40a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.787+0000 7f0b02200640 1 --2- 172.21.15.46:0/1064174456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af40a4a50 0x7f0af40a4e30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.787+0000 7f0b02200640 1 -- 172.21.15.46:0/1064174456 >> 172.21.15.46:0/1064174456 conn(0x7f0af401a0a0 msgr2=0x7f0af401a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.787+0000 7f0b02200640 1 -- 172.21.15.46:0/1064174456 shutdown_connections 2024-09-10T07:47:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.788+0000 7f0b02200640 1 -- 172.21.15.46:0/1064174456 wait complete. 2024-09-10T07:47:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.789+0000 7f0b02200640 1 Processor -- start 2024-09-10T07:47:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.790+0000 7f0b02200640 1 -- start start 2024-09-10T07:47:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.790+0000 7f0b02200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0af40a4a50 0x7f0af4016340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.790+0000 7f0b02200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af40a5370 0x7f0af4010530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.790+0000 7f0afb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0af40a4a50 0x7f0af4016340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0afb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0af40a4a50 0x7f0af4016340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43898/0 (socket says 172.21.15.46:43898) 2024-09-10T07:47:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0afb400640 1 -- 172.21.15.46:0/303978438 learned_addr learned my addr 172.21.15.46:0/303978438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0b02200640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af40a8ff0 0x7f0af4010ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0afaa00640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af40a5370 0x7f0af4010530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400e4d0 con 0x7f0af40a8ff0 2024-09-10T07:47:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400e350 con 0x7f0af40a5370 2024-09-10T07:47:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400e650 con 0x7f0af40a4a50 2024-09-10T07:47:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0afb400640 1 -- 172.21.15.46:0/303978438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af40a5370 msgr2=0x7f0af4010530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0afb400640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af40a5370 0x7f0af4010530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0afb400640 1 -- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af40a8ff0 msgr2=0x7f0af4010ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.791+0000 7f0afb400640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af40a8ff0 0x7f0af4010ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.792+0000 7f0afb400640 1 -- 172.21.15.46:0/303978438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af4160780 con 0x7f0af40a4a50 2024-09-10T07:47:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.792+0000 7f0afb400640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0af40a4a50 0x7f0af4016340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0afc051220 tx=0x7f0afc06d6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.793+0000 7f0aefe00640 1 -- 172.21.15.46:0/303978438 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0a5070 con 0x7f0af40a4a50 2024-09-10T07:47:37.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.793+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af4160a10 con 0x7f0af40a4a50 2024-09-10T07:47:37.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.794+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af4160f10 con 0x7f0af40a4a50 2024-09-10T07:47:37.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.794+0000 7f0aefe00640 1 -- 172.21.15.46:0/303978438 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc06dad0 con 0x7f0af40a4a50 2024-09-10T07:47:37.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.794+0000 7f0aefe00640 1 -- 172.21.15.46:0/303978438 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0a5070 con 0x7f0af40a4a50 2024-09-10T07:47:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.799+0000 7f0aefe00640 1 -- 172.21.15.46:0/303978438 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0afc0a8070 con 0x7f0af40a4a50 2024-09-10T07:47:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.801+0000 7f0aefe00640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ae40a2f80 0x7f0ae40a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.802+0000 7f0aefe00640 1 -- 172.21.15.46:0/303978438 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0afc095420 con 0x7f0af40a4a50 2024-09-10T07:47:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.802+0000 7f0afaa00640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ae40a2f80 0x7f0ae40a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.803+0000 7f0afaa00640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ae40a2f80 0x7f0ae40a5440 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0af4011b80 tx=0x7f0af0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.812+0000 7f0b04ddb740 1 Processor -- start 2024-09-10T07:47:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.814+0000 7f0b04ddb740 1 -- start start 2024-09-10T07:47:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.814+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3ee940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.815+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.815+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.815+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5557bd354530 con 0x5557bd3eee80 2024-09-10T07:47:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.815+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5557bd2fb860 con 0x5557bd3ee560 2024-09-10T07:47:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.815+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5557bd3504f0 con 0x5557bd3f9740 2024-09-10T07:47:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.815+0000 7f0afb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.815+0000 7f0afb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3ee940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47444/0 (socket says 172.21.15.46:47444) 2024-09-10T07:47:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.815+0000 7f0afb400640 1 -- 172.21.15.46:0/1116309077 learned_addr learned my addr 172.21.15.46:0/1116309077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.816+0000 7f0afbe00640 1 --2- 172.21.15.46:0/1116309077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.816+0000 7f0afb400640 1 -- 172.21.15.46:0/1116309077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 msgr2=0x5557bd3fbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.816+0000 7f0afb400640 1 --2- 172.21.15.46:0/1116309077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.816+0000 7f0afb400640 1 -- 172.21.15.46:0/1116309077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 msgr2=0x5557bd3f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.816+0000 7f0afb400640 1 --2- 172.21.15.46:0/1116309077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.816+0000 7f0afb400640 1 -- 172.21.15.46:0/1116309077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557bd0c5b90 con 0x5557bd3ee560 2024-09-10T07:47:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.818+0000 7f0afb400640 1 --2- 172.21.15.46:0/1116309077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3ee940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0afc072090 tx=0x7f0afc067d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f4d55c3df3fe17b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.819+0000 7f0ae3e00640 1 -- 172.21.15.46:0/1116309077 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0a5040 con 0x5557bd3ee560 2024-09-10T07:47:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.820+0000 7f0ae3e00640 1 -- 172.21.15.46:0/1116309077 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc152170 con 0x5557bd3ee560 2024-09-10T07:47:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.820+0000 7f0ae3e00640 1 -- 172.21.15.46:0/1116309077 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc11afe0 con 0x5557bd3ee560 2024-09-10T07:47:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.821+0000 7f0b04ddb740 1 -- 172.21.15.46:0/1116309077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 msgr2=0x5557bd3ee940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.821+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/1116309077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3ee940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0afc072090 tx=0x7f0afc067d60 comp rx=0 tx=0).stop 2024-09-10T07:47:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.821+0000 7f0b04ddb740 1 -- 172.21.15.46:0/1116309077 shutdown_connections 2024-09-10T07:47:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.821+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/1116309077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.822+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/1116309077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.822+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/1116309077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3ee940 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.822+0000 7f0b04ddb740 1 -- 172.21.15.46:0/1116309077 >> 172.21.15.46:0/1116309077 conn(0x5557bd3eaaf0 msgr2=0x5557bd3eaed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.822+0000 7f0b04ddb740 1 -- 172.21.15.46:0/1116309077 shutdown_connections 2024-09-10T07:47:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.822+0000 7f0b04ddb740 1 -- 172.21.15.46:0/1116309077 wait complete. 2024-09-10T07:47:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.823+0000 7f0b04ddb740 1 -- start start 2024-09-10T07:47:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.823+0000 7f0b04ddb740 1 Processor -- start 2024-09-10T07:47:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.823+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.826+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.826+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.826+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5557bd2fb860 con 0x5557bd3eee80 2024-09-10T07:47:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.826+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5557bd30ee60 con 0x5557bd3ee560 2024-09-10T07:47:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.826+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5557bd3fc0d0 con 0x5557bd3f9740 2024-09-10T07:47:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.826+0000 7f0b04ddb740 10 client.0 _ll_register_callbacks cb 0x5557bd313cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.827+0000 7f0afaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.827+0000 7f0afaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50574/0 (socket says 172.21.15.46:50574) 2024-09-10T07:47:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.827+0000 7f0afaa00640 1 -- 172.21.15.46:0/691592449 learned_addr learned my addr 172.21.15.46:0/691592449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.827+0000 7f0afb400640 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.827+0000 7f0afbe00640 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.828+0000 7f0afaa00640 1 -- 172.21.15.46:0/691592449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 msgr2=0x5557bd3f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.828+0000 7f0afaa00640 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.828+0000 7f0afaa00640 1 -- 172.21.15.46:0/691592449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 msgr2=0x5557bd3f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.828+0000 7f0afaa00640 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.828+0000 7f0afaa00640 1 -- 172.21.15.46:0/691592449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557bd31a990 con 0x5557bd3eee80 2024-09-10T07:47:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.830+0000 7f0afaa00640 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f6b70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0af0004580 tx=0x7f0af00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc7d5d87432ef70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.830+0000 7f0ae0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.831+0000 7f0ae0c00640 1 -- 172.21.15.46:0/691592449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af001e8f0 con 0x5557bd3eee80 2024-09-10T07:47:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.831+0000 7f0ae0c00640 1 -- 172.21.15.46:0/691592449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0af001ea90 con 0x5557bd3eee80 2024-09-10T07:47:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.831+0000 7f0b04ddb740 1 -- 172.21.15.46:0/691592449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0afc152170 con 0x5557bd3eee80 2024-09-10T07:47:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.833+0000 7f0ae0c00640 1 -- 172.21.15.46:0/691592449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0af001ec90 con 0x5557bd3eee80 2024-09-10T07:47:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.834+0000 7f0b04ddb740 10 client.17412 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.834+0000 7f0b04ddb740 1 -- 172.21.15.46:0/691592449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5557bd407b60 con 0x5557bd3eee80 2024-09-10T07:47:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.836+0000 7f0ae0c00640 1 -- 172.21.15.46:0/691592449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc152170 con 0x5557bd3eee80 2024-09-10T07:47:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.837+0000 7f0ae0c00640 1 -- 172.21.15.46:0/691592449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f0af0026580 con 0x5557bd3eee80 2024-09-10T07:47:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.837+0000 7f0b04ddb740 10 client.17412 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.837+0000 7f0b04ddb740 10 client.17412 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:47:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.837+0000 7f0b04ddb740 20 client.17412 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.837+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5557bd408f70 0x5557bd40b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.837+0000 7f0b04ddb740 4 client.17412 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.837+0000 7f0b04ddb740 1 -- 172.21.15.46:0/691592449 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557bd2f7310 con 0x5557bd408f70 2024-09-10T07:47:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.838+0000 7f0afb400640 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5557bd408f70 0x5557bd40b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.840+0000 7f0afb400640 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5557bd408f70 0x5557bd40b360 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f45400eb9ccf589 server_cookie=212ac6c57a14e943 in_seq=0 out_seq=0 2024-09-10T07:47:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.840+0000 7f0ae0c00640 10 client.17412 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:37.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.844+0000 7f0ae0c00640 1 -- 172.21.15.46:0/691592449 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557bd2f7310 con 0x5557bd408f70 2024-09-10T07:47:37.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.844+0000 7f0ae0c00640 10 client.17412 handle_command_reply: tid=0 2024-09-10T07:47:37.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.845+0000 7f0b04ddb740 1 client.17412 shutdown 2024-09-10T07:47:37.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.845+0000 7f0b04ddb740 1 -- 172.21.15.46:0/691592449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 msgr2=0x5557bd3f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.845+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f6b70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0af0004580 tx=0x7f0af00046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 1 -- 172.21.15.46:0/691592449 shutdown_connections 2024-09-10T07:47:37.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5557bd408f70 0x5557bd40b360 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd3f6b70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/691592449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd3f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 1 -- 172.21.15.46:0/691592449 >> 172.21.15.46:0/691592449 conn(0x5557bd3eaaf0 msgr2=0x5557bd3f0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0ae0c00640 0 client.17412 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 1 -- 172.21.15.46:0/691592449 shutdown_connections 2024-09-10T07:47:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 1 -- 172.21.15.46:0/691592449 wait complete. 2024-09-10T07:47:37.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.852+0000 7f0b04ddb740 20 client.17412 trim_cache size 0 max 16384 2024-09-10T07:47:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.891+0000 7f0b04ddb740 1 Processor -- start 2024-09-10T07:47:37.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.891+0000 7f0b04ddb740 1 -- start start 2024-09-10T07:47:37.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3ee560 0x5557bd3f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3eee80 0x5557bd3f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3f9740 0x5557bd3f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc00fa60 con 0x5557bd3ee560 2024-09-10T07:47:37.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc0115a0 con 0x5557bd3f9740 2024-09-10T07:47:37.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc010130 con 0x5557bd3eee80 2024-09-10T07:47:37.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0afbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3f9740 0x5557bd3f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0afbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3f9740 0x5557bd3f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47456/0 (socket says 172.21.15.46:47456) 2024-09-10T07:47:37.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.894+0000 7f0afbe00640 1 -- 172.21.15.46:0/3743249642 learned_addr learned my addr 172.21.15.46:0/3743249642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:37.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.893+0000 7f0afaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3eee80 0x5557bd3f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.894+0000 7f0afb400640 1 --2- 172.21.15.46:0/3743249642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3ee560 0x5557bd3f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.895+0000 7f0afaa00640 1 -- 172.21.15.46:0/3743249642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3f9740 msgr2=0x5557bd3f0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.895+0000 7f0afaa00640 1 --2- 172.21.15.46:0/3743249642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3f9740 0x5557bd3f0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.896+0000 7f0afaa00640 1 -- 172.21.15.46:0/3743249642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3ee560 msgr2=0x5557bd3f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.896+0000 7f0afaa00640 1 --2- 172.21.15.46:0/3743249642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3ee560 0x5557bd3f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.896+0000 7f0afaa00640 1 -- 172.21.15.46:0/3743249642 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557bd3e6530 con 0x5557bd3eee80 2024-09-10T07:47:37.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.897+0000 7f0afaa00640 1 --2- 172.21.15.46:0/3743249642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3eee80 0x5557bd3f06c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0af0026980 tx=0x7f0af0027f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc5f047652c3604a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.899+0000 7f0ae3400640 1 -- 172.21.15.46:0/3743249642 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0005b90 con 0x5557bd3eee80 2024-09-10T07:47:37.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.899+0000 7f0ae3400640 1 -- 172.21.15.46:0/3743249642 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af0026b70 con 0x5557bd3eee80 2024-09-10T07:47:37.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.899+0000 7f0ae3400640 1 -- 172.21.15.46:0/3743249642 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0005b90 con 0x5557bd3eee80 2024-09-10T07:47:37.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.900+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3743249642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3eee80 msgr2=0x5557bd3f06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.900+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3743249642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3eee80 0x5557bd3f06c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0af0026980 tx=0x7f0af0027f50 comp rx=0 tx=0).stop 2024-09-10T07:47:37.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.901+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3743249642 shutdown_connections 2024-09-10T07:47:37.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.901+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3743249642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3f9740 0x5557bd3f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.901+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3743249642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3eee80 0x5557bd3f06c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.902+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3743249642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3ee560 0x5557bd3f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.902+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3743249642 >> 172.21.15.46:0/3743249642 conn(0x5557bd3eaaf0 msgr2=0x5557bd3effb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.904+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3743249642 shutdown_connections 2024-09-10T07:47:37.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.904+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3743249642 wait complete. 2024-09-10T07:47:37.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.905+0000 7f0b04ddb740 1 -- start start 2024-09-10T07:47:37.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.905+0000 7f0b04ddb740 1 Processor -- start 2024-09-10T07:47:37.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.908+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd401110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.909+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd401650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.909+0000 7f0afb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd401110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.909+0000 7f0afb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd401110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47460/0 (socket says 172.21.15.46:47460) 2024-09-10T07:47:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.909+0000 7f0b04ddb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.910+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc010ed0 con 0x5557bd3eee80 2024-09-10T07:47:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.910+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc010800 con 0x5557bd3ee560 2024-09-10T07:47:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.910+0000 7f0b04ddb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc00daf0 con 0x5557bd3f9740 2024-09-10T07:47:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.910+0000 7f0afbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.910+0000 7f0afbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43948/0 (socket says 172.21.15.46:43948) 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.910+0000 7f0afbe00640 1 -- 172.21.15.46:0/3147723454 learned_addr learned my addr 172.21.15.46:0/3147723454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.910+0000 7f0b04ddb740 10 client.0 _ll_register_callbacks cb 0x7f0af00043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.912+0000 7f0afb400640 1 -- 172.21.15.46:0/3147723454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 msgr2=0x5557bd3fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.912+0000 7f0afb400640 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.912+0000 7f0afb400640 1 -- 172.21.15.46:0/3147723454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 msgr2=0x5557bd401650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.912+0000 7f0afb400640 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd401650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.912+0000 7f0afb400640 1 -- 172.21.15.46:0/3147723454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557bd3fe620 con 0x5557bd3ee560 2024-09-10T07:47:37.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.912+0000 7f0ae0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:37.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.912+0000 7f0afb400640 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd401110 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0afc11e370 tx=0x7f0afc122c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e235e60af90007 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.913+0000 7f0ae0c00640 1 -- 172.21.15.46:0/3147723454 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0a5050 con 0x5557bd3ee560 2024-09-10T07:47:37.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.913+0000 7f0ae0c00640 1 -- 172.21.15.46:0/3147723454 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0afc11aef0 con 0x5557bd3ee560 2024-09-10T07:47:37.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.914+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3147723454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af0026b70 con 0x5557bd3ee560 2024-09-10T07:47:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.916+0000 7f0ae0c00640 1 -- 172.21.15.46:0/3147723454 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0afc152170 con 0x5557bd3ee560 2024-09-10T07:47:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.916+0000 7f0b04ddb740 10 client.28112 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.916+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3147723454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5557bd474300 con 0x5557bd3ee560 2024-09-10T07:47:37.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.919+0000 7f0ae0c00640 1 -- 172.21.15.46:0/3147723454 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af0026b70 con 0x5557bd3ee560 2024-09-10T07:47:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.919+0000 7f0ae0c00640 1 -- 172.21.15.46:0/3147723454 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f0afc1090b0 con 0x5557bd3ee560 2024-09-10T07:47:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.919+0000 7f0b04ddb740 10 client.28112 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.919+0000 7f0b04ddb740 10 client.28112 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:47:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.919+0000 7f0b04ddb740 20 client.28112 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.919+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5557bd4748d0 0x5557bd476d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.919+0000 7f0b04ddb740 4 client.28112 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-09-10T07:47:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.919+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3147723454 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x5557bd361c60 con 0x5557bd4748d0 2024-09-10T07:47:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.923+0000 7f0afaa00640 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5557bd4748d0 0x5557bd476d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.926+0000 7f0afaa00640 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5557bd4748d0 0x5557bd476d00 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e59c37894fa814c server_cookie=e162160b57a5b980 in_seq=0 out_seq=0 2024-09-10T07:47:37.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:37.926+0000 7f0ae0c00640 10 client.28112 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:38.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:37 smithi046 ceph-mon[30705]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.6 MiB/s wr, 570 op/s 2024-09-10T07:47:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0ae0c00640 1 -- 172.21.15.46:0/3147723454 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5557bd361c60 con 0x5557bd4748d0 2024-09-10T07:47:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0ae0c00640 10 client.28112 handle_command_reply: tid=0 2024-09-10T07:47:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 client.28112 shutdown 2024-09-10T07:47:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3147723454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 msgr2=0x5557bd401110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd401110 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0afc11e370 tx=0x7f0afc122c50 comp rx=0 tx=0).stop 2024-09-10T07:47:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3147723454 shutdown_connections 2024-09-10T07:47:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5557bd4748d0 0x5557bd476d00 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5557bd3f9740 0x5557bd3fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd3eee80 0x5557bd401650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0ae0c00640 0 client.28112 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 --2- 172.21.15.46:0/3147723454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557bd3ee560 0x5557bd401110 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3147723454 >> 172.21.15.46:0/3147723454 conn(0x5557bd3eaaf0 msgr2=0x5557bd3f07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3147723454 shutdown_connections 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.245+0000 7f0b04ddb740 1 -- 172.21.15.46:0/3147723454 wait complete. 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b04ddb740 20 client.28112 trim_cache size 0 max 16384 2024-09-10T07:47:38.246 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 50, 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 38, 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "ed2cc4b7": { 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 38, 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 145.2, 2024-09-10T07:47:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:38.248 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:47:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ae40a2f80 msgr2=0x7f0ae40a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ae40a2f80 0x7f0ae40a5440 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0af4011b80 tx=0x7f0af0005f90 comp rx=0 tx=0).stop 2024-09-10T07:47:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0af40a4a50 msgr2=0x7f0af4016340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0af40a4a50 0x7f0af4016340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0afc051220 tx=0x7f0afc06d6d0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 shutdown_connections 2024-09-10T07:47:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0ae40a2f80 0x7f0ae40a5440 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af40a8ff0 0x7f0af4010ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af40a5370 0x7f0af4010530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 --2- 172.21.15.46:0/303978438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0af40a4a50 0x7f0af4016340 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.246+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 >> 172.21.15.46:0/303978438 conn(0x7f0af401a0a0 msgr2=0x7f0af400b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.247+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 shutdown_connections 2024-09-10T07:47:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:38.247+0000 7f0b02200640 1 -- 172.21.15.46:0/303978438 wait complete. 2024-09-10T07:47:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:37 smithi181 ceph-mon[35397]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.6 MiB/s wr, 570 op/s 2024-09-10T07:47:38.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:37 smithi063 ceph-mon[35020]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.6 MiB/s wr, 570 op/s 2024-09-10T07:47:39.501 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ed2cc4b7', quiesce took 12.5 seconds. Will release after: 12.7 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.809+0000 7fbb25800640 1 -- 172.21.15.46:0/372085375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20104010 msgr2=0x7fbb201043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.809+0000 7fbb25800640 1 --2- 172.21.15.46:0/372085375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20104010 0x7fbb201043f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009ca0 tx=0x7fbb1002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.809+0000 7fbb25800640 1 -- 172.21.15.46:0/372085375 shutdown_connections 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.809+0000 7fbb25800640 1 --2- 172.21.15.46:0/372085375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb20109140 0x7fbb20110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.809+0000 7fbb25800640 1 --2- 172.21.15.46:0/372085375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb201049c0 0x7fbb20108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.809+0000 7fbb25800640 1 --2- 172.21.15.46:0/372085375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20104010 0x7fbb201043f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.809+0000 7fbb25800640 1 -- 172.21.15.46:0/372085375 >> 172.21.15.46:0/372085375 conn(0x7fbb200ffa50 msgr2=0x7fbb20101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.809+0000 7fbb25800640 1 -- 172.21.15.46:0/372085375 shutdown_connections 2024-09-10T07:47:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.810+0000 7fbb25800640 1 -- 172.21.15.46:0/372085375 wait complete. 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.810+0000 7fbb25800640 1 Processor -- start 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb25800640 1 -- start start 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb25800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb20104010 0x7fbb2019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb25800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb201049c0 0x7fbb2019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb25800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20109140 0x7fbb2019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb25800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20114190 con 0x7fbb20109140 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb25800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20114010 con 0x7fbb201049c0 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb25800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20114310 con 0x7fbb20104010 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb20104010 0x7fbb2019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb1f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20109140 0x7fbb2019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb20104010 0x7fbb2019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43962/0 (socket says 172.21.15.46:43962) 2024-09-10T07:47:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb1ea00640 1 -- 172.21.15.46:0/1944180180 learned_addr learned my addr 172.21.15.46:0/1944180180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:39.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.811+0000 7fbb16000640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb201049c0 0x7fbb2019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb16000640 1 -- 172.21.15.46:0/1944180180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb20104010 msgr2=0x7fbb2019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb16000640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb20104010 0x7fbb2019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb16000640 1 -- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20109140 msgr2=0x7fbb2019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb16000640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20109140 0x7fbb2019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb16000640 1 -- 172.21.15.46:0/1944180180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2019f660 con 0x7fbb201049c0 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb1f400640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20109140 0x7fbb2019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb16000640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb201049c0 0x7fbb2019abf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e9f0 tx=0x7fbb0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb17e00640 1 -- 172.21.15.46:0/1944180180 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800ccc0 con 0x7fbb201049c0 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb17e00640 1 -- 172.21.15.46:0/1944180180 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb08010910 con 0x7fbb201049c0 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb25800640 1 -- 172.21.15.46:0/1944180180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2019f950 con 0x7fbb201049c0 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.812+0000 7fbb17e00640 1 -- 172.21.15.46:0/1944180180 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08018960 con 0x7fbb201049c0 2024-09-10T07:47:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.813+0000 7fbb25800640 1 -- 172.21.15.46:0/1944180180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201a7230 con 0x7fbb201049c0 2024-09-10T07:47:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.815+0000 7fbb283cc740 1 Processor -- start 2024-09-10T07:47:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb283cc740 1 -- start start 2024-09-10T07:47:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb17e00640 1 -- 172.21.15.46:0/1944180180 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb08010ac0 con 0x7fbb201049c0 2024-09-10T07:47:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f5fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f605910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f608240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb283cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e68f5613e0 con 0x55e68f5fb590 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb283cc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e68f508d80 con 0x55e68f5fac70 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb283cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e68f55d3a0 con 0x55e68f605e50 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb1f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f5fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb1f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f5fb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47492/0 (socket says 172.21.15.46:47492) 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.816+0000 7fbb1f400640 1 -- 172.21.15.46:0/3107657716 learned_addr learned my addr 172.21.15.46:0/3107657716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb16000640 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f608240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f605910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb1f400640 1 -- 172.21.15.46:0/3107657716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 msgr2=0x55e68f608240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb1f400640 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f608240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb1f400640 1 -- 172.21.15.46:0/3107657716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 msgr2=0x55e68f605910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb1f400640 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f605910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb1f400640 1 -- 172.21.15.46:0/3107657716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68f527ae0 con 0x55e68f5fac70 2024-09-10T07:47:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f605910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.817+0000 7fbb1f400640 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f5fb050 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbb18011440 tx=0x7fbb18011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a2eddaf4d29bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.818+0000 7fbb0fe00640 1 -- 172.21.15.46:0/3107657716 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18004020 con 0x55e68f5fac70 2024-09-10T07:47:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.818+0000 7fbb0fe00640 1 -- 172.21.15.46:0/3107657716 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb180041c0 con 0x55e68f5fac70 2024-09-10T07:47:39.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.818+0000 7fbb0fe00640 1 -- 172.21.15.46:0/3107657716 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800f8b0 con 0x55e68f5fac70 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.818+0000 7fbb17e00640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbb040a2b80 0x7fbb040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.818+0000 7fbb17e00640 1 -- 172.21.15.46:0/1944180180 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbb080cf7b0 con 0x7fbb201049c0 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.818+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbb040a2b80 0x7fbb040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.818+0000 7fbb283cc740 1 -- 172.21.15.46:0/3107657716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 msgr2=0x55e68f5fb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.818+0000 7fbb283cc740 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f5fb050 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbb18011440 tx=0x7fbb18011900 comp rx=0 tx=0).stop 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb283cc740 1 -- 172.21.15.46:0/3107657716 shutdown_connections 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb283cc740 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f608240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb283cc740 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f605910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb283cc740 1 --2- 172.21.15.46:0/3107657716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f5fb050 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb283cc740 1 -- 172.21.15.46:0/3107657716 >> 172.21.15.46:0/3107657716 conn(0x55e68f5f7200 msgr2=0x55e68f5f75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb283cc740 1 -- 172.21.15.46:0/3107657716 shutdown_connections 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb283cc740 1 -- 172.21.15.46:0/3107657716 wait complete. 2024-09-10T07:47:39.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbb040a2b80 0x7fbb040a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55e68f5f8060 tx=0x7fbb10005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.819+0000 7fbb283cc740 1 -- start start 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb283cc740 1 Processor -- start 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f602cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f603230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f603770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb283cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e68f608a20 con 0x55e68f5fb590 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb283cc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e68f51c0f0 con 0x55e68f5fac70 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb283cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e68f608ba0 con 0x55e68f605e50 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb283cc740 10 client.0 _ll_register_callbacks cb 0x55e68f503bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.820+0000 7fbb16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f603230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f602cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f603230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50638/0 (socket says 172.21.15.46:50638) 2024-09-10T07:47:39.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb16000640 1 -- 172.21.15.46:0/1032561271 learned_addr learned my addr 172.21.15.46:0/1032561271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:39.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb1f400640 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f603770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb16000640 1 -- 172.21.15.46:0/1032561271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 msgr2=0x55e68f603770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:39.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb16000640 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f603770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb16000640 1 -- 172.21.15.46:0/1032561271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 msgr2=0x55e68f602cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:39.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb16000640 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f602cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb16000640 1 -- 172.21.15.46:0/1032561271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68f2d23c0 con 0x55e68f5fb590 2024-09-10T07:47:39.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f602cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:39.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb16000640 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f603230 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbb0809bc70 tx=0x7fbb0809b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca33045e0dfebd11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:39.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.821+0000 7fbb0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:39.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb0cc00640 1 -- 172.21.15.46:0/1032561271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb080cfda0 con 0x55e68f5fb590 2024-09-10T07:47:39.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb0cc00640 1 -- 172.21.15.46:0/1032561271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbb0809b130 con 0x55e68f5fb590 2024-09-10T07:47:39.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb283cc740 1 -- 172.21.15.46:0/1032561271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb180041c0 con 0x55e68f5fb590 2024-09-10T07:47:39.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb0cc00640 1 -- 172.21.15.46:0/1032561271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fbb080a48a0 con 0x55e68f5fb590 2024-09-10T07:47:39.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb283cc740 10 client.17427 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb283cc740 1 -- 172.21.15.46:0/1032561271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e68f614a10 con 0x55e68f5fb590 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb0cc00640 1 -- 172.21.15.46:0/1032561271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb180041c0 con 0x55e68f5fb590 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb0cc00640 1 -- 172.21.15.46:0/1032561271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fbb08097a30 con 0x55e68f5fb590 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb283cc740 10 client.17427 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.822+0000 7fbb283cc740 10 client.17427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.823+0000 7fbb283cc740 20 client.17427 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.823+0000 7fbb283cc740 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e68f615ed0 0x55e68f618300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.823+0000 7fbb283cc740 4 client.17427 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.823+0000 7fbb283cc740 1 -- 172.21.15.46:0/1032561271 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e68f56e950 con 0x55e68f615ed0 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.823+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e68f615ed0 0x55e68f618300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.823+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e68f615ed0 0x55e68f618300 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b5ea547f7f04e99 server_cookie=d469513b4b065812 in_seq=0 out_seq=0 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.823+0000 7fbb0cc00640 10 client.17427 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:47:39.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.824+0000 7fbb0cc00640 1 -- 172.21.15.46:0/1032561271 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e68f56e950 con 0x55e68f615ed0 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.824+0000 7fbb0cc00640 10 client.17427 handle_command_reply: tid=0 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.824+0000 7fbb283cc740 1 client.17427 shutdown 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.824+0000 7fbb283cc740 1 -- 172.21.15.46:0/1032561271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 msgr2=0x55e68f603230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.824+0000 7fbb283cc740 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f603230 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbb0809bc70 tx=0x7fbb0809b640 comp rx=0 tx=0).stop 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 1 -- 172.21.15.46:0/1032561271 shutdown_connections 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e68f615ed0 0x55e68f618300 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f603770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fb590 0x55e68f603230 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 1 --2- 172.21.15.46:0/1032561271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f602cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb0cc00640 0 client.17427 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 1 -- 172.21.15.46:0/1032561271 >> 172.21.15.46:0/1032561271 conn(0x55e68f5f7200 msgr2=0x55e68f5f6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 1 -- 172.21.15.46:0/1032561271 shutdown_connections 2024-09-10T07:47:39.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 1 -- 172.21.15.46:0/1032561271 wait complete. 2024-09-10T07:47:39.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.825+0000 7fbb283cc740 20 client.17427 trim_cache size 0 max 16384 2024-09-10T07:47:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb283cc740 1 Processor -- start 2024-09-10T07:47:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb283cc740 1 -- start start 2024-09-10T07:47:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f5fac70 0x55e68f61dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f605e50 0x55e68f61e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fff50 0x55e68f6003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb283cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b0f50 con 0x55e68f605e50 2024-09-10T07:47:39.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb283cc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b2a90 con 0x55e68f5fff50 2024-09-10T07:47:39.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb283cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b1620 con 0x55e68f5fac70 2024-09-10T07:47:39.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f605e50 0x55e68f61e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f5fac70 0x55e68f61dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f605e50 0x55e68f61e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50640/0 (socket says 172.21.15.46:50640) 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb16000640 1 -- 172.21.15.46:0/749165509 learned_addr learned my addr 172.21.15.46:0/749165509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.839+0000 7fbb1f400640 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fff50 0x55e68f6003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.840+0000 7fbb1ea00640 1 -- 172.21.15.46:0/749165509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fff50 msgr2=0x55e68f6003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.840+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fff50 0x55e68f6003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.840+0000 7fbb1ea00640 1 -- 172.21.15.46:0/749165509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f605e50 msgr2=0x55e68f61e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.840+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f605e50 0x55e68f61e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.840+0000 7fbb1ea00640 1 -- 172.21.15.46:0/749165509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68f5f2770 con 0x55e68f5fac70 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.840+0000 7fbb16000640 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f605e50 0x55e68f61e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.840+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f5fac70 0x55e68f61dbd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006010 tx=0x7fbb10041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe651bf946943e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.840+0000 7fbb0f400640 1 -- 172.21.15.46:0/749165509 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100319a0 con 0x55e68f5fac70 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb0f400640 1 -- 172.21.15.46:0/749165509 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb10036050 con 0x55e68f5fac70 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb0f400640 1 -- 172.21.15.46:0/749165509 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100307e0 con 0x55e68f5fac70 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 -- 172.21.15.46:0/749165509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f5fac70 msgr2=0x55e68f61dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f5fac70 0x55e68f61dbd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006010 tx=0x7fbb10041000 comp rx=0 tx=0).stop 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 -- 172.21.15.46:0/749165509 shutdown_connections 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fff50 0x55e68f6003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f605e50 0x55e68f61e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 --2- 172.21.15.46:0/749165509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f5fac70 0x55e68f61dbd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 -- 172.21.15.46:0/749165509 >> 172.21.15.46:0/749165509 conn(0x55e68f5f7200 msgr2=0x55e68f5fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 -- 172.21.15.46:0/749165509 shutdown_connections 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.841+0000 7fbb283cc740 1 -- 172.21.15.46:0/749165509 wait complete. 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 1 -- start start 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 1 Processor -- start 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f60ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fff50 0x55e68f60e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f60adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b23c0 con 0x55e68f5fff50 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b1cf0 con 0x55e68f5fac70 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201aef30 con 0x55e68f605e50 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.842+0000 7fbb283cc740 10 client.0 _ll_register_callbacks cb 0x7fbb0800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f60ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fff50 0x55e68f60e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f60ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47510/0 (socket says 172.21.15.46:47510) 2024-09-10T07:47:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 -- 172.21.15.46:0/631661748 learned_addr learned my addr 172.21.15.46:0/631661748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1f400640 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f60adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 -- 172.21.15.46:0/631661748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 msgr2=0x55e68f60adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f60adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 -- 172.21.15.46:0/631661748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fff50 msgr2=0x55e68f60e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fff50 0x55e68f60e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 -- 172.21.15.46:0/631661748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68f5f82b0 con 0x55e68f5fac70 2024-09-10T07:47:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb16000640 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fff50 0x55e68f60e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1f400640 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f60adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb1ea00640 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f60ddf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbb1002a900 tx=0x7fbb10009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd0ce2915dad5561 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.843+0000 7fbb0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.844+0000 7fbb0cc00640 1 -- 172.21.15.46:0/631661748 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10065070 con 0x55e68f5fac70 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.844+0000 7fbb283cc740 1 -- 172.21.15.46:0/631661748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb10036050 con 0x55e68f5fac70 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.844+0000 7fbb0cc00640 1 -- 172.21.15.46:0/631661748 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbb10030760 con 0x55e68f5fac70 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.844+0000 7fbb0cc00640 1 -- 172.21.15.46:0/631661748 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fbb10071050 con 0x55e68f5fac70 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.844+0000 7fbb283cc740 10 client.28130 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.844+0000 7fbb283cc740 1 -- 172.21.15.46:0/631661748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e68f6812d0 con 0x55e68f5fac70 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.844+0000 7fbb0cc00640 1 -- 172.21.15.46:0/631661748 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb10036050 con 0x55e68f5fac70 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.845+0000 7fbb0cc00640 1 -- 172.21.15.46:0/631661748 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fbb1002fc90 con 0x55e68f5fac70 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.845+0000 7fbb283cc740 10 client.28130 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.845+0000 7fbb283cc740 10 client.28130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:47:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.845+0000 7fbb283cc740 20 client.28130 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.845+0000 7fbb283cc740 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e68f681a00 0x55e68f683e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.845+0000 7fbb283cc740 4 client.28130 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.845+0000 7fbb283cc740 1 -- 172.21.15.46:0/631661748 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e68f568e60 con 0x55e68f681a00 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.845+0000 7fbb16000640 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e68f681a00 0x55e68f683e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb16000640 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e68f681a00 0x55e68f683e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9f29c406619feaa server_cookie=b57880009e5808fa in_seq=0 out_seq=0 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb0cc00640 10 client.28130 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb0cc00640 1 -- 172.21.15.46:0/631661748 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e68f568e60 con 0x55e68f681a00 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb0cc00640 10 client.28130 handle_command_reply: tid=0 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 client.28130 shutdown 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 -- 172.21.15.46:0/631661748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 msgr2=0x55e68f60ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f60ddf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbb1002a900 tx=0x7fbb10009650 comp rx=0 tx=0).stop 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 -- 172.21.15.46:0/631661748 shutdown_connections 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55e68f681a00 0x55e68f683e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e68f605e50 0x55e68f60adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68f5fff50 0x55e68f60e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb0cc00640 0 client.28130 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:47:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 --2- 172.21.15.46:0/631661748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e68f5fac70 0x55e68f60ddf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 -- 172.21.15.46:0/631661748 >> 172.21.15.46:0/631661748 conn(0x55e68f5f7200 msgr2=0x55e68f5fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 -- 172.21.15.46:0/631661748 shutdown_connections 2024-09-10T07:47:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.846+0000 7fbb283cc740 1 -- 172.21.15.46:0/631661748 wait complete. 2024-09-10T07:47:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb283cc740 20 client.28130 trim_cache size 0 max 16384 2024-09-10T07:47:39.849 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:47:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 -- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbb040a2b80 msgr2=0x7fbb040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbb040a2b80 0x7fbb040a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55e68f5f8060 tx=0x7fbb10005d80 comp rx=0 tx=0).stop 2024-09-10T07:47:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 -- 172.21.15.46:0/1944180180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb201049c0 msgr2=0x7fbb2019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb201049c0 0x7fbb2019abf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e9f0 tx=0x7fbb0800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 -- 172.21.15.46:0/1944180180 shutdown_connections 2024-09-10T07:47:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbb040a2b80 0x7fbb040a5040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20109140 0x7fbb2019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb201049c0 0x7fbb2019abf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 --2- 172.21.15.46:0/1944180180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb20104010 0x7fbb2019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:39.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 -- 172.21.15.46:0/1944180180 >> 172.21.15.46:0/1944180180 conn(0x7fbb200ffa50 msgr2=0x7fbb20100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:39.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.847+0000 7fbb25800640 1 -- 172.21.15.46:0/1944180180 shutdown_connections 2024-09-10T07:47:39.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:39.848+0000 7fbb25800640 1 -- 172.21.15.46:0/1944180180 wait complete. 2024-09-10T07:47:40.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:39 smithi046 ceph-mon[30705]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.7 MiB/s wr, 423 op/s 2024-09-10T07:47:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:39 smithi181 ceph-mon[35397]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.7 MiB/s wr, 423 op/s 2024-09-10T07:47:40.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:39 smithi063 ceph-mon[35020]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.7 MiB/s wr, 423 op/s 2024-09-10T07:47:40.601 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 620.0 s, 283.9 tps, lat 154.170 ms stddev 185.960 2024-09-10T07:47:40.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:47:41.055 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 -- 172.21.15.46:0/2966823203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1281047e0 msgr2=0x7ff128108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 --2- 172.21.15.46:0/2966823203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1281047e0 0x7ff128108830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff110009900 tx=0x7ff11002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 -- 172.21.15.46:0/2966823203 shutdown_connections 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 --2- 172.21.15.46:0/2966823203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff128108f60 0x7ff128110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 --2- 172.21.15.46:0/2966823203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1281047e0 0x7ff128108830 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 --2- 172.21.15.46:0/2966823203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff128103e30 0x7ff128104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 -- 172.21.15.46:0/2966823203 >> 172.21.15.46:0/2966823203 conn(0x7ff1280ff8d0 msgr2=0x7ff128101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 -- 172.21.15.46:0/2966823203 shutdown_connections 2024-09-10T07:47:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.619+0000 7ff12ce00640 1 -- 172.21.15.46:0/2966823203 wait complete. 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.620+0000 7ff12ce00640 1 Processor -- start 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.620+0000 7ff12ce00640 1 -- start start 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff12ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff128103e30 0x7ff12819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff12ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1281047e0 0x7ff12819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff12ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff128108f60 0x7ff12819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff12ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff128113f80 con 0x7ff128108f60 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff12ce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff128113e00 con 0x7ff128103e30 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff12ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff128114100 con 0x7ff1281047e0 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff128108f60 0x7ff12819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff128103e30 0x7ff12819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff128103e30 0x7ff12819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47532/0 (socket says 172.21.15.46:47532) 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127400640 1 -- 172.21.15.46:0/3527160834 learned_addr learned my addr 172.21.15.46:0/3527160834 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff11fe00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1281047e0 0x7ff12819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127e00640 1 -- 172.21.15.46:0/3527160834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1281047e0 msgr2=0x7ff12819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127e00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1281047e0 0x7ff12819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127e00640 1 -- 172.21.15.46:0/3527160834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff128103e30 msgr2=0x7ff12819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127e00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff128103e30 0x7ff12819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.621+0000 7ff127e00640 1 -- 172.21.15.46:0/3527160834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12819f410 con 0x7ff128108f60 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.622+0000 7ff127400640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff128103e30 0x7ff12819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.622+0000 7ff127e00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff128108f60 0x7ff12819ed30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff12000e940 tx=0x7ff12000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.622+0000 7ff124c00640 1 -- 172.21.15.46:0/3527160834 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004d20 con 0x7ff128108f60 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.622+0000 7ff124c00640 1 -- 172.21.15.46:0/3527160834 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff120005020 con 0x7ff128108f60 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.622+0000 7ff12ce00640 1 -- 172.21.15.46:0/3527160834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12819f700 con 0x7ff128108f60 2024-09-10T07:47:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.622+0000 7ff124c00640 1 -- 172.21.15.46:0/3527160834 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200116b0 con 0x7ff128108f60 2024-09-10T07:47:41.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.622+0000 7ff12ce00640 1 -- 172.21.15.46:0/3527160834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1281a6fe0 con 0x7ff128108f60 2024-09-10T07:47:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff130e73740 1 Processor -- start 2024-09-10T07:47:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff124c00640 1 -- 172.21.15.46:0/3527160834 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1200054d0 con 0x7ff128108f60 2024-09-10T07:47:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff130e73740 1 -- start start 2024-09-10T07:47:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff130e73740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ede40 0x55aa196ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff130e73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ee760 0x55aa196f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff130e73740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196f9020 0x55aa196fb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff130e73740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aa196543e0 con 0x55aa196ede40 2024-09-10T07:47:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff130e73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aa195fbd80 con 0x55aa196ee760 2024-09-10T07:47:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff130e73740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55aa196503a0 con 0x55aa196f9020 2024-09-10T07:47:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.626+0000 7ff11fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ede40 0x55aa196ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ede40 0x55aa196ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50680/0 (socket says 172.21.15.46:50680) 2024-09-10T07:47:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 -- 172.21.15.46:0/912139890 learned_addr learned my addr 172.21.15.46:0/912139890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff127400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ee760 0x55aa196f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196f9020 0x55aa196fb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 -- 172.21.15.46:0/912139890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196f9020 msgr2=0x55aa196fb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196f9020 0x55aa196fb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 -- 172.21.15.46:0/912139890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ee760 msgr2=0x55aa196f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ee760 0x55aa196f8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 -- 172.21.15.46:0/912139890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1961aae0 con 0x55aa196ede40 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff127400640 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ee760 0x55aa196f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff124c00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff10c0a2b80 0x7ff10c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.627+0000 7ff11fe00640 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ede40 0x55aa196ee220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff11002a910 tx=0x7ff110000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb738c2950228df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.628+0000 7ff124c00640 1 -- 172.21.15.46:0/3527160834 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff1200cf410 con 0x7ff128108f60 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.628+0000 7ff127400640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff10c0a2b80 0x7ff10c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.628+0000 7ff11cc00640 1 -- 172.21.15.46:0/912139890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110044030 con 0x55aa196ede40 2024-09-10T07:47:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.628+0000 7ff11cc00640 1 -- 172.21.15.46:0/912139890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff110005020 con 0x55aa196ede40 2024-09-10T07:47:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.628+0000 7ff11cc00640 1 -- 172.21.15.46:0/912139890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110035040 con 0x55aa196ede40 2024-09-10T07:47:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 -- 172.21.15.46:0/912139890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ede40 msgr2=0x55aa196ee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff127400640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff10c0a2b80 0x7ff10c0a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55aa196eb230 tx=0x7ff11800b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ede40 0x55aa196ee220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff11002a910 tx=0x7ff110000bc0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 -- 172.21.15.46:0/912139890 shutdown_connections 2024-09-10T07:47:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196f9020 0x55aa196fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ee760 0x55aa196f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 --2- 172.21.15.46:0/912139890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ede40 0x55aa196ee220 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 -- 172.21.15.46:0/912139890 >> 172.21.15.46:0/912139890 conn(0x55aa196ea3d0 msgr2=0x55aa196ea7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 -- 172.21.15.46:0/912139890 shutdown_connections 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 -- 172.21.15.46:0/912139890 wait complete. 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.629+0000 7ff130e73740 1 -- start start 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff130e73740 1 Processor -- start 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff130e73740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff130e73740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff130e73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff130e73740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aa196fbbf0 con 0x55aa196ee760 2024-09-10T07:47:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff130e73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aa1960f0f0 con 0x55aa196f9020 2024-09-10T07:47:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff130e73740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55aa196fbd70 con 0x55aa196ede40 2024-09-10T07:47:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff130e73740 10 client.0 _ll_register_callbacks cb 0x55aa195f6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff11fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff127400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff11fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50684/0 (socket says 172.21.15.46:50684) 2024-09-10T07:47:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.630+0000 7ff11fe00640 1 -- 172.21.15.46:0/2021536116 learned_addr learned my addr 172.21.15.46:0/2021536116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff127e00640 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff127400640 1 -- 172.21.15.46:0/2021536116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 msgr2=0x55aa196f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff127400640 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff127400640 1 -- 172.21.15.46:0/2021536116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 msgr2=0x55aa196f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff127400640 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff127400640 1 -- 172.21.15.46:0/2021536116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa193c53c0 con 0x55aa196ede40 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff11fe00640 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff127400640 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196f5f60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff118012010 tx=0x7ff118004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1996f5d7a570f457 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.631+0000 7ff115600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.632+0000 7ff115600640 1 -- 172.21.15.46:0/2021536116 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11801fb40 con 0x55aa196ede40 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.632+0000 7ff130e73740 1 -- 172.21.15.46:0/2021536116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff110005020 con 0x55aa196ede40 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.632+0000 7ff115600640 1 -- 172.21.15.46:0/2021536116 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff11801fce0 con 0x55aa196ede40 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.632+0000 7ff115600640 1 -- 172.21.15.46:0/2021536116 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff118005020 con 0x55aa196ede40 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.632+0000 7ff130e73740 10 client.28087 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.632+0000 7ff130e73740 1 -- 172.21.15.46:0/2021536116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55aa19707d60 con 0x55aa196ede40 2024-09-10T07:47:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.632+0000 7ff115600640 1 -- 172.21.15.46:0/2021536116 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff110005020 con 0x55aa196ede40 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.632+0000 7ff115600640 1 -- 172.21.15.46:0/2021536116 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7ff118026070 con 0x55aa196ede40 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff130e73740 10 client.28087 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff130e73740 10 client.28087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff130e73740 20 client.28087 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff130e73740 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aa197091c0 0x55aa1970b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff130e73740 4 client.28087 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff130e73740 1 -- 172.21.15.46:0/2021536116 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa19661950 con 0x55aa197091c0 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff11fe00640 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aa197091c0 0x55aa1970b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff11fe00640 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aa197091c0 0x55aa1970b660 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c0930420e6d740 server_cookie=64d126432b0b0806 in_seq=0 out_seq=0 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.633+0000 7ff115600640 10 client.28087 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.634+0000 7ff115600640 1 -- 172.21.15.46:0/2021536116 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa19661950 con 0x55aa197091c0 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.634+0000 7ff115600640 10 client.28087 handle_command_reply: tid=0 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.634+0000 7ff130e73740 1 client.28087 shutdown 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 -- 172.21.15.46:0/2021536116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 msgr2=0x55aa196f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196f5f60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff118012010 tx=0x7ff118004730 comp rx=0 tx=0).stop 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 -- 172.21.15.46:0/2021536116 shutdown_connections 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aa197091c0 0x55aa1970b660 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 --2- 172.21.15.46:0/2021536116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196f5f60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff115600640 0 client.28087 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 -- 172.21.15.46:0/2021536116 >> 172.21.15.46:0/2021536116 conn(0x55aa196ea3d0 msgr2=0x55aa196f0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 -- 172.21.15.46:0/2021536116 shutdown_connections 2024-09-10T07:47:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 1 -- 172.21.15.46:0/2021536116 wait complete. 2024-09-10T07:47:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.635+0000 7ff130e73740 20 client.28087 trim_cache size 0 max 16384 2024-09-10T07:47:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff130e73740 1 Processor -- start 2024-09-10T07:47:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff130e73740 1 -- start start 2024-09-10T07:47:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff130e73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ede40 0x55aa19710f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff130e73740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ee760 0x55aa19711470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff130e73740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196f9020 0x55aa196ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff130e73740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281b0c50 con 0x55aa196f9020 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff130e73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281b2790 con 0x55aa196ede40 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff130e73740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281b1320 con 0x55aa196ee760 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196f9020 0x55aa196ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196f9020 0x55aa196ef890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50698/0 (socket says 172.21.15.46:50698) 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff127e00640 1 -- 172.21.15.46:0/3753512410 learned_addr learned my addr 172.21.15.46:0/3753512410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff127400640 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ede40 0x55aa19710f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.649+0000 7ff11fe00640 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ee760 0x55aa19711470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff127e00640 1 -- 172.21.15.46:0/3753512410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ee760 msgr2=0x55aa19711470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff127e00640 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ee760 0x55aa19711470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff127e00640 1 -- 172.21.15.46:0/3753512410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ede40 msgr2=0x55aa19710f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:41.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff127e00640 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ede40 0x55aa19710f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff127e00640 1 -- 172.21.15.46:0/3753512410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa196ea200 con 0x55aa196f9020 2024-09-10T07:47:41.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff127400640 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ede40 0x55aa19710f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:41.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff127e00640 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196f9020 0x55aa196ef890 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff120098310 tx=0x7ff120005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869b1852c64994f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff117e00640 1 -- 172.21.15.46:0/3753512410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200b4070 con 0x55aa196f9020 2024-09-10T07:47:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff117e00640 1 -- 172.21.15.46:0/3753512410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1200a4500 con 0x55aa196f9020 2024-09-10T07:47:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.650+0000 7ff117e00640 1 -- 172.21.15.46:0/3753512410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12001f040 con 0x55aa196f9020 2024-09-10T07:47:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 -- 172.21.15.46:0/3753512410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196f9020 msgr2=0x55aa196ef890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196f9020 0x55aa196ef890 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff120098310 tx=0x7ff120005840 comp rx=0 tx=0).stop 2024-09-10T07:47:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 -- 172.21.15.46:0/3753512410 shutdown_connections 2024-09-10T07:47:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196f9020 0x55aa196ef890 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ee760 0x55aa19711470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 --2- 172.21.15.46:0/3753512410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196ede40 0x55aa19710f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 -- 172.21.15.46:0/3753512410 >> 172.21.15.46:0/3753512410 conn(0x55aa196ea3d0 msgr2=0x55aa196f5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 -- 172.21.15.46:0/3753512410 shutdown_connections 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.651+0000 7ff130e73740 1 -- 172.21.15.46:0/3753512410 wait complete. 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 1 -- start start 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 1 Processor -- start 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281b20c0 con 0x55aa196ee760 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281b19f0 con 0x55aa196f9020 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281aec30 con 0x55aa196ede40 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.652+0000 7ff130e73740 10 client.0 _ll_register_callbacks cb 0x7ff118011600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff127e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50714/0 (socket says 172.21.15.46:50714) 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 -- 172.21.15.46:0/1405076425 learned_addr learned my addr 172.21.15.46:0/1405076425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff127400640 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 -- 172.21.15.46:0/1405076425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 msgr2=0x55aa196fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 -- 172.21.15.46:0/1405076425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 msgr2=0x55aa196fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 -- 172.21.15.46:0/1405076425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa196e9c80 con 0x55aa196ee760 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff127e00640 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff11fe00640 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196fd3c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff110004620 tx=0x7ff1100045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42289b13e7467c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.653+0000 7ff115600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.654+0000 7ff115600640 1 -- 172.21.15.46:0/1405076425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110051020 con 0x55aa196ee760 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.654+0000 7ff130e73740 1 -- 172.21.15.46:0/1405076425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1200a4500 con 0x55aa196ee760 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.654+0000 7ff115600640 1 -- 172.21.15.46:0/1405076425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff110005240 con 0x55aa196ee760 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.654+0000 7ff115600640 1 -- 172.21.15.46:0/1405076425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff110044040 con 0x55aa196ee760 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.654+0000 7ff130e73740 10 client.17451 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.654+0000 7ff130e73740 1 -- 172.21.15.46:0/1405076425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55aa196eb060 con 0x55aa196ee760 2024-09-10T07:47:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.654+0000 7ff115600640 1 -- 172.21.15.46:0/1405076425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1200a4500 con 0x55aa196ee760 2024-09-10T07:47:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.654+0000 7ff115600640 1 -- 172.21.15.46:0/1405076425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7ff11004a450 con 0x55aa196ee760 2024-09-10T07:47:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff130e73740 10 client.17451 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff130e73740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff130e73740 20 client.17451 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff130e73740 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aa19774190 0x55aa19776580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff130e73740 4 client.17451 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:47:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff130e73740 1 -- 172.21.15.46:0/1405076425 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa1965be60 con 0x55aa19774190 2024-09-10T07:47:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff127400640 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aa19774190 0x55aa19776580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff127400640 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aa19774190 0x55aa19776580 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d288a259c6966476 server_cookie=f2dc2abf5d88251b in_seq=0 out_seq=0 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.655+0000 7ff115600640 10 client.17451 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff115600640 1 -- 172.21.15.46:0/1405076425 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa1965be60 con 0x55aa19774190 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff115600640 10 client.17451 handle_command_reply: tid=0 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 client.17451 shutdown 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 -- 172.21.15.46:0/1405076425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 msgr2=0x55aa196fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196fd3c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff110004620 tx=0x7ff1100045b0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 -- 172.21.15.46:0/1405076425 shutdown_connections 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55aa19774190 0x55aa19776580 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa196f9020 0x55aa196fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa196ee760 0x55aa196fd3c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 --2- 172.21.15.46:0/1405076425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa196ede40 0x55aa196fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 -- 172.21.15.46:0/1405076425 >> 172.21.15.46:0/1405076425 conn(0x55aa196ea3d0 msgr2=0x55aa196e8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff115600640 0 client.17451 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 -- 172.21.15.46:0/1405076425 shutdown_connections 2024-09-10T07:47:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 1 -- 172.21.15.46:0/1405076425 wait complete. 2024-09-10T07:47:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.656+0000 7ff130e73740 20 client.17451 trim_cache size 0 max 16384 2024-09-10T07:47:41.661 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:47:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 -- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff10c0a2b80 msgr2=0x7ff10c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff10c0a2b80 0x7ff10c0a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55aa196eb230 tx=0x7ff11800b4a0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 -- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff128108f60 msgr2=0x7ff12819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff128108f60 0x7ff12819ed30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff12000e940 tx=0x7ff12000ee00 comp rx=0 tx=0).stop 2024-09-10T07:47:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 -- 172.21.15.46:0/3527160834 shutdown_connections 2024-09-10T07:47:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff10c0a2b80 0x7ff10c0a5040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff128108f60 0x7ff12819ed30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1281047e0 0x7ff12819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 --2- 172.21.15.46:0/3527160834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff128103e30 0x7ff12819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 -- 172.21.15.46:0/3527160834 >> 172.21.15.46:0/3527160834 conn(0x7ff1280ff8d0 msgr2=0x7ff128100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 -- 172.21.15.46:0/3527160834 shutdown_connections 2024-09-10T07:47:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:41.657+0000 7ff12ce00640 1 -- 172.21.15.46:0/3527160834 wait complete. 2024-09-10T07:47:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:41 smithi181 ceph-mon[35397]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.8 MiB/s wr, 524 op/s 2024-09-10T07:47:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:41 smithi063 ceph-mon[35020]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.8 MiB/s wr, 524 op/s 2024-09-10T07:47:42.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:41 smithi046 ceph-mon[30705]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.8 MiB/s wr, 524 op/s 2024-09-10T07:47:43.447 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:47:43.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:47:43.834 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:44.305 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:43 smithi046 ceph-mon[30705]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.0 MiB/s wr, 452 op/s 2024-09-10T07:47:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:43 smithi181 ceph-mon[35397]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.0 MiB/s wr, 452 op/s 2024-09-10T07:47:44.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:43 smithi063 ceph-mon[35020]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.0 MiB/s wr, 452 op/s 2024-09-10T07:47:44.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 -- 172.21.15.46:0/3085665511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed81012a0 msgr2=0x7f8ed8105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/3085665511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed81012a0 0x7f8ed8105770 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009900 tx=0x7f8ec002a470 comp rx=0 tx=0).stop 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 -- 172.21.15.46:0/3085665511 shutdown_connections 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/3085665511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed8105d90 0x7f8ed8106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/3085665511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed81012a0 0x7f8ed8105770 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/3085665511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed8100980 0x7f8ed8100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 -- 172.21.15.46:0/3085665511 >> 172.21.15.46:0/3085665511 conn(0x7f8ed80fc610 msgr2=0x7f8ed80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 -- 172.21.15.46:0/3085665511 shutdown_connections 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.422+0000 7f8ed7e00640 1 -- 172.21.15.46:0/3085665511 wait complete. 2024-09-10T07:47:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.423+0000 7f8ed7e00640 1 Processor -- start 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.423+0000 7f8ed7e00640 1 -- start start 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8100980 0x7f8ed81a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed81012a0 0x7f8ed81a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed8105d90 0x7f8ed819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8118400 con 0x7f8ed8100980 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8118280 con 0x7f8ed8105d90 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed7e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8118580 con 0x7f8ed81012a0 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8100980 0x7f8ed81a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8100980 0x7f8ed81a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50744/0 (socket says 172.21.15.46:50744) 2024-09-10T07:47:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed6a00640 1 -- 172.21.15.46:0/2247947947 learned_addr learned my addr 172.21.15.46:0/2247947947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ed7400640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed8105d90 0x7f8ed819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.424+0000 7f8ece000640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed81012a0 0x7f8ed81a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8ed6a00640 1 -- 172.21.15.46:0/2247947947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed81012a0 msgr2=0x7f8ed81a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed81012a0 0x7f8ed81a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8ed6a00640 1 -- 172.21.15.46:0/2247947947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed8105d90 msgr2=0x7f8ed819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed8105d90 0x7f8ed819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8ed6a00640 1 -- 172.21.15.46:0/2247947947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed819ef00 con 0x7f8ed8100980 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8100980 0x7f8ed81a45c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800ea60 tx=0x7f8ec800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8edca00640 1 -- 172.21.15.46:0/2247947947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800cde0 con 0x7f8ed8100980 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8edca00640 1 -- 172.21.15.46:0/2247947947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec80035f0 con 0x7f8ed8100980 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8edca00640 1 -- 172.21.15.46:0/2247947947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8010670 con 0x7f8ed8100980 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8ed7e00640 1 -- 172.21.15.46:0/2247947947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed819f1f0 con 0x7f8ed8100980 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.425+0000 7f8ed7e00640 1 -- 172.21.15.46:0/2247947947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed81ab370 con 0x7f8ed8100980 2024-09-10T07:47:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.428+0000 7f8ee087e740 1 Processor -- start 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.428+0000 7f8ee087e740 1 -- start start 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8edca00640 1 -- 172.21.15.46:0/2247947947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ec80037a0 con 0x7f8ed8100980 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7ccb0 0x5576a1e7d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e87950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1e8a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ee087e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5576a1de3290 con 0x5576a1e87e90 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ee087e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5576a1d8ab40 con 0x5576a1e7d5d0 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ee087e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5576a1ddf250 con 0x5576a1e7ccb0 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ece000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1e8a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ed6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e87950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ece000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1e8a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50760/0 (socket says 172.21.15.46:50760) 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ece000640 1 -- 172.21.15.46:0/1878677690 learned_addr learned my addr 172.21.15.46:0/1878677690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.429+0000 7f8ed7400640 1 --2- 172.21.15.46:0/1878677690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7ccb0 0x5576a1e7d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ed6a00640 1 -- 172.21.15.46:0/1878677690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7ccb0 msgr2=0x5576a1e7d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/1878677690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7ccb0 0x5576a1e7d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ed6a00640 1 -- 172.21.15.46:0/1878677690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 msgr2=0x5576a1e8a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/1878677690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1e8a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ed6a00640 1 -- 172.21.15.46:0/1878677690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a1b546a0 con 0x5576a1e7d5d0 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/1878677690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e87950 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80ccdb0 tx=0x7f8ec80d5ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46c26e95f3036ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ec7e00640 1 -- 172.21.15.46:0/1878677690 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80de070 con 0x5576a1e7d5d0 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ec7e00640 1 -- 172.21.15.46:0/1878677690 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec80d10d0 con 0x5576a1e7d5d0 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8edca00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ebc0a2b80 0x7f8ebc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.430+0000 7f8ec7e00640 1 -- 172.21.15.46:0/1878677690 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80da680 con 0x5576a1e7d5d0 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8edca00640 1 -- 172.21.15.46:0/2247947947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8ec80cf6f0 con 0x7f8ed8100980 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ed7400640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ebc0a2b80 0x7f8ebc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 -- 172.21.15.46:0/1878677690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 msgr2=0x5576a1e87950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1878677690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e87950 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80ccdb0 tx=0x7f8ec80d5ef0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 -- 172.21.15.46:0/1878677690 shutdown_connections 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1878677690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1e8a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1878677690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e87950 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1878677690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7ccb0 0x5576a1e7d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 -- 172.21.15.46:0/1878677690 >> 172.21.15.46:0/1878677690 conn(0x5576a1e79240 msgr2=0x5576a1e79620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 -- 172.21.15.46:0/1878677690 shutdown_connections 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.431+0000 7f8ee087e740 1 -- 172.21.15.46:0/1878677690 wait complete. 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.432+0000 7f8ed7400640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ebc0a2b80 0x7f8ebc0a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0005fd0 tx=0x7f8ed0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.432+0000 7f8ee087e740 1 -- start start 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.432+0000 7f8ee087e740 1 Processor -- start 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e7ccb0 0x5576a1e876e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e84990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e87e90 0x5576a1e84ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ee087e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5576a1e8ab00 con 0x5576a1e7ccb0 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ee087e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5576a1d9dfa0 con 0x5576a1e7d5d0 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ee087e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5576a1e8ac80 con 0x5576a1e87e90 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ee087e740 10 client.0 _ll_register_callbacks cb 0x5576a1af1ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ed6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e7ccb0 0x5576a1e876e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ece000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e84990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ece000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e84990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47606/0 (socket says 172.21.15.46:47606) 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ece000640 1 -- 172.21.15.46:0/1544983107 learned_addr learned my addr 172.21.15.46:0/1544983107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ed7400640 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e87e90 0x5576a1e84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ece000640 1 -- 172.21.15.46:0/1544983107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e87e90 msgr2=0x5576a1e84ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ece000640 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e87e90 0x5576a1e84ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ece000640 1 -- 172.21.15.46:0/1544983107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e7ccb0 msgr2=0x5576a1e876e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ece000640 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e7ccb0 0x5576a1e876e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.433+0000 7f8ece000640 1 -- 172.21.15.46:0/1544983107 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a1e856e0 con 0x5576a1e7d5d0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ece000640 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e84990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005c40 tx=0x7f8ec00042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be8b528cdf86d466 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ec4c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ec4c00640 1 -- 172.21.15.46:0/1544983107 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0035040 con 0x5576a1e7d5d0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ec4c00640 1 -- 172.21.15.46:0/1544983107 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8ec0005450 con 0x5576a1e7d5d0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ee087e740 1 -- 172.21.15.46:0/1544983107 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec80d10d0 con 0x5576a1e7d5d0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ec4c00640 1 -- 172.21.15.46:0/1544983107 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8ec004f050 con 0x5576a1e7d5d0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ee087e740 10 client.28160 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ee087e740 1 -- 172.21.15.46:0/1544983107 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5576a1e96950 con 0x5576a1e7d5d0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.434+0000 7f8ec4c00640 1 -- 172.21.15.46:0/1544983107 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec80d10d0 con 0x5576a1e7d5d0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.435+0000 7f8ec4c00640 1 -- 172.21.15.46:0/1544983107 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f8ec0003760 con 0x5576a1e7d5d0 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.435+0000 7f8ee087e740 10 client.28160 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.435+0000 7f8ee087e740 10 client.28160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.435+0000 7f8ee087e740 20 client.28160 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.435+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5576a1e97e10 0x5576a1e9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.435+0000 7f8ee087e740 4 client.28160 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.435+0000 7f8ee087e740 1 -- 172.21.15.46:0/1544983107 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576a1df0800 con 0x5576a1e97e10 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.435+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5576a1e97e10 0x5576a1e9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.436+0000 7f8ec4c00640 10 client.28160 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.436+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5576a1e97e10 0x5576a1e9a240 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f406753bd7fa50e server_cookie=2771b0f09468e2c7 in_seq=0 out_seq=0 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.436+0000 7f8ec4c00640 1 -- 172.21.15.46:0/1544983107 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576a1df0800 con 0x5576a1e97e10 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.436+0000 7f8ec4c00640 10 client.28160 handle_command_reply: tid=0 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.436+0000 7f8ee087e740 1 client.28160 shutdown 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 -- 172.21.15.46:0/1544983107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 msgr2=0x5576a1e84990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e84990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005c40 tx=0x7f8ec00042e0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 -- 172.21.15.46:0/1544983107 shutdown_connections 2024-09-10T07:47:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5576a1e97e10 0x5576a1e9a240 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e87e90 0x5576a1e84ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7d5d0 0x5576a1e84990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ec4c00640 0 client.28160 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 --2- 172.21.15.46:0/1544983107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e7ccb0 0x5576a1e876e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 -- 172.21.15.46:0/1544983107 >> 172.21.15.46:0/1544983107 conn(0x5576a1e79240 msgr2=0x5576a1e7ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 -- 172.21.15.46:0/1544983107 shutdown_connections 2024-09-10T07:47:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 1 -- 172.21.15.46:0/1544983107 wait complete. 2024-09-10T07:47:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.437+0000 7f8ee087e740 20 client.28160 trim_cache size 0 max 16384 2024-09-10T07:47:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ee087e740 1 Processor -- start 2024-09-10T07:47:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ee087e740 1 -- start start 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e80000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e9fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1ea0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ee087e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81b50f0 con 0x5576a1e87e90 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ee087e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81b6c30 con 0x5576a1e7ccb0 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ee087e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81b57c0 con 0x5576a1e7d5d0 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ed7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1ea0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ed7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1ea0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50778/0 (socket says 172.21.15.46:50778) 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ed7400640 1 -- 172.21.15.46:0/131815219 learned_addr learned my addr 172.21.15.46:0/131815219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.456+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e80000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ece000640 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e9fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ed7400640 1 -- 172.21.15.46:0/131815219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 msgr2=0x5576a1e9fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ed7400640 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e9fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ed7400640 1 -- 172.21.15.46:0/131815219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 msgr2=0x5576a1e80000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ed7400640 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e80000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ed7400640 1 -- 172.21.15.46:0/131815219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a1e83940 con 0x5576a1e87e90 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e80000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ece000640 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e9fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ed7400640 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1ea0050 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0003ef0 tx=0x7f8ed0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c71e4c9212ac2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.457+0000 7f8ec7400640 1 -- 172.21.15.46:0/131815219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0004600 con 0x5576a1e87e90 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ec7400640 1 -- 172.21.15.46:0/131815219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed00047a0 con 0x5576a1e87e90 2024-09-10T07:47:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ec7400640 1 -- 172.21.15.46:0/131815219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0005930 con 0x5576a1e87e90 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ee087e740 1 -- 172.21.15.46:0/131815219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 msgr2=0x5576a1ea0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ee087e740 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1ea0050 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0003ef0 tx=0x7f8ed0016000 comp rx=0 tx=0).stop 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ee087e740 1 -- 172.21.15.46:0/131815219 shutdown_connections 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ee087e740 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1ea0050 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ee087e740 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e9fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ee087e740 1 --2- 172.21.15.46:0/131815219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e80000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ee087e740 1 -- 172.21.15.46:0/131815219 >> 172.21.15.46:0/131815219 conn(0x5576a1e79240 msgr2=0x5576a1e7f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.458+0000 7f8ee087e740 1 -- 172.21.15.46:0/131815219 shutdown_connections 2024-09-10T07:47:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.459+0000 7f8ee087e740 1 -- 172.21.15.46:0/131815219 wait complete. 2024-09-10T07:47:44.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.459+0000 7f8ee087e740 1 -- start start 2024-09-10T07:47:44.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.459+0000 7f8ee087e740 1 Processor -- start 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e8c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ee087e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1f01ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ee087e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81b6560 con 0x5576a1e87e90 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ee087e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81b5e90 con 0x5576a1e7ccb0 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ee087e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81b30d0 con 0x5576a1e7d5d0 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ee087e740 10 client.0 _ll_register_callbacks cb 0x7f8ed00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1f01ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e8c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47614/0 (socket says 172.21.15.46:47614) 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed6a00640 1 -- 172.21.15.46:0/3149608288 learned_addr learned my addr 172.21.15.46:0/3149608288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ece000640 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e8c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed6a00640 1 -- 172.21.15.46:0/3149608288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 msgr2=0x5576a1e8c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e8c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed6a00640 1 -- 172.21.15.46:0/3149608288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 msgr2=0x5576a1f01ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1f01ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed6a00640 1 -- 172.21.15.46:0/3149608288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a1f02020 con 0x5576a1e7ccb0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.460+0000 7f8ed7400640 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1f01ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ed6a00640 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e8c090 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ec809bb00 tx=0x7f8ec80cfa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec54d5a9b9df440c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ec4c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ec4c00640 1 -- 172.21.15.46:0/3149608288 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80de040 con 0x5576a1e7ccb0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ec4c00640 1 -- 172.21.15.46:0/3149608288 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8ec80cfcf0 con 0x5576a1e7ccb0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ee087e740 1 -- 172.21.15.46:0/3149608288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed00047a0 con 0x5576a1e7ccb0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ec4c00640 1 -- 172.21.15.46:0/3149608288 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8097e10 con 0x5576a1e7ccb0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ee087e740 10 client.28166 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ee087e740 1 -- 172.21.15.46:0/3149608288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5576a1f03240 con 0x5576a1e7ccb0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.461+0000 7f8ec4c00640 1 -- 172.21.15.46:0/3149608288 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed00047a0 con 0x5576a1e7ccb0 2024-09-10T07:47:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.462+0000 7f8ec4c00640 1 -- 172.21.15.46:0/3149608288 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f8ec80cce90 con 0x5576a1e7ccb0 2024-09-10T07:47:44.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.462+0000 7f8ee087e740 10 client.28166 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:44.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.462+0000 7f8ee087e740 10 client.28166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:44.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.462+0000 7f8ee087e740 20 client.28166 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:44.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.462+0000 7f8ee087e740 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5576a1f03530 0x5576a1f05920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.462+0000 7f8ee087e740 4 client.28166 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:47:44.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.462+0000 7f8ee087e740 1 -- 172.21.15.46:0/3149608288 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5576a1dead10 con 0x5576a1f03530 2024-09-10T07:47:44.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.462+0000 7f8ece000640 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5576a1f03530 0x5576a1f05920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.463+0000 7f8ec4c00640 10 client.28166 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:44.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.463+0000 7f8ece000640 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5576a1f03530 0x5576a1f05920 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1d946cf982d79 server_cookie=fcb19d31eecbdbca in_seq=0 out_seq=0 2024-09-10T07:47:44.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ec4c00640 1 -- 172.21.15.46:0/3149608288 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5576a1dead10 con 0x5576a1f03530 2024-09-10T07:47:44.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ec4c00640 10 client.28166 handle_command_reply: tid=0 2024-09-10T07:47:44.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 client.28166 shutdown 2024-09-10T07:47:44.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 -- 172.21.15.46:0/3149608288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 msgr2=0x5576a1e8c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e8c090 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ec809bb00 tx=0x7f8ec80cfa00 comp rx=0 tx=0).stop 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 -- 172.21.15.46:0/3149608288 shutdown_connections 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5576a1f03530 0x5576a1f05920 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576a1e87e90 0x5576a1f01ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576a1e7d5d0 0x5576a1e8c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 --2- 172.21.15.46:0/3149608288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576a1e7ccb0 0x5576a1e8c090 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 -- 172.21.15.46:0/3149608288 >> 172.21.15.46:0/3149608288 conn(0x5576a1e79240 msgr2=0x5576a1e78270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ec4c00640 0 client.28166 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.464+0000 7f8ee087e740 1 -- 172.21.15.46:0/3149608288 shutdown_connections 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.465+0000 7f8ee087e740 1 -- 172.21.15.46:0/3149608288 wait complete. 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.465+0000 7f8ee087e740 20 client.28166 trim_cache size 0 max 16384 2024-09-10T07:47:44.466 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:47:44.467 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:47:44.467 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "d29d5c7e-bf7f-4ad6-9d8b-e91d8fd018cc", 2024-09-10T07:47:44.467 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:47:44.467 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:47:44.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 -- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ebc0a2b80 msgr2=0x7f8ebc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ebc0a2b80 0x7f8ebc0a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0005fd0 tx=0x7f8ed0009500 comp rx=0 tx=0).stop 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 -- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8100980 msgr2=0x7f8ed81a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8100980 0x7f8ed81a45c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800ea60 tx=0x7f8ec800ef20 comp rx=0 tx=0).stop 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 -- 172.21.15.46:0/2247947947 shutdown_connections 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ed8105d90 0x7f8ed819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed81012a0 0x7f8ed81a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ebc0a2b80 0x7f8ebc0a5040 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 --2- 172.21.15.46:0/2247947947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8100980 0x7f8ed81a45c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 -- 172.21.15.46:0/2247947947 >> 172.21.15.46:0/2247947947 conn(0x7f8ed80fc610 msgr2=0x7f8ed8103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 -- 172.21.15.46:0/2247947947 shutdown_connections 2024-09-10T07:47:44.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:44.466+0000 7f8ed7e00640 1 -- 172.21.15.46:0/2247947947 wait complete. 2024-09-10T07:47:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:45 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:47:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:45 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:45 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:47:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:45 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:47:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:45 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:45 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:47:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:45 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:47:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:45 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:45 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:47:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:45 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:47:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:45 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:45 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:45 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:47:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:45 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:45 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:47:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:45 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:47:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:45 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:45 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:45.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 625.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:47:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:46 smithi046 ceph-mon[30705]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.4 MiB/s wr, 381 op/s 2024-09-10T07:47:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.219+0000 7f72c4600640 1 -- 172.21.15.46:0/3141907469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc109140 msgr2=0x7f72bc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.219+0000 7f72c4600640 1 --2- 172.21.15.46:0/3141907469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc109140 0x7f72bc110cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009ca0 tx=0x7f72b402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.220+0000 7f72c4600640 1 -- 172.21.15.46:0/3141907469 shutdown_connections 2024-09-10T07:47:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.220+0000 7f72c4600640 1 --2- 172.21.15.46:0/3141907469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc109140 0x7f72bc110cc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.220+0000 7f72c4600640 1 --2- 172.21.15.46:0/3141907469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc1049c0 0x7f72bc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.220+0000 7f72c4600640 1 --2- 172.21.15.46:0/3141907469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72bc104010 0x7f72bc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.220+0000 7f72c4600640 1 -- 172.21.15.46:0/3141907469 >> 172.21.15.46:0/3141907469 conn(0x7f72bc0ffa50 msgr2=0x7f72bc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.220+0000 7f72c4600640 1 -- 172.21.15.46:0/3141907469 shutdown_connections 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.220+0000 7f72c4600640 1 -- 172.21.15.46:0/3141907469 wait complete. 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.221+0000 7f72c4600640 1 Processor -- start 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.221+0000 7f72c4600640 1 -- start start 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.221+0000 7f72c4600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc104010 0x7f72bc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.221+0000 7f72c4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc1049c0 0x7f72bc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.221+0000 7f72c4600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72bc109140 0x7f72bc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.221+0000 7f72c4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc114150 con 0x7f72bc1049c0 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.221+0000 7f72c4600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc113fd0 con 0x7f72bc109140 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.221+0000 7f72c4600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1142d0 con 0x7f72bc104010 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc1049c0 0x7f72bc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc104010 0x7f72bc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc104010 0x7f72bc19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53834/0 (socket says 172.21.15.46:53834) 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1c00640 1 -- 172.21.15.46:0/4004355099 learned_addr learned my addr 172.21.15.46:0/4004355099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc1049c0 0x7f72bc19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37276/0 (socket says 172.21.15.46:37276) 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c2600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72bc109140 0x7f72bc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1200640 1 -- 172.21.15.46:0/4004355099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc104010 msgr2=0x7f72bc19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1200640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc104010 0x7f72bc19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1200640 1 -- 172.21.15.46:0/4004355099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72bc109140 msgr2=0x7f72bc19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1200640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72bc109140 0x7f72bc19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1200640 1 -- 172.21.15.46:0/4004355099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc19f5a0 con 0x7f72bc1049c0 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.222+0000 7f72c1200640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc1049c0 0x7f72bc19ab30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f72a800e9f0 tx=0x7f72a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.223+0000 7f72ba000640 1 -- 172.21.15.46:0/4004355099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a800ccc0 con 0x7f72bc1049c0 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.223+0000 7f72ba000640 1 -- 172.21.15.46:0/4004355099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a8010910 con 0x7f72bc1049c0 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.223+0000 7f72c4600640 1 -- 172.21.15.46:0/4004355099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc19f890 con 0x7f72bc1049c0 2024-09-10T07:47:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.223+0000 7f72ba000640 1 -- 172.21.15.46:0/4004355099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8018960 con 0x7f72bc1049c0 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.223+0000 7f72c4600640 1 -- 172.21.15.46:0/4004355099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc1a7170 con 0x7f72bc1049c0 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.226+0000 7f72ba000640 1 -- 172.21.15.46:0/4004355099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72a8010ac0 con 0x7f72bc1049c0 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.226+0000 7f72c6fba740 1 Processor -- start 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.226+0000 7f72c6fba740 1 -- start start 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c6fba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c0571243e0 con 0x55c0571bdb90 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c6fba740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c0570cbd80 con 0x55c0571be4b0 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c6fba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0571203a0 con 0x55c0571c8d70 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72ba000640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f72a40a2b80 0x7f72a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571bdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37286/0 (socket says 172.21.15.46:37286) 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c2600640 1 -- 172.21.15.46:0/13371867 learned_addr learned my addr 172.21.15.46:0/13371867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c1200640 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72c1c00640 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.227+0000 7f72ba000640 1 -- 172.21.15.46:0/4004355099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f72a80cf740 con 0x7f72bc1049c0 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c2600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f72a40a2b80 0x7f72a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c1c00640 1 -- 172.21.15.46:0/13371867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 msgr2=0x55c0571cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c1c00640 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c1c00640 1 -- 172.21.15.46:0/13371867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 msgr2=0x55c0571bdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c1c00640 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571bdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c1c00640 1 -- 172.21.15.46:0/13371867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0570eaae0 con 0x55c0571be4b0 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c2600640 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571bdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c1c00640 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571c8830 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009c80 tx=0x7f72b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c81f2431c91989 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72c2600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f72a40a2b80 0x7f72a40a5040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f72b4004af0 tx=0x7f72b402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.228+0000 7f72aea00640 1 -- 172.21.15.46:0/13371867 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0004230 con 0x55c0571be4b0 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72aea00640 1 -- 172.21.15.46:0/13371867 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b00043d0 con 0x55c0571be4b0 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72aea00640 1 -- 172.21.15.46:0/13371867 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0004d90 con 0x55c0571be4b0 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 -- 172.21.15.46:0/13371867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 msgr2=0x55c0571c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571c8830 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009c80 tx=0x7f72b0011810 comp rx=0 tx=0).stop 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 -- 172.21.15.46:0/13371867 shutdown_connections 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571c8830 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 --2- 172.21.15.46:0/13371867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571bdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 -- 172.21.15.46:0/13371867 >> 172.21.15.46:0/13371867 conn(0x55c0571ba120 msgr2=0x55c0571ba500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 -- 172.21.15.46:0/13371867 shutdown_connections 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.229+0000 7f72c6fba740 1 -- 172.21.15.46:0/13371867 wait complete. 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.230+0000 7f72c6fba740 1 -- start start 2024-09-10T07:47:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.230+0000 7f72c6fba740 1 Processor -- start 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571be4b0 0x55c0571c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c6fba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c0571cb9e0 con 0x55c0571be4b0 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c6fba740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c0570df0f0 con 0x55c0571bdb90 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c6fba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0571cbb60 con 0x55c0571c8d70 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c6fba740 10 client.0 _ll_register_callbacks cb 0x55c0570d09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56492/0 (socket says 172.21.15.46:56492) 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1c00640 1 -- 172.21.15.46:0/776687903 learned_addr learned my addr 172.21.15.46:0/776687903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1200640 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571be4b0 0x55c0571c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c2600640 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1c00640 1 -- 172.21.15.46:0/776687903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 msgr2=0x55c0571c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1c00640 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1c00640 1 -- 172.21.15.46:0/776687903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571be4b0 msgr2=0x55c0571c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1c00640 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571be4b0 0x55c0571c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.231+0000 7f72c1c00640 1 -- 172.21.15.46:0/776687903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c056e953c0 con 0x55c0571bdb90 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.232+0000 7f72c1c00640 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571c5bb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f72b00048f0 tx=0x7f72b0019ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a9ccc970e2b4d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.232+0000 7f72a3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.232+0000 7f72a3400640 1 -- 172.21.15.46:0/776687903 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0028020 con 0x55c0571bdb90 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.232+0000 7f72c6fba740 1 -- 172.21.15.46:0/776687903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b00043d0 con 0x55c0571bdb90 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.232+0000 7f72a3400640 1 -- 172.21.15.46:0/776687903 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f72b0024620 con 0x55c0571bdb90 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.232+0000 7f72a3400640 1 -- 172.21.15.46:0/776687903 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f72b0033050 con 0x55c0571bdb90 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.232+0000 7f72c6fba740 10 client.28178 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.232+0000 7f72c6fba740 1 -- 172.21.15.46:0/776687903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55c0571d7640 con 0x55c0571bdb90 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72a3400640 1 -- 172.21.15.46:0/776687903 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b00043d0 con 0x55c0571bdb90 2024-09-10T07:47:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72a3400640 1 -- 172.21.15.46:0/776687903 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f72b0018070 con 0x55c0571bdb90 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72c6fba740 10 client.28178 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72c6fba740 10 client.28178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72c6fba740 20 client.28178 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72c6fba740 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0571d8b00 0x55c0571daef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72c6fba740 4 client.28178 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72c6fba740 1 -- 172.21.15.46:0/776687903 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c057131950 con 0x55c0571d8b00 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.233+0000 7f72c1200640 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0571d8b00 0x55c0571daef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.234+0000 7f72c1200640 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0571d8b00 0x55c0571daef0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ecb0a2e0d1e1fd server_cookie=f9cbdf19081c1a97 in_seq=0 out_seq=0 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.234+0000 7f72a3400640 10 client.28178 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.234+0000 7f72a3400640 1 -- 172.21.15.46:0/776687903 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c057131950 con 0x55c0571d8b00 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.234+0000 7f72a3400640 10 client.28178 handle_command_reply: tid=0 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 client.28178 shutdown 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 -- 172.21.15.46:0/776687903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 msgr2=0x55c0571c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571c5bb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f72b00048f0 tx=0x7f72b0019ad0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 -- 172.21.15.46:0/776687903 shutdown_connections 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0571d8b00 0x55c0571daef0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571be4b0 0x55c0571c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 --2- 172.21.15.46:0/776687903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571c5bb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 -- 172.21.15.46:0/776687903 >> 172.21.15.46:0/776687903 conn(0x55c0571ba120 msgr2=0x55c0571bfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72a3400640 0 client.28178 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 -- 172.21.15.46:0/776687903 shutdown_connections 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 1 -- 172.21.15.46:0/776687903 wait complete. 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.235+0000 7f72c6fba740 20 client.28178 trim_cache size 0 max 16384 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.253+0000 7f72c6fba740 1 Processor -- start 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.253+0000 7f72c6fba740 1 -- start start 2024-09-10T07:47:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571e0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c6fba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1b0e20 con 0x55c0571bdb90 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c6fba740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1b2960 con 0x55c0571be4b0 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c6fba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1b14f0 con 0x55c0571c8d70 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37306/0 (socket says 172.21.15.46:37306) 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1c00640 1 -- 172.21.15.46:0/1460480166 learned_addr learned my addr 172.21.15.46:0/1460480166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1200640 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571e0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c2600640 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571e0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1c00640 1 -- 172.21.15.46:0/1460480166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 msgr2=0x55c0571e0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1c00640 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571e0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1c00640 1 -- 172.21.15.46:0/1460480166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 msgr2=0x55c0571e0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1c00640 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571e0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.254+0000 7f72c1c00640 1 -- 172.21.15.46:0/1460480166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0571b9f50 con 0x55c0571bdb90 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.255+0000 7f72c1c00640 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571c4e30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f72b0019aa0 tx=0x7f72b0019a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d9c86a2c84e4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.255+0000 7f72ae000640 1 -- 172.21.15.46:0/1460480166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0028070 con 0x55c0571bdb90 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.255+0000 7f72ae000640 1 -- 172.21.15.46:0/1460480166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b001f040 con 0x55c0571bdb90 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.255+0000 7f72ae000640 1 -- 172.21.15.46:0/1460480166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00043b0 con 0x55c0571bdb90 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 -- 172.21.15.46:0/1460480166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 msgr2=0x55c0571c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571c4e30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f72b0019aa0 tx=0x7f72b0019a30 comp rx=0 tx=0).stop 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 -- 172.21.15.46:0/1460480166 shutdown_connections 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571c8d70 0x55c0571e0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571be4b0 0x55c0571e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1460480166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571bdb90 0x55c0571c4e30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 -- 172.21.15.46:0/1460480166 >> 172.21.15.46:0/1460480166 conn(0x55c0571ba120 msgr2=0x55c0571c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 -- 172.21.15.46:0/1460480166 shutdown_connections 2024-09-10T07:47:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 -- 172.21.15.46:0/1460480166 wait complete. 2024-09-10T07:47:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.256+0000 7f72c6fba740 1 -- start start 2024-09-10T07:47:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c6fba740 1 Processor -- start 2024-09-10T07:47:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571be4b0 0x55c0571d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c6fba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571c8d70 0x55c0571c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c6fba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1b2290 con 0x55c0571c8d70 2024-09-10T07:47:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c6fba740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1b1bc0 con 0x55c0571bdb90 2024-09-10T07:47:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c6fba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1aee00 con 0x55c0571be4b0 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c6fba740 10 client.0 _ll_register_callbacks cb 0x7f72b0009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571c8d70 0x55c0571c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56502/0 (socket says 172.21.15.46:56502) 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.257+0000 7f72c1c00640 1 -- 172.21.15.46:0/1790641432 learned_addr learned my addr 172.21.15.46:0/1790641432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72c1200640 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571be4b0 0x55c0571d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72c1c00640 1 -- 172.21.15.46:0/1790641432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571be4b0 msgr2=0x55c0571d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72c1c00640 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571be4b0 0x55c0571d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72c1c00640 1 -- 172.21.15.46:0/1790641432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571c8d70 msgr2=0x55c0571c23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72c1c00640 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571c8d70 0x55c0571c23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72c1c00640 1 -- 172.21.15.46:0/1790641432 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0571c2ad0 con 0x55c0571bdb90 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72c1c00640 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571d0f90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f72b0011850 tx=0x7f72b0009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6deed529fe1fd526 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72a3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.258+0000 7f72a3400640 1 -- 172.21.15.46:0/1790641432 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0041060 con 0x55c0571bdb90 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.259+0000 7f72a3400640 1 -- 172.21.15.46:0/1790641432 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f72b00053c0 con 0x55c0571bdb90 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.259+0000 7f72c6fba740 1 -- 172.21.15.46:0/1790641432 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b001f040 con 0x55c0571bdb90 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.259+0000 7f72a3400640 1 -- 172.21.15.46:0/1790641432 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f72b0028070 con 0x55c0571bdb90 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.259+0000 7f72c6fba740 10 client.28184 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.259+0000 7f72c6fba740 1 -- 172.21.15.46:0/1790641432 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55c057243100 con 0x55c0571bdb90 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.259+0000 7f72a3400640 1 -- 172.21.15.46:0/1790641432 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b001f040 con 0x55c0571bdb90 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.259+0000 7f72a3400640 1 -- 172.21.15.46:0/1790641432 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f72b0018070 con 0x55c0571bdb90 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72c6fba740 10 client.28184 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72c6fba740 10 client.28184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72c6fba740 20 client.28184 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0572437a0 0x55c057245bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72c6fba740 4 client.28184 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72c6fba740 1 -- 172.21.15.46:0/1790641432 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c05712be60 con 0x55c0572437a0 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72c1200640 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0572437a0 0x55c057245bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72c1200640 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0572437a0 0x55c057245bc0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b03330c72446be server_cookie=a34578f83efe945a in_seq=0 out_seq=0 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.260+0000 7f72a3400640 10 client.28184 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72a3400640 1 -- 172.21.15.46:0/1790641432 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c05712be60 con 0x55c0572437a0 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72a3400640 10 client.28184 handle_command_reply: tid=0 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 client.28184 shutdown 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 -- 172.21.15.46:0/1790641432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 msgr2=0x55c0571d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571d0f90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f72b0011850 tx=0x7f72b0009ef0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 -- 172.21.15.46:0/1790641432 shutdown_connections 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c0572437a0 0x55c057245bc0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0571c8d70 0x55c0571c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0571be4b0 0x55c0571d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 --2- 172.21.15.46:0/1790641432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0571bdb90 0x55c0571d0f90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72a3400640 0 client.28184 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 -- 172.21.15.46:0/1790641432 >> 172.21.15.46:0/1790641432 conn(0x55c0571ba120 msgr2=0x55c0571bf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 -- 172.21.15.46:0/1790641432 shutdown_connections 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 1 -- 172.21.15.46:0/1790641432 wait complete. 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.261+0000 7f72c6fba740 20 client.28184 trim_cache size 0 max 16384 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:47:46.274 INFO:teuthology.orchestra.run.smithi046.stdout: "d29d5c7e-bf7f-4ad6-9d8b-e91d8fd018cc": { 2024-09-10T07:47:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:47:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "d29d5c7e-bf7f-4ad6-9d8b-e91d8fd018cc", 2024-09-10T07:47:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:47:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:46.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:46.275 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 -- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f72a40a2b80 msgr2=0x7f72a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f72a40a2b80 0x7f72a40a5040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f72b4004af0 tx=0x7f72b402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 -- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc1049c0 msgr2=0x7f72bc19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc1049c0 0x7f72bc19ab30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f72a800e9f0 tx=0x7f72a800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 -- 172.21.15.46:0/4004355099 shutdown_connections 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72bc109140 0x7f72bc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f72a40a2b80 0x7f72a40a5040 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc1049c0 0x7f72bc19ab30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 --2- 172.21.15.46:0/4004355099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc104010 0x7f72bc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 -- 172.21.15.46:0/4004355099 >> 172.21.15.46:0/4004355099 conn(0x7f72bc0ffa50 msgr2=0x7f72bc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 -- 172.21.15.46:0/4004355099 shutdown_connections 2024-09-10T07:47:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:46.262+0000 7f72c4600640 1 -- 172.21.15.46:0/4004355099 wait complete. 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:46 smithi181 ceph-mon[35397]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.4 MiB/s wr, 381 op/s 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:47:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:47:46.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:47:46.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:46 smithi063 ceph-mon[35020]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.4 MiB/s wr, 381 op/s 2024-09-10T07:47:47.082 INFO:tasks.cephfs.filesystem:scrub status for tag:d29d5c7e-bf7f-4ad6-9d8b-e91d8fd018cc - {'path': '/', 'tag': 'd29d5c7e-bf7f-4ad6-9d8b-e91d8fd018cc', 'options': 'recursive,force'} 2024-09-10T07:47:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.651+0000 7f2639600640 1 -- 172.21.15.46:0/4075197839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 msgr2=0x7f26341051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.651+0000 7f2639600640 1 --2- 172.21.15.46:0/4075197839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 0x7f26341051e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2620009900 tx=0x7f262002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.652+0000 7f2639600640 1 -- 172.21.15.46:0/4075197839 shutdown_connections 2024-09-10T07:47:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.652+0000 7f2639600640 1 --2- 172.21.15.46:0/4075197839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634105720 0x7f263410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.652+0000 7f2639600640 1 --2- 172.21.15.46:0/4075197839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 0x7f26341051e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.652+0000 7f2639600640 1 --2- 172.21.15.46:0/4075197839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2634103b70 0x7f2634103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.652+0000 7f2639600640 1 -- 172.21.15.46:0/4075197839 >> 172.21.15.46:0/4075197839 conn(0x7f26340ff310 msgr2=0x7f2634101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.652+0000 7f2639600640 1 -- 172.21.15.46:0/4075197839 shutdown_connections 2024-09-10T07:47:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.652+0000 7f2639600640 1 -- 172.21.15.46:0/4075197839 wait complete. 2024-09-10T07:47:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.652+0000 7f2639600640 1 Processor -- start 2024-09-10T07:47:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2639600640 1 -- start start 2024-09-10T07:47:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2639600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634103b70 0x7f2634199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2639600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 0x7f263419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2639600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2634105720 0x7f26341a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2639600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f263410f7f0 con 0x7f2634104d60 2024-09-10T07:47:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2639600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f263410f670 con 0x7f2634105720 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2639600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f263410f970 con 0x7f2634103b70 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2633400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 0x7f263419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2633400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 0x7f263419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37320/0 (socket says 172.21.15.46:37320) 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2633400640 1 -- 172.21.15.46:0/169295336 learned_addr learned my addr 172.21.15.46:0/169295336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2633e00640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634103b70 0x7f2634199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.653+0000 7f2638c00640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2634105720 0x7f26341a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2633400640 1 -- 172.21.15.46:0/169295336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634103b70 msgr2=0x7f2634199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2633400640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634103b70 0x7f2634199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2633400640 1 -- 172.21.15.46:0/169295336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2634105720 msgr2=0x7f26341a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2633400640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2634105720 0x7f26341a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2633400640 1 -- 172.21.15.46:0/169295336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26341a1a90 con 0x7f2634104d60 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2633400640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 0x7f263419a3b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2620009230 tx=0x7f26200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2630c00640 1 -- 172.21.15.46:0/169295336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262003f030 con 0x7f2634104d60 2024-09-10T07:47:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2630c00640 1 -- 172.21.15.46:0/169295336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620003ce0 con 0x7f2634104d60 2024-09-10T07:47:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2630c00640 1 -- 172.21.15.46:0/169295336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620035070 con 0x7f2634104d60 2024-09-10T07:47:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2639600640 1 -- 172.21.15.46:0/169295336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26341a1d20 con 0x7f2634104d60 2024-09-10T07:47:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.654+0000 7f2639600640 1 -- 172.21.15.46:0/169295336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26341a21c0 con 0x7f2634104d60 2024-09-10T07:47:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.657+0000 7f263d9f7740 1 Processor -- start 2024-09-10T07:47:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.657+0000 7f2630c00640 1 -- 172.21.15.46:0/169295336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2620042030 con 0x7f2634104d60 2024-09-10T07:47:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.657+0000 7f263d9f7740 1 -- start start 2024-09-10T07:47:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184260 0x55fb04184660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb04189660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04189c30 0x55fb0418c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f263d9f7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fb040e9100 con 0x55fb04184260 2024-09-10T07:47:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f263d9f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fb040911b0 con 0x55fb04184ba0 2024-09-10T07:47:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f263d9f7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fb040e50c0 con 0x55fb04189c30 2024-09-10T07:47:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2638c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184260 0x55fb04184660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2638c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184260 0x55fb04184660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37328/0 (socket says 172.21.15.46:37328) 2024-09-10T07:47:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2638c00640 1 -- 172.21.15.46:0/4163647796 learned_addr learned my addr 172.21.15.46:0/4163647796 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2633e00640 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb04189660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2633400640 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04189c30 0x55fb0418c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2633e00640 1 -- 172.21.15.46:0/4163647796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04189c30 msgr2=0x55fb0418c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2633e00640 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04189c30 0x55fb0418c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2633e00640 1 -- 172.21.15.46:0/4163647796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184260 msgr2=0x55fb04184660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2633e00640 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184260 0x55fb04184660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2633e00640 1 -- 172.21.15.46:0/4163647796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb03dfc210 con 0x55fb04184ba0 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.658+0000 7f2638c00640 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184260 0x55fb04184660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.659+0000 7f2633e00640 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb04189660 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2624009bf0 tx=0x7f2624011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc2a84e6c673e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.659+0000 7f2630c00640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26180a2c00 0x7f26180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.659+0000 7f2629200640 1 -- 172.21.15.46:0/4163647796 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624009da0 con 0x55fb04184ba0 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.659+0000 7f2630c00640 1 -- 172.21.15.46:0/169295336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f26200ef7f0 con 0x7f2634104d60 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.659+0000 7f2629200640 1 -- 172.21.15.46:0/4163647796 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2624005020 con 0x55fb04184ba0 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.659+0000 7f2629200640 1 -- 172.21.15.46:0/4163647796 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624004b20 con 0x55fb04184ba0 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.659+0000 7f2638c00640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26180a2c00 0x7f26180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 -- 172.21.15.46:0/4163647796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 msgr2=0x55fb04189660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb04189660 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2624009bf0 tx=0x7f2624011810 comp rx=0 tx=0).stop 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 -- 172.21.15.46:0/4163647796 shutdown_connections 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04189c30 0x55fb0418c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb04189660 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 --2- 172.21.15.46:0/4163647796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184260 0x55fb04184660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 -- 172.21.15.46:0/4163647796 >> 172.21.15.46:0/4163647796 conn(0x55fb0417f8d0 msgr2=0x55fb0417fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f2638c00640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26180a2c00 0x7f26180a50c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f262c011890 tx=0x7f262c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 -- 172.21.15.46:0/4163647796 shutdown_connections 2024-09-10T07:47:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 -- 172.21.15.46:0/4163647796 wait complete. 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.660+0000 7f263d9f7740 1 -- start start 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f263d9f7740 1 Processor -- start 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0418e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb0418e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb04195910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f263d9f7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fb0418cc00 con 0x55fb04189c30 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f263d9f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fb04072090 con 0x55fb04184ba0 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f263d9f7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fb0418cd80 con 0x55fb04184260 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f263d9f7740 10 client.0 _ll_register_callbacks cb 0x55fb040c6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f2638c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb04195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f2633e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0418e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f2638c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb04195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37334/0 (socket says 172.21.15.46:37334) 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f2638c00640 1 -- 172.21.15.46:0/244776044 learned_addr learned my addr 172.21.15.46:0/244776044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.661+0000 7f2633400640 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb0418e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f2633e00640 1 -- 172.21.15.46:0/244776044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 msgr2=0x55fb0418e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f2633e00640 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb0418e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f2633e00640 1 -- 172.21.15.46:0/244776044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 msgr2=0x55fb04195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f2633e00640 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb04195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f2633e00640 1 -- 172.21.15.46:0/244776044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb03ec8620 con 0x55fb04184260 2024-09-10T07:47:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f2633e00640 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0418e390 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f262401b040 tx=0x7f2624009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47a84df05689a779 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f261d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:47:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f261d600640 1 -- 172.21.15.46:0/244776044 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624029020 con 0x55fb04184260 2024-09-10T07:47:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f263d9f7740 1 -- 172.21.15.46:0/244776044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2624005020 con 0x55fb04184260 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.662+0000 7f261d600640 1 -- 172.21.15.46:0/244776044 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2624004a90 con 0x55fb04184260 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f261d600640 1 -- 172.21.15.46:0/244776044 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2624034050 con 0x55fb04184260 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f263d9f7740 10 client.28141 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f263d9f7740 1 -- 172.21.15.46:0/244776044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55fb04198750 con 0x55fb04184260 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f261d600640 1 -- 172.21.15.46:0/244776044 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2624005020 con 0x55fb04184260 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f261d600640 1 -- 172.21.15.46:0/244776044 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f2624019450 con 0x55fb04184260 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f263d9f7740 10 client.28141 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f263d9f7740 10 client.28141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f263d9f7740 20 client.28141 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f263d9f7740 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb04199cd0 0x55fb0419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f263d9f7740 4 client.28141 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f263d9f7740 1 -- 172.21.15.46:0/244776044 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb040f6830 con 0x55fb04199cd0 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.663+0000 7f2633400640 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb04199cd0 0x55fb0419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.664+0000 7f2633400640 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb04199cd0 0x55fb0419c0c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c978fc6b2647ea2 server_cookie=caaf4220c5831f49 in_seq=0 out_seq=0 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.664+0000 7f261d600640 10 client.28141 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f261d600640 1 -- 172.21.15.46:0/244776044 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb040f6830 con 0x55fb04199cd0 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f261d600640 10 client.28141 handle_command_reply: tid=0 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 client.28141 shutdown 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 -- 172.21.15.46:0/244776044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 msgr2=0x55fb0418e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0418e390 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f262401b040 tx=0x7f2624009fd0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 -- 172.21.15.46:0/244776044 shutdown_connections 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb04199cd0 0x55fb0419c0c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb04195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb0418e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 --2- 172.21.15.46:0/244776044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0418e390 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 -- 172.21.15.46:0/244776044 >> 172.21.15.46:0/244776044 conn(0x55fb0417f8d0 msgr2=0x55fb041810e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f261d600640 0 client.28141 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 -- 172.21.15.46:0/244776044 shutdown_connections 2024-09-10T07:47:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 1 -- 172.21.15.46:0/244776044 wait complete. 2024-09-10T07:47:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.665+0000 7f263d9f7740 20 client.28141 trim_cache size 0 max 16384 2024-09-10T07:47:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.680+0000 7f263d9f7740 1 Processor -- start 2024-09-10T07:47:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.680+0000 7f263d9f7740 1 -- start start 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.680+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0417fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.680+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb041958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.680+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb041801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.680+0000 7f263d9f7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26341ac740 con 0x55fb04189c30 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.680+0000 7f263d9f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26341ae280 con 0x55fb04184ba0 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.680+0000 7f263d9f7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26341ace10 con 0x55fb04184260 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2638c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb041801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2638c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb041801f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37336/0 (socket says 172.21.15.46:37336) 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2638c00640 1 -- 172.21.15.46:0/77722610 learned_addr learned my addr 172.21.15.46:0/77722610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2633400640 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb041958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2633e00640 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0417fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2638c00640 1 -- 172.21.15.46:0/77722610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 msgr2=0x55fb0417fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2638c00640 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0417fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2638c00640 1 -- 172.21.15.46:0/77722610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 msgr2=0x55fb041958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2638c00640 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb041958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2638c00640 1 -- 172.21.15.46:0/77722610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb0417f6f0 con 0x55fb04189c30 2024-09-10T07:47:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.681+0000 7f2633400640 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb041958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f2638c00640 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb041801f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f262c003cc0 tx=0x7f262c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7aefd73644acb9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f261fe00640 1 -- 172.21.15.46:0/77722610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c00fe40 con 0x55fb04189c30 2024-09-10T07:47:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f261fe00640 1 -- 172.21.15.46:0/77722610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c003cf0 con 0x55fb04189c30 2024-09-10T07:47:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f261fe00640 1 -- 172.21.15.46:0/77722610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c004850 con 0x55fb04189c30 2024-09-10T07:47:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f263d9f7740 1 -- 172.21.15.46:0/77722610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 msgr2=0x55fb041801f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f263d9f7740 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb041801f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f262c003cc0 tx=0x7f262c016000 comp rx=0 tx=0).stop 2024-09-10T07:47:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f263d9f7740 1 -- 172.21.15.46:0/77722610 shutdown_connections 2024-09-10T07:47:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f263d9f7740 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04189c30 0x55fb041801f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f263d9f7740 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04184ba0 0x55fb041958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f263d9f7740 1 --2- 172.21.15.46:0/77722610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb0417fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.682+0000 7f263d9f7740 1 -- 172.21.15.46:0/77722610 >> 172.21.15.46:0/77722610 conn(0x55fb0417f8d0 msgr2=0x55fb0418e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 -- 172.21.15.46:0/77722610 shutdown_connections 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 -- 172.21.15.46:0/77722610 wait complete. 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 -- start start 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 Processor -- start 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb04192a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184ba0 0x55fb04192f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04189c30 0x55fb04193f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26341adbb0 con 0x55fb04184ba0 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.683+0000 7f263d9f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26341ad4e0 con 0x55fb04189c30 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f263d9f7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26341aa720 con 0x55fb04184260 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f263d9f7740 10 client.0 _ll_register_callbacks cb 0x7f262c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184ba0 0x55fb04192f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb04192a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184ba0 0x55fb04192f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37352/0 (socket says 172.21.15.46:37352) 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 -- 172.21.15.46:0/2656989134 learned_addr learned my addr 172.21.15.46:0/2656989134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2638c00640 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04189c30 0x55fb04193f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 -- 172.21.15.46:0/2656989134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 msgr2=0x55fb04192a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb04192a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 -- 172.21.15.46:0/2656989134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04189c30 msgr2=0x55fb04193f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04189c30 0x55fb04193f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 -- 172.21.15.46:0/2656989134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb04194560 con 0x55fb04184ba0 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f2633400640 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184ba0 0x55fb04192f90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26200b86f0 tx=0x7f26200b8080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a4269c2338c81e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f261d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.684+0000 7f261d600640 1 -- 172.21.15.46:0/2656989134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620049040 con 0x55fb04184ba0 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f261d600640 1 -- 172.21.15.46:0/2656989134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f262003b460 con 0x55fb04184ba0 2024-09-10T07:47:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 1 -- 172.21.15.46:0/2656989134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f262c003cf0 con 0x55fb04184ba0 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f261d600640 1 -- 172.21.15.46:0/2656989134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f26200bb680 con 0x55fb04184ba0 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 10 client.17505 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 1 -- 172.21.15.46:0/2656989134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55fb041958e0 con 0x55fb04184ba0 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f261d600640 1 -- 172.21.15.46:0/2656989134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c003cf0 con 0x55fb04184ba0 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f261d600640 1 -- 172.21.15.46:0/2656989134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f26200b8390 con 0x55fb04184ba0 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 10 client.17505 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 20 client.17505 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb04204d40 0x55fb04207160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 4 client.17505 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.685+0000 7f263d9f7740 1 -- 172.21.15.46:0/2656989134 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb040f0b80 con 0x55fb04204d40 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.686+0000 7f2633e00640 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb04204d40 0x55fb04207160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.686+0000 7f261d600640 10 client.17505 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.686+0000 7f2633e00640 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb04204d40 0x55fb04207160 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7d0bdc2539ad40 server_cookie=f50daa2cd433c6 in_seq=0 out_seq=0 2024-09-10T07:47:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f261d600640 1 -- 172.21.15.46:0/2656989134 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb040f0b80 con 0x55fb04204d40 2024-09-10T07:47:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f261d600640 10 client.17505 handle_command_reply: tid=0 2024-09-10T07:47:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 client.17505 shutdown 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 -- 172.21.15.46:0/2656989134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184ba0 msgr2=0x55fb04192f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184ba0 0x55fb04192f90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26200b86f0 tx=0x7f26200b8080 comp rx=0 tx=0).stop 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 -- 172.21.15.46:0/2656989134 shutdown_connections 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb04204d40 0x55fb04207160 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb04189c30 0x55fb04193f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb04184ba0 0x55fb04192f90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 --2- 172.21.15.46:0/2656989134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb04184260 0x55fb04192a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f261d600640 0 client.17505 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 -- 172.21.15.46:0/2656989134 >> 172.21.15.46:0/2656989134 conn(0x55fb0417f8d0 msgr2=0x55fb0417e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 -- 172.21.15.46:0/2656989134 shutdown_connections 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 1 -- 172.21.15.46:0/2656989134 wait complete. 2024-09-10T07:47:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.695+0000 7f263d9f7740 20 client.17505 trim_cache size 0 max 16384 2024-09-10T07:47:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:47:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:47:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:47:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 -- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26180a2c00 msgr2=0x7f26180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26180a2c00 0x7f26180a50c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f262c011890 tx=0x7f262c009500 comp rx=0 tx=0).stop 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 -- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 msgr2=0x7f263419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 0x7f263419a3b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2620009230 tx=0x7f26200042f0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 -- 172.21.15.46:0/169295336 shutdown_connections 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2634105720 0x7f26341a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26180a2c00 0x7f26180a50c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2634104d60 0x7f263419a3b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 --2- 172.21.15.46:0/169295336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634103b70 0x7f2634199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 -- 172.21.15.46:0/169295336 >> 172.21.15.46:0/169295336 conn(0x7f26340ff310 msgr2=0x7f2634108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 -- 172.21.15.46:0/169295336 shutdown_connections 2024-09-10T07:47:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:47:47.696+0000 7f2639600640 1 -- 172.21.15.46:0/169295336 wait complete. 2024-09-10T07:47:48.310 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:48 smithi046 ceph-mon[30705]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 4.6 MiB/s wr, 397 op/s 2024-09-10T07:47:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:48 smithi181 ceph-mon[35397]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 4.6 MiB/s wr, 397 op/s 2024-09-10T07:47:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:48 smithi063 ceph-mon[35020]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 4.6 MiB/s wr, 397 op/s 2024-09-10T07:47:50.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:50 smithi046 ceph-mon[30705]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-10T07:47:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:50 smithi181 ceph-mon[35397]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-10T07:47:50.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:50 smithi063 ceph-mon[35020]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-10T07:47:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 630.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:47:52.203 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:47:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:52 smithi181 ceph-mon[35397]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-09-10T07:47:52.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:52 smithi063 ceph-mon[35020]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-09-10T07:47:52.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:52 smithi046 ceph-mon[30705]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-09-10T07:47:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 -- 172.21.15.46:0/421783371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844112a30 msgr2=0x7fd844114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 --2- 172.21.15.46:0/421783371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844112a30 0x7fd844114e20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009ca0 tx=0x7fd83c02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 -- 172.21.15.46:0/421783371 shutdown_connections 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 --2- 172.21.15.46:0/421783371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844112a30 0x7fd844114e20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 --2- 172.21.15.46:0/421783371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd844077040 0x7fd8441124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 --2- 172.21.15.46:0/421783371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844076720 0x7fd844076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 -- 172.21.15.46:0/421783371 >> 172.21.15.46:0/421783371 conn(0x7fd8440ffa90 msgr2=0x7fd844101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 -- 172.21.15.46:0/421783371 shutdown_connections 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.412+0000 7fd84b600640 1 -- 172.21.15.46:0/421783371 wait complete. 2024-09-10T07:47:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.413+0000 7fd84b600640 1 Processor -- start 2024-09-10T07:47:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.413+0000 7fd84b600640 1 -- start start 2024-09-10T07:47:53.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd84b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844076720 0x7fd8441a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:53.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd84b600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844077040 0x7fd8441a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:53.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd84b600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd844112a30 0x7fd84419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:53.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd84b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441185d0 con 0x7fd844076720 2024-09-10T07:47:53.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd84b600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd844118450 con 0x7fd844112a30 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd84b600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd844118750 con 0x7fd844077040 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd848c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844076720 0x7fd8441a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd848c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844076720 0x7fd8441a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37384/0 (socket says 172.21.15.46:37384) 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd848c00640 1 -- 172.21.15.46:0/2987033095 learned_addr learned my addr 172.21.15.46:0/2987033095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd843e00640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844077040 0x7fd8441a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd849600640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd844112a30 0x7fd84419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd848c00640 1 -- 172.21.15.46:0/2987033095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844077040 msgr2=0x7fd8441a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd848c00640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844077040 0x7fd8441a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.414+0000 7fd848c00640 1 -- 172.21.15.46:0/2987033095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd844112a30 msgr2=0x7fd84419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.415+0000 7fd848c00640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd844112a30 0x7fd84419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.415+0000 7fd848c00640 1 -- 172.21.15.46:0/2987033095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84419efd0 con 0x7fd844076720 2024-09-10T07:47:53.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.415+0000 7fd848c00640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844076720 0x7fd8441a46e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd83800e9f0 tx=0x7fd83800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:53.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.415+0000 7fd841600640 1 -- 172.21.15.46:0/2987033095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83800cd80 con 0x7fd844076720 2024-09-10T07:47:53.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.415+0000 7fd841600640 1 -- 172.21.15.46:0/2987033095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd838005500 con 0x7fd844076720 2024-09-10T07:47:53.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.415+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84419f2c0 con 0x7fd844076720 2024-09-10T07:47:53.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.415+0000 7fd841600640 1 -- 172.21.15.46:0/2987033095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838010690 con 0x7fd844076720 2024-09-10T07:47:53.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.415+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8441ab5b0 con 0x7fd844076720 2024-09-10T07:47:53.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.416+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd844076b00 con 0x7fd844076720 2024-09-10T07:47:53.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.418+0000 7fd841600640 1 -- 172.21.15.46:0/2987033095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd8380046c0 con 0x7fd844076720 2024-09-10T07:47:53.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.420+0000 7fd841600640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd82c0a2b80 0x7fd82c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:53.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.421+0000 7fd841600640 1 -- 172.21.15.46:0/2987033095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd8380cf800 con 0x7fd844076720 2024-09-10T07:47:53.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.421+0000 7fd843e00640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd82c0a2b80 0x7fd82c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:53.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.421+0000 7fd843e00640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd82c0a2b80 0x7fd82c0a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd844079290 tx=0x7fd83000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:53.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.425+0000 7fd841600640 1 -- 172.21.15.46:0/2987033095 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd83809c380 con 0x7fd844076720 2024-09-10T07:47:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.650+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd84410dc20 con 0x7fd844076720 2024-09-10T07:47:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.651+0000 7fd841600640 1 -- 172.21.15.46:0/2987033095 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7fd83809bc80 con 0x7fd844076720 2024-09-10T07:47:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:47:53.651 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"btime":"2024-09-10T07:46:53:072255+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:46:52.230107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:47:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-09-10T07:47:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd82c0a2b80 msgr2=0x7fd82c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd82c0a2b80 0x7fd82c0a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd844079290 tx=0x7fd83000b410 comp rx=0 tx=0).stop 2024-09-10T07:47:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844076720 msgr2=0x7fd8441a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844076720 0x7fd8441a46e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd83800e9f0 tx=0x7fd83800edf0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 shutdown_connections 2024-09-10T07:47:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd844112a30 0x7fd84419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844077040 0x7fd8441a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd82c0a2b80 0x7fd82c0a5040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 --2- 172.21.15.46:0/2987033095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844076720 0x7fd8441a46e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.654+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 >> 172.21.15.46:0/2987033095 conn(0x7fd8440ffa90 msgr2=0x7fd844101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.655+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 shutdown_connections 2024-09-10T07:47:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:53.655+0000 7fd84b600640 1 -- 172.21.15.46:0/2987033095 wait complete. 2024-09-10T07:47:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:54 smithi181 ceph-mon[35397]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 538 KiB/s wr, 137 op/s 2024-09-10T07:47:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:54 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2987033095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:54 smithi063 ceph-mon[35020]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 538 KiB/s wr, 137 op/s 2024-09-10T07:47:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:54 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2987033095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:54.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:54 smithi046 ceph-mon[30705]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 538 KiB/s wr, 137 op/s 2024-09-10T07:47:54.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:54 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2987033095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:47:54.592 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id ed2cc4b7 --release --await' 2024-09-10T07:47:54.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id ed2cc4b7 --release --await 2024-09-10T07:47:55.013 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:47:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 635.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:47:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 -- 172.21.15.46:0/1372259175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f956810e3c0 msgr2=0x7f9568110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 --2- 172.21.15.46:0/1372259175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f956810e3c0 0x7f9568110810 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9560009ca0 tx=0x7f956002a950 comp rx=0 tx=0).stop 2024-09-10T07:47:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 -- 172.21.15.46:0/1372259175 shutdown_connections 2024-09-10T07:47:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 --2- 172.21.15.46:0/1372259175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f956810e3c0 0x7f9568110810 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 --2- 172.21.15.46:0/1372259175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9568074f40 0x7f956810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 --2- 172.21.15.46:0/1372259175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9568074620 0x7f9568074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 -- 172.21.15.46:0/1372259175 >> 172.21.15.46:0/1372259175 conn(0x7f95680fd8c0 msgr2=0x7f95680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 -- 172.21.15.46:0/1372259175 shutdown_connections 2024-09-10T07:47:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.635+0000 7f956cc00640 1 -- 172.21.15.46:0/1372259175 wait complete. 2024-09-10T07:47:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.636+0000 7f956cc00640 1 Processor -- start 2024-09-10T07:47:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.637+0000 7f956cc00640 1 -- start start 2024-09-10T07:47:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.637+0000 7f956cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9568074620 0x7f956819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.637+0000 7f956cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9568074f40 0x7f956819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.637+0000 7f956cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f956810e3c0 0x7f956819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.637+0000 7f956cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9568114160 con 0x7f9568074620 2024-09-10T07:47:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.637+0000 7f956cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9568113fe0 con 0x7f956810e3c0 2024-09-10T07:47:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.637+0000 7f956cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95681142e0 con 0x7f9568074f40 2024-09-10T07:47:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f9565e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9568074620 0x7f956819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f9566800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f956810e3c0 0x7f956819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f955fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9568074f40 0x7f956819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f9565e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9568074620 0x7f956819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37408/0 (socket says 172.21.15.46:37408) 2024-09-10T07:47:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f9565e00640 1 -- 172.21.15.46:0/3183625137 learned_addr learned my addr 172.21.15.46:0/3183625137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f955fe00640 1 -- 172.21.15.46:0/3183625137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f956810e3c0 msgr2=0x7f956819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f955fe00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f956810e3c0 0x7f956819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f955fe00640 1 -- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9568074620 msgr2=0x7f956819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f955fe00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9568074620 0x7f956819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f955fe00640 1 -- 172.21.15.46:0/3183625137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956819f5d0 con 0x7f9568074f40 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f9565e00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9568074620 0x7f956819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.638+0000 7f955fe00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9568074f40 0x7f956819ab60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f955000e9f0 tx=0x7f955000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.639+0000 7f955ea00640 1 -- 172.21.15.46:0/3183625137 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955000cd80 con 0x7f9568074f40 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.639+0000 7f956cc00640 1 -- 172.21.15.46:0/3183625137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956819f8c0 con 0x7f9568074f40 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.639+0000 7f955ea00640 1 -- 172.21.15.46:0/3183625137 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95500035f0 con 0x7f9568074f40 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.639+0000 7f955ea00640 1 -- 172.21.15.46:0/3183625137 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550010670 con 0x7f9568074f40 2024-09-10T07:47:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.639+0000 7f956cc00640 1 -- 172.21.15.46:0/3183625137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95681a71a0 con 0x7f9568074f40 2024-09-10T07:47:55.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f956f83c740 1 Processor -- start 2024-09-10T07:47:55.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f956f83c740 1 -- start start 2024-09-10T07:47:55.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f955ea00640 1 -- 172.21.15.46:0/3183625137 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f95500037a0 con 0x7f9568074f40 2024-09-10T07:47:55.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f956f83c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517de6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f956f83c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f956f83c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 0x5620517eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f956f83c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562051744650 con 0x5620517de310 2024-09-10T07:47:55.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f956f83c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5620516ebf10 con 0x5620517e94f0 2024-09-10T07:47:55.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.642+0000 7f956f83c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562051740610 con 0x5620517dec30 2024-09-10T07:47:55.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9566800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517de6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9566800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517de6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37422/0 (socket says 172.21.15.46:37422) 2024-09-10T07:47:55.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9566800640 1 -- 172.21.15.46:0/142810801 learned_addr learned my addr 172.21.15.46:0/142810801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:55.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9565e00640 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f955fe00640 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 0x5620517eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9566800640 1 -- 172.21.15.46:0/142810801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 msgr2=0x5620517e8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9566800640 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517e8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9566800640 1 -- 172.21.15.46:0/142810801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 msgr2=0x5620517eb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9566800640 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 0x5620517eb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.643+0000 7f9566800640 1 -- 172.21.15.46:0/142810801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562051453580 con 0x5620517de310 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.644+0000 7f9566800640 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517de6f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9560035000 tx=0x7f95600093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505c68338409d26a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.644+0000 7f9557000640 1 -- 172.21.15.46:0/142810801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560034040 con 0x5620517de310 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.644+0000 7f9557000640 1 -- 172.21.15.46:0/142810801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f956002f940 con 0x5620517de310 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.644+0000 7f9557000640 1 -- 172.21.15.46:0/142810801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560004280 con 0x5620517de310 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.644+0000 7f955ea00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f954c0a2b80 0x7f954c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.644+0000 7f955ea00640 1 -- 172.21.15.46:0/3183625137 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f95500cf6a0 con 0x7f9568074f40 2024-09-10T07:47:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f9565e00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f954c0a2b80 0x7f954c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 -- 172.21.15.46:0/142810801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 msgr2=0x5620517de6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517de6f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9560035000 tx=0x7f95600093e0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 -- 172.21.15.46:0/142810801 shutdown_connections 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 0x5620517eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517e8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 --2- 172.21.15.46:0/142810801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517de6f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 -- 172.21.15.46:0/142810801 >> 172.21.15.46:0/142810801 conn(0x5620517da8a0 msgr2=0x5620517dac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 -- 172.21.15.46:0/142810801 shutdown_connections 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f956f83c740 1 -- 172.21.15.46:0/142810801 wait complete. 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.645+0000 7f9565e00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f954c0a2b80 0x7f954c0a5040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5620517db700 tx=0x7f9558005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 1 -- start start 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 1 Processor -- start 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517dec30 0x5620517e6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x5620517e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5620517ec0c0 con 0x5620517de310 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5620516ff350 con 0x5620517dec30 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5620517ec240 con 0x5620517e94f0 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.646+0000 7f956f83c740 10 client.0 _ll_register_callbacks cb 0x562051722290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517dec30 0x5620517e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f9565e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517dec30 0x5620517e6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56628/0 (socket says 172.21.15.46:56628) 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 -- 172.21.15.46:0/3273858511 learned_addr learned my addr 172.21.15.46:0/3273858511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f9566800640 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x5620517e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 -- 172.21.15.46:0/3273858511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 msgr2=0x5620517e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x5620517e6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 -- 172.21.15.46:0/3273858511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 msgr2=0x5620517e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 -- 172.21.15.46:0/3273858511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56205151d960 con 0x5620517dec30 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f9566800640 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x5620517e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f955fe00640 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517dec30 0x5620517e6890 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f955009b820 tx=0x7f955009c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f438276f58436b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.647+0000 7f954b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f954b400640 1 -- 172.21.15.46:0/3273858511 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500a4790 con 0x5620517dec30 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f956f83c740 1 -- 172.21.15.46:0/3273858511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f956002f940 con 0x5620517dec30 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f954b400640 1 -- 172.21.15.46:0/3273858511 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f95500cf380 con 0x5620517dec30 2024-09-10T07:47:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f954b400640 1 -- 172.21.15.46:0/3273858511 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9550003b30 con 0x5620517dec30 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f956f83c740 10 client.28220 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f956f83c740 1 -- 172.21.15.46:0/3273858511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5620517f7ca0 con 0x5620517dec30 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f954b400640 1 -- 172.21.15.46:0/3273858511 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f956002f940 con 0x5620517dec30 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f954b400640 1 -- 172.21.15.46:0/3273858511 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f95500d1ba0 con 0x5620517dec30 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f956f83c740 10 client.28220 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.648+0000 7f956f83c740 10 client.28220 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.649+0000 7f956f83c740 20 client.28220 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.649+0000 7f956f83c740 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5620517f8ff0 0x5620517fb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.649+0000 7f956f83c740 4 client.28220 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.649+0000 7f956f83c740 1 -- 172.21.15.46:0/3273858511 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562051751bc0 con 0x5620517f8ff0 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.649+0000 7f9565e00640 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5620517f8ff0 0x5620517fb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.649+0000 7f9565e00640 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5620517f8ff0 0x5620517fb420 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d4a07fb19b52e1 server_cookie=2a81cd69b5a30be5 in_seq=0 out_seq=0 2024-09-10T07:47:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.649+0000 7f954b400640 10 client.28220 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:55.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.650+0000 7f954b400640 1 -- 172.21.15.46:0/3273858511 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562051751bc0 con 0x5620517f8ff0 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.650+0000 7f954b400640 10 client.28220 handle_command_reply: tid=0 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.650+0000 7f956f83c740 1 client.28220 shutdown 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.650+0000 7f956f83c740 1 -- 172.21.15.46:0/3273858511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517dec30 msgr2=0x5620517e6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.650+0000 7f956f83c740 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517dec30 0x5620517e6890 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f955009b820 tx=0x7f955009c220 comp rx=0 tx=0).stop 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 1 -- 172.21.15.46:0/3273858511 shutdown_connections 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5620517f8ff0 0x5620517fb420 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x5620517e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517dec30 0x5620517e6890 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 1 --2- 172.21.15.46:0/3273858511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f954b400640 0 client.28220 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 1 -- 172.21.15.46:0/3273858511 >> 172.21.15.46:0/3273858511 conn(0x5620517da8a0 msgr2=0x5620517d9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 1 -- 172.21.15.46:0/3273858511 shutdown_connections 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 1 -- 172.21.15.46:0/3273858511 wait complete. 2024-09-10T07:47:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.651+0000 7f956f83c740 20 client.28220 trim_cache size 0 max 16384 2024-09-10T07:47:55.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f956f83c740 1 Processor -- start 2024-09-10T07:47:55.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f956f83c740 1 -- start start 2024-09-10T07:47:55.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f956f83c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517de310 0x5620517e8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f956f83c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517dec30 0x562051800cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f956f83c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x562051801230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f956f83c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95681b0dc0 con 0x5620517dec30 2024-09-10T07:47:55.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f956f83c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95681b2900 con 0x5620517de310 2024-09-10T07:47:55.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f956f83c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95681b1490 con 0x5620517e94f0 2024-09-10T07:47:55.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f955fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517dec30 0x562051800cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f955fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517dec30 0x562051800cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37446/0 (socket says 172.21.15.46:37446) 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f955fe00640 1 -- 172.21.15.46:0/1483968974 learned_addr learned my addr 172.21.15.46:0/1483968974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.669+0000 7f9565e00640 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517de310 0x5620517e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f9566800640 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x562051801230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f955fe00640 1 -- 172.21.15.46:0/1483968974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 msgr2=0x562051801230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f955fe00640 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x562051801230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f955fe00640 1 -- 172.21.15.46:0/1483968974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517de310 msgr2=0x5620517e8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f955fe00640 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517de310 0x5620517e8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f955fe00640 1 -- 172.21.15.46:0/1483968974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620517d6260 con 0x5620517dec30 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f9565e00640 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517de310 0x5620517e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f955fe00640 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517dec30 0x562051800cf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f95500984f0 tx=0x7f955009b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dba707b9f869d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f9556600640 1 -- 172.21.15.46:0/1483968974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500a4790 con 0x5620517dec30 2024-09-10T07:47:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f9556600640 1 -- 172.21.15.46:0/1483968974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955009ce50 con 0x5620517dec30 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.670+0000 7f9556600640 1 -- 172.21.15.46:0/1483968974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550098190 con 0x5620517dec30 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 -- 172.21.15.46:0/1483968974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517dec30 msgr2=0x562051800cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517dec30 0x562051800cf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f95500984f0 tx=0x7f955009b930 comp rx=0 tx=0).stop 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 -- 172.21.15.46:0/1483968974 shutdown_connections 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517e94f0 0x562051801230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517dec30 0x562051800cf0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 --2- 172.21.15.46:0/1483968974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517de310 0x5620517e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 -- 172.21.15.46:0/1483968974 >> 172.21.15.46:0/1483968974 conn(0x5620517da8a0 msgr2=0x5620517d9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 -- 172.21.15.46:0/1483968974 shutdown_connections 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 -- 172.21.15.46:0/1483968974 wait complete. 2024-09-10T07:47:55.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.671+0000 7f956f83c740 1 -- start start 2024-09-10T07:47:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f956f83c740 1 Processor -- start 2024-09-10T07:47:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f956f83c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517edb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f956f83c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f956f83c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 0x56205185cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f956f83c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95681b2230 con 0x5620517de310 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f956f83c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95681b1b60 con 0x5620517e94f0 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f956f83c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95681aeda0 con 0x5620517dec30 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f956f83c740 10 client.0 _ll_register_callbacks cb 0x7f9550003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f9565e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517edb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f9565e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517edb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37460/0 (socket says 172.21.15.46:37460) 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.672+0000 7f9565e00640 1 -- 172.21.15.46:0/1131127808 learned_addr learned my addr 172.21.15.46:0/1131127808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f955fe00640 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f9566800640 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 0x56205185cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f9565e00640 1 -- 172.21.15.46:0/1131127808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 msgr2=0x5620517ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f9565e00640 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f9565e00640 1 -- 172.21.15.46:0/1131127808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 msgr2=0x56205185cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f9565e00640 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 0x56205185cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f9565e00640 1 -- 172.21.15.46:0/1131127808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56205185d2d0 con 0x5620517de310 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f955fe00640 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f9565e00640 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517edb20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9558005b40 tx=0x7f95580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c1c63c90781449 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.673+0000 7f954b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f954b400640 1 -- 172.21.15.46:0/1131127808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558033430 con 0x5620517de310 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f954b400640 1 -- 172.21.15.46:0/1131127808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f955802a420 con 0x5620517de310 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 1 -- 172.21.15.46:0/1131127808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f955009ce50 con 0x5620517de310 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f954b400640 1 -- 172.21.15.46:0/1131127808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f955802a620 con 0x5620517de310 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 10 client.17529 fetch_fsmap learned FSMap version 50 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 1 -- 172.21.15.46:0/1131127808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562051864700 con 0x5620517de310 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f954b400640 1 -- 172.21.15.46:0/1131127808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955009ce50 con 0x5620517de310 2024-09-10T07:47:55.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f954b400640 1 -- 172.21.15.46:0/1131127808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9558033c80 con 0x5620517de310 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 10 client.17529 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 10 client.17529 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 20 client.17529 populate_metadata read hostname 'smithi046' 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562051864e50 0x562051867270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 4 client.17529 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ed2cc4b7", "release": true, "await": true}] 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.674+0000 7f956f83c740 1 -- 172.21.15.46:0/1131127808 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ed2cc4b7", "release": true, "await": true}) -- 0x56205174c0d0 con 0x562051864e50 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.675+0000 7f955fe00640 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562051864e50 0x562051867270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.675+0000 7f955fe00640 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562051864e50 0x562051867270 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac68d6907c1cb54f server_cookie=654a864dea462b0b in_seq=0 out_seq=0 2024-09-10T07:47:55.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.675+0000 7f954b400640 10 client.17529 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.705+0000 7f954b400640 1 -- 172.21.15.46:0/1131127808 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56205174c0d0 con 0x562051864e50 2024-09-10T07:47:55.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.705+0000 7f954b400640 10 client.17529 handle_command_reply: tid=0 2024-09-10T07:47:55.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.705+0000 7f956f83c740 1 client.17529 shutdown 2024-09-10T07:47:55.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.705+0000 7f956f83c740 1 -- 172.21.15.46:0/1131127808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 msgr2=0x5620517edb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.705+0000 7f956f83c740 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517edb20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9558005b40 tx=0x7f95580046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f956f83c740 1 -- 172.21.15.46:0/1131127808 shutdown_connections 2024-09-10T07:47:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f956f83c740 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562051864e50 0x562051867270 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f956f83c740 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5620517e94f0 0x56205185cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f956f83c740 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5620517dec30 0x5620517ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f956f83c740 1 --2- 172.21.15.46:0/1131127808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620517de310 0x5620517edb20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f956f83c740 1 -- 172.21.15.46:0/1131127808 >> 172.21.15.46:0/1131127808 conn(0x5620517da8a0 msgr2=0x5620517d9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f954b400640 0 client.17529 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:47:55.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f956f83c740 1 -- 172.21.15.46:0/1131127808 shutdown_connections 2024-09-10T07:47:55.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.706+0000 7f956f83c740 1 -- 172.21.15.46:0/1131127808 wait complete. 2024-09-10T07:47:55.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.707+0000 7f956f83c740 20 client.17529 trim_cache size 0 max 16384 2024-09-10T07:47:55.707 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:47:55.707 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 50, 2024-09-10T07:47:55.707 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:47:55.707 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 40, 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "ed2cc4b7": { 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 40, 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 145.2, 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:47:55.708 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:47:55.709 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:47:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.708+0000 7f956cc00640 1 -- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f954c0a2b80 msgr2=0x7f954c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.708+0000 7f956cc00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f954c0a2b80 0x7f954c0a5040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5620517db700 tx=0x7f9558005f90 comp rx=0 tx=0).stop 2024-09-10T07:47:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.709+0000 7f956cc00640 1 -- 172.21.15.46:0/3183625137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9568074f40 msgr2=0x7f956819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.709+0000 7f956cc00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9568074f40 0x7f956819ab60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f955000e9f0 tx=0x7f955000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.709+0000 7f956cc00640 1 -- 172.21.15.46:0/3183625137 shutdown_connections 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.709+0000 7f956cc00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f956810e3c0 0x7f956819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.709+0000 7f956cc00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9568074f40 0x7f956819ab60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.709+0000 7f956cc00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f954c0a2b80 0x7f954c0a5040 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.709+0000 7f956cc00640 1 --2- 172.21.15.46:0/3183625137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9568074620 0x7f956819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.709+0000 7f956cc00640 1 -- 172.21.15.46:0/3183625137 >> 172.21.15.46:0/3183625137 conn(0x7f95680fd8c0 msgr2=0x7f95680fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.710+0000 7f956cc00640 1 -- 172.21.15.46:0/3183625137 shutdown_connections 2024-09-10T07:47:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:47:55.710+0000 7f956cc00640 1 -- 172.21.15.46:0/3183625137 wait complete. 2024-09-10T07:47:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:56 smithi181 ceph-mon[35397]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 KiB/s wr, 100 op/s 2024-09-10T07:47:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:56 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:47:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:56 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:47:56.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:56 smithi063 ceph-mon[35020]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 KiB/s wr, 100 op/s 2024-09-10T07:47:56.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:56 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:47:56.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:56 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:47:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:56 smithi046 ceph-mon[30705]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 KiB/s wr, 100 op/s 2024-09-10T07:47:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:56 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:47:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:56 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:47:57.057 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ed2cc4b7', total seconds elapsed: 30.1 2024-09-10T07:47:57.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:57 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:47:57.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:57 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:47:57.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:57 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:47:57.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:57 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:47:57.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:57 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:47:57.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:57 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:47:57.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:57 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:47:57.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:57 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:47:57.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:57 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:47:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:47:58 smithi181 ceph-mon[35397]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 187 KiB/s wr, 103 op/s 2024-09-10T07:47:58.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:47:58 smithi063 ceph-mon[35020]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 187 KiB/s wr, 103 op/s 2024-09-10T07:47:58.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:47:58 smithi046 ceph-mon[30705]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 187 KiB/s wr, 103 op/s 2024-09-10T07:48:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:00 smithi181 ceph-mon[35397]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 185 KiB/s wr, 34 op/s 2024-09-10T07:48:00.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:00 smithi063 ceph-mon[35020]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 185 KiB/s wr, 34 op/s 2024-09-10T07:48:00.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:00 smithi046 ceph-mon[30705]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 185 KiB/s wr, 34 op/s 2024-09-10T07:48:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 640.0 s, 631.2 tps, lat 723.100 ms stddev 3160.275 2024-09-10T07:48:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.947+0000 7f97d8a00640 1 -- 172.21.15.46:0/4245811012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97d4103710 msgr2=0x7f97d4109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.947+0000 7f97d8a00640 1 --2- 172.21.15.46:0/4245811012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97d4103710 0x7f97d4109fa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009900 tx=0x7f97c402a420 comp rx=0 tx=0).stop 2024-09-10T07:48:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.947+0000 7f97d8a00640 1 -- 172.21.15.46:0/4245811012 shutdown_connections 2024-09-10T07:48:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.947+0000 7f97d8a00640 1 --2- 172.21.15.46:0/4245811012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97d4103710 0x7f97d4109fa0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.947+0000 7f97d8a00640 1 --2- 172.21.15.46:0/4245811012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d4102d50 0x7f97d41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.947+0000 7f97d8a00640 1 --2- 172.21.15.46:0/4245811012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97d4101b60 0x7f97d4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.947+0000 7f97d8a00640 1 -- 172.21.15.46:0/4245811012 >> 172.21.15.46:0/4245811012 conn(0x7f97d40fd2e0 msgr2=0x7f97d40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.948+0000 7f97d8a00640 1 -- 172.21.15.46:0/4245811012 shutdown_connections 2024-09-10T07:48:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.948+0000 7f97d8a00640 1 -- 172.21.15.46:0/4245811012 wait complete. 2024-09-10T07:48:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.949+0000 7f97d8a00640 1 Processor -- start 2024-09-10T07:48:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.950+0000 7f97d8a00640 1 -- start start 2024-09-10T07:48:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.950+0000 7f97d8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97d4101b60 0x7f97d41078e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.950+0000 7f97d8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97d4102d50 0x7f97d4107e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.950+0000 7f97d8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d4103710 0x7f97d4104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.950+0000 7f97d8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97d410d860 con 0x7f97d4101b60 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.950+0000 7f97d8a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97d410d6e0 con 0x7f97d4102d50 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.950+0000 7f97d8a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97d410d9e0 con 0x7f97d4103710 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d4103710 0x7f97d4104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d4103710 0x7f97d4104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60636/0 (socket says 172.21.15.46:60636) 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 -- 172.21.15.46:0/3557601970 learned_addr learned my addr 172.21.15.46:0/3557601970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d1400640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97d4102d50 0x7f97d4107e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 -- 172.21.15.46:0/3557601970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97d4102d50 msgr2=0x7f97d4107e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97d4102d50 0x7f97d4107e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 -- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97d4101b60 msgr2=0x7f97d41078e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97d4101b60 0x7f97d41078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 -- 172.21.15.46:0/3557601970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d4104d10 con 0x7f97d4103710 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.951+0000 7f97d2800640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d4103710 0x7f97d4104450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f97c40314e0 tx=0x7f97c4032c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.952+0000 7f97caa00640 1 -- 172.21.15.46:0/3557601970 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c403f030 con 0x7f97d4103710 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.952+0000 7f97d8a00640 1 -- 172.21.15.46:0/3557601970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d41a6470 con 0x7f97d4103710 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.952+0000 7f97caa00640 1 -- 172.21.15.46:0/3557601970 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c4003ad0 con 0x7f97d4103710 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.952+0000 7f97caa00640 1 -- 172.21.15.46:0/3557601970 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4035070 con 0x7f97d4103710 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.952+0000 7f97d8a00640 1 -- 172.21.15.46:0/3557601970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d41a68a0 con 0x7f97d4103710 2024-09-10T07:48:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.952+0000 7f97d1400640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97d4102d50 0x7f97d4107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.959+0000 7f97caa00640 1 -- 172.21.15.46:0/3557601970 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97c4042030 con 0x7f97d4103710 2024-09-10T07:48:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.963+0000 7f97caa00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f97b80a2c80 0x7f97b80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.965+0000 7f97caa00640 1 -- 172.21.15.46:0/3557601970 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f97c40ef210 con 0x7f97d4103710 2024-09-10T07:48:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.966+0000 7f97d1e00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f97b80a2c80 0x7f97b80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.967+0000 7f97d1e00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f97b80a2c80 0x7f97b80a5140 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f97cc005c40 tx=0x7f97cc005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.979+0000 7f97db9d8740 1 Processor -- start 2024-09-10T07:48:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.979+0000 7f97db9d8740 1 -- start start 2024-09-10T07:48:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.979+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed6090 0x56297bed6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.979+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 0x56297bedb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.979+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bedba60 0x56297beddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.979+0000 7f97db9d8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56297be3b100 con 0x56297bedba60 2024-09-10T07:48:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.979+0000 7f97db9d8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56297bde31b0 con 0x56297bed6090 2024-09-10T07:48:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.979+0000 7f97db9d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56297be370c0 con 0x56297bed69d0 2024-09-10T07:48:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.980+0000 7f97d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bedba60 0x56297beddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.980+0000 7f97d1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 0x56297bedb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.980+0000 7f97d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bedba60 0x56297beddf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58192/0 (socket says 172.21.15.46:58192) 2024-09-10T07:48:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.980+0000 7f97d2800640 1 -- 172.21.15.46:0/561595890 learned_addr learned my addr 172.21.15.46:0/561595890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.981+0000 7f97d1e00640 1 --2- 172.21.15.46:0/561595890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed6090 0x56297bed6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.981+0000 7f97d2800640 1 -- 172.21.15.46:0/561595890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 msgr2=0x56297bedb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.981+0000 7f97d2800640 1 --2- 172.21.15.46:0/561595890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 0x56297bedb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.981+0000 7f97d2800640 1 -- 172.21.15.46:0/561595890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed6090 msgr2=0x56297bed6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.981+0000 7f97d2800640 1 --2- 172.21.15.46:0/561595890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed6090 0x56297bed6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.981+0000 7f97d2800640 1 -- 172.21.15.46:0/561595890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297bb4e210 con 0x56297bedba60 2024-09-10T07:48:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.982+0000 7f97d2800640 1 --2- 172.21.15.46:0/561595890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bedba60 0x56297beddf20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f97c40c3e30 tx=0x7f97c402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50513d6bc9eb5e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.982+0000 7f97bf000640 1 -- 172.21.15.46:0/561595890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40bb6f0 con 0x56297bedba60 2024-09-10T07:48:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.982+0000 7f97bf000640 1 -- 172.21.15.46:0/561595890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c40ece50 con 0x56297bedba60 2024-09-10T07:48:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.982+0000 7f97bf000640 1 -- 172.21.15.46:0/561595890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40bb6f0 con 0x56297bedba60 2024-09-10T07:48:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.983+0000 7f97db9d8740 1 -- 172.21.15.46:0/561595890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bedba60 msgr2=0x56297beddf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.984+0000 7f97db9d8740 1 --2- 172.21.15.46:0/561595890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bedba60 0x56297beddf20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f97c40c3e30 tx=0x7f97c402a420 comp rx=0 tx=0).stop 2024-09-10T07:48:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.984+0000 7f97db9d8740 1 -- 172.21.15.46:0/561595890 shutdown_connections 2024-09-10T07:48:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.984+0000 7f97db9d8740 1 --2- 172.21.15.46:0/561595890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bedba60 0x56297beddf20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.984+0000 7f97db9d8740 1 --2- 172.21.15.46:0/561595890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 0x56297bedb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.984+0000 7f97db9d8740 1 --2- 172.21.15.46:0/561595890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed6090 0x56297bed6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.984+0000 7f97db9d8740 1 -- 172.21.15.46:0/561595890 >> 172.21.15.46:0/561595890 conn(0x56297bed1700 msgr2=0x56297bed1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.985+0000 7f97db9d8740 1 -- 172.21.15.46:0/561595890 shutdown_connections 2024-09-10T07:48:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.985+0000 7f97db9d8740 1 -- 172.21.15.46:0/561595890 wait complete. 2024-09-10T07:48:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.985+0000 7f97db9d8740 1 -- start start 2024-09-10T07:48:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.986+0000 7f97db9d8740 1 Processor -- start 2024-09-10T07:48:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.986+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.986+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 0x56297bef1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.986+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bedba60 0x56297bef8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.986+0000 7f97db9d8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56297beef800 con 0x56297bed6090 2024-09-10T07:48:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.986+0000 7f97db9d8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56297bdc4090 con 0x56297bedba60 2024-09-10T07:48:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.986+0000 7f97db9d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56297beef980 con 0x56297bed69d0 2024-09-10T07:48:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.986+0000 7f97db9d8740 10 client.0 _ll_register_callbacks cb 0x56297be18f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.987+0000 7f97d2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bedba60 0x56297bef8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.987+0000 7f97d2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bedba60 0x56297bef8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45060/0 (socket says 172.21.15.46:45060) 2024-09-10T07:48:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.987+0000 7f97d2800640 1 -- 172.21.15.46:0/2351919583 learned_addr learned my addr 172.21.15.46:0/2351919583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.988+0000 7f97d1e00640 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.988+0000 7f97d2800640 1 -- 172.21.15.46:0/2351919583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 msgr2=0x56297bef1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.988+0000 7f97d2800640 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 0x56297bef1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.988+0000 7f97d2800640 1 -- 172.21.15.46:0/2351919583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 msgr2=0x56297bef1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.988+0000 7f97d2800640 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.988+0000 7f97d2800640 1 -- 172.21.15.46:0/2351919583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297bc1a620 con 0x56297bedba60 2024-09-10T07:48:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.989+0000 7f97d2800640 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bedba60 0x56297bef8580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f97c40b7ac0 tx=0x7f97c403beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37f1bc755f515726 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.989+0000 7f97b3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.989+0000 7f97b3400640 1 -- 172.21.15.46:0/2351919583 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40bfaa0 con 0x56297bedba60 2024-09-10T07:48:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.989+0000 7f97b3400640 1 -- 172.21.15.46:0/2351919583 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f97c40b7040 con 0x56297bedba60 2024-09-10T07:48:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.989+0000 7f97db9d8740 1 -- 172.21.15.46:0/2351919583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97c40ece50 con 0x56297bedba60 2024-09-10T07:48:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.992+0000 7f97b3400640 1 -- 172.21.15.46:0/2351919583 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f97c40c4360 con 0x56297bedba60 2024-09-10T07:48:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.992+0000 7f97db9d8740 10 client.28235 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.992+0000 7f97db9d8740 1 -- 172.21.15.46:0/2351919583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56297beef800 con 0x56297bedba60 2024-09-10T07:48:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.992+0000 7f97b3400640 1 -- 172.21.15.46:0/2351919583 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c40ece50 con 0x56297bedba60 2024-09-10T07:48:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.993+0000 7f97b3400640 1 -- 172.21.15.46:0/2351919583 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f97c40bac90 con 0x56297bedba60 2024-09-10T07:48:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.993+0000 7f97db9d8740 10 client.28235 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.993+0000 7f97db9d8740 10 client.28235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.993+0000 7f97db9d8740 20 client.28235 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.993+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56297befc6e0 0x56297befead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.993+0000 7f97db9d8740 4 client.28235 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.993+0000 7f97db9d8740 1 -- 172.21.15.46:0/2351919583 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56297be48830 con 0x56297befc6e0 2024-09-10T07:48:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.994+0000 7f97d1e00640 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56297befc6e0 0x56297befead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.995+0000 7f97d1e00640 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56297befc6e0 0x56297befead0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f1bd00df9bf5c1 server_cookie=7e22d7dcca8a4683 in_seq=0 out_seq=0 2024-09-10T07:48:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.995+0000 7f97b3400640 10 client.28235 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.997+0000 7f97b3400640 1 -- 172.21.15.46:0/2351919583 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56297be48830 con 0x56297befc6e0 2024-09-10T07:48:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.997+0000 7f97b3400640 10 client.28235 handle_command_reply: tid=0 2024-09-10T07:48:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.998+0000 7f97db9d8740 1 client.28235 shutdown 2024-09-10T07:48:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.998+0000 7f97db9d8740 1 -- 172.21.15.46:0/2351919583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bedba60 msgr2=0x56297bef8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:00.998+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bedba60 0x56297bef8580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f97c40b7ac0 tx=0x7f97c403beb0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 1 -- 172.21.15.46:0/2351919583 shutdown_connections 2024-09-10T07:48:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56297befc6e0 0x56297befead0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bedba60 0x56297bef8580 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bed69d0 0x56297bef1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2351919583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97b3400640 0 client.28235 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 1 -- 172.21.15.46:0/2351919583 >> 172.21.15.46:0/2351919583 conn(0x56297bed1700 msgr2=0x56297bed2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 1 -- 172.21.15.46:0/2351919583 shutdown_connections 2024-09-10T07:48:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 1 -- 172.21.15.46:0/2351919583 wait complete. 2024-09-10T07:48:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.001+0000 7f97db9d8740 20 client.28235 trim_cache size 0 max 16384 2024-09-10T07:48:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.036+0000 7f97db9d8740 1 Processor -- start 2024-09-10T07:48:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.036+0000 7f97db9d8740 1 -- start start 2024-09-10T07:48:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.036+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.036+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.036+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bed0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.036+0000 7f97db9d8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97b400fad0 con 0x56297bed6090 2024-09-10T07:48:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.036+0000 7f97db9d8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4011610 con 0x56297bed69d0 2024-09-10T07:48:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.036+0000 7f97db9d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97b40101a0 con 0x56297bedba60 2024-09-10T07:48:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.037+0000 7f97d2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bed0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.037+0000 7f97d2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bed0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60652/0 (socket says 172.21.15.46:60652) 2024-09-10T07:48:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.037+0000 7f97d2800640 1 -- 172.21.15.46:0/1526965427 learned_addr learned my addr 172.21.15.46:0/1526965427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.038+0000 7f97d1400640 1 --2- 172.21.15.46:0/1526965427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.038+0000 7f97d2800640 1 -- 172.21.15.46:0/1526965427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 msgr2=0x56297bef8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.038+0000 7f97d2800640 1 --2- 172.21.15.46:0/1526965427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.038+0000 7f97d2800640 1 -- 172.21.15.46:0/1526965427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 msgr2=0x56297bef03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.038+0000 7f97d2800640 1 --2- 172.21.15.46:0/1526965427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.038+0000 7f97d2800640 1 -- 172.21.15.46:0/1526965427 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297beff130 con 0x56297bedba60 2024-09-10T07:48:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.039+0000 7f97d2800640 1 --2- 172.21.15.46:0/1526965427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bed0060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f97c40b8030 tx=0x7f97c402a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc5f05bc1369ec67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.039+0000 7f97be600640 1 -- 172.21.15.46:0/1526965427 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40bfaa0 con 0x56297bedba60 2024-09-10T07:48:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.039+0000 7f97be600640 1 -- 172.21.15.46:0/1526965427 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c40ece50 con 0x56297bedba60 2024-09-10T07:48:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.039+0000 7f97be600640 1 -- 172.21.15.46:0/1526965427 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4080c80 con 0x56297bedba60 2024-09-10T07:48:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.041+0000 7f97db9d8740 1 -- 172.21.15.46:0/1526965427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 msgr2=0x56297bed0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.041+0000 7f97db9d8740 1 --2- 172.21.15.46:0/1526965427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bed0060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f97c40b8030 tx=0x7f97c402a420 comp rx=0 tx=0).stop 2024-09-10T07:48:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.045+0000 7f97db9d8740 1 -- 172.21.15.46:0/1526965427 shutdown_connections 2024-09-10T07:48:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.045+0000 7f97db9d8740 1 --2- 172.21.15.46:0/1526965427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bed0060 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.045+0000 7f97db9d8740 1 --2- 172.21.15.46:0/1526965427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.045+0000 7f97db9d8740 1 --2- 172.21.15.46:0/1526965427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.045+0000 7f97db9d8740 1 -- 172.21.15.46:0/1526965427 >> 172.21.15.46:0/1526965427 conn(0x56297bed1700 msgr2=0x56297beefce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.045+0000 7f97db9d8740 1 -- 172.21.15.46:0/1526965427 shutdown_connections 2024-09-10T07:48:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.045+0000 7f97db9d8740 1 -- 172.21.15.46:0/1526965427 wait complete. 2024-09-10T07:48:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.046+0000 7f97db9d8740 1 -- start start 2024-09-10T07:48:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.047+0000 7f97db9d8740 1 Processor -- start 2024-09-10T07:48:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.048+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.048+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.048+0000 7f97db9d8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bef6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.048+0000 7f97db9d8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4010f40 con 0x56297bed6090 2024-09-10T07:48:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.048+0000 7f97db9d8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4010870 con 0x56297bed69d0 2024-09-10T07:48:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.048+0000 7f97db9d8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97b400dab0 con 0x56297bedba60 2024-09-10T07:48:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.048+0000 7f97db9d8740 10 client.0 _ll_register_callbacks cb 0x7f97c40bb2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.049+0000 7f97d1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.049+0000 7f97d1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:45072/0 (socket says 172.21.15.46:45072) 2024-09-10T07:48:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.049+0000 7f97d1400640 1 -- 172.21.15.46:0/2442446857 learned_addr learned my addr 172.21.15.46:0/2442446857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.049+0000 7f97d2800640 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bef6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.050+0000 7f97d1400640 1 -- 172.21.15.46:0/2442446857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 msgr2=0x56297bef6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.050+0000 7f97d1400640 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bef6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.050+0000 7f97d1400640 1 -- 172.21.15.46:0/2442446857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 msgr2=0x56297bef5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.050+0000 7f97d1400640 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.050+0000 7f97d1400640 1 -- 172.21.15.46:0/2442446857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297beccd60 con 0x56297bed69d0 2024-09-10T07:48:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.051+0000 7f97d2800640 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bef6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.051+0000 7f97d1400640 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef5d70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f97d4105860 tx=0x7f97c000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b4e4ab8d4e32f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.051+0000 7f97b3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.052+0000 7f97b3400640 1 -- 172.21.15.46:0/2442446857 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c001a020 con 0x56297bed69d0 2024-09-10T07:48:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.052+0000 7f97b3400640 1 -- 172.21.15.46:0/2442446857 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f97c0016690 con 0x56297bed69d0 2024-09-10T07:48:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.052+0000 7f97db9d8740 1 -- 172.21.15.46:0/2442446857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97c40ece50 con 0x56297bed69d0 2024-09-10T07:48:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.058+0000 7f97b3400640 1 -- 172.21.15.46:0/2442446857 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f97c0010ea0 con 0x56297bed69d0 2024-09-10T07:48:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.058+0000 7f97db9d8740 10 client.28241 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.058+0000 7f97db9d8740 1 -- 172.21.15.46:0/2442446857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56297bef85a0 con 0x56297bed69d0 2024-09-10T07:48:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.058+0000 7f97b3400640 1 -- 172.21.15.46:0/2442446857 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c40ece50 con 0x56297bed69d0 2024-09-10T07:48:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.059+0000 7f97b3400640 1 -- 172.21.15.46:0/2442446857 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f97c0014c10 con 0x56297bed69d0 2024-09-10T07:48:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.059+0000 7f97db9d8740 10 client.28241 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.059+0000 7f97db9d8740 10 client.28241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.059+0000 7f97db9d8740 20 client.28241 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.059+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56297bf67860 0x56297bf69c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.060+0000 7f97db9d8740 4 client.28241 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:48:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.060+0000 7f97db9d8740 1 -- 172.21.15.46:0/2442446857 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56297be42b80 con 0x56297bf67860 2024-09-10T07:48:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.060+0000 7f97d1e00640 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56297bf67860 0x56297bf69c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.062+0000 7f97d1e00640 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56297bf67860 0x56297bf69c80 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6338dd3847a54835 server_cookie=94a6c87c50d00d83 in_seq=0 out_seq=0 2024-09-10T07:48:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.062+0000 7f97b3400640 10 client.28241 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97b3400640 1 -- 172.21.15.46:0/2442446857 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56297be42b80 con 0x56297bf67860 2024-09-10T07:48:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97b3400640 10 client.28241 handle_command_reply: tid=0 2024-09-10T07:48:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 client.28241 shutdown 2024-09-10T07:48:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 -- 172.21.15.46:0/2442446857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 msgr2=0x56297bef5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef5d70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f97d4105860 tx=0x7f97c000ef90 comp rx=0 tx=0).stop 2024-09-10T07:48:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 -- 172.21.15.46:0/2442446857 shutdown_connections 2024-09-10T07:48:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56297bf67860 0x56297bf69c80 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56297bedba60 0x56297bef6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56297bed69d0 0x56297bef5d70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 --2- 172.21.15.46:0/2442446857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56297bed6090 0x56297bef5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97b3400640 0 client.28241 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.079+0000 7f97db9d8740 1 -- 172.21.15.46:0/2442446857 >> 172.21.15.46:0/2442446857 conn(0x56297bed1700 msgr2=0x56297bed25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.081+0000 7f97db9d8740 1 -- 172.21.15.46:0/2442446857 shutdown_connections 2024-09-10T07:48:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.081+0000 7f97db9d8740 1 -- 172.21.15.46:0/2442446857 wait complete. 2024-09-10T07:48:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.082+0000 7f97db9d8740 20 client.28241 trim_cache size 0 max 16384 2024-09-10T07:48:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:48:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:48:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:48:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:48:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 -- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f97b80a2c80 msgr2=0x7f97b80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f97b80a2c80 0x7f97b80a5140 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f97cc005c40 tx=0x7f97cc005bd0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 -- 172.21.15.46:0/3557601970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d4103710 msgr2=0x7f97d4104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d4103710 0x7f97d4104450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f97c40314e0 tx=0x7f97c4032c60 comp rx=0 tx=0).stop 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 -- 172.21.15.46:0/3557601970 shutdown_connections 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d4103710 0x7f97d4104450 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97d4102d50 0x7f97d4107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f97b80a2c80 0x7f97b80a5140 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 --2- 172.21.15.46:0/3557601970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97d4101b60 0x7f97d41078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 -- 172.21.15.46:0/3557601970 >> 172.21.15.46:0/3557601970 conn(0x7f97d40fd2e0 msgr2=0x7f97d4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 -- 172.21.15.46:0/3557601970 shutdown_connections 2024-09-10T07:48:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:01.083+0000 7f97d8a00640 1 -- 172.21.15.46:0/3557601970 wait complete. 2024-09-10T07:48:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:02 smithi181 ceph-mon[35397]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-10T07:48:02.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:02 smithi063 ceph-mon[35020]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-10T07:48:02.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:02 smithi046 ceph-mon[30705]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-10T07:48:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:04 smithi181 ceph-mon[35397]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-09-10T07:48:04.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:04 smithi063 ceph-mon[35020]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-09-10T07:48:04.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:04 smithi046 ceph-mon[30705]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-09-10T07:48:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 645.0 s, 539.7 tps, lat 186.368 ms stddev 213.955 2024-09-10T07:48:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:06 smithi181 ceph-mon[35397]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-09-10T07:48:06.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:06 smithi063 ceph-mon[35020]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-09-10T07:48:06.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:06 smithi046 ceph-mon[30705]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-09-10T07:48:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:08 smithi181 ceph-mon[35397]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.1 MiB/s wr, 450 op/s 2024-09-10T07:48:08.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:08 smithi063 ceph-mon[35020]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.1 MiB/s wr, 450 op/s 2024-09-10T07:48:08.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:08 smithi046 ceph-mon[30705]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.1 MiB/s wr, 450 op/s 2024-09-10T07:48:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:10 smithi181 ceph-mon[35397]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 454 op/s 2024-09-10T07:48:10.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:10 smithi063 ceph-mon[35020]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 454 op/s 2024-09-10T07:48:10.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:10 smithi046 ceph-mon[30705]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 454 op/s 2024-09-10T07:48:10.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 650.0 s, 583.5 tps, lat 168.785 ms stddev 196.254 2024-09-10T07:48:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:12 smithi181 ceph-mon[35397]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.4 MiB/s wr, 567 op/s 2024-09-10T07:48:12.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:12 smithi063 ceph-mon[35020]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.4 MiB/s wr, 567 op/s 2024-09-10T07:48:12.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:12 smithi046 ceph-mon[30705]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.4 MiB/s wr, 567 op/s 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 -- 172.21.15.46:0/35129239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3bc109d40 msgr2=0x7fd3bc1105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/35129239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3bc109d40 0x7fd3bc1105d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009ca0 tx=0x7fd3b402a8e0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 -- 172.21.15.46:0/35129239 shutdown_connections 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/35129239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3bc109d40 0x7fd3bc1105d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/35129239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3bc109380 0x7fd3bc109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/35129239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3bc108470 0x7fd3bc108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 -- 172.21.15.46:0/35129239 >> 172.21.15.46:0/35129239 conn(0x7fd3bc077490 msgr2=0x7fd3bc0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 -- 172.21.15.46:0/35129239 shutdown_connections 2024-09-10T07:48:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.305+0000 7fd3c0a00640 1 -- 172.21.15.46:0/35129239 wait complete. 2024-09-10T07:48:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.307+0000 7fd3c0a00640 1 Processor -- start 2024-09-10T07:48:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.307+0000 7fd3c0a00640 1 -- start start 2024-09-10T07:48:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.307+0000 7fd3c0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3bc108470 0x7fd3bc19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.307+0000 7fd3c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3bc109380 0x7fd3bc19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.307+0000 7fd3c0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3bc109d40 0x7fd3bc1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.307+0000 7fd3c0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc113ec0 con 0x7fd3bc109380 2024-09-10T07:48:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.307+0000 7fd3c0a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc113d40 con 0x7fd3bc108470 2024-09-10T07:48:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.307+0000 7fd3c0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc114040 con 0x7fd3bc109d40 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3bc108470 0x7fd3bc19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3bc108470 0x7fd3bc19e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53578/0 (socket says 172.21.15.46:53578) 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 -- 172.21.15.46:0/279585493 learned_addr learned my addr 172.21.15.46:0/279585493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9400640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3bc109380 0x7fd3bc19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 -- 172.21.15.46:0/279585493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3bc109d40 msgr2=0x7fd3bc1a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3bc109d40 0x7fd3bc1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 -- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3bc109380 msgr2=0x7fd3bc19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3bc109380 0x7fd3bc19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 -- 172.21.15.46:0/279585493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc1a60a0 con 0x7fd3bc108470 2024-09-10T07:48:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9400640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3bc109380 0x7fd3bc19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.308+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3bc108470 0x7fd3bc19e480 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400b450 tx=0x7fd3a400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.309+0000 7fd3b2a00640 1 -- 172.21.15.46:0/279585493 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4015020 con 0x7fd3bc108470 2024-09-10T07:48:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.309+0000 7fd3b2a00640 1 -- 172.21.15.46:0/279585493 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a4011d70 con 0x7fd3bc108470 2024-09-10T07:48:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.309+0000 7fd3b2a00640 1 -- 172.21.15.46:0/279585493 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400f340 con 0x7fd3bc108470 2024-09-10T07:48:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.309+0000 7fd3c0a00640 1 -- 172.21.15.46:0/279585493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc1a6390 con 0x7fd3bc108470 2024-09-10T07:48:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.310+0000 7fd3c0a00640 1 -- 172.21.15.46:0/279585493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc1a6890 con 0x7fd3bc108470 2024-09-10T07:48:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.314+0000 7fd3b2a00640 1 -- 172.21.15.46:0/279585493 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd3a4018020 con 0x7fd3bc108470 2024-09-10T07:48:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.316+0000 7fd3b2a00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd3a00a2da0 0x7fd3a00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.316+0000 7fd3b2a00640 1 -- 172.21.15.46:0/279585493 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd3a40ceea0 con 0x7fd3bc108470 2024-09-10T07:48:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3c3846740 1 Processor -- start 2024-09-10T07:48:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3b9400640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd3a00a2da0 0x7fd3a00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3c3846740 1 -- start start 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 0x564fdbdb2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdb2a60 0x564fdbdb4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3c3846740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564fdbd12100 con 0x564fdbdad9d0 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3c3846740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564fdbcba1b0 con 0x564fdbdb2a60 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.317+0000 7fd3c3846740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564fdbd0e0c0 con 0x564fdbdad090 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3ba800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdb2a60 0x564fdbdb4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:52798/0 (socket says 172.21.15.46:52798) 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9e00640 1 -- 172.21.15.46:0/2630735582 learned_addr learned my addr 172.21.15.46:0/2630735582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9400640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd3a00a2da0 0x7fd3a00a5260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac005d00 tx=0x7fd3ac005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9400640 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 0x564fdbdb2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9e00640 1 -- 172.21.15.46:0/2630735582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdb2a60 msgr2=0x564fdbdb4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdb2a60 0x564fdbdb4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9e00640 1 -- 172.21.15.46:0/2630735582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 msgr2=0x564fdbdb2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 0x564fdbdb2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9e00640 1 -- 172.21.15.46:0/2630735582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fdba25210 con 0x564fdbdad090 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.318+0000 7fd3b9400640 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 0x564fdbdb2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.319+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdad490 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4097cf0 tx=0x7fd3a409af20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f10a9a281a84a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.319+0000 7fd3ab000640 1 -- 172.21.15.46:0/2630735582 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40b4070 con 0x564fdbdad090 2024-09-10T07:48:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.319+0000 7fd3ab000640 1 -- 172.21.15.46:0/2630735582 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a409b320 con 0x564fdbdad090 2024-09-10T07:48:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.319+0000 7fd3ab000640 1 -- 172.21.15.46:0/2630735582 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4097990 con 0x564fdbdad090 2024-09-10T07:48:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.320+0000 7fd3c3846740 1 -- 172.21.15.46:0/2630735582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 msgr2=0x564fdbdad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.320+0000 7fd3c3846740 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdad490 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4097cf0 tx=0x7fd3a409af20 comp rx=0 tx=0).stop 2024-09-10T07:48:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.320+0000 7fd3c3846740 1 -- 172.21.15.46:0/2630735582 shutdown_connections 2024-09-10T07:48:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.320+0000 7fd3c3846740 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdb2a60 0x564fdbdb4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.320+0000 7fd3c3846740 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 0x564fdbdb2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.320+0000 7fd3c3846740 1 --2- 172.21.15.46:0/2630735582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdad490 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.320+0000 7fd3c3846740 1 -- 172.21.15.46:0/2630735582 >> 172.21.15.46:0/2630735582 conn(0x564fdbda8700 msgr2=0x564fdbda8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.321+0000 7fd3c3846740 1 -- 172.21.15.46:0/2630735582 shutdown_connections 2024-09-10T07:48:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.321+0000 7fd3c3846740 1 -- 172.21.15.46:0/2630735582 wait complete. 2024-09-10T07:48:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.322+0000 7fd3c3846740 1 -- start start 2024-09-10T07:48:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.322+0000 7fd3c3846740 1 Processor -- start 2024-09-10T07:48:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.323+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdb7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.323+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 0x564fdbdb77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.323+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdb2a60 0x564fdbdbe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.323+0000 7fd3c3846740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564fdbdb5a30 con 0x564fdbdad9d0 2024-09-10T07:48:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.323+0000 7fd3c3846740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564fdbc9b090 con 0x564fdbdad090 2024-09-10T07:48:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.323+0000 7fd3c3846740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564fdbdb5bb0 con 0x564fdbdb2a60 2024-09-10T07:48:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdb7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdb7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53590/0 (socket says 172.21.15.46:53590) 2024-09-10T07:48:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3b9e00640 1 -- 172.21.15.46:0/3678571684 learned_addr learned my addr 172.21.15.46:0/3678571684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3ba800640 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdb2a60 0x564fdbdbe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.323+0000 7fd3c3846740 10 client.0 _ll_register_callbacks cb 0x564fdbceff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3b9e00640 1 -- 172.21.15.46:0/3678571684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdb2a60 msgr2=0x564fdbdbe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdb2a60 0x564fdbdbe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3b9e00640 1 -- 172.21.15.46:0/3678571684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 msgr2=0x564fdbdb77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 0x564fdbdb77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.324+0000 7fd3b9e00640 1 -- 172.21.15.46:0/3678571684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fdbaf1620 con 0x564fdbdad090 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.325+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdb7270 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4097780 tx=0x7fd3a409f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4268d2b7d6221a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.325+0000 7fd39f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.325+0000 7fd39f400640 1 -- 172.21.15.46:0/3678571684 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40b4040 con 0x564fdbdad090 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.325+0000 7fd3c3846740 1 -- 172.21.15.46:0/3678571684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a409b320 con 0x564fdbdad090 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.325+0000 7fd39f400640 1 -- 172.21.15.46:0/3678571684 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd3a400eb50 con 0x564fdbdad090 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.325+0000 7fd39f400640 1 -- 172.21.15.46:0/3678571684 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd3a40b4040 con 0x564fdbdad090 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.325+0000 7fd3c3846740 10 client.28250 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.325+0000 7fd3c3846740 1 -- 172.21.15.46:0/3678571684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564fdbdc15a0 con 0x564fdbdad090 2024-09-10T07:48:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd39f400640 1 -- 172.21.15.46:0/3678571684 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a409b320 con 0x564fdbdad090 2024-09-10T07:48:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd39f400640 1 -- 172.21.15.46:0/3678571684 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fd3a409a770 con 0x564fdbdad090 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd3c3846740 10 client.28250 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd3c3846740 10 client.28250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd3c3846740 20 client.28250 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd3c3846740 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564fdbdc28f0 0x564fdbdc4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd3c3846740 4 client.28250 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd3c3846740 1 -- 172.21.15.46:0/3678571684 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fdbd1f830 con 0x564fdbdc28f0 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.326+0000 7fd3b9400640 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564fdbdc28f0 0x564fdbdc4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.327+0000 7fd3b9400640 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564fdbdc28f0 0x564fdbdc4ce0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee1048b0a5eb1ad9 server_cookie=d3e85a0927400b74 in_seq=0 out_seq=0 2024-09-10T07:48:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.327+0000 7fd39f400640 10 client.28250 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd39f400640 1 -- 172.21.15.46:0/3678571684 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fdbd1f830 con 0x564fdbdc28f0 2024-09-10T07:48:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd39f400640 10 client.28250 handle_command_reply: tid=0 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 client.28250 shutdown 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 -- 172.21.15.46:0/3678571684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 msgr2=0x564fdbdb7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdb7270 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4097780 tx=0x7fd3a409f7c0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 -- 172.21.15.46:0/3678571684 shutdown_connections 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564fdbdc28f0 0x564fdbdc4ce0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdb2a60 0x564fdbdbe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdad9d0 0x564fdbdb77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 --2- 172.21.15.46:0/3678571684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdb7270 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd3c3846740 1 -- 172.21.15.46:0/3678571684 >> 172.21.15.46:0/3678571684 conn(0x564fdbda8700 msgr2=0x564fdbda9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.328+0000 7fd39f400640 0 client.28250 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.329+0000 7fd3c3846740 1 -- 172.21.15.46:0/3678571684 shutdown_connections 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.329+0000 7fd3c3846740 1 -- 172.21.15.46:0/3678571684 wait complete. 2024-09-10T07:48:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.329+0000 7fd3c3846740 20 client.28250 trim_cache size 0 max 16384 2024-09-10T07:48:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.351+0000 7fd3c3846740 1 Processor -- start 2024-09-10T07:48:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.351+0000 7fd3c3846740 1 -- start start 2024-09-10T07:48:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdb7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad9d0 0x564fdbdbe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbda7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3c3846740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1b0e10 con 0x564fdbdb2a60 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3c3846740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1b2950 con 0x564fdbdad9d0 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3c3846740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1b14e0 con 0x564fdbdad090 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3b9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad9d0 0x564fdbdbe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdb7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdb7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:52818/0 (socket says 172.21.15.46:52818) 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3b9e00640 1 -- 172.21.15.46:0/2565571693 learned_addr learned my addr 172.21.15.46:0/2565571693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3b9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad9d0 0x564fdbdbe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53604/0 (socket says 172.21.15.46:53604) 2024-09-10T07:48:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.352+0000 7fd3ba800640 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbda7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.353+0000 7fd3b9400640 1 -- 172.21.15.46:0/2565571693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 msgr2=0x564fdbdb7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.353+0000 7fd3b9400640 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdb7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.354+0000 7fd3b9400640 1 -- 172.21.15.46:0/2565571693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 msgr2=0x564fdbda7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.354+0000 7fd3b9400640 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbda7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.354+0000 7fd3b9400640 1 -- 172.21.15.46:0/2565571693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fdbdb5910 con 0x564fdbdad9d0 2024-09-10T07:48:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.354+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdb7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.355+0000 7fd3ba800640 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbda7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.355+0000 7fd3b9400640 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad9d0 0x564fdbdbe710 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x564fdbdc54e0 tx=0x7fd3ac012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0819c215866784 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.356+0000 7fd3aa600640 1 -- 172.21.15.46:0/2565571693 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0272d0 con 0x564fdbdad9d0 2024-09-10T07:48:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.356+0000 7fd3aa600640 1 -- 172.21.15.46:0/2565571693 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac02c070 con 0x564fdbdad9d0 2024-09-10T07:48:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.356+0000 7fd3aa600640 1 -- 172.21.15.46:0/2565571693 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac01fa60 con 0x564fdbdad9d0 2024-09-10T07:48:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.357+0000 7fd3c3846740 1 -- 172.21.15.46:0/2565571693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad9d0 msgr2=0x564fdbdbe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.357+0000 7fd3c3846740 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad9d0 0x564fdbdbe710 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x564fdbdc54e0 tx=0x7fd3ac012000 comp rx=0 tx=0).stop 2024-09-10T07:48:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.358+0000 7fd3c3846740 1 -- 172.21.15.46:0/2565571693 shutdown_connections 2024-09-10T07:48:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.358+0000 7fd3c3846740 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbda7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.358+0000 7fd3c3846740 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad9d0 0x564fdbdbe710 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.358+0000 7fd3c3846740 1 --2- 172.21.15.46:0/2565571693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad090 0x564fdbdb7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.358+0000 7fd3c3846740 1 -- 172.21.15.46:0/2565571693 >> 172.21.15.46:0/2565571693 conn(0x564fdbda8700 msgr2=0x564fdbda7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.358+0000 7fd3c3846740 1 -- 172.21.15.46:0/2565571693 shutdown_connections 2024-09-10T07:48:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.358+0000 7fd3c3846740 1 -- 172.21.15.46:0/2565571693 wait complete. 2024-09-10T07:48:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.358+0000 7fd3c3846740 1 -- start start 2024-09-10T07:48:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3c3846740 1 Processor -- start 2024-09-10T07:48:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdbb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad9d0 0x564fdbdbb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3c3846740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbe2b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3c3846740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1b2280 con 0x564fdbdb2a60 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3c3846740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1b1bb0 con 0x564fdbdad090 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3c3846740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1aed90 con 0x564fdbdad9d0 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3c3846740 10 client.0 _ll_register_callbacks cb 0x7fd3a40b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3b9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad9d0 0x564fdbdbb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3b9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad9d0 0x564fdbdbb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:52826/0 (socket says 172.21.15.46:52826) 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3b9400640 1 -- 172.21.15.46:0/793800242 learned_addr learned my addr 172.21.15.46:0/793800242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdbb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.359+0000 7fd3ba800640 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbe2b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd3b9400640 1 -- 172.21.15.46:0/793800242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 msgr2=0x564fdbdbb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd3b9400640 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdbb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd3b9400640 1 -- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 msgr2=0x564fdbe2b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd3b9400640 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbe2b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd3b9400640 1 -- 172.21.15.46:0/793800242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fdbdbbfa0 con 0x564fdbdad9d0 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd3ba800640 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbe2b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd3b9400640 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad9d0 0x564fdbdbb930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac014040 tx=0x7fd3ac01f4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a596db9e10000cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd39f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd39f400640 1 -- 172.21.15.46:0/793800242 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac036070 con 0x564fdbdad9d0 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd3c3846740 1 -- 172.21.15.46:0/793800242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ac02c070 con 0x564fdbdad9d0 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.360+0000 7fd39f400640 1 -- 172.21.15.46:0/793800242 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd3ac01fbb0 con 0x564fdbdad9d0 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd39f400640 1 -- 172.21.15.46:0/793800242 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd3ac042050 con 0x564fdbdad9d0 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd3c3846740 10 client.28198 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd3c3846740 1 -- 172.21.15.46:0/793800242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564fdbe2cb80 con 0x564fdbdad9d0 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd39f400640 1 -- 172.21.15.46:0/793800242 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac02c070 con 0x564fdbdad9d0 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd39f400640 1 -- 172.21.15.46:0/793800242 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fd3ac0103d0 con 0x564fdbdad9d0 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd3c3846740 10 client.28198 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd3c3846740 10 client.28198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd3c3846740 20 client.28198 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd3c3846740 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564fdbe2d4b0 0x564fdbe2f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd3c3846740 4 client.28198 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.361+0000 7fd3c3846740 1 -- 172.21.15.46:0/793800242 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fdbd19b80 con 0x564fdbe2d4b0 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.362+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564fdbe2d4b0 0x564fdbe2f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.362+0000 7fd3b9e00640 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564fdbe2d4b0 0x564fdbe2f8d0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64504e2397346fe7 server_cookie=bc9615bc50848c9f in_seq=0 out_seq=0 2024-09-10T07:48:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.362+0000 7fd39f400640 10 client.28198 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd39f400640 1 -- 172.21.15.46:0/793800242 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fdbd19b80 con 0x564fdbe2d4b0 2024-09-10T07:48:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd39f400640 10 client.28198 handle_command_reply: tid=0 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 client.28198 shutdown 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 -- 172.21.15.46:0/793800242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad9d0 msgr2=0x564fdbdbb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad9d0 0x564fdbdbb930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac014040 tx=0x7fd3ac01f4a0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 -- 172.21.15.46:0/793800242 shutdown_connections 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564fdbe2d4b0 0x564fdbe2f8d0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fdbdb2a60 0x564fdbe2b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fdbdad9d0 0x564fdbdbb930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 --2- 172.21.15.46:0/793800242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fdbdad090 0x564fdbdbb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 -- 172.21.15.46:0/793800242 >> 172.21.15.46:0/793800242 conn(0x564fdbda8700 msgr2=0x564fdbda7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd39f400640 0 client.28198 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 -- 172.21.15.46:0/793800242 shutdown_connections 2024-09-10T07:48:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.372+0000 7fd3c3846740 1 -- 172.21.15.46:0/793800242 wait complete. 2024-09-10T07:48:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.373+0000 7fd3c3846740 20 client.28198 trim_cache size 0 max 16384 2024-09-10T07:48:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:48:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:48:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:48:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:48:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.373+0000 7fd3c0a00640 1 -- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd3a00a2da0 msgr2=0x7fd3a00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.373+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd3a00a2da0 0x7fd3a00a5260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac005d00 tx=0x7fd3ac005c90 comp rx=0 tx=0).stop 2024-09-10T07:48:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.373+0000 7fd3c0a00640 1 -- 172.21.15.46:0/279585493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3bc108470 msgr2=0x7fd3bc19e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.373+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3bc108470 0x7fd3bc19e480 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400b450 tx=0x7fd3a400b910 comp rx=0 tx=0).stop 2024-09-10T07:48:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.374+0000 7fd3c0a00640 1 -- 172.21.15.46:0/279585493 shutdown_connections 2024-09-10T07:48:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.374+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3bc109d40 0x7fd3bc1a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.374+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd3a00a2da0 0x7fd3a00a5260 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.374+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3bc109380 0x7fd3bc19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.374+0000 7fd3c0a00640 1 --2- 172.21.15.46:0/279585493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3bc108470 0x7fd3bc19e480 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.374+0000 7fd3c0a00640 1 -- 172.21.15.46:0/279585493 >> 172.21.15.46:0/279585493 conn(0x7fd3bc077490 msgr2=0x7fd3bc10c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.374+0000 7fd3c0a00640 1 -- 172.21.15.46:0/279585493 shutdown_connections 2024-09-10T07:48:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:14.374+0000 7fd3c0a00640 1 -- 172.21.15.46:0/279585493 wait complete. 2024-09-10T07:48:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:14 smithi181 ceph-mon[35397]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 588 op/s 2024-09-10T07:48:14.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:14 smithi063 ceph-mon[35020]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 588 op/s 2024-09-10T07:48:14.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:14 smithi046 ceph-mon[30705]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 588 op/s 2024-09-10T07:48:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 655.0 s, 610.6 tps, lat 162.373 ms stddev 202.176 2024-09-10T07:48:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:16 smithi181 ceph-mon[35397]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.5 MiB/s wr, 499 op/s 2024-09-10T07:48:16.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:16 smithi063 ceph-mon[35020]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.5 MiB/s wr, 499 op/s 2024-09-10T07:48:16.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:16 smithi046 ceph-mon[30705]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.5 MiB/s wr, 499 op/s 2024-09-10T07:48:17.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:48:18.307 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:18.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:18 smithi063 ceph-mon[35020]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.1 MiB/s wr, 615 op/s 2024-09-10T07:48:18.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:18 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:48:18.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:18 smithi046 ceph-mon[30705]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.1 MiB/s wr, 615 op/s 2024-09-10T07:48:18.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:18 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:48:18.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:18 smithi181 ceph-mon[35397]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.1 MiB/s wr, 615 op/s 2024-09-10T07:48:18.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:18 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:48:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 -- 172.21.15.46:0/1039977142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01082a0 msgr2=0x7f10d010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 --2- 172.21.15.46:0/1039977142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01082a0 0x7f10d010ec40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009cd0 tx=0x7f10c802a910 comp rx=0 tx=0).stop 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 -- 172.21.15.46:0/1039977142 shutdown_connections 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 --2- 172.21.15.46:0/1039977142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01082a0 0x7f10d010ec40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 --2- 172.21.15.46:0/1039977142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d01036e0 0x7f10d0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 --2- 172.21.15.46:0/1039977142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d0102d30 0x7f10d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 -- 172.21.15.46:0/1039977142 >> 172.21.15.46:0/1039977142 conn(0x7f10d00fc7d0 msgr2=0x7f10d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 -- 172.21.15.46:0/1039977142 shutdown_connections 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.494+0000 7f10d7a00640 1 -- 172.21.15.46:0/1039977142 wait complete. 2024-09-10T07:48:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.497+0000 7f10d7a00640 1 Processor -- start 2024-09-10T07:48:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.498+0000 7f10d7a00640 1 -- start start 2024-09-10T07:48:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.498+0000 7f10d7a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d0102d30 0x7f10d0073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.498+0000 7f10d7a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01036e0 0x7f10d0073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10d7a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d01082a0 0x7f10d006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10d7a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10d010ff00 con 0x7f10d0102d30 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01036e0 0x7f10d0073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01036e0 0x7f10d0073ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:32918/0 (socket says 172.21.15.46:32918) 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10cfe00640 1 -- 172.21.15.46:0/4080920313 learned_addr learned my addr 172.21.15.46:0/4080920313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10d5a00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d01082a0 0x7f10d006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f10d010fd80 con 0x7f10d01036e0 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0110080 con 0x7f10d01082a0 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.499+0000 7f10d5000640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d0102d30 0x7f10d0073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.500+0000 7f10cfe00640 1 -- 172.21.15.46:0/4080920313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d01082a0 msgr2=0x7f10d006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.500+0000 7f10cfe00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d01082a0 0x7f10d006cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.500+0000 7f10cfe00640 1 -- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d0102d30 msgr2=0x7f10d0073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.500+0000 7f10cfe00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d0102d30 0x7f10d0073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.500+0000 7f10cfe00640 1 -- 172.21.15.46:0/4080920313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d006d490 con 0x7f10d01036e0 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.501+0000 7f10cfe00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01036e0 0x7f10d0073ec0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00d740 tx=0x7f10bc00dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.502+0000 7f10cd600640 1 -- 172.21.15.46:0/4080920313 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc017070 con 0x7f10d01036e0 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.502+0000 7f10cd600640 1 -- 172.21.15.46:0/4080920313 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10bc00de50 con 0x7f10d01036e0 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.502+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d006d780 con 0x7f10d01036e0 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.502+0000 7f10cd600640 1 -- 172.21.15.46:0/4080920313 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00f6b0 con 0x7f10d01036e0 2024-09-10T07:48:19.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.502+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d0071790 con 0x7f10d01036e0 2024-09-10T07:48:19.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.509+0000 7f10da608740 1 Processor -- start 2024-09-10T07:48:19.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.511+0000 7f10da608740 1 -- start start 2024-09-10T07:48:19.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.511+0000 7f10da608740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dd7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.511+0000 7f10cd600640 1 -- 172.21.15.46:0/4080920313 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f10bc01a070 con 0x7f10d01036e0 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.517+0000 7f10da608740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.517+0000 7f10da608740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.517+0000 7f10da608740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d989d3e3e0 con 0x55d989de2d70 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.517+0000 7f10da608740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d989ce5d80 con 0x55d989dd7b90 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.517+0000 7f10da608740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d989d3a3a0 con 0x55d989dd84b0 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.517+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dd7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.517+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dd7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:32924/0 (socket says 172.21.15.46:32924) 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.517+0000 7f10d5a00640 1 -- 172.21.15.46:0/1293555864 learned_addr learned my addr 172.21.15.46:0/1293555864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.513+0000 7f10cd600640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10b40a2d40 0x7f10b40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.518+0000 7f10cfe00640 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.519+0000 7f10cfe00640 1 -- 172.21.15.46:0/1293555864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 msgr2=0x55d989dd7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.519+0000 7f10cfe00640 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dd7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.519+0000 7f10d5000640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10b40a2d40 0x7f10b40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.519+0000 7f10cfe00640 1 -- 172.21.15.46:0/1293555864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 msgr2=0x55d989de5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.519+0000 7f10cfe00640 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.519+0000 7f10cfe00640 1 -- 172.21.15.46:0/1293555864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d989d04ae0 con 0x55d989dd84b0 2024-09-10T07:48:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.519+0000 7f10d5a00640 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dd7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.519+0000 7f10cd600640 1 -- 172.21.15.46:0/4080920313 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f10bc0cecd0 con 0x7f10d01036e0 2024-09-10T07:48:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.520+0000 7f10cfe00640 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de2830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0cc760 tx=0x7f10bc023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd3d03ed8ce3446 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.520+0000 7f10c6000640 1 -- 172.21.15.46:0/1293555864 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc009c30 con 0x55d989dd84b0 2024-09-10T07:48:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.520+0000 7f10c6000640 1 -- 172.21.15.46:0/1293555864 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10bc09b850 con 0x55d989dd84b0 2024-09-10T07:48:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.520+0000 7f10c6000640 1 -- 172.21.15.46:0/1293555864 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc0a3dc0 con 0x55d989dd84b0 2024-09-10T07:48:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.521+0000 7f10da608740 1 -- 172.21.15.46:0/1293555864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 msgr2=0x55d989de2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.521+0000 7f10da608740 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de2830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0cc760 tx=0x7f10bc023000 comp rx=0 tx=0).stop 2024-09-10T07:48:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.521+0000 7f10da608740 1 -- 172.21.15.46:0/1293555864 shutdown_connections 2024-09-10T07:48:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.521+0000 7f10da608740 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.521+0000 7f10da608740 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de2830 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.521+0000 7f10da608740 1 --2- 172.21.15.46:0/1293555864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dd7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.521+0000 7f10da608740 1 -- 172.21.15.46:0/1293555864 >> 172.21.15.46:0/1293555864 conn(0x55d989dd4120 msgr2=0x55d989dd4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.522+0000 7f10da608740 1 -- 172.21.15.46:0/1293555864 shutdown_connections 2024-09-10T07:48:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.522+0000 7f10da608740 1 -- 172.21.15.46:0/1293555864 wait complete. 2024-09-10T07:48:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.523+0000 7f10da608740 1 -- start start 2024-09-10T07:48:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.523+0000 7f10da608740 1 Processor -- start 2024-09-10T07:48:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.523+0000 7f10d5000640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10b40a2d40 0x7f10b40a5200 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f10c0005fd0 tx=0x7f10c0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.525+0000 7f10da608740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989ddfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.526+0000 7f10da608740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.526+0000 7f10d5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989ddfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.526+0000 7f10d5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989ddfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:32940/0 (socket says 172.21.15.46:32940) 2024-09-10T07:48:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.527+0000 7f10d5000640 1 -- 172.21.15.46:0/1753463509 learned_addr learned my addr 172.21.15.46:0/1753463509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.527+0000 7f10cfe00640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.527+0000 7f10da608740 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.527+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d989de59e0 con 0x55d989de2d70 2024-09-10T07:48:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.527+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d989cf90f0 con 0x55d989dd7b90 2024-09-10T07:48:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.527+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d989de5b60 con 0x55d989dd84b0 2024-09-10T07:48:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.528+0000 7f10d5a00640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.528+0000 7f10da608740 10 client.0 _ll_register_callbacks cb 0x55d989cea9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.529+0000 7f10d5000640 1 -- 172.21.15.46:0/1753463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 msgr2=0x55d989de0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.529+0000 7f10d5000640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.529+0000 7f10d5000640 1 -- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 msgr2=0x55d989de0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.529+0000 7f10d5000640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.529+0000 7f10d5000640 1 -- 172.21.15.46:0/1753463509 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d989aaf3c0 con 0x55d989dd7b90 2024-09-10T07:48:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.529+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10bc09b850 con 0x55d989dd7b90 2024-09-10T07:48:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.530+0000 7f10baa00640 10 client.28268 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.530+0000 7f10d5000640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989ddfd00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f10c00117f0 tx=0x7f10c0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50b89555c9f8c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.531+0000 7f10cfe00640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.531+0000 7f10d5a00640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.531+0000 7f10baa00640 1 -- 172.21.15.46:0/1753463509 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c002c050 con 0x55d989dd7b90 2024-09-10T07:48:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.531+0000 7f10baa00640 1 -- 172.21.15.46:0/1753463509 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f10c0010b50 con 0x55d989dd7b90 2024-09-10T07:48:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.532+0000 7f10baa00640 1 -- 172.21.15.46:0/1753463509 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f10c0022070 con 0x55d989dd7b90 2024-09-10T07:48:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.532+0000 7f10da608740 10 client.28268 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.532+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=50}) -- 0x55d989df1760 con 0x55d989dd7b90 2024-09-10T07:48:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.532+0000 7f10baa00640 1 -- 172.21.15.46:0/1753463509 <== mon.2 v2:172.21.15.63:3300/0 4 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f10c000f040 con 0x55d989dd7b90 2024-09-10T07:48:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.533+0000 7f10da608740 10 client.28268 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.533+0000 7f10da608740 10 client.28268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.533+0000 7f10da608740 20 client.28268 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.533+0000 7f10da608740 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d989df2b10 0x55d989df4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.533+0000 7f10da608740 4 client.28268 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.534+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d989d4b950 con 0x55d989df2b10 2024-09-10T07:48:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.534+0000 7f10cfe00640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d989df2b10 0x55d989df4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.538+0000 7f10cfe00640 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d989df2b10 0x55d989df4f00 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d363c2e0a7929fb7 server_cookie=bcf073b4a8b9e051 in_seq=0 out_seq=0 2024-09-10T07:48:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.538+0000 7f10baa00640 10 client.28268 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.541+0000 7f10baa00640 1 -- 172.21.15.46:0/1753463509 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d989d4b950 con 0x55d989df2b10 2024-09-10T07:48:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.541+0000 7f10baa00640 10 client.28268 handle_command_reply: tid=0 2024-09-10T07:48:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.541+0000 7f10da608740 1 client.28268 shutdown 2024-09-10T07:48:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.542+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 msgr2=0x55d989ddfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.542+0000 7f10da608740 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989ddfd00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f10c00117f0 tx=0x7f10c0004730 comp rx=0 tx=0).stop 2024-09-10T07:48:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.542+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 shutdown_connections 2024-09-10T07:48:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.542+0000 7f10da608740 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d989df2b10 0x55d989df4f00 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.542+0000 7f10da608740 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.542+0000 7f10da608740 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.542+0000 7f10da608740 1 --2- 172.21.15.46:0/1753463509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989ddfd00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.542+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 >> 172.21.15.46:0/1753463509 conn(0x55d989dd4120 msgr2=0x55d989dd9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.543+0000 7f10baa00640 0 client.28268 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.544+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 shutdown_connections 2024-09-10T07:48:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.544+0000 7f10da608740 1 -- 172.21.15.46:0/1753463509 wait complete. 2024-09-10T07:48:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.544+0000 7f10da608740 20 client.28268 trim_cache size 0 max 16384 2024-09-10T07:48:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.593+0000 7f10da608740 1 Processor -- start 2024-09-10T07:48:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.593+0000 7f10da608740 1 -- start start 2024-09-10T07:48:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.593+0000 7f10da608740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dda5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.593+0000 7f10da608740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.593+0000 7f10da608740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.593+0000 7f10da608740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01d7370 con 0x55d989de2d70 2024-09-10T07:48:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.593+0000 7f10da608740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01d8eb0 con 0x55d989dd7b90 2024-09-10T07:48:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.593+0000 7f10da608740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01d7a40 con 0x55d989dd84b0 2024-09-10T07:48:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.594+0000 7f10d5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dda5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.594+0000 7f10d5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dda5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:32948/0 (socket says 172.21.15.46:32948) 2024-09-10T07:48:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.594+0000 7f10d5000640 1 -- 172.21.15.46:0/828802613 learned_addr learned my addr 172.21.15.46:0/828802613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.594+0000 7f10cfe00640 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.594+0000 7f10d5a00640 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.595+0000 7f10d5a00640 1 -- 172.21.15.46:0/828802613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 msgr2=0x55d989de0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.595+0000 7f10d5a00640 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.595+0000 7f10d5a00640 1 -- 172.21.15.46:0/828802613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 msgr2=0x55d989dda5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.595+0000 7f10d5a00640 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dda5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.595+0000 7f10d5a00640 1 -- 172.21.15.46:0/828802613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d989dd52b0 con 0x55d989de2d70 2024-09-10T07:48:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.596+0000 7f10d5a00640 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55d989dd54a0 tx=0x7f10c80099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248a769eeaba0e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.596+0000 7f10c5600640 1 -- 172.21.15.46:0/828802613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8031460 con 0x55d989de2d70 2024-09-10T07:48:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.597+0000 7f10c5600640 1 -- 172.21.15.46:0/828802613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c8036040 con 0x55d989de2d70 2024-09-10T07:48:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.597+0000 7f10c5600640 1 -- 172.21.15.46:0/828802613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c80043a0 con 0x55d989de2d70 2024-09-10T07:48:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.598+0000 7f10da608740 1 -- 172.21.15.46:0/828802613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 msgr2=0x55d989de0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.598+0000 7f10da608740 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55d989dd54a0 tx=0x7f10c80099c0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.598+0000 7f10da608740 1 -- 172.21.15.46:0/828802613 shutdown_connections 2024-09-10T07:48:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.598+0000 7f10da608740 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989de2d70 0x55d989de0790 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.598+0000 7f10da608740 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989dd84b0 0x55d989de0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.598+0000 7f10da608740 1 --2- 172.21.15.46:0/828802613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd7b90 0x55d989dda5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.598+0000 7f10da608740 1 -- 172.21.15.46:0/828802613 >> 172.21.15.46:0/828802613 conn(0x55d989dd4120 msgr2=0x55d989de1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.599+0000 7f10da608740 1 -- 172.21.15.46:0/828802613 shutdown_connections 2024-09-10T07:48:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.599+0000 7f10da608740 1 -- 172.21.15.46:0/828802613 wait complete. 2024-09-10T07:48:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.602+0000 7f10da608740 1 -- start start 2024-09-10T07:48:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.603+0000 7f10da608740 1 Processor -- start 2024-09-10T07:48:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.604+0000 7f10da608740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989dd7b90 0x55d989de6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.604+0000 7f10da608740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd84b0 0x55d989de7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.604+0000 7f10da608740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989de2d70 0x55d989de7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.606+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd84b0 0x55d989de7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.607+0000 7f10da608740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01d87e0 con 0x55d989dd7b90 2024-09-10T07:48:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.607+0000 7f10da608740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01d8110 con 0x55d989dd84b0 2024-09-10T07:48:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.607+0000 7f10da608740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01d5350 con 0x55d989de2d70 2024-09-10T07:48:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.607+0000 7f10da608740 10 client.0 _ll_register_callbacks cb 0x7f10c0011470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.608+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd84b0 0x55d989de7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:32952/0 (socket says 172.21.15.46:32952) 2024-09-10T07:48:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.608+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989de2d70 0x55d989de7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.609+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989de2d70 0x55d989de7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:55756/0 (socket says 172.21.15.46:55756) 2024-09-10T07:48:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.609+0000 7f10d5a00640 1 -- 172.21.15.46:0/3551774366 learned_addr learned my addr 172.21.15.46:0/3551774366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.610+0000 7f10d5a00640 1 -- 172.21.15.46:0/3551774366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd84b0 msgr2=0x55d989de7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.610+0000 7f10d5000640 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989dd7b90 0x55d989de6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.610+0000 7f10d5a00640 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd84b0 0x55d989de7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.610+0000 7f10d5a00640 1 -- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989dd7b90 msgr2=0x55d989de6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.611+0000 7f10cfe00640 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd84b0 0x55d989de7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T07:48:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.611+0000 7f10d5a00640 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989dd7b90 0x55d989de6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.611+0000 7f10d5a00640 1 -- 172.21.15.46:0/3551774366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d989e5c9a0 con 0x55d989de2d70 2024-09-10T07:48:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.611+0000 7f10d5000640 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989dd7b90 0x55d989de6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.613+0000 7f10d5a00640 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989de2d70 0x55d989de7840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f10c8031f00 tx=0x7f10c8031f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6edbe7a09354cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.613+0000 7f10baa00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.613+0000 7f10baa00640 1 -- 172.21.15.46:0/3551774366 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8051020 con 0x55d989de2d70 2024-09-10T07:48:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.614+0000 7f10baa00640 1 -- 172.21.15.46:0/3551774366 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f10c8031870 con 0x55d989de2d70 2024-09-10T07:48:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.614+0000 7f10da608740 1 -- 172.21.15.46:0/3551774366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10c8036040 con 0x55d989de2d70 2024-09-10T07:48:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.619+0000 7f10baa00640 1 -- 172.21.15.46:0/3551774366 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f10c8004000 con 0x55d989de2d70 2024-09-10T07:48:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.619+0000 7f10da608740 10 client.28216 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.619+0000 7f10da608740 1 -- 172.21.15.46:0/3551774366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d989e5dcb0 con 0x55d989de2d70 2024-09-10T07:48:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.622+0000 7f10baa00640 1 -- 172.21.15.46:0/3551774366 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c8004680 con 0x55d989de2d70 2024-09-10T07:48:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.622+0000 7f10baa00640 1 -- 172.21.15.46:0/3551774366 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f10c802fa80 con 0x55d989de2d70 2024-09-10T07:48:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.623+0000 7f10da608740 10 client.28216 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.623+0000 7f10da608740 10 client.28216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.623+0000 7f10da608740 20 client.28216 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.623+0000 7f10da608740 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d989e5e320 0x55d989e60710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.623+0000 7f10da608740 4 client.28216 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:48:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.623+0000 7f10da608740 1 -- 172.21.15.46:0/3551774366 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f10c8036040 con 0x55d989e5e320 2024-09-10T07:48:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.624+0000 7f10d5000640 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d989e5e320 0x55d989e60710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.625+0000 7f10d5000640 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d989e5e320 0x55d989e60710 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3b05aa7af9bab9 server_cookie=43e62356e49cc6bf in_seq=0 out_seq=0 2024-09-10T07:48:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.625+0000 7f10baa00640 10 client.28216 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.626+0000 7f10baa00640 1 -- 172.21.15.46:0/3551774366 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f10c8036040 con 0x55d989e5e320 2024-09-10T07:48:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.626+0000 7f10baa00640 10 client.28216 handle_command_reply: tid=0 2024-09-10T07:48:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.626+0000 7f10da608740 1 client.28216 shutdown 2024-09-10T07:48:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.626+0000 7f10da608740 1 -- 172.21.15.46:0/3551774366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989de2d70 msgr2=0x55d989de7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.626+0000 7f10da608740 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989de2d70 0x55d989de7840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f10c8031f00 tx=0x7f10c8031f30 comp rx=0 tx=0).stop 2024-09-10T07:48:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.627+0000 7f10da608740 1 -- 172.21.15.46:0/3551774366 shutdown_connections 2024-09-10T07:48:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.627+0000 7f10da608740 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d989e5e320 0x55d989e60710 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.627+0000 7f10da608740 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d989de2d70 0x55d989de7840 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.627+0000 7f10baa00640 0 client.28216 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.627+0000 7f10da608740 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d989dd84b0 0x55d989de7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.627+0000 7f10da608740 1 --2- 172.21.15.46:0/3551774366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d989dd7b90 0x55d989de6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.627+0000 7f10da608740 1 -- 172.21.15.46:0/3551774366 >> 172.21.15.46:0/3551774366 conn(0x55d989dd4120 msgr2=0x55d989dd29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.628+0000 7f10da608740 1 -- 172.21.15.46:0/3551774366 shutdown_connections 2024-09-10T07:48:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.628+0000 7f10da608740 1 -- 172.21.15.46:0/3551774366 wait complete. 2024-09-10T07:48:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.628+0000 7f10da608740 20 client.28216 trim_cache size 0 max 16384 2024-09-10T07:48:19.629 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:48:19.629 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:48:19.629 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:48:19.629 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:48:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.630+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10b40a2d40 msgr2=0x7f10b40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.630+0000 7f10d7a00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10b40a2d40 0x7f10b40a5200 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f10c0005fd0 tx=0x7f10c0005850 comp rx=0 tx=0).stop 2024-09-10T07:48:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.630+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01036e0 msgr2=0x7f10d0073ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.630+0000 7f10d7a00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01036e0 0x7f10d0073ec0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00d740 tx=0x7f10bc00dc00 comp rx=0 tx=0).stop 2024-09-10T07:48:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.631+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 shutdown_connections 2024-09-10T07:48:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.631+0000 7f10d7a00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10d01082a0 0x7f10d006cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.631+0000 7f10d7a00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10d01036e0 0x7f10d0073ec0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.631+0000 7f10d7a00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f10b40a2d40 0x7f10b40a5200 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.631+0000 7f10d7a00640 1 --2- 172.21.15.46:0/4080920313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d0102d30 0x7f10d0073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.631+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 >> 172.21.15.46:0/4080920313 conn(0x7f10d00fc7d0 msgr2=0x7f10d0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.631+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 shutdown_connections 2024-09-10T07:48:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:19.631+0000 7f10d7a00640 1 -- 172.21.15.46:0/4080920313 wait complete. 2024-09-10T07:48:20.396 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:20 smithi046 ceph-mon[30705]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-09-10T07:48:20.396 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:20 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:48:20.396 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:20 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:48:20.396 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:20 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:48:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:20 smithi181 ceph-mon[35397]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-09-10T07:48:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:20 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:48:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:20 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:48:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:20 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:48:20.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:20 smithi063 ceph-mon[35020]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-09-10T07:48:20.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:20 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:48:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:20 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:48:20.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:20 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:48:20.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 660.0 s, 659.4 tps, lat 153.036 ms stddev 171.182 2024-09-10T07:48:20.999 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:48:20.999 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:48:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:22.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:22 smithi046 ceph-mon[30705]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 543 op/s 2024-09-10T07:48:22.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:22 smithi181 ceph-mon[35397]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 543 op/s 2024-09-10T07:48:22.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:22 smithi063 ceph-mon[35020]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.8 MiB/s wr, 543 op/s 2024-09-10T07:48:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:24 smithi181 ceph-mon[35397]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.5 MiB/s wr, 568 op/s 2024-09-10T07:48:24.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:24 smithi063 ceph-mon[35020]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.5 MiB/s wr, 568 op/s 2024-09-10T07:48:24.661 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:24 smithi046 ceph-mon[30705]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.5 MiB/s wr, 568 op/s 2024-09-10T07:48:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 665.0 s, 516.2 tps, lat 192.898 ms stddev 273.810 2024-09-10T07:48:25.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:25 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1451118339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:48:25.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:25 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1451118339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:48:25.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:25 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1451118339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:48:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:48:25.963 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"btime":"2024-09-10T07:46:53:072255+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:46:52.230107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:48:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 -- 172.21.15.46:0/2517150935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c1027a0 msgr2=0x7f9c4c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 --2- 172.21.15.46:0/2517150935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c1027a0 0x7f9c4c1067f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009900 tx=0x7f9c3802a410 comp rx=0 tx=0).stop 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 -- 172.21.15.46:0/2517150935 shutdown_connections 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 --2- 172.21.15.46:0/2517150935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c106f20 0x7f9c4c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 --2- 172.21.15.46:0/2517150935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c1027a0 0x7f9c4c1067f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 --2- 172.21.15.46:0/2517150935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c4c101df0 0x7f9c4c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 -- 172.21.15.46:0/2517150935 >> 172.21.15.46:0/2517150935 conn(0x7f9c4c0fd8b0 msgr2=0x7f9c4c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 -- 172.21.15.46:0/2517150935 shutdown_connections 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.834+0000 7f9c52000640 1 -- 172.21.15.46:0/2517150935 wait complete. 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.836+0000 7f9c52000640 1 Processor -- start 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.838+0000 7f9c52000640 1 -- start start 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.839+0000 7f9c52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c101df0 0x7f9c4c10f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.839+0000 7f9c52000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c4c1027a0 0x7f9c4c10fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.839+0000 7f9c52000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c106f20 0x7f9c4c110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.839+0000 7f9c52000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c114210 con 0x7f9c4c1027a0 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.839+0000 7f9c52000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c114090 con 0x7f9c4c101df0 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.839+0000 7f9c52000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c114390 con 0x7f9c4c106f20 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.841+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c101df0 0x7f9c4c10f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.841+0000 7f9c4aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c4c1027a0 0x7f9c4c10fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.841+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c101df0 0x7f9c4c10f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:32984/0 (socket says 172.21.15.46:32984) 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.841+0000 7f9c4b400640 1 -- 172.21.15.46:0/1451118339 learned_addr learned my addr 172.21.15.46:0/1451118339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.841+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c106f20 0x7f9c4c110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.842+0000 7f9c4be00640 1 -- 172.21.15.46:0/1451118339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c101df0 msgr2=0x7f9c4c10f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.842+0000 7f9c4be00640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c101df0 0x7f9c4c10f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.842+0000 7f9c4be00640 1 -- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c4c1027a0 msgr2=0x7f9c4c10fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.843+0000 7f9c4be00640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c4c1027a0 0x7f9c4c10fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.843+0000 7f9c4be00640 1 -- 172.21.15.46:0/1451118339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c1aad20 con 0x7f9c4c106f20 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.843+0000 7f9c4aa00640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c4c1027a0 0x7f9c4c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.843+0000 7f9c4be00640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c106f20 0x7f9c4c110140 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400ea80 tx=0x7f9c4400ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.843+0000 7f9c4b400640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c101df0 0x7f9c4c10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.844+0000 7f9c43e00640 1 -- 172.21.15.46:0/1451118339 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400cea0 con 0x7f9c4c106f20 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.844+0000 7f9c43e00640 1 -- 172.21.15.46:0/1451118339 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c44005020 con 0x7f9c4c106f20 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.844+0000 7f9c43e00640 1 -- 172.21.15.46:0/1451118339 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44010690 con 0x7f9c4c106f20 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.844+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c1ab010 con 0x7f9c4c106f20 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.844+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c1ab4d0 con 0x7f9c4c106f20 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.847+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c1032d0 con 0x7f9c4c106f20 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.851+0000 7f9c43e00640 1 -- 172.21.15.46:0/1451118339 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9c44005520 con 0x7f9c4c106f20 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.853+0000 7f9c43e00640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c300a2c00 0x7f9c300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.853+0000 7f9c43e00640 1 -- 172.21.15.46:0/1451118339 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9c440cf8e0 con 0x7f9c4c106f20 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.854+0000 7f9c4b400640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c300a2c00 0x7f9c300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.856+0000 7f9c4b400640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c300a2c00 0x7f9c300a50c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009e10 tx=0x7f9c3c0091c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:23.863+0000 7f9c43e00640 1 -- 172.21.15.46:0/1451118339 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c4409c4c0 con 0x7f9c4c106f20 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.266+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c4c02d7c0 con 0x7f9c4c106f20 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.267+0000 7f9c43e00640 1 -- 172.21.15.46:0/1451118339 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7f9c4409bdc0 con 0x7f9c4c106f20 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c300a2c00 msgr2=0x7f9c300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c300a2c00 0x7f9c300a50c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009e10 tx=0x7f9c3c0091c0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c106f20 msgr2=0x7f9c4c110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c106f20 0x7f9c4c110140 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400ea80 tx=0x7f9c4400ef40 comp rx=0 tx=0).stop 2024-09-10T07:48:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 shutdown_connections 2024-09-10T07:48:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c106f20 0x7f9c4c110140 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9c300a2c00 0x7f9c300a50c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c4c1027a0 0x7f9c4c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 --2- 172.21.15.46:0/1451118339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c101df0 0x7f9c4c10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 >> 172.21.15.46:0/1451118339 conn(0x7f9c4c0fd8b0 msgr2=0x7f9c4c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 shutdown_connections 2024-09-10T07:48:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:24.269+0000 7f9c52000640 1 -- 172.21.15.46:0/1451118339 wait complete. 2024-09-10T07:48:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:26 smithi181 ceph-mon[35397]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 MiB/s wr, 461 op/s 2024-09-10T07:48:26.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:26 smithi063 ceph-mon[35020]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 MiB/s wr, 461 op/s 2024-09-10T07:48:26.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:26 smithi046 ceph-mon[30705]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 MiB/s wr, 461 op/s 2024-09-10T07:48:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.598+0000 7fc302e00640 1 -- 172.21.15.46:0/4210151628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc102d50 msgr2=0x7fc2fc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.598+0000 7fc302e00640 1 --2- 172.21.15.46:0/4210151628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc102d50 0x7fc2fc1031d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009900 tx=0x7fc2ec02a410 comp rx=0 tx=0).stop 2024-09-10T07:48:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.599+0000 7fc302e00640 1 -- 172.21.15.46:0/4210151628 shutdown_connections 2024-09-10T07:48:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.599+0000 7fc302e00640 1 --2- 172.21.15.46:0/4210151628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc103710 0x7fc2fc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.599+0000 7fc302e00640 1 --2- 172.21.15.46:0/4210151628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc102d50 0x7fc2fc1031d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.599+0000 7fc302e00640 1 --2- 172.21.15.46:0/4210151628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2fc101b60 0x7fc2fc101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.599+0000 7fc302e00640 1 -- 172.21.15.46:0/4210151628 >> 172.21.15.46:0/4210151628 conn(0x7fc2fc0fd2c0 msgr2=0x7fc2fc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.601+0000 7fc302e00640 1 -- 172.21.15.46:0/4210151628 shutdown_connections 2024-09-10T07:48:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.601+0000 7fc302e00640 1 -- 172.21.15.46:0/4210151628 wait complete. 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.602+0000 7fc302e00640 1 Processor -- start 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.603+0000 7fc302e00640 1 -- start start 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.603+0000 7fc302e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2fc101b60 0x7fc2fc075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.603+0000 7fc302e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc102d50 0x7fc2fc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.603+0000 7fc302e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc103710 0x7fc2fc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.603+0000 7fc302e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc10d860 con 0x7fc2fc102d50 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.603+0000 7fc302e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc10d6e0 con 0x7fc2fc103710 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.603+0000 7fc300e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc103710 0x7fc2fc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2fc101b60 0x7fc2fc075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc300e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc103710 0x7fc2fc074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42530/0 (socket says 172.21.15.46:42530) 2024-09-10T07:48:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc300e00640 1 -- 172.21.15.46:0/2358082463 learned_addr learned my addr 172.21.15.46:0/2358082463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc10d9e0 con 0x7fc2fc101b60 2024-09-10T07:48:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc2fb400640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc102d50 0x7fc2fc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc300e00640 1 -- 172.21.15.46:0/2358082463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2fc101b60 msgr2=0x7fc2fc075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc300e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2fc101b60 0x7fc2fc075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc300e00640 1 -- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc102d50 msgr2=0x7fc2fc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc300e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc102d50 0x7fc2fc0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.604+0000 7fc300e00640 1 -- 172.21.15.46:0/2358082463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc19be70 con 0x7fc2fc103710 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.605+0000 7fc300e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc103710 0x7fc2fc074b20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400eaa0 tx=0x7fc2f400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.605+0000 7fc2f8c00640 1 -- 172.21.15.46:0/2358082463 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4004d20 con 0x7fc2fc103710 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.605+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc19c160 con 0x7fc2fc103710 2024-09-10T07:48:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.605+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc19c700 con 0x7fc2fc103710 2024-09-10T07:48:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.608+0000 7fc2f8c00640 1 -- 172.21.15.46:0/2358082463 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f4005020 con 0x7fc2fc103710 2024-09-10T07:48:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.608+0000 7fc2f8c00640 1 -- 172.21.15.46:0/2358082463 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40116d0 con 0x7fc2fc103710 2024-09-10T07:48:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.611+0000 7fc2f8c00640 1 -- 172.21.15.46:0/2358082463 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc2f40118f0 con 0x7fc2fc103710 2024-09-10T07:48:27.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.615+0000 7fc2f8c00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2e00a2cb0 0x7fc2e00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.616+0000 7fc305c55740 1 Processor -- start 2024-09-10T07:48:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.616+0000 7fc305c55740 1 -- start start 2024-09-10T07:48:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.616+0000 7fc305c55740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df6652490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.616+0000 7fc305c55740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df6657490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.617+0000 7fc305c55740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df6659f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.617+0000 7fc305c55740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564df65b7100 con 0x564df66529d0 2024-09-10T07:48:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.617+0000 7fc305c55740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564df655f1b0 con 0x564df6657a60 2024-09-10T07:48:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.617+0000 7fc305c55740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564df65b30c0 con 0x564df6652090 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.617+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df6652490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.617+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df6652490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34368/0 (socket says 172.21.15.46:34368) 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.617+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1627254240 learned_addr learned my addr 172.21.15.46:0/1627254240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.618+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1627254240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 msgr2=0x564df6659f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.618+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1627254240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df6659f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.618+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1627254240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 msgr2=0x564df6657490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.618+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1627254240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df6657490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.618+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1627254240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df62ca210 con 0x564df6652090 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.618+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2e00a2cb0 0x7fc2e00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.618+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1627254240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df6652490 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009b50 tx=0x7fc2e8010620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21186dc3913b6a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.619+0000 7fc2f8c00640 1 -- 172.21.15.46:0/2358082463 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc2f40d0500 con 0x7fc2fc103710 2024-09-10T07:48:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.622+0000 7fc2f1600640 1 -- 172.21.15.46:0/1627254240 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e801a030 con 0x564df6652090 2024-09-10T07:48:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.622+0000 7fc2f1600640 1 -- 172.21.15.46:0/1627254240 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e8009e50 con 0x564df6652090 2024-09-10T07:48:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.622+0000 7fc2f1600640 1 -- 172.21.15.46:0/1627254240 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e801f7b0 con 0x564df6652090 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.623+0000 7fc305c55740 1 -- 172.21.15.46:0/1627254240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 msgr2=0x564df6652490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.623+0000 7fc305c55740 1 --2- 172.21.15.46:0/1627254240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df6652490 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009b50 tx=0x7fc2e8010620 comp rx=0 tx=0).stop 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.623+0000 7fc305c55740 1 -- 172.21.15.46:0/1627254240 shutdown_connections 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.623+0000 7fc305c55740 1 --2- 172.21.15.46:0/1627254240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df6659f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.623+0000 7fc305c55740 1 --2- 172.21.15.46:0/1627254240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df6657490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.623+0000 7fc305c55740 1 --2- 172.21.15.46:0/1627254240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df6652490 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.623+0000 7fc305c55740 1 -- 172.21.15.46:0/1627254240 >> 172.21.15.46:0/1627254240 conn(0x564df664d700 msgr2=0x564df664db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.624+0000 7fc305c55740 1 -- 172.21.15.46:0/1627254240 shutdown_connections 2024-09-10T07:48:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.624+0000 7fc305c55740 1 -- 172.21.15.46:0/1627254240 wait complete. 2024-09-10T07:48:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.625+0000 7fc305c55740 1 -- start start 2024-09-10T07:48:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.625+0000 7fc305c55740 1 Processor -- start 2024-09-10T07:48:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.628+0000 7fc305c55740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df6652090 0x564df665c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.628+0000 7fc305c55740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df66529d0 0x564df665c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.628+0000 7fc305c55740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.629+0000 7fc300e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.629+0000 7fc300e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66637e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42544/0 (socket says 172.21.15.46:42544) 2024-09-10T07:48:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.629+0000 7fc300e00640 1 -- 172.21.15.46:0/89373896 learned_addr learned my addr 172.21.15.46:0/89373896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.629+0000 7fc2fb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df66529d0 0x564df665c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.629+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564df665aa80 con 0x564df6652090 2024-09-10T07:48:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.630+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564df6540090 con 0x564df6657a60 2024-09-10T07:48:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.630+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564df665ac00 con 0x564df66529d0 2024-09-10T07:48:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.631+0000 7fc305c55740 10 client.0 _ll_register_callbacks cb 0x564df6594f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.631+0000 7fc2fb400640 1 -- 172.21.15.46:0/89373896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 msgr2=0x564df66637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.631+0000 7fc2fb400640 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.631+0000 7fc2fb400640 1 -- 172.21.15.46:0/89373896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df6652090 msgr2=0x564df665c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.631+0000 7fc2fb400640 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df6652090 0x564df665c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.631+0000 7fc2fb400640 1 -- 172.21.15.46:0/89373896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df6396620 con 0x564df66529d0 2024-09-10T07:48:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.632+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2e00a2cb0 0x7fc2e00a5170 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009b50 tx=0x7fc2e8010620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.632+0000 7fc2fb400640 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df66529d0 0x564df665c7a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc075d10 tx=0x7fc2ec0095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f762523652817725 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.632+0000 7fc2e6000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.632+0000 7fc2e6000640 1 -- 172.21.15.46:0/89373896 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec040070 con 0x564df66529d0 2024-09-10T07:48:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.633+0000 7fc2e6000640 1 -- 172.21.15.46:0/89373896 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc2ec005200 con 0x564df66529d0 2024-09-10T07:48:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.633+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e8009e50 con 0x564df66529d0 2024-09-10T07:48:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.633+0000 7fc2e6000640 1 -- 172.21.15.46:0/89373896 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec04e050 con 0x564df66529d0 2024-09-10T07:48:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.635+0000 7fc305c55740 10 client.28231 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.635+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564df6666600 con 0x564df66529d0 2024-09-10T07:48:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.636+0000 7fc2e6000640 1 -- 172.21.15.46:0/89373896 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec005420 con 0x564df66529d0 2024-09-10T07:48:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.636+0000 7fc2e6000640 1 -- 172.21.15.46:0/89373896 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fc2ec031440 con 0x564df66529d0 2024-09-10T07:48:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.637+0000 7fc305c55740 10 client.28231 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.637+0000 7fc305c55740 10 client.28231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.637+0000 7fc305c55740 20 client.28231 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.637+0000 7fc305c55740 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564df6667cd0 0x564df666a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.637+0000 7fc305c55740 4 client.28231 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.637+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc2e8009e50 con 0x564df6667cd0 2024-09-10T07:48:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.640+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564df6667cd0 0x564df666a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.642+0000 7fc2e6000640 10 client.28231 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.642+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564df6667cd0 0x564df666a0c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d8a65545c3ee5 server_cookie=d564f53758adb3ec in_seq=0 out_seq=0 2024-09-10T07:48:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.645+0000 7fc2e6000640 1 -- 172.21.15.46:0/89373896 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc2e8009e50 con 0x564df6667cd0 2024-09-10T07:48:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.645+0000 7fc2e6000640 10 client.28231 handle_command_reply: tid=0 2024-09-10T07:48:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.645+0000 7fc305c55740 1 client.28231 shutdown 2024-09-10T07:48:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df66529d0 msgr2=0x564df665c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc305c55740 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df66529d0 0x564df665c7a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc075d10 tx=0x7fc2ec0095d0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 shutdown_connections 2024-09-10T07:48:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc305c55740 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564df6667cd0 0x564df666a0c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc305c55740 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc2e6000640 0 client.28231 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc305c55740 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df66529d0 0x564df665c7a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc305c55740 1 --2- 172.21.15.46:0/89373896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df6652090 0x564df665c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.646+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 >> 172.21.15.46:0/89373896 conn(0x564df664d700 msgr2=0x564df664ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.647+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 shutdown_connections 2024-09-10T07:48:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.647+0000 7fc305c55740 1 -- 172.21.15.46:0/89373896 wait complete. 2024-09-10T07:48:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.647+0000 7fc305c55740 20 client.28231 trim_cache size 0 max 16384 2024-09-10T07:48:27.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:48:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.680+0000 7fc305c55740 1 Processor -- start 2024-09-10T07:48:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.680+0000 7fc305c55740 1 -- start start 2024-09-10T07:48:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.680+0000 7fc305c55740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df665c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.680+0000 7fc305c55740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df66637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.680+0000 7fc305c55740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.681+0000 7fc305c55740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc00fb00 con 0x564df66529d0 2024-09-10T07:48:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.681+0000 7fc305c55740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc011640 con 0x564df6657a60 2024-09-10T07:48:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.681+0000 7fc305c55740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc0101d0 con 0x564df6652090 2024-09-10T07:48:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.681+0000 7fc300e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.681+0000 7fc300e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42560/0 (socket says 172.21.15.46:42560) 2024-09-10T07:48:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.681+0000 7fc300e00640 1 -- 172.21.15.46:0/1465566318 learned_addr learned my addr 172.21.15.46:0/1465566318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.681+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df665c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.681+0000 7fc2fb400640 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df66637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.682+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1465566318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 msgr2=0x564df66486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.682+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.682+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1465566318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 msgr2=0x564df66637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.682+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df66637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.682+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1465566318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df6652eb0 con 0x564df6652090 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.683+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df665c720 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009150 tx=0x7fc2e800b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9f6a3e3572b4425 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.683+0000 7fc2f0c00640 1 -- 172.21.15.46:0/1465566318 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e801a070 con 0x564df6652090 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.683+0000 7fc2f0c00640 1 -- 172.21.15.46:0/1465566318 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e8009e50 con 0x564df6652090 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.683+0000 7fc2f0c00640 1 -- 172.21.15.46:0/1465566318 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e802bd10 con 0x564df6652090 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.684+0000 7fc305c55740 1 -- 172.21.15.46:0/1465566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 msgr2=0x564df665c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.684+0000 7fc305c55740 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df665c720 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009150 tx=0x7fc2e800b000 comp rx=0 tx=0).stop 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.684+0000 7fc305c55740 1 -- 172.21.15.46:0/1465566318 shutdown_connections 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.684+0000 7fc305c55740 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6657a60 0x564df66486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.684+0000 7fc305c55740 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df66637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.684+0000 7fc305c55740 1 --2- 172.21.15.46:0/1465566318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6652090 0x564df665c720 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.684+0000 7fc305c55740 1 -- 172.21.15.46:0/1465566318 >> 172.21.15.46:0/1465566318 conn(0x564df664d700 msgr2=0x564df665c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:27.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.685+0000 7fc305c55740 1 -- 172.21.15.46:0/1465566318 shutdown_connections 2024-09-10T07:48:27.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.685+0000 7fc305c55740 1 -- 172.21.15.46:0/1465566318 wait complete. 2024-09-10T07:48:27.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.686+0000 7fc305c55740 1 -- start start 2024-09-10T07:48:27.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.686+0000 7fc305c55740 1 Processor -- start 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc305c55740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6652090 0x564df665ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc305c55740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df665f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6652090 0x564df665ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6652090 0x564df665ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42566/0 (socket says 172.21.15.46:42566) 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc305c55740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6657a60 0x564df66622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc2fbe00640 1 -- 172.21.15.46:0/3207645082 learned_addr learned my addr 172.21.15.46:0/3207645082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc010f70 con 0x564df66529d0 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc0108a0 con 0x564df6652090 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.687+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc00dae0 con 0x564df6657a60 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.688+0000 7fc300e00640 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6657a60 0x564df66622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.688+0000 7fc305c55740 10 client.0 _ll_register_callbacks cb 0x7fc2ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.688+0000 7fc300e00640 1 -- 172.21.15.46:0/3207645082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6652090 msgr2=0x564df665ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.688+0000 7fc300e00640 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6652090 0x564df665ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.688+0000 7fc300e00640 1 -- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 msgr2=0x564df665f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.689+0000 7fc2fb400640 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df665f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.691+0000 7fc300e00640 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df665f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.691+0000 7fc300e00640 1 -- 172.21.15.46:0/3207645082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df66628d0 con 0x564df6657a60 2024-09-10T07:48:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.691+0000 7fc2fb400640 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df665f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.691+0000 7fc300e00640 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6657a60 0x564df66622c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400efd0 tx=0x7fc2f409d0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4037b2b1108b82a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.691+0000 7fc2e6000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.691+0000 7fc2e6000640 1 -- 172.21.15.46:0/3207645082 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40b5060 con 0x564df6657a60 2024-09-10T07:48:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.691+0000 7fc2e6000640 1 -- 172.21.15.46:0/3207645082 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc2f4011be0 con 0x564df6657a60 2024-09-10T07:48:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.691+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e8009e50 con 0x564df6657a60 2024-09-10T07:48:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.692+0000 7fc2e6000640 1 -- 172.21.15.46:0/3207645082 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc2f401f040 con 0x564df6657a60 2024-09-10T07:48:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.692+0000 7fc305c55740 10 client.28237 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.692+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564df66d2980 con 0x564df6657a60 2024-09-10T07:48:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.692+0000 7fc2e6000640 1 -- 172.21.15.46:0/3207645082 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e8009e50 con 0x564df6657a60 2024-09-10T07:48:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.692+0000 7fc2e6000640 1 -- 172.21.15.46:0/3207645082 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fc2f400f040 con 0x564df6657a60 2024-09-10T07:48:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.693+0000 7fc305c55740 10 client.28237 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.693+0000 7fc305c55740 10 client.28237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.693+0000 7fc305c55740 20 client.28237 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.693+0000 7fc305c55740 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564df66d31d0 0x564df66d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.693+0000 7fc305c55740 4 client.28237 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:48:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.693+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564df6648d80 con 0x564df66d31d0 2024-09-10T07:48:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.694+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564df66d31d0 0x564df66d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.695+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564df66d31d0 0x564df66d55f0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b97e94d264c187cf server_cookie=d6d5f4b096127960 in_seq=0 out_seq=0 2024-09-10T07:48:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.697+0000 7fc2e6000640 10 client.28237 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc2e6000640 1 -- 172.21.15.46:0/3207645082 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564df6648d80 con 0x564df66d31d0 2024-09-10T07:48:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc2e6000640 10 client.28237 handle_command_reply: tid=0 2024-09-10T07:48:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 client.28237 shutdown 2024-09-10T07:48:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6657a60 msgr2=0x564df66622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6657a60 0x564df66622c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400efd0 tx=0x7fc2f409d0e0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 shutdown_connections 2024-09-10T07:48:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564df66d31d0 0x564df66d55f0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564df6657a60 0x564df66622c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564df66529d0 0x564df665f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc2e6000640 0 client.28237 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 --2- 172.21.15.46:0/3207645082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564df6652090 0x564df665ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.746+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 >> 172.21.15.46:0/3207645082 conn(0x564df664d700 msgr2=0x564df664c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.747+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 shutdown_connections 2024-09-10T07:48:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.747+0000 7fc305c55740 1 -- 172.21.15.46:0/3207645082 wait complete. 2024-09-10T07:48:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.747+0000 7fc305c55740 20 client.28237 trim_cache size 0 max 16384 2024-09-10T07:48:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:48:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:48:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:48:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.748+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2e00a2cb0 msgr2=0x7fc2e00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.748+0000 7fc302e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2e00a2cb0 0x7fc2e00a5170 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009b50 tx=0x7fc2e8010620 comp rx=0 tx=0).stop 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.748+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc103710 msgr2=0x7fc2fc074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc103710 0x7fc2fc074b20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400eaa0 tx=0x7fc2f400eea0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 shutdown_connections 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2fc103710 0x7fc2fc074b20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc2e00a2cb0 0x7fc2e00a5170 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc102d50 0x7fc2fc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 --2- 172.21.15.46:0/2358082463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2fc101b60 0x7fc2fc075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 >> 172.21.15.46:0/2358082463 conn(0x7fc2fc0fd2c0 msgr2=0x7fc2fc1060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 shutdown_connections 2024-09-10T07:48:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:27.749+0000 7fc302e00640 1 -- 172.21.15.46:0/2358082463 wait complete. 2024-09-10T07:48:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:28.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:28 smithi046 ceph-mon[30705]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 MiB/s wr, 696 op/s 2024-09-10T07:48:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:28 smithi181 ceph-mon[35397]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 MiB/s wr, 696 op/s 2024-09-10T07:48:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:28 smithi063 ceph-mon[35020]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 MiB/s wr, 696 op/s 2024-09-10T07:48:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.180+0000 7fb140e00640 1 -- 172.21.15.46:0/44672921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c103ff0 msgr2=0x7fb13c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.180+0000 7fb140e00640 1 --2- 172.21.15.46:0/44672921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c103ff0 0x7fb13c1043d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009900 tx=0x7fb12c02a410 comp rx=0 tx=0).stop 2024-09-10T07:48:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.180+0000 7fb140e00640 1 -- 172.21.15.46:0/44672921 shutdown_connections 2024-09-10T07:48:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.180+0000 7fb140e00640 1 --2- 172.21.15.46:0/44672921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c109120 0x7fb13c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.180+0000 7fb140e00640 1 --2- 172.21.15.46:0/44672921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c1049a0 0x7fb13c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.180+0000 7fb140e00640 1 --2- 172.21.15.46:0/44672921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c103ff0 0x7fb13c1043d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.180+0000 7fb140e00640 1 -- 172.21.15.46:0/44672921 >> 172.21.15.46:0/44672921 conn(0x7fb13c0ffa90 msgr2=0x7fb13c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.181+0000 7fb140e00640 1 -- 172.21.15.46:0/44672921 shutdown_connections 2024-09-10T07:48:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.181+0000 7fb140e00640 1 -- 172.21.15.46:0/44672921 wait complete. 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.182+0000 7fb140e00640 1 Processor -- start 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.182+0000 7fb140e00640 1 -- start start 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.183+0000 7fb140e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c103ff0 0x7fb13c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.183+0000 7fb140e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c1049a0 0x7fb13c076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.183+0000 7fb140e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c109120 0x7fb13c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.183+0000 7fb140e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1141a0 con 0x7fb13c103ff0 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.183+0000 7fb140e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c114020 con 0x7fb13c109120 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.183+0000 7fb140e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c114320 con 0x7fb13c1049a0 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.184+0000 7fb13a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c109120 0x7fb13c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.184+0000 7fb13a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c109120 0x7fb13c077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42582/0 (socket says 172.21.15.46:42582) 2024-09-10T07:48:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.184+0000 7fb139400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c1049a0 0x7fb13c076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.184+0000 7fb13a800640 1 -- 172.21.15.46:0/2683411301 learned_addr learned my addr 172.21.15.46:0/2683411301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.184+0000 7fb139e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c103ff0 0x7fb13c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.185+0000 7fb139400640 1 -- 172.21.15.46:0/2683411301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c109120 msgr2=0x7fb13c077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.185+0000 7fb139400640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c109120 0x7fb13c077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.185+0000 7fb139400640 1 -- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c103ff0 msgr2=0x7fb13c076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.185+0000 7fb139400640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c103ff0 0x7fb13c076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.185+0000 7fb139400640 1 -- 172.21.15.46:0/2683411301 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c1aaca0 con 0x7fb13c1049a0 2024-09-10T07:48:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.185+0000 7fb139400640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c1049a0 0x7fb13c076c40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb12800ea20 tx=0x7fb12800eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.186+0000 7fb132a00640 1 -- 172.21.15.46:0/2683411301 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12800ccc0 con 0x7fb13c1049a0 2024-09-10T07:48:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.186+0000 7fb132a00640 1 -- 172.21.15.46:0/2683411301 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12800ce60 con 0x7fb13c1049a0 2024-09-10T07:48:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.186+0000 7fb132a00640 1 -- 172.21.15.46:0/2683411301 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1280188b0 con 0x7fb13c1049a0 2024-09-10T07:48:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.186+0000 7fb140e00640 1 -- 172.21.15.46:0/2683411301 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c1aaf90 con 0x7fb13c1049a0 2024-09-10T07:48:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.187+0000 7fb140e00640 1 -- 172.21.15.46:0/2683411301 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c1ab3f0 con 0x7fb13c1049a0 2024-09-10T07:48:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.194+0000 7fb132a00640 1 -- 172.21.15.46:0/2683411301 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb128018a50 con 0x7fb13c1049a0 2024-09-10T07:48:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.196+0000 7fb14384d740 1 Processor -- start 2024-09-10T07:48:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.196+0000 7fb14384d740 1 -- start start 2024-09-10T07:48:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.196+0000 7fb14384d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe68fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.196+0000 7fb14384d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 0x55ebe6907910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.196+0000 7fb14384d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.196+0000 7fb14384d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ebe68633e0 con 0x55ebe68fd590 2024-09-10T07:48:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.196+0000 7fb14384d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ebe680ad80 con 0x55ebe6907e50 2024-09-10T07:48:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.196+0000 7fb14384d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ebe685f3a0 con 0x55ebe68fcc70 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb139400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb13a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe68fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb13a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe68fd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34430/0 (socket says 172.21.15.46:34430) 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb139400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42598/0 (socket says 172.21.15.46:42598) 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb13a800640 1 -- 172.21.15.46:0/3461313695 learned_addr learned my addr 172.21.15.46:0/3461313695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb139e00640 1 --2- 172.21.15.46:0/3461313695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 0x55ebe6907910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb13a800640 1 -- 172.21.15.46:0/3461313695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 msgr2=0x55ebe690a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb13a800640 1 --2- 172.21.15.46:0/3461313695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb13a800640 1 -- 172.21.15.46:0/3461313695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 msgr2=0x55ebe6907910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb13a800640 1 --2- 172.21.15.46:0/3461313695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 0x55ebe6907910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.197+0000 7fb13a800640 1 -- 172.21.15.46:0/3461313695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebe6829ae0 con 0x55ebe68fcc70 2024-09-10T07:48:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.198+0000 7fb132a00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1200a2c00 0x7fb1200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.198+0000 7fb13a800640 1 --2- 172.21.15.46:0/3461313695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe68fd050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb134009fa0 tx=0x7fb134011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bc2e105df7bcdec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.198+0000 7fb132a00640 1 -- 172.21.15.46:0/2683411301 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb1280cf620 con 0x7fb13c1049a0 2024-09-10T07:48:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.198+0000 7fb127400640 1 -- 172.21.15.46:0/3461313695 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134012710 con 0x55ebe68fcc70 2024-09-10T07:48:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.198+0000 7fb139e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1200a2c00 0x7fb1200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.198+0000 7fb127400640 1 -- 172.21.15.46:0/3461313695 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb134012d90 con 0x55ebe68fcc70 2024-09-10T07:48:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.198+0000 7fb127400640 1 -- 172.21.15.46:0/3461313695 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13401a3f0 con 0x55ebe68fcc70 2024-09-10T07:48:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.199+0000 7fb14384d740 1 -- 172.21.15.46:0/3461313695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 msgr2=0x55ebe68fd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.199+0000 7fb139e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1200a2c00 0x7fb1200a50c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb12c000c00 tx=0x7fb12c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.199+0000 7fb14384d740 1 --2- 172.21.15.46:0/3461313695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe68fd050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb134009fa0 tx=0x7fb134011870 comp rx=0 tx=0).stop 2024-09-10T07:48:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.200+0000 7fb14384d740 1 -- 172.21.15.46:0/3461313695 shutdown_connections 2024-09-10T07:48:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.200+0000 7fb14384d740 1 --2- 172.21.15.46:0/3461313695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.200+0000 7fb14384d740 1 --2- 172.21.15.46:0/3461313695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 0x55ebe6907910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.200+0000 7fb14384d740 1 --2- 172.21.15.46:0/3461313695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe68fd050 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.200+0000 7fb14384d740 1 -- 172.21.15.46:0/3461313695 >> 172.21.15.46:0/3461313695 conn(0x55ebe68f9200 msgr2=0x55ebe68f95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.200+0000 7fb14384d740 1 -- 172.21.15.46:0/3461313695 shutdown_connections 2024-09-10T07:48:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.200+0000 7fb14384d740 1 -- 172.21.15.46:0/3461313695 wait complete. 2024-09-10T07:48:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.200+0000 7fb14384d740 1 -- start start 2024-09-10T07:48:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.201+0000 7fb14384d740 1 Processor -- start 2024-09-10T07:48:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.201+0000 7fb14384d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe68fcc70 0x55ebe6904d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.201+0000 7fb14384d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe6905250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.201+0000 7fb14384d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe6907e50 0x55ebe6905790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.201+0000 7fb14384d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ebe690aa20 con 0x55ebe6907e50 2024-09-10T07:48:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.201+0000 7fb14384d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ebe681e0f0 con 0x55ebe68fcc70 2024-09-10T07:48:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.201+0000 7fb14384d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ebe690aba0 con 0x55ebe68fd590 2024-09-10T07:48:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.201+0000 7fb14384d740 10 client.0 _ll_register_callbacks cb 0x55ebe6805bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe68fcc70 0x55ebe6904d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe68fcc70 0x55ebe6904d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42612/0 (socket says 172.21.15.46:42612) 2024-09-10T07:48:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139e00640 1 -- 172.21.15.46:0/1420113993 learned_addr learned my addr 172.21.15.46:0/1420113993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb13a800640 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe6907e50 0x55ebe6905790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139400640 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe6905250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139e00640 1 -- 172.21.15.46:0/1420113993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 msgr2=0x55ebe6905250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139e00640 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe6905250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139e00640 1 -- 172.21.15.46:0/1420113993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe6907e50 msgr2=0x55ebe6905790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139e00640 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe6907e50 0x55ebe6905790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.202+0000 7fb139e00640 1 -- 172.21.15.46:0/1420113993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebe65d43c0 con 0x55ebe68fcc70 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.203+0000 7fb13a800640 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe6907e50 0x55ebe6905790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.203+0000 7fb11be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.203+0000 7fb139e00640 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe68fcc70 0x55ebe6904d10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009900 tx=0x7fb12c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a82099f54882ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.203+0000 7fb11be00640 1 -- 172.21.15.46:0/1420113993 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c04e070 con 0x55ebe68fcc70 2024-09-10T07:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.203+0000 7fb11be00640 1 -- 172.21.15.46:0/1420113993 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb12c0048c0 con 0x55ebe68fcc70 2024-09-10T07:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.203+0000 7fb14384d740 1 -- 172.21.15.46:0/1420113993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb134012d90 con 0x55ebe68fcc70 2024-09-10T07:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.204+0000 7fb11be00640 1 -- 172.21.15.46:0/1420113993 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb12c05a050 con 0x55ebe68fcc70 2024-09-10T07:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.204+0000 7fb14384d740 10 client.28304 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.204+0000 7fb14384d740 1 -- 172.21.15.46:0/1420113993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ebe6916ab0 con 0x55ebe68fcc70 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.204+0000 7fb11be00640 1 -- 172.21.15.46:0/1420113993 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb134012d90 con 0x55ebe68fcc70 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.204+0000 7fb11be00640 1 -- 172.21.15.46:0/1420113993 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fb12c0035f0 con 0x55ebe68fcc70 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.205+0000 7fb14384d740 10 client.28304 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.205+0000 7fb14384d740 10 client.28304 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.205+0000 7fb14384d740 20 client.28304 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.205+0000 7fb14384d740 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55ebe6917f10 0x55ebe691a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.205+0000 7fb14384d740 4 client.28304 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.205+0000 7fb14384d740 1 -- 172.21.15.46:0/1420113993 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebe6870950 con 0x55ebe6917f10 2024-09-10T07:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.206+0000 7fb139400640 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55ebe6917f10 0x55ebe691a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.206+0000 7fb139400640 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55ebe6917f10 0x55ebe691a320 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7534e073f026c30 server_cookie=4d5d11fb7d1c3b27 in_seq=0 out_seq=0 2024-09-10T07:48:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.206+0000 7fb11be00640 10 client.28304 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:48:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.207+0000 7fb11be00640 1 -- 172.21.15.46:0/1420113993 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ebe6870950 con 0x55ebe6917f10 2024-09-10T07:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.207+0000 7fb11be00640 10 client.28304 handle_command_reply: tid=0 2024-09-10T07:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.207+0000 7fb14384d740 1 client.28304 shutdown 2024-09-10T07:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.207+0000 7fb14384d740 1 -- 172.21.15.46:0/1420113993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe68fcc70 msgr2=0x55ebe6904d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.207+0000 7fb14384d740 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe68fcc70 0x55ebe6904d10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009900 tx=0x7fb12c009260 comp rx=0 tx=0).stop 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 1 -- 172.21.15.46:0/1420113993 shutdown_connections 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55ebe6917f10 0x55ebe691a320 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe6907e50 0x55ebe6905790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe6905250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 1 --2- 172.21.15.46:0/1420113993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe68fcc70 0x55ebe6904d10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 1 -- 172.21.15.46:0/1420113993 >> 172.21.15.46:0/1420113993 conn(0x55ebe68f9200 msgr2=0x55ebe68fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb11be00640 0 client.28304 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 1 -- 172.21.15.46:0/1420113993 shutdown_connections 2024-09-10T07:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 1 -- 172.21.15.46:0/1420113993 wait complete. 2024-09-10T07:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.208+0000 7fb14384d740 20 client.28304 trim_cache size 0 max 16384 2024-09-10T07:48:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.239+0000 7fb14384d740 1 Processor -- start 2024-09-10T07:48:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.240+0000 7fb14384d740 1 -- start start 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.240+0000 7fb14384d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fcc70 0x55ebe68ff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.240+0000 7fb14384d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe68f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.240+0000 7fb14384d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe691fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.240+0000 7fb14384d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1b51f0 con 0x55ebe68fcc70 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.240+0000 7fb14384d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1b6d30 con 0x55ebe6907e50 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.240+0000 7fb14384d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1b58c0 con 0x55ebe68fd590 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.241+0000 7fb13a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe691fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.241+0000 7fb13a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe691fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42622/0 (socket says 172.21.15.46:42622) 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.241+0000 7fb13a800640 1 -- 172.21.15.46:0/3556159161 learned_addr learned my addr 172.21.15.46:0/3556159161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.241+0000 7fb139e00640 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fcc70 0x55ebe68ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.242+0000 7fb139400640 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe68f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.244+0000 7fb139400640 1 -- 172.21.15.46:0/3556159161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 msgr2=0x55ebe691fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.244+0000 7fb139400640 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe691fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.244+0000 7fb139400640 1 -- 172.21.15.46:0/3556159161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fcc70 msgr2=0x55ebe68ff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.244+0000 7fb139400640 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fcc70 0x55ebe68ff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.244+0000 7fb139400640 1 -- 172.21.15.46:0/3556159161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebe68fd3c0 con 0x55ebe68fd590 2024-09-10T07:48:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.245+0000 7fb13a800640 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe691fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.245+0000 7fb139400640 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe68f8270 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb1280ccdf0 tx=0x7fb1280d1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=416d6bc567c83fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.246+0000 7fb126a00640 1 -- 172.21.15.46:0/3556159161 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12809bb00 con 0x55ebe68fd590 2024-09-10T07:48:29.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.248+0000 7fb126a00640 1 -- 172.21.15.46:0/3556159161 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12809feb0 con 0x55ebe68fd590 2024-09-10T07:48:29.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.248+0000 7fb126a00640 1 -- 172.21.15.46:0/3556159161 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128098170 con 0x55ebe68fd590 2024-09-10T07:48:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.249+0000 7fb14384d740 1 -- 172.21.15.46:0/3556159161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 msgr2=0x55ebe68f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.249+0000 7fb14384d740 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe68f8270 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb1280ccdf0 tx=0x7fb1280d1040 comp rx=0 tx=0).stop 2024-09-10T07:48:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.252+0000 7fb14384d740 1 -- 172.21.15.46:0/3556159161 shutdown_connections 2024-09-10T07:48:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.252+0000 7fb14384d740 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe691fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.252+0000 7fb14384d740 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fd590 0x55ebe68f8270 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.252+0000 7fb14384d740 1 --2- 172.21.15.46:0/3556159161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fcc70 0x55ebe68ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.252+0000 7fb14384d740 1 -- 172.21.15.46:0/3556159161 >> 172.21.15.46:0/3556159161 conn(0x55ebe68f9200 msgr2=0x55ebe68f7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.252+0000 7fb14384d740 1 -- 172.21.15.46:0/3556159161 shutdown_connections 2024-09-10T07:48:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.252+0000 7fb14384d740 1 -- 172.21.15.46:0/3556159161 wait complete. 2024-09-10T07:48:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.253+0000 7fb14384d740 1 -- start start 2024-09-10T07:48:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.253+0000 7fb14384d740 1 Processor -- start 2024-09-10T07:48:29.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.256+0000 7fb14384d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe69102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.256+0000 7fb14384d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 0x55ebe690b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.256+0000 7fb14384d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.257+0000 7fb14384d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1b6660 con 0x55ebe68fd590 2024-09-10T07:48:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.257+0000 7fb14384d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1b5f90 con 0x55ebe6907e50 2024-09-10T07:48:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.258+0000 7fb139e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe69102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.258+0000 7fb139e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe69102d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34458/0 (socket says 172.21.15.46:34458) 2024-09-10T07:48:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.258+0000 7fb14384d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1b31d0 con 0x55ebe68fcc70 2024-09-10T07:48:29.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.258+0000 7fb14384d740 10 client.0 _ll_register_callbacks cb 0x7fb12c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.258+0000 7fb13a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.258+0000 7fb13a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42628/0 (socket says 172.21.15.46:42628) 2024-09-10T07:48:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.258+0000 7fb13a800640 1 -- 172.21.15.46:0/2737132941 learned_addr learned my addr 172.21.15.46:0/2737132941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.259+0000 7fb139400640 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 0x55ebe690b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.261+0000 7fb13a800640 1 -- 172.21.15.46:0/2737132941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 msgr2=0x55ebe69102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.261+0000 7fb13a800640 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe69102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.261+0000 7fb13a800640 1 -- 172.21.15.46:0/2737132941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 msgr2=0x55ebe690b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.261+0000 7fb13a800640 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 0x55ebe690b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.261+0000 7fb13a800640 1 -- 172.21.15.46:0/2737132941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebe690c080 con 0x55ebe6907e50 2024-09-10T07:48:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.261+0000 7fb11be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.262+0000 7fb13a800640 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690bb40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb134009f70 tx=0x7fb134004f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb04124dc49125d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.262+0000 7fb11be00640 1 -- 172.21.15.46:0/2737132941 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13404b030 con 0x55ebe6907e50 2024-09-10T07:48:29.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.262+0000 7fb14384d740 1 -- 172.21.15.46:0/2737132941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12809feb0 con 0x55ebe6907e50 2024-09-10T07:48:29.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.262+0000 7fb11be00640 1 -- 172.21.15.46:0/2737132941 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb13404c070 con 0x55ebe6907e50 2024-09-10T07:48:29.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.263+0000 7fb11be00640 1 -- 172.21.15.46:0/2737132941 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb134058050 con 0x55ebe6907e50 2024-09-10T07:48:29.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.263+0000 7fb14384d740 10 client.28310 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:29.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.263+0000 7fb14384d740 1 -- 172.21.15.46:0/2737132941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ebe69828a0 con 0x55ebe6907e50 2024-09-10T07:48:29.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.264+0000 7fb11be00640 1 -- 172.21.15.46:0/2737132941 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12809feb0 con 0x55ebe6907e50 2024-09-10T07:48:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.264+0000 7fb11be00640 1 -- 172.21.15.46:0/2737132941 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fb134005640 con 0x55ebe6907e50 2024-09-10T07:48:29.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.264+0000 7fb14384d740 10 client.28310 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.264+0000 7fb14384d740 10 client.28310 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:48:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.264+0000 7fb14384d740 20 client.28310 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.264+0000 7fb14384d740 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55ebe6982c40 0x55ebe6985030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.264+0000 7fb14384d740 4 client.28310 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:48:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.264+0000 7fb14384d740 1 -- 172.21.15.46:0/2737132941 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebe686ae60 con 0x55ebe6982c40 2024-09-10T07:48:29.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.265+0000 7fb139e00640 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55ebe6982c40 0x55ebe6985030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.267+0000 7fb139e00640 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55ebe6982c40 0x55ebe6985030 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a312c139bc7116e server_cookie=42a5e2e5a71051cd in_seq=0 out_seq=0 2024-09-10T07:48:29.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.267+0000 7fb11be00640 10 client.28310 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:48:29.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb11be00640 1 -- 172.21.15.46:0/2737132941 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebe686ae60 con 0x55ebe6982c40 2024-09-10T07:48:29.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb11be00640 10 client.28310 handle_command_reply: tid=0 2024-09-10T07:48:29.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 client.28310 shutdown 2024-09-10T07:48:29.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 -- 172.21.15.46:0/2737132941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 msgr2=0x55ebe690bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690bb40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb134009f70 tx=0x7fb134004f80 comp rx=0 tx=0).stop 2024-09-10T07:48:29.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 -- 172.21.15.46:0/2737132941 shutdown_connections 2024-09-10T07:48:29.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55ebe6982c40 0x55ebe6985030 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ebe6907e50 0x55ebe690bb40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebe68fd590 0x55ebe690b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 --2- 172.21.15.46:0/2737132941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ebe68fcc70 0x55ebe69102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.269+0000 7fb14384d740 1 -- 172.21.15.46:0/2737132941 >> 172.21.15.46:0/2737132941 conn(0x55ebe68f9200 msgr2=0x55ebe69054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.273+0000 7fb11be00640 0 client.28310 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:48:29.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.273+0000 7fb14384d740 1 -- 172.21.15.46:0/2737132941 shutdown_connections 2024-09-10T07:48:29.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.273+0000 7fb14384d740 1 -- 172.21.15.46:0/2737132941 wait complete. 2024-09-10T07:48:29.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.275+0000 7fb14384d740 20 client.28310 trim_cache size 0 max 16384 2024-09-10T07:48:29.276 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:48:29.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 -- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1200a2c00 msgr2=0x7fb1200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1200a2c00 0x7fb1200a50c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb12c000c00 tx=0x7fb12c005b50 comp rx=0 tx=0).stop 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 -- 172.21.15.46:0/2683411301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c1049a0 msgr2=0x7fb13c076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c1049a0 0x7fb13c076c40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb12800ea20 tx=0x7fb12800eee0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 -- 172.21.15.46:0/2683411301 shutdown_connections 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb13c109120 0x7fb13c077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c1049a0 0x7fb13c076c40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb1200a2c00 0x7fb1200a50c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 --2- 172.21.15.46:0/2683411301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c103ff0 0x7fb13c076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.276+0000 7fb140e00640 1 -- 172.21.15.46:0/2683411301 >> 172.21.15.46:0/2683411301 conn(0x7fb13c0ffa90 msgr2=0x7fb13c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.277+0000 7fb140e00640 1 -- 172.21.15.46:0/2683411301 shutdown_connections 2024-09-10T07:48:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:29.277+0000 7fb140e00640 1 -- 172.21.15.46:0/2683411301 wait complete. 2024-09-10T07:48:30.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:48:30.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:30 smithi046 ceph-mon[30705]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 MiB/s wr, 563 op/s 2024-09-10T07:48:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 670.0 s, 499.4 tps, lat 202.310 ms stddev 233.452 2024-09-10T07:48:30.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:30 smithi181 ceph-mon[35397]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 MiB/s wr, 563 op/s 2024-09-10T07:48:30.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:30 smithi063 ceph-mon[35020]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 MiB/s wr, 563 op/s 2024-09-10T07:48:31.475 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:32.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:32 smithi181 ceph-mon[35397]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 626 op/s 2024-09-10T07:48:32.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:32 smithi063 ceph-mon[35020]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 626 op/s 2024-09-10T07:48:32.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:32 smithi046 ceph-mon[30705]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 626 op/s 2024-09-10T07:48:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:34 smithi046 ceph-mon[30705]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.6 MiB/s wr, 626 op/s 2024-09-10T07:48:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:34 smithi181 ceph-mon[35397]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.6 MiB/s wr, 626 op/s 2024-09-10T07:48:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:34 smithi063 ceph-mon[35020]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.6 MiB/s wr, 626 op/s 2024-09-10T07:48:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.984+0000 7eff22800640 1 -- 172.21.15.46:0/3252745415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 msgr2=0x7eff1c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.984+0000 7eff22800640 1 --2- 172.21.15.46:0/3252745415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 0x7eff1c1043f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009900 tx=0x7eff0c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.985+0000 7eff22800640 1 -- 172.21.15.46:0/3252745415 shutdown_connections 2024-09-10T07:48:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.985+0000 7eff22800640 1 --2- 172.21.15.46:0/3252745415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff1c109140 0x7eff1c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.985+0000 7eff22800640 1 --2- 172.21.15.46:0/3252745415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff1c1049c0 0x7eff1c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.985+0000 7eff22800640 1 --2- 172.21.15.46:0/3252745415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 0x7eff1c1043f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.985+0000 7eff22800640 1 -- 172.21.15.46:0/3252745415 >> 172.21.15.46:0/3252745415 conn(0x7eff1c0ffa50 msgr2=0x7eff1c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.985+0000 7eff22800640 1 -- 172.21.15.46:0/3252745415 shutdown_connections 2024-09-10T07:48:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.985+0000 7eff22800640 1 -- 172.21.15.46:0/3252745415 wait complete. 2024-09-10T07:48:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.986+0000 7eff22800640 1 Processor -- start 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.986+0000 7eff22800640 1 -- start start 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.987+0000 7eff22800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 0x7eff1c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.987+0000 7eff22800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff1c1049c0 0x7eff1c076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.987+0000 7eff22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff1c109140 0x7eff1c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.987+0000 7eff22800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c114210 con 0x7eff1c109140 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.987+0000 7eff22800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c114090 con 0x7eff1c104010 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.987+0000 7eff22800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c114390 con 0x7eff1c1049c0 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.987+0000 7eff1aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff1c1049c0 0x7eff1c076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff1c1049c0 0x7eff1c076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34488/0 (socket says 172.21.15.46:34488) 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1aa00640 1 -- 172.21.15.46:0/335755180 learned_addr learned my addr 172.21.15.46:0/335755180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1be00640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff1c109140 0x7eff1c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1b400640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 0x7eff1c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1aa00640 1 -- 172.21.15.46:0/335755180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 msgr2=0x7eff1c076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1aa00640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 0x7eff1c076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1aa00640 1 -- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff1c109140 msgr2=0x7eff1c077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1aa00640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff1c109140 0x7eff1c077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.988+0000 7eff1aa00640 1 -- 172.21.15.46:0/335755180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c19a490 con 0x7eff1c1049c0 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.989+0000 7eff1b400640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 0x7eff1c076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.989+0000 7eff1be00640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff1c109140 0x7eff1c077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.989+0000 7eff1aa00640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff1c1049c0 0x7eff1c076c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7eff0400b4d0 tx=0x7eff0400b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.989+0000 7eff13e00640 1 -- 172.21.15.46:0/335755180 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04014070 con 0x7eff1c1049c0 2024-09-10T07:48:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.989+0000 7eff13e00640 1 -- 172.21.15.46:0/335755180 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff04010d50 con 0x7eff1c1049c0 2024-09-10T07:48:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.989+0000 7eff13e00640 1 -- 172.21.15.46:0/335755180 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04017360 con 0x7eff1c1049c0 2024-09-10T07:48:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.991+0000 7eff22800640 1 -- 172.21.15.46:0/335755180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c19a780 con 0x7eff1c1049c0 2024-09-10T07:48:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.991+0000 7eff22800640 1 -- 172.21.15.46:0/335755180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c19ac70 con 0x7eff1c1049c0 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.996+0000 7eff252bb740 1 Processor -- start 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.996+0000 7eff252bb740 1 -- start start 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.996+0000 7eff252bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff1be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff1be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42646/0 (socket says 172.21.15.46:42646) 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff252bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff1be00640 1 -- 172.21.15.46:0/2016444272 learned_addr learned my addr 172.21.15.46:0/2016444272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff1b400640 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff252bb740 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff252bb740 1 -- 172.21.15.46:0/2016444272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55768b9123e0 con 0x55768b9b6e50 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff252bb740 1 -- 172.21.15.46:0/2016444272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55768b8b9d80 con 0x55768b9abc70 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.997+0000 7eff252bb740 1 -- 172.21.15.46:0/2016444272 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55768b90e3a0 con 0x55768b9ac590 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.998+0000 7eff1b400640 1 -- 172.21.15.46:0/2016444272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 msgr2=0x55768b9ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.998+0000 7eff1b400640 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.998+0000 7eff1b400640 1 -- 172.21.15.46:0/2016444272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 msgr2=0x55768b9b9240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.998+0000 7eff1b400640 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9b9240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.998+0000 7eff1b400640 1 -- 172.21.15.46:0/2016444272 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55768b8d8ae0 con 0x55768b9ac590 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.999+0000 7eff1b400640 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b6910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7eff0c02a910 tx=0x7eff0c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64196f07f32cdf3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:34.999+0000 7eff13e00640 1 -- 172.21.15.46:0/335755180 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff04017070 con 0x7eff1c1049c0 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.000+0000 7eff10c00640 1 -- 172.21.15.46:0/2016444272 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c03f070 con 0x55768b9ac590 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.000+0000 7eff10c00640 1 -- 172.21.15.46:0/2016444272 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0c005520 con 0x55768b9ac590 2024-09-10T07:48:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.000+0000 7eff10c00640 1 -- 172.21.15.46:0/2016444272 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c035030 con 0x55768b9ac590 2024-09-10T07:48:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.001+0000 7eff252bb740 1 -- 172.21.15.46:0/2016444272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 msgr2=0x55768b9b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.001+0000 7eff252bb740 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b6910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7eff0c02a910 tx=0x7eff0c0040c0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.001+0000 7eff252bb740 1 -- 172.21.15.46:0/2016444272 shutdown_connections 2024-09-10T07:48:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.001+0000 7eff252bb740 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.001+0000 7eff252bb740 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b6910 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.001+0000 7eff252bb740 1 --2- 172.21.15.46:0/2016444272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.001+0000 7eff252bb740 1 -- 172.21.15.46:0/2016444272 >> 172.21.15.46:0/2016444272 conn(0x55768b9a8200 msgr2=0x55768b9a85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.003+0000 7eff13e00640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff000a2c80 0x7eff000a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.003+0000 7eff252bb740 1 -- 172.21.15.46:0/2016444272 shutdown_connections 2024-09-10T07:48:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.003+0000 7eff13e00640 1 -- 172.21.15.46:0/335755180 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7eff040ce5b0 con 0x7eff1c1049c0 2024-09-10T07:48:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.003+0000 7eff252bb740 1 -- 172.21.15.46:0/2016444272 wait complete. 2024-09-10T07:48:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.004+0000 7eff252bb740 1 -- start start 2024-09-10T07:48:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.004+0000 7eff252bb740 1 Processor -- start 2024-09-10T07:48:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff1b400640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff000a2c80 0x7eff000a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff252bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff252bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff252bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff252bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55768b9b9a20 con 0x55768b9b6e50 2024-09-10T07:48:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff252bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55768b8cd0f0 con 0x55768b9abc70 2024-09-10T07:48:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff252bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55768b9b9ba0 con 0x55768b9ac590 2024-09-10T07:48:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff252bb740 10 client.0 _ll_register_callbacks cb 0x55768b8b4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.005+0000 7eff1aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.006+0000 7eff1aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34514/0 (socket says 172.21.15.46:34514) 2024-09-10T07:48:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.006+0000 7eff1aa00640 1 -- 172.21.15.46:0/546351436 learned_addr learned my addr 172.21.15.46:0/546351436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.006+0000 7eff1be00640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.007+0000 7eff1aa00640 1 -- 172.21.15.46:0/546351436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 msgr2=0x55768b9b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.008+0000 7eff1b400640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.008+0000 7eff1aa00640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.008+0000 7eff1aa00640 1 -- 172.21.15.46:0/546351436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 msgr2=0x55768b9b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.008+0000 7eff1aa00640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.008+0000 7eff1aa00640 1 -- 172.21.15.46:0/546351436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768b6833c0 con 0x55768b9ac590 2024-09-10T07:48:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.008+0000 7eff1be00640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.008+0000 7eff1b400640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.012+0000 7eff1aa00640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b4370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff04097400 tx=0x7eff0409a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5eb1897e3b6b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.012+0000 7eff09600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.012+0000 7eff1b400640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff000a2c80 0x7eff000a5140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff0c031550 tx=0x7eff0c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.012+0000 7eff09600640 1 -- 172.21.15.46:0/546351436 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0401e050 con 0x55768b9ac590 2024-09-10T07:48:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.012+0000 7eff09600640 1 -- 172.21.15.46:0/546351436 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7eff0400f530 con 0x55768b9ac590 2024-09-10T07:48:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.025+0000 7eff252bb740 1 -- 172.21.15.46:0/546351436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff0c005520 con 0x55768b9ac590 2024-09-10T07:48:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.027+0000 7eff09600640 1 -- 172.21.15.46:0/546351436 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7eff0409aa30 con 0x55768b9ac590 2024-09-10T07:48:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.027+0000 7eff252bb740 10 client.28267 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:35.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.027+0000 7eff252bb740 1 -- 172.21.15.46:0/546351436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55768b9c5be0 con 0x55768b9ac590 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.028+0000 7eff09600640 1 -- 172.21.15.46:0/546351436 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0c005520 con 0x55768b9ac590 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.028+0000 7eff09600640 1 -- 172.21.15.46:0/546351436 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7eff04099e80 con 0x55768b9ac590 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.028+0000 7eff252bb740 10 client.28267 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.028+0000 7eff252bb740 10 client.28267 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.028+0000 7eff252bb740 20 client.28267 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.028+0000 7eff252bb740 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55768b9c70a0 0x55768b9c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.028+0000 7eff252bb740 4 client.28267 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.028+0000 7eff252bb740 1 -- 172.21.15.46:0/546351436 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55768b91f950 con 0x55768b9c70a0 2024-09-10T07:48:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.029+0000 7eff1b400640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55768b9c70a0 0x55768b9c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.034+0000 7eff1b400640 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55768b9c70a0 0x55768b9c9490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4e64b72c33102c server_cookie=ee63f75a107c966 in_seq=0 out_seq=0 2024-09-10T07:48:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.034+0000 7eff09600640 10 client.28267 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:48:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.035+0000 7eff09600640 1 -- 172.21.15.46:0/546351436 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55768b91f950 con 0x55768b9c70a0 2024-09-10T07:48:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.035+0000 7eff09600640 10 client.28267 handle_command_reply: tid=0 2024-09-10T07:48:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.036+0000 7eff252bb740 1 client.28267 shutdown 2024-09-10T07:48:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.036+0000 7eff252bb740 1 -- 172.21.15.46:0/546351436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 msgr2=0x55768b9b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.036+0000 7eff252bb740 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b4370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff04097400 tx=0x7eff0409a630 comp rx=0 tx=0).stop 2024-09-10T07:48:35.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.038+0000 7eff252bb740 1 -- 172.21.15.46:0/546351436 shutdown_connections 2024-09-10T07:48:35.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.038+0000 7eff252bb740 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.038+0000 7eff252bb740 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55768b9c70a0 0x55768b9c9490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.038+0000 7eff252bb740 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9b4370 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.038+0000 7eff252bb740 1 --2- 172.21.15.46:0/546351436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9abc70 0x55768b9b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.038+0000 7eff252bb740 1 -- 172.21.15.46:0/546351436 >> 172.21.15.46:0/546351436 conn(0x55768b9a8200 msgr2=0x55768b9adee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:35.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.038+0000 7eff09600640 0 client.28267 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:48:35.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.038+0000 7eff252bb740 1 -- 172.21.15.46:0/546351436 shutdown_connections 2024-09-10T07:48:35.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.039+0000 7eff252bb740 1 -- 172.21.15.46:0/546351436 wait complete. 2024-09-10T07:48:35.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.040+0000 7eff252bb740 20 client.28267 trim_cache size 0 max 16384 2024-09-10T07:48:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.075+0000 7eff252bb740 1 Processor -- start 2024-09-10T07:48:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.075+0000 7eff252bb740 1 -- start start 2024-09-10T07:48:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.075+0000 7eff252bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9abc70 0x55768b9b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.075+0000 7eff252bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.075+0000 7eff252bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9b6e50 0x55768b9cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.075+0000 7eff252bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1acb30 con 0x55768b9abc70 2024-09-10T07:48:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.075+0000 7eff252bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1ae670 con 0x55768b9b6e50 2024-09-10T07:48:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.075+0000 7eff252bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1ad200 con 0x55768b9ac590 2024-09-10T07:48:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.076+0000 7eff1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9abc70 0x55768b9b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.076+0000 7eff1be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9b6e50 0x55768b9cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.076+0000 7eff1be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9b6e50 0x55768b9cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42668/0 (socket says 172.21.15.46:42668) 2024-09-10T07:48:35.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.076+0000 7eff1be00640 1 -- 172.21.15.46:0/2163868716 learned_addr learned my addr 172.21.15.46:0/2163868716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.077+0000 7eff1be00640 1 -- 172.21.15.46:0/2163868716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 msgr2=0x55768b9ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.077+0000 7eff1be00640 1 --2- 172.21.15.46:0/2163868716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.077+0000 7eff1be00640 1 -- 172.21.15.46:0/2163868716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9abc70 msgr2=0x55768b9b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.077+0000 7eff1be00640 1 --2- 172.21.15.46:0/2163868716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9abc70 0x55768b9b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.077+0000 7eff1be00640 1 -- 172.21.15.46:0/2163868716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55768b9a3770 con 0x55768b9b6e50 2024-09-10T07:48:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.078+0000 7eff1be00640 1 --2- 172.21.15.46:0/2163868716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9b6e50 0x55768b9cf2a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7eff14033870 tx=0x7eff14033d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c85c98ed4d1214e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.079+0000 7eff0be00640 1 -- 172.21.15.46:0/2163868716 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14004b20 con 0x55768b9b6e50 2024-09-10T07:48:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.079+0000 7eff0be00640 1 -- 172.21.15.46:0/2163868716 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff14004cc0 con 0x55768b9b6e50 2024-09-10T07:48:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.079+0000 7eff0be00640 1 -- 172.21.15.46:0/2163868716 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14005910 con 0x55768b9b6e50 2024-09-10T07:48:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.080+0000 7eff252bb740 1 -- 172.21.15.46:0/2163868716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9b6e50 msgr2=0x55768b9cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.080+0000 7eff252bb740 1 --2- 172.21.15.46:0/2163868716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9b6e50 0x55768b9cf2a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7eff14033870 tx=0x7eff14033d30 comp rx=0 tx=0).stop 2024-09-10T07:48:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.080+0000 7eff252bb740 1 -- 172.21.15.46:0/2163868716 shutdown_connections 2024-09-10T07:48:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.080+0000 7eff252bb740 1 --2- 172.21.15.46:0/2163868716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9b6e50 0x55768b9cf2a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.080+0000 7eff252bb740 1 --2- 172.21.15.46:0/2163868716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9ac590 0x55768b9ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.080+0000 7eff252bb740 1 --2- 172.21.15.46:0/2163868716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9abc70 0x55768b9b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.080+0000 7eff252bb740 1 -- 172.21.15.46:0/2163868716 >> 172.21.15.46:0/2163868716 conn(0x55768b9a8200 msgr2=0x55768b9b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.080+0000 7eff252bb740 1 -- 172.21.15.46:0/2163868716 shutdown_connections 2024-09-10T07:48:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.081+0000 7eff252bb740 1 -- 172.21.15.46:0/2163868716 wait complete. 2024-09-10T07:48:35.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.081+0000 7eff252bb740 1 -- start start 2024-09-10T07:48:35.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.081+0000 7eff252bb740 1 Processor -- start 2024-09-10T07:48:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.083+0000 7eff252bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9abc70 0x55768b9baae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.083+0000 7eff252bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9ac590 0x55768b9bb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.084+0000 7eff252bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9bb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.084+0000 7eff1b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9abc70 0x55768b9baae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.084+0000 7eff1aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9ac590 0x55768b9bb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.085+0000 7eff1aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9ac590 0x55768b9bb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42672/0 (socket says 172.21.15.46:42672) 2024-09-10T07:48:35.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.086+0000 7eff1b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9abc70 0x55768b9baae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34532/0 (socket says 172.21.15.46:34532) 2024-09-10T07:48:35.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.086+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1adfa0 con 0x55768b9b6e50 2024-09-10T07:48:35.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.086+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1ad8d0 con 0x55768b9ac590 2024-09-10T07:48:35.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.086+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1a2a30 con 0x55768b9abc70 2024-09-10T07:48:35.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.086+0000 7eff1aa00640 1 -- 172.21.15.46:0/183605907 learned_addr learned my addr 172.21.15.46:0/183605907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:35.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.087+0000 7eff252bb740 10 client.0 _ll_register_callbacks cb 0x7eff0400f440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:35.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.088+0000 7eff1be00640 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.089+0000 7eff1aa00640 1 -- 172.21.15.46:0/183605907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9abc70 msgr2=0x55768b9baae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:35.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.089+0000 7eff1aa00640 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9abc70 0x55768b9baae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.089+0000 7eff1aa00640 1 -- 172.21.15.46:0/183605907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 msgr2=0x55768b9bb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:35.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.089+0000 7eff1aa00640 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9bb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.089+0000 7eff1aa00640 1 -- 172.21.15.46:0/183605907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768b9a8e90 con 0x55768b9ac590 2024-09-10T07:48:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.091+0000 7eff1aa00640 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9ac590 0x55768b9bb020 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7eff040973b0 tx=0x7eff040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0419b8265af467 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.091+0000 7eff09600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.092+0000 7eff09600640 1 -- 172.21.15.46:0/183605907 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0401e070 con 0x55768b9ac590 2024-09-10T07:48:35.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.092+0000 7eff09600640 1 -- 172.21.15.46:0/183605907 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7eff04010870 con 0x55768b9ac590 2024-09-10T07:48:35.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.093+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff14004cc0 con 0x55768b9ac590 2024-09-10T07:48:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.093+0000 7eff09600640 1 -- 172.21.15.46:0/183605907 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7eff0409a800 con 0x55768b9ac590 2024-09-10T07:48:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.094+0000 7eff252bb740 10 client.28322 fetch_fsmap learned FSMap version 50 2024-09-10T07:48:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.094+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55768ba31da0 con 0x55768b9ac590 2024-09-10T07:48:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.095+0000 7eff09600640 1 -- 172.21.15.46:0/183605907 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff14004cc0 con 0x55768b9ac590 2024-09-10T07:48:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.096+0000 7eff09600640 1 -- 172.21.15.46:0/183605907 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7eff040998f0 con 0x55768b9ac590 2024-09-10T07:48:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.096+0000 7eff252bb740 10 client.28322 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T07:48:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.097+0000 7eff252bb740 10 client.28322 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:48:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.097+0000 7eff252bb740 20 client.28322 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.097+0000 7eff252bb740 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55768ba323c0 0x55768ba347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.097+0000 7eff252bb740 4 client.28322 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:48:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.097+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55768b919e60 con 0x55768ba323c0 2024-09-10T07:48:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.099+0000 7eff1b400640 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55768ba323c0 0x55768ba347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.099+0000 7eff1b400640 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55768ba323c0 0x55768ba347e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2655efa60e79e29b server_cookie=bc6b23e64a5a2297 in_seq=0 out_seq=0 2024-09-10T07:48:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.100+0000 7eff09600640 10 client.28322 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:48:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.100+0000 7eff09600640 1 -- 172.21.15.46:0/183605907 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55768b919e60 con 0x55768ba323c0 2024-09-10T07:48:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.100+0000 7eff09600640 10 client.28322 handle_command_reply: tid=0 2024-09-10T07:48:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.100+0000 7eff252bb740 1 client.28322 shutdown 2024-09-10T07:48:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.101+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9ac590 msgr2=0x55768b9bb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.101+0000 7eff252bb740 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9ac590 0x55768b9bb020 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7eff040973b0 tx=0x7eff040046c0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 shutdown_connections 2024-09-10T07:48:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff252bb740 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55768b9b6e50 0x55768b9bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff252bb740 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55768b9ac590 0x55768b9bb020 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff252bb740 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55768ba323c0 0x55768ba347e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff252bb740 1 --2- 172.21.15.46:0/183605907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55768b9abc70 0x55768b9baae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 >> 172.21.15.46:0/183605907 conn(0x55768b9a8200 msgr2=0x55768b9a6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff09600640 0 client.28322 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:48:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 shutdown_connections 2024-09-10T07:48:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.102+0000 7eff252bb740 1 -- 172.21.15.46:0/183605907 wait complete. 2024-09-10T07:48:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.103+0000 7eff252bb740 20 client.28322 trim_cache size 0 max 16384 2024-09-10T07:48:35.103 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:48:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 -- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff000a2c80 msgr2=0x7eff000a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff000a2c80 0x7eff000a5140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff0c031550 tx=0x7eff0c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 -- 172.21.15.46:0/335755180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff1c1049c0 msgr2=0x7eff1c076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff1c1049c0 0x7eff1c076c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7eff0400b4d0 tx=0x7eff0400b990 comp rx=0 tx=0).stop 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 -- 172.21.15.46:0/335755180 shutdown_connections 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7eff000a2c80 0x7eff000a5140 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff1c109140 0x7eff1c077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff1c1049c0 0x7eff1c076c40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 --2- 172.21.15.46:0/335755180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c104010 0x7eff1c076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.104+0000 7eff22800640 1 -- 172.21.15.46:0/335755180 >> 172.21.15.46:0/335755180 conn(0x7eff1c0ffa50 msgr2=0x7eff1c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.105+0000 7eff22800640 1 -- 172.21.15.46:0/335755180 shutdown_connections 2024-09-10T07:48:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:35.105+0000 7eff22800640 1 -- 172.21.15.46:0/335755180 wait complete. 2024-09-10T07:48:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 675.0 s, 477.4 tps, lat 203.419 ms stddev 286.480 2024-09-10T07:48:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:36 smithi181 ceph-mon[35397]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 MiB/s wr, 488 op/s 2024-09-10T07:48:36.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:36 smithi063 ceph-mon[35020]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 MiB/s wr, 488 op/s 2024-09-10T07:48:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:36 smithi046 ceph-mon[30705]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 MiB/s wr, 488 op/s 2024-09-10T07:48:36.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:48:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:38 smithi181 ceph-mon[35397]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.3 MiB/s wr, 688 op/s 2024-09-10T07:48:38.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:38 smithi063 ceph-mon[35020]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.3 MiB/s wr, 688 op/s 2024-09-10T07:48:38.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:38 smithi046 ceph-mon[30705]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.3 MiB/s wr, 688 op/s 2024-09-10T07:48:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.681+0000 7f9379200640 1 -- 172.21.15.46:0/1191591491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9374100980 msgr2=0x7f9374100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.681+0000 7f9379200640 1 --2- 172.21.15.46:0/1191591491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9374100980 0x7f9374100d60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9364009900 tx=0x7f936402a480 comp rx=0 tx=0).stop 2024-09-10T07:48:39.182 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:48:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.684+0000 7f9379200640 1 -- 172.21.15.46:0/1191591491 shutdown_connections 2024-09-10T07:48:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.684+0000 7f9379200640 1 --2- 172.21.15.46:0/1191591491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9374105d90 0x7f9374106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.684+0000 7f9379200640 1 --2- 172.21.15.46:0/1191591491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93741012a0 0x7f9374105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.684+0000 7f9379200640 1 --2- 172.21.15.46:0/1191591491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9374100980 0x7f9374100d60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.684+0000 7f9379200640 1 -- 172.21.15.46:0/1191591491 >> 172.21.15.46:0/1191591491 conn(0x7f93740fc610 msgr2=0x7f93740fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.686+0000 7f9379200640 1 -- 172.21.15.46:0/1191591491 shutdown_connections 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.686+0000 7f9379200640 1 -- 172.21.15.46:0/1191591491 wait complete. 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.687+0000 7f9379200640 1 Processor -- start 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.688+0000 7f9379200640 1 -- start start 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.690+0000 7f9379200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9374100980 0x7f9374073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.691+0000 7f9379200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93741012a0 0x7f937406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.691+0000 7f9379200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9374105d90 0x7f937406cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.691+0000 7f9379200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9374118400 con 0x7f9374105d90 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.692+0000 7f9372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93741012a0 0x7f937406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.692+0000 7f9372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93741012a0 0x7f937406caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34394/0 (socket says 172.21.15.46:34394) 2024-09-10T07:48:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.692+0000 7f9372a00640 1 -- 172.21.15.46:0/1559738247 learned_addr learned my addr 172.21.15.46:0/1559738247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.692+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9374118280 con 0x7f93741012a0 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.692+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9374118580 con 0x7f9374100980 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.693+0000 7f9373e00640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9374105d90 0x7f937406cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.694+0000 7f9372a00640 1 -- 172.21.15.46:0/1559738247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9374100980 msgr2=0x7f9374073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.694+0000 7f9373400640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9374100980 0x7f9374073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.694+0000 7f9372a00640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9374100980 0x7f9374073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.694+0000 7f9372a00640 1 -- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9374105d90 msgr2=0x7f937406cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.694+0000 7f9372a00640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9374105d90 0x7f937406cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.694+0000 7f9372a00640 1 -- 172.21.15.46:0/1559738247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937406d5d0 con 0x7f93741012a0 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.695+0000 7f9373400640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9374100980 0x7f9374073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.696+0000 7f9372a00640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93741012a0 0x7f937406caa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f936000b3f0 tx=0x7f936000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.696+0000 7f936be00640 1 -- 172.21.15.46:0/1559738247 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360015020 con 0x7f93741012a0 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.697+0000 7f936be00640 1 -- 172.21.15.46:0/1559738247 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9360011d10 con 0x7f93741012a0 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.697+0000 7f936be00640 1 -- 172.21.15.46:0/1559738247 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936000f2e0 con 0x7f93741012a0 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.697+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937406d8c0 con 0x7f93741012a0 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.697+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9374071880 con 0x7f93741012a0 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.704+0000 7f937d27f740 1 Processor -- start 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.705+0000 7f937d27f740 1 -- start start 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.706+0000 7f936be00640 1 -- 172.21.15.46:0/1559738247 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9360018020 con 0x7f93741012a0 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.708+0000 7f936be00640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93580a2cc0 0x7f93580a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.708+0000 7f936be00640 1 -- 172.21.15.46:0/1559738247 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f93600cebd0 con 0x7f93741012a0 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.710+0000 7f937d27f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a96220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.710+0000 7f937d27f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31aa0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.710+0000 7f937d27f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31aa3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.710+0000 7f937d27f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557a319fc3e0 con 0x557a31aa1020 2024-09-10T07:48:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.711+0000 7f937d27f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557a319a3d80 con 0x557a31a96760 2024-09-10T07:48:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.712+0000 7f937d27f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557a319f83a0 con 0x557a31a95e40 2024-09-10T07:48:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.712+0000 7f9373400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a96220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.712+0000 7f9373400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a96220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53202/0 (socket says 172.21.15.46:53202) 2024-09-10T07:48:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.712+0000 7f9373400640 1 -- 172.21.15.46:0/1009155563 learned_addr learned my addr 172.21.15.46:0/1009155563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.713+0000 7f9373e00640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93580a2cc0 0x7f93580a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.714+0000 7f9373400640 1 -- 172.21.15.46:0/1009155563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 msgr2=0x557a31aa0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.714+0000 7f9373400640 1 --2- 172.21.15.46:0/1009155563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31aa0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.714+0000 7f9373400640 1 -- 172.21.15.46:0/1009155563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 msgr2=0x557a31aa3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.714+0000 7f9373400640 1 --2- 172.21.15.46:0/1009155563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31aa3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.715+0000 7f9373400640 1 -- 172.21.15.46:0/1009155563 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a319c2ae0 con 0x557a31a95e40 2024-09-10T07:48:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.715+0000 7f9373400640 1 --2- 172.21.15.46:0/1009155563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a96220 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9364031510 tx=0x7f9364004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a1dad383c395ede server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.716+0000 7f9368c00640 1 -- 172.21.15.46:0/1009155563 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936403f070 con 0x557a31a95e40 2024-09-10T07:48:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.716+0000 7f9368c00640 1 -- 172.21.15.46:0/1009155563 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9364005520 con 0x557a31a95e40 2024-09-10T07:48:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.716+0000 7f9368c00640 1 -- 172.21.15.46:0/1009155563 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364035030 con 0x557a31a95e40 2024-09-10T07:48:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.717+0000 7f937d27f740 1 -- 172.21.15.46:0/1009155563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 msgr2=0x557a31a96220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.717+0000 7f937d27f740 1 --2- 172.21.15.46:0/1009155563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a96220 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9364031510 tx=0x7f9364004040 comp rx=0 tx=0).stop 2024-09-10T07:48:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.717+0000 7f9373e00640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93580a2cc0 0x7f93580a5180 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f936c005fd0 tx=0x7f936c009bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.717+0000 7f937d27f740 1 -- 172.21.15.46:0/1009155563 shutdown_connections 2024-09-10T07:48:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.717+0000 7f937d27f740 1 --2- 172.21.15.46:0/1009155563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31aa3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.717+0000 7f937d27f740 1 --2- 172.21.15.46:0/1009155563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31aa0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.717+0000 7f937d27f740 1 --2- 172.21.15.46:0/1009155563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a96220 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.717+0000 7f937d27f740 1 -- 172.21.15.46:0/1009155563 >> 172.21.15.46:0/1009155563 conn(0x557a31a923d0 msgr2=0x557a31a927b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.721+0000 7f937d27f740 1 -- 172.21.15.46:0/1009155563 shutdown_connections 2024-09-10T07:48:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.721+0000 7f937d27f740 1 -- 172.21.15.46:0/1009155563 wait complete. 2024-09-10T07:48:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.721+0000 7f937d27f740 1 -- start start 2024-09-10T07:48:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.722+0000 7f937d27f740 1 Processor -- start 2024-09-10T07:48:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.723+0000 7f937d27f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a9df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.724+0000 7f937d27f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.724+0000 7f9373400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a9df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.724+0000 7f9373400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a9df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53210/0 (socket says 172.21.15.46:53210) 2024-09-10T07:48:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.724+0000 7f9373400640 1 -- 172.21.15.46:0/2177261971 learned_addr learned my addr 172.21.15.46:0/2177261971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.725+0000 7f9372a00640 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.725+0000 7f937d27f740 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31a9e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.725+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557a31aa3bf0 con 0x557a31aa1020 2024-09-10T07:48:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.725+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557a319b70f0 con 0x557a31a96760 2024-09-10T07:48:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.725+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557a31aa3d70 con 0x557a31a95e40 2024-09-10T07:48:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.726+0000 7f9373e00640 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31a9e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.726+0000 7f937d27f740 10 client.0 _ll_register_callbacks cb 0x557a3199ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.726+0000 7f9373400640 1 -- 172.21.15.46:0/2177261971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 msgr2=0x557a31a9e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.726+0000 7f9373400640 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.727+0000 7f9373400640 1 -- 172.21.15.46:0/2177261971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 msgr2=0x557a31a9e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.727+0000 7f9373400640 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31a9e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.727+0000 7f9373400640 1 -- 172.21.15.46:0/2177261971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3176d3c0 con 0x557a31a95e40 2024-09-10T07:48:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.728+0000 7f9373400640 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a9df00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f93640096c0 tx=0x7f9364004550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5cd77feb1cef463 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.728+0000 7f935d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.729+0000 7f935d600640 1 -- 172.21.15.46:0/2177261971 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936404c070 con 0x557a31a95e40 2024-09-10T07:48:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.729+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9364005520 con 0x557a31a95e40 2024-09-10T07:48:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.729+0000 7f935d600640 1 -- 172.21.15.46:0/2177261971 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9364004d40 con 0x557a31a95e40 2024-09-10T07:48:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.730+0000 7f935d600640 1 -- 172.21.15.46:0/2177261971 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f936403f050 con 0x557a31a95e40 2024-09-10T07:48:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.730+0000 7f937d27f740 10 client.28279 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.730+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x557a31aa3bf0 con 0x557a31a95e40 2024-09-10T07:48:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.731+0000 7f935d600640 1 -- 172.21.15.46:0/2177261971 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9364005520 con 0x557a31a95e40 2024-09-10T07:48:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.731+0000 7f935d600640 1 -- 172.21.15.46:0/2177261971 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7f93640035f0 con 0x557a31a95e40 2024-09-10T07:48:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.731+0000 7f937d27f740 10 client.28279 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.731+0000 7f937d27f740 10 client.28279 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:48:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.731+0000 7f937d27f740 20 client.28279 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.731+0000 7f937d27f740 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557a31ab0dc0 0x557a31ab31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.734+0000 7f9372a00640 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557a31ab0dc0 0x557a31ab31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.732+0000 7f937d27f740 4 client.28279 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.734+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a31a09950 con 0x557a31ab0dc0 2024-09-10T07:48:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.735+0000 7f9372a00640 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557a31ab0dc0 0x557a31ab31b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa006aa67ab5da34 server_cookie=f640416a100ff55e in_seq=0 out_seq=0 2024-09-10T07:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.735+0000 7f935d600640 10 client.28279 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.737+0000 7f935d600640 1 -- 172.21.15.46:0/2177261971 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a31a09950 con 0x557a31ab0dc0 2024-09-10T07:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.737+0000 7f935d600640 10 client.28279 handle_command_reply: tid=0 2024-09-10T07:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.737+0000 7f937d27f740 1 client.28279 shutdown 2024-09-10T07:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 msgr2=0x557a31a9df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a9df00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f93640096c0 tx=0x7f9364004550 comp rx=0 tx=0).stop 2024-09-10T07:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 shutdown_connections 2024-09-10T07:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557a31ab0dc0 0x557a31ab31b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31a9e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 --2- 172.21.15.46:0/2177261971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a95e40 0x557a31a9df00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 >> 172.21.15.46:0/2177261971 conn(0x557a31a923d0 msgr2=0x557a31a980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f935d600640 0 client.28279 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.739+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 shutdown_connections 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.741+0000 7f937d27f740 1 -- 172.21.15.46:0/2177261971 wait complete. 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.741+0000 7f937d27f740 20 client.28279 trim_cache size 0 max 16384 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.775+0000 7f937d27f740 1 Processor -- start 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.775+0000 7f937d27f740 1 -- start start 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.775+0000 7f937d27f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a95e40 0x557a31a98860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.775+0000 7f937d27f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a96760 0x557a31ab88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.775+0000 7f937d27f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31ab8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.775+0000 7f937d27f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93741bd9e0 con 0x557a31aa1020 2024-09-10T07:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.775+0000 7f937d27f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93741bf520 con 0x557a31a95e40 2024-09-10T07:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.775+0000 7f937d27f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f93741be0b0 con 0x557a31a96760 2024-09-10T07:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.776+0000 7f9373400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a95e40 0x557a31a98860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.776+0000 7f9373400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a95e40 0x557a31a98860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34420/0 (socket says 172.21.15.46:34420) 2024-09-10T07:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.776+0000 7f9373400640 1 -- 172.21.15.46:0/1169079088 learned_addr learned my addr 172.21.15.46:0/1169079088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.776+0000 7f9372a00640 1 --2- 172.21.15.46:0/1169079088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a96760 0x557a31ab88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.777+0000 7f9373400640 1 -- 172.21.15.46:0/1169079088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a96760 msgr2=0x557a31ab88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.777+0000 7f9373400640 1 --2- 172.21.15.46:0/1169079088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a96760 0x557a31ab88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.777+0000 7f9373400640 1 -- 172.21.15.46:0/1169079088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 msgr2=0x557a31ab8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.777+0000 7f9373400640 1 --2- 172.21.15.46:0/1169079088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31ab8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.777+0000 7f9373400640 1 -- 172.21.15.46:0/1169079088 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a31a93060 con 0x557a31a95e40 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.777+0000 7f9373400640 1 --2- 172.21.15.46:0/1169079088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a95e40 0x557a31a98860 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9364053040 tx=0x7f936402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c98fef5eeb2ea47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.778+0000 7f935fe00640 1 -- 172.21.15.46:0/1169079088 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936404c040 con 0x557a31a95e40 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.778+0000 7f935fe00640 1 -- 172.21.15.46:0/1169079088 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9364003c00 con 0x557a31a95e40 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.778+0000 7f935fe00640 1 -- 172.21.15.46:0/1169079088 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364035040 con 0x557a31a95e40 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.779+0000 7f937d27f740 1 -- 172.21.15.46:0/1169079088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a95e40 msgr2=0x557a31a98860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.779+0000 7f937d27f740 1 --2- 172.21.15.46:0/1169079088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a95e40 0x557a31a98860 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9364053040 tx=0x7f936402a480 comp rx=0 tx=0).stop 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.779+0000 7f937d27f740 1 -- 172.21.15.46:0/1169079088 shutdown_connections 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.779+0000 7f937d27f740 1 --2- 172.21.15.46:0/1169079088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31aa1020 0x557a31ab8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.779+0000 7f937d27f740 1 --2- 172.21.15.46:0/1169079088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31a96760 0x557a31ab88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.779+0000 7f937d27f740 1 --2- 172.21.15.46:0/1169079088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a95e40 0x557a31a98860 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.779+0000 7f937d27f740 1 -- 172.21.15.46:0/1169079088 >> 172.21.15.46:0/1169079088 conn(0x557a31a923d0 msgr2=0x557a31a9cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.779+0000 7f937d27f740 1 -- 172.21.15.46:0/1169079088 shutdown_connections 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.780+0000 7f937d27f740 1 -- 172.21.15.46:0/1169079088 wait complete. 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.781+0000 7f937d27f740 1 -- start start 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.781+0000 7f937d27f740 1 Processor -- start 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.781+0000 7f937d27f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31a95e40 0x557a31aa9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.782+0000 7f937d27f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.782+0000 7f9373400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31a95e40 0x557a31aa9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.782+0000 7f937d27f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31aa1020 0x557a31a9b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.782+0000 7f9373400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31a95e40 0x557a31aa9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35888/0 (socket says 172.21.15.46:35888) 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.782+0000 7f9372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.783+0000 7f9372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34434/0 (socket says 172.21.15.46:34434) 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.783+0000 7f9373400640 1 -- 172.21.15.46:0/1178743673 learned_addr learned my addr 172.21.15.46:0/1178743673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.783+0000 7f9373e00640 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31aa1020 0x557a31a9b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.783+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93741bee50 con 0x557a31a95e40 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.783+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93741be780 con 0x557a31a96760 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.783+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f93741bb9c0 con 0x557a31aa1020 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.783+0000 7f937d27f740 10 client.0 _ll_register_callbacks cb 0x7f9364009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.784+0000 7f9373e00640 1 -- 172.21.15.46:0/1178743673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 msgr2=0x557a31a9ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.784+0000 7f9373e00640 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.784+0000 7f9373e00640 1 -- 172.21.15.46:0/1178743673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31a95e40 msgr2=0x557a31aa9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.784+0000 7f9373e00640 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31a95e40 0x557a31aa9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.784+0000 7f9373e00640 1 -- 172.21.15.46:0/1178743673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a31a9b9c0 con 0x557a31aa1020 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.784+0000 7f9372a00640 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.785+0000 7f935d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.785+0000 7f9373e00640 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31aa1020 0x557a31a9b170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f936c009f10 tx=0x7f936c0054a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18884e58d7b9451 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.788+0000 7f935d600640 1 -- 172.21.15.46:0/1178743673 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c02c060 con 0x557a31aa1020 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.789+0000 7f935d600640 1 -- 172.21.15.46:0/1178743673 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f936c024070 con 0x557a31aa1020 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.789+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9364003c00 con 0x557a31aa1020 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.789+0000 7f935d600640 1 -- 172.21.15.46:0/1178743673 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f936c02c060 con 0x557a31aa1020 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.789+0000 7f937d27f740 10 client.28285 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.790+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x557a31b1b7e0 con 0x557a31aa1020 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.790+0000 7f935d600640 1 -- 172.21.15.46:0/1178743673 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9364003c00 con 0x557a31aa1020 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.790+0000 7f935d600640 1 -- 172.21.15.46:0/1178743673 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7f936c01e810 con 0x557a31aa1020 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.790+0000 7f937d27f740 10 client.28285 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.791+0000 7f937d27f740 10 client.28285 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.791+0000 7f937d27f740 20 client.28285 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.791+0000 7f937d27f740 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557a31b1bf40 0x557a31b1e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.791+0000 7f937d27f740 4 client.28285 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.791+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a31a03e60 con 0x557a31b1bf40 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.791+0000 7f9373400640 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557a31b1bf40 0x557a31b1e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.792+0000 7f9373400640 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557a31b1bf40 0x557a31b1e360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b3bf399ecfe92e1 server_cookie=f9346ef65df40883 in_seq=0 out_seq=0 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.792+0000 7f935d600640 10 client.28285 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.792+0000 7f935d600640 1 -- 172.21.15.46:0/1178743673 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a31a03e60 con 0x557a31b1bf40 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.792+0000 7f935d600640 10 client.28285 handle_command_reply: tid=0 2024-09-10T07:48:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 client.28285 shutdown 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31aa1020 msgr2=0x557a31a9b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31aa1020 0x557a31a9b170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f936c009f10 tx=0x7f936c0054a0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 shutdown_connections 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x557a31b1bf40 0x557a31b1e360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a31aa1020 0x557a31a9b170 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557a31a96760 0x557a31a9ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f935d600640 0 client.28285 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 --2- 172.21.15.46:0/1178743673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a31a95e40 0x557a31aa9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.793+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 >> 172.21.15.46:0/1178743673 conn(0x557a31a923d0 msgr2=0x557a31a90c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.794+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 shutdown_connections 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.794+0000 7f937d27f740 1 -- 172.21.15.46:0/1178743673 wait complete. 2024-09-10T07:48:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.794+0000 7f937d27f740 20 client.28285 trim_cache size 0 max 16384 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.795+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93580a2cc0 msgr2=0x7f93580a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.795+0000 7f9379200640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93580a2cc0 0x7f93580a5180 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f936c005fd0 tx=0x7f936c009bf0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.795+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93741012a0 msgr2=0x7f937406caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.795+0000 7f9379200640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93741012a0 0x7f937406caa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f936000b3f0 tx=0x7f936000b8b0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.796+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 shutdown_connections 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.796+0000 7f9379200640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f93580a2cc0 0x7f93580a5180 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.796+0000 7f9379200640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9374105d90 0x7f937406cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.796+0000 7f9379200640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93741012a0 0x7f937406caa0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.796+0000 7f9379200640 1 --2- 172.21.15.46:0/1559738247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9374100980 0x7f9374073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.796+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 >> 172.21.15.46:0/1559738247 conn(0x7f93740fc610 msgr2=0x7f937410f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.797+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 shutdown_connections 2024-09-10T07:48:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:38.797+0000 7f9379200640 1 -- 172.21.15.46:0/1559738247 wait complete. 2024-09-10T07:48:39.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:39 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:48:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:39 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:48:39.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:39 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:48:40.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:48:40.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:40 smithi046 ceph-mon[30705]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-09-10T07:48:40.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 680.0 s, 540.4 tps, lat 190.816 ms stddev 215.514 2024-09-10T07:48:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:40 smithi181 ceph-mon[35397]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-09-10T07:48:40.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:40 smithi063 ceph-mon[35020]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-09-10T07:48:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.966+0000 7fb848a00640 1 -- 172.21.15.46:0/102800322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb844102d10 msgr2=0x7fb844105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.966+0000 7fb848a00640 1 --2- 172.21.15.46:0/102800322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb844102d10 0x7fb844105180 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb830009860 tx=0x7fb83002a340 comp rx=0 tx=0).stop 2024-09-10T07:48:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.967+0000 7fb848a00640 1 -- 172.21.15.46:0/102800322 shutdown_connections 2024-09-10T07:48:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.967+0000 7fb848a00640 1 --2- 172.21.15.46:0/102800322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441056c0 0x7fb844105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.967+0000 7fb848a00640 1 --2- 172.21.15.46:0/102800322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb844102d10 0x7fb844105180 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.967+0000 7fb848a00640 1 --2- 172.21.15.46:0/102800322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8441003c0 0x7fb8441027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.967+0000 7fb848a00640 1 -- 172.21.15.46:0/102800322 >> 172.21.15.46:0/102800322 conn(0x7fb8440fc1b0 msgr2=0x7fb8440fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.967+0000 7fb848a00640 1 -- 172.21.15.46:0/102800322 shutdown_connections 2024-09-10T07:48:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.967+0000 7fb848a00640 1 -- 172.21.15.46:0/102800322 wait complete. 2024-09-10T07:48:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.969+0000 7fb848a00640 1 Processor -- start 2024-09-10T07:48:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.969+0000 7fb848a00640 1 -- start start 2024-09-10T07:48:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.969+0000 7fb848a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441003c0 0x7fb84419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.969+0000 7fb848a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844102d10 0x7fb84419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb848a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8441056c0 0x7fb8441a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb841e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441003c0 0x7fb84419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb841e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441003c0 0x7fb84419e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35900/0 (socket says 172.21.15.46:35900) 2024-09-10T07:48:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb841e00640 1 -- 172.21.15.46:0/1440870996 learned_addr learned my addr 172.21.15.46:0/1440870996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb841400640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844102d10 0x7fb84419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb842800640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8441056c0 0x7fb8441a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb844113e50 con 0x7fb8441003c0 2024-09-10T07:48:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb844113cd0 con 0x7fb844102d10 2024-09-10T07:48:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.970+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb844113fd0 con 0x7fb8441056c0 2024-09-10T07:48:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.971+0000 7fb841400640 1 -- 172.21.15.46:0/1440870996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8441056c0 msgr2=0x7fb8441a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.971+0000 7fb841400640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8441056c0 0x7fb8441a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.971+0000 7fb841400640 1 -- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441003c0 msgr2=0x7fb84419e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.971+0000 7fb841400640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441003c0 0x7fb84419e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.971+0000 7fb841400640 1 -- 172.21.15.46:0/1440870996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8441a5ff0 con 0x7fb844102d10 2024-09-10T07:48:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.971+0000 7fb841400640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844102d10 0x7fb84419e910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb830035040 tx=0x7fb830033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.972+0000 7fb83aa00640 1 -- 172.21.15.46:0/1440870996 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830037040 con 0x7fb844102d10 2024-09-10T07:48:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.972+0000 7fb83aa00640 1 -- 172.21.15.46:0/1440870996 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb830004050 con 0x7fb844102d10 2024-09-10T07:48:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.972+0000 7fb83aa00640 1 -- 172.21.15.46:0/1440870996 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830003a10 con 0x7fb844102d10 2024-09-10T07:48:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.972+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8441a6280 con 0x7fb844102d10 2024-09-10T07:48:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.972+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8441a6720 con 0x7fb844102d10 2024-09-10T07:48:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.981+0000 7fb83aa00640 1 -- 172.21.15.46:0/1440870996 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb830042030 con 0x7fb844102d10 2024-09-10T07:48:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.984+0000 7fb83aa00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb8280a2d80 0x7fb8280a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.984+0000 7fb83aa00640 1 -- 172.21.15.46:0/1440870996 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb8300ee850 con 0x7fb844102d10 2024-09-10T07:48:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.987+0000 7fb84b9db740 1 Processor -- start 2024-09-10T07:48:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.989+0000 7fb84b9db740 1 -- start start 2024-09-10T07:48:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.989+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e525490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.992+0000 7fb841e00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb8280a2d80 0x7fb8280a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.992+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 0x563f5e52a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.992+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e52cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.992+0000 7fb84b9db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563f5e48a100 con 0x563f5e525090 2024-09-10T07:48:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.992+0000 7fb84b9db740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563f5e4321b0 con 0x563f5e52aa60 2024-09-10T07:48:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.992+0000 7fb84b9db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563f5e4860c0 con 0x563f5e5259d0 2024-09-10T07:48:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.993+0000 7fb841400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 0x563f5e52a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.993+0000 7fb842800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e52cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.993+0000 7fb842800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e52cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34460/0 (socket says 172.21.15.46:34460) 2024-09-10T07:48:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.993+0000 7fb842800640 1 -- 172.21.15.46:0/3927269911 learned_addr learned my addr 172.21.15.46:0/3927269911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.994+0000 7fb841e00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb8280a2d80 0x7fb8280a5240 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb834009df0 tx=0x7fb834009930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.994+0000 7fb841e00640 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e525490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.994+0000 7fb842800640 1 -- 172.21.15.46:0/3927269911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 msgr2=0x563f5e52a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.995+0000 7fb842800640 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 0x563f5e52a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.995+0000 7fb842800640 1 -- 172.21.15.46:0/3927269911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 msgr2=0x563f5e525490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.997+0000 7fb842800640 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e525490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.997+0000 7fb842800640 1 -- 172.21.15.46:0/3927269911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5e19d210 con 0x563f5e52aa60 2024-09-10T07:48:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.997+0000 7fb841400640 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 0x563f5e52a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.997+0000 7fb841e00640 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e525490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:40.998+0000 7fb842800640 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e52cf20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00e580 tx=0x7fb83c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee155149bad6cef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.001+0000 7fb82f000640 1 -- 172.21.15.46:0/3927269911 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c003970 con 0x563f5e52aa60 2024-09-10T07:48:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.001+0000 7fb82f000640 1 -- 172.21.15.46:0/3927269911 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c00d830 con 0x563f5e52aa60 2024-09-10T07:48:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.001+0000 7fb82f000640 1 -- 172.21.15.46:0/3927269911 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0044f0 con 0x563f5e52aa60 2024-09-10T07:48:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.003+0000 7fb84b9db740 1 -- 172.21.15.46:0/3927269911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 msgr2=0x563f5e52cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.003+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e52cf20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00e580 tx=0x7fb83c0095c0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.004+0000 7fb84b9db740 1 -- 172.21.15.46:0/3927269911 shutdown_connections 2024-09-10T07:48:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.004+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e52cf20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.004+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 0x563f5e52a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.004+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3927269911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e525490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.004+0000 7fb84b9db740 1 -- 172.21.15.46:0/3927269911 >> 172.21.15.46:0/3927269911 conn(0x563f5e520700 msgr2=0x563f5e520b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.005+0000 7fb84b9db740 1 -- 172.21.15.46:0/3927269911 shutdown_connections 2024-09-10T07:48:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.005+0000 7fb84b9db740 1 -- 172.21.15.46:0/3927269911 wait complete. 2024-09-10T07:48:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.005+0000 7fb84b9db740 1 -- start start 2024-09-10T07:48:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.006+0000 7fb84b9db740 1 Processor -- start 2024-09-10T07:48:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e52f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e52f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e536730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb84b9db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563f5e52da30 con 0x563f5e5259d0 2024-09-10T07:48:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb84b9db740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563f5e413090 con 0x563f5e52aa60 2024-09-10T07:48:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb84b9db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563f5e52dbb0 con 0x563f5e525090 2024-09-10T07:48:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb842800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e536730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb842800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e536730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34470/0 (socket says 172.21.15.46:34470) 2024-09-10T07:48:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.008+0000 7fb842800640 1 -- 172.21.15.46:0/2566170104 learned_addr learned my addr 172.21.15.46:0/2566170104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb841400640 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e52f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb841e00640 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e52f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb842800640 1 -- 172.21.15.46:0/2566170104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 msgr2=0x563f5e52f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb842800640 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e52f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb842800640 1 -- 172.21.15.46:0/2566170104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 msgr2=0x563f5e52f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb842800640 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e52f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb842800640 1 -- 172.21.15.46:0/2566170104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5e269620 con 0x563f5e52aa60 2024-09-10T07:48:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb842800640 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e536730 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00eed0 tx=0x7fb83c00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=484c2691ddc0eb50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.009+0000 7fb823400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.010+0000 7fb823400640 1 -- 172.21.15.46:0/2566170104 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c027070 con 0x563f5e52aa60 2024-09-10T07:48:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.010+0000 7fb84b9db740 10 client.0 _ll_register_callbacks cb 0x563f5e467f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.010+0000 7fb84b9db740 1 -- 172.21.15.46:0/2566170104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83c00d830 con 0x563f5e52aa60 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.010+0000 7fb823400640 1 -- 172.21.15.46:0/2566170104 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb83c003b80 con 0x563f5e52aa60 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.011+0000 7fb823400640 1 -- 172.21.15.46:0/2566170104 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb83c031050 con 0x563f5e52aa60 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.011+0000 7fb84b9db740 10 client.28349 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.011+0000 7fb84b9db740 1 -- 172.21.15.46:0/2566170104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=51}) -- 0x563f5e539300 con 0x563f5e52aa60 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.012+0000 7fb823400640 1 -- 172.21.15.46:0/2566170104 <== mon.2 v2:172.21.15.63:3300/0 4 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7fb83c00b040 con 0x563f5e52aa60 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.013+0000 7fb84b9db740 10 client.28349 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.013+0000 7fb84b9db740 10 client.28349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.013+0000 7fb84b9db740 20 client.28349 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.013+0000 7fb84b9db740 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563f5e53a6b0 0x563f5e53cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.013+0000 7fb84b9db740 4 client.28349 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.013+0000 7fb84b9db740 1 -- 172.21.15.46:0/2566170104 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f5e497830 con 0x563f5e53a6b0 2024-09-10T07:48:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.013+0000 7fb841e00640 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563f5e53a6b0 0x563f5e53cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.015+0000 7fb841e00640 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563f5e53a6b0 0x563f5e53cb00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=206b68d3d5b12c30 server_cookie=9c69e93768f9cd1 in_seq=0 out_seq=0 2024-09-10T07:48:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.015+0000 7fb823400640 10 client.28349 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.018+0000 7fb823400640 1 -- 172.21.15.46:0/2566170104 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f5e497830 con 0x563f5e53a6b0 2024-09-10T07:48:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.018+0000 7fb823400640 10 client.28349 handle_command_reply: tid=0 2024-09-10T07:48:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.019+0000 7fb84b9db740 1 client.28349 shutdown 2024-09-10T07:48:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.019+0000 7fb84b9db740 1 -- 172.21.15.46:0/2566170104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 msgr2=0x563f5e536730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.019+0000 7fb84b9db740 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e536730 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00eed0 tx=0x7fb83c00dc10 comp rx=0 tx=0).stop 2024-09-10T07:48:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.020+0000 7fb84b9db740 1 -- 172.21.15.46:0/2566170104 shutdown_connections 2024-09-10T07:48:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.020+0000 7fb84b9db740 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563f5e53a6b0 0x563f5e53cb00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.020+0000 7fb84b9db740 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e536730 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.020+0000 7fb84b9db740 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e52f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.020+0000 7fb84b9db740 1 --2- 172.21.15.46:0/2566170104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e52f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.020+0000 7fb84b9db740 1 -- 172.21.15.46:0/2566170104 >> 172.21.15.46:0/2566170104 conn(0x563f5e520700 msgr2=0x563f5e521f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.020+0000 7fb823400640 0 client.28349 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.022+0000 7fb84b9db740 1 -- 172.21.15.46:0/2566170104 shutdown_connections 2024-09-10T07:48:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.022+0000 7fb84b9db740 1 -- 172.21.15.46:0/2566170104 wait complete. 2024-09-10T07:48:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.023+0000 7fb84b9db740 20 client.28349 trim_cache size 0 max 16384 2024-09-10T07:48:41.070 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.085+0000 7fb84b9db740 1 Processor -- start 2024-09-10T07:48:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.085+0000 7fb84b9db740 1 -- start start 2024-09-10T07:48:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.085+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e521af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.085+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e5366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.085+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e522030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.085+0000 7fb84b9db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441b0c70 con 0x563f5e5259d0 2024-09-10T07:48:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.085+0000 7fb84b9db740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441b27b0 con 0x563f5e52aa60 2024-09-10T07:48:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.085+0000 7fb84b9db740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441b1340 con 0x563f5e525090 2024-09-10T07:48:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.086+0000 7fb841400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e5366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.086+0000 7fb841e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e521af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.086+0000 7fb841e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e521af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53286/0 (socket says 172.21.15.46:53286) 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.086+0000 7fb841e00640 1 -- 172.21.15.46:0/1565453911 learned_addr learned my addr 172.21.15.46:0/1565453911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.086+0000 7fb842800640 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e522030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.086+0000 7fb841e00640 1 -- 172.21.15.46:0/1565453911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 msgr2=0x563f5e522030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.086+0000 7fb841e00640 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e522030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.087+0000 7fb841e00640 1 -- 172.21.15.46:0/1565453911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 msgr2=0x563f5e5366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.087+0000 7fb841e00640 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e5366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.087+0000 7fb841e00640 1 -- 172.21.15.46:0/1565453911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5e53d180 con 0x563f5e525090 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.087+0000 7fb841400640 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e5366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.087+0000 7fb841e00640 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e521af0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x563f5e52e780 tx=0x7fb834016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45431662d9da2d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.087+0000 7fb82e600640 1 -- 172.21.15.46:0/1565453911 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83404e070 con 0x563f5e525090 2024-09-10T07:48:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.087+0000 7fb82e600640 1 -- 172.21.15.46:0/1565453911 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8340278e0 con 0x563f5e525090 2024-09-10T07:48:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.087+0000 7fb82e600640 1 -- 172.21.15.46:0/1565453911 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83401da10 con 0x563f5e525090 2024-09-10T07:48:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.089+0000 7fb84b9db740 1 -- 172.21.15.46:0/1565453911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 msgr2=0x563f5e521af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.090+0000 7fb84b9db740 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e521af0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x563f5e52e780 tx=0x7fb834016000 comp rx=0 tx=0).stop 2024-09-10T07:48:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.090+0000 7fb84b9db740 1 -- 172.21.15.46:0/1565453911 shutdown_connections 2024-09-10T07:48:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.090+0000 7fb84b9db740 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e522030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.090+0000 7fb84b9db740 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e5259d0 0x563f5e5366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.090+0000 7fb84b9db740 1 --2- 172.21.15.46:0/1565453911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e525090 0x563f5e521af0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.090+0000 7fb84b9db740 1 -- 172.21.15.46:0/1565453911 >> 172.21.15.46:0/1565453911 conn(0x563f5e520700 msgr2=0x563f5e51f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.090+0000 7fb84b9db740 1 -- 172.21.15.46:0/1565453911 shutdown_connections 2024-09-10T07:48:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.094+0000 7fb84b9db740 1 -- 172.21.15.46:0/1565453911 wait complete. 2024-09-10T07:48:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.095+0000 7fb84b9db740 1 -- start start 2024-09-10T07:48:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.095+0000 7fb84b9db740 1 Processor -- start 2024-09-10T07:48:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.096+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e533df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.096+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 0x563f5e534330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.096+0000 7fb84b9db740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e5352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb841e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e533df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb842800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e5352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb842800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e5352f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34484/0 (socket says 172.21.15.46:34484) 2024-09-10T07:48:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb842800640 1 -- 172.21.15.46:0/3249985894 learned_addr learned my addr 172.21.15.46:0/3249985894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441b20e0 con 0x563f5e525090 2024-09-10T07:48:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441b1a10 con 0x563f5e52aa60 2024-09-10T07:48:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441aec50 con 0x563f5e5259d0 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb842800640 1 -- 172.21.15.46:0/3249985894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 msgr2=0x563f5e534330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb842800640 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 0x563f5e534330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb842800640 1 -- 172.21.15.46:0/3249985894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 msgr2=0x563f5e533df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.097+0000 7fb842800640 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e533df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.098+0000 7fb842800640 1 -- 172.21.15.46:0/3249985894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5e535900 con 0x563f5e52aa60 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.098+0000 7fb84b9db740 10 client.0 _ll_register_callbacks cb 0x7fb830004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.098+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8340278e0 con 0x563f5e52aa60 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.098+0000 7fb842800640 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e5352f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0042c0 tx=0x7fb83c010040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3223242e79de75e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.098+0000 7fb823400640 10 client.28355 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.099+0000 7fb823400640 1 -- 172.21.15.46:0/3249985894 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c044050 con 0x563f5e52aa60 2024-09-10T07:48:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.099+0000 7fb823400640 1 -- 172.21.15.46:0/3249985894 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb83c01c070 con 0x563f5e52aa60 2024-09-10T07:48:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.099+0000 7fb823400640 1 -- 172.21.15.46:0/3249985894 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb83c027040 con 0x563f5e52aa60 2024-09-10T07:48:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.105+0000 7fb84b9db740 10 client.28355 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.105+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=51}) -- 0x563f5e5a53e0 con 0x563f5e52aa60 2024-09-10T07:48:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.105+0000 7fb823400640 1 -- 172.21.15.46:0/3249985894 <== mon.2 v2:172.21.15.63:3300/0 4 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7fb83c00b050 con 0x563f5e52aa60 2024-09-10T07:48:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.106+0000 7fb84b9db740 10 client.28355 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.106+0000 7fb84b9db740 10 client.28355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.106+0000 7fb84b9db740 20 client.28355 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.106+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563f5e5a5c20 0x563f5e5a8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.106+0000 7fb84b9db740 4 client.28355 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:48:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.106+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f5e491b80 con 0x563f5e5a5c20 2024-09-10T07:48:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.106+0000 7fb841e00640 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563f5e5a5c20 0x563f5e5a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.110+0000 7fb841e00640 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563f5e5a5c20 0x563f5e5a8030 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef16b566bb67aa82 server_cookie=1f16d98c2dee29a8 in_seq=0 out_seq=0 2024-09-10T07:48:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.110+0000 7fb823400640 10 client.28355 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.158+0000 7fb823400640 1 -- 172.21.15.46:0/3249985894 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f5e491b80 con 0x563f5e5a5c20 2024-09-10T07:48:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.158+0000 7fb823400640 10 client.28355 handle_command_reply: tid=0 2024-09-10T07:48:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.158+0000 7fb84b9db740 1 client.28355 shutdown 2024-09-10T07:48:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 msgr2=0x563f5e5352f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e5352f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0042c0 tx=0x7fb83c010040 comp rx=0 tx=0).stop 2024-09-10T07:48:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 shutdown_connections 2024-09-10T07:48:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x563f5e5a5c20 0x563f5e5a8030 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f5e52aa60 0x563f5e5352f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563f5e5259d0 0x563f5e534330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb84b9db740 1 --2- 172.21.15.46:0/3249985894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f5e525090 0x563f5e533df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb823400640 0 client.28355 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.159+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 >> 172.21.15.46:0/3249985894 conn(0x563f5e520700 msgr2=0x563f5e520ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.160+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 shutdown_connections 2024-09-10T07:48:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.160+0000 7fb84b9db740 1 -- 172.21.15.46:0/3249985894 wait complete. 2024-09-10T07:48:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.160+0000 7fb84b9db740 20 client.28355 trim_cache size 0 max 16384 2024-09-10T07:48:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:48:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:48:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:48:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:48:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb8280a2d80 msgr2=0x7fb8280a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb8280a2d80 0x7fb8280a5240 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb834009df0 tx=0x7fb834009930 comp rx=0 tx=0).stop 2024-09-10T07:48:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844102d10 msgr2=0x7fb84419e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844102d10 0x7fb84419e910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb830035040 tx=0x7fb830033d10 comp rx=0 tx=0).stop 2024-09-10T07:48:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 shutdown_connections 2024-09-10T07:48:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8441056c0 0x7fb8441a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844102d10 0x7fb84419e910 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb8280a2d80 0x7fb8280a5240 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 --2- 172.21.15.46:0/1440870996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441003c0 0x7fb84419e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.162+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 >> 172.21.15.46:0/1440870996 conn(0x7fb8440fc1b0 msgr2=0x7fb844100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.164+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 shutdown_connections 2024-09-10T07:48:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:41.164+0000 7fb848a00640 1 -- 172.21.15.46:0/1440870996 wait complete. 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 -- 172.21.15.46:0/3177624694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d0102b90 msgr2=0x7fa3d0102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 --2- 172.21.15.46:0/3177624694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d0102b90 0x7fa3d0102f70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009900 tx=0x7fa3c002a480 comp rx=0 tx=0).stop 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 -- 172.21.15.46:0/3177624694 shutdown_connections 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 --2- 172.21.15.46:0/3177624694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d010bfc0 0x7fa3d010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 --2- 172.21.15.46:0/3177624694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d01034b0 0x7fa3d010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 --2- 172.21.15.46:0/3177624694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d0102b90 0x7fa3d0102f70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 -- 172.21.15.46:0/3177624694 >> 172.21.15.46:0/3177624694 conn(0x7fa3d00fc610 msgr2=0x7fa3d00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 -- 172.21.15.46:0/3177624694 shutdown_connections 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.995+0000 7fa3d5000640 1 -- 172.21.15.46:0/3177624694 wait complete. 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.996+0000 7fa3d5000640 1 Processor -- start 2024-09-10T07:48:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.996+0000 7fa3d5000640 1 -- start start 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.996+0000 7fa3d5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0102b90 0x7fa3d01007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.996+0000 7fa3d5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d01034b0 0x7fa3d0100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.996+0000 7fa3d5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d010bfc0 0x7fa3d0101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.996+0000 7fa3d5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0111e10 con 0x7fa3d01034b0 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.996+0000 7fa3d5000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0111c90 con 0x7fa3d010bfc0 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.996+0000 7fa3d5000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0111f90 con 0x7fa3d0102b90 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d010bfc0 0x7fa3d0101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d010bfc0 0x7fa3d0101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34498/0 (socket says 172.21.15.46:34498) 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0102b90 0x7fa3d01007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 -- 172.21.15.46:0/1508269211 learned_addr learned my addr 172.21.15.46:0/1508269211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d01034b0 0x7fa3d0100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 -- 172.21.15.46:0/1508269211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0102b90 msgr2=0x7fa3d01007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0102b90 0x7fa3d01007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 -- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d01034b0 msgr2=0x7fa3d0100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d01034b0 0x7fa3d0100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 -- 172.21.15.46:0/1508269211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d01a6790 con 0x7fa3d010bfc0 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3cfe00640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d010bfc0 0x7fa3d0101270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800eb10 tx=0x7fa3c800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.997+0000 7fa3ccc00640 1 -- 172.21.15.46:0/1508269211 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8004b20 con 0x7fa3d010bfc0 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.998+0000 7fa3d5000640 1 -- 172.21.15.46:0/1508269211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d01a6a80 con 0x7fa3d010bfc0 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.998+0000 7fa3ccc00640 1 -- 172.21.15.46:0/1508269211 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c8004cc0 con 0x7fa3d010bfc0 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.998+0000 7fa3ccc00640 1 -- 172.21.15.46:0/1508269211 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8010760 con 0x7fa3d010bfc0 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:41.998+0000 7fa3d5000640 1 -- 172.21.15.46:0/1508269211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d01a6fc0 con 0x7fa3d010bfc0 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3d908d740 1 Processor -- start 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3ccc00640 1 -- 172.21.15.46:0/1508269211 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa3c8005020 con 0x7fa3d010bfc0 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3d908d740 1 -- start start 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22104e40 0x562d22105220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22105760 0x562d2210fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22110020 0x562d22112410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3d908d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d2206b3e0 con 0x562d22110020 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3d908d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562d22012d80 con 0x562d22105760 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.001+0000 7fa3d908d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562d220673a0 con 0x562d22104e40 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22104e40 0x562d22105220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22104e40 0x562d22105220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53312/0 (socket says 172.21.15.46:53312) 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 -- 172.21.15.46:0/556998079 learned_addr learned my addr 172.21.15.46:0/556998079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22110020 0x562d22112410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3cf400640 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22105760 0x562d2210fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 -- 172.21.15.46:0/556998079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22105760 msgr2=0x562d2210fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22105760 0x562d2210fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 -- 172.21.15.46:0/556998079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22110020 msgr2=0x562d22112410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22110020 0x562d22112410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 -- 172.21.15.46:0/556998079 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d22031ae0 con 0x562d22104e40 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22104e40 0x562d22105220 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80058c0 tx=0x7fa3b8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e731ea20df400e2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3ccc00640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3b40a2b80 0x7fa3b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.002+0000 7fa3c4c00640 1 -- 172.21.15.46:0/556998079 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8004030 con 0x562d22104e40 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3c4c00640 1 -- 172.21.15.46:0/556998079 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b8005500 con 0x562d22104e40 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3c4c00640 1 -- 172.21.15.46:0/556998079 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8004ca0 con 0x562d22104e40 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3ccc00640 1 -- 172.21.15.46:0/1508269211 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa3c80cf910 con 0x7fa3d010bfc0 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3cf400640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3b40a2b80 0x7fa3b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3d908d740 1 -- 172.21.15.46:0/556998079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22104e40 msgr2=0x562d22105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3d908d740 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22104e40 0x562d22105220 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80058c0 tx=0x7fa3b8005850 comp rx=0 tx=0).stop 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3d908d740 1 -- 172.21.15.46:0/556998079 shutdown_connections 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3d908d740 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22110020 0x562d22112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3d908d740 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22105760 0x562d2210fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3d908d740 1 --2- 172.21.15.46:0/556998079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22104e40 0x562d22105220 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3d908d740 1 -- 172.21.15.46:0/556998079 >> 172.21.15.46:0/556998079 conn(0x562d221013d0 msgr2=0x562d221017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.003+0000 7fa3d908d740 1 -- 172.21.15.46:0/556998079 shutdown_connections 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.005+0000 7fa3d908d740 1 -- 172.21.15.46:0/556998079 wait complete. 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.006+0000 7fa3d908d740 1 -- start start 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.006+0000 7fa3d908d740 1 Processor -- start 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d2210cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22105760 0x562d2210d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d2210d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3d908d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d22112bf0 con 0x562d22104e40 2024-09-10T07:48:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3d908d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562d220260f0 con 0x562d22110020 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3d908d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562d22112d70 con 0x562d22105760 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3d908d740 10 client.0 _ll_register_callbacks cb 0x562d2200dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22105760 0x562d2210d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22105760 0x562d2210d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53328/0 (socket says 172.21.15.46:53328) 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.007+0000 7fa3c7e00640 1 -- 172.21.15.46:0/931146610 learned_addr learned my addr 172.21.15.46:0/931146610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.008+0000 7fa3c7e00640 1 -- 172.21.15.46:0/931146610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 msgr2=0x562d2210d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.008+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d2210d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.008+0000 7fa3c7e00640 1 -- 172.21.15.46:0/931146610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 msgr2=0x562d2210cea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.008+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d2210cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.008+0000 7fa3c7e00640 1 -- 172.21.15.46:0/931146610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d21ddc3c0 con 0x562d22105760 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.008+0000 7fa3cf400640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3b40a2b80 0x7fa3b40a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0000c00 tx=0x7fa3c0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.010+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22105760 0x562d2210d3e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8004880 tx=0x7fa3b8005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b2e25e378c894d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.010+0000 7fa3bd600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.011+0000 7fa3bd600640 1 -- 172.21.15.46:0/931146610 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8004c80 con 0x562d22105760 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.011+0000 7fa3bd600640 1 -- 172.21.15.46:0/931146610 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa3b8004e20 con 0x562d22105760 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.012+0000 7fa3d908d740 1 -- 172.21.15.46:0/931146610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b8005500 con 0x562d22105760 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.012+0000 7fa3bd600640 1 -- 172.21.15.46:0/931146610 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa3b800f790 con 0x562d22105760 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.013+0000 7fa3d908d740 10 client.28309 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.013+0000 7fa3d908d740 1 -- 172.21.15.46:0/931146610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562d221273d0 con 0x562d22105760 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.015+0000 7fa3bd600640 1 -- 172.21.15.46:0/931146610 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b8005500 con 0x562d22105760 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.015+0000 7fa3bd600640 1 -- 172.21.15.46:0/931146610 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7fa3b8027070 con 0x562d22105760 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.015+0000 7fa3d908d740 10 client.28309 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.015+0000 7fa3d908d740 10 client.28309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.015+0000 7fa3d908d740 20 client.28309 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.015+0000 7fa3d908d740 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x562d22128720 0x562d2212ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.015+0000 7fa3d908d740 4 client.28309 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.015+0000 7fa3d908d740 1 -- 172.21.15.46:0/931146610 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d22078950 con 0x562d22128720 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.018+0000 7fa3cf400640 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x562d22128720 0x562d2212ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.019+0000 7fa3cf400640 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x562d22128720 0x562d2212ab10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2b06056c84bd72a server_cookie=57dd128f866b823 in_seq=0 out_seq=0 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.019+0000 7fa3bd600640 10 client.28309 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.020+0000 7fa3bd600640 1 -- 172.21.15.46:0/931146610 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d22078950 con 0x562d22128720 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.020+0000 7fa3bd600640 10 client.28309 handle_command_reply: tid=0 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 client.28309 shutdown 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 -- 172.21.15.46:0/931146610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22105760 msgr2=0x562d2210d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22105760 0x562d2210d3e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8004880 tx=0x7fa3b8005970 comp rx=0 tx=0).stop 2024-09-10T07:48:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 -- 172.21.15.46:0/931146610 shutdown_connections 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x562d22128720 0x562d2212ab10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d2210d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22105760 0x562d2210d3e0 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8004880 tx=0x7fa3b8005970 comp rx=0 tx=0).stop 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 --2- 172.21.15.46:0/931146610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d2210cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.021+0000 7fa3d908d740 1 -- 172.21.15.46:0/931146610 >> 172.21.15.46:0/931146610 conn(0x562d221013d0 msgr2=0x562d221070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.022+0000 7fa3bd600640 0 client.28309 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.023+0000 7fa3d908d740 1 -- 172.21.15.46:0/931146610 shutdown_connections 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.024+0000 7fa3d908d740 1 -- 172.21.15.46:0/931146610 wait complete. 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.024+0000 7fa3d908d740 20 client.28309 trim_cache size 0 max 16384 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.056+0000 7fa3d908d740 1 Processor -- start 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.056+0000 7fa3d908d740 1 -- start start 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.056+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d221078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.056+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22130340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.056+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d22130880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.056+0000 7fa3d908d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01ac8b0 con 0x562d22104e40 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.056+0000 7fa3d908d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01ae3f0 con 0x562d22110020 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.056+0000 7fa3d908d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01acf80 con 0x562d22101e20 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.057+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22130340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.057+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22130340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34530/0 (socket says 172.21.15.46:34530) 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.057+0000 7fa3c7e00640 1 -- 172.21.15.46:0/1559569230 learned_addr learned my addr 172.21.15.46:0/1559569230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.057+0000 7fa3c7e00640 1 -- 172.21.15.46:0/1559569230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 msgr2=0x562d22130880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.057+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/1559569230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d22130880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.057+0000 7fa3c7e00640 1 -- 172.21.15.46:0/1559569230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 msgr2=0x562d221078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.057+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/1559569230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d221078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.057+0000 7fa3c7e00640 1 -- 172.21.15.46:0/1559569230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d22105590 con 0x562d22110020 2024-09-10T07:48:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.058+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/1559569230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22130340 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8029790 tx=0x7fa3b8011760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450ae4e108469975 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.059+0000 7fa3bfe00640 1 -- 172.21.15.46:0/1559569230 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b805c070 con 0x562d22110020 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.059+0000 7fa3bfe00640 1 -- 172.21.15.46:0/1559569230 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b800fe10 con 0x562d22110020 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.059+0000 7fa3bfe00640 1 -- 172.21.15.46:0/1559569230 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80048b0 con 0x562d22110020 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.060+0000 7fa3d908d740 1 -- 172.21.15.46:0/1559569230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 msgr2=0x562d22130340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.060+0000 7fa3d908d740 1 --2- 172.21.15.46:0/1559569230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22130340 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8029790 tx=0x7fa3b8011760 comp rx=0 tx=0).stop 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.061+0000 7fa3d908d740 1 -- 172.21.15.46:0/1559569230 shutdown_connections 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.061+0000 7fa3d908d740 1 --2- 172.21.15.46:0/1559569230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d22130880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.061+0000 7fa3d908d740 1 --2- 172.21.15.46:0/1559569230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22130340 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.061+0000 7fa3d908d740 1 --2- 172.21.15.46:0/1559569230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d221078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.061+0000 7fa3d908d740 1 -- 172.21.15.46:0/1559569230 >> 172.21.15.46:0/1559569230 conn(0x562d221013d0 msgr2=0x562d2210bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.061+0000 7fa3d908d740 1 -- 172.21.15.46:0/1559569230 shutdown_connections 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.061+0000 7fa3d908d740 1 -- 172.21.15.46:0/1559569230 wait complete. 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.062+0000 7fa3d908d740 1 -- start start 2024-09-10T07:48:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.062+0000 7fa3d908d740 1 Processor -- start 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.064+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d221184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d22118a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d221184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d221184d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:53346/0 (socket says 172.21.15.46:53346) 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3d908d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22113890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d22118a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3cf400640 1 -- 172.21.15.46:0/3788170956 learned_addr learned my addr 172.21.15.46:0/3788170956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01add20 con 0x562d22104e40 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01ad650 con 0x562d22110020 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01aa890 con 0x562d22101e20 2024-09-10T07:48:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.065+0000 7fa3d908d740 10 client.0 _ll_register_callbacks cb 0x7fa3b8009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.069+0000 7fa3cfe00640 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.655 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:48:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.069+0000 7fa3cf400640 1 -- 172.21.15.46:0/3788170956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 msgr2=0x562d22113890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.069+0000 7fa3cf400640 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22113890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.069+0000 7fa3cf400640 1 -- 172.21.15.46:0/3788170956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 msgr2=0x562d22118a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.069+0000 7fa3cf400640 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d22118a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.069+0000 7fa3cf400640 1 -- 172.21.15.46:0/3788170956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d221140c0 con 0x562d22101e20 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.069+0000 7fa3cf400640 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d221184d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0031550 tx=0x7fa3c002a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa458ad1ee3814c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.069+0000 7fa3bd600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.074+0000 7fa3bd600640 1 -- 172.21.15.46:0/3788170956 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0035040 con 0x562d22101e20 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.075+0000 7fa3bd600640 1 -- 172.21.15.46:0/3788170956 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa3c00046c0 con 0x562d22101e20 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.075+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b800fe10 con 0x562d22101e20 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.075+0000 7fa3bd600640 1 -- 172.21.15.46:0/3788170956 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa3c0060050 con 0x562d22101e20 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.076+0000 7fa3d908d740 10 client.28312 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.076+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562d22120d10 con 0x562d22101e20 2024-09-10T07:48:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.076+0000 7fa3bd600640 1 -- 172.21.15.46:0/3788170956 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b800fe10 con 0x562d22101e20 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.078+0000 7fa3bd600640 1 -- 172.21.15.46:0/3788170956 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7fa3c0032b00 con 0x562d22101e20 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.078+0000 7fa3d908d740 10 client.28312 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.078+0000 7fa3d908d740 10 client.28312 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.078+0000 7fa3d908d740 20 client.28312 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.078+0000 7fa3d908d740 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x562d2218b2d0 0x562d2218d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.078+0000 7fa3d908d740 4 client.28312 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.078+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d22072e60 con 0x562d2218b2d0 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.079+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x562d2218b2d0 0x562d2218d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.080+0000 7fa3c7e00640 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x562d2218b2d0 0x562d2218d6c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77dc33eb543a47c4 server_cookie=88feec538ca162b4 in_seq=0 out_seq=0 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.080+0000 7fa3bd600640 10 client.28312 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3bd600640 1 -- 172.21.15.46:0/3788170956 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d22072e60 con 0x562d2218b2d0 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3bd600640 10 client.28312 handle_command_reply: tid=0 2024-09-10T07:48:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 client.28312 shutdown 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 msgr2=0x562d221184d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d221184d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0031550 tx=0x7fa3c002a480 comp rx=0 tx=0).stop 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 shutdown_connections 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x562d2218b2d0 0x562d2218d6c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562d22110020 0x562d22113890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d22104e40 0x562d22118a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 --2- 172.21.15.46:0/3788170956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562d22101e20 0x562d221184d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 >> 172.21.15.46:0/3788170956 conn(0x562d221013d0 msgr2=0x562d2210ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.081+0000 7fa3bd600640 0 client.28312 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.082+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 shutdown_connections 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.082+0000 7fa3d908d740 1 -- 172.21.15.46:0/3788170956 wait complete. 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.082+0000 7fa3d908d740 20 client.28312 trim_cache size 0 max 16384 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 -- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3b40a2b80 msgr2=0x7fa3b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3b40a2b80 0x7fa3b40a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0000c00 tx=0x7fa3c0005b50 comp rx=0 tx=0).stop 2024-09-10T07:48:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 -- 172.21.15.46:0/1508269211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d010bfc0 msgr2=0x7fa3d0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d010bfc0 0x7fa3d0101270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800eb10 tx=0x7fa3c800c2d0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 -- 172.21.15.46:0/1508269211 shutdown_connections 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d010bfc0 0x7fa3d0101270 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa3b40a2b80 0x7fa3b40a5040 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d01034b0 0x7fa3d0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 --2- 172.21.15.46:0/1508269211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0102b90 0x7fa3d01007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 -- 172.21.15.46:0/1508269211 >> 172.21.15.46:0/1508269211 conn(0x7fa3d00fc610 msgr2=0x7fa3d00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 -- 172.21.15.46:0/1508269211 shutdown_connections 2024-09-10T07:48:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:42.083+0000 7fa3d5000640 1 -- 172.21.15.46:0/1508269211 wait complete. 2024-09-10T07:48:42.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:42 smithi046 ceph-mon[30705]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 510 op/s 2024-09-10T07:48:42.660 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:42 smithi063 ceph-mon[35020]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 510 op/s 2024-09-10T07:48:42.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:42 smithi181 ceph-mon[35397]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 510 op/s 2024-09-10T07:48:43.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:48:43.858 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.1 2024-09-10T07:48:43.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:48:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:44 smithi181 ceph-mon[35397]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.5 MiB/s wr, 539 op/s 2024-09-10T07:48:44.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:44 smithi063 ceph-mon[35020]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.5 MiB/s wr, 539 op/s 2024-09-10T07:48:44.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:44 smithi046 ceph-mon[30705]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.5 MiB/s wr, 539 op/s 2024-09-10T07:48:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:44.861 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 685.0 s, 457.0 tps, lat 213.052 ms stddev 262.582 2024-09-10T07:48:46.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:46 smithi046 ceph-mon[30705]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 440 op/s 2024-09-10T07:48:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.340+0000 7fcd0f600640 1 -- 172.21.15.46:0/594204015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd08074e30 msgr2=0x7fcd0810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.340+0000 7fcd0f600640 1 --2- 172.21.15.46:0/594204015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd08074e30 0x7fcd0810df00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009900 tx=0x7fccf402a480 comp rx=0 tx=0).stop 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.340+0000 7fcd0f600640 1 -- 172.21.15.46:0/594204015 shutdown_connections 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.341+0000 7fcd0f600640 1 --2- 172.21.15.46:0/594204015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd0810e440 0x7fcd08110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.341+0000 7fcd0f600640 1 --2- 172.21.15.46:0/594204015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd08074e30 0x7fcd0810df00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.341+0000 7fcd0f600640 1 --2- 172.21.15.46:0/594204015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd08074510 0x7fcd080748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.341+0000 7fcd0f600640 1 -- 172.21.15.46:0/594204015 >> 172.21.15.46:0/594204015 conn(0x7fcd080fd8e0 msgr2=0x7fcd080ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.342+0000 7fcd0f600640 1 -- 172.21.15.46:0/594204015 shutdown_connections 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.342+0000 7fcd0f600640 1 -- 172.21.15.46:0/594204015 wait complete. 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.344+0000 7fcd0f600640 1 Processor -- start 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.345+0000 7fcd0f600640 1 -- start start 2024-09-10T07:48:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.345+0000 7fcd0f600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd08074510 0x7fcd0810dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.345+0000 7fcd0f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd08074e30 0x7fcd0810bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.345+0000 7fcd0f600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0810e440 0x7fcd0810c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.345+0000 7fcd0f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081021f0 con 0x7fcd08074e30 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.345+0000 7fcd0f600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08102070 con 0x7fcd0810e440 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.345+0000 7fcd0f600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08102370 con 0x7fcd08074510 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.345+0000 7fcd0cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd08074510 0x7fcd0810dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd08074510 0x7fcd0810dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38396/0 (socket says 172.21.15.46:38396) 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0cc00640 1 -- 172.21.15.46:0/3538202939 learned_addr learned my addr 172.21.15.46:0/3538202939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0d600640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0810e440 0x7fcd0810c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0cc00640 1 -- 172.21.15.46:0/3538202939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0810e440 msgr2=0x7fcd0810c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0cc00640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0810e440 0x7fcd0810c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0cc00640 1 -- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd08074e30 msgr2=0x7fcd0810bdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0cc00640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd08074e30 0x7fcd0810bdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0cc00640 1 -- 172.21.15.46:0/3538202939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0810cb70 con 0x7fcd08074510 2024-09-10T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.346+0000 7fcd0cc00640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd08074510 0x7fcd0810dcb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00d950 tx=0x7fccfc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.347+0000 7fcd05600640 1 -- 172.21.15.46:0/3538202939 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc017070 con 0x7fcd08074510 2024-09-10T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.347+0000 7fcd05600640 1 -- 172.21.15.46:0/3538202939 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccfc009c30 con 0x7fcd08074510 2024-09-10T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.347+0000 7fcd05600640 1 -- 172.21.15.46:0/3538202939 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00f6b0 con 0x7fcd08074510 2024-09-10T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.347+0000 7fcd0f600640 1 -- 172.21.15.46:0/3538202939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd08113af0 con 0x7fcd08074510 2024-09-10T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.347+0000 7fcd0f600640 1 -- 172.21.15.46:0/3538202939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd08114050 con 0x7fcd08074510 2024-09-10T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.351+0000 7fcd05600640 1 -- 172.21.15.46:0/3538202939 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fccfc0109d0 con 0x7fcd08074510 2024-09-10T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.351+0000 7fcd12008740 1 Processor -- start 2024-09-10T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.352+0000 7fcd12008740 1 -- start start 2024-09-10T07:48:46.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.352+0000 7fcd12008740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259dc70 0x55fad259e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd05600640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fccf00a2c00 0x7fccf00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd0d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259dc70 0x55fad259e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd0d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259dc70 0x55fad259e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43136/0 (socket says 172.21.15.46:43136) 2024-09-10T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd12008740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259e590 0x55fad25a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd12008740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 0x55fad25ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd12008740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fad25043e0 con 0x55fad259e590 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd12008740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fad24abd80 con 0x55fad259dc70 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd12008740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fad25003a0 con 0x55fad25a8e50 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd05600640 1 -- 172.21.15.46:0/3538202939 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fccfc0ceff0 con 0x7fcd08074510 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd0d600640 1 -- 172.21.15.46:0/2785116832 learned_addr learned my addr 172.21.15.46:0/2785116832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.353+0000 7fcd0cc00640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fccf00a2c00 0x7fccf00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fcd0d600640 1 -- 172.21.15.46:0/2785116832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 msgr2=0x55fad25ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fcd0d600640 1 --2- 172.21.15.46:0/2785116832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 0x55fad25ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fcd0d600640 1 -- 172.21.15.46:0/2785116832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259e590 msgr2=0x55fad25a8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fcd0d600640 1 --2- 172.21.15.46:0/2785116832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259e590 0x55fad25a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fcd0d600640 1 -- 172.21.15.46:0/2785116832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad24caae0 con 0x55fad259dc70 2024-09-10T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fcd0cc00640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fccf00a2c00 0x7fccf00a50c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0058c0 tx=0x7fccfc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fcd0d600640 1 --2- 172.21.15.46:0/2785116832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259dc70 0x55fad259e050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd00011490 tx=0x7fcd00011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcdc2801c026d55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fccf9c00640 1 -- 172.21.15.46:0/2785116832 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00004b20 con 0x55fad259dc70 2024-09-10T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fccf9c00640 1 -- 172.21.15.46:0/2785116832 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd00004cc0 con 0x55fad259dc70 2024-09-10T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.354+0000 7fccf9c00640 1 -- 172.21.15.46:0/2785116832 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000f4e0 con 0x55fad259dc70 2024-09-10T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.355+0000 7fcd12008740 1 -- 172.21.15.46:0/2785116832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259dc70 msgr2=0x55fad259e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.355+0000 7fcd12008740 1 --2- 172.21.15.46:0/2785116832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259dc70 0x55fad259e050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd00011490 tx=0x7fcd00011950 comp rx=0 tx=0).stop 2024-09-10T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.356+0000 7fcd12008740 1 -- 172.21.15.46:0/2785116832 shutdown_connections 2024-09-10T07:48:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.356+0000 7fcd12008740 1 --2- 172.21.15.46:0/2785116832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 0x55fad25ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.357+0000 7fcd12008740 1 --2- 172.21.15.46:0/2785116832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259e590 0x55fad25a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.357+0000 7fcd12008740 1 --2- 172.21.15.46:0/2785116832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259dc70 0x55fad259e050 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.357+0000 7fcd12008740 1 -- 172.21.15.46:0/2785116832 >> 172.21.15.46:0/2785116832 conn(0x55fad259a200 msgr2=0x55fad259a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.358+0000 7fcd12008740 1 -- 172.21.15.46:0/2785116832 shutdown_connections 2024-09-10T07:48:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.358+0000 7fcd12008740 1 -- 172.21.15.46:0/2785116832 wait complete. 2024-09-10T07:48:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 1 -- start start 2024-09-10T07:48:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 1 Processor -- start 2024-09-10T07:48:46.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259e590 0x55fad25a6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad25a8e50 0x55fad25a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fad25aba20 con 0x55fad259dc70 2024-09-10T07:48:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fad24bf0f0 con 0x55fad25a8e50 2024-09-10T07:48:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fad25abba0 con 0x55fad259e590 2024-09-10T07:48:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.359+0000 7fcd12008740 10 client.0 _ll_register_callbacks cb 0x55fad24a6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.360+0000 7fcd07e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259e590 0x55fad25a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.360+0000 7fcd07e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259e590 0x55fad25a6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38422/0 (socket says 172.21.15.46:38422) 2024-09-10T07:48:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.360+0000 7fcd07e00640 1 -- 172.21.15.46:0/1466644543 learned_addr learned my addr 172.21.15.46:0/1466644543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.361+0000 7fcd0d600640 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad25a8e50 0x55fad25a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.361+0000 7fcd0d600640 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.361+0000 7fcd07e00640 1 -- 172.21.15.46:0/1466644543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad25a8e50 msgr2=0x55fad25a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.361+0000 7fcd07e00640 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad25a8e50 0x55fad25a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.361+0000 7fcd07e00640 1 -- 172.21.15.46:0/1466644543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 msgr2=0x55fad25a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.361+0000 7fcd07e00640 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.361+0000 7fcd07e00640 1 -- 172.21.15.46:0/1466644543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad22753c0 con 0x55fad259e590 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.362+0000 7fcd07e00640 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259e590 0x55fad25a6220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fccf402a9a0 tx=0x7fccf4003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=472e851ee19265ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.362+0000 7fccee000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.362+0000 7fccee000640 1 -- 172.21.15.46:0/1466644543 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf403e070 con 0x55fad259e590 2024-09-10T07:48:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.363+0000 7fccee000640 1 -- 172.21.15.46:0/1466644543 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fccf4003dd0 con 0x55fad259e590 2024-09-10T07:48:46.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.363+0000 7fcd12008740 1 -- 172.21.15.46:0/1466644543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd00004cc0 con 0x55fad259e590 2024-09-10T07:48:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fccee000640 1 -- 172.21.15.46:0/1466644543 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fccf4048050 con 0x55fad259e590 2024-09-10T07:48:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fcd12008740 10 client.28321 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fcd12008740 1 -- 172.21.15.46:0/1466644543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55fad25b79f0 con 0x55fad259e590 2024-09-10T07:48:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fccee000640 1 -- 172.21.15.46:0/1466644543 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd00004cc0 con 0x55fad259e590 2024-09-10T07:48:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fccee000640 1 -- 172.21.15.46:0/1466644543 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7fccf40048c0 con 0x55fad259e590 2024-09-10T07:48:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fcd12008740 10 client.28321 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fcd12008740 10 client.28321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fcd12008740 20 client.28321 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.365+0000 7fcd12008740 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fad25b8eb0 0x55fad25bb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.366+0000 7fcd12008740 4 client.28321 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.366+0000 7fcd12008740 1 -- 172.21.15.46:0/1466644543 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad2511950 con 0x55fad25b8eb0 2024-09-10T07:48:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.366+0000 7fcd0d600640 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fad25b8eb0 0x55fad25bb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.368+0000 7fccee000640 10 client.28321 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.370+0000 7fcd0d600640 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fad25b8eb0 0x55fad25bb2c0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526e75f0da49c5d1 server_cookie=afd1e4503a83b0a9 in_seq=0 out_seq=0 2024-09-10T07:48:46.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.373+0000 7fccee000640 1 -- 172.21.15.46:0/1466644543 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fad2511950 con 0x55fad25b8eb0 2024-09-10T07:48:46.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.373+0000 7fccee000640 10 client.28321 handle_command_reply: tid=0 2024-09-10T07:48:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.373+0000 7fcd12008740 1 client.28321 shutdown 2024-09-10T07:48:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 -- 172.21.15.46:0/1466644543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259e590 msgr2=0x55fad25a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259e590 0x55fad25a6220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fccf402a9a0 tx=0x7fccf4003820 comp rx=0 tx=0).stop 2024-09-10T07:48:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 -- 172.21.15.46:0/1466644543 shutdown_connections 2024-09-10T07:48:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fad25b8eb0 0x55fad25bb2c0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad25a8e50 0x55fad25a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259e590 0x55fad25a6220 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 --2- 172.21.15.46:0/1466644543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 -- 172.21.15.46:0/1466644543 >> 172.21.15.46:0/1466644543 conn(0x55fad259a200 msgr2=0x55fad259ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fccee000640 0 client.28321 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.376+0000 7fcd12008740 1 -- 172.21.15.46:0/1466644543 shutdown_connections 2024-09-10T07:48:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.377+0000 7fcd12008740 1 -- 172.21.15.46:0/1466644543 wait complete. 2024-09-10T07:48:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.377+0000 7fcd12008740 20 client.28321 trim_cache size 0 max 16384 2024-09-10T07:48:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.398+0000 7fcd12008740 1 Processor -- start 2024-09-10T07:48:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.398+0000 7fcd12008740 1 -- start start 2024-09-10T07:48:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.398+0000 7fcd12008740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259dc70 0x55fad25a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.398+0000 7fcd12008740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.398+0000 7fcd12008740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad25a8e50 0x55fad25c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.398+0000 7fcd12008740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081bdbe0 con 0x55fad25a8e50 2024-09-10T07:48:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.398+0000 7fcd12008740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081bf720 con 0x55fad259e590 2024-09-10T07:48:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.398+0000 7fcd12008740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081be2b0 con 0x55fad259dc70 2024-09-10T07:48:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.399+0000 7fcd0d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259dc70 0x55fad25a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.399+0000 7fcd0d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259dc70 0x55fad25a65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38430/0 (socket says 172.21.15.46:38430) 2024-09-10T07:48:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.399+0000 7fcd0d600640 1 -- 172.21.15.46:0/1987432405 learned_addr learned my addr 172.21.15.46:0/1987432405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.399+0000 7fcd0d600640 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad25a8e50 0x55fad25c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.400+0000 7fcd07e00640 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.401+0000 7fcd07e00640 1 -- 172.21.15.46:0/1987432405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259dc70 msgr2=0x55fad25a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.401+0000 7fcd07e00640 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259dc70 0x55fad25a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.401+0000 7fcd07e00640 1 -- 172.21.15.46:0/1987432405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad25a8e50 msgr2=0x55fad25c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.402+0000 7fcd07e00640 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad25a8e50 0x55fad25c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.402+0000 7fcd07e00640 1 -- 172.21.15.46:0/1987432405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad259a030 con 0x55fad259e590 2024-09-10T07:48:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.403+0000 7fcd07e00640 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a7c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fccf4004d10 tx=0x7fccf4003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60eab54a1ed9274 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.403+0000 7fccf9200640 1 -- 172.21.15.46:0/1987432405 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf403e070 con 0x55fad259e590 2024-09-10T07:48:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.403+0000 7fccf9200640 1 -- 172.21.15.46:0/1987432405 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf4009520 con 0x55fad259e590 2024-09-10T07:48:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.403+0000 7fccf9200640 1 -- 172.21.15.46:0/1987432405 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4035040 con 0x55fad259e590 2024-09-10T07:48:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 -- 172.21.15.46:0/1987432405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 msgr2=0x55fad25a7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a7c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fccf4004d10 tx=0x7fccf4003910 comp rx=0 tx=0).stop 2024-09-10T07:48:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 -- 172.21.15.46:0/1987432405 shutdown_connections 2024-09-10T07:48:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad25a8e50 0x55fad25c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a7c50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 --2- 172.21.15.46:0/1987432405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad259dc70 0x55fad25a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 -- 172.21.15.46:0/1987432405 >> 172.21.15.46:0/1987432405 conn(0x55fad259a200 msgr2=0x55fad25a7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 -- 172.21.15.46:0/1987432405 shutdown_connections 2024-09-10T07:48:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.405+0000 7fcd12008740 1 -- 172.21.15.46:0/1987432405 wait complete. 2024-09-10T07:48:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.406+0000 7fcd12008740 1 -- start start 2024-09-10T07:48:46.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.406+0000 7fcd12008740 1 Processor -- start 2024-09-10T07:48:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.406+0000 7fcd12008740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.406+0000 7fcd12008740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd12008740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 0x55fad25a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd12008740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081bf050 con 0x55fad259dc70 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd12008740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081be980 con 0x55fad259e590 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd12008740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0811be30 con 0x55fad25a8e50 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd12008740 10 client.0 _ll_register_callbacks cb 0x7fcd000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd07e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd07e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43172/0 (socket says 172.21.15.46:43172) 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd07e00640 1 -- 172.21.15.46:0/3122567504 learned_addr learned my addr 172.21.15.46:0/3122567504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd0d600640 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd0d600640 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 0x55fad25a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd07e00640 1 -- 172.21.15.46:0/3122567504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 msgr2=0x55fad25a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd07e00640 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 0x55fad25a2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd07e00640 1 -- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 msgr2=0x55fad25b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd07e00640 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd07e00640 1 -- 172.21.15.46:0/3122567504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad259adb0 con 0x55fad259e590 2024-09-10T07:48:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.407+0000 7fcd0d600640 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.408+0000 7fcd07e00640 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a29b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009850 tx=0x7fccf4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bebbe06bc628f40e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.408+0000 7fccee000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.408+0000 7fccee000640 1 -- 172.21.15.46:0/3122567504 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4035070 con 0x55fad259e590 2024-09-10T07:48:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.408+0000 7fccee000640 1 -- 172.21.15.46:0/3122567504 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fccf40048c0 con 0x55fad259e590 2024-09-10T07:48:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.408+0000 7fcd12008740 1 -- 172.21.15.46:0/3122567504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccf4009520 con 0x55fad259e590 2024-09-10T07:48:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.408+0000 7fccee000640 1 -- 172.21.15.46:0/3122567504 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fccf4064050 con 0x55fad259e590 2024-09-10T07:48:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.408+0000 7fcd12008740 10 client.28385 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.408+0000 7fcd12008740 1 -- 172.21.15.46:0/3122567504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55fad2595770 con 0x55fad259e590 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.409+0000 7fccee000640 1 -- 172.21.15.46:0/3122567504 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf4009520 con 0x55fad259e590 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.409+0000 7fccee000640 1 -- 172.21.15.46:0/3122567504 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7fccf4005320 con 0x55fad259e590 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.409+0000 7fcd12008740 10 client.28385 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.409+0000 7fcd12008740 10 client.28385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.409+0000 7fcd12008740 20 client.28385 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.409+0000 7fcd12008740 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fad2623e60 0x55fad2626280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.409+0000 7fcd12008740 4 client.28385 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.409+0000 7fcd12008740 1 -- 172.21.15.46:0/3122567504 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fad250be60 con 0x55fad2623e60 2024-09-10T07:48:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.410+0000 7fcd0d600640 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fad2623e60 0x55fad2626280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.410+0000 7fcd0d600640 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fad2623e60 0x55fad2626280 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1d26ae35398dfe server_cookie=b38b1f4417399b1 in_seq=0 out_seq=0 2024-09-10T07:48:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.411+0000 7fccee000640 10 client.28385 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.412+0000 7fccee000640 1 -- 172.21.15.46:0/3122567504 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fad250be60 con 0x55fad2623e60 2024-09-10T07:48:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.412+0000 7fccee000640 10 client.28385 handle_command_reply: tid=0 2024-09-10T07:48:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.412+0000 7fcd12008740 1 client.28385 shutdown 2024-09-10T07:48:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fcd12008740 1 -- 172.21.15.46:0/3122567504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 msgr2=0x55fad25a29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fcd12008740 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a29b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009850 tx=0x7fccf4005fd0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fcd12008740 1 -- 172.21.15.46:0/3122567504 shutdown_connections 2024-09-10T07:48:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fcd12008740 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fad2623e60 0x55fad2626280 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fcd12008740 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fad25a8e50 0x55fad25a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fcd12008740 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fad259e590 0x55fad25a29b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fcd12008740 1 --2- 172.21.15.46:0/3122567504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad259dc70 0x55fad25b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fccee000640 0 client.28385 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.414+0000 7fcd12008740 1 -- 172.21.15.46:0/3122567504 >> 172.21.15.46:0/3122567504 conn(0x55fad259a200 msgr2=0x55fad25a7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:46.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.415+0000 7fcd12008740 1 -- 172.21.15.46:0/3122567504 shutdown_connections 2024-09-10T07:48:46.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.415+0000 7fcd12008740 1 -- 172.21.15.46:0/3122567504 wait complete. 2024-09-10T07:48:46.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.415+0000 7fcd12008740 20 client.28385 trim_cache size 0 max 16384 2024-09-10T07:48:46.416 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:48:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.416+0000 7fcd0f600640 1 -- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fccf00a2c00 msgr2=0x7fccf00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.416+0000 7fcd0f600640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fccf00a2c00 0x7fccf00a50c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0058c0 tx=0x7fccfc005f90 comp rx=0 tx=0).stop 2024-09-10T07:48:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.416+0000 7fcd0f600640 1 -- 172.21.15.46:0/3538202939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd08074510 msgr2=0x7fcd0810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.416+0000 7fcd0f600640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd08074510 0x7fcd0810dcb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00d950 tx=0x7fccfc00de10 comp rx=0 tx=0).stop 2024-09-10T07:48:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.417+0000 7fcd0f600640 1 -- 172.21.15.46:0/3538202939 shutdown_connections 2024-09-10T07:48:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.417+0000 7fcd0f600640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0810e440 0x7fcd0810c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.417+0000 7fcd0f600640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fccf00a2c00 0x7fccf00a50c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.417+0000 7fcd0f600640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd08074e30 0x7fcd0810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.417+0000 7fcd0f600640 1 --2- 172.21.15.46:0/3538202939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd08074510 0x7fcd0810dcb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.417+0000 7fcd0f600640 1 -- 172.21.15.46:0/3538202939 >> 172.21.15.46:0/3538202939 conn(0x7fcd080fd8e0 msgr2=0x7fcd080ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.417+0000 7fcd0f600640 1 -- 172.21.15.46:0/3538202939 shutdown_connections 2024-09-10T07:48:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:46.418+0000 7fcd0f600640 1 -- 172.21.15.46:0/3538202939 wait complete. 2024-09-10T07:48:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:46 smithi181 ceph-mon[35397]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 440 op/s 2024-09-10T07:48:46.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:46 smithi063 ceph-mon[35020]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 440 op/s 2024-09-10T07:48:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.620+0000 7f1d9b000640 1 -- 172.21.15.46:0/1158984362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 msgr2=0x7f1d9406e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.620+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1158984362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 0x7f1d9406e8d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009930 tx=0x7f1d8802a440 comp rx=0 tx=0).stop 2024-09-10T07:48:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.622+0000 7f1d9b000640 1 -- 172.21.15.46:0/1158984362 shutdown_connections 2024-09-10T07:48:47.623 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:47 smithi046 ceph-mon[30705]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.7 MiB/s wr, 635 op/s 2024-09-10T07:48:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.622+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1158984362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d94114ab0 0x7f1d94116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.622+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1158984362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d9406ee10 0x7f1d94114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.622+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1158984362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 0x7f1d9406e8d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.622+0000 7f1d9b000640 1 -- 172.21.15.46:0/1158984362 >> 172.21.15.46:0/1158984362 conn(0x7f1d94073180 msgr2=0x7f1d94073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.622+0000 7f1d9b000640 1 -- 172.21.15.46:0/1158984362 shutdown_connections 2024-09-10T07:48:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.623+0000 7f1d9b000640 1 -- 172.21.15.46:0/1158984362 wait complete. 2024-09-10T07:48:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.624+0000 7f1d9b000640 1 Processor -- start 2024-09-10T07:48:47.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.625+0000 7f1d9b000640 1 -- start start 2024-09-10T07:48:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.627+0000 7f1d9b000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 0x7f1d94114500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.627+0000 7f1d9b000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d9406ee10 0x7f1d94111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.627+0000 7f1d9b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d94114ab0 0x7f1d94111d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.628+0000 7f1d99200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d9406ee10 0x7f1d94111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.628+0000 7f1d99200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d9406ee10 0x7f1d94111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38456/0 (socket says 172.21.15.46:38456) 2024-09-10T07:48:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.628+0000 7f1d99c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 0x7f1d94114500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.628+0000 7f1d99c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 0x7f1d94114500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43188/0 (socket says 172.21.15.46:43188) 2024-09-10T07:48:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.628+0000 7f1d99200640 1 -- 172.21.15.46:0/1480776983 learned_addr learned my addr 172.21.15.46:0/1480776983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.629+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9411c7f0 con 0x7f1d94114ab0 2024-09-10T07:48:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.629+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9411c670 con 0x7f1d9406e4f0 2024-09-10T07:48:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.629+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9411c970 con 0x7f1d9406ee10 2024-09-10T07:48:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.629+0000 7f1d9a600640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d94114ab0 0x7f1d94111d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.630+0000 7f1d99c00640 1 -- 172.21.15.46:0/1480776983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d9406ee10 msgr2=0x7f1d94111600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.630+0000 7f1d99c00640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d9406ee10 0x7f1d94111600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.630+0000 7f1d99c00640 1 -- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d94114ab0 msgr2=0x7f1d94111d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.630+0000 7f1d99c00640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d94114ab0 0x7f1d94111d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.630+0000 7f1d99c00640 1 -- 172.21.15.46:0/1480776983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d941b34d0 con 0x7f1d9406e4f0 2024-09-10T07:48:47.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.630+0000 7f1d99200640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d9406ee10 0x7f1d94111600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:47.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.630+0000 7f1d99c00640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 0x7f1d94114500 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009850 tx=0x7f1d88003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:47.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.631+0000 7f1d92000640 1 -- 172.21.15.46:0/1480776983 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8803f070 con 0x7f1d9406e4f0 2024-09-10T07:48:47.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.631+0000 7f1d92000640 1 -- 172.21.15.46:0/1480776983 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d88031c30 con 0x7f1d9406e4f0 2024-09-10T07:48:47.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.631+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d941b3700 con 0x7f1d9406e4f0 2024-09-10T07:48:47.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.631+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d941b3c40 con 0x7f1d9406e4f0 2024-09-10T07:48:47.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.631+0000 7f1d92000640 1 -- 172.21.15.46:0/1480776983 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88035030 con 0x7f1d9406e4f0 2024-09-10T07:48:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.634+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9406e8d0 con 0x7f1d9406e4f0 2024-09-10T07:48:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.637+0000 7f1d92000640 1 -- 172.21.15.46:0/1480776983 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d88042070 con 0x7f1d9406e4f0 2024-09-10T07:48:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.639+0000 7f1d92000640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d7c0a2c00 0x7f1d7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.639+0000 7f1d92000640 1 -- 172.21.15.46:0/1480776983 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f1d880efd70 con 0x7f1d9406e4f0 2024-09-10T07:48:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.640+0000 7f1d99200640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d7c0a2c00 0x7f1d7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.646+0000 7f1d99200640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d7c0a2c00 0x7f1d7c0a50c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1d9406dc70 tx=0x7f1d8000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:47.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.646+0000 7f1d92000640 1 -- 172.21.15.46:0/1480776983 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d880f4070 con 0x7f1d9406e4f0 2024-09-10T07:48:47.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:47 smithi181 ceph-mon[35397]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.7 MiB/s wr, 635 op/s 2024-09-10T07:48:47.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:47 smithi063 ceph-mon[35020]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.7 MiB/s wr, 635 op/s 2024-09-10T07:48:47.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.982+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d941b3fb0 con 0x7f1d9406e4f0 2024-09-10T07:48:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.983+0000 7f1d92000640 1 -- 172.21.15.46:0/1480776983 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11973 (secure 0 0 0) 0x7f1d880bc2a0 con 0x7f1d9406e4f0 2024-09-10T07:48:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:48:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 51 2024-09-10T07:48:47.985 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"btime":"2024-09-10T07:48:38:221420+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:48:37.235788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:48:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.986+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d7c0a2c00 msgr2=0x7f1d7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.986+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d7c0a2c00 0x7f1d7c0a50c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1d9406dc70 tx=0x7f1d8000b4a0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.986+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 msgr2=0x7f1d94114500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.986+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 0x7f1d94114500 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009850 tx=0x7f1d88003c10 comp rx=0 tx=0).stop 2024-09-10T07:48:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.987+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 shutdown_connections 2024-09-10T07:48:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.987+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f1d7c0a2c00 0x7f1d7c0a50c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.987+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d94114ab0 0x7f1d94111d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.987+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d9406ee10 0x7f1d94111600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.987+0000 7f1d9b000640 1 --2- 172.21.15.46:0/1480776983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d9406e4f0 0x7f1d94114500 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.987+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 >> 172.21.15.46:0/1480776983 conn(0x7f1d94073180 msgr2=0x7f1d9406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.987+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 shutdown_connections 2024-09-10T07:48:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:47.987+0000 7f1d9b000640 1 -- 172.21.15.46:0/1480776983 wait complete. 2024-09-10T07:48:48.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:48 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1480776983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:48:48.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:48 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1480776983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:48:48.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:48 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1480776983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:48:49.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:49 smithi181 ceph-mon[35397]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 459 op/s 2024-09-10T07:48:49.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:49 smithi063 ceph-mon[35020]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 459 op/s 2024-09-10T07:48:49.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:49 smithi046 ceph-mon[30705]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 459 op/s 2024-09-10T07:48:50.148 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:48:50.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:48:50.394 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0' 2024-09-10T07:48:50.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0 2024-09-10T07:48:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 690.0 s, 490.8 tps, lat 202.355 ms stddev 261.878 2024-09-10T07:48:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:52.134 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:51 smithi046 ceph-mon[30705]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-09-10T07:48:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.307+0000 7f3b41600640 1 -- 172.21.15.46:0/3584162494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c06ea30 msgr2=0x7f3b3c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.307+0000 7f3b41600640 1 --2- 172.21.15.46:0/3584162494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c06ea30 0x7f3b3c06ee70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009be0 tx=0x7f3b2402a6d0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.307+0000 7f3b41600640 1 -- 172.21.15.46:0/3584162494 shutdown_connections 2024-09-10T07:48:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.307+0000 7f3b41600640 1 --2- 172.21.15.46:0/3584162494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b3c11b140 0x7f3b3c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.307+0000 7f3b41600640 1 --2- 172.21.15.46:0/3584162494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c06ea30 0x7f3b3c06ee70 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009be0 tx=0x7f3b2402a6d0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.307+0000 7f3b41600640 1 --2- 172.21.15.46:0/3584162494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c110190 0x7f3b3c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.307+0000 7f3b41600640 1 -- 172.21.15.46:0/3584162494 >> 172.21.15.46:0/3584162494 conn(0x7f3b3c073180 msgr2=0x7f3b3c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.308+0000 7f3b41600640 1 -- 172.21.15.46:0/3584162494 shutdown_connections 2024-09-10T07:48:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.308+0000 7f3b41600640 1 -- 172.21.15.46:0/3584162494 wait complete. 2024-09-10T07:48:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.309+0000 7f3b41600640 1 Processor -- start 2024-09-10T07:48:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.310+0000 7f3b41600640 1 -- start start 2024-09-10T07:48:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.311+0000 7f3b41600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b3c110190 0x7f3b3c1a7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.312+0000 7f3b41600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c11b140 0x7f3b3c1a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.312+0000 7f3b41600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c1abc00 0x7f3b3c1ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.312+0000 7f3b41600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c120b90 con 0x7f3b3c110190 2024-09-10T07:48:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.312+0000 7f3b41600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c120a10 con 0x7f3b3c11b140 2024-09-10T07:48:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.312+0000 7f3b41600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c120d10 con 0x7f3b3c1abc00 2024-09-10T07:48:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.312+0000 7f3b33400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c11b140 0x7f3b3c1a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.312+0000 7f3b33400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c11b140 0x7f3b3c1a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43204/0 (socket says 172.21.15.46:43204) 2024-09-10T07:48:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.312+0000 7f3b33400640 1 -- 172.21.15.46:0/3889968815 learned_addr learned my addr 172.21.15.46:0/3889968815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.313+0000 7f3b40c00640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c1abc00 0x7f3b3c1ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.313+0000 7f3b3be00640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b3c110190 0x7f3b3c1a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.314+0000 7f3b40c00640 1 -- 172.21.15.46:0/3889968815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c11b140 msgr2=0x7f3b3c1a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.314+0000 7f3b40c00640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c11b140 0x7f3b3c1a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.314+0000 7f3b40c00640 1 -- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b3c110190 msgr2=0x7f3b3c1a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.314+0000 7f3b40c00640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b3c110190 0x7f3b3c1a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.314+0000 7f3b40c00640 1 -- 172.21.15.46:0/3889968815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3c1ac6c0 con 0x7f3b3c1abc00 2024-09-10T07:48:52.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.315+0000 7f3b33400640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c11b140 0x7f3b3c1a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.317+0000 7f3b40c00640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c1abc00 0x7f3b3c1ac0b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400e710 tx=0x7f3b3400ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.317+0000 7f3b39600640 1 -- 172.21.15.46:0/3889968815 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34004220 con 0x7f3b3c1abc00 2024-09-10T07:48:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.317+0000 7f3b39600640 1 -- 172.21.15.46:0/3889968815 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3400cd90 con 0x7f3b3c1abc00 2024-09-10T07:48:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.318+0000 7f3b39600640 1 -- 172.21.15.46:0/3889968815 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34004a00 con 0x7f3b3c1abc00 2024-09-10T07:48:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.318+0000 7f3b41600640 1 -- 172.21.15.46:0/3889968815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3c1ac9b0 con 0x7f3b3c1abc00 2024-09-10T07:48:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.319+0000 7f3b41600640 1 -- 172.21.15.46:0/3889968815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3c1b4140 con 0x7f3b3c1abc00 2024-09-10T07:48:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.349+0000 7f3b39600640 1 -- 172.21.15.46:0/3889968815 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b34005020 con 0x7f3b3c1abc00 2024-09-10T07:48:52.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.351+0000 7f3b39600640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b1c0a2b80 0x7f3b1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.352+0000 7f3b455e1740 1 Processor -- start 2024-09-10T07:48:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.352+0000 7f3b455e1740 1 -- start start 2024-09-10T07:48:52.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.352+0000 7f3b3be00640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b1c0a2b80 0x7f3b1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.352+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 0x5618df222090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.353+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df22c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.353+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df22f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.353+0000 7f3b455e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5618df188290 con 0x5618df2225d0 2024-09-10T07:48:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.353+0000 7f3b455e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5618df12fb40 con 0x5618df221cb0 2024-09-10T07:48:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.353+0000 7f3b455e1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5618df184250 con 0x5618df22ce90 2024-09-10T07:48:52.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.353+0000 7f3b40c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df22f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.353+0000 7f3b40c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df22f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38482/0 (socket says 172.21.15.46:38482) 2024-09-10T07:48:52.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.353+0000 7f3b40c00640 1 -- 172.21.15.46:0/3770172738 learned_addr learned my addr 172.21.15.46:0/3770172738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:52.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.354+0000 7f3b3be00640 1 --2- 172.21.15.46:0/3770172738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 0x5618df222090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.355+0000 7f3b40c00640 1 -- 172.21.15.46:0/3770172738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 msgr2=0x5618df222090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:52.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.355+0000 7f3b40c00640 1 --2- 172.21.15.46:0/3770172738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 0x5618df222090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.355+0000 7f3b40c00640 1 -- 172.21.15.46:0/3770172738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 msgr2=0x5618df22c950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:48:52.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.355+0000 7f3b40c00640 1 --2- 172.21.15.46:0/3770172738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df22c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.355+0000 7f3b40c00640 1 -- 172.21.15.46:0/3770172738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618deef96a0 con 0x5618df22ce90 2024-09-10T07:48:52.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.358+0000 7f3b3be00640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b1c0a2b80 0x7f3b1c0a5040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005fd0 tx=0x7f3b2c009a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.359+0000 7f3b39600640 1 -- 172.21.15.46:0/3889968815 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b34099e60 con 0x7f3b3c1abc00 2024-09-10T07:48:52.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.359+0000 7f3b40c00640 1 --2- 172.21.15.46:0/3770172738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df22f280 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400c6a0 tx=0x7f3b340a3480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24883dd94708947f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.359+0000 7f3b31200640 1 -- 172.21.15.46:0/3770172738 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340987f0 con 0x5618df22ce90 2024-09-10T07:48:52.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.360+0000 7f3b31200640 1 -- 172.21.15.46:0/3770172738 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b34016ec0 con 0x5618df22ce90 2024-09-10T07:48:52.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.361+0000 7f3b455e1740 1 -- 172.21.15.46:0/3770172738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 msgr2=0x5618df22f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.361+0000 7f3b455e1740 1 --2- 172.21.15.46:0/3770172738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df22f280 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400c6a0 tx=0x7f3b340a3480 comp rx=0 tx=0).stop 2024-09-10T07:48:52.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.363+0000 7f3b455e1740 1 -- 172.21.15.46:0/3770172738 shutdown_connections 2024-09-10T07:48:52.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.363+0000 7f3b455e1740 1 --2- 172.21.15.46:0/3770172738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df22f280 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.363+0000 7f3b455e1740 1 --2- 172.21.15.46:0/3770172738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df22c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.363+0000 7f3b455e1740 1 --2- 172.21.15.46:0/3770172738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 0x5618df222090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.363+0000 7f3b455e1740 1 -- 172.21.15.46:0/3770172738 >> 172.21.15.46:0/3770172738 conn(0x5618df21e240 msgr2=0x5618df21e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.364+0000 7f3b455e1740 1 -- 172.21.15.46:0/3770172738 shutdown_connections 2024-09-10T07:48:52.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.364+0000 7f3b455e1740 1 -- 172.21.15.46:0/3770172738 wait complete. 2024-09-10T07:48:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.365+0000 7f3b455e1740 1 -- start start 2024-09-10T07:48:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.365+0000 7f3b455e1740 1 Processor -- start 2024-09-10T07:48:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.368+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 0x5618df22c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.369+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df2299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.371+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df229f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.371+0000 7f3b455e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5618df22fac0 con 0x5618df2225d0 2024-09-10T07:48:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.371+0000 7f3b40c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df229f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.371+0000 7f3b40c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df229f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38496/0 (socket says 172.21.15.46:38496) 2024-09-10T07:48:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.371+0000 7f3b40c00640 1 -- 172.21.15.46:0/2969063689 learned_addr learned my addr 172.21.15.46:0/2969063689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.371+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 0x5618df22c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.372+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5618df142fa0 con 0x5618df221cb0 2024-09-10T07:48:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.372+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5618df22fc40 con 0x5618df22ce90 2024-09-10T07:48:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.372+0000 7f3b40c00640 1 -- 172.21.15.46:0/2969063689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 msgr2=0x5618df22c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.372+0000 7f3b40c00640 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 0x5618df22c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.372+0000 7f3b40c00640 1 -- 172.21.15.46:0/2969063689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 msgr2=0x5618df2299d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.372+0000 7f3b40c00640 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df2299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.372+0000 7f3b40c00640 1 -- 172.21.15.46:0/2969063689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618df22a6f0 con 0x5618df22ce90 2024-09-10T07:48:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.373+0000 7f3b455e1740 10 client.0 _ll_register_callbacks cb 0x5618dee96ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.373+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b34016ec0 con 0x5618df22ce90 2024-09-10T07:48:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.374+0000 7f3b29600640 10 client.28339 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.374+0000 7f3b40c00640 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df229f10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5618df222c70 tx=0x7f3b340cd380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2237b8aae79b472e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.374+0000 7f3b29600640 1 -- 172.21.15.46:0/2969063689 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340a1050 con 0x5618df22ce90 2024-09-10T07:48:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.375+0000 7f3b29600640 1 -- 172.21.15.46:0/2969063689 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b340a49d0 con 0x5618df22ce90 2024-09-10T07:48:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.375+0000 7f3b29600640 1 -- 172.21.15.46:0/2969063689 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3b340a1050 con 0x5618df22ce90 2024-09-10T07:48:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.375+0000 7f3b455e1740 10 client.28339 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.375+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=51}) -- 0x5618df24c620 con 0x5618df22ce90 2024-09-10T07:48:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.376+0000 7f3b29600640 1 -- 172.21.15.46:0/2969063689 <== mon.1 v2:172.21.15.181:3300/0 4 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7f3b3401f070 con 0x5618df22ce90 2024-09-10T07:48:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.376+0000 7f3b455e1740 10 client.28339 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.376+0000 7f3b455e1740 10 client.28339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.376+0000 7f3b455e1740 20 client.28339 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.376+0000 7f3b455e1740 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5618df24d970 0x5618df24fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.377+0000 7f3b455e1740 4 client.28339 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.377+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618df195800 con 0x5618df24d970 2024-09-10T07:48:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.379+0000 7f3b3be00640 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5618df24d970 0x5618df24fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.380+0000 7f3b3be00640 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5618df24d970 0x5618df24fd60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10dbcbdad8da5140 server_cookie=baa1ee56cfcac75 in_seq=0 out_seq=0 2024-09-10T07:48:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.380+0000 7f3b29600640 10 client.28339 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.382+0000 7f3b29600640 1 -- 172.21.15.46:0/2969063689 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618df195800 con 0x5618df24d970 2024-09-10T07:48:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.382+0000 7f3b29600640 10 client.28339 handle_command_reply: tid=0 2024-09-10T07:48:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.382+0000 7f3b455e1740 1 client.28339 shutdown 2024-09-10T07:48:52.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.382+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 msgr2=0x5618df229f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.382+0000 7f3b455e1740 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df229f10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5618df222c70 tx=0x7f3b340cd380 comp rx=0 tx=0).stop 2024-09-10T07:48:52.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.382+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 shutdown_connections 2024-09-10T07:48:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.382+0000 7f3b455e1740 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5618df24d970 0x5618df24fd60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.383+0000 7f3b29600640 0 client.28339 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.383+0000 7f3b455e1740 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df22ce90 0x5618df229f10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.383+0000 7f3b455e1740 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df2299d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.383+0000 7f3b455e1740 1 --2- 172.21.15.46:0/2969063689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df221cb0 0x5618df22c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.383+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 >> 172.21.15.46:0/2969063689 conn(0x5618df21e240 msgr2=0x5618df223f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.383+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 shutdown_connections 2024-09-10T07:48:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.384+0000 7f3b455e1740 1 -- 172.21.15.46:0/2969063689 wait complete. 2024-09-10T07:48:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.384+0000 7f3b455e1740 20 client.28339 trim_cache size 0 max 16384 2024-09-10T07:48:52.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:51 smithi181 ceph-mon[35397]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-09-10T07:48:52.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:51 smithi063 ceph-mon[35020]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-09-10T07:48:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.429+0000 7f3b455e1740 1 Processor -- start 2024-09-10T07:48:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.429+0000 7f3b455e1740 1 -- start start 2024-09-10T07:48:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.430+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df255390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.430+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df2225d0 0x5618df2558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.430+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df22ce90 0x5618df226c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.430+0000 7f3b455e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b5410 con 0x5618df22ce90 2024-09-10T07:48:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.430+0000 7f3b455e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b6f50 con 0x5618df2225d0 2024-09-10T07:48:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.430+0000 7f3b455e1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b5ae0 con 0x5618df221cb0 2024-09-10T07:48:52.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.431+0000 7f3b33400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df2225d0 0x5618df2558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.431+0000 7f3b33400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df2225d0 0x5618df2558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43248/0 (socket says 172.21.15.46:43248) 2024-09-10T07:48:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.431+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df255390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.431+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df255390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38500/0 (socket says 172.21.15.46:38500) 2024-09-10T07:48:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.431+0000 7f3b3be00640 1 -- 172.21.15.46:0/595185982 learned_addr learned my addr 172.21.15.46:0/595185982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.432+0000 7f3b3be00640 1 -- 172.21.15.46:0/595185982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df2225d0 msgr2=0x5618df2558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.432+0000 7f3b3be00640 1 --2- 172.21.15.46:0/595185982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df2225d0 0x5618df2558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.432+0000 7f3b3be00640 1 -- 172.21.15.46:0/595185982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df22ce90 msgr2=0x5618df226c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.432+0000 7f3b3be00640 1 --2- 172.21.15.46:0/595185982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df22ce90 0x5618df226c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.432+0000 7f3b3be00640 1 -- 172.21.15.46:0/595185982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618df22f5d0 con 0x5618df221cb0 2024-09-10T07:48:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.433+0000 7f3b3be00640 1 --2- 172.21.15.46:0/595185982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df255390 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0131d0 tx=0x7f3b2c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24262a30fa9fed8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.434+0000 7f3b2be00640 1 -- 172.21.15.46:0/595185982 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00fe60 con 0x5618df221cb0 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.434+0000 7f3b2be00640 1 -- 172.21.15.46:0/595185982 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c011b50 con 0x5618df221cb0 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.435+0000 7f3b455e1740 1 -- 172.21.15.46:0/595185982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 msgr2=0x5618df255390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.435+0000 7f3b2be00640 1 -- 172.21.15.46:0/595185982 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c020b20 con 0x5618df221cb0 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.435+0000 7f3b455e1740 1 --2- 172.21.15.46:0/595185982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df255390 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0131d0 tx=0x7f3b2c009260 comp rx=0 tx=0).stop 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.436+0000 7f3b455e1740 1 -- 172.21.15.46:0/595185982 shutdown_connections 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.436+0000 7f3b455e1740 1 --2- 172.21.15.46:0/595185982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df22ce90 0x5618df226c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.436+0000 7f3b455e1740 1 --2- 172.21.15.46:0/595185982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df2225d0 0x5618df2558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.436+0000 7f3b455e1740 1 --2- 172.21.15.46:0/595185982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df255390 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.436+0000 7f3b455e1740 1 -- 172.21.15.46:0/595185982 >> 172.21.15.46:0/595185982 conn(0x5618df21e240 msgr2=0x5618df22ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.437+0000 7f3b455e1740 1 -- 172.21.15.46:0/595185982 shutdown_connections 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.437+0000 7f3b455e1740 1 -- 172.21.15.46:0/595185982 wait complete. 2024-09-10T07:48:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.438+0000 7f3b455e1740 1 -- start start 2024-09-10T07:48:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.438+0000 7f3b455e1740 1 Processor -- start 2024-09-10T07:48:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.444+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df231590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.444+0000 7f3b455e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df231ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.445+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df231590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.445+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df231590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38516/0 (socket says 172.21.15.46:38516) 2024-09-10T07:48:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.445+0000 7f3b3be00640 1 -- 172.21.15.46:0/1873052553 learned_addr learned my addr 172.21.15.46:0/1873052553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.445+0000 7f3b455e1740 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df22ce90 0x5618df2b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.445+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b6880 con 0x5618df2225d0 2024-09-10T07:48:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.445+0000 7f3b40c00640 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df22ce90 0x5618df2b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.445+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1b61b0 con 0x5618df22ce90 2024-09-10T07:48:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.445+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c118210 con 0x5618df221cb0 2024-09-10T07:48:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.446+0000 7f3b33400640 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df231ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.446+0000 7f3b3be00640 1 -- 172.21.15.46:0/1873052553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df22ce90 msgr2=0x5618df2b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.446+0000 7f3b3be00640 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df22ce90 0x5618df2b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.446+0000 7f3b3be00640 1 -- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 msgr2=0x5618df231ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.446+0000 7f3b3be00640 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df231ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.446+0000 7f3b3be00640 1 -- 172.21.15.46:0/1873052553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618df2b1a90 con 0x5618df221cb0 2024-09-10T07:48:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.447+0000 7f3b33400640 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df231ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.447+0000 7f3b3be00640 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df231590 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c011fb0 tx=0x7f3b2c00fe80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ad5d3daed3a148c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.447+0000 7f3b455e1740 10 client.0 _ll_register_callbacks cb 0x7f3b34067100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.447+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2c011b50 con 0x5618df221cb0 2024-09-10T07:48:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.447+0000 7f3b29600640 10 client.28345 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:48:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.447+0000 7f3b29600640 1 -- 172.21.15.46:0/1873052553 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c040050 con 0x5618df221cb0 2024-09-10T07:48:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.449+0000 7f3b29600640 1 -- 172.21.15.46:0/1873052553 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b2c011670 con 0x5618df221cb0 2024-09-10T07:48:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.449+0000 7f3b29600640 1 -- 172.21.15.46:0/1873052553 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3b2c036070 con 0x5618df221cb0 2024-09-10T07:48:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.449+0000 7f3b455e1740 10 client.28345 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.449+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=51}) -- 0x5618df2b8cb0 con 0x5618df221cb0 2024-09-10T07:48:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.450+0000 7f3b29600640 1 -- 172.21.15.46:0/1873052553 <== mon.1 v2:172.21.15.181:3300/0 4 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7f3b2c027030 con 0x5618df221cb0 2024-09-10T07:48:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.450+0000 7f3b455e1740 10 client.28345 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.450+0000 7f3b455e1740 10 client.28345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.450+0000 7f3b455e1740 20 client.28345 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.450+0000 7f3b455e1740 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5618df2b92c0 0x5618df2bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.450+0000 7f3b455e1740 4 client.28345 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:48:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.450+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5618df18fd10 con 0x5618df2b92c0 2024-09-10T07:48:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.452+0000 7f3b33400640 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5618df2b92c0 0x5618df2bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.454+0000 7f3b33400640 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5618df2b92c0 0x5618df2bb6b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785f9b6bb82cf4f0 server_cookie=90b7030a7b77a085 in_seq=0 out_seq=0 2024-09-10T07:48:52.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.454+0000 7f3b29600640 10 client.28345 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.458+0000 7f3b29600640 1 -- 172.21.15.46:0/1873052553 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618df18fd10 con 0x5618df2b92c0 2024-09-10T07:48:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.458+0000 7f3b29600640 10 client.28345 handle_command_reply: tid=0 2024-09-10T07:48:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.458+0000 7f3b455e1740 1 client.28345 shutdown 2024-09-10T07:48:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 msgr2=0x5618df231590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df231590 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c011fb0 tx=0x7f3b2c00fe80 comp rx=0 tx=0).stop 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 shutdown_connections 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5618df2b92c0 0x5618df2bb6b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618df22ce90 0x5618df2b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5618df2225d0 0x5618df231ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 --2- 172.21.15.46:0/1873052553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618df221cb0 0x5618df231590 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 >> 172.21.15.46:0/1873052553 conn(0x5618df21e240 msgr2=0x5618df223700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b29600640 0 client.28345 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 shutdown_connections 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.459+0000 7f3b455e1740 1 -- 172.21.15.46:0/1873052553 wait complete. 2024-09-10T07:48:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.460+0000 7f3b455e1740 20 client.28345 trim_cache size 0 max 16384 2024-09-10T07:48:52.461 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:48:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:48:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6c436a4d-3cf8-4264-9954-ee89c5dbbc79", 2024-09-10T07:48:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:48:52.461 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:48:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 -- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b1c0a2b80 msgr2=0x7f3b1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b1c0a2b80 0x7f3b1c0a5040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005fd0 tx=0x7f3b2c009a80 comp rx=0 tx=0).stop 2024-09-10T07:48:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 -- 172.21.15.46:0/3889968815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c1abc00 msgr2=0x7f3b3c1ac0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c1abc00 0x7f3b3c1ac0b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400e710 tx=0x7f3b3400ebd0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 -- 172.21.15.46:0/3889968815 shutdown_connections 2024-09-10T07:48:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c1abc00 0x7f3b3c1ac0b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c11b140 0x7f3b3c1a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b1c0a2b80 0x7f3b1c0a5040 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 --2- 172.21.15.46:0/3889968815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b3c110190 0x7f3b3c1a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 -- 172.21.15.46:0/3889968815 >> 172.21.15.46:0/3889968815 conn(0x7f3b3c073180 msgr2=0x7f3b3c119590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 -- 172.21.15.46:0/3889968815 shutdown_connections 2024-09-10T07:48:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:52.463+0000 7f3b41600640 1 -- 172.21.15.46:0/3889968815 wait complete. 2024-09-10T07:48:53.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:52 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:48:53.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:52 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:53.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:52 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:48:53.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:52 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:48:53.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:52 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:53.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:52 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:48:53.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:52 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:53.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:52 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:48:53.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:52 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:48:53.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:52 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:53.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:52 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:48:53.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:52 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:53.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:52 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:48:53.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:52 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:48:53.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:52 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:48:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:48:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:48:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 531 op/s 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:53 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 531 op/s 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:53 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 531 op/s 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:53 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:48:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.465+0000 7fd526600640 1 -- 172.21.15.46:0/1186094301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5200736c0 msgr2=0x7fd520119130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.465+0000 7fd526600640 1 --2- 172.21.15.46:0/1186094301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5200736c0 0x7fd520119130 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009900 tx=0x7fd50c02a410 comp rx=0 tx=0).stop 2024-09-10T07:48:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.466+0000 7fd526600640 1 -- 172.21.15.46:0/1186094301 shutdown_connections 2024-09-10T07:48:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.466+0000 7fd526600640 1 --2- 172.21.15.46:0/1186094301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5200736c0 0x7fd520119130 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.466+0000 7fd526600640 1 --2- 172.21.15.46:0/1186094301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd520072d20 0x7fd520073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.466+0000 7fd526600640 1 --2- 172.21.15.46:0/1186094301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd520109700 0x7fd520109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.466+0000 7fd526600640 1 -- 172.21.15.46:0/1186094301 >> 172.21.15.46:0/1186094301 conn(0x7fd52006ece0 msgr2=0x7fd520071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.466+0000 7fd526600640 1 -- 172.21.15.46:0/1186094301 shutdown_connections 2024-09-10T07:48:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.466+0000 7fd526600640 1 -- 172.21.15.46:0/1186094301 wait complete. 2024-09-10T07:48:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.467+0000 7fd526600640 1 Processor -- start 2024-09-10T07:48:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.469+0000 7fd526600640 1 -- start start 2024-09-10T07:48:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd526600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd520072d20 0x7fd5201a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd526600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5200736c0 0x7fd5201a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd526600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520109700 0x7fd5201ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd526600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd52011c6b0 con 0x7fd520072d20 2024-09-10T07:48:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd526600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd52011c530 con 0x7fd5200736c0 2024-09-10T07:48:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd526600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd52011c830 con 0x7fd520109700 2024-09-10T07:48:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5200736c0 0x7fd5201a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd51fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520109700 0x7fd5201ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5200736c0 0x7fd5201a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43256/0 (socket says 172.21.15.46:43256) 2024-09-10T07:48:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.470+0000 7fd51ea00640 1 -- 172.21.15.46:0/1634949408 learned_addr learned my addr 172.21.15.46:0/1634949408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.471+0000 7fd51fe00640 1 -- 172.21.15.46:0/1634949408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5200736c0 msgr2=0x7fd5201a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.471+0000 7fd51fe00640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5200736c0 0x7fd5201a7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.471+0000 7fd51fe00640 1 -- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd520072d20 msgr2=0x7fd5201a6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.471+0000 7fd51fe00640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd520072d20 0x7fd5201a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.471+0000 7fd51fe00640 1 -- 172.21.15.46:0/1634949408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5201ae910 con 0x7fd520109700 2024-09-10T07:48:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.475+0000 7fd51fe00640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520109700 0x7fd5201ae270 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009850 tx=0x7fd50c032d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.475+0000 7fd517e00640 1 -- 172.21.15.46:0/1634949408 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c050030 con 0x7fd520109700 2024-09-10T07:48:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.475+0000 7fd517e00640 1 -- 172.21.15.46:0/1634949408 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd50c003ce0 con 0x7fd520109700 2024-09-10T07:48:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.475+0000 7fd517e00640 1 -- 172.21.15.46:0/1634949408 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c035070 con 0x7fd520109700 2024-09-10T07:48:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.476+0000 7fd526600640 1 -- 172.21.15.46:0/1634949408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd520110920 con 0x7fd520109700 2024-09-10T07:48:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.476+0000 7fd526600640 1 -- 172.21.15.46:0/1634949408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd520110e40 con 0x7fd520109700 2024-09-10T07:48:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.481+0000 7fd517e00640 1 -- 172.21.15.46:0/1634949408 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd50c053030 con 0x7fd520109700 2024-09-10T07:48:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.485+0000 7fd517e00640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5040a2e70 0x7fd5040a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.485+0000 7fd517e00640 1 -- 172.21.15.46:0/1634949408 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd50c100810 con 0x7fd520109700 2024-09-10T07:48:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.487+0000 7fd51f400640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5040a2e70 0x7fd5040a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.489+0000 7fd51f400640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5040a2e70 0x7fd5040a5330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd5201a81d0 tx=0x7fd518009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.490+0000 7fd529617740 1 Processor -- start 2024-09-10T07:48:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.493+0000 7fd529617740 1 -- start start 2024-09-10T07:48:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.493+0000 7fd529617740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 0x55f439ffd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.493+0000 7fd529617740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a002490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.493+0000 7fd529617740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 0x55f43a004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.493+0000 7fd529617740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f439f62100 con 0x55f43a002a60 2024-09-10T07:48:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.493+0000 7fd529617740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f439f0a1b0 con 0x55f439ffd9d0 2024-09-10T07:48:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.493+0000 7fd529617740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f439f5e0c0 con 0x55f439ffd090 2024-09-10T07:48:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.494+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a002490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.494+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a002490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43266/0 (socket says 172.21.15.46:43266) 2024-09-10T07:48:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.494+0000 7fd51ea00640 1 -- 172.21.15.46:0/2944982286 learned_addr learned my addr 172.21.15.46:0/2944982286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.494+0000 7fd51ea00640 1 -- 172.21.15.46:0/2944982286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 msgr2=0x55f439ffd490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:48:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.494+0000 7fd51ea00640 1 --2- 172.21.15.46:0/2944982286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 0x55f439ffd490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.494+0000 7fd51ea00640 1 -- 172.21.15.46:0/2944982286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 msgr2=0x55f43a004f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.494+0000 7fd51ea00640 1 --2- 172.21.15.46:0/2944982286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 0x55f43a004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.494+0000 7fd51ea00640 1 -- 172.21.15.46:0/2944982286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f439c75210 con 0x55f439ffd9d0 2024-09-10T07:48:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.495+0000 7fd51ea00640 1 --2- 172.21.15.46:0/2944982286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a002490 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd510007840 tx=0x7fd510007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9bdd3e5b4506c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.495+0000 7fd514c00640 1 -- 172.21.15.46:0/2944982286 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51000d950 con 0x55f439ffd9d0 2024-09-10T07:48:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.498+0000 7fd514c00640 1 -- 172.21.15.46:0/2944982286 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd510018070 con 0x55f439ffd9d0 2024-09-10T07:48:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.498+0000 7fd514c00640 1 -- 172.21.15.46:0/2944982286 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51000d950 con 0x55f439ffd9d0 2024-09-10T07:48:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.500+0000 7fd529617740 1 -- 172.21.15.46:0/2944982286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 msgr2=0x55f43a002490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.500+0000 7fd529617740 1 --2- 172.21.15.46:0/2944982286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a002490 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd510007840 tx=0x7fd510007d00 comp rx=0 tx=0).stop 2024-09-10T07:48:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.502+0000 7fd529617740 1 -- 172.21.15.46:0/2944982286 shutdown_connections 2024-09-10T07:48:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.502+0000 7fd529617740 1 --2- 172.21.15.46:0/2944982286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 0x55f43a004f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.502+0000 7fd529617740 1 --2- 172.21.15.46:0/2944982286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a002490 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.502+0000 7fd529617740 1 --2- 172.21.15.46:0/2944982286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 0x55f439ffd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.502+0000 7fd529617740 1 -- 172.21.15.46:0/2944982286 >> 172.21.15.46:0/2944982286 conn(0x55f439ff8700 msgr2=0x55f439ff8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.502+0000 7fd529617740 1 -- 172.21.15.46:0/2944982286 shutdown_connections 2024-09-10T07:48:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.502+0000 7fd529617740 1 -- 172.21.15.46:0/2944982286 wait complete. 2024-09-10T07:48:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.503+0000 7fd529617740 1 -- start start 2024-09-10T07:48:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.503+0000 7fd529617740 1 Processor -- start 2024-09-10T07:48:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd529617740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a007310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd529617740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd9d0 0x55f43a007850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd529617740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f43a002a60 0x55f43a00e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd529617740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f43a005a30 con 0x55f439ffd090 2024-09-10T07:48:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd529617740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f439eeb090 con 0x55f43a002a60 2024-09-10T07:48:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd51fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f43a002a60 0x55f43a00e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd51fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f43a002a60 0x55f43a00e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43272/0 (socket says 172.21.15.46:43272) 2024-09-10T07:48:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd51fe00640 1 -- 172.21.15.46:0/3302966342 learned_addr learned my addr 172.21.15.46:0/3302966342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd529617740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f43a005bb0 con 0x55f439ffd9d0 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd51f400640 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a007310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.504+0000 7fd529617740 10 client.0 _ll_register_callbacks cb 0x55f439f3ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd51ea00640 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd9d0 0x55f43a007850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd51fe00640 1 -- 172.21.15.46:0/3302966342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd9d0 msgr2=0x55f43a007850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd51fe00640 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd9d0 0x55f43a007850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd51fe00640 1 -- 172.21.15.46:0/3302966342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 msgr2=0x55f43a007310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd51fe00640 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a007310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd51fe00640 1 -- 172.21.15.46:0/3302966342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f439d41620 con 0x55f43a002a60 2024-09-10T07:48:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd51fe00640 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f43a002a60 0x55f43a00e820 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0ccc20 tx=0x7fd50c0c8fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb67e869d7fa3a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd509600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd509600640 1 -- 172.21.15.46:0/3302966342 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c05a040 con 0x55f43a002a60 2024-09-10T07:48:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd509600640 1 -- 172.21.15.46:0/3302966342 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd50c04c470 con 0x55f43a002a60 2024-09-10T07:48:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.505+0000 7fd529617740 1 -- 172.21.15.46:0/3302966342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd510018070 con 0x55f43a002a60 2024-09-10T07:48:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.508+0000 7fd509600640 1 -- 172.21.15.46:0/3302966342 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd50c0cd390 con 0x55f43a002a60 2024-09-10T07:48:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.510+0000 7fd529617740 10 client.28418 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.510+0000 7fd529617740 1 -- 172.21.15.46:0/3302966342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f43a011610 con 0x55f43a002a60 2024-09-10T07:48:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.511+0000 7fd509600640 1 -- 172.21.15.46:0/3302966342 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd510018070 con 0x55f43a002a60 2024-09-10T07:48:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.511+0000 7fd509600640 1 -- 172.21.15.46:0/3302966342 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7fd50c0ccef0 con 0x55f43a002a60 2024-09-10T07:48:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.511+0000 7fd529617740 10 client.28418 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.511+0000 7fd529617740 10 client.28418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.511+0000 7fd529617740 20 client.28418 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.511+0000 7fd529617740 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f43a012d90 0x55f43a015180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.511+0000 7fd529617740 4 client.28418 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.511+0000 7fd529617740 1 -- 172.21.15.46:0/3302966342 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f439f6f830 con 0x55f43a012d90 2024-09-10T07:48:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.514+0000 7fd51f400640 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f43a012d90 0x55f43a015180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.515+0000 7fd51f400640 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f43a012d90 0x55f43a015180 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2179211ea8f6d3f3 server_cookie=eae5fe3c97983612 in_seq=0 out_seq=0 2024-09-10T07:48:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.515+0000 7fd509600640 10 client.28418 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.516+0000 7fd509600640 1 -- 172.21.15.46:0/3302966342 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f439f6f830 con 0x55f43a012d90 2024-09-10T07:48:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.516+0000 7fd509600640 10 client.28418 handle_command_reply: tid=0 2024-09-10T07:48:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 client.28418 shutdown 2024-09-10T07:48:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 -- 172.21.15.46:0/3302966342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f43a002a60 msgr2=0x55f43a00e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f43a002a60 0x55f43a00e820 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0ccc20 tx=0x7fd50c0c8fd0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 -- 172.21.15.46:0/3302966342 shutdown_connections 2024-09-10T07:48:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f43a012d90 0x55f43a015180 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f43a002a60 0x55f43a00e820 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd9d0 0x55f43a007850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 --2- 172.21.15.46:0/3302966342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a007310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd529617740 1 -- 172.21.15.46:0/3302966342 >> 172.21.15.46:0/3302966342 conn(0x55f439ff8700 msgr2=0x55f439ff9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.517+0000 7fd509600640 0 client.28418 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.520+0000 7fd529617740 1 -- 172.21.15.46:0/3302966342 shutdown_connections 2024-09-10T07:48:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.520+0000 7fd529617740 1 -- 172.21.15.46:0/3302966342 wait complete. 2024-09-10T07:48:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.520+0000 7fd529617740 20 client.28418 trim_cache size 0 max 16384 2024-09-10T07:48:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.562+0000 7fd529617740 1 Processor -- start 2024-09-10T07:48:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.562+0000 7fd529617740 1 -- start start 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.562+0000 7fd529617740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 0x55f439ff38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.562+0000 7fd529617740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.562+0000 7fd529617740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 0x55f439ff95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.562+0000 7fd529617740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b0e50 con 0x55f43a002a60 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.562+0000 7fd529617740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b2990 con 0x55f439ffd9d0 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.562+0000 7fd529617740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b1520 con 0x55f439ffd090 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.563+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.563+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43288/0 (socket says 172.21.15.46:43288) 2024-09-10T07:48:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.563+0000 7fd51ea00640 1 -- 172.21.15.46:0/1673843982 learned_addr learned my addr 172.21.15.46:0/1673843982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.563+0000 7fd51f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 0x55f439ff38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.563+0000 7fd51fe00640 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 0x55f439ff95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.565+0000 7fd51ea00640 1 -- 172.21.15.46:0/1673843982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 msgr2=0x55f439ff38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.565+0000 7fd51ea00640 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 0x55f439ff38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.565+0000 7fd51ea00640 1 -- 172.21.15.46:0/1673843982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 msgr2=0x55f439ff95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.565+0000 7fd51ea00640 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 0x55f439ff95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.565+0000 7fd51ea00640 1 -- 172.21.15.46:0/1673843982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f439ff8520 con 0x55f439ffd9d0 2024-09-10T07:48:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.565+0000 7fd51f400640 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 0x55f439ff38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.566+0000 7fd51ea00640 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00e7e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd510017000 tx=0x7fd510007580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=696602caaaec2264 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.566+0000 7fd50be00640 1 -- 172.21.15.46:0/1673843982 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51000db50 con 0x55f439ffd9d0 2024-09-10T07:48:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.567+0000 7fd50be00640 1 -- 172.21.15.46:0/1673843982 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd510018050 con 0x55f439ffd9d0 2024-09-10T07:48:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.567+0000 7fd50be00640 1 -- 172.21.15.46:0/1673843982 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51000db50 con 0x55f439ffd9d0 2024-09-10T07:48:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.568+0000 7fd529617740 1 -- 172.21.15.46:0/1673843982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 msgr2=0x55f43a00e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.568+0000 7fd529617740 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00e7e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd510017000 tx=0x7fd510007580 comp rx=0 tx=0).stop 2024-09-10T07:48:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.571+0000 7fd529617740 1 -- 172.21.15.46:0/1673843982 shutdown_connections 2024-09-10T07:48:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.571+0000 7fd529617740 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f43a002a60 0x55f439ff95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.571+0000 7fd529617740 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00e7e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.571+0000 7fd529617740 1 --2- 172.21.15.46:0/1673843982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f439ffd090 0x55f439ff38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.571+0000 7fd529617740 1 -- 172.21.15.46:0/1673843982 >> 172.21.15.46:0/1673843982 conn(0x55f439ff8700 msgr2=0x55f43a007310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.572+0000 7fd529617740 1 -- 172.21.15.46:0/1673843982 shutdown_connections 2024-09-10T07:48:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.572+0000 7fd529617740 1 -- 172.21.15.46:0/1673843982 wait complete. 2024-09-10T07:48:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.572+0000 7fd529617740 1 -- start start 2024-09-10T07:48:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.573+0000 7fd529617740 1 Processor -- start 2024-09-10T07:48:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.574+0000 7fd529617740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a00b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.574+0000 7fd529617740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.575+0000 7fd529617740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f43a002a60 0x55f43a07ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.575+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.575+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43296/0 (socket says 172.21.15.46:43296) 2024-09-10T07:48:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.575+0000 7fd51ea00640 1 -- 172.21.15.46:0/2268841024 learned_addr learned my addr 172.21.15.46:0/2268841024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.576+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b22c0 con 0x55f439ffd090 2024-09-10T07:48:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.576+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201b1bf0 con 0x55f439ffd9d0 2024-09-10T07:48:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.576+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201aee30 con 0x55f43a002a60 2024-09-10T07:48:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.576+0000 7fd529617740 10 client.0 _ll_register_callbacks cb 0x7fd50c032ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.577+0000 7fd51fe00640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f43a002a60 0x55f43a07ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.577+0000 7fd51ea00640 1 -- 172.21.15.46:0/2268841024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f43a002a60 msgr2=0x55f43a07ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.577+0000 7fd51f400640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a00b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.577+0000 7fd51ea00640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f43a002a60 0x55f43a07ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.577+0000 7fd51ea00640 1 -- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 msgr2=0x55f43a00b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.578+0000 7fd51ea00640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a00b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.578+0000 7fd51ea00640 1 -- 172.21.15.46:0/2268841024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43a07bf50 con 0x55f439ffd9d0 2024-09-10T07:48:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.578+0000 7fd51fe00640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f43a002a60 0x55f43a07ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.578+0000 7fd51f400640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a00b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.579+0000 7fd51ea00640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00bbf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd510006fd0 tx=0x7fd510007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9f6b7ad2151fea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.579+0000 7fd509600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.580+0000 7fd509600640 1 -- 172.21.15.46:0/2268841024 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510033060 con 0x55f439ffd9d0 2024-09-10T07:48:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.580+0000 7fd509600640 1 -- 172.21.15.46:0/2268841024 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd51000dd60 con 0x55f439ffd9d0 2024-09-10T07:48:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.582+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd510018050 con 0x55f439ffd9d0 2024-09-10T07:48:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.582+0000 7fd509600640 1 -- 172.21.15.46:0/2268841024 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd510004040 con 0x55f439ffd9d0 2024-09-10T07:48:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.584+0000 7fd529617740 10 client.28424 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.584+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f43a07d100 con 0x55f439ffd9d0 2024-09-10T07:48:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.584+0000 7fd509600640 1 -- 172.21.15.46:0/2268841024 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5100036b0 con 0x55f439ffd9d0 2024-09-10T07:48:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.585+0000 7fd509600640 1 -- 172.21.15.46:0/2268841024 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7fd51002e450 con 0x55f439ffd9d0 2024-09-10T07:48:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.585+0000 7fd529617740 10 client.28424 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.585+0000 7fd529617740 10 client.28424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.585+0000 7fd529617740 20 client.28424 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.585+0000 7fd529617740 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f43a07d810 0x55f43a07fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.585+0000 7fd529617740 4 client.28424 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:48:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.585+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd510018050 con 0x55f43a07d810 2024-09-10T07:48:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.587+0000 7fd51f400640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f43a07d810 0x55f43a07fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.588+0000 7fd51f400640 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f43a07d810 0x55f43a07fc30 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce7fdba71e8237d server_cookie=16746ea91248d089 in_seq=0 out_seq=0 2024-09-10T07:48:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.588+0000 7fd509600640 10 client.28424 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.615+0000 7fd509600640 1 -- 172.21.15.46:0/2268841024 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd510018050 con 0x55f43a07d810 2024-09-10T07:48:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.615+0000 7fd509600640 10 client.28424 handle_command_reply: tid=0 2024-09-10T07:48:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.615+0000 7fd529617740 1 client.28424 shutdown 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.615+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 msgr2=0x55f43a00bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.615+0000 7fd529617740 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00bbf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd510006fd0 tx=0x7fd510007d00 comp rx=0 tx=0).stop 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.616+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 shutdown_connections 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.616+0000 7fd529617740 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f43a07d810 0x55f43a07fc30 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.616+0000 7fd509600640 0 client.28424 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.616+0000 7fd529617740 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f43a002a60 0x55f43a07ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.616+0000 7fd529617740 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f439ffd9d0 0x55f43a00bbf0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.616+0000 7fd529617740 1 --2- 172.21.15.46:0/2268841024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f439ffd090 0x55f43a00b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.616+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 >> 172.21.15.46:0/2268841024 conn(0x55f439ff8700 msgr2=0x55f439ff7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.617+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 shutdown_connections 2024-09-10T07:48:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.617+0000 7fd529617740 1 -- 172.21.15.46:0/2268841024 wait complete. 2024-09-10T07:48:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.617+0000 7fd529617740 20 client.28424 trim_cache size 0 max 16384 2024-09-10T07:48:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:48:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:48:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:48:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:48:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.619+0000 7fd526600640 1 -- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5040a2e70 msgr2=0x7fd5040a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.619+0000 7fd526600640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5040a2e70 0x7fd5040a5330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd5201a81d0 tx=0x7fd518009250 comp rx=0 tx=0).stop 2024-09-10T07:48:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.620+0000 7fd526600640 1 -- 172.21.15.46:0/1634949408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520109700 msgr2=0x7fd5201ae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.620+0000 7fd526600640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520109700 0x7fd5201ae270 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009850 tx=0x7fd50c032d80 comp rx=0 tx=0).stop 2024-09-10T07:48:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.620+0000 7fd526600640 1 -- 172.21.15.46:0/1634949408 shutdown_connections 2024-09-10T07:48:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.620+0000 7fd526600640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd520109700 0x7fd5201ae270 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.620+0000 7fd526600640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5200736c0 0x7fd5201a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.620+0000 7fd526600640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5040a2e70 0x7fd5040a5330 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.620+0000 7fd526600640 1 --2- 172.21.15.46:0/1634949408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd520072d20 0x7fd5201a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.620+0000 7fd526600640 1 -- 172.21.15.46:0/1634949408 >> 172.21.15.46:0/1634949408 conn(0x7fd52006ece0 msgr2=0x7fd520071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.621+0000 7fd526600640 1 -- 172.21.15.46:0/1634949408 shutdown_connections 2024-09-10T07:48:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:48:54.621+0000 7fd526600640 1 -- 172.21.15.46:0/1634949408 wait complete. 2024-09-10T07:48:55.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:48:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.095+0000 7f2e78800640 1 -- 172.21.15.46:0/3950018583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 msgr2=0x7f2e701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.095+0000 7f2e78800640 1 --2- 172.21.15.46:0/3950018583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 0x7f2e701067e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2e60013f10 tx=0x7f2e60032970 comp rx=0 tx=0).stop 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.096+0000 7f2e78800640 1 -- 172.21.15.46:0/3950018583 shutdown_connections 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.096+0000 7f2e78800640 1 --2- 172.21.15.46:0/3950018583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70106f10 0x7f2e7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.096+0000 7f2e78800640 1 --2- 172.21.15.46:0/3950018583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 0x7f2e701067e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.096+0000 7f2e78800640 1 --2- 172.21.15.46:0/3950018583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70101de0 0x7f2e701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.096+0000 7f2e78800640 1 -- 172.21.15.46:0/3950018583 >> 172.21.15.46:0/3950018583 conn(0x7f2e700fd8e0 msgr2=0x7f2e700ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.100+0000 7f2e78800640 1 -- 172.21.15.46:0/3950018583 shutdown_connections 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.100+0000 7f2e78800640 1 -- 172.21.15.46:0/3950018583 wait complete. 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.102+0000 7f2e78800640 1 Processor -- start 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.103+0000 7f2e78800640 1 -- start start 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.103+0000 7f2e78800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70101de0 0x7f2e70196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.103+0000 7f2e78800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 0x7f2e701966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.103+0000 7f2e78800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70106f10 0x7f2e7019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.103+0000 7f2e78800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7010fd90 con 0x7f2e70101de0 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.103+0000 7f2e78800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7010fc10 con 0x7f2e70102790 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.103+0000 7f2e78800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7010ff10 con 0x7f2e70106f10 2024-09-10T07:48:55.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.104+0000 7f2e75400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 0x7f2e701966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.104+0000 7f2e75400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 0x7f2e701966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43318/0 (socket says 172.21.15.46:43318) 2024-09-10T07:48:55.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.104+0000 7f2e75400640 1 -- 172.21.15.46:0/3862108477 learned_addr learned my addr 172.21.15.46:0/3862108477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:55.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.104+0000 7f2e76800640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70106f10 0x7f2e7019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.104+0000 7f2e75e00640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70101de0 0x7f2e70196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.105+0000 7f2e75400640 1 -- 172.21.15.46:0/3862108477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70106f10 msgr2=0x7f2e7019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.105+0000 7f2e75400640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70106f10 0x7f2e7019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.105+0000 7f2e75400640 1 -- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70101de0 msgr2=0x7f2e70196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.105+0000 7f2e75400640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70101de0 0x7f2e70196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.105+0000 7f2e75400640 1 -- 172.21.15.46:0/3862108477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7019b120 con 0x7f2e70102790 2024-09-10T07:48:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.106+0000 7f2e75400640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 0x7f2e701966b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2e60032ee0 tx=0x7f2e60013580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.107+0000 7f2e6aa00640 1 -- 172.21.15.46:0/3862108477 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60046070 con 0x7f2e70102790 2024-09-10T07:48:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.107+0000 7f2e6aa00640 1 -- 172.21.15.46:0/3862108477 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e60004a30 con 0x7f2e70102790 2024-09-10T07:48:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.107+0000 7f2e6aa00640 1 -- 172.21.15.46:0/3862108477 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60038950 con 0x7f2e70102790 2024-09-10T07:48:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.107+0000 7f2e78800640 1 -- 172.21.15.46:0/3862108477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7019b3b0 con 0x7f2e70102790 2024-09-10T07:48:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.107+0000 7f2e78800640 1 -- 172.21.15.46:0/3862108477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e701a2cf0 con 0x7f2e70102790 2024-09-10T07:48:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.115+0000 7f2e6aa00640 1 -- 172.21.15.46:0/3862108477 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2e6003f040 con 0x7f2e70102790 2024-09-10T07:48:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.117+0000 7f2e6aa00640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e580a2d90 0x7f2e580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.118+0000 7f2e6aa00640 1 -- 172.21.15.46:0/3862108477 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2e600f8040 con 0x7f2e70102790 2024-09-10T07:48:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.120+0000 7f2e75e00640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e580a2d90 0x7f2e580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.122+0000 7f2e7b1ba740 1 Processor -- start 2024-09-10T07:48:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.122+0000 7f2e7b1ba740 1 -- start start 2024-09-10T07:48:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.122+0000 7f2e75e00640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e580a2d90 0x7f2e580a5250 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005e10 tx=0x7f2e64005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.122+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3a99790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.123+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.123+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.124+0000 7f2e75400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.124+0000 7f2e75400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43330/0 (socket says 172.21.15.46:43330) 2024-09-10T07:48:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.124+0000 7f2e75400640 1 -- 172.21.15.46:0/4016106895 learned_addr learned my addr 172.21.15.46:0/4016106895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.124+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/4016106895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5590c39ff530 con 0x5590c3a993b0 2024-09-10T07:48:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.124+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/4016106895 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590c39a6860 con 0x5590c3a99cd0 2024-09-10T07:48:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.124+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/4016106895 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5590c39fb4f0 con 0x5590c3aa4590 2024-09-10T07:48:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.124+0000 7f2e76800640 1 --2- 172.21.15.46:0/4016106895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.128+0000 7f2e76800640 1 -- 172.21.15.46:0/4016106895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 msgr2=0x5590c3aa4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.128+0000 7f2e76800640 1 --2- 172.21.15.46:0/4016106895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.128+0000 7f2e76800640 1 -- 172.21.15.46:0/4016106895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 msgr2=0x5590c3a99790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.128+0000 7f2e76800640 1 --2- 172.21.15.46:0/4016106895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3a99790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.128+0000 7f2e76800640 1 -- 172.21.15.46:0/4016106895 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c3770b90 con 0x5590c3aa4590 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.128+0000 7f2e76800640 1 --2- 172.21.15.46:0/4016106895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa6980 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009eb0 tx=0x7f2e6c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=699906435471cced server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.129+0000 7f2e5f000640 1 -- 172.21.15.46:0/4016106895 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c016070 con 0x5590c3aa4590 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.129+0000 7f2e5f000640 1 -- 172.21.15.46:0/4016106895 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e6c004d20 con 0x5590c3aa4590 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.129+0000 7f2e5f000640 1 -- 172.21.15.46:0/4016106895 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c012810 con 0x5590c3aa4590 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.130+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/4016106895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 msgr2=0x5590c3aa6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.130+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/4016106895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa6980 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009eb0 tx=0x7f2e6c010740 comp rx=0 tx=0).stop 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.132+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/4016106895 shutdown_connections 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.132+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/4016106895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa6980 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.132+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/4016106895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.132+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/4016106895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3a99790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.132+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/4016106895 >> 172.21.15.46:0/4016106895 conn(0x5590c3a95940 msgr2=0x5590c3a95d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.132+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/4016106895 shutdown_connections 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.132+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/4016106895 wait complete. 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.133+0000 7f2e7b1ba740 1 -- start start 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.133+0000 7f2e7b1ba740 1 Processor -- start 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.135+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.135+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.135+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e75400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5590c3aa6fb0 con 0x5590c3a993b0 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590c39b9e60 con 0x5590c3a99cd0 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5590c3aa7130 con 0x5590c3aa4590 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e7b1ba740 10 client.0 _ll_register_callbacks cb 0x5590c39becc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e76800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e76800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38600/0 (socket says 172.21.15.46:38600) 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.136+0000 7f2e76800640 1 -- 172.21.15.46:0/2312694349 learned_addr learned my addr 172.21.15.46:0/2312694349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.137+0000 7f2e75400640 1 -- 172.21.15.46:0/2312694349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 msgr2=0x5590c3aa2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.137+0000 7f2e75400640 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.137+0000 7f2e75400640 1 -- 172.21.15.46:0/2312694349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 msgr2=0x5590c3aa16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.137+0000 7f2e75400640 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.137+0000 7f2e75400640 1 -- 172.21.15.46:0/2312694349 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c39c5990 con 0x5590c3a99cd0 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.137+0000 7f2e53400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.137+0000 7f2e75400640 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa1c20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2e600c4450 tx=0x7f2e600c3ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914c12196d42a874 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.138+0000 7f2e53400640 1 -- 172.21.15.46:0/2312694349 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600e2060 con 0x5590c3a99cd0 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.138+0000 7f2e53400640 1 -- 172.21.15.46:0/2312694349 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2e600dd070 con 0x5590c3a99cd0 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.138+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/2312694349 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e6c004d20 con 0x5590c3a99cd0 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.138+0000 7f2e53400640 1 -- 172.21.15.46:0/2312694349 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2e600e2060 con 0x5590c3a99cd0 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.153+0000 7f2e7b1ba740 10 client.28436 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.153+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/2312694349 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5590c3ab2e80 con 0x5590c3a99cd0 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.153+0000 7f2e53400640 1 -- 172.21.15.46:0/2312694349 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e6c004d20 con 0x5590c3a99cd0 2024-09-10T07:48:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.153+0000 7f2e53400640 1 -- 172.21.15.46:0/2312694349 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7f2e60053030 con 0x5590c3a99cd0 2024-09-10T07:48:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.154+0000 7f2e7b1ba740 10 client.28436 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.154+0000 7f2e7b1ba740 10 client.28436 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:48:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.155+0000 7f2e7b1ba740 20 client.28436 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.155+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c3ab42c0 0x5590c3ab66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.155+0000 7f2e7b1ba740 4 client.28436 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.155+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/2312694349 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590c39a2310 con 0x5590c3ab42c0 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.156+0000 7f2e75e00640 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c3ab42c0 0x5590c3ab66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.159+0000 7f2e75e00640 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c3ab42c0 0x5590c3ab66b0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e71d8724eacf28b server_cookie=bf2dbb6faa254fab in_seq=0 out_seq=0 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.159+0000 7f2e53400640 10 client.28436 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.160+0000 7f2e53400640 1 -- 172.21.15.46:0/2312694349 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590c39a2310 con 0x5590c3ab42c0 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.161+0000 7f2e53400640 10 client.28436 handle_command_reply: tid=0 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.161+0000 7f2e7b1ba740 1 client.28436 shutdown 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.161+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/2312694349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 msgr2=0x5590c3aa1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.161+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa1c20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2e600c4450 tx=0x7f2e600c3ee0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.162+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/2312694349 shutdown_connections 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.162+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c3ab42c0 0x5590c3ab66b0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.162+0000 7f2e53400640 0 client.28436 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.162+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3aa2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.162+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3aa1c20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.162+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/2312694349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.162+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/2312694349 >> 172.21.15.46:0/2312694349 conn(0x5590c3a95940 msgr2=0x5590c3a94c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.163+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/2312694349 shutdown_connections 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.163+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/2312694349 wait complete. 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.163+0000 7f2e7b1ba740 20 client.28436 trim_cache size 0 max 16384 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.202+0000 7f2e7b1ba740 1 Processor -- start 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.202+0000 7f2e7b1ba740 1 -- start start 2024-09-10T07:48:55.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.203+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.203+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3a9be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.203+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3abbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.203+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e701ac9a0 con 0x5590c3a993b0 2024-09-10T07:48:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.203+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e701ae4e0 con 0x5590c3a99cd0 2024-09-10T07:48:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.203+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e701ad070 con 0x5590c3aa4590 2024-09-10T07:48:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.203+0000 7f2e75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e75400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3a9be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52784/0 (socket says 172.21.15.46:52784) 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e75e00640 1 -- 172.21.15.46:0/865691637 learned_addr learned my addr 172.21.15.46:0/865691637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e76800640 1 --2- 172.21.15.46:0/865691637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3abbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e75400640 1 -- 172.21.15.46:0/865691637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 msgr2=0x5590c3abbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e75400640 1 --2- 172.21.15.46:0/865691637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3abbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e75400640 1 -- 172.21.15.46:0/865691637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 msgr2=0x5590c3aa34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e75400640 1 --2- 172.21.15.46:0/865691637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.204+0000 7f2e75400640 1 -- 172.21.15.46:0/865691637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c3ab6a20 con 0x5590c3a99cd0 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.206+0000 7f2e75400640 1 --2- 172.21.15.46:0/865691637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3a9be60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x5590c3aa2c90 tx=0x7f2e600c3f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83594483a5df6ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.206+0000 7f2e5e600640 1 -- 172.21.15.46:0/865691637 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600dc030 con 0x5590c3a99cd0 2024-09-10T07:48:55.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.206+0000 7f2e5e600640 1 -- 172.21.15.46:0/865691637 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e60013b10 con 0x5590c3a99cd0 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.206+0000 7f2e5e600640 1 -- 172.21.15.46:0/865691637 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60038af0 con 0x5590c3a99cd0 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.207+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/865691637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 msgr2=0x5590c3a9be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.207+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/865691637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3a9be60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x5590c3aa2c90 tx=0x7f2e600c3f50 comp rx=0 tx=0).stop 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.207+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/865691637 shutdown_connections 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.207+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/865691637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3aa4590 0x5590c3abbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.207+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/865691637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3a99cd0 0x5590c3a9be60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.207+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/865691637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aa34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.207+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/865691637 >> 172.21.15.46:0/865691637 conn(0x5590c3a95940 msgr2=0x5590c3aa1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.208+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/865691637 shutdown_connections 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.208+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/865691637 wait complete. 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.209+0000 7f2e7b1ba740 1 -- start start 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.209+0000 7f2e7b1ba740 1 Processor -- start 2024-09-10T07:48:55.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.209+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.209+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3a99cd0 0x5590c3aa8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e7b1ba740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3aa4590 0x5590c3aa9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e701ade10 con 0x5590c3a993b0 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e701ad740 con 0x5590c3aa4590 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e7b1ba740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e701aa980 con 0x5590c3a99cd0 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e7b1ba740 10 client.0 _ll_register_callbacks cb 0x7f2e600c4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e75400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3a99cd0 0x5590c3aa8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e76800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3aa4590 0x5590c3aa9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e76800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3aa4590 0x5590c3aa9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:43352/0 (socket says 172.21.15.46:43352) 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e76800640 1 -- 172.21.15.46:0/1018039456 learned_addr learned my addr 172.21.15.46:0/1018039456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e75400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3a99cd0 0x5590c3aa8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:38608/0 (socket says 172.21.15.46:38608) 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e75e00640 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e76800640 1 -- 172.21.15.46:0/1018039456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3a99cd0 msgr2=0x5590c3aa8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e76800640 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3a99cd0 0x5590c3aa8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e76800640 1 -- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 msgr2=0x5590c3aac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e76800640 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e76800640 1 -- 172.21.15.46:0/1018039456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c3aa9670 con 0x5590c3aa4590 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.210+0000 7f2e75e00640 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.211+0000 7f2e76800640 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3aa4590 0x5590c3aa9130 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0049d0 tx=0x7f2e6c004c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd98c5629b43a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.211+0000 7f2e53400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.212+0000 7f2e53400640 1 -- 172.21.15.46:0/1018039456 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c016070 con 0x5590c3aa4590 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.212+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/1018039456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e60013b10 con 0x5590c3aa4590 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.212+0000 7f2e53400640 1 -- 172.21.15.46:0/1018039456 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2e6c005240 con 0x5590c3aa4590 2024-09-10T07:48:55.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.212+0000 7f2e53400640 1 -- 172.21.15.46:0/1018039456 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2e6c005480 con 0x5590c3aa4590 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.212+0000 7f2e7b1ba740 10 client.28442 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.212+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/1018039456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5590c3b1fcd0 con 0x5590c3aa4590 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.213+0000 7f2e53400640 1 -- 172.21.15.46:0/1018039456 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e60013b10 con 0x5590c3aa4590 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.213+0000 7f2e53400640 1 -- 172.21.15.46:0/1018039456 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7f2e6c00f040 con 0x5590c3aa4590 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.213+0000 7f2e7b1ba740 10 client.28442 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.213+0000 7f2e7b1ba740 10 client.28442 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.213+0000 7f2e7b1ba740 20 client.28442 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.213+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c3b202d0 0x5590c3b226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.213+0000 7f2e7b1ba740 4 client.28442 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}] 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.213+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/1018039456 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}) -- 0x5590c3a0cc60 con 0x5590c3b202d0 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.217+0000 7f2e75e00640 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c3b202d0 0x5590c3b226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.220+0000 7f2e75e00640 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c3b202d0 0x5590c3b226f0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b26a31d1ed1005 server_cookie=98167772f5a0c7ca in_seq=0 out_seq=0 2024-09-10T07:48:55.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.220+0000 7f2e53400640 10 client.28442 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e53400640 1 -- 172.21.15.46:0/1018039456 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5590c3a0cc60 con 0x5590c3b202d0 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e53400640 10 client.28442 handle_command_reply: tid=0 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 client.28442 shutdown 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/1018039456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3aa4590 msgr2=0x5590c3aa9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3aa4590 0x5590c3aa9130 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0049d0 tx=0x7f2e6c004c30 comp rx=0 tx=0).stop 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/1018039456 shutdown_connections 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5590c3b202d0 0x5590c3b226f0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c3aa4590 0x5590c3aa9130 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5590c3a99cd0 0x5590c3aa8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 --2- 172.21.15.46:0/1018039456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c3a993b0 0x5590c3aac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e53400640 0 client.28442 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/1018039456 >> 172.21.15.46:0/1018039456 conn(0x5590c3a95940 msgr2=0x5590c3aa16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/1018039456 shutdown_connections 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 1 -- 172.21.15.46:0/1018039456 wait complete. 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.387+0000 7f2e7b1ba740 20 client.28442 trim_cache size 0 max 16384 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 51, 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 42, 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stdout: "c01c2732": { 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 42, 2024-09-10T07:48:55.622 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 138.1, 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:48:55.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:48:55.624 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:48:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 -- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e580a2d90 msgr2=0x7f2e580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e580a2d90 0x7f2e580a5250 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005e10 tx=0x7f2e64005da0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 -- 172.21.15.46:0/3862108477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 msgr2=0x7f2e701966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 0x7f2e701966b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2e60032ee0 tx=0x7f2e60013580 comp rx=0 tx=0).stop 2024-09-10T07:48:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 -- 172.21.15.46:0/3862108477 shutdown_connections 2024-09-10T07:48:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e70106f10 0x7f2e7019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e70102790 0x7f2e701966b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2e580a2d90 0x7f2e580a5250 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 --2- 172.21.15.46:0/3862108477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70101de0 0x7f2e70196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 -- 172.21.15.46:0/3862108477 >> 172.21.15.46:0/3862108477 conn(0x7f2e700fd8e0 msgr2=0x7f2e700fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.388+0000 7f2e78800640 1 -- 172.21.15.46:0/3862108477 shutdown_connections 2024-09-10T07:48:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:55.389+0000 7f2e78800640 1 -- 172.21.15.46:0/3862108477 wait complete. 2024-09-10T07:48:55.625 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 695.0 s, 437.8 tps, lat 218.132 ms stddev 285.087 2024-09-10T07:48:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:55 smithi181 ceph-mon[35397]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 418 op/s 2024-09-10T07:48:56.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:55 smithi063 ceph-mon[35020]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 418 op/s 2024-09-10T07:48:56.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:55 smithi046 ceph-mon[30705]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 418 op/s 2024-09-10T07:48:56.443 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c01c2732', quiesce took 12.6 seconds. Will release after: 5.500000000000002 2024-09-10T07:48:56.473 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:48:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.114+0000 7f13c6e00640 1 -- 172.21.15.46:0/3445320921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01019c0 msgr2=0x7f13c0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.114+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3445320921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01019c0 0x7f13c0101e20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009930 tx=0x7f13ac02a440 comp rx=0 tx=0).stop 2024-09-10T07:48:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.114+0000 7f13c6e00640 1 -- 172.21.15.46:0/3445320921 shutdown_connections 2024-09-10T07:48:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.114+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3445320921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13c0102360 0x7f13c010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.114+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3445320921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01019c0 0x7f13c0101e20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.114+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3445320921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c01079c0 0x7f13c0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.114+0000 7f13c6e00640 1 -- 172.21.15.46:0/3445320921 >> 172.21.15.46:0/3445320921 conn(0x7f13c00fd6e0 msgr2=0x7f13c00ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.115+0000 7f13c6e00640 1 -- 172.21.15.46:0/3445320921 shutdown_connections 2024-09-10T07:48:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.115+0000 7f13c6e00640 1 -- 172.21.15.46:0/3445320921 wait complete. 2024-09-10T07:48:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.116+0000 7f13c6e00640 1 Processor -- start 2024-09-10T07:48:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.116+0000 7f13c6e00640 1 -- start start 2024-09-10T07:48:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.116+0000 7f13c6e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13c01019c0 0x7f13c019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.116+0000 7f13c6e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0102360 0x7f13c019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.116+0000 7f13c6e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01079c0 0x7f13c019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.116+0000 7f13c6e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0111dc0 con 0x7f13c01079c0 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.116+0000 7f13c6e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0111c40 con 0x7f13c0102360 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.116+0000 7f13c6e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0111f40 con 0x7f13c01019c0 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0102360 0x7f13c019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01079c0 0x7f13c019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0102360 0x7f13c019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37046/0 (socket says 172.21.15.46:37046) 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 -- 172.21.15.46:0/3339492416 learned_addr learned my addr 172.21.15.46:0/3339492416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5a00640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13c01019c0 0x7f13c019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 -- 172.21.15.46:0/3339492416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13c01019c0 msgr2=0x7f13c019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13c01019c0 0x7f13c019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 -- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01079c0 msgr2=0x7f13c019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01079c0 0x7f13c019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 -- 172.21.15.46:0/3339492416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c019f440 con 0x7f13c0102360 2024-09-10T07:48:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c6400640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01079c0 0x7f13c019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.117+0000 7f13c5000640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0102360 0x7f13c019a9d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009230 tx=0x7f13ac004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.118+0000 7f13be000640 1 -- 172.21.15.46:0/3339492416 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac03f030 con 0x7f13c0102360 2024-09-10T07:48:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.118+0000 7f13c6e00640 1 -- 172.21.15.46:0/3339492416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c019f6d0 con 0x7f13c0102360 2024-09-10T07:48:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.118+0000 7f13be000640 1 -- 172.21.15.46:0/3339492416 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac003ad0 con 0x7f13c0102360 2024-09-10T07:48:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.118+0000 7f13be000640 1 -- 172.21.15.46:0/3339492416 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac035070 con 0x7f13c0102360 2024-09-10T07:48:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.118+0000 7f13c6e00640 1 -- 172.21.15.46:0/3339492416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c01a7010 con 0x7f13c0102360 2024-09-10T07:48:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13caea9740 1 Processor -- start 2024-09-10T07:48:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13caea9740 1 -- start start 2024-09-10T07:48:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13be000640 1 -- 172.21.15.46:0/3339492416 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f13ac042030 con 0x7f13c0102360 2024-09-10T07:48:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13caea9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f0d50 0x564c812f1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13caea9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c812fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13caea9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812fe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13caea9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564c812573e0 con 0x564c812f1670 2024-09-10T07:48:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13caea9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564c811fed80 con 0x564c812fbf30 2024-09-10T07:48:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.121+0000 7f13caea9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564c812533a0 con 0x564c812f0d50 2024-09-10T07:48:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c812fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812fe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37056/0 (socket says 172.21.15.46:37056) 2024-09-10T07:48:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 -- 172.21.15.46:0/2818016634 learned_addr learned my addr 172.21.15.46:0/2818016634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c6400640 1 --2- 172.21.15.46:0/2818016634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f0d50 0x564c812f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 -- 172.21.15.46:0/2818016634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f0d50 msgr2=0x564c812f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 --2- 172.21.15.46:0/2818016634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f0d50 0x564c812f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 -- 172.21.15.46:0/2818016634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 msgr2=0x564c812fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 --2- 172.21.15.46:0/2818016634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c812fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 -- 172.21.15.46:0/2818016634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c8121dae0 con 0x564c812fbf30 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.122+0000 7f13c5000640 1 --2- 172.21.15.46:0/2818016634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812fe320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0f5c20 tx=0x7f13ac0f5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15d128d9d65db50c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.123+0000 7f13b2a00640 1 -- 172.21.15.46:0/2818016634 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0fe070 con 0x564c812fbf30 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.123+0000 7f13be000640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f13a80a2c00 0x7f13a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.123+0000 7f13b2a00640 1 -- 172.21.15.46:0/2818016634 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac0f7c90 con 0x564c812fbf30 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.123+0000 7f13b2a00640 1 -- 172.21.15.46:0/2818016634 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0fa750 con 0x564c812fbf30 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.123+0000 7f13be000640 1 -- 172.21.15.46:0/3339492416 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f13ac0ef860 con 0x7f13c0102360 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.123+0000 7f13c6400640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f13a80a2c00 0x7f13a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 -- 172.21.15.46:0/2818016634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 msgr2=0x564c812fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 --2- 172.21.15.46:0/2818016634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812fe320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0f5c20 tx=0x7f13ac0f5d10 comp rx=0 tx=0).stop 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 -- 172.21.15.46:0/2818016634 shutdown_connections 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 --2- 172.21.15.46:0/2818016634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812fe320 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 --2- 172.21.15.46:0/2818016634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c812fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 --2- 172.21.15.46:0/2818016634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f0d50 0x564c812f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 -- 172.21.15.46:0/2818016634 >> 172.21.15.46:0/2818016634 conn(0x564c812ed2e0 msgr2=0x564c812ed6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 -- 172.21.15.46:0/2818016634 shutdown_connections 2024-09-10T07:48:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13c6400640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f13a80a2c00 0x7f13a80a50c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f13b8005fd0 tx=0x7f13b8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 -- 172.21.15.46:0/2818016634 wait complete. 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.124+0000 7f13caea9740 1 -- start start 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13caea9740 1 Processor -- start 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13caea9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c812f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13caea9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f1670 0x564c812f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13caea9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 0x564c812f9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13caea9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564c812fe8c0 con 0x564c812f0d50 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13caea9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564c812120f0 con 0x564c812f1670 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13caea9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564c812fea40 con 0x564c812fbf30 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13caea9740 10 client.0 _ll_register_callbacks cb 0x564c812039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13c5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c812f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13c5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f1670 0x564c812f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.125+0000 7f13c5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f1670 0x564c812f92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37062/0 (socket says 172.21.15.46:37062) 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c5000640 1 -- 172.21.15.46:0/800741868 learned_addr learned my addr 172.21.15.46:0/800741868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c6400640 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 0x564c812f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c5a00640 1 -- 172.21.15.46:0/800741868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 msgr2=0x564c812f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c5a00640 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 0x564c812f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c5a00640 1 -- 172.21.15.46:0/800741868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f1670 msgr2=0x564c812f92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c5a00640 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f1670 0x564c812f92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c5a00640 1 -- 172.21.15.46:0/800741868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c80fc83c0 con 0x564c812f0d50 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c5000640 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f1670 0x564c812f92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13c5a00640 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c812f8d90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f13b4005fd0 tx=0x7f13b400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e2b1398c3afe7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13a7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.126+0000 7f13a7400640 1 -- 172.21.15.46:0/800741868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4004020 con 0x564c812f0d50 2024-09-10T07:48:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13caea9740 1 -- 172.21.15.46:0/800741868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ac0f7c90 con 0x564c812f0d50 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13a7400640 1 -- 172.21.15.46:0/800741868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f13b40041c0 con 0x564c812f0d50 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13a7400640 1 -- 172.21.15.46:0/800741868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f13b4024050 con 0x564c812f0d50 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13caea9740 10 client.17685 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13caea9740 1 -- 172.21.15.46:0/800741868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564c8130a520 con 0x564c812f0d50 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13a7400640 1 -- 172.21.15.46:0/800741868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac0f7c90 con 0x564c812f0d50 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13a7400640 1 -- 172.21.15.46:0/800741868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7f13b4004be0 con 0x564c812f0d50 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13caea9740 10 client.17685 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.127+0000 7f13caea9740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.128+0000 7f13caea9740 20 client.17685 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.128+0000 7f13caea9740 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564c8130ba30 0x564c8130ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.128+0000 7f13caea9740 4 client.17685 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:57.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.128+0000 7f13caea9740 1 -- 172.21.15.46:0/800741868 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c81264950 con 0x564c8130ba30 2024-09-10T07:48:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.128+0000 7f13c5000640 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564c8130ba30 0x564c8130ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.128+0000 7f13c5000640 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564c8130ba30 0x564c8130ded0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413a48a2ae056ffa server_cookie=8bef78867de80576 in_seq=0 out_seq=0 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.128+0000 7f13a7400640 10 client.17685 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13a7400640 1 -- 172.21.15.46:0/800741868 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c81264950 con 0x564c8130ba30 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13a7400640 10 client.17685 handle_command_reply: tid=0 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 client.17685 shutdown 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 -- 172.21.15.46:0/800741868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 msgr2=0x564c812f8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c812f8d90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f13b4005fd0 tx=0x7f13b400bf90 comp rx=0 tx=0).stop 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 -- 172.21.15.46:0/800741868 shutdown_connections 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564c8130ba30 0x564c8130ded0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 0x564c812f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f1670 0x564c812f92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 --2- 172.21.15.46:0/800741868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c812f8d90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13caea9740 1 -- 172.21.15.46:0/800741868 >> 172.21.15.46:0/800741868 conn(0x564c812ed2e0 msgr2=0x564c812f3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.129+0000 7f13a7400640 0 client.17685 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:57.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.130+0000 7f13caea9740 1 -- 172.21.15.46:0/800741868 shutdown_connections 2024-09-10T07:48:57.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.130+0000 7f13caea9740 1 -- 172.21.15.46:0/800741868 wait complete. 2024-09-10T07:48:57.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.130+0000 7f13caea9740 20 client.17685 trim_cache size 0 max 16384 2024-09-10T07:48:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.148+0000 7f13caea9740 1 Processor -- start 2024-09-10T07:48:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.148+0000 7f13caea9740 1 -- start start 2024-09-10T07:48:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.148+0000 7f13caea9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f0d50 0x564c812fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.148+0000 7f13caea9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c813137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.148+0000 7f13caea9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 0x564c81313ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.148+0000 7f13caea9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b0d20 con 0x564c812f1670 2024-09-10T07:48:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.148+0000 7f13caea9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b2860 con 0x564c812f0d50 2024-09-10T07:48:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.148+0000 7f13caea9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b13f0 con 0x564c812fbf30 2024-09-10T07:48:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c813137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c813137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47236/0 (socket says 172.21.15.46:47236) 2024-09-10T07:48:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 -- 172.21.15.46:0/3932285084 learned_addr learned my addr 172.21.15.46:0/3932285084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5a00640 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f0d50 0x564c812fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c6400640 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 0x564c81313ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 -- 172.21.15.46:0/3932285084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 msgr2=0x564c81313ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 0x564c81313ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 -- 172.21.15.46:0/3932285084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f0d50 msgr2=0x564c812fad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f0d50 0x564c812fad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 -- 172.21.15.46:0/3932285084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c812ed110 con 0x564c812f1670 2024-09-10T07:48:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.149+0000 7f13c5000640 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c813137a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009900 tx=0x7f13ac0f5f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1788139fef60e60a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13b2000640 1 -- 172.21.15.46:0/3932285084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac049040 con 0x564c812f1670 2024-09-10T07:48:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13b2000640 1 -- 172.21.15.46:0/3932285084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac0bbd90 con 0x564c812f1670 2024-09-10T07:48:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13b2000640 1 -- 172.21.15.46:0/3932285084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0fe040 con 0x564c812f1670 2024-09-10T07:48:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13caea9740 1 -- 172.21.15.46:0/3932285084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 msgr2=0x564c813137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13caea9740 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c813137a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009900 tx=0x7f13ac0f5f60 comp rx=0 tx=0).stop 2024-09-10T07:48:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13caea9740 1 -- 172.21.15.46:0/3932285084 shutdown_connections 2024-09-10T07:48:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13caea9740 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812fbf30 0x564c81313ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13caea9740 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f1670 0x564c813137a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13caea9740 1 --2- 172.21.15.46:0/3932285084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812f0d50 0x564c812fad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13caea9740 1 -- 172.21.15.46:0/3932285084 >> 172.21.15.46:0/3932285084 conn(0x564c812ed2e0 msgr2=0x564c812fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.150+0000 7f13caea9740 1 -- 172.21.15.46:0/3932285084 shutdown_connections 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.151+0000 7f13caea9740 1 -- 172.21.15.46:0/3932285084 wait complete. 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.151+0000 7f13caea9740 1 -- start start 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.151+0000 7f13caea9740 1 Processor -- start 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.151+0000 7f13caea9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c813041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.151+0000 7f13caea9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f1670 0x564c812f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13caea9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13caea9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b2190 con 0x564c812f0d50 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13caea9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b1ac0 con 0x564c812fbf30 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13caea9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01aed00 con 0x564c812f1670 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13caea9740 10 client.0 _ll_register_callbacks cb 0x7f13b400b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c813041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c813041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47240/0 (socket says 172.21.15.46:47240) 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5a00640 1 -- 172.21.15.46:0/3418208418 learned_addr learned my addr 172.21.15.46:0/3418208418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5000640 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f1670 0x564c812f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c6400640 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5a00640 1 -- 172.21.15.46:0/3418208418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f1670 msgr2=0x564c812f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5a00640 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f1670 0x564c812f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5a00640 1 -- 172.21.15.46:0/3418208418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 msgr2=0x564c812f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5a00640 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c5a00640 1 -- 172.21.15.46:0/3418208418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c812f6350 con 0x564c812f0d50 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.152+0000 7f13c6400640 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.153+0000 7f13c5a00640 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c813041d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f13b4011da0 tx=0x7f13b4010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde707deddd2d791 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.153+0000 7f13a7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:48:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.153+0000 7f13a7400640 1 -- 172.21.15.46:0/3418208418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4010c90 con 0x564c812f0d50 2024-09-10T07:48:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.153+0000 7f13a7400640 1 -- 172.21.15.46:0/3418208418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f13b4010e30 con 0x564c812f0d50 2024-09-10T07:48:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.153+0000 7f13caea9740 1 -- 172.21.15.46:0/3418208418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ac0bbd90 con 0x564c812f0d50 2024-09-10T07:48:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.153+0000 7f13a7400640 1 -- 172.21.15.46:0/3418208418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f13b4004fd0 con 0x564c812f0d50 2024-09-10T07:48:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.153+0000 7f13caea9740 10 client.17691 fetch_fsmap learned FSMap version 51 2024-09-10T07:48:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.153+0000 7f13caea9740 1 -- 172.21.15.46:0/3418208418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564c813764e0 con 0x564c812f0d50 2024-09-10T07:48:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13a7400640 1 -- 172.21.15.46:0/3418208418 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac0bbd90 con 0x564c812f0d50 2024-09-10T07:48:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13a7400640 1 -- 172.21.15.46:0/3418208418 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) ==== 8374+0+0 (secure 0 0 0) 0x7f13b4031430 con 0x564c812f0d50 2024-09-10T07:48:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13caea9740 10 client.17691 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:48:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13caea9740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13caea9740 20 client.17691 populate_metadata read hostname 'smithi046' 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13caea9740 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564c81376c90 0x564c813790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13caea9740 4 client.17691 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13caea9740 1 -- 172.21.15.46:0/3418208418 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c8125ee60 con 0x564c81376c90 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.154+0000 7f13c5000640 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564c81376c90 0x564c813790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.155+0000 7f13a7400640 10 client.17691 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.155+0000 7f13c5000640 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564c81376c90 0x564c813790b0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf009424822d4c1 server_cookie=c3bd6e6204416799 in_seq=0 out_seq=0 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.155+0000 7f13a7400640 1 -- 172.21.15.46:0/3418208418 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564c8125ee60 con 0x564c81376c90 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.155+0000 7f13a7400640 10 client.17691 handle_command_reply: tid=0 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.155+0000 7f13caea9740 1 client.17691 shutdown 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.155+0000 7f13caea9740 1 -- 172.21.15.46:0/3418208418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 msgr2=0x564c813041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.155+0000 7f13caea9740 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c813041d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f13b4011da0 tx=0x7f13b4010b20 comp rx=0 tx=0).stop 2024-09-10T07:48:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 1 -- 172.21.15.46:0/3418208418 shutdown_connections 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564c81376c90 0x564c813790b0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c812fbf30 0x564c812f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c812f1670 0x564c812f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 1 --2- 172.21.15.46:0/3418208418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c812f0d50 0x564c813041d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13a7400640 0 client.17691 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 1 -- 172.21.15.46:0/3418208418 >> 172.21.15.46:0/3418208418 conn(0x564c812ed2e0 msgr2=0x564c812fae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 1 -- 172.21.15.46:0/3418208418 shutdown_connections 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 1 -- 172.21.15.46:0/3418208418 wait complete. 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.156+0000 7f13caea9740 20 client.17691 trim_cache size 0 max 16384 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:48:57.159 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:48:57.160 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:48:57.160 INFO:teuthology.orchestra.run.smithi046.stdout: "6c436a4d-3cf8-4264-9954-ee89c5dbbc79": { 2024-09-10T07:48:57.160 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:48:57.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6c436a4d-3cf8-4264-9954-ee89c5dbbc79", 2024-09-10T07:48:57.160 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:48:57.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:48:57.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:48:57.160 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 -- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f13a80a2c00 msgr2=0x7f13a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f13a80a2c00 0x7f13a80a50c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f13b8005fd0 tx=0x7f13b8009600 comp rx=0 tx=0).stop 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 -- 172.21.15.46:0/3339492416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0102360 msgr2=0x7f13c019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0102360 0x7f13c019a9d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009230 tx=0x7f13ac004270 comp rx=0 tx=0).stop 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 -- 172.21.15.46:0/3339492416 shutdown_connections 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f13a80a2c00 0x7f13a80a50c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c01079c0 0x7f13c019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0102360 0x7f13c019a9d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 --2- 172.21.15.46:0/3339492416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13c01019c0 0x7f13c019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 -- 172.21.15.46:0/3339492416 >> 172.21.15.46:0/3339492416 conn(0x7f13c00fd6e0 msgr2=0x7f13c010d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 -- 172.21.15.46:0/3339492416 shutdown_connections 2024-09-10T07:48:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:48:57.157+0000 7f13c6e00640 1 -- 172.21.15.46:0/3339492416 wait complete. 2024-09-10T07:48:57.938 INFO:tasks.cephfs.filesystem:scrub status for tag:6c436a4d-3cf8-4264-9954-ee89c5dbbc79 - {'path': '/', 'tag': '6c436a4d-3cf8-4264-9954-ee89c5dbbc79', 'options': 'recursive,force'} 2024-09-10T07:48:58.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:57 smithi046 ceph-mon[30705]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.8 MiB/s wr, 649 op/s 2024-09-10T07:48:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:57 smithi181 ceph-mon[35397]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.8 MiB/s wr, 649 op/s 2024-09-10T07:48:58.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:57 smithi063 ceph-mon[35020]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.8 MiB/s wr, 649 op/s 2024-09-10T07:48:59.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:58 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:48:59.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:58 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:48:59.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:58 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:49:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:48:59 smithi181 ceph-mon[35397]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-10T07:49:00.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:48:59 smithi063 ceph-mon[35020]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-10T07:49:00.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:48:59 smithi046 ceph-mon[30705]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-10T07:49:00.602 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 700.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:49:01.944 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:49:02.336 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:01 smithi181 ceph-mon[35397]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-09-10T07:49:02.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:01 smithi063 ceph-mon[35020]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-09-10T07:49:02.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:01 smithi046 ceph-mon[30705]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.942+0000 7f227d200640 1 -- 172.21.15.46:0/2920296442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 msgr2=0x7f2278074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.942+0000 7f227d200640 1 --2- 172.21.15.46:0/2920296442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 0x7f2278074a00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2268009900 tx=0x7f226802a480 comp rx=0 tx=0).stop 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.943+0000 7f227d200640 1 -- 172.21.15.46:0/2920296442 shutdown_connections 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.943+0000 7f227d200640 1 --2- 172.21.15.46:0/2920296442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227810e3c0 0x7f2278110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.943+0000 7f227d200640 1 --2- 172.21.15.46:0/2920296442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2278074f40 0x7f227810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.943+0000 7f227d200640 1 --2- 172.21.15.46:0/2920296442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 0x7f2278074a00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.943+0000 7f227d200640 1 -- 172.21.15.46:0/2920296442 >> 172.21.15.46:0/2920296442 conn(0x7f22780fd8c0 msgr2=0x7f22780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.943+0000 7f227d200640 1 -- 172.21.15.46:0/2920296442 shutdown_connections 2024-09-10T07:49:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.943+0000 7f227d200640 1 -- 172.21.15.46:0/2920296442 wait complete. 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.944+0000 7f227d200640 1 Processor -- start 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.944+0000 7f227d200640 1 -- start start 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f227d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 0x7f227819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f227d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2278074f40 0x7f227819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f227d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227810e3c0 0x7f227819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f227d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22781141a0 con 0x7f227810e3c0 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f227d200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2278114020 con 0x7f2278074f40 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f227d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2278114320 con 0x7f2278074620 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 0x7f227819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 0x7f227819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60804/0 (socket says 172.21.15.46:60804) 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275e00640 1 -- 172.21.15.46:0/1076436259 learned_addr learned my addr 172.21.15.46:0/1076436259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275400640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2278074f40 0x7f227819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2276800640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227810e3c0 0x7f227819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275e00640 1 -- 172.21.15.46:0/1076436259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2278074f40 msgr2=0x7f227819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275e00640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2278074f40 0x7f227819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275e00640 1 -- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227810e3c0 msgr2=0x7f227819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275e00640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227810e3c0 0x7f227819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275e00640 1 -- 172.21.15.46:0/1076436259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227819f6d0 con 0x7f2278074620 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.945+0000 7f2275400640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2278074f40 0x7f227819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.946+0000 7f2275e00640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 0x7f227819a720 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2268004770 tx=0x7f2268003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.946+0000 7f226ea00640 1 -- 172.21.15.46:0/1076436259 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226803f070 con 0x7f2278074620 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.946+0000 7f226ea00640 1 -- 172.21.15.46:0/1076436259 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2268031ca0 con 0x7f2278074620 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.946+0000 7f226ea00640 1 -- 172.21.15.46:0/1076436259 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268035030 con 0x7f2278074620 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.946+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227819f960 con 0x7f2278074620 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.946+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f227819fdf0 con 0x7f2278074620 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.947+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2278074a00 con 0x7f2278074620 2024-09-10T07:49:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.949+0000 7f226ea00640 1 -- 172.21.15.46:0/1076436259 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2268042070 con 0x7f2278074620 2024-09-10T07:49:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.951+0000 7f226ea00640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f225c0a2c00 0x7f225c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.951+0000 7f226ea00640 1 -- 172.21.15.46:0/1076436259 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f22680f0170 con 0x7f2278074620 2024-09-10T07:49:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.951+0000 7f2275400640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f225c0a2c00 0x7f225c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.952+0000 7f2275400640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f225c0a2c00 0x7f225c0a50c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f227810f7d0 tx=0x7f2264005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:02.953+0000 7f226ea00640 1 -- 172.21.15.46:0/1076436259 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22680bccf0 con 0x7f2278074620 2024-09-10T07:49:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.178+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f227819ba80 con 0x7f2278074620 2024-09-10T07:49:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.180+0000 7f226ea00640 1 -- 172.21.15.46:0/1076436259 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11972 (secure 0 0 0) 0x7f227819ba80 con 0x7f2278074620 2024-09-10T07:49:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:49:03.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":52,"btime":"2024-09-10T07:48:57:965518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:48:57.232057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:49:03.181 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 52 2024-09-10T07:49:03.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f225c0a2c00 msgr2=0x7f225c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:03.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f225c0a2c00 0x7f225c0a50c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f227810f7d0 tx=0x7f2264005f90 comp rx=0 tx=0).stop 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 msgr2=0x7f227819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 0x7f227819a720 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2268004770 tx=0x7f2268003c10 comp rx=0 tx=0).stop 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 shutdown_connections 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f225c0a2c00 0x7f225c0a50c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227810e3c0 0x7f227819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2278074f40 0x7f227819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 --2- 172.21.15.46:0/1076436259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2278074620 0x7f227819a720 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 >> 172.21.15.46:0/1076436259 conn(0x7f22780fd8c0 msgr2=0x7f22780fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 shutdown_connections 2024-09-10T07:49:03.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:03.182+0000 7f227d200640 1 -- 172.21.15.46:0/1076436259 wait complete. 2024-09-10T07:49:03.961 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id c01c2732 --release --await' 2024-09-10T07:49:03.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id c01c2732 --release --await 2024-09-10T07:49:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:03 smithi181 ceph-mon[35397]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 395 op/s 2024-09-10T07:49:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:03 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1076436259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:03 smithi063 ceph-mon[35020]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 395 op/s 2024-09-10T07:49:04.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:03 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1076436259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:04.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:03 smithi046 ceph-mon[30705]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 395 op/s 2024-09-10T07:49:04.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:03 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1076436259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.915+0000 7ff344e00640 1 -- 172.21.15.46:0/552159784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340103e30 msgr2=0x7ff340104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.915+0000 7ff344e00640 1 --2- 172.21.15.46:0/552159784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340103e30 0x7ff340104210 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff330009900 tx=0x7ff33002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.915+0000 7ff344e00640 1 -- 172.21.15.46:0/552159784 shutdown_connections 2024-09-10T07:49:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.915+0000 7ff344e00640 1 --2- 172.21.15.46:0/552159784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff340108f60 0x7ff340110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.915+0000 7ff344e00640 1 --2- 172.21.15.46:0/552159784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3401047e0 0x7ff340108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.915+0000 7ff344e00640 1 --2- 172.21.15.46:0/552159784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340103e30 0x7ff340104210 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.915+0000 7ff344e00640 1 -- 172.21.15.46:0/552159784 >> 172.21.15.46:0/552159784 conn(0x7ff3400ff8d0 msgr2=0x7ff340101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.916+0000 7ff344e00640 1 -- 172.21.15.46:0/552159784 shutdown_connections 2024-09-10T07:49:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.916+0000 7ff344e00640 1 -- 172.21.15.46:0/552159784 wait complete. 2024-09-10T07:49:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.917+0000 7ff344e00640 1 Processor -- start 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.917+0000 7ff344e00640 1 -- start start 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.917+0000 7ff344e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff340103e30 0x7ff34019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.917+0000 7ff344e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3401047e0 0x7ff34019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.917+0000 7ff344e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340108f60 0x7ff34019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.917+0000 7ff344e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff340114010 con 0x7ff340108f60 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff344e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff340113e90 con 0x7ff340103e30 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff344e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff340114190 con 0x7ff3401047e0 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff33fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340108f60 0x7ff34019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff337e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3401047e0 0x7ff34019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff33f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff340103e30 0x7ff34019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff33fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340108f60 0x7ff34019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47270/0 (socket says 172.21.15.46:47270) 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff337e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3401047e0 0x7ff34019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60822/0 (socket says 172.21.15.46:60822) 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff33fe00640 1 -- 172.21.15.46:0/2010918144 learned_addr learned my addr 172.21.15.46:0/2010918144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff337e00640 1 -- 172.21.15.46:0/2010918144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff340103e30 msgr2=0x7ff34019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff337e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff340103e30 0x7ff34019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff337e00640 1 -- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340108f60 msgr2=0x7ff34019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff337e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340108f60 0x7ff34019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.918+0000 7ff337e00640 1 -- 172.21.15.46:0/2010918144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34019f430 con 0x7ff3401047e0 2024-09-10T07:49:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.919+0000 7ff337e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3401047e0 0x7ff34019a9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9f0 tx=0x7ff32800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.919+0000 7ff33cc00640 1 -- 172.21.15.46:0/2010918144 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32800ccc0 con 0x7ff3401047e0 2024-09-10T07:49:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.919+0000 7ff344e00640 1 -- 172.21.15.46:0/2010918144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34019f720 con 0x7ff3401047e0 2024-09-10T07:49:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.919+0000 7ff33cc00640 1 -- 172.21.15.46:0/2010918144 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff328005020 con 0x7ff3401047e0 2024-09-10T07:49:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.919+0000 7ff33cc00640 1 -- 172.21.15.46:0/2010918144 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3280049e0 con 0x7ff3401047e0 2024-09-10T07:49:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.919+0000 7ff344e00640 1 -- 172.21.15.46:0/2010918144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3401a7000 con 0x7ff3401047e0 2024-09-10T07:49:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.922+0000 7ff349056740 1 Processor -- start 2024-09-10T07:49:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.922+0000 7ff349056740 1 -- start start 2024-09-10T07:49:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.922+0000 7ff33cc00640 1 -- 172.21.15.46:0/2010918144 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff328005560 con 0x7ff3401047e0 2024-09-10T07:49:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.922+0000 7ff349056740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc8693900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.922+0000 7ff349056740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc869e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff349056740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 0x55ccc86a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff349056740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ccc85f9650 con 0x55ccc8693520 2024-09-10T07:49:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff349056740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ccc85a0f10 con 0x55ccc8693e40 2024-09-10T07:49:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff349056740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ccc85f5610 con 0x55ccc869e700 2024-09-10T07:49:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc8693900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc869e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc869e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37114/0 (socket says 172.21.15.46:37114) 2024-09-10T07:49:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33f400640 1 -- 172.21.15.46:0/2800901671 learned_addr learned my addr 172.21.15.46:0/2800901671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff337e00640 1 --2- 172.21.15.46:0/2800901671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 0x55ccc86a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33f400640 1 -- 172.21.15.46:0/2800901671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 msgr2=0x55ccc86a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33f400640 1 --2- 172.21.15.46:0/2800901671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 0x55ccc86a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33f400640 1 -- 172.21.15.46:0/2800901671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 msgr2=0x55ccc8693900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33f400640 1 --2- 172.21.15.46:0/2800901671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc8693900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.923+0000 7ff33f400640 1 -- 172.21.15.46:0/2800901671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc8308580 con 0x55ccc8693e40 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.924+0000 7ff33f400640 1 --2- 172.21.15.46:0/2800901671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc869e1c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff330009900 tx=0x7ff330004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab700350e0162575 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.924+0000 7ff334c00640 1 -- 172.21.15.46:0/2800901671 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330035040 con 0x55ccc8693e40 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.924+0000 7ff334c00640 1 -- 172.21.15.46:0/2800901671 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff330005520 con 0x55ccc8693e40 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.924+0000 7ff334c00640 1 -- 172.21.15.46:0/2800901671 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330003850 con 0x55ccc8693e40 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.924+0000 7ff33cc00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3240a2b80 0x7ff3240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff33cc00640 1 -- 172.21.15.46:0/2010918144 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff3280cf390 con 0x7ff3401047e0 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff33fe00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3240a2b80 0x7ff3240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 -- 172.21.15.46:0/2800901671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 msgr2=0x55ccc869e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 --2- 172.21.15.46:0/2800901671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc869e1c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff330009900 tx=0x7ff330004040 comp rx=0 tx=0).stop 2024-09-10T07:49:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 -- 172.21.15.46:0/2800901671 shutdown_connections 2024-09-10T07:49:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 --2- 172.21.15.46:0/2800901671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 0x55ccc86a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 --2- 172.21.15.46:0/2800901671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc869e1c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 --2- 172.21.15.46:0/2800901671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc8693900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 -- 172.21.15.46:0/2800901671 >> 172.21.15.46:0/2800901671 conn(0x55ccc868fab0 msgr2=0x55ccc868fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 -- 172.21.15.46:0/2800901671 shutdown_connections 2024-09-10T07:49:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.925+0000 7ff349056740 1 -- 172.21.15.46:0/2800901671 wait complete. 2024-09-10T07:49:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff33fe00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3240a2b80 0x7ff3240a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff338009270 tx=0x7ff338009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 1 -- start start 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 1 Processor -- start 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc869b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693e40 0x55ccc869bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc869c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ccc86a12d0 con 0x55ccc8693520 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ccc85b4350 con 0x55ccc869e700 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ccc86a1450 con 0x55ccc8693e40 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.926+0000 7ff349056740 10 client.0 _ll_register_callbacks cb 0x55ccc85d7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc869c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc869c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:37120/0 (socket says 172.21.15.46:37120) 2024-09-10T07:49:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 -- 172.21.15.46:0/1042169709 learned_addr learned my addr 172.21.15.46:0/1042169709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33f400640 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc869b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff337e00640 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693e40 0x55ccc869bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 -- 172.21.15.46:0/1042169709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693e40 msgr2=0x55ccc869bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693e40 0x55ccc869bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 -- 172.21.15.46:0/1042169709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 msgr2=0x55ccc869b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc869b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 -- 172.21.15.46:0/1042169709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc83d2960 con 0x55ccc869e700 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff33fe00640 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc869c020 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff338011900 tx=0x7ff338004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c77d820ee3fb82b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.927+0000 7ff32d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.928+0000 7ff32d600640 1 -- 172.21.15.46:0/1042169709 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338020bc0 con 0x55ccc869e700 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.928+0000 7ff349056740 1 -- 172.21.15.46:0/1042169709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff330005520 con 0x55ccc869e700 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.928+0000 7ff32d600640 1 -- 172.21.15.46:0/1042169709 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff338020d60 con 0x55ccc869e700 2024-09-10T07:49:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.928+0000 7ff32d600640 1 -- 172.21.15.46:0/1042169709 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff338029440 con 0x55ccc869e700 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.928+0000 7ff349056740 10 client.28478 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.928+0000 7ff349056740 1 -- 172.21.15.46:0/1042169709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ccc86acf50 con 0x55ccc869e700 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.928+0000 7ff32d600640 1 -- 172.21.15.46:0/1042169709 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff330005520 con 0x55ccc869e700 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.929+0000 7ff32d600640 1 -- 172.21.15.46:0/1042169709 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7ff338027070 con 0x55ccc869e700 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.929+0000 7ff349056740 10 client.28478 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.929+0000 7ff349056740 10 client.28478 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.929+0000 7ff349056740 20 client.28478 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.929+0000 7ff349056740 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ccc86ae360 0x55ccc86b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.929+0000 7ff349056740 4 client.28478 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.929+0000 7ff349056740 1 -- 172.21.15.46:0/1042169709 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccc8606bc0 con 0x55ccc86ae360 2024-09-10T07:49:04.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.929+0000 7ff33f400640 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ccc86ae360 0x55ccc86b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.930+0000 7ff33f400640 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ccc86ae360 0x55ccc86b0800 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ac51579b719f7a server_cookie=1147bd43349651be in_seq=0 out_seq=0 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.930+0000 7ff32d600640 10 client.28478 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.930+0000 7ff32d600640 1 -- 172.21.15.46:0/1042169709 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccc8606bc0 con 0x55ccc86ae360 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.930+0000 7ff32d600640 10 client.28478 handle_command_reply: tid=0 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.930+0000 7ff349056740 1 client.28478 shutdown 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 -- 172.21.15.46:0/1042169709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 msgr2=0x55ccc869c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc869c020 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff338011900 tx=0x7ff338004a40 comp rx=0 tx=0).stop 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 -- 172.21.15.46:0/1042169709 shutdown_connections 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ccc86ae360 0x55ccc86b0800 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc869c020 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693e40 0x55ccc869bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 --2- 172.21.15.46:0/1042169709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc869b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 -- 172.21.15.46:0/1042169709 >> 172.21.15.46:0/1042169709 conn(0x55ccc868fab0 msgr2=0x55ccc868ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:04.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff32d600640 0 client.28478 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:04.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 -- 172.21.15.46:0/1042169709 shutdown_connections 2024-09-10T07:49:04.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 1 -- 172.21.15.46:0/1042169709 wait complete. 2024-09-10T07:49:04.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.931+0000 7ff349056740 20 client.28478 trim_cache size 0 max 16384 2024-09-10T07:49:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.949+0000 7ff349056740 1 Processor -- start 2024-09-10T07:49:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.949+0000 7ff349056740 1 -- start start 2024-09-10T07:49:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.949+0000 7ff349056740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693520 0x55ccc869a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.949+0000 7ff349056740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693e40 0x55ccc8695f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.949+0000 7ff349056740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc86b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.949+0000 7ff349056740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3401b0c50 con 0x55ccc8693e40 2024-09-10T07:49:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.949+0000 7ff349056740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3401b2790 con 0x55ccc869e700 2024-09-10T07:49:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.949+0000 7ff349056740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3401b1320 con 0x55ccc8693520 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff337e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693e40 0x55ccc8695f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693520 0x55ccc869a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693520 0x55ccc869a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60862/0 (socket says 172.21.15.46:60862) 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 -- 172.21.15.46:0/2662103149 learned_addr learned my addr 172.21.15.46:0/2662103149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff337e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693e40 0x55ccc8695f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47294/0 (socket says 172.21.15.46:47294) 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33fe00640 1 --2- 172.21.15.46:0/2662103149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc86b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 -- 172.21.15.46:0/2662103149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 msgr2=0x55ccc86b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 --2- 172.21.15.46:0/2662103149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc86b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 -- 172.21.15.46:0/2662103149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693e40 msgr2=0x55ccc8695f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 --2- 172.21.15.46:0/2662103149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693e40 0x55ccc8695f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 -- 172.21.15.46:0/2662103149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc868f8e0 con 0x55ccc8693520 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.950+0000 7ff33f400640 1 --2- 172.21.15.46:0/2662103149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693520 0x55ccc869a4e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55ccc8690910 tx=0x7ff330005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d85f524b52c42a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff32fe00640 1 -- 172.21.15.46:0/2662103149 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330035040 con 0x55ccc8693520 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff32fe00640 1 -- 172.21.15.46:0/2662103149 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3300046c0 con 0x55ccc8693520 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff32fe00640 1 -- 172.21.15.46:0/2662103149 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300038e0 con 0x55ccc8693520 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff349056740 1 -- 172.21.15.46:0/2662103149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693520 msgr2=0x55ccc869a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff349056740 1 --2- 172.21.15.46:0/2662103149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693520 0x55ccc869a4e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55ccc8690910 tx=0x7ff330005fd0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff349056740 1 -- 172.21.15.46:0/2662103149 shutdown_connections 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff349056740 1 --2- 172.21.15.46:0/2662103149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc869e700 0x55ccc86b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff349056740 1 --2- 172.21.15.46:0/2662103149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693e40 0x55ccc8695f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff349056740 1 --2- 172.21.15.46:0/2662103149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc8693520 0x55ccc869a4e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff349056740 1 -- 172.21.15.46:0/2662103149 >> 172.21.15.46:0/2662103149 conn(0x55ccc868fab0 msgr2=0x55ccc868ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.951+0000 7ff349056740 1 -- 172.21.15.46:0/2662103149 shutdown_connections 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.952+0000 7ff349056740 1 -- 172.21.15.46:0/2662103149 wait complete. 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.952+0000 7ff349056740 1 -- start start 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.952+0000 7ff349056740 1 Processor -- start 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff349056740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc86a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff349056740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc86a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff349056740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 0x55ccc86a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff349056740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3401b20c0 con 0x55ccc8693520 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff349056740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3401b19f0 con 0x55ccc8693e40 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff349056740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3401aec30 con 0x55ccc869e700 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff349056740 10 client.0 _ll_register_callbacks cb 0x7ff338011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc86a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff337e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc86a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc86a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47306/0 (socket says 172.21.15.46:47306) 2024-09-10T07:49:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33f400640 1 -- 172.21.15.46:0/129932057 learned_addr learned my addr 172.21.15.46:0/129932057 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33fe00640 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 0x55ccc86a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33f400640 1 -- 172.21.15.46:0/129932057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 msgr2=0x55ccc86a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33f400640 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 0x55ccc86a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33f400640 1 -- 172.21.15.46:0/129932057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 msgr2=0x55ccc86a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33f400640 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc86a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff33f400640 1 -- 172.21.15.46:0/129932057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc86a2af0 con 0x55ccc8693520 2024-09-10T07:49:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.953+0000 7ff337e00640 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc86a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.954+0000 7ff33f400640 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc86a6c40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff330031e40 tx=0x7ff33002ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4fe78de6996973 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.954+0000 7ff32d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:49:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.954+0000 7ff32d600640 1 -- 172.21.15.46:0/129932057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330035070 con 0x55ccc8693520 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.954+0000 7ff32d600640 1 -- 172.21.15.46:0/129932057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff3300317b0 con 0x55ccc8693520 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.954+0000 7ff349056740 1 -- 172.21.15.46:0/129932057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3300046c0 con 0x55ccc8693520 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.954+0000 7ff32d600640 1 -- 172.21.15.46:0/129932057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff33006a050 con 0x55ccc8693520 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.954+0000 7ff349056740 10 client.17715 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.954+0000 7ff349056740 1 -- 172.21.15.46:0/129932057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ccc8718f80 con 0x55ccc8693520 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff32d600640 1 -- 172.21.15.46:0/129932057 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3300046c0 con 0x55ccc8693520 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff32d600640 1 -- 172.21.15.46:0/129932057 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7ff330050480 con 0x55ccc8693520 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff349056740 10 client.17715 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff349056740 10 client.17715 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff349056740 20 client.17715 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff349056740 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ccc8719150 0x55ccc871b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff349056740 4 client.17715 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c01c2732", "release": true, "await": true}] 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff349056740 1 -- 172.21.15.46:0/129932057 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c01c2732", "release": true, "await": true}) -- 0x55ccc86010d0 con 0x55ccc8719150 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.955+0000 7ff337e00640 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ccc8719150 0x55ccc871b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.956+0000 7ff337e00640 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ccc8719150 0x55ccc871b540 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73dca48b65fdb89 server_cookie=7fd9af8bd9c32ddb in_seq=0 out_seq=0 2024-09-10T07:49:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.956+0000 7ff32d600640 10 client.17715 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.993+0000 7ff32d600640 1 -- 172.21.15.46:0/129932057 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ccc86010d0 con 0x55ccc8719150 2024-09-10T07:49:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.993+0000 7ff32d600640 10 client.17715 handle_command_reply: tid=0 2024-09-10T07:49:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.994+0000 7ff349056740 1 client.17715 shutdown 2024-09-10T07:49:04.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff349056740 1 -- 172.21.15.46:0/129932057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 msgr2=0x55ccc86a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff349056740 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc86a6c40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff330031e40 tx=0x7ff33002ab90 comp rx=0 tx=0).stop 2024-09-10T07:49:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff349056740 1 -- 172.21.15.46:0/129932057 shutdown_connections 2024-09-10T07:49:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff349056740 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ccc8719150 0x55ccc871b540 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff349056740 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccc869e700 0x55ccc86a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff349056740 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ccc8693e40 0x55ccc86a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff349056740 1 --2- 172.21.15.46:0/129932057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccc8693520 0x55ccc86a6c40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff32d600640 0 client.17715 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.998+0000 7ff349056740 1 -- 172.21.15.46:0/129932057 >> 172.21.15.46:0/129932057 conn(0x55ccc868fab0 msgr2=0x55ccc869b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.999+0000 7ff349056740 1 -- 172.21.15.46:0/129932057 shutdown_connections 2024-09-10T07:49:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:04.999+0000 7ff349056740 1 -- 172.21.15.46:0/129932057 wait complete. 2024-09-10T07:49:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.000+0000 7ff349056740 20 client.17715 trim_cache size 0 max 16384 2024-09-10T07:49:05.000 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 52, 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 44, 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "c01c2732": { 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 44, 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 138.1, 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:49:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.001+0000 7ff344e00640 1 -- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3240a2b80 msgr2=0x7ff3240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.001+0000 7ff344e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3240a2b80 0x7ff3240a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff338009270 tx=0x7ff338009680 comp rx=0 tx=0).stop 2024-09-10T07:49:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.001+0000 7ff344e00640 1 -- 172.21.15.46:0/2010918144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3401047e0 msgr2=0x7ff34019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.001+0000 7ff344e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3401047e0 0x7ff34019a9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9f0 tx=0x7ff32800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.004+0000 7ff344e00640 1 -- 172.21.15.46:0/2010918144 shutdown_connections 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.004+0000 7ff344e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3240a2b80 0x7ff3240a5040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.004+0000 7ff344e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff340108f60 0x7ff34019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.004+0000 7ff344e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3401047e0 0x7ff34019a9c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.004+0000 7ff344e00640 1 --2- 172.21.15.46:0/2010918144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff340103e30 0x7ff34019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.004+0000 7ff344e00640 1 -- 172.21.15.46:0/2010918144 >> 172.21.15.46:0/2010918144 conn(0x7ff3400ff8d0 msgr2=0x7ff340100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:05.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.004+0000 7ff344e00640 1 -- 172.21.15.46:0/2010918144 shutdown_connections 2024-09-10T07:49:05.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:05.004+0000 7ff344e00640 1 -- 172.21.15.46:0/2010918144 wait complete. 2024-09-10T07:49:05.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 705.0 s, 77.1 tps, lat 2589.283 ms stddev 4349.344 2024-09-10T07:49:06.101 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c01c2732', total seconds elapsed: 22.2 2024-09-10T07:49:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:06 smithi181 ceph-mon[35397]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-09-10T07:49:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:06 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:49:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:06 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:49:06.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:06 smithi063 ceph-mon[35020]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-09-10T07:49:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:06 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:49:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:06 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:49:06.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:06 smithi046 ceph-mon[30705]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-09-10T07:49:06.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:06 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:49:06.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:06 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:49:07.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:07 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:49:07.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:07 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:49:07.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:07 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:49:07.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:07 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:49:07.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:07 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:49:07.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:07 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:49:07.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:07 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:49:07.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:07 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:49:07.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:07 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:49:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.820+0000 7f9eee200640 1 -- 172.21.15.46:0/1916942837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee8109790 msgr2=0x7f9ee8109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.820+0000 7f9eee200640 1 --2- 172.21.15.46:0/1916942837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee8109790 0x7f9ee8109c10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009860 tx=0x7f9ed402a340 comp rx=0 tx=0).stop 2024-09-10T07:49:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.820+0000 7f9eee200640 1 -- 172.21.15.46:0/1916942837 shutdown_connections 2024-09-10T07:49:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.820+0000 7f9eee200640 1 --2- 172.21.15.46:0/1916942837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee810a150 0x7f9ee81109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.820+0000 7f9eee200640 1 --2- 172.21.15.46:0/1916942837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee8109790 0x7f9ee8109c10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.820+0000 7f9eee200640 1 --2- 172.21.15.46:0/1916942837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee8073730 0x7f9ee8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.820+0000 7f9eee200640 1 -- 172.21.15.46:0/1916942837 >> 172.21.15.46:0/1916942837 conn(0x7f9ee806ef20 msgr2=0x7f9ee8071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.821+0000 7f9eee200640 1 -- 172.21.15.46:0/1916942837 shutdown_connections 2024-09-10T07:49:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.821+0000 7f9eee200640 1 -- 172.21.15.46:0/1916942837 wait complete. 2024-09-10T07:49:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.823+0000 7f9eee200640 1 Processor -- start 2024-09-10T07:49:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.824+0000 7f9eee200640 1 -- start start 2024-09-10T07:49:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9eee200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee8073730 0x7f9ee81a2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9eee200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee8109790 0x7f9ee81a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9eee200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee810a150 0x7f9ee81a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9eee200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8113ea0 con 0x7f9ee810a150 2024-09-10T07:49:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9eee200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8113d20 con 0x7f9ee8073730 2024-09-10T07:49:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9eee200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8114020 con 0x7f9ee8109790 2024-09-10T07:49:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9ee7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee8073730 0x7f9ee81a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9ee6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee8109790 0x7f9ee81a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9ee6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee8109790 0x7f9ee81a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40728/0 (socket says 172.21.15.46:40728) 2024-09-10T07:49:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.825+0000 7f9ee6a00640 1 -- 172.21.15.46:0/1533367955 learned_addr learned my addr 172.21.15.46:0/1533367955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.826+0000 7f9ee6a00640 1 -- 172.21.15.46:0/1533367955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee8073730 msgr2=0x7f9ee81a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.826+0000 7f9ee6a00640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee8073730 0x7f9ee81a2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.826+0000 7f9ee6a00640 1 -- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee810a150 msgr2=0x7f9ee81a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.826+0000 7f9ee6a00640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee810a150 0x7f9ee81a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.826+0000 7f9ee6a00640 1 -- 172.21.15.46:0/1533367955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee81aa530 con 0x7f9ee8109790 2024-09-10T07:49:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.826+0000 7f9ee6a00640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee8109790 0x7f9ee81a2e50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4031000 tx=0x7f9ed4033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.827+0000 7f9edfe00640 1 -- 172.21.15.46:0/1533367955 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4037040 con 0x7f9ee8109790 2024-09-10T07:49:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.827+0000 7f9edfe00640 1 -- 172.21.15.46:0/1533367955 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4030850 con 0x7f9ee8109790 2024-09-10T07:49:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.827+0000 7f9edfe00640 1 -- 172.21.15.46:0/1533367955 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4037040 con 0x7f9ee8109790 2024-09-10T07:49:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.827+0000 7f9eee200640 1 -- 172.21.15.46:0/1533367955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee8199580 con 0x7f9ee8109790 2024-09-10T07:49:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.827+0000 7f9eee200640 1 -- 172.21.15.46:0/1533367955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee8199aa0 con 0x7f9ee8109790 2024-09-10T07:49:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.832+0000 7f9ef11dd740 1 Processor -- start 2024-09-10T07:49:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.832+0000 7f9ef11dd740 1 -- start start 2024-09-10T07:49:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.833+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f61490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.833+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f619d0 0x558fa9f66490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.833+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f66a60 0x558fa9f68f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.833+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558fa9ec6100 con 0x558fa9f61090 2024-09-10T07:49:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.833+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558fa9e6e1b0 con 0x558fa9f619d0 2024-09-10T07:49:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.833+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558fa9ec20c0 con 0x558fa9f66a60 2024-09-10T07:49:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.834+0000 7f9ee7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f619d0 0x558fa9f66490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.834+0000 7f9ee7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f619d0 0x558fa9f66490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42532/0 (socket says 172.21.15.46:42532) 2024-09-10T07:49:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.834+0000 7f9ee7400640 1 -- 172.21.15.46:0/3258409648 learned_addr learned my addr 172.21.15.46:0/3258409648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.834+0000 7f9ee7400640 1 -- 172.21.15.46:0/3258409648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f66a60 msgr2=0x558fa9f68f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:49:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.834+0000 7f9ee7400640 1 --2- 172.21.15.46:0/3258409648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f66a60 0x558fa9f68f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.834+0000 7f9ee7400640 1 -- 172.21.15.46:0/3258409648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 msgr2=0x558fa9f61490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.834+0000 7f9ee7400640 1 --2- 172.21.15.46:0/3258409648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f61490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.834+0000 7f9ee7400640 1 -- 172.21.15.46:0/3258409648 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa9bd9210 con 0x558fa9f619d0 2024-09-10T07:49:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.835+0000 7f9ee7400640 1 --2- 172.21.15.46:0/3258409648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f619d0 0x558fa9f66490 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800b7f0 tx=0x7f9ed800bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7c929499066916 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.836+0000 7f9edfe00640 1 -- 172.21.15.46:0/1533367955 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ed4005030 con 0x7f9ee8109790 2024-09-10T07:49:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.836+0000 7f9ed3e00640 1 -- 172.21.15.46:0/3258409648 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800d8d0 con 0x558fa9f619d0 2024-09-10T07:49:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.836+0000 7f9ed3e00640 1 -- 172.21.15.46:0/3258409648 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8018070 con 0x558fa9f619d0 2024-09-10T07:49:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.836+0000 7f9ed3e00640 1 -- 172.21.15.46:0/3258409648 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80123e0 con 0x558fa9f619d0 2024-09-10T07:49:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.838+0000 7f9edfe00640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9ecc0a2f10 0x7f9ecc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.838+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3258409648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f619d0 msgr2=0x558fa9f66490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.838+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3258409648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f619d0 0x558fa9f66490 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800b7f0 tx=0x7f9ed800bcb0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.838+0000 7f9edfe00640 1 -- 172.21.15.46:0/1533367955 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9ed40eec30 con 0x7f9ee8109790 2024-09-10T07:49:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.839+0000 7f9ee7e00640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9ecc0a2f10 0x7f9ecc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.839+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3258409648 shutdown_connections 2024-09-10T07:49:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.839+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3258409648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f66a60 0x558fa9f68f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.839+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3258409648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f619d0 0x558fa9f66490 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.839+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3258409648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f61490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.839+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3258409648 >> 172.21.15.46:0/3258409648 conn(0x558fa9f5c700 msgr2=0x558fa9f5cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.840+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3258409648 shutdown_connections 2024-09-10T07:49:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.840+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3258409648 wait complete. 2024-09-10T07:49:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.841+0000 7f9ef11dd740 1 -- start start 2024-09-10T07:49:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.841+0000 7f9ef11dd740 1 Processor -- start 2024-09-10T07:49:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.842+0000 7f9ee7e00640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9ecc0a2f10 0x7f9ecc0a53d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000afd0 tx=0x7f9ee000c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.842+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.842+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 0x558fa9f6b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.842+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f72880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.843+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558fa9f69a30 con 0x558fa9f619d0 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.843+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558fa9e4f090 con 0x558fa9f66a60 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.843+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558fa9f69bb0 con 0x558fa9f61090 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.843+0000 7f9ef11dd740 10 client.0 _ll_register_callbacks cb 0x558fa9ea3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.843+0000 7f9ee7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f72880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.843+0000 7f9ee7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f72880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42536/0 (socket says 172.21.15.46:42536) 2024-09-10T07:49:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.843+0000 7f9ee7e00640 1 -- 172.21.15.46:0/1259209200 learned_addr learned my addr 172.21.15.46:0/1259209200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.844+0000 7f9ee7400640 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.844+0000 7f9ee6a00640 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 0x558fa9f6b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.845+0000 7f9ee7e00640 1 -- 172.21.15.46:0/1259209200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 msgr2=0x558fa9f6b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.845+0000 7f9ee7e00640 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.845+0000 7f9ee7e00640 1 -- 172.21.15.46:0/1259209200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 msgr2=0x558fa9f6b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.845+0000 7f9ee7e00640 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 0x558fa9f6b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.845+0000 7f9ee7e00640 1 -- 172.21.15.46:0/1259209200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa9ca5620 con 0x558fa9f66a60 2024-09-10T07:49:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.846+0000 7f9ee7e00640 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f72880 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000abd0 tx=0x7f9ee000aec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af86c8643583da24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.846+0000 7f9ed0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.847+0000 7f9ed0c00640 1 -- 172.21.15.46:0/1259209200 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0026060 con 0x558fa9f66a60 2024-09-10T07:49:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.847+0000 7f9ed0c00640 1 -- 172.21.15.46:0/1259209200 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9ee0008dd0 con 0x558fa9f66a60 2024-09-10T07:49:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.848+0000 7f9ef11dd740 1 -- 172.21.15.46:0/1259209200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed8018070 con 0x558fa9f66a60 2024-09-10T07:49:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.852+0000 7f9ed0c00640 1 -- 172.21.15.46:0/1259209200 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9ee0021410 con 0x558fa9f66a60 2024-09-10T07:49:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.852+0000 7f9ef11dd740 10 client.28496 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.852+0000 7f9ef11dd740 1 -- 172.21.15.46:0/1259209200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x558fa9f75720 con 0x558fa9f66a60 2024-09-10T07:49:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.853+0000 7f9ed0c00640 1 -- 172.21.15.46:0/1259209200 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8018070 con 0x558fa9f66a60 2024-09-10T07:49:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.853+0000 7f9ed0c00640 1 -- 172.21.15.46:0/1259209200 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f9ee0016850 con 0x558fa9f66a60 2024-09-10T07:49:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.853+0000 7f9ef11dd740 10 client.28496 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.853+0000 7f9ef11dd740 10 client.28496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.853+0000 7f9ef11dd740 20 client.28496 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.854+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558fa9f76da0 0x558fa9f79190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.854+0000 7f9ef11dd740 4 client.28496 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.854+0000 7f9ef11dd740 1 -- 172.21.15.46:0/1259209200 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fa9ed3830 con 0x558fa9f76da0 2024-09-10T07:49:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.854+0000 7f9ee7400640 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558fa9f76da0 0x558fa9f79190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.855+0000 7f9ee7400640 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558fa9f76da0 0x558fa9f79190 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db6ea9b70ef2ca5 server_cookie=f0f1c74cdee3cc8b in_seq=0 out_seq=0 2024-09-10T07:49:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.855+0000 7f9ed0c00640 10 client.28496 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.856+0000 7f9ed0c00640 1 -- 172.21.15.46:0/1259209200 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fa9ed3830 con 0x558fa9f76da0 2024-09-10T07:49:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.856+0000 7f9ed0c00640 10 client.28496 handle_command_reply: tid=0 2024-09-10T07:49:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 client.28496 shutdown 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 -- 172.21.15.46:0/1259209200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 msgr2=0x558fa9f72880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f72880 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000abd0 tx=0x7f9ee000aec0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 -- 172.21.15.46:0/1259209200 shutdown_connections 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558fa9f76da0 0x558fa9f79190 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f72880 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 0x558fa9f6b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/1259209200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ef11dd740 1 -- 172.21.15.46:0/1259209200 >> 172.21.15.46:0/1259209200 conn(0x558fa9f5c700 msgr2=0x558fa9f5df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.857+0000 7f9ed0c00640 0 client.28496 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.858+0000 7f9ef11dd740 1 -- 172.21.15.46:0/1259209200 shutdown_connections 2024-09-10T07:49:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.858+0000 7f9ef11dd740 1 -- 172.21.15.46:0/1259209200 wait complete. 2024-09-10T07:49:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.858+0000 7f9ef11dd740 20 client.28496 trim_cache size 0 max 16384 2024-09-10T07:49:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.884+0000 7f9ef11dd740 1 Processor -- start 2024-09-10T07:49:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.884+0000 7f9ef11dd740 1 -- start start 2024-09-10T07:49:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.884+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f6b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.884+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f619d0 0x558fa9f72840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.884+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ee6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f619d0 0x558fa9f72840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ee6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f619d0 0x558fa9f72840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40746/0 (socket says 172.21.15.46:40746) 2024-09-10T07:49:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ee7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f6b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ee7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f6b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40728/0 (socket says 172.21.15.46:40728) 2024-09-10T07:49:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ee7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ee7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:42542/0 (socket says 172.21.15.46:42542) 2024-09-10T07:49:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81b9650 con 0x558fa9f61090 2024-09-10T07:49:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81bb190 con 0x558fa9f66a60 2024-09-10T07:49:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81b9d20 con 0x558fa9f619d0 2024-09-10T07:49:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.885+0000 7f9ee6a00640 1 -- 172.21.15.46:0/4220431344 learned_addr learned my addr 172.21.15.46:0/4220431344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.886+0000 7f9ee7400640 1 -- 172.21.15.46:0/4220431344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f619d0 msgr2=0x558fa9f72840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.886+0000 7f9ee7400640 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f619d0 0x558fa9f72840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.886+0000 7f9ee7400640 1 -- 172.21.15.46:0/4220431344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 msgr2=0x558fa9f576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.886+0000 7f9ee7400640 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.886+0000 7f9ee7400640 1 -- 172.21.15.46:0/4220431344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa9f5c520 con 0x558fa9f61090 2024-09-10T07:49:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.886+0000 7f9ee6a00640 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f619d0 0x558fa9f72840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.886+0000 7f9ee7e00640 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.887+0000 7f9ee7400640 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f6b7c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800d530 tx=0x7f9ed8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726a2f7368fb8f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.887+0000 7f9ed3400640 1 -- 172.21.15.46:0/4220431344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed802b070 con 0x558fa9f61090 2024-09-10T07:49:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.887+0000 7f9ed3400640 1 -- 172.21.15.46:0/4220431344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8018040 con 0x558fa9f61090 2024-09-10T07:49:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.887+0000 7f9ed3400640 1 -- 172.21.15.46:0/4220431344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8026780 con 0x558fa9f61090 2024-09-10T07:49:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.888+0000 7f9ef11dd740 1 -- 172.21.15.46:0/4220431344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 msgr2=0x558fa9f6b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.888+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f6b7c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800d530 tx=0x7f9ed8008040 comp rx=0 tx=0).stop 2024-09-10T07:49:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.888+0000 7f9ef11dd740 1 -- 172.21.15.46:0/4220431344 shutdown_connections 2024-09-10T07:49:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.888+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.888+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f619d0 0x558fa9f72840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.888+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/4220431344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f61090 0x558fa9f6b7c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.888+0000 7f9ef11dd740 1 -- 172.21.15.46:0/4220431344 >> 172.21.15.46:0/4220431344 conn(0x558fa9f5c700 msgr2=0x558fa9f6b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.888+0000 7f9ef11dd740 1 -- 172.21.15.46:0/4220431344 shutdown_connections 2024-09-10T07:49:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.889+0000 7f9ef11dd740 1 -- 172.21.15.46:0/4220431344 wait complete. 2024-09-10T07:49:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.889+0000 7f9ef11dd740 1 -- start start 2024-09-10T07:49:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.889+0000 7f9ef11dd740 1 Processor -- start 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 0x558fa9f70250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ef11dd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f71260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81baac0 con 0x558fa9f619d0 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81ba3f0 con 0x558fa9f66a60 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ef11dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81b7650 con 0x558fa9f61090 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ef11dd740 10 client.0 _ll_register_callbacks cb 0x7f9ee000a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ee7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ee7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:40758/0 (socket says 172.21.15.46:40758) 2024-09-10T07:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ee6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 0x558fa9f70250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ee7400640 1 -- 172.21.15.46:0/3093008969 learned_addr learned my addr 172.21.15.46:0/3093008969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.890+0000 7f9ee7e00640 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f71260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.891+0000 7f9ee7400640 1 -- 172.21.15.46:0/3093008969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 msgr2=0x558fa9f71260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.891+0000 7f9ee7400640 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f71260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.891+0000 7f9ee7400640 1 -- 172.21.15.46:0/3093008969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 msgr2=0x558fa9f70250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.891+0000 7f9ee7400640 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 0x558fa9f70250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.891+0000 7f9ee7400640 1 -- 172.21.15.46:0/3093008969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa9f71870 con 0x558fa9f61090 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.891+0000 7f9ee7400640 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6fd10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed8013870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed30eb14987336dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.891+0000 7f9ed0c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.892+0000 7f9ed0c00640 1 -- 172.21.15.46:0/3093008969 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed802b040 con 0x558fa9f61090 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.892+0000 7f9ed0c00640 1 -- 172.21.15.46:0/3093008969 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9ed800d8d0 con 0x558fa9f61090 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.892+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3093008969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed8018040 con 0x558fa9f61090 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.892+0000 7f9ed0c00640 1 -- 172.21.15.46:0/3093008969 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9ed803f050 con 0x558fa9f61090 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.892+0000 7f9ef11dd740 10 client.28429 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.892+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3093008969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x558fa9f72b80 con 0x558fa9f61090 2024-09-10T07:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.892+0000 7f9ed0c00640 1 -- 172.21.15.46:0/3093008969 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8018040 con 0x558fa9f61090 2024-09-10T07:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.893+0000 7f9ed0c00640 1 -- 172.21.15.46:0/3093008969 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f9ed8026b00 con 0x558fa9f61090 2024-09-10T07:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.894+0000 7f9ef11dd740 10 client.28429 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.894+0000 7f9ef11dd740 10 client.28429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.894+0000 7f9ef11dd740 20 client.28429 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.894+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558fa9fe20b0 0x558fa9fe44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.894+0000 7f9ef11dd740 4 client.28429 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.894+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3093008969 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558fa9f57d80 con 0x558fa9fe20b0 2024-09-10T07:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.894+0000 7f9ee6a00640 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558fa9fe20b0 0x558fa9fe44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.895+0000 7f9ee6a00640 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558fa9fe20b0 0x558fa9fe44d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a8a7831078ccbc server_cookie=10e5f775901afb73 in_seq=0 out_seq=0 2024-09-10T07:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.895+0000 7f9ed0c00640 10 client.28429 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.905+0000 7f9ed0c00640 1 -- 172.21.15.46:0/3093008969 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558fa9f57d80 con 0x558fa9fe20b0 2024-09-10T07:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.905+0000 7f9ed0c00640 10 client.28429 handle_command_reply: tid=0 2024-09-10T07:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.905+0000 7f9ef11dd740 1 client.28429 shutdown 2024-09-10T07:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3093008969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 msgr2=0x558fa9f6fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6fd10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed8013870 comp rx=0 tx=0).stop 2024-09-10T07:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3093008969 shutdown_connections 2024-09-10T07:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558fa9fe20b0 0x558fa9fe44d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fa9f66a60 0x558fa9f71260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fa9f619d0 0x558fa9f70250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ef11dd740 1 --2- 172.21.15.46:0/3093008969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558fa9f61090 0x558fa9f6fd10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ed0c00640 0 client.28429 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.906+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3093008969 >> 172.21.15.46:0/3093008969 conn(0x558fa9f5c700 msgr2=0x558fa9f5b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.907+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3093008969 shutdown_connections 2024-09-10T07:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.907+0000 7f9ef11dd740 1 -- 172.21.15.46:0/3093008969 wait complete. 2024-09-10T07:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.907+0000 7f9ef11dd740 20 client.28429 trim_cache size 0 max 16384 2024-09-10T07:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 -- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9ecc0a2f10 msgr2=0x7f9ecc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9ecc0a2f10 0x7f9ecc0a53d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000afd0 tx=0x7f9ee000c040 comp rx=0 tx=0).stop 2024-09-10T07:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 -- 172.21.15.46:0/1533367955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee8109790 msgr2=0x7f9ee81a2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee8109790 0x7f9ee81a2e50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4031000 tx=0x7f9ed4033d10 comp rx=0 tx=0).stop 2024-09-10T07:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 -- 172.21.15.46:0/1533367955 shutdown_connections 2024-09-10T07:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9ecc0a2f10 0x7f9ecc0a53d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee810a150 0x7f9ee81a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee8109790 0x7f9ee81a2e50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 --2- 172.21.15.46:0/1533367955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee8073730 0x7f9ee81a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 -- 172.21.15.46:0/1533367955 >> 172.21.15.46:0/1533367955 conn(0x7f9ee806ef20 msgr2=0x7f9ee810cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 -- 172.21.15.46:0/1533367955 shutdown_connections 2024-09-10T07:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:07.908+0000 7f9eee200640 1 -- 172.21.15.46:0/1533367955 wait complete. 2024-09-10T07:49:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:08 smithi181 ceph-mon[35397]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-09-10T07:49:08.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:08 smithi063 ceph-mon[35020]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-09-10T07:49:08.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:08 smithi046 ceph-mon[30705]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-09-10T07:49:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:10 smithi181 ceph-mon[35397]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 487 KiB/s wr, 76 op/s 2024-09-10T07:49:10.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:10 smithi063 ceph-mon[35020]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 487 KiB/s wr, 76 op/s 2024-09-10T07:49:10.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:10 smithi046 ceph-mon[30705]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 487 KiB/s wr, 76 op/s 2024-09-10T07:49:10.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 710.0 s, 548.0 tps, lat 198.326 ms stddev 501.037 2024-09-10T07:49:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:12 smithi181 ceph-mon[35397]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T07:49:12.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:12 smithi063 ceph-mon[35020]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T07:49:12.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:12 smithi046 ceph-mon[30705]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T07:49:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:14 smithi181 ceph-mon[35397]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.1 MiB/s wr, 275 op/s 2024-09-10T07:49:14.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:14 smithi063 ceph-mon[35020]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.1 MiB/s wr, 275 op/s 2024-09-10T07:49:14.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:14 smithi046 ceph-mon[30705]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.1 MiB/s wr, 275 op/s 2024-09-10T07:49:15.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 715.0 s, 679.0 tps, lat 149.223 ms stddev 165.278 2024-09-10T07:49:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:16 smithi181 ceph-mon[35397]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.1 MiB/s wr, 274 op/s 2024-09-10T07:49:16.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:16 smithi063 ceph-mon[35020]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.1 MiB/s wr, 274 op/s 2024-09-10T07:49:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:16 smithi046 ceph-mon[30705]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.1 MiB/s wr, 274 op/s 2024-09-10T07:49:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:18 smithi181 ceph-mon[35397]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.7 MiB/s wr, 483 op/s 2024-09-10T07:49:18.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:18 smithi063 ceph-mon[35020]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.7 MiB/s wr, 483 op/s 2024-09-10T07:49:18.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:18 smithi046 ceph-mon[30705]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.7 MiB/s wr, 483 op/s 2024-09-10T07:49:20.320 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:20 smithi046 ceph-mon[30705]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 462 op/s 2024-09-10T07:49:20.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:20 smithi181 ceph-mon[35397]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 462 op/s 2024-09-10T07:49:20.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:20 smithi063 ceph-mon[35020]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 462 op/s 2024-09-10T07:49:20.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 720.0 s, 639.4 tps, lat 152.756 ms stddev 167.970 2024-09-10T07:49:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 -- 172.21.15.46:0/3872610748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f041810e130 msgr2=0x7f0418110520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 --2- 172.21.15.46:0/3872610748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f041810e130 0x7f0418110520 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0410009ca0 tx=0x7f041002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 -- 172.21.15.46:0/3872610748 shutdown_connections 2024-09-10T07:49:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 --2- 172.21.15.46:0/3872610748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f041810e130 0x7f0418110520 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 --2- 172.21.15.46:0/3872610748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0418074c50 0x7f04180750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 --2- 172.21.15.46:0/3872610748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0418076230 0x7f0418074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 -- 172.21.15.46:0/3872610748 >> 172.21.15.46:0/3872610748 conn(0x7f04180fd2c0 msgr2=0x7f04180ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 -- 172.21.15.46:0/3872610748 shutdown_connections 2024-09-10T07:49:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.157+0000 7f041ec00640 1 -- 172.21.15.46:0/3872610748 wait complete. 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.159+0000 7f041ec00640 1 Processor -- start 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.159+0000 7f041ec00640 1 -- start start 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.159+0000 7f041ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0418074c50 0x7f041819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.159+0000 7f041ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0418076230 0x7f041819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.159+0000 7f041ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f041810e130 0x7f04181a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.159+0000 7f041ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0418113e40 con 0x7f0418076230 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.159+0000 7f041ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0418113cc0 con 0x7f0418074c50 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.159+0000 7f041ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0418113fc0 con 0x7f041810e130 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0418074c50 0x7f041819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0418074c50 0x7f041819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52160/0 (socket says 172.21.15.46:52160) 2024-09-10T07:49:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417e00640 1 -- 172.21.15.46:0/1528305413 learned_addr learned my addr 172.21.15.46:0/1528305413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f041cc00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f041810e130 0x7f04181a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417400640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0418076230 0x7f041819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417e00640 1 -- 172.21.15.46:0/1528305413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f041810e130 msgr2=0x7f04181a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417e00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f041810e130 0x7f04181a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417e00640 1 -- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0418076230 msgr2=0x7f041819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417e00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0418076230 0x7f041819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417e00640 1 -- 172.21.15.46:0/1528305413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181a5ff0 con 0x7f0418074c50 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.160+0000 7f0417400640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0418076230 0x7f041819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.161+0000 7f0417e00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0418074c50 0x7f041819e3d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f040800ea60 tx=0x7f040800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.161+0000 7f0414c00640 1 -- 172.21.15.46:0/1528305413 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040800cde0 con 0x7f0418074c50 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.161+0000 7f0414c00640 1 -- 172.21.15.46:0/1528305413 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04080035f0 con 0x7f0418074c50 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.161+0000 7f0414c00640 1 -- 172.21.15.46:0/1528305413 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408010670 con 0x7f0418074c50 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.161+0000 7f041ec00640 1 -- 172.21.15.46:0/1528305413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181a62e0 con 0x7f0418074c50 2024-09-10T07:49:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.161+0000 7f041ec00640 1 -- 172.21.15.46:0/1528305413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a67e0 con 0x7f0418074c50 2024-09-10T07:49:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.164+0000 7f0414c00640 1 -- 172.21.15.46:0/1528305413 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04080037a0 con 0x7f0418074c50 2024-09-10T07:49:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.166+0000 7f0414c00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f03fc0a2cb0 0x7f03fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.166+0000 7f0421bea740 1 Processor -- start 2024-09-10T07:49:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.166+0000 7f0414c00640 1 -- 172.21.15.46:0/1528305413 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f04080cf720 con 0x7f0418074c50 2024-09-10T07:49:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.166+0000 7f0417400640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f03fc0a2cb0 0x7f03fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.166+0000 7f0421bea740 1 -- start start 2024-09-10T07:49:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0421bea740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dc0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0421bea740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 0x55a835dc5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0421bea740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dc7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0421bea740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a835d25100 con 0x55a835dc5a60 2024-09-10T07:49:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0421bea740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a835ccd1b0 con 0x55a835dc0090 2024-09-10T07:49:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0421bea740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a835d210c0 con 0x55a835dc09d0 2024-09-10T07:49:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0417e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dc0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0417e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dc0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52166/0 (socket says 172.21.15.46:52166) 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.167+0000 7f0417e00640 1 -- 172.21.15.46:0/2633035133 learned_addr learned my addr 172.21.15.46:0/2633035133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f0417400640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f03fc0a2cb0 0x7f03fc0a5170 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f041819f8e0 tx=0x7f0404005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f0417400640 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 0x55a835dc5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f041cc00640 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dc7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f0417e00640 1 -- 172.21.15.46:0/2633035133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 msgr2=0x55a835dc5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f0417e00640 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 0x55a835dc5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f0417e00640 1 -- 172.21.15.46:0/2633035133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 msgr2=0x55a835dc7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f0417e00640 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dc7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f0417e00640 1 -- 172.21.15.46:0/2633035133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a835a38210 con 0x55a835dc0090 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f041cc00640 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dc7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f0417e00640 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dc0490 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f04080a42e0 tx=0x7f040809b500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172b26e94763156a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f040d200640 1 -- 172.21.15.46:0/2633035133 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040809ff50 con 0x55a835dc0090 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f040d200640 1 -- 172.21.15.46:0/2633035133 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04080a4810 con 0x55a835dc0090 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.168+0000 7f040d200640 1 -- 172.21.15.46:0/2633035133 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040809aff0 con 0x55a835dc0090 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.169+0000 7f0421bea740 1 -- 172.21.15.46:0/2633035133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 msgr2=0x55a835dc0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.169+0000 7f0421bea740 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dc0490 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f04080a42e0 tx=0x7f040809b500 comp rx=0 tx=0).stop 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.169+0000 7f0421bea740 1 -- 172.21.15.46:0/2633035133 shutdown_connections 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.169+0000 7f0421bea740 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dc7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.169+0000 7f0421bea740 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 0x55a835dc5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.169+0000 7f0421bea740 1 --2- 172.21.15.46:0/2633035133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dc0490 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.169+0000 7f0421bea740 1 -- 172.21.15.46:0/2633035133 >> 172.21.15.46:0/2633035133 conn(0x55a835dbb700 msgr2=0x55a835dbbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.170+0000 7f0421bea740 1 -- 172.21.15.46:0/2633035133 shutdown_connections 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.170+0000 7f0421bea740 1 -- 172.21.15.46:0/2633035133 wait complete. 2024-09-10T07:49:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.170+0000 7f0421bea740 1 -- start start 2024-09-10T07:49:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0421bea740 1 Processor -- start 2024-09-10T07:49:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0421bea740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0421bea740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 0x55a835dca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0421bea740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dd17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0421bea740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a835dc89d0 con 0x55a835dc5a60 2024-09-10T07:49:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0421bea740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a835cae090 con 0x55a835dc0090 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0421bea740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a835dc8b50 con 0x55a835dc09d0 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0421bea740 10 client.0 _ll_register_callbacks cb 0x55a835d02f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.171+0000 7f0417e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 0x55a835dca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52168/0 (socket says 172.21.15.46:52168) 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417e00640 1 -- 172.21.15.46:0/1620257525 learned_addr learned my addr 172.21.15.46:0/1620257525 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f041cc00640 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dd17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417e00640 1 -- 172.21.15.46:0/1620257525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 msgr2=0x55a835dca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417e00640 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 0x55a835dca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417e00640 1 -- 172.21.15.46:0/1620257525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 msgr2=0x55a835dd17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417e00640 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dd17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417e00640 1 -- 172.21.15.46:0/1620257525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a835b04620 con 0x55a835dc0090 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0417e00640 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dca240 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f040801f000 tx=0x7f04080a0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f6858f1f9adedd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.172+0000 7f0401600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.173+0000 7f0401600640 1 -- 172.21.15.46:0/1620257525 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04080ba050 con 0x55a835dc0090 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.173+0000 7f0421bea740 1 -- 172.21.15.46:0/1620257525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04080a4810 con 0x55a835dc0090 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.173+0000 7f0401600640 1 -- 172.21.15.46:0/1620257525 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f040809aff0 con 0x55a835dc0090 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.173+0000 7f0401600640 1 -- 172.21.15.46:0/1620257525 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f040809b1f0 con 0x55a835dc0090 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.173+0000 7f0421bea740 10 client.28514 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.173+0000 7f0421bea740 1 -- 172.21.15.46:0/1620257525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55a835dd4520 con 0x55a835dc0090 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.173+0000 7f0401600640 1 -- 172.21.15.46:0/1620257525 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04080a4810 con 0x55a835dc0090 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.174+0000 7f0401600640 1 -- 172.21.15.46:0/1620257525 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f040809aaf0 con 0x55a835dc0090 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.174+0000 7f0421bea740 10 client.28514 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.174+0000 7f0421bea740 10 client.28514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.174+0000 7f0421bea740 20 client.28514 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.174+0000 7f0421bea740 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a835dd5870 0x55a835dd7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.174+0000 7f0421bea740 4 client.28514 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.174+0000 7f0421bea740 1 -- 172.21.15.46:0/1620257525 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a835d32830 con 0x55a835dd5870 2024-09-10T07:49:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.174+0000 7f0417400640 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a835dd5870 0x55a835dd7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.175+0000 7f0417400640 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a835dd5870 0x55a835dd7c60 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd510599af4c3be6 server_cookie=750c43771c167b3 in_seq=0 out_seq=0 2024-09-10T07:49:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.175+0000 7f0401600640 10 client.28514 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.176+0000 7f0401600640 1 -- 172.21.15.46:0/1620257525 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a835d32830 con 0x55a835dd5870 2024-09-10T07:49:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.176+0000 7f0401600640 10 client.28514 handle_command_reply: tid=0 2024-09-10T07:49:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 client.28514 shutdown 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 -- 172.21.15.46:0/1620257525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 msgr2=0x55a835dca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dca240 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f040801f000 tx=0x7f04080a0040 comp rx=0 tx=0).stop 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 -- 172.21.15.46:0/1620257525 shutdown_connections 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a835dd5870 0x55a835dd7c60 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0401600640 0 client.28514 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc5a60 0x55a835dd17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc09d0 0x55a835dca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 --2- 172.21.15.46:0/1620257525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc0090 0x55a835dca240 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 -- 172.21.15.46:0/1620257525 >> 172.21.15.46:0/1620257525 conn(0x55a835dbb700 msgr2=0x55a835dbcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 -- 172.21.15.46:0/1620257525 shutdown_connections 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.177+0000 7f0421bea740 1 -- 172.21.15.46:0/1620257525 wait complete. 2024-09-10T07:49:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.178+0000 7f0421bea740 20 client.28514 trim_cache size 0 max 16384 2024-09-10T07:49:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.201+0000 7f0421bea740 1 Processor -- start 2024-09-10T07:49:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.201+0000 7f0421bea740 1 -- start start 2024-09-10T07:49:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.201+0000 7f0421bea740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.201+0000 7f0421bea740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 0x55a835dd1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0421bea740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 0x55a835db68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0421bea740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b0db0 con 0x55a835dc09d0 2024-09-10T07:49:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0421bea740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b28f0 con 0x55a835dc5a60 2024-09-10T07:49:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0421bea740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b1480 con 0x55a835dc0090 2024-09-10T07:49:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 0x55a835dd1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:41056/0 (socket says 172.21.15.46:41056) 2024-09-10T07:49:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417e00640 1 -- 172.21.15.46:0/1370261199 learned_addr learned my addr 172.21.15.46:0/1370261199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f041cc00640 1 --2- 172.21.15.46:0/1370261199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 0x55a835db68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417e00640 1 -- 172.21.15.46:0/1370261199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 msgr2=0x55a835db68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417e00640 1 --2- 172.21.15.46:0/1370261199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 0x55a835db68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417e00640 1 -- 172.21.15.46:0/1370261199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 msgr2=0x55a835dd1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417e00640 1 --2- 172.21.15.46:0/1370261199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 0x55a835dd1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.202+0000 7f0417e00640 1 -- 172.21.15.46:0/1370261199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a835dd28d0 con 0x55a835dc0090 2024-09-10T07:49:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.203+0000 7f0417e00640 1 --2- 172.21.15.46:0/1370261199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dca700 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0408085080 tx=0x7f040809b500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71e795fd6eb3d0dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.203+0000 7f0403e00640 1 -- 172.21.15.46:0/1370261199 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04080a4810 con 0x55a835dc0090 2024-09-10T07:49:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.203+0000 7f0403e00640 1 -- 172.21.15.46:0/1370261199 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040807f070 con 0x55a835dc0090 2024-09-10T07:49:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.203+0000 7f0403e00640 1 -- 172.21.15.46:0/1370261199 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040809aa60 con 0x55a835dc0090 2024-09-10T07:49:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 -- 172.21.15.46:0/1370261199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 msgr2=0x55a835dca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 --2- 172.21.15.46:0/1370261199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dca700 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0408085080 tx=0x7f040809b500 comp rx=0 tx=0).stop 2024-09-10T07:49:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 -- 172.21.15.46:0/1370261199 shutdown_connections 2024-09-10T07:49:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 --2- 172.21.15.46:0/1370261199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 0x55a835db68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 --2- 172.21.15.46:0/1370261199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 0x55a835dd1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 --2- 172.21.15.46:0/1370261199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dca700 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 -- 172.21.15.46:0/1370261199 >> 172.21.15.46:0/1370261199 conn(0x55a835dbb700 msgr2=0x55a835dc9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 -- 172.21.15.46:0/1370261199 shutdown_connections 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 -- 172.21.15.46:0/1370261199 wait complete. 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 -- start start 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.204+0000 7f0421bea740 1 Processor -- start 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0421bea740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0421bea740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 0x55a835dcee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0421bea740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 0x55a835dcfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0421bea740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b2220 con 0x55a835dc09d0 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0421bea740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b1b50 con 0x55a835dc5a60 2024-09-10T07:49:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0421bea740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04181aed90 con 0x55a835dc0090 2024-09-10T07:49:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0421bea740 10 client.0 _ll_register_callbacks cb 0x7f040809ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:41072/0 (socket says 172.21.15.46:41072) 2024-09-10T07:49:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417e00640 1 -- 172.21.15.46:0/538023896 learned_addr learned my addr 172.21.15.46:0/538023896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f041cc00640 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 0x55a835dcfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417400640 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 0x55a835dcee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417e00640 1 -- 172.21.15.46:0/538023896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 msgr2=0x55a835dcfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417e00640 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 0x55a835dcfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417e00640 1 -- 172.21.15.46:0/538023896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 msgr2=0x55a835dcee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417e00640 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 0x55a835dcee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.205+0000 7f0417e00640 1 -- 172.21.15.46:0/538023896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a835dd04b0 con 0x55a835dc0090 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.206+0000 7f0417e00640 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dce950 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0408099e50 tx=0x7f040809b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f99f258c984f0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.206+0000 7f0401600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.206+0000 7f0401600640 1 -- 172.21.15.46:0/538023896 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04080a4810 con 0x55a835dc0090 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.206+0000 7f0421bea740 1 -- 172.21.15.46:0/538023896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040807f070 con 0x55a835dc0090 2024-09-10T07:49:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.206+0000 7f0401600640 1 -- 172.21.15.46:0/538023896 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0408004210 con 0x55a835dc0090 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.206+0000 7f0401600640 1 -- 172.21.15.46:0/538023896 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0408003c10 con 0x55a835dc0090 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.206+0000 7f0421bea740 10 client.28447 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.206+0000 7f0421bea740 1 -- 172.21.15.46:0/538023896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55a835dd04b0 con 0x55a835dc0090 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.208+0000 7f0401600640 1 -- 172.21.15.46:0/538023896 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040807f070 con 0x55a835dc0090 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.208+0000 7f0401600640 1 -- 172.21.15.46:0/538023896 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f04080ab070 con 0x55a835dc0090 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.208+0000 7f0421bea740 10 client.28447 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.208+0000 7f0421bea740 10 client.28447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.208+0000 7f0421bea740 20 client.28447 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.208+0000 7f0421bea740 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a835e404d0 0x55a835e428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.208+0000 7f0421bea740 4 client.28447 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:49:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.208+0000 7f0421bea740 1 -- 172.21.15.46:0/538023896 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a835d2cb80 con 0x55a835e404d0 2024-09-10T07:49:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.210+0000 7f0417400640 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a835e404d0 0x55a835e428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.213+0000 7f0401600640 10 client.28447 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.213+0000 7f0417400640 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a835e404d0 0x55a835e428c0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30156c13dc802e67 server_cookie=e5e56c0edaa065a4 in_seq=0 out_seq=0 2024-09-10T07:49:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0401600640 1 -- 172.21.15.46:0/538023896 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a835d2cb80 con 0x55a835e404d0 2024-09-10T07:49:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0401600640 10 client.28447 handle_command_reply: tid=0 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 client.28447 shutdown 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 -- 172.21.15.46:0/538023896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 msgr2=0x55a835dce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dce950 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0408099e50 tx=0x7f040809b630 comp rx=0 tx=0).stop 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 -- 172.21.15.46:0/538023896 shutdown_connections 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a835e404d0 0x55a835e428c0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a835dc5a60 0x55a835dcfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a835dc09d0 0x55a835dcee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 --2- 172.21.15.46:0/538023896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a835dc0090 0x55a835dce950 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 -- 172.21.15.46:0/538023896 >> 172.21.15.46:0/538023896 conn(0x55a835dbb700 msgr2=0x55a835dba060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0401600640 0 client.28447 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.218+0000 7f0421bea740 1 -- 172.21.15.46:0/538023896 shutdown_connections 2024-09-10T07:49:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.219+0000 7f0421bea740 1 -- 172.21.15.46:0/538023896 wait complete. 2024-09-10T07:49:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.219+0000 7f0421bea740 20 client.28447 trim_cache size 0 max 16384 2024-09-10T07:49:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:49:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:49:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:49:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:49:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 -- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f03fc0a2cb0 msgr2=0x7f03fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f03fc0a2cb0 0x7f03fc0a5170 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f041819f8e0 tx=0x7f0404005f90 comp rx=0 tx=0).stop 2024-09-10T07:49:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 -- 172.21.15.46:0/1528305413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0418074c50 msgr2=0x7f041819e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0418074c50 0x7f041819e3d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f040800ea60 tx=0x7f040800ef20 comp rx=0 tx=0).stop 2024-09-10T07:49:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 -- 172.21.15.46:0/1528305413 shutdown_connections 2024-09-10T07:49:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f041810e130 0x7f04181a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f03fc0a2cb0 0x7f03fc0a5170 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0418076230 0x7f041819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 --2- 172.21.15.46:0/1528305413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0418074c50 0x7f041819e3d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 -- 172.21.15.46:0/1528305413 >> 172.21.15.46:0/1528305413 conn(0x7f04180fd2c0 msgr2=0x7f04181014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 -- 172.21.15.46:0/1528305413 shutdown_connections 2024-09-10T07:49:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:21.221+0000 7f041ec00640 1 -- 172.21.15.46:0/1528305413 wait complete. 2024-09-10T07:49:21.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:49:21.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:49:21.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:49:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:22 smithi181 ceph-mon[35397]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.9 MiB/s wr, 503 op/s 2024-09-10T07:49:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:22 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:49:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:22 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:49:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:22 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:49:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:22 smithi063 ceph-mon[35020]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.9 MiB/s wr, 503 op/s 2024-09-10T07:49:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:22 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:49:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:22 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:49:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:22 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:49:22.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:22 smithi046 ceph-mon[30705]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.9 MiB/s wr, 503 op/s 2024-09-10T07:49:22.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:22 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:49:22.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:22 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:49:22.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:22 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:49:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:24 smithi181 ceph-mon[35397]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.5 MiB/s wr, 628 op/s 2024-09-10T07:49:24.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:24 smithi063 ceph-mon[35020]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.5 MiB/s wr, 628 op/s 2024-09-10T07:49:24.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:24 smithi046 ceph-mon[30705]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.5 MiB/s wr, 628 op/s 2024-09-10T07:49:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 725.0 s, 443.4 tps, lat 227.653 ms stddev 284.614 2024-09-10T07:49:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:26 smithi181 ceph-mon[35397]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 457 op/s 2024-09-10T07:49:26.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:26 smithi063 ceph-mon[35020]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 457 op/s 2024-09-10T07:49:26.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:26 smithi046 ceph-mon[30705]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 457 op/s 2024-09-10T07:49:27.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:49:28.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:28 smithi181 ceph-mon[35397]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.6 MiB/s wr, 611 op/s 2024-09-10T07:49:28.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:28 smithi063 ceph-mon[35020]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.6 MiB/s wr, 611 op/s 2024-09-10T07:49:28.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:28 smithi046 ceph-mon[30705]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.6 MiB/s wr, 611 op/s 2024-09-10T07:49:29.935 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:30.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:30 smithi046 ceph-mon[30705]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2024-09-10T07:49:30.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:30 smithi181 ceph-mon[35397]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2024-09-10T07:49:30.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:30 smithi063 ceph-mon[35020]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2024-09-10T07:49:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 730.0 s, 478.6 tps, lat 201.800 ms stddev 226.790 2024-09-10T07:49:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.718+0000 7fba1cc00640 1 -- 172.21.15.46:0/4074719737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 msgr2=0x7fba18119410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.718+0000 7fba1cc00640 1 --2- 172.21.15.46:0/4074719737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 0x7fba18119410 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fba10009be0 tx=0x7fba1002a6e0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.720+0000 7fba1cc00640 1 -- 172.21.15.46:0/4074719737 shutdown_connections 2024-09-10T07:49:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.720+0000 7fba1cc00640 1 --2- 172.21.15.46:0/4074719737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 0x7fba18119410 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.720+0000 7fba1cc00640 1 --2- 172.21.15.46:0/4074719737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1810c590 0x7fba1810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.720+0000 7fba1cc00640 1 --2- 172.21.15.46:0/4074719737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba18112640 0x7fba18112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.720+0000 7fba1cc00640 1 -- 172.21.15.46:0/4074719737 >> 172.21.15.46:0/4074719737 conn(0x7fba1806cdc0 msgr2=0x7fba1806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.720+0000 7fba1cc00640 1 -- 172.21.15.46:0/4074719737 shutdown_connections 2024-09-10T07:49:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.720+0000 7fba1cc00640 1 -- 172.21.15.46:0/4074719737 wait complete. 2024-09-10T07:49:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.721+0000 7fba1cc00640 1 Processor -- start 2024-09-10T07:49:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.722+0000 7fba1cc00640 1 -- start start 2024-09-10T07:49:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.723+0000 7fba1cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1810c590 0x7fba181a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.723+0000 7fba1cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 0x7fba181a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.723+0000 7fba1cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18112640 0x7fba181a7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.723+0000 7fba15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1810c590 0x7fba181a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.723+0000 7fba15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1810c590 0x7fba181a2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38494/0 (socket says 172.21.15.46:38494) 2024-09-10T07:49:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.726+0000 7fba15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 0x7fba181a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.726+0000 7fba15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 0x7fba181a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52138/0 (socket says 172.21.15.46:52138) 2024-09-10T07:49:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.727+0000 7fba15e00640 1 -- 172.21.15.46:0/2117536840 learned_addr learned my addr 172.21.15.46:0/2117536840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.727+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba1811c9a0 con 0x7fba1810cf30 2024-09-10T07:49:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.727+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba1811c820 con 0x7fba1810c590 2024-09-10T07:49:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.727+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fba1811cb20 con 0x7fba18112640 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.727+0000 7fba16800640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18112640 0x7fba181a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.728+0000 7fba16800640 1 -- 172.21.15.46:0/2117536840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1810c590 msgr2=0x7fba181a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.728+0000 7fba16800640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1810c590 0x7fba181a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.728+0000 7fba16800640 1 -- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 msgr2=0x7fba181a3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.728+0000 7fba16800640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 0x7fba181a3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.728+0000 7fba16800640 1 -- 172.21.15.46:0/2117536840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba181a7cf0 con 0x7fba18112640 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.728+0000 7fba15e00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1810c590 0x7fba181a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.728+0000 7fba15400640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 0x7fba181a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.728+0000 7fba16800640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18112640 0x7fba181a7610 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fba10034040 tx=0x7fba10030be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.729+0000 7fba0ea00640 1 -- 172.21.15.46:0/2117536840 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10036040 con 0x7fba18112640 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.729+0000 7fba0ea00640 1 -- 172.21.15.46:0/2117536840 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba100483a0 con 0x7fba18112640 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.729+0000 7fba0ea00640 1 -- 172.21.15.46:0/2117536840 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1002f600 con 0x7fba18112640 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.729+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba181a7f80 con 0x7fba18112640 2024-09-10T07:49:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.729+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba181a8380 con 0x7fba18112640 2024-09-10T07:49:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.735+0000 7fba0ea00640 1 -- 172.21.15.46:0/2117536840 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba10050060 con 0x7fba18112640 2024-09-10T07:49:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.737+0000 7fba0ea00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb9fc0a2cd0 0x7fb9fc0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.737+0000 7fba0ea00640 1 -- 172.21.15.46:0/2117536840 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fba100f7d10 con 0x7fba18112640 2024-09-10T07:49:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.738+0000 7fba15e00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb9fc0a2cd0 0x7fb9fc0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.740+0000 7fba15e00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb9fc0a2cd0 0x7fb9fc0a5190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fba04005fd0 tx=0x7fba04005de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.741+0000 7fba1f85b740 1 Processor -- start 2024-09-10T07:49:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.742+0000 7fba1f85b740 1 -- start start 2024-09-10T07:49:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.742+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e6141b90 0x55e7e6141f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.743+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 0x55e7e614c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.743+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.743+0000 7fba1f85b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e7e60a83e0 con 0x55e7e6141b90 2024-09-10T07:49:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.743+0000 7fba1f85b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e7e604fd80 con 0x55e7e614cd70 2024-09-10T07:49:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.743+0000 7fba1f85b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e7e60a43a0 con 0x55e7e61424b0 2024-09-10T07:49:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.743+0000 7fba15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e6141b90 0x55e7e6141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.743+0000 7fba15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e6141b90 0x55e7e6141f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52148/0 (socket says 172.21.15.46:52148) 2024-09-10T07:49:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.743+0000 7fba15e00640 1 -- 172.21.15.46:0/1870148327 learned_addr learned my addr 172.21.15.46:0/1870148327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.744+0000 7fba16800640 1 --2- 172.21.15.46:0/1870148327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.744+0000 7fba16800640 1 -- 172.21.15.46:0/1870148327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 msgr2=0x55e7e614c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.744+0000 7fba16800640 1 --2- 172.21.15.46:0/1870148327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 0x55e7e614c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.744+0000 7fba16800640 1 -- 172.21.15.46:0/1870148327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e6141b90 msgr2=0x55e7e6141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.744+0000 7fba16800640 1 --2- 172.21.15.46:0/1870148327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e6141b90 0x55e7e6141f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.744+0000 7fba16800640 1 -- 172.21.15.46:0/1870148327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e606eae0 con 0x55e7e614cd70 2024-09-10T07:49:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.745+0000 7fba16800640 1 --2- 172.21.15.46:0/1870148327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614f160 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fba100f5490 tx=0x7fba100c4080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=872b8d88dc061926 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.745+0000 7fba03000640 1 -- 172.21.15.46:0/1870148327 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100c41f0 con 0x55e7e614cd70 2024-09-10T07:49:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.745+0000 7fba03000640 1 -- 172.21.15.46:0/1870148327 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba100c85a0 con 0x55e7e614cd70 2024-09-10T07:49:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.746+0000 7fba1f85b740 1 -- 172.21.15.46:0/1870148327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 msgr2=0x55e7e614f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.746+0000 7fba1f85b740 1 --2- 172.21.15.46:0/1870148327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614f160 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fba100f5490 tx=0x7fba100c4080 comp rx=0 tx=0).stop 2024-09-10T07:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.746+0000 7fba1f85b740 1 -- 172.21.15.46:0/1870148327 shutdown_connections 2024-09-10T07:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.746+0000 7fba1f85b740 1 --2- 172.21.15.46:0/1870148327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614f160 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.746+0000 7fba1f85b740 1 --2- 172.21.15.46:0/1870148327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 0x55e7e614c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.746+0000 7fba1f85b740 1 --2- 172.21.15.46:0/1870148327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e6141b90 0x55e7e6141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.747+0000 7fba1f85b740 1 -- 172.21.15.46:0/1870148327 >> 172.21.15.46:0/1870148327 conn(0x55e7e613e120 msgr2=0x55e7e613e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.747+0000 7fba1f85b740 1 -- 172.21.15.46:0/1870148327 shutdown_connections 2024-09-10T07:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.747+0000 7fba1f85b740 1 -- 172.21.15.46:0/1870148327 wait complete. 2024-09-10T07:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.750+0000 7fba1f85b740 1 -- start start 2024-09-10T07:49:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.750+0000 7fba1f85b740 1 Processor -- start 2024-09-10T07:49:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.751+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6141b90 0x55e7e6149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.751+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e61424b0 0x55e7e614a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.751+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.751+0000 7fba1f85b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e7e614f700 con 0x55e7e61424b0 2024-09-10T07:49:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.751+0000 7fba1f85b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e7e60630f0 con 0x55e7e614cd70 2024-09-10T07:49:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.751+0000 7fba1f85b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e7e614f880 con 0x55e7e6141b90 2024-09-10T07:49:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.751+0000 7fba1f85b740 10 client.0 _ll_register_callbacks cb 0x55e7e60549e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.752+0000 7fba15e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6141b90 0x55e7e6149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.752+0000 7fba15e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6141b90 0x55e7e6149bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34578/0 (socket says 172.21.15.46:34578) 2024-09-10T07:49:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.752+0000 7fba16800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.752+0000 7fba16800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38516/0 (socket says 172.21.15.46:38516) 2024-09-10T07:49:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.752+0000 7fba16800640 1 -- 172.21.15.46:0/4002219778 learned_addr learned my addr 172.21.15.46:0/4002219778 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.753+0000 7fba15400640 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e61424b0 0x55e7e614a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.753+0000 7fba16800640 1 -- 172.21.15.46:0/4002219778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6141b90 msgr2=0x55e7e6149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.753+0000 7fba16800640 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6141b90 0x55e7e6149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.753+0000 7fba16800640 1 -- 172.21.15.46:0/4002219778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e61424b0 msgr2=0x55e7e614a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.753+0000 7fba16800640 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e61424b0 0x55e7e614a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.753+0000 7fba16800640 1 -- 172.21.15.46:0/4002219778 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e5e193c0 con 0x55e7e614cd70 2024-09-10T07:49:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.753+0000 7fba16800640 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614a670 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fba100c0bc0 tx=0x7fba10009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f70916750f3b8f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.753+0000 7fb9f7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.754+0000 7fb9f7400640 1 -- 172.21.15.46:0/4002219778 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100cce60 con 0x55e7e614cd70 2024-09-10T07:49:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.754+0000 7fb9f7400640 1 -- 172.21.15.46:0/4002219778 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fba100e40c0 con 0x55e7e614cd70 2024-09-10T07:49:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.757+0000 7fba1f85b740 1 -- 172.21.15.46:0/4002219778 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba100c85a0 con 0x55e7e614cd70 2024-09-10T07:49:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.757+0000 7fb9f7400640 1 -- 172.21.15.46:0/4002219778 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fba100e42c0 con 0x55e7e614cd70 2024-09-10T07:49:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.757+0000 7fba1f85b740 10 client.28526 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.757+0000 7fba1f85b740 1 -- 172.21.15.46:0/4002219778 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e7e614f700 con 0x55e7e614cd70 2024-09-10T07:49:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.757+0000 7fb9f7400640 1 -- 172.21.15.46:0/4002219778 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba100c85a0 con 0x55e7e614cd70 2024-09-10T07:49:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.760+0000 7fb9f7400640 1 -- 172.21.15.46:0/4002219778 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7fba100ccad0 con 0x55e7e614cd70 2024-09-10T07:49:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.760+0000 7fba1f85b740 10 client.28526 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.760+0000 7fba1f85b740 10 client.28526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.760+0000 7fba1f85b740 20 client.28526 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.760+0000 7fba1f85b740 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e7e616d550 0x55e7e616f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.760+0000 7fba1f85b740 4 client.28526 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.760+0000 7fba1f85b740 1 -- 172.21.15.46:0/4002219778 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7e60b5950 con 0x55e7e616d550 2024-09-10T07:49:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.761+0000 7fba15e00640 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e7e616d550 0x55e7e616f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.763+0000 7fba15e00640 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e7e616d550 0x55e7e616f9b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3518e5f319a251 server_cookie=b418027158338f3b in_seq=0 out_seq=0 2024-09-10T07:49:31.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.763+0000 7fb9f7400640 10 client.28526 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:31.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.766+0000 7fb9f7400640 1 -- 172.21.15.46:0/4002219778 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7e60b5950 con 0x55e7e616d550 2024-09-10T07:49:31.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.766+0000 7fb9f7400640 10 client.28526 handle_command_reply: tid=0 2024-09-10T07:49:31.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.766+0000 7fba1f85b740 1 client.28526 shutdown 2024-09-10T07:49:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.766+0000 7fba1f85b740 1 -- 172.21.15.46:0/4002219778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 msgr2=0x55e7e614a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.766+0000 7fba1f85b740 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614a670 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fba100c0bc0 tx=0x7fba10009520 comp rx=0 tx=0).stop 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 1 -- 172.21.15.46:0/4002219778 shutdown_connections 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e7e616d550 0x55e7e616f9b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e614cd70 0x55e7e614a670 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e61424b0 0x55e7e614a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 1 --2- 172.21.15.46:0/4002219778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6141b90 0x55e7e6149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 1 -- 172.21.15.46:0/4002219778 >> 172.21.15.46:0/4002219778 conn(0x55e7e613e120 msgr2=0x55e7e6143e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fb9f7400640 0 client.28526 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 1 -- 172.21.15.46:0/4002219778 shutdown_connections 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 1 -- 172.21.15.46:0/4002219778 wait complete. 2024-09-10T07:49:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.767+0000 7fba1f85b740 20 client.28526 trim_cache size 0 max 16384 2024-09-10T07:49:31.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.804+0000 7fba1f85b740 1 Processor -- start 2024-09-10T07:49:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.804+0000 7fba1f85b740 1 -- start start 2024-09-10T07:49:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.804+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e6141b90 0x55e7e614b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.804+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 0x55e7e614a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.804+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 0x55e7e6175280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.804+0000 7fba1f85b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba181b5220 con 0x55e7e614cd70 2024-09-10T07:49:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.804+0000 7fba1f85b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba181b6d60 con 0x55e7e6141b90 2024-09-10T07:49:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.804+0000 7fba1f85b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fba181b58f0 con 0x55e7e61424b0 2024-09-10T07:49:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e6141b90 0x55e7e614b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e6141b90 0x55e7e614b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38520/0 (socket says 172.21.15.46:38520) 2024-09-10T07:49:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15e00640 1 -- 172.21.15.46:0/2240047672 learned_addr learned my addr 172.21.15.46:0/2240047672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15400640 1 --2- 172.21.15.46:0/2240047672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 0x55e7e614a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15e00640 1 -- 172.21.15.46:0/2240047672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 msgr2=0x55e7e614a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15e00640 1 --2- 172.21.15.46:0/2240047672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 0x55e7e614a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15e00640 1 -- 172.21.15.46:0/2240047672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 msgr2=0x55e7e6175280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15e00640 1 --2- 172.21.15.46:0/2240047672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 0x55e7e6175280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.805+0000 7fba15e00640 1 -- 172.21.15.46:0/2240047672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e613df50 con 0x55e7e6141b90 2024-09-10T07:49:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.807+0000 7fba15e00640 1 --2- 172.21.15.46:0/2240047672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e6141b90 0x55e7e614b450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55e7e61701f0 tx=0x7fba04010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e3911133074d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:31.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.807+0000 7fba02600640 1 -- 172.21.15.46:0/2240047672 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0401d430 con 0x55e7e6141b90 2024-09-10T07:49:31.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.808+0000 7fba02600640 1 -- 172.21.15.46:0/2240047672 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0401dab0 con 0x55e7e6141b90 2024-09-10T07:49:31.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.808+0000 7fba02600640 1 -- 172.21.15.46:0/2240047672 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04031b20 con 0x55e7e6141b90 2024-09-10T07:49:31.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.809+0000 7fba1f85b740 1 -- 172.21.15.46:0/2240047672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e6141b90 msgr2=0x55e7e614b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.809+0000 7fba1f85b740 1 --2- 172.21.15.46:0/2240047672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e6141b90 0x55e7e614b450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55e7e61701f0 tx=0x7fba04010000 comp rx=0 tx=0).stop 2024-09-10T07:49:31.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.809+0000 7fba1f85b740 1 -- 172.21.15.46:0/2240047672 shutdown_connections 2024-09-10T07:49:31.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.809+0000 7fba1f85b740 1 --2- 172.21.15.46:0/2240047672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 0x55e7e6175280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.809+0000 7fba1f85b740 1 --2- 172.21.15.46:0/2240047672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e61424b0 0x55e7e614a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.809+0000 7fba1f85b740 1 --2- 172.21.15.46:0/2240047672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e6141b90 0x55e7e614b450 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.809+0000 7fba1f85b740 1 -- 172.21.15.46:0/2240047672 >> 172.21.15.46:0/2240047672 conn(0x55e7e613e120 msgr2=0x55e7e6143e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:31.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.810+0000 7fba1f85b740 1 -- 172.21.15.46:0/2240047672 shutdown_connections 2024-09-10T07:49:31.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.810+0000 7fba1f85b740 1 -- 172.21.15.46:0/2240047672 wait complete. 2024-09-10T07:49:31.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.811+0000 7fba1f85b740 1 -- start start 2024-09-10T07:49:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.812+0000 7fba1f85b740 1 Processor -- start 2024-09-10T07:49:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.812+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e61424b0 0x55e7e6146c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.813+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 0x55e7e6147150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.813+0000 7fba1f85b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6147690 0x55e7e61d7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.814+0000 7fba15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e61424b0 0x55e7e6146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.814+0000 7fba15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e61424b0 0x55e7e6146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38532/0 (socket says 172.21.15.46:38532) 2024-09-10T07:49:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.815+0000 7fba15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 0x55e7e6147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.815+0000 7fba16800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6147690 0x55e7e61d7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.815+0000 7fba16800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6147690 0x55e7e61d7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34598/0 (socket says 172.21.15.46:34598) 2024-09-10T07:49:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.815+0000 7fba1f85b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba181b6690 con 0x55e7e614cd70 2024-09-10T07:49:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.815+0000 7fba1f85b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba181b5fc0 con 0x55e7e61424b0 2024-09-10T07:49:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.815+0000 7fba1f85b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fba181b3200 con 0x55e7e6147690 2024-09-10T07:49:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.815+0000 7fba1f85b740 10 client.0 _ll_register_callbacks cb 0x7fba100c0860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.816+0000 7fba15e00640 1 -- 172.21.15.46:0/859341742 learned_addr learned my addr 172.21.15.46:0/859341742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.816+0000 7fba16800640 1 -- 172.21.15.46:0/859341742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e61424b0 msgr2=0x55e7e6146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.816+0000 7fba16800640 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e61424b0 0x55e7e6146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.816+0000 7fba16800640 1 -- 172.21.15.46:0/859341742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 msgr2=0x55e7e6147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.816+0000 7fba16800640 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 0x55e7e6147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.816+0000 7fba16800640 1 -- 172.21.15.46:0/859341742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e61d7790 con 0x55e7e6147690 2024-09-10T07:49:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.817+0000 7fb9f7400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.817+0000 7fba16800640 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6147690 0x55e7e61d7250 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fba100e4460 tx=0x7fba10009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a8765f8d455c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.817+0000 7fb9f7400640 1 -- 172.21.15.46:0/859341742 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100c3b50 con 0x55e7e6147690 2024-09-10T07:49:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.817+0000 7fba1f85b740 1 -- 172.21.15.46:0/859341742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba0401dab0 con 0x55e7e6147690 2024-09-10T07:49:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.818+0000 7fb9f7400640 1 -- 172.21.15.46:0/859341742 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fba10048550 con 0x55e7e6147690 2024-09-10T07:49:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.818+0000 7fb9f7400640 1 -- 172.21.15.46:0/859341742 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fba100c3b50 con 0x55e7e6147690 2024-09-10T07:49:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.818+0000 7fba1f85b740 10 client.28462 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.818+0000 7fba1f85b740 1 -- 172.21.15.46:0/859341742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e7e61d7790 con 0x55e7e6147690 2024-09-10T07:49:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.819+0000 7fb9f7400640 1 -- 172.21.15.46:0/859341742 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0401dab0 con 0x55e7e6147690 2024-09-10T07:49:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.819+0000 7fb9f7400640 1 -- 172.21.15.46:0/859341742 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7fba10004c60 con 0x55e7e6147690 2024-09-10T07:49:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.819+0000 7fba1f85b740 10 client.28462 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.819+0000 7fba1f85b740 10 client.28462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.819+0000 7fba1f85b740 20 client.28462 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.819+0000 7fba1f85b740 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e7e61d8c30 0x55e7e61db050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.821+0000 7fba1f85b740 4 client.28462 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:49:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.821+0000 7fba1f85b740 1 -- 172.21.15.46:0/859341742 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7e60afe60 con 0x55e7e61d8c30 2024-09-10T07:49:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.821+0000 7fba15e00640 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e7e61d8c30 0x55e7e61db050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.822+0000 7fb9f7400640 10 client.28462 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.823+0000 7fba15e00640 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e7e61d8c30 0x55e7e61db050 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3dc49f41788f865 server_cookie=ac6372d73885f52a in_seq=0 out_seq=0 2024-09-10T07:49:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.823+0000 7fb9f7400640 1 -- 172.21.15.46:0/859341742 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e7e60afe60 con 0x55e7e61d8c30 2024-09-10T07:49:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.823+0000 7fb9f7400640 10 client.28462 handle_command_reply: tid=0 2024-09-10T07:49:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.823+0000 7fba1f85b740 1 client.28462 shutdown 2024-09-10T07:49:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fba1f85b740 1 -- 172.21.15.46:0/859341742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6147690 msgr2=0x55e7e61d7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fba1f85b740 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6147690 0x55e7e61d7250 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fba100e4460 tx=0x7fba10009500 comp rx=0 tx=0).stop 2024-09-10T07:49:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fba1f85b740 1 -- 172.21.15.46:0/859341742 shutdown_connections 2024-09-10T07:49:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fba1f85b740 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e7e61d8c30 0x55e7e61db050 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fba1f85b740 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e6147690 0x55e7e61d7250 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fba1f85b740 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7e614cd70 0x55e7e6147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fb9f7400640 0 client.28462 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fba1f85b740 1 --2- 172.21.15.46:0/859341742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7e61424b0 0x55e7e6146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.824+0000 7fba1f85b740 1 -- 172.21.15.46:0/859341742 >> 172.21.15.46:0/859341742 conn(0x55e7e613e120 msgr2=0x55e7e61435e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.826+0000 7fba1f85b740 1 -- 172.21.15.46:0/859341742 shutdown_connections 2024-09-10T07:49:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.827+0000 7fba1f85b740 1 -- 172.21.15.46:0/859341742 wait complete. 2024-09-10T07:49:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.827+0000 7fba1f85b740 20 client.28462 trim_cache size 0 max 16384 2024-09-10T07:49:31.827 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:49:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:49:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:49:31.827 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:49:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.830+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb9fc0a2cd0 msgr2=0x7fb9fc0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.830+0000 7fba1cc00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb9fc0a2cd0 0x7fb9fc0a5190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fba04005fd0 tx=0x7fba04005de0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.830+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18112640 msgr2=0x7fba181a7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.830+0000 7fba1cc00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18112640 0x7fba181a7610 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fba10034040 tx=0x7fba10030be0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.831+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 shutdown_connections 2024-09-10T07:49:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.831+0000 7fba1cc00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba18112640 0x7fba181a7610 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.831+0000 7fba1cc00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb9fc0a2cd0 0x7fb9fc0a5190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.831+0000 7fba1cc00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba1810cf30 0x7fba181a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.831+0000 7fba1cc00640 1 --2- 172.21.15.46:0/2117536840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1810c590 0x7fba181a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.831+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 >> 172.21.15.46:0/2117536840 conn(0x7fba1806cdc0 msgr2=0x7fba1810ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.831+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 shutdown_connections 2024-09-10T07:49:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:31.831+0000 7fba1cc00640 1 -- 172.21.15.46:0/2117536840 wait complete. 2024-09-10T07:49:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:32 smithi181 ceph-mon[35397]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.9 MiB/s wr, 512 op/s 2024-09-10T07:49:32.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:32 smithi063 ceph-mon[35020]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.9 MiB/s wr, 512 op/s 2024-09-10T07:49:32.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:32 smithi046 ceph-mon[30705]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.9 MiB/s wr, 512 op/s 2024-09-10T07:49:33.676 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:49:33.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:49:34.340 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:34 smithi181 ceph-mon[35397]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 661 op/s 2024-09-10T07:49:34.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:34 smithi063 ceph-mon[35020]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 661 op/s 2024-09-10T07:49:34.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:34 smithi046 ceph-mon[30705]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 661 op/s 2024-09-10T07:49:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.441+0000 7f67df200640 1 -- 172.21.15.46:0/1971957502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 msgr2=0x7f67d8109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.441+0000 7f67df200640 1 --2- 172.21.15.46:0/1971957502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 0x7f67d8109db0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009ca0 tx=0x7f67d402a8f0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.445+0000 7f67df200640 1 -- 172.21.15.46:0/1971957502 shutdown_connections 2024-09-10T07:49:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.445+0000 7f67df200640 1 --2- 172.21.15.46:0/1971957502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 0x7f67d8109db0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.445+0000 7f67df200640 1 --2- 172.21.15.46:0/1971957502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67d8101580 0x7f67d8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.445+0000 7f67df200640 1 --2- 172.21.15.46:0/1971957502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d8103050 0x7f67d8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.445+0000 7f67df200640 1 -- 172.21.15.46:0/1971957502 >> 172.21.15.46:0/1971957502 conn(0x7f67d80fd0d0 msgr2=0x7f67d80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.445+0000 7f67df200640 1 -- 172.21.15.46:0/1971957502 shutdown_connections 2024-09-10T07:49:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.445+0000 7f67df200640 1 -- 172.21.15.46:0/1971957502 wait complete. 2024-09-10T07:49:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.450+0000 7f67df200640 1 Processor -- start 2024-09-10T07:49:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.454+0000 7f67df200640 1 -- start start 2024-09-10T07:49:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.454+0000 7f67df200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d8101580 0x7f67d819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.454+0000 7f67df200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 0x7f67d819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.454+0000 7f67df200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67d8103050 0x7f67d81a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.454+0000 7f67df200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810d7c0 con 0x7f67d8101580 2024-09-10T07:49:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.454+0000 7f67df200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810d640 con 0x7f67d8101f40 2024-09-10T07:49:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.454+0000 7f67df200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810d940 con 0x7f67d8103050 2024-09-10T07:49:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67dd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 0x7f67d819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67dd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 0x7f67d819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38558/0 (socket says 172.21.15.46:38558) 2024-09-10T07:49:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67dd400640 1 -- 172.21.15.46:0/2929995831 learned_addr learned my addr 172.21.15.46:0/2929995831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67de800640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67d8103050 0x7f67d81a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67dd400640 1 -- 172.21.15.46:0/2929995831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67d8103050 msgr2=0x7f67d81a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67dd400640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67d8103050 0x7f67d81a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67dd400640 1 -- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d8101580 msgr2=0x7f67d819a000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:49:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67dd400640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d8101580 0x7f67d819a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.455+0000 7f67dd400640 1 -- 172.21.15.46:0/2929995831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d81a1c20 con 0x7f67d8101f40 2024-09-10T07:49:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.456+0000 7f67dd400640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 0x7f67d819a540 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f67c800e9f0 tx=0x7f67c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.457+0000 7f67d2a00640 1 -- 172.21.15.46:0/2929995831 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c800cd80 con 0x7f67d8101f40 2024-09-10T07:49:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.457+0000 7f67df200640 1 -- 172.21.15.46:0/2929995831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d81a1f10 con 0x7f67d8101f40 2024-09-10T07:49:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.457+0000 7f67d2a00640 1 -- 172.21.15.46:0/2929995831 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c8010910 con 0x7f67d8101f40 2024-09-10T07:49:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.457+0000 7f67df200640 1 -- 172.21.15.46:0/2929995831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81a24a0 con 0x7f67d8101f40 2024-09-10T07:49:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.458+0000 7f67d2a00640 1 -- 172.21.15.46:0/2929995831 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8026060 con 0x7f67d8101f40 2024-09-10T07:49:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.464+0000 7f67d2a00640 1 -- 172.21.15.46:0/2929995831 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67c8018910 con 0x7f67d8101f40 2024-09-10T07:49:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.467+0000 7f67e3652740 1 Processor -- start 2024-09-10T07:49:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.467+0000 7f67e3652740 1 -- start start 2024-09-10T07:49:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.468+0000 7f67d2a00640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67c00a2b80 0x7f67c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.469+0000 7f67d2a00640 1 -- 172.21.15.46:0/2929995831 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f67c80cf8e0 con 0x7f67d8101f40 2024-09-10T07:49:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.470+0000 7f67e3652740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 0x5630d91c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.470+0000 7f67e3652740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.470+0000 7f67e3652740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.471+0000 7f67de800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.471+0000 7f67dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.471+0000 7f67de800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38574/0 (socket says 172.21.15.46:38574) 2024-09-10T07:49:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.471+0000 7f67de800640 1 -- 172.21.15.46:0/756197020 learned_addr learned my addr 172.21.15.46:0/756197020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.472+0000 7f67dde00640 1 --2- 172.21.15.46:0/756197020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 0x5630d91c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.472+0000 7f67e3652740 1 -- 172.21.15.46:0/756197020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630d9126100 con 0x5630d91c1ba0 2024-09-10T07:49:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.472+0000 7f67e3652740 1 -- 172.21.15.46:0/756197020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5630d90ce1b0 con 0x5630d91c6c30 2024-09-10T07:49:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.472+0000 7f67e3652740 1 -- 172.21.15.46:0/756197020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5630d91220c0 con 0x5630d91c1260 2024-09-10T07:49:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.472+0000 7f67de800640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67c00a2b80 0x7f67c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.473+0000 7f67dd400640 1 -- 172.21.15.46:0/756197020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 msgr2=0x5630d91c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.473+0000 7f67dd400640 1 --2- 172.21.15.46:0/756197020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 0x5630d91c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.473+0000 7f67dd400640 1 -- 172.21.15.46:0/756197020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 msgr2=0x5630d91c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.473+0000 7f67dd400640 1 --2- 172.21.15.46:0/756197020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.473+0000 7f67dd400640 1 -- 172.21.15.46:0/756197020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d8e39210 con 0x5630d91c1ba0 2024-09-10T07:49:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.474+0000 7f67dd400640 1 --2- 172.21.15.46:0/756197020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91c6660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f67c80cd060 tx=0x7f67c809cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e3ffa4b3bac5dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.475+0000 7f67c7000640 1 -- 172.21.15.46:0/756197020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c809a5c0 con 0x5630d91c1ba0 2024-09-10T07:49:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.475+0000 7f67c7000640 1 -- 172.21.15.46:0/756197020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c8098930 con 0x5630d91c1ba0 2024-09-10T07:49:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.475+0000 7f67c7000640 1 -- 172.21.15.46:0/756197020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8098010 con 0x5630d91c1ba0 2024-09-10T07:49:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.475+0000 7f67de800640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67c00a2b80 0x7f67c00a5040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009a60 tx=0x7f67d402a8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.476+0000 7f67e3652740 1 -- 172.21.15.46:0/756197020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 msgr2=0x5630d91c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.476+0000 7f67e3652740 1 --2- 172.21.15.46:0/756197020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91c6660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f67c80cd060 tx=0x7f67c809cb00 comp rx=0 tx=0).stop 2024-09-10T07:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.479+0000 7f67e3652740 1 -- 172.21.15.46:0/756197020 shutdown_connections 2024-09-10T07:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.479+0000 7f67e3652740 1 --2- 172.21.15.46:0/756197020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.480+0000 7f67e3652740 1 --2- 172.21.15.46:0/756197020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91c6660 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.480+0000 7f67e3652740 1 --2- 172.21.15.46:0/756197020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 0x5630d91c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.480+0000 7f67e3652740 1 -- 172.21.15.46:0/756197020 >> 172.21.15.46:0/756197020 conn(0x5630d91bc8d0 msgr2=0x5630d91bccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.481+0000 7f67e3652740 1 -- 172.21.15.46:0/756197020 shutdown_connections 2024-09-10T07:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.481+0000 7f67e3652740 1 -- 172.21.15.46:0/756197020 wait complete. 2024-09-10T07:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.482+0000 7f67e3652740 1 -- start start 2024-09-10T07:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.482+0000 7f67e3652740 1 Processor -- start 2024-09-10T07:49:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.483+0000 7f67e3652740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1260 0x5630d91cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.483+0000 7f67e3652740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1ba0 0x5630d91cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.483+0000 7f67e3652740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c6c30 0x5630d91d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.483+0000 7f67dde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1260 0x5630d91cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.483+0000 7f67dd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1ba0 0x5630d91cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.483+0000 7f67dd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1ba0 0x5630d91cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34636/0 (socket says 172.21.15.46:34636) 2024-09-10T07:49:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.483+0000 7f67dde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1260 0x5630d91cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38576/0 (socket says 172.21.15.46:38576) 2024-09-10T07:49:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.486+0000 7f67e3652740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630d91c9ba0 con 0x5630d91c6c30 2024-09-10T07:49:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.486+0000 7f67e3652740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5630d90af090 con 0x5630d91c1260 2024-09-10T07:49:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.486+0000 7f67e3652740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5630d91c9d20 con 0x5630d91c1ba0 2024-09-10T07:49:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.486+0000 7f67e3652740 10 client.0 _ll_register_callbacks cb 0x5630d9103f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.486+0000 7f67dd400640 1 -- 172.21.15.46:0/824570944 learned_addr learned my addr 172.21.15.46:0/824570944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.487+0000 7f67dde00640 1 -- 172.21.15.46:0/824570944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1ba0 msgr2=0x5630d91cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.487+0000 7f67dde00640 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1ba0 0x5630d91cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.487+0000 7f67dde00640 1 -- 172.21.15.46:0/824570944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c6c30 msgr2=0x5630d91d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.487+0000 7f67dde00640 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c6c30 0x5630d91d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.487+0000 7f67dde00640 1 -- 172.21.15.46:0/824570944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d8f05620 con 0x5630d91c1260 2024-09-10T07:49:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.488+0000 7f67dde00640 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1260 0x5630d91cb320 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f67d81029c0 tx=0x7f67cc009cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bdd11b4adf8a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.488+0000 7f67bb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.489+0000 7f67bb400640 1 -- 172.21.15.46:0/824570944 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc016a30 con 0x5630d91c1260 2024-09-10T07:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.489+0000 7f67e3652740 1 -- 172.21.15.46:0/824570944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c8098930 con 0x5630d91c1260 2024-09-10T07:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.489+0000 7f67bb400640 1 -- 172.21.15.46:0/824570944 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f67cc016bd0 con 0x5630d91c1260 2024-09-10T07:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.489+0000 7f67bb400640 1 -- 172.21.15.46:0/824570944 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f67cc016dd0 con 0x5630d91c1260 2024-09-10T07:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.489+0000 7f67e3652740 10 client.28544 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.489+0000 7f67e3652740 1 -- 172.21.15.46:0/824570944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5630d91d5760 con 0x5630d91c1260 2024-09-10T07:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.490+0000 7f67bb400640 1 -- 172.21.15.46:0/824570944 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67cc02c070 con 0x5630d91c1260 2024-09-10T07:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.490+0000 7f67bb400640 1 -- 172.21.15.46:0/824570944 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f67cc006730 con 0x5630d91c1260 2024-09-10T07:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.490+0000 7f67e3652740 10 client.28544 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.490+0000 7f67e3652740 10 client.28544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.490+0000 7f67e3652740 20 client.28544 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.490+0000 7f67e3652740 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5630d91d6f10 0x5630d91d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.490+0000 7f67e3652740 4 client.28544 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.490+0000 7f67e3652740 1 -- 172.21.15.46:0/824570944 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630d9133830 con 0x5630d91d6f10 2024-09-10T07:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.491+0000 7f67dd400640 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5630d91d6f10 0x5630d91d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.492+0000 7f67dd400640 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5630d91d6f10 0x5630d91d93b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee8fc8748961cde server_cookie=ae4674baee1c2f02 in_seq=0 out_seq=0 2024-09-10T07:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.492+0000 7f67bb400640 10 client.28544 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.493+0000 7f67bb400640 1 -- 172.21.15.46:0/824570944 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f67c80abe80 con 0x5630d91d6f10 2024-09-10T07:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.493+0000 7f67bb400640 10 client.28544 handle_command_reply: tid=0 2024-09-10T07:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.494+0000 7f67e3652740 1 client.28544 shutdown 2024-09-10T07:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 -- 172.21.15.46:0/824570944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1260 msgr2=0x5630d91cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1260 0x5630d91cb320 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f67d81029c0 tx=0x7f67cc009cb0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 -- 172.21.15.46:0/824570944 shutdown_connections 2024-09-10T07:49:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5630d91d6f10 0x5630d91d93b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c6c30 0x5630d91d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1ba0 0x5630d91cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 --2- 172.21.15.46:0/824570944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1260 0x5630d91cb320 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 -- 172.21.15.46:0/824570944 >> 172.21.15.46:0/824570944 conn(0x5630d91bc8d0 msgr2=0x5630d91be0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67bb400640 0 client.28544 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 -- 172.21.15.46:0/824570944 shutdown_connections 2024-09-10T07:49:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.495+0000 7f67e3652740 1 -- 172.21.15.46:0/824570944 wait complete. 2024-09-10T07:49:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.497+0000 7f67e3652740 20 client.28544 trim_cache size 0 max 16384 2024-09-10T07:49:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.538+0000 7f67e3652740 1 Processor -- start 2024-09-10T07:49:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.538+0000 7f67e3652740 1 -- start start 2024-09-10T07:49:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.539+0000 7f67e3652740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 0x5630d91bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.539+0000 7f67e3652740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.539+0000 7f67e3652740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.540+0000 7f67e3652740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc00fb50 con 0x5630d91c1ba0 2024-09-10T07:49:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.540+0000 7f67e3652740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc011690 con 0x5630d91c6c30 2024-09-10T07:49:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.540+0000 7f67e3652740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc010220 con 0x5630d91c1260 2024-09-10T07:49:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.540+0000 7f67de800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.540+0000 7f67de800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91bd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38584/0 (socket says 172.21.15.46:38584) 2024-09-10T07:49:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.540+0000 7f67de800640 1 -- 172.21.15.46:0/2176119915 learned_addr learned my addr 172.21.15.46:0/2176119915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.540+0000 7f67dd400640 1 --2- 172.21.15.46:0/2176119915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.542+0000 7f67de800640 1 -- 172.21.15.46:0/2176119915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 msgr2=0x5630d91bccb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:49:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.542+0000 7f67de800640 1 --2- 172.21.15.46:0/2176119915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 0x5630d91bccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.542+0000 7f67de800640 1 -- 172.21.15.46:0/2176119915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 msgr2=0x5630d91d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.542+0000 7f67de800640 1 --2- 172.21.15.46:0/2176119915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.542+0000 7f67de800640 1 -- 172.21.15.46:0/2176119915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d91bc6f0 con 0x5630d91c6c30 2024-09-10T07:49:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.544+0000 7f67de800640 1 --2- 172.21.15.46:0/2176119915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91bd1f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f67d4006010 tx=0x7f67d4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7eea008a6e15567 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.545+0000 7f67c6600640 1 -- 172.21.15.46:0/2176119915 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4047040 con 0x5630d91c6c30 2024-09-10T07:49:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.546+0000 7f67c6600640 1 -- 172.21.15.46:0/2176119915 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4051070 con 0x5630d91c6c30 2024-09-10T07:49:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.546+0000 7f67c6600640 1 -- 172.21.15.46:0/2176119915 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d402fc50 con 0x5630d91c6c30 2024-09-10T07:49:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.547+0000 7f67e3652740 1 -- 172.21.15.46:0/2176119915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 msgr2=0x5630d91bd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.547+0000 7f67e3652740 1 --2- 172.21.15.46:0/2176119915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91bd1f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f67d4006010 tx=0x7f67d4005ef0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.548+0000 7f67e3652740 1 -- 172.21.15.46:0/2176119915 shutdown_connections 2024-09-10T07:49:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.548+0000 7f67e3652740 1 --2- 172.21.15.46:0/2176119915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c6c30 0x5630d91bd1f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.548+0000 7f67e3652740 1 --2- 172.21.15.46:0/2176119915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1ba0 0x5630d91d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.548+0000 7f67e3652740 1 --2- 172.21.15.46:0/2176119915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c1260 0x5630d91bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.548+0000 7f67e3652740 1 -- 172.21.15.46:0/2176119915 >> 172.21.15.46:0/2176119915 conn(0x5630d91bc8d0 msgr2=0x5630d91bba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.551+0000 7f67e3652740 1 -- 172.21.15.46:0/2176119915 shutdown_connections 2024-09-10T07:49:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.551+0000 7f67e3652740 1 -- 172.21.15.46:0/2176119915 wait complete. 2024-09-10T07:49:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.552+0000 7f67e3652740 1 -- start start 2024-09-10T07:49:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.552+0000 7f67e3652740 1 Processor -- start 2024-09-10T07:49:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.553+0000 7f67e3652740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1260 0x5630d91cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.553+0000 7f67e3652740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1ba0 0x5630d91cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.553+0000 7f67e3652740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c6c30 0x5630d923fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.554+0000 7f67dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1260 0x5630d91cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67dd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1ba0 0x5630d91cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67dd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1ba0 0x5630d91cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38596/0 (socket says 172.21.15.46:38596) 2024-09-10T07:49:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1260 0x5630d91cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52206/0 (socket says 172.21.15.46:52206) 2024-09-10T07:49:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67e3652740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc010fc0 con 0x5630d91c1260 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67e3652740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc0108f0 con 0x5630d91c1ba0 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67e3652740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc00db30 con 0x5630d91c6c30 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67e3652740 10 client.0 _ll_register_callbacks cb 0x7f67cc0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67de800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c6c30 0x5630d923fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67de800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c6c30 0x5630d923fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34660/0 (socket says 172.21.15.46:34660) 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.555+0000 7f67de800640 1 -- 172.21.15.46:0/2141884467 learned_addr learned my addr 172.21.15.46:0/2141884467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.556+0000 7f67de800640 1 -- 172.21.15.46:0/2141884467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1ba0 msgr2=0x5630d91cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.556+0000 7f67de800640 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1ba0 0x5630d91cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.556+0000 7f67de800640 1 -- 172.21.15.46:0/2141884467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1260 msgr2=0x5630d91cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.556+0000 7f67de800640 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1260 0x5630d91cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.556+0000 7f67de800640 1 -- 172.21.15.46:0/2141884467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d9240180 con 0x5630d91c6c30 2024-09-10T07:49:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.557+0000 7f67de800640 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c6c30 0x5630d923fc40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f67d4031be0 tx=0x7f67d402ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c7dff7db6667a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.557+0000 7f67bb400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.559+0000 7f67bb400640 1 -- 172.21.15.46:0/2141884467 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4059020 con 0x5630d91c6c30 2024-09-10T07:49:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.559+0000 7f67bb400640 1 -- 172.21.15.46:0/2141884467 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f67d4004240 con 0x5630d91c6c30 2024-09-10T07:49:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.559+0000 7f67e3652740 1 -- 172.21.15.46:0/2141884467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d4051070 con 0x5630d91c6c30 2024-09-10T07:49:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.559+0000 7f67bb400640 1 -- 172.21.15.46:0/2141884467 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f67d4004440 con 0x5630d91c6c30 2024-09-10T07:49:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.559+0000 7f67e3652740 10 client.28477 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.559+0000 7f67e3652740 1 -- 172.21.15.46:0/2141884467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5630d9241300 con 0x5630d91c6c30 2024-09-10T07:49:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.561+0000 7f67bb400640 1 -- 172.21.15.46:0/2141884467 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4051070 con 0x5630d91c6c30 2024-09-10T07:49:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.561+0000 7f67bb400640 1 -- 172.21.15.46:0/2141884467 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f67d403b430 con 0x5630d91c6c30 2024-09-10T07:49:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.561+0000 7f67e3652740 10 client.28477 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.561+0000 7f67e3652740 10 client.28477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.561+0000 7f67e3652740 20 client.28477 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.562+0000 7f67e3652740 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5630d9241c00 0x5630d9244020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.562+0000 7f67e3652740 4 client.28477 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:49:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.562+0000 7f67e3652740 1 -- 172.21.15.46:0/2141884467 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630d912db80 con 0x5630d9241c00 2024-09-10T07:49:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.562+0000 7f67dde00640 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5630d9241c00 0x5630d9244020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.565+0000 7f67dde00640 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5630d9241c00 0x5630d9244020 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f563d43e847b2369 server_cookie=a8613582ea7e3ac6 in_seq=0 out_seq=0 2024-09-10T07:49:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.565+0000 7f67bb400640 10 client.28477 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.655+0000 7f67bb400640 1 -- 172.21.15.46:0/2141884467 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630d912db80 con 0x5630d9241c00 2024-09-10T07:49:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.655+0000 7f67bb400640 10 client.28477 handle_command_reply: tid=0 2024-09-10T07:49:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.656+0000 7f67e3652740 1 client.28477 shutdown 2024-09-10T07:49:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.657+0000 7f67e3652740 1 -- 172.21.15.46:0/2141884467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c6c30 msgr2=0x5630d923fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.657+0000 7f67e3652740 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c6c30 0x5630d923fc40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f67d4031be0 tx=0x7f67d402ff70 comp rx=0 tx=0).stop 2024-09-10T07:49:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.657+0000 7f67e3652740 1 -- 172.21.15.46:0/2141884467 shutdown_connections 2024-09-10T07:49:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.657+0000 7f67e3652740 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5630d9241c00 0x5630d9244020 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.657+0000 7f67e3652740 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5630d91c6c30 0x5630d923fc40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.657+0000 7f67e3652740 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630d91c1ba0 0x5630d91cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.657+0000 7f67e3652740 1 --2- 172.21.15.46:0/2141884467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630d91c1260 0x5630d91cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.657+0000 7f67e3652740 1 -- 172.21.15.46:0/2141884467 >> 172.21.15.46:0/2141884467 conn(0x5630d91bc8d0 msgr2=0x5630d91ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.658+0000 7f67bb400640 0 client.28477 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.658+0000 7f67e3652740 1 -- 172.21.15.46:0/2141884467 shutdown_connections 2024-09-10T07:49:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.658+0000 7f67e3652740 1 -- 172.21.15.46:0/2141884467 wait complete. 2024-09-10T07:49:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.658+0000 7f67e3652740 20 client.28477 trim_cache size 0 max 16384 2024-09-10T07:49:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:49:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:49:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:49:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:49:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.659+0000 7f67df200640 1 -- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67c00a2b80 msgr2=0x7f67c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.659+0000 7f67df200640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67c00a2b80 0x7f67c00a5040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009a60 tx=0x7f67d402a8f0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.659+0000 7f67df200640 1 -- 172.21.15.46:0/2929995831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 msgr2=0x7f67d819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.659+0000 7f67df200640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 0x7f67d819a540 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f67c800e9f0 tx=0x7f67c800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.660+0000 7f67df200640 1 -- 172.21.15.46:0/2929995831 shutdown_connections 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.660+0000 7f67df200640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67d8103050 0x7f67d81a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.660+0000 7f67df200640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8101f40 0x7f67d819a540 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.660+0000 7f67df200640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f67c00a2b80 0x7f67c00a5040 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.660+0000 7f67df200640 1 --2- 172.21.15.46:0/2929995831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d8101580 0x7f67d819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.660+0000 7f67df200640 1 -- 172.21.15.46:0/2929995831 >> 172.21.15.46:0/2929995831 conn(0x7f67d80fd0d0 msgr2=0x7f67d8074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.660+0000 7f67df200640 1 -- 172.21.15.46:0/2929995831 shutdown_connections 2024-09-10T07:49:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:34.660+0000 7f67df200640 1 -- 172.21.15.46:0/2929995831 wait complete. 2024-09-10T07:49:35.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.128+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1322446534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 msgr2=0x7fb4dc1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.128+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1322446534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 0x7fb4dc1095d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009ca0 tx=0x7fb4cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.129+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1322446534 shutdown_connections 2024-09-10T07:49:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.129+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1322446534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc116ea0 0x7fb4dc119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.129+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1322446534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4dc109b10 0x7fb4dc116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.129+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1322446534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 0x7fb4dc1095d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.129+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1322446534 >> 172.21.15.46:0/1322446534 conn(0x7fb4dc06e140 msgr2=0x7fb4dc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.129+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1322446534 shutdown_connections 2024-09-10T07:49:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.130+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1322446534 wait complete. 2024-09-10T07:49:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.133+0000 7fb4e2a00640 1 Processor -- start 2024-09-10T07:49:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.134+0000 7fb4e2a00640 1 -- start start 2024-09-10T07:49:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.134+0000 7fb4e2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 0x7fb4dc1a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.134+0000 7fb4e2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4dc109b10 0x7fb4dc1a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.134+0000 7fb4e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc116ea0 0x7fb4dc1a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.136+0000 7fb4dbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 0x7fb4dc1a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.136+0000 7fb4e2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc11ca00 con 0x7fb4dc116ea0 2024-09-10T07:49:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.136+0000 7fb4dbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 0x7fb4dc1a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38614/0 (socket says 172.21.15.46:38614) 2024-09-10T07:49:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.136+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc11c880 con 0x7fb4dc1091f0 2024-09-10T07:49:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.136+0000 7fb4dbe00640 1 -- 172.21.15.46:0/1879355356 learned_addr learned my addr 172.21.15.46:0/1879355356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.136+0000 7fb4db400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4dc109b10 0x7fb4dc1a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.136+0000 7fb4e0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc116ea0 0x7fb4dc1a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.136+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc11cb80 con 0x7fb4dc109b10 2024-09-10T07:49:35.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.137+0000 7fb4db400640 1 -- 172.21.15.46:0/1879355356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 msgr2=0x7fb4dc1a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.137+0000 7fb4db400640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 0x7fb4dc1a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.138+0000 7fb4db400640 1 -- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc116ea0 msgr2=0x7fb4dc1a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.138+0000 7fb4db400640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc116ea0 0x7fb4dc1a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.138+0000 7fb4db400640 1 -- 172.21.15.46:0/1879355356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc1a7df0 con 0x7fb4dc109b10 2024-09-10T07:49:35.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.138+0000 7fb4e0a00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc116ea0 0x7fb4dc1a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:35.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.139+0000 7fb4db400640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4dc109b10 0x7fb4dc1a3380 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8005fd0 tx=0x7fb4c800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.139+0000 7fb4d8c00640 1 -- 172.21.15.46:0/1879355356 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8009c30 con 0x7fb4dc109b10 2024-09-10T07:49:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.140+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc1a80e0 con 0x7fb4dc109b10 2024-09-10T07:49:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.140+0000 7fb4d8c00640 1 -- 172.21.15.46:0/1879355356 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c8009dd0 con 0x7fb4dc109b10 2024-09-10T07:49:35.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.140+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc1af9c0 con 0x7fb4dc109b10 2024-09-10T07:49:35.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.140+0000 7fb4d8c00640 1 -- 172.21.15.46:0/1879355356 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8010460 con 0x7fb4dc109b10 2024-09-10T07:49:35.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.143+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc005290 con 0x7fb4dc109b10 2024-09-10T07:49:35.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.146+0000 7fb4d8c00640 1 -- 172.21.15.46:0/1879355356 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb4c8010600 con 0x7fb4dc109b10 2024-09-10T07:49:35.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.149+0000 7fb4d8c00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.150+0000 7fb4d8c00640 1 -- 172.21.15.46:0/1879355356 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb4c80cf1e0 con 0x7fb4dc109b10 2024-09-10T07:49:35.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.150+0000 7fb4dbe00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.153+0000 7fb4dbe00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006010 tx=0x7fb4cc009860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.156+0000 7fb4d8c00640 1 -- 172.21.15.46:0/1879355356 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4c8017020 con 0x7fb4dc109b10 2024-09-10T07:49:35.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:49:35.533 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":52,"btime":"2024-09-10T07:48:57:965518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:48:57.232057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 52 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.527+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4bc005580 con 0x7fb4dc109b10 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.528+0000 7fb4d8c00640 1 -- 172.21.15.46:0/1879355356 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11972 (secure 0 0 0) 0x7fb4c809b710 con 0x7fb4dc109b10 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.533+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4c00a2c00 msgr2=0x7fb4c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.533+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006010 tx=0x7fb4cc009860 comp rx=0 tx=0).stop 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.533+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4dc109b10 msgr2=0x7fb4dc1a3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.533+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4dc109b10 0x7fb4dc1a3380 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8005fd0 tx=0x7fb4c800dde0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.534+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 shutdown_connections 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.534+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.534+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc116ea0 0x7fb4dc1a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.534+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4dc109b10 0x7fb4dc1a3380 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.534+0000 7fb4e2a00640 1 --2- 172.21.15.46:0/1879355356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4dc1091f0 0x7fb4dc1a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.534+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 >> 172.21.15.46:0/1879355356 conn(0x7fb4dc06e140 msgr2=0x7fb4dc117380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.534+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 shutdown_connections 2024-09-10T07:49:35.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:35.534+0000 7fb4e2a00640 1 -- 172.21.15.46:0/1879355356 wait complete. 2024-09-10T07:49:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 735.0 s, 470.6 tps, lat 214.901 ms stddev 237.645 2024-09-10T07:49:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:36 smithi181 ceph-mon[35397]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-09-10T07:49:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:36 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1879355356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:36.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:36 smithi063 ceph-mon[35020]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-09-10T07:49:36.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:36 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1879355356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:36.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:36 smithi046 ceph-mon[30705]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-09-10T07:49:36.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:36 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1879355356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:37.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:49:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:38.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:38 smithi046 ceph-mon[30705]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 6.5 MiB/s wr, 638 op/s 2024-09-10T07:49:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:38 smithi181 ceph-mon[35397]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 6.5 MiB/s wr, 638 op/s 2024-09-10T07:49:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:38 smithi063 ceph-mon[35020]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 6.5 MiB/s wr, 638 op/s 2024-09-10T07:49:39.717 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T07:49:39.717 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.5 2024-09-10T07:49:39.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:49:40.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 -- 172.21.15.46:0/3778877250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994112a50 msgr2=0x7f0994114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 --2- 172.21.15.46:0/3778877250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994112a50 0x7f0994114e40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f098c009ca0 tx=0x7f098c02a8b0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 -- 172.21.15.46:0/3778877250 shutdown_connections 2024-09-10T07:49:40.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 --2- 172.21.15.46:0/3778877250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994112a50 0x7f0994114e40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 --2- 172.21.15.46:0/3778877250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0994077040 0x7f0994112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 --2- 172.21.15.46:0/3778877250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0994076720 0x7f0994076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 -- 172.21.15.46:0/3778877250 >> 172.21.15.46:0/3778877250 conn(0x7f09940ffa50 msgr2=0x7f0994101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 -- 172.21.15.46:0/3778877250 shutdown_connections 2024-09-10T07:49:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.248+0000 7f0999000640 1 -- 172.21.15.46:0/3778877250 wait complete. 2024-09-10T07:49:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.249+0000 7f0999000640 1 Processor -- start 2024-09-10T07:49:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.249+0000 7f0999000640 1 -- start start 2024-09-10T07:49:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0999000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0994076720 0x7f09941a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0999000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994077040 0x7f09941a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0999000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0994112a50 0x7f099419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0999000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09941185e0 con 0x7f0994112a50 2024-09-10T07:49:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0999000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0994118460 con 0x7f0994077040 2024-09-10T07:49:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0999000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0994118760 con 0x7f0994076720 2024-09-10T07:49:40.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0991e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0994076720 0x7f09941a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0991e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0994076720 0x7f09941a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:45000/0 (socket says 172.21.15.46:45000) 2024-09-10T07:49:40.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0991e00640 1 -- 172.21.15.46:0/3243818045 learned_addr learned my addr 172.21.15.46:0/3243818045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:40.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.250+0000 7f0991400640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994077040 0x7f09941a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.251+0000 7f0992800640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0994112a50 0x7f099419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.251+0000 7f0991e00640 1 -- 172.21.15.46:0/3243818045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994077040 msgr2=0x7f09941a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.251+0000 7f0991e00640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994077040 0x7f09941a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.251+0000 7f0991e00640 1 -- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0994112a50 msgr2=0x7f099419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.251+0000 7f0991e00640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0994112a50 0x7f099419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.251+0000 7f0991e00640 1 -- 172.21.15.46:0/3243818045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099419f0a0 con 0x7f0994076720 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.251+0000 7f0991e00640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0994076720 0x7f09941a47b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f098400e9f0 tx=0x7f098400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.252+0000 7f098aa00640 1 -- 172.21.15.46:0/3243818045 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098400cd80 con 0x7f0994076720 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.252+0000 7f098aa00640 1 -- 172.21.15.46:0/3243818045 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0984010910 con 0x7f0994076720 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.252+0000 7f098aa00640 1 -- 172.21.15.46:0/3243818045 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984018990 con 0x7f0994076720 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.252+0000 7f0999000640 1 -- 172.21.15.46:0/3243818045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099419f390 con 0x7f0994076720 2024-09-10T07:49:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.252+0000 7f0999000640 1 -- 172.21.15.46:0/3243818045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09941ab680 con 0x7f0994076720 2024-09-10T07:49:40.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.256+0000 7f098aa00640 1 -- 172.21.15.46:0/3243818045 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0984010ac0 con 0x7f0994076720 2024-09-10T07:49:40.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.256+0000 7f099bbfe740 1 Processor -- start 2024-09-10T07:49:40.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.257+0000 7f099bbfe740 1 -- start start 2024-09-10T07:49:40.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.257+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815dc70 0x55796815e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.257+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815e590 0x557968168910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.257+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x55796816b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.257+0000 7f099bbfe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5579680c43e0 con 0x55796815e590 2024-09-10T07:49:40.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.257+0000 7f099bbfe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55796806bd80 con 0x557968168e50 2024-09-10T07:49:40.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.257+0000 7f099bbfe740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5579680c03a0 con 0x55796815dc70 2024-09-10T07:49:40.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0991400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x55796816b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0991400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x55796816b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48590/0 (socket says 172.21.15.46:48590) 2024-09-10T07:49:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0991400640 1 -- 172.21.15.46:0/3013223076 learned_addr learned my addr 172.21.15.46:0/3013223076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f098aa00640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f09780a2d50 0x7f09780a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f098aa00640 1 -- 172.21.15.46:0/3243818045 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f09840cf6b0 con 0x7f0994076720 2024-09-10T07:49:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0992800640 1 --2- 172.21.15.46:0/3013223076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815dc70 0x55796815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0991400640 1 -- 172.21.15.46:0/3013223076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815dc70 msgr2=0x55796815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0991400640 1 --2- 172.21.15.46:0/3013223076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815dc70 0x55796815e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0991400640 1 -- 172.21.15.46:0/3013223076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815e590 msgr2=0x557968168910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0991400640 1 --2- 172.21.15.46:0/3013223076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815e590 0x557968168910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.258+0000 7f0991400640 1 -- 172.21.15.46:0/3013223076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55796808aae0 con 0x557968168e50 2024-09-10T07:49:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.259+0000 7f0991400640 1 --2- 172.21.15.46:0/3013223076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x55796816b240 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f098000bbf0 tx=0x7f09800107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d8b03c14d57094 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.260+0000 7f097f400640 1 -- 172.21.15.46:0/3013223076 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980005020 con 0x557968168e50 2024-09-10T07:49:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.260+0000 7f097f400640 1 -- 172.21.15.46:0/3013223076 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09800047a0 con 0x557968168e50 2024-09-10T07:49:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.260+0000 7f097f400640 1 -- 172.21.15.46:0/3013223076 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980005020 con 0x557968168e50 2024-09-10T07:49:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.261+0000 7f0992800640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f09780a2d50 0x7f09780a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.264+0000 7f099bbfe740 1 -- 172.21.15.46:0/3013223076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 msgr2=0x55796816b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.264+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3013223076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x55796816b240 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f098000bbf0 tx=0x7f09800107a0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.264+0000 7f0992800640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f09780a2d50 0x7f09780a5210 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f098c006010 tx=0x7f098c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.265+0000 7f099bbfe740 1 -- 172.21.15.46:0/3013223076 shutdown_connections 2024-09-10T07:49:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.265+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3013223076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x55796816b240 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.265+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3013223076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815e590 0x557968168910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.265+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3013223076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815dc70 0x55796815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.266+0000 7f099bbfe740 1 -- 172.21.15.46:0/3013223076 >> 172.21.15.46:0/3013223076 conn(0x55796815a200 msgr2=0x55796815a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.266+0000 7f099bbfe740 1 -- 172.21.15.46:0/3013223076 shutdown_connections 2024-09-10T07:49:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.266+0000 7f099bbfe740 1 -- 172.21.15.46:0/3013223076 wait complete. 2024-09-10T07:49:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.268+0000 7f099bbfe740 1 -- start start 2024-09-10T07:49:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.268+0000 7f099bbfe740 1 Processor -- start 2024-09-10T07:49:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x557968165db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 0x557968166830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f0991e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x557968165db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f0991400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f099bbfe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55796816ba20 con 0x557968168e50 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f099bbfe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55796807f0f0 con 0x55796815dc70 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f099bbfe740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55796816bba0 con 0x55796815e590 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f0991400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681662f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:45008/0 (socket says 172.21.15.46:45008) 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f0991400640 1 -- 172.21.15.46:0/3453185023 learned_addr learned my addr 172.21.15.46:0/3453185023 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f099bbfe740 10 client.0 _ll_register_callbacks cb 0x557968066bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f0992800640 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 0x557968166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.269+0000 7f0991400640 1 -- 172.21.15.46:0/3453185023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 msgr2=0x557968165db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.270+0000 7f0991400640 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x557968165db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.270+0000 7f0991400640 1 -- 172.21.15.46:0/3453185023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 msgr2=0x557968166830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.270+0000 7f0991400640 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 0x557968166830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.270+0000 7f0991400640 1 -- 172.21.15.46:0/3453185023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557967e353c0 con 0x55796815e590 2024-09-10T07:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.270+0000 7f0991400640 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681662f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0980005bc0 tx=0x7f098000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8016e1f33c3ef82c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.270+0000 7f0973e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.271+0000 7f0973e00640 1 -- 172.21.15.46:0/3453185023 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098002a060 con 0x55796815e590 2024-09-10T07:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.271+0000 7f0973e00640 1 -- 172.21.15.46:0/3453185023 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0980004be0 con 0x55796815e590 2024-09-10T07:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.271+0000 7f099bbfe740 1 -- 172.21.15.46:0/3453185023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09800047a0 con 0x55796815e590 2024-09-10T07:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.271+0000 7f0973e00640 1 -- 172.21.15.46:0/3453185023 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0980031050 con 0x55796815e590 2024-09-10T07:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.271+0000 7f099bbfe740 10 client.28495 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.271+0000 7f099bbfe740 1 -- 172.21.15.46:0/3453185023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557968177ac0 con 0x55796815e590 2024-09-10T07:49:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.272+0000 7f0973e00640 1 -- 172.21.15.46:0/3453185023 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09800047a0 con 0x55796815e590 2024-09-10T07:49:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.272+0000 7f0973e00640 1 -- 172.21.15.46:0/3453185023 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f0980011a80 con 0x55796815e590 2024-09-10T07:49:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.272+0000 7f099bbfe740 10 client.28495 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.272+0000 7f099bbfe740 10 client.28495 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:49:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.272+0000 7f099bbfe740 20 client.28495 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.272+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x557968178ed0 0x55796817b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.272+0000 7f099bbfe740 4 client.28495 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.272+0000 7f099bbfe740 1 -- 172.21.15.46:0/3453185023 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579680d1950 con 0x557968178ed0 2024-09-10T07:49:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.273+0000 7f0991e00640 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x557968178ed0 0x55796817b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.276+0000 7f0991e00640 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x557968178ed0 0x55796817b2c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85482e02f66b2a20 server_cookie=28e96d92c68c511c in_seq=0 out_seq=0 2024-09-10T07:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.276+0000 7f0973e00640 10 client.28495 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.277+0000 7f0973e00640 1 -- 172.21.15.46:0/3453185023 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579680d1950 con 0x557968178ed0 2024-09-10T07:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.277+0000 7f0973e00640 10 client.28495 handle_command_reply: tid=0 2024-09-10T07:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.277+0000 7f099bbfe740 1 client.28495 shutdown 2024-09-10T07:49:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.277+0000 7f099bbfe740 1 -- 172.21.15.46:0/3453185023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 msgr2=0x5579681662f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.277+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681662f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0980005bc0 tx=0x7f098000b870 comp rx=0 tx=0).stop 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f099bbfe740 1 -- 172.21.15.46:0/3453185023 shutdown_connections 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x557968178ed0 0x55796817b2c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 0x557968166830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681662f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f099bbfe740 1 --2- 172.21.15.46:0/3453185023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x557968165db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f099bbfe740 1 -- 172.21.15.46:0/3453185023 >> 172.21.15.46:0/3453185023 conn(0x55796815a200 msgr2=0x55796815ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f0973e00640 0 client.28495 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f099bbfe740 1 -- 172.21.15.46:0/3453185023 shutdown_connections 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.278+0000 7f099bbfe740 1 -- 172.21.15.46:0/3453185023 wait complete. 2024-09-10T07:49:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.279+0000 7f099bbfe740 20 client.28495 trim_cache size 0 max 16384 2024-09-10T07:49:40.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.301+0000 7f099bbfe740 1 Processor -- start 2024-09-10T07:49:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.301+0000 7f099bbfe740 1 -- start start 2024-09-10T07:49:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815dc70 0x557968165080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x557968180890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f099bbfe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b5370 con 0x55796815dc70 2024-09-10T07:49:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f099bbfe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b6eb0 con 0x557968168e50 2024-09-10T07:49:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f099bbfe740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b5a40 con 0x55796815e590 2024-09-10T07:49:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f0991400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f0992800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x557968180890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f0992800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x557968180890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48602/0 (socket says 172.21.15.46:48602) 2024-09-10T07:49:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f0992800640 1 -- 172.21.15.46:0/1890981636 learned_addr learned my addr 172.21.15.46:0/1890981636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.302+0000 7f0991400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:45014/0 (socket says 172.21.15.46:45014) 2024-09-10T07:49:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.303+0000 7f0991e00640 1 --2- 172.21.15.46:0/1890981636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815dc70 0x557968165080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.303+0000 7f0991400640 1 -- 172.21.15.46:0/1890981636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 msgr2=0x557968180890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.303+0000 7f0991400640 1 --2- 172.21.15.46:0/1890981636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x557968180890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.303+0000 7f0991400640 1 -- 172.21.15.46:0/1890981636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815dc70 msgr2=0x557968165080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.303+0000 7f0991400640 1 --2- 172.21.15.46:0/1890981636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815dc70 0x557968165080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.303+0000 7f0991400640 1 -- 172.21.15.46:0/1890981636 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557968155770 con 0x55796815e590 2024-09-10T07:49:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.304+0000 7f0991400640 1 --2- 172.21.15.46:0/1890981636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681606d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0980004e30 tx=0x7f098000b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac1f4efec67bcecf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.304+0000 7f097ea00640 1 -- 172.21.15.46:0/1890981636 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980004640 con 0x55796815e590 2024-09-10T07:49:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.304+0000 7f097ea00640 1 -- 172.21.15.46:0/1890981636 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09800047e0 con 0x55796815e590 2024-09-10T07:49:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.304+0000 7f097ea00640 1 -- 172.21.15.46:0/1890981636 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980004640 con 0x55796815e590 2024-09-10T07:49:40.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.305+0000 7f099bbfe740 1 -- 172.21.15.46:0/1890981636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 msgr2=0x5579681606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.305+0000 7f099bbfe740 1 --2- 172.21.15.46:0/1890981636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681606d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0980004e30 tx=0x7f098000b800 comp rx=0 tx=0).stop 2024-09-10T07:49:40.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.306+0000 7f099bbfe740 1 -- 172.21.15.46:0/1890981636 shutdown_connections 2024-09-10T07:49:40.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.306+0000 7f099bbfe740 1 --2- 172.21.15.46:0/1890981636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557968168e50 0x557968180890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.306+0000 7f099bbfe740 1 --2- 172.21.15.46:0/1890981636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x5579681606d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.306+0000 7f099bbfe740 1 --2- 172.21.15.46:0/1890981636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55796815dc70 0x557968165080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.306+0000 7f099bbfe740 1 -- 172.21.15.46:0/1890981636 >> 172.21.15.46:0/1890981636 conn(0x55796815a200 msgr2=0x557968159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.306+0000 7f099bbfe740 1 -- 172.21.15.46:0/1890981636 shutdown_connections 2024-09-10T07:49:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.306+0000 7f099bbfe740 1 -- 172.21.15.46:0/1890981636 wait complete. 2024-09-10T07:49:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.307+0000 7f099bbfe740 1 -- start start 2024-09-10T07:49:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.307+0000 7f099bbfe740 1 Processor -- start 2024-09-10T07:49:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.309+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x5579681710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.309+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x557968171630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.309+0000 7f099bbfe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 0x55796816df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.309+0000 7f0991e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x5579681710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.309+0000 7f0991e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x5579681710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48608/0 (socket says 172.21.15.46:48608) 2024-09-10T07:49:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.310+0000 7f0991400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x557968171630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.310+0000 7f0991400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x557968171630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:45020/0 (socket says 172.21.15.46:45020) 2024-09-10T07:49:40.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.310+0000 7f099bbfe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b67e0 con 0x557968168e50 2024-09-10T07:49:40.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.310+0000 7f099bbfe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b6110 con 0x55796815dc70 2024-09-10T07:49:40.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.310+0000 7f0991e00640 1 -- 172.21.15.46:0/2148753272 learned_addr learned my addr 172.21.15.46:0/2148753272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:40.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.311+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f09941b3350 con 0x55796815e590 2024-09-10T07:49:40.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.311+0000 7f099bbfe740 10 client.0 _ll_register_callbacks cb 0x7f098000ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.312+0000 7f0992800640 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 0x55796816df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.312+0000 7f0991e00640 1 -- 172.21.15.46:0/2148753272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 msgr2=0x557968171630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.312+0000 7f0991e00640 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x557968171630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.312+0000 7f0991e00640 1 -- 172.21.15.46:0/2148753272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 msgr2=0x55796816df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.312+0000 7f0991e00640 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 0x55796816df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.312+0000 7f0991e00640 1 -- 172.21.15.46:0/2148753272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55796816e700 con 0x55796815dc70 2024-09-10T07:49:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.312+0000 7f0991e00640 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x5579681710f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0984098500 tx=0x7f0984000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eeb6b5327fa8b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.312+0000 7f0973e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.313+0000 7f0973e00640 1 -- 172.21.15.46:0/2148753272 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09840b9060 con 0x55796815dc70 2024-09-10T07:49:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.313+0000 7f0973e00640 1 -- 172.21.15.46:0/2148753272 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f09840cf370 con 0x55796815dc70 2024-09-10T07:49:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.313+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09800047e0 con 0x55796815dc70 2024-09-10T07:49:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.313+0000 7f0973e00640 1 -- 172.21.15.46:0/2148753272 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f098409ce60 con 0x55796815dc70 2024-09-10T07:49:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.313+0000 7f099bbfe740 10 client.28574 fetch_fsmap learned FSMap version 52 2024-09-10T07:49:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.313+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5579681e44f0 con 0x55796815dc70 2024-09-10T07:49:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.316+0000 7f0973e00640 1 -- 172.21.15.46:0/2148753272 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09800047e0 con 0x55796815dc70 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.316+0000 7f0973e00640 1 -- 172.21.15.46:0/2148753272 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f0984018b30 con 0x55796815dc70 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.317+0000 7f099bbfe740 10 client.28574 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.317+0000 7f099bbfe740 10 client.28574 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.317+0000 7f099bbfe740 20 client.28574 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.317+0000 7f099bbfe740 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5579681e4bc0 0x5579681e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.317+0000 7f099bbfe740 4 client.28574 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.317+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579680cbe60 con 0x5579681e4bc0 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.317+0000 7f0991400640 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5579681e4bc0 0x5579681e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.318+0000 7f0973e00640 10 client.28574 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:49:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.320+0000 7f0991400640 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5579681e4bc0 0x5579681e6fe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e77cebdfd6d34cb0 server_cookie=a0e57b9bf098ffad in_seq=0 out_seq=0 2024-09-10T07:49:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.321+0000 7f0973e00640 1 -- 172.21.15.46:0/2148753272 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579680cbe60 con 0x5579681e4bc0 2024-09-10T07:49:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.321+0000 7f0973e00640 10 client.28574 handle_command_reply: tid=0 2024-09-10T07:49:40.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.321+0000 7f099bbfe740 1 client.28574 shutdown 2024-09-10T07:49:40.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 msgr2=0x5579681710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f099bbfe740 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x5579681710f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0984098500 tx=0x7f0984000bc0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 shutdown_connections 2024-09-10T07:49:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f099bbfe740 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x5579681e4bc0 0x5579681e6fe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f0973e00640 0 client.28574 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:49:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f099bbfe740 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557968168e50 0x55796816df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f099bbfe740 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55796815e590 0x557968171630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f099bbfe740 1 --2- 172.21.15.46:0/2148753272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55796815dc70 0x5579681710f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.322+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 >> 172.21.15.46:0/2148753272 conn(0x55796815a200 msgr2=0x557968165d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.323+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 shutdown_connections 2024-09-10T07:49:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.323+0000 7f099bbfe740 1 -- 172.21.15.46:0/2148753272 wait complete. 2024-09-10T07:49:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.324+0000 7f099bbfe740 20 client.28574 trim_cache size 0 max 16384 2024-09-10T07:49:40.325 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:49:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 -- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f09780a2d50 msgr2=0x7f09780a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f09780a2d50 0x7f09780a5210 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f098c006010 tx=0x7f098c0095c0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 -- 172.21.15.46:0/3243818045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0994076720 msgr2=0x7f09941a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0994076720 0x7f09941a47b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f098400e9f0 tx=0x7f098400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 -- 172.21.15.46:0/3243818045 shutdown_connections 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f09780a2d50 0x7f09780a5210 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0994112a50 0x7f099419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994077040 0x7f09941a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 --2- 172.21.15.46:0/3243818045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0994076720 0x7f09941a47b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.327+0000 7f0999000640 1 -- 172.21.15.46:0/3243818045 >> 172.21.15.46:0/3243818045 conn(0x7f09940ffa50 msgr2=0x7f0994101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.329+0000 7f0999000640 1 -- 172.21.15.46:0/3243818045 shutdown_connections 2024-09-10T07:49:40.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:40.329+0000 7f0999000640 1 -- 172.21.15.46:0/3243818045 wait complete. 2024-09-10T07:49:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 740.0 s, 458.4 tps, lat 222.847 ms stddev 292.003 2024-09-10T07:49:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:40 smithi181 ceph-mon[35397]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-09-10T07:49:40.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:40 smithi063 ceph-mon[35020]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-09-10T07:49:40.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:40 smithi046 ceph-mon[30705]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-09-10T07:49:41.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:49:41.898 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:42.450 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:42 smithi181 ceph-mon[35397]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-09-10T07:49:42.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:42 smithi046 ceph-mon[30705]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-09-10T07:49:42.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:42 smithi063 ceph-mon[35020]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-09-10T07:49:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.916+0000 7f660fa00640 1 -- 172.21.15.46:0/1788796458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608106f10 msgr2=0x7f660810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.916+0000 7f660fa00640 1 --2- 172.21.15.46:0/1788796458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608106f10 0x7f660810ea90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009900 tx=0x7f65f802a440 comp rx=0 tx=0).stop 2024-09-10T07:49:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.917+0000 7f660fa00640 1 -- 172.21.15.46:0/1788796458 shutdown_connections 2024-09-10T07:49:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.917+0000 7f660fa00640 1 --2- 172.21.15.46:0/1788796458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608106f10 0x7f660810ea90 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.917+0000 7f660fa00640 1 --2- 172.21.15.46:0/1788796458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608102790 0x7f66081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.917+0000 7f660fa00640 1 --2- 172.21.15.46:0/1788796458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608101de0 0x7f66081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.917+0000 7f660fa00640 1 -- 172.21.15.46:0/1788796458 >> 172.21.15.46:0/1788796458 conn(0x7f66080fd8c0 msgr2=0x7f66080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.918+0000 7f660fa00640 1 -- 172.21.15.46:0/1788796458 shutdown_connections 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.918+0000 7f660fa00640 1 -- 172.21.15.46:0/1788796458 wait complete. 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.920+0000 7f660fa00640 1 Processor -- start 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.921+0000 7f660fa00640 1 -- start start 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660fa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608101de0 0x7f660810bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660fa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608102790 0x7f660810c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608106f10 0x7f660810c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6608076100 con 0x7f6608106f10 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660fa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6608075f80 con 0x7f6608101de0 2024-09-10T07:49:43.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660fa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6608076280 con 0x7f6608102790 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608101de0 0x7f660810bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608101de0 0x7f660810bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:48634/0 (socket says 172.21.15.46:48634) 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.922+0000 7f660d000640 1 -- 172.21.15.46:0/1453343233 learned_addr learned my addr 172.21.15.46:0/1453343233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.923+0000 7f660da00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608106f10 0x7f660810c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.923+0000 7f660d000640 1 -- 172.21.15.46:0/1453343233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608102790 msgr2=0x7f660810c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.923+0000 7f660d000640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608102790 0x7f660810c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.923+0000 7f660d000640 1 -- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608106f10 msgr2=0x7f660810c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.924+0000 7f660d000640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608106f10 0x7f660810c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.924+0000 7f660d000640 1 -- 172.21.15.46:0/1453343233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66081aad40 con 0x7f6608101de0 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.925+0000 7f660da00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608106f10 0x7f660810c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.926+0000 7f660d000640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608101de0 0x7f660810bd00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f660000ebf0 tx=0x7f660000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.927+0000 7f6605600640 1 -- 172.21.15.46:0/1453343233 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600010630 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.927+0000 7f6605600640 1 -- 172.21.15.46:0/1453343233 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6600010cb0 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.927+0000 7f6605600640 1 -- 172.21.15.46:0/1453343233 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600018bc0 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.927+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66081aafd0 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.928+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66081ab530 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.932+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66081021c0 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.933+0000 7f6605600640 1 -- 172.21.15.46:0/1453343233 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66000173d0 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.935+0000 7f6605600640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f65ec0a2c40 0x7f65ec0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.936+0000 7f6607e00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f65ec0a2c40 0x7f65ec0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.938+0000 7f6605600640 1 -- 172.21.15.46:0/1453343233 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f66000cfa20 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.938+0000 7f6607e00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f65ec0a2c40 0x7f65ec0a5100 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6608108540 tx=0x7f65f4006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:42.939+0000 7f6605600640 1 -- 172.21.15.46:0/1453343233 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f660009c600 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.315+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f660802d7c0 con 0x7f6608101de0 2024-09-10T07:49:43.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.318+0000 7f6605600640 1 -- 172.21.15.46:0/1453343233 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11973 (secure 0 0 0) 0x7f660009bf00 con 0x7f6608101de0 2024-09-10T07:49:43.485 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:49:43.485 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"btime":"2024-09-10T07:49:43:169804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:49:43.058969+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:49:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 53 2024-09-10T07:49:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.328+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f65ec0a2c40 msgr2=0x7f65ec0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.328+0000 7f660fa00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f65ec0a2c40 0x7f65ec0a5100 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6608108540 tx=0x7f65f4006c90 comp rx=0 tx=0).stop 2024-09-10T07:49:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.328+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608101de0 msgr2=0x7f660810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.328+0000 7f660fa00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608101de0 0x7f660810bd00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f660000ebf0 tx=0x7f660000c6a0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.329+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 shutdown_connections 2024-09-10T07:49:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.329+0000 7f660fa00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f65ec0a2c40 0x7f65ec0a5100 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.329+0000 7f660fa00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608106f10 0x7f660810c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.329+0000 7f660fa00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608102790 0x7f660810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.329+0000 7f660fa00640 1 --2- 172.21.15.46:0/1453343233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6608101de0 0x7f660810bd00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.329+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 >> 172.21.15.46:0/1453343233 conn(0x7f66080fd8c0 msgr2=0x7f66080feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.329+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 shutdown_connections 2024-09-10T07:49:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:43.331+0000 7f660fa00640 1 -- 172.21.15.46:0/1453343233 wait complete. 2024-09-10T07:49:44.414 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0' 2024-09-10T07:49:44.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0 2024-09-10T07:49:44.469 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:44.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:44 smithi046 ceph-mon[30705]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-09-10T07:49:44.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:44 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:49:44.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:44 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1453343233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:44 smithi181 ceph-mon[35397]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-09-10T07:49:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:44 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:49:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:44 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1453343233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:44 smithi063 ceph-mon[35020]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-09-10T07:49:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:44 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:49:44.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:44 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1453343233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:49:45.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 745.0 s, 502.0 tps, lat 201.644 ms stddev 236.017 2024-09-10T07:49:46.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.980+0000 7f2da5200640 1 -- 172.21.15.46:0/658792656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da006c260 msgr2=0x7f2da006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.980+0000 7f2da5200640 1 --2- 172.21.15.46:0/658792656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da006c260 0x7f2da006c6a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009900 tx=0x7f2d8802a490 comp rx=0 tx=0).stop 2024-09-10T07:49:46.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.980+0000 7f2da5200640 1 -- 172.21.15.46:0/658792656 shutdown_connections 2024-09-10T07:49:46.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.980+0000 7f2da5200640 1 --2- 172.21.15.46:0/658792656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da01137e0 0x7f2da0115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.980+0000 7f2da5200640 1 --2- 172.21.15.46:0/658792656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da006c260 0x7f2da006c6a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.980+0000 7f2da5200640 1 --2- 172.21.15.46:0/658792656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2da0073f30 0x7f2da006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.981+0000 7f2da5200640 1 -- 172.21.15.46:0/658792656 >> 172.21.15.46:0/658792656 conn(0x7f2da00fc610 msgr2=0x7f2da00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.981+0000 7f2da5200640 1 -- 172.21.15.46:0/658792656 shutdown_connections 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.981+0000 7f2da5200640 1 -- 172.21.15.46:0/658792656 wait complete. 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.983+0000 7f2da5200640 1 Processor -- start 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.983+0000 7f2da5200640 1 -- start start 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.985+0000 7f2da5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da006c260 0x7f2da01a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.985+0000 7f2da5200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0073f30 0x7f2da01a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.985+0000 7f2da5200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2da01137e0 0x7f2da019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.986+0000 7f2d97e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0073f30 0x7f2da01a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.986+0000 7f2d97e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0073f30 0x7f2da01a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52432/0 (socket says 172.21.15.46:52432) 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.986+0000 7f2d9fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2da01137e0 0x7f2da019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.986+0000 7f2da5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0118430 con 0x7f2da006c260 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.986+0000 7f2d9fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2da01137e0 0x7f2da019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35680/0 (socket says 172.21.15.46:35680) 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.986+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01182b0 con 0x7f2da0073f30 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.986+0000 7f2d9fe00640 1 -- 172.21.15.46:0/4092683370 learned_addr learned my addr 172.21.15.46:0/4092683370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.986+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01185b0 con 0x7f2da01137e0 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.987+0000 7f2d97e00640 1 -- 172.21.15.46:0/4092683370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2da01137e0 msgr2=0x7f2da019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.987+0000 7f2d97e00640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2da01137e0 0x7f2da019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.987+0000 7f2d97e00640 1 -- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da006c260 msgr2=0x7f2da01a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.987+0000 7f2d97e00640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da006c260 0x7f2da01a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.987+0000 7f2d97e00640 1 -- 172.21.15.46:0/4092683370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da019efb0 con 0x7f2da0073f30 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.987+0000 7f2d97e00640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0073f30 0x7f2da01a4bb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2d8802a9b0 tx=0x7f2d88003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.988+0000 7f2d9cc00640 1 -- 172.21.15.46:0/4092683370 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8803f070 con 0x7f2da0073f30 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.988+0000 7f2d9cc00640 1 -- 172.21.15.46:0/4092683370 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d88031c90 con 0x7f2da0073f30 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.988+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da019f240 con 0x7f2da0073f30 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.988+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da01ab420 con 0x7f2da0073f30 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.988+0000 7f2d9cc00640 1 -- 172.21.15.46:0/4092683370 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88035030 con 0x7f2da0073f30 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.992+0000 7f2d9cc00640 1 -- 172.21.15.46:0/4092683370 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2d88042070 con 0x7f2da0073f30 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.993+0000 7f2da91bb740 1 Processor -- start 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.993+0000 7f2d9cc00640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d840a7090 0x7f2d840a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.993+0000 7f2da91bb740 1 -- start start 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.994+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 0x558a73f22220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.994+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f22760 0x558a73f2cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.994+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f2f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.994+0000 7f2da91bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558a73e883e0 con 0x558a73f2d020 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.994+0000 7f2da91bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558a73e2fd80 con 0x558a73f22760 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.994+0000 7f2d9cc00640 1 -- 172.21.15.46:0/4092683370 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2d880ef610 con 0x7f2da0073f30 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.994+0000 7f2da91bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558a73e843a0 con 0x558a73f21e40 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.994+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d840a7090 0x7f2d840a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.995+0000 7f2d97e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f22760 0x558a73f2cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.995+0000 7f2d97e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f22760 0x558a73f2cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52448/0 (socket says 172.21.15.46:52448) 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.995+0000 7f2d97e00640 1 -- 172.21.15.46:0/1343278467 learned_addr learned my addr 172.21.15.46:0/1343278467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.995+0000 7f2d97e00640 1 -- 172.21.15.46:0/1343278467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 msgr2=0x558a73f22220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.995+0000 7f2d97e00640 1 --2- 172.21.15.46:0/1343278467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 0x558a73f22220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.995+0000 7f2d97e00640 1 -- 172.21.15.46:0/1343278467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 msgr2=0x558a73f2f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.995+0000 7f2d97e00640 1 --2- 172.21.15.46:0/1343278467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f2f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.995+0000 7f2d97e00640 1 -- 172.21.15.46:0/1343278467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a73e4eae0 con 0x558a73f22760 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.996+0000 7f2d97e00640 1 --2- 172.21.15.46:0/1343278467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f22760 0x558a73f2cae0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2d880c5310 tx=0x7f2d88003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04e4968663421af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.997+0000 7f2d94c00640 1 -- 172.21.15.46:0/1343278467 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d880da070 con 0x558a73f22760 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.997+0000 7f2d94c00640 1 -- 172.21.15.46:0/1343278467 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d880bcbd0 con 0x558a73f22760 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.997+0000 7f2d94c00640 1 -- 172.21.15.46:0/1343278467 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88049070 con 0x558a73f22760 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.998+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d840a7090 0x7f2d840a9550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2d98010270 tx=0x7f2d98009710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.998+0000 7f2da91bb740 1 -- 172.21.15.46:0/1343278467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f22760 msgr2=0x558a73f2cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:45.999+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1343278467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f22760 0x558a73f2cae0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2d880c5310 tx=0x7f2d88003f90 comp rx=0 tx=0).stop 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.000+0000 7f2da91bb740 1 -- 172.21.15.46:0/1343278467 shutdown_connections 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.000+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1343278467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f2f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.000+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1343278467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f22760 0x558a73f2cae0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.000+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1343278467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 0x558a73f22220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.000+0000 7f2da91bb740 1 -- 172.21.15.46:0/1343278467 >> 172.21.15.46:0/1343278467 conn(0x558a73f1e3d0 msgr2=0x558a73f1e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.000+0000 7f2da91bb740 1 -- 172.21.15.46:0/1343278467 shutdown_connections 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.000+0000 7f2da91bb740 1 -- 172.21.15.46:0/1343278467 wait complete. 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.001+0000 7f2da91bb740 1 -- start start 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.001+0000 7f2da91bb740 1 Processor -- start 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.001+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 0x558a73f29f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.001+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f22760 0x558a73f2a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.001+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f2d020 0x558a73f2aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.001+0000 7f2da91bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558a73f2f9b0 con 0x558a73f22760 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.001+0000 7f2da91bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558a73e430f0 con 0x558a73f2d020 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.001+0000 7f2da91bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558a73f2fb30 con 0x558a73f21e40 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.002+0000 7f2da91bb740 10 client.0 _ll_register_callbacks cb 0x558a73e2abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.002+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 0x558a73f29f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.002+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 0x558a73f29f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35708/0 (socket says 172.21.15.46:35708) 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.002+0000 7f2d9f400640 1 -- 172.21.15.46:0/1119065489 learned_addr learned my addr 172.21.15.46:0/1119065489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.003+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f2d020 0x558a73f2aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.003+0000 7f2d97e00640 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f22760 0x558a73f2a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.003+0000 7f2d9fe00640 1 -- 172.21.15.46:0/1119065489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 msgr2=0x558a73f29f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.003+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 0x558a73f29f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.003+0000 7f2d9fe00640 1 -- 172.21.15.46:0/1119065489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f22760 msgr2=0x558a73f2a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.003+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f22760 0x558a73f2a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.003+0000 7f2d9fe00640 1 -- 172.21.15.46:0/1119065489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a73bf93c0 con 0x558a73f2d020 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.003+0000 7f2d97e00640 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f22760 0x558a73f2a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.004+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f2d020 0x558a73f2aa00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2d98010b00 tx=0x7f2d98004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae494b5262bdbd50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.004+0000 7f2d8d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.004+0000 7f2d8d600640 1 -- 172.21.15.46:0/1119065489 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98027060 con 0x558a73f2d020 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.004+0000 7f2d8d600640 1 -- 172.21.15.46:0/1119065489 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2d98017ba0 con 0x558a73f2d020 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.004+0000 7f2da91bb740 1 -- 172.21.15.46:0/1119065489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d880bcbd0 con 0x558a73f2d020 2024-09-10T07:49:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.004+0000 7f2d8d600640 1 -- 172.21.15.46:0/1119065489 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2d9802e050 con 0x558a73f2d020 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.004+0000 7f2da91bb740 10 client.28592 fetch_fsmap learned FSMap version 53 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.004+0000 7f2da91bb740 1 -- 172.21.15.46:0/1119065489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x558a73f3bad0 con 0x558a73f2d020 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.005+0000 7f2d8d600640 1 -- 172.21.15.46:0/1119065489 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d880bcbd0 con 0x558a73f2d020 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.006+0000 7f2d8d600640 1 -- 172.21.15.46:0/1119065489 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 53) ==== 8374+0+0 (secure 0 0 0) 0x7f2d98021070 con 0x558a73f2d020 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.006+0000 7f2da91bb740 10 client.28592 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.006+0000 7f2da91bb740 10 client.28592 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.006+0000 7f2da91bb740 20 client.28592 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.006+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558a73f3cfe0 0x558a73f3f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.006+0000 7f2da91bb740 4 client.28592 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.006+0000 7f2da91bb740 1 -- 172.21.15.46:0/1119065489 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a73e95950 con 0x558a73f3cfe0 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.007+0000 7f2d9f400640 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558a73f3cfe0 0x558a73f3f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.007+0000 7f2d9f400640 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558a73f3cfe0 0x558a73f3f440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c24cce2577b41f22 server_cookie=ddf33a4d1f27f70f in_seq=0 out_seq=0 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.007+0000 7f2d8d600640 10 client.28592 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.008+0000 7f2d8d600640 1 -- 172.21.15.46:0/1119065489 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a73e95950 con 0x558a73f3cfe0 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.008+0000 7f2d8d600640 10 client.28592 handle_command_reply: tid=0 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.008+0000 7f2da91bb740 1 client.28592 shutdown 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2da91bb740 1 -- 172.21.15.46:0/1119065489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f2d020 msgr2=0x558a73f2aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f2d020 0x558a73f2aa00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2d98010b00 tx=0x7f2d98004a40 comp rx=0 tx=0).stop 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2da91bb740 1 -- 172.21.15.46:0/1119065489 shutdown_connections 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f2d020 0x558a73f2aa00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f22760 0x558a73f2a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558a73f3cfe0 0x558a73f3f440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2da91bb740 1 --2- 172.21.15.46:0/1119065489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f21e40 0x558a73f29f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2da91bb740 1 -- 172.21.15.46:0/1119065489 >> 172.21.15.46:0/1119065489 conn(0x558a73f1e3d0 msgr2=0x558a73f240b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.009+0000 7f2d8d600640 0 client.28592 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.010+0000 7f2da91bb740 1 -- 172.21.15.46:0/1119065489 shutdown_connections 2024-09-10T07:49:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.010+0000 7f2da91bb740 1 -- 172.21.15.46:0/1119065489 wait complete. 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.010+0000 7f2da91bb740 20 client.28592 trim_cache size 0 max 16384 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.029+0000 7f2da91bb740 1 Processor -- start 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.029+0000 7f2da91bb740 1 -- start start 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f44d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f45250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2da91bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c00fb20 con 0x558a73f2d020 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2da91bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c011690 con 0x558a73f21e40 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2da91bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0101f0 con 0x558a73f22760 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f248a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52454/0 (socket says 172.21.15.46:52454) 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.030+0000 7f2d9f400640 1 -- 172.21.15.46:0/3998564667 learned_addr learned my addr 172.21.15.46:0/3998564667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d97e00640 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f44d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d9f400640 1 -- 172.21.15.46:0/3998564667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 msgr2=0x558a73f44d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f45250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d9f400640 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f44d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d9f400640 1 -- 172.21.15.46:0/3998564667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 msgr2=0x558a73f45250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d9f400640 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f45250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d9f400640 1 -- 172.21.15.46:0/3998564667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a73f1f060 con 0x558a73f21e40 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d97e00640 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f44d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d9f400640 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f248a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x558a73f3fc50 tx=0x7f2d90009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b25fd3bf1910b65e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d8fe00640 1 -- 172.21.15.46:0/3998564667 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90004050 con 0x558a73f21e40 2024-09-10T07:49:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d8fe00640 1 -- 172.21.15.46:0/3998564667 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d900041f0 con 0x558a73f21e40 2024-09-10T07:49:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.031+0000 7f2d8fe00640 1 -- 172.21.15.46:0/3998564667 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9001a620 con 0x558a73f21e40 2024-09-10T07:49:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.032+0000 7f2da91bb740 1 -- 172.21.15.46:0/3998564667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 msgr2=0x558a73f248a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.032+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f248a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x558a73f3fc50 tx=0x7f2d90009720 comp rx=0 tx=0).stop 2024-09-10T07:49:46.053 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 -- 172.21.15.46:0/3998564667 shutdown_connections 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f45250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f44d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3998564667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f248a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 -- 172.21.15.46:0/3998564667 >> 172.21.15.46:0/3998564667 conn(0x558a73f1e3d0 msgr2=0x558a73f2c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 -- 172.21.15.46:0/3998564667 shutdown_connections 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 -- 172.21.15.46:0/3998564667 wait complete. 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 -- start start 2024-09-10T07:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.033+0000 7f2da91bb740 1 Processor -- start 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f34d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2da91bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f31d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2da91bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c010fc0 con 0x558a73f2d020 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2da91bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0108c0 con 0x558a73f21e40 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2da91bb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c00db00 con 0x558a73f22760 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2da91bb740 10 client.0 _ll_register_callbacks cb 0x7f2d98010f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f34d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f34d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52464/0 (socket says 172.21.15.46:52464) 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2d9f400640 1 -- 172.21.15.46:0/3181213927 learned_addr learned my addr 172.21.15.46:0/3181213927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2d97e00640 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.034+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f31d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d9f400640 1 -- 172.21.15.46:0/3181213927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 msgr2=0x558a73f352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d9f400640 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d9f400640 1 -- 172.21.15.46:0/3181213927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 msgr2=0x558a73f31d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d9f400640 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f31d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d9f400640 1 -- 172.21.15.46:0/3181213927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a73f35930 con 0x558a73f21e40 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d9fe00640 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f31d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d9f400640 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f34d80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009cd0 tx=0x7f2d90009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d20209afc01e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d8d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d8d600640 1 -- 172.21.15.46:0/3181213927 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d900035f0 con 0x558a73f21e40 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.035+0000 7f2d8d600640 1 -- 172.21.15.46:0/3181213927 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2d90003790 con 0x558a73f21e40 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.036+0000 7f2da91bb740 1 -- 172.21.15.46:0/3181213927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d900041f0 con 0x558a73f21e40 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.036+0000 7f2d8d600640 1 -- 172.21.15.46:0/3181213927 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2d90033050 con 0x558a73f21e40 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.036+0000 7f2da91bb740 10 client.28598 fetch_fsmap learned FSMap version 53 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.036+0000 7f2da91bb740 1 -- 172.21.15.46:0/3181213927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x558a73fb09c0 con 0x558a73f21e40 2024-09-10T07:49:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.036+0000 7f2d8d600640 1 -- 172.21.15.46:0/3181213927 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d900041f0 con 0x558a73f21e40 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2d8d600640 1 -- 172.21.15.46:0/3181213927 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 53) ==== 8374+0+0 (secure 0 0 0) 0x7f2d90010430 con 0x558a73f21e40 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2da91bb740 10 client.28598 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2da91bb740 10 client.28598 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2da91bb740 20 client.28598 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558a73fb10e0 0x558a73fb3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2da91bb740 4 client.28598 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2da91bb740 1 -- 172.21.15.46:0/3181213927 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a73e8fe60 con 0x558a73fb10e0 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2d97e00640 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558a73fb10e0 0x558a73fb3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2d97e00640 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558a73fb10e0 0x558a73fb3500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fa6134619deb1d4b server_cookie=570a471f6b5f0d7c in_seq=0 out_seq=0 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.037+0000 7f2d8d600640 10 client.28598 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2d8d600640 1 -- 172.21.15.46:0/3181213927 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a73e8fe60 con 0x558a73fb10e0 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2d8d600640 10 client.28598 handle_command_reply: tid=0 2024-09-10T07:49:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 client.28598 shutdown 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 -- 172.21.15.46:0/3181213927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 msgr2=0x558a73f34d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f34d80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009cd0 tx=0x7f2d90009b00 comp rx=0 tx=0).stop 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 -- 172.21.15.46:0/3181213927 shutdown_connections 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a73f2d020 0x558a73f31d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x558a73fb10e0 0x558a73fb3500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a73f22760 0x558a73f352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2d8d600640 0 client.28598 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 --2- 172.21.15.46:0/3181213927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558a73f21e40 0x558a73f34d80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 -- 172.21.15.46:0/3181213927 >> 172.21.15.46:0/3181213927 conn(0x558a73f1e3d0 msgr2=0x558a73f29da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 -- 172.21.15.46:0/3181213927 shutdown_connections 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 1 -- 172.21.15.46:0/3181213927 wait complete. 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.038+0000 7f2da91bb740 20 client.28598 trim_cache size 0 max 16384 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d840a7090 msgr2=0x7f2d840a9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d840a7090 0x7f2d840a9550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2d98010270 tx=0x7f2d98009710 comp rx=0 tx=0).stop 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0073f30 msgr2=0x7f2da01a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0073f30 0x7f2da01a4bb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2d8802a9b0 tx=0x7f2d88003b90 comp rx=0 tx=0).stop 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 shutdown_connections 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2da01137e0 0x7f2da019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0073f30 0x7f2da01a4bb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2d840a7090 0x7f2d840a9550 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 --2- 172.21.15.46:0/4092683370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da006c260 0x7f2da01a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 >> 172.21.15.46:0/4092683370 conn(0x7f2da00fc610 msgr2=0x7f2da0072fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.044+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 shutdown_connections 2024-09-10T07:49:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:46.045+0000 7f2da5200640 1 -- 172.21.15.46:0/4092683370 wait complete. 2024-09-10T07:49:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:46 smithi046 ceph-mon[30705]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 4.2 MiB/s wr, 432 op/s 2024-09-10T07:49:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:46 smithi181 ceph-mon[35397]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 4.2 MiB/s wr, 432 op/s 2024-09-10T07:49:46.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:46 smithi063 ceph-mon[35020]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 4.2 MiB/s wr, 432 op/s 2024-09-10T07:49:47.269 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:47.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:49:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.883+0000 7f8a62c00640 1 -- 172.21.15.46:0/2174238868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c101b60 msgr2=0x7f8a5c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.883+0000 7f8a62c00640 1 --2- 172.21.15.46:0/2174238868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c101b60 0x7f8a5c101f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009900 tx=0x7f8a4c02a410 comp rx=0 tx=0).stop 2024-09-10T07:49:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.883+0000 7f8a62c00640 1 -- 172.21.15.46:0/2174238868 shutdown_connections 2024-09-10T07:49:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.883+0000 7f8a62c00640 1 --2- 172.21.15.46:0/2174238868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5c103710 0x7f8a5c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.883+0000 7f8a62c00640 1 --2- 172.21.15.46:0/2174238868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a5c102d50 0x7f8a5c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.883+0000 7f8a62c00640 1 --2- 172.21.15.46:0/2174238868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c101b60 0x7f8a5c101f60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.883+0000 7f8a62c00640 1 -- 172.21.15.46:0/2174238868 >> 172.21.15.46:0/2174238868 conn(0x7f8a5c0fd320 msgr2=0x7f8a5c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.884+0000 7f8a62c00640 1 -- 172.21.15.46:0/2174238868 shutdown_connections 2024-09-10T07:49:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.884+0000 7f8a62c00640 1 -- 172.21.15.46:0/2174238868 wait complete. 2024-09-10T07:49:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.886+0000 7f8a62c00640 1 Processor -- start 2024-09-10T07:49:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.886+0000 7f8a62c00640 1 -- start start 2024-09-10T07:49:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a62c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5c101b60 0x7f8a5c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a62c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a5c102d50 0x7f8a5c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a62c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c103710 0x7f8a5c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a62c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10d8b0 con 0x7f8a5c102d50 2024-09-10T07:49:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a62c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10d730 con 0x7f8a5c103710 2024-09-10T07:49:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a62c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c10da30 con 0x7f8a5c101b60 2024-09-10T07:49:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a60c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c103710 0x7f8a5c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a60c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c103710 0x7f8a5c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52482/0 (socket says 172.21.15.46:52482) 2024-09-10T07:49:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a60c00640 1 -- 172.21.15.46:0/3973859192 learned_addr learned my addr 172.21.15.46:0/3973859192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a5be00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5c101b60 0x7f8a5c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a53e00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a5c102d50 0x7f8a5c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a60c00640 1 -- 172.21.15.46:0/3973859192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5c101b60 msgr2=0x7f8a5c075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a60c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5c101b60 0x7f8a5c075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.887+0000 7f8a60c00640 1 -- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a5c102d50 msgr2=0x7f8a5c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a60c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a5c102d50 0x7f8a5c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a60c00640 1 -- 172.21.15.46:0/3973859192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c0753b0 con 0x7f8a5c103710 2024-09-10T07:49:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a5be00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5c101b60 0x7f8a5c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a60c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c103710 0x7f8a5c074b20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400eaa0 tx=0x7f8a5400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a59600640 1 -- 172.21.15.46:0/3973859192 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400cdb0 con 0x7f8a5c103710 2024-09-10T07:49:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a59600640 1 -- 172.21.15.46:0/3973859192 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a54005020 con 0x7f8a5c103710 2024-09-10T07:49:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a59600640 1 -- 172.21.15.46:0/3973859192 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a540116b0 con 0x7f8a5c103710 2024-09-10T07:49:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a62c00640 1 -- 172.21.15.46:0/3973859192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c1a1f90 con 0x7f8a5c103710 2024-09-10T07:49:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.888+0000 7f8a62c00640 1 -- 172.21.15.46:0/3973859192 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c1a2420 con 0x7f8a5c103710 2024-09-10T07:49:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.901+0000 7f8a65a86740 1 Processor -- start 2024-09-10T07:49:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.904+0000 7f8a59600640 1 -- 172.21.15.46:0/3973859192 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8a54005560 con 0x7f8a5c103710 2024-09-10T07:49:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.906+0000 7f8a65a86740 1 -- start start 2024-09-10T07:49:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.906+0000 7f8a59600640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8a400a2d00 0x7f8a400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.906+0000 7f8a59600640 1 -- 172.21.15.46:0/3973859192 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8a540cf8b0 con 0x7f8a5c103710 2024-09-10T07:49:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.907+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d658a0 0x55db08d65ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.907+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d661e0 0x55db08d6aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.907+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d6b270 0x55db08d6d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.908+0000 7f8a5be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d658a0 0x55db08d65ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.908+0000 7f8a5be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d658a0 0x55db08d65ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52494/0 (socket says 172.21.15.46:52494) 2024-09-10T07:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.908+0000 7f8a65a86740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55db08cc2100 con 0x55db08d6b270 2024-09-10T07:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.908+0000 7f8a65a86740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55db08c6a1b0 con 0x55db08d658a0 2024-09-10T07:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.908+0000 7f8a65a86740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55db08cbe0c0 con 0x55db08d661e0 2024-09-10T07:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.908+0000 7f8a5be00640 1 -- 172.21.15.46:0/2663340872 learned_addr learned my addr 172.21.15.46:0/2663340872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.909+0000 7f8a60c00640 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d6b270 0x55db08d6d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.909+0000 7f8a5be00640 1 -- 172.21.15.46:0/2663340872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d661e0 msgr2=0x55db08d6aca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.909+0000 7f8a5be00640 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d661e0 0x55db08d6aca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.909+0000 7f8a5be00640 1 -- 172.21.15.46:0/2663340872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d6b270 msgr2=0x55db08d6d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.909+0000 7f8a5be00640 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d6b270 0x55db08d6d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.910+0000 7f8a53e00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8a400a2d00 0x7f8a400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.910+0000 7f8a5be00640 1 -- 172.21.15.46:0/2663340872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db089d5210 con 0x55db08d658a0 2024-09-10T07:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.910+0000 7f8a60c00640 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d6b270 0x55db08d6d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.911+0000 7f8a5be00640 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d658a0 0x55db08d65ca0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006010 tx=0x7f8a4c004210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec12348b8e3f5e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.911+0000 7f8a51200640 1 -- 172.21.15.46:0/2663340872 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c03f070 con 0x55db08d658a0 2024-09-10T07:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.912+0000 7f8a51200640 1 -- 172.21.15.46:0/2663340872 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c005520 con 0x55db08d658a0 2024-09-10T07:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.912+0000 7f8a53e00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8a400a2d00 0x7f8a400a51c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55db08d66c10 tx=0x7f8a44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.914+0000 7f8a65a86740 1 -- 172.21.15.46:0/2663340872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d658a0 msgr2=0x55db08d65ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.914+0000 7f8a51200640 1 -- 172.21.15.46:0/2663340872 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c03f070 con 0x55db08d658a0 2024-09-10T07:49:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.914+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d658a0 0x55db08d65ca0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006010 tx=0x7f8a4c004210 comp rx=0 tx=0).stop 2024-09-10T07:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.914+0000 7f8a65a86740 1 -- 172.21.15.46:0/2663340872 shutdown_connections 2024-09-10T07:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.914+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d6b270 0x55db08d6d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.914+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d661e0 0x55db08d6aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.914+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2663340872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d658a0 0x55db08d65ca0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.914+0000 7f8a65a86740 1 -- 172.21.15.46:0/2663340872 >> 172.21.15.46:0/2663340872 conn(0x55db08d58700 msgr2=0x55db08d58b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.915+0000 7f8a65a86740 1 -- 172.21.15.46:0/2663340872 shutdown_connections 2024-09-10T07:49:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.916+0000 7f8a65a86740 1 -- 172.21.15.46:0/2663340872 wait complete. 2024-09-10T07:49:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.916+0000 7f8a65a86740 1 -- start start 2024-09-10T07:49:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.917+0000 7f8a65a86740 1 Processor -- start 2024-09-10T07:49:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.918+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d6f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.918+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d6fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d76f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.918+0000 7f8a5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d6f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a53e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d6fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a53e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d6fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52496/0 (socket says 172.21.15.46:52496) 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a60c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d76f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a60c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d76f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35734/0 (socket says 172.21.15.46:35734) 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a65a86740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55db08d6e230 con 0x55db08d658a0 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a65a86740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55db08c4b090 con 0x55db08d661e0 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a65a86740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55db08d6e3b0 con 0x55db08d6b270 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a65a86740 10 client.0 _ll_register_callbacks cb 0x55db08c9ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d6f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54316/0 (socket says 172.21.15.46:54316) 2024-09-10T07:49:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.919+0000 7f8a5be00640 1 -- 172.21.15.46:0/839885591 learned_addr learned my addr 172.21.15.46:0/839885591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.920+0000 7f8a60c00640 1 -- 172.21.15.46:0/839885591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 msgr2=0x55db08d6fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.920+0000 7f8a60c00640 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d6fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.920+0000 7f8a60c00640 1 -- 172.21.15.46:0/839885591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 msgr2=0x55db08d6f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.920+0000 7f8a60c00640 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d6f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.920+0000 7f8a60c00640 1 -- 172.21.15.46:0/839885591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db08aa1620 con 0x55db08d6b270 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.920+0000 7f8a5be00640 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d6f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.920+0000 7f8a53e00640 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d6fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.921+0000 7f8a60c00640 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d76f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400efd0 tx=0x7f8a5409be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1815e102a877ee3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.921+0000 7f8a49600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.921+0000 7f8a49600640 1 -- 172.21.15.46:0/839885591 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5409b1e0 con 0x55db08d6b270 2024-09-10T07:49:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.921+0000 7f8a49600640 1 -- 172.21.15.46:0/839885591 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8a5409b380 con 0x55db08d6b270 2024-09-10T07:49:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.921+0000 7f8a65a86740 1 -- 172.21.15.46:0/839885591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c005520 con 0x55db08d6b270 2024-09-10T07:49:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.922+0000 7f8a49600640 1 -- 172.21.15.46:0/839885591 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8a54097ae0 con 0x55db08d6b270 2024-09-10T07:49:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.922+0000 7f8a65a86740 10 client.28522 fetch_fsmap learned FSMap version 53 2024-09-10T07:49:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.922+0000 7f8a65a86740 1 -- 172.21.15.46:0/839885591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55db08d79db0 con 0x55db08d6b270 2024-09-10T07:49:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.922+0000 7f8a49600640 1 -- 172.21.15.46:0/839885591 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a54097c80 con 0x55db08d6b270 2024-09-10T07:49:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.923+0000 7f8a49600640 1 -- 172.21.15.46:0/839885591 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 53) ==== 8374+0+0 (secure 0 0 0) 0x7f8a5400f040 con 0x55db08d6b270 2024-09-10T07:49:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.923+0000 7f8a65a86740 10 client.28522 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T07:49:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.923+0000 7f8a65a86740 10 client.28522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.923+0000 7f8a65a86740 20 client.28522 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.923+0000 7f8a65a86740 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55db08d7b4a0 0x55db08d7d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.923+0000 7f8a65a86740 4 client.28522 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.923+0000 7f8a65a86740 1 -- 172.21.15.46:0/839885591 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8a4c005520 con 0x55db08d7b4a0 2024-09-10T07:49:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.923+0000 7f8a5be00640 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55db08d7b4a0 0x55db08d7d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.929+0000 7f8a49600640 10 client.28522 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.929+0000 7f8a5be00640 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55db08d7b4a0 0x55db08d7d890 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b72c35029200b2c server_cookie=6d1899ec9fe23c03 in_seq=0 out_seq=0 2024-09-10T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.931+0000 7f8a49600640 1 -- 172.21.15.46:0/839885591 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8a4c005520 con 0x55db08d7b4a0 2024-09-10T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.931+0000 7f8a49600640 10 client.28522 handle_command_reply: tid=0 2024-09-10T07:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.931+0000 7f8a65a86740 1 client.28522 shutdown 2024-09-10T07:49:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.932+0000 7f8a65a86740 1 -- 172.21.15.46:0/839885591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 msgr2=0x55db08d76f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.932+0000 7f8a65a86740 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d76f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400efd0 tx=0x7f8a5409be50 comp rx=0 tx=0).stop 2024-09-10T07:49:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.932+0000 7f8a65a86740 1 -- 172.21.15.46:0/839885591 shutdown_connections 2024-09-10T07:49:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.932+0000 7f8a65a86740 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55db08d7b4a0 0x55db08d7d890 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.932+0000 7f8a65a86740 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d76f30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.932+0000 7f8a65a86740 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d6fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.932+0000 7f8a65a86740 1 --2- 172.21.15.46:0/839885591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d6f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.932+0000 7f8a65a86740 1 -- 172.21.15.46:0/839885591 >> 172.21.15.46:0/839885591 conn(0x55db08d58700 msgr2=0x55db08d59f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.933+0000 7f8a49600640 0 client.28522 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.935+0000 7f8a65a86740 1 -- 172.21.15.46:0/839885591 shutdown_connections 2024-09-10T07:49:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.935+0000 7f8a65a86740 1 -- 172.21.15.46:0/839885591 wait complete. 2024-09-10T07:49:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.936+0000 7f8a65a86740 20 client.28522 trim_cache size 0 max 16384 2024-09-10T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.985+0000 7f8a65a86740 1 Processor -- start 2024-09-10T07:49:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.985+0000 7f8a65a86740 1 -- start start 2024-09-10T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.987+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.987+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.988+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d57770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.988+0000 7f8a53e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.988+0000 7f8a53e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d76ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52498/0 (socket says 172.21.15.46:52498) 2024-09-10T07:49:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.989+0000 7f8a65a86740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a85b0 con 0x55db08d658a0 2024-09-10T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.989+0000 7f8a53e00640 1 -- 172.21.15.46:0/1801557494 learned_addr learned my addr 172.21.15.46:0/1801557494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.989+0000 7f8a65a86740 1 -- 172.21.15.46:0/1801557494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1aa120 con 0x55db08d661e0 2024-09-10T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.990+0000 7f8a60c00640 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d57770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.990+0000 7f8a65a86740 1 -- 172.21.15.46:0/1801557494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a8c80 con 0x55db08d6b270 2024-09-10T07:49:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.990+0000 7f8a5be00640 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.990+0000 7f8a60c00640 1 -- 172.21.15.46:0/1801557494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 msgr2=0x55db08d76ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.991+0000 7f8a60c00640 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d76ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.991+0000 7f8a60c00640 1 -- 172.21.15.46:0/1801557494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 msgr2=0x55db08d536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.991+0000 7f8a60c00640 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.991+0000 7f8a60c00640 1 -- 172.21.15.46:0/1801557494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db08d7dc00 con 0x55db08d6b270 2024-09-10T07:49:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.994+0000 7f8a53e00640 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d76ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.994+0000 7f8a60c00640 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d57770 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8a540214b0 tx=0x7f8a5409b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f019ad260b0a4767 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.995+0000 7f8a4be00640 1 -- 172.21.15.46:0/1801557494 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a540a0140 con 0x55db08d6b270 2024-09-10T07:49:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.995+0000 7f8a4be00640 1 -- 172.21.15.46:0/1801557494 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a540051d0 con 0x55db08d6b270 2024-09-10T07:49:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.995+0000 7f8a4be00640 1 -- 172.21.15.46:0/1801557494 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5409ac50 con 0x55db08d6b270 2024-09-10T07:49:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.996+0000 7f8a65a86740 1 -- 172.21.15.46:0/1801557494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 msgr2=0x55db08d57770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.996+0000 7f8a65a86740 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d57770 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8a540214b0 tx=0x7f8a5409b820 comp rx=0 tx=0).stop 2024-09-10T07:49:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.997+0000 7f8a65a86740 1 -- 172.21.15.46:0/1801557494 shutdown_connections 2024-09-10T07:49:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.998+0000 7f8a65a86740 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d6b270 0x55db08d57770 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.998+0000 7f8a65a86740 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d661e0 0x55db08d76ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.998+0000 7f8a65a86740 1 --2- 172.21.15.46:0/1801557494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.998+0000 7f8a65a86740 1 -- 172.21.15.46:0/1801557494 >> 172.21.15.46:0/1801557494 conn(0x55db08d58700 msgr2=0x55db08d57060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:47.998+0000 7f8a65a86740 1 -- 172.21.15.46:0/1801557494 shutdown_connections 2024-09-10T07:49:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.000+0000 7f8a65a86740 1 -- 172.21.15.46:0/1801557494 wait complete. 2024-09-10T07:49:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.000+0000 7f8a65a86740 1 -- start start 2024-09-10T07:49:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.000+0000 7f8a65a86740 1 Processor -- start 2024-09-10T07:49:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.002+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d74530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.002+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d661e0 0x55db08d74a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.002+0000 7f8a65a86740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d6b270 0x55db08d75a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.002+0000 7f8a60c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d6b270 0x55db08d75a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.002+0000 7f8a60c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d6b270 0x55db08d75a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52514/0 (socket says 172.21.15.46:52514) 2024-09-10T07:49:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.002+0000 7f8a60c00640 1 -- 172.21.15.46:0/2995928878 learned_addr learned my addr 172.21.15.46:0/2995928878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.002+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a9a50 con 0x55db08d658a0 2024-09-10T07:49:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.002+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a9350 con 0x55db08d6b270 2024-09-10T07:49:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.003+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1a6590 con 0x55db08d661e0 2024-09-10T07:49:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.003+0000 7f8a60c00640 1 -- 172.21.15.46:0/2995928878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d661e0 msgr2=0x55db08d74a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:49:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.003+0000 7f8a60c00640 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d661e0 0x55db08d74a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.003+0000 7f8a60c00640 1 -- 172.21.15.46:0/2995928878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 msgr2=0x55db08d74530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.003+0000 7f8a60c00640 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d74530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.003+0000 7f8a60c00640 1 -- 172.21.15.46:0/2995928878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db08d76090 con 0x55db08d6b270 2024-09-10T07:49:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.006+0000 7f8a65a86740 10 client.0 _ll_register_callbacks cb 0x7f8a5409bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.006+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a540051d0 con 0x55db08d6b270 2024-09-10T07:49:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.008+0000 7f8a49600640 10 client.28613 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.008+0000 7f8a60c00640 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d6b270 0x55db08d75a80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8a5409ba90 tx=0x7f8a540a4b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=878b9b139eb8a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.008+0000 7f8a49600640 1 -- 172.21.15.46:0/2995928878 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54082050 con 0x55db08d6b270 2024-09-10T07:49:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.009+0000 7f8a49600640 1 -- 172.21.15.46:0/2995928878 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8a540a14d0 con 0x55db08d6b270 2024-09-10T07:49:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.009+0000 7f8a49600640 1 -- 172.21.15.46:0/2995928878 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8a5409b4d0 con 0x55db08d6b270 2024-09-10T07:49:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.009+0000 7f8a65a86740 10 client.28613 fetch_fsmap learned FSMap version 53 2024-09-10T07:49:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.009+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=53}) -- 0x55db08d76090 con 0x55db08d6b270 2024-09-10T07:49:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.010+0000 7f8a49600640 1 -- 172.21.15.46:0/2995928878 <== mon.2 v2:172.21.15.63:3300/0 4 ==== fsmap(e 53) ==== 8374+0+0 (secure 0 0 0) 0x7f8a5409ac50 con 0x55db08d6b270 2024-09-10T07:49:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.011+0000 7f8a65a86740 10 client.28613 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T07:49:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.011+0000 7f8a65a86740 10 client.28613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.011+0000 7f8a65a86740 20 client.28613 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.011+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55db08d62f90 0x55db08df10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.011+0000 7f8a65a86740 4 client.28613 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:49:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.011+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db08d6b030 con 0x55db08d62f90 2024-09-10T07:49:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.012+0000 7f8a5be00640 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55db08d62f90 0x55db08df10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.013+0000 7f8a5be00640 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55db08d62f90 0x55db08df10c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7495caf9da2c5a19 server_cookie=fd8f0f81804d9628 in_seq=0 out_seq=0 2024-09-10T07:49:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.013+0000 7f8a49600640 10 client.28613 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.123+0000 7f3cc3200640 1 -- 172.21.15.46:0/2164448212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4076720 msgr2=0x7f3cc4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.123+0000 7f3cc3200640 1 --2- 172.21.15.46:0/2164448212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4076720 0x7f3cc4076b00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009900 tx=0x7f3cbc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.124+0000 7f3cc3200640 1 -- 172.21.15.46:0/2164448212 shutdown_connections 2024-09-10T07:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.124+0000 7f3cc3200640 1 --2- 172.21.15.46:0/2164448212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4112870 0x7f3cc4114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.124+0000 7f3cc3200640 1 --2- 172.21.15.46:0/2164448212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cc4077040 0x7f3cc4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.124+0000 7f3cc3200640 1 --2- 172.21.15.46:0/2164448212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4076720 0x7f3cc4076b00 secure :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009900 tx=0x7f3cbc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.124+0000 7f3cc3200640 1 -- 172.21.15.46:0/2164448212 >> 172.21.15.46:0/2164448212 conn(0x7f3cc40ff910 msgr2=0x7f3cc4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.128+0000 7f3cc3200640 1 -- 172.21.15.46:0/2164448212 shutdown_connections 2024-09-10T07:49:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.130+0000 7f3cc3200640 1 -- 172.21.15.46:0/2164448212 wait complete. 2024-09-10T07:49:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.131+0000 7f3cc3200640 1 Processor -- start 2024-09-10T07:49:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.132+0000 7f3cc3200640 1 -- start start 2024-09-10T07:49:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.134+0000 7f3cc3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4077040 0x7f3cc4112080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.134+0000 7f3cc3200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4112870 0x7f3cc410b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.135+0000 7f3cc1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4077040 0x7f3cc4112080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.135+0000 7f3cc1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4077040 0x7f3cc4112080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52536/0 (socket says 172.21.15.46:52536) 2024-09-10T07:49:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.135+0000 7f3cc3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cc410b650 0x7f3cc410bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.135+0000 7f3cc1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4112870 0x7f3cc410b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.135+0000 7f3cc1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4112870 0x7f3cc410b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35782/0 (socket says 172.21.15.46:35782) 2024-09-10T07:49:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.136+0000 7f3cc1e00640 1 -- 172.21.15.46:0/52074493 learned_addr learned my addr 172.21.15.46:0/52074493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.136+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc4118430 con 0x7f3cc410b650 2024-09-10T07:49:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.136+0000 7f3cc2800640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cc410b650 0x7f3cc410bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.136+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41182b0 con 0x7f3cc4077040 2024-09-10T07:49:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.136+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41185b0 con 0x7f3cc4112870 2024-09-10T07:49:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.137+0000 7f3cc1e00640 1 -- 172.21.15.46:0/52074493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4112870 msgr2=0x7f3cc410b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.137+0000 7f3cc1e00640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4112870 0x7f3cc410b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.137+0000 7f3cc1e00640 1 -- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cc410b650 msgr2=0x7f3cc410bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.137+0000 7f3cc1e00640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cc410b650 0x7f3cc410bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.138+0000 7f3cc1e00640 1 -- 172.21.15.46:0/52074493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc410fce0 con 0x7f3cc4077040 2024-09-10T07:49:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.140+0000 7f3cc1e00640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4077040 0x7f3cc4112080 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc004770 tx=0x7f3cbc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.142+0000 7f3cbaa00640 1 -- 172.21.15.46:0/52074493 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc03f020 con 0x7f3cc4077040 2024-09-10T07:49:48.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.142+0000 7f3cbaa00640 1 -- 172.21.15.46:0/52074493 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cbc031ca0 con 0x7f3cc4077040 2024-09-10T07:49:48.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.142+0000 7f3cbaa00640 1 -- 172.21.15.46:0/52074493 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc035070 con 0x7f3cc4077040 2024-09-10T07:49:48.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.142+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc410ff70 con 0x7f3cc4077040 2024-09-10T07:49:48.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.144+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc4110400 con 0x7f3cc4077040 2024-09-10T07:49:48.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.157+0000 7f3ccba17740 1 Processor -- start 2024-09-10T07:49:48.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.157+0000 7f3ccba17740 1 -- start start 2024-09-10T07:49:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.157+0000 7f3cbaa00640 1 -- 172.21.15.46:0/52074493 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3cbc042020 con 0x7f3cc4077040 2024-09-10T07:49:48.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.161+0000 7f3ccba17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.162+0000 7f3cbaa00640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ca80a2cb0 0x7f3ca80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.162+0000 7f3cc2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.163+0000 7f3cbaa00640 1 -- 172.21.15.46:0/52074493 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3cbc0ef740 con 0x7f3cc4077040 2024-09-10T07:49:48.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.164+0000 7f3cc2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35794/0 (socket says 172.21.15.46:35794) 2024-09-10T07:49:48.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.164+0000 7f3cc2800640 1 -- 172.21.15.46:0/3585248907 learned_addr learned my addr 172.21.15.46:0/3585248907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:48.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.165+0000 7f3cc1e00640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ca80a2cb0 0x7f3ca80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.165+0000 7f3ccba17740 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7d40 0x557bef0d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.165+0000 7f3ccba17740 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef0d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.165+0000 7f3ccba17740 1 -- 172.21.15.46:0/3585248907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557bef01d530 con 0x557bef0b7d40 2024-09-10T07:49:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.165+0000 7f3ccba17740 1 -- 172.21.15.46:0/3585248907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557beefc4860 con 0x557bef0d3750 2024-09-10T07:49:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.165+0000 7f3ccba17740 1 -- 172.21.15.46:0/3585248907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557bef0194f0 con 0x557bef0b7420 2024-09-10T07:49:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.165+0000 7f3cc2800640 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef0d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.166+0000 7f3cc2800640 1 -- 172.21.15.46:0/3585248907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 msgr2=0x557bef0d5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.166+0000 7f3cc2800640 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef0d5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.166+0000 7f3cc2800640 1 -- 172.21.15.46:0/3585248907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7d40 msgr2=0x557bef0d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.166+0000 7f3cc1400640 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7d40 0x557bef0d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.168+0000 7f3cc2800640 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7d40 0x557bef0d3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.168+0000 7f3cc1e00640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ca80a2cb0 0x7f3ca80a5170 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0bb8c0 tx=0x7f3cbc02a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.169+0000 7f3cc2800640 1 -- 172.21.15.46:0/3585248907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557beed8eb90 con 0x557bef0b7420 2024-09-10T07:49:48.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.170+0000 7f3cc2800640 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0b7800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0098b0 tx=0x7f3cac009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9338e780ad1f58f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.171+0000 7f3cb3400640 1 -- 172.21.15.46:0/3585248907 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac014e00 con 0x557bef0b7420 2024-09-10T07:49:48.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.172+0000 7f3cb3400640 1 -- 172.21.15.46:0/3585248907 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cac014110 con 0x557bef0b7420 2024-09-10T07:49:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.173+0000 7f3ccba17740 1 -- 172.21.15.46:0/3585248907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 msgr2=0x557bef0b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.173+0000 7f3ccba17740 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0b7800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0098b0 tx=0x7f3cac009bf0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.174+0000 7f3ccba17740 1 -- 172.21.15.46:0/3585248907 shutdown_connections 2024-09-10T07:49:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.174+0000 7f3ccba17740 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef0d5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.174+0000 7f3ccba17740 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7d40 0x557bef0d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.174+0000 7f3ccba17740 1 --2- 172.21.15.46:0/3585248907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0b7800 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.174+0000 7f3ccba17740 1 -- 172.21.15.46:0/3585248907 >> 172.21.15.46:0/3585248907 conn(0x557bef0b3af0 msgr2=0x557bef0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.175+0000 7f3ccba17740 1 -- 172.21.15.46:0/3585248907 shutdown_connections 2024-09-10T07:49:48.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.175+0000 7f3ccba17740 1 -- 172.21.15.46:0/3585248907 wait complete. 2024-09-10T07:49:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.178+0000 7f3ccba17740 1 -- start start 2024-09-10T07:49:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.179+0000 7f3ccba17740 1 Processor -- start 2024-09-10T07:49:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.180+0000 7f3ccba17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.181+0000 7f3cc2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.181+0000 7f3cc2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35810/0 (socket says 172.21.15.46:35810) 2024-09-10T07:49:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.181+0000 7f3cc2800640 1 -- 172.21.15.46:0/454122309 learned_addr learned my addr 172.21.15.46:0/454122309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.181+0000 7f3ccba17740 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0b7d40 0x557bef0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.182+0000 7f3ccba17740 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0d3750 0x557bef0d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.182+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557bef0d63c0 con 0x557bef0d3750 2024-09-10T07:49:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.182+0000 7f3cc1400640 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0b7d40 0x557bef0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.183+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557beefd7e60 con 0x557bef0b7d40 2024-09-10T07:49:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.183+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557bef0d6540 con 0x557bef0b7420 2024-09-10T07:49:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.185+0000 7f3cc2800640 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0d3750 0x557bef0d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.185+0000 7f3cc2800640 1 -- 172.21.15.46:0/454122309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0b7d40 msgr2=0x557bef0d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.185+0000 7f3cc2800640 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0b7d40 0x557bef0d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.185+0000 7f3cc2800640 1 -- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0d3750 msgr2=0x557bef0d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.185+0000 7f3cc2800640 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0d3750 0x557bef0d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.185+0000 7f3cc2800640 1 -- 172.21.15.46:0/454122309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557beefe3990 con 0x557bef0b7420 2024-09-10T07:49:48.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.187+0000 7f3ccba17740 10 client.0 _ll_register_callbacks cb 0x557beefdccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:48.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.187+0000 7f3ca7e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:48.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.187+0000 7f3cc2800640 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0d06f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3cac01c040 tx=0x7f3cac005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c699d11e1e7c28e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.188+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557beefc0310 con 0x557bef0b7420 2024-09-10T07:49:48.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.189+0000 7f3ca7e00640 1 -- 172.21.15.46:0/454122309 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac028070 con 0x557bef0b7420 2024-09-10T07:49:48.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.189+0000 7f3ca7e00640 1 -- 172.21.15.46:0/454122309 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3cac005480 con 0x557bef0b7420 2024-09-10T07:49:48.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.189+0000 7f3ca7e00640 1 -- 172.21.15.46:0/454122309 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3cac036050 con 0x557bef0b7420 2024-09-10T07:49:48.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.189+0000 7f3ccba17740 10 client.28537 fetch_fsmap learned FSMap version 53 2024-09-10T07:49:48.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.189+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=53}) -- 0x557bef0e1f20 con 0x557bef0b7420 2024-09-10T07:49:48.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.190+0000 7f3ca7e00640 1 -- 172.21.15.46:0/454122309 <== mon.1 v2:172.21.15.181:3300/0 4 ==== fsmap(e 53) ==== 8374+0+0 (secure 0 0 0) 0x7f3cac00f040 con 0x557bef0b7420 2024-09-10T07:49:48.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.191+0000 7f3ccba17740 10 client.28537 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T07:49:48.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.191+0000 7f3ccba17740 10 client.28537 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:49:48.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.191+0000 7f3ccba17740 20 client.28537 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:48.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.191+0000 7f3ccba17740 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557bef0e3290 0x557bef0e5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.191+0000 7f3ccba17740 4 client.28537 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:48.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.191+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bef02ac60 con 0x557bef0e3290 2024-09-10T07:49:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.191+0000 7f3cc1400640 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557bef0e3290 0x557bef0e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.192+0000 7f3cc1400640 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557bef0e3290 0x557bef0e5680 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b7a5a982dd4d00 server_cookie=dc79d985958be561 in_seq=0 out_seq=0 2024-09-10T07:49:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.192+0000 7f3ca7e00640 10 client.28537 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:48.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.193+0000 7f3ca7e00640 1 -- 172.21.15.46:0/454122309 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557bef02ac60 con 0x557bef0e3290 2024-09-10T07:49:48.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.193+0000 7f3ca7e00640 10 client.28537 handle_command_reply: tid=0 2024-09-10T07:49:48.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.194+0000 7f3ccba17740 1 client.28537 shutdown 2024-09-10T07:49:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.195+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 msgr2=0x557bef0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.195+0000 7f3ccba17740 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0d06f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3cac01c040 tx=0x7f3cac005cf0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.195+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 shutdown_connections 2024-09-10T07:49:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.195+0000 7f3ccba17740 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557bef0e3290 0x557bef0e5680 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.195+0000 7f3ccba17740 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0d3750 0x557bef0d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.195+0000 7f3ccba17740 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0b7d40 0x557bef0d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.195+0000 7f3ccba17740 1 --2- 172.21.15.46:0/454122309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7420 0x557bef0d06f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.195+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 >> 172.21.15.46:0/454122309 conn(0x557bef0b3af0 msgr2=0x557bef0b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.196+0000 7f3ca7e00640 0 client.28537 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.197+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 shutdown_connections 2024-09-10T07:49:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.200+0000 7f3ccba17740 1 -- 172.21.15.46:0/454122309 wait complete. 2024-09-10T07:49:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.200+0000 7f3ccba17740 20 client.28537 trim_cache size 0 max 16384 2024-09-10T07:49:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.240+0000 7f3ccba17740 1 Processor -- start 2024-09-10T07:49:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.240+0000 7f3ccba17740 1 -- start start 2024-09-10T07:49:48.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3ccba17740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3ccba17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0dee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3ccba17740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef0df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3ccba17740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41b7970 con 0x557bef0b7420 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3ccba17740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41b5730 con 0x557bef0d3750 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3ccba17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41b72a0 con 0x557bef0b7d40 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3cc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3cc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54378/0 (socket says 172.21.15.46:54378) 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.241+0000 7f3cc2800640 1 -- 172.21.15.46:0/111334618 learned_addr learned my addr 172.21.15.46:0/111334618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.242+0000 7f3cc2800640 1 --2- 172.21.15.46:0/111334618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef0df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.242+0000 7f3cc2800640 1 -- 172.21.15.46:0/111334618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 msgr2=0x557bef0dee10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.242+0000 7f3cc2800640 1 --2- 172.21.15.46:0/111334618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0dee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.242+0000 7f3cc2800640 1 -- 172.21.15.46:0/111334618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 msgr2=0x557bef0df350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.242+0000 7f3cc2800640 1 --2- 172.21.15.46:0/111334618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef0df350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.242+0000 7f3cc2800640 1 -- 172.21.15.46:0/111334618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bef0ae160 con 0x557bef0b7420 2024-09-10T07:49:48.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.242+0000 7f3cc2800640 1 --2- 172.21.15.46:0/111334618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0d0fc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3cac004c30 tx=0x7f3cac024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c8d302a6089360 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.243+0000 7f3cb2a00640 1 -- 172.21.15.46:0/111334618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac028020 con 0x557bef0b7420 2024-09-10T07:49:48.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.243+0000 7f3cb2a00640 1 -- 172.21.15.46:0/111334618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557beefc0310 con 0x557bef0b7420 2024-09-10T07:49:48.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.243+0000 7f3cb2a00640 1 -- 172.21.15.46:0/111334618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac02d460 con 0x557bef0b7420 2024-09-10T07:49:48.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.245+0000 7f3ccba17740 1 -- 172.21.15.46:0/111334618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 msgr2=0x557bef0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.245+0000 7f3ccba17740 1 --2- 172.21.15.46:0/111334618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0d0fc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3cac004c30 tx=0x7f3cac024000 comp rx=0 tx=0).stop 2024-09-10T07:49:48.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.245+0000 7f3ccba17740 1 -- 172.21.15.46:0/111334618 shutdown_connections 2024-09-10T07:49:48.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.245+0000 7f3ccba17740 1 --2- 172.21.15.46:0/111334618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef0df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.245+0000 7f3ccba17740 1 --2- 172.21.15.46:0/111334618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0dee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.245+0000 7f3ccba17740 1 --2- 172.21.15.46:0/111334618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0d0fc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.245+0000 7f3ccba17740 1 -- 172.21.15.46:0/111334618 >> 172.21.15.46:0/111334618 conn(0x557bef0b3af0 msgr2=0x557bef0de700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.246+0000 7f3ccba17740 1 -- 172.21.15.46:0/111334618 shutdown_connections 2024-09-10T07:49:48.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.246+0000 7f3ccba17740 1 -- 172.21.15.46:0/111334618 wait complete. 2024-09-10T07:49:48.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.246+0000 7f3ccba17740 1 -- start start 2024-09-10T07:49:48.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.246+0000 7f3ccba17740 1 Processor -- start 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.247+0000 7f3ccba17740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.247+0000 7f3ccba17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.247+0000 7f3ccba17740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.247+0000 7f3ccba17740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41b64d0 con 0x557bef0b7420 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.247+0000 7f3ccba17740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41b5e00 con 0x557bef0d3750 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.247+0000 7f3ccba17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41b6bd0 con 0x557bef0b7d40 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.247+0000 7f3ccba17740 10 client.0 _ll_register_callbacks cb 0x7f3cac0146d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.248+0000 7f3cc1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.248+0000 7f3cc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.248+0000 7f3cc1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0e6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35826/0 (socket says 172.21.15.46:35826) 2024-09-10T07:49:48.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.248+0000 7f3cc1400640 1 -- 172.21.15.46:0/177128230 learned_addr learned my addr 172.21.15.46:0/177128230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:48.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.248+0000 7f3cc2800640 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.248+0000 7f3cc1400640 1 -- 172.21.15.46:0/177128230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 msgr2=0x557bef14cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.248+0000 7f3cc1400640 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef14cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.248+0000 7f3cc1400640 1 -- 172.21.15.46:0/177128230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 msgr2=0x557bef0e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.249+0000 7f3cc1400640 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.249+0000 7f3cc1400640 1 -- 172.21.15.46:0/177128230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bef14d110 con 0x557bef0b7d40 2024-09-10T07:49:48.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.249+0000 7f3cc2800640 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:48.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.249+0000 7f3cc1400640 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0e6cb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3cc410c960 tx=0x7f3cb4005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a335bf43ae6bca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.249+0000 7f3ca7e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:48.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.250+0000 7f3ca7e00640 1 -- 172.21.15.46:0/177128230 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4004220 con 0x557bef0b7d40 2024-09-10T07:49:48.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.250+0000 7f3ca7e00640 1 -- 172.21.15.46:0/177128230 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3cb40043c0 con 0x557bef0b7d40 2024-09-10T07:49:48.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.250+0000 7f3ccba17740 1 -- 172.21.15.46:0/177128230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557beefc0310 con 0x557bef0b7d40 2024-09-10T07:49:48.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.251+0000 7f3ca7e00640 1 -- 172.21.15.46:0/177128230 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3cb402a050 con 0x557bef0b7d40 2024-09-10T07:49:48.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.251+0000 7f3ccba17740 10 client.28540 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:48.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.251+0000 7f3ccba17740 1 -- 172.21.15.46:0/177128230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557bef14e300 con 0x557bef0b7d40 2024-09-10T07:49:48.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.251+0000 7f3ca7e00640 1 -- 172.21.15.46:0/177128230 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557beefc0310 con 0x557bef0b7d40 2024-09-10T07:49:48.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.252+0000 7f3ca7e00640 1 -- 172.21.15.46:0/177128230 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f3cb401b3c0 con 0x557bef0b7d40 2024-09-10T07:49:48.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.252+0000 7f3ccba17740 10 client.28540 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:48.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.252+0000 7f3ccba17740 10 client.28540 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:49:48.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.252+0000 7f3ccba17740 20 client.28540 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:48.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.252+0000 7f3ccba17740 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557bef14ea70 0x557bef150e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.252+0000 7f3ccba17740 4 client.28540 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}] 2024-09-10T07:49:48.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.252+0000 7f3ccba17740 1 -- 172.21.15.46:0/177128230 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}) -- 0x557bef024fb0 con 0x557bef14ea70 2024-09-10T07:49:48.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.254+0000 7f3cc2800640 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557bef14ea70 0x557bef150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.254+0000 7f3cc2800640 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557bef14ea70 0x557bef150e90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36509602f042eaa server_cookie=4a678de87f8e51a0 in_seq=0 out_seq=0 2024-09-10T07:49:48.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.255+0000 7f3ca7e00640 10 client.28540 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.264+0000 7f8a49600640 1 -- 172.21.15.46:0/2995928878 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db08d6b030 con 0x55db08d62f90 2024-09-10T07:49:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.264+0000 7f8a49600640 10 client.28613 handle_command_reply: tid=0 2024-09-10T07:49:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.265+0000 7f8a65a86740 1 client.28613 shutdown 2024-09-10T07:49:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.265+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d6b270 msgr2=0x55db08d75a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.265+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d6b270 0x55db08d75a80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8a5409ba90 tx=0x7f8a540a4b80 comp rx=0 tx=0).stop 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.265+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 shutdown_connections 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.265+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55db08d62f90 0x55db08df10c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.265+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db08d6b270 0x55db08d75a80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.265+0000 7f8a49600640 0 client.28613 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.266+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db08d661e0 0x55db08d74a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.266+0000 7f8a65a86740 1 --2- 172.21.15.46:0/2995928878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db08d658a0 0x55db08d74530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.266+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 >> 172.21.15.46:0/2995928878 conn(0x55db08d58700 msgr2=0x55db08d57880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.266+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 shutdown_connections 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.266+0000 7f8a65a86740 1 -- 172.21.15.46:0/2995928878 wait complete. 2024-09-10T07:49:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.266+0000 7f8a65a86740 20 client.28613 trim_cache size 0 max 16384 2024-09-10T07:49:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:49:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:49:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:49:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.267+0000 7f8a62c00640 1 -- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8a400a2d00 msgr2=0x7f8a400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.267+0000 7f8a62c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8a400a2d00 0x7f8a400a51c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55db08d66c10 tx=0x7f8a44008040 comp rx=0 tx=0).stop 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.267+0000 7f8a62c00640 1 -- 172.21.15.46:0/3973859192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c103710 msgr2=0x7f8a5c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.267+0000 7f8a62c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c103710 0x7f8a5c074b20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400eaa0 tx=0x7f8a5400ef60 comp rx=0 tx=0).stop 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.268+0000 7f8a62c00640 1 -- 172.21.15.46:0/3973859192 shutdown_connections 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.268+0000 7f8a62c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5c103710 0x7f8a5c074b20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.268+0000 7f8a62c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8a400a2d00 0x7f8a400a51c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.268+0000 7f8a62c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a5c102d50 0x7f8a5c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.268+0000 7f8a62c00640 1 --2- 172.21.15.46:0/3973859192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5c101b60 0x7f8a5c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.268+0000 7f8a62c00640 1 -- 172.21.15.46:0/3973859192 >> 172.21.15.46:0/3973859192 conn(0x7f8a5c0fd320 msgr2=0x7f8a5c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.268+0000 7f8a62c00640 1 -- 172.21.15.46:0/3973859192 shutdown_connections 2024-09-10T07:49:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:49:48.268+0000 7f8a62c00640 1 -- 172.21.15.46:0/3973859192 wait complete. 2024-09-10T07:49:48.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:48 smithi046 ceph-mon[30705]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-09-10T07:49:48.439 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:48.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.442+0000 7f3ca7e00640 1 -- 172.21.15.46:0/177128230 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557bef024fb0 con 0x557bef14ea70 2024-09-10T07:49:48.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.442+0000 7f3ca7e00640 10 client.28540 handle_command_reply: tid=0 2024-09-10T07:49:48.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.442+0000 7f3ccba17740 1 client.28540 shutdown 2024-09-10T07:49:48.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 -- 172.21.15.46:0/177128230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 msgr2=0x557bef0e6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0e6cb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3cc410c960 tx=0x7f3cb4005410 comp rx=0 tx=0).stop 2024-09-10T07:49:48.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 -- 172.21.15.46:0/177128230 shutdown_connections 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557bef14ea70 0x557bef150e90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557bef0d3750 0x557bef14cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bef0b7d40 0x557bef0e6cb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 --2- 172.21.15.46:0/177128230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bef0b7420 0x557bef0e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 -- 172.21.15.46:0/177128230 >> 172.21.15.46:0/177128230 conn(0x557bef0b3af0 msgr2=0x557bef0def20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ca7e00640 0 client.28540 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 -- 172.21.15.46:0/177128230 shutdown_connections 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 1 -- 172.21.15.46:0/177128230 wait complete. 2024-09-10T07:49:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.443+0000 7f3ccba17740 20 client.28540 trim_cache size 0 max 16384 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 54, 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 46, 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "c6666156": { 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 46, 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:49:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 141.5, 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:48.445 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:49:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ca80a2cb0 msgr2=0x7f3ca80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ca80a2cb0 0x7f3ca80a5170 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0bb8c0 tx=0x7f3cbc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4077040 msgr2=0x7f3cc4112080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4077040 0x7f3cc4112080 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc004770 tx=0x7f3cbc003c10 comp rx=0 tx=0).stop 2024-09-10T07:49:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 shutdown_connections 2024-09-10T07:49:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3ca80a2cb0 0x7f3ca80a5170 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cc410b650 0x7f3cc410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc4112870 0x7f3cc410b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 --2- 172.21.15.46:0/52074493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cc4077040 0x7f3cc4112080 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 >> 172.21.15.46:0/52074493 conn(0x7f3cc40ff910 msgr2=0x7f3cc4114630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 shutdown_connections 2024-09-10T07:49:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:48.444+0000 7f3cc3200640 1 -- 172.21.15.46:0/52074493 wait complete. 2024-09-10T07:49:48.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:48 smithi181 ceph-mon[35397]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-09-10T07:49:48.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:48 smithi063 ceph-mon[35020]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-09-10T07:49:49.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:49 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:49:49.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:49 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:49:49.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:49 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:49:50.168 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c6666156', quiesce took 10.5 seconds. Will release after: 11.0 2024-09-10T07:49:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 -- 172.21.15.46:0/1024064548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac076720 msgr2=0x7f7aac076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/1024064548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac076720 0x7f7aac076b00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009860 tx=0x7f7a9c02a340 comp rx=0 tx=0).stop 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 -- 172.21.15.46:0/1024064548 shutdown_connections 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/1024064548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac112a50 0x7f7aac114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/1024064548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac077040 0x7f7aac112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/1024064548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac076720 0x7f7aac076b00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 -- 172.21.15.46:0/1024064548 >> 172.21.15.46:0/1024064548 conn(0x7f7aac0ffa50 msgr2=0x7f7aac101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 -- 172.21.15.46:0/1024064548 shutdown_connections 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.367+0000 7f7ab1e00640 1 -- 172.21.15.46:0/1024064548 wait complete. 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.368+0000 7f7ab1e00640 1 Processor -- start 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.368+0000 7f7ab1e00640 1 -- start start 2024-09-10T07:49:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7ab1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac076720 0x7f7aac1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7ab1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac077040 0x7f7aac1a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7ab1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac112a50 0x7f7aac19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7ab1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1185c0 con 0x7f7aac076720 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7ab1e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac118440 con 0x7f7aac112a50 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7ab1e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac118740 con 0x7f7aac077040 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac076720 0x7f7aac1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac076720 0x7f7aac1a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54404/0 (socket says 172.21.15.46:54404) 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aaaa00640 1 -- 172.21.15.46:0/968115304 learned_addr learned my addr 172.21.15.46:0/968115304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aa2000640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac077040 0x7f7aac1a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aab400640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac112a50 0x7f7aac19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aaaa00640 1 -- 172.21.15.46:0/968115304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac077040 msgr2=0x7f7aac1a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac077040 0x7f7aac1a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aaaa00640 1 -- 172.21.15.46:0/968115304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac112a50 msgr2=0x7f7aac19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac112a50 0x7f7aac19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aaaa00640 1 -- 172.21.15.46:0/968115304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac19f000 con 0x7f7aac076720 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.369+0000 7f7aab400640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac112a50 0x7f7aac19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.370+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac076720 0x7f7aac1a4710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c035040 tx=0x7f7a9c033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.370+0000 7f7ab0a00640 1 -- 172.21.15.46:0/968115304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c037040 con 0x7f7aac076720 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.370+0000 7f7ab1e00640 1 -- 172.21.15.46:0/968115304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac19f290 con 0x7f7aac076720 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.370+0000 7f7ab0a00640 1 -- 172.21.15.46:0/968115304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c040070 con 0x7f7aac076720 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.370+0000 7f7ab0a00640 1 -- 172.21.15.46:0/968115304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0039f0 con 0x7f7aac076720 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.370+0000 7f7ab1e00640 1 -- 172.21.15.46:0/968115304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac1ab5e0 con 0x7f7aac076720 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab0a00640 1 -- 172.21.15.46:0/968115304 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a9c003b90 con 0x7f7aac076720 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab4a60740 1 Processor -- start 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab4a60740 1 -- start start 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc9369d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc936a7e50 0x55fc936aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fc936033e0 con 0x55fc9369d590 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc935aad80 con 0x55fc9369cc70 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fc935ff3a0 con 0x55fc936a7e50 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54410/0 (socket says 172.21.15.46:54410) 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.374+0000 7f7aaaa00640 1 -- 172.21.15.46:0/996269156 learned_addr learned my addr 172.21.15.46:0/996269156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aab400640 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc9369d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aa2000640 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc936a7e50 0x55fc936aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aaaa00640 1 -- 172.21.15.46:0/996269156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc936a7e50 msgr2=0x55fc936aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc936a7e50 0x55fc936aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aaaa00640 1 -- 172.21.15.46:0/996269156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 msgr2=0x55fc9369d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc9369d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aaaa00640 1 -- 172.21.15.46:0/996269156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc935c9ae0 con 0x55fc9369d590 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aab400640 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc9369d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.375+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a7910 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0f73d0 tx=0x7f7a9c0f74c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e58b3ee29c33b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.376+0000 7f7a9be00640 1 -- 172.21.15.46:0/996269156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0efa80 con 0x55fc9369d590 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.376+0000 7f7a9be00640 1 -- 172.21.15.46:0/996269156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0fc030 con 0x55fc9369d590 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.376+0000 7f7a9be00640 1 -- 172.21.15.46:0/996269156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c101860 con 0x55fc9369d590 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.376+0000 7f7ab0a00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2c00 0x7f7a900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.376+0000 7f7ab0a00640 1 -- 172.21.15.46:0/968115304 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7a9c0ef3d0 con 0x7f7aac076720 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.376+0000 7f7aab400640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2c00 0x7f7a900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 -- 172.21.15.46:0/996269156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 msgr2=0x55fc936a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a7910 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0f73d0 tx=0x7f7a9c0f74c0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 -- 172.21.15.46:0/996269156 shutdown_connections 2024-09-10T07:49:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc936a7e50 0x55fc936aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a7910 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/996269156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc9369d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 -- 172.21.15.46:0/996269156 >> 172.21.15.46:0/996269156 conn(0x55fc93699200 msgr2=0x55fc936995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 -- 172.21.15.46:0/996269156 shutdown_connections 2024-09-10T07:49:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.377+0000 7f7ab4a60740 1 -- 172.21.15.46:0/996269156 wait complete. 2024-09-10T07:49:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.378+0000 7f7aab400640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2c00 0x7f7a900a50c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7aac06b460 tx=0x7f7aa4009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.378+0000 7f7ab4a60740 1 -- start start 2024-09-10T07:49:50.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:50 smithi046 ceph-mon[30705]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-09-10T07:49:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.378+0000 7f7ab4a60740 1 Processor -- start 2024-09-10T07:49:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.378+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.378+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.378+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc936a7e50 0x55fc936a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.378+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fc936aa7e0 con 0x55fc9369d590 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.378+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc935be0f0 con 0x55fc936a7e50 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fc936aa960 con 0x55fc9369cc70 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7ab4a60740 10 client.0 _ll_register_callbacks cb 0x55fc935a5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35866/0 (socket says 172.21.15.46:35866) 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7aaaa00640 1 -- 172.21.15.46:0/217203318 learned_addr learned my addr 172.21.15.46:0/217203318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7aa2000640 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7aab400640 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc936a7e50 0x55fc936a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7aaaa00640 1 -- 172.21.15.46:0/217203318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc936a7e50 msgr2=0x55fc936a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.379+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc936a7e50 0x55fc936a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.380+0000 7f7aaaa00640 1 -- 172.21.15.46:0/217203318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 msgr2=0x55fc936a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.380+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.380+0000 7f7aaaa00640 1 -- 172.21.15.46:0/217203318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc933743c0 con 0x55fc9369cc70 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.380+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a4cf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0bb840 tx=0x7f7a9c0efd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2562c7b45af2462 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.380+0000 7f7a98c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.380+0000 7f7a98c00640 1 -- 172.21.15.46:0/217203318 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0b7b50 con 0x55fc9369cc70 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.380+0000 7f7a98c00640 1 -- 172.21.15.46:0/217203318 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7a9c0b9bb0 con 0x55fc9369cc70 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.380+0000 7f7ab4a60740 1 -- 172.21.15.46:0/217203318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9c0fc030 con 0x55fc9369cc70 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7a98c00640 1 -- 172.21.15.46:0/217203318 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c0bb210 con 0x55fc9369cc70 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7ab4a60740 10 client.28552 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7ab4a60740 1 -- 172.21.15.46:0/217203318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55fc936aa7e0 con 0x55fc9369cc70 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7a98c00640 1 -- 172.21.15.46:0/217203318 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0fc030 con 0x55fc9369cc70 2024-09-10T07:49:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7a98c00640 1 -- 172.21.15.46:0/217203318 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f7a9c03a070 con 0x55fc9369cc70 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7ab4a60740 10 client.28552 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7ab4a60740 10 client.28552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7ab4a60740 20 client.28552 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55fc936b79d0 0x55fc936b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7ab4a60740 4 client.28552 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.381+0000 7f7ab4a60740 1 -- 172.21.15.46:0/217203318 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc93610950 con 0x55fc936b79d0 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.382+0000 7f7aa2000640 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55fc936b79d0 0x55fc936b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.382+0000 7f7aa2000640 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55fc936b79d0 0x55fc936b9e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba9171b65b40e10 server_cookie=8a7450518abb459f in_seq=0 out_seq=0 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.382+0000 7f7a98c00640 10 client.28552 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.383+0000 7f7a98c00640 1 -- 172.21.15.46:0/217203318 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc93610950 con 0x55fc936b79d0 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.383+0000 7f7a98c00640 10 client.28552 handle_command_reply: tid=0 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.383+0000 7f7ab4a60740 1 client.28552 shutdown 2024-09-10T07:49:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 -- 172.21.15.46:0/217203318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 msgr2=0x55fc936a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a4cf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0bb840 tx=0x7f7a9c0efd40 comp rx=0 tx=0).stop 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 -- 172.21.15.46:0/217203318 shutdown_connections 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55fc936b79d0 0x55fc936b9e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc936a7e50 0x55fc936a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc9369d590 0x55fc936a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/217203318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a4cf0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 -- 172.21.15.46:0/217203318 >> 172.21.15.46:0/217203318 conn(0x55fc93699200 msgr2=0x55fc93698380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7a98c00640 0 client.28552 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 -- 172.21.15.46:0/217203318 shutdown_connections 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 1 -- 172.21.15.46:0/217203318 wait complete. 2024-09-10T07:49:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.384+0000 7f7ab4a60740 20 client.28552 trim_cache size 0 max 16384 2024-09-10T07:49:50.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.399+0000 7f7ab4a60740 1 Processor -- start 2024-09-10T07:49:50.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.399+0000 7f7ab4a60740 1 -- start start 2024-09-10T07:49:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.399+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.399+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369d590 0x55fc936bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.399+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.399+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b5380 con 0x55fc936a7e50 2024-09-10T07:49:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.399+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b6f50 con 0x55fc9369d590 2024-09-10T07:49:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.399+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b5a50 con 0x55fc9369cc70 2024-09-10T07:49:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aaaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35874/0 (socket says 172.21.15.46:35874) 2024-09-10T07:49:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aaaa00640 1 -- 172.21.15.46:0/694957788 learned_addr learned my addr 172.21.15.46:0/694957788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aa2000640 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369d590 0x55fc936bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aaaa00640 1 -- 172.21.15.46:0/694957788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369d590 msgr2=0x55fc936bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369d590 0x55fc936bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aaaa00640 1 -- 172.21.15.46:0/694957788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 msgr2=0x55fc936bfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936bfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aaaa00640 1 -- 172.21.15.46:0/694957788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc93694770 con 0x55fc9369cc70 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aa2000640 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369d590 0x55fc936bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.400+0000 7f7aab400640 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.401+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a6c60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0b7d10 tx=0x7f7a9c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb7e4a37327f5c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.401+0000 7f7a9b400640 1 -- 172.21.15.46:0/694957788 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0bb3f0 con 0x55fc9369cc70 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.401+0000 7f7a9b400640 1 -- 172.21.15.46:0/694957788 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0fc070 con 0x55fc9369cc70 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.401+0000 7f7a9b400640 1 -- 172.21.15.46:0/694957788 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0bb680 con 0x55fc9369cc70 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 -- 172.21.15.46:0/694957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 msgr2=0x55fc936a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a6c60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0b7d10 tx=0x7f7a9c005740 comp rx=0 tx=0).stop 2024-09-10T07:49:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 -- 172.21.15.46:0/694957788 shutdown_connections 2024-09-10T07:49:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369d590 0x55fc936bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/694957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369cc70 0x55fc936a6c60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 -- 172.21.15.46:0/694957788 >> 172.21.15.46:0/694957788 conn(0x55fc93699200 msgr2=0x55fc936a6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 -- 172.21.15.46:0/694957788 shutdown_connections 2024-09-10T07:49:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 -- 172.21.15.46:0/694957788 wait complete. 2024-09-10T07:49:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.402+0000 7f7ab4a60740 1 -- start start 2024-09-10T07:49:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7ab4a60740 1 Processor -- start 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc936b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369d590 0x55fc936ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7ab4a60740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b6880 con 0x55fc936a7e50 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b6120 con 0x55fc9369cc70 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7ab4a60740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b3360 con 0x55fc9369d590 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7ab4a60740 10 client.0 _ll_register_callbacks cb 0x7f7aa4009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7aab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7aa2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369d590 0x55fc936ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7aab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936ab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54428/0 (socket says 172.21.15.46:54428) 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7aab400640 1 -- 172.21.15.46:0/3345956214 learned_addr learned my addr 172.21.15.46:0/3345956214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.403+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc936b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.404+0000 7f7aab400640 1 -- 172.21.15.46:0/3345956214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369d590 msgr2=0x55fc936ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.404+0000 7f7aab400640 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369d590 0x55fc936ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.404+0000 7f7aab400640 1 -- 172.21.15.46:0/3345956214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 msgr2=0x55fc936b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.404+0000 7f7aab400640 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc936b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.404+0000 7f7aab400640 1 -- 172.21.15.46:0/3345956214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc936ac060 con 0x55fc936a7e50 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.404+0000 7f7aab400640 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936ab970 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7aac07aa10 tx=0x7f7aa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3252da9671cee9ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.404+0000 7f7a98c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.404+0000 7f7a98c00640 1 -- 172.21.15.46:0/3345956214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa402d060 con 0x55fc936a7e50 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7a98c00640 1 -- 172.21.15.46:0/3345956214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7aa4028600 con 0x55fc936a7e50 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7ab4a60740 1 -- 172.21.15.46:0/3345956214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9c0fc070 con 0x55fc936a7e50 2024-09-10T07:49:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7a98c00640 1 -- 172.21.15.46:0/3345956214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7aa4034050 con 0x55fc936a7e50 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7ab4a60740 10 client.17817 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7ab4a60740 1 -- 172.21.15.46:0/3345956214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55fc937225c0 con 0x55fc936a7e50 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7a98c00640 1 -- 172.21.15.46:0/3345956214 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0fc070 con 0x55fc936a7e50 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7a98c00640 1 -- 172.21.15.46:0/3345956214 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f7aa401a040 con 0x55fc936a7e50 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7ab4a60740 10 client.17817 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7ab4a60740 10 client.17817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7ab4a60740 20 client.17817 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.405+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55fc93722d40 0x55fc93725160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.406+0000 7f7ab4a60740 4 client.17817 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.406+0000 7f7ab4a60740 1 -- 172.21.15.46:0/3345956214 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc936ba1b0 con 0x55fc93722d40 2024-09-10T07:49:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.406+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55fc93722d40 0x55fc93725160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.406+0000 7f7aaaa00640 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55fc93722d40 0x55fc93725160 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94dcc0eb7e8aaad0 server_cookie=2cdf6488c9059a48 in_seq=0 out_seq=0 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.406+0000 7f7a98c00640 10 client.17817 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7a98c00640 1 -- 172.21.15.46:0/3345956214 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc936ba1b0 con 0x55fc93722d40 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7a98c00640 10 client.17817 handle_command_reply: tid=0 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 client.17817 shutdown 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 -- 172.21.15.46:0/3345956214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 msgr2=0x55fc936ab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936ab970 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7aac07aa10 tx=0x7f7aa4004a40 comp rx=0 tx=0).stop 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 -- 172.21.15.46:0/3345956214 shutdown_connections 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55fc93722d40 0x55fc93725160 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc936a7e50 0x55fc936ab970 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc9369d590 0x55fc936ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 --2- 172.21.15.46:0/3345956214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9369cc70 0x55fc936b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7a98c00640 0 client.17817 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:49:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 -- 172.21.15.46:0/3345956214 >> 172.21.15.46:0/3345956214 conn(0x55fc93699200 msgr2=0x55fc936a6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 -- 172.21.15.46:0/3345956214 shutdown_connections 2024-09-10T07:49:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 1 -- 172.21.15.46:0/3345956214 wait complete. 2024-09-10T07:49:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.407+0000 7f7ab4a60740 20 client.17817 trim_cache size 0 max 16384 2024-09-10T07:49:50.410 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 -- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2c00 msgr2=0x7f7a900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2c00 0x7f7a900a50c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7aac06b460 tx=0x7f7aa4009780 comp rx=0 tx=0).stop 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 -- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac076720 msgr2=0x7f7aac1a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac076720 0x7f7aac1a4710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c035040 tx=0x7f7a9c033d90 comp rx=0 tx=0).stop 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 -- 172.21.15.46:0/968115304 shutdown_connections 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aac112a50 0x7f7aac19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aac077040 0x7f7aac1a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f7a900a2c00 0x7f7a900a50c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 --2- 172.21.15.46:0/968115304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aac076720 0x7f7aac1a4710 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.408+0000 7f7ab1e00640 1 -- 172.21.15.46:0/968115304 >> 172.21.15.46:0/968115304 conn(0x7f7aac0ffa50 msgr2=0x7f7aac101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.409+0000 7f7ab1e00640 1 -- 172.21.15.46:0/968115304 shutdown_connections 2024-09-10T07:49:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:50.409+0000 7f7ab1e00640 1 -- 172.21.15.46:0/968115304 wait complete. 2024-09-10T07:49:50.601 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 750.0 s, 204.1 tps, lat 237.980 ms stddev 247.943 2024-09-10T07:49:50.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:50 smithi181 ceph-mon[35397]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-09-10T07:49:50.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:50 smithi063 ceph-mon[35020]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-09-10T07:49:51.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:49:51.635 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:52.362 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:52 smithi046 ceph-mon[30705]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.2 MiB/s wr, 434 op/s 2024-09-10T07:49:52.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:52 smithi181 ceph-mon[35397]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.2 MiB/s wr, 434 op/s 2024-09-10T07:49:52.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:52 smithi063 ceph-mon[35020]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.2 MiB/s wr, 434 op/s 2024-09-10T07:49:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 -- 172.21.15.46:0/2573208368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 msgr2=0x7fc0bc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 --2- 172.21.15.46:0/2573208368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 0x7fc0bc1021d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009900 tx=0x7fc0ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 -- 172.21.15.46:0/2573208368 shutdown_connections 2024-09-10T07:49:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 --2- 172.21.15.46:0/2573208368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0bc106f20 0x7fc0bc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 --2- 172.21.15.46:0/2573208368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc1027a0 0x7fc0bc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 --2- 172.21.15.46:0/2573208368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 0x7fc0bc1021d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 -- 172.21.15.46:0/2573208368 >> 172.21.15.46:0/2573208368 conn(0x7fc0bc0fd8b0 msgr2=0x7fc0bc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 -- 172.21.15.46:0/2573208368 shutdown_connections 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.333+0000 7fc0c1800640 1 -- 172.21.15.46:0/2573208368 wait complete. 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.334+0000 7fc0c1800640 1 Processor -- start 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.334+0000 7fc0c1800640 1 -- start start 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.334+0000 7fc0c1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 0x7fc0bc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.334+0000 7fc0c1800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0bc1027a0 0x7fc0bc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.334+0000 7fc0c1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106f20 0x7fc0bc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.334+0000 7fc0c1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc111fd0 con 0x7fc0bc101df0 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.334+0000 7fc0c1800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc111e50 con 0x7fc0bc106f20 2024-09-10T07:49:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.334+0000 7fc0c1800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc112150 con 0x7fc0bc1027a0 2024-09-10T07:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 0x7fc0bc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 0x7fc0bc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54448/0 (socket says 172.21.15.46:54448) 2024-09-10T07:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0baa00640 1 -- 172.21.15.46:0/3577572649 learned_addr learned my addr 172.21.15.46:0/3577572649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0bb400640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106f20 0x7fc0bc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0b2000640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0bc1027a0 0x7fc0bc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0bb400640 1 -- 172.21.15.46:0/3577572649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0bc1027a0 msgr2=0x7fc0bc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0bb400640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0bc1027a0 0x7fc0bc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0bb400640 1 -- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 msgr2=0x7fc0bc1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0bb400640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 0x7fc0bc1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0bb400640 1 -- 172.21.15.46:0/3577572649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc19d3a0 con 0x7fc0bc106f20 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0baa00640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 0x7fc0bc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.335+0000 7fc0bb400640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106f20 0x7fc0bc19ccc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009fd0 tx=0x7fc0b400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.336+0000 7fc0b3e00640 1 -- 172.21.15.46:0/3577572649 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b400ce20 con 0x7fc0bc106f20 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.336+0000 7fc0c1800640 1 -- 172.21.15.46:0/3577572649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc19d690 con 0x7fc0bc106f20 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.336+0000 7fc0b3e00640 1 -- 172.21.15.46:0/3577572649 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b4005020 con 0x7fc0bc106f20 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.336+0000 7fc0c1800640 1 -- 172.21.15.46:0/3577572649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc1a4f70 con 0x7fc0bc106f20 2024-09-10T07:49:52.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.336+0000 7fc0b3e00640 1 -- 172.21.15.46:0/3577572649 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4010720 con 0x7fc0bc106f20 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.339+0000 7fc0c424c740 1 Processor -- start 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.339+0000 7fc0b3e00640 1 -- 172.21.15.46:0/3577572649 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc0b4005540 con 0x7fc0bc106f20 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.339+0000 7fc0c424c740 1 -- start start 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59378ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e59387d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59388260 0x563e5938a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0c424c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563e592df3e0 con 0x563e59378b10 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0c424c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563e59286d80 con 0x563e59379430 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0c424c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563e592db3a0 con 0x563e59388260 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59378ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59378ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54460/0 (socket says 172.21.15.46:54460) 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0b2000640 1 -- 172.21.15.46:0/824973772 learned_addr learned my addr 172.21.15.46:0/824973772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0baa00640 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e59387d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0b2000640 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59388260 0x563e5938a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0baa00640 1 -- 172.21.15.46:0/824973772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59388260 msgr2=0x563e5938a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0baa00640 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59388260 0x563e5938a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0baa00640 1 -- 172.21.15.46:0/824973772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 msgr2=0x563e59378ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0baa00640 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59378ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0baa00640 1 -- 172.21.15.46:0/824973772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e592a5ae0 con 0x563e59379430 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0b2000640 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59388260 0x563e5938a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.340+0000 7fc0b3e00640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc0a00a2b80 0x7fc0a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.341+0000 7fc0b3e00640 1 -- 172.21.15.46:0/3577572649 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc0b40cf9a0 con 0x7fc0bc106f20 2024-09-10T07:49:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.341+0000 7fc0baa00640 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e59387d20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac031550 tx=0x7fc0ac005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913f228b4aefbf5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.341+0000 7fc0b2000640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc0a00a2b80 0x7fc0a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.341+0000 7fc0abe00640 1 -- 172.21.15.46:0/824973772 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac040070 con 0x563e59379430 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.341+0000 7fc0abe00640 1 -- 172.21.15.46:0/824973772 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac0047a0 con 0x563e59379430 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.341+0000 7fc0abe00640 1 -- 172.21.15.46:0/824973772 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac035040 con 0x563e59379430 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 -- 172.21.15.46:0/824973772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 msgr2=0x563e59387d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e59387d20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac031550 tx=0x7fc0ac005dd0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0b2000640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc0a00a2b80 0x7fc0a00a5040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x563e593762f0 tx=0x7fc0a4015040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 -- 172.21.15.46:0/824973772 shutdown_connections 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59388260 0x563e5938a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e59387d20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 --2- 172.21.15.46:0/824973772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59378ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 -- 172.21.15.46:0/824973772 >> 172.21.15.46:0/824973772 conn(0x563e59375200 msgr2=0x563e593755e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 -- 172.21.15.46:0/824973772 shutdown_connections 2024-09-10T07:49:52.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.342+0000 7fc0c424c740 1 -- 172.21.15.46:0/824973772 wait complete. 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.343+0000 7fc0c424c740 1 -- start start 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.343+0000 7fc0c424c740 1 Processor -- start 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59385180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59379430 0x563e593856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e59385c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0c424c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563e5938ae30 con 0x563e59378b10 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0c424c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563e5929a0f0 con 0x563e59388260 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0c424c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563e5938afb0 con 0x563e59379430 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0c424c740 10 client.0 _ll_register_callbacks cb 0x563e59281bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59385180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0b2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59379430 0x563e593856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0b2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59379430 0x563e593856c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35912/0 (socket says 172.21.15.46:35912) 2024-09-10T07:49:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0b2000640 1 -- 172.21.15.46:0/2659569720 learned_addr learned my addr 172.21.15.46:0/2659569720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0bb400640 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e59385c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0b2000640 1 -- 172.21.15.46:0/2659569720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 msgr2=0x563e59385c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0b2000640 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e59385c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0b2000640 1 -- 172.21.15.46:0/2659569720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 msgr2=0x563e59385180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0b2000640 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59385180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0b2000640 1 -- 172.21.15.46:0/2659569720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e590503c0 con 0x563e59379430 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.344+0000 7fc0baa00640 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59385180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.345+0000 7fc0b2000640 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59379430 0x563e593856c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40038d0 tx=0x7fc0a4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0cd489407b409 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.345+0000 7fc0a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.345+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2659569720 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4010c70 con 0x563e59379430 2024-09-10T07:49:52.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.345+0000 7fc0c424c740 1 -- 172.21.15.46:0/2659569720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0ac0047a0 con 0x563e59379430 2024-09-10T07:49:52.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.345+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2659569720 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc0a4023420 con 0x563e59379430 2024-09-10T07:49:52.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.345+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2659569720 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc0a4023620 con 0x563e59379430 2024-09-10T07:49:52.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.345+0000 7fc0c424c740 10 client.28570 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:52.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.345+0000 7fc0c424c740 1 -- 172.21.15.46:0/2659569720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x563e59396fc0 con 0x563e59379430 2024-09-10T07:49:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2659569720 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac0047a0 con 0x563e59379430 2024-09-10T07:49:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2659569720 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fc0a402b840 con 0x563e59379430 2024-09-10T07:49:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0c424c740 10 client.28570 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0c424c740 10 client.28570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:49:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0c424c740 20 client.28570 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0c424c740 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x563e59398470 0x563e5939a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0c424c740 4 client.28570 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0c424c740 1 -- 172.21.15.46:0/2659569720 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e592ec950 con 0x563e59398470 2024-09-10T07:49:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.346+0000 7fc0baa00640 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x563e59398470 0x563e5939a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.347+0000 7fc0baa00640 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x563e59398470 0x563e5939a900 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe85c7f00800bae server_cookie=20959074f42803e5 in_seq=0 out_seq=0 2024-09-10T07:49:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.347+0000 7fc0a8c00640 10 client.28570 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:49:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2659569720 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e592ec950 con 0x563e59398470 2024-09-10T07:49:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0a8c00640 10 client.28570 handle_command_reply: tid=0 2024-09-10T07:49:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 client.28570 shutdown 2024-09-10T07:49:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 -- 172.21.15.46:0/2659569720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59379430 msgr2=0x563e593856c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59379430 0x563e593856c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40038d0 tx=0x7fc0a4005f90 comp rx=0 tx=0).stop 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 -- 172.21.15.46:0/2659569720 shutdown_connections 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x563e59398470 0x563e5939a900 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e59385c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59379430 0x563e593856c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0a8c00640 0 client.28570 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 --2- 172.21.15.46:0/2659569720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59378b10 0x563e59385180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 -- 172.21.15.46:0/2659569720 >> 172.21.15.46:0/2659569720 conn(0x563e59375200 msgr2=0x563e5937af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 -- 172.21.15.46:0/2659569720 shutdown_connections 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 1 -- 172.21.15.46:0/2659569720 wait complete. 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.348+0000 7fc0c424c740 20 client.28570 trim_cache size 0 max 16384 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.363+0000 7fc0c424c740 1 Processor -- start 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.363+0000 7fc0c424c740 1 -- start start 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.363+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e593870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.363+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e593a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.363+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59388260 0x563e593a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.363+0000 7fc0c424c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1aca80 con 0x563e59388260 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.363+0000 7fc0c424c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ae650 con 0x563e59379430 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.363+0000 7fc0c424c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ad150 con 0x563e59378b10 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0b2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e593a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0baa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e593870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0b2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e593a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52672/0 (socket says 172.21.15.46:52672) 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0b2000640 1 -- 172.21.15.46:0/3585317725 learned_addr learned my addr 172.21.15.46:0/3585317725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0bb400640 1 --2- 172.21.15.46:0/3585317725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59388260 0x563e593a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0baa00640 1 -- 172.21.15.46:0/3585317725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 msgr2=0x563e593a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0baa00640 1 --2- 172.21.15.46:0/3585317725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e593a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0baa00640 1 -- 172.21.15.46:0/3585317725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59388260 msgr2=0x563e593a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0baa00640 1 --2- 172.21.15.46:0/3585317725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59388260 0x563e593a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0baa00640 1 -- 172.21.15.46:0/3585317725 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e59375030 con 0x563e59378b10 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.364+0000 7fc0baa00640 1 --2- 172.21.15.46:0/3585317725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e593870f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac004770 tx=0x7fc0ac047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc584eb841d9fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0ab400640 1 -- 172.21.15.46:0/3585317725 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac040040 con 0x563e59378b10 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0ab400640 1 -- 172.21.15.46:0/3585317725 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac031b80 con 0x563e59378b10 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0ab400640 1 -- 172.21.15.46:0/3585317725 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac035040 con 0x563e59378b10 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 -- 172.21.15.46:0/3585317725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 msgr2=0x563e593870f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3585317725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e593870f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac004770 tx=0x7fc0ac047040 comp rx=0 tx=0).stop 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 -- 172.21.15.46:0/3585317725 shutdown_connections 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3585317725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59388260 0x563e593a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3585317725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59379430 0x563e593a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3585317725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e593870f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 -- 172.21.15.46:0/3585317725 >> 172.21.15.46:0/3585317725 conn(0x563e59375200 msgr2=0x563e593869e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 -- 172.21.15.46:0/3585317725 shutdown_connections 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.365+0000 7fc0c424c740 1 -- 172.21.15.46:0/3585317725 wait complete. 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.366+0000 7fc0c424c740 1 -- start start 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.366+0000 7fc0c424c740 1 Processor -- start 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.366+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e59390a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.366+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59379430 0x563e5938bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.366+0000 7fc0c424c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e5938c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.366+0000 7fc0c424c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1adf80 con 0x563e59379430 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.366+0000 7fc0c424c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ad820 con 0x563e59388260 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.366+0000 7fc0c424c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1aaa60 con 0x563e59378b10 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0c424c740 10 client.0 _ll_register_callbacks cb 0x7fc0a4004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59379430 0x563e5938bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0baa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e59390a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59379430 0x563e5938bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54488/0 (socket says 172.21.15.46:54488) 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0baa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e59390a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35928/0 (socket says 172.21.15.46:35928) 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0b2000640 1 -- 172.21.15.46:0/3084019972 learned_addr learned my addr 172.21.15.46:0/3084019972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0bb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e5938c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0baa00640 1 -- 172.21.15.46:0/3084019972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 msgr2=0x563e5938c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0baa00640 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e5938c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0baa00640 1 -- 172.21.15.46:0/3084019972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59379430 msgr2=0x563e5938bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0baa00640 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59379430 0x563e5938bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0baa00640 1 -- 172.21.15.46:0/3084019972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e59370770 con 0x563e59378b10 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0b2000640 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59379430 0x563e5938bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.367+0000 7fc0bb400640 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e5938c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0baa00640 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e59390a10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac004770 tx=0x7fc0ac000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32d9d89f2cd2b26b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0a8c00640 1 -- 172.21.15.46:0/3084019972 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac057070 con 0x563e59378b10 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0a8c00640 1 -- 172.21.15.46:0/3084019972 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc0ac0037f0 con 0x563e59378b10 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0c424c740 1 -- 172.21.15.46:0/3084019972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0ac031b80 con 0x563e59378b10 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0a8c00640 1 -- 172.21.15.46:0/3084019972 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc0ac063050 con 0x563e59378b10 2024-09-10T07:49:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0c424c740 10 client.28576 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0c424c740 1 -- 172.21.15.46:0/3084019972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x563e59403070 con 0x563e59378b10 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.368+0000 7fc0a8c00640 1 -- 172.21.15.46:0/3084019972 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac031b80 con 0x563e59378b10 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.369+0000 7fc0a8c00640 1 -- 172.21.15.46:0/3084019972 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fc0ac03c440 con 0x563e59378b10 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.369+0000 7fc0c424c740 10 client.28576 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.369+0000 7fc0c424c740 10 client.28576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.369+0000 7fc0c424c740 20 client.28576 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.369+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x563e59403340 0x563e59405730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.369+0000 7fc0c424c740 4 client.28576 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:49:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.369+0000 7fc0c424c740 1 -- 172.21.15.46:0/3084019972 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e592e6e60 con 0x563e59403340 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.369+0000 7fc0b2000640 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x563e59403340 0x563e59405730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0b2000640 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x563e59403340 0x563e59405730 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e717bc77f251a1de server_cookie=c873d3c036d9d39a in_seq=0 out_seq=0 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0a8c00640 10 client.28576 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0a8c00640 1 -- 172.21.15.46:0/3084019972 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e592e6e60 con 0x563e59403340 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0a8c00640 10 client.28576 handle_command_reply: tid=0 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 client.28576 shutdown 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 -- 172.21.15.46:0/3084019972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 msgr2=0x563e59390a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e59390a10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac004770 tx=0x7fc0ac000bc0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 -- 172.21.15.46:0/3084019972 shutdown_connections 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x563e59403340 0x563e59405730 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563e59388260 0x563e5938c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e59379430 0x563e5938bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 --2- 172.21.15.46:0/3084019972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e59378b10 0x563e59390a10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0a8c00640 0 client.28576 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 -- 172.21.15.46:0/3084019972 >> 172.21.15.46:0/3084019972 conn(0x563e59375200 msgr2=0x563e59387200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 -- 172.21.15.46:0/3084019972 shutdown_connections 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.370+0000 7fc0c424c740 1 -- 172.21.15.46:0/3084019972 wait complete. 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c424c740 20 client.28576 trim_cache size 0 max 16384 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:49:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 -- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc0a00a2b80 msgr2=0x7fc0a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc0a00a2b80 0x7fc0a00a5040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x563e593762f0 tx=0x7fc0a4015040 comp rx=0 tx=0).stop 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 -- 172.21.15.46:0/3577572649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106f20 msgr2=0x7fc0bc19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106f20 0x7fc0bc19ccc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009fd0 tx=0x7fc0b400ef10 comp rx=0 tx=0).stop 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 -- 172.21.15.46:0/3577572649 shutdown_connections 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106f20 0x7fc0bc19ccc0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0bc1027a0 0x7fc0bc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc0a00a2b80 0x7fc0a00a5040 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 --2- 172.21.15.46:0/3577572649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0bc101df0 0x7fc0bc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 -- 172.21.15.46:0/3577572649 >> 172.21.15.46:0/3577572649 conn(0x7fc0bc0fd8b0 msgr2=0x7fc0bc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 -- 172.21.15.46:0/3577572649 shutdown_connections 2024-09-10T07:49:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:52.371+0000 7fc0c1800640 1 -- 172.21.15.46:0/3577572649 wait complete. 2024-09-10T07:49:53.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:49:53.902 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:54.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:54 smithi046 ceph-mon[30705]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.2 MiB/s wr, 454 op/s 2024-09-10T07:49:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 -- 172.21.15.46:0/1631875621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 msgr2=0x7fb57c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 --2- 172.21.15.46:0/1631875621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 0x7fb57c10ea90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb574009ca0 tx=0x7fb57402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 -- 172.21.15.46:0/1631875621 shutdown_connections 2024-09-10T07:49:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 --2- 172.21.15.46:0/1631875621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 0x7fb57c10ea90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 --2- 172.21.15.46:0/1631875621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb57c102790 0x7fb57c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 --2- 172.21.15.46:0/1631875621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57c101de0 0x7fb57c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 -- 172.21.15.46:0/1631875621 >> 172.21.15.46:0/1631875621 conn(0x7fb57c0fd8c0 msgr2=0x7fb57c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 -- 172.21.15.46:0/1631875621 shutdown_connections 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.501+0000 7fb583800640 1 -- 172.21.15.46:0/1631875621 wait complete. 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.502+0000 7fb583800640 1 Processor -- start 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.502+0000 7fb583800640 1 -- start start 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb583800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57c101de0 0x7fb57c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb583800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb57c102790 0x7fb57c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb583800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 0x7fb57c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb583800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c111fc0 con 0x7fb57c106f10 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb583800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c111e40 con 0x7fb57c102790 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb583800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c112140 con 0x7fb57c101de0 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb581800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 0x7fb57c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb57be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb57c102790 0x7fb57c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb581800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 0x7fb57c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54500/0 (socket says 172.21.15.46:54500) 2024-09-10T07:49:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb581800640 1 -- 172.21.15.46:0/211930170 learned_addr learned my addr 172.21.15.46:0/211930170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb57be00640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb57c102790 0x7fb57c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52698/0 (socket says 172.21.15.46:52698) 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb580e00640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57c101de0 0x7fb57c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb581800640 1 -- 172.21.15.46:0/211930170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57c101de0 msgr2=0x7fb57c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb581800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57c101de0 0x7fb57c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb581800640 1 -- 172.21.15.46:0/211930170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb57c102790 msgr2=0x7fb57c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb581800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb57c102790 0x7fb57c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.503+0000 7fb581800640 1 -- 172.21.15.46:0/211930170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c19d410 con 0x7fb57c106f10 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.504+0000 7fb581800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 0x7fb57c19cd30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb574005e30 tx=0x7fb574003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.504+0000 7fb579600640 1 -- 172.21.15.46:0/211930170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574031650 con 0x7fb57c106f10 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.504+0000 7fb579600640 1 -- 172.21.15.46:0/211930170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5740317f0 con 0x7fb57c106f10 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.504+0000 7fb583800640 1 -- 172.21.15.46:0/211930170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c19d6a0 con 0x7fb57c106f10 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.504+0000 7fb579600640 1 -- 172.21.15.46:0/211930170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57402f9d0 con 0x7fb57c106f10 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.504+0000 7fb583800640 1 -- 172.21.15.46:0/211930170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c1a4fe0 con 0x7fb57c106f10 2024-09-10T07:49:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.507+0000 7fb579600640 1 -- 172.21.15.46:0/211930170 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb57402fbf0 con 0x7fb57c106f10 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.507+0000 7fb586237740 1 Processor -- start 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb586237740 1 -- start start 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb586237740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e6642a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb586237740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e66434910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb586237740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66434e50 0x559e66437240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb586237740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e663903e0 con 0x559e66429c70 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb586237740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559e66337d80 con 0x559e6642a590 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb586237740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559e6638c3a0 con 0x559e66434e50 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb57be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e6642a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb57be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e6642a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54514/0 (socket says 172.21.15.46:54514) 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb57be00640 1 -- 172.21.15.46:0/3544081099 learned_addr learned my addr 172.21.15.46:0/3544081099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.508+0000 7fb580e00640 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e66434910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb57be00640 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66434e50 0x559e66437240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb57be00640 1 -- 172.21.15.46:0/3544081099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66434e50 msgr2=0x559e66437240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb57be00640 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66434e50 0x559e66437240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb57be00640 1 -- 172.21.15.46:0/3544081099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 msgr2=0x559e66434910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb579600640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5640a2b80 0x7fb5640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb57be00640 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e66434910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb57be00640 1 -- 172.21.15.46:0/3544081099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e66356ae0 con 0x559e66429c70 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb580e00640 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e66434910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb579600640 1 -- 172.21.15.46:0/211930170 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb5740f0870 con 0x7fb57c106f10 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb57be00640 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e6642a050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb5680058c0 tx=0x7fb568005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac3f94d253a2d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.509+0000 7fb581800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5640a2b80 0x7fb5640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb56e000640 1 -- 172.21.15.46:0/3544081099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568004050 con 0x559e66429c70 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb56e000640 1 -- 172.21.15.46:0/3544081099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5680041f0 con 0x559e66429c70 2024-09-10T07:49:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb56e000640 1 -- 172.21.15.46:0/3544081099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56800f700 con 0x559e66429c70 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb586237740 1 -- 172.21.15.46:0/3544081099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 msgr2=0x559e6642a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb586237740 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e6642a050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb5680058c0 tx=0x7fb568005850 comp rx=0 tx=0).stop 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb581800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5640a2b80 0x7fb5640a5040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5740049d0 tx=0x7fb5740093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb586237740 1 -- 172.21.15.46:0/3544081099 shutdown_connections 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb586237740 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66434e50 0x559e66437240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb586237740 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e66434910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb586237740 1 --2- 172.21.15.46:0/3544081099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e6642a050 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.510+0000 7fb586237740 1 -- 172.21.15.46:0/3544081099 >> 172.21.15.46:0/3544081099 conn(0x559e66426200 msgr2=0x559e664265e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.511+0000 7fb586237740 1 -- 172.21.15.46:0/3544081099 shutdown_connections 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.511+0000 7fb586237740 1 -- 172.21.15.46:0/3544081099 wait complete. 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.511+0000 7fb586237740 1 -- start start 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.511+0000 7fb586237740 1 Processor -- start 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb586237740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e66431cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb586237740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6642a590 0x559e66432210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb586237740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e66432750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb586237740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e66437a20 con 0x559e66429c70 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb586237740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559e6634b0f0 con 0x559e66434e50 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb586237740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559e66437ba0 con 0x559e6642a590 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb586237740 10 client.0 _ll_register_callbacks cb 0x559e66332bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb57be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e66431cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb57be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e66431cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54520/0 (socket says 172.21.15.46:54520) 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb57be00640 1 -- 172.21.15.46:0/3019330230 learned_addr learned my addr 172.21.15.46:0/3019330230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.512+0000 7fb580e00640 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6642a590 0x559e66432210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb57be00640 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e66432750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb57be00640 1 -- 172.21.15.46:0/3019330230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6642a590 msgr2=0x559e66432210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb57be00640 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6642a590 0x559e66432210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb57be00640 1 -- 172.21.15.46:0/3019330230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 msgr2=0x559e66432750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb57be00640 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e66432750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb57be00640 1 -- 172.21.15.46:0/3019330230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e661013c0 con 0x559e66429c70 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb580e00640 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6642a590 0x559e66432210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb57be00640 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e66431cd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb568003e80 tx=0x7fb568005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca60f4c61d74fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb562a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb562a00640 1 -- 172.21.15.46:0/3019330230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56802e060 con 0x559e66429c70 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb562a00640 1 -- 172.21.15.46:0/3019330230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb56800f500 con 0x559e66429c70 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.513+0000 7fb586237740 1 -- 172.21.15.46:0/3019330230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5680041f0 con 0x559e66429c70 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb562a00640 1 -- 172.21.15.46:0/3019330230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb568024070 con 0x559e66429c70 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb586237740 10 client.17847 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb586237740 1 -- 172.21.15.46:0/3019330230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559e664439f0 con 0x559e66429c70 2024-09-10T07:49:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb562a00640 1 -- 172.21.15.46:0/3019330230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5680041f0 con 0x559e66429c70 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb562a00640 1 -- 172.21.15.46:0/3019330230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fb56801e070 con 0x559e66429c70 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb586237740 10 client.17847 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb586237740 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb586237740 20 client.17847 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb586237740 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e66444e50 0x559e66447240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb586237740 4 client.17847 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.514+0000 7fb586237740 1 -- 172.21.15.46:0/3019330230 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6639d950 con 0x559e66444e50 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.515+0000 7fb580e00640 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e66444e50 0x559e66447240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.515+0000 7fb580e00640 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e66444e50 0x559e66447240 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e826dfa8ffb9c81 server_cookie=c5cb01fd8eb6a727 in_seq=0 out_seq=0 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.515+0000 7fb562a00640 10 client.17847 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb562a00640 1 -- 172.21.15.46:0/3019330230 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e6639d950 con 0x559e66444e50 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb562a00640 10 client.17847 handle_command_reply: tid=0 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 client.17847 shutdown 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 -- 172.21.15.46:0/3019330230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 msgr2=0x559e66431cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e66431cd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb568003e80 tx=0x7fb568005f90 comp rx=0 tx=0).stop 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 -- 172.21.15.46:0/3019330230 shutdown_connections 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e66444e50 0x559e66447240 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e66432750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e6642a590 0x559e66432210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 --2- 172.21.15.46:0/3019330230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66429c70 0x559e66431cd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb562a00640 0 client.17847 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 -- 172.21.15.46:0/3019330230 >> 172.21.15.46:0/3019330230 conn(0x559e66426200 msgr2=0x559e6642bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 -- 172.21.15.46:0/3019330230 shutdown_connections 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.516+0000 7fb586237740 1 -- 172.21.15.46:0/3019330230 wait complete. 2024-09-10T07:49:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.517+0000 7fb586237740 20 client.17847 trim_cache size 0 max 16384 2024-09-10T07:49:54.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb586237740 1 Processor -- start 2024-09-10T07:49:54.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb586237740 1 -- start start 2024-09-10T07:49:54.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb586237740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 0x559e66433c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb586237740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e6642a590 0x559e6643d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb586237740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e6643d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb586237740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1aec10 con 0x559e6642a590 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb586237740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1b07e0 con 0x559e66434e50 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb586237740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1af2e0 con 0x559e66429c70 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb580e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e6642a590 0x559e6643d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb580e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e6642a590 0x559e6643d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54526/0 (socket says 172.21.15.46:54526) 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.531+0000 7fb580e00640 1 -- 172.21.15.46:0/2544961218 learned_addr learned my addr 172.21.15.46:0/2544961218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb57be00640 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 0x559e66433c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb580e00640 1 -- 172.21.15.46:0/2544961218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 msgr2=0x559e66433c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb57be00640 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e6643d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb580e00640 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 0x559e66433c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb580e00640 1 -- 172.21.15.46:0/2544961218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 msgr2=0x559e6643d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb580e00640 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e6643d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb580e00640 1 -- 172.21.15.46:0/2544961218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e66426030 con 0x559e6642a590 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb57be00640 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e6643d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb580e00640 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e6642a590 0x559e6643d1d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb570005d00 tx=0x7fb570005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842141c9b64973d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb56d600640 1 -- 172.21.15.46:0/2544961218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570010630 con 0x559e6642a590 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb56d600640 1 -- 172.21.15.46:0/2544961218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb570010cb0 con 0x559e6642a590 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.532+0000 7fb56d600640 1 -- 172.21.15.46:0/2544961218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57001edb0 con 0x559e6642a590 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 -- 172.21.15.46:0/2544961218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e6642a590 msgr2=0x559e6643d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e6642a590 0x559e6643d1d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb570005d00 tx=0x7fb570005f90 comp rx=0 tx=0).stop 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 -- 172.21.15.46:0/2544961218 shutdown_connections 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e66434e50 0x559e6643d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e6642a590 0x559e6643d1d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 --2- 172.21.15.46:0/2544961218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 0x559e66433c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 -- 172.21.15.46:0/2544961218 >> 172.21.15.46:0/2544961218 conn(0x559e66426200 msgr2=0x559e66433530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 -- 172.21.15.46:0/2544961218 shutdown_connections 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.533+0000 7fb586237740 1 -- 172.21.15.46:0/2544961218 wait complete. 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 1 -- start start 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 1 Processor -- start 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 0x559e6642df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e6642e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66434e50 0x559e6642eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1b0110 con 0x559e66434e50 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1af9b0 con 0x559e6642a590 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1acbf0 con 0x559e66429c70 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb586237740 10 client.0 _ll_register_callbacks cb 0x7fb570009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.534+0000 7fb57be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66434e50 0x559e6642eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb580e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e6642e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66434e50 0x559e6642eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54542/0 (socket says 172.21.15.46:54542) 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 -- 172.21.15.46:0/3842053657 learned_addr learned my addr 172.21.15.46:0/3842053657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 0x559e6642df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 -- 172.21.15.46:0/3842053657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 msgr2=0x559e6642df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 0x559e6642df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 -- 172.21.15.46:0/3842053657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 msgr2=0x559e6642e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e6642e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 -- 172.21.15.46:0/3842053657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6642cae0 con 0x559e66434e50 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb580e00640 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e6642e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb57be00640 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66434e50 0x559e6642eab0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb568010940 tx=0x7fb568005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f6ded96d67789e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.535+0000 7fb562a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb562a00640 1 -- 172.21.15.46:0/3842053657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56802e040 con 0x559e66434e50 2024-09-10T07:49:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 1 -- 172.21.15.46:0/3842053657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb570010cb0 con 0x559e66434e50 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb562a00640 1 -- 172.21.15.46:0/3842053657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb56800f500 con 0x559e66434e50 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb562a00640 1 -- 172.21.15.46:0/3842053657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb568044050 con 0x559e66434e50 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 10 client.17853 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 1 -- 172.21.15.46:0/3842053657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559e66426e90 con 0x559e66434e50 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb562a00640 1 -- 172.21.15.46:0/3842053657 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb570010cb0 con 0x559e66434e50 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb562a00640 1 -- 172.21.15.46:0/3842053657 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fb56801e040 con 0x559e66434e50 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 10 client.17853 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 20 client.17853 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e664b41b0 0x559e664b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 4 client.17853 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.536+0000 7fb586237740 1 -- 172.21.15.46:0/3842053657 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e66397e60 con 0x559e664b41b0 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.537+0000 7fb580e00640 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e664b41b0 0x559e664b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.537+0000 7fb580e00640 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e664b41b0 0x559e664b65d0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc053ec188dbdde server_cookie=83abf6f20f158a8f in_seq=0 out_seq=0 2024-09-10T07:49:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.537+0000 7fb562a00640 10 client.17853 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb562a00640 1 -- 172.21.15.46:0/3842053657 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e66397e60 con 0x559e664b41b0 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb562a00640 10 client.17853 handle_command_reply: tid=0 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 client.17853 shutdown 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 -- 172.21.15.46:0/3842053657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66434e50 msgr2=0x559e6642eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66434e50 0x559e6642eab0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb568010940 tx=0x7fb568005f90 comp rx=0 tx=0).stop 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 -- 172.21.15.46:0/3842053657 shutdown_connections 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e664b41b0 0x559e664b65d0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e66434e50 0x559e6642eab0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e6642a590 0x559e6642e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 --2- 172.21.15.46:0/3842053657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e66429c70 0x559e6642df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 -- 172.21.15.46:0/3842053657 >> 172.21.15.46:0/3842053657 conn(0x559e66426200 msgr2=0x559e66433d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb562a00640 0 client.17853 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 -- 172.21.15.46:0/3842053657 shutdown_connections 2024-09-10T07:49:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 1 -- 172.21.15.46:0/3842053657 wait complete. 2024-09-10T07:49:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.538+0000 7fb586237740 20 client.17853 trim_cache size 0 max 16384 2024-09-10T07:49:54.547 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:49:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 -- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5640a2b80 msgr2=0x7fb5640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5640a2b80 0x7fb5640a5040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5740049d0 tx=0x7fb5740093d0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 -- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 msgr2=0x7fb57c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 0x7fb57c19cd30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb574005e30 tx=0x7fb574003d00 comp rx=0 tx=0).stop 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 -- 172.21.15.46:0/211930170 shutdown_connections 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb5640a2b80 0x7fb5640a5040 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c106f10 0x7fb57c19cd30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb57c102790 0x7fb57c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 --2- 172.21.15.46:0/211930170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57c101de0 0x7fb57c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 -- 172.21.15.46:0/211930170 >> 172.21.15.46:0/211930170 conn(0x7fb57c0fd8c0 msgr2=0x7fb57c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 -- 172.21.15.46:0/211930170 shutdown_connections 2024-09-10T07:49:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:54.539+0000 7fb583800640 1 -- 172.21.15.46:0/211930170 wait complete. 2024-09-10T07:49:54.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:54 smithi181 ceph-mon[35397]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.2 MiB/s wr, 454 op/s 2024-09-10T07:49:54.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:54 smithi063 ceph-mon[35020]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.2 MiB/s wr, 454 op/s 2024-09-10T07:49:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 755.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:49:56.336 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:49:56.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:49:56.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:56 smithi181 ceph-mon[35397]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-09-10T07:49:56.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:56 smithi063 ceph-mon[35020]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-09-10T07:49:56.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:56 smithi046 ceph-mon[30705]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-09-10T07:49:56.728 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 -- 172.21.15.46:0/1585546492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e181047e0 msgr2=0x7f0e18108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1585546492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e181047e0 0x7f0e18108830 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009900 tx=0x7f0e0002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 -- 172.21.15.46:0/1585546492 shutdown_connections 2024-09-10T07:49:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1585546492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e18108f60 0x7f0e18110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1585546492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e181047e0 0x7f0e18108830 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1585546492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18103e30 0x7f0e18104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 -- 172.21.15.46:0/1585546492 >> 172.21.15.46:0/1585546492 conn(0x7f0e180ff910 msgr2=0x7f0e18101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 -- 172.21.15.46:0/1585546492 shutdown_connections 2024-09-10T07:49:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.304+0000 7f0e17e00640 1 -- 172.21.15.46:0/1585546492 wait complete. 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.305+0000 7f0e17e00640 1 Processor -- start 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.305+0000 7f0e17e00640 1 -- start start 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e17e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18103e30 0x7f0e1819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e17e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e181047e0 0x7f0e1819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e18108f60 0x7f0e1819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e17e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18113f80 con 0x7f0e18108f60 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e17e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18113e00 con 0x7f0e18103e30 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e17e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18114100 con 0x7f0e181047e0 2024-09-10T07:49:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18103e30 0x7f0e1819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e16a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18103e30 0x7f0e1819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56708/0 (socket says 172.21.15.46:56708) 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e181047e0 0x7f0e1819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e16a00640 1 -- 172.21.15.46:0/1874011798 learned_addr learned my addr 172.21.15.46:0/1874011798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.306+0000 7f0e17400640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e18108f60 0x7f0e1819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e16a00640 1 -- 172.21.15.46:0/1874011798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e181047e0 msgr2=0x7f0e1819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e16a00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e181047e0 0x7f0e1819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e16a00640 1 -- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e18108f60 msgr2=0x7f0e1819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e16a00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e18108f60 0x7f0e1819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e16a00640 1 -- 172.21.15.46:0/1874011798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1819f490 con 0x7f0e18103e30 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e181047e0 0x7f0e1819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e16a00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18103e30 0x7f0e1819a4e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800ea60 tx=0x7f0e0800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e0f400640 1 -- 172.21.15.46:0/1874011798 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0800cde0 con 0x7f0e18103e30 2024-09-10T07:49:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e0f400640 1 -- 172.21.15.46:0/1874011798 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e080035f0 con 0x7f0e18103e30 2024-09-10T07:49:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e17e00640 1 -- 172.21.15.46:0/1874011798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1819f780 con 0x7f0e18103e30 2024-09-10T07:49:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e0f400640 1 -- 172.21.15.46:0/1874011798 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08010670 con 0x7f0e18103e30 2024-09-10T07:49:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.307+0000 7f0e17e00640 1 -- 172.21.15.46:0/1874011798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e181a7060 con 0x7f0e18103e30 2024-09-10T07:49:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.310+0000 7f0e2002a740 1 Processor -- start 2024-09-10T07:49:57.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.310+0000 7f0e0f400640 1 -- 172.21.15.46:0/1874011798 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e080037a0 con 0x7f0e18103e30 2024-09-10T07:49:57.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e2002a740 1 -- start start 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e41950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e44280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e2002a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ff1d9d290 con 0x556ff1e375d0 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e2002a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556ff1d44b40 con 0x556ff1e41e90 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e2002a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556ff1d99250 con 0x556ff1e36cb0 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e16a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e41950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e17400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e17400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e37090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48760/0 (socket says 172.21.15.46:48760) 2024-09-10T07:49:57.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e17400640 1 -- 172.21.15.46:0/1414417050 learned_addr learned my addr 172.21.15.46:0/1414417050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:57.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.311+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/1414417050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e44280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e17400640 1 -- 172.21.15.46:0/1414417050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 msgr2=0x556ff1e44280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e17400640 1 --2- 172.21.15.46:0/1414417050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e44280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e17400640 1 -- 172.21.15.46:0/1414417050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 msgr2=0x556ff1e41950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e17400640 1 --2- 172.21.15.46:0/1414417050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e41950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e17400640 1 -- 172.21.15.46:0/1414417050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff1b0e6a0 con 0x556ff1e36cb0 2024-09-10T07:49:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e0f400640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0dfc0a2c00 0x7f0dfc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e0f400640 1 -- 172.21.15.46:0/1874011798 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0e080cf6b0 con 0x7f0e18103e30 2024-09-10T07:49:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e17400640 1 --2- 172.21.15.46:0/1414417050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e37090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009f50 tx=0x7f0e100118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144ccd932b65fbf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.312+0000 7f0e16a00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0dfc0a2c00 0x7f0dfc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.313+0000 7f0e07400640 1 -- 172.21.15.46:0/1414417050 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10004d20 con 0x556ff1e36cb0 2024-09-10T07:49:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.313+0000 7f0e07400640 1 -- 172.21.15.46:0/1414417050 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e10004050 con 0x556ff1e36cb0 2024-09-10T07:49:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.313+0000 7f0e07400640 1 -- 172.21.15.46:0/1414417050 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10005260 con 0x556ff1e36cb0 2024-09-10T07:49:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.313+0000 7f0e2002a740 1 -- 172.21.15.46:0/1414417050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 msgr2=0x556ff1e37090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.313+0000 7f0e2002a740 1 --2- 172.21.15.46:0/1414417050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e37090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009f50 tx=0x7f0e100118d0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.313+0000 7f0e16a00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0dfc0a2c00 0x7f0dfc0a50c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0e080a4270 tx=0x7f0e080d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 -- 172.21.15.46:0/1414417050 shutdown_connections 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 --2- 172.21.15.46:0/1414417050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e44280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 --2- 172.21.15.46:0/1414417050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e41950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 --2- 172.21.15.46:0/1414417050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e37090 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 -- 172.21.15.46:0/1414417050 >> 172.21.15.46:0/1414417050 conn(0x556ff1e33240 msgr2=0x556ff1e33620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 -- 172.21.15.46:0/1414417050 shutdown_connections 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 -- 172.21.15.46:0/1414417050 wait complete. 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 -- start start 2024-09-10T07:49:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.314+0000 7f0e2002a740 1 Processor -- start 2024-09-10T07:49:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e41730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e3e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e2002a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ff1e44b00 con 0x556ff1e375d0 2024-09-10T07:49:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e2002a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556ff1d57fa0 con 0x556ff1e41e90 2024-09-10T07:49:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e2002a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556ff1e44c80 con 0x556ff1e36cb0 2024-09-10T07:49:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e2002a740 10 client.0 _ll_register_callbacks cb 0x556ff1aabba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e3e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e3e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51094/0 (socket says 172.21.15.46:51094) 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e17400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e41730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e0fe00640 1 -- 172.21.15.46:0/2062720827 learned_addr learned my addr 172.21.15.46:0/2062720827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.315+0000 7f0e17400640 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0e0fe00640 1 -- 172.21.15.46:0/2062720827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 msgr2=0x556ff1e41730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e41730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0e0fe00640 1 -- 172.21.15.46:0/2062720827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 msgr2=0x556ff1e3ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e3ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0e0fe00640 1 -- 172.21.15.46:0/2062720827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff1e3f720 con 0x556ff1e375d0 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0e17400640 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e3e9e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009900 tx=0x7f0e000317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38af537a9a668e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0dfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:49:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0dfbe00640 1 -- 172.21.15.46:0/2062720827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00035040 con 0x556ff1e375d0 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0dfbe00640 1 -- 172.21.15.46:0/2062720827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0e00004450 con 0x556ff1e375d0 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.316+0000 7f0e2002a740 1 -- 172.21.15.46:0/2062720827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e10004050 con 0x556ff1e375d0 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0dfbe00640 1 -- 172.21.15.46:0/2062720827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0e00050050 con 0x556ff1e375d0 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0e2002a740 10 client.17865 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0e2002a740 1 -- 172.21.15.46:0/2062720827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556ff1e50980 con 0x556ff1e375d0 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0dfbe00640 1 -- 172.21.15.46:0/2062720827 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e10004050 con 0x556ff1e375d0 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0dfbe00640 1 -- 172.21.15.46:0/2062720827 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f0e00004c60 con 0x556ff1e375d0 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0e2002a740 10 client.17865 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0e2002a740 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0e2002a740 20 client.17865 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0e2002a740 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ff1e51e40 0x556ff1e54230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0e2002a740 4 client.17865 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.317+0000 7f0e2002a740 1 -- 172.21.15.46:0/2062720827 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ff1daa800 con 0x556ff1e51e40 2024-09-10T07:49:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.318+0000 7f0e17400640 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ff1e51e40 0x556ff1e54230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.318+0000 7f0e17400640 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ff1e51e40 0x556ff1e54230 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af7197efd7a1b7e server_cookie=23adb2fab2cc93ce in_seq=0 out_seq=0 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.318+0000 7f0dfbe00640 10 client.17865 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0dfbe00640 1 -- 172.21.15.46:0/2062720827 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ff1daa800 con 0x556ff1e51e40 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0dfbe00640 10 client.17865 handle_command_reply: tid=0 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 client.17865 shutdown 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 -- 172.21.15.46:0/2062720827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 msgr2=0x556ff1e3e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e3e9e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009900 tx=0x7f0e000317e0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 -- 172.21.15.46:0/2062720827 shutdown_connections 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ff1e51e40 0x556ff1e54230 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e3e9e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 --2- 172.21.15.46:0/2062720827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e41730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0dfbe00640 0 client.17865 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 -- 172.21.15.46:0/2062720827 >> 172.21.15.46:0/2062720827 conn(0x556ff1e33240 msgr2=0x556ff1e38f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 -- 172.21.15.46:0/2062720827 shutdown_connections 2024-09-10T07:49:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.319+0000 7f0e2002a740 1 -- 172.21.15.46:0/2062720827 wait complete. 2024-09-10T07:49:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.320+0000 7f0e2002a740 20 client.17865 trim_cache size 0 max 16384 2024-09-10T07:49:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.338+0000 7f0e2002a740 1 Processor -- start 2024-09-10T07:49:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.338+0000 7f0e2002a740 1 -- start start 2024-09-10T07:49:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.338+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e59b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.338+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e375d0 0x556ff1e5a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.338+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e41e90 0x556ff1e3bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e2002a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181b0cf0 con 0x556ff1e41e90 2024-09-10T07:49:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e2002a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181b28c0 con 0x556ff1e375d0 2024-09-10T07:49:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e2002a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181b13c0 con 0x556ff1e36cb0 2024-09-10T07:49:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e41e90 0x556ff1e3bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e375d0 0x556ff1e5a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e375d0 0x556ff1e5a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56730/0 (socket says 172.21.15.46:56730) 2024-09-10T07:49:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e41e90 0x556ff1e3bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51110/0 (socket says 172.21.15.46:51110) 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e0fe00640 1 -- 172.21.15.46:0/1181348930 learned_addr learned my addr 172.21.15.46:0/1181348930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e17400640 1 --2- 172.21.15.46:0/1181348930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e59b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e0fe00640 1 -- 172.21.15.46:0/1181348930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 msgr2=0x556ff1e59b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/1181348930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e59b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e0fe00640 1 -- 172.21.15.46:0/1181348930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e41e90 msgr2=0x556ff1e3bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/1181348930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e41e90 0x556ff1e3bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.339+0000 7f0e0fe00640 1 -- 172.21.15.46:0/1181348930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff1e32ce0 con 0x556ff1e375d0 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.340+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/1181348930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e375d0 0x556ff1e5a040 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0e00033000 tx=0x7f0e0002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=195ec6d95527230c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.340+0000 7f0e06a00640 1 -- 172.21.15.46:0/1181348930 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00035070 con 0x556ff1e375d0 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.340+0000 7f0e06a00640 1 -- 172.21.15.46:0/1181348930 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e00003900 con 0x556ff1e375d0 2024-09-10T07:49:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.340+0000 7f0e06a00640 1 -- 172.21.15.46:0/1181348930 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00004490 con 0x556ff1e375d0 2024-09-10T07:49:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 -- 172.21.15.46:0/1181348930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e375d0 msgr2=0x556ff1e5a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 --2- 172.21.15.46:0/1181348930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e375d0 0x556ff1e5a040 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0e00033000 tx=0x7f0e0002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 -- 172.21.15.46:0/1181348930 shutdown_connections 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 --2- 172.21.15.46:0/1181348930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e41e90 0x556ff1e3bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 --2- 172.21.15.46:0/1181348930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e375d0 0x556ff1e5a040 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 --2- 172.21.15.46:0/1181348930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e59b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 -- 172.21.15.46:0/1181348930 >> 172.21.15.46:0/1181348930 conn(0x556ff1e33240 msgr2=0x556ff1e40a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 -- 172.21.15.46:0/1181348930 shutdown_connections 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 -- 172.21.15.46:0/1181348930 wait complete. 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 -- start start 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.341+0000 7f0e2002a740 1 Processor -- start 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e4a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e4a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e2002a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e46ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e2002a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181b21f0 con 0x556ff1e375d0 2024-09-10T07:49:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e2002a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181b1a90 con 0x556ff1e41e90 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e2002a740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181aecd0 con 0x556ff1e36cb0 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e2002a740 10 client.0 _ll_register_callbacks cb 0x7f0e00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e4a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e4a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51118/0 (socket says 172.21.15.46:51118) 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e0fe00640 1 -- 172.21.15.46:0/3410570178 learned_addr learned my addr 172.21.15.46:0/3410570178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e17400640 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e4a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.342+0000 7f0e17400640 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e46ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e0fe00640 1 -- 172.21.15.46:0/3410570178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 msgr2=0x556ff1e4a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e4a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e0fe00640 1 -- 172.21.15.46:0/3410570178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 msgr2=0x556ff1e46ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e46ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e0fe00640 1 -- 172.21.15.46:0/3410570178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff1e476a0 con 0x556ff1e375d0 2024-09-10T07:49:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e17400640 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e46ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e0fe00640 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e4a5f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0e000056c0 tx=0x7f0e00004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfbd6b6a9ad2d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0dfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0dfbe00640 1 -- 172.21.15.46:0/3410570178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0004d040 con 0x556ff1e375d0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0dfbe00640 1 -- 172.21.15.46:0/3410570178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0e0003d450 con 0x556ff1e375d0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e2002a740 1 -- 172.21.15.46:0/3410570178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e00003900 con 0x556ff1e375d0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0dfbe00640 1 -- 172.21.15.46:0/3410570178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0e00035040 con 0x556ff1e375d0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e2002a740 10 client.17871 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.343+0000 7f0e2002a740 1 -- 172.21.15.46:0/3410570178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556ff1ebd620 con 0x556ff1e375d0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0dfbe00640 1 -- 172.21.15.46:0/3410570178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e00003900 con 0x556ff1e375d0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0dfbe00640 1 -- 172.21.15.46:0/3410570178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f0e000474c0 con 0x556ff1e375d0 2024-09-10T07:49:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0e2002a740 10 client.17871 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0e2002a740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0e2002a740 20 client.17871 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0e2002a740 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ff1ebdc50 0x556ff1ec0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0e2002a740 4 client.17871 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:49:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0e2002a740 1 -- 172.21.15.46:0/3410570178 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556ff1da4d10 con 0x556ff1ebdc50 2024-09-10T07:49:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.344+0000 7f0e17400640 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ff1ebdc50 0x556ff1ec0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.345+0000 7f0e17400640 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ff1ebdc50 0x556ff1ec0070 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e8387b280b0598 server_cookie=f3ba9ef6f34ff02 in_seq=0 out_seq=0 2024-09-10T07:49:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.345+0000 7f0dfbe00640 10 client.17871 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.346+0000 7f0dfbe00640 1 -- 172.21.15.46:0/3410570178 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556ff1da4d10 con 0x556ff1ebdc50 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.346+0000 7f0dfbe00640 10 client.17871 handle_command_reply: tid=0 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.346+0000 7f0e2002a740 1 client.17871 shutdown 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 -- 172.21.15.46:0/3410570178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 msgr2=0x556ff1e4a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e4a5f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0e000056c0 tx=0x7f0e00004480 comp rx=0 tx=0).stop 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 -- 172.21.15.46:0/3410570178 shutdown_connections 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556ff1ebdc50 0x556ff1ec0070 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556ff1e41e90 0x556ff1e46ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ff1e375d0 0x556ff1e4a5f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 --2- 172.21.15.46:0/3410570178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556ff1e36cb0 0x556ff1e4a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 -- 172.21.15.46:0/3410570178 >> 172.21.15.46:0/3410570178 conn(0x556ff1e33240 msgr2=0x556ff1e31a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0dfbe00640 0 client.17871 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.347+0000 7f0e2002a740 1 -- 172.21.15.46:0/3410570178 shutdown_connections 2024-09-10T07:49:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.348+0000 7f0e2002a740 1 -- 172.21.15.46:0/3410570178 wait complete. 2024-09-10T07:49:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.348+0000 7f0e2002a740 20 client.17871 trim_cache size 0 max 16384 2024-09-10T07:49:57.350 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:49:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:49:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4f823651-711e-4891-8594-f2bf5d36b851", 2024-09-10T07:49:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:49:57.350 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 -- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0dfc0a2c00 msgr2=0x7f0dfc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0dfc0a2c00 0x7f0dfc0a50c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0e080a4270 tx=0x7f0e080d5530 comp rx=0 tx=0).stop 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 -- 172.21.15.46:0/1874011798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18103e30 msgr2=0x7f0e1819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18103e30 0x7f0e1819a4e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800ea60 tx=0x7f0e0800ef20 comp rx=0 tx=0).stop 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 -- 172.21.15.46:0/1874011798 shutdown_connections 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0dfc0a2c00 0x7f0dfc0a50c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e18108f60 0x7f0e1819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e181047e0 0x7f0e1819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 --2- 172.21.15.46:0/1874011798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18103e30 0x7f0e1819a4e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 -- 172.21.15.46:0/1874011798 >> 172.21.15.46:0/1874011798 conn(0x7f0e180ff910 msgr2=0x7f0e18100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 -- 172.21.15.46:0/1874011798 shutdown_connections 2024-09-10T07:49:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:57.349+0000 7f0e17e00640 1 -- 172.21.15.46:0/1874011798 wait complete. 2024-09-10T07:49:58.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:49:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-09-10T07:49:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:49:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:49:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:49:58.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:49:58.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:49:58.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:58 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-09-10T07:49:58.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:49:58.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:49:58.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:49:58.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:49:58.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:49:58.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:58 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-09-10T07:49:58.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:49:58.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:49:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:49:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:49:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:49:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:58 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.305+0000 7fcf9d200640 1 -- 172.21.15.46:0/910436194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98077040 msgr2=0x7fcf981124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.305+0000 7fcf9d200640 1 --2- 172.21.15.46:0/910436194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98077040 0x7fcf981124f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009900 tx=0x7fcf8002a400 comp rx=0 tx=0).stop 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.305+0000 7fcf9d200640 1 -- 172.21.15.46:0/910436194 shutdown_connections 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.305+0000 7fcf9d200640 1 --2- 172.21.15.46:0/910436194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf98112a30 0x7fcf98114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.305+0000 7fcf9d200640 1 --2- 172.21.15.46:0/910436194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98077040 0x7fcf981124f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.305+0000 7fcf9d200640 1 --2- 172.21.15.46:0/910436194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf98076720 0x7fcf98076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.305+0000 7fcf9d200640 1 -- 172.21.15.46:0/910436194 >> 172.21.15.46:0/910436194 conn(0x7fcf980ffa90 msgr2=0x7fcf98101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.305+0000 7fcf9d200640 1 -- 172.21.15.46:0/910436194 shutdown_connections 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.306+0000 7fcf9d200640 1 -- 172.21.15.46:0/910436194 wait complete. 2024-09-10T07:49:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.306+0000 7fcf9d200640 1 Processor -- start 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf9d200640 1 -- start start 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf9d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf98076720 0x7fcf981a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf9d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf98077040 0x7fcf981a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf9d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98112a30 0x7fcf9819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf9d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98118600 con 0x7fcf98112a30 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf9d200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98118480 con 0x7fcf98077040 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf9d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98118780 con 0x7fcf98076720 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf96800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98112a30 0x7fcf9819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf96800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98112a30 0x7fcf9819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51136/0 (socket says 172.21.15.46:51136) 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf96800640 1 -- 172.21.15.46:0/541136128 learned_addr learned my addr 172.21.15.46:0/541136128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf98077040 0x7fcf981a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.307+0000 7fcf95e00640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf98076720 0x7fcf981a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf96800640 1 -- 172.21.15.46:0/541136128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf98076720 msgr2=0x7fcf981a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf96800640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf98076720 0x7fcf981a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf96800640 1 -- 172.21.15.46:0/541136128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf98077040 msgr2=0x7fcf981a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf96800640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf98077040 0x7fcf981a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf96800640 1 -- 172.21.15.46:0/541136128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9819efc0 con 0x7fcf98112a30 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf96800640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98112a30 0x7fcf9819e890 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000eb10 tx=0x7fcf9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf8ea00640 1 -- 172.21.15.46:0/541136128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004b20 con 0x7fcf98112a30 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf8ea00640 1 -- 172.21.15.46:0/541136128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90004cc0 con 0x7fcf98112a30 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf8ea00640 1 -- 172.21.15.46:0/541136128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90010790 con 0x7fcf98112a30 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf9d200640 1 -- 172.21.15.46:0/541136128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9819f2b0 con 0x7fcf98112a30 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.308+0000 7fcf9d200640 1 -- 172.21.15.46:0/541136128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981ab5a0 con 0x7fcf98112a30 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf9fc89740 1 Processor -- start 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf8ea00640 1 -- 172.21.15.46:0/541136128 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf90005020 con 0x7fcf98112a30 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf9fc89740 1 -- start start 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c957f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c9584b0 0x55a42c962830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c962d70 0x55a42c965160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a42c8be3e0 con 0x55a42c9584b0 2024-09-10T07:49:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a42c865d80 con 0x55a42c957b90 2024-09-10T07:49:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.312+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a42c8ba3a0 con 0x55a42c962d70 2024-09-10T07:49:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c9584b0 0x55a42c962830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c9584b0 0x55a42c962830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51142/0 (socket says 172.21.15.46:51142) 2024-09-10T07:49:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf8fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c957f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 -- 172.21.15.46:0/1981086615 learned_addr learned my addr 172.21.15.46:0/1981086615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c962d70 0x55a42c965160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 -- 172.21.15.46:0/1981086615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c962d70 msgr2=0x55a42c965160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c962d70 0x55a42c965160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 -- 172.21.15.46:0/1981086615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 msgr2=0x55a42c957f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c957f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 -- 172.21.15.46:0/1981086615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42c884ae0 con 0x55a42c9584b0 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c962d70 0x55a42c965160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf8ea00640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf7c0a2c00 0x7fcf7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.313+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c9584b0 0x55a42c962830 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009c90 tx=0x7fcf88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb8f34165f439f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.314+0000 7fcf8ea00640 1 -- 172.21.15.46:0/541136128 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcf900cfa70 con 0x7fcf98112a30 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.314+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf7c0a2c00 0x7fcf7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.314+0000 7fcf87400640 1 -- 172.21.15.46:0/1981086615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88004230 con 0x55a42c9584b0 2024-09-10T07:49:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.314+0000 7fcf87400640 1 -- 172.21.15.46:0/1981086615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf880043d0 con 0x55a42c9584b0 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.314+0000 7fcf87400640 1 -- 172.21.15.46:0/1981086615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88019740 con 0x55a42c9584b0 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1981086615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c9584b0 msgr2=0x55a42c962830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c9584b0 0x55a42c962830 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009c90 tx=0x7fcf88011810 comp rx=0 tx=0).stop 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf7c0a2c00 0x7fcf7c0a50c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcf80000c00 tx=0x7fcf80041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1981086615 shutdown_connections 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c962d70 0x55a42c965160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c9584b0 0x55a42c962830 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1981086615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c957f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1981086615 >> 172.21.15.46:0/1981086615 conn(0x55a42c954120 msgr2=0x55a42c954500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1981086615 shutdown_connections 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1981086615 wait complete. 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 -- start start 2024-09-10T07:49:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.315+0000 7fcf9fc89740 1 Processor -- start 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c9601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a42c9659e0 con 0x55a42c962d70 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a42c8790f0 con 0x55a42c957b90 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a42c965b60 con 0x55a42c9584b0 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf9fc89740 10 client.0 _ll_register_callbacks cb 0x55a42c86a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf96800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf96800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51144/0 (socket says 172.21.15.46:51144) 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf96800640 1 -- 172.21.15.46:0/1099795319 learned_addr learned my addr 172.21.15.46:0/1099795319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.316+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf96800640 1 -- 172.21.15.46:0/1099795319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 msgr2=0x55a42c9601b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf96800640 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c9601b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf96800640 1 -- 172.21.15.46:0/1099795319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 msgr2=0x55a42c95fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf96800640 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf96800640 1 -- 172.21.15.46:0/1099795319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42c62f3c0 con 0x55a42c962d70 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf96800640 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9606f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcf900a4630 tx=0x7fcf9009b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef83d773b39c2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf7be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf7be00640 1 -- 172.21.15.46:0/1099795319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf900a48d0 con 0x55a42c962d70 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1099795319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf880043d0 con 0x55a42c962d70 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.317+0000 7fcf7be00640 1 -- 172.21.15.46:0/1099795319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcf900bab30 con 0x55a42c962d70 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf7be00640 1 -- 172.21.15.46:0/1099795319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fcf900a4b60 con 0x55a42c962d70 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf9fc89740 10 client.17883 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1099795319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a42c971700 con 0x55a42c962d70 2024-09-10T07:49:59.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf7be00640 1 -- 172.21.15.46:0/1099795319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf900b7020 con 0x55a42c962d70 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf7be00640 1 -- 172.21.15.46:0/1099795319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fcf90005310 con 0x55a42c962d70 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf9fc89740 10 client.17883 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf9fc89740 10 client.17883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf9fc89740 20 client.17883 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.318+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a42c972bc0 0x55a42c974fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.319+0000 7fcf9fc89740 4 client.17883 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.319+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1099795319 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a42c8cb950 con 0x55a42c972bc0 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.319+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a42c972bc0 0x55a42c974fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.319+0000 7fcf7be00640 10 client.17883 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.319+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a42c972bc0 0x55a42c974fd0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6929f468de68a77 server_cookie=c5d2e7bc9b8fcd65 in_seq=0 out_seq=0 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf7be00640 1 -- 172.21.15.46:0/1099795319 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a42c8cb950 con 0x55a42c972bc0 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf7be00640 10 client.17883 handle_command_reply: tid=0 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 client.17883 shutdown 2024-09-10T07:49:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1099795319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 msgr2=0x55a42c9606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9606f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcf900a4630 tx=0x7fcf9009b910 comp rx=0 tx=0).stop 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1099795319 shutdown_connections 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a42c972bc0 0x55a42c974fd0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9606f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c9601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1099795319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.320+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1099795319 >> 172.21.15.46:0/1099795319 conn(0x55a42c954120 msgr2=0x55a42c959e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.321+0000 7fcf7be00640 0 client.17883 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.321+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1099795319 shutdown_connections 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.321+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1099795319 wait complete. 2024-09-10T07:49:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.321+0000 7fcf9fc89740 20 client.17883 trim_cache size 0 max 16384 2024-09-10T07:49:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.342+0000 7fcf9fc89740 1 Processor -- start 2024-09-10T07:49:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.342+0000 7fcf9fc89740 1 -- start start 2024-09-10T07:49:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c954500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b5200 con 0x55a42c962d70 2024-09-10T07:49:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b6dd0 con 0x55a42c957b90 2024-09-10T07:49:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b58d0 con 0x55a42c9584b0 2024-09-10T07:49:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf95e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf96800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf95e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56812/0 (socket says 172.21.15.46:56812) 2024-09-10T07:49:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf95e00640 1 -- 172.21.15.46:0/1778014275 learned_addr learned my addr 172.21.15.46:0/1778014275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.343+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c954500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf95e00640 1 -- 172.21.15.46:0/1778014275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 msgr2=0x55a42c954500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c954500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf95e00640 1 -- 172.21.15.46:0/1778014275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 msgr2=0x55a42c9530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf95e00640 1 -- 172.21.15.46:0/1778014275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42c95ec20 con 0x55a42c957b90 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf96800640 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95a5f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcf88003f90 tx=0x7fcf880093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a47aa8084f7c9b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf86a00640 1 -- 172.21.15.46:0/1778014275 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8801f040 con 0x55a42c957b90 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf86a00640 1 -- 172.21.15.46:0/1778014275 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf8802a070 con 0x55a42c957b90 2024-09-10T07:49:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.344+0000 7fcf86a00640 1 -- 172.21.15.46:0/1778014275 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88004580 con 0x55a42c957b90 2024-09-10T07:49:59.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1778014275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 msgr2=0x55a42c95a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95a5f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcf88003f90 tx=0x7fcf880093f0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1778014275 shutdown_connections 2024-09-10T07:49:59.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c9530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c954500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1778014275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c95a5f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1778014275 >> 172.21.15.46:0/1778014275 conn(0x55a42c954120 msgr2=0x55a42c9529c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1778014275 shutdown_connections 2024-09-10T07:49:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.345+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1778014275 wait complete. 2024-09-10T07:49:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.346+0000 7fcf9fc89740 1 -- start start 2024-09-10T07:49:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.346+0000 7fcf9fc89740 1 Processor -- start 2024-09-10T07:49:59.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.346+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c966550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.346+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c966a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf9fc89740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c96abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b6700 con 0x55a42c962d70 2024-09-10T07:49:59.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b5fa0 con 0x55a42c957b90 2024-09-10T07:49:59.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf9fc89740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b31e0 con 0x55a42c9584b0 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf9fc89740 10 client.0 _ll_register_callbacks cb 0x7fcf9009bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf95e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c966550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf95e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c966550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56822/0 (socket says 172.21.15.46:56822) 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf95e00640 1 -- 172.21.15.46:0/1659681078 learned_addr learned my addr 172.21.15.46:0/1659681078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf96800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c96abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c966a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf95e00640 1 -- 172.21.15.46:0/1659681078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 msgr2=0x55a42c966a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c966a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf95e00640 1 -- 172.21.15.46:0/1659681078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 msgr2=0x55a42c96abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c96abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.347+0000 7fcf95e00640 1 -- 172.21.15.46:0/1659681078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42c96b2c0 con 0x55a42c957b90 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.348+0000 7fcf95e00640 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c966550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcf88011850 tx=0x7fcf88003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4681a94dbb1ee7d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.348+0000 7fcf7be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:49:59.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.348+0000 7fcf7be00640 1 -- 172.21.15.46:0/1659681078 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88036070 con 0x55a42c957b90 2024-09-10T07:49:59.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.348+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1659681078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf8802a070 con 0x55a42c957b90 2024-09-10T07:49:59.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.348+0000 7fcf7be00640 1 -- 172.21.15.46:0/1659681078 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fcf88005330 con 0x55a42c957b90 2024-09-10T07:49:59.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.348+0000 7fcf7be00640 1 -- 172.21.15.46:0/1659681078 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fcf88042050 con 0x55a42c957b90 2024-09-10T07:49:59.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.348+0000 7fcf9fc89740 10 client.28715 fetch_fsmap learned FSMap version 54 2024-09-10T07:49:59.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.348+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1659681078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a42c9dd420 con 0x55a42c957b90 2024-09-10T07:49:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf7be00640 1 -- 172.21.15.46:0/1659681078 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf8802a070 con 0x55a42c957b90 2024-09-10T07:49:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf7be00640 1 -- 172.21.15.46:0/1659681078 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fcf88018070 con 0x55a42c957b90 2024-09-10T07:49:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf9fc89740 10 client.28715 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf9fc89740 10 client.28715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf9fc89740 20 client.28715 populate_metadata read hostname 'smithi046' 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a42c9dd8a0 0x55a42c9dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf9fc89740 4 client.28715 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1659681078 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a42c8c5e60 con 0x55a42c9dd8a0 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.349+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a42c9dd8a0 0x55a42c9dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf8fe00640 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a42c9dd8a0 0x55a42c9dfc90 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27b00e80908c5ec server_cookie=f449017a78dfe99f in_seq=0 out_seq=0 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf7be00640 10 client.28715 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf7be00640 1 -- 172.21.15.46:0/1659681078 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a42c8c5e60 con 0x55a42c9dd8a0 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf7be00640 10 client.28715 handle_command_reply: tid=0 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf9fc89740 1 client.28715 shutdown 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1659681078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 msgr2=0x55a42c966550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c966550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcf88011850 tx=0x7fcf88003ef0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1659681078 shutdown_connections 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55a42c9dd8a0 0x55a42c9dfc90 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.350+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42c962d70 0x55a42c96abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.351+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a42c9584b0 0x55a42c966a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.351+0000 7fcf9fc89740 1 --2- 172.21.15.46:0/1659681078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a42c957b90 0x55a42c966550 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.351+0000 7fcf7be00640 0 client.28715 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.351+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1659681078 >> 172.21.15.46:0/1659681078 conn(0x55a42c954120 msgr2=0x55a42c9531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.351+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1659681078 shutdown_connections 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.351+0000 7fcf9fc89740 1 -- 172.21.15.46:0/1659681078 wait complete. 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.351+0000 7fcf9fc89740 20 client.28715 trim_cache size 0 max 16384 2024-09-10T07:49:59.352 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout: "4f823651-711e-4891-8594-f2bf5d36b851": { 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "4f823651-711e-4891-8594-f2bf5d36b851", 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:49:59.353 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.351+0000 7fcf9d200640 1 -- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf7c0a2c00 msgr2=0x7fcf7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf7c0a2c00 0x7fcf7c0a50c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcf80000c00 tx=0x7fcf80041040 comp rx=0 tx=0).stop 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 -- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98112a30 msgr2=0x7fcf9819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98112a30 0x7fcf9819e890 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000eb10 tx=0x7fcf9000c2d0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 -- 172.21.15.46:0/541136128 shutdown_connections 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fcf7c0a2c00 0x7fcf7c0a50c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf98112a30 0x7fcf9819e890 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf98077040 0x7fcf981a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 --2- 172.21.15.46:0/541136128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf98076720 0x7fcf981a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 -- 172.21.15.46:0/541136128 >> 172.21.15.46:0/541136128 conn(0x7fcf980ffa90 msgr2=0x7fcf98101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 -- 172.21.15.46:0/541136128 shutdown_connections 2024-09-10T07:49:59.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:49:59.352+0000 7fcf9d200640 1 -- 172.21.15.46:0/541136128 wait complete. 2024-09-10T07:49:59.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:59 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:59 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:49:59.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:59 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:59 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:49:59.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:59 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:49:59.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:59 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:49:59 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:59 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:59 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:49:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:59 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:59 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:49:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:59 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:49:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:59 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:49:59 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:59 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:59 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:49:59.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:59 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:59 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:49:59.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:59 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:49:59.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:59 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:49:59.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:49:59 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:50:00.366 INFO:tasks.cephfs.filesystem:scrub status for tag:4f823651-711e-4891-8594-f2bf5d36b851 - {'path': '/', 'tag': '4f823651-711e-4891-8594-f2bf5d36b851', 'options': 'recursive,force'} 2024-09-10T07:50:00.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 760.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:50:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:00 smithi181 ceph-mon[35397]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-10T07:50:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:00 smithi181 ceph-mon[35397]: overall HEALTH_OK 2024-09-10T07:50:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:00 smithi063 ceph-mon[35020]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-10T07:50:00.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:00 smithi063 ceph-mon[35020]: overall HEALTH_OK 2024-09-10T07:50:00.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:00 smithi046 ceph-mon[30705]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-10T07:50:00.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:00 smithi046 ceph-mon[30705]: overall HEALTH_OK 2024-09-10T07:50:01.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 -- 172.21.15.46:0/4289253935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c4105910 msgr2=0x7fd5c410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 --2- 172.21.15.46:0/4289253935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c4105910 0x7fd5c410c1a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009ca0 tx=0x7fd5c002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 -- 172.21.15.46:0/4289253935 shutdown_connections 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 --2- 172.21.15.46:0/4289253935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c4105910 0x7fd5c410c1a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 --2- 172.21.15.46:0/4289253935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c4104f50 0x7fd5c41053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 --2- 172.21.15.46:0/4289253935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c4103d60 0x7fd5c4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 -- 172.21.15.46:0/4289253935 >> 172.21.15.46:0/4289253935 conn(0x7fd5c40ff4e0 msgr2=0x7fd5c4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 -- 172.21.15.46:0/4289253935 shutdown_connections 2024-09-10T07:50:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.398+0000 7fd5cc800640 1 -- 172.21.15.46:0/4289253935 wait complete. 2024-09-10T07:50:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.400+0000 7fd5cc800640 1 Processor -- start 2024-09-10T07:50:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.400+0000 7fd5cc800640 1 -- start start 2024-09-10T07:50:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.400+0000 7fd5cc800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c4103d60 0x7fd5c4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.400+0000 7fd5cc800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c4104f50 0x7fd5c419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.400+0000 7fd5cc800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c4105910 0x7fd5c41a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.400+0000 7fd5cc800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c410fa30 con 0x7fd5c4104f50 2024-09-10T07:50:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.400+0000 7fd5cc800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c410f8b0 con 0x7fd5c4105910 2024-09-10T07:50:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.400+0000 7fd5cc800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c410fbb0 con 0x7fd5c4103d60 2024-09-10T07:50:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c4104f50 0x7fd5c419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c4103d60 0x7fd5c4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5ca800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c4105910 0x7fd5c41a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c4103d60 0x7fd5c4199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48858/0 (socket says 172.21.15.46:48858) 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 -- 172.21.15.46:0/1046139216 learned_addr learned my addr 172.21.15.46:0/1046139216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 -- 172.21.15.46:0/1046139216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c4105910 msgr2=0x7fd5c41a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c4105910 0x7fd5c41a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 -- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c4104f50 msgr2=0x7fd5c419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c4104f50 0x7fd5c419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 -- 172.21.15.46:0/1046139216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c41a1bb0 con 0x7fd5c4103d60 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.401+0000 7fd5c9e00640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c4103d60 0x7fd5c4199f90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9c0 tx=0x7fd5b800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.402+0000 7fd5bea00640 1 -- 172.21.15.46:0/1046139216 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800cd80 con 0x7fd5c4103d60 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.402+0000 7fd5bea00640 1 -- 172.21.15.46:0/1046139216 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b80035f0 con 0x7fd5c4103d60 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.402+0000 7fd5cc800640 1 -- 172.21.15.46:0/1046139216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c41a1ea0 con 0x7fd5c4103d60 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.402+0000 7fd5bea00640 1 -- 172.21.15.46:0/1046139216 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8010690 con 0x7fd5c4103d60 2024-09-10T07:50:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.402+0000 7fd5cc800640 1 -- 172.21.15.46:0/1046139216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c41a2300 con 0x7fd5c4103d60 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5cf7e8740 1 Processor -- start 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5cf7e8740 1 -- start start 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5bea00640 1 -- 172.21.15.46:0/1046139216 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5b80037a0 con 0x7fd5c4103d60 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f2090 0x5629f87f2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f29d0 0x5629f87f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f87f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629f8757100 con 0x5629f87f2090 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5629f86ff1b0 con 0x5629f87f29d0 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.405+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5629f87530c0 con 0x5629f87f7a60 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f2090 0x5629f87f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f2090 0x5629f87f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51164/0 (socket says 172.21.15.46:51164) 2024-09-10T07:50:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5ca800640 1 -- 172.21.15.46:0/2541706268 learned_addr learned my addr 172.21.15.46:0/2541706268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5c9e00640 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f29d0 0x5629f87f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5c9400640 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f87f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5ca800640 1 -- 172.21.15.46:0/2541706268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 msgr2=0x5629f87f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f87f9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5ca800640 1 -- 172.21.15.46:0/2541706268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f29d0 msgr2=0x5629f87f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f29d0 0x5629f87f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5ca800640 1 -- 172.21.15.46:0/2541706268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f846a210 con 0x5629f87f2090 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.406+0000 7fd5c9e00640 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f29d0 0x5629f87f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.407+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f2090 0x5629f87f2490 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009ca0 tx=0x7fd5c00093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f39a5cec684a595 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.407+0000 7fd5b3000640 1 -- 172.21.15.46:0/2541706268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0031650 con 0x5629f87f2090 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.407+0000 7fd5bea00640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5ac0a2cb0 0x7fd5ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.407+0000 7fd5b3000640 1 -- 172.21.15.46:0/2541706268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c0036030 con 0x5629f87f2090 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.407+0000 7fd5b3000640 1 -- 172.21.15.46:0/2541706268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0004740 con 0x5629f87f2090 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.407+0000 7fd5bea00640 1 -- 172.21.15.46:0/1046139216 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd5b80cf720 con 0x7fd5c4103d60 2024-09-10T07:50:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.407+0000 7fd5c9e00640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5ac0a2cb0 0x7fd5ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2541706268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f2090 msgr2=0x5629f87f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f2090 0x5629f87f2490 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009ca0 tx=0x7fd5c00093c0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2541706268 shutdown_connections 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f87f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f29d0 0x5629f87f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2541706268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f2090 0x5629f87f2490 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5c9e00640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5ac0a2cb0 0x7fd5ac0a5170 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80a42e0 tx=0x7fd5b80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2541706268 >> 172.21.15.46:0/2541706268 conn(0x5629f87ed700 msgr2=0x5629f87edb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2541706268 shutdown_connections 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2541706268 wait complete. 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.408+0000 7fd5cf7e8740 1 -- start start 2024-09-10T07:50:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5cf7e8740 1 Processor -- start 2024-09-10T07:50:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f2090 0x5629f87fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f87fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f88035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629f87fa850 con 0x5629f87f29d0 2024-09-10T07:50:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5629f86e0090 con 0x5629f87f2090 2024-09-10T07:50:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5629f87fa9d0 con 0x5629f87f7a60 2024-09-10T07:50:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5cf7e8740 10 client.0 _ll_register_callbacks cb 0x5629f8734f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f87fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f87fc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51176/0 (socket says 172.21.15.46:51176) 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5c9400640 1 -- 172.21.15.46:0/535559225 learned_addr learned my addr 172.21.15.46:0/535559225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.409+0000 7fd5ca800640 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f2090 0x5629f87fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5ca800640 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f88035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5c9400640 1 -- 172.21.15.46:0/535559225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 msgr2=0x5629f88035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5c9400640 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f88035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5c9400640 1 -- 172.21.15.46:0/535559225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f2090 msgr2=0x5629f87fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5c9400640 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f2090 0x5629f87fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5c9400640 1 -- 172.21.15.46:0/535559225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f8536620 con 0x5629f87f29d0 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5ca800640 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f88035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5c9400640 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f87fc600 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4012720 tx=0x7fd5b4012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91f9bdccfe84207 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5a7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5a7400640 1 -- 172.21.15.46:0/535559225 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b400f920 con 0x5629f87f29d0 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5a7400640 1 -- 172.21.15.46:0/535559225 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd5b400fac0 con 0x5629f87f29d0 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.410+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/535559225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c0036030 con 0x5629f87f29d0 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5a7400640 1 -- 172.21.15.46:0/535559225 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd5b400fd00 con 0x5629f87f29d0 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5cf7e8740 10 client.17901 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/535559225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5629f8806490 con 0x5629f87f29d0 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5a7400640 1 -- 172.21.15.46:0/535559225 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c0036030 con 0x5629f87f29d0 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5a7400640 1 -- 172.21.15.46:0/535559225 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fd5b4010550 con 0x5629f87f29d0 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5cf7e8740 10 client.17901 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5cf7e8740 10 client.17901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5cf7e8740 20 client.17901 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629f8807b10 0x5629f8809f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5cf7e8740 4 client.17901 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.411+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/535559225 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629f8764830 con 0x5629f8807b10 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.412+0000 7fd5ca800640 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629f8807b10 0x5629f8809f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.412+0000 7fd5ca800640 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629f8807b10 0x5629f8809f60 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2a9d26d2bebd98 server_cookie=a9f990dcd74770c7 in_seq=0 out_seq=0 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.412+0000 7fd5a7400640 10 client.17901 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5a7400640 1 -- 172.21.15.46:0/535559225 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629f8764830 con 0x5629f8807b10 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5a7400640 10 client.17901 handle_command_reply: tid=0 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 client.17901 shutdown 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/535559225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 msgr2=0x5629f87fc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f87fc600 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4012720 tx=0x7fd5b4012be0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/535559225 shutdown_connections 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629f8807b10 0x5629f8809f60 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f7a60 0x5629f88035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f87fc600 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5a7400640 0 client.17901 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/535559225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f2090 0x5629f87fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/535559225 >> 172.21.15.46:0/535559225 conn(0x5629f87ed700 msgr2=0x5629f87eef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/535559225 shutdown_connections 2024-09-10T07:50:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/535559225 wait complete. 2024-09-10T07:50:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.413+0000 7fd5cf7e8740 20 client.17901 trim_cache size 0 max 16384 2024-09-10T07:50:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5cf7e8740 1 Processor -- start 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5cf7e8740 1 -- start start 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f87ec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8803590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 0x5629f880e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41ac800 con 0x5629f87f29d0 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41ae3d0 con 0x5629f87f7a60 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41aced0 con 0x5629f87f2090 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.429+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8803590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8803590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51192/0 (socket says 172.21.15.46:51192) 2024-09-10T07:50:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5c9400640 1 -- 172.21.15.46:0/2538696727 learned_addr learned my addr 172.21.15.46:0/2538696727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f87ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 0x5629f880e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5c9400640 1 -- 172.21.15.46:0/2538696727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 msgr2=0x5629f87ec770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5c9400640 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f87ec770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5c9400640 1 -- 172.21.15.46:0/2538696727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 msgr2=0x5629f880e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5c9400640 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 0x5629f880e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5c9400640 1 -- 172.21.15.46:0/2538696727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f87ed520 con 0x5629f87f29d0 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f87ec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.430+0000 7fd5c9400640 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8803590 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4012040 tx=0x7fd5b400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c27e808ef6aa78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5b2600640 1 -- 172.21.15.46:0/2538696727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4010d20 con 0x5629f87f29d0 2024-09-10T07:50:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5b2600640 1 -- 172.21.15.46:0/2538696727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b400f920 con 0x5629f87f29d0 2024-09-10T07:50:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5b2600640 1 -- 172.21.15.46:0/2538696727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b40035f0 con 0x5629f87f29d0 2024-09-10T07:50:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2538696727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 msgr2=0x5629f8803590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8803590 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4012040 tx=0x7fd5b400f810 comp rx=0 tx=0).stop 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2538696727 shutdown_connections 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 0x5629f880e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8803590 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2538696727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f87ec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.431+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2538696727 >> 172.21.15.46:0/2538696727 conn(0x5629f87ed700 msgr2=0x5629f87ec060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.432+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2538696727 shutdown_connections 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.432+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2538696727 wait complete. 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.432+0000 7fd5cf7e8740 1 -- start start 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.432+0000 7fd5cf7e8740 1 Processor -- start 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f8800aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8800fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5cf7e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 0x5629f8801ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41add00 con 0x5629f87f29d0 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41ad5a0 con 0x5629f87f7a60 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5cf7e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41aa7e0 con 0x5629f87f2090 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5cf7e8740 10 client.0 _ll_register_callbacks cb 0x7fd5b4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8800fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8800fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51202/0 (socket says 172.21.15.46:51202) 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5c9400640 1 -- 172.21.15.46:0/2300114026 learned_addr learned my addr 172.21.15.46:0/2300114026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f8800aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 0x5629f8801ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5c9400640 1 -- 172.21.15.46:0/2300114026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 msgr2=0x5629f8800aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5c9400640 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f8800aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5c9400640 1 -- 172.21.15.46:0/2300114026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 msgr2=0x5629f8801ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5c9400640 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 0x5629f8801ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.433+0000 7fd5c9400640 1 -- 172.21.15.46:0/2300114026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f8802600 con 0x5629f87f29d0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.434+0000 7fd5c9400640 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8800fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40126d0 tx=0x7fd5b4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb91daa47b1945f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.434+0000 7fd5a7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.434+0000 7fd5a7400640 1 -- 172.21.15.46:0/2300114026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4010430 con 0x5629f87f29d0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.434+0000 7fd5a7400640 1 -- 172.21.15.46:0/2300114026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd5b40105d0 con 0x5629f87f29d0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.434+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2300114026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b400f920 con 0x5629f87f29d0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.434+0000 7fd5a7400640 1 -- 172.21.15.46:0/2300114026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd5b40035f0 con 0x5629f87f29d0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.434+0000 7fd5cf7e8740 10 client.17907 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.434+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2300114026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5629f8803910 con 0x5629f87f29d0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5a7400640 1 -- 172.21.15.46:0/2300114026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b400f920 con 0x5629f87f29d0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5a7400640 1 -- 172.21.15.46:0/2300114026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fd5b4028030 con 0x5629f87f29d0 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5cf7e8740 10 client.17907 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5cf7e8740 10 client.17907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5cf7e8740 20 client.17907 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629f887b810 0x5629f887dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5cf7e8740 4 client.17907 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:50:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2300114026 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629f875eb80 con 0x5629f887b810 2024-09-10T07:50:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.435+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629f887b810 0x5629f887dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.436+0000 7fd5ca800640 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629f887b810 0x5629f887dc00 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61081a1731dd4b7 server_cookie=2cd394a53a3b1ea in_seq=0 out_seq=0 2024-09-10T07:50:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.436+0000 7fd5a7400640 10 client.17907 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5a7400640 1 -- 172.21.15.46:0/2300114026 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629f875eb80 con 0x5629f887b810 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5a7400640 10 client.17907 handle_command_reply: tid=0 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 client.17907 shutdown 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2300114026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 msgr2=0x5629f8800fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8800fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40126d0 tx=0x7fd5b4009640 comp rx=0 tx=0).stop 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2300114026 shutdown_connections 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5629f887b810 0x5629f887dc00 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629f87f7a60 0x5629f8801ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f87f29d0 0x5629f8800fe0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 --2- 172.21.15.46:0/2300114026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629f87f2090 0x5629f8800aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5a7400640 0 client.17907 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.445+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2300114026 >> 172.21.15.46:0/2300114026 conn(0x5629f87ed700 msgr2=0x5629f87ec880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.446+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2300114026 shutdown_connections 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.446+0000 7fd5cf7e8740 1 -- 172.21.15.46:0/2300114026 wait complete. 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.446+0000 7fd5cf7e8740 20 client.17907 trim_cache size 0 max 16384 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:50:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:50:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.446+0000 7fd5cc800640 1 -- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5ac0a2cb0 msgr2=0x7fd5ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.446+0000 7fd5cc800640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5ac0a2cb0 0x7fd5ac0a5170 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80a42e0 tx=0x7fd5b80d5530 comp rx=0 tx=0).stop 2024-09-10T07:50:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.446+0000 7fd5cc800640 1 -- 172.21.15.46:0/1046139216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c4103d60 msgr2=0x7fd5c4199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.446+0000 7fd5cc800640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c4103d60 0x7fd5c4199f90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9c0 tx=0x7fd5b800ee80 comp rx=0 tx=0).stop 2024-09-10T07:50:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.447+0000 7fd5cc800640 1 -- 172.21.15.46:0/1046139216 shutdown_connections 2024-09-10T07:50:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.447+0000 7fd5cc800640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c4105910 0x7fd5c41a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.447+0000 7fd5cc800640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd5ac0a2cb0 0x7fd5ac0a5170 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.447+0000 7fd5cc800640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c4104f50 0x7fd5c419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.447+0000 7fd5cc800640 1 --2- 172.21.15.46:0/1046139216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c4103d60 0x7fd5c4199f90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.447+0000 7fd5cc800640 1 -- 172.21.15.46:0/1046139216 >> 172.21.15.46:0/1046139216 conn(0x7fd5c40ff4e0 msgr2=0x7fd5c41081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.447+0000 7fd5cc800640 1 -- 172.21.15.46:0/1046139216 shutdown_connections 2024-09-10T07:50:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:01.447+0000 7fd5cc800640 1 -- 172.21.15.46:0/1046139216 wait complete. 2024-09-10T07:50:01.579 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 -- 172.21.15.46:0/1247778145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 msgr2=0x7ffaf81043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 --2- 172.21.15.46:0/1247778145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 0x7ffaf81043e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009900 tx=0x7ffaec02a400 comp rx=0 tx=0).stop 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 -- 172.21.15.46:0/1247778145 shutdown_connections 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 --2- 172.21.15.46:0/1247778145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaf8109130 0x7ffaf8110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 --2- 172.21.15.46:0/1247778145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf81049b0 0x7ffaf8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 --2- 172.21.15.46:0/1247778145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 0x7ffaf81043e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 -- 172.21.15.46:0/1247778145 >> 172.21.15.46:0/1247778145 conn(0x7ffaf80ffa20 msgr2=0x7ffaf8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 -- 172.21.15.46:0/1247778145 shutdown_connections 2024-09-10T07:50:02.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.237+0000 7ffaff400640 1 -- 172.21.15.46:0/1247778145 wait complete. 2024-09-10T07:50:02.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.238+0000 7ffaff400640 1 Processor -- start 2024-09-10T07:50:02.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.238+0000 7ffaff400640 1 -- start start 2024-09-10T07:50:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.238+0000 7ffaff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 0x7ffaf819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.238+0000 7ffaff400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaf81049b0 0x7ffaf819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.238+0000 7ffaff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf8109130 0x7ffaf819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.238+0000 7ffaff400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8114180 con 0x7ffaf8104000 2024-09-10T07:50:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.238+0000 7ffaff400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8114000 con 0x7ffaf81049b0 2024-09-10T07:50:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.238+0000 7ffaff400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8114300 con 0x7ffaf8109130 2024-09-10T07:50:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffafca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 0x7ffaf819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffafca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 0x7ffaf819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51214/0 (socket says 172.21.15.46:51214) 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffafca00640 1 -- 172.21.15.46:0/3330950990 learned_addr learned my addr 172.21.15.46:0/3330950990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffaf7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaf81049b0 0x7ffaf819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffafd400640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf8109130 0x7ffaf819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffaf7e00640 1 -- 172.21.15.46:0/3330950990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf8109130 msgr2=0x7ffaf819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffaf7e00640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf8109130 0x7ffaf819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffaf7e00640 1 -- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 msgr2=0x7ffaf819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffaf7e00640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 0x7ffaf819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffaf7e00640 1 -- 172.21.15.46:0/3330950990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf819f650 con 0x7ffaf81049b0 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffafd400640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf8109130 0x7ffaf819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.239+0000 7ffaf7e00640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaf81049b0 0x7ffaf819abe0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffae400e9f0 tx=0x7ffae400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.240+0000 7ffaf5600640 1 -- 172.21.15.46:0/3330950990 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae400cd80 con 0x7ffaf81049b0 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.240+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf819f940 con 0x7ffaf81049b0 2024-09-10T07:50:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.240+0000 7ffaf5600640 1 -- 172.21.15.46:0/3330950990 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae40035f0 con 0x7ffaf81049b0 2024-09-10T07:50:02.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.240+0000 7ffaf5600640 1 -- 172.21.15.46:0/3330950990 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4010670 con 0x7ffaf81049b0 2024-09-10T07:50:02.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.240+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf81a7220 con 0x7ffaf81049b0 2024-09-10T07:50:02.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.241+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf81043e0 con 0x7ffaf81049b0 2024-09-10T07:50:02.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.243+0000 7ffaf5600640 1 -- 172.21.15.46:0/3330950990 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffae40037a0 con 0x7ffaf81049b0 2024-09-10T07:50:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.245+0000 7ffaf5600640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffae00a2b80 0x7ffae00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.245+0000 7ffaf5600640 1 -- 172.21.15.46:0/3330950990 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ffae40cf840 con 0x7ffaf81049b0 2024-09-10T07:50:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.245+0000 7ffafca00640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffae00a2b80 0x7ffae00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.246+0000 7ffafca00640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffae00a2b80 0x7ffae00a5040 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0096c0 tx=0x7ffaec0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.246+0000 7ffaf5600640 1 -- 172.21.15.46:0/3330950990 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffae409c450 con 0x7ffaf81049b0 2024-09-10T07:50:02.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:02 smithi046 ceph-mon[30705]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 646 KiB/s wr, 113 op/s 2024-09-10T07:50:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.473+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaf819ba00 con 0x7ffaf81049b0 2024-09-10T07:50:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.474+0000 7ffaf5600640 1 -- 172.21.15.46:0/3330950990 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11977 (secure 0 0 0) 0x7ffae409bd50 con 0x7ffaf81049b0 2024-09-10T07:50:02.474 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:50:02.474 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"btime":"2024-09-10T07:49:48:222949+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:49:48.058024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:50:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 54 2024-09-10T07:50:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffae00a2b80 msgr2=0x7ffae00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffae00a2b80 0x7ffae00a5040 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0096c0 tx=0x7ffaec0046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaf81049b0 msgr2=0x7ffaf819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaf81049b0 0x7ffaf819abe0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffae400e9f0 tx=0x7ffae400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 shutdown_connections 2024-09-10T07:50:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf8109130 0x7ffaf819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaf81049b0 0x7ffaf819abe0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ffae00a2b80 0x7ffae00a5040 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 --2- 172.21.15.46:0/3330950990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf8104000 0x7ffaf819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 >> 172.21.15.46:0/3330950990 conn(0x7ffaf80ffa20 msgr2=0x7ffaf8100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 shutdown_connections 2024-09-10T07:50:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:02.477+0000 7ffaff400640 1 -- 172.21.15.46:0/3330950990 wait complete. 2024-09-10T07:50:02.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:02 smithi181 ceph-mon[35397]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 646 KiB/s wr, 113 op/s 2024-09-10T07:50:02.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:02 smithi063 ceph-mon[35020]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 646 KiB/s wr, 113 op/s 2024-09-10T07:50:03.356 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id c6666156 --release --await' 2024-09-10T07:50:03.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id c6666156 --release --await 2024-09-10T07:50:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:03 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3330950990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:03.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:03 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3330950990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:03.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:03 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3330950990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:04.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:04 smithi046 ceph-mon[30705]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 557 KiB/s wr, 164 op/s 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 -- 172.21.15.46:0/565143971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac077040 msgr2=0x7f71ac112350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 --2- 172.21.15.46:0/565143971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac077040 0x7f71ac112350 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7194009900 tx=0x7f719402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 -- 172.21.15.46:0/565143971 shutdown_connections 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 --2- 172.21.15.46:0/565143971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71ac112890 0x7f71ac114c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 --2- 172.21.15.46:0/565143971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac077040 0x7f71ac112350 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 --2- 172.21.15.46:0/565143971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ac076720 0x7f71ac076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 -- 172.21.15.46:0/565143971 >> 172.21.15.46:0/565143971 conn(0x7f71ac0ff8b0 msgr2=0x7f71ac101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 -- 172.21.15.46:0/565143971 shutdown_connections 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.381+0000 7f71ab200640 1 -- 172.21.15.46:0/565143971 wait complete. 2024-09-10T07:50:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.382+0000 7f71ab200640 1 Processor -- start 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71ab200640 1 -- start start 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71ab200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ac076720 0x7f71ac1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71ab200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71ac077040 0x7f71ac1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71ab200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac112890 0x7f71ac19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71ab200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac118540 con 0x7f71ac112890 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71ab200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1183c0 con 0x7f71ac077040 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71ab200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1186c0 con 0x7f71ac076720 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71aa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac112890 0x7f71ac19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71a9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ac076720 0x7f71ac1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71aa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac112890 0x7f71ac19e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51228/0 (socket says 172.21.15.46:51228) 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.383+0000 7f71aa800640 1 -- 172.21.15.46:0/1681103618 learned_addr learned my addr 172.21.15.46:0/1681103618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71a3e00640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71ac077040 0x7f71ac1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71aa800640 1 -- 172.21.15.46:0/1681103618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ac076720 msgr2=0x7f71ac1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71aa800640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ac076720 0x7f71ac1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71aa800640 1 -- 172.21.15.46:0/1681103618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71ac077040 msgr2=0x7f71ac1a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71aa800640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71ac077040 0x7f71ac1a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71aa800640 1 -- 172.21.15.46:0/1681103618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac19f020 con 0x7f71ac112890 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71aa800640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac112890 0x7f71ac19e8f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f71a400dac0 tx=0x7f71a400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71a2a00640 1 -- 172.21.15.46:0/1681103618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4017030 con 0x7f71ac112890 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71a2a00640 1 -- 172.21.15.46:0/1681103618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4003e50 con 0x7f71ac112890 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71ab200640 1 -- 172.21.15.46:0/1681103618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac19f310 con 0x7f71ac112890 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71a2a00640 1 -- 172.21.15.46:0/1681103618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4010460 con 0x7f71ac112890 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.384+0000 7f71ab200640 1 -- 172.21.15.46:0/1681103618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1ab600 con 0x7f71ac112890 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71b35e8740 1 Processor -- start 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71a2a00640 1 -- 172.21.15.46:0/1681103618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71a401a070 con 0x7f71ac112890 2024-09-10T07:50:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71b35e8740 1 -- start start 2024-09-10T07:50:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x56040758a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758ae40 0x5604075951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560407595700 0x560407597af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71b35e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604074f0650 con 0x560407595700 2024-09-10T07:50:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71b35e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560407497f10 con 0x56040758ae40 2024-09-10T07:50:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.388+0000 7f71b35e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5604074ec610 con 0x56040758a520 2024-09-10T07:50:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758ae40 0x5604075951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560407595700 0x560407597af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758ae40 0x5604075951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56910/0 (socket says 172.21.15.46:56910) 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a9e00640 1 -- 172.21.15.46:0/1114143759 learned_addr learned my addr 172.21.15.46:0/1114143759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a3e00640 1 --2- 172.21.15.46:0/1114143759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x56040758a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a9e00640 1 -- 172.21.15.46:0/1114143759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 msgr2=0x56040758a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a9e00640 1 --2- 172.21.15.46:0/1114143759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x56040758a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a9e00640 1 -- 172.21.15.46:0/1114143759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560407595700 msgr2=0x560407597af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a9e00640 1 --2- 172.21.15.46:0/1114143759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560407595700 0x560407597af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a9e00640 1 -- 172.21.15.46:0/1114143759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604071ff580 con 0x56040758ae40 2024-09-10T07:50:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.389+0000 7f71a2a00640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f71900a2b80 0x7f71900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.390+0000 7f71a2a00640 1 -- 172.21.15.46:0/1681103618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f71a40cf1b0 con 0x7f71ac112890 2024-09-10T07:50:04.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.390+0000 7f71a9e00640 1 --2- 172.21.15.46:0/1114143759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758ae40 0x5604075951c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f719c009ca0 tx=0x7f719c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=750edea0723a3c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:04.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.390+0000 7f71a3e00640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f71900a2b80 0x7f71900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.390+0000 7f719b000640 1 -- 172.21.15.46:0/1114143759 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c005220 con 0x56040758ae40 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.390+0000 7f719b000640 1 -- 172.21.15.46:0/1114143759 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719c004c80 con 0x56040758ae40 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.390+0000 7f719b000640 1 -- 172.21.15.46:0/1114143759 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c0197d0 con 0x56040758ae40 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 -- 172.21.15.46:0/1114143759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758ae40 msgr2=0x5604075951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1114143759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758ae40 0x5604075951c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f719c009ca0 tx=0x7f719c011810 comp rx=0 tx=0).stop 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71a3e00640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f71900a2b80 0x7f71900a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f71ac1a00e0 tx=0x7f7194040040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 -- 172.21.15.46:0/1114143759 shutdown_connections 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1114143759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560407595700 0x560407597af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1114143759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758ae40 0x5604075951c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1114143759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x56040758a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 -- 172.21.15.46:0/1114143759 >> 172.21.15.46:0/1114143759 conn(0x560407586ab0 msgr2=0x560407586e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 -- 172.21.15.46:0/1114143759 shutdown_connections 2024-09-10T07:50:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 -- 172.21.15.46:0/1114143759 wait complete. 2024-09-10T07:50:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.391+0000 7f71b35e8740 1 -- start start 2024-09-10T07:50:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71b35e8740 1 Processor -- start 2024-09-10T07:50:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407592540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x560407592a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 0x560407592fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71b35e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604075982d0 con 0x56040758ae40 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71b35e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5604074ab350 con 0x560407595700 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71b35e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560407598450 con 0x56040758a520 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71b35e8740 10 client.0 _ll_register_callbacks cb 0x5604074ce290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x560407592a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x560407592a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51240/0 (socket says 172.21.15.46:51240) 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.392+0000 7f71a3e00640 1 -- 172.21.15.46:0/1410253844 learned_addr learned my addr 172.21.15.46:0/1410253844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71aa800640 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 0x560407592fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71a9e00640 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407592540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71a3e00640 1 -- 172.21.15.46:0/1410253844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 msgr2=0x560407592540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71a3e00640 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407592540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71a3e00640 1 -- 172.21.15.46:0/1410253844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 msgr2=0x560407592fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71a3e00640 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 0x560407592fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71a3e00640 1 -- 172.21.15.46:0/1410253844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604072c9960 con 0x56040758ae40 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71a9e00640 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407592540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71a3e00640 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x560407592a80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7194009900 tx=0x7f719402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22912a9246f3780 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f718f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f718f400640 1 -- 172.21.15.46:0/1410253844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719404f030 con 0x56040758ae40 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.393+0000 7f71b35e8740 1 -- 172.21.15.46:0/1410253844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f719c004c80 con 0x56040758ae40 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f718f400640 1 -- 172.21.15.46:0/1410253844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7194003b40 con 0x56040758ae40 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f718f400640 1 -- 172.21.15.46:0/1410253844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7194045070 con 0x56040758ae40 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f71b35e8740 10 client.17925 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f71b35e8740 1 -- 172.21.15.46:0/1410253844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5604075a3e90 con 0x56040758ae40 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f718f400640 1 -- 172.21.15.46:0/1410253844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719c004c80 con 0x56040758ae40 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f718f400640 1 -- 172.21.15.46:0/1410253844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f719403a040 con 0x56040758ae40 2024-09-10T07:50:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f71b35e8740 10 client.17925 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f71b35e8740 10 client.17925 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f71b35e8740 20 client.17925 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5604075a52a0 0x5604075a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f71b35e8740 4 client.17925 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.394+0000 7f71b35e8740 1 -- 172.21.15.46:0/1410253844 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604074fdbc0 con 0x5604075a52a0 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.395+0000 7f71a9e00640 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5604075a52a0 0x5604075a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.395+0000 7f71a9e00640 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5604075a52a0 0x5604075a7760 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6285c740efd81b1 server_cookie=e8d30e47cd54781f in_seq=0 out_seq=0 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.395+0000 7f718f400640 10 client.17925 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f718f400640 1 -- 172.21.15.46:0/1410253844 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604074fdbc0 con 0x5604075a52a0 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f718f400640 10 client.17925 handle_command_reply: tid=0 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 client.17925 shutdown 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 -- 172.21.15.46:0/1410253844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 msgr2=0x560407592a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x560407592a80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7194009900 tx=0x7f719402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 -- 172.21.15.46:0/1410253844 shutdown_connections 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5604075a52a0 0x5604075a7760 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 0x560407592fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x560407592a80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1410253844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407592540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f718f400640 0 client.17925 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.396+0000 7f71b35e8740 1 -- 172.21.15.46:0/1410253844 >> 172.21.15.46:0/1410253844 conn(0x560407586ab0 msgr2=0x560407585cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.397+0000 7f71b35e8740 1 -- 172.21.15.46:0/1410253844 shutdown_connections 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.397+0000 7f71b35e8740 1 -- 172.21.15.46:0/1410253844 wait complete. 2024-09-10T07:50:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.397+0000 7f71b35e8740 20 client.17925 trim_cache size 0 max 16384 2024-09-10T07:50:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.415+0000 7f71b35e8740 1 Processor -- start 2024-09-10T07:50:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.415+0000 7f71b35e8740 1 -- start start 2024-09-10T07:50:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.415+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758a520 0x560407591480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.415+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x5604075ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.415+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560407595700 0x5604075ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.415+0000 7f71b35e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1b5120 con 0x56040758ae40 2024-09-10T07:50:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.415+0000 7f71b35e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1b6cf0 con 0x56040758a520 2024-09-10T07:50:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.415+0000 7f71b35e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1b57f0 con 0x560407595700 2024-09-10T07:50:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x5604075ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560407595700 0x5604075ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71a9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758a520 0x560407591480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560407595700 0x5604075ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48932/0 (socket says 172.21.15.46:48932) 2024-09-10T07:50:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x5604075ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51246/0 (socket says 172.21.15.46:51246) 2024-09-10T07:50:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 -- 172.21.15.46:0/1371752854 learned_addr learned my addr 172.21.15.46:0/1371752854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 -- 172.21.15.46:0/1371752854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758a520 msgr2=0x560407591480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 --2- 172.21.15.46:0/1371752854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758a520 0x560407591480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 -- 172.21.15.46:0/1371752854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 msgr2=0x5604075ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 --2- 172.21.15.46:0/1371752854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x5604075ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 -- 172.21.15.46:0/1371752854 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604075868e0 con 0x560407595700 2024-09-10T07:50:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.416+0000 7f71aa800640 1 --2- 172.21.15.46:0/1371752854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560407595700 0x5604075ad570 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b6f0 tx=0x7f71a4086190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7c499c9f98a4034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.417+0000 7f719a600640 1 -- 172.21.15.46:0/1371752854 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40b4070 con 0x560407595700 2024-09-10T07:50:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.417+0000 7f719a600640 1 -- 172.21.15.46:0/1371752854 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a409f9e0 con 0x560407595700 2024-09-10T07:50:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.417+0000 7f719a600640 1 -- 172.21.15.46:0/1371752854 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4084bc0 con 0x560407595700 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 -- 172.21.15.46:0/1371752854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560407595700 msgr2=0x5604075ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1371752854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560407595700 0x5604075ad570 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b6f0 tx=0x7f71a4086190 comp rx=0 tx=0).stop 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 -- 172.21.15.46:0/1371752854 shutdown_connections 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1371752854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560407595700 0x5604075ad570 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1371752854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x5604075ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 --2- 172.21.15.46:0/1371752854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040758a520 0x560407591480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 -- 172.21.15.46:0/1371752854 >> 172.21.15.46:0/1371752854 conn(0x560407586ab0 msgr2=0x560407585cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 -- 172.21.15.46:0/1371752854 shutdown_connections 2024-09-10T07:50:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 -- 172.21.15.46:0/1371752854 wait complete. 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.418+0000 7f71b35e8740 1 -- start start 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71b35e8740 1 Processor -- start 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407599bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x56040759a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71b35e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 0x56040760f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71b35e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1b6620 con 0x56040758ae40 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71b35e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1b5ec0 con 0x560407595700 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71b35e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1b3100 con 0x56040758a520 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71b35e8740 10 client.0 _ll_register_callbacks cb 0x7f7194009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x56040759a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71a9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407599bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x56040759a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51252/0 (socket says 172.21.15.46:51252) 2024-09-10T07:50:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71a3e00640 1 -- 172.21.15.46:0/3105970331 learned_addr learned my addr 172.21.15.46:0/3105970331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.419+0000 7f71a9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407599bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48940/0 (socket says 172.21.15.46:48940) 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f71aa800640 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 0x56040760f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f71a9e00640 1 -- 172.21.15.46:0/3105970331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 msgr2=0x56040760f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f71a9e00640 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 0x56040760f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f71a9e00640 1 -- 172.21.15.46:0/3105970331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 msgr2=0x56040759a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f71a9e00640 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x56040759a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f71a9e00640 1 -- 172.21.15.46:0/3105970331 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040760f5f0 con 0x56040758a520 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f71a3e00640 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x56040759a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f71a9e00640 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407599bb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f719c01b040 tx=0x7f719c005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35dedc991d4ec9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.420+0000 7f718f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f718f400640 1 -- 172.21.15.46:0/3105970331 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c030020 con 0x56040758a520 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f71b35e8740 1 -- 172.21.15.46:0/3105970331 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71a409f9e0 con 0x56040758a520 2024-09-10T07:50:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f718f400640 1 -- 172.21.15.46:0/3105970331 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f719c005390 con 0x56040758a520 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f718f400640 1 -- 172.21.15.46:0/3105970331 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f719c03b050 con 0x56040758a520 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f71b35e8740 10 client.28666 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f71b35e8740 1 -- 172.21.15.46:0/3105970331 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5604076108f0 con 0x56040758a520 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f718f400640 1 -- 172.21.15.46:0/3105970331 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a409f9e0 con 0x56040758a520 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f718f400640 1 -- 172.21.15.46:0/3105970331 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f719c02c5a0 con 0x56040758a520 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f71b35e8740 10 client.28666 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f71b35e8740 10 client.28666 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.421+0000 7f71b35e8740 20 client.28666 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.422+0000 7f71b35e8740 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560407610e70 0x560407613260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.422+0000 7f71b35e8740 4 client.28666 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c6666156", "release": true, "await": true}] 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.422+0000 7f71b35e8740 1 -- 172.21.15.46:0/3105970331 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c6666156", "release": true, "await": true}) -- 0x5604074f80d0 con 0x560407610e70 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.422+0000 7f71a3e00640 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560407610e70 0x560407613260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.422+0000 7f71a3e00640 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560407610e70 0x560407613260 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9625c9f8c0acbc server_cookie=b17a3e1f5b22d6d8 in_seq=0 out_seq=0 2024-09-10T07:50:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.422+0000 7f718f400640 10 client.28666 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.455+0000 7f718f400640 1 -- 172.21.15.46:0/3105970331 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5604074f80d0 con 0x560407610e70 2024-09-10T07:50:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.455+0000 7f718f400640 10 client.28666 handle_command_reply: tid=0 2024-09-10T07:50:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.455+0000 7f71b35e8740 1 client.28666 shutdown 2024-09-10T07:50:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.455+0000 7f71b35e8740 1 -- 172.21.15.46:0/3105970331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 msgr2=0x560407599bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.455+0000 7f71b35e8740 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407599bb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f719c01b040 tx=0x7f719c005d90 comp rx=0 tx=0).stop 2024-09-10T07:50:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.456+0000 7f71b35e8740 1 -- 172.21.15.46:0/3105970331 shutdown_connections 2024-09-10T07:50:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.456+0000 7f71b35e8740 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560407610e70 0x560407613260 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.456+0000 7f71b35e8740 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560407595700 0x56040760f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.456+0000 7f71b35e8740 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56040758ae40 0x56040759a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.456+0000 7f71b35e8740 1 --2- 172.21.15.46:0/3105970331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56040758a520 0x560407599bb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.456+0000 7f71b35e8740 1 -- 172.21.15.46:0/3105970331 >> 172.21.15.46:0/3105970331 conn(0x560407586ab0 msgr2=0x560407593bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.456+0000 7f718f400640 0 client.28666 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:04.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.457+0000 7f71b35e8740 1 -- 172.21.15.46:0/3105970331 shutdown_connections 2024-09-10T07:50:04.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.457+0000 7f71b35e8740 1 -- 172.21.15.46:0/3105970331 wait complete. 2024-09-10T07:50:04.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.457+0000 7f71b35e8740 20 client.28666 trim_cache size 0 max 16384 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 54, 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 48, 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "c6666156": { 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 48, 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:50:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 141.5, 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:04.460 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:50:04.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 -- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f71900a2b80 msgr2=0x7f71900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f71900a2b80 0x7f71900a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f71ac1a00e0 tx=0x7f7194040040 comp rx=0 tx=0).stop 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 -- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac112890 msgr2=0x7f71ac19e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac112890 0x7f71ac19e8f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f71a400dac0 tx=0x7f71a400df80 comp rx=0 tx=0).stop 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 -- 172.21.15.46:0/1681103618 shutdown_connections 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f71900a2b80 0x7f71900a5040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ac112890 0x7f71ac19e8f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71ac077040 0x7f71ac1a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 --2- 172.21.15.46:0/1681103618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71ac076720 0x7f71ac1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.459+0000 7f71ab200640 1 -- 172.21.15.46:0/1681103618 >> 172.21.15.46:0/1681103618 conn(0x7f71ac0ff8b0 msgr2=0x7f71ac101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.460+0000 7f71ab200640 1 -- 172.21.15.46:0/1681103618 shutdown_connections 2024-09-10T07:50:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:04.460+0000 7f71ab200640 1 -- 172.21.15.46:0/1681103618 wait complete. 2024-09-10T07:50:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:04 smithi181 ceph-mon[35397]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 557 KiB/s wr, 164 op/s 2024-09-10T07:50:04.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:04 smithi063 ceph-mon[35020]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 557 KiB/s wr, 164 op/s 2024-09-10T07:50:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 765.0 s, 137.7 tps, lat 2541.452 ms stddev 5817.177 2024-09-10T07:50:05.674 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c6666156', total seconds elapsed: 26.0 2024-09-10T07:50:05.675 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.5 2024-09-10T07:50:05.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:50:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:06.608 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:06 smithi046 ceph-mon[30705]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.0 KiB/s wr, 98 op/s 2024-09-10T07:50:06.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:06 smithi181 ceph-mon[35397]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.0 KiB/s wr, 98 op/s 2024-09-10T07:50:06.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:06 smithi063 ceph-mon[35020]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.0 KiB/s wr, 98 op/s 2024-09-10T07:50:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.593+0000 7f3cd3400640 1 -- 172.21.15.46:0/782734997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ccc112a40 msgr2=0x7f3ccc114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.593+0000 7f3cd3400640 1 --2- 172.21.15.46:0/782734997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ccc112a40 0x7f3ccc114e30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009900 tx=0x7f3cc402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.597+0000 7f3cd3400640 1 -- 172.21.15.46:0/782734997 shutdown_connections 2024-09-10T07:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.597+0000 7f3cd3400640 1 --2- 172.21.15.46:0/782734997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ccc112a40 0x7f3ccc114e30 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.597+0000 7f3cd3400640 1 --2- 172.21.15.46:0/782734997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ccc077040 0x7f3ccc112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.597+0000 7f3cd3400640 1 --2- 172.21.15.46:0/782734997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc076720 0x7f3ccc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.597+0000 7f3cd3400640 1 -- 172.21.15.46:0/782734997 >> 172.21.15.46:0/782734997 conn(0x7f3ccc0ffa20 msgr2=0x7f3ccc101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.599+0000 7f3cd3400640 1 -- 172.21.15.46:0/782734997 shutdown_connections 2024-09-10T07:50:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.599+0000 7f3cd3400640 1 -- 172.21.15.46:0/782734997 wait complete. 2024-09-10T07:50:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.600+0000 7f3cd3400640 1 Processor -- start 2024-09-10T07:50:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.600+0000 7f3cd3400640 1 -- start start 2024-09-10T07:50:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.601+0000 7f3cd3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ccc076720 0x7f3ccc073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.601+0000 7f3cd3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc077040 0x7f3ccc073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.601+0000 7f3cd3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ccc112a40 0x7f3ccc06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.601+0000 7f3cd3400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc118640 con 0x7f3ccc077040 2024-09-10T07:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.601+0000 7f3cd3400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc1184c0 con 0x7f3ccc076720 2024-09-10T07:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.601+0000 7f3cd3400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc1187c0 con 0x7f3ccc112a40 2024-09-10T07:50:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.602+0000 7f3cd1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ccc112a40 0x7f3ccc06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.602+0000 7f3cd1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ccc112a40 0x7f3ccc06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48418/0 (socket says 172.21.15.46:48418) 2024-09-10T07:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.602+0000 7f3cd1400640 1 -- 172.21.15.46:0/4105941691 learned_addr learned my addr 172.21.15.46:0/4105941691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.602+0000 7f3cd0a00640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ccc076720 0x7f3ccc073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.603+0000 7f3cd1400640 1 -- 172.21.15.46:0/4105941691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ccc076720 msgr2=0x7f3ccc073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.603+0000 7f3cd1400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ccc076720 0x7f3ccc073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.603+0000 7f3cd1400640 1 -- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc077040 msgr2=0x7f3ccc073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.603+0000 7f3cd1400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc077040 0x7f3ccc073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.603+0000 7f3cd1400640 1 -- 172.21.15.46:0/4105941691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc06d3b0 con 0x7f3ccc112a40 2024-09-10T07:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.604+0000 7f3cd1400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ccc112a40 0x7f3ccc06cb80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009230 tx=0x7f3cc4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.604+0000 7f3cc9600640 1 -- 172.21.15.46:0/4105941691 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc403f070 con 0x7f3ccc112a40 2024-09-10T07:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.605+0000 7f3cc9600640 1 -- 172.21.15.46:0/4105941691 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc4031c90 con 0x7f3ccc112a40 2024-09-10T07:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.605+0000 7f3cc9600640 1 -- 172.21.15.46:0/4105941691 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4035030 con 0x7f3ccc112a40 2024-09-10T07:50:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.605+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ccc06d640 con 0x7f3ccc112a40 2024-09-10T07:50:07.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.605+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ccc071610 con 0x7f3ccc112a40 2024-09-10T07:50:07.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.609+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ccc077e60 con 0x7f3ccc112a40 2024-09-10T07:50:07.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.610+0000 7f3cc9600640 1 -- 172.21.15.46:0/4105941691 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3cc4042070 con 0x7f3ccc112a40 2024-09-10T07:50:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.612+0000 7f3cc9600640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3cb00a2c00 0x7f3cb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.612+0000 7f3cc9600640 1 -- 172.21.15.46:0/4105941691 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3cc40b9e00 con 0x7f3ccc112a40 2024-09-10T07:50:07.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.613+0000 7f3cd0a00640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3cb00a2c00 0x7f3cb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:07.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.616+0000 7f3cd0a00640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3cb00a2c00 0x7f3cb00a50c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009f20 tx=0x7f3cbc0094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:07.619+0000 7f3cc9600640 1 -- 172.21.15.46:0/4105941691 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3cc40bc460 con 0x7f3ccc112a40 2024-09-10T07:50:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.021+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ccc0718c0 con 0x7f3ccc112a40 2024-09-10T07:50:08.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.022+0000 7f3cc9600640 1 -- 172.21.15.46:0/4105941691 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11977 (secure 0 0 0) 0x7f3cc40c0110 con 0x7f3ccc112a40 2024-09-10T07:50:08.022 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:50:08.022 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"btime":"2024-09-10T07:49:48:222949+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:49:48.058024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:50:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 54 2024-09-10T07:50:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3cb00a2c00 msgr2=0x7f3cb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3cb00a2c00 0x7f3cb00a50c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009f20 tx=0x7f3cbc0094f0 comp rx=0 tx=0).stop 2024-09-10T07:50:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ccc112a40 msgr2=0x7f3ccc06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ccc112a40 0x7f3ccc06cb80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009230 tx=0x7f3cc4003b90 comp rx=0 tx=0).stop 2024-09-10T07:50:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 shutdown_connections 2024-09-10T07:50:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ccc112a40 0x7f3ccc06cb80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3cb00a2c00 0x7f3cb00a50c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc077040 0x7f3ccc073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 --2- 172.21.15.46:0/4105941691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ccc076720 0x7f3ccc073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.026+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 >> 172.21.15.46:0/4105941691 conn(0x7f3ccc0ffa20 msgr2=0x7f3ccc101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.027+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 shutdown_connections 2024-09-10T07:50:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:08.027+0000 7f3cd3400640 1 -- 172.21.15.46:0/4105941691 wait complete. 2024-09-10T07:50:08.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:08 smithi046 ceph-mon[30705]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 438 KiB/s wr, 145 op/s 2024-09-10T07:50:08.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:08 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/4105941691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:08 smithi181 ceph-mon[35397]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 438 KiB/s wr, 145 op/s 2024-09-10T07:50:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:08 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/4105941691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:08.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:08 smithi063 ceph-mon[35020]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 438 KiB/s wr, 145 op/s 2024-09-10T07:50:08.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:08 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/4105941691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:09.471 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0' 2024-09-10T07:50:09.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0 2024-09-10T07:50:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:10.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 770.0 s, 449.4 tps, lat 220.982 ms stddev 254.687 2024-09-10T07:50:10.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:10 smithi181 ceph-mon[35397]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 683 KiB/s wr, 181 op/s 2024-09-10T07:50:10.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:10 smithi063 ceph-mon[35020]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 683 KiB/s wr, 181 op/s 2024-09-10T07:50:10.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:10 smithi046 ceph-mon[30705]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 683 KiB/s wr, 181 op/s 2024-09-10T07:50:11.617 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:11 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:50:11.617 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:11 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:50:11.618 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:11 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:50:11.618 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:11 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:50:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:11 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:50:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:11 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:50:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:11 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:50:11.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:11 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:50:11.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:11 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:50:11.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:11 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:50:11.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:11 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:50:11.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:11 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.616+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3969663696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b4c096480 msgr2=0x7f5b4c004d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.616+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3969663696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b4c096480 0x7f5b4c004d70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5b54067960 tx=0x7f5b54096dd0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.617+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3969663696 shutdown_connections 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.617+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3969663696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b4c098d40 0x7f5b4c0991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.617+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3969663696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b4c0968f0 0x7f5b4c0052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.617+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3969663696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b4c096480 0x7f5b4c004d70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.617+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3969663696 >> 172.21.15.46:0/3969663696 conn(0x7f5b4c00a1f0 msgr2=0x7f5b4c00a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.617+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3969663696 shutdown_connections 2024-09-10T07:50:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.617+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3969663696 wait complete. 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.618+0000 7f5b5ac00640 1 Processor -- start 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.618+0000 7f5b5ac00640 1 -- start start 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.618+0000 7f5b5ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b4c096480 0x7f5b4c12f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.618+0000 7f5b5ac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b4c0968f0 0x7f5b4c12fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.618+0000 7f5b5ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b4c098d40 0x7f5b4c129a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.618+0000 7f5b5ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a1720 con 0x7f5b4c096480 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.618+0000 7f5b5ac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a15a0 con 0x7f5b4c0968f0 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.618+0000 7f5b5ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a18a0 con 0x7f5b4c098d40 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b58e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b4c0968f0 0x7f5b4c12fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b58e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b4c0968f0 0x7f5b4c12fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34144/0 (socket says 172.21.15.46:34144) 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b58e00640 1 -- 172.21.15.46:0/3769242832 learned_addr learned my addr 172.21.15.46:0/3769242832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:11.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b5a200640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b4c098d40 0x7f5b4c129a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b59800640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b4c096480 0x7f5b4c12f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b58e00640 1 -- 172.21.15.46:0/3769242832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b4c098d40 msgr2=0x7f5b4c129a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b58e00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b4c098d40 0x7f5b4c129a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b58e00640 1 -- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b4c096480 msgr2=0x7f5b4c12f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b58e00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b4c096480 0x7f5b4c12f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b58e00640 1 -- 172.21.15.46:0/3769242832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c12a1c0 con 0x7f5b4c0968f0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.619+0000 7f5b59800640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b4c096480 0x7f5b4c12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.620+0000 7f5b58e00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b4c0968f0 0x7f5b4c12fe10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5b4001fb40 tx=0x7f5b4001d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.620+0000 7f5b52000640 1 -- 172.21.15.46:0/3769242832 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4001dd20 con 0x7f5b4c0968f0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.620+0000 7f5b52000640 1 -- 172.21.15.46:0/3769242832 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40021920 con 0x7f5b4c0968f0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.620+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3769242832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c12a4b0 con 0x7f5b4c0968f0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.620+0000 7f5b52000640 1 -- 172.21.15.46:0/3769242832 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40029990 con 0x7f5b4c0968f0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.620+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3769242832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1367a0 con 0x7f5b4c0968f0 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.629+0000 7f5b5ec6c740 1 Processor -- start 2024-09-10T07:50:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.629+0000 7f5b5ec6c740 1 -- start start 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.629+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d94940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b52000640 1 -- 172.21.15.46:0/3769242832 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b40021ad0 con 0x7f5b4c0968f0 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839d9f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9f740 0x55c839da1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c839cfa530 con 0x55c839d94560 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c839ca1860 con 0x55c839d94e80 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c839cf64f0 con 0x55c839d9f740 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b5a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d94940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b5a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d94940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51556/0 (socket says 172.21.15.46:51556) 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b5a200640 1 -- 172.21.15.46:0/4260828887 learned_addr learned my addr 172.21.15.46:0/4260828887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b58e00640 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9f740 0x55c839da1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.630+0000 7f5b59800640 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839d9f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.631+0000 7f5b58e00640 1 -- 172.21.15.46:0/4260828887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 msgr2=0x55c839d9f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.631+0000 7f5b58e00640 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839d9f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.631+0000 7f5b58e00640 1 -- 172.21.15.46:0/4260828887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 msgr2=0x55c839d94940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.631+0000 7f5b58e00640 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d94940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.631+0000 7f5b58e00640 1 -- 172.21.15.46:0/4260828887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c839a6bb90 con 0x55c839d9f740 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.631+0000 7f5b58e00640 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9f740 0x55c839da1b30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5b400ddf20 tx=0x7f5b400e8670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f845b5f373af73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.632+0000 7f5b46a00640 1 -- 172.21.15.46:0/4260828887 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400ef070 con 0x55c839d9f740 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.632+0000 7f5b52000640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5b3c0a2c90 0x7f5b3c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.632+0000 7f5b52000640 1 -- 172.21.15.46:0/3769242832 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5b400e0860 con 0x7f5b4c0968f0 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.632+0000 7f5b46a00640 1 -- 172.21.15.46:0/4260828887 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b400e27b0 con 0x55c839d9f740 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.632+0000 7f5b46a00640 1 -- 172.21.15.46:0/4260828887 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400ebb30 con 0x55c839d9f740 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.632+0000 7f5b5a200640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5b3c0a2c90 0x7f5b3c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.633+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/4260828887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9f740 msgr2=0x55c839da1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.633+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9f740 0x55c839da1b30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5b400ddf20 tx=0x7f5b400e8670 comp rx=0 tx=0).stop 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.633+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/4260828887 shutdown_connections 2024-09-10T07:50:11.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.633+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9f740 0x55c839da1b30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.633+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839d9f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.633+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/4260828887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d94940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.633+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/4260828887 >> 172.21.15.46:0/4260828887 conn(0x55c839d90af0 msgr2=0x55c839d90ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.633+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/4260828887 shutdown_connections 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.634+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/4260828887 wait complete. 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.634+0000 7f5b5a200640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5b3c0a2c90 0x7f5b3c0a5150 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5b48006aa0 tx=0x7f5b480090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.634+0000 7f5b5ec6c740 1 -- start start 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.634+0000 7f5b5ec6c740 1 Processor -- start 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94560 0x55c839d9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d94e80 0x55c839d9ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d9d1f0 0x55c839dab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c839da2310 con 0x55c839d9d1f0 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c839cb4e60 con 0x55c839d94560 2024-09-10T07:50:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c839da2490 con 0x55c839d94e80 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b5ec6c740 10 client.0 _ll_register_callbacks cb 0x55c839cb9cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b58e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d94e80 0x55c839d9ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b59800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94560 0x55c839d9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b59800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94560 0x55c839d9c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34170/0 (socket says 172.21.15.46:34170) 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.635+0000 7f5b59800640 1 -- 172.21.15.46:0/3467447418 learned_addr learned my addr 172.21.15.46:0/3467447418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.636+0000 7f5b5a200640 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d9d1f0 0x55c839dab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.636+0000 7f5b58e00640 1 -- 172.21.15.46:0/3467447418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94560 msgr2=0x55c839d9c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.636+0000 7f5b58e00640 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94560 0x55c839d9c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.636+0000 7f5b58e00640 1 -- 172.21.15.46:0/3467447418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d9d1f0 msgr2=0x55c839dab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.636+0000 7f5b58e00640 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d9d1f0 0x55c839dab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.636+0000 7f5b58e00640 1 -- 172.21.15.46:0/3467447418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c839cc0990 con 0x55c839d94e80 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.637+0000 7f5b59800640 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94560 0x55c839d9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.637+0000 7f5b58e00640 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d94e80 0x55c839d9ccb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55c839d91ba0 tx=0x7f5b400eb770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317b38285b5bab43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.637+0000 7f5b3b400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.638+0000 7f5b3b400640 1 -- 172.21.15.46:0/3467447418 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400ef070 con 0x55c839d94e80 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.638+0000 7f5b3b400640 1 -- 172.21.15.46:0/3467447418 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5b400a9350 con 0x55c839d94e80 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.638+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/3467447418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b400e27b0 con 0x55c839d94e80 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.638+0000 7f5b3b400640 1 -- 172.21.15.46:0/3467447418 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5b400a8f80 con 0x55c839d94e80 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.638+0000 7f5b5ec6c740 10 client.28681 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.638+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/3467447418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c839da2310 con 0x55c839d94e80 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.639+0000 7f5b3b400640 1 -- 172.21.15.46:0/3467447418 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b400e27b0 con 0x55c839d94e80 2024-09-10T07:50:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.639+0000 7f5b3b400640 1 -- 172.21.15.46:0/3467447418 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f5b400acf40 con 0x55c839d94e80 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.639+0000 7f5b5ec6c740 10 client.28681 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.639+0000 7f5b5ec6c740 10 client.28681 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.639+0000 7f5b5ec6c740 20 client.28681 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.639+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c839daf6e0 0x55c839db1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.640+0000 7f5b5ec6c740 4 client.28681 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.640+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/3467447418 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c839c9d310 con 0x55c839daf6e0 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.640+0000 7f5b59800640 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c839daf6e0 0x55c839db1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.641+0000 7f5b59800640 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c839daf6e0 0x55c839db1ad0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d6967f350cd183 server_cookie=13d96120798a0ae8 in_seq=0 out_seq=0 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.641+0000 7f5b3b400640 10 client.28681 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.642+0000 7f5b3b400640 1 -- 172.21.15.46:0/3467447418 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c839c9d310 con 0x55c839daf6e0 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.642+0000 7f5b3b400640 10 client.28681 handle_command_reply: tid=0 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.642+0000 7f5b5ec6c740 1 client.28681 shutdown 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.642+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/3467447418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d94e80 msgr2=0x55c839d9ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.642+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d94e80 0x55c839d9ccb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55c839d91ba0 tx=0x7f5b400eb770 comp rx=0 tx=0).stop 2024-09-10T07:50:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.642+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/3467447418 shutdown_connections 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.642+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c839daf6e0 0x55c839db1ad0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.643+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d9d1f0 0x55c839dab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.643+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d94e80 0x55c839d9ccb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.643+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/3467447418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94560 0x55c839d9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.643+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/3467447418 >> 172.21.15.46:0/3467447418 conn(0x55c839d90af0 msgr2=0x55c839d96850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.643+0000 7f5b3b400640 0 client.28681 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.643+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/3467447418 shutdown_connections 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.643+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/3467447418 wait complete. 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.643+0000 7f5b5ec6c740 20 client.28681 trim_cache size 0 max 16384 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.671+0000 7f5b5ec6c740 1 Processor -- start 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.672+0000 7f5b5ec6c740 1 -- start start 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.672+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d9b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.672+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839dab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.672+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 0x55c839db71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.672+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1405d0 con 0x55c839d94560 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.672+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1421a0 con 0x55c839d94e80 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.672+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c140ca0 con 0x55c839d9d1f0 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.673+0000 7f5b59800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d9b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.673+0000 7f5b59800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d9b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51578/0 (socket says 172.21.15.46:51578) 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.673+0000 7f5b59800640 1 -- 172.21.15.46:0/2063764887 learned_addr learned my addr 172.21.15.46:0/2063764887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.674+0000 7f5b59800640 1 -- 172.21.15.46:0/2063764887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 msgr2=0x55c839db71f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.674+0000 7f5b59800640 1 --2- 172.21.15.46:0/2063764887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 0x55c839db71f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.674+0000 7f5b59800640 1 -- 172.21.15.46:0/2063764887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 msgr2=0x55c839dab5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.674+0000 7f5b58e00640 1 --2- 172.21.15.46:0/2063764887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839dab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.674+0000 7f5b59800640 1 --2- 172.21.15.46:0/2063764887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839dab5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.674+0000 7f5b59800640 1 -- 172.21.15.46:0/2063764887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c839d94cb0 con 0x55c839d94560 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.675+0000 7f5b59800640 1 --2- 172.21.15.46:0/2063764887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d9b6b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5b54065360 tx=0x7f5b54099040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1e4b5e4155a0e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.675+0000 7f5b46000640 1 -- 172.21.15.46:0/2063764887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540b8070 con 0x55c839d94560 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.676+0000 7f5b46000640 1 -- 172.21.15.46:0/2063764887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5409b040 con 0x55c839d94560 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.676+0000 7f5b46000640 1 -- 172.21.15.46:0/2063764887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54062a50 con 0x55c839d94560 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.678+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/2063764887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 msgr2=0x55c839d9b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.678+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/2063764887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d9b6b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5b54065360 tx=0x7f5b54099040 comp rx=0 tx=0).stop 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.680+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/2063764887 shutdown_connections 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.680+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/2063764887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 0x55c839db71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.680+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/2063764887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839dab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.680+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/2063764887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839d9b6b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.680+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/2063764887 >> 172.21.15.46:0/2063764887 conn(0x55c839d90af0 msgr2=0x55c839d8fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.680+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/2063764887 shutdown_connections 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.680+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/2063764887 wait complete. 2024-09-10T07:50:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.680+0000 7f5b5ec6c740 1 -- start start 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.681+0000 7f5b5ec6c740 1 Processor -- start 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.682+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839da7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.682+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839da3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.682+0000 7f5b5ec6c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 0x55c839da3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.682+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c141ad0 con 0x55c839d94560 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.682+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c141370 con 0x55c839d94e80 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.682+0000 7f5b5ec6c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c13e5b0 con 0x55c839d9d1f0 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.683+0000 7f5b5ec6c740 10 client.0 _ll_register_callbacks cb 0x7f5b48009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.683+0000 7f5b59800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839da7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.683+0000 7f5b59800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839da7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51592/0 (socket says 172.21.15.46:51592) 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.683+0000 7f5b59800640 1 -- 172.21.15.46:0/1809897958 learned_addr learned my addr 172.21.15.46:0/1809897958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.684+0000 7f5b5a200640 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 0x55c839da3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.684+0000 7f5b59800640 1 -- 172.21.15.46:0/1809897958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 msgr2=0x55c839da3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.685+0000 7f5b59800640 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 0x55c839da3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.685+0000 7f5b59800640 1 -- 172.21.15.46:0/1809897958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 msgr2=0x55c839da3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.685+0000 7f5b59800640 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839da3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.685+0000 7f5b59800640 1 -- 172.21.15.46:0/1809897958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c839da4050 con 0x55c839d94560 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.685+0000 7f5b5a200640 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 0x55c839da3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.686+0000 7f5b3b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.686+0000 7f5b59800640 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839da7f50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5b54051790 tx=0x7f5b540626c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d830a49164f45b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.687+0000 7f5b3b400640 1 -- 172.21.15.46:0/1809897958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540c3060 con 0x55c839d94560 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.687+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/1809897958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b5409b040 con 0x55c839d94560 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.688+0000 7f5b3b400640 1 -- 172.21.15.46:0/1809897958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5b54063250 con 0x55c839d94560 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.688+0000 7f5b3b400640 1 -- 172.21.15.46:0/1809897958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5b540c3060 con 0x55c839d94560 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.688+0000 7f5b5ec6c740 10 client.17946 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.688+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/1809897958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c839e1a270 con 0x55c839d94560 2024-09-10T07:50:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.703+0000 7f5b3b400640 1 -- 172.21.15.46:0/1809897958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b54063450 con 0x55c839d94560 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.703+0000 7f5b3b400640 1 -- 172.21.15.46:0/1809897958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f5b540be470 con 0x55c839d94560 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.703+0000 7f5b5ec6c740 10 client.17946 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.703+0000 7f5b5ec6c740 10 client.17946 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.703+0000 7f5b5ec6c740 20 client.17946 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.703+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c839e1a8b0 0x55c839e1cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.703+0000 7f5b5ec6c740 4 client.17946 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}] 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.703+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/1809897958 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}) -- 0x7f5b5409b040 con 0x55c839e1a8b0 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.704+0000 7f5b58e00640 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c839e1a8b0 0x55c839e1cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.705+0000 7f5b58e00640 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c839e1a8b0 0x55c839e1cca0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=563c6845d6d3554a server_cookie=9eb61b029078c636 in_seq=0 out_seq=0 2024-09-10T07:50:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.705+0000 7f5b3b400640 10 client.17946 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.953+0000 7f5b3b400640 1 -- 172.21.15.46:0/1809897958 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f5b5409b040 con 0x55c839e1a8b0 2024-09-10T07:50:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.953+0000 7f5b3b400640 10 client.17946 handle_command_reply: tid=0 2024-09-10T07:50:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.953+0000 7f5b5ec6c740 1 client.17946 shutdown 2024-09-10T07:50:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/1809897958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 msgr2=0x55c839da7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839da7f50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5b54051790 tx=0x7f5b540626c0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/1809897958 shutdown_connections 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55c839e1a8b0 0x55c839e1cca0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c839d9d1f0 0x55c839da3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c839d94e80 0x55c839da3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 --2- 172.21.15.46:0/1809897958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c839d94560 0x55c839da7f50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b3b400640 0 client.17946 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/1809897958 >> 172.21.15.46:0/1809897958 conn(0x55c839d90af0 msgr2=0x55c839d95fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/1809897958 shutdown_connections 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 1 -- 172.21.15.46:0/1809897958 wait complete. 2024-09-10T07:50:11.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.954+0000 7f5b5ec6c740 20 client.17946 trim_cache size 0 max 16384 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 54, 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 50, 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "ec6517": { 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 50, 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:50:11.955 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 141.5, 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:11.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:11.957 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:50:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5b3c0a2c90 msgr2=0x7f5b3c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5b3c0a2c90 0x7f5b3c0a5150 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5b48006aa0 tx=0x7f5b480090e0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3769242832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b4c0968f0 msgr2=0x7f5b4c12fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b4c0968f0 0x7f5b4c12fe10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5b4001fb40 tx=0x7f5b4001d2e0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3769242832 shutdown_connections 2024-09-10T07:50:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b4c098d40 0x7f5b4c129a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b4c0968f0 0x7f5b4c12fe10 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5b3c0a2c90 0x7f5b3c0a5150 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 --2- 172.21.15.46:0/3769242832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b4c096480 0x7f5b4c12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3769242832 >> 172.21.15.46:0/3769242832 conn(0x7f5b4c00a1f0 msgr2=0x7f5b4c093920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3769242832 shutdown_connections 2024-09-10T07:50:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:11.955+0000 7f5b5ac00640 1 -- 172.21.15.46:0/3769242832 wait complete. 2024-09-10T07:50:12.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:12 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:50:12.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:12 smithi181 ceph-mon[35397]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-09-10T07:50:12.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:12 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:50:12.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:12 smithi063 ceph-mon[35020]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-09-10T07:50:12.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:12 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:50:12.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:12 smithi046 ceph-mon[30705]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-09-10T07:50:12.886 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ec6517', quiesce took 7.2 seconds. Will release after: 14.3 2024-09-10T07:50:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.581+0000 7f9f28c00640 1 -- 172.21.15.46:0/2380762442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24104cf0 msgr2=0x7f9f241070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.581+0000 7f9f28c00640 1 --2- 172.21.15.46:0/2380762442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24104cf0 0x7f9f241070e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009860 tx=0x7f9f1002a340 comp rx=0 tx=0).stop 2024-09-10T07:50:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.582+0000 7f9f28c00640 1 -- 172.21.15.46:0/2380762442 shutdown_connections 2024-09-10T07:50:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.582+0000 7f9f28c00640 1 --2- 172.21.15.46:0/2380762442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f24107620 0x7f9f24109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.582+0000 7f9f28c00640 1 --2- 172.21.15.46:0/2380762442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24104cf0 0x7f9f241070e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.582+0000 7f9f28c00640 1 --2- 172.21.15.46:0/2380762442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f241023c0 0x7f9f241047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.582+0000 7f9f28c00640 1 -- 172.21.15.46:0/2380762442 >> 172.21.15.46:0/2380762442 conn(0x7f9f240fbfa0 msgr2=0x7f9f240fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.582+0000 7f9f28c00640 1 -- 172.21.15.46:0/2380762442 shutdown_connections 2024-09-10T07:50:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.582+0000 7f9f28c00640 1 -- 172.21.15.46:0/2380762442 wait complete. 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f28c00640 1 Processor -- start 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f28c00640 1 -- start start 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f28c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f241023c0 0x7f9f24199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f28c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f24104cf0 0x7f9f2419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f28c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24107620 0x7f9f241a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f28c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410d600 con 0x7f9f24107620 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f28c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410d480 con 0x7f9f241023c0 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f28c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410d780 con 0x7f9f24104cf0 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24107620 0x7f9f241a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f24104cf0 0x7f9f2419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f24104cf0 0x7f9f2419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48474/0 (socket says 172.21.15.46:48474) 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f22a00640 1 -- 172.21.15.46:0/922599976 learned_addr learned my addr 172.21.15.46:0/922599976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24107620 0x7f9f241a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51612/0 (socket says 172.21.15.46:51612) 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.583+0000 7f9f23400640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f241023c0 0x7f9f24199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f22a00640 1 -- 172.21.15.46:0/922599976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f241023c0 msgr2=0x7f9f24199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f22a00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f241023c0 0x7f9f24199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f22a00640 1 -- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24107620 msgr2=0x7f9f241a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f22a00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24107620 0x7f9f241a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f22a00640 1 -- 172.21.15.46:0/922599976 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f241a1b40 con 0x7f9f24104cf0 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f22a00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f24104cf0 0x7f9f2419a460 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9f100305b0 tx=0x7f9f10033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f1be00640 1 -- 172.21.15.46:0/922599976 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1003f020 con 0x7f9f24104cf0 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f28c00640 1 -- 172.21.15.46:0/922599976 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241a1dd0 con 0x7f9f24104cf0 2024-09-10T07:50:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f1be00640 1 -- 172.21.15.46:0/922599976 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f10030850 con 0x7f9f24104cf0 2024-09-10T07:50:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.584+0000 7f9f1be00640 1 -- 172.21.15.46:0/922599976 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10037040 con 0x7f9f24104cf0 2024-09-10T07:50:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.585+0000 7f9f28c00640 1 -- 172.21.15.46:0/922599976 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a2260 con 0x7f9f24104cf0 2024-09-10T07:50:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.587+0000 7f9f2d212740 1 Processor -- start 2024-09-10T07:50:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f2d212740 1 -- start start 2024-09-10T07:50:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f1be00640 1 -- 172.21.15.46:0/922599976 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f10042020 con 0x7f9f24104cf0 2024-09-10T07:50:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce537660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce53c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 0x5632ce53f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f2d212740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632ce49c100 con 0x5632ce537ba0 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f2d212740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5632ce4441b0 con 0x5632ce53cc30 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f2d212740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5632ce4980c0 con 0x5632ce537260 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f23400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce53c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce537660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 0x5632ce53f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce537660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48482/0 (socket says 172.21.15.46:48482) 2024-09-10T07:50:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.588+0000 7f9f23e00640 1 -- 172.21.15.46:0/3249681058 learned_addr learned my addr 172.21.15.46:0/3249681058 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f23e00640 1 -- 172.21.15.46:0/3249681058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 msgr2=0x5632ce53f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f23e00640 1 --2- 172.21.15.46:0/3249681058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 0x5632ce53f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f23e00640 1 -- 172.21.15.46:0/3249681058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 msgr2=0x5632ce53c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f23e00640 1 --2- 172.21.15.46:0/3249681058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce53c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f23e00640 1 -- 172.21.15.46:0/3249681058 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ce1af210 con 0x5632ce537260 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f23e00640 1 --2- 172.21.15.46:0/3249681058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce537660 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00e4d0 tx=0x7f9f1c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4749d76364ca295 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f0fe00640 1 -- 172.21.15.46:0/3249681058 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c003970 con 0x5632ce537260 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f0fe00640 1 -- 172.21.15.46:0/3249681058 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1c00d770 con 0x5632ce537260 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f0fe00640 1 -- 172.21.15.46:0/3249681058 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c004480 con 0x5632ce537260 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.589+0000 7f9f1be00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f080a2b80 0x7f9f080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f1be00640 1 -- 172.21.15.46:0/922599976 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9f100ef770 con 0x7f9f24104cf0 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f23400640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f080a2b80 0x7f9f080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 -- 172.21.15.46:0/3249681058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 msgr2=0x5632ce537660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3249681058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce537660 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00e4d0 tx=0x7f9f1c0095c0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 -- 172.21.15.46:0/3249681058 shutdown_connections 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3249681058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 0x5632ce53f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3249681058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce53c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3249681058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce537660 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 -- 172.21.15.46:0/3249681058 >> 172.21.15.46:0/3249681058 conn(0x5632ce5328d0 msgr2=0x5632ce532cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 -- 172.21.15.46:0/3249681058 shutdown_connections 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f2d212740 1 -- 172.21.15.46:0/3249681058 wait complete. 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.590+0000 7f9f23400640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f080a2b80 0x7f9f080a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x5632ce5385d0 tx=0x7f9f14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.591+0000 7f9f2d212740 1 -- start start 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.591+0000 7f9f2d212740 1 Processor -- start 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce5413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce5418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce53cc30 0x5632ce548930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f2d212740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632ce53fc00 con 0x5632ce53cc30 2024-09-10T07:50:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f2d212740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5632ce425090 con 0x5632ce537ba0 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f2d212740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5632ce53fd80 con 0x5632ce537260 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f2d212740 10 client.0 _ll_register_callbacks cb 0x5632ce479f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce53cc30 0x5632ce548930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce5418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f23400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce5413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce5418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:34224/0 (socket says 172.21.15.46:34224) 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f23400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce5413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48484/0 (socket says 172.21.15.46:48484) 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f22a00640 1 -- 172.21.15.46:0/3423884565 learned_addr learned my addr 172.21.15.46:0/3423884565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f22a00640 1 -- 172.21.15.46:0/3423884565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 msgr2=0x5632ce5413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f22a00640 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce5413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f22a00640 1 -- 172.21.15.46:0/3423884565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce53cc30 msgr2=0x5632ce548930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f22a00640 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce53cc30 0x5632ce548930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.592+0000 7f9f22a00640 1 -- 172.21.15.46:0/3423884565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ce27b620 con 0x5632ce537ba0 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f22a00640 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce5418f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9f100c4330 tx=0x7f9f10005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affeeb50d21d3a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3423884565 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100bb9c0 con 0x5632ce537ba0 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f2d212740 1 -- 172.21.15.46:0/3423884565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f1c00d770 con 0x5632ce537ba0 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3423884565 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9f100ef400 con 0x5632ce537ba0 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3423884565 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9f100d2070 con 0x5632ce537ba0 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f2d212740 10 client.28784 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f2d212740 1 -- 172.21.15.46:0/3423884565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5632ce54b720 con 0x5632ce537ba0 2024-09-10T07:50:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3423884565 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1c00d770 con 0x5632ce537ba0 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.593+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3423884565 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f9f100f2ce0 con 0x5632ce537ba0 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f2d212740 10 client.28784 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f2d212740 10 client.28784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f2d212740 20 client.28784 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5632ce54cef0 0x5632ce54f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f2d212740 4 client.28784 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f2d212740 1 -- 172.21.15.46:0/3423884565 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632ce4a9830 con 0x5632ce54cef0 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f23400640 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5632ce54cef0 0x5632ce54f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f23400640 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5632ce54cef0 0x5632ce54f330 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ed67c0dcba899e server_cookie=c098426295feee04 in_seq=0 out_seq=0 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.594+0000 7f9f0cc00640 10 client.28784 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3423884565 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632ce4a9830 con 0x5632ce54cef0 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f0cc00640 10 client.28784 handle_command_reply: tid=0 2024-09-10T07:50:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f2d212740 1 client.28784 shutdown 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f2d212740 1 -- 172.21.15.46:0/3423884565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 msgr2=0x5632ce5418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce5418f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9f100c4330 tx=0x7f9f10005760 comp rx=0 tx=0).stop 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f2d212740 1 -- 172.21.15.46:0/3423884565 shutdown_connections 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5632ce54cef0 0x5632ce54f330 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce53cc30 0x5632ce548930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce5418f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3423884565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce5413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.596+0000 7f9f2d212740 1 -- 172.21.15.46:0/3423884565 >> 172.21.15.46:0/3423884565 conn(0x5632ce5328d0 msgr2=0x5632ce5340e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.595+0000 7f9f0cc00640 0 client.28784 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.596+0000 7f9f2d212740 1 -- 172.21.15.46:0/3423884565 shutdown_connections 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.596+0000 7f9f2d212740 1 -- 172.21.15.46:0/3423884565 wait complete. 2024-09-10T07:50:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.596+0000 7f9f2d212740 20 client.28784 trim_cache size 0 max 16384 2024-09-10T07:50:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f2d212740 1 Processor -- start 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f2d212740 1 -- start start 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce538d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce539280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 0x5632ce5488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f2d212740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ac610 con 0x5632ce537ba0 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f2d212740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ae1e0 con 0x5632ce53cc30 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f2d212740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241acce0 con 0x5632ce537260 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.611+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce539280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce539280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51644/0 (socket says 172.21.15.46:51644) 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f22a00640 1 -- 172.21.15.46:0/1349930173 learned_addr learned my addr 172.21.15.46:0/1349930173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f23e00640 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 0x5632ce5488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f23400640 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce538d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f22a00640 1 -- 172.21.15.46:0/1349930173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 msgr2=0x5632ce538d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f22a00640 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce538d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f22a00640 1 -- 172.21.15.46:0/1349930173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 msgr2=0x5632ce5488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f22a00640 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 0x5632ce5488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f22a00640 1 -- 172.21.15.46:0/1349930173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ce54f9b0 con 0x5632ce537ba0 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f22a00640 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce539280 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9f100b9ea0 tx=0x7f9f1002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a156ca460cae7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f0f400640 1 -- 172.21.15.46:0/1349930173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100d8070 con 0x5632ce537ba0 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f0f400640 1 -- 172.21.15.46:0/1349930173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f100a6040 con 0x5632ce537ba0 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.612+0000 7f9f0f400640 1 -- 172.21.15.46:0/1349930173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100bb550 con 0x5632ce537ba0 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 -- 172.21.15.46:0/1349930173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 msgr2=0x5632ce539280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce539280 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9f100b9ea0 tx=0x7f9f1002a340 comp rx=0 tx=0).stop 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 -- 172.21.15.46:0/1349930173 shutdown_connections 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce53cc30 0x5632ce5488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537ba0 0x5632ce539280 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 --2- 172.21.15.46:0/1349930173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce537260 0x5632ce538d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 -- 172.21.15.46:0/1349930173 >> 172.21.15.46:0/1349930173 conn(0x5632ce5328d0 msgr2=0x5632ce5413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 -- 172.21.15.46:0/1349930173 shutdown_connections 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 -- 172.21.15.46:0/1349930173 wait complete. 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.613+0000 7f9f2d212740 1 -- start start 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f2d212740 1 Processor -- start 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537260 0x5632ce545d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce546290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f2d212740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce53cc30 0x5632ce547250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f2d212740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241adb10 con 0x5632ce537260 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f2d212740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ad3b0 con 0x5632ce537ba0 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f2d212740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241aa5f0 con 0x5632ce53cc30 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f2d212740 10 client.0 _ll_register_callbacks cb 0x7f9f100097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f23400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537260 0x5632ce545d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f23400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537260 0x5632ce545d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51656/0 (socket says 172.21.15.46:51656) 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f23400640 1 -- 172.21.15.46:0/3815713408 learned_addr learned my addr 172.21.15.46:0/3815713408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f22a00640 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce546290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.614+0000 7f9f23e00640 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce53cc30 0x5632ce547250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f23400640 1 -- 172.21.15.46:0/3815713408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce53cc30 msgr2=0x5632ce547250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f23400640 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce53cc30 0x5632ce547250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f23400640 1 -- 172.21.15.46:0/3815713408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 msgr2=0x5632ce546290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f23400640 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce546290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f23400640 1 -- 172.21.15.46:0/3815713408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ce538830 con 0x5632ce537260 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f23e00640 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce53cc30 0x5632ce547250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f23400640 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537260 0x5632ce545d50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009460 tx=0x7f9f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60715f8364a75009 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3815713408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14003bc0 con 0x5632ce537260 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3815713408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f9f14003d60 con 0x5632ce537260 2024-09-10T07:50:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f2d212740 1 -- 172.21.15.46:0/3815713408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f100a6040 con 0x5632ce537260 2024-09-10T07:50:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.615+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3815713408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9f1403c050 con 0x5632ce537260 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f2d212740 10 client.17964 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f2d212740 1 -- 172.21.15.46:0/3815713408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5632ce548a70 con 0x5632ce537260 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3815713408 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f100a6040 con 0x5632ce537260 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3815713408 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f9f14004c90 con 0x5632ce537260 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f2d212740 10 client.17964 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f2d212740 10 client.17964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f2d212740 20 client.17964 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5632ce5b8140 0x5632ce5ba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f2d212740 4 client.17964 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f2d212740 1 -- 172.21.15.46:0/3815713408 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632ce4a3b80 con 0x5632ce5b8140 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.616+0000 7f9f22a00640 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5632ce5b8140 0x5632ce5ba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.617+0000 7f9f22a00640 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5632ce5b8140 0x5632ce5ba560 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b162527496a03651 server_cookie=a730e3475871c035 in_seq=0 out_seq=0 2024-09-10T07:50:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.617+0000 7f9f0cc00640 10 client.17964 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f0cc00640 1 -- 172.21.15.46:0/3815713408 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632ce4a3b80 con 0x5632ce5b8140 2024-09-10T07:50:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f0cc00640 10 client.17964 handle_command_reply: tid=0 2024-09-10T07:50:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 client.17964 shutdown 2024-09-10T07:50:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 -- 172.21.15.46:0/3815713408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537260 msgr2=0x5632ce545d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537260 0x5632ce545d50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009460 tx=0x7f9f140046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 -- 172.21.15.46:0/3815713408 shutdown_connections 2024-09-10T07:50:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5632ce5b8140 0x5632ce5ba560 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5632ce53cc30 0x5632ce547250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5632ce537ba0 0x5632ce546290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f0cc00640 0 client.17964 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 --2- 172.21.15.46:0/3815713408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ce537260 0x5632ce545d50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.626+0000 7f9f2d212740 1 -- 172.21.15.46:0/3815713408 >> 172.21.15.46:0/3815713408 conn(0x5632ce5328d0 msgr2=0x5632ce541350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f2d212740 1 -- 172.21.15.46:0/3815713408 shutdown_connections 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f2d212740 1 -- 172.21.15.46:0/3815713408 wait complete. 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f2d212740 20 client.17964 trim_cache size 0 max 16384 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:50:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:50:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:50:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 -- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f080a2b80 msgr2=0x7f9f080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f080a2b80 0x7f9f080a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x5632ce5385d0 tx=0x7f9f14005f90 comp rx=0 tx=0).stop 2024-09-10T07:50:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 -- 172.21.15.46:0/922599976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f24104cf0 msgr2=0x7f9f2419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f24104cf0 0x7f9f2419a460 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9f100305b0 tx=0x7f9f10033d90 comp rx=0 tx=0).stop 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 -- 172.21.15.46:0/922599976 shutdown_connections 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f9f080a2b80 0x7f9f080a5040 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f24107620 0x7f9f241a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f24104cf0 0x7f9f2419a460 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 --2- 172.21.15.46:0/922599976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f241023c0 0x7f9f24199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.627+0000 7f9f28c00640 1 -- 172.21.15.46:0/922599976 >> 172.21.15.46:0/922599976 conn(0x7f9f240fbfa0 msgr2=0x7f9f24101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.628+0000 7f9f28c00640 1 -- 172.21.15.46:0/922599976 shutdown_connections 2024-09-10T07:50:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:14.628+0000 7f9f28c00640 1 -- 172.21.15.46:0/922599976 wait complete. 2024-09-10T07:50:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:14 smithi181 ceph-mon[35397]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.8 MiB/s wr, 355 op/s 2024-09-10T07:50:14.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:14 smithi063 ceph-mon[35020]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.8 MiB/s wr, 355 op/s 2024-09-10T07:50:14.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:14 smithi046 ceph-mon[30705]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.8 MiB/s wr, 355 op/s 2024-09-10T07:50:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 775.0 s, 113.5 tps, lat 200.588 ms stddev 284.637 2024-09-10T07:50:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:16 smithi181 ceph-mon[35397]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-09-10T07:50:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:16 smithi063 ceph-mon[35020]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-09-10T07:50:16.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:16 smithi046 ceph-mon[30705]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-09-10T07:50:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:18 smithi181 ceph-mon[35397]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-09-10T07:50:18.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:18 smithi063 ceph-mon[35020]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-09-10T07:50:18.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:18 smithi046 ceph-mon[30705]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-09-10T07:50:19.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:19 smithi181 ceph-mon[35397]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 2.5 MiB/s wr, 268 op/s 2024-09-10T07:50:19.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:19 smithi063 ceph-mon[35020]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 2.5 MiB/s wr, 268 op/s 2024-09-10T07:50:19.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:19 smithi046 ceph-mon[30705]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 2.5 MiB/s wr, 268 op/s 2024-09-10T07:50:20.604 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 780.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:50:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:21 smithi046 ceph-mon[30705]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 230 op/s 2024-09-10T07:50:22.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:21 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:50:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:21 smithi181 ceph-mon[35397]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 230 op/s 2024-09-10T07:50:22.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:21 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:50:22.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:21 smithi063 ceph-mon[35020]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 230 op/s 2024-09-10T07:50:22.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:21 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:50:23.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:22 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:50:23.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:22 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:50:23.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:22 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:50:23.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:22 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:50:23.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:22 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:50:23.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:22 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:50:23.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:22 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:50:23.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:22 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:50:23.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:22 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:50:24.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:23 smithi181 ceph-mon[35397]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-10T07:50:24.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:23 smithi063 ceph-mon[35020]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-10T07:50:24.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:23 smithi046 ceph-mon[30705]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-10T07:50:25.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 785.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:50:26.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:25 smithi181 ceph-mon[35397]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 206 KiB/s wr, 33 op/s 2024-09-10T07:50:26.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:25 smithi063 ceph-mon[35020]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 206 KiB/s wr, 33 op/s 2024-09-10T07:50:26.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:25 smithi046 ceph-mon[30705]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 206 KiB/s wr, 33 op/s 2024-09-10T07:50:27.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:50:27.587 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 -- 172.21.15.46:0/820770197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308103d60 msgr2=0x7fa308104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 --2- 172.21.15.46:0/820770197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308103d60 0x7fa308104160 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009ca0 tx=0x7fa2fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 -- 172.21.15.46:0/820770197 shutdown_connections 2024-09-10T07:50:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 --2- 172.21.15.46:0/820770197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa308105910 0x7fa30810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 --2- 172.21.15.46:0/820770197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa308104f50 0x7fa3081053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 --2- 172.21.15.46:0/820770197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308103d60 0x7fa308104160 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 -- 172.21.15.46:0/820770197 >> 172.21.15.46:0/820770197 conn(0x7fa3080ff4e0 msgr2=0x7fa308101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 -- 172.21.15.46:0/820770197 shutdown_connections 2024-09-10T07:50:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.751+0000 7fa30f800640 1 -- 172.21.15.46:0/820770197 wait complete. 2024-09-10T07:50:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.752+0000 7fa30f800640 1 Processor -- start 2024-09-10T07:50:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.752+0000 7fa30f800640 1 -- start start 2024-09-10T07:50:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30f800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa308103d60 0x7fa308199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30f800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa308104f50 0x7fa30819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308105910 0x7fa3081a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa30810f9e0 con 0x7fa308105910 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30f800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa30810f860 con 0x7fa308104f50 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30f800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa30810fb60 con 0x7fa308103d60 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308105910 0x7fa3081a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa308103d60 0x7fa308199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa307e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa308104f50 0x7fa30819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa308103d60 0x7fa308199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:59982/0 (socket says 172.21.15.46:59982) 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30ce00640 1 -- 172.21.15.46:0/3600387857 learned_addr learned my addr 172.21.15.46:0/3600387857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30d800640 1 -- 172.21.15.46:0/3600387857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa308103d60 msgr2=0x7fa308199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30d800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa308103d60 0x7fa308199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30d800640 1 -- 172.21.15.46:0/3600387857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa308104f50 msgr2=0x7fa30819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30d800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa308104f50 0x7fa30819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.753+0000 7fa30d800640 1 -- 172.21.15.46:0/3600387857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3081a1bd0 con 0x7fa308105910 2024-09-10T07:50:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.754+0000 7fa30d800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308105910 0x7fa3081a1560 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa30000e970 tx=0x7fa30000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.754+0000 7fa305600640 1 -- 172.21.15.46:0/3600387857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30000ccc0 con 0x7fa308105910 2024-09-10T07:50:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.754+0000 7fa305600640 1 -- 172.21.15.46:0/3600387857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa300005020 con 0x7fa308105910 2024-09-10T07:50:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.754+0000 7fa305600640 1 -- 172.21.15.46:0/3600387857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300004a00 con 0x7fa308105910 2024-09-10T07:50:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.754+0000 7fa30f800640 1 -- 172.21.15.46:0/3600387857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3081a1ec0 con 0x7fa308105910 2024-09-10T07:50:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.754+0000 7fa30f800640 1 -- 172.21.15.46:0/3600387857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3081a23d0 con 0x7fa308105910 2024-09-10T07:50:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.757+0000 7fa3126d3740 1 Processor -- start 2024-09-10T07:50:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.757+0000 7fa305600640 1 -- 172.21.15.46:0/3600387857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa3000051d0 con 0x7fa308105910 2024-09-10T07:50:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.757+0000 7fa3126d3740 1 -- start start 2024-09-10T07:50:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.757+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07045f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.757+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f9d0 0x55e070464490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e070466f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa3126d3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e0703c4100 con 0x55e07045f090 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa3126d3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e07036c1b0 con 0x55e07045f9d0 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa3126d3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e0703c00c0 con 0x55e070464a60 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07045f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07045f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57504/0 (socket says 172.21.15.46:57504) 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 -- 172.21.15.46:0/2443855496 learned_addr learned my addr 172.21.15.46:0/2443855496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa30ce00640 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f9d0 0x55e070464490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e070466f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 -- 172.21.15.46:0/2443855496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 msgr2=0x55e070466f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e070466f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 -- 172.21.15.46:0/2443855496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f9d0 msgr2=0x55e070464490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f9d0 0x55e070464490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 -- 172.21.15.46:0/2443855496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0700d7210 con 0x55e07045f090 2024-09-10T07:50:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa307e00640 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07045f490 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40135a0 tx=0x7fa2f4009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed781bdb1b22966 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.758+0000 7fa305600640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2f00a2cb0 0x7fa2f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.759+0000 7fa305600640 1 -- 172.21.15.46:0/3600387857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa3000cf2a0 con 0x7fa308105910 2024-09-10T07:50:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.759+0000 7fa2fa000640 1 -- 172.21.15.46:0/2443855496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4004050 con 0x55e07045f090 2024-09-10T07:50:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.759+0000 7fa2fa000640 1 -- 172.21.15.46:0/2443855496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f40041f0 con 0x55e07045f090 2024-09-10T07:50:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.759+0000 7fa2fa000640 1 -- 172.21.15.46:0/2443855496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f400f780 con 0x55e07045f090 2024-09-10T07:50:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.759+0000 7fa30ce00640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2f00a2cb0 0x7fa2f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.759+0000 7fa3126d3740 1 -- 172.21.15.46:0/2443855496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 msgr2=0x55e07045f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.759+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07045f490 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40135a0 tx=0x7fa2f4009850 comp rx=0 tx=0).stop 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa30ce00640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2f00a2cb0 0x7fa2f00a5170 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55e070460400 tx=0x7fa2fc03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 -- 172.21.15.46:0/2443855496 shutdown_connections 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e070466f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f9d0 0x55e070464490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2443855496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07045f490 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 -- 172.21.15.46:0/2443855496 >> 172.21.15.46:0/2443855496 conn(0x55e07045a700 msgr2=0x55e07045ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 -- 172.21.15.46:0/2443855496 shutdown_connections 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 -- 172.21.15.46:0/2443855496 wait complete. 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 -- start start 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.760+0000 7fa3126d3740 1 Processor -- start 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 0x55e070469160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e0704696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e070464a60 0x55e0704706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa3126d3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e0704679d0 con 0x55e070464a60 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa3126d3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e07034d090 con 0x55e07045f090 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa3126d3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e070467b50 con 0x55e07045f9d0 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa3126d3740 10 client.0 _ll_register_callbacks cb 0x55e0703a1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa30d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e070464a60 0x55e0704706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa307e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e0704696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa30d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e070464a60 0x55e0704706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57510/0 (socket says 172.21.15.46:57510) 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa30d800640 1 -- 172.21.15.46:0/2547703244 learned_addr learned my addr 172.21.15.46:0/2547703244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa30ce00640 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 0x55e070469160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa30d800640 1 -- 172.21.15.46:0/2547703244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 msgr2=0x55e0704696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.761+0000 7fa30d800640 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e0704696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa30d800640 1 -- 172.21.15.46:0/2547703244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 msgr2=0x55e070469160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa30d800640 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 0x55e070469160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa30d800640 1 -- 172.21.15.46:0/2547703244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0701a3620 con 0x55e070464a60 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa30d800640 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e070464a60 0x55e0704706e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa300000c00 tx=0x7fa3000056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1bd1824643daed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa2eea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa2eea00640 1 -- 172.21.15.46:0/2547703244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30001c070 con 0x55e070464a60 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa2eea00640 1 -- 172.21.15.46:0/2547703244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa3000a3f00 con 0x55e070464a60 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa3126d3740 1 -- 172.21.15.46:0/2547703244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f40041f0 con 0x55e070464a60 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa2eea00640 1 -- 172.21.15.46:0/2547703244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa30009fad0 con 0x55e070464a60 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa3126d3740 10 client.17976 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa3126d3740 1 -- 172.21.15.46:0/2547703244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e070473540 con 0x55e070464a60 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.762+0000 7fa2eea00640 1 -- 172.21.15.46:0/2547703244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f40041f0 con 0x55e070464a60 2024-09-10T07:50:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.763+0000 7fa2eea00640 1 -- 172.21.15.46:0/2547703244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fa30009ab70 con 0x55e070464a60 2024-09-10T07:50:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.763+0000 7fa3126d3740 10 client.17976 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.763+0000 7fa3126d3740 10 client.17976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.763+0000 7fa3126d3740 20 client.17976 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.763+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e070474c10 0x55e070477000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.763+0000 7fa3126d3740 4 client.17976 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.763+0000 7fa3126d3740 1 -- 172.21.15.46:0/2547703244 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0703d1830 con 0x55e070474c10 2024-09-10T07:50:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.763+0000 7fa30ce00640 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e070474c10 0x55e070477000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.764+0000 7fa30ce00640 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e070474c10 0x55e070477000 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752fc76a52dce4bc server_cookie=e9a7ee65798c9ba6 in_seq=0 out_seq=0 2024-09-10T07:50:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.764+0000 7fa2eea00640 10 client.17976 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.764+0000 7fa2eea00640 1 -- 172.21.15.46:0/2547703244 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0703d1830 con 0x55e070474c10 2024-09-10T07:50:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.764+0000 7fa2eea00640 10 client.17976 handle_command_reply: tid=0 2024-09-10T07:50:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.764+0000 7fa3126d3740 1 client.17976 shutdown 2024-09-10T07:50:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.764+0000 7fa3126d3740 1 -- 172.21.15.46:0/2547703244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e070464a60 msgr2=0x55e0704706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.764+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e070464a60 0x55e0704706e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa300000c00 tx=0x7fa3000056a0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 1 -- 172.21.15.46:0/2547703244 shutdown_connections 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e070474c10 0x55e070477000 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e070464a60 0x55e0704706e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e0704696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2547703244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 0x55e070469160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa2eea00640 0 client.17976 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 1 -- 172.21.15.46:0/2547703244 >> 172.21.15.46:0/2547703244 conn(0x55e07045a700 msgr2=0x55e07045bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 1 -- 172.21.15.46:0/2547703244 shutdown_connections 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 1 -- 172.21.15.46:0/2547703244 wait complete. 2024-09-10T07:50:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.765+0000 7fa3126d3740 20 client.17976 trim_cache size 0 max 16384 2024-09-10T07:50:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.780+0000 7fa3126d3740 1 Processor -- start 2024-09-10T07:50:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.780+0000 7fa3126d3740 1 -- start start 2024-09-10T07:50:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.780+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 0x55e07045b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.780+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f9d0 0x55e0704706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.780+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e07045baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.780+0000 7fa3126d3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081ac900 con 0x55e07045f9d0 2024-09-10T07:50:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.780+0000 7fa3126d3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081ae4d0 con 0x55e07045f090 2024-09-10T07:50:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.780+0000 7fa3126d3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081acfd0 con 0x55e070464a60 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f9d0 0x55e0704706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa30ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 0x55e07045b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f9d0 0x55e0704706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57512/0 (socket says 172.21.15.46:57512) 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 -- 172.21.15.46:0/2685888076 learned_addr learned my addr 172.21.15.46:0/2685888076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa30d800640 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e07045baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 -- 172.21.15.46:0/2685888076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 msgr2=0x55e07045baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e07045baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 -- 172.21.15.46:0/2685888076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 msgr2=0x55e07045b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 0x55e07045b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 -- 172.21.15.46:0/2685888076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e07045a520 con 0x55e07045f9d0 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa30d800640 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e07045baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.781+0000 7fa307e00640 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f9d0 0x55e0704706a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009730 tx=0x7fa2f4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facc8f122c5ba63e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa2f9600640 1 -- 172.21.15.46:0/2685888076 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40050b0 con 0x55e07045f9d0 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa2f9600640 1 -- 172.21.15.46:0/2685888076 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f40035f0 con 0x55e07045f9d0 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa2f9600640 1 -- 172.21.15.46:0/2685888076 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f400f7c0 con 0x55e07045f9d0 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 -- 172.21.15.46:0/2685888076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f9d0 msgr2=0x55e0704706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f9d0 0x55e0704706a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009730 tx=0x7fa2f4009660 comp rx=0 tx=0).stop 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 -- 172.21.15.46:0/2685888076 shutdown_connections 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e070464a60 0x55e07045baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f9d0 0x55e0704706a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 --2- 172.21.15.46:0/2685888076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e07045f090 0x55e07045b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 -- 172.21.15.46:0/2685888076 >> 172.21.15.46:0/2685888076 conn(0x55e07045a700 msgr2=0x55e070469160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 -- 172.21.15.46:0/2685888076 shutdown_connections 2024-09-10T07:50:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.782+0000 7fa3126d3740 1 -- 172.21.15.46:0/2685888076 wait complete. 2024-09-10T07:50:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 1 -- start start 2024-09-10T07:50:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 1 Processor -- start 2024-09-10T07:50:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07046da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e07046dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e070464a60 0x55e07046ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081ade00 con 0x55e07045f090 2024-09-10T07:50:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081ad6a0 con 0x55e070464a60 2024-09-10T07:50:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081aa8e0 con 0x55e07045f9d0 2024-09-10T07:50:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa3126d3740 10 client.0 _ll_register_callbacks cb 0x7fa300097380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.783+0000 7fa30ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07046da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07046da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57516/0 (socket says 172.21.15.46:57516) 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30ce00640 1 -- 172.21.15.46:0/1672808001 learned_addr learned my addr 172.21.15.46:0/1672808001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30d800640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e070464a60 0x55e07046ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa307e00640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e07046dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30ce00640 1 -- 172.21.15.46:0/1672808001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 msgr2=0x55e07046dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30ce00640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e07046dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30ce00640 1 -- 172.21.15.46:0/1672808001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e070464a60 msgr2=0x55e07046ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30ce00640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e070464a60 0x55e07046ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30ce00640 1 -- 172.21.15.46:0/1672808001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e07046f570 con 0x55e07045f090 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30d800640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e070464a60 0x55e07046ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa307e00640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e07046dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa30ce00640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07046da60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006010 tx=0x7fa2fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60161af71631db34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa2eea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa2eea00640 1 -- 172.21.15.46:0/1672808001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc054430 con 0x55e07045f090 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa2eea00640 1 -- 172.21.15.46:0/1672808001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa2fc0545d0 con 0x55e07045f090 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.784+0000 7fa3126d3740 1 -- 172.21.15.46:0/1672808001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f40035f0 con 0x55e07045f090 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa2eea00640 1 -- 172.21.15.46:0/1672808001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa2fc061050 con 0x55e07045f090 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa3126d3740 10 client.17982 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa3126d3740 1 -- 172.21.15.46:0/1672808001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e070470980 con 0x55e07045f090 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa2eea00640 1 -- 172.21.15.46:0/1672808001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f40035f0 con 0x55e07045f090 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa2eea00640 1 -- 172.21.15.46:0/1672808001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fa2fc030cc0 con 0x55e07045f090 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa3126d3740 10 client.17982 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa3126d3740 10 client.17982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa3126d3740 20 client.17982 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa3126d3740 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0704dff50 0x55e0704e2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa3126d3740 4 client.17982 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa3126d3740 1 -- 172.21.15.46:0/1672808001 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0703cbb80 con 0x55e0704dff50 2024-09-10T07:50:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.785+0000 7fa307e00640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0704dff50 0x55e0704e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.786+0000 7fa307e00640 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0704dff50 0x55e0704e2370 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf32b9c559bd289d server_cookie=a293391c54ca8a0b in_seq=0 out_seq=0 2024-09-10T07:50:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.786+0000 7fa2eea00640 10 client.17982 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa2eea00640 1 -- 172.21.15.46:0/1672808001 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0703cbb80 con 0x55e0704dff50 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa2eea00640 10 client.17982 handle_command_reply: tid=0 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 client.17982 shutdown 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 -- 172.21.15.46:0/1672808001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 msgr2=0x55e07046da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07046da60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006010 tx=0x7fa2fc0095c0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 -- 172.21.15.46:0/1672808001 shutdown_connections 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55e0704dff50 0x55e0704e2370 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e070464a60 0x55e07046ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e07045f9d0 0x55e07046dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 --2- 172.21.15.46:0/1672808001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e07045f090 0x55e07046da60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa2eea00640 0 client.17982 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 -- 172.21.15.46:0/1672808001 >> 172.21.15.46:0/1672808001 conn(0x55e07045a700 msgr2=0x55e070459060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 -- 172.21.15.46:0/1672808001 shutdown_connections 2024-09-10T07:50:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 1 -- 172.21.15.46:0/1672808001 wait complete. 2024-09-10T07:50:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.795+0000 7fa3126d3740 20 client.17982 trim_cache size 0 max 16384 2024-09-10T07:50:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:50:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:50:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:50:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:50:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 -- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2f00a2cb0 msgr2=0x7fa2f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2f00a2cb0 0x7fa2f00a5170 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55e070460400 tx=0x7fa2fc03d040 comp rx=0 tx=0).stop 2024-09-10T07:50:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 -- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308105910 msgr2=0x7fa3081a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308105910 0x7fa3081a1560 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa30000e970 tx=0x7fa30000ee30 comp rx=0 tx=0).stop 2024-09-10T07:50:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 -- 172.21.15.46:0/3600387857 shutdown_connections 2024-09-10T07:50:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa2f00a2cb0 0x7fa2f00a5170 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa308105910 0x7fa3081a1560 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa308104f50 0x7fa30819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 --2- 172.21.15.46:0/3600387857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa308103d60 0x7fa308199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 -- 172.21.15.46:0/3600387857 >> 172.21.15.46:0/3600387857 conn(0x7fa3080ff4e0 msgr2=0x7fa308108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 -- 172.21.15.46:0/3600387857 shutdown_connections 2024-09-10T07:50:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:27.796+0000 7fa30f800640 1 -- 172.21.15.46:0/3600387857 wait complete. 2024-09-10T07:50:28.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:27 smithi046 ceph-mon[30705]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 207 KiB/s wr, 34 op/s 2024-09-10T07:50:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 -- 172.21.15.46:0/3257334744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 msgr2=0x7f5170110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 --2- 172.21.15.46:0/3257334744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 0x7f5170110ae0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5168009ca0 tx=0x7f516802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 -- 172.21.15.46:0/3257334744 shutdown_connections 2024-09-10T07:50:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 --2- 172.21.15.46:0/3257334744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 0x7f5170110ae0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 --2- 172.21.15.46:0/3257334744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51701047e0 0x7f5170108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 --2- 172.21.15.46:0/3257334744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5170103e30 0x7f5170104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 -- 172.21.15.46:0/3257334744 >> 172.21.15.46:0/3257334744 conn(0x7f51700ff8d0 msgr2=0x7f5170101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 -- 172.21.15.46:0/3257334744 shutdown_connections 2024-09-10T07:50:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.317+0000 7f5174a00640 1 -- 172.21.15.46:0/3257334744 wait complete. 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.318+0000 7f5174a00640 1 Processor -- start 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5174a00640 1 -- start start 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5174a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5170103e30 0x7f517019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5174a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51701047e0 0x7f517019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5174a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 0x7f517019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5174a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5170113fc0 con 0x7f5170108f60 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5174a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5170113e40 con 0x7f5170103e30 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5174a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5170114140 con 0x7f51701047e0 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5167e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51701047e0 0x7f517019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f516fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 0x7f517019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f5167e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51701047e0 0x7f517019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60024/0 (socket says 172.21.15.46:60024) 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f5167e00640 1 -- 172.21.15.46:0/1486817816 learned_addr learned my addr 172.21.15.46:0/1486817816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.319+0000 7f516f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5170103e30 0x7f517019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f5167e00640 1 -- 172.21.15.46:0/1486817816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5170103e30 msgr2=0x7f517019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f5167e00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5170103e30 0x7f517019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f5167e00640 1 -- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 msgr2=0x7f517019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f5167e00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 0x7f517019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f5167e00640 1 -- 172.21.15.46:0/1486817816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517019f3b0 con 0x7f51701047e0 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f516fe00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 0x7f517019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f516f400640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5170103e30 0x7f517019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.320+0000 7f5167e00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51701047e0 0x7f517019a940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f515800e9f0 tx=0x7f515800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.321+0000 7f516cc00640 1 -- 172.21.15.46:0/1486817816 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515800cd80 con 0x7f51701047e0 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.321+0000 7f516cc00640 1 -- 172.21.15.46:0/1486817816 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5158010910 con 0x7f51701047e0 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.321+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517019f6a0 con 0x7f51701047e0 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.321+0000 7f516cc00640 1 -- 172.21.15.46:0/1486817816 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158018990 con 0x7f51701047e0 2024-09-10T07:50:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.321+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51701a6f80 con 0x7f51701047e0 2024-09-10T07:50:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.322+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5170104210 con 0x7f51701047e0 2024-09-10T07:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.325+0000 7f516cc00640 1 -- 172.21.15.46:0/1486817816 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5158010ac0 con 0x7f51701047e0 2024-09-10T07:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.326+0000 7f516cc00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51540a2b80 0x7f51540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.326+0000 7f516cc00640 1 -- 172.21.15.46:0/1486817816 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f51580cfab0 con 0x7f51701047e0 2024-09-10T07:50:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.326+0000 7f516f400640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51540a2b80 0x7f51540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.327+0000 7f516f400640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51540a2b80 0x7f51540a5040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5160009fc0 tx=0x7f5160005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.327+0000 7f516cc00640 1 -- 172.21.15.46:0/1486817816 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f515809bfe0 con 0x7f51701047e0 2024-09-10T07:50:28.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:27 smithi181 ceph-mon[35397]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 207 KiB/s wr, 34 op/s 2024-09-10T07:50:28.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:27 smithi063 ceph-mon[35020]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 207 KiB/s wr, 34 op/s 2024-09-10T07:50:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.552+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f517010c1a0 con 0x7f51701047e0 2024-09-10T07:50:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.553+0000 7f516cc00640 1 -- 172.21.15.46:0/1486817816 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11977 (secure 0 0 0) 0x7f51580a02e0 con 0x7f51701047e0 2024-09-10T07:50:28.553 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:50:28.553 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"btime":"2024-09-10T07:49:48:222949+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:49:48.058024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:50:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 54 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.555+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51540a2b80 msgr2=0x7f51540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.555+0000 7f5174a00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51540a2b80 0x7f51540a5040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5160009fc0 tx=0x7f5160005f90 comp rx=0 tx=0).stop 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.555+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51701047e0 msgr2=0x7f517019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.555+0000 7f5174a00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51701047e0 0x7f517019a940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f515800e9f0 tx=0x7f515800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.555+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 shutdown_connections 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.555+0000 7f5174a00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51540a2b80 0x7f51540a5040 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.555+0000 7f5174a00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108f60 0x7f517019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.556+0000 7f5174a00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51701047e0 0x7f517019a940 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.556+0000 7f5174a00640 1 --2- 172.21.15.46:0/1486817816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5170103e30 0x7f517019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.556+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 >> 172.21.15.46:0/1486817816 conn(0x7f51700ff8d0 msgr2=0x7f5170100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.556+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 shutdown_connections 2024-09-10T07:50:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:28.556+0000 7f5174a00640 1 -- 172.21.15.46:0/1486817816 wait complete. 2024-09-10T07:50:29.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:28 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1486817816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:29.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:28 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1486817816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:29.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:28 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1486817816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:29.485 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id ec6517 --release --await' 2024-09-10T07:50:29.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id ec6517 --release --await 2024-09-10T07:50:29.907 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:30.240 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:29 smithi046 ceph-mon[30705]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T07:50:30.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:50:30.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:29 smithi181 ceph-mon[35397]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T07:50:30.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:29 smithi063 ceph-mon[35020]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T07:50:30.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 -- 172.21.15.46:0/2626839210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c103ff0 msgr2=0x7f783c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 --2- 172.21.15.46:0/2626839210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c103ff0 0x7f783c1043d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f782c009900 tx=0x7f782c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 -- 172.21.15.46:0/2626839210 shutdown_connections 2024-09-10T07:50:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 --2- 172.21.15.46:0/2626839210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f783c109120 0x7f783c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 --2- 172.21.15.46:0/2626839210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c1049a0 0x7f783c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 --2- 172.21.15.46:0/2626839210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c103ff0 0x7f783c1043d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 -- 172.21.15.46:0/2626839210 >> 172.21.15.46:0/2626839210 conn(0x7f783c0ffa90 msgr2=0x7f783c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 -- 172.21.15.46:0/2626839210 shutdown_connections 2024-09-10T07:50:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.511+0000 7f7842800640 1 -- 172.21.15.46:0/2626839210 wait complete. 2024-09-10T07:50:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.512+0000 7f7842800640 1 Processor -- start 2024-09-10T07:50:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.512+0000 7f7842800640 1 -- start start 2024-09-10T07:50:30.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f7842800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f783c103ff0 0x7f783c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f7842800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c1049a0 0x7f783c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f7842800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c109120 0x7f783c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f7842800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1141a0 con 0x7f783c1049a0 2024-09-10T07:50:30.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f7842800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f783c114020 con 0x7f783c109120 2024-09-10T07:50:30.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f7842800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f783c114320 con 0x7f783c103ff0 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f783aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c1049a0 0x7f783c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f783b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f783c103ff0 0x7f783c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f783aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c1049a0 0x7f783c19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57568/0 (socket says 172.21.15.46:57568) 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f783aa00640 1 -- 172.21.15.46:0/2395272999 learned_addr learned my addr 172.21.15.46:0/2395272999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.513+0000 7f783be00640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c109120 0x7f783c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.514+0000 7f783aa00640 1 -- 172.21.15.46:0/2395272999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f783c103ff0 msgr2=0x7f783c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.514+0000 7f783aa00640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f783c103ff0 0x7f783c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.514+0000 7f783aa00640 1 -- 172.21.15.46:0/2395272999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c109120 msgr2=0x7f783c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.514+0000 7f783aa00640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c109120 0x7f783c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.514+0000 7f783aa00640 1 -- 172.21.15.46:0/2395272999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c19f690 con 0x7f783c1049a0 2024-09-10T07:50:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.514+0000 7f783be00640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c109120 0x7f783c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.514+0000 7f783b400640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f783c103ff0 0x7f783c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.514+0000 7f783aa00640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c1049a0 0x7f783c19ac20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f782800e9f0 tx=0x7f782800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.515+0000 7f7833e00640 1 -- 172.21.15.46:0/2395272999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782800cd80 con 0x7f783c1049a0 2024-09-10T07:50:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.515+0000 7f7833e00640 1 -- 172.21.15.46:0/2395272999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7828010910 con 0x7f783c1049a0 2024-09-10T07:50:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.515+0000 7f7842800640 1 -- 172.21.15.46:0/2395272999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c19f980 con 0x7f783c1049a0 2024-09-10T07:50:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.515+0000 7f7833e00640 1 -- 172.21.15.46:0/2395272999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828018990 con 0x7f783c1049a0 2024-09-10T07:50:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.515+0000 7f7842800640 1 -- 172.21.15.46:0/2395272999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c1a7260 con 0x7f783c1049a0 2024-09-10T07:50:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f784526c740 1 Processor -- start 2024-09-10T07:50:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f7833e00640 1 -- 172.21.15.46:0/2395272999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7828010ac0 con 0x7f783c1049a0 2024-09-10T07:50:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f784526c740 1 -- start start 2024-09-10T07:50:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f784526c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423a310 0x56232423a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f784526c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423ac30 0x562324244fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f784526c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f784526c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5623241a0650 con 0x56232423ac30 2024-09-10T07:50:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f784526c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562324147f10 con 0x56232423a310 2024-09-10T07:50:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.518+0000 7f784526c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56232419c610 con 0x5623242454f0 2024-09-10T07:50:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423ac30 0x562324244fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423ac30 0x562324244fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57578/0 (socket says 172.21.15.46:57578) 2024-09-10T07:50:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783b400640 1 -- 172.21.15.46:0/2659115674 learned_addr learned my addr 172.21.15.46:0/2659115674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423a310 0x56232423a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783aa00640 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783be00640 1 -- 172.21.15.46:0/2659115674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 msgr2=0x5623242478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783be00640 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783be00640 1 -- 172.21.15.46:0/2659115674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423ac30 msgr2=0x562324244fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783be00640 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423ac30 0x562324244fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783be00640 1 -- 172.21.15.46:0/2659115674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562323eaf580 con 0x56232423a310 2024-09-10T07:50:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.519+0000 7f783aa00640 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.520+0000 7f783be00640 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423a310 0x56232423a6f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7834010710 tx=0x7f7834010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997de795cbe91c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.520+0000 7f7830c00640 1 -- 172.21.15.46:0/2659115674 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834004b20 con 0x56232423a310 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.520+0000 7f7833e00640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78200a2c00 0x7f78200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.520+0000 7f7830c00640 1 -- 172.21.15.46:0/2659115674 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783400edb0 con 0x56232423a310 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.520+0000 7f7830c00640 1 -- 172.21.15.46:0/2659115674 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834005690 con 0x56232423a310 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.520+0000 7f7833e00640 1 -- 172.21.15.46:0/2395272999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f78280cf690 con 0x7f783c1049a0 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f783b400640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78200a2c00 0x7f78200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f784526c740 1 -- 172.21.15.46:0/2659115674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423a310 msgr2=0x56232423a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f784526c740 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423a310 0x56232423a6f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7834010710 tx=0x7f7834010bd0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f784526c740 1 -- 172.21.15.46:0/2659115674 shutdown_connections 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f784526c740 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f784526c740 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423ac30 0x562324244fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f784526c740 1 --2- 172.21.15.46:0/2659115674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423a310 0x56232423a6f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f784526c740 1 -- 172.21.15.46:0/2659115674 >> 172.21.15.46:0/2659115674 conn(0x5623242368a0 msgr2=0x562324236c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.521+0000 7f784526c740 1 -- 172.21.15.46:0/2659115674 shutdown_connections 2024-09-10T07:50:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.522+0000 7f783b400640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78200a2c00 0x7f78200a50c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x562324237700 tx=0x7f782c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.522+0000 7f784526c740 1 -- 172.21.15.46:0/2659115674 wait complete. 2024-09-10T07:50:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.522+0000 7f784526c740 1 -- start start 2024-09-10T07:50:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.522+0000 7f784526c740 1 Processor -- start 2024-09-10T07:50:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f784526c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x5623242423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f784526c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x5623242428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f784526c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x562324242e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f784526c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5623242480c0 con 0x56232423a310 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f784526c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56232415b350 con 0x56232423ac30 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f784526c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562324248240 con 0x5623242454f0 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f784526c740 10 client.0 _ll_register_callbacks cb 0x56232417e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f783aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x5623242428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f783b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x5623242423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f783aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x5623242428e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33148/0 (socket says 172.21.15.46:33148) 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f783aa00640 1 -- 172.21.15.46:0/2780936339 learned_addr learned my addr 172.21.15.46:0/2780936339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f783be00640 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x562324242e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f783aa00640 1 -- 172.21.15.46:0/2780936339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 msgr2=0x562324242e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.523+0000 7f783aa00640 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x562324242e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f783aa00640 1 -- 172.21.15.46:0/2780936339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 msgr2=0x5623242423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f783aa00640 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x5623242423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f783aa00640 1 -- 172.21.15.46:0/2780936339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562323f79960 con 0x56232423ac30 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f783be00640 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x562324242e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f783aa00640 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x5623242428e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f782809b8c0 tx=0x7f7828011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beaa78b0590dd0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f7825600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f7825600640 1 -- 172.21.15.46:0/2780936339 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78280a4780 con 0x56232423ac30 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f784526c740 1 -- 172.21.15.46:0/2780936339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783400edb0 con 0x56232423ac30 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f7825600640 1 -- 172.21.15.46:0/2780936339 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f78280d1b50 con 0x56232423ac30 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f7825600640 1 -- 172.21.15.46:0/2780936339 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f78280d1d50 con 0x56232423ac30 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f784526c740 10 client.28826 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.524+0000 7f784526c740 1 -- 172.21.15.46:0/2780936339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562324253c80 con 0x56232423ac30 2024-09-10T07:50:30.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f7825600640 1 -- 172.21.15.46:0/2780936339 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783400edb0 con 0x56232423ac30 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f7825600640 1 -- 172.21.15.46:0/2780936339 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f78280ba270 con 0x56232423ac30 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f784526c740 10 client.28826 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f784526c740 10 client.28826 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f784526c740 20 client.28826 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f784526c740 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562324255140 0x562324257530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f784526c740 4 client.28826 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f784526c740 1 -- 172.21.15.46:0/2780936339 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623241adbc0 con 0x562324255140 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.525+0000 7f783b400640 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562324255140 0x562324257530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.526+0000 7f783b400640 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562324255140 0x562324257530 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b230c67a77be016 server_cookie=12563e086eac9c6b in_seq=0 out_seq=0 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.526+0000 7f7825600640 10 client.28826 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f7825600640 1 -- 172.21.15.46:0/2780936339 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623241adbc0 con 0x562324255140 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f7825600640 10 client.28826 handle_command_reply: tid=0 2024-09-10T07:50:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 client.28826 shutdown 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 -- 172.21.15.46:0/2780936339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 msgr2=0x5623242428e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x5623242428e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f782809b8c0 tx=0x7f7828011e20 comp rx=0 tx=0).stop 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 -- 172.21.15.46:0/2780936339 shutdown_connections 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562324255140 0x562324257530 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x562324242e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x5623242428e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f7825600640 0 client.28826 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 --2- 172.21.15.46:0/2780936339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x5623242423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.527+0000 7f784526c740 1 -- 172.21.15.46:0/2780936339 >> 172.21.15.46:0/2780936339 conn(0x5623242368a0 msgr2=0x56232423c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.528+0000 7f784526c740 1 -- 172.21.15.46:0/2780936339 shutdown_connections 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.528+0000 7f784526c740 1 -- 172.21.15.46:0/2780936339 wait complete. 2024-09-10T07:50:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.528+0000 7f784526c740 20 client.28826 trim_cache size 0 max 16384 2024-09-10T07:50:30.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.546+0000 7f784526c740 1 Processor -- start 2024-09-10T07:50:30.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.546+0000 7f784526c740 1 -- start start 2024-09-10T07:50:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.546+0000 7f784526c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56232423a310 0x56232423cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.546+0000 7f784526c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x562324241250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.546+0000 7f784526c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623242454f0 0x56232425cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.546+0000 7f784526c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b0d80 con 0x5623242454f0 2024-09-10T07:50:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.546+0000 7f784526c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b2950 con 0x56232423ac30 2024-09-10T07:50:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.546+0000 7f784526c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b1450 con 0x56232423a310 2024-09-10T07:50:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623242454f0 0x56232425cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56232423a310 0x56232423cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x562324241250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623242454f0 0x56232425cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57596/0 (socket says 172.21.15.46:57596) 2024-09-10T07:50:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783be00640 1 -- 172.21.15.46:0/1981613095 learned_addr learned my addr 172.21.15.46:0/1981613095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783aa00640 1 -- 172.21.15.46:0/1981613095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56232423a310 msgr2=0x56232423cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783aa00640 1 --2- 172.21.15.46:0/1981613095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56232423a310 0x56232423cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783aa00640 1 -- 172.21.15.46:0/1981613095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623242454f0 msgr2=0x56232425cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783aa00640 1 --2- 172.21.15.46:0/1981613095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623242454f0 0x56232425cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.547+0000 7f783aa00640 1 -- 172.21.15.46:0/1981613095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562324257b60 con 0x56232423ac30 2024-09-10T07:50:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.548+0000 7f783aa00640 1 --2- 172.21.15.46:0/1981613095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x562324241250 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f782809bb50 tx=0x7f782800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e0a7f9b4e8e617 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.548+0000 7f7827e00640 1 -- 172.21.15.46:0/1981613095 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78280ba2e0 con 0x56232423ac30 2024-09-10T07:50:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.548+0000 7f7827e00640 1 -- 172.21.15.46:0/1981613095 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782809b010 con 0x56232423ac30 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.548+0000 7f7827e00640 1 -- 172.21.15.46:0/1981613095 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828018d90 con 0x56232423ac30 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 -- 172.21.15.46:0/1981613095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 msgr2=0x562324241250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 --2- 172.21.15.46:0/1981613095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x562324241250 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f782809bb50 tx=0x7f782800cf20 comp rx=0 tx=0).stop 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 -- 172.21.15.46:0/1981613095 shutdown_connections 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 --2- 172.21.15.46:0/1981613095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623242454f0 0x56232425cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 --2- 172.21.15.46:0/1981613095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x562324241250 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 --2- 172.21.15.46:0/1981613095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56232423a310 0x56232423cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 -- 172.21.15.46:0/1981613095 >> 172.21.15.46:0/1981613095 conn(0x5623242368a0 msgr2=0x562324240b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 -- 172.21.15.46:0/1981613095 shutdown_connections 2024-09-10T07:50:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 -- 172.21.15.46:0/1981613095 wait complete. 2024-09-10T07:50:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.549+0000 7f784526c740 1 -- start start 2024-09-10T07:50:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.550+0000 7f784526c740 1 Processor -- start 2024-09-10T07:50:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.550+0000 7f784526c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x56232424d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.550+0000 7f784526c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x56232424dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.550+0000 7f784526c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.550+0000 7f784526c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b2280 con 0x56232423a310 2024-09-10T07:50:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.550+0000 7f784526c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b1b20 con 0x56232423ac30 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.550+0000 7f784526c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1aed60 con 0x5623242454f0 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.550+0000 7f784526c740 10 client.0 _ll_register_callbacks cb 0x7f7828009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x56232424d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x56232424dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x56232424dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33162/0 (socket says 172.21.15.46:33162) 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x56232424d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57612/0 (socket says 172.21.15.46:57612) 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 -- 172.21.15.46:0/1244982298 learned_addr learned my addr 172.21.15.46:0/1244982298 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783be00640 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 -- 172.21.15.46:0/1244982298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 msgr2=0x5623242499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 -- 172.21.15.46:0/1244982298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 msgr2=0x56232424d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x56232424d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 -- 172.21.15.46:0/1244982298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562324237530 con 0x56232423ac30 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783be00640 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f783aa00640 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x56232424dac0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7828099e70 tx=0x7f782800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f0e2a4f99c4dee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.551+0000 7f7825600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.552+0000 7f7825600640 1 -- 172.21.15.46:0/1244982298 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828018d30 con 0x56232423ac30 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.552+0000 7f7825600640 1 -- 172.21.15.46:0/1244982298 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f7828010430 con 0x56232423ac30 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.552+0000 7f784526c740 1 -- 172.21.15.46:0/1244982298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f782809b010 con 0x56232423ac30 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.552+0000 7f7825600640 1 -- 172.21.15.46:0/1244982298 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f782809b330 con 0x56232423ac30 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.552+0000 7f784526c740 10 client.28832 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.552+0000 7f784526c740 1 -- 172.21.15.46:0/1244982298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562324237530 con 0x56232423ac30 2024-09-10T07:50:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.552+0000 7f7825600640 1 -- 172.21.15.46:0/1244982298 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782809b010 con 0x56232423ac30 2024-09-10T07:50:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.553+0000 7f7825600640 1 -- 172.21.15.46:0/1244982298 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f78280861f0 con 0x56232423ac30 2024-09-10T07:50:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.553+0000 7f784526c740 10 client.28832 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.553+0000 7f784526c740 10 client.28832 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:50:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.553+0000 7f784526c740 20 client.28832 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.553+0000 7f784526c740 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5623242c0960 0x5623242c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.553+0000 7f784526c740 4 client.28832 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ec6517", "release": true, "await": true}] 2024-09-10T07:50:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.553+0000 7f784526c740 1 -- 172.21.15.46:0/1244982298 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ec6517", "release": true, "await": true}) -- 0x56232423dfc0 con 0x5623242c0960 2024-09-10T07:50:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.553+0000 7f783b400640 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5623242c0960 0x5623242c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.554+0000 7f7825600640 10 client.28832 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.554+0000 7f783b400640 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5623242c0960 0x5623242c2d80 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a26abb3d90d27c server_cookie=16f5ab8759ca44df in_seq=0 out_seq=0 2024-09-10T07:50:30.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.585+0000 7f7825600640 1 -- 172.21.15.46:0/1244982298 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56232423dfc0 con 0x5623242c0960 2024-09-10T07:50:30.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.585+0000 7f7825600640 10 client.28832 handle_command_reply: tid=0 2024-09-10T07:50:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.585+0000 7f784526c740 1 client.28832 shutdown 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f784526c740 1 -- 172.21.15.46:0/1244982298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 msgr2=0x56232424dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f784526c740 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x56232424dac0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7828099e70 tx=0x7f782800cf90 comp rx=0 tx=0).stop 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f784526c740 1 -- 172.21.15.46:0/1244982298 shutdown_connections 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f784526c740 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5623242c0960 0x5623242c2d80 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f784526c740 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5623242454f0 0x5623242499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f784526c740 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56232423ac30 0x56232424dac0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f7825600640 0 client.28832 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f784526c740 1 --2- 172.21.15.46:0/1244982298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56232423a310 0x56232424d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.587+0000 7f784526c740 1 -- 172.21.15.46:0/1244982298 >> 172.21.15.46:0/1244982298 conn(0x5623242368a0 msgr2=0x562324242b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.588+0000 7f784526c740 1 -- 172.21.15.46:0/1244982298 shutdown_connections 2024-09-10T07:50:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.588+0000 7f784526c740 1 -- 172.21.15.46:0/1244982298 wait complete. 2024-09-10T07:50:30.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.589+0000 7f784526c740 20 client.28832 trim_cache size 0 max 16384 2024-09-10T07:50:30.590 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:50:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 54, 2024-09-10T07:50:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 52, 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "ec6517": { 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 52, 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 141.5, 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:50:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:50:30.592 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.590+0000 7f7842800640 1 -- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78200a2c00 msgr2=0x7f78200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.590+0000 7f7842800640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78200a2c00 0x7f78200a50c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x562324237700 tx=0x7f782c031510 comp rx=0 tx=0).stop 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.590+0000 7f7842800640 1 -- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c1049a0 msgr2=0x7f783c19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.590+0000 7f7842800640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c1049a0 0x7f783c19ac20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f782800e9f0 tx=0x7f782800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.591+0000 7f7842800640 1 -- 172.21.15.46:0/2395272999 shutdown_connections 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.591+0000 7f7842800640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c109120 0x7f783c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.591+0000 7f7842800640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f78200a2c00 0x7f78200a50c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.591+0000 7f7842800640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f783c1049a0 0x7f783c19ac20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.591+0000 7f7842800640 1 --2- 172.21.15.46:0/2395272999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f783c103ff0 0x7f783c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.591+0000 7f7842800640 1 -- 172.21.15.46:0/2395272999 >> 172.21.15.46:0/2395272999 conn(0x7f783c0ffa90 msgr2=0x7f783c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.591+0000 7f7842800640 1 -- 172.21.15.46:0/2395272999 shutdown_connections 2024-09-10T07:50:30.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:30.591+0000 7f7842800640 1 -- 172.21.15.46:0/2395272999 wait complete. 2024-09-10T07:50:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 790.0 s, 6.8 tps, lat 18476.185 ms stddev 3221.910 2024-09-10T07:50:32.030 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ec6517', total seconds elapsed: 26.4 2024-09-10T07:50:32.078 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:32.263 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:32 smithi046 ceph-mon[30705]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T07:50:32.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:32 smithi181 ceph-mon[35397]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T07:50:32.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:32 smithi063 ceph-mon[35020]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T07:50:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.864+0000 7fb313400640 1 -- 172.21.15.46:0/849508331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 msgr2=0x7fb30c1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.864+0000 7fb313400640 1 --2- 172.21.15.46:0/849508331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 0x7fb30c1095d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb300009860 tx=0x7fb30002a340 comp rx=0 tx=0).stop 2024-09-10T07:50:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.865+0000 7fb313400640 1 -- 172.21.15.46:0/849508331 shutdown_connections 2024-09-10T07:50:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.865+0000 7fb313400640 1 --2- 172.21.15.46:0/849508331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c116ea0 0x7fb30c119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.865+0000 7fb313400640 1 --2- 172.21.15.46:0/849508331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c109b10 0x7fb30c116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.865+0000 7fb313400640 1 --2- 172.21.15.46:0/849508331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 0x7fb30c1095d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.865+0000 7fb313400640 1 -- 172.21.15.46:0/849508331 >> 172.21.15.46:0/849508331 conn(0x7fb30c06e2d0 msgr2=0x7fb30c06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.865+0000 7fb313400640 1 -- 172.21.15.46:0/849508331 shutdown_connections 2024-09-10T07:50:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.865+0000 7fb313400640 1 -- 172.21.15.46:0/849508331 wait complete. 2024-09-10T07:50:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.867+0000 7fb313400640 1 Processor -- start 2024-09-10T07:50:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.868+0000 7fb313400640 1 -- start start 2024-09-10T07:50:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.868+0000 7fb313400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 0x7fb30c10c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.868+0000 7fb313400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c109b10 0x7fb30c10cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.868+0000 7fb313400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c116ea0 0x7fb30c10d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.869+0000 7fb310a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 0x7fb30c10c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.869+0000 7fb310a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 0x7fb30c10c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33176/0 (socket says 172.21.15.46:33176) 2024-09-10T07:50:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.869+0000 7fb311400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c116ea0 0x7fb30c10d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.869+0000 7fb311400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c116ea0 0x7fb30c10d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60060/0 (socket says 172.21.15.46:60060) 2024-09-10T07:50:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.869+0000 7fb311400640 1 -- 172.21.15.46:0/708134346 learned_addr learned my addr 172.21.15.46:0/708134346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.870+0000 7fb30be00640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c109b10 0x7fb30c10cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.870+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c11c9d0 con 0x7fb30c109b10 2024-09-10T07:50:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.870+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c11c850 con 0x7fb30c1091f0 2024-09-10T07:50:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.870+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c11cb50 con 0x7fb30c116ea0 2024-09-10T07:50:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.871+0000 7fb311400640 1 -- 172.21.15.46:0/708134346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 msgr2=0x7fb30c10c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.871+0000 7fb311400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 0x7fb30c10c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.871+0000 7fb311400640 1 -- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c109b10 msgr2=0x7fb30c10cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.871+0000 7fb311400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c109b10 0x7fb30c10cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.871+0000 7fb311400640 1 -- 172.21.15.46:0/708134346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c1b3570 con 0x7fb30c116ea0 2024-09-10T07:50:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.873+0000 7fb30be00640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c109b10 0x7fb30c10cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.873+0000 7fb311400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c116ea0 0x7fb30c10d060 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c420 tx=0x7fb30400c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.877+0000 7fb309600640 1 -- 172.21.15.46:0/708134346 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304003970 con 0x7fb30c116ea0 2024-09-10T07:50:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.878+0000 7fb309600640 1 -- 172.21.15.46:0/708134346 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30400b690 con 0x7fb30c116ea0 2024-09-10T07:50:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.878+0000 7fb309600640 1 -- 172.21.15.46:0/708134346 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304004420 con 0x7fb30c116ea0 2024-09-10T07:50:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.878+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c1b3860 con 0x7fb30c116ea0 2024-09-10T07:50:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.878+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1b3da0 con 0x7fb30c116ea0 2024-09-10T07:50:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.891+0000 7fb309600640 1 -- 172.21.15.46:0/708134346 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb304004600 con 0x7fb30c116ea0 2024-09-10T07:50:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.897+0000 7fb315ec8740 1 Processor -- start 2024-09-10T07:50:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.897+0000 7fb309600640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2f40a2cd0 0x7fb2f40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.898+0000 7fb309600640 1 -- 172.21.15.46:0/708134346 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb3040d0540 con 0x7fb30c116ea0 2024-09-10T07:50:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.899+0000 7fb30be00640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2f40a2cd0 0x7fb2f40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.900+0000 7fb315ec8740 1 -- start start 2024-09-10T07:50:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.900+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.900+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.900+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.900+0000 7fb315ec8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5645f91233e0 con 0x5645f91c7d70 2024-09-10T07:50:32.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.900+0000 7fb315ec8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5645f90cad80 con 0x5645f91bd4b0 2024-09-10T07:50:32.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.900+0000 7fb315ec8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5645f911f3a0 con 0x5645f91bcb90 2024-09-10T07:50:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.904+0000 7fb310a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.904+0000 7fb310a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91bcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60074/0 (socket says 172.21.15.46:60074) 2024-09-10T07:50:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.904+0000 7fb310a00640 1 -- 172.21.15.46:0/1000742659 learned_addr learned my addr 172.21.15.46:0/1000742659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.905+0000 7fb30be00640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2f40a2cd0 0x7fb2f40a5190 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb30c10dda0 tx=0x7fb2f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.905+0000 7fb30be00640 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.905+0000 7fb311400640 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.905+0000 7fb310a00640 1 -- 172.21.15.46:0/1000742659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 msgr2=0x5645f91c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.905+0000 7fb310a00640 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.905+0000 7fb310a00640 1 -- 172.21.15.46:0/1000742659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 msgr2=0x5645f91ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.905+0000 7fb310a00640 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.905+0000 7fb310a00640 1 -- 172.21.15.46:0/1000742659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f90e9ae0 con 0x5645f91bcb90 2024-09-10T07:50:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.906+0000 7fb30be00640 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.907+0000 7fb310a00640 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91bcf70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb300009780 tx=0x7fb300009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb43c977d1dd805 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.907+0000 7fb2fdc00640 1 -- 172.21.15.46:0/1000742659 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300041020 con 0x5645f91bcb90 2024-09-10T07:50:32.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.908+0000 7fb2fdc00640 1 -- 172.21.15.46:0/1000742659 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb300030e50 con 0x5645f91bcb90 2024-09-10T07:50:32.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.908+0000 7fb2fdc00640 1 -- 172.21.15.46:0/1000742659 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300037040 con 0x5645f91bcb90 2024-09-10T07:50:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.909+0000 7fb315ec8740 1 -- 172.21.15.46:0/1000742659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 msgr2=0x5645f91bcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.909+0000 7fb315ec8740 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91bcf70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb300009780 tx=0x7fb300009510 comp rx=0 tx=0).stop 2024-09-10T07:50:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.909+0000 7fb315ec8740 1 -- 172.21.15.46:0/1000742659 shutdown_connections 2024-09-10T07:50:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.909+0000 7fb315ec8740 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.909+0000 7fb315ec8740 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.909+0000 7fb315ec8740 1 --2- 172.21.15.46:0/1000742659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91bcf70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.909+0000 7fb315ec8740 1 -- 172.21.15.46:0/1000742659 >> 172.21.15.46:0/1000742659 conn(0x5645f91b9120 msgr2=0x5645f91b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.910+0000 7fb315ec8740 1 -- 172.21.15.46:0/1000742659 shutdown_connections 2024-09-10T07:50:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.910+0000 7fb315ec8740 1 -- 172.21.15.46:0/1000742659 wait complete. 2024-09-10T07:50:32.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.910+0000 7fb315ec8740 1 -- start start 2024-09-10T07:50:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.911+0000 7fb315ec8740 1 Processor -- start 2024-09-10T07:50:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.914+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.914+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.914+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.914+0000 7fb315ec8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5645f91ca9e0 con 0x5645f91c7d70 2024-09-10T07:50:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.914+0000 7fb315ec8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5645f90de0f0 con 0x5645f91bd4b0 2024-09-10T07:50:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.914+0000 7fb315ec8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5645f91cab60 con 0x5645f91bcb90 2024-09-10T07:50:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.914+0000 7fb315ec8740 10 client.0 _ll_register_callbacks cb 0x5645f90cf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.916+0000 7fb311400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.916+0000 7fb310a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.916+0000 7fb310a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60080/0 (socket says 172.21.15.46:60080) 2024-09-10T07:50:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.916+0000 7fb310a00640 1 -- 172.21.15.46:0/3815646003 learned_addr learned my addr 172.21.15.46:0/3815646003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.917+0000 7fb30be00640 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.917+0000 7fb311400640 1 -- 172.21.15.46:0/3815646003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 msgr2=0x5645f91c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.917+0000 7fb311400640 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.917+0000 7fb311400640 1 -- 172.21.15.46:0/3815646003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 msgr2=0x5645f91c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.917+0000 7fb311400640 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.917+0000 7fb311400640 1 -- 172.21.15.46:0/3815646003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f8e943c0 con 0x5645f91c7d70 2024-09-10T07:50:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.917+0000 7fb30be00640 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.917+0000 7fb310a00640 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.920+0000 7fb311400640 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c5620 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb3040cdc90 tx=0x7fb3040099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59299ad0106ba72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.920+0000 7fb2f2000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.922+0000 7fb2f2000640 1 -- 172.21.15.46:0/3815646003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3040b4020 con 0x5645f91c7d70 2024-09-10T07:50:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.923+0000 7fb2f2000640 1 -- 172.21.15.46:0/3815646003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb30400b840 con 0x5645f91c7d70 2024-09-10T07:50:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.923+0000 7fb315ec8740 1 -- 172.21.15.46:0/3815646003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb300030e50 con 0x5645f91c7d70 2024-09-10T07:50:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.923+0000 7fb2f2000640 1 -- 172.21.15.46:0/3815646003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb3040a0dd0 con 0x5645f91c7d70 2024-09-10T07:50:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.923+0000 7fb315ec8740 10 client.18018 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:32.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.923+0000 7fb315ec8740 1 -- 172.21.15.46:0/3815646003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5645f91d6670 con 0x5645f91c7d70 2024-09-10T07:50:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.924+0000 7fb2f2000640 1 -- 172.21.15.46:0/3815646003 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb300030e50 con 0x5645f91c7d70 2024-09-10T07:50:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.925+0000 7fb2f2000640 1 -- 172.21.15.46:0/3815646003 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fb30400b040 con 0x5645f91c7d70 2024-09-10T07:50:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.925+0000 7fb315ec8740 10 client.18018 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.925+0000 7fb315ec8740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.925+0000 7fb315ec8740 20 client.18018 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.925+0000 7fb315ec8740 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645f91d7a10 0x5645f91d9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.925+0000 7fb315ec8740 4 client.18018 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.925+0000 7fb315ec8740 1 -- 172.21.15.46:0/3815646003 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645f9130950 con 0x5645f91d7a10 2024-09-10T07:50:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.926+0000 7fb310a00640 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645f91d7a10 0x5645f91d9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.930+0000 7fb310a00640 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645f91d7a10 0x5645f91d9eb0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257ce06b3cf049ae server_cookie=f50ef922ffd4ba9f in_seq=0 out_seq=0 2024-09-10T07:50:32.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.930+0000 7fb2f2000640 10 client.18018 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.934+0000 7fb2f2000640 1 -- 172.21.15.46:0/3815646003 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645f9130950 con 0x5645f91d7a10 2024-09-10T07:50:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.934+0000 7fb2f2000640 10 client.18018 handle_command_reply: tid=0 2024-09-10T07:50:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.934+0000 7fb315ec8740 1 client.18018 shutdown 2024-09-10T07:50:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.934+0000 7fb315ec8740 1 -- 172.21.15.46:0/3815646003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 msgr2=0x5645f91c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.934+0000 7fb315ec8740 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c5620 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb3040cdc90 tx=0x7fb3040099a0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb315ec8740 1 -- 172.21.15.46:0/3815646003 shutdown_connections 2024-09-10T07:50:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb315ec8740 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645f91d7a10 0x5645f91d9eb0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb315ec8740 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c5620 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb315ec8740 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb315ec8740 1 --2- 172.21.15.46:0/3815646003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb315ec8740 1 -- 172.21.15.46:0/3815646003 >> 172.21.15.46:0/3815646003 conn(0x5645f91b9120 msgr2=0x5645f91bee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb2f2000640 0 client.18018 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb315ec8740 1 -- 172.21.15.46:0/3815646003 shutdown_connections 2024-09-10T07:50:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.935+0000 7fb315ec8740 1 -- 172.21.15.46:0/3815646003 wait complete. 2024-09-10T07:50:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.936+0000 7fb315ec8740 20 client.18018 trim_cache size 0 max 16384 2024-09-10T07:50:32.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.977+0000 7fb315ec8740 1 Processor -- start 2024-09-10T07:50:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.977+0000 7fb315ec8740 1 -- start start 2024-09-10T07:50:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.977+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91bcb90 0x5645f91bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.977+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.977+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91c7d70 0x5645f91dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.977+0000 7fb315ec8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1bda10 con 0x5645f91bcb90 2024-09-10T07:50:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.977+0000 7fb315ec8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1bf5e0 con 0x5645f91bd4b0 2024-09-10T07:50:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.977+0000 7fb315ec8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1be0e0 con 0x5645f91c7d70 2024-09-10T07:50:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.978+0000 7fb310a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91bcb90 0x5645f91bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.978+0000 7fb311400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91c7d70 0x5645f91dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.978+0000 7fb311400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91c7d70 0x5645f91dfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:60086/0 (socket says 172.21.15.46:60086) 2024-09-10T07:50:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.978+0000 7fb311400640 1 -- 172.21.15.46:0/4081359699 learned_addr learned my addr 172.21.15.46:0/4081359699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.978+0000 7fb30be00640 1 --2- 172.21.15.46:0/4081359699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.979+0000 7fb311400640 1 -- 172.21.15.46:0/4081359699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 msgr2=0x5645f91df780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.979+0000 7fb311400640 1 --2- 172.21.15.46:0/4081359699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91df780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.979+0000 7fb311400640 1 -- 172.21.15.46:0/4081359699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91bcb90 msgr2=0x5645f91bf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.979+0000 7fb311400640 1 --2- 172.21.15.46:0/4081359699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91bcb90 0x5645f91bf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.979+0000 7fb311400640 1 -- 172.21.15.46:0/4081359699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f91b8f50 con 0x5645f91c7d70 2024-09-10T07:50:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.980+0000 7fb311400640 1 --2- 172.21.15.46:0/4081359699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91c7d70 0x5645f91dfcc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb30409c9b0 tx=0x7fb30409af30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f50971d2aa0dd56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.980+0000 7fb2fd200640 1 -- 172.21.15.46:0/4081359699 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3040b4070 con 0x5645f91c7d70 2024-09-10T07:50:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.980+0000 7fb2fd200640 1 -- 172.21.15.46:0/4081359699 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3040cf040 con 0x5645f91c7d70 2024-09-10T07:50:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.980+0000 7fb2fd200640 1 -- 172.21.15.46:0/4081359699 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30400b840 con 0x5645f91c7d70 2024-09-10T07:50:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.981+0000 7fb315ec8740 1 -- 172.21.15.46:0/4081359699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91c7d70 msgr2=0x5645f91dfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.981+0000 7fb315ec8740 1 --2- 172.21.15.46:0/4081359699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91c7d70 0x5645f91dfcc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb30409c9b0 tx=0x7fb30409af30 comp rx=0 tx=0).stop 2024-09-10T07:50:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.982+0000 7fb315ec8740 1 -- 172.21.15.46:0/4081359699 shutdown_connections 2024-09-10T07:50:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.982+0000 7fb315ec8740 1 --2- 172.21.15.46:0/4081359699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91c7d70 0x5645f91dfcc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.982+0000 7fb315ec8740 1 --2- 172.21.15.46:0/4081359699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.982+0000 7fb315ec8740 1 --2- 172.21.15.46:0/4081359699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91bcb90 0x5645f91bf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.982+0000 7fb315ec8740 1 -- 172.21.15.46:0/4081359699 >> 172.21.15.46:0/4081359699 conn(0x5645f91b9120 msgr2=0x5645f91c3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.982+0000 7fb315ec8740 1 -- 172.21.15.46:0/4081359699 shutdown_connections 2024-09-10T07:50:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.982+0000 7fb315ec8740 1 -- 172.21.15.46:0/4081359699 wait complete. 2024-09-10T07:50:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.982+0000 7fb315ec8740 1 -- start start 2024-09-10T07:50:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.983+0000 7fb315ec8740 1 Processor -- start 2024-09-10T07:50:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.984+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.984+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb315ec8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb310a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb315ec8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1bef10 con 0x5645f91c7d70 2024-09-10T07:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb315ec8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1be7b0 con 0x5645f91bd4b0 2024-09-10T07:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb315ec8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1bb9f0 con 0x5645f91bcb90 2024-09-10T07:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb30be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb30be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33208/0 (socket says 172.21.15.46:33208) 2024-09-10T07:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb315ec8740 10 client.0 _ll_register_callbacks cb 0x7fb30409c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.985+0000 7fb30be00640 1 -- 172.21.15.46:0/2292399243 learned_addr learned my addr 172.21.15.46:0/2292399243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.986+0000 7fb310a00640 1 -- 172.21.15.46:0/2292399243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 msgr2=0x5645f91c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.986+0000 7fb310a00640 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.986+0000 7fb310a00640 1 -- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 msgr2=0x5645f91c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.986+0000 7fb311400640 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.986+0000 7fb310a00640 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.986+0000 7fb310a00640 1 -- 172.21.15.46:0/2292399243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f91b9db0 con 0x5645f91bcb90 2024-09-10T07:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.986+0000 7fb311400640 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.987+0000 7fb310a00640 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91d05b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5645f91d7df0 tx=0x7fb300000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb164c181d65a7be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.987+0000 7fb2f2000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.987+0000 7fb2f2000640 1 -- 172.21.15.46:0/2292399243 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300041040 con 0x5645f91bcb90 2024-09-10T07:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.988+0000 7fb2f2000640 1 -- 172.21.15.46:0/2292399243 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb300004340 con 0x5645f91bcb90 2024-09-10T07:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.988+0000 7fb315ec8740 1 -- 172.21.15.46:0/2292399243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3040cf040 con 0x5645f91bcb90 2024-09-10T07:50:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.989+0000 7fb2f2000640 1 -- 172.21.15.46:0/2292399243 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb300037040 con 0x5645f91bcb90 2024-09-10T07:50:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.989+0000 7fb315ec8740 10 client.28753 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.989+0000 7fb315ec8740 1 -- 172.21.15.46:0/2292399243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5645f9242750 con 0x5645f91bcb90 2024-09-10T07:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.990+0000 7fb2f2000640 1 -- 172.21.15.46:0/2292399243 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3040cf040 con 0x5645f91bcb90 2024-09-10T07:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.991+0000 7fb2f2000640 1 -- 172.21.15.46:0/2292399243 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fb300050030 con 0x5645f91bcb90 2024-09-10T07:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.991+0000 7fb315ec8740 10 client.28753 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.991+0000 7fb315ec8740 10 client.28753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.991+0000 7fb315ec8740 20 client.28753 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.991+0000 7fb315ec8740 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645f9242920 0x5645f9244d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.991+0000 7fb315ec8740 4 client.28753 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.991+0000 7fb315ec8740 1 -- 172.21.15.46:0/2292399243 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645f912ae60 con 0x5645f9242920 2024-09-10T07:50:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.991+0000 7fb30be00640 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645f9242920 0x5645f9244d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.994+0000 7fb30be00640 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645f9242920 0x5645f9244d10 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b338f47df3af905 server_cookie=e36ca87a57a31e6 in_seq=0 out_seq=0 2024-09-10T07:50:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.994+0000 7fb2f2000640 10 client.28753 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb2f2000640 1 -- 172.21.15.46:0/2292399243 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5645f912ae60 con 0x5645f9242920 2024-09-10T07:50:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb2f2000640 10 client.28753 handle_command_reply: tid=0 2024-09-10T07:50:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 client.28753 shutdown 2024-09-10T07:50:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 -- 172.21.15.46:0/2292399243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 msgr2=0x5645f91d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91d05b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5645f91d7df0 tx=0x7fb300000bc0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 -- 172.21.15.46:0/2292399243 shutdown_connections 2024-09-10T07:50:32.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645f9242920 0x5645f9244d10 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f91c7d70 0x5645f91c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645f91bd4b0 0x5645f91c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 --2- 172.21.15.46:0/2292399243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645f91bcb90 0x5645f91d05b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb315ec8740 1 -- 172.21.15.46:0/2292399243 >> 172.21.15.46:0/2292399243 conn(0x5645f91b9120 msgr2=0x5645f91c4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.995+0000 7fb2f2000640 0 client.28753 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.998+0000 7fb315ec8740 1 -- 172.21.15.46:0/2292399243 shutdown_connections 2024-09-10T07:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.998+0000 7fb315ec8740 1 -- 172.21.15.46:0/2292399243 wait complete. 2024-09-10T07:50:32.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:32.998+0000 7fb315ec8740 20 client.28753 trim_cache size 0 max 16384 2024-09-10T07:50:33.001 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:50:33.001 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:50:33.001 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:50:33.001 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:50:33.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.004+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2f40a2cd0 msgr2=0x7fb2f40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.004+0000 7fb313400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2f40a2cd0 0x7fb2f40a5190 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb30c10dda0 tx=0x7fb2f8005f90 comp rx=0 tx=0).stop 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.004+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c116ea0 msgr2=0x7fb30c10d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.004+0000 7fb313400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c116ea0 0x7fb30c10d060 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c420 tx=0x7fb30400c8e0 comp rx=0 tx=0).stop 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.005+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 shutdown_connections 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.005+0000 7fb313400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c116ea0 0x7fb30c10d060 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.005+0000 7fb313400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb2f40a2cd0 0x7fb2f40a5190 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.005+0000 7fb313400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb30c109b10 0x7fb30c10cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.005+0000 7fb313400640 1 --2- 172.21.15.46:0/708134346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb30c1091f0 0x7fb30c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.005+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 >> 172.21.15.46:0/708134346 conn(0x7fb30c06e2d0 msgr2=0x7fb30c1173c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.006+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 shutdown_connections 2024-09-10T07:50:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:33.006+0000 7fb313400640 1 -- 172.21.15.46:0/708134346 wait complete. 2024-09-10T07:50:33.903 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:50:33.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:50:34.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:34 smithi046 ceph-mon[30705]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 318 KiB/s wr, 51 op/s 2024-09-10T07:50:34.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:34 smithi181 ceph-mon[35397]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 318 KiB/s wr, 51 op/s 2024-09-10T07:50:34.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:34 smithi063 ceph-mon[35020]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 318 KiB/s wr, 51 op/s 2024-09-10T07:50:34.601 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.477+0000 7ff4bee00640 1 -- 172.21.15.46:0/336635179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 msgr2=0x7ff4b81067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.477+0000 7ff4b6a00640 1 -- 172.21.15.46:0/336635179 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8035020 con 0x7ff4b8102780 2024-09-10T07:50:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.477+0000 7ff4bee00640 1 --2- 172.21.15.46:0/336635179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 0x7ff4b81067d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009900 tx=0x7ff4a802a410 comp rx=0 tx=0).stop 2024-09-10T07:50:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.478+0000 7ff4bee00640 1 -- 172.21.15.46:0/336635179 shutdown_connections 2024-09-10T07:50:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.478+0000 7ff4bee00640 1 --2- 172.21.15.46:0/336635179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8106f00 0x7ff4b810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.478+0000 7ff4bee00640 1 --2- 172.21.15.46:0/336635179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 0x7ff4b81067d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.478+0000 7ff4bee00640 1 --2- 172.21.15.46:0/336635179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b8101dd0 0x7ff4b81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.478+0000 7ff4bee00640 1 -- 172.21.15.46:0/336635179 >> 172.21.15.46:0/336635179 conn(0x7ff4b80fd890 msgr2=0x7ff4b80ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.478+0000 7ff4bee00640 1 -- 172.21.15.46:0/336635179 shutdown_connections 2024-09-10T07:50:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.478+0000 7ff4bee00640 1 -- 172.21.15.46:0/336635179 wait complete. 2024-09-10T07:50:35.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.480+0000 7ff4bee00640 1 Processor -- start 2024-09-10T07:50:35.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.482+0000 7ff4bee00640 1 -- start start 2024-09-10T07:50:35.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.483+0000 7ff4bee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101dd0 0x7ff4b819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.483+0000 7ff4bee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 0x7ff4b819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.483+0000 7ff4b7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 0x7ff4b819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.483+0000 7ff4b7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 0x7ff4b819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:33232/0 (socket says 172.21.15.46:33232) 2024-09-10T07:50:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.484+0000 7ff4b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101dd0 0x7ff4b819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:35.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.484+0000 7ff4b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101dd0 0x7ff4b819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57694/0 (socket says 172.21.15.46:57694) 2024-09-10T07:50:35.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.483+0000 7ff4bee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b8106f00 0x7ff4b819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:35.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.484+0000 7ff4b7400640 1 -- 172.21.15.46:0/241486845 learned_addr learned my addr 172.21.15.46:0/241486845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:35.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.484+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b8112070 con 0x7ff4b8101dd0 2024-09-10T07:50:35.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.484+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b8111ef0 con 0x7ff4b8102780 2024-09-10T07:50:35.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.484+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81121f0 con 0x7ff4b8106f00 2024-09-10T07:50:35.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.485+0000 7ff4b7400640 1 -- 172.21.15.46:0/241486845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b8106f00 msgr2=0x7ff4b819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:35.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.485+0000 7ff4b7400640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b8106f00 0x7ff4b819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.485+0000 7ff4b7400640 1 -- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101dd0 msgr2=0x7ff4b819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:35.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.485+0000 7ff4b7400640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101dd0 0x7ff4b819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.485+0000 7ff4b7400640 1 -- 172.21.15.46:0/241486845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b819f580 con 0x7ff4b8102780 2024-09-10T07:50:35.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.486+0000 7ff4b7400640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 0x7ff4b819ab10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005ab0 tx=0x7ff4a8030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:35.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.487+0000 7ff4b4c00640 1 -- 172.21.15.46:0/241486845 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a803f030 con 0x7ff4b8102780 2024-09-10T07:50:35.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.487+0000 7ff4b4c00640 1 -- 172.21.15.46:0/241486845 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a803b460 con 0x7ff4b8102780 2024-09-10T07:50:35.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.487+0000 7ff4b4c00640 1 -- 172.21.15.46:0/241486845 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8035070 con 0x7ff4b8102780 2024-09-10T07:50:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.487+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b819f810 con 0x7ff4b8102780 2024-09-10T07:50:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.488+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b819fca0 con 0x7ff4b8102780 2024-09-10T07:50:35.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.496+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b819b930 con 0x7ff4b8102780 2024-09-10T07:50:35.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.496+0000 7ff4b4c00640 1 -- 172.21.15.46:0/241486845 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4a8042030 con 0x7ff4b8102780 2024-09-10T07:50:35.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.500+0000 7ff4b4c00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff49c0a2d20 0x7ff49c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:35.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.501+0000 7ff4b4c00640 1 -- 172.21.15.46:0/241486845 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff4a80f3060 con 0x7ff4b8102780 2024-09-10T07:50:35.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.501+0000 7ff4b7e00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff49c0a2d20 0x7ff49c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:35.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.502+0000 7ff4b7e00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff49c0a2d20 0x7ff49c0a51e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009990 tx=0x7ff4a4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:35.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.506+0000 7ff4b4c00640 1 -- 172.21.15.46:0/241486845 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4a80bbca0 con 0x7ff4b8102780 2024-09-10T07:50:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 795.0 s, 445.2 tps, lat 789.514 ms stddev 3272.522 2024-09-10T07:50:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.954+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4b81a7430 con 0x7ff4b8102780 2024-09-10T07:50:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.955+0000 7ff4b4c00640 1 -- 172.21.15.46:0/241486845 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11977 (secure 0 0 0) 0x7ff4a80bb5a0 con 0x7ff4b8102780 2024-09-10T07:50:35.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:50:35.962 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":54,"btime":"2024-09-10T07:49:48:222949+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:49:48.058024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:50:35.963 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 54 2024-09-10T07:50:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff49c0a2d20 msgr2=0x7ff49c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff49c0a2d20 0x7ff49c0a51e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009990 tx=0x7ff4a4009110 comp rx=0 tx=0).stop 2024-09-10T07:50:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 msgr2=0x7ff4b819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 0x7ff4b819ab10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005ab0 tx=0x7ff4a8030650 comp rx=0 tx=0).stop 2024-09-10T07:50:35.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 shutdown_connections 2024-09-10T07:50:35.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b8106f00 0x7ff4b819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4b8102780 0x7ff4b819ab10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff49c0a2d20 0x7ff49c0a51e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 --2- 172.21.15.46:0/241486845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101dd0 0x7ff4b819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:35.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 >> 172.21.15.46:0/241486845 conn(0x7ff4b80fd890 msgr2=0x7ff4b80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:35.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 shutdown_connections 2024-09-10T07:50:35.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:35.965+0000 7ff4bee00640 1 -- 172.21.15.46:0/241486845 wait complete. 2024-09-10T07:50:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:36 smithi181 ceph-mon[35397]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 318 KiB/s wr, 49 op/s 2024-09-10T07:50:36.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:36 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/241486845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:36.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:36 smithi063 ceph-mon[35020]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 318 KiB/s wr, 49 op/s 2024-09-10T07:50:36.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:36 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/241486845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:36 smithi046 ceph-mon[30705]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 318 KiB/s wr, 49 op/s 2024-09-10T07:50:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:36 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/241486845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:50:38.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:38 smithi181 ceph-mon[35397]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-09-10T07:50:38.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:38 smithi063 ceph-mon[35020]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-09-10T07:50:38.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:38 smithi046 ceph-mon[30705]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-09-10T07:50:39.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:50:39.845 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:40.281 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:40 smithi046 ceph-mon[30705]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-09-10T07:50:40.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:40 smithi181 ceph-mon[35397]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-09-10T07:50:40.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:40 smithi063 ceph-mon[35020]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-09-10T07:50:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 800.0 s, 437.2 tps, lat 224.189 ms stddev 275.022 2024-09-10T07:50:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.971+0000 7f5a09000640 1 -- 172.21.15.46:0/2921651281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 msgr2=0x7f5a04101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.971+0000 7f5a09000640 1 --2- 172.21.15.46:0/2921651281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 0x7f5a04101eb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009900 tx=0x7f59ec02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.972+0000 7f5a09000640 1 -- 172.21.15.46:0/2921651281 shutdown_connections 2024-09-10T07:50:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.972+0000 7f5a09000640 1 --2- 172.21.15.46:0/2921651281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a041023f0 0x7f5a04108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.972+0000 7f5a09000640 1 --2- 172.21.15.46:0/2921651281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 0x7f5a04101eb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.972+0000 7f5a09000640 1 --2- 172.21.15.46:0/2921651281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a04100840 0x7f5a04100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.972+0000 7f5a09000640 1 -- 172.21.15.46:0/2921651281 >> 172.21.15.46:0/2921651281 conn(0x7f5a040fbfa0 msgr2=0x7f5a040fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.972+0000 7f5a09000640 1 -- 172.21.15.46:0/2921651281 shutdown_connections 2024-09-10T07:50:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.972+0000 7f5a09000640 1 -- 172.21.15.46:0/2921651281 wait complete. 2024-09-10T07:50:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.973+0000 7f5a09000640 1 Processor -- start 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.973+0000 7f5a09000640 1 -- start start 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.973+0000 7f5a09000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a04100840 0x7f5a041047d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.973+0000 7f5a09000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 0x7f5a04102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.973+0000 7f5a09000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a041023f0 0x7f5a04103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.973+0000 7f5a09000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0410c510 con 0x7f5a04100840 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.973+0000 7f5a09000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0410c390 con 0x7f5a041023f0 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.973+0000 7f5a09000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0410c690 con 0x7f5a04101a30 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 0x7f5a04102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a03400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a04100840 0x7f5a041047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 0x7f5a04102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48422/0 (socket says 172.21.15.46:48422) 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 -- 172.21.15.46:0/4220843330 learned_addr learned my addr 172.21.15.46:0/4220843330 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a03e00640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a041023f0 0x7f5a04103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 -- 172.21.15.46:0/4220843330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a041023f0 msgr2=0x7f5a04103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a041023f0 0x7f5a04103350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 -- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a04100840 msgr2=0x7f5a041047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a04100840 0x7f5a041047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 -- 172.21.15.46:0/4220843330 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a04103be0 con 0x7f5a04101a30 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.974+0000 7f5a02a00640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 0x7f5a04102e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f59ec031440 tx=0x7f59ec032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.975+0000 7f59fbe00640 1 -- 172.21.15.46:0/4220843330 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec03f030 con 0x7f5a04101a30 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.975+0000 7f59fbe00640 1 -- 172.21.15.46:0/4220843330 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ec003ce0 con 0x7f5a04101a30 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.975+0000 7f5a09000640 1 -- 172.21.15.46:0/4220843330 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a041a1df0 con 0x7f5a04101a30 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.975+0000 7f59fbe00640 1 -- 172.21.15.46:0/4220843330 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec035070 con 0x7f5a04101a30 2024-09-10T07:50:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.975+0000 7f5a09000640 1 -- 172.21.15.46:0/4220843330 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a041a2260 con 0x7f5a04101a30 2024-09-10T07:50:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.980+0000 7f59fbe00640 1 -- 172.21.15.46:0/4220843330 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f59ec042030 con 0x7f5a04101a30 2024-09-10T07:50:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.980+0000 7f5a0d49e740 1 Processor -- start 2024-09-10T07:50:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.981+0000 7f5a0d49e740 1 -- start start 2024-09-10T07:50:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.981+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d81e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.981+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d86e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.981+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d89900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.981+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bea8cde100 con 0x55bea8d87440 2024-09-10T07:50:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.981+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bea8c861b0 con 0x55bea8d823b0 2024-09-10T07:50:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.981+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bea8cda0c0 con 0x55bea8d81a70 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.981+0000 7f5a03400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d86e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d86e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56070/0 (socket says 172.21.15.46:56070) 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03400640 1 -- 172.21.15.46:0/2504081361 learned_addr learned my addr 172.21.15.46:0/2504081361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03e00640 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d81e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a02a00640 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d89900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03400640 1 -- 172.21.15.46:0/2504081361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 msgr2=0x55bea8d81e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03400640 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d81e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03400640 1 -- 172.21.15.46:0/2504081361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 msgr2=0x55bea8d89900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03400640 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d89900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03400640 1 -- 172.21.15.46:0/2504081361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea89f1210 con 0x55bea8d823b0 2024-09-10T07:50:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f59fbe00640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59e80a2c00 0x7f59e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f5a03400640 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d86e70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f59fc011470 tx=0x7f59fc011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71b18188b2cedb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.982+0000 7f59fbe00640 1 -- 172.21.15.46:0/4220843330 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f59ec0efaf0 con 0x7f5a04101a30 2024-09-10T07:50:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.983+0000 7f5a03e00640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59e80a2c00 0x7f59e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.983+0000 7f59f8c00640 1 -- 172.21.15.46:0/2504081361 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc003970 con 0x55bea8d823b0 2024-09-10T07:50:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.983+0000 7f59f8c00640 1 -- 172.21.15.46:0/2504081361 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59fc004020 con 0x55bea8d823b0 2024-09-10T07:50:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.983+0000 7f59f8c00640 1 -- 172.21.15.46:0/2504081361 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc003970 con 0x55bea8d823b0 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 -- 172.21.15.46:0/2504081361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 msgr2=0x55bea8d86e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d86e70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f59fc011470 tx=0x7f59fc011930 comp rx=0 tx=0).stop 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a03e00640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59e80a2c00 0x7f59e80a50c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009db0 tx=0x7f59f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 -- 172.21.15.46:0/2504081361 shutdown_connections 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d89900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d86e70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/2504081361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d81e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 -- 172.21.15.46:0/2504081361 >> 172.21.15.46:0/2504081361 conn(0x55bea8d7d0e0 msgr2=0x55bea8d7d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 -- 172.21.15.46:0/2504081361 shutdown_connections 2024-09-10T07:50:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.984+0000 7f5a0d49e740 1 -- 172.21.15.46:0/2504081361 wait complete. 2024-09-10T07:50:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.985+0000 7f5a0d49e740 1 -- start start 2024-09-10T07:50:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.986+0000 7f5a0d49e740 1 Processor -- start 2024-09-10T07:50:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d8bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d8c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a03400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d8bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a03400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d8bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48436/0 (socket says 172.21.15.46:48436) 2024-09-10T07:50:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bea8d8a410 con 0x55bea8d87440 2024-09-10T07:50:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bea8c67090 con 0x55bea8d823b0 2024-09-10T07:50:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bea8d8a590 con 0x55bea8d81a70 2024-09-10T07:50:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.987+0000 7f5a03400640 1 -- 172.21.15.46:0/485125492 learned_addr learned my addr 172.21.15.46:0/485125492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a0d49e740 10 client.0 _ll_register_callbacks cb 0x55bea8cbbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a03e00640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a02a00640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d8c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a03400640 1 -- 172.21.15.46:0/485125492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 msgr2=0x55bea8d8c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a03400640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d8c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a03400640 1 -- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 msgr2=0x55bea8d931a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a03400640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d931a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a03400640 1 -- 172.21.15.46:0/485125492 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea8abd620 con 0x55bea8d81a70 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a02a00640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d8c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.988+0000 7f5a03e00640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.989+0000 7f5a03400640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d8bc20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f59fc011440 tx=0x7f59fc004bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197e68418de4033c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.989+0000 7f59f1600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.989+0000 7f59f1600640 1 -- 172.21.15.46:0/485125492 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc028020 con 0x55bea8d81a70 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.989+0000 7f59f1600640 1 -- 172.21.15.46:0/485125492 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f59fc016040 con 0x55bea8d81a70 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.989+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59fc004020 con 0x55bea8d81a70 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.990+0000 7f59f1600640 1 -- 172.21.15.46:0/485125492 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f59fc003970 con 0x55bea8d81a70 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.990+0000 7f5a0d49e740 10 client.28765 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.990+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55bea8d95f70 con 0x55bea8d81a70 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.990+0000 7f59f1600640 1 -- 172.21.15.46:0/485125492 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59fc00f670 con 0x55bea8d81a70 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.990+0000 7f59f1600640 1 -- 172.21.15.46:0/485125492 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f59fc005040 con 0x55bea8d81a70 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.991+0000 7f5a0d49e740 10 client.28765 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.991+0000 7f5a0d49e740 10 client.28765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.991+0000 7f5a0d49e740 20 client.28765 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.991+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bea8d97a00 0x55bea8d99ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.991+0000 7f5a0d49e740 4 client.28765 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.991+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f59fc004020 con 0x55bea8d97a00 2024-09-10T07:50:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.991+0000 7f5a02a00640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bea8d97a00 0x55bea8d99ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.992+0000 7f5a02a00640 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bea8d97a00 0x55bea8d99ea0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9880674dc9fd2e85 server_cookie=ae7a8ab02466e709 in_seq=0 out_seq=0 2024-09-10T07:50:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.992+0000 7f59f1600640 10 client.28765 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f59f1600640 1 -- 172.21.15.46:0/485125492 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f59fc004020 con 0x55bea8d97a00 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f59f1600640 10 client.28765 handle_command_reply: tid=0 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 client.28765 shutdown 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 msgr2=0x55bea8d8bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d8bc20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f59fc011440 tx=0x7f59fc004bf0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 shutdown_connections 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bea8d97a00 0x55bea8d99ea0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d8c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f59f1600640 0 client.28765 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/485125492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d8bc20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 >> 172.21.15.46:0/485125492 conn(0x55bea8d7d0e0 msgr2=0x55bea8d7e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 shutdown_connections 2024-09-10T07:50:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 1 -- 172.21.15.46:0/485125492 wait complete. 2024-09-10T07:50:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:40.993+0000 7f5a0d49e740 20 client.28765 trim_cache size 0 max 16384 2024-09-10T07:50:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.029+0000 7f5a0d49e740 1 Processor -- start 2024-09-10T07:50:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.029+0000 7f5a0d49e740 1 -- start start 2024-09-10T07:50:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.030+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d93160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a03400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d7da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a041a8380 con 0x55bea8d87440 2024-09-10T07:50:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a041a9f50 con 0x55bea8d823b0 2024-09-10T07:50:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a041a8a50 con 0x55bea8d81a70 2024-09-10T07:50:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a03400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d7d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48452/0 (socket says 172.21.15.46:48452) 2024-09-10T07:50:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a03400640 1 -- 172.21.15.46:0/855657901 learned_addr learned my addr 172.21.15.46:0/855657901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.032+0000 7f5a03400640 1 -- 172.21.15.46:0/855657901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 msgr2=0x55bea8d93160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.031+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d93160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.032+0000 7f5a03400640 1 --2- 172.21.15.46:0/855657901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d93160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.032+0000 7f5a03400640 1 -- 172.21.15.46:0/855657901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 msgr2=0x55bea8d7da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.032+0000 7f5a03400640 1 --2- 172.21.15.46:0/855657901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d7da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.032+0000 7f5a03400640 1 -- 172.21.15.46:0/855657901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea8d82890 con 0x55bea8d81a70 2024-09-10T07:50:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.032+0000 7f5a02a00640 1 --2- 172.21.15.46:0/855657901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.033+0000 7f5a03400640 1 --2- 172.21.15.46:0/855657901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d7d4c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f59fc02f040 tx=0x7f59fc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eaaabdaf348887a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.034+0000 7f59f3e00640 1 -- 172.21.15.46:0/855657901 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc009e60 con 0x55bea8d81a70 2024-09-10T07:50:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.035+0000 7f59f3e00640 1 -- 172.21.15.46:0/855657901 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59fc03a070 con 0x55bea8d81a70 2024-09-10T07:50:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.035+0000 7f59f3e00640 1 -- 172.21.15.46:0/855657901 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00f860 con 0x55bea8d81a70 2024-09-10T07:50:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.040+0000 7f5a0d49e740 1 -- 172.21.15.46:0/855657901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 msgr2=0x55bea8d7d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.041+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/855657901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d7d4c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f59fc02f040 tx=0x7f59fc005fd0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.042+0000 7f5a0d49e740 1 -- 172.21.15.46:0/855657901 shutdown_connections 2024-09-10T07:50:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.042+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/855657901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d7da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.042+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/855657901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.042+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/855657901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d7d4c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.042+0000 7f5a0d49e740 1 -- 172.21.15.46:0/855657901 >> 172.21.15.46:0/855657901 conn(0x55bea8d7d0e0 msgr2=0x55bea8d86c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.042+0000 7f5a0d49e740 1 -- 172.21.15.46:0/855657901 shutdown_connections 2024-09-10T07:50:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.042+0000 7f5a0d49e740 1 -- 172.21.15.46:0/855657901 wait complete. 2024-09-10T07:50:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.043+0000 7f5a0d49e740 1 -- start start 2024-09-10T07:50:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.043+0000 7f5a0d49e740 1 Processor -- start 2024-09-10T07:50:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.044+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.044+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.044+0000 7f5a0d49e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d91a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d909f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56104/0 (socket says 172.21.15.46:56104) 2024-09-10T07:50:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a041a9880 con 0x55bea8d87440 2024-09-10T07:50:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a041a9120 con 0x55bea8d823b0 2024-09-10T07:50:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a0d49e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a041a6360 con 0x55bea8d81a70 2024-09-10T07:50:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a0d49e740 10 client.0 _ll_register_callbacks cb 0x7f59f4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d91a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a03400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a03400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d904b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48456/0 (socket says 172.21.15.46:48456) 2024-09-10T07:50:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.045+0000 7f5a03400640 1 -- 172.21.15.46:0/993469315 learned_addr learned my addr 172.21.15.46:0/993469315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.047+0000 7f5a02a00640 1 -- 172.21.15.46:0/993469315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 msgr2=0x55bea8d904b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.047+0000 7f5a02a00640 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d904b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.047+0000 7f5a02a00640 1 -- 172.21.15.46:0/993469315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 msgr2=0x55bea8d91a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.047+0000 7f5a02a00640 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d91a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.047+0000 7f5a02a00640 1 -- 172.21.15.46:0/993469315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea8d92010 con 0x55bea8d823b0 2024-09-10T07:50:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.048+0000 7f5a02a00640 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d909f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0ed2c0 tx=0x7f59ec02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877707649132e70c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.048+0000 7f59f1600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:50:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.050+0000 7f59f1600640 1 -- 172.21.15.46:0/993469315 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec049040 con 0x55bea8d823b0 2024-09-10T07:50:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.050+0000 7f59f1600640 1 -- 172.21.15.46:0/993469315 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f59ec0ba2d0 con 0x55bea8d823b0 2024-09-10T07:50:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.050+0000 7f5a0d49e740 1 -- 172.21.15.46:0/993469315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59fc03a070 con 0x55bea8d823b0 2024-09-10T07:50:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.059+0000 7f59f1600640 1 -- 172.21.15.46:0/993469315 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f59ec0dc0b0 con 0x55bea8d823b0 2024-09-10T07:50:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.059+0000 7f5a0d49e740 10 client.28868 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.059+0000 7f5a0d49e740 1 -- 172.21.15.46:0/993469315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55bea8d93300 con 0x55bea8d823b0 2024-09-10T07:50:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.060+0000 7f59f1600640 1 -- 172.21.15.46:0/993469315 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59fc03a070 con 0x55bea8d823b0 2024-09-10T07:50:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.060+0000 7f59f1600640 1 -- 172.21.15.46:0/993469315 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f59ec003800 con 0x55bea8d823b0 2024-09-10T07:50:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.060+0000 7f5a0d49e740 10 client.28868 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.060+0000 7f5a0d49e740 10 client.28868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.060+0000 7f5a0d49e740 20 client.28868 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.060+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bea8e02ca0 0x55bea8e050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.060+0000 7f5a0d49e740 4 client.28868 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:50:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.060+0000 7f5a0d49e740 1 -- 172.21.15.46:0/993469315 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bea8ceb830 con 0x55bea8e02ca0 2024-09-10T07:50:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.061+0000 7f5a03400640 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bea8e02ca0 0x55bea8e050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.068+0000 7f5a03400640 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bea8e02ca0 0x55bea8e050c0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad02251a1bbf9fd server_cookie=6592c967797fdeda in_seq=0 out_seq=0 2024-09-10T07:50:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.068+0000 7f59f1600640 10 client.28868 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.098+0000 7f59f1600640 1 -- 172.21.15.46:0/993469315 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bea8ceb830 con 0x55bea8e02ca0 2024-09-10T07:50:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.098+0000 7f59f1600640 10 client.28868 handle_command_reply: tid=0 2024-09-10T07:50:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 client.28868 shutdown 2024-09-10T07:50:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 -- 172.21.15.46:0/993469315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 msgr2=0x55bea8d909f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d909f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0ed2c0 tx=0x7f59ec02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 -- 172.21.15.46:0/993469315 shutdown_connections 2024-09-10T07:50:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55bea8e02ca0 0x55bea8e050c0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea8d87440 0x55bea8d91a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bea8d823b0 0x55bea8d909f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 --2- 172.21.15.46:0/993469315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bea8d81a70 0x55bea8d904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f59f1600640 0 client.28868 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.099+0000 7f5a0d49e740 1 -- 172.21.15.46:0/993469315 >> 172.21.15.46:0/993469315 conn(0x55bea8d7d0e0 msgr2=0x55bea8d731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.100+0000 7f5a0d49e740 1 -- 172.21.15.46:0/993469315 shutdown_connections 2024-09-10T07:50:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.100+0000 7f5a0d49e740 1 -- 172.21.15.46:0/993469315 wait complete. 2024-09-10T07:50:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.100+0000 7f5a0d49e740 20 client.28868 trim_cache size 0 max 16384 2024-09-10T07:50:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:50:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:50:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:50:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:50:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 -- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59e80a2c00 msgr2=0x7f59e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59e80a2c00 0x7f59e80a50c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009db0 tx=0x7f59f4005f90 comp rx=0 tx=0).stop 2024-09-10T07:50:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 -- 172.21.15.46:0/4220843330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 msgr2=0x7f5a04102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 0x7f5a04102e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f59ec031440 tx=0x7f59ec032ce0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 -- 172.21.15.46:0/4220843330 shutdown_connections 2024-09-10T07:50:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a041023f0 0x7f5a04103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04101a30 0x7f5a04102e10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f59e80a2c00 0x7f59e80a50c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 --2- 172.21.15.46:0/4220843330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a04100840 0x7f5a041047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 -- 172.21.15.46:0/4220843330 >> 172.21.15.46:0/4220843330 conn(0x7f5a040fbfa0 msgr2=0x7f5a04069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.102+0000 7f5a09000640 1 -- 172.21.15.46:0/4220843330 shutdown_connections 2024-09-10T07:50:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:41.103+0000 7f5a09000640 1 -- 172.21.15.46:0/4220843330 wait complete. 2024-09-10T07:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.286+0000 7f2829600640 1 -- 172.21.15.46:0/204478883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f282410e330 msgr2=0x7f2824110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.286+0000 7f2829600640 1 --2- 172.21.15.46:0/204478883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f282410e330 0x7f2824110720 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f281c009ca0 tx=0x7f281c02a8f0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.287+0000 7f2829600640 1 -- 172.21.15.46:0/204478883 shutdown_connections 2024-09-10T07:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.287+0000 7f2829600640 1 --2- 172.21.15.46:0/204478883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f282410e330 0x7f2824110720 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.287+0000 7f2829600640 1 --2- 172.21.15.46:0/204478883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2824101130 0x7f282410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.287+0000 7f2829600640 1 --2- 172.21.15.46:0/204478883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2824100780 0x7f2824100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.287+0000 7f2829600640 1 -- 172.21.15.46:0/204478883 >> 172.21.15.46:0/204478883 conn(0x7f28240fc610 msgr2=0x7f28240fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.288+0000 7f2829600640 1 -- 172.21.15.46:0/204478883 shutdown_connections 2024-09-10T07:50:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.288+0000 7f2829600640 1 -- 172.21.15.46:0/204478883 wait complete. 2024-09-10T07:50:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.290+0000 7f2829600640 1 Processor -- start 2024-09-10T07:50:41.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.290+0000 7f2829600640 1 -- start start 2024-09-10T07:50:41.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.291+0000 7f2829600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824100780 0x7f2824073950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.291+0000 7f2829600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2824101130 0x7f2824073e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.292+0000 7f2823e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824100780 0x7f2824073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.294+0000 7f2823e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824100780 0x7f2824073950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48478/0 (socket says 172.21.15.46:48478) 2024-09-10T07:50:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.294+0000 7f2823e00640 1 -- 172.21.15.46:0/3731604110 learned_addr learned my addr 172.21.15.46:0/3731604110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.292+0000 7f2823400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2824101130 0x7f2824073e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.294+0000 7f2829600640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f282410e330 0x7f282406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.294+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2824114070 con 0x7f2824101130 2024-09-10T07:50:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.294+0000 7f2828c00640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f282410e330 0x7f282406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.294+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2824113ef0 con 0x7f282410e330 2024-09-10T07:50:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.294+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28241141f0 con 0x7f2824100780 2024-09-10T07:50:41.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.295+0000 7f2823e00640 1 -- 172.21.15.46:0/3731604110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f282410e330 msgr2=0x7f282406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.295+0000 7f2823e00640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f282410e330 0x7f282406cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.295+0000 7f2823e00640 1 -- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2824101130 msgr2=0x7f2824073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.295+0000 7f2823e00640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2824101130 0x7f2824073e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.295+0000 7f2823e00640 1 -- 172.21.15.46:0/3731604110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282406d4d0 con 0x7f2824100780 2024-09-10T07:50:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.296+0000 7f2823e00640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824100780 0x7f2824073950 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f281400c890 tx=0x7f281400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.296+0000 7f2820c00640 1 -- 172.21.15.46:0/3731604110 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814004220 con 0x7f2824100780 2024-09-10T07:50:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.296+0000 7f2820c00640 1 -- 172.21.15.46:0/3731604110 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28140043c0 con 0x7f2824100780 2024-09-10T07:50:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.296+0000 7f2820c00640 1 -- 172.21.15.46:0/3731604110 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28140049c0 con 0x7f2824100780 2024-09-10T07:50:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.297+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282406d7c0 con 0x7f2824100780 2024-09-10T07:50:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.297+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28240716b0 con 0x7f2824100780 2024-09-10T07:50:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.310+0000 7f282d6c6740 1 Processor -- start 2024-09-10T07:50:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.311+0000 7f282d6c6740 1 -- start start 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.311+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38bd60 0x56430f38c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.311+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 0x56430f39f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.311+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f39f830 0x56430f3a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.311+0000 7f282d6c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56430f2f23e0 con 0x56430f38c680 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.311+0000 7f282d6c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56430f299d80 con 0x56430f38bd60 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.311+0000 7f282d6c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56430f2ee3a0 con 0x56430f39f830 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.311+0000 7f2820c00640 1 -- 172.21.15.46:0/3731604110 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2814005120 con 0x7f2824100780 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.312+0000 7f2828c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38bd60 0x56430f38c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.312+0000 7f2828c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38bd60 0x56430f38c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56140/0 (socket says 172.21.15.46:56140) 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.312+0000 7f2828c00640 1 -- 172.21.15.46:0/1841147161 learned_addr learned my addr 172.21.15.46:0/1841147161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.312+0000 7f2823e00640 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 0x56430f39f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.313+0000 7f2823400640 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f39f830 0x56430f3a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.313+0000 7f2828c00640 1 -- 172.21.15.46:0/1841147161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f39f830 msgr2=0x56430f3a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.313+0000 7f2828c00640 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f39f830 0x56430f3a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.313+0000 7f2828c00640 1 -- 172.21.15.46:0/1841147161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 msgr2=0x56430f39f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.313+0000 7f2828c00640 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 0x56430f39f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.313+0000 7f2828c00640 1 -- 172.21.15.46:0/1841147161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56430f2b8ae0 con 0x56430f38bd60 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.313+0000 7f2823e00640 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 0x56430f39f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.314+0000 7f2828c00640 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38bd60 0x56430f38c140 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f281c009ca0 tx=0x7f281c009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870f24d04a396a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.314+0000 7f2819200640 1 -- 172.21.15.46:0/1841147161 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c031be0 con 0x56430f38bd60 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.314+0000 7f2819200640 1 -- 172.21.15.46:0/1841147161 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c036040 con 0x56430f38bd60 2024-09-10T07:50:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.314+0000 7f2819200640 1 -- 172.21.15.46:0/1841147161 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c004280 con 0x56430f38bd60 2024-09-10T07:50:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.315+0000 7f282d6c6740 1 -- 172.21.15.46:0/1841147161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38bd60 msgr2=0x56430f38c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.315+0000 7f282d6c6740 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38bd60 0x56430f38c140 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f281c009ca0 tx=0x7f281c009830 comp rx=0 tx=0).stop 2024-09-10T07:50:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.316+0000 7f282d6c6740 1 -- 172.21.15.46:0/1841147161 shutdown_connections 2024-09-10T07:50:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.316+0000 7f282d6c6740 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f39f830 0x56430f3a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.316+0000 7f282d6c6740 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 0x56430f39f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.316+0000 7f282d6c6740 1 --2- 172.21.15.46:0/1841147161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38bd60 0x56430f38c140 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.316+0000 7f282d6c6740 1 -- 172.21.15.46:0/1841147161 >> 172.21.15.46:0/1841147161 conn(0x56430f3883d0 msgr2=0x56430f3887b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.316+0000 7f2820c00640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28080a2cd0 0x7f28080a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.316+0000 7f282d6c6740 1 -- 172.21.15.46:0/1841147161 shutdown_connections 2024-09-10T07:50:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.319+0000 7f2820c00640 1 -- 172.21.15.46:0/3731604110 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28140cf280 con 0x7f2824100780 2024-09-10T07:50:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.321+0000 7f282d6c6740 1 -- 172.21.15.46:0/1841147161 wait complete. 2024-09-10T07:50:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.321+0000 7f282d6c6740 1 -- start start 2024-09-10T07:50:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.321+0000 7f282d6c6740 1 Processor -- start 2024-09-10T07:50:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.322+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f39c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.322+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38c680 0x56430f39cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.322+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f39f830 0x56430f39d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f2823e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f39c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f2823e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f39c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48506/0 (socket says 172.21.15.46:48506) 2024-09-10T07:50:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f2823e00640 1 -- 172.21.15.46:0/515014882 learned_addr learned my addr 172.21.15.46:0/515014882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f2823400640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28080a2cd0 0x7f28080a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56430f3a2340 con 0x56430f39f830 2024-09-10T07:50:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56430f2ad0f0 con 0x56430f38c680 2024-09-10T07:50:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56430f3a24c0 con 0x56430f38bd60 2024-09-10T07:50:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f282d6c6740 10 client.0 _ll_register_callbacks cb 0x56430f294bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.323+0000 7f2828c00640 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f39f830 0x56430f39d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.324+0000 7f2823e00640 1 -- 172.21.15.46:0/515014882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38c680 msgr2=0x56430f39cd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:50:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.324+0000 7f2823e00640 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38c680 0x56430f39cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.324+0000 7f2823e00640 1 -- 172.21.15.46:0/515014882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f39f830 msgr2=0x56430f39d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.324+0000 7f2823e00640 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f39f830 0x56430f39d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.324+0000 7f2823e00640 1 -- 172.21.15.46:0/515014882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56430f0633c0 con 0x56430f38bd60 2024-09-10T07:50:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.324+0000 7f2828c00640 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f39f830 0x56430f39d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.325+0000 7f280d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.325+0000 7f2823e00640 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f39c810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f28140981e0 tx=0x7f2814004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f7474742f6447b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.325+0000 7f280d600640 1 -- 172.21.15.46:0/515014882 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281401d040 con 0x56430f38bd60 2024-09-10T07:50:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.325+0000 7f280d600640 1 -- 172.21.15.46:0/515014882 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f28140a3f40 con 0x56430f38bd60 2024-09-10T07:50:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.326+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281c036040 con 0x56430f38bd60 2024-09-10T07:50:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.330+0000 7f2823400640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28080a2cd0 0x7f28080a5190 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x56430f39ddc0 tx=0x7f2810005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.330+0000 7f280d600640 1 -- 172.21.15.46:0/515014882 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f281401d040 con 0x56430f38bd60 2024-09-10T07:50:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.330+0000 7f282d6c6740 10 client.28783 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.330+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56430f38ead0 con 0x56430f38bd60 2024-09-10T07:50:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.330+0000 7f280d600640 1 -- 172.21.15.46:0/515014882 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c036040 con 0x56430f38bd60 2024-09-10T07:50:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.331+0000 7f280d600640 1 -- 172.21.15.46:0/515014882 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f281409ad80 con 0x56430f38bd60 2024-09-10T07:50:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.331+0000 7f282d6c6740 10 client.28783 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.331+0000 7f282d6c6740 10 client.28783 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:50:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.331+0000 7f282d6c6740 20 client.28783 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.331+0000 7f282d6c6740 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56430f38fe20 0x56430f3922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.331+0000 7f282d6c6740 4 client.28783 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.331+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56430f2ff950 con 0x56430f38fe20 2024-09-10T07:50:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.331+0000 7f2823400640 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56430f38fe20 0x56430f3922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.333+0000 7f2823400640 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56430f38fe20 0x56430f3922b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c9f62ea55b3037c server_cookie=ba2c72428f53dae1 in_seq=0 out_seq=0 2024-09-10T07:50:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.333+0000 7f280d600640 10 client.28783 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:50:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.334+0000 7f280d600640 1 -- 172.21.15.46:0/515014882 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56430f2ff950 con 0x56430f38fe20 2024-09-10T07:50:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.334+0000 7f280d600640 10 client.28783 handle_command_reply: tid=0 2024-09-10T07:50:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.334+0000 7f282d6c6740 1 client.28783 shutdown 2024-09-10T07:50:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.334+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 msgr2=0x56430f39c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.334+0000 7f282d6c6740 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f39c810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f28140981e0 tx=0x7f2814004570 comp rx=0 tx=0).stop 2024-09-10T07:50:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.334+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 shutdown_connections 2024-09-10T07:50:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f282d6c6740 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56430f38fe20 0x56430f3922b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f282d6c6740 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f39f830 0x56430f39d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f282d6c6740 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f38c680 0x56430f39cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f282d6c6740 1 --2- 172.21.15.46:0/515014882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f39c810 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 >> 172.21.15.46:0/515014882 conn(0x56430f3883d0 msgr2=0x56430f38e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f280d600640 0 client.28783 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:50:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 shutdown_connections 2024-09-10T07:50:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f282d6c6740 1 -- 172.21.15.46:0/515014882 wait complete. 2024-09-10T07:50:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.335+0000 7f282d6c6740 20 client.28783 trim_cache size 0 max 16384 2024-09-10T07:50:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.366+0000 7f282d6c6740 1 Processor -- start 2024-09-10T07:50:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.367+0000 7f282d6c6740 1 -- start start 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.367+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f38df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.367+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 0x56430f399a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.368+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f399fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.368+0000 7f282d6c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b5070 con 0x56430f38c680 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.368+0000 7f282d6c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b6c40 con 0x56430f39f830 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.368+0000 7f282d6c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b5740 con 0x56430f38bd60 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.369+0000 7f2823e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f38df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.369+0000 7f2828c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f399fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.369+0000 7f2828c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f399fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56162/0 (socket says 172.21.15.46:56162) 2024-09-10T07:50:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.369+0000 7f2828c00640 1 -- 172.21.15.46:0/2347718675 learned_addr learned my addr 172.21.15.46:0/2347718675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.371+0000 7f2828c00640 1 -- 172.21.15.46:0/2347718675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 msgr2=0x56430f38df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.371+0000 7f2828c00640 1 --2- 172.21.15.46:0/2347718675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f38df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.371+0000 7f2828c00640 1 -- 172.21.15.46:0/2347718675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 msgr2=0x56430f399a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.371+0000 7f2828c00640 1 --2- 172.21.15.46:0/2347718675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 0x56430f399a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.371+0000 7f2828c00640 1 -- 172.21.15.46:0/2347718675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56430f38d2b0 con 0x56430f39f830 2024-09-10T07:50:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.373+0000 7f2828c00640 1 --2- 172.21.15.46:0/2347718675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f399fb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f281c02b0d0 tx=0x7f281c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5b86bd2867f3da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.373+0000 7f280fe00640 1 -- 172.21.15.46:0/2347718675 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c031480 con 0x56430f39f830 2024-09-10T07:50:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.373+0000 7f280fe00640 1 -- 172.21.15.46:0/2347718675 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c036040 con 0x56430f39f830 2024-09-10T07:50:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.373+0000 7f280fe00640 1 -- 172.21.15.46:0/2347718675 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c004560 con 0x56430f39f830 2024-09-10T07:50:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.375+0000 7f282d6c6740 1 -- 172.21.15.46:0/2347718675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 msgr2=0x56430f399fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.375+0000 7f282d6c6740 1 --2- 172.21.15.46:0/2347718675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f399fb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f281c02b0d0 tx=0x7f281c009a20 comp rx=0 tx=0).stop 2024-09-10T07:50:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.375+0000 7f282d6c6740 1 -- 172.21.15.46:0/2347718675 shutdown_connections 2024-09-10T07:50:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.375+0000 7f282d6c6740 1 --2- 172.21.15.46:0/2347718675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f399fb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.375+0000 7f282d6c6740 1 --2- 172.21.15.46:0/2347718675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38c680 0x56430f399a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.375+0000 7f282d6c6740 1 --2- 172.21.15.46:0/2347718675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38bd60 0x56430f38df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.375+0000 7f282d6c6740 1 -- 172.21.15.46:0/2347718675 >> 172.21.15.46:0/2347718675 conn(0x56430f3883d0 msgr2=0x56430f38d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.378+0000 7f282d6c6740 1 -- 172.21.15.46:0/2347718675 shutdown_connections 2024-09-10T07:50:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.378+0000 7f282d6c6740 1 -- 172.21.15.46:0/2347718675 wait complete. 2024-09-10T07:50:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.378+0000 7f282d6c6740 1 -- start start 2024-09-10T07:50:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.378+0000 7f282d6c6740 1 Processor -- start 2024-09-10T07:50:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.379+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38bd60 0x56430f3b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.379+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38c680 0x56430f3b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.379+0000 7f282d6c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f3b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.379+0000 7f282d6c6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b6570 con 0x56430f38bd60 2024-09-10T07:50:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.379+0000 7f282d6c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b5e10 con 0x56430f39f830 2024-09-10T07:50:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.379+0000 7f282d6c6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b3050 con 0x56430f38c680 2024-09-10T07:50:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.379+0000 7f282d6c6740 10 client.0 _ll_register_callbacks cb 0x7f2814097c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2828c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f3b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2828c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f3b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56164/0 (socket says 172.21.15.46:56164) 2024-09-10T07:50:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2828c00640 1 -- 172.21.15.46:0/801088675 learned_addr learned my addr 172.21.15.46:0/801088675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2823400640 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38c680 0x56430f3b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2828c00640 1 -- 172.21.15.46:0/801088675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38c680 msgr2=0x56430f3b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2828c00640 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38c680 0x56430f3b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2828c00640 1 -- 172.21.15.46:0/801088675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38bd60 msgr2=0x56430f3b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2828c00640 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38bd60 0x56430f3b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.380+0000 7f2828c00640 1 -- 172.21.15.46:0/801088675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56430f4114f0 con 0x56430f39f830 2024-09-10T07:50:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.382+0000 7f2828c00640 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f3b1cf0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f281c009ca0 tx=0x7f281c02a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=841b18b00d7bd4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.382+0000 7f280d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:50:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.383+0000 7f280d600640 1 -- 172.21.15.46:0/801088675 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c059060 con 0x56430f39f830 2024-09-10T07:50:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.383+0000 7f280d600640 1 -- 172.21.15.46:0/801088675 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f281c003b80 con 0x56430f39f830 2024-09-10T07:50:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.383+0000 7f282d6c6740 1 -- 172.21.15.46:0/801088675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281c036040 con 0x56430f39f830 2024-09-10T07:50:41.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.385+0000 7f280d600640 1 -- 172.21.15.46:0/801088675 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f281c003d80 con 0x56430f39f830 2024-09-10T07:50:41.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.385+0000 7f282d6c6740 10 client.28880 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:41.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.385+0000 7f282d6c6740 1 -- 172.21.15.46:0/801088675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56430f4127b0 con 0x56430f39f830 2024-09-10T07:50:41.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.387+0000 7f280d600640 1 -- 172.21.15.46:0/801088675 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c036040 con 0x56430f39f830 2024-09-10T07:50:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.387+0000 7f280d600640 1 -- 172.21.15.46:0/801088675 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f281c02f640 con 0x56430f39f830 2024-09-10T07:50:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.387+0000 7f282d6c6740 10 client.28880 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.387+0000 7f282d6c6740 10 client.28880 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:50:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.387+0000 7f282d6c6740 20 client.28880 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.387+0000 7f282d6c6740 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56430f412a40 0x56430f414e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.387+0000 7f282d6c6740 4 client.28880 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:50:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.387+0000 7f282d6c6740 1 -- 172.21.15.46:0/801088675 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56430f2f9e60 con 0x56430f412a40 2024-09-10T07:50:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.388+0000 7f2823e00640 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56430f412a40 0x56430f414e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:41.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.391+0000 7f2823e00640 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56430f412a40 0x56430f414e30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8470b527f54a28f5 server_cookie=998638668fab2a26 in_seq=0 out_seq=0 2024-09-10T07:50:41.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.391+0000 7f280d600640 10 client.28880 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:50:41.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.393+0000 7f280d600640 1 -- 172.21.15.46:0/801088675 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56430f2f9e60 con 0x56430f412a40 2024-09-10T07:50:41.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.393+0000 7f280d600640 10 client.28880 handle_command_reply: tid=0 2024-09-10T07:50:41.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.393+0000 7f282d6c6740 1 client.28880 shutdown 2024-09-10T07:50:41.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.393+0000 7f282d6c6740 1 -- 172.21.15.46:0/801088675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 msgr2=0x56430f3b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.393+0000 7f282d6c6740 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f3b1cf0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f281c009ca0 tx=0x7f281c02a8f0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.393+0000 7f282d6c6740 1 -- 172.21.15.46:0/801088675 shutdown_connections 2024-09-10T07:50:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.393+0000 7f282d6c6740 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56430f412a40 0x56430f414e30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.394+0000 7f282d6c6740 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56430f39f830 0x56430f3b1cf0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.394+0000 7f282d6c6740 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56430f38c680 0x56430f3b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.394+0000 7f282d6c6740 1 --2- 172.21.15.46:0/801088675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430f38bd60 0x56430f3b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.394+0000 7f282d6c6740 1 -- 172.21.15.46:0/801088675 >> 172.21.15.46:0/801088675 conn(0x56430f3883d0 msgr2=0x56430f38e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.394+0000 7f280d600640 0 client.28880 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:50:41.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.394+0000 7f282d6c6740 1 -- 172.21.15.46:0/801088675 shutdown_connections 2024-09-10T07:50:41.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.395+0000 7f282d6c6740 1 -- 172.21.15.46:0/801088675 wait complete. 2024-09-10T07:50:41.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.395+0000 7f282d6c6740 20 client.28880 trim_cache size 0 max 16384 2024-09-10T07:50:41.396 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:50:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28080a2cd0 msgr2=0x7f28080a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28080a2cd0 0x7f28080a5190 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x56430f39ddc0 tx=0x7f2810005f90 comp rx=0 tx=0).stop 2024-09-10T07:50:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824100780 msgr2=0x7f2824073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824100780 0x7f2824073950 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f281400c890 tx=0x7f281400cd50 comp rx=0 tx=0).stop 2024-09-10T07:50:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 shutdown_connections 2024-09-10T07:50:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f282410e330 0x7f282406cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f28080a2cd0 0x7f28080a5190 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2824101130 0x7f2824073e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 --2- 172.21.15.46:0/3731604110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2824100780 0x7f2824073950 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 >> 172.21.15.46:0/3731604110 conn(0x7f28240fc610 msgr2=0x7f28240fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 shutdown_connections 2024-09-10T07:50:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:41.396+0000 7f2829600640 1 -- 172.21.15.46:0/3731604110 wait complete. 2024-09-10T07:50:42.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:42 smithi046 ceph-mon[30705]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-09-10T07:50:42.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:42 smithi181 ceph-mon[35397]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-09-10T07:50:42.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:42 smithi063 ceph-mon[35020]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-09-10T07:50:43.306 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:50:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:44.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:44 smithi181 ceph-mon[35397]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.5 MiB/s wr, 447 op/s 2024-09-10T07:50:44.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:44 smithi063 ceph-mon[35020]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.5 MiB/s wr, 447 op/s 2024-09-10T07:50:44.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:44 smithi046 ceph-mon[30705]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.5 MiB/s wr, 447 op/s 2024-09-10T07:50:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.951+0000 7f81f8c00640 1 -- 172.21.15.46:0/2430951912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec0960d0 msgr2=0x7f81ec0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.951+0000 7f81f8c00640 1 --2- 172.21.15.46:0/2430951912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec0960d0 0x7f81ec0a24a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f81f4067960 tx=0x7f81f4096da0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.951+0000 7f81f8c00640 1 -- 172.21.15.46:0/2430951912 shutdown_connections 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.951+0000 7f81f8c00640 1 --2- 172.21.15.46:0/2430951912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec0960d0 0x7f81ec0a24a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.951+0000 7f81f8c00640 1 --2- 172.21.15.46:0/2430951912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ec095710 0x7f81ec095b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.951+0000 7f81f8c00640 1 --2- 172.21.15.46:0/2430951912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ec09b660 0x7f81ec09ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.951+0000 7f81f8c00640 1 -- 172.21.15.46:0/2430951912 >> 172.21.15.46:0/2430951912 conn(0x7f81ec00c9a0 msgr2=0x7f81ec00cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.952+0000 7f81f8c00640 1 -- 172.21.15.46:0/2430951912 shutdown_connections 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.952+0000 7f81f8c00640 1 -- 172.21.15.46:0/2430951912 wait complete. 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.953+0000 7f81f8c00640 1 Processor -- start 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.954+0000 7f81f8c00640 1 -- start start 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.955+0000 7f81f8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec095710 0x7f81ec140710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.955+0000 7f81f8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ec0960d0 0x7f81ec140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.955+0000 7f81f8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ec09b660 0x7f81ec13a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.955+0000 7f81f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec095710 0x7f81ec140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.955+0000 7f81f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec095710 0x7f81ec140710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56188/0 (socket says 172.21.15.46:56188) 2024-09-10T07:50:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.956+0000 7f81f2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ec09b660 0x7f81ec13a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.956+0000 7f81f2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ec09b660 0x7f81ec13a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48548/0 (socket says 172.21.15.46:48548) 2024-09-10T07:50:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.956+0000 7f81f1e00640 1 -- 172.21.15.46:0/3669405402 learned_addr learned my addr 172.21.15.46:0/3669405402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:45.259 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:50:45.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.956+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec0a36e0 con 0x7f81ec0960d0 2024-09-10T07:50:45.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.956+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec0a3560 con 0x7f81ec095710 2024-09-10T07:50:45.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.956+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec0a3860 con 0x7f81ec09b660 2024-09-10T07:50:45.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.957+0000 7f81f2800640 1 -- 172.21.15.46:0/3669405402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec095710 msgr2=0x7f81ec140710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.957+0000 7f81f2800640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec095710 0x7f81ec140710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.957+0000 7f81f2800640 1 -- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ec0960d0 msgr2=0x7f81ec140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.958+0000 7f81f2800640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ec0960d0 0x7f81ec140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.959+0000 7f81f2800640 1 -- 172.21.15.46:0/3669405402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec13b000 con 0x7f81ec09b660 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.959+0000 7f81f2800640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ec09b660 0x7f81ec13a8d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000b60 tx=0x7f81f40a06c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.960+0000 7f81eaa00640 1 -- 172.21.15.46:0/3669405402 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f40a0a80 con 0x7f81ec09b660 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.960+0000 7f81eaa00640 1 -- 172.21.15.46:0/3669405402 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81f40a0c20 con 0x7f81ec09b660 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.961+0000 7f81eaa00640 1 -- 172.21.15.46:0/3669405402 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f409e3c0 con 0x7f81ec09b660 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.962+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ec13b290 con 0x7f81ec09b660 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.962+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ec1475e0 con 0x7f81ec09b660 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.967+0000 7f81eaa00640 1 -- 172.21.15.46:0/3669405402 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f81f409d030 con 0x7f81ec09b660 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.969+0000 7f81eaa00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81d80a2ce0 0x7f81d80a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.969+0000 7f81eaa00640 1 -- 172.21.15.46:0/3669405402 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f81f41538b0 con 0x7f81ec09b660 2024-09-10T07:50:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.970+0000 7f81f1e00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81d80a2ce0 0x7f81d80a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.970+0000 7f81fb686740 1 Processor -- start 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.971+0000 7f81fb686740 1 -- start start 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.971+0000 7f81fb686740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f32d90 0x5602e1f33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.971+0000 7f81fb686740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f336b0 0x5602e1f46320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.971+0000 7f81fb686740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f46860 0x5602e1f48c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.971+0000 7f81fb686740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5602e1e993e0 con 0x5602e1f336b0 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.971+0000 7f81fb686740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5602e1e40d80 con 0x5602e1f46860 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.971+0000 7f81fb686740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5602e1e953a0 con 0x5602e1f32d90 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f46860 0x5602e1f48c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f46860 0x5602e1f48c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56204/0 (socket says 172.21.15.46:56204) 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f2800640 1 -- 172.21.15.46:0/3441222891 learned_addr learned my addr 172.21.15.46:0/3441222891 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f2800640 1 -- 172.21.15.46:0/3441222891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f32d90 msgr2=0x5602e1f33170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f1e00640 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f32d90 0x5602e1f33170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f2800640 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f32d90 0x5602e1f33170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f2800640 1 -- 172.21.15.46:0/3441222891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f336b0 msgr2=0x5602e1f46320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f2800640 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f336b0 0x5602e1f46320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.972+0000 7f81f2800640 1 -- 172.21.15.46:0/3441222891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e1e5fae0 con 0x5602e1f46860 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.973+0000 7f81f1e00640 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f32d90 0x5602e1f33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.973+0000 7f81f2800640 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f46860 0x5602e1f48c50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f81f411fa30 tx=0x7f81f4068300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e789db5ef91ec619 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.974+0000 7f81f1e00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81d80a2ce0 0x7f81d80a51a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f81e4005fd0 tx=0x7f81e400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.974+0000 7f81df000640 1 -- 172.21.15.46:0/3441222891 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f411fd30 con 0x5602e1f46860 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.974+0000 7f81df000640 1 -- 172.21.15.46:0/3441222891 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81f409b020 con 0x5602e1f46860 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.974+0000 7f81df000640 1 -- 172.21.15.46:0/3441222891 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f411f450 con 0x5602e1f46860 2024-09-10T07:50:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.975+0000 7f81fb686740 1 -- 172.21.15.46:0/3441222891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f46860 msgr2=0x5602e1f48c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.975+0000 7f81fb686740 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f46860 0x5602e1f48c50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f81f411fa30 tx=0x7f81f4068300 comp rx=0 tx=0).stop 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.976+0000 7f81fb686740 1 -- 172.21.15.46:0/3441222891 shutdown_connections 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.976+0000 7f81fb686740 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f46860 0x5602e1f48c50 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.976+0000 7f81fb686740 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f336b0 0x5602e1f46320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.976+0000 7f81fb686740 1 --2- 172.21.15.46:0/3441222891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f32d90 0x5602e1f33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.976+0000 7f81fb686740 1 -- 172.21.15.46:0/3441222891 >> 172.21.15.46:0/3441222891 conn(0x5602e1f2f400 msgr2=0x5602e1f2f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.977+0000 7f81fb686740 1 -- 172.21.15.46:0/3441222891 shutdown_connections 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.977+0000 7f81fb686740 1 -- 172.21.15.46:0/3441222891 wait complete. 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.978+0000 7f81fb686740 1 -- start start 2024-09-10T07:50:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.978+0000 7f81fb686740 1 Processor -- start 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.979+0000 7f81fb686740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.980+0000 7f81fb686740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f40660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.980+0000 7f81fb686740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1f45e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.980+0000 7f81fb686740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5602e1f35ab0 con 0x5602e1f46860 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.980+0000 7f81fb686740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5602e1e540f0 con 0x5602e1f32d90 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.980+0000 7f81fb686740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5602e1f35c30 con 0x5602e1f336b0 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.980+0000 7f81fb686740 10 client.0 _ll_register_callbacks cb 0x5602e1e3bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.981+0000 7f81f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.981+0000 7f81f1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f40660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.981+0000 7f81f1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f40660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48576/0 (socket says 172.21.15.46:48576) 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.981+0000 7f81f1400640 1 -- 172.21.15.46:0/2596938140 learned_addr learned my addr 172.21.15.46:0/2596938140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.982+0000 7f81f1400640 1 -- 172.21.15.46:0/2596938140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 msgr2=0x5602e1f40120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.982+0000 7f81f1400640 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f40120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.982+0000 7f81f1400640 1 -- 172.21.15.46:0/2596938140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 msgr2=0x5602e1f45e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.982+0000 7f81f1400640 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1f45e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.982+0000 7f81f1400640 1 -- 172.21.15.46:0/2596938140 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602e1c0a3c0 con 0x5602e1f336b0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.982+0000 7f81f1400640 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f40660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009620 tx=0x7f81e0009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d36a64378e0cc7e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.982+0000 7f81d3400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.983+0000 7f81d3400640 1 -- 172.21.15.46:0/2596938140 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e001b020 con 0x5602e1f336b0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.983+0000 7f81d3400640 1 -- 172.21.15.46:0/2596938140 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f81e0005220 con 0x5602e1f336b0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.984+0000 7f81fb686740 1 -- 172.21.15.46:0/2596938140 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81f409b020 con 0x5602e1f336b0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.984+0000 7f81d3400640 1 -- 172.21.15.46:0/2596938140 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f81e0005420 con 0x5602e1f336b0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.984+0000 7f81fb686740 10 client.28798 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.984+0000 7f81fb686740 1 -- 172.21.15.46:0/2596938140 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5602e1f5da80 con 0x5602e1f336b0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.985+0000 7f81d3400640 1 -- 172.21.15.46:0/2596938140 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81f409b020 con 0x5602e1f336b0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.986+0000 7f81d3400640 1 -- 172.21.15.46:0/2596938140 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f81e000c940 con 0x5602e1f336b0 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.986+0000 7f81fb686740 10 client.28798 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.986+0000 7f81fb686740 10 client.28798 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:50:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.986+0000 7f81fb686740 20 client.28798 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.986+0000 7f81fb686740 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5602e1f5ef90 0x5602e1f613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.986+0000 7f81fb686740 4 client.28798 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.986+0000 7f81fb686740 1 -- 172.21.15.46:0/2596938140 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602e1ea6950 con 0x5602e1f5ef90 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.988+0000 7f81f1e00640 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5602e1f5ef90 0x5602e1f613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.989+0000 7f81f1e00640 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5602e1f5ef90 0x5602e1f613f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da15f14bc4d51429 server_cookie=9a37e8f530616133 in_seq=0 out_seq=0 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.989+0000 7f81d3400640 10 client.28798 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.990+0000 7f81d3400640 1 -- 172.21.15.46:0/2596938140 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602e1ea6950 con 0x5602e1f5ef90 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.990+0000 7f81d3400640 10 client.28798 handle_command_reply: tid=0 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.991+0000 7f81fb686740 1 client.28798 shutdown 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.991+0000 7f81fb686740 1 -- 172.21.15.46:0/2596938140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 msgr2=0x5602e1f40660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.991+0000 7f81fb686740 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f40660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009620 tx=0x7f81e0009ae0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81fb686740 1 -- 172.21.15.46:0/2596938140 shutdown_connections 2024-09-10T07:50:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81fb686740 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1f45e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81fb686740 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5602e1f5ef90 0x5602e1f613f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81fb686740 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f40660 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81fb686740 1 --2- 172.21.15.46:0/2596938140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f40120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81fb686740 1 -- 172.21.15.46:0/2596938140 >> 172.21.15.46:0/2596938140 conn(0x5602e1f2f400 msgr2=0x5602e1f2e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81d3400640 0 client.28798 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81fb686740 1 -- 172.21.15.46:0/2596938140 shutdown_connections 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.993+0000 7f81fb686740 1 -- 172.21.15.46:0/2596938140 wait complete. 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:44.994+0000 7f81fb686740 20 client.28798 trim_cache size 0 max 16384 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.027+0000 7f81fb686740 1 Processor -- start 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.027+0000 7f81fb686740 1 -- start start 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.027+0000 7f81fb686740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f45da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.027+0000 7f81fb686740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f2e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.027+0000 7f81fb686740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1f66cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.028+0000 7f81fb686740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec151220 con 0x5602e1f46860 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.028+0000 7f81fb686740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec152df0 con 0x5602e1f32d90 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.028+0000 7f81fb686740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1518f0 con 0x5602e1f336b0 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.028+0000 7f81f2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1f66cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.028+0000 7f81f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f45da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.028+0000 7f81f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f45da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56224/0 (socket says 172.21.15.46:56224) 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.028+0000 7f81f1e00640 1 -- 172.21.15.46:0/1468045082 learned_addr learned my addr 172.21.15.46:0/1468045082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.029+0000 7f81f1e00640 1 -- 172.21.15.46:0/1468045082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 msgr2=0x5602e1f2e470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.029+0000 7f81f1e00640 1 --2- 172.21.15.46:0/1468045082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f2e470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.029+0000 7f81f1e00640 1 -- 172.21.15.46:0/1468045082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 msgr2=0x5602e1f66cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.029+0000 7f81f1e00640 1 --2- 172.21.15.46:0/1468045082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1f66cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.029+0000 7f81f1e00640 1 -- 172.21.15.46:0/1468045082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e1f409d0 con 0x5602e1f32d90 2024-09-10T07:50:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.029+0000 7f81f1e00640 1 --2- 172.21.15.46:0/1468045082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f45da0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f81e400c010 tx=0x7f81e4018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4449fbd91944245f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.030+0000 7f81de600640 1 -- 172.21.15.46:0/1468045082 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011cf0 con 0x5602e1f32d90 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.030+0000 7f81de600640 1 -- 172.21.15.46:0/1468045082 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81e401fe00 con 0x5602e1f32d90 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.030+0000 7f81de600640 1 -- 172.21.15.46:0/1468045082 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4004a00 con 0x5602e1f32d90 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.031+0000 7f81fb686740 1 -- 172.21.15.46:0/1468045082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 msgr2=0x5602e1f45da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.031+0000 7f81fb686740 1 --2- 172.21.15.46:0/1468045082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f45da0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f81e400c010 tx=0x7f81e4018000 comp rx=0 tx=0).stop 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.032+0000 7f81fb686740 1 -- 172.21.15.46:0/1468045082 shutdown_connections 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.032+0000 7f81fb686740 1 --2- 172.21.15.46:0/1468045082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1f66cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.032+0000 7f81fb686740 1 --2- 172.21.15.46:0/1468045082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f2e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.032+0000 7f81fb686740 1 --2- 172.21.15.46:0/1468045082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f45da0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.032+0000 7f81fb686740 1 -- 172.21.15.46:0/1468045082 >> 172.21.15.46:0/1468045082 conn(0x5602e1f2f400 msgr2=0x5602e1f2dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.032+0000 7f81fb686740 1 -- 172.21.15.46:0/1468045082 shutdown_connections 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.032+0000 7f81fb686740 1 -- 172.21.15.46:0/1468045082 wait complete. 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.033+0000 7f81fb686740 1 -- start start 2024-09-10T07:50:45.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.033+0000 7f81fb686740 1 Processor -- start 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.034+0000 7f81fb686740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.034+0000 7f81fb686740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f6f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.034+0000 7f81fb686740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1fc2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.034+0000 7f81f1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f6f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.034+0000 7f81f1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f6f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48588/0 (socket says 172.21.15.46:48588) 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81f2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1fc2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f6f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:56234/0 (socket says 172.21.15.46:56234) 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81f2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1fc2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51368/0 (socket says 172.21.15.46:51368) 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81f1400640 1 -- 172.21.15.46:0/3582275372 learned_addr learned my addr 172.21.15.46:0/3582275372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec152720 con 0x5602e1f46860 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec151fc0 con 0x5602e1f32d90 2024-09-10T07:50:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec14f2b0 con 0x5602e1f336b0 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.035+0000 7f81fb686740 10 client.0 _ll_register_callbacks cb 0x7f81e000f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.036+0000 7f81f1e00640 1 -- 172.21.15.46:0/3582275372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 msgr2=0x5602e1f6f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.036+0000 7f81f1e00640 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f6f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.036+0000 7f81f1e00640 1 -- 172.21.15.46:0/3582275372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 msgr2=0x5602e1fc2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.036+0000 7f81f1e00640 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1fc2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.036+0000 7f81f1e00640 1 -- 172.21.15.46:0/3582275372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602e1fc3240 con 0x5602e1f32d90 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.038+0000 7f81f1e00640 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f6f0a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f81e4005480 tx=0x7f81e4011620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445b185e5fce71e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.038+0000 7f81d3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.039+0000 7f81d3400640 1 -- 172.21.15.46:0/3582275372 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e401f920 con 0x5602e1f32d90 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.039+0000 7f81d3400640 1 -- 172.21.15.46:0/3582275372 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f81e402d040 con 0x5602e1f32d90 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.040+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e401fe00 con 0x5602e1f32d90 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.040+0000 7f81d3400640 1 -- 172.21.15.46:0/3582275372 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f81e4042050 con 0x5602e1f32d90 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.040+0000 7f81fb686740 10 client.28895 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.040+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5602e1fca5c0 con 0x5602e1f32d90 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.041+0000 7f81d3400640 1 -- 172.21.15.46:0/3582275372 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81e401fe00 con 0x5602e1f32d90 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.042+0000 7f81d3400640 1 -- 172.21.15.46:0/3582275372 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f81e4031440 con 0x5602e1f32d90 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.042+0000 7f81fb686740 10 client.28895 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.042+0000 7f81fb686740 10 client.28895 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.042+0000 7f81fb686740 20 client.28895 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.042+0000 7f81fb686740 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5602e1fcac20 0x5602e1fcd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.042+0000 7f81fb686740 4 client.28895 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.042+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602e1ea0e60 con 0x5602e1fcac20 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.043+0000 7f81f1400640 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5602e1fcac20 0x5602e1fcd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.043+0000 7f81f1400640 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5602e1fcac20 0x5602e1fcd010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aa2f6c84763fe78f server_cookie=55c0e7b400557497 in_seq=0 out_seq=0 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.043+0000 7f81d3400640 10 client.28895 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.044+0000 7f81d3400640 1 -- 172.21.15.46:0/3582275372 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602e1ea0e60 con 0x5602e1fcac20 2024-09-10T07:50:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.044+0000 7f81d3400640 10 client.28895 handle_command_reply: tid=0 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.044+0000 7f81fb686740 1 client.28895 shutdown 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.044+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 msgr2=0x5602e1f6f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.044+0000 7f81fb686740 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f6f0a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f81e4005480 tx=0x7f81e4011620 comp rx=0 tx=0).stop 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.045+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 shutdown_connections 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.045+0000 7f81fb686740 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e1f46860 0x5602e1fc2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.045+0000 7f81fb686740 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x5602e1fcac20 0x5602e1fcd010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.045+0000 7f81fb686740 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602e1f336b0 0x5602e1f6f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.045+0000 7f81fb686740 1 --2- 172.21.15.46:0/3582275372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5602e1f32d90 0x5602e1f6f0a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.045+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 >> 172.21.15.46:0/3582275372 conn(0x5602e1f2f400 msgr2=0x5602e1f35050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.045+0000 7f81d3400640 0 client.28895 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.046+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 shutdown_connections 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.046+0000 7f81fb686740 1 -- 172.21.15.46:0/3582275372 wait complete. 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.046+0000 7f81fb686740 20 client.28895 trim_cache size 0 max 16384 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81d80a2ce0 msgr2=0x7f81d80a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81d80a2ce0 0x7f81d80a51a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f81e4005fd0 tx=0x7f81e400b480 comp rx=0 tx=0).stop 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ec09b660 msgr2=0x7f81ec13a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ec09b660 0x7f81ec13a8d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000b60 tx=0x7f81f40a06c0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 shutdown_connections 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ec09b660 0x7f81ec13a8d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f81d80a2ce0 0x7f81d80a51a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ec0960d0 0x7f81ec140c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 --2- 172.21.15.46:0/3669405402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81ec095710 0x7f81ec140710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.047+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 >> 172.21.15.46:0/3669405402 conn(0x7f81ec00c9a0 msgr2=0x7f81ec094210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.048+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 shutdown_connections 2024-09-10T07:50:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:45.048+0000 7f81f8c00640 1 -- 172.21.15.46:0/3669405402 wait complete. 2024-09-10T07:50:45.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 805.0 s, 506.2 tps, lat 201.301 ms stddev 253.632 2024-09-10T07:50:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:46 smithi181 ceph-mon[35397]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 4.2 MiB/s wr, 407 op/s 2024-09-10T07:50:46.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:46 smithi063 ceph-mon[35020]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 4.2 MiB/s wr, 407 op/s 2024-09-10T07:50:46.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:46 smithi046 ceph-mon[30705]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 4.2 MiB/s wr, 407 op/s 2024-09-10T07:50:46.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:50:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:48 smithi181 ceph-mon[35397]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-09-10T07:50:48.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:48 smithi063 ceph-mon[35020]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-09-10T07:50:48.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:48 smithi046 ceph-mon[30705]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-09-10T07:50:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.237+0000 7f5337e00640 1 -- 172.21.15.46:0/383802751 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c035020 con 0x7f5338074510 2024-09-10T07:50:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.238+0000 7f533fe00640 1 -- 172.21.15.46:0/383802751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5338074510 msgr2=0x7f53380748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.238+0000 7f533fe00640 1 --2- 172.21.15.46:0/383802751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5338074510 0x7f53380748f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f532c009900 tx=0x7f532c02a480 comp rx=0 tx=0).stop 2024-09-10T07:50:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.238+0000 7f533fe00640 1 -- 172.21.15.46:0/383802751 shutdown_connections 2024-09-10T07:50:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.238+0000 7f533fe00640 1 --2- 172.21.15.46:0/383802751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f533810e450 0x7f5338110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.238+0000 7f533fe00640 1 --2- 172.21.15.46:0/383802751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338074e30 0x7f533810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.238+0000 7f533fe00640 1 --2- 172.21.15.46:0/383802751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5338074510 0x7f53380748f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.239+0000 7f533fe00640 1 -- 172.21.15.46:0/383802751 >> 172.21.15.46:0/383802751 conn(0x7f53380fd8b0 msgr2=0x7f53380ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.241+0000 7f533fe00640 1 -- 172.21.15.46:0/383802751 shutdown_connections 2024-09-10T07:50:49.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.241+0000 7f533fe00640 1 -- 172.21.15.46:0/383802751 wait complete. 2024-09-10T07:50:49.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.242+0000 7f533fe00640 1 Processor -- start 2024-09-10T07:50:49.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.242+0000 7f533fe00640 1 -- start start 2024-09-10T07:50:49.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.243+0000 7f533fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338074510 0x7f533811a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.243+0000 7f533fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5338074e30 0x7f533811a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.243+0000 7f533fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533810e450 0x7f53381132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5338074e30 0x7f533811a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53381035a0 con 0x7f5338074510 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5338103420 con 0x7f5338074e30 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5338103720 con 0x7f533810e450 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533810e450 0x7f53381132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533810e450 0x7f53381132d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43402/0 (socket says 172.21.15.46:43402) 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533de00640 1 -- 172.21.15.46:0/1182551614 learned_addr learned my addr 172.21.15.46:0/1182551614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533d400640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338074510 0x7f533811a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.244+0000 7f533de00640 1 -- 172.21.15.46:0/1182551614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5338074e30 msgr2=0x7f533811a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.245+0000 7f533de00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5338074e30 0x7f533811a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.245+0000 7f533de00640 1 -- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338074510 msgr2=0x7f533811a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.245+0000 7f533de00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338074510 0x7f533811a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.245+0000 7f533de00640 1 -- 172.21.15.46:0/1182551614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5338113b70 con 0x7f533810e450 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.245+0000 7f533d400640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338074510 0x7f533811a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.245+0000 7f533ca00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5338074e30 0x7f533811a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.246+0000 7f533de00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533810e450 0x7f53381132d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f533000e9d0 tx=0x7f533000ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.246+0000 7f5336000640 1 -- 172.21.15.46:0/1182551614 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533000cd80 con 0x7f533810e450 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.246+0000 7f533fe00640 1 -- 172.21.15.46:0/1182551614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5338113e60 con 0x7f533810e450 2024-09-10T07:50:49.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.246+0000 7f533fe00640 1 -- 172.21.15.46:0/1182551614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5338117f40 con 0x7f533810e450 2024-09-10T07:50:49.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.246+0000 7f5336000640 1 -- 172.21.15.46:0/1182551614 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5330005020 con 0x7f533810e450 2024-09-10T07:50:49.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.246+0000 7f5336000640 1 -- 172.21.15.46:0/1182551614 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53300116d0 con 0x7f533810e450 2024-09-10T07:50:49.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.249+0000 7f53427bf740 1 Processor -- start 2024-09-10T07:50:49.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.250+0000 7f5336000640 1 -- 172.21.15.46:0/1182551614 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5330011930 con 0x7f533810e450 2024-09-10T07:50:49.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.250+0000 7f53427bf740 1 -- start start 2024-09-10T07:50:49.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.252+0000 7f53427bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.252+0000 7f53427bf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 0x5563486f2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.252+0000 7f53427bf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x5563486f5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.252+0000 7f53427bf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55634864e3e0 con 0x5563486e8590 2024-09-10T07:50:49.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.252+0000 7f53427bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5563485f5d80 con 0x5563486e7c70 2024-09-10T07:50:49.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.252+0000 7f53427bf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55634864a3a0 con 0x5563486f2e50 2024-09-10T07:50:49.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.252+0000 7f533ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f533ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486e8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38518/0 (socket says 172.21.15.46:38518) 2024-09-10T07:50:49.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f533ca00640 1 -- 172.21.15.46:0/4042419767 learned_addr learned my addr 172.21.15.46:0/4042419767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f5336000640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53200a2c00 0x7f53200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f533ca00640 1 --2- 172.21.15.46:0/4042419767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x5563486f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f5336000640 1 -- 172.21.15.46:0/1182551614 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f53300cf860 con 0x7f533810e450 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f533ca00640 1 -- 172.21.15.46:0/4042419767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 msgr2=0x5563486f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f533ca00640 1 --2- 172.21.15.46:0/4042419767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x5563486f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f533ca00640 1 -- 172.21.15.46:0/4042419767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 msgr2=0x5563486f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.253+0000 7f533ca00640 1 --2- 172.21.15.46:0/4042419767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 0x5563486f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.254+0000 7f533ca00640 1 -- 172.21.15.46:0/4042419767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556348614ae0 con 0x5563486e7c70 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.254+0000 7f533ca00640 1 --2- 172.21.15.46:0/4042419767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486e8050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5324015880 tx=0x7f53240095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2abce989064860b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.254+0000 7f533de00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53200a2c00 0x7f53200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.254+0000 7f532a600640 1 -- 172.21.15.46:0/4042419767 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324004030 con 0x5563486e7c70 2024-09-10T07:50:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.255+0000 7f532a600640 1 -- 172.21.15.46:0/4042419767 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532400ebd0 con 0x5563486e7c70 2024-09-10T07:50:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.255+0000 7f532a600640 1 -- 172.21.15.46:0/4042419767 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324003850 con 0x5563486e7c70 2024-09-10T07:50:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 -- 172.21.15.46:0/4042419767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 msgr2=0x5563486e8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 --2- 172.21.15.46:0/4042419767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486e8050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5324015880 tx=0x7f53240095d0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f533de00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53200a2c00 0x7f53200a50c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5330000f80 tx=0x7f533000c7f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 -- 172.21.15.46:0/4042419767 shutdown_connections 2024-09-10T07:50:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 --2- 172.21.15.46:0/4042419767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x5563486f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 --2- 172.21.15.46:0/4042419767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 0x5563486f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 --2- 172.21.15.46:0/4042419767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486e8050 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 -- 172.21.15.46:0/4042419767 >> 172.21.15.46:0/4042419767 conn(0x5563486e4200 msgr2=0x5563486e45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 -- 172.21.15.46:0/4042419767 shutdown_connections 2024-09-10T07:50:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.256+0000 7f53427bf740 1 -- 172.21.15.46:0/4042419767 wait complete. 2024-09-10T07:50:49.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.257+0000 7f53427bf740 1 -- start start 2024-09-10T07:50:49.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.257+0000 7f53427bf740 1 Processor -- start 2024-09-10T07:50:49.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.258+0000 7f53427bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486efe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.258+0000 7f53427bf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 0x5563486f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.258+0000 7f53427bf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x5563486f0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.258+0000 7f533ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.258+0000 7f533d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 0x5563486f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.258+0000 7f533ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486efe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38528/0 (socket says 172.21.15.46:38528) 2024-09-10T07:50:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.258+0000 7f533ca00640 1 -- 172.21.15.46:0/2533918184 learned_addr learned my addr 172.21.15.46:0/2533918184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.259+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5563486f5a20 con 0x5563486e8590 2024-09-10T07:50:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.259+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5563486090f0 con 0x5563486e7c70 2024-09-10T07:50:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.259+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5563486f5ba0 con 0x5563486f2e50 2024-09-10T07:50:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.259+0000 7f53427bf740 10 client.0 _ll_register_callbacks cb 0x5563485f0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.259+0000 7f533ca00640 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x5563486f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.259+0000 7f533d400640 1 -- 172.21.15.46:0/2533918184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 msgr2=0x5563486f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.260+0000 7f533d400640 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x5563486f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.260+0000 7f533d400640 1 -- 172.21.15.46:0/2533918184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 msgr2=0x5563486efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.260+0000 7f533d400640 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.260+0000 7f533d400640 1 -- 172.21.15.46:0/2533918184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563483bf3c0 con 0x5563486e8590 2024-09-10T07:50:49.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.260+0000 7f533ca00640 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.260+0000 7f533d400640 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 0x5563486f0340 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f532c02aa70 tx=0x7f532c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd1232a977548d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.260+0000 7f531ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.261+0000 7f531ea00640 1 -- 172.21.15.46:0/2533918184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c035030 con 0x5563486e8590 2024-09-10T07:50:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.261+0000 7f531ea00640 1 -- 172.21.15.46:0/2533918184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f532c0048c0 con 0x5563486e8590 2024-09-10T07:50:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.262+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532400ebd0 con 0x5563486e8590 2024-09-10T07:50:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.262+0000 7f531ea00640 1 -- 172.21.15.46:0/2533918184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f532c005d50 con 0x5563486e8590 2024-09-10T07:50:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.262+0000 7f53427bf740 10 client.18060 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:49.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.262+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556348701b60 con 0x5563486e8590 2024-09-10T07:50:49.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.263+0000 7f531ea00640 1 -- 172.21.15.46:0/2533918184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532400ebd0 con 0x5563486e8590 2024-09-10T07:50:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.263+0000 7f531ea00640 1 -- 172.21.15.46:0/2533918184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f532c0314d0 con 0x5563486e8590 2024-09-10T07:50:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.263+0000 7f53427bf740 10 client.18060 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.263+0000 7f53427bf740 10 client.18060 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:50:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.263+0000 7f53427bf740 20 client.18060 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.263+0000 7f53427bf740 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556348703020 0x556348705480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.263+0000 7f53427bf740 4 client.18060 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:49.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.263+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55634865b950 con 0x556348703020 2024-09-10T07:50:49.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.264+0000 7f533ca00640 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556348703020 0x556348705480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.265+0000 7f533ca00640 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556348703020 0x556348705480 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61144924714c3c3a server_cookie=311b09ed87e61cd2 in_seq=0 out_seq=0 2024-09-10T07:50:49.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.265+0000 7f531ea00640 10 client.18060 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:50:49.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.266+0000 7f531ea00640 1 -- 172.21.15.46:0/2533918184 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55634865b950 con 0x556348703020 2024-09-10T07:50:49.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.266+0000 7f531ea00640 10 client.18060 handle_command_reply: tid=0 2024-09-10T07:50:49.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.266+0000 7f53427bf740 1 client.18060 shutdown 2024-09-10T07:50:49.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.267+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 msgr2=0x5563486f0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.267+0000 7f53427bf740 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 0x5563486f0340 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f532c02aa70 tx=0x7f532c005270 comp rx=0 tx=0).stop 2024-09-10T07:50:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.268+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 shutdown_connections 2024-09-10T07:50:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.268+0000 7f53427bf740 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x556348703020 0x556348705480 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.268+0000 7f53427bf740 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x5563486f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.268+0000 7f53427bf740 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e8590 0x5563486f0340 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.268+0000 7f53427bf740 1 --2- 172.21.15.46:0/2533918184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.268+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 >> 172.21.15.46:0/2533918184 conn(0x5563486e4200 msgr2=0x5563486e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.268+0000 7f531ea00640 0 client.18060 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:50:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.269+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 shutdown_connections 2024-09-10T07:50:49.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.269+0000 7f53427bf740 1 -- 172.21.15.46:0/2533918184 wait complete. 2024-09-10T07:50:49.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.269+0000 7f53427bf740 20 client.18060 trim_cache size 0 max 16384 2024-09-10T07:50:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.294+0000 7f53427bf740 1 Processor -- start 2024-09-10T07:50:49.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.294+0000 7f53427bf740 1 -- start start 2024-09-10T07:50:49.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.294+0000 7f53427bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.295+0000 7f53427bf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486e8590 0x55634870ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.295+0000 7f53427bf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486f2e50 0x55634870b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.295+0000 7f53427bf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f531800fb20 con 0x5563486f2e50 2024-09-10T07:50:49.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.295+0000 7f53427bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53180116f0 con 0x5563486e7c70 2024-09-10T07:50:49.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.295+0000 7f53427bf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53180101f0 con 0x5563486e8590 2024-09-10T07:50:49.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.295+0000 7f533d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486e8590 0x55634870ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.295+0000 7f533d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486e8590 0x55634870ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43436/0 (socket says 172.21.15.46:43436) 2024-09-10T07:50:49.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.296+0000 7f533d400640 1 -- 172.21.15.46:0/1224833465 learned_addr learned my addr 172.21.15.46:0/1224833465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:49.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.298+0000 7f533ca00640 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.298+0000 7f533ca00640 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486f2e50 0x55634870b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.298+0000 7f533d400640 1 -- 172.21.15.46:0/1224833465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 msgr2=0x5563486ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.298+0000 7f533d400640 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486ea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.298+0000 7f533d400640 1 -- 172.21.15.46:0/1224833465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486f2e50 msgr2=0x55634870b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.298+0000 7f533d400640 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486f2e50 0x55634870b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.298+0000 7f533d400640 1 -- 172.21.15.46:0/1224833465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556348705b00 con 0x5563486e8590 2024-09-10T07:50:49.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.299+0000 7f533d400640 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486e8590 0x55634870ad90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f532c005ab0 tx=0x7f532c02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dea1b3f919e1140 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.299+0000 7f5329c00640 1 -- 172.21.15.46:0/1224833465 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c035070 con 0x5563486e8590 2024-09-10T07:50:49.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.299+0000 7f5329c00640 1 -- 172.21.15.46:0/1224833465 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532c031ae0 con 0x5563486e8590 2024-09-10T07:50:49.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.299+0000 7f5329c00640 1 -- 172.21.15.46:0/1224833465 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0035f0 con 0x5563486e8590 2024-09-10T07:50:49.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.301+0000 7f53427bf740 1 -- 172.21.15.46:0/1224833465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486e8590 msgr2=0x55634870ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.301+0000 7f53427bf740 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486e8590 0x55634870ad90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f532c005ab0 tx=0x7f532c02a480 comp rx=0 tx=0).stop 2024-09-10T07:50:49.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.301+0000 7f53427bf740 1 -- 172.21.15.46:0/1224833465 shutdown_connections 2024-09-10T07:50:49.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.301+0000 7f53427bf740 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486f2e50 0x55634870b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.301+0000 7f53427bf740 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486e8590 0x55634870ad90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.301+0000 7f53427bf740 1 --2- 172.21.15.46:0/1224833465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e7c70 0x5563486ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.301+0000 7f53427bf740 1 -- 172.21.15.46:0/1224833465 >> 172.21.15.46:0/1224833465 conn(0x5563486e4200 msgr2=0x5563486f1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.302+0000 7f53427bf740 1 -- 172.21.15.46:0/1224833465 shutdown_connections 2024-09-10T07:50:49.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.302+0000 7f53427bf740 1 -- 172.21.15.46:0/1224833465 wait complete. 2024-09-10T07:50:49.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.302+0000 7f53427bf740 1 -- start start 2024-09-10T07:50:49.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.303+0000 7f53427bf740 1 Processor -- start 2024-09-10T07:50:49.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.304+0000 7f53427bf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e7c70 0x5563486f7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.304+0000 7f53427bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e8590 0x5563486f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.304+0000 7f53427bf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x55634876cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.305+0000 7f53427bf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5318011020 con 0x5563486e7c70 2024-09-10T07:50:49.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.305+0000 7f53427bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53180108c0 con 0x5563486e8590 2024-09-10T07:50:49.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.305+0000 7f53427bf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f531800db00 con 0x5563486f2e50 2024-09-10T07:50:49.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.305+0000 7f533d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e8590 0x5563486f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.305+0000 7f533d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e8590 0x5563486f75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38544/0 (socket says 172.21.15.46:38544) 2024-09-10T07:50:49.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.305+0000 7f533d400640 1 -- 172.21.15.46:0/2434633928 learned_addr learned my addr 172.21.15.46:0/2434633928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:49.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.305+0000 7f53427bf740 10 client.0 _ll_register_callbacks cb 0x7f5324005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:49.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.306+0000 7f533ca00640 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x55634876cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.306+0000 7f533d400640 1 -- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 msgr2=0x55634876cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.306+0000 7f533d400640 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x55634876cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.306+0000 7f533d400640 1 -- 172.21.15.46:0/2434633928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e7c70 msgr2=0x5563486f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.306+0000 7f533d400640 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e7c70 0x5563486f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.306+0000 7f533d400640 1 -- 172.21.15.46:0/2434633928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563486eac60 con 0x5563486e8590 2024-09-10T07:50:49.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.307+0000 7f533ca00640 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x55634876cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:49.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.308+0000 7f533d400640 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e8590 0x5563486f75a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f532c031870 tx=0x7f532c031ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5aae2c87574eefe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.308+0000 7f531ea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:50:49.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.308+0000 7f531ea00640 1 -- 172.21.15.46:0/2434633928 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c035040 con 0x5563486e8590 2024-09-10T07:50:49.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.308+0000 7f531ea00640 1 -- 172.21.15.46:0/2434633928 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f532c004ad0 con 0x5563486e8590 2024-09-10T07:50:49.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.308+0000 7f53427bf740 1 -- 172.21.15.46:0/2434633928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532c031ae0 con 0x5563486e8590 2024-09-10T07:50:49.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.309+0000 7f531ea00640 1 -- 172.21.15.46:0/2434633928 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f532c066050 con 0x5563486e8590 2024-09-10T07:50:49.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.310+0000 7f53427bf740 10 client.28910 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:49.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.310+0000 7f53427bf740 1 -- 172.21.15.46:0/2434633928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5563486df770 con 0x5563486e8590 2024-09-10T07:50:49.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.310+0000 7f531ea00640 1 -- 172.21.15.46:0/2434633928 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532c031ae0 con 0x5563486e8590 2024-09-10T07:50:49.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.314+0000 7f531ea00640 1 -- 172.21.15.46:0/2434633928 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f532c054030 con 0x5563486e8590 2024-09-10T07:50:49.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.314+0000 7f53427bf740 10 client.28910 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:49.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.314+0000 7f53427bf740 10 client.28910 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:50:49.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.314+0000 7f53427bf740 20 client.28910 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:49.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.314+0000 7f53427bf740 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55634876e6e0 0x556348770b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.314+0000 7f53427bf740 4 client.28910 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:50:49.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.314+0000 7f53427bf740 1 -- 172.21.15.46:0/2434633928 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556348655e60 con 0x55634876e6e0 2024-09-10T07:50:49.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.319+0000 7f533ca00640 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55634876e6e0 0x556348770b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.320+0000 7f533ca00640 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55634876e6e0 0x556348770b00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=290ac5b6e04a794c server_cookie=792fe7920b5a6ac1 in_seq=0 out_seq=0 2024-09-10T07:50:49.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.320+0000 7f531ea00640 10 client.28910 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:50:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.320+0000 7f531ea00640 1 -- 172.21.15.46:0/2434633928 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556348655e60 con 0x55634876e6e0 2024-09-10T07:50:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.320+0000 7f531ea00640 10 client.28910 handle_command_reply: tid=0 2024-09-10T07:50:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.320+0000 7f53427bf740 1 client.28910 shutdown 2024-09-10T07:50:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f53427bf740 1 -- 172.21.15.46:0/2434633928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e8590 msgr2=0x5563486f75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f53427bf740 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e8590 0x5563486f75a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f532c031870 tx=0x7f532c031ec0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f53427bf740 1 -- 172.21.15.46:0/2434633928 shutdown_connections 2024-09-10T07:50:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f53427bf740 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55634876e6e0 0x556348770b00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f531ea00640 0 client.28910 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:50:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f53427bf740 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563486f2e50 0x55634876cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f53427bf740 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563486e8590 0x5563486f75a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f53427bf740 1 --2- 172.21.15.46:0/2434633928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563486e7c70 0x5563486f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.321+0000 7f53427bf740 1 -- 172.21.15.46:0/2434633928 >> 172.21.15.46:0/2434633928 conn(0x5563486e4200 msgr2=0x5563486e2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.322+0000 7f53427bf740 1 -- 172.21.15.46:0/2434633928 shutdown_connections 2024-09-10T07:50:49.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.322+0000 7f53427bf740 1 -- 172.21.15.46:0/2434633928 wait complete. 2024-09-10T07:50:49.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.322+0000 7f53427bf740 20 client.28910 trim_cache size 0 max 16384 2024-09-10T07:50:49.323 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:50:49.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 -- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53200a2c00 msgr2=0x7f53200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53200a2c00 0x7f53200a50c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5330000f80 tx=0x7f533000c7f0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 -- 172.21.15.46:0/1182551614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533810e450 msgr2=0x7f53381132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533810e450 0x7f53381132d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f533000e9d0 tx=0x7f533000ee90 comp rx=0 tx=0).stop 2024-09-10T07:50:49.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 -- 172.21.15.46:0/1182551614 shutdown_connections 2024-09-10T07:50:49.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533810e450 0x7f53381132d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5338074e30 0x7f533811a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f53200a2c00 0x7f53200a50c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 --2- 172.21.15.46:0/1182551614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338074510 0x7f533811a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 -- 172.21.15.46:0/1182551614 >> 172.21.15.46:0/1182551614 conn(0x7f53380fd8b0 msgr2=0x7f53380ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 -- 172.21.15.46:0/1182551614 shutdown_connections 2024-09-10T07:50:49.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:49.327+0000 7f533fe00640 1 -- 172.21.15.46:0/1182551614 wait complete. 2024-09-10T07:50:50.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:50 smithi046 ceph-mon[30705]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-09-10T07:50:50.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:50:50.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:50 smithi181 ceph-mon[35397]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-09-10T07:50:50.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:50 smithi063 ceph-mon[35020]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-09-10T07:50:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 810.0 s, 461.4 tps, lat 217.810 ms stddev 258.321 2024-09-10T07:50:51.072 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:52.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:52 smithi181 ceph-mon[35397]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.5 MiB/s wr, 451 op/s 2024-09-10T07:50:52.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:52 smithi063 ceph-mon[35020]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.5 MiB/s wr, 451 op/s 2024-09-10T07:50:52.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:52 smithi046 ceph-mon[30705]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.5 MiB/s wr, 451 op/s 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.586+0000 7f29d1000640 1 -- 172.21.15.46:0/476605751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 msgr2=0x7f29cc110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.586+0000 7f29d1000640 1 --2- 172.21.15.46:0/476605751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 0x7f29cc110740 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009900 tx=0x7f29b802a410 comp rx=0 tx=0).stop 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.587+0000 7f29d1000640 1 -- 172.21.15.46:0/476605751 shutdown_connections 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.587+0000 7f29d1000640 1 --2- 172.21.15.46:0/476605751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 0x7f29cc110740 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.587+0000 7f29d1000640 1 --2- 172.21.15.46:0/476605751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29cc101150 0x7f29cc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.587+0000 7f29d1000640 1 --2- 172.21.15.46:0/476605751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc1007a0 0x7f29cc100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.587+0000 7f29d1000640 1 -- 172.21.15.46:0/476605751 >> 172.21.15.46:0/476605751 conn(0x7f29cc0fc610 msgr2=0x7f29cc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.588+0000 7f29d1000640 1 -- 172.21.15.46:0/476605751 shutdown_connections 2024-09-10T07:50:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.589+0000 7f29d1000640 1 -- 172.21.15.46:0/476605751 wait complete. 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.590+0000 7f29d1000640 1 Processor -- start 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.594+0000 7f29d1000640 1 -- start start 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.595+0000 7f29d1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29cc1007a0 0x7f29cc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.595+0000 7f29d1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc101150 0x7f29cc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.595+0000 7f29d1000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 0x7f29cc19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.595+0000 7f29d1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc114060 con 0x7f29cc101150 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.595+0000 7f29d1000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc113ee0 con 0x7f29cc1007a0 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.595+0000 7f29d1000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1141e0 con 0x7f29cc10e350 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.596+0000 7f29cbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 0x7f29cc19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.596+0000 7f29cbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 0x7f29cc19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43462/0 (socket says 172.21.15.46:43462) 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.596+0000 7f29caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc101150 0x7f29cc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.596+0000 7f29cbe00640 1 -- 172.21.15.46:0/2918740031 learned_addr learned my addr 172.21.15.46:0/2918740031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.596+0000 7f29cb400640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29cc1007a0 0x7f29cc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.597+0000 7f29cbe00640 1 -- 172.21.15.46:0/2918740031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29cc1007a0 msgr2=0x7f29cc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.597+0000 7f29cbe00640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29cc1007a0 0x7f29cc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.597+0000 7f29cbe00640 1 -- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc101150 msgr2=0x7f29cc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.597+0000 7f29cbe00640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc101150 0x7f29cc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.597+0000 7f29cbe00640 1 -- 172.21.15.46:0/2918740031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc19f4b0 con 0x7f29cc10e350 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.597+0000 7f29cb400640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29cc1007a0 0x7f29cc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.597+0000 7f29cbe00640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 0x7f29cc19edd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f29b8004770 tx=0x7f29b8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.598+0000 7f29c3e00640 1 -- 172.21.15.46:0/2918740031 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b803f070 con 0x7f29cc10e350 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.598+0000 7f29c3e00640 1 -- 172.21.15.46:0/2918740031 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b8031c90 con 0x7f29cc10e350 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.598+0000 7f29c3e00640 1 -- 172.21.15.46:0/2918740031 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8035030 con 0x7f29cc10e350 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.598+0000 7f29d1000640 1 -- 172.21.15.46:0/2918740031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29cc19f740 con 0x7f29cc10e350 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.599+0000 7f29d1000640 1 -- 172.21.15.46:0/2918740031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29cc19fbd0 con 0x7f29cc10e350 2024-09-10T07:50:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.603+0000 7f29c3e00640 1 -- 172.21.15.46:0/2918740031 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f29b8042070 con 0x7f29cc10e350 2024-09-10T07:50:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.605+0000 7f29d4fc4740 1 Processor -- start 2024-09-10T07:50:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.605+0000 7f29d4fc4740 1 -- start start 2024-09-10T07:50:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.606+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019530 0x564756019910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.606+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 0x564756014e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.606+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56475602d2a0 0x56475602f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.606+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564755f773e0 con 0x56475602d2a0 2024-09-10T07:50:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.606+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564755f1ed80 con 0x564756019530 2024-09-10T07:50:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.606+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564755f733a0 con 0x564756019e50 2024-09-10T07:50:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.607+0000 7f29caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56475602d2a0 0x56475602f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.607+0000 7f29caa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019530 0x564756019910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.607+0000 7f29caa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019530 0x564756019910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38580/0 (socket says 172.21.15.46:38580) 2024-09-10T07:50:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.607+0000 7f29caa00640 1 -- 172.21.15.46:0/1585084368 learned_addr learned my addr 172.21.15.46:0/1585084368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.608+0000 7f29cb400640 1 --2- 172.21.15.46:0/1585084368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 0x564756014e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.608+0000 7f29caa00640 1 -- 172.21.15.46:0/1585084368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 msgr2=0x564756014e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.608+0000 7f29caa00640 1 --2- 172.21.15.46:0/1585084368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 0x564756014e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.608+0000 7f29caa00640 1 -- 172.21.15.46:0/1585084368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019530 msgr2=0x564756019910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.608+0000 7f29caa00640 1 --2- 172.21.15.46:0/1585084368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019530 0x564756019910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.608+0000 7f29c3e00640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29b00a2cd0 0x7f29b00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.608+0000 7f29caa00640 1 -- 172.21.15.46:0/1585084368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564755f3dae0 con 0x56475602d2a0 2024-09-10T07:50:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.609+0000 7f29c3e00640 1 -- 172.21.15.46:0/2918740031 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f29b80ef5a0 con 0x7f29cc10e350 2024-09-10T07:50:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.609+0000 7f29caa00640 1 --2- 172.21.15.46:0/1585084368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56475602d2a0 0x56475602f690 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00ef30 tx=0x7f29bc014bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185821f422dbd2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.610+0000 7f29cbe00640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29b00a2cd0 0x7f29b00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.610+0000 7f29c0c00640 1 -- 172.21.15.46:0/1585084368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc004030 con 0x56475602d2a0 2024-09-10T07:50:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.610+0000 7f29c0c00640 1 -- 172.21.15.46:0/1585084368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29bc00bba0 con 0x56475602d2a0 2024-09-10T07:50:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.610+0000 7f29c0c00640 1 -- 172.21.15.46:0/1585084368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0039e0 con 0x56475602d2a0 2024-09-10T07:50:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.611+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1585084368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56475602d2a0 msgr2=0x56475602f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.611+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/1585084368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56475602d2a0 0x56475602f690 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00ef30 tx=0x7f29bc014bc0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.611+0000 7f29cbe00640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29b00a2cd0 0x7f29b00a5190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f29b80bb780 tx=0x7f29b8003f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.612+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1585084368 shutdown_connections 2024-09-10T07:50:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.612+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/1585084368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56475602d2a0 0x56475602f690 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.612+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/1585084368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 0x564756014e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.612+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/1585084368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019530 0x564756019910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.612+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1585084368 >> 172.21.15.46:0/1585084368 conn(0x564756015be0 msgr2=0x564756015fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.615+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1585084368 shutdown_connections 2024-09-10T07:50:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.615+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1585084368 wait complete. 2024-09-10T07:50:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.615+0000 7f29d4fc4740 1 -- start start 2024-09-10T07:50:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.618+0000 7f29d4fc4740 1 Processor -- start 2024-09-10T07:50:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.621+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x564756012370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.621+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x5647560128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.621+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56475602d2a0 0x564756012df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.621+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56475602fc90 con 0x564756019530 2024-09-10T07:50:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.621+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564755f320f0 con 0x564756019e50 2024-09-10T07:50:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.621+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56475602fe10 con 0x56475602d2a0 2024-09-10T07:50:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.621+0000 7f29d4fc4740 10 client.0 _ll_register_callbacks cb 0x564755f19bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.622+0000 7f29cb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x5647560128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.623+0000 7f29cb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x5647560128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38584/0 (socket says 172.21.15.46:38584) 2024-09-10T07:50:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.623+0000 7f29cb400640 1 -- 172.21.15.46:0/2839262546 learned_addr learned my addr 172.21.15.46:0/2839262546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.623+0000 7f29caa00640 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56475602d2a0 0x564756012df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.624+0000 7f29caa00640 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x564756012370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.624+0000 7f29caa00640 1 -- 172.21.15.46:0/2839262546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 msgr2=0x5647560128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.624+0000 7f29caa00640 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x5647560128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.624+0000 7f29caa00640 1 -- 172.21.15.46:0/2839262546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 msgr2=0x564756012370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.624+0000 7f29caa00640 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x564756012370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.624+0000 7f29caa00640 1 -- 172.21.15.46:0/2839262546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564755ce83c0 con 0x56475602d2a0 2024-09-10T07:50:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.626+0000 7f29caa00640 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56475602d2a0 0x564756012df0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f29bc005fd0 tx=0x7f29bc004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0402321c4874fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.626+0000 7f29b5600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.626+0000 7f29b5600640 1 -- 172.21.15.46:0/2839262546 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00b690 con 0x56475602d2a0 2024-09-10T07:50:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.626+0000 7f29b5600640 1 -- 172.21.15.46:0/2839262546 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f29bc00b830 con 0x56475602d2a0 2024-09-10T07:50:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.626+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2839262546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29bc00bba0 con 0x56475602d2a0 2024-09-10T07:50:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.627+0000 7f29b5600640 1 -- 172.21.15.46:0/2839262546 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f29bc031050 con 0x56475602d2a0 2024-09-10T07:50:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.627+0000 7f29d4fc4740 10 client.28828 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.627+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2839262546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56475603bdd0 con 0x56475602d2a0 2024-09-10T07:50:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.631+0000 7f29b5600640 1 -- 172.21.15.46:0/2839262546 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29bc00bba0 con 0x56475602d2a0 2024-09-10T07:50:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.632+0000 7f29b5600640 1 -- 172.21.15.46:0/2839262546 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f29bc00f040 con 0x56475602d2a0 2024-09-10T07:50:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.632+0000 7f29d4fc4740 10 client.28828 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.632+0000 7f29d4fc4740 10 client.28828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:50:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.633+0000 7f29d4fc4740 20 client.28828 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.633+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x56475603d180 0x56475603f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.633+0000 7f29d4fc4740 4 client.28828 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.633+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2839262546 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564755f84950 con 0x56475603d180 2024-09-10T07:50:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.634+0000 7f29cb400640 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x56475603d180 0x56475603f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.635+0000 7f29cb400640 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x56475603d180 0x56475603f570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c731405cc82f4d0c server_cookie=95c061153b4c75de in_seq=0 out_seq=0 2024-09-10T07:50:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.635+0000 7f29b5600640 10 client.28828 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:50:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.636+0000 7f29b5600640 1 -- 172.21.15.46:0/2839262546 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564755f84950 con 0x56475603d180 2024-09-10T07:50:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.636+0000 7f29b5600640 10 client.28828 handle_command_reply: tid=0 2024-09-10T07:50:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.637+0000 7f29d4fc4740 1 client.28828 shutdown 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.637+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2839262546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56475602d2a0 msgr2=0x564756012df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.637+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56475602d2a0 0x564756012df0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f29bc005fd0 tx=0x7f29bc004590 comp rx=0 tx=0).stop 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.638+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2839262546 shutdown_connections 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.638+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x56475603d180 0x56475603f570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.638+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56475602d2a0 0x564756012df0 secure :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f29bc005fd0 tx=0x7f29bc004590 comp rx=0 tx=0).stop 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.638+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x5647560128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.638+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2839262546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x564756012370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.638+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2839262546 >> 172.21.15.46:0/2839262546 conn(0x564756015be0 msgr2=0x56475601b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.640+0000 7f29b5600640 0 client.28828 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:50:52.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.640+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2839262546 shutdown_connections 2024-09-10T07:50:52.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.642+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2839262546 wait complete. 2024-09-10T07:50:52.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.645+0000 7f29d4fc4740 20 client.28828 trim_cache size 0 max 16384 2024-09-10T07:50:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.675+0000 7f29d4fc4740 1 Processor -- start 2024-09-10T07:50:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.676+0000 7f29d4fc4740 1 -- start start 2024-09-10T07:50:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.677+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019530 0x56475600c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.677+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x564756012c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.677+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647560110a0 0x5647560142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.677+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1ac8f0 con 0x5647560110a0 2024-09-10T07:50:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.677+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1ae4c0 con 0x564756019e50 2024-09-10T07:50:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.677+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1acfc0 con 0x564756019530 2024-09-10T07:50:52.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.678+0000 7f29caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019530 0x56475600c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.678+0000 7f29caa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019530 0x56475600c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43490/0 (socket says 172.21.15.46:43490) 2024-09-10T07:50:52.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.678+0000 7f29cb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x564756012c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.678+0000 7f29cb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x564756012c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38590/0 (socket says 172.21.15.46:38590) 2024-09-10T07:50:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.678+0000 7f29cb400640 1 -- 172.21.15.46:0/1764754341 learned_addr learned my addr 172.21.15.46:0/1764754341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.680+0000 7f29cb400640 1 -- 172.21.15.46:0/1764754341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019530 msgr2=0x56475600c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.681+0000 7f29cb400640 1 --2- 172.21.15.46:0/1764754341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019530 0x56475600c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.681+0000 7f29cb400640 1 -- 172.21.15.46:0/1764754341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647560110a0 msgr2=0x5647560142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.681+0000 7f29cb400640 1 --2- 172.21.15.46:0/1764754341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647560110a0 0x5647560142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.681+0000 7f29cb400640 1 -- 172.21.15.46:0/1764754341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564756014ce0 con 0x564756019e50 2024-09-10T07:50:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.681+0000 7f29caa00640 1 --2- 172.21.15.46:0/1764754341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019530 0x56475600c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.682+0000 7f29cb400640 1 --2- 172.21.15.46:0/1764754341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x564756012c40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x564756011c00 tx=0x7f29c4009970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79f6eaf79b9e990a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.684+0000 7f29b7e00640 1 -- 172.21.15.46:0/1764754341 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4027070 con 0x564756019e50 2024-09-10T07:50:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.684+0000 7f29b7e00640 1 -- 172.21.15.46:0/1764754341 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c4022470 con 0x564756019e50 2024-09-10T07:50:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.684+0000 7f29b7e00640 1 -- 172.21.15.46:0/1764754341 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c400be60 con 0x564756019e50 2024-09-10T07:50:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.686+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1764754341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 msgr2=0x564756012c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.686+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/1764754341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x564756012c40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x564756011c00 tx=0x7f29c4009970 comp rx=0 tx=0).stop 2024-09-10T07:50:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.689+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1764754341 shutdown_connections 2024-09-10T07:50:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.689+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/1764754341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647560110a0 0x5647560142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.689+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/1764754341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564756019e50 0x564756012c40 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.689+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/1764754341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019530 0x56475600c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.689+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1764754341 >> 172.21.15.46:0/1764754341 conn(0x564756015be0 msgr2=0x564756013bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.690+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1764754341 shutdown_connections 2024-09-10T07:50:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.690+0000 7f29d4fc4740 1 -- 172.21.15.46:0/1764754341 wait complete. 2024-09-10T07:50:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.690+0000 7f29d4fc4740 1 -- start start 2024-09-10T07:50:52.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.691+0000 7f29d4fc4740 1 Processor -- start 2024-09-10T07:50:52.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.692+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647560110a0 0x564756030e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.692+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x5647560313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.692+0000 7f29d4fc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 0x564756031900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.692+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1addf0 con 0x564756019530 2024-09-10T07:50:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.692+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1ad690 con 0x5647560110a0 2024-09-10T07:50:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.692+0000 7f29d4fc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1aa8d0 con 0x564756019e50 2024-09-10T07:50:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.692+0000 7f29d4fc4740 10 client.0 _ll_register_callbacks cb 0x7f29bc014630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:52.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.693+0000 7f29cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x5647560313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.693+0000 7f29cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x5647560313c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45794/0 (socket says 172.21.15.46:45794) 2024-09-10T07:50:52.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.693+0000 7f29cb400640 1 -- 172.21.15.46:0/2345715092 learned_addr learned my addr 172.21.15.46:0/2345715092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:52.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.694+0000 7f29caa00640 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647560110a0 0x564756030e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.694+0000 7f29cb400640 1 -- 172.21.15.46:0/2345715092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 msgr2=0x564756031900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.694+0000 7f29cb400640 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 0x564756031900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.694+0000 7f29cb400640 1 -- 172.21.15.46:0/2345715092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647560110a0 msgr2=0x564756030e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.694+0000 7f29cb400640 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647560110a0 0x564756030e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.694+0000 7f29cb400640 1 -- 172.21.15.46:0/2345715092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647560a6d30 con 0x564756019530 2024-09-10T07:50:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.695+0000 7f29cb400640 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x5647560313c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f29c40176f0 tx=0x7f29c4017ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d20962c9846dcba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.696+0000 7f29b5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:52.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.697+0000 7f29b5600640 1 -- 172.21.15.46:0/2345715092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4030060 con 0x564756019530 2024-09-10T07:50:52.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.697+0000 7f29b5600640 1 -- 172.21.15.46:0/2345715092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f29c4017e20 con 0x564756019530 2024-09-10T07:50:52.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.697+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2345715092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c4022470 con 0x564756019530 2024-09-10T07:50:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.698+0000 7f29b5600640 1 -- 172.21.15.46:0/2345715092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f29c400be60 con 0x564756019530 2024-09-10T07:50:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.700+0000 7f29d4fc4740 10 client.18075 fetch_fsmap learned FSMap version 54 2024-09-10T07:50:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.700+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2345715092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5647560a8010 con 0x564756019530 2024-09-10T07:50:52.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.701+0000 7f29b5600640 1 -- 172.21.15.46:0/2345715092 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c4022470 con 0x564756019530 2024-09-10T07:50:52.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.701+0000 7f29b5600640 1 -- 172.21.15.46:0/2345715092 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f29c4033050 con 0x564756019530 2024-09-10T07:50:52.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.701+0000 7f29d4fc4740 10 client.18075 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T07:50:52.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.701+0000 7f29d4fc4740 10 client.18075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:50:52.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.701+0000 7f29d4fc4740 20 client.18075 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.701+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5647560a86a0 0x5647560aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.701+0000 7f29d4fc4740 4 client.18075 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:50:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.701+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2345715092 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564755f7ee60 con 0x5647560a86a0 2024-09-10T07:50:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.702+0000 7f29caa00640 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5647560a86a0 0x5647560aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.702+0000 7f29caa00640 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5647560a86a0 0x5647560aaac0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=318549a18535a347 server_cookie=ab554642a99d13a8 in_seq=0 out_seq=0 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.703+0000 7f29b5600640 10 client.18075 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.703+0000 7f29b5600640 1 -- 172.21.15.46:0/2345715092 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564755f7ee60 con 0x5647560a86a0 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.703+0000 7f29b5600640 10 client.18075 handle_command_reply: tid=0 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.703+0000 7f29d4fc4740 1 client.18075 shutdown 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.703+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2345715092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 msgr2=0x5647560313c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.703+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x5647560313c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f29c40176f0 tx=0x7f29c4017ab0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.703+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2345715092 shutdown_connections 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.704+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x5647560a86a0 0x5647560aaac0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.704+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564756019e50 0x564756031900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.704+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564756019530 0x5647560313c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.704+0000 7f29d4fc4740 1 --2- 172.21.15.46:0/2345715092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647560110a0 0x564756030e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.704+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2345715092 >> 172.21.15.46:0/2345715092 conn(0x564756015be0 msgr2=0x564756013bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.704+0000 7f29b5600640 0 client.18075 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:50:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.704+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2345715092 shutdown_connections 2024-09-10T07:50:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.704+0000 7f29d4fc4740 1 -- 172.21.15.46:0/2345715092 wait complete. 2024-09-10T07:50:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.705+0000 7f29d4fc4740 20 client.18075 trim_cache size 0 max 16384 2024-09-10T07:50:52.707 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:50:52.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.707+0000 7f29d1000640 1 -- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29b00a2cd0 msgr2=0x7f29b00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.707+0000 7f29d1000640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29b00a2cd0 0x7f29b00a5190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f29b80bb780 tx=0x7f29b8003f90 comp rx=0 tx=0).stop 2024-09-10T07:50:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.707+0000 7f29d1000640 1 -- 172.21.15.46:0/2918740031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 msgr2=0x7f29cc19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.707+0000 7f29d1000640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 0x7f29cc19edd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f29b8004770 tx=0x7f29b8003b90 comp rx=0 tx=0).stop 2024-09-10T07:50:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.708+0000 7f29d1000640 1 -- 172.21.15.46:0/2918740031 shutdown_connections 2024-09-10T07:50:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.708+0000 7f29d1000640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29cc10e350 0x7f29cc19edd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.708+0000 7f29d1000640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f29b00a2cd0 0x7f29b00a5190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.708+0000 7f29d1000640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc101150 0x7f29cc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.708+0000 7f29d1000640 1 --2- 172.21.15.46:0/2918740031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29cc1007a0 0x7f29cc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.708+0000 7f29d1000640 1 -- 172.21.15.46:0/2918740031 >> 172.21.15.46:0/2918740031 conn(0x7f29cc0fc610 msgr2=0x7f29cc0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.709+0000 7f29d1000640 1 -- 172.21.15.46:0/2918740031 shutdown_connections 2024-09-10T07:50:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:52.709+0000 7f29d1000640 1 -- 172.21.15.46:0/2918740031 wait complete. 2024-09-10T07:50:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:54 smithi181 ceph-mon[35397]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-09-10T07:50:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:54 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:50:54.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:54 smithi063 ceph-mon[35020]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-09-10T07:50:54.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:54 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:50:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:54 smithi046 ceph-mon[30705]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-09-10T07:50:54.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:54 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:50:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.790+0000 7fe967200640 1 -- 172.21.15.46:0/303061609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe96810dbb0 msgr2=0x7fe968110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.790+0000 7fe967200640 1 --2- 172.21.15.46:0/303061609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe96810dbb0 0x7fe968110070 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe950009860 tx=0x7fe95002a380 comp rx=0 tx=0).stop 2024-09-10T07:50:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.791+0000 7fe967200640 1 -- 172.21.15.46:0/303061609 shutdown_connections 2024-09-10T07:50:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.791+0000 7fe967200640 1 --2- 172.21.15.46:0/303061609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe96810dbb0 0x7fe968110070 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.791+0000 7fe967200640 1 --2- 172.21.15.46:0/303061609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968073680 0x7fe96810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.791+0000 7fe967200640 1 --2- 172.21.15.46:0/303061609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe968072cb0 0x7fe9680730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.791+0000 7fe967200640 1 -- 172.21.15.46:0/303061609 >> 172.21.15.46:0/303061609 conn(0x7fe96806eb50 msgr2=0x7fe968070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.798+0000 7fe967200640 1 -- 172.21.15.46:0/303061609 shutdown_connections 2024-09-10T07:50:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.798+0000 7fe967200640 1 -- 172.21.15.46:0/303061609 wait complete. 2024-09-10T07:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.799+0000 7fe967200640 1 Processor -- start 2024-09-10T07:50:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.805+0000 7fe967200640 1 -- start start 2024-09-10T07:50:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.806+0000 7fe967200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968073680 0x7fe96819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.806+0000 7fe967200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe96810dbb0 0x7fe96819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.807+0000 7fe965e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968073680 0x7fe96819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.807+0000 7fe967200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9681a3a10 0x7fe9681a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.807+0000 7fe965e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968073680 0x7fe96819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38620/0 (socket says 172.21.15.46:38620) 2024-09-10T07:50:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.807+0000 7fe967200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe968113650 con 0x7fe9681a3a10 2024-09-10T07:50:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.807+0000 7fe967200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681134d0 con 0x7fe968073680 2024-09-10T07:50:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.807+0000 7fe967200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681137d0 con 0x7fe96810dbb0 2024-09-10T07:50:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.807+0000 7fe965e00640 1 -- 172.21.15.46:0/4156161626 learned_addr learned my addr 172.21.15.46:0/4156161626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.808+0000 7fe965400640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe96810dbb0 0x7fe96819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.808+0000 7fe965e00640 1 -- 172.21.15.46:0/4156161626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe96810dbb0 msgr2=0x7fe96819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.808+0000 7fe965e00640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe96810dbb0 0x7fe96819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.808+0000 7fe965e00640 1 -- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9681a3a10 msgr2=0x7fe9681a5e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:50:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.808+0000 7fe965e00640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9681a3a10 0x7fe9681a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.808+0000 7fe965e00640 1 -- 172.21.15.46:0/4156161626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9681a64a0 con 0x7fe968073680 2024-09-10T07:50:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.809+0000 7fe965400640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe96810dbb0 0x7fe96819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.809+0000 7fe965e00640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968073680 0x7fe96819e6f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe96001fd30 tx=0x7fe96001fe20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.810+0000 7fe95ea00640 1 -- 172.21.15.46:0/4156161626 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960004b20 con 0x7fe968073680 2024-09-10T07:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.810+0000 7fe95ea00640 1 -- 172.21.15.46:0/4156161626 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe960004cc0 con 0x7fe968073680 2024-09-10T07:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.810+0000 7fe95ea00640 1 -- 172.21.15.46:0/4156161626 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960028440 con 0x7fe968073680 2024-09-10T07:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.810+0000 7fe967200640 1 -- 172.21.15.46:0/4156161626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9681a6790 con 0x7fe968073680 2024-09-10T07:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.810+0000 7fe967200640 1 -- 172.21.15.46:0/4156161626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9681a6d10 con 0x7fe968073680 2024-09-10T07:50:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.815+0000 7fe95ea00640 1 -- 172.21.15.46:0/4156161626 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe96001db10 con 0x7fe968073680 2024-09-10T07:50:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.820+0000 7fe970061740 1 Processor -- start 2024-09-10T07:50:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.820+0000 7fe95ea00640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe94c0a2c00 0x7fe94c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.820+0000 7fe95ea00640 1 -- 172.21.15.46:0/4156161626 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe9600e0040 con 0x7fe968073680 2024-09-10T07:50:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.821+0000 7fe970061740 1 -- start start 2024-09-10T07:50:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.821+0000 7fe970061740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67260 0x564d7cc67660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.821+0000 7fe970061740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67ba0 0x564d7cc6c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.821+0000 7fe970061740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc6cc30 0x564d7cc6f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.821+0000 7fe970061740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564d7cbcc100 con 0x564d7cc67ba0 2024-09-10T07:50:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.821+0000 7fe970061740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d7cb741b0 con 0x564d7cc67260 2024-09-10T07:50:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.821+0000 7fe970061740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564d7cbc80c0 con 0x564d7cc6cc30 2024-09-10T07:50:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc6cc30 0x564d7cc6f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc6cc30 0x564d7cc6f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43520/0 (socket says 172.21.15.46:43520) 2024-09-10T07:50:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 -- 172.21.15.46:0/726169018 learned_addr learned my addr 172.21.15.46:0/726169018 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe94c0a2c00 0x7fe94c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe965e00640 1 --2- 172.21.15.46:0/726169018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67260 0x564d7cc67660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 -- 172.21.15.46:0/726169018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67260 msgr2=0x564d7cc67660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 --2- 172.21.15.46:0/726169018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67260 0x564d7cc67660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 -- 172.21.15.46:0/726169018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67ba0 msgr2=0x564d7cc6c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 --2- 172.21.15.46:0/726169018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67ba0 0x564d7cc6c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.822+0000 7fe966800640 1 -- 172.21.15.46:0/726169018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7c8df210 con 0x564d7cc6cc30 2024-09-10T07:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.826+0000 7fe966800640 1 --2- 172.21.15.46:0/726169018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc6cc30 0x564d7cc6f0f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe950005b60 tx=0x7fe950030fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cedbb15737442cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.830+0000 7fe966800640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe94c0a2c00 0x7fe94c0a50c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe950005e00 tx=0x7fe950044000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.830+0000 7fe957400640 1 -- 172.21.15.46:0/726169018 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95004d070 con 0x564d7cc6cc30 2024-09-10T07:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.830+0000 7fe957400640 1 -- 172.21.15.46:0/726169018 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe950044070 con 0x564d7cc6cc30 2024-09-10T07:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.830+0000 7fe957400640 1 -- 172.21.15.46:0/726169018 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950037040 con 0x564d7cc6cc30 2024-09-10T07:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.831+0000 7fe970061740 1 -- 172.21.15.46:0/726169018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc6cc30 msgr2=0x564d7cc6f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.831+0000 7fe970061740 1 --2- 172.21.15.46:0/726169018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc6cc30 0x564d7cc6f0f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe950005b60 tx=0x7fe950030fc0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.832+0000 7fe970061740 1 -- 172.21.15.46:0/726169018 shutdown_connections 2024-09-10T07:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.832+0000 7fe970061740 1 --2- 172.21.15.46:0/726169018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc6cc30 0x564d7cc6f0f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.832+0000 7fe970061740 1 --2- 172.21.15.46:0/726169018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67ba0 0x564d7cc6c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.832+0000 7fe970061740 1 --2- 172.21.15.46:0/726169018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67260 0x564d7cc67660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.832+0000 7fe970061740 1 -- 172.21.15.46:0/726169018 >> 172.21.15.46:0/726169018 conn(0x564d7cc628d0 msgr2=0x564d7cc62cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.833+0000 7fe970061740 1 -- 172.21.15.46:0/726169018 shutdown_connections 2024-09-10T07:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.833+0000 7fe970061740 1 -- 172.21.15.46:0/726169018 wait complete. 2024-09-10T07:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.833+0000 7fe970061740 1 -- start start 2024-09-10T07:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.834+0000 7fe970061740 1 Processor -- start 2024-09-10T07:50:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.835+0000 7fe970061740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc71470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.836+0000 7fe970061740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.836+0000 7fe970061740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 0x564d7cc789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.836+0000 7fe970061740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564d7cc6fba0 con 0x564d7cc67260 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.836+0000 7fe965400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.836+0000 7fe965400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43526/0 (socket says 172.21.15.46:43526) 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.836+0000 7fe965400640 1 -- 172.21.15.46:0/28975967 learned_addr learned my addr 172.21.15.46:0/28975967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.836+0000 7fe965e00640 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc71470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.836+0000 7fe966800640 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 0x564d7cc789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.837+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d7cb55090 con 0x564d7cc6cc30 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.837+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x564d7cc6fd20 con 0x564d7cc67ba0 2024-09-10T07:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.837+0000 7fe970061740 10 client.0 _ll_register_callbacks cb 0x564d7cba9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.838+0000 7fe965400640 1 -- 172.21.15.46:0/28975967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 msgr2=0x564d7cc789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.838+0000 7fe965400640 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 0x564d7cc789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.838+0000 7fe965400640 1 -- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 msgr2=0x564d7cc71470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.838+0000 7fe965400640 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc71470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.838+0000 7fe965400640 1 -- 172.21.15.46:0/28975967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7c9ab620 con 0x564d7cc67ba0 2024-09-10T07:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.838+0000 7fe965e00640 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc71470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.839+0000 7fe94be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.839+0000 7fe965400640 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc719b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe95800af30 tx=0x7fe95800c5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8a558b15c38a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.839+0000 7fe94be00640 1 -- 172.21.15.46:0/28975967 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958015070 con 0x564d7cc67ba0 2024-09-10T07:50:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.840+0000 7fe94be00640 1 -- 172.21.15.46:0/28975967 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe9580115f0 con 0x564d7cc67ba0 2024-09-10T07:50:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.840+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe950044070 con 0x564d7cc67ba0 2024-09-10T07:50:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.840+0000 7fe94be00640 1 -- 172.21.15.46:0/28975967 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe958021050 con 0x564d7cc67ba0 2024-09-10T07:50:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.841+0000 7fe970061740 10 client.28840 fetch_fsmap learned FSMap version 55 2024-09-10T07:50:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.841+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564d7cc7b8b0 con 0x564d7cc67ba0 2024-09-10T07:50:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.841+0000 7fe94be00640 1 -- 172.21.15.46:0/28975967 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe950044070 con 0x564d7cc67ba0 2024-09-10T07:50:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.842+0000 7fe94be00640 1 -- 172.21.15.46:0/28975967 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8386+0+0 (secure 0 0 0) 0x7fe958019aa0 con 0x564d7cc67ba0 2024-09-10T07:50:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.842+0000 7fe970061740 10 client.28840 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T07:50:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.842+0000 7fe970061740 10 client.28840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.842+0000 7fe970061740 20 client.28840 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.843+0000 7fe970061740 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564d7cc7cf80 0x564d7cc7f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.845+0000 7fe970061740 4 client.28840 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.845+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d7cbd9830 con 0x564d7cc7cf80 2024-09-10T07:50:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.845+0000 7fe965e00640 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564d7cc7cf80 0x564d7cc7f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.846+0000 7fe965e00640 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564d7cc7cf80 0x564d7cc7f370 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753bbf19f6114c03 server_cookie=c045cae94b053810 in_seq=0 out_seq=0 2024-09-10T07:50:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.846+0000 7fe94be00640 10 client.28840 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.847+0000 7fe94be00640 1 -- 172.21.15.46:0/28975967 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d7cbd9830 con 0x564d7cc7cf80 2024-09-10T07:50:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.847+0000 7fe94be00640 10 client.28840 handle_command_reply: tid=0 2024-09-10T07:50:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 client.28840 shutdown 2024-09-10T07:50:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 msgr2=0x564d7cc719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc719b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe95800af30 tx=0x7fe95800c5e0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 shutdown_connections 2024-09-10T07:50:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564d7cc7cf80 0x564d7cc7f370 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 0x564d7cc789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc719b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 --2- 172.21.15.46:0/28975967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc71470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 >> 172.21.15.46:0/28975967 conn(0x564d7cc628d0 msgr2=0x564d7cc640e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.848+0000 7fe94be00640 0 client.28840 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.849+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 shutdown_connections 2024-09-10T07:50:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.849+0000 7fe970061740 1 -- 172.21.15.46:0/28975967 wait complete. 2024-09-10T07:50:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.849+0000 7fe970061740 20 client.28840 trim_cache size 0 max 16384 2024-09-10T07:50:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.866+0000 7fe970061740 1 Processor -- start 2024-09-10T07:50:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.866+0000 7fe970061740 1 -- start start 2024-09-10T07:50:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.866+0000 7fe970061740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67260 0x564d7cc63cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.866+0000 7fe970061740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67ba0 0x564d7cc789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.866+0000 7fe970061740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc6cc30 0x564d7cc64200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.866+0000 7fe970061740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b1200 con 0x564d7cc6cc30 2024-09-10T07:50:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.866+0000 7fe970061740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b2e00 con 0x564d7cc67ba0 2024-09-10T07:50:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.866+0000 7fe970061740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b18d0 con 0x564d7cc67260 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67ba0 0x564d7cc789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe966800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc6cc30 0x564d7cc64200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67ba0 0x564d7cc789b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:38646/0 (socket says 172.21.15.46:38646) 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965400640 1 -- 172.21.15.46:0/2893338484 learned_addr learned my addr 172.21.15.46:0/2893338484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965e00640 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67260 0x564d7cc63cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965400640 1 -- 172.21.15.46:0/2893338484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67260 msgr2=0x564d7cc63cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965400640 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67260 0x564d7cc63cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965400640 1 -- 172.21.15.46:0/2893338484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc6cc30 msgr2=0x564d7cc64200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965400640 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc6cc30 0x564d7cc64200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe965400640 1 -- 172.21.15.46:0/2893338484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7cc626f0 con 0x564d7cc67ba0 2024-09-10T07:50:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.867+0000 7fe966800640 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc6cc30 0x564d7cc64200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.868+0000 7fe965400640 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67ba0 0x564d7cc789b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe958008010 tx=0x7fe95800a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a38497439dbdf8ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.868+0000 7fe956a00640 1 -- 172.21.15.46:0/2893338484 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958015040 con 0x564d7cc67ba0 2024-09-10T07:50:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.868+0000 7fe956a00640 1 -- 172.21.15.46:0/2893338484 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95801e070 con 0x564d7cc67ba0 2024-09-10T07:50:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.868+0000 7fe956a00640 1 -- 172.21.15.46:0/2893338484 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958003690 con 0x564d7cc67ba0 2024-09-10T07:50:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 -- 172.21.15.46:0/2893338484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67ba0 msgr2=0x564d7cc789b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67ba0 0x564d7cc789b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe958008010 tx=0x7fe95800a290 comp rx=0 tx=0).stop 2024-09-10T07:50:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 -- 172.21.15.46:0/2893338484 shutdown_connections 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc6cc30 0x564d7cc64200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc67ba0 0x564d7cc789b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 --2- 172.21.15.46:0/2893338484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67260 0x564d7cc63cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 -- 172.21.15.46:0/2893338484 >> 172.21.15.46:0/2893338484 conn(0x564d7cc628d0 msgr2=0x564d7cc61a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 -- 172.21.15.46:0/2893338484 shutdown_connections 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 -- 172.21.15.46:0/2893338484 wait complete. 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 -- start start 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.869+0000 7fe970061740 1 Processor -- start 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe970061740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe970061740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc76710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe970061740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 0x564d7cc77720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe970061740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b2730 con 0x564d7cc67260 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe970061740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b1fd0 con 0x564d7cc6cc30 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe970061740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681af1e0 con 0x564d7cc67ba0 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe970061740 10 client.0 _ll_register_callbacks cb 0x7fe958006ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc76710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc76710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43548/0 (socket says 172.21.15.46:43548) 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965400640 1 -- 172.21.15.46:0/3959174898 learned_addr learned my addr 172.21.15.46:0/3959174898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe966800640 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 0x564d7cc77720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965400640 1 -- 172.21.15.46:0/3959174898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 msgr2=0x564d7cc77720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965400640 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 0x564d7cc77720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965400640 1 -- 172.21.15.46:0/3959174898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 msgr2=0x564d7cc761d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965400640 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc761d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965400640 1 -- 172.21.15.46:0/3959174898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7cc77d30 con 0x564d7cc67ba0 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.870+0000 7fe965e00640 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.871+0000 7fe965400640 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc76710 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe9580186c0 tx=0x7fe9580186f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0636a817c40221e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.871+0000 7fe94be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.871+0000 7fe94be00640 1 -- 172.21.15.46:0/3959174898 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958033060 con 0x564d7cc67ba0 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.871+0000 7fe970061740 1 -- 172.21.15.46:0/3959174898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95801e070 con 0x564d7cc67ba0 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.871+0000 7fe94be00640 1 -- 172.21.15.46:0/3959174898 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe958004040 con 0x564d7cc67ba0 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.871+0000 7fe94be00640 1 -- 172.21.15.46:0/3959174898 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe95803a050 con 0x564d7cc67ba0 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.871+0000 7fe970061740 10 client.28846 fetch_fsmap learned FSMap version 55 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.871+0000 7fe970061740 1 -- 172.21.15.46:0/3959174898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564d7cce7d20 con 0x564d7cc67ba0 2024-09-10T07:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe94be00640 1 -- 172.21.15.46:0/3959174898 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95801e070 con 0x564d7cc67ba0 2024-09-10T07:50:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe94be00640 1 -- 172.21.15.46:0/3959174898 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 55) ==== 8386+0+0 (secure 0 0 0) 0x7fe9580193b0 con 0x564d7cc67ba0 2024-09-10T07:50:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe970061740 10 client.28846 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T07:50:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe970061740 10 client.28846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe970061740 20 client.28846 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe970061740 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564d7cce85f0 0x564d7cceaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe970061740 4 client.28846 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:50:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe970061740 1 -- 172.21.15.46:0/3959174898 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d7cbd3b80 con 0x564d7cce85f0 2024-09-10T07:50:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.872+0000 7fe965e00640 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564d7cce85f0 0x564d7cceaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.873+0000 7fe94be00640 10 client.28846 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:54.873+0000 7fe965e00640 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564d7cce85f0 0x564d7cceaa10 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be1bd3cb3185fe4 server_cookie=c0eb2b571b2b6286 in_seq=0 out_seq=0 2024-09-10T07:50:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.044+0000 7fe94be00640 1 -- 172.21.15.46:0/3959174898 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d7cbd3b80 con 0x564d7cce85f0 2024-09-10T07:50:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.044+0000 7fe94be00640 10 client.28846 handle_command_reply: tid=0 2024-09-10T07:50:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.044+0000 7fe970061740 1 client.28846 shutdown 2024-09-10T07:50:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.044+0000 7fe970061740 1 -- 172.21.15.46:0/3959174898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 msgr2=0x564d7cc76710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.044+0000 7fe970061740 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc76710 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe9580186c0 tx=0x7fe9580186f0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.045+0000 7fe970061740 1 -- 172.21.15.46:0/3959174898 shutdown_connections 2024-09-10T07:50:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.045+0000 7fe970061740 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x564d7cce85f0 0x564d7cceaa10 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.045+0000 7fe970061740 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d7cc6cc30 0x564d7cc77720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.045+0000 7fe970061740 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d7cc67ba0 0x564d7cc76710 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.045+0000 7fe970061740 1 --2- 172.21.15.46:0/3959174898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d7cc67260 0x564d7cc761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.045+0000 7fe970061740 1 -- 172.21.15.46:0/3959174898 >> 172.21.15.46:0/3959174898 conn(0x564d7cc628d0 msgr2=0x564d7cc62cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.045+0000 7fe94be00640 0 client.28846 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.046+0000 7fe970061740 1 -- 172.21.15.46:0/3959174898 shutdown_connections 2024-09-10T07:50:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.046+0000 7fe970061740 1 -- 172.21.15.46:0/3959174898 wait complete. 2024-09-10T07:50:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.048+0000 7fe970061740 20 client.28846 trim_cache size 0 max 16384 2024-09-10T07:50:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:50:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:50:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:50:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:50:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.049+0000 7fe967200640 1 -- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe94c0a2c00 msgr2=0x7fe94c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.049+0000 7fe967200640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe94c0a2c00 0x7fe94c0a50c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe950005e00 tx=0x7fe950044000 comp rx=0 tx=0).stop 2024-09-10T07:50:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.049+0000 7fe967200640 1 -- 172.21.15.46:0/4156161626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968073680 msgr2=0x7fe96819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.049+0000 7fe967200640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968073680 0x7fe96819e6f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe96001fd30 tx=0x7fe96001fe20 comp rx=0 tx=0).stop 2024-09-10T07:50:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.050+0000 7fe967200640 1 -- 172.21.15.46:0/4156161626 shutdown_connections 2024-09-10T07:50:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.050+0000 7fe967200640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe94c0a2c00 0x7fe94c0a50c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.050+0000 7fe967200640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9681a3a10 0x7fe9681a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.050+0000 7fe967200640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe96810dbb0 0x7fe96819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.050+0000 7fe967200640 1 --2- 172.21.15.46:0/4156161626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968073680 0x7fe96819e6f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.050+0000 7fe967200640 1 -- 172.21.15.46:0/4156161626 >> 172.21.15.46:0/4156161626 conn(0x7fe96806eb50 msgr2=0x7fe96810c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.050+0000 7fe967200640 1 -- 172.21.15.46:0/4156161626 shutdown_connections 2024-09-10T07:50:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:50:55.050+0000 7fe967200640 1 -- 172.21.15.46:0/4156161626 wait complete. 2024-09-10T07:50:55.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:50:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 815.0 s, 520.1 tps, lat 194.402 ms stddev 234.309 2024-09-10T07:50:56.316 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:50:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:56 smithi181 ceph-mon[35397]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-09-10T07:50:56.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:56 smithi063 ceph-mon[35020]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-09-10T07:50:56.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:56 smithi046 ceph-mon[30705]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-09-10T07:50:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.214+0000 7f3abf200640 1 -- 172.21.15.46:0/1098822066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 msgr2=0x7f3ac0107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.214+0000 7f3abf200640 1 --2- 172.21.15.46:0/1098822066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 0x7f3ac0107da0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009c10 tx=0x7f3ab802a700 comp rx=0 tx=0).stop 2024-09-10T07:50:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.215+0000 7f3abf200640 1 -- 172.21.15.46:0/1098822066 shutdown_connections 2024-09-10T07:50:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.215+0000 7f3abf200640 1 --2- 172.21.15.46:0/1098822066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac0102360 0x7f3ac010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.215+0000 7f3abf200640 1 --2- 172.21.15.46:0/1098822066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ac01019c0 0x7f3ac0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.215+0000 7f3abf200640 1 --2- 172.21.15.46:0/1098822066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 0x7f3ac0107da0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.215+0000 7f3abf200640 1 -- 172.21.15.46:0/1098822066 >> 172.21.15.46:0/1098822066 conn(0x7f3ac00fd740 msgr2=0x7f3ac00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.215+0000 7f3abf200640 1 -- 172.21.15.46:0/1098822066 shutdown_connections 2024-09-10T07:50:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.215+0000 7f3abf200640 1 -- 172.21.15.46:0/1098822066 wait complete. 2024-09-10T07:50:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.216+0000 7f3abf200640 1 Processor -- start 2024-09-10T07:50:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.217+0000 7f3abf200640 1 -- start start 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.217+0000 7f3abf200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ac01019c0 0x7f3ac01a3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.217+0000 7f3abf200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac0102360 0x7f3ac01a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.217+0000 7f3abf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 0x7f3ac010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.217+0000 7f3abf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0111d90 con 0x7f3ac01079c0 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.217+0000 7f3abf200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0111c10 con 0x7f3ac01019c0 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.217+0000 7f3abf200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0111f10 con 0x7f3ac0102360 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.217+0000 7f3abde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ac01019c0 0x7f3ac01a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ac01019c0 0x7f3ac01a3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59540/0 (socket says 172.21.15.46:59540) 2024-09-10T07:50:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abde00640 1 -- 172.21.15.46:0/110833691 learned_addr learned my addr 172.21.15.46:0/110833691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abd400640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac0102360 0x7f3ac01a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abe800640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 0x7f3ac010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abde00640 1 -- 172.21.15.46:0/110833691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac0102360 msgr2=0x7f3ac01a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abde00640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac0102360 0x7f3ac01a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abde00640 1 -- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 msgr2=0x7f3ac010da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abde00640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 0x7f3ac010da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abde00640 1 -- 172.21.15.46:0/110833691 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac010e1e0 con 0x7f3ac01019c0 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.218+0000 7f3abe800640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 0x7f3ac010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.219+0000 7f3abd400640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac0102360 0x7f3ac01a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:57.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.219+0000 7f3abde00640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ac01019c0 0x7f3ac01a3500 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009be0 tx=0x7f3ab8030980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.220+0000 7f3ab6a00640 1 -- 172.21.15.46:0/110833691 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8041020 con 0x7f3ac01019c0 2024-09-10T07:50:57.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.220+0000 7f3ab6a00640 1 -- 172.21.15.46:0/110833691 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab8030e30 con 0x7f3ac01019c0 2024-09-10T07:50:57.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.220+0000 7f3ab6a00640 1 -- 172.21.15.46:0/110833691 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8036040 con 0x7f3ac01019c0 2024-09-10T07:50:57.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.221+0000 7f3abf200640 1 -- 172.21.15.46:0/110833691 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac010e470 con 0x7f3ac01019c0 2024-09-10T07:50:57.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.222+0000 7f3abf200640 1 -- 172.21.15.46:0/110833691 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac01ab430 con 0x7f3ac01019c0 2024-09-10T07:50:57.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.228+0000 7f3ab6a00640 1 -- 172.21.15.46:0/110833691 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ab8044020 con 0x7f3ac01019c0 2024-09-10T07:50:57.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.230+0000 7f3ab6a00640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3aa40a2d50 0x7f3aa40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.231+0000 7f3ab6a00640 1 -- 172.21.15.46:0/110833691 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ab80f00e0 con 0x7f3ac01019c0 2024-09-10T07:50:57.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.232+0000 7f3abd400640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3aa40a2d50 0x7f3aa40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.233+0000 7f3abd400640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3aa40a2d50 0x7f3aa40a5210 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0005fd0 tx=0x7f3ab0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.234+0000 7f3ac780c740 1 Processor -- start 2024-09-10T07:50:57.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.235+0000 7f3ac780c740 1 -- start start 2024-09-10T07:50:57.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.236+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84870220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.236+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac8487aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.236+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac8487d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.236+0000 7f3ac780c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac847d63e0 con 0x55ac84870760 2024-09-10T07:50:57.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.236+0000 7f3ac780c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ac8477dd80 con 0x55ac8486fe40 2024-09-10T07:50:57.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.236+0000 7f3ac780c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ac847d23a0 con 0x55ac8487b020 2024-09-10T07:50:57.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84870220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abe800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac8487d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abe800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac8487d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:32950/0 (socket says 172.21.15.46:32950) 2024-09-10T07:50:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abe800640 1 -- 172.21.15.46:0/2222211482 learned_addr learned my addr 172.21.15.46:0/2222211482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abd400640 1 --2- 172.21.15.46:0/2222211482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac8487aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abe800640 1 -- 172.21.15.46:0/2222211482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 msgr2=0x55ac84870220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abe800640 1 --2- 172.21.15.46:0/2222211482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84870220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abe800640 1 -- 172.21.15.46:0/2222211482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 msgr2=0x55ac8487aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abe800640 1 --2- 172.21.15.46:0/2222211482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac8487aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.237+0000 7f3abe800640 1 -- 172.21.15.46:0/2222211482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8479cae0 con 0x55ac8487b020 2024-09-10T07:50:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.238+0000 7f3abe800640 1 --2- 172.21.15.46:0/2222211482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac8487d410 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00bca0 tx=0x7f3aac00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce5c9b8b0a7c2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.240+0000 7f3aab400640 1 -- 172.21.15.46:0/2222211482 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac015070 con 0x55ac8487b020 2024-09-10T07:50:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.240+0000 7f3aab400640 1 -- 172.21.15.46:0/2222211482 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac005500 con 0x55ac8487b020 2024-09-10T07:50:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.240+0000 7f3aab400640 1 -- 172.21.15.46:0/2222211482 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0116a0 con 0x55ac8487b020 2024-09-10T07:50:57.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.245+0000 7f3ac780c740 1 -- 172.21.15.46:0/2222211482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 msgr2=0x55ac8487d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.245+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2222211482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac8487d410 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00bca0 tx=0x7f3aac00f730 comp rx=0 tx=0).stop 2024-09-10T07:50:57.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.246+0000 7f3ac780c740 1 -- 172.21.15.46:0/2222211482 shutdown_connections 2024-09-10T07:50:57.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.246+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2222211482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac8487d410 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.246+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2222211482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac8487aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.246+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2222211482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84870220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.246+0000 7f3ac780c740 1 -- 172.21.15.46:0/2222211482 >> 172.21.15.46:0/2222211482 conn(0x55ac8486c3d0 msgr2=0x55ac8486c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.246+0000 7f3ac780c740 1 -- 172.21.15.46:0/2222211482 shutdown_connections 2024-09-10T07:50:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.246+0000 7f3ac780c740 1 -- 172.21.15.46:0/2222211482 wait complete. 2024-09-10T07:50:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.247+0000 7f3ac780c740 1 -- start start 2024-09-10T07:50:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.247+0000 7f3ac780c740 1 Processor -- start 2024-09-10T07:50:57.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.248+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84877fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.248+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac848784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.248+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac84878a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.249+0000 7f3ac780c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac8487dbf0 con 0x55ac84870760 2024-09-10T07:50:57.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.249+0000 7f3ac780c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ac847910f0 con 0x55ac8486fe40 2024-09-10T07:50:57.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.249+0000 7f3ac780c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ac8487dd70 con 0x55ac8487b020 2024-09-10T07:50:57.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.249+0000 7f3abde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84877fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.249+0000 7f3abde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84877fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59556/0 (socket says 172.21.15.46:59556) 2024-09-10T07:50:57.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.249+0000 7f3abde00640 1 -- 172.21.15.46:0/2858089504 learned_addr learned my addr 172.21.15.46:0/2858089504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:57.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.250+0000 7f3ac780c740 10 client.0 _ll_register_callbacks cb 0x55ac84778bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:57.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.250+0000 7f3abd400640 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac848784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.250+0000 7f3abe800640 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac84878a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.251+0000 7f3abde00640 1 -- 172.21.15.46:0/2858089504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 msgr2=0x55ac84878a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.251+0000 7f3abde00640 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac84878a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.251+0000 7f3abde00640 1 -- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 msgr2=0x55ac848784e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.251+0000 7f3abde00640 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac848784e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.251+0000 7f3abde00640 1 -- 172.21.15.46:0/2858089504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac845473c0 con 0x55ac8486fe40 2024-09-10T07:50:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.251+0000 7f3abd400640 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac848784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.252+0000 7f3a9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:50:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.252+0000 7f3abde00640 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84877fa0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80ed800 tx=0x7f3ab80bbf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3719ba0017ca181 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.253+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2858089504 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80c0910 con 0x55ac8486fe40 2024-09-10T07:50:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.253+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2858089504 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ab803cbc0 con 0x55ac8486fe40 2024-09-10T07:50:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.253+0000 7f3ac780c740 1 -- 172.21.15.46:0/2858089504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac005500 con 0x55ac8486fe40 2024-09-10T07:50:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.254+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2858089504 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3ab80c0910 con 0x55ac8486fe40 2024-09-10T07:50:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.254+0000 7f3ac780c740 10 client.28955 fetch_fsmap learned FSMap version 55 2024-09-10T07:50:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.254+0000 7f3ac780c740 1 -- 172.21.15.46:0/2858089504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ac84889c80 con 0x55ac8486fe40 2024-09-10T07:50:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.255+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2858089504 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac005500 con 0x55ac8486fe40 2024-09-10T07:50:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.256+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2858089504 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 55) ==== 8386+0+0 (secure 0 0 0) 0x7f3ab8004c50 con 0x55ac8486fe40 2024-09-10T07:50:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.256+0000 7f3ac780c740 10 client.28955 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T07:50:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.256+0000 7f3ac780c740 10 client.28955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.256+0000 7f3ac780c740 20 client.28955 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.256+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ac8488af40 0x55ac8488d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.256+0000 7f3ac780c740 4 client.28955 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.256+0000 7f3ac780c740 1 -- 172.21.15.46:0/2858089504 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac847e3950 con 0x55ac8488af40 2024-09-10T07:50:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.257+0000 7f3abd400640 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ac8488af40 0x55ac8488d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.262+0000 7f3abd400640 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ac8488af40 0x55ac8488d3a0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96b83a5dd7215f51 server_cookie=36cea65b16dd62d2 in_seq=0 out_seq=0 2024-09-10T07:50:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.262+0000 7f3a9fe00640 10 client.28955 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:57.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.266+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2858089504 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac847e3950 con 0x55ac8488af40 2024-09-10T07:50:57.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.269+0000 7f3a9fe00640 10 client.28955 handle_command_reply: tid=0 2024-09-10T07:50:57.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.269+0000 7f3ac780c740 1 client.28955 shutdown 2024-09-10T07:50:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3ac780c740 1 -- 172.21.15.46:0/2858089504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 msgr2=0x55ac84877fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84877fa0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80ed800 tx=0x7f3ab80bbf80 comp rx=0 tx=0).stop 2024-09-10T07:50:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3ac780c740 1 -- 172.21.15.46:0/2858089504 shutdown_connections 2024-09-10T07:50:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ac8488af40 0x55ac8488d3a0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac84878a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac84870760 0x55ac848784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2858089504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac8486fe40 0x55ac84877fa0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3a9fe00640 0 client.28955 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.270+0000 7f3ac780c740 1 -- 172.21.15.46:0/2858089504 >> 172.21.15.46:0/2858089504 conn(0x55ac8486c3d0 msgr2=0x55ac84872100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.271+0000 7f3ac780c740 1 -- 172.21.15.46:0/2858089504 shutdown_connections 2024-09-10T07:50:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.271+0000 7f3ac780c740 1 -- 172.21.15.46:0/2858089504 wait complete. 2024-09-10T07:50:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.271+0000 7f3ac780c740 20 client.28955 trim_cache size 0 max 16384 2024-09-10T07:50:57.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.301+0000 7f3ac780c740 1 Processor -- start 2024-09-10T07:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.301+0000 7f3ac780c740 1 -- start start 2024-09-10T07:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.301+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8486fe40 0x55ac84878670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.301+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84892c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.301+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8487b020 0x55ac848931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.301+0000 7f3ac780c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01b4fc0 con 0x55ac8487b020 2024-09-10T07:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.301+0000 7f3ac780c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01b6bc0 con 0x55ac84870760 2024-09-10T07:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.302+0000 7f3ac780c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01b5690 con 0x55ac8486fe40 2024-09-10T07:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.302+0000 7f3abde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8486fe40 0x55ac84878670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.302+0000 7f3abd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84892c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.302+0000 7f3abd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84892c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59562/0 (socket says 172.21.15.46:59562) 2024-09-10T07:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.302+0000 7f3abd400640 1 -- 172.21.15.46:0/1094661375 learned_addr learned my addr 172.21.15.46:0/1094661375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.302+0000 7f3abde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8486fe40 0x55ac84878670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:32964/0 (socket says 172.21.15.46:32964) 2024-09-10T07:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.303+0000 7f3abde00640 1 -- 172.21.15.46:0/1094661375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 msgr2=0x55ac84892c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.303+0000 7f3abde00640 1 --2- 172.21.15.46:0/1094661375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84892c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.303+0000 7f3abde00640 1 -- 172.21.15.46:0/1094661375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8487b020 msgr2=0x55ac848931b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.303+0000 7f3abde00640 1 --2- 172.21.15.46:0/1094661375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8487b020 0x55ac848931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.303+0000 7f3abde00640 1 -- 172.21.15.46:0/1094661375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8486d060 con 0x55ac8486fe40 2024-09-10T07:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.303+0000 7f3abde00640 1 --2- 172.21.15.46:0/1094661375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8486fe40 0x55ac84878670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80bc4f0 tx=0x7f3ab80bc6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c6db631fbf3f7eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.304+0000 7f3aaaa00640 1 -- 172.21.15.46:0/1094661375 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80c51d0 con 0x55ac8486fe40 2024-09-10T07:50:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.304+0000 7f3aaaa00640 1 -- 172.21.15.46:0/1094661375 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab80d5070 con 0x55ac8486fe40 2024-09-10T07:50:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.304+0000 7f3aaaa00640 1 -- 172.21.15.46:0/1094661375 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8004da0 con 0x55ac8486fe40 2024-09-10T07:50:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.305+0000 7f3ac780c740 1 -- 172.21.15.46:0/1094661375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8486fe40 msgr2=0x55ac84878670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.305+0000 7f3ac780c740 1 --2- 172.21.15.46:0/1094661375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8486fe40 0x55ac84878670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80bc4f0 tx=0x7f3ab80bc6e0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.305+0000 7f3ac780c740 1 -- 172.21.15.46:0/1094661375 shutdown_connections 2024-09-10T07:50:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.305+0000 7f3ac780c740 1 --2- 172.21.15.46:0/1094661375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8487b020 0x55ac848931b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.305+0000 7f3ac780c740 1 --2- 172.21.15.46:0/1094661375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84892c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.305+0000 7f3ac780c740 1 --2- 172.21.15.46:0/1094661375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8486fe40 0x55ac84878670 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.305+0000 7f3ac780c740 1 -- 172.21.15.46:0/1094661375 >> 172.21.15.46:0/1094661375 conn(0x55ac8486c3d0 msgr2=0x55ac84877f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.306+0000 7f3ac780c740 1 -- 172.21.15.46:0/1094661375 shutdown_connections 2024-09-10T07:50:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.306+0000 7f3ac780c740 1 -- 172.21.15.46:0/1094661375 wait complete. 2024-09-10T07:50:57.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.307+0000 7f3ac780c740 1 -- start start 2024-09-10T07:50:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.307+0000 7f3ac780c740 1 Processor -- start 2024-09-10T07:50:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.309+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8486fe40 0x55ac848f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.309+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84883020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3ac780c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac848837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3ac780c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01b64f0 con 0x55ac8486fe40 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3ac780c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01b5d90 con 0x55ac84870760 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3ac780c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01b3050 con 0x55ac8487b020 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3ac780c740 10 client.0 _ll_register_callbacks cb 0x7f3ab80b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.309+0000 7f3abde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8486fe40 0x55ac848f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3abde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8486fe40 0x55ac848f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43314/0 (socket says 172.21.15.46:43314) 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3abde00640 1 -- 172.21.15.46:0/2580878660 learned_addr learned my addr 172.21.15.46:0/2580878660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3abd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84883020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.310+0000 7f3abe800640 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac848837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.311+0000 7f3abde00640 1 -- 172.21.15.46:0/2580878660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 msgr2=0x55ac848837b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.311+0000 7f3abde00640 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac848837b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.311+0000 7f3abde00640 1 -- 172.21.15.46:0/2580878660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 msgr2=0x55ac84883020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.311+0000 7f3abde00640 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84883020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.311+0000 7f3abde00640 1 -- 172.21.15.46:0/2580878660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac848ee9d0 con 0x55ac8486fe40 2024-09-10T07:50:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.313+0000 7f3abd400640 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84883020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.313+0000 7f3abde00640 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8486fe40 0x55ac848f1650 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80c4ca0 tx=0x7f3ab802a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd7dc56d626f8e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.313+0000 7f3a9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:50:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.313+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2580878660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80bc330 con 0x55ac8486fe40 2024-09-10T07:50:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.313+0000 7f3ac780c740 1 -- 172.21.15.46:0/2580878660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab80d5070 con 0x55ac8486fe40 2024-09-10T07:50:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.314+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2580878660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ab80b8300 con 0x55ac8486fe40 2024-09-10T07:50:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.316+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2580878660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3ab80bbae0 con 0x55ac8486fe40 2024-09-10T07:50:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.316+0000 7f3ac780c740 10 client.18102 fetch_fsmap learned FSMap version 55 2024-09-10T07:50:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.316+0000 7f3ac780c740 1 -- 172.21.15.46:0/2580878660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ac848ee9d0 con 0x55ac8486fe40 2024-09-10T07:50:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.317+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2580878660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab80d5070 con 0x55ac8486fe40 2024-09-10T07:50:57.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.317+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2580878660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) ==== 8386+0+0 (secure 0 0 0) 0x7f3ab80cd030 con 0x55ac8486fe40 2024-09-10T07:50:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.317+0000 7f3ac780c740 10 client.18102 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T07:50:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.317+0000 7f3ac780c740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:50:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.317+0000 7f3ac780c740 20 client.18102 populate_metadata read hostname 'smithi046' 2024-09-10T07:50:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.317+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ac84907240 0x55ac84909630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.317+0000 7f3ac780c740 4 client.18102 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:50:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.317+0000 7f3ac780c740 1 -- 172.21.15.46:0/2580878660 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac847dde60 con 0x55ac84907240 2024-09-10T07:50:57.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.325+0000 7f3abd400640 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ac84907240 0x55ac84909630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.326+0000 7f3a9fe00640 10 client.18102 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.326+0000 7f3abd400640 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ac84907240 0x55ac84909630 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ddce9f9cdbbee6 server_cookie=221e7634f34baef8 in_seq=0 out_seq=0 2024-09-10T07:50:57.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3a9fe00640 1 -- 172.21.15.46:0/2580878660 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac847dde60 con 0x55ac84907240 2024-09-10T07:50:57.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3a9fe00640 10 client.18102 handle_command_reply: tid=0 2024-09-10T07:50:57.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 client.18102 shutdown 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 -- 172.21.15.46:0/2580878660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8486fe40 msgr2=0x55ac848f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8486fe40 0x55ac848f1650 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80c4ca0 tx=0x7f3ab802a700 comp rx=0 tx=0).stop 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 -- 172.21.15.46:0/2580878660 shutdown_connections 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55ac84907240 0x55ac84909630 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac8487b020 0x55ac848837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac84870760 0x55ac84883020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 --2- 172.21.15.46:0/2580878660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac8486fe40 0x55ac848f1650 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3a9fe00640 0 client.18102 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:50:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.328+0000 7f3ac780c740 1 -- 172.21.15.46:0/2580878660 >> 172.21.15.46:0/2580878660 conn(0x55ac8486c3d0 msgr2=0x55ac84872100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.329+0000 7f3ac780c740 1 -- 172.21.15.46:0/2580878660 shutdown_connections 2024-09-10T07:50:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.329+0000 7f3ac780c740 1 -- 172.21.15.46:0/2580878660 wait complete. 2024-09-10T07:50:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.330+0000 7f3ac780c740 20 client.18102 trim_cache size 0 max 16384 2024-09-10T07:50:57.330 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:50:57.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 -- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3aa40a2d50 msgr2=0x7f3aa40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3aa40a2d50 0x7f3aa40a5210 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0005fd0 tx=0x7f3ab0005a20 comp rx=0 tx=0).stop 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 -- 172.21.15.46:0/110833691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ac01019c0 msgr2=0x7f3ac01a3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ac01019c0 0x7f3ac01a3500 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009be0 tx=0x7f3ab8030980 comp rx=0 tx=0).stop 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 -- 172.21.15.46:0/110833691 shutdown_connections 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3aa40a2d50 0x7f3aa40a5210 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac01079c0 0x7f3ac010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac0102360 0x7f3ac01a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 --2- 172.21.15.46:0/110833691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ac01019c0 0x7f3ac01a3500 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 -- 172.21.15.46:0/110833691 >> 172.21.15.46:0/110833691 conn(0x7f3ac00fd740 msgr2=0x7f3ac0074450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.331+0000 7f3abf200640 1 -- 172.21.15.46:0/110833691 shutdown_connections 2024-09-10T07:50:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:50:57.332+0000 7f3abf200640 1 -- 172.21.15.46:0/110833691 wait complete. 2024-09-10T07:50:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:58 smithi181 ceph-mon[35397]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-10T07:50:58.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:58 smithi063 ceph-mon[35020]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-10T07:50:58.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:58 smithi046 ceph-mon[30705]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-10T07:50:59.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:50:59 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:50:59.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:50:59 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:50:59.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:50:59 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:00.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:00 smithi046 ceph-mon[30705]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 4.5 MiB/s wr, 469 op/s 2024-09-10T07:51:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 820.0 s, 457.5 tps, lat 222.336 ms stddev 253.821 2024-09-10T07:51:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:00 smithi181 ceph-mon[35397]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 4.5 MiB/s wr, 469 op/s 2024-09-10T07:51:00.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:00 smithi063 ceph-mon[35020]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 4.5 MiB/s wr, 469 op/s 2024-09-10T07:51:00.687 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:51:00.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:51:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:51:02.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:02 smithi046 ceph-mon[30705]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T07:51:02.564 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:51:02.564 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:51:02.564 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "8f96c08f-1cd7-4ca4-aabd-527fd6439e14", 2024-09-10T07:51:02.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:51:02.564 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:51:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.161+0000 7fb028400640 1 -- 172.21.15.46:0/2064260969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 msgr2=0x7fb02011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.161+0000 7fb028400640 1 --2- 172.21.15.46:0/2064260969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 0x7fb02011d530 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb018009ca0 tx=0x7fb01802a8b0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.161+0000 7fb028400640 1 -- 172.21.15.46:0/2064260969 shutdown_connections 2024-09-10T07:51:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.161+0000 7fb028400640 1 --2- 172.21.15.46:0/2064260969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 0x7fb02011d530 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.161+0000 7fb028400640 1 --2- 172.21.15.46:0/2064260969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb020109830 0x7fb020109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.161+0000 7fb028400640 1 --2- 172.21.15.46:0/2064260969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb020109450 0x7fb020110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.161+0000 7fb028400640 1 -- 172.21.15.46:0/2064260969 >> 172.21.15.46:0/2064260969 conn(0x7fb02006e140 msgr2=0x7fb02006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.163+0000 7fb028400640 1 -- 172.21.15.46:0/2064260969 shutdown_connections 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.163+0000 7fb028400640 1 -- 172.21.15.46:0/2064260969 wait complete. 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.164+0000 7fb028400640 1 Processor -- start 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.164+0000 7fb028400640 1 -- start start 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.165+0000 7fb028400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb020109450 0x7fb02011a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.165+0000 7fb028400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb020109830 0x7fb02011aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.165+0000 7fb028400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 0x7fb0201145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.165+0000 7fb025000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb020109830 0x7fb02011aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.166+0000 7fb025000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb020109830 0x7fb02011aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:32988/0 (socket says 172.21.15.46:32988) 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.167+0000 7fb025000640 1 -- 172.21.15.46:0/90719511 learned_addr learned my addr 172.21.15.46:0/90719511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.167+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb020111040 con 0x7fb020109450 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.167+0000 7fb026400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 0x7fb0201145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.167+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb020110ec0 con 0x7fb02011b140 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.167+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201111c0 con 0x7fb020109830 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.168+0000 7fb025a00640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb020109450 0x7fb02011a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.168+0000 7fb026400640 1 -- 172.21.15.46:0/90719511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb020109830 msgr2=0x7fb02011aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.168+0000 7fb026400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb020109830 0x7fb02011aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.168+0000 7fb026400640 1 -- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb020109450 msgr2=0x7fb02011a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.168+0000 7fb026400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb020109450 0x7fb02011a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.168+0000 7fb026400640 1 -- 172.21.15.46:0/90719511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb020114e60 con 0x7fb02011b140 2024-09-10T07:51:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.168+0000 7fb026400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 0x7fb0201145a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb018009bf0 tx=0x7fb018003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.169+0000 7fb01e000640 1 -- 172.21.15.46:0/90719511 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018030b30 con 0x7fb02011b140 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.169+0000 7fb01e000640 1 -- 172.21.15.46:0/90719511 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb018040070 con 0x7fb02011b140 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.169+0000 7fb01e000640 1 -- 172.21.15.46:0/90719511 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018004330 con 0x7fb02011b140 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.170+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0201150f0 con 0x7fb02011b140 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.170+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0201bc5a0 con 0x7fb02011b140 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.176+0000 7fb01e000640 1 -- 172.21.15.46:0/90719511 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0180044d0 con 0x7fb02011b140 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.179+0000 7fb02b032740 1 Processor -- start 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.179+0000 7fb02b032740 1 -- start start 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.179+0000 7fb02b032740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb4ac0 0x557fc0fb4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.179+0000 7fb02b032740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fbf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.179+0000 7fb02b032740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fbfca0 0x557fc0fc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.179+0000 7fb02b032740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fc0f1b290 con 0x557fc0fb4ac0 2024-09-10T07:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.179+0000 7fb02b032740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557fc0ec2b40 con 0x557fc0fbfca0 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.179+0000 7fb02b032740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557fc0f17250 con 0x557fc0fb53e0 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.180+0000 7fb025000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fbfca0 0x557fc0fc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.180+0000 7fb025000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fbfca0 0x557fc0fc2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59598/0 (socket says 172.21.15.46:59598) 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.180+0000 7fb025000640 1 -- 172.21.15.46:0/1666601611 learned_addr learned my addr 172.21.15.46:0/1666601611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.180+0000 7fb025a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fbf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.180+0000 7fb01e000640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb0080a2d10 0x7fb0080a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.180+0000 7fb025000640 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb4ac0 0x557fc0fb4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.181+0000 7fb01e000640 1 -- 172.21.15.46:0/90719511 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb0180f0420 con 0x7fb02011b140 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.181+0000 7fb025a00640 1 -- 172.21.15.46:0/1666601611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fbfca0 msgr2=0x557fc0fc2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.181+0000 7fb025a00640 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fbfca0 0x557fc0fc2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.181+0000 7fb025a00640 1 -- 172.21.15.46:0/1666601611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb4ac0 msgr2=0x557fc0fb4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.181+0000 7fb025a00640 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb4ac0 0x557fc0fb4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.181+0000 7fb025a00640 1 -- 172.21.15.46:0/1666601611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc0c8c6a0 con 0x557fc0fb53e0 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.182+0000 7fb025000640 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb4ac0 0x557fc0fb4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.184+0000 7fb025a00640 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fbf760 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb014009b90 tx=0x7fb014010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d9705a5d5977e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.184+0000 7fb012a00640 1 -- 172.21.15.46:0/1666601611 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014004020 con 0x557fc0fb53e0 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.185+0000 7fb012a00640 1 -- 172.21.15.46:0/1666601611 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0140041c0 con 0x557fc0fb53e0 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.185+0000 7fb026400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb0080a2d10 0x7fb0080a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.186+0000 7fb02b032740 1 -- 172.21.15.46:0/1666601611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 msgr2=0x557fc0fbf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.186+0000 7fb02b032740 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fbf760 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb014009b90 tx=0x7fb014010740 comp rx=0 tx=0).stop 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.186+0000 7fb02b032740 1 -- 172.21.15.46:0/1666601611 shutdown_connections 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.187+0000 7fb02b032740 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fbfca0 0x557fc0fc2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.187+0000 7fb02b032740 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fbf760 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.187+0000 7fb02b032740 1 --2- 172.21.15.46:0/1666601611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb4ac0 0x557fc0fb4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.187+0000 7fb02b032740 1 -- 172.21.15.46:0/1666601611 >> 172.21.15.46:0/1666601611 conn(0x557fc0fb1050 msgr2=0x557fc0fb1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.187+0000 7fb026400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb0080a2d10 0x7fb0080a51d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb0180b8d60 tx=0x7fb018030dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.190+0000 7fb02b032740 1 -- 172.21.15.46:0/1666601611 shutdown_connections 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.190+0000 7fb02b032740 1 -- 172.21.15.46:0/1666601611 wait complete. 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.194+0000 7fb02b032740 1 -- start start 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.197+0000 7fb02b032740 1 Processor -- start 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.203+0000 7fb02b032740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.204+0000 7fb02b032740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fbc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.204+0000 7fb02b032740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 0x557fc0fbcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.204+0000 7fb025000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.204+0000 7fb025000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59600/0 (socket says 172.21.15.46:59600) 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.204+0000 7fb025000640 1 -- 172.21.15.46:0/2788939353 learned_addr learned my addr 172.21.15.46:0/2788939353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.205+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fc0fc29b0 con 0x557fc0fb53e0 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.205+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557fc0ed5fa0 con 0x557fc0fb4ac0 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.205+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557fc0fc2b30 con 0x557fc0fbfca0 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.205+0000 7fb02b032740 10 client.0 _ll_register_callbacks cb 0x557fc0c29ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.205+0000 7fb025a00640 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fbc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.205+0000 7fb025000640 1 -- 172.21.15.46:0/2788939353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 msgr2=0x557fc0fbcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.206+0000 7fb025000640 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 0x557fc0fbcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.206+0000 7fb025000640 1 -- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 msgr2=0x557fc0fbc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.206+0000 7fb025000640 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fbc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.206+0000 7fb025000640 1 -- 172.21.15.46:0/2788939353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc0fbd6d0 con 0x557fc0fb4ac0 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.206+0000 7fb025a00640 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fbc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.206+0000 7fb025000640 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbf650 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009820 tx=0x7fb00c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62906c1d6513c4ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.206+0000 7fb007400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.206+0000 7fb007400640 1 -- 172.21.15.46:0/2788939353 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c021020 con 0x557fc0fb4ac0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.207+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557fc0f28800 con 0x557fc0fb4ac0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.207+0000 7fb007400640 1 -- 172.21.15.46:0/2788939353 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb00c00fe00 con 0x557fc0fb4ac0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.207+0000 7fb007400640 1 -- 172.21.15.46:0/2788939353 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb00c005020 con 0x557fc0fb4ac0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.207+0000 7fb02b032740 10 client.28973 fetch_fsmap learned FSMap version 56 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.207+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x557fc0fceaf0 con 0x557fc0fb4ac0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.207+0000 7fb007400640 1 -- 172.21.15.46:0/2788939353 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557fc0f28800 con 0x557fc0fb4ac0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.208+0000 7fb007400640 1 -- 172.21.15.46:0/2788939353 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 56) ==== 8398+0+0 (secure 0 0 0) 0x7fb00c01d410 con 0x557fc0fb4ac0 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.208+0000 7fb02b032740 10 client.28973 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.208+0000 7fb02b032740 10 client.28973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:02.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.208+0000 7fb02b032740 20 client.28973 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.208+0000 7fb02b032740 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc0fcffb0 0x557fc0fd23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.208+0000 7fb02b032740 4 client.28973 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.208+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fc0f22d10 con 0x557fc0fcffb0 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.209+0000 7fb025a00640 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc0fcffb0 0x557fc0fd23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.209+0000 7fb025a00640 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc0fcffb0 0x557fc0fd23c0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48dc80c665a41e37 server_cookie=cd44739adcf51ed9 in_seq=0 out_seq=0 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.209+0000 7fb007400640 10 client.28973 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.210+0000 7fb007400640 1 -- 172.21.15.46:0/2788939353 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fc0f22d10 con 0x557fc0fcffb0 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.210+0000 7fb007400640 10 client.28973 handle_command_reply: tid=0 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 client.28973 shutdown 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 msgr2=0x557fc0fbf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbf650 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009820 tx=0x7fb00c00f790 comp rx=0 tx=0).stop 2024-09-10T07:51:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 shutdown_connections 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc0fcffb0 0x557fc0fd23c0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 0x557fc0fbcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb007400640 0 client.28973 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fbc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 --2- 172.21.15.46:0/2788939353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbf650 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.211+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 >> 172.21.15.46:0/2788939353 conn(0x557fc0fb1050 msgr2=0x557fc0fb6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.212+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 shutdown_connections 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.212+0000 7fb02b032740 1 -- 172.21.15.46:0/2788939353 wait complete. 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.212+0000 7fb02b032740 20 client.28973 trim_cache size 0 max 16384 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.250+0000 7fb02b032740 1 Processor -- start 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.254+0000 7fb02b032740 1 -- start start 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.254+0000 7fb02b032740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.254+0000 7fb02b032740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fd7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.254+0000 7fb02b032740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fbfca0 0x557fc0fd81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.254+0000 7fb02b032740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201c6150 con 0x557fc0fbfca0 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.254+0000 7fb02b032740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201c7de0 con 0x557fc0fb4ac0 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.254+0000 7fb02b032740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201c6820 con 0x557fc0fb53e0 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.255+0000 7fb025000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbe870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59612/0 (socket says 172.21.15.46:59612) 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025000640 1 -- 172.21.15.46:0/2826705577 learned_addr learned my addr 172.21.15.46:0/2826705577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025000640 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fbfca0 0x557fc0fd81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025a00640 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fd7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025000640 1 -- 172.21.15.46:0/2826705577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 msgr2=0x557fc0fd7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025000640 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fd7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025000640 1 -- 172.21.15.46:0/2826705577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fbfca0 msgr2=0x557fc0fd81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025000640 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fbfca0 0x557fc0fd81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.256+0000 7fb025000640 1 -- 172.21.15.46:0/2826705577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc0fbae50 con 0x557fc0fb4ac0 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.257+0000 7fb025000640 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbe870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00fd10 tx=0x7fb00c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d392a7d9345d696 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.257+0000 7fb012000640 1 -- 172.21.15.46:0/2826705577 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c021020 con 0x557fc0fb4ac0 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.257+0000 7fb012000640 1 -- 172.21.15.46:0/2826705577 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c00fe00 con 0x557fc0fb4ac0 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.257+0000 7fb012000640 1 -- 172.21.15.46:0/2826705577 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c011290 con 0x557fc0fb4ac0 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.258+0000 7fb02b032740 1 -- 172.21.15.46:0/2826705577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 msgr2=0x557fc0fbe870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.258+0000 7fb02b032740 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbe870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00fd10 tx=0x7fb00c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.260+0000 7fb02b032740 1 -- 172.21.15.46:0/2826705577 shutdown_connections 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.260+0000 7fb02b032740 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fbfca0 0x557fc0fd81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.260+0000 7fb02b032740 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fb53e0 0x557fc0fd7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.260+0000 7fb02b032740 1 --2- 172.21.15.46:0/2826705577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fbe870 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.260+0000 7fb02b032740 1 -- 172.21.15.46:0/2826705577 >> 172.21.15.46:0/2826705577 conn(0x557fc0fb1050 msgr2=0x557fc0fbe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.260+0000 7fb02b032740 1 -- 172.21.15.46:0/2826705577 shutdown_connections 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.260+0000 7fb02b032740 1 -- 172.21.15.46:0/2826705577 wait complete. 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.260+0000 7fb02b032740 1 -- start start 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.261+0000 7fb02b032740 1 Processor -- start 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb02b032740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fc8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb02b032740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fc59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb02b032740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 0x557fc0fc5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb02b032740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201c7710 con 0x557fc0fb53e0 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb02b032740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201c6fb0 con 0x557fc0fb4ac0 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb02b032740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201c4130 con 0x557fc0fbfca0 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb02b032740 10 client.0 _ll_register_callbacks cb 0x7fb00c0119d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb025000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fc8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb025000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fc8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59626/0 (socket says 172.21.15.46:59626) 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb025000640 1 -- 172.21.15.46:0/4183134434 learned_addr learned my addr 172.21.15.46:0/4183134434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.265+0000 7fb025000640 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 0x557fc0fc5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.267+0000 7fb025a00640 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fc59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.267+0000 7fb025a00640 1 -- 172.21.15.46:0/4183134434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 msgr2=0x557fc0fc5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.267+0000 7fb025a00640 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 0x557fc0fc5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.267+0000 7fb025a00640 1 -- 172.21.15.46:0/4183134434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 msgr2=0x557fc0fc8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.267+0000 7fb025a00640 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fc8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.267+0000 7fb025a00640 1 -- 172.21.15.46:0/4183134434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc0fc6790 con 0x557fc0fb53e0 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.268+0000 7fb025a00640 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fc59c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb0140059d0 tx=0x7fb0140095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4967ca7b54dcd96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.268+0000 7fb007400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.269+0000 7fb007400640 1 -- 172.21.15.46:0/4183134434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014012de0 con 0x557fc0fb53e0 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.269+0000 7fb02b032740 1 -- 172.21.15.46:0/4183134434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb00c00fe00 con 0x557fc0fb53e0 2024-09-10T07:51:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.269+0000 7fb007400640 1 -- 172.21.15.46:0/4183134434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fb014005040 con 0x557fc0fb53e0 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.270+0000 7fb007400640 1 -- 172.21.15.46:0/4183134434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb014037050 con 0x557fc0fb53e0 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.272+0000 7fb02b032740 10 client.18117 fetch_fsmap learned FSMap version 56 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.272+0000 7fb02b032740 1 -- 172.21.15.46:0/4183134434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x557fc103ad80 con 0x557fc0fb53e0 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.273+0000 7fb007400640 1 -- 172.21.15.46:0/4183134434 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c00fe00 con 0x557fc0fb53e0 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.273+0000 7fb007400640 1 -- 172.21.15.46:0/4183134434 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 56) ==== 8398+0+0 (secure 0 0 0) 0x7fb01400f040 con 0x557fc0fb53e0 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.273+0000 7fb02b032740 10 client.18117 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.273+0000 7fb02b032740 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.273+0000 7fb02b032740 20 client.18117 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.273+0000 7fb02b032740 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc103b250 0x557fc103d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.273+0000 7fb02b032740 4 client.18117 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.273+0000 7fb02b032740 1 -- 172.21.15.46:0/4183134434 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557fc0e9efb0 con 0x557fc103b250 2024-09-10T07:51:02.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.275+0000 7fb025000640 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc103b250 0x557fc103d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.276+0000 7fb025000640 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc103b250 0x557fc103d670 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73f73d670a108a9 server_cookie=3f2f6664851a0c4f in_seq=0 out_seq=0 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.276+0000 7fb007400640 10 client.18117 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.280+0000 7fb007400640 1 -- 172.21.15.46:0/4183134434 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557fc0e9efb0 con 0x557fc103b250 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.280+0000 7fb007400640 10 client.18117 handle_command_reply: tid=0 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.280+0000 7fb02b032740 1 client.18117 shutdown 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 -- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 msgr2=0x557fc0fc59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fc59c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb0140059d0 tx=0x7fb0140095b0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 -- 172.21.15.46:0/4183134434 shutdown_connections 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x557fc103b250 0x557fc103d670 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557fc0fbfca0 0x557fc0fc5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fc0fb53e0 0x557fc0fc59c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 --2- 172.21.15.46:0/4183134434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557fc0fb4ac0 0x557fc0fc8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 -- 172.21.15.46:0/4183134434 >> 172.21.15.46:0/4183134434 conn(0x557fc0fb1050 msgr2=0x557fc0fb01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:02.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb007400640 0 client.18117 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 -- 172.21.15.46:0/4183134434 shutdown_connections 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.281+0000 7fb02b032740 1 -- 172.21.15.46:0/4183134434 wait complete. 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.282+0000 7fb02b032740 20 client.18117 trim_cache size 0 max 16384 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb0080a2d10 msgr2=0x7fb0080a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb0080a2d10 0x7fb0080a51d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb0180b8d60 tx=0x7fb018030dd0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 msgr2=0x7fb0201145a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 0x7fb0201145a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb018009bf0 tx=0x7fb018003ba0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 shutdown_connections 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb02011b140 0x7fb0201145a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb020109830 0x7fb02011aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fb0080a2d10 0x7fb0080a51d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 --2- 172.21.15.46:0/90719511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb020109450 0x7fb02011a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 >> 172.21.15.46:0/90719511 conn(0x7fb02006e140 msgr2=0x7fb020070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 shutdown_connections 2024-09-10T07:51:02.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:02.283+0000 7fb028400640 1 -- 172.21.15.46:0/90719511 wait complete. 2024-09-10T07:51:02.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:02 smithi181 ceph-mon[35397]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T07:51:02.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:02 smithi063 ceph-mon[35020]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T07:51:03.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:03 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:51:03.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:03 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:03.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:03 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:51:03.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:03 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:51:03.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:03 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:51:03.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:03 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:03.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:03 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:51:03.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:51:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:03 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:51:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:03 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:03.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:03 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:51:03.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:03 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:51:03.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:03 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:51:03.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:03 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:03.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:03 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:51:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:03 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:51:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:03 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:03 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:51:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:03 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:51:03.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:03 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:51:03.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:03 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:03.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:03 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:51:04.271 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:51:04.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:04 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:51:04.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:04 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:51:04.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:04 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:04 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:04 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:04 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:04 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:04 smithi046 ceph-mon[30705]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 5.2 MiB/s wr, 527 op/s 2024-09-10T07:51:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:04 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:51:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:04 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:51:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:04 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:04 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:04 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:04 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:04 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:04 smithi181 ceph-mon[35397]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 5.2 MiB/s wr, 527 op/s 2024-09-10T07:51:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:04 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:51:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:04 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:51:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:04 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:04 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:04 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:04 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:04 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:51:04.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:04 smithi063 ceph-mon[35020]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 5.2 MiB/s wr, 527 op/s 2024-09-10T07:51:05.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 825.0 s, 476.0 tps, lat 209.416 ms stddev 255.168 2024-09-10T07:51:06.451 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:06 smithi063 ceph-mon[35020]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.2 MiB/s wr, 466 op/s 2024-09-10T07:51:06.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:06 smithi181 ceph-mon[35397]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.2 MiB/s wr, 466 op/s 2024-09-10T07:51:06.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:06 smithi046 ceph-mon[30705]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.2 MiB/s wr, 466 op/s 2024-09-10T07:51:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.981+0000 7f6fa3600640 1 -- 172.21.15.46:0/1694373577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f9c106f00 msgr2=0x7f6f9c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.981+0000 7f6fa3600640 1 --2- 172.21.15.46:0/1694373577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f9c106f00 0x7f6f9c10ea80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009860 tx=0x7f6f8c02a360 comp rx=0 tx=0).stop 2024-09-10T07:51:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.981+0000 7f6fa3600640 1 -- 172.21.15.46:0/1694373577 shutdown_connections 2024-09-10T07:51:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.981+0000 7f6fa3600640 1 --2- 172.21.15.46:0/1694373577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f9c106f00 0x7f6f9c10ea80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.981+0000 7f6fa3600640 1 --2- 172.21.15.46:0/1694373577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9c102780 0x7f6f9c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.981+0000 7f6fa3600640 1 --2- 172.21.15.46:0/1694373577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f9c101dd0 0x7f6f9c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.981+0000 7f6fa3600640 1 -- 172.21.15.46:0/1694373577 >> 172.21.15.46:0/1694373577 conn(0x7f6f9c0fd8b0 msgr2=0x7f6f9c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.982+0000 7f6fa3600640 1 -- 172.21.15.46:0/1694373577 shutdown_connections 2024-09-10T07:51:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.982+0000 7f6fa3600640 1 -- 172.21.15.46:0/1694373577 wait complete. 2024-09-10T07:51:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.983+0000 7f6fa3600640 1 Processor -- start 2024-09-10T07:51:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.984+0000 7f6fa3600640 1 -- start start 2024-09-10T07:51:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.985+0000 7f6fa3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f9c101dd0 0x7f6f9c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.985+0000 7f6fa3600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f9c102780 0x7f6f9c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.985+0000 7f6fa3600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9c106f00 0x7f6f9c10dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.986+0000 7f6fa0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f9c101dd0 0x7f6f9c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.986+0000 7f6fa1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9c106f00 0x7f6f9c10dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.986+0000 7f6fa1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9c106f00 0x7f6f9c10dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56938/0 (socket says 172.21.15.46:56938) 2024-09-10T07:51:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.986+0000 7f6fa1600640 1 -- 172.21.15.46:0/3349734812 learned_addr learned my addr 172.21.15.46:0/3349734812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.986+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c111f80 con 0x7f6f9c101dd0 2024-09-10T07:51:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.985+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f9c102780 0x7f6f9c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.986+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c111e00 con 0x7f6f9c102780 2024-09-10T07:51:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.986+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c112100 con 0x7f6f9c106f00 2024-09-10T07:51:06.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.987+0000 7f6fa1600640 1 -- 172.21.15.46:0/3349734812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f9c102780 msgr2=0x7f6f9c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:06.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.987+0000 7f6fa1600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f9c102780 0x7f6f9c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:06.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.987+0000 7f6fa1600640 1 -- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f9c101dd0 msgr2=0x7f6f9c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:06.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.987+0000 7f6fa1600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f9c101dd0 0x7f6f9c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:06.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.987+0000 7f6fa1600640 1 -- 172.21.15.46:0/3349734812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c10e600 con 0x7f6f9c106f00 2024-09-10T07:51:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.991+0000 7f6fa1600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9c106f00 0x7f6f9c10dd40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c02a950 tx=0x7f6f8c033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.992+0000 7f6f99600640 1 -- 172.21.15.46:0/3349734812 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c03f020 con 0x7f6f9c106f00 2024-09-10T07:51:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.992+0000 7f6f99600640 1 -- 172.21.15.46:0/3349734812 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c0307f0 con 0x7f6f9c106f00 2024-09-10T07:51:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.992+0000 7f6f99600640 1 -- 172.21.15.46:0/3349734812 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c037040 con 0x7f6f9c106f00 2024-09-10T07:51:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.992+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9c10e890 con 0x7f6f9c106f00 2024-09-10T07:51:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.992+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9c19ad30 con 0x7f6f9c106f00 2024-09-10T07:51:06.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.997+0000 7f6fa60a1740 1 Processor -- start 2024-09-10T07:51:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.998+0000 7f6f99600640 1 -- 172.21.15.46:0/3349734812 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f8c042020 con 0x7f6f9c106f00 2024-09-10T07:51:07.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.002+0000 7f6f99600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f800a2d10 0x7f6f800a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.002+0000 7f6f99600640 1 -- 172.21.15.46:0/3349734812 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6f8c0eedf0 con 0x7f6f9c106f00 2024-09-10T07:51:07.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.002+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f800a2d10 0x7f6f800a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:06.998+0000 7f6fa60a1740 1 -- start start 2024-09-10T07:51:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.003+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c84b0 0x5645690d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690d2d70 0x5645690d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56948/0 (socket says 172.21.15.46:56948) 2024-09-10T07:51:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa0c00640 1 -- 172.21.15.46:0/3360446716 learned_addr learned my addr 172.21.15.46:0/3360446716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3360446716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56456902e3e0 con 0x5645690d2d70 2024-09-10T07:51:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3360446716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564568fd5d80 con 0x5645690c84b0 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3360446716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56456902a3a0 con 0x5645690c7b90 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c84b0 0x5645690d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.004+0000 7f6fa1600640 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690d2d70 0x5645690d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.005+0000 7f6fa0c00640 1 -- 172.21.15.46:0/3360446716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c84b0 msgr2=0x5645690d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.005+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c84b0 0x5645690d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.005+0000 7f6fa0c00640 1 -- 172.21.15.46:0/3360446716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690d2d70 msgr2=0x5645690d5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.005+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690d2d70 0x5645690d5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.005+0000 7f6fa0c00640 1 -- 172.21.15.46:0/3360446716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564568ff4ae0 con 0x5645690c7b90 2024-09-10T07:51:07.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.005+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690c7f70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400e5d0 tx=0x7f6f940095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0c67bb38aaf4e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.005+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f800a2d10 0x7f6f800a51d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c197890 tx=0x7f6f88009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.006+0000 7f6f91c00640 1 -- 172.21.15.46:0/3360446716 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400dad0 con 0x5645690c7b90 2024-09-10T07:51:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.006+0000 7f6f91c00640 1 -- 172.21.15.46:0/3360446716 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9401c070 con 0x5645690c7b90 2024-09-10T07:51:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.006+0000 7f6f91c00640 1 -- 172.21.15.46:0/3360446716 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94016380 con 0x5645690c7b90 2024-09-10T07:51:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.007+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3360446716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 msgr2=0x5645690c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.007+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690c7f70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400e5d0 tx=0x7f6f940095c0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.007+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3360446716 shutdown_connections 2024-09-10T07:51:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.007+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690d2d70 0x5645690d5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.007+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c84b0 0x5645690d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.007+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3360446716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690c7f70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.007+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3360446716 >> 172.21.15.46:0/3360446716 conn(0x5645690c4120 msgr2=0x5645690c4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:07.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.008+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3360446716 shutdown_connections 2024-09-10T07:51:07.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.008+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3360446716 wait complete. 2024-09-10T07:51:07.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.008+0000 7f6fa60a1740 1 -- start start 2024-09-10T07:51:07.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.009+0000 7f6fa60a1740 1 Processor -- start 2024-09-10T07:51:07.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.010+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c7b90 0x5645690cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.010+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.011+0000 7f6fa0c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c7b90 0x5645690cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.011+0000 7f6fa0c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c7b90 0x5645690cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49888/0 (socket says 172.21.15.46:49888) 2024-09-10T07:51:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.013+0000 7f6fa0c00640 1 -- 172.21.15.46:0/3207646479 learned_addr learned my addr 172.21.15.46:0/3207646479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.013+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690d2d70 0x5645690d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.011+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.013+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5645690d59e0 con 0x5645690c84b0 2024-09-10T07:51:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.014+0000 7f6fa1600640 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690d2d70 0x5645690d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.014+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564568fe90f0 con 0x5645690c7b90 2024-09-10T07:51:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.014+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5645690d5b60 con 0x5645690d2d70 2024-09-10T07:51:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.014+0000 7f6fa60a1740 10 client.0 _ll_register_callbacks cb 0x564568fda9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.015+0000 7f6fa0c00640 1 -- 172.21.15.46:0/3207646479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690d2d70 msgr2=0x5645690d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.015+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690d2d70 0x5645690d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.015+0000 7f6fa0c00640 1 -- 172.21.15.46:0/3207646479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 msgr2=0x5645690d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.015+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.015+0000 7f6fa0c00640 1 -- 172.21.15.46:0/3207646479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564568d9f3c0 con 0x5645690c7b90 2024-09-10T07:51:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.015+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.015+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c7b90 0x5645690cfc50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000f80 tx=0x7f6f94017650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bebf1860ecc237f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.015+0000 7f6f86000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.017+0000 7f6f86000640 1 -- 172.21.15.46:0/3207646479 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400d8d0 con 0x5645690c7b90 2024-09-10T07:51:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.017+0000 7f6f86000640 1 -- 172.21.15.46:0/3207646479 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6f9400da70 con 0x5645690c7b90 2024-09-10T07:51:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.018+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9401c070 con 0x5645690c7b90 2024-09-10T07:51:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.019+0000 7f6f86000640 1 -- 172.21.15.46:0/3207646479 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f6f9400dc70 con 0x5645690c7b90 2024-09-10T07:51:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.019+0000 7f6fa60a1740 10 client.28991 fetch_fsmap learned FSMap version 56 2024-09-10T07:51:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.019+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5645690e1670 con 0x5645690c7b90 2024-09-10T07:51:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.020+0000 7f6f86000640 1 -- 172.21.15.46:0/3207646479 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9401c070 con 0x5645690c7b90 2024-09-10T07:51:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.021+0000 7f6f86000640 1 -- 172.21.15.46:0/3207646479 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 56) ==== 8398+0+0 (secure 0 0 0) 0x7f6f9400b040 con 0x5645690c7b90 2024-09-10T07:51:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.021+0000 7f6fa60a1740 10 client.28991 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:51:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.021+0000 7f6fa60a1740 10 client.28991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.021+0000 7f6fa60a1740 20 client.28991 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.021+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645690e2ad0 0x5645690e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.022+0000 7f6fa60a1740 4 client.28991 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.022+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56456903b950 con 0x5645690e2ad0 2024-09-10T07:51:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.022+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645690e2ad0 0x5645690e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.023+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645690e2ad0 0x5645690e4ec0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53a227f6a7540d23 server_cookie=41c1f137c0806a97 in_seq=0 out_seq=0 2024-09-10T07:51:07.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.023+0000 7f6f86000640 10 client.28991 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.025+0000 7f6f86000640 1 -- 172.21.15.46:0/3207646479 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56456903b950 con 0x5645690e2ad0 2024-09-10T07:51:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.025+0000 7f6f86000640 10 client.28991 handle_command_reply: tid=0 2024-09-10T07:51:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 client.28991 shutdown 2024-09-10T07:51:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c7b90 msgr2=0x5645690cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c7b90 0x5645690cfc50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000f80 tx=0x7f6f94017650 comp rx=0 tx=0).stop 2024-09-10T07:51:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 shutdown_connections 2024-09-10T07:51:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5645690e2ad0 0x5645690e4ec0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690d2d70 0x5645690d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3207646479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690c7b90 0x5645690cfc50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6f86000640 0 client.28991 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.026+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 >> 172.21.15.46:0/3207646479 conn(0x5645690c4120 msgr2=0x5645690c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.027+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 shutdown_connections 2024-09-10T07:51:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.027+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3207646479 wait complete. 2024-09-10T07:51:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.027+0000 7f6fa60a1740 20 client.28991 trim_cache size 0 max 16384 2024-09-10T07:51:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.061+0000 7f6fa60a1740 1 Processor -- start 2024-09-10T07:51:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.061+0000 7f6fa60a1740 1 -- start start 2024-09-10T07:51:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.062+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.062+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.062+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690eaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.062+0000 7f6fa60a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c1bdc40 con 0x5645690c84b0 2024-09-10T07:51:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.062+0000 7f6fa60a1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c1bf8d0 con 0x5645690d2d70 2024-09-10T07:51:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.062+0000 7f6fa60a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c1be310 con 0x5645690c7b90 2024-09-10T07:51:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56978/0 (socket says 172.21.15.46:56978) 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa0c00640 1 -- 172.21.15.46:0/2020568093 learned_addr learned my addr 172.21.15.46:0/2020568093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa1600640 1 --2- 172.21.15.46:0/2020568093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690eaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa0c00640 1 -- 172.21.15.46:0/2020568093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 msgr2=0x5645690eaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/2020568093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690eaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa0c00640 1 -- 172.21.15.46:0/2020568093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 msgr2=0x5645690ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/2020568093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.063+0000 7f6fa0c00640 1 -- 172.21.15.46:0/2020568093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645690c52b0 con 0x5645690c7b90 2024-09-10T07:51:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.064+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/2020568093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690ca5f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6f94004200 tx=0x7f6f9400dfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50082201cbc23345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.064+0000 7f6f91200640 1 -- 172.21.15.46:0/2020568093 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94039070 con 0x5645690c7b90 2024-09-10T07:51:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.064+0000 7f6f91200640 1 -- 172.21.15.46:0/2020568093 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9401c070 con 0x5645690c7b90 2024-09-10T07:51:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.064+0000 7f6f91200640 1 -- 172.21.15.46:0/2020568093 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94005070 con 0x5645690c7b90 2024-09-10T07:51:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.065+0000 7f6fa60a1740 1 -- 172.21.15.46:0/2020568093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 msgr2=0x5645690ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.065+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/2020568093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690ca5f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6f94004200 tx=0x7f6f9400dfd0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.066+0000 7f6fa60a1740 1 -- 172.21.15.46:0/2020568093 shutdown_connections 2024-09-10T07:51:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.066+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/2020568093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.066+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/2020568093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.066+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/2020568093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690ca5f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.066+0000 7f6fa60a1740 1 -- 172.21.15.46:0/2020568093 >> 172.21.15.46:0/2020568093 conn(0x5645690c4120 msgr2=0x5645690ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.068+0000 7f6fa60a1740 1 -- 172.21.15.46:0/2020568093 shutdown_connections 2024-09-10T07:51:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.068+0000 7f6fa60a1740 1 -- 172.21.15.46:0/2020568093 wait complete. 2024-09-10T07:51:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.068+0000 7f6fa60a1740 1 -- start start 2024-09-10T07:51:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.069+0000 7f6fa60a1740 1 Processor -- start 2024-09-10T07:51:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.069+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.070+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690db5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.070+0000 7f6fa60a1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.070+0000 7f6fa60a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c1bf200 con 0x5645690c84b0 2024-09-10T07:51:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.070+0000 7f6fa60a1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c1beaa0 con 0x5645690d2d70 2024-09-10T07:51:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.070+0000 7f6fa60a1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c1bbc20 con 0x5645690c7b90 2024-09-10T07:51:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.070+0000 7f6fa60a1740 10 client.0 _ll_register_callbacks cb 0x7f6f94009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:07.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.072+0000 7f6fa1600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690d64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.072+0000 7f6fa1600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690d64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49900/0 (socket says 172.21.15.46:49900) 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.072+0000 7f6fa1600640 1 -- 172.21.15.46:0/3567942926 learned_addr learned my addr 172.21.15.46:0/3567942926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.073+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690db5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.074+0000 7f6f9be00640 1 -- 172.21.15.46:0/3567942926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 msgr2=0x5645690db080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.074+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690db080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.074+0000 7f6f9be00640 1 -- 172.21.15.46:0/3567942926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 msgr2=0x5645690d64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.074+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690d64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.074+0000 7f6f9be00640 1 -- 172.21.15.46:0/3567942926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645690d6ce0 con 0x5645690c84b0 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.075+0000 7f6f9be00640 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690db5c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5645690d1290 tx=0x7f6f880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff11e677e441f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.075+0000 7f6f86000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:51:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.078+0000 7f6f86000640 1 -- 172.21.15.46:0/3567942926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8801b040 con 0x5645690c84b0 2024-09-10T07:51:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.078+0000 7f6f86000640 1 -- 172.21.15.46:0/3567942926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f6f88022430 con 0x5645690c84b0 2024-09-10T07:51:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.079+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3567942926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9401c070 con 0x5645690c84b0 2024-09-10T07:51:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.079+0000 7f6f86000640 1 -- 172.21.15.46:0/3567942926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f6f88022630 con 0x5645690c84b0 2024-09-10T07:51:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.080+0000 7f6fa60a1740 10 client.18135 fetch_fsmap learned FSMap version 56 2024-09-10T07:51:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.080+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3567942926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56456914d0b0 con 0x5645690c84b0 2024-09-10T07:51:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.080+0000 7f6f86000640 1 -- 172.21.15.46:0/3567942926 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9401c070 con 0x5645690c84b0 2024-09-10T07:51:07.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.081+0000 7f6f86000640 1 -- 172.21.15.46:0/3567942926 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 56) ==== 8398+0+0 (secure 0 0 0) 0x7f6f8802f070 con 0x5645690c84b0 2024-09-10T07:51:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.081+0000 7f6fa60a1740 10 client.18135 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:51:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.081+0000 7f6fa60a1740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.081+0000 7f6fa60a1740 20 client.18135 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.081+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56456914d7e0 0x56456914fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.081+0000 7f6fa60a1740 4 client.18135 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:51:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.081+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3567942926 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564569035e60 con 0x56456914d7e0 2024-09-10T07:51:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.083+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56456914d7e0 0x56456914fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.085+0000 7f6fa0c00640 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56456914d7e0 0x56456914fc00 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db147693cc9b73fe server_cookie=bd426cb632d79f94 in_seq=0 out_seq=0 2024-09-10T07:51:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.085+0000 7f6f86000640 10 client.18135 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.085+0000 7f6f86000640 1 -- 172.21.15.46:0/3567942926 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564569035e60 con 0x56456914d7e0 2024-09-10T07:51:07.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.085+0000 7f6f86000640 10 client.18135 handle_command_reply: tid=0 2024-09-10T07:51:07.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.085+0000 7f6fa60a1740 1 client.18135 shutdown 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 msgr2=0x5645690db5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690db5c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5645690d1290 tx=0x7f6f880046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3567942926 shutdown_connections 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56456914d7e0 0x56456914fc00 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645690d2d70 0x5645690d64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645690c84b0 0x5645690db5c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 --2- 172.21.15.46:0/3567942926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645690c7b90 0x5645690db080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3567942926 >> 172.21.15.46:0/3567942926 conn(0x5645690c4120 msgr2=0x5645690c29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6f86000640 0 client.18135 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.086+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3567942926 shutdown_connections 2024-09-10T07:51:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.087+0000 7f6fa60a1740 1 -- 172.21.15.46:0/3567942926 wait complete. 2024-09-10T07:51:07.088 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:51:07.088 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:51:07.088 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:51:07.088 INFO:teuthology.orchestra.run.smithi046.stdout: "8f96c08f-1cd7-4ca4-aabd-527fd6439e14": { 2024-09-10T07:51:07.088 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:51:07.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "8f96c08f-1cd7-4ca4-aabd-527fd6439e14", 2024-09-10T07:51:07.088 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:51:07.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:51:07.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:51:07.089 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:51:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.087+0000 7f6fa60a1740 20 client.18135 trim_cache size 0 max 16384 2024-09-10T07:51:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f800a2d10 msgr2=0x7f6f800a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f800a2d10 0x7f6f800a51d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c197890 tx=0x7f6f88009110 comp rx=0 tx=0).stop 2024-09-10T07:51:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9c106f00 msgr2=0x7f6f9c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9c106f00 0x7f6f9c10dd40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c02a950 tx=0x7f6f8c033d10 comp rx=0 tx=0).stop 2024-09-10T07:51:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 shutdown_connections 2024-09-10T07:51:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9c106f00 0x7f6f9c10dd40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f9c102780 0x7f6f9c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f6f800a2d10 0x7f6f800a51d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 --2- 172.21.15.46:0/3349734812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f9c101dd0 0x7f6f9c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.091+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 >> 172.21.15.46:0/3349734812 conn(0x7f6f9c0fd8b0 msgr2=0x7f6f9c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.092+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 shutdown_connections 2024-09-10T07:51:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:07.092+0000 7f6fa3600640 1 -- 172.21.15.46:0/3349734812 wait complete. 2024-09-10T07:51:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 -- 172.21.15.46:0/3331641431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64077df0 msgr2=0x7f5d64078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 --2- 172.21.15.46:0/3331641431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64077df0 0x7f5d64078250 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009860 tx=0x7f5d5002a340 comp rx=0 tx=0).stop 2024-09-10T07:51:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 -- 172.21.15.46:0/3331641431 shutdown_connections 2024-09-10T07:51:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 --2- 172.21.15.46:0/3331641431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d64078790 0x7f5d64079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 --2- 172.21.15.46:0/3331641431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64077df0 0x7f5d64078250 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 --2- 172.21.15.46:0/3331641431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d64076bb0 0x7f5d64076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 -- 172.21.15.46:0/3331641431 >> 172.21.15.46:0/3331641431 conn(0x7f5d640ff2f0 msgr2=0x7f5d64101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 -- 172.21.15.46:0/3331641431 shutdown_connections 2024-09-10T07:51:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.260+0000 7f5d69e00640 1 -- 172.21.15.46:0/3331641431 wait complete. 2024-09-10T07:51:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.261+0000 7f5d69e00640 1 Processor -- start 2024-09-10T07:51:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.261+0000 7f5d69e00640 1 -- start start 2024-09-10T07:51:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64076bb0 0x7f5d6419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d64077df0 0x7f5d6419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d64078790 0x7f5d641a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d64113c20 con 0x7f5d64077df0 2024-09-10T07:51:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5d64113aa0 con 0x7f5d64076bb0 2024-09-10T07:51:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d64113da0 con 0x7f5d64078790 2024-09-10T07:51:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d68a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64076bb0 0x7f5d6419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d64078790 0x7f5d641a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d64078790 0x7f5d641a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57012/0 (socket says 172.21.15.46:57012) 2024-09-10T07:51:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d68a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64076bb0 0x7f5d6419e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49924/0 (socket says 172.21.15.46:49924) 2024-09-10T07:51:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d69400640 1 -- 172.21.15.46:0/2905595505 learned_addr learned my addr 172.21.15.46:0/2905595505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.262+0000 7f5d63e00640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d64077df0 0x7f5d6419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d69400640 1 -- 172.21.15.46:0/2905595505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64076bb0 msgr2=0x7f5d6419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d69400640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64076bb0 0x7f5d6419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d69400640 1 -- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d64077df0 msgr2=0x7f5d6419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d69400640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d64077df0 0x7f5d6419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d69400640 1 -- 172.21.15.46:0/2905595505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d641a5ed0 con 0x7f5d64078790 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d69400640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d64078790 0x7f5d641a5860 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009fd0 tx=0x7f5d5c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d61600640 1 -- 172.21.15.46:0/2905595505 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c004b20 con 0x7f5d64078790 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d69e00640 1 -- 172.21.15.46:0/2905595505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d641a61c0 con 0x7f5d64078790 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.263+0000 7f5d61600640 1 -- 172.21.15.46:0/2905595505 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5c003e50 con 0x7f5d64078790 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.264+0000 7f5d61600640 1 -- 172.21.15.46:0/2905595505 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00b630 con 0x7f5d64078790 2024-09-10T07:51:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.264+0000 7f5d69e00640 1 -- 172.21.15.46:0/2905595505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d641a66e0 con 0x7f5d64078790 2024-09-10T07:51:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.267+0000 7f5d61600640 1 -- 172.21.15.46:0/2905595505 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d5c01d020 con 0x7f5d64078790 2024-09-10T07:51:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.268+0000 7f5d6e3fa740 1 Processor -- start 2024-09-10T07:51:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.269+0000 7f5d6e3fa740 1 -- start start 2024-09-10T07:51:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.269+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x56057660b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x560576610660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 0x5605766130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560576570100 con 0x56057660b260 2024-09-10T07:51:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5605765181b0 con 0x56057660bba0 2024-09-10T07:51:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56057656c0c0 con 0x560576610c30 2024-09-10T07:51:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d61600640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5d4c0a2c40 0x7f5d4c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d61600640 1 -- 172.21.15.46:0/2905595505 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5d5c0cf380 con 0x7f5d64078790 2024-09-10T07:51:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x56057660b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x56057660b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35188/0 (socket says 172.21.15.46:35188) 2024-09-10T07:51:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.270+0000 7f5d63e00640 1 -- 172.21.15.46:0/2232513114 learned_addr learned my addr 172.21.15.46:0/2232513114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.271+0000 7f5d63e00640 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 0x5605766130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.271+0000 7f5d68a00640 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x560576610660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.272+0000 7f5d68a00640 1 -- 172.21.15.46:0/2232513114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 msgr2=0x5605766130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.272+0000 7f5d68a00640 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 0x5605766130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.272+0000 7f5d68a00640 1 -- 172.21.15.46:0/2232513114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 msgr2=0x56057660b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.272+0000 7f5d68a00640 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x56057660b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.272+0000 7f5d68a00640 1 -- 172.21.15.46:0/2232513114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560576283210 con 0x56057660bba0 2024-09-10T07:51:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.272+0000 7f5d63e00640 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 0x5605766130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.272+0000 7f5d69400640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5d4c0a2c40 0x7f5d4c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.273+0000 7f5d68a00640 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x560576610660 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800ab40 tx=0x7f5d5800f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f827715fed442e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.273+0000 7f5d55c00640 1 -- 172.21.15.46:0/2232513114 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58004020 con 0x56057660bba0 2024-09-10T07:51:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.273+0000 7f5d55c00640 1 -- 172.21.15.46:0/2232513114 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d580041c0 con 0x56057660bba0 2024-09-10T07:51:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.273+0000 7f5d55c00640 1 -- 172.21.15.46:0/2232513114 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d580187c0 con 0x56057660bba0 2024-09-10T07:51:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.274+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/2232513114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 msgr2=0x560576610660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.274+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x560576610660 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800ab40 tx=0x7f5d5800f3c0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.274+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/2232513114 shutdown_connections 2024-09-10T07:51:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.274+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 0x5605766130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.274+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x560576610660 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.274+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/2232513114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x56057660b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.274+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/2232513114 >> 172.21.15.46:0/2232513114 conn(0x5605766068d0 msgr2=0x560576606cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.276+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/2232513114 shutdown_connections 2024-09-10T07:51:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.276+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/2232513114 wait complete. 2024-09-10T07:51:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.277+0000 7f5d69400640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5d4c0a2c40 0x7f5d4c0a5100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00cd00 tx=0x7f5d5c004170 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.277+0000 7f5d6e3fa740 1 -- start start 2024-09-10T07:51:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.278+0000 7f5d6e3fa740 1 Processor -- start 2024-09-10T07:51:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.278+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660b260 0x560576615480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x5605766159c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 0x56057661ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560576613c00 con 0x560576610c30 2024-09-10T07:51:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5605764f9090 con 0x56057660bba0 2024-09-10T07:51:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560576613d80 con 0x56057660b260 2024-09-10T07:51:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d6e3fa740 10 client.0 _ll_register_callbacks cb 0x56057654df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d63e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660b260 0x560576615480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d63e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660b260 0x560576615480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57034/0 (socket says 172.21.15.46:57034) 2024-09-10T07:51:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d63e00640 1 -- 172.21.15.46:0/542355001 learned_addr learned my addr 172.21.15.46:0/542355001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d63e00640 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 0x56057661ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.279+0000 7f5d68a00640 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x5605766159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d63e00640 1 -- 172.21.15.46:0/542355001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 msgr2=0x5605766159c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d63e00640 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x5605766159c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d63e00640 1 -- 172.21.15.46:0/542355001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 msgr2=0x56057661ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d63e00640 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 0x56057661ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d63e00640 1 -- 172.21.15.46:0/542355001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56057634f620 con 0x56057660b260 2024-09-10T07:51:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d68a00640 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x5605766159c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d63e00640 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660b260 0x560576615480 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009980 tx=0x7f5d50000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bac58c3113fcc2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d4a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d4a000640 1 -- 172.21.15.46:0/542355001 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50037040 con 0x56057660b260 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.280+0000 7f5d4a000640 1 -- 172.21.15.46:0/542355001 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5d50004460 con 0x56057660b260 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.281+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/542355001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d580041c0 con 0x56057660b260 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.281+0000 7f5d4a000640 1 -- 172.21.15.46:0/542355001 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f5d5004e050 con 0x56057660b260 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.281+0000 7f5d6e3fa740 10 client.28900 fetch_fsmap learned FSMap version 56 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.281+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/542355001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5605766308c0 con 0x56057660b260 2024-09-10T07:51:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.281+0000 7f5d4a000640 1 -- 172.21.15.46:0/542355001 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d580041c0 con 0x56057660b260 2024-09-10T07:51:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.281+0000 7f5d4a000640 1 -- 172.21.15.46:0/542355001 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 56) ==== 8398+0+0 (secure 0 0 0) 0x7f5d5003d030 con 0x56057660b260 2024-09-10T07:51:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.282+0000 7f5d6e3fa740 10 client.28900 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:51:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.282+0000 7f5d6e3fa740 10 client.28900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.282+0000 7f5d6e3fa740 20 client.28900 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.282+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560576631f20 0x560576634370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.282+0000 7f5d6e3fa740 4 client.28900 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.282+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/542355001 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56057657d830 con 0x560576631f20 2024-09-10T07:51:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.283+0000 7f5d68a00640 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560576631f20 0x560576634370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.284+0000 7f5d68a00640 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560576631f20 0x560576634370 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f249f7c4e32f389c server_cookie=86ae5e28a89846e6 in_seq=0 out_seq=0 2024-09-10T07:51:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.284+0000 7f5d4a000640 10 client.28900 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d4a000640 1 -- 172.21.15.46:0/542355001 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56057657d830 con 0x560576631f20 2024-09-10T07:51:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d4a000640 10 client.28900 handle_command_reply: tid=0 2024-09-10T07:51:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 client.28900 shutdown 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/542355001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660b260 msgr2=0x560576615480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660b260 0x560576615480 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009980 tx=0x7f5d50000bc0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/542355001 shutdown_connections 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x560576631f20 0x560576634370 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 0x56057661ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x5605766159c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/542355001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660b260 0x560576615480 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/542355001 >> 172.21.15.46:0/542355001 conn(0x5605766068d0 msgr2=0x5605766080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.286+0000 7f5d4a000640 0 client.28900 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.287+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/542355001 shutdown_connections 2024-09-10T07:51:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.287+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/542355001 wait complete. 2024-09-10T07:51:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.287+0000 7f5d6e3fa740 20 client.28900 trim_cache size 0 max 16384 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.307+0000 7f5d6e3fa740 1 Processor -- start 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.307+0000 7f5d6e3fa740 1 -- start start 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.307+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x560576607780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.307+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x56057661c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.307+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 0x560576607cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.307+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641a82f0 con 0x56057660b260 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.307+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641a9f80 con 0x56057660bba0 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.307+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641a89c0 con 0x560576610c30 2024-09-10T07:51:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x56057661c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x56057661c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49948/0 (socket says 172.21.15.46:49948) 2024-09-10T07:51:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 -- 172.21.15.46:0/3690749898 learned_addr learned my addr 172.21.15.46:0/3690749898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x560576607780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 -- 172.21.15.46:0/3690749898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 msgr2=0x560576607cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 --2- 172.21.15.46:0/3690749898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 0x560576607cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 -- 172.21.15.46:0/3690749898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 msgr2=0x560576607780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 --2- 172.21.15.46:0/3690749898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x560576607780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 -- 172.21.15.46:0/3690749898 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605766066f0 con 0x56057660bba0 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.308+0000 7f5d68a00640 1 --2- 172.21.15.46:0/3690749898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x56057661c9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800ab40 tx=0x7f5d58012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0982efb6c0b0988 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.309+0000 7f5d55200640 1 -- 172.21.15.46:0/3690749898 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5800a740 con 0x56057660bba0 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.309+0000 7f5d55200640 1 -- 172.21.15.46:0/3690749898 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d58031070 con 0x56057660bba0 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.309+0000 7f5d55200640 1 -- 172.21.15.46:0/3690749898 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58005470 con 0x56057660bba0 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.309+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/3690749898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 msgr2=0x56057661c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.309+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/3690749898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x56057661c9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800ab40 tx=0x7f5d58012040 comp rx=0 tx=0).stop 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.310+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/3690749898 shutdown_connections 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.310+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/3690749898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560576610c30 0x560576607cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.310+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/3690749898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660bba0 0x56057661c9c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.310+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/3690749898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56057660b260 0x560576607780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.310+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/3690749898 >> 172.21.15.46:0/3690749898 conn(0x5605766068d0 msgr2=0x560576615230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.310+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/3690749898 shutdown_connections 2024-09-10T07:51:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.310+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/3690749898 wait complete. 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.310+0000 7f5d6e3fa740 1 -- start start 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d6e3fa740 1 Processor -- start 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660b260 0x560576619860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660bba0 0x560576619da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d6e3fa740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 0x56057669ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641a98b0 con 0x560576610c30 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641a9150 con 0x56057660b260 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d6e3fa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d64073db0 con 0x56057660bba0 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d6e3fa740 10 client.0 _ll_register_callbacks cb 0x7f5d5c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d68a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660bba0 0x560576619da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d68a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660bba0 0x560576619da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:57058/0 (socket says 172.21.15.46:57058) 2024-09-10T07:51:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.311+0000 7f5d68a00640 1 -- 172.21.15.46:0/1138893832 learned_addr learned my addr 172.21.15.46:0/1138893832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d63e00640 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 0x56057669ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d63e00640 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660b260 0x560576619860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d68a00640 1 -- 172.21.15.46:0/1138893832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660b260 msgr2=0x560576619860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d68a00640 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660b260 0x560576619860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d68a00640 1 -- 172.21.15.46:0/1138893832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 msgr2=0x56057669ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d68a00640 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 0x56057669ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d68a00640 1 -- 172.21.15.46:0/1138893832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56057669b180 con 0x56057660bba0 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d68a00640 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660bba0 0x560576619da0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d5801a040 tx=0x7f5d58004100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c19209d72f433431 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.312+0000 7f5d4a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.313+0000 7f5d4a000640 1 -- 172.21.15.46:0/1138893832 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d580051b0 con 0x56057660bba0 2024-09-10T07:51:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.313+0000 7f5d4a000640 1 -- 172.21.15.46:0/1138893832 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5d58005350 con 0x56057660bba0 2024-09-10T07:51:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.313+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/1138893832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d58031070 con 0x56057660bba0 2024-09-10T07:51:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.313+0000 7f5d4a000640 1 -- 172.21.15.46:0/1138893832 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f5d58046050 con 0x56057660bba0 2024-09-10T07:51:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.313+0000 7f5d6e3fa740 10 client.28903 fetch_fsmap learned FSMap version 56 2024-09-10T07:51:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.313+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/1138893832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56057669c430 con 0x56057660bba0 2024-09-10T07:51:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.313+0000 7f5d4a000640 1 -- 172.21.15.46:0/1138893832 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d58031070 con 0x56057660bba0 2024-09-10T07:51:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.314+0000 7f5d4a000640 1 -- 172.21.15.46:0/1138893832 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 56) ==== 8398+0+0 (secure 0 0 0) 0x7f5d58018440 con 0x56057660bba0 2024-09-10T07:51:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.314+0000 7f5d6e3fa740 10 client.28903 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:51:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.314+0000 7f5d6e3fa740 10 client.28903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.314+0000 7f5d6e3fa740 20 client.28903 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.315+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56057669cb20 0x56057669ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.315+0000 7f5d6e3fa740 4 client.28903 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:51:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.315+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/1138893832 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560576577b80 con 0x56057669cb20 2024-09-10T07:51:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.316+0000 7f5d63e00640 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56057669cb20 0x56057669ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.318+0000 7f5d63e00640 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56057669cb20 0x56057669ef40 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60338c87afd1a097 server_cookie=484b6d147ea0562e in_seq=0 out_seq=0 2024-09-10T07:51:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.318+0000 7f5d4a000640 10 client.28903 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.345+0000 7f5d4a000640 1 -- 172.21.15.46:0/1138893832 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560576577b80 con 0x56057669cb20 2024-09-10T07:51:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.345+0000 7f5d4a000640 10 client.28903 handle_command_reply: tid=0 2024-09-10T07:51:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.345+0000 7f5d6e3fa740 1 client.28903 shutdown 2024-09-10T07:51:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.348+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/1138893832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660bba0 msgr2=0x560576619da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.348+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660bba0 0x560576619da0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d5801a040 tx=0x7f5d58004100 comp rx=0 tx=0).stop 2024-09-10T07:51:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.348+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/1138893832 shutdown_connections 2024-09-10T07:51:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.348+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56057669cb20 0x56057669ef40 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.348+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560576610c30 0x56057669ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.348+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56057660bba0 0x560576619da0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.348+0000 7f5d6e3fa740 1 --2- 172.21.15.46:0/1138893832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56057660b260 0x560576619860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.348+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/1138893832 >> 172.21.15.46:0/1138893832 conn(0x5605766068d0 msgr2=0x560576607750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.350+0000 7f5d4a000640 0 client.28903 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.350+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/1138893832 shutdown_connections 2024-09-10T07:51:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.350+0000 7f5d6e3fa740 1 -- 172.21.15.46:0/1138893832 wait complete. 2024-09-10T07:51:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.350+0000 7f5d6e3fa740 20 client.28903 trim_cache size 0 max 16384 2024-09-10T07:51:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:51:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:51:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:51:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:51:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 -- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5d4c0a2c40 msgr2=0x7f5d4c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5d4c0a2c40 0x7f5d4c0a5100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00cd00 tx=0x7f5d5c004170 comp rx=0 tx=0).stop 2024-09-10T07:51:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 -- 172.21.15.46:0/2905595505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d64078790 msgr2=0x7f5d641a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d64078790 0x7f5d641a5860 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009fd0 tx=0x7f5d5c00c800 comp rx=0 tx=0).stop 2024-09-10T07:51:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 -- 172.21.15.46:0/2905595505 shutdown_connections 2024-09-10T07:51:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d64078790 0x7f5d641a5860 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5d4c0a2c40 0x7f5d4c0a5100 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d64077df0 0x7f5d6419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 --2- 172.21.15.46:0/2905595505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d64076bb0 0x7f5d6419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.353+0000 7f5d69e00640 1 -- 172.21.15.46:0/2905595505 >> 172.21.15.46:0/2905595505 conn(0x7f5d640ff2f0 msgr2=0x7f5d64100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.354+0000 7f5d69e00640 1 -- 172.21.15.46:0/2905595505 shutdown_connections 2024-09-10T07:51:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:08.355+0000 7f5d69e00640 1 -- 172.21.15.46:0/2905595505 wait complete. 2024-09-10T07:51:08.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:08 smithi046 ceph-mon[30705]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 5.0 MiB/s wr, 610 op/s 2024-09-10T07:51:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:08 smithi181 ceph-mon[35397]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 5.0 MiB/s wr, 610 op/s 2024-09-10T07:51:08.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:08 smithi063 ceph-mon[35020]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 5.0 MiB/s wr, 610 op/s 2024-09-10T07:51:09.061 INFO:tasks.cephfs.filesystem:scrub status for tag:8f96c08f-1cd7-4ca4-aabd-527fd6439e14 - {'path': '/', 'tag': '8f96c08f-1cd7-4ca4-aabd-527fd6439e14', 'options': 'recursive,force'} 2024-09-10T07:51:10.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 830.0 s, 601.8 tps, lat 166.973 ms stddev 246.254 2024-09-10T07:51:10.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:10 smithi181 ceph-mon[35397]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 4.3 MiB/s wr, 544 op/s 2024-09-10T07:51:10.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:10 smithi063 ceph-mon[35020]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 4.3 MiB/s wr, 544 op/s 2024-09-10T07:51:10.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:10 smithi046 ceph-mon[30705]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 4.3 MiB/s wr, 544 op/s 2024-09-10T07:51:12.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:12 smithi181 ceph-mon[35397]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.0 MiB/s wr, 506 op/s 2024-09-10T07:51:12.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:12 smithi063 ceph-mon[35020]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.0 MiB/s wr, 506 op/s 2024-09-10T07:51:12.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:12 smithi046 ceph-mon[30705]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.0 MiB/s wr, 506 op/s 2024-09-10T07:51:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:14 smithi181 ceph-mon[35397]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.3 MiB/s wr, 620 op/s 2024-09-10T07:51:14.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:14 smithi063 ceph-mon[35020]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.3 MiB/s wr, 620 op/s 2024-09-10T07:51:14.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:14 smithi046 ceph-mon[30705]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.3 MiB/s wr, 620 op/s 2024-09-10T07:51:15.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 835.0 s, 571.8 tps, lat 162.270 ms stddev 207.034 2024-09-10T07:51:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:16 smithi181 ceph-mon[35397]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 4.2 MiB/s wr, 532 op/s 2024-09-10T07:51:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:16 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:51:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:16 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:51:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:16 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:51:16.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:16 smithi063 ceph-mon[35020]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 4.2 MiB/s wr, 532 op/s 2024-09-10T07:51:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:16 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:51:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:16 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:51:16.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:16 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:51:16.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:16 smithi046 ceph-mon[30705]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 4.2 MiB/s wr, 532 op/s 2024-09-10T07:51:16.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:16 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:51:16.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:16 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:51:16.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:16 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:51:17.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:17 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:51:17.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:17 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:51:17.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:17 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:51:17.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:17 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:51:17.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:17 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:51:17.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:17 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:51:18.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:18 smithi046 ceph-mon[30705]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-09-10T07:51:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:18 smithi181 ceph-mon[35397]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-09-10T07:51:18.652 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:18 smithi063 ceph-mon[35020]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-09-10T07:51:20.601 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 840.0 s, 603.2 tps, lat 173.041 ms stddev 267.894 2024-09-10T07:51:20.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:20 smithi181 ceph-mon[35397]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-09-10T07:51:20.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:20 smithi063 ceph-mon[35020]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-09-10T07:51:20.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:20 smithi046 ceph-mon[30705]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-09-10T07:51:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.651+0000 7fdad9400640 1 -- 172.21.15.46:0/2725360096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad4105910 msgr2=0x7fdad410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.651+0000 7fdad9400640 1 --2- 172.21.15.46:0/2725360096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad4105910 0x7fdad410c1a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009900 tx=0x7fdac002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.653+0000 7fdad9400640 1 -- 172.21.15.46:0/2725360096 shutdown_connections 2024-09-10T07:51:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.653+0000 7fdad9400640 1 --2- 172.21.15.46:0/2725360096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad4105910 0x7fdad410c1a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.653+0000 7fdad9400640 1 --2- 172.21.15.46:0/2725360096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad4104f50 0x7fdad41053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.653+0000 7fdad9400640 1 --2- 172.21.15.46:0/2725360096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4103d60 0x7fdad4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.653+0000 7fdad9400640 1 -- 172.21.15.46:0/2725360096 >> 172.21.15.46:0/2725360096 conn(0x7fdad40ff4e0 msgr2=0x7fdad4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.655+0000 7fdad9400640 1 -- 172.21.15.46:0/2725360096 shutdown_connections 2024-09-10T07:51:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.655+0000 7fdad9400640 1 -- 172.21.15.46:0/2725360096 wait complete. 2024-09-10T07:51:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.656+0000 7fdad9400640 1 Processor -- start 2024-09-10T07:51:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.656+0000 7fdad9400640 1 -- start start 2024-09-10T07:51:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.657+0000 7fdad9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad4103d60 0x7fdad419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.658+0000 7fdad9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4104f50 0x7fdad419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.658+0000 7fdad9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad4105910 0x7fdad41a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.658+0000 7fdad2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad4103d60 0x7fdad419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.658+0000 7fdad2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad4103d60 0x7fdad419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34548/0 (socket says 172.21.15.46:34548) 2024-09-10T07:51:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.658+0000 7fdad2800640 1 -- 172.21.15.46:0/266090712 learned_addr learned my addr 172.21.15.46:0/266090712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.658+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdad410fa80 con 0x7fdad4104f50 2024-09-10T07:51:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.658+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdad410f900 con 0x7fdad4105910 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.658+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdad410fc00 con 0x7fdad4103d60 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.659+0000 7fdad3200640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad4105910 0x7fdad41a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.659+0000 7fdad2800640 1 -- 172.21.15.46:0/266090712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad4105910 msgr2=0x7fdad41a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.659+0000 7fdad2800640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad4105910 0x7fdad41a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.659+0000 7fdad2800640 1 -- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4104f50 msgr2=0x7fdad419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.659+0000 7fdad2800640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4104f50 0x7fdad419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.659+0000 7fdad2800640 1 -- 172.21.15.46:0/266090712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad41a1c90 con 0x7fdad4103d60 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.659+0000 7fdad2800640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad4103d60 0x7fdad419a070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00b9c0 tx=0x7fdacc00be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.659+0000 7fdacb400640 1 -- 172.21.15.46:0/266090712 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc019070 con 0x7fdad4103d60 2024-09-10T07:51:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.660+0000 7fdacb400640 1 -- 172.21.15.46:0/266090712 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc00cba0 con 0x7fdad4103d60 2024-09-10T07:51:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.660+0000 7fdacb400640 1 -- 172.21.15.46:0/266090712 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc012670 con 0x7fdad4103d60 2024-09-10T07:51:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.660+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41a1f80 con 0x7fdad4103d60 2024-09-10T07:51:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.660+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41a2470 con 0x7fdad4103d60 2024-09-10T07:51:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.666+0000 7fdadc3c2740 1 Processor -- start 2024-09-10T07:51:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.667+0000 7fdadc3c2740 1 -- start start 2024-09-10T07:51:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.667+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d367090 0x55fb9d367490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.667+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 0x55fb9d36c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.667+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d36ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.667+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fb9d2cc100 con 0x55fb9d367090 2024-09-10T07:51:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.667+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fb9d2741b0 con 0x55fb9d36ca60 2024-09-10T07:51:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.667+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fb9d2c80c0 con 0x55fb9d3679d0 2024-09-10T07:51:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.669+0000 7fdad1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d36ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.669+0000 7fdad1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d36ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47434/0 (socket says 172.21.15.46:47434) 2024-09-10T07:51:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.669+0000 7fdad1e00640 1 -- 172.21.15.46:0/2828469465 learned_addr learned my addr 172.21.15.46:0/2828469465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.669+0000 7fdad3200640 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d367090 0x55fb9d367490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.670+0000 7fdacb400640 1 -- 172.21.15.46:0/266090712 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdacc014440 con 0x7fdad4103d60 2024-09-10T07:51:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.671+0000 7fdad1e00640 1 -- 172.21.15.46:0/2828469465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 msgr2=0x55fb9d36c490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:51:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.671+0000 7fdad2800640 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 0x55fb9d36c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.671+0000 7fdad1e00640 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 0x55fb9d36c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.671+0000 7fdad1e00640 1 -- 172.21.15.46:0/2828469465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d367090 msgr2=0x55fb9d367490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.671+0000 7fdad1e00640 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d367090 0x55fb9d367490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.671+0000 7fdad1e00640 1 -- 172.21.15.46:0/2828469465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb9cfdf210 con 0x55fb9d36ca60 2024-09-10T07:51:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.671+0000 7fdad1e00640 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d36ef20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009be0 tx=0x7fdac400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e3199b6bc2be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.672+0000 7fdabf400640 1 -- 172.21.15.46:0/2828469465 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4005020 con 0x55fb9d36ca60 2024-09-10T07:51:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.672+0000 7fdabf400640 1 -- 172.21.15.46:0/2828469465 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac40047a0 con 0x55fb9d36ca60 2024-09-10T07:51:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.672+0000 7fdabf400640 1 -- 172.21.15.46:0/2828469465 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4010690 con 0x55fb9d36ca60 2024-09-10T07:51:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.672+0000 7fdacb400640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdab80a2d00 0x7fdab80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.674+0000 7fdacb400640 1 -- 172.21.15.46:0/266090712 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdacc0ce8c0 con 0x7fdad4103d60 2024-09-10T07:51:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.675+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2828469465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 msgr2=0x55fb9d36ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.675+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d36ef20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009be0 tx=0x7fdac400e720 comp rx=0 tx=0).stop 2024-09-10T07:51:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.675+0000 7fdad3200640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdab80a2d00 0x7fdab80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.677+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2828469465 shutdown_connections 2024-09-10T07:51:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.677+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d36ef20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.677+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 0x55fb9d36c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.677+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2828469465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d367090 0x55fb9d367490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.677+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2828469465 >> 172.21.15.46:0/2828469465 conn(0x55fb9d362700 msgr2=0x55fb9d362b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.677+0000 7fdad3200640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdab80a2d00 0x7fdab80a51c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdac0031440 tx=0x7fdac0004300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.679+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2828469465 shutdown_connections 2024-09-10T07:51:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.679+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2828469465 wait complete. 2024-09-10T07:51:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.679+0000 7fdadc3c2740 1 -- start start 2024-09-10T07:51:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.680+0000 7fdadc3c2740 1 Processor -- start 2024-09-10T07:51:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d379a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 0x55fb9d379f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d380fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fb9d36fa30 con 0x55fb9d3679d0 2024-09-10T07:51:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fb9d255090 con 0x55fb9d36ca60 2024-09-10T07:51:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fb9d36fbb0 con 0x55fb9d367090 2024-09-10T07:51:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdadc3c2740 10 client.0 _ll_register_callbacks cb 0x55fb9d2a9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdad3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d380fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdad3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d380fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47442/0 (socket says 172.21.15.46:47442) 2024-09-10T07:51:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.681+0000 7fdad3200640 1 -- 172.21.15.46:0/2670085829 learned_addr learned my addr 172.21.15.46:0/2670085829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.682+0000 7fdad1e00640 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 0x55fb9d379f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.683+0000 7fdad3200640 1 -- 172.21.15.46:0/2670085829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 msgr2=0x55fb9d379a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:51:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.683+0000 7fdad3200640 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d379a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.683+0000 7fdad3200640 1 -- 172.21.15.46:0/2670085829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 msgr2=0x55fb9d379f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.683+0000 7fdad3200640 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 0x55fb9d379f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.683+0000 7fdad3200640 1 -- 172.21.15.46:0/2670085829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb9d0ab620 con 0x55fb9d36ca60 2024-09-10T07:51:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.683+0000 7fdad3200640 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d380fb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdac0033000 tx=0x7fdac0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea9b1a2f2f692e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.683+0000 7fdab3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.684+0000 7fdab3e00640 1 -- 172.21.15.46:0/2670085829 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac004c070 con 0x55fb9d36ca60 2024-09-10T07:51:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.684+0000 7fdab3e00640 1 -- 172.21.15.46:0/2670085829 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdac00328c0 con 0x55fb9d36ca60 2024-09-10T07:51:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.685+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2670085829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac40047a0 con 0x55fb9d36ca60 2024-09-10T07:51:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.687+0000 7fdab3e00640 1 -- 172.21.15.46:0/2670085829 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fdac0056050 con 0x55fb9d36ca60 2024-09-10T07:51:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.687+0000 7fdadc3c2740 10 client.29027 fetch_fsmap learned FSMap version 56 2024-09-10T07:51:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.687+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2670085829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55fb9d383e10 con 0x55fb9d36ca60 2024-09-10T07:51:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.688+0000 7fdab3e00640 1 -- 172.21.15.46:0/2670085829 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac40047a0 con 0x55fb9d36ca60 2024-09-10T07:51:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.691+0000 7fdab3e00640 1 -- 172.21.15.46:0/2670085829 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 56) ==== 8398+0+0 (secure 0 0 0) 0x7fdac0003bd0 con 0x55fb9d36ca60 2024-09-10T07:51:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.691+0000 7fdadc3c2740 10 client.29027 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:51:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.691+0000 7fdadc3c2740 10 client.29027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.691+0000 7fdadc3c2740 20 client.29027 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.691+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb9d385550 0x55fb9d3879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.691+0000 7fdadc3c2740 4 client.29027 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.691+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2670085829 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb9d2d9830 con 0x55fb9d385550 2024-09-10T07:51:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.693+0000 7fdad2800640 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb9d385550 0x55fb9d3879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.694+0000 7fdad2800640 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb9d385550 0x55fb9d3879a0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7314e50ef542a1a0 server_cookie=16f06558a55a2fe9 in_seq=0 out_seq=0 2024-09-10T07:51:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.694+0000 7fdab3e00640 10 client.29027 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.695+0000 7fdab3e00640 1 -- 172.21.15.46:0/2670085829 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb9d2d9830 con 0x55fb9d385550 2024-09-10T07:51:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.695+0000 7fdab3e00640 10 client.29027 handle_command_reply: tid=0 2024-09-10T07:51:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.695+0000 7fdadc3c2740 1 client.29027 shutdown 2024-09-10T07:51:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2670085829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 msgr2=0x55fb9d380fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d380fb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdac0033000 tx=0x7fdac0009260 comp rx=0 tx=0).stop 2024-09-10T07:51:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2670085829 shutdown_connections 2024-09-10T07:51:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb9d385550 0x55fb9d3879a0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d380fb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 0x55fb9d379f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2670085829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d379a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdab3e00640 0 client.29027 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.696+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2670085829 >> 172.21.15.46:0/2670085829 conn(0x55fb9d362700 msgr2=0x55fb9d363f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.698+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2670085829 shutdown_connections 2024-09-10T07:51:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.698+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2670085829 wait complete. 2024-09-10T07:51:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.698+0000 7fdadc3c2740 20 client.29027 trim_cache size 0 max 16384 2024-09-10T07:51:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.741+0000 7fdadc3c2740 1 Processor -- start 2024-09-10T07:51:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdadc3c2740 1 -- start start 2024-09-10T07:51:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d36c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 0x55fb9d380f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d3635b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41a83d0 con 0x55fb9d3679d0 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41aa060 con 0x55fb9d36ca60 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41a8aa0 con 0x55fb9d367090 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdad3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d3635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdad2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d36c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdad3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d3635b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47452/0 (socket says 172.21.15.46:47452) 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdad2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d36c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:34566/0 (socket says 172.21.15.46:34566) 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdad3200640 1 -- 172.21.15.46:0/1666141427 learned_addr learned my addr 172.21.15.46:0/1666141427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.742+0000 7fdad1e00640 1 --2- 172.21.15.46:0/1666141427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 0x55fb9d380f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.743+0000 7fdad2800640 1 -- 172.21.15.46:0/1666141427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 msgr2=0x55fb9d3635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.743+0000 7fdad2800640 1 --2- 172.21.15.46:0/1666141427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d3635b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.743+0000 7fdad2800640 1 -- 172.21.15.46:0/1666141427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 msgr2=0x55fb9d380f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.743+0000 7fdad2800640 1 --2- 172.21.15.46:0/1666141427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 0x55fb9d380f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.743+0000 7fdad2800640 1 -- 172.21.15.46:0/1666141427 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb9d368660 con 0x55fb9d367090 2024-09-10T07:51:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.743+0000 7fdad2800640 1 --2- 172.21.15.46:0/1666141427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d36c1d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0cc090 tx=0x7fdacc0d0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e240fda6a8b2e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.743+0000 7fdabea00640 1 -- 172.21.15.46:0/1666141427 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0a3a10 con 0x55fb9d367090 2024-09-10T07:51:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.744+0000 7fdabea00640 1 -- 172.21.15.46:0/1666141427 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc098e40 con 0x55fb9d367090 2024-09-10T07:51:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.744+0000 7fdabea00640 1 -- 172.21.15.46:0/1666141427 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc097410 con 0x55fb9d367090 2024-09-10T07:51:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.744+0000 7fdadc3c2740 1 -- 172.21.15.46:0/1666141427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 msgr2=0x55fb9d36c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.744+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/1666141427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d36c1d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0cc090 tx=0x7fdacc0d0040 comp rx=0 tx=0).stop 2024-09-10T07:51:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.745+0000 7fdadc3c2740 1 -- 172.21.15.46:0/1666141427 shutdown_connections 2024-09-10T07:51:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.745+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/1666141427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d36ca60 0x55fb9d3635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.745+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/1666141427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d3679d0 0x55fb9d380f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.745+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/1666141427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d367090 0x55fb9d36c1d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.745+0000 7fdadc3c2740 1 -- 172.21.15.46:0/1666141427 >> 172.21.15.46:0/1666141427 conn(0x55fb9d362700 msgr2=0x55fb9d379a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.745+0000 7fdadc3c2740 1 -- 172.21.15.46:0/1666141427 shutdown_connections 2024-09-10T07:51:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.745+0000 7fdadc3c2740 1 -- 172.21.15.46:0/1666141427 wait complete. 2024-09-10T07:51:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.746+0000 7fdadc3c2740 1 -- start start 2024-09-10T07:51:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.746+0000 7fdadc3c2740 1 Processor -- start 2024-09-10T07:51:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.746+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d367090 0x55fb9d37c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.746+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 0x55fb9d37c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdadc3c2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d36ca60 0x55fb9d37f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41a9990 con 0x55fb9d36ca60 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41a9230 con 0x55fb9d367090 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdadc3c2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41a6460 con 0x55fb9d3679d0 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdadc3c2740 10 client.0 _ll_register_callbacks cb 0x7fdac4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdad2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d367090 0x55fb9d37c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdad2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d367090 0x55fb9d37c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47466/0 (socket says 172.21.15.46:47466) 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdad2800640 1 -- 172.21.15.46:0/2705006470 learned_addr learned my addr 172.21.15.46:0/2705006470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdad3200640 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d36ca60 0x55fb9d37f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.747+0000 7fdad1e00640 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 0x55fb9d37c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdad2800640 1 -- 172.21.15.46:0/2705006470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 msgr2=0x55fb9d37c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdad2800640 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 0x55fb9d37c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdad2800640 1 -- 172.21.15.46:0/2705006470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d36ca60 msgr2=0x55fb9d37f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdad2800640 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d36ca60 0x55fb9d37f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdad2800640 1 -- 172.21.15.46:0/2705006470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb9d37fd90 con 0x55fb9d367090 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdab3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdad2800640 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d367090 0x55fb9d37c2c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdacc097770 tx=0x7fdacc09aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687b3b68c307591f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdab3e00640 1 -- 172.21.15.46:0/2705006470 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0b8070 con 0x55fb9d367090 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdab3e00640 1 -- 172.21.15.46:0/2705006470 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fdacc0149b0 con 0x55fb9d367090 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.748+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2705006470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdacc098e40 con 0x55fb9d367090 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdab3e00640 1 -- 172.21.15.46:0/2705006470 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fdacc0b8070 con 0x55fb9d367090 2024-09-10T07:51:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdadc3c2740 10 client.29033 fetch_fsmap learned FSMap version 56 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2705006470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55fb9d3810a0 con 0x55fb9d367090 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdab3e00640 1 -- 172.21.15.46:0/2705006470 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc098e40 con 0x55fb9d367090 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdab3e00640 1 -- 172.21.15.46:0/2705006470 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 56) ==== 8398+0+0 (secure 0 0 0) 0x7fdacc0a3610 con 0x55fb9d367090 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdadc3c2740 10 client.29033 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdadc3c2740 10 client.29033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdadc3c2740 20 client.29033 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb9d401740 0x55fb9d403b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdadc3c2740 4 client.29033 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:51:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.749+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2705006470 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb9d2d3b80 con 0x55fb9d401740 2024-09-10T07:51:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.750+0000 7fdad1e00640 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb9d401740 0x55fb9d403b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.750+0000 7fdab3e00640 10 client.29033 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.750+0000 7fdad1e00640 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb9d401740 0x55fb9d403b60 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b240eef4c10f35db server_cookie=8a2dc186db445f4e in_seq=0 out_seq=0 2024-09-10T07:51:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdab3e00640 1 -- 172.21.15.46:0/2705006470 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb9d2d3b80 con 0x55fb9d401740 2024-09-10T07:51:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdab3e00640 10 client.29033 handle_command_reply: tid=0 2024-09-10T07:51:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 client.29033 shutdown 2024-09-10T07:51:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2705006470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d367090 msgr2=0x55fb9d37c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d367090 0x55fb9d37c2c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdacc097770 tx=0x7fdacc09aca0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2705006470 shutdown_connections 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fb9d401740 0x55fb9d403b60 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb9d36ca60 0x55fb9d37f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb9d3679d0 0x55fb9d37c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 --2- 172.21.15.46:0/2705006470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb9d367090 0x55fb9d37c2c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.761+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2705006470 >> 172.21.15.46:0/2705006470 conn(0x55fb9d362700 msgr2=0x55fb9d379a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.762+0000 7fdab3e00640 0 client.29033 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.762+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2705006470 shutdown_connections 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.762+0000 7fdadc3c2740 1 -- 172.21.15.46:0/2705006470 wait complete. 2024-09-10T07:51:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.762+0000 7fdadc3c2740 20 client.29033 trim_cache size 0 max 16384 2024-09-10T07:51:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:51:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:51:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:51:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:51:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdab80a2d00 msgr2=0x7fdab80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdab80a2d00 0x7fdab80a51c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdac0031440 tx=0x7fdac0004300 comp rx=0 tx=0).stop 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad4103d60 msgr2=0x7fdad419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad4103d60 0x7fdad419a070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00b9c0 tx=0x7fdacc00be80 comp rx=0 tx=0).stop 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 shutdown_connections 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad4105910 0x7fdad41a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fdab80a2d00 0x7fdab80a51c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4104f50 0x7fdad419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 --2- 172.21.15.46:0/266090712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad4103d60 0x7fdad419a070 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.763+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 >> 172.21.15.46:0/266090712 conn(0x7fdad40ff4e0 msgr2=0x7fdad41081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.764+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 shutdown_connections 2024-09-10T07:51:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:21.764+0000 7fdad9400640 1 -- 172.21.15.46:0/266090712 wait complete. 2024-09-10T07:51:22.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:22 smithi181 ceph-mon[35397]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-09-10T07:51:22.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:22 smithi063 ceph-mon[35020]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-09-10T07:51:22.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:22 smithi046 ceph-mon[30705]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-09-10T07:51:23.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:23 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:51:23.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:23 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:51:23.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:23 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:51:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:24 smithi181 ceph-mon[35397]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 5.6 MiB/s wr, 569 op/s 2024-09-10T07:51:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:51:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:51:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:24 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:51:24.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:24 smithi063 ceph-mon[35020]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 5.6 MiB/s wr, 569 op/s 2024-09-10T07:51:24.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:51:24.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:51:24.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:24 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:51:24.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:24 smithi046 ceph-mon[30705]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 5.6 MiB/s wr, 569 op/s 2024-09-10T07:51:24.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:51:24.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:51:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:24 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:51:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 845.0 s, 461.4 tps, lat 217.912 ms stddev 287.998 2024-09-10T07:51:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:26 smithi181 ceph-mon[35397]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-09-10T07:51:26.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:26 smithi063 ceph-mon[35020]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-09-10T07:51:26.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:26 smithi046 ceph-mon[30705]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-09-10T07:51:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:28 smithi181 ceph-mon[35397]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-09-10T07:51:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:28 smithi063 ceph-mon[35020]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-09-10T07:51:28.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:28 smithi046 ceph-mon[30705]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-09-10T07:51:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 850.0 s, 481.6 tps, lat 204.502 ms stddev 231.092 2024-09-10T07:51:30.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:30 smithi181 ceph-mon[35397]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.6 MiB/s wr, 497 op/s 2024-09-10T07:51:30.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:30 smithi063 ceph-mon[35020]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.6 MiB/s wr, 497 op/s 2024-09-10T07:51:30.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:30 smithi046 ceph-mon[30705]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.6 MiB/s wr, 497 op/s 2024-09-10T07:51:32.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:32 smithi181 ceph-mon[35397]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-09-10T07:51:32.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:32 smithi063 ceph-mon[35020]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-09-10T07:51:32.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:32 smithi046 ceph-mon[30705]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-09-10T07:51:34.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:34 smithi181 ceph-mon[35397]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.9 MiB/s wr, 538 op/s 2024-09-10T07:51:34.652 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:34 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:34.653 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:34 smithi063 ceph-mon[35020]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.9 MiB/s wr, 538 op/s 2024-09-10T07:51:34.653 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:34 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:34 smithi046 ceph-mon[30705]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.9 MiB/s wr, 538 op/s 2024-09-10T07:51:34.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:34 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 -- 172.21.15.46:0/191792379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe62c103d80 msgr2=0x7fe62c104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 --2- 172.21.15.46:0/191792379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe62c103d80 0x7fe62c104180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009900 tx=0x7fe61c02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 -- 172.21.15.46:0/191792379 shutdown_connections 2024-09-10T07:51:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 --2- 172.21.15.46:0/191792379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c105930 0x7fe62c10c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 --2- 172.21.15.46:0/191792379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe62c104f70 0x7fe62c1053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 --2- 172.21.15.46:0/191792379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe62c103d80 0x7fe62c104180 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 -- 172.21.15.46:0/191792379 >> 172.21.15.46:0/191792379 conn(0x7fe62c0ff4e0 msgr2=0x7fe62c101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 -- 172.21.15.46:0/191792379 shutdown_connections 2024-09-10T07:51:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.973+0000 7fe630e00640 1 -- 172.21.15.46:0/191792379 wait complete. 2024-09-10T07:51:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.976+0000 7fe630e00640 1 Processor -- start 2024-09-10T07:51:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.976+0000 7fe630e00640 1 -- start start 2024-09-10T07:51:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe630e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe62c103d80 0x7fe62c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe630e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c104f70 0x7fe62c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe630e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe62c105930 0x7fe62c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe630e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10fa50 con 0x7fe62c103d80 2024-09-10T07:51:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe630e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10f8d0 con 0x7fe62c105930 2024-09-10T07:51:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe630e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10fbd0 con 0x7fe62c104f70 2024-09-10T07:51:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe62a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe62c105930 0x7fe62c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe62a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe62c105930 0x7fe62c1a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47602/0 (socket says 172.21.15.46:47602) 2024-09-10T07:51:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.977+0000 7fe62a800640 1 -- 172.21.15.46:0/1755860543 learned_addr learned my addr 172.21.15.46:0/1755860543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.980+0000 7fe629400640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c104f70 0x7fe62c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.981+0000 7fe629400640 1 -- 172.21.15.46:0/1755860543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe62c105930 msgr2=0x7fe62c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.981+0000 7fe629400640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe62c105930 0x7fe62c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.981+0000 7fe629400640 1 -- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe62c103d80 msgr2=0x7fe62c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.981+0000 7fe629400640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe62c103d80 0x7fe62c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.981+0000 7fe629400640 1 -- 172.21.15.46:0/1755860543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c1a1c30 con 0x7fe62c104f70 2024-09-10T07:51:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.981+0000 7fe629400640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c104f70 0x7fe62c19a550 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe61800e9f0 tx=0x7fe61800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.982+0000 7fe622a00640 1 -- 172.21.15.46:0/1755860543 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61800ccc0 con 0x7fe62c104f70 2024-09-10T07:51:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.982+0000 7fe622a00640 1 -- 172.21.15.46:0/1755860543 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe618010910 con 0x7fe62c104f70 2024-09-10T07:51:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.982+0000 7fe622a00640 1 -- 172.21.15.46:0/1755860543 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618018960 con 0x7fe62c104f70 2024-09-10T07:51:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.982+0000 7fe630e00640 1 -- 172.21.15.46:0/1755860543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c076690 con 0x7fe62c104f70 2024-09-10T07:51:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.982+0000 7fe630e00640 1 -- 172.21.15.46:0/1755860543 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c076b90 con 0x7fe62c104f70 2024-09-10T07:51:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.989+0000 7fe622a00640 1 -- 172.21.15.46:0/1755860543 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe618010ac0 con 0x7fe62c104f70 2024-09-10T07:51:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.991+0000 7fe633dff740 1 Processor -- start 2024-09-10T07:51:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.991+0000 7fe633dff740 1 -- start start 2024-09-10T07:51:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.991+0000 7fe633dff740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b126490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.991+0000 7fe633dff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b12b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.991+0000 7fe633dff740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 0x55fe0b12df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.991+0000 7fe633dff740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fe0b08b100 con 0x55fe0b12ba60 2024-09-10T07:51:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.991+0000 7fe633dff740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe0b0331b0 con 0x55fe0b126090 2024-09-10T07:51:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.991+0000 7fe633dff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fe0b0870c0 con 0x55fe0b1269d0 2024-09-10T07:51:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.992+0000 7fe622a00640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2cb0 0x7fe6100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.992+0000 7fe622a00640 1 -- 172.21.15.46:0/1755860543 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe6180cf750 con 0x7fe62c104f70 2024-09-10T07:51:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.992+0000 7fe629400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 0x55fe0b12df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.993+0000 7fe629400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 0x55fe0b12df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45728/0 (socket says 172.21.15.46:45728) 2024-09-10T07:51:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.993+0000 7fe629400640 1 -- 172.21.15.46:0/4047010917 learned_addr learned my addr 172.21.15.46:0/4047010917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.993+0000 7fe62a800640 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b126490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.993+0000 7fe62a800640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2cb0 0x7fe6100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.994+0000 7fe629e00640 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b12b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.994+0000 7fe629e00640 1 -- 172.21.15.46:0/4047010917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 msgr2=0x55fe0b126490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.994+0000 7fe629e00640 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b126490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.994+0000 7fe629e00640 1 -- 172.21.15.46:0/4047010917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 msgr2=0x55fe0b12df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.994+0000 7fe629e00640 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 0x55fe0b12df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.994+0000 7fe629e00640 1 -- 172.21.15.46:0/4047010917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0ad9e210 con 0x55fe0b1269d0 2024-09-10T07:51:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.996+0000 7fe629e00640 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b12b490 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe61c02a910 tx=0x7fe61c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1423bc699bbec94e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.996+0000 7fe617400640 1 -- 172.21.15.46:0/4047010917 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c03e070 con 0x55fe0b1269d0 2024-09-10T07:51:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.996+0000 7fe617400640 1 -- 172.21.15.46:0/4047010917 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61c005530 con 0x55fe0b1269d0 2024-09-10T07:51:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.996+0000 7fe617400640 1 -- 172.21.15.46:0/4047010917 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c03e070 con 0x55fe0b1269d0 2024-09-10T07:51:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.998+0000 7fe633dff740 1 -- 172.21.15.46:0/4047010917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 msgr2=0x55fe0b12b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.998+0000 7fe633dff740 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b12b490 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe61c02a910 tx=0x7fe61c000bc0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.998+0000 7fe633dff740 1 -- 172.21.15.46:0/4047010917 shutdown_connections 2024-09-10T07:51:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.998+0000 7fe633dff740 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 0x55fe0b12df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.998+0000 7fe633dff740 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b12b490 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.998+0000 7fe633dff740 1 --2- 172.21.15.46:0/4047010917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b126490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.999+0000 7fe633dff740 1 -- 172.21.15.46:0/4047010917 >> 172.21.15.46:0/4047010917 conn(0x55fe0b121700 msgr2=0x55fe0b121b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:34.999+0000 7fe62a800640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2cb0 0x7fe6100a5170 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe624014fd0 tx=0x7fe624009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.001+0000 7fe633dff740 1 -- 172.21.15.46:0/4047010917 shutdown_connections 2024-09-10T07:51:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.001+0000 7fe633dff740 1 -- 172.21.15.46:0/4047010917 wait complete. 2024-09-10T07:51:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.002+0000 7fe633dff740 1 -- start start 2024-09-10T07:51:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.002+0000 7fe633dff740 1 Processor -- start 2024-09-10T07:51:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.003+0000 7fe633dff740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b1301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.003+0000 7fe633dff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b130720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.003+0000 7fe633dff740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 0x55fe0b137760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.004+0000 7fe629e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b1301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.004+0000 7fe629e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b1301e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47626/0 (socket says 172.21.15.46:47626) 2024-09-10T07:51:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe629e00640 1 -- 172.21.15.46:0/1239881082 learned_addr learned my addr 172.21.15.46:0/1239881082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fe0b12e9d0 con 0x55fe0b12ba60 2024-09-10T07:51:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe0b014090 con 0x55fe0b126090 2024-09-10T07:51:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fe0b12eb50 con 0x55fe0b1269d0 2024-09-10T07:51:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe629400640 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b130720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe633dff740 10 client.0 _ll_register_callbacks cb 0x55fe0b068f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe629400640 1 -- 172.21.15.46:0/1239881082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 msgr2=0x55fe0b1301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe629400640 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b1301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe629400640 1 -- 172.21.15.46:0/1239881082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 msgr2=0x55fe0b137760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe629400640 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 0x55fe0b137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.006+0000 7fe629400640 1 -- 172.21.15.46:0/1239881082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0ae6a620 con 0x55fe0b1269d0 2024-09-10T07:51:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.007+0000 7fe629400640 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b130720 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe6180a4370 tx=0x7fe618010ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca8b69f6e38966a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.007+0000 7fe60be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:51:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.007+0000 7fe60be00640 1 -- 172.21.15.46:0/1239881082 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61809b280 con 0x55fe0b1269d0 2024-09-10T07:51:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.007+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61c005530 con 0x55fe0b1269d0 2024-09-10T07:51:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.008+0000 7fe60be00640 1 -- 172.21.15.46:0/1239881082 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe618010430 con 0x55fe0b1269d0 2024-09-10T07:51:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.010+0000 7fe60be00640 1 -- 172.21.15.46:0/1239881082 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe6180a48a0 con 0x55fe0b1269d0 2024-09-10T07:51:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.010+0000 7fe633dff740 10 client.28927 fetch_fsmap learned FSMap version 57 2024-09-10T07:51:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.010+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55fe0b13a670 con 0x55fe0b1269d0 2024-09-10T07:51:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.010+0000 7fe60be00640 1 -- 172.21.15.46:0/1239881082 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61c005530 con 0x55fe0b1269d0 2024-09-10T07:51:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.012+0000 7fe60be00640 1 -- 172.21.15.46:0/1239881082 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7fe61809aaf0 con 0x55fe0b1269d0 2024-09-10T07:51:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.012+0000 7fe633dff740 10 client.28927 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:51:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.012+0000 7fe633dff740 10 client.28927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.012+0000 7fe633dff740 20 client.28927 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.012+0000 7fe633dff740 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fe0b13bd40 0x55fe0b13e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.012+0000 7fe633dff740 4 client.28927 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.012+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe0b098830 con 0x55fe0b13bd40 2024-09-10T07:51:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.013+0000 7fe629e00640 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fe0b13bd40 0x55fe0b13e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.017+0000 7fe629e00640 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fe0b13bd40 0x55fe0b13e1d0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31425c1f325333c9 server_cookie=3fb6c44371440aaa in_seq=0 out_seq=0 2024-09-10T07:51:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.017+0000 7fe60be00640 10 client.28927 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.018+0000 7fe60be00640 1 -- 172.21.15.46:0/1239881082 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe0b098830 con 0x55fe0b13bd40 2024-09-10T07:51:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.019+0000 7fe60be00640 10 client.28927 handle_command_reply: tid=0 2024-09-10T07:51:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.019+0000 7fe633dff740 1 client.28927 shutdown 2024-09-10T07:51:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.019+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 msgr2=0x55fe0b130720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.019+0000 7fe633dff740 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b130720 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe6180a4370 tx=0x7fe618010ef0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.019+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 shutdown_connections 2024-09-10T07:51:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.019+0000 7fe633dff740 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fe0b13bd40 0x55fe0b13e1d0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.020+0000 7fe633dff740 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b12ba60 0x55fe0b137760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.020+0000 7fe60be00640 0 client.28927 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.020+0000 7fe633dff740 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b130720 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.020+0000 7fe633dff740 1 --2- 172.21.15.46:0/1239881082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b126090 0x55fe0b1301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.020+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 >> 172.21.15.46:0/1239881082 conn(0x55fe0b121700 msgr2=0x55fe0b122f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.021+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 shutdown_connections 2024-09-10T07:51:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.021+0000 7fe633dff740 1 -- 172.21.15.46:0/1239881082 wait complete. 2024-09-10T07:51:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.021+0000 7fe633dff740 20 client.28927 trim_cache size 0 max 16384 2024-09-10T07:51:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.061+0000 7fe633dff740 1 Processor -- start 2024-09-10T07:51:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.061+0000 7fe633dff740 1 -- start start 2024-09-10T07:51:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.062+0000 7fe633dff740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b121700 0x55fe0b122af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.062+0000 7fe633dff740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b126090 0x55fe0b137720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.063+0000 7fe633dff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b123030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.063+0000 7fe633dff740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c00f9c0 con 0x55fe0b126090 2024-09-10T07:51:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.063+0000 7fe633dff740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c0115f0 con 0x55fe0b121700 2024-09-10T07:51:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.063+0000 7fe633dff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c010090 con 0x55fe0b1269d0 2024-09-10T07:51:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.063+0000 7fe629e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b121700 0x55fe0b122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.063+0000 7fe62a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.063+0000 7fe629400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b126090 0x55fe0b137720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.064+0000 7fe629e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b121700 0x55fe0b122af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:47628/0 (socket says 172.21.15.46:47628) 2024-09-10T07:51:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.064+0000 7fe629e00640 1 -- 172.21.15.46:0/1963883430 learned_addr learned my addr 172.21.15.46:0/1963883430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.065+0000 7fe62a800640 1 -- 172.21.15.46:0/1963883430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b121700 msgr2=0x55fe0b122af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.065+0000 7fe62a800640 1 --2- 172.21.15.46:0/1963883430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b121700 0x55fe0b122af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.065+0000 7fe62a800640 1 -- 172.21.15.46:0/1963883430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b126090 msgr2=0x55fe0b137720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.066+0000 7fe62a800640 1 --2- 172.21.15.46:0/1963883430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b126090 0x55fe0b137720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.066+0000 7fe62a800640 1 -- 172.21.15.46:0/1963883430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0b126eb0 con 0x55fe0b1269d0 2024-09-10T07:51:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.066+0000 7fe629400640 1 --2- 172.21.15.46:0/1963883430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b126090 0x55fe0b137720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.067+0000 7fe62a800640 1 --2- 172.21.15.46:0/1963883430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b123030 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe624016010 tx=0x7fe624004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf162fdef725a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.067+0000 7fe616a00640 1 -- 172.21.15.46:0/1963883430 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624032070 con 0x55fe0b1269d0 2024-09-10T07:51:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.067+0000 7fe616a00640 1 -- 172.21.15.46:0/1963883430 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe624024ce0 con 0x55fe0b1269d0 2024-09-10T07:51:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.067+0000 7fe616a00640 1 -- 172.21.15.46:0/1963883430 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62402d910 con 0x55fe0b1269d0 2024-09-10T07:51:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.068+0000 7fe633dff740 1 -- 172.21.15.46:0/1963883430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 msgr2=0x55fe0b123030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.068+0000 7fe633dff740 1 --2- 172.21.15.46:0/1963883430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b123030 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe624016010 tx=0x7fe624004a40 comp rx=0 tx=0).stop 2024-09-10T07:51:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.070+0000 7fe633dff740 1 -- 172.21.15.46:0/1963883430 shutdown_connections 2024-09-10T07:51:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.070+0000 7fe633dff740 1 --2- 172.21.15.46:0/1963883430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b1269d0 0x55fe0b123030 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.070+0000 7fe633dff740 1 --2- 172.21.15.46:0/1963883430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b126090 0x55fe0b137720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.070+0000 7fe633dff740 1 --2- 172.21.15.46:0/1963883430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b121700 0x55fe0b122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.070+0000 7fe633dff740 1 -- 172.21.15.46:0/1963883430 >> 172.21.15.46:0/1963883430 conn(0x55fe0b127490 msgr2=0x55fe0b120880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.071+0000 7fe633dff740 1 -- 172.21.15.46:0/1963883430 shutdown_connections 2024-09-10T07:51:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.071+0000 7fe633dff740 1 -- 172.21.15.46:0/1963883430 wait complete. 2024-09-10T07:51:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.071+0000 7fe633dff740 1 -- start start 2024-09-10T07:51:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.072+0000 7fe633dff740 1 Processor -- start 2024-09-10T07:51:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.073+0000 7fe633dff740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b121700 0x55fe0b1351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.074+0000 7fe633dff740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b126090 0x55fe0b135720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.074+0000 7fe633dff740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b1269d0 0x55fe0b1366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.074+0000 7fe633dff740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c010f20 con 0x55fe0b121700 2024-09-10T07:51:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.074+0000 7fe633dff740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c010820 con 0x55fe0b1269d0 2024-09-10T07:51:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.074+0000 7fe633dff740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c00da50 con 0x55fe0b126090 2024-09-10T07:51:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.074+0000 7fe633dff740 10 client.0 _ll_register_callbacks cb 0x7fe618097ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe629e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b121700 0x55fe0b1351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe629400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b126090 0x55fe0b135720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe629400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b126090 0x55fe0b135720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33660/0 (socket says 172.21.15.46:33660) 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe629400640 1 -- 172.21.15.46:0/3316911238 learned_addr learned my addr 172.21.15.46:0/3316911238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe62a800640 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b1269d0 0x55fe0b1366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe62a800640 1 -- 172.21.15.46:0/3316911238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b126090 msgr2=0x55fe0b135720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe62a800640 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b126090 0x55fe0b135720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe62a800640 1 -- 172.21.15.46:0/3316911238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b121700 msgr2=0x55fe0b1351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe62a800640 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b121700 0x55fe0b1351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.075+0000 7fe62a800640 1 -- 172.21.15.46:0/3316911238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0b136cf0 con 0x55fe0b1269d0 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.076+0000 7fe62a800640 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b1269d0 0x55fe0b1366e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe62400c010 tx=0x7fe624024e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f69b71ab541d6d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.076+0000 7fe60be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.077+0000 7fe60be00640 1 -- 172.21.15.46:0/3316911238 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624032040 con 0x55fe0b1269d0 2024-09-10T07:51:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.077+0000 7fe60be00640 1 -- 172.21.15.46:0/3316911238 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe624024800 con 0x55fe0b1269d0 2024-09-10T07:51:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.078+0000 7fe633dff740 1 -- 172.21.15.46:0/3316911238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe624024ce0 con 0x55fe0b1269d0 2024-09-10T07:51:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.079+0000 7fe60be00640 1 -- 172.21.15.46:0/3316911238 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe624024a00 con 0x55fe0b1269d0 2024-09-10T07:51:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.079+0000 7fe633dff740 10 client.29051 fetch_fsmap learned FSMap version 57 2024-09-10T07:51:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.079+0000 7fe633dff740 1 -- 172.21.15.46:0/3316911238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55fe0b1a6e00 con 0x55fe0b1269d0 2024-09-10T07:51:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.080+0000 7fe60be00640 1 -- 172.21.15.46:0/3316911238 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe624024ce0 con 0x55fe0b1269d0 2024-09-10T07:51:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.080+0000 7fe60be00640 1 -- 172.21.15.46:0/3316911238 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7fe6240388a0 con 0x55fe0b1269d0 2024-09-10T07:51:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.080+0000 7fe633dff740 10 client.29051 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:51:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.080+0000 7fe633dff740 10 client.29051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.080+0000 7fe633dff740 20 client.29051 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.080+0000 7fe633dff740 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fe0b12ba60 0x55fe0b1a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.080+0000 7fe633dff740 4 client.29051 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:51:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.080+0000 7fe633dff740 1 -- 172.21.15.46:0/3316911238 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe0b092b80 con 0x55fe0b12ba60 2024-09-10T07:51:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.081+0000 7fe629e00640 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fe0b12ba60 0x55fe0b1a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.082+0000 7fe629e00640 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fe0b12ba60 0x55fe0b1a9740 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73e0b95f7e088a2 server_cookie=3ae66498ec141ef1 in_seq=0 out_seq=0 2024-09-10T07:51:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.082+0000 7fe60be00640 10 client.29051 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.261+0000 7fe60be00640 1 -- 172.21.15.46:0/3316911238 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe0b092b80 con 0x55fe0b12ba60 2024-09-10T07:51:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.261+0000 7fe60be00640 10 client.29051 handle_command_reply: tid=0 2024-09-10T07:51:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.261+0000 7fe633dff740 1 client.29051 shutdown 2024-09-10T07:51:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 -- 172.21.15.46:0/3316911238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b1269d0 msgr2=0x55fe0b1366e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b1269d0 0x55fe0b1366e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe62400c010 tx=0x7fe624024e90 comp rx=0 tx=0).stop 2024-09-10T07:51:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 -- 172.21.15.46:0/3316911238 shutdown_connections 2024-09-10T07:51:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55fe0b12ba60 0x55fe0b1a9740 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe60be00640 0 client.29051 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe0b1269d0 0x55fe0b1366e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe0b126090 0x55fe0b135720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 --2- 172.21.15.46:0/3316911238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe0b121700 0x55fe0b1351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 -- 172.21.15.46:0/3316911238 >> 172.21.15.46:0/3316911238 conn(0x55fe0b127490 msgr2=0x55fe0b120880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 -- 172.21.15.46:0/3316911238 shutdown_connections 2024-09-10T07:51:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.262+0000 7fe633dff740 1 -- 172.21.15.46:0/3316911238 wait complete. 2024-09-10T07:51:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.263+0000 7fe633dff740 20 client.29051 trim_cache size 0 max 16384 2024-09-10T07:51:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:51:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:51:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:51:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:51:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 -- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2cb0 msgr2=0x7fe6100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2cb0 0x7fe6100a5170 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe624014fd0 tx=0x7fe624009500 comp rx=0 tx=0).stop 2024-09-10T07:51:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 -- 172.21.15.46:0/1755860543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c104f70 msgr2=0x7fe62c19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c104f70 0x7fe62c19a550 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe61800e9f0 tx=0x7fe61800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 -- 172.21.15.46:0/1755860543 shutdown_connections 2024-09-10T07:51:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe62c105930 0x7fe62c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c104f70 0x7fe62c19a550 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe6100a2cb0 0x7fe6100a5170 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 --2- 172.21.15.46:0/1755860543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe62c103d80 0x7fe62c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.266+0000 7fe630e00640 1 -- 172.21.15.46:0/1755860543 >> 172.21.15.46:0/1755860543 conn(0x7fe62c0ff4e0 msgr2=0x7fe62c108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.267+0000 7fe630e00640 1 -- 172.21.15.46:0/1755860543 shutdown_connections 2024-09-10T07:51:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:35.267+0000 7fe630e00640 1 -- 172.21.15.46:0/1755860543 wait complete. 2024-09-10T07:51:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 855.0 s, 537.2 tps, lat 183.155 ms stddev 191.981 2024-09-10T07:51:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:36 smithi181 ceph-mon[35397]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.3 MiB/s wr, 474 op/s 2024-09-10T07:51:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:36 smithi063 ceph-mon[35020]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.3 MiB/s wr, 474 op/s 2024-09-10T07:51:36.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:36 smithi046 ceph-mon[30705]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.3 MiB/s wr, 474 op/s 2024-09-10T07:51:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:38 smithi181 ceph-mon[35397]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-09-10T07:51:38.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:38 smithi063 ceph-mon[35020]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-09-10T07:51:38.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:38 smithi046 ceph-mon[30705]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-09-10T07:51:39.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:51:39.796 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:51:40.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:40 smithi046 ceph-mon[30705]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.8 MiB/s wr, 516 op/s 2024-09-10T07:51:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 860.0 s, 504.2 tps, lat 193.269 ms stddev 225.389 2024-09-10T07:51:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:40 smithi181 ceph-mon[35397]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.8 MiB/s wr, 516 op/s 2024-09-10T07:51:40.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:40 smithi063 ceph-mon[35020]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.8 MiB/s wr, 516 op/s 2024-09-10T07:51:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.669+0000 7fad15400640 1 -- 172.21.15.46:0/2073258854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad100745c0 msgr2=0x7fad100749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.669+0000 7fad15400640 1 --2- 172.21.15.46:0/2073258854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad100745c0 0x7fad100749a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fad00009900 tx=0x7fad0002a4b0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.669+0000 7fad15400640 1 -- 172.21.15.46:0/2073258854 shutdown_connections 2024-09-10T07:51:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.669+0000 7fad15400640 1 --2- 172.21.15.46:0/2073258854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1010e420 0x7fad10110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.669+0000 7fad15400640 1 --2- 172.21.15.46:0/2073258854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad10074ee0 0x7fad1010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.669+0000 7fad15400640 1 --2- 172.21.15.46:0/2073258854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad100745c0 0x7fad100749a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.669+0000 7fad15400640 1 -- 172.21.15.46:0/2073258854 >> 172.21.15.46:0/2073258854 conn(0x7fad100fd8c0 msgr2=0x7fad100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.669+0000 7fad15400640 1 -- 172.21.15.46:0/2073258854 shutdown_connections 2024-09-10T07:51:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.670+0000 7fad15400640 1 -- 172.21.15.46:0/2073258854 wait complete. 2024-09-10T07:51:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.671+0000 7fad15400640 1 Processor -- start 2024-09-10T07:51:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.671+0000 7fad15400640 1 -- start start 2024-09-10T07:51:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad15400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad100745c0 0x7fad1019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad15400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad10074ee0 0x7fad1019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1010e420 0x7fad101964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad15400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad101044b0 con 0x7fad1010e420 2024-09-10T07:51:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad15400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad10104330 con 0x7fad10074ee0 2024-09-10T07:51:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad15400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad10104630 con 0x7fad100745c0 2024-09-10T07:51:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad0e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad100745c0 0x7fad1019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad0e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad100745c0 0x7fad1019bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43928/0 (socket says 172.21.15.46:43928) 2024-09-10T07:51:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.672+0000 7fad0e800640 1 -- 172.21.15.46:0/3992525824 learned_addr learned my addr 172.21.15.46:0/3992525824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:41.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.673+0000 7fad0de00640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad10074ee0 0x7fad1019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.673+0000 7fad0f200640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1010e420 0x7fad101964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.674+0000 7fad0e800640 1 -- 172.21.15.46:0/3992525824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad10074ee0 msgr2=0x7fad1019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.675+0000 7fad0e800640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad10074ee0 0x7fad1019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.675+0000 7fad0e800640 1 -- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1010e420 msgr2=0x7fad101964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.675+0000 7fad0e800640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1010e420 0x7fad101964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.675+0000 7fad0e800640 1 -- 172.21.15.46:0/3992525824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad10196c70 con 0x7fad100745c0 2024-09-10T07:51:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.675+0000 7fad0f200640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1010e420 0x7fad101964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.677+0000 7fad0de00640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad10074ee0 0x7fad1019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.677+0000 7fad0e800640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad100745c0 0x7fad1019bfc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad00004770 tx=0x7fad00003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.678+0000 7fad07400640 1 -- 172.21.15.46:0/3992525824 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0003f070 con 0x7fad100745c0 2024-09-10T07:51:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.678+0000 7fad07400640 1 -- 172.21.15.46:0/3992525824 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad00031ca0 con 0x7fad100745c0 2024-09-10T07:51:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.678+0000 7fad07400640 1 -- 172.21.15.46:0/3992525824 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00035030 con 0x7fad100745c0 2024-09-10T07:51:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.679+0000 7fad15400640 1 -- 172.21.15.46:0/3992525824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad10196f00 con 0x7fad100745c0 2024-09-10T07:51:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.679+0000 7fad15400640 1 -- 172.21.15.46:0/3992525824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a2e20 con 0x7fad100745c0 2024-09-10T07:51:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.687+0000 7fad17fc4740 1 Processor -- start 2024-09-10T07:51:41.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.689+0000 7fad07400640 1 -- 172.21.15.46:0/3992525824 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad00042070 con 0x7fad100745c0 2024-09-10T07:51:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.690+0000 7fad17fc4740 1 -- start start 2024-09-10T07:51:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.690+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818049b90 0x559818049f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.690+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x559818054830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.690+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818054d70 0x559818057160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.690+0000 7fad17fc4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559817fb03e0 con 0x559818054d70 2024-09-10T07:51:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.690+0000 7fad17fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559817f57d80 con 0x559818049b90 2024-09-10T07:51:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.690+0000 7fad17fc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559817fac3a0 con 0x55981804a4b0 2024-09-10T07:51:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x559818054830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x559818054830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43934/0 (socket says 172.21.15.46:43934) 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0e800640 1 -- 172.21.15.46:0/3243263592 learned_addr learned my addr 172.21.15.46:0/3243263592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0f200640 1 --2- 172.21.15.46:0/3243263592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818049b90 0x559818049f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0e800640 1 -- 172.21.15.46:0/3243263592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818049b90 msgr2=0x559818049f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0e800640 1 --2- 172.21.15.46:0/3243263592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818049b90 0x559818049f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0e800640 1 -- 172.21.15.46:0/3243263592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818054d70 msgr2=0x559818057160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0e800640 1 --2- 172.21.15.46:0/3243263592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818054d70 0x559818057160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.691+0000 7fad0e800640 1 -- 172.21.15.46:0/3243263592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559817f76ae0 con 0x55981804a4b0 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.692+0000 7fad0e800640 1 --2- 172.21.15.46:0/3243263592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x559818054830 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fad000f5c10 tx=0x7fad000f5d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76738906dc85f2c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.692+0000 7facfb400640 1 -- 172.21.15.46:0/3243263592 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000fe070 con 0x55981804a4b0 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.693+0000 7facfb400640 1 -- 172.21.15.46:0/3243263592 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad000f7cb0 con 0x55981804a4b0 2024-09-10T07:51:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.693+0000 7facfb400640 1 -- 172.21.15.46:0/3243263592 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000f97a0 con 0x55981804a4b0 2024-09-10T07:51:41.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.697+0000 7fad17fc4740 1 -- 172.21.15.46:0/3243263592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 msgr2=0x559818054830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.697+0000 7fad17fc4740 1 --2- 172.21.15.46:0/3243263592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x559818054830 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fad000f5c10 tx=0x7fad000f5d00 comp rx=0 tx=0).stop 2024-09-10T07:51:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.692+0000 7fad07400640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2cd0 0x7facf40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.697+0000 7fad07400640 1 -- 172.21.15.46:0/3992525824 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fad000ef900 con 0x7fad100745c0 2024-09-10T07:51:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.697+0000 7fad0f200640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2cd0 0x7facf40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.698+0000 7fad17fc4740 1 -- 172.21.15.46:0/3243263592 shutdown_connections 2024-09-10T07:51:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.698+0000 7fad17fc4740 1 --2- 172.21.15.46:0/3243263592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818054d70 0x559818057160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.698+0000 7fad17fc4740 1 --2- 172.21.15.46:0/3243263592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x559818054830 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.698+0000 7fad17fc4740 1 --2- 172.21.15.46:0/3243263592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818049b90 0x559818049f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.698+0000 7fad17fc4740 1 -- 172.21.15.46:0/3243263592 >> 172.21.15.46:0/3243263592 conn(0x559818046120 msgr2=0x559818046500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:41.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.698+0000 7fad17fc4740 1 -- 172.21.15.46:0/3243263592 shutdown_connections 2024-09-10T07:51:41.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.699+0000 7fad17fc4740 1 -- 172.21.15.46:0/3243263592 wait complete. 2024-09-10T07:51:41.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.699+0000 7fad0f200640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2cd0 0x7facf40a5190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fad1006b460 tx=0x7fad0800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:41.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.699+0000 7fad17fc4740 1 -- start start 2024-09-10T07:51:41.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.700+0000 7fad17fc4740 1 Processor -- start 2024-09-10T07:51:41.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.705+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 0x559818051c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.705+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x5598180521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818054d70 0x559818052700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad17fc4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559818057700 con 0x559818049b90 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad17fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559817f6b0f0 con 0x559818054d70 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad17fc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559818057880 con 0x55981804a4b0 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad17fc4740 10 client.0 _ll_register_callbacks cb 0x559817f5c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad0f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818054d70 0x559818052700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad0f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818054d70 0x559818052700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:57998/0 (socket says 172.21.15.46:57998) 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad0f200640 1 -- 172.21.15.46:0/2806537005 learned_addr learned my addr 172.21.15.46:0/2806537005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad0e800640 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 0x559818051c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.706+0000 7fad0de00640 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x5598180521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.707+0000 7fad0f200640 1 -- 172.21.15.46:0/2806537005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 msgr2=0x5598180521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.707+0000 7fad0f200640 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x5598180521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.707+0000 7fad0f200640 1 -- 172.21.15.46:0/2806537005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 msgr2=0x559818051c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.707+0000 7fad0f200640 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 0x559818051c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.707+0000 7fad0f200640 1 -- 172.21.15.46:0/2806537005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559817d213c0 con 0x559818054d70 2024-09-10T07:51:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.707+0000 7fad0f200640 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818054d70 0x559818052700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fad080186a0 tx=0x7fad08004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae4ccbafae78e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.708+0000 7facefe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.708+0000 7facefe00640 1 -- 172.21.15.46:0/2806537005 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08027060 con 0x559818054d70 2024-09-10T07:51:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.708+0000 7facefe00640 1 -- 172.21.15.46:0/2806537005 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fad080167e0 con 0x559818054d70 2024-09-10T07:51:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.708+0000 7fad17fc4740 1 -- 172.21.15.46:0/2806537005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad000f7cb0 con 0x559818054d70 2024-09-10T07:51:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.709+0000 7facefe00640 1 -- 172.21.15.46:0/2806537005 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fad080169e0 con 0x559818054d70 2024-09-10T07:51:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.709+0000 7fad17fc4740 10 client.29063 fetch_fsmap learned FSMap version 58 2024-09-10T07:51:41.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.709+0000 7fad17fc4740 1 -- 172.21.15.46:0/2806537005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559818063440 con 0x559818054d70 2024-09-10T07:51:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.710+0000 7facefe00640 1 -- 172.21.15.46:0/2806537005 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad000f7cb0 con 0x559818054d70 2024-09-10T07:51:41.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.710+0000 7facefe00640 1 -- 172.21.15.46:0/2806537005 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fad0802b3a0 con 0x559818054d70 2024-09-10T07:51:41.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.712+0000 7fad17fc4740 10 client.29063 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T07:51:41.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.712+0000 7fad17fc4740 10 client.29063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:41.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.712+0000 7fad17fc4740 20 client.29063 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.712+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559818064850 0x559818066c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.712+0000 7fad17fc4740 4 client.29063 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.712+0000 7fad17fc4740 1 -- 172.21.15.46:0/2806537005 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559817fbd950 con 0x559818064850 2024-09-10T07:51:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.713+0000 7fad0e800640 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559818064850 0x559818066c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.714+0000 7fad0e800640 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559818064850 0x559818066c60 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12318f451ea40d42 server_cookie=8f7a0ee66eeaa0d8 in_seq=0 out_seq=0 2024-09-10T07:51:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.714+0000 7facefe00640 10 client.29063 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.715+0000 7facefe00640 1 -- 172.21.15.46:0/2806537005 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559817fbd950 con 0x559818064850 2024-09-10T07:51:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.715+0000 7facefe00640 10 client.29063 handle_command_reply: tid=0 2024-09-10T07:51:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.716+0000 7fad17fc4740 1 client.29063 shutdown 2024-09-10T07:51:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.716+0000 7fad17fc4740 1 -- 172.21.15.46:0/2806537005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818054d70 msgr2=0x559818052700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.716+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818054d70 0x559818052700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fad080186a0 tx=0x7fad08004a80 comp rx=0 tx=0).stop 2024-09-10T07:51:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7fad17fc4740 1 -- 172.21.15.46:0/2806537005 shutdown_connections 2024-09-10T07:51:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559818064850 0x559818066c60 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559818054d70 0x559818052700 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7facefe00640 0 client.29063 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55981804a4b0 0x5598180521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2806537005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 0x559818051c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7fad17fc4740 1 -- 172.21.15.46:0/2806537005 >> 172.21.15.46:0/2806537005 conn(0x559818046120 msgr2=0x55981804be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7fad17fc4740 1 -- 172.21.15.46:0/2806537005 shutdown_connections 2024-09-10T07:51:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.717+0000 7fad17fc4740 1 -- 172.21.15.46:0/2806537005 wait complete. 2024-09-10T07:51:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.718+0000 7fad17fc4740 20 client.29063 trim_cache size 0 max 16384 2024-09-10T07:51:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.759+0000 7fad17fc4740 1 Processor -- start 2024-09-10T07:51:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.759+0000 7fad17fc4740 1 -- start start 2024-09-10T07:51:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818049b90 0x559818052550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x5598180450b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818054d70 0x55981806c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad17fc4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b53b0 con 0x559818054d70 2024-09-10T07:51:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad17fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b7010 con 0x55981804a4b0 2024-09-10T07:51:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad17fc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b5a80 con 0x559818049b90 2024-09-10T07:51:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad0e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818049b90 0x559818052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x5598180450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad0e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818049b90 0x559818052550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43946/0 (socket says 172.21.15.46:43946) 2024-09-10T07:51:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad0e800640 1 -- 172.21.15.46:0/1392627291 learned_addr learned my addr 172.21.15.46:0/1392627291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.760+0000 7fad0de00640 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x5598180450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58000/0 (socket says 172.21.15.46:58000) 2024-09-10T07:51:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.761+0000 7fad0de00640 1 -- 172.21.15.46:0/1392627291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818049b90 msgr2=0x559818052550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.761+0000 7fad0de00640 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818049b90 0x559818052550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.761+0000 7fad0de00640 1 -- 172.21.15.46:0/1392627291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818054d70 msgr2=0x55981806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.761+0000 7fad0de00640 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818054d70 0x55981806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.761+0000 7fad0de00640 1 -- 172.21.15.46:0/1392627291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598180672e0 con 0x55981804a4b0 2024-09-10T07:51:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.761+0000 7fad0de00640 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x5598180450b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x559818051510 tx=0x7facfc0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b76ad848b1b5066 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.761+0000 7fad0e800640 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818049b90 0x559818052550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.762+0000 7facfaa00640 1 -- 172.21.15.46:0/1392627291 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc005220 con 0x55981804a4b0 2024-09-10T07:51:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.762+0000 7facfaa00640 1 -- 172.21.15.46:0/1392627291 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc0116d0 con 0x55981804a4b0 2024-09-10T07:51:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.762+0000 7facfaa00640 1 -- 172.21.15.46:0/1392627291 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc004ca0 con 0x55981804a4b0 2024-09-10T07:51:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.763+0000 7fad17fc4740 1 -- 172.21.15.46:0/1392627291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 msgr2=0x5598180450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.763+0000 7fad17fc4740 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x5598180450b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x559818051510 tx=0x7facfc0039c0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.763+0000 7fad17fc4740 1 -- 172.21.15.46:0/1392627291 shutdown_connections 2024-09-10T07:51:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.763+0000 7fad17fc4740 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818054d70 0x55981806c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.763+0000 7fad17fc4740 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x5598180450b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.763+0000 7fad17fc4740 1 --2- 172.21.15.46:0/1392627291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818049b90 0x559818052550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.763+0000 7fad17fc4740 1 -- 172.21.15.46:0/1392627291 >> 172.21.15.46:0/1392627291 conn(0x559818046120 msgr2=0x5598180449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.764+0000 7fad17fc4740 1 -- 172.21.15.46:0/1392627291 shutdown_connections 2024-09-10T07:51:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.764+0000 7fad17fc4740 1 -- 172.21.15.46:0/1392627291 wait complete. 2024-09-10T07:51:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.765+0000 7fad17fc4740 1 -- start start 2024-09-10T07:51:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.765+0000 7fad17fc4740 1 Processor -- start 2024-09-10T07:51:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.766+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 0x559818058a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.766+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x559818058fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.766+0000 7fad17fc4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818054d70 0x559818059500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.766+0000 7fad17fc4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b6910 con 0x559818049b90 2024-09-10T07:51:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.766+0000 7fad17fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b6210 con 0x55981804a4b0 2024-09-10T07:51:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.766+0000 7fad17fc4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad100737b0 con 0x559818054d70 2024-09-10T07:51:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.766+0000 7fad17fc4740 10 client.0 _ll_register_callbacks cb 0x7fad0800ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.766+0000 7fad0e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 0x559818058a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.767+0000 7fad0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x559818058fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.767+0000 7fad0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x559818058fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58012/0 (socket says 172.21.15.46:58012) 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.767+0000 7fad0de00640 1 -- 172.21.15.46:0/2915419668 learned_addr learned my addr 172.21.15.46:0/2915419668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.767+0000 7fad0de00640 1 -- 172.21.15.46:0/2915419668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818054d70 msgr2=0x559818059500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.767+0000 7fad0de00640 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818054d70 0x559818059500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.767+0000 7fad0de00640 1 -- 172.21.15.46:0/2915419668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 msgr2=0x559818058a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.767+0000 7fad0de00640 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 0x559818058a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.767+0000 7fad0de00640 1 -- 172.21.15.46:0/2915419668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598180ce750 con 0x55981804a4b0 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.769+0000 7fad0de00640 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x559818058fc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7facfc004b50 tx=0x7facfc004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1436d025645bb1ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:41.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.769+0000 7facefe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.771+0000 7facefe00640 1 -- 172.21.15.46:0/2915419668 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc03c060 con 0x55981804a4b0 2024-09-10T07:51:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.771+0000 7facefe00640 1 -- 172.21.15.46:0/2915419668 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7facfc0046e0 con 0x55981804a4b0 2024-09-10T07:51:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.772+0000 7fad17fc4740 1 -- 172.21.15.46:0/2915419668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc0116d0 con 0x55981804a4b0 2024-09-10T07:51:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.773+0000 7facefe00640 1 -- 172.21.15.46:0/2915419668 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7facfc005020 con 0x55981804a4b0 2024-09-10T07:51:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.773+0000 7fad17fc4740 10 client.29069 fetch_fsmap learned FSMap version 58 2024-09-10T07:51:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.773+0000 7fad17fc4740 1 -- 172.21.15.46:0/2915419668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5598180cfa40 con 0x55981804a4b0 2024-09-10T07:51:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.773+0000 7facefe00640 1 -- 172.21.15.46:0/2915419668 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc0116d0 con 0x55981804a4b0 2024-09-10T07:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.774+0000 7facefe00640 1 -- 172.21.15.46:0/2915419668 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7facfc016ac0 con 0x55981804a4b0 2024-09-10T07:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.774+0000 7fad17fc4740 10 client.29069 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T07:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.774+0000 7fad17fc4740 10 client.29069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.774+0000 7fad17fc4740 20 client.29069 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.774+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5598180d0080 0x5598180d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.774+0000 7fad17fc4740 4 client.29069 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.774+0000 7fad17fc4740 1 -- 172.21.15.46:0/2915419668 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559817fb7e60 con 0x5598180d0080 2024-09-10T07:51:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.775+0000 7fad0e800640 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5598180d0080 0x5598180d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.776+0000 7fad0e800640 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5598180d0080 0x5598180d24a0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19baf2cc9a49724e server_cookie=b62b49c54f462ba4 in_seq=0 out_seq=0 2024-09-10T07:51:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.776+0000 7facefe00640 10 client.29069 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.777+0000 7facefe00640 1 -- 172.21.15.46:0/2915419668 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559817fb7e60 con 0x5598180d0080 2024-09-10T07:51:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.777+0000 7facefe00640 10 client.29069 handle_command_reply: tid=0 2024-09-10T07:51:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.777+0000 7fad17fc4740 1 client.29069 shutdown 2024-09-10T07:51:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.777+0000 7fad17fc4740 1 -- 172.21.15.46:0/2915419668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 msgr2=0x559818058fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.777+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x559818058fc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7facfc004b50 tx=0x7facfc004b80 comp rx=0 tx=0).stop 2024-09-10T07:51:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 1 -- 172.21.15.46:0/2915419668 shutdown_connections 2024-09-10T07:51:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5598180d0080 0x5598180d24a0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559818054d70 0x559818059500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55981804a4b0 0x559818058fc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 1 --2- 172.21.15.46:0/2915419668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559818049b90 0x559818058a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 1 -- 172.21.15.46:0/2915419668 >> 172.21.15.46:0/2915419668 conn(0x559818046120 msgr2=0x5598180449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7facefe00640 0 client.29069 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 1 -- 172.21.15.46:0/2915419668 shutdown_connections 2024-09-10T07:51:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 1 -- 172.21.15.46:0/2915419668 wait complete. 2024-09-10T07:51:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.778+0000 7fad17fc4740 20 client.29069 trim_cache size 0 max 16384 2024-09-10T07:51:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:51:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:51:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:51:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:51:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 -- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2cd0 msgr2=0x7facf40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2cd0 0x7facf40a5190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fad1006b460 tx=0x7fad0800d040 comp rx=0 tx=0).stop 2024-09-10T07:51:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 -- 172.21.15.46:0/3992525824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad100745c0 msgr2=0x7fad1019bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad100745c0 0x7fad1019bfc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad00004770 tx=0x7fad00003c10 comp rx=0 tx=0).stop 2024-09-10T07:51:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 -- 172.21.15.46:0/3992525824 shutdown_connections 2024-09-10T07:51:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7facf40a2cd0 0x7facf40a5190 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1010e420 0x7fad101964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad10074ee0 0x7fad1019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 --2- 172.21.15.46:0/3992525824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad100745c0 0x7fad1019bfc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 -- 172.21.15.46:0/3992525824 >> 172.21.15.46:0/3992525824 conn(0x7fad100fd8c0 msgr2=0x7fad100fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.783+0000 7fad15400640 1 -- 172.21.15.46:0/3992525824 shutdown_connections 2024-09-10T07:51:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:41.784+0000 7fad15400640 1 -- 172.21.15.46:0/3992525824 wait complete. 2024-09-10T07:51:42.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:42 smithi181 ceph-mon[35397]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 4.0 MiB/s wr, 460 op/s 2024-09-10T07:51:42.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:42 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:42.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:42 smithi063 ceph-mon[35020]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 4.0 MiB/s wr, 460 op/s 2024-09-10T07:51:42.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:42 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:42.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:42 smithi046 ceph-mon[30705]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 4.0 MiB/s wr, 460 op/s 2024-09-10T07:51:42.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:42 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:43.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:43 smithi181 ceph-mon[35397]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-09-10T07:51:43.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:43 smithi063 ceph-mon[35020]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-09-10T07:51:43.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:43 smithi046 ceph-mon[30705]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-09-10T07:51:45.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 865.0 s, 416.2 tps, lat 250.692 ms stddev 349.829 2024-09-10T07:51:46.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:46 smithi046 ceph-mon[30705]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 5.4 MiB/s wr, 588 op/s 2024-09-10T07:51:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:46 smithi181 ceph-mon[35397]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 5.4 MiB/s wr, 588 op/s 2024-09-10T07:51:46.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:46 smithi063 ceph-mon[35020]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 5.4 MiB/s wr, 588 op/s 2024-09-10T07:51:46.443 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:51:46.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:51:47.153 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:51:47.321 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:47 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:47.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:47 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:47.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:47 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:48.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.082+0000 7f3d3f400640 1 -- 172.21.15.46:0/3013219203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109e00 msgr2=0x7f3d3810e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.082+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3013219203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109e00 0x7f3d3810e350 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400b3b0 tx=0x7f3d2402ac20 comp rx=0 tx=0).stop 2024-09-10T07:51:48.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.083+0000 7f3d3f400640 1 -- 172.21.15.46:0/3013219203 shutdown_connections 2024-09-10T07:51:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.083+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3013219203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3810ea80 0x7f3d381154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.085+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3013219203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109e00 0x7f3d3810e350 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.085+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3013219203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d38109450 0x7f3d38109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.085+0000 7f3d3f400640 1 -- 172.21.15.46:0/3013219203 >> 172.21.15.46:0/3013219203 conn(0x7f3d3806e2d0 msgr2=0x7f3d3806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.085+0000 7f3d3f400640 1 -- 172.21.15.46:0/3013219203 shutdown_connections 2024-09-10T07:51:48.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.086+0000 7f3d3f400640 1 -- 172.21.15.46:0/3013219203 wait complete. 2024-09-10T07:51:48.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.088+0000 7f3d3f400640 1 Processor -- start 2024-09-10T07:51:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.089+0000 7f3d3f400640 1 -- start start 2024-09-10T07:51:48.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.090+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109450 0x7f3d381a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.090+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d38109e00 0x7f3d381a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109450 0x7f3d381a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3810ea80 0x7f3d381abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38118580 con 0x7f3d3810ea80 2024-09-10T07:51:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3f400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38118400 con 0x7f3d38109450 2024-09-10T07:51:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3f400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38118700 con 0x7f3d38109e00 2024-09-10T07:51:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109450 0x7f3d381a7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49548/0 (socket says 172.21.15.46:49548) 2024-09-10T07:51:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3ca00640 1 -- 172.21.15.46:0/3135885046 learned_addr learned my addr 172.21.15.46:0/3135885046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3ca00640 1 -- 172.21.15.46:0/3135885046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d38109e00 msgr2=0x7f3d381a77a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T07:51:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3ca00640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d38109e00 0x7f3d381a77a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3ca00640 1 -- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3810ea80 msgr2=0x7f3d381abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3ca00640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3810ea80 0x7f3d381abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.091+0000 7f3d3ca00640 1 -- 172.21.15.46:0/3135885046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d381ac210 con 0x7f3d38109450 2024-09-10T07:51:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.092+0000 7f3d3ca00640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109450 0x7f3d381a7260 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000efb0 tx=0x7f3d3000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.092+0000 7f3d35600640 1 -- 172.21.15.46:0/3135885046 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3001b070 con 0x7f3d38109450 2024-09-10T07:51:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.092+0000 7f3d35600640 1 -- 172.21.15.46:0/3135885046 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3000ba50 con 0x7f3d38109450 2024-09-10T07:51:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.092+0000 7f3d35600640 1 -- 172.21.15.46:0/3135885046 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30016c60 con 0x7f3d38109450 2024-09-10T07:51:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.092+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d381ac500 con 0x7f3d38109450 2024-09-10T07:51:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.092+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d381b3de0 con 0x7f3d38109450 2024-09-10T07:51:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.094+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d38109830 con 0x7f3d38109450 2024-09-10T07:51:48.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.102+0000 7f3d35600640 1 -- 172.21.15.46:0/3135885046 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d3001e070 con 0x7f3d38109450 2024-09-10T07:51:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.105+0000 7f3d35600640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d1c0a2cd0 0x7f3d1c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.106+0000 7f3d35600640 1 -- 172.21.15.46:0/3135885046 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3d300cf2f0 con 0x7f3d38109450 2024-09-10T07:51:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.106+0000 7f3d37e00640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d1c0a2cd0 0x7f3d1c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.108+0000 7f3d37e00640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d1c0a2cd0 0x7f3d1c0a5190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3d2402b140 tx=0x7f3d24000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.109+0000 7f3d35600640 1 -- 172.21.15.46:0/3135885046 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d3009bed0 con 0x7f3d38109450 2024-09-10T07:51:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:48 smithi181 ceph-mon[35397]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 5.7 MiB/s wr, 639 op/s 2024-09-10T07:51:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:48 smithi063 ceph-mon[35020]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 5.7 MiB/s wr, 639 op/s 2024-09-10T07:51:48.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:48 smithi046 ceph-mon[30705]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 5.7 MiB/s wr, 639 op/s 2024-09-10T07:51:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.489+0000 7f5152a00640 1 -- 172.21.15.46:0/2889448861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 msgr2=0x7f514c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.489+0000 7f5152a00640 1 --2- 172.21.15.46:0/2889448861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 0x7f514c101eb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5138009860 tx=0x7f513802a340 comp rx=0 tx=0).stop 2024-09-10T07:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.493+0000 7f5152a00640 1 -- 172.21.15.46:0/2889448861 shutdown_connections 2024-09-10T07:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.493+0000 7f5152a00640 1 --2- 172.21.15.46:0/2889448861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c1023f0 0x7f514c108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.493+0000 7f5152a00640 1 --2- 172.21.15.46:0/2889448861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 0x7f514c101eb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.493+0000 7f5152a00640 1 --2- 172.21.15.46:0/2889448861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514c100840 0x7f514c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.493+0000 7f5152a00640 1 -- 172.21.15.46:0/2889448861 >> 172.21.15.46:0/2889448861 conn(0x7f514c0fbfa0 msgr2=0x7f514c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.495+0000 7f5152a00640 1 -- 172.21.15.46:0/2889448861 shutdown_connections 2024-09-10T07:51:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.495+0000 7f5152a00640 1 -- 172.21.15.46:0/2889448861 wait complete. 2024-09-10T07:51:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.496+0000 7f5152a00640 1 Processor -- start 2024-09-10T07:51:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.497+0000 7f5152a00640 1 -- start start 2024-09-10T07:51:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.497+0000 7f5152a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c100840 0x7f514c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.497+0000 7f5152a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 0x7f514c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.497+0000 7f5152a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514c1023f0 0x7f514c10fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.498+0000 7f5150c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 0x7f514c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.498+0000 7f5151600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c100840 0x7f514c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.498+0000 7f5150c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 0x7f514c0711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49568/0 (socket says 172.21.15.46:49568) 2024-09-10T07:51:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.498+0000 7f5151600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c100840 0x7f514c070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56526/0 (socket says 172.21.15.46:56526) 2024-09-10T07:51:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.498+0000 7f5152000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514c1023f0 0x7f514c10fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.498+0000 7f5150c00640 1 -- 172.21.15.46:0/2776136995 learned_addr learned my addr 172.21.15.46:0/2776136995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.498+0000 7f5152a00640 1 -- 172.21.15.46:0/2776136995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f514c10b390 con 0x7f514c1023f0 2024-09-10T07:51:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f5152a00640 1 -- 172.21.15.46:0/2776136995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f514c10b210 con 0x7f514c101a30 2024-09-10T07:51:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f5152a00640 1 -- 172.21.15.46:0/2776136995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f514c10b510 con 0x7f514c100840 2024-09-10T07:51:48.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.499+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d381a85c0 con 0x7f3d38109450 2024-09-10T07:51:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f3d35600640 1 -- 172.21.15.46:0/3135885046 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11987 (secure 0 0 0) 0x7f3d3009b7d0 con 0x7f3d38109450 2024-09-10T07:51:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f5150c00640 1 -- 172.21.15.46:0/2776136995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c100840 msgr2=0x7f514c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f5150c00640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c100840 0x7f514c070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f5150c00640 1 -- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514c1023f0 msgr2=0x7f514c10fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f5150c00640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514c1023f0 0x7f514c10fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f5150c00640 1 -- 172.21.15.46:0/2776136995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c1102c0 con 0x7f514c101a30 2024-09-10T07:51:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.502+0000 7f5152000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514c1023f0 0x7f514c10fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.503+0000 7f5150c00640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 0x7f514c0711a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5138035040 tx=0x7f5138033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.503+0000 7f514a000640 1 -- 172.21.15.46:0/2776136995 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138037040 con 0x7f514c101a30 2024-09-10T07:51:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.503+0000 7f514a000640 1 -- 172.21.15.46:0/2776136995 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5138030850 con 0x7f514c101a30 2024-09-10T07:51:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.503+0000 7f514a000640 1 -- 172.21.15.46:0/2776136995 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138004920 con 0x7f514c101a30 2024-09-10T07:51:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.503+0000 7f5152a00640 1 -- 172.21.15.46:0/2776136995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c110550 con 0x7f514c101a30 2024-09-10T07:51:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.503+0000 7f5152a00640 1 -- 172.21.15.46:0/2776136995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c1109f0 con 0x7f514c101a30 2024-09-10T07:51:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:51:48.506 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"btime":"2024-09-10T07:51:46:016468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:51:45.545089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:51:48.506 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2024-09-10T07:51:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.507+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d1c0a2cd0 msgr2=0x7f3d1c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.507+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d1c0a2cd0 0x7f3d1c0a5190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3d2402b140 tx=0x7f3d24000bc0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.507+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109450 msgr2=0x7f3d381a7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.507+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109450 0x7f3d381a7260 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000efb0 tx=0x7f3d3000c800 comp rx=0 tx=0).stop 2024-09-10T07:51:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.508+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 shutdown_connections 2024-09-10T07:51:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.508+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d1c0a2cd0 0x7f3d1c0a5190 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.508+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3810ea80 0x7f3d381abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.508+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d38109e00 0x7f3d381a77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.508+0000 7f3d3f400640 1 --2- 172.21.15.46:0/3135885046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d38109450 0x7f3d381a7260 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.508+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 >> 172.21.15.46:0/3135885046 conn(0x7f3d3806e2d0 msgr2=0x7f3d3810c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.508+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 shutdown_connections 2024-09-10T07:51:48.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:48.508+0000 7f3d3f400640 1 -- 172.21.15.46:0/3135885046 wait complete. 2024-09-10T07:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.511+0000 7f514a000640 1 -- 172.21.15.46:0/2776136995 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5138004050 con 0x7f514c101a30 2024-09-10T07:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.512+0000 7f5156dd9740 1 Processor -- start 2024-09-10T07:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.512+0000 7f5156dd9740 1 -- start start 2024-09-10T07:51:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.512+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec439670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec43e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5152000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec439670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5152000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec439670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56540/0 (socket says 172.21.15.46:56540) 2024-09-10T07:51:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5152000640 1 -- 172.21.15.46:0/1152056534 learned_addr learned my addr 172.21.15.46:0/1152056534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5156dd9740 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec43ec40 0x562dec441100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5156dd9740 1 -- 172.21.15.46:0/1152056534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562dec39e100 con 0x562dec43ec40 2024-09-10T07:51:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5156dd9740 1 -- 172.21.15.46:0/1152056534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562dec3461b0 con 0x562dec439bb0 2024-09-10T07:51:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5156dd9740 1 -- 172.21.15.46:0/1152056534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562dec39a0c0 con 0x562dec439270 2024-09-10T07:51:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.513+0000 7f5151600640 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec43e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.514+0000 7f514a000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51340a2c00 0x7f51340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.514+0000 7f5151600640 1 -- 172.21.15.46:0/1152056534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 msgr2=0x562dec439670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.514+0000 7f5151600640 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec439670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.514+0000 7f5151600640 1 -- 172.21.15.46:0/1152056534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec43ec40 msgr2=0x562dec441100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.514+0000 7f5151600640 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec43ec40 0x562dec441100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.514+0000 7f5151600640 1 -- 172.21.15.46:0/1152056534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dec0b1210 con 0x562dec439bb0 2024-09-10T07:51:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.515+0000 7f5151600640 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec43e670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f514000ab20 tx=0x7f514000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7635710d844e57e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.515+0000 7f513e600640 1 -- 172.21.15.46:0/1152056534 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140004020 con 0x562dec439bb0 2024-09-10T07:51:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.515+0000 7f513e600640 1 -- 172.21.15.46:0/1152056534 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5140005500 con 0x562dec439bb0 2024-09-10T07:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.515+0000 7f513e600640 1 -- 172.21.15.46:0/1152056534 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140004dc0 con 0x562dec439bb0 2024-09-10T07:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.516+0000 7f5156dd9740 1 -- 172.21.15.46:0/1152056534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 msgr2=0x562dec43e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.516+0000 7f5156dd9740 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec43e670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f514000ab20 tx=0x7f514000f3c0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.518+0000 7f514a000640 1 -- 172.21.15.46:0/2776136995 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f51380ef360 con 0x7f514c101a30 2024-09-10T07:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.519+0000 7f5156dd9740 1 -- 172.21.15.46:0/1152056534 shutdown_connections 2024-09-10T07:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.519+0000 7f5156dd9740 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec43ec40 0x562dec441100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.519+0000 7f5156dd9740 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec43e670 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.519+0000 7f5156dd9740 1 --2- 172.21.15.46:0/1152056534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec439670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.519+0000 7f5156dd9740 1 -- 172.21.15.46:0/1152056534 >> 172.21.15.46:0/1152056534 conn(0x562dec4348e0 msgr2=0x562dec434ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.519+0000 7f5156dd9740 1 -- 172.21.15.46:0/1152056534 shutdown_connections 2024-09-10T07:51:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.519+0000 7f5156dd9740 1 -- 172.21.15.46:0/1152056534 wait complete. 2024-09-10T07:51:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.520+0000 7f5156dd9740 1 -- start start 2024-09-10T07:51:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.520+0000 7f5156dd9740 1 Processor -- start 2024-09-10T07:51:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.520+0000 7f5152000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51340a2c00 0x7f51340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.521+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 0x562dec443480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.522+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec43ec40 0x562dec44aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5150c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5156dd9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562dec441c10 con 0x562dec439270 2024-09-10T07:51:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5156dd9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562dec327090 con 0x562dec43ec40 2024-09-10T07:51:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5156dd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562dec441d90 con 0x562dec439bb0 2024-09-10T07:51:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5152000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec43ec40 0x562dec44aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5152000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec43ec40 0x562dec44aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49584/0 (socket says 172.21.15.46:49584) 2024-09-10T07:51:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5152000640 1 -- 172.21.15.46:0/533230309 learned_addr learned my addr 172.21.15.46:0/533230309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.523+0000 7f5156dd9740 10 client.0 _ll_register_callbacks cb 0x562dec37bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.524+0000 7f5152000640 1 -- 172.21.15.46:0/533230309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 msgr2=0x562dec4439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.524+0000 7f5152000640 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.524+0000 7f5152000640 1 -- 172.21.15.46:0/533230309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 msgr2=0x562dec443480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.524+0000 7f5152000640 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 0x562dec443480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.524+0000 7f5152000640 1 -- 172.21.15.46:0/533230309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dec17d620 con 0x562dec43ec40 2024-09-10T07:51:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.524+0000 7f5150c00640 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.524+0000 7f5152000640 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec43ec40 0x562dec44aa00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f514400d820 tx=0x7f514400dce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd7bca01be5c8db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.524+0000 7f5132a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.525+0000 7f5132a00640 1 -- 172.21.15.46:0/533230309 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144023070 con 0x562dec43ec40 2024-09-10T07:51:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.525+0000 7f5132a00640 1 -- 172.21.15.46:0/533230309 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f514400bdf0 con 0x562dec43ec40 2024-09-10T07:51:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.525+0000 7f5156dd9740 1 -- 172.21.15.46:0/533230309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5140005500 con 0x562dec43ec40 2024-09-10T07:51:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.526+0000 7f5152000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51340a2c00 0x7f51340a50c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5144025000 tx=0x7f5144022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.529+0000 7f5132a00640 1 -- 172.21.15.46:0/533230309 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f514402d020 con 0x562dec43ec40 2024-09-10T07:51:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.529+0000 7f5156dd9740 10 client.29087 fetch_fsmap learned FSMap version 59 2024-09-10T07:51:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.529+0000 7f5156dd9740 1 -- 172.21.15.46:0/533230309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x562dec44d8c0 con 0x562dec43ec40 2024-09-10T07:51:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.532+0000 7f5132a00640 1 -- 172.21.15.46:0/533230309 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5140005500 con 0x562dec43ec40 2024-09-10T07:51:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.532+0000 7f5132a00640 1 -- 172.21.15.46:0/533230309 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 59) ==== 8406+0+0 (secure 0 0 0) 0x7f5144031070 con 0x562dec43ec40 2024-09-10T07:51:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.533+0000 7f5156dd9740 10 client.29087 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:51:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.533+0000 7f5156dd9740 10 client.29087 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.533+0000 7f5156dd9740 20 client.29087 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.533+0000 7f5156dd9740 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562dec44f110 0x562dec4515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.533+0000 7f5156dd9740 4 client.29087 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.533+0000 7f5156dd9740 1 -- 172.21.15.46:0/533230309 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dec3ab830 con 0x562dec44f110 2024-09-10T07:51:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.533+0000 7f5151600640 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562dec44f110 0x562dec4515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.535+0000 7f5151600640 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562dec44f110 0x562dec4515d0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de002817c0935325 server_cookie=d8f54d013d5375f8 in_seq=0 out_seq=0 2024-09-10T07:51:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.535+0000 7f5132a00640 10 client.29087 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.536+0000 7f5132a00640 1 -- 172.21.15.46:0/533230309 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562dec3ab830 con 0x562dec44f110 2024-09-10T07:51:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.536+0000 7f5132a00640 10 client.29087 handle_command_reply: tid=0 2024-09-10T07:51:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.539+0000 7f5156dd9740 1 client.29087 shutdown 2024-09-10T07:51:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5156dd9740 1 -- 172.21.15.46:0/533230309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec43ec40 msgr2=0x562dec44aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5156dd9740 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec43ec40 0x562dec44aa00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f514400d820 tx=0x7f514400dce0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5156dd9740 1 -- 172.21.15.46:0/533230309 shutdown_connections 2024-09-10T07:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5156dd9740 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562dec44f110 0x562dec4515d0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5156dd9740 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec43ec40 0x562dec44aa00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5156dd9740 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5156dd9740 1 --2- 172.21.15.46:0/533230309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 0x562dec443480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5156dd9740 1 -- 172.21.15.46:0/533230309 >> 172.21.15.46:0/533230309 conn(0x562dec4348e0 msgr2=0x562dec4360f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.540+0000 7f5132a00640 0 client.29087 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.541+0000 7f5156dd9740 1 -- 172.21.15.46:0/533230309 shutdown_connections 2024-09-10T07:51:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.541+0000 7f5156dd9740 1 -- 172.21.15.46:0/533230309 wait complete. 2024-09-10T07:51:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.541+0000 7f5156dd9740 20 client.29087 trim_cache size 0 max 16384 2024-09-10T07:51:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.596+0000 7f5156dd9740 1 Processor -- start 2024-09-10T07:51:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.596+0000 7f5156dd9740 1 -- start start 2024-09-10T07:51:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.598+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec4348e0 0x562dec435790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.598+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec44a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.598+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec435cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.598+0000 7f5156dd9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f514c1c1c60 con 0x562dec4348e0 2024-09-10T07:51:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.598+0000 7f5156dd9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f514c1c3950 con 0x562dec439bb0 2024-09-10T07:51:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.598+0000 7f5156dd9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f514c1c2330 con 0x562dec439270 2024-09-10T07:51:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.598+0000 7f5152000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec435cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.598+0000 7f5152000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec435cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49596/0 (socket says 172.21.15.46:49596) 2024-09-10T07:51:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.599+0000 7f5150c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec44a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.599+0000 7f5152000640 1 -- 172.21.15.46:0/4037374280 learned_addr learned my addr 172.21.15.46:0/4037374280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.599+0000 7f5151600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec4348e0 0x562dec435790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.599+0000 7f5152000640 1 -- 172.21.15.46:0/4037374280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 msgr2=0x562dec44a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.599+0000 7f5152000640 1 --2- 172.21.15.46:0/4037374280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec44a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.599+0000 7f5152000640 1 -- 172.21.15.46:0/4037374280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec4348e0 msgr2=0x562dec435790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.599+0000 7f5152000640 1 --2- 172.21.15.46:0/4037374280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec4348e0 0x562dec435790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.599+0000 7f5152000640 1 -- 172.21.15.46:0/4037374280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dec434700 con 0x562dec439bb0 2024-09-10T07:51:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.600+0000 7f5152000640 1 --2- 172.21.15.46:0/4037374280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec435cd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5144001220 tx=0x7f514400d3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e80c639f6d3494 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.601+0000 7f513dc00640 1 -- 172.21.15.46:0/4037374280 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144037070 con 0x562dec439bb0 2024-09-10T07:51:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.601+0000 7f513dc00640 1 -- 172.21.15.46:0/4037374280 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5144024020 con 0x562dec439bb0 2024-09-10T07:51:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.601+0000 7f513dc00640 1 -- 172.21.15.46:0/4037374280 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144023040 con 0x562dec439bb0 2024-09-10T07:51:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.602+0000 7f5156dd9740 1 -- 172.21.15.46:0/4037374280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 msgr2=0x562dec435cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.602+0000 7f5156dd9740 1 --2- 172.21.15.46:0/4037374280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec435cd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5144001220 tx=0x7f514400d3e0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.602+0000 7f5156dd9740 1 -- 172.21.15.46:0/4037374280 shutdown_connections 2024-09-10T07:51:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.602+0000 7f5156dd9740 1 --2- 172.21.15.46:0/4037374280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec439bb0 0x562dec435cd0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.602+0000 7f5156dd9740 1 --2- 172.21.15.46:0/4037374280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439270 0x562dec44a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.602+0000 7f5156dd9740 1 --2- 172.21.15.46:0/4037374280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec4348e0 0x562dec435790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.602+0000 7f5156dd9740 1 -- 172.21.15.46:0/4037374280 >> 172.21.15.46:0/4037374280 conn(0x562dec43e7d0 msgr2=0x562dec443480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.603+0000 7f5156dd9740 1 -- 172.21.15.46:0/4037374280 shutdown_connections 2024-09-10T07:51:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.603+0000 7f5156dd9740 1 -- 172.21.15.46:0/4037374280 wait complete. 2024-09-10T07:51:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.604+0000 7f5156dd9740 1 -- start start 2024-09-10T07:51:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.605+0000 7f5156dd9740 1 Processor -- start 2024-09-10T07:51:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.606+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec4348e0 0x562dec448350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.606+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 0x562dec448890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.606+0000 7f5156dd9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5152000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5152000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56556/0 (socket says 172.21.15.46:56556) 2024-09-10T07:51:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5152000640 1 -- 172.21.15.46:0/2994718198 learned_addr learned my addr 172.21.15.46:0/2994718198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f514c1c31c0 con 0x562dec439270 2024-09-10T07:51:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5150c00640 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 0x562dec448890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5151600640 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec4348e0 0x562dec448350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f514c1c2ac0 con 0x562dec4348e0 2024-09-10T07:51:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f514c118f10 con 0x562dec439bb0 2024-09-10T07:51:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5156dd9740 10 client.0 _ll_register_callbacks cb 0x7f514400f460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5152000640 1 -- 172.21.15.46:0/2994718198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec4348e0 msgr2=0x562dec448350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5152000640 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec4348e0 0x562dec448350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5152000640 1 -- 172.21.15.46:0/2994718198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 msgr2=0x562dec448890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5152000640 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 0x562dec448890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.607+0000 7f5152000640 1 -- 172.21.15.46:0/2994718198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dec449eb0 con 0x562dec439bb0 2024-09-10T07:51:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.608+0000 7f5152000640 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4498a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5144026040 tx=0x7f5144009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=682a5d97ca254762 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.608+0000 7f5132a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:51:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.609+0000 7f5132a00640 1 -- 172.21.15.46:0/2994718198 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144023050 con 0x562dec439bb0 2024-09-10T07:51:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.609+0000 7f5132a00640 1 -- 172.21.15.46:0/2994718198 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5144004be0 con 0x562dec439bb0 2024-09-10T07:51:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.609+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5144024020 con 0x562dec439bb0 2024-09-10T07:51:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.609+0000 7f5132a00640 1 -- 172.21.15.46:0/2994718198 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f5144037040 con 0x562dec439bb0 2024-09-10T07:51:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.611+0000 7f5156dd9740 10 client.28966 fetch_fsmap learned FSMap version 59 2024-09-10T07:51:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.611+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x562dec4ba150 con 0x562dec439bb0 2024-09-10T07:51:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.611+0000 7f5132a00640 1 -- 172.21.15.46:0/2994718198 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5144024020 con 0x562dec439bb0 2024-09-10T07:51:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.614+0000 7f5132a00640 1 -- 172.21.15.46:0/2994718198 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 59) ==== 8406+0+0 (secure 0 0 0) 0x7f5144031070 con 0x562dec439bb0 2024-09-10T07:51:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.614+0000 7f5156dd9740 10 client.28966 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:51:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.614+0000 7f5156dd9740 10 client.28966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:51:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.614+0000 7f5156dd9740 20 client.28966 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.614+0000 7f5156dd9740 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562dec43ec40 0x562dec4bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.614+0000 7f5156dd9740 4 client.28966 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:51:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.614+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562dec3a5b80 con 0x562dec43ec40 2024-09-10T07:51:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.615+0000 7f5151600640 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562dec43ec40 0x562dec4bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.618+0000 7f5132a00640 10 client.28966 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.618+0000 7f5151600640 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562dec43ec40 0x562dec4bcae0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8a4940c3763130 server_cookie=f275d8a463f097e4 in_seq=0 out_seq=0 2024-09-10T07:51:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.666+0000 7f5132a00640 1 -- 172.21.15.46:0/2994718198 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562dec3a5b80 con 0x562dec43ec40 2024-09-10T07:51:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.666+0000 7f5132a00640 10 client.28966 handle_command_reply: tid=0 2024-09-10T07:51:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.666+0000 7f5156dd9740 1 client.28966 shutdown 2024-09-10T07:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 msgr2=0x562dec4498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5156dd9740 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4498a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5144026040 tx=0x7f5144009fd0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 shutdown_connections 2024-09-10T07:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5156dd9740 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x562dec43ec40 0x562dec4bcae0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5156dd9740 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562dec439bb0 0x562dec4498a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5156dd9740 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dec439270 0x562dec448890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5156dd9740 1 --2- 172.21.15.46:0/2994718198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562dec4348e0 0x562dec448350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 >> 172.21.15.46:0/2994718198 conn(0x562dec43e7d0 msgr2=0x562dec433290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.667+0000 7f5132a00640 0 client.28966 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:51:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.668+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 shutdown_connections 2024-09-10T07:51:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.668+0000 7f5156dd9740 1 -- 172.21.15.46:0/2994718198 wait complete. 2024-09-10T07:51:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.669+0000 7f5156dd9740 20 client.28966 trim_cache size 0 max 16384 2024-09-10T07:51:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:51:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:51:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:51:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:51:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 -- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51340a2c00 msgr2=0x7f51340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51340a2c00 0x7f51340a50c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5144025000 tx=0x7f5144022000 comp rx=0 tx=0).stop 2024-09-10T07:51:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 -- 172.21.15.46:0/2776136995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 msgr2=0x7f514c0711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 0x7f514c0711a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5138035040 tx=0x7f5138033d90 comp rx=0 tx=0).stop 2024-09-10T07:51:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 -- 172.21.15.46:0/2776136995 shutdown_connections 2024-09-10T07:51:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f51340a2c00 0x7f51340a50c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514c1023f0 0x7f514c10fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f514c101a30 0x7f514c0711a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.670+0000 7f513f000640 1 --2- 172.21.15.46:0/2776136995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f514c100840 0x7f514c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.671+0000 7f513f000640 1 -- 172.21.15.46:0/2776136995 >> 172.21.15.46:0/2776136995 conn(0x7f514c0fbfa0 msgr2=0x7f514c104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.673+0000 7f513f000640 1 -- 172.21.15.46:0/2776136995 shutdown_connections 2024-09-10T07:51:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:51:48.674+0000 7f513f000640 1 -- 172.21.15.46:0/2776136995 wait complete. 2024-09-10T07:51:49.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:49 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3135885046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:51:49.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:49 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3135885046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:51:49.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:49 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3135885046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:51:49.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:51:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:51:50.320 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:50 smithi046 ceph-mon[30705]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-09-10T07:51:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:50 smithi181 ceph-mon[35397]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-09-10T07:51:50.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:50 smithi063 ceph-mon[35020]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-09-10T07:51:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 870.0 s, 461.4 tps, lat 218.826 ms stddev 240.121 2024-09-10T07:51:51.888 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T07:51:51.889 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-09-10T07:51:51.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:51:52.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:52 smithi046 ceph-mon[30705]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-09-10T07:51:52.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:52 smithi181 ceph-mon[35397]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-09-10T07:51:52.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:52 smithi063 ceph-mon[35020]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-09-10T07:51:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.458+0000 7ff368a00640 1 -- 172.21.15.46:0/2732553809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3641091f0 msgr2=0x7ff3641095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.458+0000 7ff368a00640 1 --2- 172.21.15.46:0/2732553809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3641091f0 0x7ff3641095d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff354009900 tx=0x7ff35402a4a0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.459+0000 7ff368a00640 1 -- 172.21.15.46:0/2732553809 shutdown_connections 2024-09-10T07:51:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.459+0000 7ff368a00640 1 --2- 172.21.15.46:0/2732553809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff364114ab0 0x7ff364116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.459+0000 7ff368a00640 1 --2- 172.21.15.46:0/2732553809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff364109b10 0x7ff364114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.459+0000 7ff368a00640 1 --2- 172.21.15.46:0/2732553809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3641091f0 0x7ff3641095d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.459+0000 7ff368a00640 1 -- 172.21.15.46:0/2732553809 >> 172.21.15.46:0/2732553809 conn(0x7ff36406e140 msgr2=0x7ff36406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.461+0000 7ff368a00640 1 -- 172.21.15.46:0/2732553809 shutdown_connections 2024-09-10T07:51:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.462+0000 7ff368a00640 1 -- 172.21.15.46:0/2732553809 wait complete. 2024-09-10T07:51:53.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.463+0000 7ff368a00640 1 Processor -- start 2024-09-10T07:51:53.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.464+0000 7ff368a00640 1 -- start start 2024-09-10T07:51:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.465+0000 7ff368a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3641091f0 0x7ff3641a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.465+0000 7ff368a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364109b10 0x7ff3641a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.465+0000 7ff368a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff364114ab0 0x7ff3641a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.465+0000 7ff368a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff36411a850 con 0x7ff3641091f0 2024-09-10T07:51:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.465+0000 7ff368a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff36411a6d0 con 0x7ff364114ab0 2024-09-10T07:51:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.465+0000 7ff368a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff36411a9d0 con 0x7ff364109b10 2024-09-10T07:51:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.466+0000 7ff361400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364109b10 0x7ff3641a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.466+0000 7ff361400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364109b10 0x7ff3641a34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56584/0 (socket says 172.21.15.46:56584) 2024-09-10T07:51:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.466+0000 7ff361400640 1 -- 172.21.15.46:0/1521829469 learned_addr learned my addr 172.21.15.46:0/1521829469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.466+0000 7ff362800640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff364114ab0 0x7ff3641a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.467+0000 7ff361400640 1 -- 172.21.15.46:0/1521829469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff364114ab0 msgr2=0x7ff3641a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.467+0000 7ff361400640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff364114ab0 0x7ff3641a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.467+0000 7ff361400640 1 -- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3641091f0 msgr2=0x7ff3641a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.467+0000 7ff361400640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3641091f0 0x7ff3641a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.467+0000 7ff361400640 1 -- 172.21.15.46:0/1521829469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3641a7f30 con 0x7ff364109b10 2024-09-10T07:51:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.467+0000 7ff361400640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364109b10 0x7ff3641a34c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff350009f20 tx=0x7ff35000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.468+0000 7ff35aa00640 1 -- 172.21.15.46:0/1521829469 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350024070 con 0x7ff364109b10 2024-09-10T07:51:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.468+0000 7ff35aa00640 1 -- 172.21.15.46:0/1521829469 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35001f440 con 0x7ff364109b10 2024-09-10T07:51:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.468+0000 7ff35aa00640 1 -- 172.21.15.46:0/1521829469 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35001d5a0 con 0x7ff364109b10 2024-09-10T07:51:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.468+0000 7ff368a00640 1 -- 172.21.15.46:0/1521829469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3641a8220 con 0x7ff364109b10 2024-09-10T07:51:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.469+0000 7ff368a00640 1 -- 172.21.15.46:0/1521829469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff36410c510 con 0x7ff364109b10 2024-09-10T07:51:53.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.474+0000 7ff36b422740 1 Processor -- start 2024-09-10T07:51:53.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.476+0000 7ff35aa00640 1 -- 172.21.15.46:0/1521829469 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff350027070 con 0x7ff364109b10 2024-09-10T07:51:53.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.476+0000 7ff36b422740 1 -- start start 2024-09-10T07:51:53.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.476+0000 7ff36b422740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bec70 0x55f7849bf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.476+0000 7ff36b422740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bf590 0x55f7849c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.476+0000 7ff36b422740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f7849cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.476+0000 7ff36b422740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f7849253e0 con 0x55f7849bec70 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff36b422740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f7848ccd80 con 0x55f7849bf590 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff36b422740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f7849213a0 con 0x55f7849c9e50 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bf590 0x55f7849c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bf590 0x55f7849c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49620/0 (socket says 172.21.15.46:49620) 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361e00640 1 -- 172.21.15.46:0/3433117637 learned_addr learned my addr 172.21.15.46:0/3433117637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361400640 1 --2- 172.21.15.46:0/3433117637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f7849cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361e00640 1 -- 172.21.15.46:0/3433117637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 msgr2=0x55f7849cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361e00640 1 --2- 172.21.15.46:0/3433117637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f7849cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361e00640 1 -- 172.21.15.46:0/3433117637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bec70 msgr2=0x55f7849bf050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361e00640 1 --2- 172.21.15.46:0/3433117637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bec70 0x55f7849bf050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.477+0000 7ff361e00640 1 -- 172.21.15.46:0/3433117637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7848ebae0 con 0x55f7849bf590 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.478+0000 7ff361e00640 1 --2- 172.21.15.46:0/3433117637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bf590 0x55f7849c9910 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff354009230 tx=0x7ff354005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=288d4c1d86cbc5ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.478+0000 7ff35aa00640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3480a2c40 0x7ff3480a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.478+0000 7ff34f000640 1 -- 172.21.15.46:0/3433117637 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354060030 con 0x55f7849bf590 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.478+0000 7ff35aa00640 1 -- 172.21.15.46:0/1521829469 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff3500d7930 con 0x7ff364109b10 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.478+0000 7ff34f000640 1 -- 172.21.15.46:0/3433117637 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3540035f0 con 0x55f7849bf590 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.479+0000 7ff36b422740 1 -- 172.21.15.46:0/3433117637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bf590 msgr2=0x55f7849c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.479+0000 7ff36b422740 1 --2- 172.21.15.46:0/3433117637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bf590 0x55f7849c9910 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff354009230 tx=0x7ff354005760 comp rx=0 tx=0).stop 2024-09-10T07:51:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.481+0000 7ff36b422740 1 -- 172.21.15.46:0/3433117637 shutdown_connections 2024-09-10T07:51:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.482+0000 7ff36b422740 1 --2- 172.21.15.46:0/3433117637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f7849cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.482+0000 7ff36b422740 1 --2- 172.21.15.46:0/3433117637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bf590 0x55f7849c9910 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.482+0000 7ff36b422740 1 --2- 172.21.15.46:0/3433117637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bec70 0x55f7849bf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.482+0000 7ff36b422740 1 -- 172.21.15.46:0/3433117637 >> 172.21.15.46:0/3433117637 conn(0x55f7849bb200 msgr2=0x55f7849bb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.482+0000 7ff36b422740 1 -- 172.21.15.46:0/3433117637 shutdown_connections 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.482+0000 7ff36b422740 1 -- 172.21.15.46:0/3433117637 wait complete. 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.483+0000 7ff36b422740 1 -- start start 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.483+0000 7ff362800640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3480a2c40 0x7ff3480a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.483+0000 7ff36b422740 1 Processor -- start 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff36b422740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff36b422740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bf590 0x55f7849c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff36b422740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849c9e50 0x55f7849c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff36b422740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f7849cca10 con 0x55f7849c9e50 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff36b422740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f7848e00f0 con 0x55f7849bec70 2024-09-10T07:51:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff36b422740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f7849ccb90 con 0x55f7849bf590 2024-09-10T07:51:53.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff36b422740 10 client.0 _ll_register_callbacks cb 0x55f7848c7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff361400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bf590 0x55f7849c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff361400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bf590 0x55f7849c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56604/0 (socket says 172.21.15.46:56604) 2024-09-10T07:51:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff361400640 1 -- 172.21.15.46:0/1418115783 learned_addr learned my addr 172.21.15.46:0/1418115783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.484+0000 7ff361e00640 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff362800640 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849c9e50 0x55f7849c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff361400640 1 -- 172.21.15.46:0/1418115783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 msgr2=0x55f7849c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff361400640 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff361400640 1 -- 172.21.15.46:0/1418115783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849c9e50 msgr2=0x55f7849c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff361400640 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849c9e50 0x55f7849c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff361400640 1 -- 172.21.15.46:0/1418115783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7846963c0 con 0x55f7849bf590 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff361400640 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bf590 0x55f7849c7150 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff35000c6a0 tx=0x7ff350017f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454ce91ee3874bff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff343400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff343400640 1 -- 172.21.15.46:0/1418115783 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3500a3950 con 0x55f7849bf590 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.485+0000 7ff343400640 1 -- 172.21.15.46:0/1418115783 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff3500a0480 con 0x55f7849bf590 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.486+0000 7ff36b422740 1 -- 172.21.15.46:0/1418115783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3540035f0 con 0x55f7849bf590 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.486+0000 7ff343400640 1 -- 172.21.15.46:0/1418115783 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff3500a3770 con 0x55f7849bf590 2024-09-10T07:51:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.486+0000 7ff36b422740 10 client.28978 fetch_fsmap learned FSMap version 59 2024-09-10T07:51:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.486+0000 7ff36b422740 1 -- 172.21.15.46:0/1418115783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f7849d89c0 con 0x55f7849bf590 2024-09-10T07:51:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.487+0000 7ff343400640 1 -- 172.21.15.46:0/1418115783 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3540035f0 con 0x55f7849bf590 2024-09-10T07:51:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.488+0000 7ff343400640 1 -- 172.21.15.46:0/1418115783 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 59) ==== 8406+0+0 (secure 0 0 0) 0x7ff3500111a0 con 0x55f7849bf590 2024-09-10T07:51:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.488+0000 7ff36b422740 10 client.28978 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:51:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.488+0000 7ff36b422740 10 client.28978 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:51:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.488+0000 7ff36b422740 20 client.28978 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.488+0000 7ff36b422740 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55f7849da330 0x55f7849dc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.488+0000 7ff36b422740 4 client.28978 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.488+0000 7ff36b422740 1 -- 172.21.15.46:0/1418115783 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f784932950 con 0x55f7849da330 2024-09-10T07:51:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.489+0000 7ff362800640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3480a2c40 0x7ff3480a5100 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff36406b430 tx=0x7ff35c00c7f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.490+0000 7ff361e00640 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55f7849da330 0x55f7849dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.491+0000 7ff343400640 10 client.28978 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:51:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.491+0000 7ff361e00640 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55f7849da330 0x55f7849dc770 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65ae97d93c2da045 server_cookie=5874015cee3c5d36 in_seq=0 out_seq=0 2024-09-10T07:51:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.492+0000 7ff343400640 1 -- 172.21.15.46:0/1418115783 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f784932950 con 0x55f7849da330 2024-09-10T07:51:53.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.492+0000 7ff343400640 10 client.28978 handle_command_reply: tid=0 2024-09-10T07:51:53.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.493+0000 7ff36b422740 1 client.28978 shutdown 2024-09-10T07:51:53.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.493+0000 7ff36b422740 1 -- 172.21.15.46:0/1418115783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bf590 msgr2=0x55f7849c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.493+0000 7ff36b422740 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bf590 0x55f7849c7150 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff35000c6a0 tx=0x7ff350017f90 comp rx=0 tx=0).stop 2024-09-10T07:51:53.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.493+0000 7ff36b422740 1 -- 172.21.15.46:0/1418115783 shutdown_connections 2024-09-10T07:51:53.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.494+0000 7ff36b422740 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55f7849da330 0x55f7849dc770 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.494+0000 7ff36b422740 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849c9e50 0x55f7849c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.494+0000 7ff36b422740 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bf590 0x55f7849c7150 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.494+0000 7ff36b422740 1 --2- 172.21.15.46:0/1418115783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.494+0000 7ff343400640 0 client.28978 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:51:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.494+0000 7ff36b422740 1 -- 172.21.15.46:0/1418115783 >> 172.21.15.46:0/1418115783 conn(0x55f7849bb200 msgr2=0x55f7849c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.494+0000 7ff36b422740 1 -- 172.21.15.46:0/1418115783 shutdown_connections 2024-09-10T07:51:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.495+0000 7ff36b422740 1 -- 172.21.15.46:0/1418115783 wait complete. 2024-09-10T07:51:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.495+0000 7ff36b422740 20 client.28978 trim_cache size 0 max 16384 2024-09-10T07:51:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.523+0000 7ff36b422740 1 Processor -- start 2024-09-10T07:51:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.524+0000 7ff36b422740 1 -- start start 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.525+0000 7ff36b422740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bec70 0x55f7849c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.525+0000 7ff36b422740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.526+0000 7ff361e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bec70 0x55f7849c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.526+0000 7ff361e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bec70 0x55f7849c0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:56610/0 (socket says 172.21.15.46:56610) 2024-09-10T07:51:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.526+0000 7ff361e00640 1 -- 172.21.15.46:0/645764739 learned_addr learned my addr 172.21.15.46:0/645764739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.526+0000 7ff36b422740 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849c9e50 0x55f7849e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.527+0000 7ff362800640 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849c9e50 0x55f7849e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.527+0000 7ff36b422740 1 -- 172.21.15.46:0/645764739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641b5400 con 0x55f7849bf590 2024-09-10T07:51:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.527+0000 7ff36b422740 1 -- 172.21.15.46:0/645764739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641b70f0 con 0x55f7849c9e50 2024-09-10T07:51:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.528+0000 7ff36b422740 1 -- 172.21.15.46:0/645764739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641b5ad0 con 0x55f7849bec70 2024-09-10T07:51:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.528+0000 7ff361400640 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.529+0000 7ff362800640 1 -- 172.21.15.46:0/645764739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bec70 msgr2=0x55f7849c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.529+0000 7ff362800640 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bec70 0x55f7849c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.529+0000 7ff362800640 1 -- 172.21.15.46:0/645764739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 msgr2=0x55f7849e2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.529+0000 7ff362800640 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849e2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.529+0000 7ff362800640 1 -- 172.21.15.46:0/645764739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7849bb030 con 0x55f7849c9e50 2024-09-10T07:51:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.529+0000 7ff361e00640 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bec70 0x55f7849c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.529+0000 7ff361400640 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.530+0000 7ff362800640 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849c9e50 0x55f7849e2d60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff3640743f0 tx=0x7ff35c009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64343217acba3052 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.530+0000 7ff34e600640 1 -- 172.21.15.46:0/645764739 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c019410 con 0x55f7849c9e50 2024-09-10T07:51:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.530+0000 7ff34e600640 1 -- 172.21.15.46:0/645764739 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c019a90 con 0x55f7849c9e50 2024-09-10T07:51:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.530+0000 7ff34e600640 1 -- 172.21.15.46:0/645764739 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c025ac0 con 0x55f7849c9e50 2024-09-10T07:51:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.532+0000 7ff36b422740 1 -- 172.21.15.46:0/645764739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849c9e50 msgr2=0x55f7849e2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.532+0000 7ff36b422740 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849c9e50 0x55f7849e2d60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff3640743f0 tx=0x7ff35c009430 comp rx=0 tx=0).stop 2024-09-10T07:51:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.533+0000 7ff36b422740 1 -- 172.21.15.46:0/645764739 shutdown_connections 2024-09-10T07:51:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.533+0000 7ff36b422740 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849c9e50 0x55f7849e2d60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.533+0000 7ff36b422740 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.533+0000 7ff36b422740 1 --2- 172.21.15.46:0/645764739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849bec70 0x55f7849c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.533+0000 7ff36b422740 1 -- 172.21.15.46:0/645764739 >> 172.21.15.46:0/645764739 conn(0x55f7849bb200 msgr2=0x55f7849c06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.533+0000 7ff36b422740 1 -- 172.21.15.46:0/645764739 shutdown_connections 2024-09-10T07:51:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.533+0000 7ff36b422740 1 -- 172.21.15.46:0/645764739 wait complete. 2024-09-10T07:51:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.534+0000 7ff36b422740 1 -- start start 2024-09-10T07:51:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.534+0000 7ff36b422740 1 Processor -- start 2024-09-10T07:51:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.535+0000 7ff36b422740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.536+0000 7ff36b422740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.536+0000 7ff36b422740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f784a437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.536+0000 7ff36b422740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641b6960 con 0x55f7849bf590 2024-09-10T07:51:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.536+0000 7ff36b422740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641b6260 con 0x55f7849bec70 2024-09-10T07:51:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.536+0000 7ff36b422740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641b33e0 con 0x55f7849c9e50 2024-09-10T07:51:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.536+0000 7ff36b422740 10 client.0 _ll_register_callbacks cb 0x7ff350012820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.537+0000 7ff361e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.537+0000 7ff361400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.537+0000 7ff361e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49654/0 (socket says 172.21.15.46:49654) 2024-09-10T07:51:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.537+0000 7ff361e00640 1 -- 172.21.15.46:0/2401387769 learned_addr learned my addr 172.21.15.46:0/2401387769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.538+0000 7ff362800640 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f784a437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.538+0000 7ff361e00640 1 -- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 msgr2=0x55f784a437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.538+0000 7ff361e00640 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f784a437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.538+0000 7ff361e00640 1 -- 172.21.15.46:0/2401387769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 msgr2=0x55f7849cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.539+0000 7ff361e00640 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.539+0000 7ff361e00640 1 -- 172.21.15.46:0/2401387769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f784a43d30 con 0x55f7849bec70 2024-09-10T07:51:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.539+0000 7ff361400640 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.539+0000 7ff362800640 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f784a437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.540+0000 7ff361e00640 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849cf3a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff35402aa90 tx=0x7ff354009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13047b00ca40bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.540+0000 7ff343400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:51:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.540+0000 7ff343400640 1 -- 172.21.15.46:0/2401387769 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354060040 con 0x55f7849bec70 2024-09-10T07:51:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.540+0000 7ff343400640 1 -- 172.21.15.46:0/2401387769 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff3540040c0 con 0x55f7849bec70 2024-09-10T07:51:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.541+0000 7ff36b422740 1 -- 172.21.15.46:0/2401387769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff35c019a90 con 0x55f7849bec70 2024-09-10T07:51:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.541+0000 7ff343400640 1 -- 172.21.15.46:0/2401387769 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff354005850 con 0x55f7849bec70 2024-09-10T07:51:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.541+0000 7ff36b422740 10 client.29108 fetch_fsmap learned FSMap version 59 2024-09-10T07:51:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.541+0000 7ff36b422740 1 -- 172.21.15.46:0/2401387769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f784a44f40 con 0x55f7849bec70 2024-09-10T07:51:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.542+0000 7ff343400640 1 -- 172.21.15.46:0/2401387769 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c019a90 con 0x55f7849bec70 2024-09-10T07:51:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.542+0000 7ff343400640 1 -- 172.21.15.46:0/2401387769 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 59) ==== 8406+0+0 (secure 0 0 0) 0x7ff350012ce0 con 0x55f7849bec70 2024-09-10T07:51:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.542+0000 7ff36b422740 10 client.29108 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:51:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.542+0000 7ff36b422740 10 client.29108 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:51:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.542+0000 7ff36b422740 20 client.29108 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.542+0000 7ff36b422740 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55f784a45220 0x55f784a47610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.542+0000 7ff36b422740 4 client.29108 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:51:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.542+0000 7ff36b422740 1 -- 172.21.15.46:0/2401387769 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f78492ce60 con 0x55f784a45220 2024-09-10T07:51:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.543+0000 7ff361400640 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55f784a45220 0x55f784a47610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.544+0000 7ff361400640 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55f784a45220 0x55f784a47610 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b56c12e9beedfced server_cookie=c8ffebd40fab05f2 in_seq=0 out_seq=0 2024-09-10T07:51:53.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.544+0000 7ff343400640 10 client.29108 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.544+0000 7ff343400640 1 -- 172.21.15.46:0/2401387769 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f78492ce60 con 0x55f784a45220 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.544+0000 7ff343400640 10 client.29108 handle_command_reply: tid=0 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.544+0000 7ff36b422740 1 client.29108 shutdown 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.545+0000 7ff36b422740 1 -- 172.21.15.46:0/2401387769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 msgr2=0x55f7849cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.545+0000 7ff36b422740 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849cf3a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff35402aa90 tx=0x7ff354009260 comp rx=0 tx=0).stop 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.545+0000 7ff36b422740 1 -- 172.21.15.46:0/2401387769 shutdown_connections 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.545+0000 7ff36b422740 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x55f784a45220 0x55f784a47610 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.545+0000 7ff36b422740 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f7849c9e50 0x55f784a437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.545+0000 7ff36b422740 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7849bf590 0x55f7849cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.545+0000 7ff343400640 0 client.29108 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:51:53.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.545+0000 7ff36b422740 1 --2- 172.21.15.46:0/2401387769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7849bec70 0x55f7849cf3a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.546+0000 7ff36b422740 1 -- 172.21.15.46:0/2401387769 >> 172.21.15.46:0/2401387769 conn(0x55f7849bb200 msgr2=0x55f7849ba380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.546+0000 7ff36b422740 1 -- 172.21.15.46:0/2401387769 shutdown_connections 2024-09-10T07:51:53.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.546+0000 7ff36b422740 1 -- 172.21.15.46:0/2401387769 wait complete. 2024-09-10T07:51:53.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.547+0000 7ff36b422740 20 client.29108 trim_cache size 0 max 16384 2024-09-10T07:51:53.547 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 -- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3480a2c40 msgr2=0x7ff3480a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3480a2c40 0x7ff3480a5100 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff36406b430 tx=0x7ff35c00c7f0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 -- 172.21.15.46:0/1521829469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364109b10 msgr2=0x7ff3641a34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364109b10 0x7ff3641a34c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff350009f20 tx=0x7ff35000ee80 comp rx=0 tx=0).stop 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 -- 172.21.15.46:0/1521829469 shutdown_connections 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff364114ab0 0x7ff3641a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364109b10 0x7ff3641a34c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff3480a2c40 0x7ff3480a5100 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 --2- 172.21.15.46:0/1521829469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3641091f0 0x7ff3641a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 -- 172.21.15.46:0/1521829469 >> 172.21.15.46:0/1521829469 conn(0x7ff36406e140 msgr2=0x7ff364070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 -- 172.21.15.46:0/1521829469 shutdown_connections 2024-09-10T07:51:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:53.548+0000 7ff368a00640 1 -- 172.21.15.46:0/1521829469 wait complete. 2024-09-10T07:51:54.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:54 smithi181 ceph-mon[35397]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-09-10T07:51:54.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:54 smithi063 ceph-mon[35020]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-09-10T07:51:54.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:54 smithi046 ceph-mon[30705]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 5.5 MiB/s wr, 612 op/s 2024-09-10T07:51:55.080 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:51:55.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:51:55.305 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:55 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:55.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:55 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:55.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:55 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:51:55.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 875.0 s, 365.0 tps, lat 262.909 ms stddev 301.063 2024-09-10T07:51:56.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:56 smithi181 ceph-mon[35397]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-09-10T07:51:56.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:56 smithi063 ceph-mon[35020]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-09-10T07:51:56.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:56 smithi046 ceph-mon[30705]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-09-10T07:51:56.693 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 -- 172.21.15.46:0/533480795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 msgr2=0x7f5e6810e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 --2- 172.21.15.46:0/533480795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 0x7f5e6810e350 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009860 tx=0x7f5e5402a340 comp rx=0 tx=0).stop 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 -- 172.21.15.46:0/533480795 shutdown_connections 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 --2- 172.21.15.46:0/533480795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6810e890 0x7f5e6810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 --2- 172.21.15.46:0/533480795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 0x7f5e6810e350 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 --2- 172.21.15.46:0/533480795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e68109450 0x7f5e68109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 -- 172.21.15.46:0/533480795 >> 172.21.15.46:0/533480795 conn(0x7f5e6806e2d0 msgr2=0x7f5e6806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 -- 172.21.15.46:0/533480795 shutdown_connections 2024-09-10T07:51:56.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.810+0000 7f5e70000640 1 -- 172.21.15.46:0/533480795 wait complete. 2024-09-10T07:51:56.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.811+0000 7f5e70000640 1 Processor -- start 2024-09-10T07:51:56.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.811+0000 7f5e70000640 1 -- start start 2024-09-10T07:51:56.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e70000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e68109450 0x7f5e681188e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e70000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 0x7f5e68118e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e70000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e6810e890 0x7f5e68119360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e70000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68116580 con 0x7f5e68109450 2024-09-10T07:51:56.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e70000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68116400 con 0x7f5e68109d70 2024-09-10T07:51:56.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e70000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6811cc20 con 0x7f5e6810e890 2024-09-10T07:51:56.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e6e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e6810e890 0x7f5e68119360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e6cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 0x7f5e68118e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e6e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e6810e890 0x7f5e68119360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50972/0 (socket says 172.21.15.46:50972) 2024-09-10T07:51:56.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e6cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 0x7f5e68118e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:51946/0 (socket says 172.21.15.46:51946) 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.812+0000 7f5e6e000640 1 -- 172.21.15.46:0/3606009294 learned_addr learned my addr 172.21.15.46:0/3606009294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.813+0000 7f5e6d600640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e68109450 0x7f5e681188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.813+0000 7f5e6cc00640 1 -- 172.21.15.46:0/3606009294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e6810e890 msgr2=0x7f5e68119360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.813+0000 7f5e6cc00640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e6810e890 0x7f5e68119360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.813+0000 7f5e6cc00640 1 -- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e68109450 msgr2=0x7f5e681188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.813+0000 7f5e6cc00640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e68109450 0x7f5e681188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.813+0000 7f5e6cc00640 1 -- 172.21.15.46:0/3606009294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e681b3560 con 0x7f5e68109d70 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.813+0000 7f5e6d600640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e68109450 0x7f5e681188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:56.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.813+0000 7f5e6cc00640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 0x7f5e68118e20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009860 tx=0x7f5e54033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.814+0000 7f5e66000640 1 -- 172.21.15.46:0/3606009294 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54037040 con 0x7f5e68109d70 2024-09-10T07:51:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.814+0000 7f5e66000640 1 -- 172.21.15.46:0/3606009294 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e54040070 con 0x7f5e68109d70 2024-09-10T07:51:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.814+0000 7f5e66000640 1 -- 172.21.15.46:0/3606009294 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54003930 con 0x7f5e68109d70 2024-09-10T07:51:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.814+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e681b37f0 con 0x7f5e68109d70 2024-09-10T07:51:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.814+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681b3c80 con 0x7f5e68109d70 2024-09-10T07:51:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.815+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e68073f40 con 0x7f5e68109d70 2024-09-10T07:51:56.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.819+0000 7f5e66000640 1 -- 172.21.15.46:0/3606009294 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e54030670 con 0x7f5e68109d70 2024-09-10T07:51:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.821+0000 7f5e66000640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e500a2c00 0x7f5e500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.821+0000 7f5e66000640 1 -- 172.21.15.46:0/3606009294 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f5e540ef510 con 0x7f5e68109d70 2024-09-10T07:51:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.823+0000 7f5e66000640 1 -- 172.21.15.46:0/3606009294 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e54049060 con 0x7f5e68109d70 2024-09-10T07:51:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.823+0000 7f5e6d600640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e500a2c00 0x7f5e500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:56.825+0000 7f5e6d600640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e500a2c00 0x7f5e500a50c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009c20 tx=0x7f5e5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.208+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e681b3fa0 con 0x7f5e68109d70 2024-09-10T07:51:57.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.210+0000 7f5e66000640 1 -- 172.21.15.46:0/3606009294 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11986 (secure 0 0 0) 0x7f5e540bc120 con 0x7f5e68109d70 2024-09-10T07:51:57.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:51:57.213 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 60 2024-09-10T07:51:57.214 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":60,"btime":"2024-09-10T07:51:54:063936+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:51:53.066973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:51:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.216+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e500a2c00 msgr2=0x7f5e500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.216+0000 7f5e70000640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e500a2c00 0x7f5e500a50c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009c20 tx=0x7f5e5c005f90 comp rx=0 tx=0).stop 2024-09-10T07:51:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.216+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 msgr2=0x7f5e68118e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.216+0000 7f5e70000640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 0x7f5e68118e20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009860 tx=0x7f5e54033b60 comp rx=0 tx=0).stop 2024-09-10T07:51:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.219+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 shutdown_connections 2024-09-10T07:51:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.219+0000 7f5e70000640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e6810e890 0x7f5e68119360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.219+0000 7f5e70000640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e68109d70 0x7f5e68118e20 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.219+0000 7f5e70000640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f5e500a2c00 0x7f5e500a50c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.219+0000 7f5e70000640 1 --2- 172.21.15.46:0/3606009294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e68109450 0x7f5e681188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.219+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 >> 172.21.15.46:0/3606009294 conn(0x7f5e6806e2d0 msgr2=0x7f5e68072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.219+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 shutdown_connections 2024-09-10T07:51:57.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:57.220+0000 7f5e70000640 1 -- 172.21.15.46:0/3606009294 wait complete. 2024-09-10T07:51:58.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:58 smithi046 ceph-mon[30705]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.8 MiB/s wr, 518 op/s 2024-09-10T07:51:58.159 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:51:58 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3606009294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:51:58.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:58 smithi181 ceph-mon[35397]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.8 MiB/s wr, 518 op/s 2024-09-10T07:51:58.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:51:58 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3606009294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:51:58.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:58 smithi063 ceph-mon[35020]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.8 MiB/s wr, 518 op/s 2024-09-10T07:51:58.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:51:58 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3606009294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:51:59.115 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-09-10T07:51:59.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.092+0000 7f3816a00640 1 -- 172.21.15.46:0/1093134178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 msgr2=0x7f3810116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.092+0000 7f3816a00640 1 --2- 172.21.15.46:0/1093134178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 0x7f3810116ea0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3808009ca0 tx=0x7f380802a940 comp rx=0 tx=0).stop 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.096+0000 7f3816a00640 1 -- 172.21.15.46:0/1093134178 shutdown_connections 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.096+0000 7f3816a00640 1 --2- 172.21.15.46:0/1093134178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 0x7f3810116ea0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.096+0000 7f3816a00640 1 --2- 172.21.15.46:0/1093134178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3810109b10 0x7f3810114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.096+0000 7f3816a00640 1 --2- 172.21.15.46:0/1093134178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38101091f0 0x7f38101095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.096+0000 7f3816a00640 1 -- 172.21.15.46:0/1093134178 >> 172.21.15.46:0/1093134178 conn(0x7f381006e140 msgr2=0x7f381006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.096+0000 7f3816a00640 1 -- 172.21.15.46:0/1093134178 shutdown_connections 2024-09-10T07:51:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.096+0000 7f3816a00640 1 -- 172.21.15.46:0/1093134178 wait complete. 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.097+0000 7f3816a00640 1 Processor -- start 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.098+0000 7f3816a00640 1 -- start start 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.099+0000 7f3816a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38101091f0 0x7f38101a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.099+0000 7f3816a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3810109b10 0x7f38101a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.100+0000 7f380fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38101091f0 0x7f38101a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.100+0000 7f380fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38101091f0 0x7f38101a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:51958/0 (socket says 172.21.15.46:51958) 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.100+0000 7f380fe00640 1 -- 172.21.15.46:0/3438368739 learned_addr learned my addr 172.21.15.46:0/3438368739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.100+0000 7f380f400640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3810109b10 0x7f38101a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.100+0000 7f3816a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 0x7f38101a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.101+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f381011a770 con 0x7f3810109b10 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.101+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f381011a5f0 con 0x7f38101091f0 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.101+0000 7f3814a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 0x7f38101a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.101+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f381011a8f0 con 0x7f3810114ab0 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.102+0000 7f3814a00640 1 -- 172.21.15.46:0/3438368739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38101091f0 msgr2=0x7f38101a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.102+0000 7f3814a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38101091f0 0x7f38101a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.102+0000 7f3814a00640 1 -- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3810109b10 msgr2=0x7f38101a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.102+0000 7f3814a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3810109b10 0x7f38101a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.102+0000 7f3814a00640 1 -- 172.21.15.46:0/3438368739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38101a4c00 con 0x7f3810114ab0 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.104+0000 7f3814a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 0x7f38101a44d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f380802af30 tx=0x7f380802f700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.106+0000 7f380cc00640 1 -- 172.21.15.46:0/3438368739 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808034040 con 0x7f3810114ab0 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.106+0000 7f380cc00640 1 -- 172.21.15.46:0/3438368739 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380802fe10 con 0x7f3810114ab0 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.106+0000 7f380cc00640 1 -- 172.21.15.46:0/3438368739 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808004540 con 0x7f3810114ab0 2024-09-10T07:51:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.106+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38101a4e90 con 0x7f3810114ab0 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.106+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381010a300 con 0x7f3810114ab0 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.113+0000 7f3819657740 1 Processor -- start 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.123+0000 7f380cc00640 1 -- 172.21.15.46:0/3438368739 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3808037020 con 0x7f3810114ab0 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.125+0000 7f3819657740 1 -- start start 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.127+0000 7f3819657740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd95370f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.127+0000 7f3819657740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd95384120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.127+0000 7f3819657740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95386a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.127+0000 7f3819657740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dd952d73e0 con 0x55dd953714b0 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.127+0000 7f3819657740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd9527ed80 con 0x55dd95384660 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.127+0000 7f3819657740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dd952d33a0 con 0x55dd95370b90 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.128+0000 7f380f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd95370f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.128+0000 7f380cc00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37f4004aa0 0x7f37f4006f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.128+0000 7f380f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd95370f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50988/0 (socket says 172.21.15.46:50988) 2024-09-10T07:51:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.128+0000 7f380f400640 1 -- 172.21.15.46:0/3859041411 learned_addr learned my addr 172.21.15.46:0/3859041411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.128+0000 7f380f400640 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95386a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.128+0000 7f380cc00640 1 -- 172.21.15.46:0/3438368739 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f38080f0110 con 0x7f3810114ab0 2024-09-10T07:51:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.130+0000 7f3814a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37f4004aa0 0x7f37f4006f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.130+0000 7f380fe00640 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd95384120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.133+0000 7f380f400640 1 -- 172.21.15.46:0/3859041411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 msgr2=0x55dd95386a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.133+0000 7f380f400640 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95386a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.133+0000 7f380f400640 1 -- 172.21.15.46:0/3859041411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 msgr2=0x55dd95384120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.133+0000 7f380f400640 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd95384120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.133+0000 7f3814a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37f4004aa0 0x7f37f4006f60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3808005e30 tx=0x7f380803c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.133+0000 7f380f400640 1 -- 172.21.15.46:0/3859041411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9529dae0 con 0x55dd95370b90 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.134+0000 7f380fe00640 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd95384120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.134+0000 7f380f400640 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd95370f70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0058c0 tx=0x7f37fc009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=931bcc2b3f85a984 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.135+0000 7f3805200640 1 -- 172.21.15.46:0/3859041411 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc004030 con 0x55dd95370b90 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.136+0000 7f3805200640 1 -- 172.21.15.46:0/3859041411 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37fc00fd20 con 0x55dd95370b90 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.136+0000 7f3805200640 1 -- 172.21.15.46:0/3859041411 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc003950 con 0x55dd95370b90 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.137+0000 7f3819657740 1 -- 172.21.15.46:0/3859041411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 msgr2=0x55dd95370f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.137+0000 7f3819657740 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd95370f70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0058c0 tx=0x7f37fc009770 comp rx=0 tx=0).stop 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.137+0000 7f3819657740 1 -- 172.21.15.46:0/3859041411 shutdown_connections 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.137+0000 7f3819657740 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95386a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.137+0000 7f3819657740 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd95384120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.137+0000 7f3819657740 1 --2- 172.21.15.46:0/3859041411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd95370f70 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.137+0000 7f3819657740 1 -- 172.21.15.46:0/3859041411 >> 172.21.15.46:0/3859041411 conn(0x55dd9536d200 msgr2=0x55dd9536d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.138+0000 7f3819657740 1 -- 172.21.15.46:0/3859041411 shutdown_connections 2024-09-10T07:51:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.138+0000 7f3819657740 1 -- 172.21.15.46:0/3859041411 wait complete. 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.138+0000 7f3819657740 1 -- start start 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.139+0000 7f3819657740 1 Processor -- start 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.140+0000 7f3819657740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd9537e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.140+0000 7f3819657740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd9537e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.141+0000 7f3819657740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 0x55dd95383e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.141+0000 7f3819657740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dd95373af0 con 0x55dd953714b0 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.141+0000 7f3819657740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd952920f0 con 0x55dd95370b90 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.141+0000 7f3819657740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55dd95373c70 con 0x55dd95384660 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.141+0000 7f3819657740 10 client.0 _ll_register_callbacks cb 0x55dd95279bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.141+0000 7f380fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd9537e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.142+0000 7f380fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd9537e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43002/0 (socket says 172.21.15.46:43002) 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.142+0000 7f380fe00640 1 -- 172.21.15.46:0/882555862 learned_addr learned my addr 172.21.15.46:0/882555862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.142+0000 7f380f400640 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 0x55dd95383e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.142+0000 7f380f400640 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd9537e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.143+0000 7f380fe00640 1 -- 172.21.15.46:0/882555862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 msgr2=0x55dd95383e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.143+0000 7f380fe00640 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 0x55dd95383e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.143+0000 7f380fe00640 1 -- 172.21.15.46:0/882555862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 msgr2=0x55dd9537e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.143+0000 7f380fe00640 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd9537e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.143+0000 7f380fe00640 1 -- 172.21.15.46:0/882555862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd950483c0 con 0x55dd953714b0 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.147+0000 7f380fe00640 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd9537e6a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f380000ba40 tx=0x7f380000bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd01e8cf1ccc71b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.147+0000 7f37f9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.148+0000 7f37f9600640 1 -- 172.21.15.46:0/882555862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380001b670 con 0x55dd953714b0 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.149+0000 7f37f9600640 1 -- 172.21.15.46:0/882555862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f380001b470 con 0x55dd953714b0 2024-09-10T07:51:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.149+0000 7f3819657740 1 -- 172.21.15.46:0/882555862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37fc00fd20 con 0x55dd953714b0 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.150+0000 7f37f9600640 1 -- 172.21.15.46:0/882555862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f380001b670 con 0x55dd953714b0 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.150+0000 7f3819657740 10 client.18207 fetch_fsmap learned FSMap version 60 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.150+0000 7f3819657740 1 -- 172.21.15.46:0/882555862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55dd9539bae0 con 0x55dd953714b0 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.151+0000 7f37f9600640 1 -- 172.21.15.46:0/882555862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37fc00fd20 con 0x55dd953714b0 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.154+0000 7f37f9600640 1 -- 172.21.15.46:0/882555862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f380001b810 con 0x55dd953714b0 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.155+0000 7f3819657740 10 client.18207 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.155+0000 7f3819657740 10 client.18207 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.155+0000 7f3819657740 20 client.18207 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.155+0000 7f3819657740 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55dd9539cf90 0x55dd9539f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.155+0000 7f3819657740 4 client.18207 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.155+0000 7f3819657740 1 -- 172.21.15.46:0/882555862 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd952e4950 con 0x55dd9539cf90 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.155+0000 7f380f400640 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55dd9539cf90 0x55dd9539f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.156+0000 7f380f400640 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55dd9539cf90 0x55dd9539f3b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4dff44882c110e4 server_cookie=30186ca566a8ccf9 in_seq=0 out_seq=0 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.156+0000 7f37f9600640 10 client.18207 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.157+0000 7f37f9600640 1 -- 172.21.15.46:0/882555862 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd952e4950 con 0x55dd9539cf90 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.157+0000 7f37f9600640 10 client.18207 handle_command_reply: tid=0 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.157+0000 7f3819657740 1 client.18207 shutdown 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f3819657740 1 -- 172.21.15.46:0/882555862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 msgr2=0x55dd9537e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f3819657740 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd9537e6a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f380000ba40 tx=0x7f380000bf00 comp rx=0 tx=0).stop 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f3819657740 1 -- 172.21.15.46:0/882555862 shutdown_connections 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f3819657740 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55dd9539cf90 0x55dd9539f3b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f3819657740 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 0x55dd95383e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f3819657740 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd9537e6a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f37f9600640 0 client.18207 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f3819657740 1 --2- 172.21.15.46:0/882555862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd9537e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.158+0000 7f3819657740 1 -- 172.21.15.46:0/882555862 >> 172.21.15.46:0/882555862 conn(0x55dd9536d200 msgr2=0x55dd9536c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.159+0000 7f3819657740 1 -- 172.21.15.46:0/882555862 shutdown_connections 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.159+0000 7f3819657740 1 -- 172.21.15.46:0/882555862 wait complete. 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.160+0000 7f3819657740 20 client.18207 trim_cache size 0 max 16384 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.190+0000 7f3819657740 1 Processor -- start 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.190+0000 7f3819657740 1 -- start start 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.190+0000 7f3819657740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd95383de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.190+0000 7f3819657740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.190+0000 7f3819657740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 0x55dd953a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.190+0000 7f3819657740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b5190 con 0x55dd953714b0 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.190+0000 7f3819657740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b6e50 con 0x55dd95370b90 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.190+0000 7f3819657740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b5860 con 0x55dd95384660 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.191+0000 7f380f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd95383de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.191+0000 7f380f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd95383de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:51980/0 (socket says 172.21.15.46:51980) 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.191+0000 7f380f400640 1 -- 172.21.15.46:0/4188483555 learned_addr learned my addr 172.21.15.46:0/4188483555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.191+0000 7f380f400640 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 0x55dd953a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.192+0000 7f380fe00640 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.193+0000 7f380fe00640 1 -- 172.21.15.46:0/4188483555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 msgr2=0x55dd953a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.193+0000 7f380fe00640 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 0x55dd953a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.193+0000 7f380fe00640 1 -- 172.21.15.46:0/4188483555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 msgr2=0x55dd95383de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.193+0000 7f380fe00640 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd95383de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.193+0000 7f380fe00640 1 -- 172.21.15.46:0/4188483555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9537ea10 con 0x55dd953714b0 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.194+0000 7f380fe00640 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953a4c60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f380000ba10 tx=0x7f380000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f771fc20de988f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.194+0000 7f37fbe00640 1 -- 172.21.15.46:0/4188483555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800003a80 con 0x55dd953714b0 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.195+0000 7f37fbe00640 1 -- 172.21.15.46:0/4188483555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3800016070 con 0x55dd953714b0 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.195+0000 7f37fbe00640 1 -- 172.21.15.46:0/4188483555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000049b0 con 0x55dd953714b0 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.195+0000 7f3819657740 1 -- 172.21.15.46:0/4188483555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 msgr2=0x55dd953a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.195+0000 7f3819657740 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953a4c60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f380000ba10 tx=0x7f380000b260 comp rx=0 tx=0).stop 2024-09-10T07:51:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.196+0000 7f3819657740 1 -- 172.21.15.46:0/4188483555 shutdown_connections 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.196+0000 7f3819657740 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95384660 0x55dd953a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.196+0000 7f3819657740 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953a4c60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.196+0000 7f3819657740 1 --2- 172.21.15.46:0/4188483555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95370b90 0x55dd95383de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.196+0000 7f3819657740 1 -- 172.21.15.46:0/4188483555 >> 172.21.15.46:0/4188483555 conn(0x55dd9536d200 msgr2=0x55dd95372e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.196+0000 7f3819657740 1 -- 172.21.15.46:0/4188483555 shutdown_connections 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.196+0000 7f3819657740 1 -- 172.21.15.46:0/4188483555 wait complete. 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.197+0000 7f3819657740 1 -- start start 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.197+0000 7f3819657740 1 Processor -- start 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f3819657740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd953ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f3819657740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f3819657740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95406ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f3819657740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b66c0 con 0x55dd953714b0 2024-09-10T07:51:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f3819657740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b5ff0 con 0x55dd95384660 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f3819657740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b3170 con 0x55dd95370b90 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f3819657740 10 client.0 _ll_register_callbacks cb 0x7f3800004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f380f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95406ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.198+0000 7f380f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95406ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:51994/0 (socket says 172.21.15.46:51994) 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.199+0000 7f380f400640 1 -- 172.21.15.46:0/2136076266 learned_addr learned my addr 172.21.15.46:0/2136076266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.199+0000 7f380f400640 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd953ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.199+0000 7f380fe00640 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.200+0000 7f380fe00640 1 -- 172.21.15.46:0/2136076266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 msgr2=0x55dd953ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.200+0000 7f380fe00640 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd953ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.200+0000 7f380fe00640 1 -- 172.21.15.46:0/2136076266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 msgr2=0x55dd95406ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.201+0000 7f380fe00640 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95406ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.201+0000 7f380fe00640 1 -- 172.21.15.46:0/2136076266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd95406ff0 con 0x55dd953714b0 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.201+0000 7f380f400640 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95406ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.202+0000 7f380fe00640 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953ad370 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55dd9535eed0 tx=0x7f3800012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3466a8ce2e0eee98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.202+0000 7f37f9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.202+0000 7f37f9600640 1 -- 172.21.15.46:0/2136076266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380003b060 con 0x55dd953714b0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.203+0000 7f37f9600640 1 -- 172.21.15.46:0/2136076266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3800033070 con 0x55dd953714b0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.203+0000 7f3819657740 1 -- 172.21.15.46:0/2136076266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3800016070 con 0x55dd953714b0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.204+0000 7f37f9600640 1 -- 172.21.15.46:0/2136076266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f380003b060 con 0x55dd953714b0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.204+0000 7f3819657740 10 client.18213 fetch_fsmap learned FSMap version 60 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.204+0000 7f3819657740 1 -- 172.21.15.46:0/2136076266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55dd954082f0 con 0x55dd953714b0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.205+0000 7f37f9600640 1 -- 172.21.15.46:0/2136076266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3800016070 con 0x55dd953714b0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.205+0000 7f37f9600640 1 -- 172.21.15.46:0/2136076266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f3800012cc0 con 0x55dd953714b0 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.206+0000 7f3819657740 10 client.18213 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.206+0000 7f3819657740 10 client.18213 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:51:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.207+0000 7f3819657740 20 client.18213 populate_metadata read hostname 'smithi046' 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.207+0000 7f3819657740 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55dd95408850 0x55dd9540ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.208+0000 7f3819657740 4 client.18213 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.208+0000 7f3819657740 1 -- 172.21.15.46:0/2136076266 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd952dee60 con 0x55dd95408850 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.208+0000 7f380f400640 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55dd95408850 0x55dd9540ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.211+0000 7f380f400640 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55dd95408850 0x55dd9540ac40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=feccb817e965566f server_cookie=6d10cc7a1f7f4162 in_seq=0 out_seq=0 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.211+0000 7f37f9600640 10 client.18213 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.211+0000 7f37f9600640 1 -- 172.21.15.46:0/2136076266 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd952dee60 con 0x55dd95408850 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.211+0000 7f37f9600640 10 client.18213 handle_command_reply: tid=0 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.211+0000 7f3819657740 1 client.18213 shutdown 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f3819657740 1 -- 172.21.15.46:0/2136076266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 msgr2=0x55dd953ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f3819657740 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953ad370 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55dd9535eed0 tx=0x7f3800012a30 comp rx=0 tx=0).stop 2024-09-10T07:51:59.162 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:51:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f3819657740 1 -- 172.21.15.46:0/2136076266 shutdown_connections 2024-09-10T07:51:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f3819657740 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd95384660 0x55dd95406ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f3819657740 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd953714b0 0x55dd953ad370 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f3819657740 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55dd95408850 0x55dd9540ac40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f3819657740 1 --2- 172.21.15.46:0/2136076266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dd95370b90 0x55dd953ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f3819657740 1 -- 172.21.15.46:0/2136076266 >> 172.21.15.46:0/2136076266 conn(0x55dd9536d200 msgr2=0x55dd9537ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.215+0000 7f37f9600640 0 client.18213 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.216+0000 7f3819657740 1 -- 172.21.15.46:0/2136076266 shutdown_connections 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.216+0000 7f3819657740 1 -- 172.21.15.46:0/2136076266 wait complete. 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.217+0000 7f3819657740 20 client.18213 trim_cache size 0 max 16384 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37f4004aa0 msgr2=0x7f37f4006f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37f4004aa0 0x7f37f4006f60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3808005e30 tx=0x7f380803c040 comp rx=0 tx=0).stop 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 msgr2=0x7f38101a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 0x7f38101a44d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f380802af30 tx=0x7f380802f700 comp rx=0 tx=0).stop 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 shutdown_connections 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3810114ab0 0x7f38101a44d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f37f4004aa0 0x7f37f4006f60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3810109b10 0x7f38101a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 --2- 172.21.15.46:0/3438368739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38101091f0 0x7f38101a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.218+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 >> 172.21.15.46:0/3438368739 conn(0x7f381006e140 msgr2=0x7f3810071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.219+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 shutdown_connections 2024-09-10T07:51:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:51:58.219+0000 7f3816a00640 1 -- 172.21.15.46:0/3438368739 wait complete. 2024-09-10T07:52:00.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:00 smithi181 ceph-mon[35397]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-09-10T07:52:00.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:00 smithi063 ceph-mon[35020]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-09-10T07:52:00.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:00 smithi046 ceph-mon[30705]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-09-10T07:52:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 880.0 s, 454.4 tps, lat 231.849 ms stddev 282.808 2024-09-10T07:52:00.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:52:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.131+0000 7f2c05600640 1 -- 172.21.15.46:0/914509827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00102600 msgr2=0x7f2c0006a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.131+0000 7f2c05600640 1 --2- 172.21.15.46:0/914509827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00102600 0x7f2c0006a0c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009ca0 tx=0x7f2bf802a8d0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.133+0000 7f2c05600640 1 -- 172.21.15.46:0/914509827 shutdown_connections 2024-09-10T07:52:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.133+0000 7f2c05600640 1 --2- 172.21.15.46:0/914509827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00102600 0x7f2c0006a0c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.133+0000 7f2c05600640 1 --2- 172.21.15.46:0/914509827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00101c40 0x7f2c001020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.133+0000 7f2c05600640 1 --2- 172.21.15.46:0/914509827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c00100a50 0x7f2c00100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.133+0000 7f2c05600640 1 -- 172.21.15.46:0/914509827 >> 172.21.15.46:0/914509827 conn(0x7f2c000fc1b0 msgr2=0x7f2c000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.136+0000 7f2c05600640 1 -- 172.21.15.46:0/914509827 shutdown_connections 2024-09-10T07:52:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.136+0000 7f2c05600640 1 -- 172.21.15.46:0/914509827 wait complete. 2024-09-10T07:52:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.137+0000 7f2c05600640 1 Processor -- start 2024-09-10T07:52:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.138+0000 7f2c05600640 1 -- start start 2024-09-10T07:52:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.138+0000 7f2c05600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00100a50 0x7f2c0019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.138+0000 7f2c05600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00101c40 0x7f2c0019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.138+0000 7f2c05600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c00102600 0x7f2c001a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.138+0000 7f2c05600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0010fac0 con 0x7f2c00100a50 2024-09-10T07:52:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.138+0000 7f2c05600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0010f940 con 0x7f2c00102600 2024-09-10T07:52:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.139+0000 7f2c05600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0010fc40 con 0x7f2c00101c40 2024-09-10T07:52:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.139+0000 7f2bff400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c00102600 0x7f2c001a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.139+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00101c40 0x7f2c0019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.139+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00101c40 0x7f2c0019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:51038/0 (socket says 172.21.15.46:51038) 2024-09-10T07:52:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.139+0000 7f2bfe000640 1 -- 172.21.15.46:0/2082304210 learned_addr learned my addr 172.21.15.46:0/2082304210 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.140+0000 7f2bfe000640 1 -- 172.21.15.46:0/2082304210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c00102600 msgr2=0x7f2c001a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.140+0000 7f2bfe000640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c00102600 0x7f2c001a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.140+0000 7f2bfe000640 1 -- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00100a50 msgr2=0x7f2c0019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.140+0000 7f2bfe000640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00100a50 0x7f2c0019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.140+0000 7f2bfe000640 1 -- 172.21.15.46:0/2082304210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c001a6000 con 0x7f2c00101c40 2024-09-10T07:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.144+0000 7f2bfe000640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00101c40 0x7f2c0019e920 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00b4a0 tx=0x7f2bec00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.145+0000 7f2bf7400640 1 -- 172.21.15.46:0/2082304210 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec014070 con 0x7f2c00101c40 2024-09-10T07:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.145+0000 7f2c05600640 1 -- 172.21.15.46:0/2082304210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c001a62f0 con 0x7f2c00101c40 2024-09-10T07:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.145+0000 7f2c05600640 1 -- 172.21.15.46:0/2082304210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001a6910 con 0x7f2c00101c40 2024-09-10T07:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.147+0000 7f2bf7400640 1 -- 172.21.15.46:0/2082304210 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bec010af0 con 0x7f2c00101c40 2024-09-10T07:52:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.147+0000 7f2bf7400640 1 -- 172.21.15.46:0/2082304210 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec017d10 con 0x7f2c00101c40 2024-09-10T07:52:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.163+0000 7f2c08493740 1 Processor -- start 2024-09-10T07:52:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.163+0000 7f2bf7400640 1 -- 172.21.15.46:0/2082304210 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2bec017070 con 0x7f2c00101c40 2024-09-10T07:52:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.163+0000 7f2c08493740 1 -- start start 2024-09-10T07:52:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.166+0000 7f2c08493740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.166+0000 7f2c08493740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288bb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.166+0000 7f2c08493740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288bba60 0x55cb288bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.166+0000 7f2c08493740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cb2881b100 con 0x55cb288bba60 2024-09-10T07:52:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.166+0000 7f2c08493740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb287c31b0 con 0x55cb288b69d0 2024-09-10T07:52:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.166+0000 7f2c08493740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cb288170c0 con 0x55cb288b6090 2024-09-10T07:52:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.167+0000 7f2bf7400640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2be40a2cb0 0x7f2be40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.167+0000 7f2bff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.167+0000 7f2bff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:51044/0 (socket says 172.21.15.46:51044) 2024-09-10T07:52:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.167+0000 7f2bff400640 1 -- 172.21.15.46:0/425425386 learned_addr learned my addr 172.21.15.46:0/425425386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.168+0000 7f2bfea00640 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288bb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.168+0000 7f2bff400640 1 -- 172.21.15.46:0/425425386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 msgr2=0x55cb288bb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.168+0000 7f2bff400640 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288bb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.168+0000 7f2bff400640 1 -- 172.21.15.46:0/425425386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288bba60 msgr2=0x55cb288bdf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:52:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.169+0000 7f2bf7400640 1 -- 172.21.15.46:0/2082304210 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2bec0ce750 con 0x7f2c00101c40 2024-09-10T07:52:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.170+0000 7f2bfe000640 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288bba60 0x55cb288bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.170+0000 7f2bff400640 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288bba60 0x55cb288bdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.170+0000 7f2bff400640 1 -- 172.21.15.46:0/425425386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2852e210 con 0x55cb288b6090 2024-09-10T07:52:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.171+0000 7f2bff400640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2be40a2cb0 0x7f2be40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.175+0000 7f2bff400640 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b6490 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8005e30 tx=0x7f2bf8031bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e8ce4958dcfbdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.175+0000 7f2beb400640 1 -- 172.21.15.46:0/425425386 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80315a0 con 0x55cb288b6090 2024-09-10T07:52:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.176+0000 7f2beb400640 1 -- 172.21.15.46:0/425425386 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8036070 con 0x55cb288b6090 2024-09-10T07:52:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.177+0000 7f2c08493740 1 -- 172.21.15.46:0/425425386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 msgr2=0x55cb288b6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.177+0000 7f2c08493740 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b6490 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8005e30 tx=0x7f2bf8031bc0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.177+0000 7f2bff400640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2be40a2cb0 0x7f2be40a5170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8047000 tx=0x7f2bf8030c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.178+0000 7f2c08493740 1 -- 172.21.15.46:0/425425386 shutdown_connections 2024-09-10T07:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.178+0000 7f2c08493740 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288bba60 0x55cb288bdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.178+0000 7f2c08493740 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288bb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.178+0000 7f2c08493740 1 --2- 172.21.15.46:0/425425386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b6490 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.178+0000 7f2c08493740 1 -- 172.21.15.46:0/425425386 >> 172.21.15.46:0/425425386 conn(0x55cb288b1700 msgr2=0x55cb288b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.178+0000 7f2c08493740 1 -- 172.21.15.46:0/425425386 shutdown_connections 2024-09-10T07:52:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.181+0000 7f2c08493740 1 -- 172.21.15.46:0/425425386 wait complete. 2024-09-10T07:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.181+0000 7f2c08493740 1 -- start start 2024-09-10T07:52:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.182+0000 7f2c08493740 1 Processor -- start 2024-09-10T07:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.183+0000 7f2c08493740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b6090 0x55cb288d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.183+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b6090 0x55cb288d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.184+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b6090 0x55cb288d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43040/0 (socket says 172.21.15.46:43040) 2024-09-10T07:52:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.185+0000 7f2bfea00640 1 -- 172.21.15.46:0/745747490 learned_addr learned my addr 172.21.15.46:0/745747490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.185+0000 7f2c08493740 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.185+0000 7f2c08493740 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.185+0000 7f2bfe000640 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.185+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cb288cfae0 con 0x55cb288b6090 2024-09-10T07:52:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.185+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb287a4090 con 0x55cb288b69d0 2024-09-10T07:52:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.185+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cb288cfc60 con 0x55cb288bba60 2024-09-10T07:52:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.185+0000 7f2bff400640 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.186+0000 7f2c08493740 10 client.0 _ll_register_callbacks cb 0x55cb287f8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.187+0000 7f2bfe000640 1 -- 172.21.15.46:0/745747490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 msgr2=0x55cb288d8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.187+0000 7f2bfe000640 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.187+0000 7f2bfe000640 1 -- 172.21.15.46:0/745747490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b6090 msgr2=0x55cb288d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.187+0000 7f2bfe000640 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b6090 0x55cb288d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.187+0000 7f2bfe000640 1 -- 172.21.15.46:0/745747490 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb285fa620 con 0x55cb288b69d0 2024-09-10T07:52:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.187+0000 7f2bfea00640 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b6090 0x55cb288d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.188+0000 7f2bfe000640 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288d17c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0cbf20 tx=0x7f2bec09b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45c8e24df81806d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.188+0000 7f2bdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.190+0000 7f2bdfe00640 1 -- 172.21.15.46:0/745747490 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec01e070 con 0x55cb288b69d0 2024-09-10T07:52:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.190+0000 7f2bdfe00640 1 -- 172.21.15.46:0/745747490 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2bec0972a0 con 0x55cb288b69d0 2024-09-10T07:52:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.190+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf8036070 con 0x55cb288b69d0 2024-09-10T07:52:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.190+0000 7f2bdfe00640 1 -- 172.21.15.46:0/745747490 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2bec0b8050 con 0x55cb288b69d0 2024-09-10T07:52:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.191+0000 7f2c08493740 10 client.29141 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.191+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55cb288db650 con 0x55cb288b69d0 2024-09-10T07:52:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.193+0000 7f2bdfe00640 1 -- 172.21.15.46:0/745747490 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8036070 con 0x55cb288b69d0 2024-09-10T07:52:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.193+0000 7f2bdfe00640 1 -- 172.21.15.46:0/745747490 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f2bec096980 con 0x55cb288b69d0 2024-09-10T07:52:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.193+0000 7f2c08493740 10 client.29141 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.193+0000 7f2c08493740 10 client.29141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.193+0000 7f2c08493740 20 client.29141 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.193+0000 7f2c08493740 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb288dcd20 0x55cb288df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.193+0000 7f2c08493740 4 client.29141 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.193+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb28828830 con 0x55cb288dcd20 2024-09-10T07:52:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.195+0000 7f2bfea00640 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb288dcd20 0x55cb288df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.196+0000 7f2bfea00640 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb288dcd20 0x55cb288df1a0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eaea6e8bde5db7a server_cookie=4c312e7db102fd87 in_seq=0 out_seq=0 2024-09-10T07:52:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.196+0000 7f2bdfe00640 10 client.29141 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.198+0000 7f2bdfe00640 1 -- 172.21.15.46:0/745747490 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb28828830 con 0x55cb288dcd20 2024-09-10T07:52:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.199+0000 7f2bdfe00640 10 client.29141 handle_command_reply: tid=0 2024-09-10T07:52:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.199+0000 7f2c08493740 1 client.29141 shutdown 2024-09-10T07:52:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.199+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 msgr2=0x55cb288d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.199+0000 7f2c08493740 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288d17c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0cbf20 tx=0x7f2bec09b470 comp rx=0 tx=0).stop 2024-09-10T07:52:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 shutdown_connections 2024-09-10T07:52:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2c08493740 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb288dcd20 0x55cb288df1a0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2bdfe00640 0 client.29141 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2c08493740 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2c08493740 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b69d0 0x55cb288d17c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2c08493740 1 --2- 172.21.15.46:0/745747490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b6090 0x55cb288d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 >> 172.21.15.46:0/745747490 conn(0x55cb288b1700 msgr2=0x55cb288b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 shutdown_connections 2024-09-10T07:52:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.200+0000 7f2c08493740 1 -- 172.21.15.46:0/745747490 wait complete. 2024-09-10T07:52:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.201+0000 7f2c08493740 20 client.29141 trim_cache size 0 max 16384 2024-09-10T07:52:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.243+0000 7f2c08493740 1 Processor -- start 2024-09-10T07:52:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.243+0000 7f2c08493740 1 -- start start 2024-09-10T07:52:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.243+0000 7f2c08493740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.244+0000 7f2c08493740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 0x55cb288d8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.244+0000 7f2c08493740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288bba60 0x55cb288b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.244+0000 7f2c08493740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2be000fc10 con 0x55cb288b69d0 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.244+0000 7f2c08493740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00118d0 con 0x55cb288bba60 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.244+0000 7f2c08493740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00102e0 con 0x55cb288b6090 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.244+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.244+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:51062/0 (socket says 172.21.15.46:51062) 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.244+0000 7f2bfea00640 1 -- 172.21.15.46:0/4131662734 learned_addr learned my addr 172.21.15.46:0/4131662734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.245+0000 7f2bff400640 1 --2- 172.21.15.46:0/4131662734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288bba60 0x55cb288b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.245+0000 7f2bfea00640 1 -- 172.21.15.46:0/4131662734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288bba60 msgr2=0x55cb288b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.245+0000 7f2bfea00640 1 --2- 172.21.15.46:0/4131662734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288bba60 0x55cb288b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.245+0000 7f2bfea00640 1 -- 172.21.15.46:0/4131662734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 msgr2=0x55cb288d8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.245+0000 7f2bfea00640 1 --2- 172.21.15.46:0/4131662734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 0x55cb288d8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.245+0000 7f2bfea00640 1 -- 172.21.15.46:0/4131662734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb288d0680 con 0x55cb288b6090 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.248+0000 7f2bfea00640 1 --2- 172.21.15.46:0/4131662734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b2af0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55cb288df9e0 tx=0x7f2bf0009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29db902c743f2099 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.249+0000 7f2beaa00640 1 -- 172.21.15.46:0/4131662734 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00142d0 con 0x55cb288b6090 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.249+0000 7f2beaa00640 1 -- 172.21.15.46:0/4131662734 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0005500 con 0x55cb288b6090 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.249+0000 7f2beaa00640 1 -- 172.21.15.46:0/4131662734 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf001b590 con 0x55cb288b6090 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 -- 172.21.15.46:0/4131662734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 msgr2=0x55cb288b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 --2- 172.21.15.46:0/4131662734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b2af0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55cb288df9e0 tx=0x7f2bf0009550 comp rx=0 tx=0).stop 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 -- 172.21.15.46:0/4131662734 shutdown_connections 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 --2- 172.21.15.46:0/4131662734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288bba60 0x55cb288b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 --2- 172.21.15.46:0/4131662734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 0x55cb288d8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 --2- 172.21.15.46:0/4131662734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288b6090 0x55cb288b2af0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 -- 172.21.15.46:0/4131662734 >> 172.21.15.46:0/4131662734 conn(0x55cb288b1700 msgr2=0x55cb288b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 -- 172.21.15.46:0/4131662734 shutdown_connections 2024-09-10T07:52:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.250+0000 7f2c08493740 1 -- 172.21.15.46:0/4131662734 wait complete. 2024-09-10T07:52:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.251+0000 7f2c08493740 1 -- start start 2024-09-10T07:52:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.251+0000 7f2c08493740 1 Processor -- start 2024-09-10T07:52:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.253+0000 7f2c08493740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b6090 0x55cb288d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.253+0000 7f2c08493740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 0x55cb288d6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.253+0000 7f2c08493740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.253+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b6090 0x55cb288d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.253+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b6090 0x55cb288d5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52042/0 (socket says 172.21.15.46:52042) 2024-09-10T07:52:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.254+0000 7f2bff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.254+0000 7f2bff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:51066/0 (socket says 172.21.15.46:51066) 2024-09-10T07:52:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.255+0000 7f2c08493740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0011140 con 0x55cb288b69d0 2024-09-10T07:52:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.255+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0010a70 con 0x55cb288b6090 2024-09-10T07:52:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.255+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2be000dbf0 con 0x55cb288bba60 2024-09-10T07:52:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.255+0000 7f2bfea00640 1 -- 172.21.15.46:0/650388148 learned_addr learned my addr 172.21.15.46:0/650388148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.256+0000 7f2c08493740 10 client.0 _ll_register_callbacks cb 0x7f2bec096ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.257+0000 7f2bfe000640 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 0x55cb288d6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.257+0000 7f2bff400640 1 -- 172.21.15.46:0/650388148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b6090 msgr2=0x55cb288d5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.257+0000 7f2bff400640 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b6090 0x55cb288d5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.257+0000 7f2bff400640 1 -- 172.21.15.46:0/650388148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 msgr2=0x55cb288d6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.257+0000 7f2bff400640 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 0x55cb288d6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.257+0000 7f2bff400640 1 -- 172.21.15.46:0/650388148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb288d7b30 con 0x55cb288bba60 2024-09-10T07:52:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.259+0000 7f2bfea00640 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b6090 0x55cb288d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.259+0000 7f2bff400640 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d7520 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80319f0 tx=0x7f2bf8036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f825e1459d3d7f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.259+0000 7f2bdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.260+0000 7f2bdfe00640 1 -- 172.21.15.46:0/650388148 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8060020 con 0x55cb288bba60 2024-09-10T07:52:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.260+0000 7f2bdfe00640 1 -- 172.21.15.46:0/650388148 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2bf8003970 con 0x55cb288bba60 2024-09-10T07:52:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.260+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf0005500 con 0x55cb288bba60 2024-09-10T07:52:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.260+0000 7f2bdfe00640 1 -- 172.21.15.46:0/650388148 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2bf806b050 con 0x55cb288bba60 2024-09-10T07:52:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.260+0000 7f2c08493740 10 client.29023 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.260+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55cb28947c60 con 0x55cb288bba60 2024-09-10T07:52:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.260+0000 7f2bdfe00640 1 -- 172.21.15.46:0/650388148 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0005500 con 0x55cb288bba60 2024-09-10T07:52:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.261+0000 7f2bdfe00640 1 -- 172.21.15.46:0/650388148 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f2bf8003c00 con 0x55cb288bba60 2024-09-10T07:52:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.262+0000 7f2c08493740 10 client.29023 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.262+0000 7f2c08493740 10 client.29023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.262+0000 7f2c08493740 20 client.29023 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.262+0000 7f2c08493740 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb289482d0 0x55cb2894a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.262+0000 7f2c08493740 4 client.29023 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:52:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.262+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb28822b80 con 0x55cb289482d0 2024-09-10T07:52:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.263+0000 7f2bfea00640 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb289482d0 0x55cb2894a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.264+0000 7f2bfea00640 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb289482d0 0x55cb2894a6c0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3010d671829ac5e9 server_cookie=3a5d11b37ef6b78d in_seq=0 out_seq=0 2024-09-10T07:52:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.264+0000 7f2bdfe00640 10 client.29023 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.284+0000 7f2bdfe00640 1 -- 172.21.15.46:0/650388148 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb28822b80 con 0x55cb289482d0 2024-09-10T07:52:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.284+0000 7f2bdfe00640 10 client.29023 handle_command_reply: tid=0 2024-09-10T07:52:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.284+0000 7f2c08493740 1 client.29023 shutdown 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.284+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 msgr2=0x55cb288d7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.284+0000 7f2c08493740 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d7520 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80319f0 tx=0x7f2bf8036000 comp rx=0 tx=0).stop 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.285+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 shutdown_connections 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.285+0000 7f2c08493740 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55cb289482d0 0x55cb2894a6c0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.285+0000 7f2c08493740 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cb288bba60 0x55cb288d7520 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.285+0000 7f2c08493740 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb288b69d0 0x55cb288d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.285+0000 7f2c08493740 1 --2- 172.21.15.46:0/650388148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb288b6090 0x55cb288d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.285+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 >> 172.21.15.46:0/650388148 conn(0x55cb288b1700 msgr2=0x55cb288b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.285+0000 7f2bdfe00640 0 client.29023 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.286+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 shutdown_connections 2024-09-10T07:52:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.286+0000 7f2c08493740 1 -- 172.21.15.46:0/650388148 wait complete. 2024-09-10T07:52:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.287+0000 7f2c08493740 20 client.29023 trim_cache size 0 max 16384 2024-09-10T07:52:02.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:02 smithi181 ceph-mon[35397]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.0 MiB/s wr, 410 op/s 2024-09-10T07:52:02.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:02 smithi063 ceph-mon[35020]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.0 MiB/s wr, 410 op/s 2024-09-10T07:52:02.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:02 smithi046 ceph-mon[30705]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.0 MiB/s wr, 410 op/s 2024-09-10T07:52:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:52:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:52:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:52:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:52:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.622+0000 7f2c05600640 1 -- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2be40a2cb0 msgr2=0x7f2be40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.623+0000 7f2c05600640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2be40a2cb0 0x7f2be40a5170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8047000 tx=0x7f2bf8030c80 comp rx=0 tx=0).stop 2024-09-10T07:52:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.623+0000 7f2c05600640 1 -- 172.21.15.46:0/2082304210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00101c40 msgr2=0x7f2c0019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.623+0000 7f2c05600640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00101c40 0x7f2c0019e920 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00b4a0 tx=0x7f2bec00b960 comp rx=0 tx=0).stop 2024-09-10T07:52:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.624+0000 7f2c05600640 1 -- 172.21.15.46:0/2082304210 shutdown_connections 2024-09-10T07:52:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.624+0000 7f2c05600640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c00102600 0x7f2c001a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.624+0000 7f2c05600640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c00101c40 0x7f2c0019e920 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.624+0000 7f2c05600640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f2be40a2cb0 0x7f2be40a5170 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.624+0000 7f2c05600640 1 --2- 172.21.15.46:0/2082304210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00100a50 0x7f2c0019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.624+0000 7f2c05600640 1 -- 172.21.15.46:0/2082304210 >> 172.21.15.46:0/2082304210 conn(0x7f2c000fc1b0 msgr2=0x7f2c00102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.624+0000 7f2c05600640 1 -- 172.21.15.46:0/2082304210 shutdown_connections 2024-09-10T07:52:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:02.624+0000 7f2c05600640 1 -- 172.21.15.46:0/2082304210 wait complete. 2024-09-10T07:52:02.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.467+0000 7f8d01800640 1 -- 172.21.15.46:0/2754398345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc101380 msgr2=0x7f8cfc10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.467+0000 7f8d01800640 1 --2- 172.21.15.46:0/2754398345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc101380 0x7f8cfc10dfb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009900 tx=0x7f8ce802a410 comp rx=0 tx=0).stop 2024-09-10T07:52:02.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.469+0000 7f8d01800640 1 -- 172.21.15.46:0/2754398345 shutdown_connections 2024-09-10T07:52:02.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.469+0000 7f8d01800640 1 --2- 172.21.15.46:0/2754398345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cfc10e4f0 0x7f8cfc110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.469+0000 7f8d01800640 1 --2- 172.21.15.46:0/2754398345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc101380 0x7f8cfc10dfb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.469+0000 7f8d01800640 1 --2- 172.21.15.46:0/2754398345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cfc1009d0 0x7f8cfc100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.469+0000 7f8d01800640 1 -- 172.21.15.46:0/2754398345 >> 172.21.15.46:0/2754398345 conn(0x7f8cfc0fc7d0 msgr2=0x7f8cfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.470+0000 7f8d01800640 1 -- 172.21.15.46:0/2754398345 shutdown_connections 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.470+0000 7f8d01800640 1 -- 172.21.15.46:0/2754398345 wait complete. 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.474+0000 7f8d01800640 1 Processor -- start 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.474+0000 7f8d01800640 1 -- start start 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.475+0000 7f8d01800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc1009d0 0x7f8cfc107f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.475+0000 7f8d01800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cfc101380 0x7f8cfc108470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.475+0000 7f8d01800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cfc10e4f0 0x7f8cfc10dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8cfaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc1009d0 0x7f8cfc107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8cfaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc1009d0 0x7f8cfc107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:51078/0 (socket says 172.21.15.46:51078) 2024-09-10T07:52:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8cfa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cfc101380 0x7f8cfc108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8cfa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cfc101380 0x7f8cfc108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52046/0 (socket says 172.21.15.46:52046) 2024-09-10T07:52:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8d01800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc06bfa0 con 0x7f8cfc10e4f0 2024-09-10T07:52:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8d01800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc06be20 con 0x7f8cfc101380 2024-09-10T07:52:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8d01800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc06c120 con 0x7f8cfc1009d0 2024-09-10T07:52:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8cfaa00640 1 -- 172.21.15.46:0/748863591 learned_addr learned my addr 172.21.15.46:0/748863591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.476+0000 7f8cfb400640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cfc10e4f0 0x7f8cfc10dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.477+0000 7f8cfaa00640 1 -- 172.21.15.46:0/748863591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cfc101380 msgr2=0x7f8cfc108470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.477+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cfc101380 0x7f8cfc108470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.477+0000 7f8cfaa00640 1 -- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cfc10e4f0 msgr2=0x7f8cfc10dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.477+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cfc10e4f0 0x7f8cfc10dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.477+0000 7f8cfaa00640 1 -- 172.21.15.46:0/748863591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc103080 con 0x7f8cfc1009d0 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.477+0000 7f8cfa000640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cfc101380 0x7f8cfc108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.477+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc1009d0 0x7f8cfc107f30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00b450 tx=0x7f8cec00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.478+0000 7f8cf3400640 1 -- 172.21.15.46:0/748863591 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec015020 con 0x7f8cfc1009d0 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.478+0000 7f8d01800640 1 -- 172.21.15.46:0/748863591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc103310 con 0x7f8cfc1009d0 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.478+0000 7f8d01800640 1 -- 172.21.15.46:0/748863591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc103820 con 0x7f8cfc1009d0 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.478+0000 7f8cf3400640 1 -- 172.21.15.46:0/748863591 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cec011d70 con 0x7f8cfc1009d0 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.478+0000 7f8cf3400640 1 -- 172.21.15.46:0/748863591 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00f340 con 0x7f8cfc1009d0 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.484+0000 7f8cf3400640 1 -- 172.21.15.46:0/748863591 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8cec018020 con 0x7f8cfc1009d0 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.486+0000 7f8cf3400640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ce00a2b80 0x7f8ce00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.486+0000 7f8cf3400640 1 -- 172.21.15.46:0/748863591 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8cec0cece0 con 0x7f8cfc1009d0 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.487+0000 7f8cfa000640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ce00a2b80 0x7f8ce00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.487+0000 7f8d0429b740 1 Processor -- start 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.489+0000 7f8d0429b740 1 -- start start 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.489+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbb4e0 0x558261fbb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.490+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbbe00 0x558261fc6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.490+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x558261fc8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.490+0000 7f8d0429b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558261f20260 con 0x558261fc66c0 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.490+0000 7f8d0429b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558261ec7500 con 0x558261fbb4e0 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.490+0000 7f8d0429b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558261f1c220 con 0x558261fbbe00 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x558261fc8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x558261fc8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43090/0 (socket says 172.21.15.46:43090) 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfb400640 1 -- 172.21.15.46:0/355206382 learned_addr learned my addr 172.21.15.46:0/355206382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/355206382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbb4e0 0x558261fbb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfaa00640 1 -- 172.21.15.46:0/355206382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbbe00 msgr2=0x558261fc6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/355206382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbbe00 0x558261fc6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfaa00640 1 -- 172.21.15.46:0/355206382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 msgr2=0x558261fc8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/355206382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x558261fc8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfaa00640 1 -- 172.21.15.46:0/355206382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558261c91ba0 con 0x558261fbb4e0 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.491+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/355206382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbb4e0 0x558261fbb8c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00be10 tx=0x7f8cec09abe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b98404362ecebd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.492+0000 7f8ce7e00640 1 -- 172.21.15.46:0/355206382 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0a3e30 con 0x558261fbb4e0 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.492+0000 7f8ce7e00640 1 -- 172.21.15.46:0/355206382 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cec099260 con 0x558261fbb4e0 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.492+0000 7f8ce7e00640 1 -- 172.21.15.46:0/355206382 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0bb070 con 0x558261fbb4e0 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8d0429b740 1 -- 172.21.15.46:0/355206382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbb4e0 msgr2=0x558261fbb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8cfa000640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ce00a2b80 0x7f8ce00a5040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8005ab0 tx=0x7f8ce8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8d0429b740 1 --2- 172.21.15.46:0/355206382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbb4e0 0x558261fbb8c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00be10 tx=0x7f8cec09abe0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8d0429b740 1 -- 172.21.15.46:0/355206382 shutdown_connections 2024-09-10T07:52:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8d0429b740 1 --2- 172.21.15.46:0/355206382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x558261fc8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8d0429b740 1 --2- 172.21.15.46:0/355206382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbbe00 0x558261fc6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8d0429b740 1 --2- 172.21.15.46:0/355206382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbb4e0 0x558261fbb8c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8d0429b740 1 -- 172.21.15.46:0/355206382 >> 172.21.15.46:0/355206382 conn(0x558261fb7a70 msgr2=0x558261fb7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.493+0000 7f8d0429b740 1 -- 172.21.15.46:0/355206382 shutdown_connections 2024-09-10T07:52:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.494+0000 7f8d0429b740 1 -- 172.21.15.46:0/355206382 wait complete. 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.494+0000 7f8d0429b740 1 -- start start 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.495+0000 7f8d0429b740 1 Processor -- start 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.497+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbb4e0 0x558261fc35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.498+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 0x558261fc3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.498+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fc66c0 0x558261fc4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.498+0000 7f8d0429b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558261fc9290 con 0x558261fbb4e0 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.498+0000 7f8d0429b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558261edabd0 con 0x558261fbbe00 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.498+0000 7f8d0429b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558261fc9410 con 0x558261fc66c0 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.498+0000 7f8cfb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fc66c0 0x558261fc4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8cfb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fc66c0 0x558261fc4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:51100/0 (socket says 172.21.15.46:51100) 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8cfb400640 1 -- 172.21.15.46:0/297142043 learned_addr learned my addr 172.21.15.46:0/297142043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8d0429b740 10 client.0 _ll_register_callbacks cb 0x558261c2f0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8cfb400640 1 -- 172.21.15.46:0/297142043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 msgr2=0x558261fc3b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8cfb400640 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 0x558261fc3b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8cfb400640 1 -- 172.21.15.46:0/297142043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbb4e0 msgr2=0x558261fc35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8cfb400640 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbb4e0 0x558261fc35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8cfb400640 1 -- 172.21.15.46:0/297142043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558261fd2700 con 0x558261fc66c0 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.499+0000 7f8cfb400640 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fc66c0 0x558261fc4040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4011710 tx=0x7f8cf4011bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0bd373c29f911f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.500+0000 7f8ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.500+0000 7f8ce4c00640 1 -- 172.21.15.46:0/297142043 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400fae0 con 0x558261fc66c0 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.500+0000 7f8ce4c00640 1 -- 172.21.15.46:0/297142043 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8cf400fc80 con 0x558261fc66c0 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.501+0000 7f8d0429b740 1 -- 172.21.15.46:0/297142043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cec099260 con 0x558261fc66c0 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.501+0000 7f8ce4c00640 1 -- 172.21.15.46:0/297142043 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8cf400fe80 con 0x558261fc66c0 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.501+0000 7f8d0429b740 10 client.29032 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.501+0000 7f8d0429b740 1 -- 172.21.15.46:0/297142043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x558261fd5120 con 0x558261fc66c0 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.501+0000 7f8ce4c00640 1 -- 172.21.15.46:0/297142043 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cec099260 con 0x558261fc66c0 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.502+0000 7f8ce4c00640 1 -- 172.21.15.46:0/297142043 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f8cf40126b0 con 0x558261fc66c0 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.504+0000 7f8d0429b740 10 client.29032 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.504+0000 7f8d0429b740 10 client.29032 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.504+0000 7f8d0429b740 20 client.29032 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.504+0000 7f8d0429b740 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558261fd6af0 0x558261fd8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.504+0000 7f8d0429b740 4 client.29032 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.504+0000 7f8d0429b740 1 -- 172.21.15.46:0/297142043 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558261f2d990 con 0x558261fd6af0 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.506+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558261fd6af0 0x558261fd8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.507+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558261fd6af0 0x558261fd8fb0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def193ca90790edb server_cookie=84ea5523554fd8ff in_seq=0 out_seq=0 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.507+0000 7f8ce4c00640 10 client.29032 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.513+0000 7f8ce4c00640 1 -- 172.21.15.46:0/297142043 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558261f2d990 con 0x558261fd6af0 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.513+0000 7f8ce4c00640 10 client.29032 handle_command_reply: tid=0 2024-09-10T07:52:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.513+0000 7f8d0429b740 1 client.29032 shutdown 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.514+0000 7f8d0429b740 1 -- 172.21.15.46:0/297142043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fc66c0 msgr2=0x558261fc4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.514+0000 7f8d0429b740 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fc66c0 0x558261fc4040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4011710 tx=0x7f8cf4011bd0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.515+0000 7f8d0429b740 1 -- 172.21.15.46:0/297142043 shutdown_connections 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.515+0000 7f8d0429b740 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558261fd6af0 0x558261fd8fb0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.515+0000 7f8d0429b740 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fc66c0 0x558261fc4040 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.515+0000 7f8ce4c00640 0 client.29032 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.515+0000 7f8d0429b740 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 0x558261fc3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.515+0000 7f8d0429b740 1 --2- 172.21.15.46:0/297142043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbb4e0 0x558261fc35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.515+0000 7f8d0429b740 1 -- 172.21.15.46:0/297142043 >> 172.21.15.46:0/297142043 conn(0x558261fb7a70 msgr2=0x558261fbd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.517+0000 7f8d0429b740 1 -- 172.21.15.46:0/297142043 shutdown_connections 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.517+0000 7f8d0429b740 1 -- 172.21.15.46:0/297142043 wait complete. 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.517+0000 7f8d0429b740 20 client.29032 trim_cache size 0 max 16384 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.555+0000 7f8d0429b740 1 Processor -- start 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.556+0000 7f8d0429b740 1 -- start start 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fbdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbbe00 0x558261fb6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fc66c0 0x558261fde8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8d0429b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1bdac0 con 0x558261fbbe00 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8d0429b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1bf780 con 0x558261fc66c0 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8d0429b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1be190 con 0x558261fbb4e0 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8cfaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fbdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8cfb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fc66c0 0x558261fde8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8cfaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fbdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:51102/0 (socket says 172.21.15.46:51102) 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.557+0000 7f8cfaa00640 1 -- 172.21.15.46:0/678253144 learned_addr learned my addr 172.21.15.46:0/678253144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.558+0000 7f8cfb400640 1 -- 172.21.15.46:0/678253144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 msgr2=0x558261fbdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.558+0000 7f8cfb400640 1 --2- 172.21.15.46:0/678253144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fbdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.558+0000 7f8cfa000640 1 --2- 172.21.15.46:0/678253144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbbe00 0x558261fb6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.558+0000 7f8cfb400640 1 -- 172.21.15.46:0/678253144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbbe00 msgr2=0x558261fb6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.558+0000 7f8cfb400640 1 --2- 172.21.15.46:0/678253144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbbe00 0x558261fb6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.558+0000 7f8cfb400640 1 -- 172.21.15.46:0/678253144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558261fd4f50 con 0x558261fc66c0 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.558+0000 7f8cfb400640 1 --2- 172.21.15.46:0/678253144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fc66c0 0x558261fde8c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4012b50 tx=0x7f8cf4012f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19aebec358218b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.559+0000 7f8ce7400640 1 -- 172.21.15.46:0/678253144 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf402c070 con 0x558261fc66c0 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.559+0000 7f8ce7400640 1 -- 172.21.15.46:0/678253144 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf4003e50 con 0x558261fc66c0 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.559+0000 7f8ce7400640 1 -- 172.21.15.46:0/678253144 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf402c070 con 0x558261fc66c0 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.560+0000 7f8d0429b740 1 -- 172.21.15.46:0/678253144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fc66c0 msgr2=0x558261fde8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.560+0000 7f8d0429b740 1 --2- 172.21.15.46:0/678253144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fc66c0 0x558261fde8c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4012b50 tx=0x7f8cf4012f50 comp rx=0 tx=0).stop 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.560+0000 7f8d0429b740 1 -- 172.21.15.46:0/678253144 shutdown_connections 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.560+0000 7f8d0429b740 1 --2- 172.21.15.46:0/678253144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fc66c0 0x558261fde8c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.560+0000 7f8d0429b740 1 --2- 172.21.15.46:0/678253144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fbbe00 0x558261fb6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.560+0000 7f8d0429b740 1 --2- 172.21.15.46:0/678253144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fbdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.560+0000 7f8d0429b740 1 -- 172.21.15.46:0/678253144 >> 172.21.15.46:0/678253144 conn(0x558261fb7a70 msgr2=0x558261fb63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.562+0000 7f8d0429b740 1 -- 172.21.15.46:0/678253144 shutdown_connections 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.562+0000 7f8d0429b740 1 -- 172.21.15.46:0/678253144 wait complete. 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.562+0000 7f8d0429b740 1 -- start start 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.563+0000 7f8d0429b740 1 Processor -- start 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.565+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fcac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.565+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 0x558261fcb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.565+0000 7f8d0429b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x55826203a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.566+0000 7f8d0429b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1beff0 con 0x558261fc66c0 2024-09-10T07:52:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.566+0000 7f8d0429b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1be920 con 0x558261fbbe00 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.566+0000 7f8d0429b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1bbaa0 con 0x558261fbb4e0 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.566+0000 7f8d0429b740 10 client.0 _ll_register_callbacks cb 0x7f8cf4009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.566+0000 7f8cfa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 0x558261fcb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.566+0000 7f8cfa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 0x558261fcb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52078/0 (socket says 172.21.15.46:52078) 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.566+0000 7f8cfa000640 1 -- 172.21.15.46:0/2389455703 learned_addr learned my addr 172.21.15.46:0/2389455703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.567+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fcac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.568+0000 7f8cfb400640 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x55826203a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.568+0000 7f8cfaa00640 1 -- 172.21.15.46:0/2389455703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 msgr2=0x558261fcb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.568+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 0x558261fcb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.569+0000 7f8cfaa00640 1 -- 172.21.15.46:0/2389455703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 msgr2=0x55826203a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.569+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x55826203a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.569+0000 7f8cfaa00640 1 -- 172.21.15.46:0/2389455703 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558261fb1fe0 con 0x558261fbb4e0 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.572+0000 7f8cfaa00640 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fcac70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0cc4b0 tx=0x7f8cec097590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b93ebfab283af548 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.572+0000 7f8ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.579+0000 7f8ce4c00640 1 -- 172.21.15.46:0/2389455703 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec084020 con 0x558261fbb4e0 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.579+0000 7f8ce4c00640 1 -- 172.21.15.46:0/2389455703 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8cec0b9e00 con 0x558261fbb4e0 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.579+0000 7f8d0429b740 1 -- 172.21.15.46:0/2389455703 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf4003e50 con 0x558261fbb4e0 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.580+0000 7f8ce4c00640 1 -- 172.21.15.46:0/2389455703 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8cec0a3e30 con 0x558261fbb4e0 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.580+0000 7f8d0429b740 10 client.29038 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.580+0000 7f8d0429b740 1 -- 172.21.15.46:0/2389455703 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5582620420c0 con 0x558261fbb4e0 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.580+0000 7f8ce4c00640 1 -- 172.21.15.46:0/2389455703 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf4003e50 con 0x558261fbb4e0 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.581+0000 7f8ce4c00640 1 -- 172.21.15.46:0/2389455703 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f8cec09f160 con 0x558261fbb4e0 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.581+0000 7f8d0429b740 10 client.29038 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.581+0000 7f8d0429b740 10 client.29038 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.581+0000 7f8d0429b740 20 client.29038 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.581+0000 7f8d0429b740 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558262042460 0x558262044850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.581+0000 7f8d0429b740 4 client.29038 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-09-10T07:52:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.581+0000 7f8d0429b740 1 -- 172.21.15.46:0/2389455703 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x558261f27ce0 con 0x558262042460 2024-09-10T07:52:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.585+0000 7f8cfa000640 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558262042460 0x558262044850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.587+0000 7f8cfa000640 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558262042460 0x558262044850 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af2880f4dbb566ae server_cookie=8ecc05fdae0cd1cb in_seq=0 out_seq=0 2024-09-10T07:52:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.587+0000 7f8ce4c00640 10 client.29038 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.818+0000 7f8ce4c00640 1 -- 172.21.15.46:0/2389455703 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558261f27ce0 con 0x558262042460 2024-09-10T07:52:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.818+0000 7f8ce4c00640 10 client.29038 handle_command_reply: tid=0 2024-09-10T07:52:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.818+0000 7f8d0429b740 1 client.29038 shutdown 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.818+0000 7f8d0429b740 1 -- 172.21.15.46:0/2389455703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 msgr2=0x558261fcac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fcac70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0cc4b0 tx=0x7f8cec097590 comp rx=0 tx=0).stop 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 -- 172.21.15.46:0/2389455703 shutdown_connections 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x558262042460 0x558262044850 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558261fc66c0 0x55826203a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558261fbbe00 0x558261fcb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8ce4c00640 0 client.29038 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 --2- 172.21.15.46:0/2389455703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558261fbb4e0 0x558261fcac70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 -- 172.21.15.46:0/2389455703 >> 172.21.15.46:0/2389455703 conn(0x558261fb7a70 msgr2=0x558261fb6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 -- 172.21.15.46:0/2389455703 shutdown_connections 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 1 -- 172.21.15.46:0/2389455703 wait complete. 2024-09-10T07:52:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.819+0000 7f8d0429b740 20 client.29038 trim_cache size 0 max 16384 2024-09-10T07:52:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:52:02.820 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 60, 2024-09-10T07:52:02.820 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:52:02.820 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 54, 2024-09-10T07:52:02.820 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:52:02.820 INFO:teuthology.orchestra.run.smithi046.stdout: "67ec92b3": { 2024-09-10T07:52:02.820 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 54, 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 139.8, 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:52:02.821 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:02.822 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:52:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 -- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ce00a2b80 msgr2=0x7f8ce00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ce00a2b80 0x7f8ce00a5040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8005ab0 tx=0x7f8ce8005850 comp rx=0 tx=0).stop 2024-09-10T07:52:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 -- 172.21.15.46:0/748863591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc1009d0 msgr2=0x7f8cfc107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc1009d0 0x7f8cfc107f30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00b450 tx=0x7f8cec00b910 comp rx=0 tx=0).stop 2024-09-10T07:52:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 -- 172.21.15.46:0/748863591 shutdown_connections 2024-09-10T07:52:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8ce00a2b80 0x7f8ce00a5040 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cfc10e4f0 0x7f8cfc10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cfc101380 0x7f8cfc108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 --2- 172.21.15.46:0/748863591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cfc1009d0 0x7f8cfc107f30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.820+0000 7f8d01800640 1 -- 172.21.15.46:0/748863591 >> 172.21.15.46:0/748863591 conn(0x7f8cfc0fc7d0 msgr2=0x7f8cfc110670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.821+0000 7f8d01800640 1 -- 172.21.15.46:0/748863591 shutdown_connections 2024-09-10T07:52:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:02.821+0000 7f8d01800640 1 -- 172.21.15.46:0/748863591 wait complete. 2024-09-10T07:52:03.629 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '67ec92b3', quiesce took 11.7 seconds. Will release after: 8.150000000000002 2024-09-10T07:52:04.204 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:04 smithi046 ceph-mon[30705]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.221+0000 7f30a0a00640 1 -- 172.21.15.46:0/3033894499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f309c1049a0 msgr2=0x7f309c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.221+0000 7f30a0a00640 1 --2- 172.21.15.46:0/3033894499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f309c1049a0 0x7f309c1089f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3084009860 tx=0x7f308402a340 comp rx=0 tx=0).stop 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.221+0000 7f30a0a00640 1 -- 172.21.15.46:0/3033894499 shutdown_connections 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.221+0000 7f30a0a00640 1 --2- 172.21.15.46:0/3033894499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c109120 0x7f309c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.221+0000 7f30a0a00640 1 --2- 172.21.15.46:0/3033894499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f309c1049a0 0x7f309c1089f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.221+0000 7f30a0a00640 1 --2- 172.21.15.46:0/3033894499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f309c103ff0 0x7f309c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.221+0000 7f30a0a00640 1 -- 172.21.15.46:0/3033894499 >> 172.21.15.46:0/3033894499 conn(0x7f309c0ffa90 msgr2=0x7f309c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.222+0000 7f30a0a00640 1 -- 172.21.15.46:0/3033894499 shutdown_connections 2024-09-10T07:52:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.222+0000 7f30a0a00640 1 -- 172.21.15.46:0/3033894499 wait complete. 2024-09-10T07:52:04.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.223+0000 7f30a0a00640 1 Processor -- start 2024-09-10T07:52:04.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.223+0000 7f30a0a00640 1 -- start start 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.223+0000 7f30a0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c103ff0 0x7f309c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.223+0000 7f30a0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f309c1049a0 0x7f309c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f30a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f309c109120 0x7f309c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f30a0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f309c114120 con 0x7f309c103ff0 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f30a0a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f309c113fa0 con 0x7f309c109120 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f30a0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1142a0 con 0x7f309c1049a0 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3099e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c103ff0 0x7f309c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3099e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c103ff0 0x7f309c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43142/0 (socket says 172.21.15.46:43142) 2024-09-10T07:52:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3099e00640 1 -- 172.21.15.46:0/190168442 learned_addr learned my addr 172.21.15.46:0/190168442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:04.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f309a800640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f309c109120 0x7f309c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3093e00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f309c1049a0 0x7f309c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3099e00640 1 -- 172.21.15.46:0/190168442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f309c1049a0 msgr2=0x7f309c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3099e00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f309c1049a0 0x7f309c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3099e00640 1 -- 172.21.15.46:0/190168442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f309c109120 msgr2=0x7f309c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3099e00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f309c109120 0x7f309c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3099e00640 1 -- 172.21.15.46:0/190168442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c19f670 con 0x7f309c103ff0 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.224+0000 7f3093e00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f309c1049a0 0x7f309c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.225+0000 7f3099e00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c103ff0 0x7f309c19a6c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f308c00e9f0 tx=0x7f308c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.225+0000 7f3092a00640 1 -- 172.21.15.46:0/190168442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00cd80 con 0x7f309c103ff0 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.225+0000 7f3092a00640 1 -- 172.21.15.46:0/190168442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308c010910 con 0x7f309c103ff0 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.225+0000 7f3092a00640 1 -- 172.21.15.46:0/190168442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c018990 con 0x7f309c103ff0 2024-09-10T07:52:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.225+0000 7f30a0a00640 1 -- 172.21.15.46:0/190168442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309c19f960 con 0x7f309c103ff0 2024-09-10T07:52:04.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.225+0000 7f30a0a00640 1 -- 172.21.15.46:0/190168442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309c1a7240 con 0x7f309c103ff0 2024-09-10T07:52:04.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.228+0000 7f30a34b1740 1 Processor -- start 2024-09-10T07:52:04.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.228+0000 7f3092a00640 1 -- 172.21.15.46:0/190168442 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f308c010ac0 con 0x7f309c103ff0 2024-09-10T07:52:04.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.228+0000 7f30a34b1740 1 -- start start 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e2590 0x5638718ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718ece50 0x5638718ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f30a34b1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5638718483e0 con 0x5638718e2590 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f30a34b1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5638717efd80 con 0x5638718e1c70 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f30a34b1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5638718443a0 con 0x5638718ece50 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f3099e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e2590 0x5638718ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f3099e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e2590 0x5638718ec910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43150/0 (socket says 172.21.15.46:43150) 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f3099e00640 1 -- 172.21.15.46:0/2586471411 learned_addr learned my addr 172.21.15.46:0/2586471411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f309a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.229+0000 7f3093e00640 1 --2- 172.21.15.46:0/2586471411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718ece50 0x5638718ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f309a800640 1 -- 172.21.15.46:0/2586471411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718ece50 msgr2=0x5638718ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f309a800640 1 --2- 172.21.15.46:0/2586471411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718ece50 0x5638718ef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f309a800640 1 -- 172.21.15.46:0/2586471411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e2590 msgr2=0x5638718ec910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f309a800640 1 --2- 172.21.15.46:0/2586471411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e2590 0x5638718ec910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f309a800640 1 -- 172.21.15.46:0/2586471411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387180eae0 con 0x5638718e1c70 2024-09-10T07:52:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f309a800640 1 --2- 172.21.15.46:0/2586471411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e2050 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3094005bd0 tx=0x7f30940095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a433c862f6854b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f308b000640 1 -- 172.21.15.46:0/2586471411 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094004b20 con 0x5638718e1c70 2024-09-10T07:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f308b000640 1 -- 172.21.15.46:0/2586471411 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094003e50 con 0x5638718e1c70 2024-09-10T07:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f308b000640 1 -- 172.21.15.46:0/2586471411 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094004020 con 0x5638718e1c70 2024-09-10T07:52:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.230+0000 7f3092a00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30800a2b80 0x7f30800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f3092a00640 1 -- 172.21.15.46:0/190168442 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f308c0cf730 con 0x7f309c103ff0 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f3099e00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30800a2b80 0x7f30800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 -- 172.21.15.46:0/2586471411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 msgr2=0x5638718e2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2586471411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e2050 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3094005bd0 tx=0x7f30940095c0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 -- 172.21.15.46:0/2586471411 shutdown_connections 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2586471411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718ece50 0x5638718ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2586471411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e2590 0x5638718ec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2586471411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e2050 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 -- 172.21.15.46:0/2586471411 >> 172.21.15.46:0/2586471411 conn(0x5638718de200 msgr2=0x5638718de5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 -- 172.21.15.46:0/2586471411 shutdown_connections 2024-09-10T07:52:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.231+0000 7f30a34b1740 1 -- 172.21.15.46:0/2586471411 wait complete. 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.232+0000 7f3099e00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30800a2b80 0x7f30800a5040 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f308c0cf9d0 tx=0x7f308c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.232+0000 7f30a34b1740 1 -- start start 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.232+0000 7f30a34b1740 1 Processor -- start 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e1c70 0x5638718e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e2590 0x5638718ea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x5638718ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f30a34b1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5638718efa20 con 0x5638718ece50 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f30a34b1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5638718030f0 con 0x5638718e2590 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f30a34b1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5638718efba0 con 0x5638718e1c70 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f30a34b1740 10 client.0 _ll_register_callbacks cb 0x5638717eabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f3093e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e2590 0x5638718ea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f309a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e1c70 0x5638718e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f3093e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e2590 0x5638718ea280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52120/0 (socket says 172.21.15.46:52120) 2024-09-10T07:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f3093e00640 1 -- 172.21.15.46:0/2841690553 learned_addr learned my addr 172.21.15.46:0/2841690553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:04.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f309a800640 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x5638718ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f3093e00640 1 -- 172.21.15.46:0/2841690553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e1c70 msgr2=0x5638718e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f3093e00640 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e1c70 0x5638718e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f3093e00640 1 -- 172.21.15.46:0/2841690553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 msgr2=0x5638718ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f3093e00640 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x5638718ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f3093e00640 1 -- 172.21.15.46:0/2841690553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638715b93c0 con 0x5638718e2590 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.233+0000 7f309a800640 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x5638718ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.234+0000 7f3093e00640 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e2590 0x5638718ea280 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3084031000 tx=0x7f30840045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1910798b2cf5e030 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.234+0000 7f307f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.234+0000 7f307f400640 1 -- 172.21.15.46:0/2841690553 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084037040 con 0x5638718e2590 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.234+0000 7f30a34b1740 1 -- 172.21.15.46:0/2841690553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3094003e50 con 0x5638718e2590 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.234+0000 7f307f400640 1 -- 172.21.15.46:0/2841690553 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3084030960 con 0x5638718e2590 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.234+0000 7f307f400640 1 -- 172.21.15.46:0/2841690553 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f308404c050 con 0x5638718e2590 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f30a34b1740 10 client.29171 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:04.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f30a34b1740 1 -- 172.21.15.46:0/2841690553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5638718fbab0 con 0x5638718e2590 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f307f400640 1 -- 172.21.15.46:0/2841690553 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094003e50 con 0x5638718e2590 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f307f400640 1 -- 172.21.15.46:0/2841690553 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f308403d7c0 con 0x5638718e2590 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f30a34b1740 10 client.29171 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f30a34b1740 10 client.29171 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f30a34b1740 20 client.29171 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5638718fcfb0 0x5638718ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f30a34b1740 4 client.29171 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.235+0000 7f30a34b1740 1 -- 172.21.15.46:0/2841690553 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563871855950 con 0x5638718fcfb0 2024-09-10T07:52:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.236+0000 7f309a800640 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5638718fcfb0 0x5638718ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.236+0000 7f309a800640 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5638718fcfb0 0x5638718ff3d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b081149d12ff5e6 server_cookie=bfec66adf2e38a5a in_seq=0 out_seq=0 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.236+0000 7f307f400640 10 client.29171 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f307f400640 1 -- 172.21.15.46:0/2841690553 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563871855950 con 0x5638718fcfb0 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f307f400640 10 client.29171 handle_command_reply: tid=0 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 client.29171 shutdown 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 -- 172.21.15.46:0/2841690553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e2590 msgr2=0x5638718ea280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e2590 0x5638718ea280 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3084031000 tx=0x7f30840045f0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 -- 172.21.15.46:0/2841690553 shutdown_connections 2024-09-10T07:52:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x5638718fcfb0 0x5638718ff3d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x5638718ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e2590 0x5638718ea280 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2841690553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e1c70 0x5638718e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f307f400640 0 client.29171 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:52:04.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 -- 172.21.15.46:0/2841690553 >> 172.21.15.46:0/2841690553 conn(0x5638718de200 msgr2=0x5638718dd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:04.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 -- 172.21.15.46:0/2841690553 shutdown_connections 2024-09-10T07:52:04.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.237+0000 7f30a34b1740 1 -- 172.21.15.46:0/2841690553 wait complete. 2024-09-10T07:52:04.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.238+0000 7f30a34b1740 20 client.29171 trim_cache size 0 max 16384 2024-09-10T07:52:04.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.251+0000 7f30a34b1740 1 Processor -- start 2024-09-10T07:52:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.251+0000 7f30a34b1740 1 -- start start 2024-09-10T07:52:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718ea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x563871904ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f30a34b1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b0e00 con 0x5638718ece50 2024-09-10T07:52:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f30a34b1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b2ac0 con 0x5638718e1c70 2024-09-10T07:52:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f30a34b1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b14d0 con 0x5638718e2590 2024-09-10T07:52:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f309a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x563871904ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f309a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f3093e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f3093e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718ea610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:51166/0 (socket says 172.21.15.46:51166) 2024-09-10T07:52:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f3093e00640 1 -- 172.21.15.46:0/2791025736 learned_addr learned my addr 172.21.15.46:0/2791025736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f309a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:52128/0 (socket says 172.21.15.46:52128) 2024-09-10T07:52:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f3093e00640 1 -- 172.21.15.46:0/2791025736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 msgr2=0x5638718e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f3093e00640 1 --2- 172.21.15.46:0/2791025736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f3093e00640 1 -- 172.21.15.46:0/2791025736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 msgr2=0x563871904ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f3093e00640 1 --2- 172.21.15.46:0/2791025736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x563871904ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.252+0000 7f3093e00640 1 -- 172.21.15.46:0/2791025736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638718d9770 con 0x5638718e2590 2024-09-10T07:52:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.253+0000 7f3093e00640 1 --2- 172.21.15.46:0/2791025736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718ea610 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3084000c00 tx=0x7f3084005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b251f9337e3a693 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.253+0000 7f308a600640 1 -- 172.21.15.46:0/2791025736 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084037070 con 0x5638718e2590 2024-09-10T07:52:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.253+0000 7f308a600640 1 -- 172.21.15.46:0/2791025736 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3084055070 con 0x5638718e2590 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.253+0000 7f308a600640 1 -- 172.21.15.46:0/2791025736 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084030760 con 0x5638718e2590 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 -- 172.21.15.46:0/2791025736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 msgr2=0x5638718ea610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2791025736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718ea610 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3084000c00 tx=0x7f3084005a90 comp rx=0 tx=0).stop 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 -- 172.21.15.46:0/2791025736 shutdown_connections 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2791025736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718ece50 0x563871904ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2791025736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718ea610 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2791025736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718e1c70 0x5638718e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 -- 172.21.15.46:0/2791025736 >> 172.21.15.46:0/2791025736 conn(0x5638718de200 msgr2=0x5638718ebdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 -- 172.21.15.46:0/2791025736 shutdown_connections 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 -- 172.21.15.46:0/2791025736 wait complete. 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.254+0000 7f30a34b1740 1 -- start start 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f30a34b1740 1 Processor -- start 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e1c70 0x5638718e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f30a34b1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718ece50 0x5638718e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f30a34b1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b2330 con 0x5638718e1c70 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f30a34b1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b1c60 con 0x5638718ece50 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f30a34b1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1aede0 con 0x5638718e2590 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f30a34b1740 10 client.0 _ll_register_callbacks cb 0x7f308c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f309a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e1c70 0x5638718e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.255+0000 7f309a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e1c70 0x5638718e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43168/0 (socket says 172.21.15.46:43168) 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f309a800640 1 -- 172.21.15.46:0/2610636801 learned_addr learned my addr 172.21.15.46:0/2610636801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f3093e00640 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f309a800640 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718ece50 0x5638718e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f3093e00640 1 -- 172.21.15.46:0/2610636801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718ece50 msgr2=0x5638718e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f3093e00640 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718ece50 0x5638718e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f3093e00640 1 -- 172.21.15.46:0/2610636801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e1c70 msgr2=0x5638718e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f3093e00640 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e1c70 0x5638718e6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f3093e00640 1 -- 172.21.15.46:0/2610636801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638718dee90 con 0x5638718e2590 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f3093e00640 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718e71d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5638718e2a20 tx=0x7f3084030a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c9fde6d55465fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.256+0000 7f307f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.257+0000 7f307f400640 1 -- 172.21.15.46:0/2610636801 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084037040 con 0x5638718e2590 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.257+0000 7f30a34b1740 1 -- 172.21.15.46:0/2610636801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3084055070 con 0x5638718e2590 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.257+0000 7f307f400640 1 -- 172.21.15.46:0/2610636801 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f308403d660 con 0x5638718e2590 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.257+0000 7f307f400640 1 -- 172.21.15.46:0/2610636801 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f3084069050 con 0x5638718e2590 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.257+0000 7f30a34b1740 10 client.29056 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.257+0000 7f30a34b1740 1 -- 172.21.15.46:0/2610636801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x563871967ce0 con 0x5638718e2590 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.257+0000 7f307f400640 1 -- 172.21.15.46:0/2610636801 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3084055070 con 0x5638718e2590 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f307f400640 1 -- 172.21.15.46:0/2610636801 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f3084032bc0 con 0x5638718e2590 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f30a34b1740 10 client.29056 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f30a34b1740 10 client.29056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:52:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f30a34b1740 20 client.29056 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563871968450 0x56387196a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f30a34b1740 4 client.29056 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f30a34b1740 1 -- 172.21.15.46:0/2610636801 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56387184fe60 con 0x563871968450 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f309a800640 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563871968450 0x56387196a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f309a800640 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563871968450 0x56387196a870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c211ac35243e3de0 server_cookie=7ea5590ab85e2855 in_seq=0 out_seq=0 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.258+0000 7f307f400640 10 client.29056 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f307f400640 1 -- 172.21.15.46:0/2610636801 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56387184fe60 con 0x563871968450 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f307f400640 10 client.29056 handle_command_reply: tid=0 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 client.29056 shutdown 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 -- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 msgr2=0x5638718e71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718e71d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5638718e2a20 tx=0x7f3084030a10 comp rx=0 tx=0).stop 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 -- 172.21.15.46:0/2610636801 shutdown_connections 2024-09-10T07:52:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x563871968450 0x56387196a870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5638718ece50 0x5638718e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638718e2590 0x5638718e71d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 --2- 172.21.15.46:0/2610636801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638718e1c70 0x5638718e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 -- 172.21.15.46:0/2610636801 >> 172.21.15.46:0/2610636801 conn(0x5638718de200 msgr2=0x5638718dcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f307f400640 0 client.29056 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:52:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 -- 172.21.15.46:0/2610636801 shutdown_connections 2024-09-10T07:52:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 1 -- 172.21.15.46:0/2610636801 wait complete. 2024-09-10T07:52:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.259+0000 7f30a34b1740 20 client.29056 trim_cache size 0 max 16384 2024-09-10T07:52:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:52:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 -- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30800a2b80 msgr2=0x7f30800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30800a2b80 0x7f30800a5040 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f308c0cf9d0 tx=0x7f308c0d5530 comp rx=0 tx=0).stop 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 -- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c103ff0 msgr2=0x7f309c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c103ff0 0x7f309c19a6c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f308c00e9f0 tx=0x7f308c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 -- 172.21.15.46:0/190168442 shutdown_connections 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f309c109120 0x7f309c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f309c1049a0 0x7f309c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30800a2b80 0x7f30800a5040 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 --2- 172.21.15.46:0/190168442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309c103ff0 0x7f309c19a6c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 -- 172.21.15.46:0/190168442 >> 172.21.15.46:0/190168442 conn(0x7f309c0ffa90 msgr2=0x7f309c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 -- 172.21.15.46:0/190168442 shutdown_connections 2024-09-10T07:52:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:04.260+0000 7f30a0a00640 1 -- 172.21.15.46:0/190168442 wait complete. 2024-09-10T07:52:04.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:04 smithi181 ceph-mon[35397]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-09-10T07:52:04.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:04 smithi063 ceph-mon[35020]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-09-10T07:52:05.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:52:05.441 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 885.0 s, 142.9 tps, lat 259.445 ms stddev 298.077 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 -- 172.21.15.46:0/582822146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86ec1079c0 msgr2=0x7f86ec107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 --2- 172.21.15.46:0/582822146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86ec1079c0 0x7f86ec107da0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009930 tx=0x7f86dc02a420 comp rx=0 tx=0).stop 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 -- 172.21.15.46:0/582822146 shutdown_connections 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 --2- 172.21.15.46:0/582822146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec102360 0x7f86ec10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 --2- 172.21.15.46:0/582822146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86ec1019c0 0x7f86ec101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 --2- 172.21.15.46:0/582822146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86ec1079c0 0x7f86ec107da0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 -- 172.21.15.46:0/582822146 >> 172.21.15.46:0/582822146 conn(0x7f86ec0fd740 msgr2=0x7f86ec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 -- 172.21.15.46:0/582822146 shutdown_connections 2024-09-10T07:52:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.028+0000 7f86eb200640 1 -- 172.21.15.46:0/582822146 wait complete. 2024-09-10T07:52:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.029+0000 7f86eb200640 1 Processor -- start 2024-09-10T07:52:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.029+0000 7f86eb200640 1 -- start start 2024-09-10T07:52:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86eb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86ec1019c0 0x7f86ec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86eb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86ec102360 0x7f86ec19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86eb200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec1079c0 0x7f86ec19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86eb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec111e10 con 0x7f86ec102360 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86eb200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec111c90 con 0x7f86ec1079c0 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86eb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec111f90 con 0x7f86ec1019c0 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86ec102360 0x7f86ec19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86ec1019c0 0x7f86ec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86ec1019c0 0x7f86ec19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48078/0 (socket says 172.21.15.46:48078) 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9e00640 1 -- 172.21.15.46:0/2331703326 learned_addr learned my addr 172.21.15.46:0/2331703326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86ea800640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec1079c0 0x7f86ec19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9e00640 1 -- 172.21.15.46:0/2331703326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec1079c0 msgr2=0x7f86ec19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9e00640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec1079c0 0x7f86ec19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9e00640 1 -- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86ec102360 msgr2=0x7f86ec19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9e00640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86ec102360 0x7f86ec19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.030+0000 7f86e9e00640 1 -- 172.21.15.46:0/2331703326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec19f4a0 con 0x7f86ec1019c0 2024-09-10T07:52:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.031+0000 7f86e9e00640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86ec1019c0 0x7f86ec19a4f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f86dc02a940 tx=0x7f86dc003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.031+0000 7f86e2a00640 1 -- 172.21.15.46:0/2331703326 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc03f070 con 0x7f86ec1019c0 2024-09-10T07:52:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.031+0000 7f86eb200640 1 -- 172.21.15.46:0/2331703326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec19f730 con 0x7f86ec1019c0 2024-09-10T07:52:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.031+0000 7f86eb200640 1 -- 172.21.15.46:0/2331703326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec1a7070 con 0x7f86ec1019c0 2024-09-10T07:52:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.031+0000 7f86e2a00640 1 -- 172.21.15.46:0/2331703326 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86dc031c90 con 0x7f86ec1019c0 2024-09-10T07:52:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.031+0000 7f86e2a00640 1 -- 172.21.15.46:0/2331703326 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc049060 con 0x7f86ec1019c0 2024-09-10T07:52:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.034+0000 7f86f3cae740 1 Processor -- start 2024-09-10T07:52:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86f3cae740 1 -- start start 2024-09-10T07:52:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86e2a00640 1 -- 172.21.15.46:0/2331703326 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86dc042070 con 0x7f86ec1019c0 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f285220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 0x55a37f292410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86f3cae740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a37f1eb3e0 con 0x55a37f284e40 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86f3cae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a37f192d80 con 0x55a37f290020 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86f3cae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a37f1e73a0 con 0x55a37f285760 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f285220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f285220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35862/0 (socket says 172.21.15.46:35862) 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86ea800640 1 -- 172.21.15.46:0/4108183931 learned_addr learned my addr 172.21.15.46:0/4108183931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86e9e00640 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.035+0000 7f86e9400640 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 0x55a37f292410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86ea800640 1 -- 172.21.15.46:0/4108183931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 msgr2=0x55a37f28fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86ea800640 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86ea800640 1 -- 172.21.15.46:0/4108183931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 msgr2=0x55a37f292410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86ea800640 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 0x55a37f292410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86ea800640 1 -- 172.21.15.46:0/4108183931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a37f1b1ae0 con 0x55a37f284e40 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86e9e00640 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86ea800640 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f285220 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009f40 tx=0x7f86e4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1820df1bcbac2c7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86d7400640 1 -- 172.21.15.46:0/4108183931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4003b70 con 0x55a37f284e40 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86e2a00640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86d00a2c00 0x7f86d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86d7400640 1 -- 172.21.15.46:0/4108183931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e4003d10 con 0x55a37f284e40 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.036+0000 7f86d7400640 1 -- 172.21.15.46:0/4108183931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4011a40 con 0x55a37f284e40 2024-09-10T07:52:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86e2a00640 1 -- 172.21.15.46:0/2331703326 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f86dc0ef700 con 0x7f86ec1019c0 2024-09-10T07:52:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86e9e00640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86d00a2c00 0x7f86d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86f3cae740 1 -- 172.21.15.46:0/4108183931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 msgr2=0x55a37f285220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86f3cae740 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f285220 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009f40 tx=0x7f86e4010860 comp rx=0 tx=0).stop 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86f3cae740 1 -- 172.21.15.46:0/4108183931 shutdown_connections 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86f3cae740 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 0x55a37f292410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86f3cae740 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86f3cae740 1 --2- 172.21.15.46:0/4108183931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f285220 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86f3cae740 1 -- 172.21.15.46:0/4108183931 >> 172.21.15.46:0/4108183931 conn(0x55a37f2813d0 msgr2=0x55a37f2817b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.037+0000 7f86f3cae740 1 -- 172.21.15.46:0/4108183931 shutdown_connections 2024-09-10T07:52:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.038+0000 7f86e9e00640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86d00a2c00 0x7f86d00a50c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0b7fe0 tx=0x7f86dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.038+0000 7f86f3cae740 1 -- 172.21.15.46:0/4108183931 wait complete. 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.038+0000 7f86f3cae740 1 -- start start 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.038+0000 7f86f3cae740 1 Processor -- start 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f290020 0x55a37f28d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86f3cae740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a37f292bf0 con 0x55a37f290020 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86f3cae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a37f1a60f0 con 0x55a37f284e40 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86f3cae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a37f292d70 con 0x55a37f285760 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86f3cae740 10 client.0 _ll_register_callbacks cb 0x55a37f18dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86e9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86ea800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86e9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48088/0 (socket says 172.21.15.46:48088) 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86e9400640 1 -- 172.21.15.46:0/1042889621 learned_addr learned my addr 172.21.15.46:0/1042889621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86ea800640 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f290020 0x55a37f28d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86e9400640 1 -- 172.21.15.46:0/1042889621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 msgr2=0x55a37f28cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86e9400640 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.039+0000 7f86e9400640 1 -- 172.21.15.46:0/1042889621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f290020 msgr2=0x55a37f28d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86e9400640 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f290020 0x55a37f28d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86e9400640 1 -- 172.21.15.46:0/1042889621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a37ef5c3c0 con 0x55a37f285760 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86e9400640 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28d4b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f86d8012730 tx=0x7f86d8012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337c52c5804bb5b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86cbe00640 1 -- 172.21.15.46:0/1042889621 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8004050 con 0x55a37f285760 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86f3cae740 1 -- 172.21.15.46:0/1042889621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e4003d10 con 0x55a37f285760 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86cbe00640 1 -- 172.21.15.46:0/1042889621 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f86d80041f0 con 0x55a37f285760 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86cbe00640 1 -- 172.21.15.46:0/1042889621 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f86d8025050 con 0x55a37f285760 2024-09-10T07:52:06.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86f3cae740 10 client.29068 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:06.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.040+0000 7f86f3cae740 1 -- 172.21.15.46:0/1042889621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55a37f29ed40 con 0x55a37f285760 2024-09-10T07:52:06.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.041+0000 7f86cbe00640 1 -- 172.21.15.46:0/1042889621 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e4003d10 con 0x55a37f285760 2024-09-10T07:52:06.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.041+0000 7f86cbe00640 1 -- 172.21.15.46:0/1042889621 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f86d8003a40 con 0x55a37f285760 2024-09-10T07:52:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.041+0000 7f86f3cae740 10 client.29068 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.041+0000 7f86f3cae740 10 client.29068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:52:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.041+0000 7f86f3cae740 20 client.29068 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.041+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a37f2a01a0 0x55a37f2a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.041+0000 7f86f3cae740 4 client.29068 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.041+0000 7f86f3cae740 1 -- 172.21.15.46:0/1042889621 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a37f1f8950 con 0x55a37f2a01a0 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.042+0000 7f86ea800640 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a37f2a01a0 0x55a37f2a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.042+0000 7f86ea800640 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a37f2a01a0 0x55a37f2a25c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f526fe8851893a8f server_cookie=b45b3fba53ffa19a in_seq=0 out_seq=0 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.042+0000 7f86cbe00640 10 client.29068 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86cbe00640 1 -- 172.21.15.46:0/1042889621 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a37f1f8950 con 0x55a37f2a01a0 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86cbe00640 10 client.29068 handle_command_reply: tid=0 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 client.29068 shutdown 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 -- 172.21.15.46:0/1042889621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 msgr2=0x55a37f28d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28d4b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f86d8012730 tx=0x7f86d8012bf0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 -- 172.21.15.46:0/1042889621 shutdown_connections 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a37f2a01a0 0x55a37f2a25c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f290020 0x55a37f28d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f28d4b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1042889621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86cbe00640 0 client.29068 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:52:06.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 -- 172.21.15.46:0/1042889621 >> 172.21.15.46:0/1042889621 conn(0x55a37f2813d0 msgr2=0x55a37f2870b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 -- 172.21.15.46:0/1042889621 shutdown_connections 2024-09-10T07:52:06.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.043+0000 7f86f3cae740 1 -- 172.21.15.46:0/1042889621 wait complete. 2024-09-10T07:52:06.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.044+0000 7f86f3cae740 20 client.29068 trim_cache size 0 max 16384 2024-09-10T07:52:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.057+0000 7f86f3cae740 1 Processor -- start 2024-09-10T07:52:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86f3cae740 1 -- start start 2024-09-10T07:52:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f285760 0x55a37f28d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f290020 0x55a37f2a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86f3cae740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1b0ce0 con 0x55a37f285760 2024-09-10T07:52:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86f3cae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1b29a0 con 0x55a37f284e40 2024-09-10T07:52:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86f3cae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1b13b0 con 0x55a37f290020 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f285760 0x55a37f28d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f285760 0x55a37f28d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35880/0 (socket says 172.21.15.46:35880) 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86e9400640 1 -- 172.21.15.46:0/1145699426 learned_addr learned my addr 172.21.15.46:0/1145699426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86ea800640 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.058+0000 7f86ea800640 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f290020 0x55a37f2a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86e9400640 1 -- 172.21.15.46:0/1145699426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f290020 msgr2=0x55a37f2a7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86e9400640 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f290020 0x55a37f2a7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86e9400640 1 -- 172.21.15.46:0/1145699426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 msgr2=0x55a37f28b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86e9400640 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86e9400640 1 -- 172.21.15.46:0/1145699426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a37f285590 con 0x55a37f285760 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86ea800640 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86e9400640 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f285760 0x55a37f28d840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f86d8003e60 tx=0x7f86d8010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98aa7fd360e655a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86d6a00640 1 -- 172.21.15.46:0/1145699426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d80050e0 con 0x55a37f285760 2024-09-10T07:52:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86d6a00640 1 -- 172.21.15.46:0/1145699426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d802e070 con 0x55a37f285760 2024-09-10T07:52:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.059+0000 7f86d6a00640 1 -- 172.21.15.46:0/1145699426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d800fe10 con 0x55a37f285760 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 -- 172.21.15.46:0/1145699426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f285760 msgr2=0x55a37f28d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f285760 0x55a37f28d840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f86d8003e60 tx=0x7f86d8010690 comp rx=0 tx=0).stop 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 -- 172.21.15.46:0/1145699426 shutdown_connections 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f290020 0x55a37f2a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f285760 0x55a37f28d840 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1145699426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f284e40 0x55a37f28b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 -- 172.21.15.46:0/1145699426 >> 172.21.15.46:0/1145699426 conn(0x55a37f2813d0 msgr2=0x55a37f28eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 -- 172.21.15.46:0/1145699426 shutdown_connections 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.060+0000 7f86f3cae740 1 -- 172.21.15.46:0/1145699426 wait complete. 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.061+0000 7f86f3cae740 1 -- start start 2024-09-10T07:52:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.061+0000 7f86f3cae740 1 Processor -- start 2024-09-10T07:52:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f293cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f294210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86f3cae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 0x55a37f294750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86f3cae740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1b2210 con 0x55a37f284e40 2024-09-10T07:52:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86f3cae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1b1b40 con 0x55a37f290020 2024-09-10T07:52:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86f3cae740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1aecc0 con 0x55a37f285760 2024-09-10T07:52:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86f3cae740 10 client.0 _ll_register_callbacks cb 0x7f86d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f293cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86e9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f294210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86e9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f294210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48106/0 (socket says 172.21.15.46:48106) 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86e9400640 1 -- 172.21.15.46:0/1405696986 learned_addr learned my addr 172.21.15.46:0/1405696986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86ea800640 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 0x55a37f294750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86ea800640 1 -- 172.21.15.46:0/1405696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 msgr2=0x55a37f294210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86ea800640 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f294210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86ea800640 1 -- 172.21.15.46:0/1405696986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 msgr2=0x55a37f294750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86ea800640 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 0x55a37f294750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86ea800640 1 -- 172.21.15.46:0/1405696986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a37f309e40 con 0x55a37f284e40 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.062+0000 7f86e9400640 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f294210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.063+0000 7f86ea800640 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f293cd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55a37f2a2e10 tx=0x7f86e4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b9d9b67a2415bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.063+0000 7f86cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.063+0000 7f86cbe00640 1 -- 172.21.15.46:0/1405696986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4024040 con 0x55a37f284e40 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.063+0000 7f86cbe00640 1 -- 172.21.15.46:0/1405696986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f86e40118d0 con 0x55a37f284e40 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.063+0000 7f86f3cae740 1 -- 172.21.15.46:0/1405696986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d802e070 con 0x55a37f284e40 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.063+0000 7f86cbe00640 1 -- 172.21.15.46:0/1405696986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f86e403e050 con 0x55a37f284e40 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.063+0000 7f86f3cae740 10 client.18261 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.063+0000 7f86f3cae740 1 -- 172.21.15.46:0/1405696986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55a37f30b0d0 con 0x55a37f284e40 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86cbe00640 1 -- 172.21.15.46:0/1405696986 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d802e070 con 0x55a37f284e40 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86cbe00640 1 -- 172.21.15.46:0/1405696986 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f86e4035070 con 0x55a37f284e40 2024-09-10T07:52:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86f3cae740 10 client.18261 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86f3cae740 10 client.18261 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86f3cae740 20 client.18261 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a37f30b610 0x55a37f30da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86f3cae740 4 client.18261 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86f3cae740 1 -- 172.21.15.46:0/1405696986 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a37f1f2e60 con 0x55a37f30b610 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.064+0000 7f86e9400640 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a37f30b610 0x55a37f30da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86e9400640 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a37f30b610 0x55a37f30da00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b921548b91fe6a1 server_cookie=fe749393353afa48 in_seq=0 out_seq=0 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86cbe00640 10 client.18261 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86cbe00640 1 -- 172.21.15.46:0/1405696986 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a37f1f2e60 con 0x55a37f30b610 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86cbe00640 10 client.18261 handle_command_reply: tid=0 2024-09-10T07:52:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 client.18261 shutdown 2024-09-10T07:52:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 -- 172.21.15.46:0/1405696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 msgr2=0x55a37f293cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f293cd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55a37f2a2e10 tx=0x7f86e4010860 comp rx=0 tx=0).stop 2024-09-10T07:52:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 -- 172.21.15.46:0/1405696986 shutdown_connections 2024-09-10T07:52:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x55a37f30b610 0x55a37f30da00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a37f290020 0x55a37f294750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a37f285760 0x55a37f294210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86cbe00640 0 client.18261 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:52:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 --2- 172.21.15.46:0/1405696986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a37f284e40 0x55a37f293cd0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 -- 172.21.15.46:0/1405696986 >> 172.21.15.46:0/1405696986 conn(0x55a37f2813d0 msgr2=0x55a37f27fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.065+0000 7f86f3cae740 1 -- 172.21.15.46:0/1405696986 shutdown_connections 2024-09-10T07:52:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86f3cae740 1 -- 172.21.15.46:0/1405696986 wait complete. 2024-09-10T07:52:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86f3cae740 20 client.18261 trim_cache size 0 max 16384 2024-09-10T07:52:06.069 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:52:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 -- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86d00a2c00 msgr2=0x7f86d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86d00a2c00 0x7f86d00a50c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0b7fe0 tx=0x7f86dc009260 comp rx=0 tx=0).stop 2024-09-10T07:52:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 -- 172.21.15.46:0/2331703326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86ec1019c0 msgr2=0x7f86ec19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86ec1019c0 0x7f86ec19a4f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f86dc02a940 tx=0x7f86dc003b20 comp rx=0 tx=0).stop 2024-09-10T07:52:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 -- 172.21.15.46:0/2331703326 shutdown_connections 2024-09-10T07:52:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec1079c0 0x7f86ec19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f86d00a2c00 0x7f86d00a50c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86ec102360 0x7f86ec19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 --2- 172.21.15.46:0/2331703326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86ec1019c0 0x7f86ec19a4f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.066+0000 7f86eb200640 1 -- 172.21.15.46:0/2331703326 >> 172.21.15.46:0/2331703326 conn(0x7f86ec0fd740 msgr2=0x7f86ec10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.067+0000 7f86eb200640 1 -- 172.21.15.46:0/2331703326 shutdown_connections 2024-09-10T07:52:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:06.067+0000 7f86eb200640 1 -- 172.21.15.46:0/2331703326 wait complete. 2024-09-10T07:52:06.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:06 smithi181 ceph-mon[35397]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-09-10T07:52:06.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:06 smithi063 ceph-mon[35020]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-09-10T07:52:06.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:06 smithi046 ceph-mon[30705]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-09-10T07:52:07.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:52:07.431 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:08.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 -- 172.21.15.46:0/830165748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 msgr2=0x7ff8501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 --2- 172.21.15.46:0/830165748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 0x7ff8501021c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff840009900 tx=0x7ff84002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 -- 172.21.15.46:0/830165748 shutdown_connections 2024-09-10T07:52:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 --2- 172.21.15.46:0/830165748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff850106f10 0x7ff85010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 --2- 172.21.15.46:0/830165748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850102790 0x7ff8501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 --2- 172.21.15.46:0/830165748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 0x7ff8501021c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 -- 172.21.15.46:0/830165748 >> 172.21.15.46:0/830165748 conn(0x7ff8500fd8c0 msgr2=0x7ff8500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 -- 172.21.15.46:0/830165748 shutdown_connections 2024-09-10T07:52:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.346+0000 7ff856c00640 1 -- 172.21.15.46:0/830165748 wait complete. 2024-09-10T07:52:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.347+0000 7ff856c00640 1 Processor -- start 2024-09-10T07:52:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.347+0000 7ff856c00640 1 -- start start 2024-09-10T07:52:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.347+0000 7ff856c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 0x7ff850198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.347+0000 7ff856c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff850102790 0x7ff8501989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.347+0000 7ff856c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850106f10 0x7ff85019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.347+0000 7ff856c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff850111f70 con 0x7ff850101de0 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.347+0000 7ff856c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff850111df0 con 0x7ff850106f10 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.347+0000 7ff856c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501120f0 con 0x7ff850102790 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff84fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 0x7ff850198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff847400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff850102790 0x7ff8501989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff84fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 0x7ff850198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35900/0 (socket says 172.21.15.46:35900) 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff84fe00640 1 -- 172.21.15.46:0/2107991675 learned_addr learned my addr 172.21.15.46:0/2107991675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff854c00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850106f10 0x7ff85019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff847400640 1 -- 172.21.15.46:0/2107991675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850106f10 msgr2=0x7ff85019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff847400640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850106f10 0x7ff85019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff847400640 1 -- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 msgr2=0x7ff850198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff847400640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 0x7ff850198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.348+0000 7ff847400640 1 -- 172.21.15.46:0/2107991675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85019d420 con 0x7ff850102790 2024-09-10T07:52:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.349+0000 7ff847400640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff850102790 0x7ff8501989b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff83800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.349+0000 7ff84d600640 1 -- 172.21.15.46:0/2107991675 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838009c30 con 0x7ff850102790 2024-09-10T07:52:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.349+0000 7ff856c00640 1 -- 172.21.15.46:0/2107991675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85019d710 con 0x7ff850102790 2024-09-10T07:52:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.349+0000 7ff84d600640 1 -- 172.21.15.46:0/2107991675 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff838009dd0 con 0x7ff850102790 2024-09-10T07:52:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.349+0000 7ff84d600640 1 -- 172.21.15.46:0/2107991675 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838010460 con 0x7ff850102790 2024-09-10T07:52:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.349+0000 7ff856c00640 1 -- 172.21.15.46:0/2107991675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8501a4ff0 con 0x7ff850102790 2024-09-10T07:52:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.353+0000 7ff8597d6740 1 Processor -- start 2024-09-10T07:52:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.353+0000 7ff8597d6740 1 -- start start 2024-09-10T07:52:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.353+0000 7ff84d600640 1 -- 172.21.15.46:0/2107991675 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff838003ef0 con 0x7ff850102790 2024-09-10T07:52:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.353+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b8208050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8208590 0x55f6b8212910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 0x55f6b8215240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff8597d6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f6b816e3e0 con 0x55f6b8207c70 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff8597d6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8115d80 con 0x55f6b8212e50 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff8597d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f6b816a3a0 con 0x55f6b8208590 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff854c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b8208050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff854c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b8208050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35902/0 (socket says 172.21.15.46:35902) 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff854c00640 1 -- 172.21.15.46:0/2923122581 learned_addr learned my addr 172.21.15.46:0/2923122581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff847400640 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 0x55f6b8215240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff84fe00640 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8208590 0x55f6b8212910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff854c00640 1 -- 172.21.15.46:0/2923122581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8208590 msgr2=0x55f6b8212910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff854c00640 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8208590 0x55f6b8212910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff854c00640 1 -- 172.21.15.46:0/2923122581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 msgr2=0x55f6b8215240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff854c00640 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 0x55f6b8215240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.354+0000 7ff854c00640 1 -- 172.21.15.46:0/2923122581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b8134ae0 con 0x55f6b8207c70 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.355+0000 7ff854c00640 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b8208050 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff848009f50 tx=0x7ff8480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8cc9c77ff7f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.355+0000 7ff845200640 1 -- 172.21.15.46:0/2923122581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848004d20 con 0x55f6b8207c70 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.355+0000 7ff845200640 1 -- 172.21.15.46:0/2923122581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff848004050 con 0x55f6b8207c70 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.355+0000 7ff845200640 1 -- 172.21.15.46:0/2923122581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848005280 con 0x55f6b8207c70 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.355+0000 7ff84d600640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8340a2b80 0x7ff8340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff84d600640 1 -- 172.21.15.46:0/2107991675 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff8380cef60 con 0x7ff850102790 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff84fe00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8340a2b80 0x7ff8340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 -- 172.21.15.46:0/2923122581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 msgr2=0x55f6b8208050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b8208050 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff848009f50 tx=0x7ff8480118d0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 -- 172.21.15.46:0/2923122581 shutdown_connections 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 0x55f6b8215240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8208590 0x55f6b8212910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 --2- 172.21.15.46:0/2923122581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b8208050 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 -- 172.21.15.46:0/2923122581 >> 172.21.15.46:0/2923122581 conn(0x55f6b8204200 msgr2=0x55f6b82045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 -- 172.21.15.46:0/2923122581 shutdown_connections 2024-09-10T07:52:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.356+0000 7ff8597d6740 1 -- 172.21.15.46:0/2923122581 wait complete. 2024-09-10T07:52:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.357+0000 7ff84fe00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8340a2b80 0x7ff8340a5040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x55f6b8205060 tx=0x7ff840005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.357+0000 7ff8597d6740 1 -- start start 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.357+0000 7ff8597d6740 1 Processor -- start 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b820fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b8210240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8212e50 0x55f6b8210780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff8597d6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8219e30 con 0x55f6b8207c70 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff8597d6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f6b81290f0 con 0x55f6b8208590 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff8597d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8219fb0 con 0x55f6b8212e50 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff8597d6740 10 client.0 _ll_register_callbacks cb 0x55f6b8110bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff84fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b820fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff84fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b820fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35904/0 (socket says 172.21.15.46:35904) 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff84fe00640 1 -- 172.21.15.46:0/1369576026 learned_addr learned my addr 172.21.15.46:0/1369576026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff847400640 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b8210240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff854c00640 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8212e50 0x55f6b8210780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff84fe00640 1 -- 172.21.15.46:0/1369576026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8212e50 msgr2=0x55f6b8210780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff84fe00640 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8212e50 0x55f6b8210780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff84fe00640 1 -- 172.21.15.46:0/1369576026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 msgr2=0x55f6b8210240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff84fe00640 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b8210240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.358+0000 7ff84fe00640 1 -- 172.21.15.46:0/1369576026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b7edf3c0 con 0x55f6b8207c70 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.359+0000 7ff84fe00640 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b820fd00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff8400096c0 tx=0x7ff840009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d52c5ac35157da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.359+0000 7ff83d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.359+0000 7ff83d600640 1 -- 172.21.15.46:0/1369576026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84004d030 con 0x55f6b8207c70 2024-09-10T07:52:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.359+0000 7ff83d600640 1 -- 172.21.15.46:0/1369576026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff840003800 con 0x55f6b8207c70 2024-09-10T07:52:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.359+0000 7ff8597d6740 1 -- 172.21.15.46:0/1369576026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff848004050 con 0x55f6b8207c70 2024-09-10T07:52:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.359+0000 7ff83d600640 1 -- 172.21.15.46:0/1369576026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7ff84005a050 con 0x55f6b8207c70 2024-09-10T07:52:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.359+0000 7ff8597d6740 10 client.18273 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.359+0000 7ff8597d6740 1 -- 172.21.15.46:0/1369576026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f6b8225e40 con 0x55f6b8207c70 2024-09-10T07:52:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff83d600640 1 -- 172.21.15.46:0/1369576026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff848004050 con 0x55f6b8207c70 2024-09-10T07:52:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff83d600640 1 -- 172.21.15.46:0/1369576026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7ff84003dac0 con 0x55f6b8207c70 2024-09-10T07:52:08.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff8597d6740 10 client.18273 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff8597d6740 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff8597d6740 20 client.18273 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f6b82273d0 0x55f6b8229830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff8597d6740 4 client.18273 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff8597d6740 1 -- 172.21.15.46:0/1369576026 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6b817b950 con 0x55f6b82273d0 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.360+0000 7ff847400640 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f6b82273d0 0x55f6b8229830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.361+0000 7ff847400640 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f6b82273d0 0x55f6b8229830 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91ea2e6503a2745 server_cookie=7596afdc198e2a74 in_seq=0 out_seq=0 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.361+0000 7ff83d600640 10 client.18273 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.361+0000 7ff83d600640 1 -- 172.21.15.46:0/1369576026 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6b817b950 con 0x55f6b82273d0 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.361+0000 7ff83d600640 10 client.18273 handle_command_reply: tid=0 2024-09-10T07:52:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 client.18273 shutdown 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 -- 172.21.15.46:0/1369576026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 msgr2=0x55f6b820fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b820fd00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff8400096c0 tx=0x7ff840009260 comp rx=0 tx=0).stop 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 -- 172.21.15.46:0/1369576026 shutdown_connections 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f6b82273d0 0x55f6b8229830 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8212e50 0x55f6b8210780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b8210240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1369576026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8207c70 0x55f6b820fd00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 -- 172.21.15.46:0/1369576026 >> 172.21.15.46:0/1369576026 conn(0x55f6b8204200 msgr2=0x55f6b8203380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff83d600640 0 client.18273 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 -- 172.21.15.46:0/1369576026 shutdown_connections 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 1 -- 172.21.15.46:0/1369576026 wait complete. 2024-09-10T07:52:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.362+0000 7ff8597d6740 20 client.18273 trim_cache size 0 max 16384 2024-09-10T07:52:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.376+0000 7ff8597d6740 1 Processor -- start 2024-09-10T07:52:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.376+0000 7ff8597d6740 1 -- start start 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.376+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b8209dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.376+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b822f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.376+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8212e50 0x55f6b822f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.376+0000 7ff8597d6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501aca30 con 0x55f6b8212e50 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.376+0000 7ff8597d6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501ae6f0 con 0x55f6b8208590 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.376+0000 7ff8597d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501ad100 con 0x55f6b8207c70 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b822f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b822f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49446/0 (socket says 172.21.15.46:49446) 2024-09-10T07:52:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 -- 172.21.15.46:0/1756316705 learned_addr learned my addr 172.21.15.46:0/1756316705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff854c00640 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8212e50 0x55f6b822f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff84fe00640 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b8209dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 -- 172.21.15.46:0/1756316705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 msgr2=0x55f6b8209dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b8209dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 -- 172.21.15.46:0/1756316705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8212e50 msgr2=0x55f6b822f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8212e50 0x55f6b822f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 -- 172.21.15.46:0/1756316705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b8204e90 con 0x55f6b8208590 2024-09-10T07:52:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.377+0000 7ff847400640 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b822f100 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff83809b190 tx=0x7ff838097040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6836de6ab3ae7d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:08.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:08 smithi046 ceph-mon[30705]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.4 MiB/s wr, 391 op/s 2024-09-10T07:52:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.378+0000 7ff83fe00640 1 -- 172.21.15.46:0/1756316705 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838020040 con 0x55f6b8208590 2024-09-10T07:52:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.378+0000 7ff83fe00640 1 -- 172.21.15.46:0/1756316705 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83809b490 con 0x55f6b8208590 2024-09-10T07:52:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.378+0000 7ff83fe00640 1 -- 172.21.15.46:0/1756316705 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380cc790 con 0x55f6b8208590 2024-09-10T07:52:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 -- 172.21.15.46:0/1756316705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 msgr2=0x55f6b822f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b822f100 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff83809b190 tx=0x7ff838097040 comp rx=0 tx=0).stop 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 -- 172.21.15.46:0/1756316705 shutdown_connections 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8212e50 0x55f6b822f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8208590 0x55f6b822f100 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1756316705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b8209dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 -- 172.21.15.46:0/1756316705 >> 172.21.15.46:0/1756316705 conn(0x55f6b8204200 msgr2=0x55f6b82096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 -- 172.21.15.46:0/1756316705 shutdown_connections 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 -- 172.21.15.46:0/1756316705 wait complete. 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.379+0000 7ff8597d6740 1 -- start start 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff8597d6740 1 Processor -- start 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b821b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8208590 0x55f6b821b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff8597d6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 0x55f6b82910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff8597d6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501adf60 con 0x55f6b8208590 2024-09-10T07:52:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff8597d6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501ad890 con 0x55f6b8212e50 2024-09-10T07:52:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff8597d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501aaa10 con 0x55f6b8207c70 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff8597d6740 10 client.0 _ll_register_callbacks cb 0x7ff840009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff84fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b821b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.380+0000 7ff847400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8208590 0x55f6b821b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff847400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8208590 0x55f6b821b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35914/0 (socket says 172.21.15.46:35914) 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff847400640 1 -- 172.21.15.46:0/1322895885 learned_addr learned my addr 172.21.15.46:0/1322895885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff84fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b821b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48150/0 (socket says 172.21.15.46:48150) 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff854c00640 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 0x55f6b82910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff847400640 1 -- 172.21.15.46:0/1322895885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 msgr2=0x55f6b821b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff847400640 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b821b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff847400640 1 -- 172.21.15.46:0/1322895885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 msgr2=0x55f6b82910d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff847400640 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 0x55f6b82910d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff847400640 1 -- 172.21.15.46:0/1322895885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b821c060 con 0x55f6b8208590 2024-09-10T07:52:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff847400640 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8208590 0x55f6b821b7d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff8380978f0 tx=0x7ff8380cf290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da62c7d9acfc3ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.381+0000 7ff83d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.382+0000 7ff83d600640 1 -- 172.21.15.46:0/1322895885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380b9060 con 0x55f6b8208590 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.382+0000 7ff83d600640 1 -- 172.21.15.46:0/1322895885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff8380cc790 con 0x55f6b8208590 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.382+0000 7ff8597d6740 1 -- 172.21.15.46:0/1322895885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff83809b490 con 0x55f6b8208590 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.382+0000 7ff83d600640 1 -- 172.21.15.46:0/1322895885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7ff838020040 con 0x55f6b8208590 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.382+0000 7ff8597d6740 10 client.18279 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.382+0000 7ff8597d6740 1 -- 172.21.15.46:0/1322895885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f6b82926e0 con 0x55f6b8208590 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.382+0000 7ff83d600640 1 -- 172.21.15.46:0/1322895885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83809b490 con 0x55f6b8208590 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.382+0000 7ff83d600640 1 -- 172.21.15.46:0/1322895885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7ff8380a3bc0 con 0x55f6b8208590 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.383+0000 7ff8597d6740 10 client.18279 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.383+0000 7ff8597d6740 10 client.18279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.383+0000 7ff8597d6740 20 client.18279 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.383+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f6b8292e60 0x55f6b8295280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.383+0000 7ff8597d6740 4 client.18279 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:52:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.383+0000 7ff8597d6740 1 -- 172.21.15.46:0/1322895885 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6b8175e60 con 0x55f6b8292e60 2024-09-10T07:52:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.383+0000 7ff84fe00640 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f6b8292e60 0x55f6b8295280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff83d600640 10 client.18279 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff84fe00640 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f6b8292e60 0x55f6b8295280 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b720055f52043523 server_cookie=6bcbd1b7f53d7e01 in_seq=0 out_seq=0 2024-09-10T07:52:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff83d600640 1 -- 172.21.15.46:0/1322895885 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6b8175e60 con 0x55f6b8292e60 2024-09-10T07:52:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff83d600640 10 client.18279 handle_command_reply: tid=0 2024-09-10T07:52:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 client.18279 shutdown 2024-09-10T07:52:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 -- 172.21.15.46:0/1322895885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8208590 msgr2=0x55f6b821b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8208590 0x55f6b821b7d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff8380978f0 tx=0x7ff8380cf290 comp rx=0 tx=0).stop 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 -- 172.21.15.46:0/1322895885 shutdown_connections 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55f6b8292e60 0x55f6b8295280 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6b8212e50 0x55f6b82910d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6b8208590 0x55f6b821b7d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 --2- 172.21.15.46:0/1322895885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f6b8207c70 0x55f6b821b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.384+0000 7ff8597d6740 1 -- 172.21.15.46:0/1322895885 >> 172.21.15.46:0/1322895885 conn(0x55f6b8204200 msgr2=0x55f6b8202b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.385+0000 7ff83d600640 0 client.18279 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.385+0000 7ff8597d6740 1 -- 172.21.15.46:0/1322895885 shutdown_connections 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.385+0000 7ff8597d6740 1 -- 172.21.15.46:0/1322895885 wait complete. 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.385+0000 7ff8597d6740 20 client.18279 trim_cache size 0 max 16384 2024-09-10T07:52:08.387 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:52:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 -- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8340a2b80 msgr2=0x7ff8340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8340a2b80 0x7ff8340a5040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x55f6b8205060 tx=0x7ff840005a20 comp rx=0 tx=0).stop 2024-09-10T07:52:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 -- 172.21.15.46:0/2107991675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff850102790 msgr2=0x7ff8501989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff850102790 0x7ff8501989b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff83800dde0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 -- 172.21.15.46:0/2107991675 shutdown_connections 2024-09-10T07:52:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850106f10 0x7ff85019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff850102790 0x7ff8501989b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff8340a2b80 0x7ff8340a5040 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 --2- 172.21.15.46:0/2107991675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850101de0 0x7ff850198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 -- 172.21.15.46:0/2107991675 >> 172.21.15.46:0/2107991675 conn(0x7ff8500fd8c0 msgr2=0x7ff8500fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 -- 172.21.15.46:0/2107991675 shutdown_connections 2024-09-10T07:52:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:08.386+0000 7ff856c00640 1 -- 172.21.15.46:0/2107991675 wait complete. 2024-09-10T07:52:08.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:08 smithi181 ceph-mon[35397]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.4 MiB/s wr, 391 op/s 2024-09-10T07:52:08.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:08 smithi063 ceph-mon[35020]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.4 MiB/s wr, 391 op/s 2024-09-10T07:52:10.167 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T07:52:10.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T07:52:10.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:10 smithi181 ceph-mon[35397]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.3 MiB/s wr, 278 op/s 2024-09-10T07:52:10.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:10 smithi063 ceph-mon[35020]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.3 MiB/s wr, 278 op/s 2024-09-10T07:52:10.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:10 smithi046 ceph-mon[30705]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.3 MiB/s wr, 278 op/s 2024-09-10T07:52:10.561 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:10.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 890.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:52:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 -- 172.21.15.46:0/546290023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda841019c0 msgr2=0x7fda84101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 --2- 172.21.15.46:0/546290023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda841019c0 0x7fda84101e20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009930 tx=0x7fda6c02a420 comp rx=0 tx=0).stop 2024-09-10T07:52:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 -- 172.21.15.46:0/546290023 shutdown_connections 2024-09-10T07:52:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 --2- 172.21.15.46:0/546290023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda84102360 0x7fda8410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 --2- 172.21.15.46:0/546290023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda841019c0 0x7fda84101e20 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 --2- 172.21.15.46:0/546290023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda841079c0 0x7fda84107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 -- 172.21.15.46:0/546290023 >> 172.21.15.46:0/546290023 conn(0x7fda840fd740 msgr2=0x7fda840ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 -- 172.21.15.46:0/546290023 shutdown_connections 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.121+0000 7fda8a600640 1 -- 172.21.15.46:0/546290023 wait complete. 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.122+0000 7fda8a600640 1 Processor -- start 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.122+0000 7fda8a600640 1 -- start start 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.122+0000 7fda8a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda841019c0 0x7fda8419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.122+0000 7fda8a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda84102360 0x7fda8419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.122+0000 7fda8a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda841079c0 0x7fda8419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.122+0000 7fda8a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda84111d90 con 0x7fda84102360 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.122+0000 7fda8a600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda84111c10 con 0x7fda841019c0 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.122+0000 7fda8a600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda84111f10 con 0x7fda841079c0 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda841019c0 0x7fda8419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda841019c0 0x7fda8419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49484/0 (socket says 172.21.15.46:49484) 2024-09-10T07:52:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 -- 172.21.15.46:0/134906001 learned_addr learned my addr 172.21.15.46:0/134906001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89c00640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda841079c0 0x7fda8419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda7be00640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda84102360 0x7fda8419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 -- 172.21.15.46:0/134906001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda841079c0 msgr2=0x7fda8419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda841079c0 0x7fda8419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 -- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda84102360 msgr2=0x7fda8419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda84102360 0x7fda8419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 -- 172.21.15.46:0/134906001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8419f420 con 0x7fda841019c0 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda7be00640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda84102360 0x7fda8419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.123+0000 7fda89200640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda841019c0 0x7fda8419a470 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e9f0 tx=0x7fda7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.124+0000 7fda82000640 1 -- 172.21.15.46:0/134906001 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7400cd80 con 0x7fda841019c0 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.124+0000 7fda82000640 1 -- 172.21.15.46:0/134906001 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda74010910 con 0x7fda841019c0 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.124+0000 7fda8a600640 1 -- 172.21.15.46:0/134906001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8419f710 con 0x7fda841019c0 2024-09-10T07:52:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.124+0000 7fda82000640 1 -- 172.21.15.46:0/134906001 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740188f0 con 0x7fda841019c0 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.124+0000 7fda8a600640 1 -- 172.21.15.46:0/134906001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda841a6ff0 con 0x7fda841019c0 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda8e664740 1 Processor -- start 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda8e664740 1 -- start start 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda8e664740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ecb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda8e664740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08ed5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda82000640 1 -- 172.21.15.46:0/134906001 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda74010ac0 con 0x7fda841019c0 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda8e664740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ed5e90 0x556f08ed8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda8e664740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556f08e31290 con 0x556f08ecacb0 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda8e664740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556f08dd8b40 con 0x556f08ed5e90 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda8e664740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f08e2d250 con 0x556f08ecb5d0 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda89c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ecb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda89c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ecb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35934/0 (socket says 172.21.15.46:35934) 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.127+0000 7fda89c00640 1 -- 172.21.15.46:0/1192173065 learned_addr learned my addr 172.21.15.46:0/1192173065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda7be00640 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ed5e90 0x556f08ed8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda89200640 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08ed5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda89c00640 1 -- 172.21.15.46:0/1192173065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 msgr2=0x556f08ed5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda89c00640 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08ed5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda89c00640 1 -- 172.21.15.46:0/1192173065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ed5e90 msgr2=0x556f08ed8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda89c00640 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ed5e90 0x556f08ed8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda89c00640 1 -- 172.21.15.46:0/1192173065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f08ba26a0 con 0x556f08ecacb0 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda89c00640 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ecb090 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009fd0 tx=0x7fda7c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29537c23aaa8ba73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.128+0000 7fda7a000640 1 -- 172.21.15.46:0/1192173065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c004d20 con 0x556f08ecacb0 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.129+0000 7fda7a000640 1 -- 172.21.15.46:0/1192173065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda7c004050 con 0x556f08ecacb0 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.129+0000 7fda7a000640 1 -- 172.21.15.46:0/1192173065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c005220 con 0x556f08ecacb0 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.129+0000 7fda82000640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda680a2b80 0x7fda680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.129+0000 7fda8e664740 1 -- 172.21.15.46:0/1192173065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 msgr2=0x556f08ecb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.129+0000 7fda8e664740 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ecb090 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009fd0 tx=0x7fda7c0118d0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.129+0000 7fda89200640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda680a2b80 0x7fda680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.129+0000 7fda82000640 1 -- 172.21.15.46:0/134906001 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fda740cf7e0 con 0x7fda841019c0 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.130+0000 7fda8e664740 1 -- 172.21.15.46:0/1192173065 shutdown_connections 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.130+0000 7fda8e664740 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ed5e90 0x556f08ed8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.130+0000 7fda8e664740 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08ed5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.130+0000 7fda8e664740 1 --2- 172.21.15.46:0/1192173065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ecb090 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.130+0000 7fda8e664740 1 -- 172.21.15.46:0/1192173065 >> 172.21.15.46:0/1192173065 conn(0x556f08ec7240 msgr2=0x556f08ec7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.130+0000 7fda8e664740 1 -- 172.21.15.46:0/1192173065 shutdown_connections 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.130+0000 7fda89200640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda680a2b80 0x7fda680a5040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda7409b960 tx=0x7fda7409bb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.130+0000 7fda8e664740 1 -- 172.21.15.46:0/1192173065 wait complete. 2024-09-10T07:52:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 1 -- start start 2024-09-10T07:52:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 1 Processor -- start 2024-09-10T07:52:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ed56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ecb5d0 0x556f08ed2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ed5e90 0x556f08ed2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556f08ed8b00 con 0x556f08ecacb0 2024-09-10T07:52:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556f08debfa0 con 0x556f08ecb5d0 2024-09-10T07:52:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556f08ed8c80 con 0x556f08ed5e90 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.131+0000 7fda8e664740 10 client.0 _ll_register_callbacks cb 0x556f08b3fba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda7be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ecb5d0 0x556f08ed2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda89c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ed56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda7be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ecb5d0 0x556f08ed2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49500/0 (socket says 172.21.15.46:49500) 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda7be00640 1 -- 172.21.15.46:0/2964289036 learned_addr learned my addr 172.21.15.46:0/2964289036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda89c00640 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ed5e90 0x556f08ed2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda7be00640 1 -- 172.21.15.46:0/2964289036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ed5e90 msgr2=0x556f08ed2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda7be00640 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ed5e90 0x556f08ed2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda7be00640 1 -- 172.21.15.46:0/2964289036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 msgr2=0x556f08ed56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda7be00640 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ed56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda7be00640 1 -- 172.21.15.46:0/2964289036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f08ed36c0 con 0x556f08ecb5d0 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.132+0000 7fda89c00640 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ed56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.133+0000 7fda7be00640 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ecb5d0 0x556f08ed2950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009930 tx=0x7fda6c031860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c7bb297483a6b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.133+0000 7fda72a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.135+0000 7fda72a00640 1 -- 172.21.15.46:0/2964289036 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c035040 con 0x556f08ecb5d0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.135+0000 7fda8e664740 1 -- 172.21.15.46:0/2964289036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda7c004050 con 0x556f08ecb5d0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.135+0000 7fda72a00640 1 -- 172.21.15.46:0/2964289036 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fda6c004240 con 0x556f08ecb5d0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.135+0000 7fda72a00640 1 -- 172.21.15.46:0/2964289036 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fda6c04e050 con 0x556f08ecb5d0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.135+0000 7fda8e664740 10 client.29225 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.135+0000 7fda8e664740 1 -- 172.21.15.46:0/2964289036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x556f08ee4940 con 0x556f08ecb5d0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.135+0000 7fda72a00640 1 -- 172.21.15.46:0/2964289036 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda7c004050 con 0x556f08ecb5d0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.135+0000 7fda72a00640 1 -- 172.21.15.46:0/2964289036 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7fda6c004850 con 0x556f08ecb5d0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda8e664740 10 client.29225 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda8e664740 10 client.29225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda8e664740 20 client.29225 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda8e664740 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556f08ee5e10 0x556f08ee8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda8e664740 4 client.29225 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda8e664740 1 -- 172.21.15.46:0/2964289036 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f08e3e800 con 0x556f08ee5e10 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda89c00640 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556f08ee5e10 0x556f08ee8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda89c00640 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556f08ee5e10 0x556f08ee8290 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18f7aba1f495aad server_cookie=fe732151f7586ed7 in_seq=0 out_seq=0 2024-09-10T07:52:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.136+0000 7fda72a00640 10 client.29225 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.137+0000 7fda72a00640 1 -- 172.21.15.46:0/2964289036 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f08e3e800 con 0x556f08ee5e10 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.137+0000 7fda72a00640 10 client.29225 handle_command_reply: tid=0 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.137+0000 7fda8e664740 1 client.29225 shutdown 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.137+0000 7fda8e664740 1 -- 172.21.15.46:0/2964289036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ecb5d0 msgr2=0x556f08ed2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.137+0000 7fda8e664740 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ecb5d0 0x556f08ed2950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009930 tx=0x7fda6c031860 comp rx=0 tx=0).stop 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 1 -- 172.21.15.46:0/2964289036 shutdown_connections 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556f08ee5e10 0x556f08ee8290 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ed5e90 0x556f08ed2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ecb5d0 0x556f08ed2950 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 1 --2- 172.21.15.46:0/2964289036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08ed56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda72a00640 0 client.29225 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 1 -- 172.21.15.46:0/2964289036 >> 172.21.15.46:0/2964289036 conn(0x556f08ec7240 msgr2=0x556f08eccf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 1 -- 172.21.15.46:0/2964289036 shutdown_connections 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 1 -- 172.21.15.46:0/2964289036 wait complete. 2024-09-10T07:52:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.138+0000 7fda8e664740 20 client.29225 trim_cache size 0 max 16384 2024-09-10T07:52:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.156+0000 7fda8e664740 1 Processor -- start 2024-09-10T07:52:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.156+0000 7fda8e664740 1 -- start start 2024-09-10T07:52:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda8e664740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ec6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda8e664740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eedba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda8e664740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08eee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda8e664740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda841b0cf0 con 0x556f08ecacb0 2024-09-10T07:52:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda8e664740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda841b29b0 con 0x556f08ec7240 2024-09-10T07:52:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda8e664740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda841b13c0 con 0x556f08ecb5d0 2024-09-10T07:52:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eedba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eedba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35950/0 (socket says 172.21.15.46:35950) 2024-09-10T07:52:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda7be00640 1 -- 172.21.15.46:0/3944384578 learned_addr learned my addr 172.21.15.46:0/3944384578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda89c00640 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ec6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda89c00640 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08eee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda7be00640 1 -- 172.21.15.46:0/3944384578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 msgr2=0x556f08eee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.157+0000 7fda7be00640 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08eee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.158+0000 7fda7be00640 1 -- 172.21.15.46:0/3944384578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 msgr2=0x556f08ec6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.158+0000 7fda7be00640 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ec6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.158+0000 7fda7be00640 1 -- 172.21.15.46:0/3944384578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f08ecb3c0 con 0x556f08ecacb0 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.158+0000 7fda7be00640 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eedba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fda6c001eb0 tx=0x7fda6c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e29cdf99c7cd8dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.158+0000 7fda79600640 1 -- 172.21.15.46:0/3944384578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c035070 con 0x556f08ecacb0 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.158+0000 7fda79600640 1 -- 172.21.15.46:0/3944384578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6c0035f0 con 0x556f08ecacb0 2024-09-10T07:52:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.158+0000 7fda79600640 1 -- 172.21.15.46:0/3944384578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0052e0 con 0x556f08ecacb0 2024-09-10T07:52:11.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 -- 172.21.15.46:0/3944384578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 msgr2=0x556f08eedba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eedba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fda6c001eb0 tx=0x7fda6c005a20 comp rx=0 tx=0).stop 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 -- 172.21.15.46:0/3944384578 shutdown_connections 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08eee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eedba0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 --2- 172.21.15.46:0/3944384578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ec6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 -- 172.21.15.46:0/3944384578 >> 172.21.15.46:0/3944384578 conn(0x556f08ed2ff0 msgr2=0x556f08ec5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 -- 172.21.15.46:0/3944384578 shutdown_connections 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.159+0000 7fda8e664740 1 -- 172.21.15.46:0/3944384578 wait complete. 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 1 -- start start 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 1 Processor -- start 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ede3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eda700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08edac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda841b2220 con 0x556f08ecacb0 2024-09-10T07:52:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda841b1b50 con 0x556f08ec7240 2024-09-10T07:52:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda841aecd0 con 0x556f08ecb5d0 2024-09-10T07:52:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.160+0000 7fda8e664740 10 client.0 _ll_register_callbacks cb 0x7fda74005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eda700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ede3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eda700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35958/0 (socket says 172.21.15.46:35958) 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ede3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49518/0 (socket says 172.21.15.46:49518) 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda7be00640 1 -- 172.21.15.46:0/2980598008 learned_addr learned my addr 172.21.15.46:0/2980598008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08edac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 -- 172.21.15.46:0/2980598008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 msgr2=0x556f08edac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08edac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 -- 172.21.15.46:0/2980598008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 msgr2=0x556f08eda700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eda700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 -- 172.21.15.46:0/2980598008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f08edb200 con 0x556f08ec7240 2024-09-10T07:52:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda89c00640 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ede3c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda7c006010 tx=0x7fda7c003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b2d500e0b12b8c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.161+0000 7fda72a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.162+0000 7fda72a00640 1 -- 172.21.15.46:0/2980598008 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c004510 con 0x556f08ec7240 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.162+0000 7fda72a00640 1 -- 172.21.15.46:0/2980598008 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fda7c0046b0 con 0x556f08ec7240 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.162+0000 7fda8e664740 1 -- 172.21.15.46:0/2980598008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda6c0035f0 con 0x556f08ec7240 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.162+0000 7fda72a00640 1 -- 172.21.15.46:0/2980598008 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fda7c043050 con 0x556f08ec7240 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.162+0000 7fda8e664740 10 client.29231 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.162+0000 7fda8e664740 1 -- 172.21.15.46:0/2980598008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x556f08f51950 con 0x556f08ec7240 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.162+0000 7fda72a00640 1 -- 172.21.15.46:0/2980598008 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6c0035f0 con 0x556f08ec7240 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.163+0000 7fda72a00640 1 -- 172.21.15.46:0/2980598008 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7fda7c0284e0 con 0x556f08ec7240 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.163+0000 7fda8e664740 10 client.29231 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.163+0000 7fda8e664740 10 client.29231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.163+0000 7fda8e664740 20 client.29231 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.163+0000 7fda8e664740 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556f08ed5e90 0x556f08f53fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.163+0000 7fda8e664740 4 client.29231 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T07:52:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.163+0000 7fda8e664740 1 -- 172.21.15.46:0/2980598008 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f08e38d10 con 0x556f08ed5e90 2024-09-10T07:52:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.163+0000 7fda7be00640 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556f08ed5e90 0x556f08f53fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda72a00640 10 client.29231 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda7be00640 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556f08ed5e90 0x556f08f53fb0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2246bc939af2a398 server_cookie=6fce065f3ed22f63 in_seq=0 out_seq=0 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda72a00640 1 -- 172.21.15.46:0/2980598008 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f08e38d10 con 0x556f08ed5e90 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda72a00640 10 client.29231 handle_command_reply: tid=0 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 client.29231 shutdown 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 -- 172.21.15.46:0/2980598008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 msgr2=0x556f08ede3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ede3c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda7c006010 tx=0x7fda7c003e60 comp rx=0 tx=0).stop 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 -- 172.21.15.46:0/2980598008 shutdown_connections 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x556f08ed5e90 0x556f08f53fb0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f08ecb5d0 0x556f08edac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f08ecacb0 0x556f08eda700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 --2- 172.21.15.46:0/2980598008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f08ec7240 0x556f08ede3c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.164+0000 7fda8e664740 1 -- 172.21.15.46:0/2980598008 >> 172.21.15.46:0/2980598008 conn(0x556f08ed2ff0 msgr2=0x556f08ec6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.165+0000 7fda72a00640 0 client.29231 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.165+0000 7fda8e664740 1 -- 172.21.15.46:0/2980598008 shutdown_connections 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.165+0000 7fda8e664740 1 -- 172.21.15.46:0/2980598008 wait complete. 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.165+0000 7fda8e664740 20 client.29231 trim_cache size 0 max 16384 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "5797ec9b-6d2d-4df8-a357-71c2d3be57d2", 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-09-10T07:52:11.167 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:52:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 -- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda680a2b80 msgr2=0x7fda680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda680a2b80 0x7fda680a5040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda7409b960 tx=0x7fda7409bb40 comp rx=0 tx=0).stop 2024-09-10T07:52:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 -- 172.21.15.46:0/134906001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda841019c0 msgr2=0x7fda8419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda841019c0 0x7fda8419a470 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e9f0 tx=0x7fda7400eeb0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 -- 172.21.15.46:0/134906001 shutdown_connections 2024-09-10T07:52:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda841079c0 0x7fda8419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fda680a2b80 0x7fda680a5040 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda84102360 0x7fda8419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 --2- 172.21.15.46:0/134906001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda841019c0 0x7fda8419a470 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.166+0000 7fda8a600640 1 -- 172.21.15.46:0/134906001 >> 172.21.15.46:0/134906001 conn(0x7fda840fd740 msgr2=0x7fda8410d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.167+0000 7fda8a600640 1 -- 172.21.15.46:0/134906001 shutdown_connections 2024-09-10T07:52:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:11.167+0000 7fda8a600640 1 -- 172.21.15.46:0/134906001 wait complete. 2024-09-10T07:52:11.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:52:11.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:52:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub queued for path: / 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: active paths [/] 2024-09-10T07:52:12.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:12 smithi181 ceph-mon[35397]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub queued for path: / 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:52:12.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:52:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:52:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:52:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: active paths [/] 2024-09-10T07:52:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:12 smithi063 ceph-mon[35020]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-10T07:52:12.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub queued for path: / 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: active paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:12 smithi046 ceph-mon[30705]: scrub summary: idle+waiting paths [/] 2024-09-10T07:52:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 -- 172.21.15.46:0/1575447322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 msgr2=0x7f8b10076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 --2- 172.21.15.46:0/1575447322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 0x7f8b10076b00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009900 tx=0x7f8b0402a410 comp rx=0 tx=0).stop 2024-09-10T07:52:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 -- 172.21.15.46:0/1575447322 shutdown_connections 2024-09-10T07:52:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 --2- 172.21.15.46:0/1575447322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b10112a30 0x7f8b10114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 --2- 172.21.15.46:0/1575447322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10077040 0x7f8b101124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 --2- 172.21.15.46:0/1575447322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 0x7f8b10076b00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 -- 172.21.15.46:0/1575447322 >> 172.21.15.46:0/1575447322 conn(0x7f8b100ffa90 msgr2=0x7f8b10101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 -- 172.21.15.46:0/1575447322 shutdown_connections 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.787+0000 7f8b17e00640 1 -- 172.21.15.46:0/1575447322 wait complete. 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.789+0000 7f8b17e00640 1 Processor -- start 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.789+0000 7f8b17e00640 1 -- start start 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.789+0000 7f8b17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 0x7f8b101a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.789+0000 7f8b17e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b10077040 0x7f8b101a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.789+0000 7f8b17e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10112a30 0x7f8b1019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.789+0000 7f8b17e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101185f0 con 0x7f8b10076720 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.789+0000 7f8b17e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10118470 con 0x7f8b10077040 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.789+0000 7f8b17e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10118770 con 0x7f8b10112a30 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 0x7f8b101a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 0x7f8b101a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35968/0 (socket says 172.21.15.46:35968) 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 -- 172.21.15.46:0/2307033732 learned_addr learned my addr 172.21.15.46:0/2307033732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b14a00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b10077040 0x7f8b101a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15e00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10112a30 0x7f8b1019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 -- 172.21.15.46:0/2307033732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10112a30 msgr2=0x7f8b1019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10112a30 0x7f8b1019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 -- 172.21.15.46:0/2307033732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b10077040 msgr2=0x7f8b101a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b10077040 0x7f8b101a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 -- 172.21.15.46:0/2307033732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1019f090 con 0x7f8b10076720 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b14a00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b10077040 0x7f8b101a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.790+0000 7f8b15400640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 0x7f8b101a47a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8b04004770 tx=0x7f8b04003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.791+0000 7f8b0e000640 1 -- 172.21.15.46:0/2307033732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0403f070 con 0x7f8b10076720 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.791+0000 7f8b0e000640 1 -- 172.21.15.46:0/2307033732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b04031ca0 con 0x7f8b10076720 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.791+0000 7f8b0e000640 1 -- 172.21.15.46:0/2307033732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04035030 con 0x7f8b10076720 2024-09-10T07:52:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.791+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1019f320 con 0x7f8b10076720 2024-09-10T07:52:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.791+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b101ab670 con 0x7f8b10076720 2024-09-10T07:52:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.792+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b10076b00 con 0x7f8b10076720 2024-09-10T07:52:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.794+0000 7f8b0e000640 1 -- 172.21.15.46:0/2307033732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b04042070 con 0x7f8b10076720 2024-09-10T07:52:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.795+0000 7f8b0e000640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8af80a2c00 0x7f8af80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.795+0000 7f8b0e000640 1 -- 172.21.15.46:0/2307033732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f8b040efac0 con 0x7f8b10076720 2024-09-10T07:52:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.796+0000 7f8b14a00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8af80a2c00 0x7f8af80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.796+0000 7f8b14a00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8af80a2c00 0x7f8af80a50c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8b10079320 tx=0x7f8afc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:12.800+0000 7f8b0e000640 1 -- 172.21.15.46:0/2307033732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b040bbff0 con 0x7f8b10076720 2024-09-10T07:52:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:13.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.052+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b1010dbc0 con 0x7f8b10076720 2024-09-10T07:52:13.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.053+0000 7f8b0e000640 1 -- 172.21.15.46:0/2307033732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11986 (secure 0 0 0) 0x7f8b040c02f0 con 0x7f8b10076720 2024-09-10T07:52:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:13.053 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":60,"btime":"2024-09-10T07:51:54:063936+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:51:53.066973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:52:13.054 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 60 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8af80a2c00 msgr2=0x7f8af80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8af80a2c00 0x7f8af80a50c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8b10079320 tx=0x7f8afc00b410 comp rx=0 tx=0).stop 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 msgr2=0x7f8b101a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 0x7f8b101a47a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8b04004770 tx=0x7f8b04003c10 comp rx=0 tx=0).stop 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 shutdown_connections 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b10112a30 0x7f8b1019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b10077040 0x7f8b101a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f8af80a2c00 0x7f8af80a50c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 --2- 172.21.15.46:0/2307033732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10076720 0x7f8b101a47a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.055+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 >> 172.21.15.46:0/2307033732 conn(0x7f8b100ffa90 msgr2=0x7f8b10101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.056+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 shutdown_connections 2024-09-10T07:52:13.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.056+0000 7f8b17e00640 1 -- 172.21.15.46:0/2307033732 wait complete. 2024-09-10T07:52:13.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:13 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2307033732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:13.399 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:13 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2307033732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:13.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:13 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2307033732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 -- 172.21.15.46:0/3304276480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed41019c0 msgr2=0x7f3ed4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/3304276480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed41019c0 0x7f3ed4101e20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009860 tx=0x7f3ebc02a340 comp rx=0 tx=0).stop 2024-09-10T07:52:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 -- 172.21.15.46:0/3304276480 shutdown_connections 2024-09-10T07:52:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/3304276480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed4102360 0x7f3ed410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/3304276480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed41019c0 0x7f3ed4101e20 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/3304276480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed41079c0 0x7f3ed4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 -- 172.21.15.46:0/3304276480 >> 172.21.15.46:0/3304276480 conn(0x7f3ed40fd740 msgr2=0x7f3ed40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 -- 172.21.15.46:0/3304276480 shutdown_connections 2024-09-10T07:52:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.897+0000 7f3ed3e00640 1 -- 172.21.15.46:0/3304276480 wait complete. 2024-09-10T07:52:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.898+0000 7f3ed3e00640 1 Processor -- start 2024-09-10T07:52:13.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.898+0000 7f3ed3e00640 1 -- start start 2024-09-10T07:52:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.898+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed41019c0 0x7f3ed419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed4102360 0x7f3ed419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed41079c0 0x7f3ed419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4111dc0 con 0x7f3ed4102360 2024-09-10T07:52:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4111c40 con 0x7f3ed41079c0 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4111f40 con 0x7f3ed41019c0 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed41019c0 0x7f3ed419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed4102360 0x7f3ed419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed41019c0 0x7f3ed419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48212/0 (socket says 172.21.15.46:48212) 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed2a00640 1 -- 172.21.15.46:0/53803706 learned_addr learned my addr 172.21.15.46:0/53803706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed3400640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed41079c0 0x7f3ed419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed2a00640 1 -- 172.21.15.46:0/53803706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed41079c0 msgr2=0x7f3ed419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed41079c0 0x7f3ed419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed2a00640 1 -- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed4102360 msgr2=0x7f3ed419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed4102360 0x7f3ed419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.899+0000 7f3ed2a00640 1 -- 172.21.15.46:0/53803706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed419f470 con 0x7f3ed41019c0 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.900+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed41019c0 0x7f3ed419a4c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e9f0 tx=0x7f3ec400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.900+0000 7f3ecb400640 1 -- 172.21.15.46:0/53803706 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400cd80 con 0x7f3ed41019c0 2024-09-10T07:52:13.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.900+0000 7f3ecb400640 1 -- 172.21.15.46:0/53803706 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec4005500 con 0x7f3ed41019c0 2024-09-10T07:52:13.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.900+0000 7f3ecb400640 1 -- 172.21.15.46:0/53803706 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4010690 con 0x7f3ed41019c0 2024-09-10T07:52:13.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.900+0000 7f3ed3e00640 1 -- 172.21.15.46:0/53803706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed419f760 con 0x7f3ed41019c0 2024-09-10T07:52:13.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.900+0000 7f3ed3e00640 1 -- 172.21.15.46:0/53803706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed41a7040 con 0x7f3ed41019c0 2024-09-10T07:52:13.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.903+0000 7f3edc248740 1 Processor -- start 2024-09-10T07:52:13.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.904+0000 7f3edc248740 1 -- start start 2024-09-10T07:52:13.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.904+0000 7f3ecb400640 1 -- 172.21.15.46:0/53803706 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ec40046c0 con 0x7f3ed41019c0 2024-09-10T07:52:13.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.904+0000 7f3edc248740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d306134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.904+0000 7f3edc248740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 0x55d30613e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.904+0000 7f3edc248740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306141320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.904+0000 7f3edc248740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d30609a3e0 con 0x55d306134670 2024-09-10T07:52:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.904+0000 7f3edc248740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d306041d80 con 0x55d30613ef30 2024-09-10T07:52:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.904+0000 7f3edc248740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d3060963a0 con 0x55d306133d50 2024-09-10T07:52:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ecbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ecbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306141320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49558/0 (socket says 172.21.15.46:49558) 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ecbe00640 1 -- 172.21.15.46:0/3788611315 learned_addr learned my addr 172.21.15.46:0/3788611315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 0x55d30613e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ed3400640 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d306134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ecbe00640 1 -- 172.21.15.46:0/3788611315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 msgr2=0x55d306134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d306134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ecbe00640 1 -- 172.21.15.46:0/3788611315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 msgr2=0x55d30613e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 0x55d30613e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ecbe00640 1 -- 172.21.15.46:0/3788611315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d306060ae0 con 0x55d30613ef30 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.905+0000 7f3ed3400640 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d306134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.906+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306141320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005e00 tx=0x7f3ebc009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ced230b357d7056 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:13.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.906+0000 7f3ec3400640 1 -- 172.21.15.46:0/3788611315 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc037040 con 0x55d30613ef30 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.906+0000 7f3ec3400640 1 -- 172.21.15.46:0/3788611315 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ebc030e40 con 0x55d30613ef30 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.906+0000 7f3ec3400640 1 -- 172.21.15.46:0/3788611315 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc003a90 con 0x55d30613ef30 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.906+0000 7f3ecb400640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3eb80a2c00 0x7f3eb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.906+0000 7f3ecb400640 1 -- 172.21.15.46:0/53803706 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ec40cf710 con 0x7f3ed41019c0 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.906+0000 7f3ed3400640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3eb80a2c00 0x7f3eb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3edc248740 1 -- 172.21.15.46:0/3788611315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 msgr2=0x55d306141320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3edc248740 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306141320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005e00 tx=0x7f3ebc009510 comp rx=0 tx=0).stop 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3edc248740 1 -- 172.21.15.46:0/3788611315 shutdown_connections 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3edc248740 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306141320 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3edc248740 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 0x55d30613e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3edc248740 1 --2- 172.21.15.46:0/3788611315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d306134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3edc248740 1 -- 172.21.15.46:0/3788611315 >> 172.21.15.46:0/3788611315 conn(0x55d3061302e0 msgr2=0x55d3061306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3ed3400640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3eb80a2c00 0x7f3eb80a50c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc005fd0 tx=0x7f3ecc010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:13.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.907+0000 7f3edc248740 1 -- 172.21.15.46:0/3788611315 shutdown_connections 2024-09-10T07:52:13.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.908+0000 7f3edc248740 1 -- 172.21.15.46:0/3788611315 wait complete. 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.908+0000 7f3edc248740 1 -- start start 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3edc248740 1 Processor -- start 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3edc248740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306133d50 0x55d30613bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3edc248740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d30613c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3edc248740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d30613ef30 0x55d30613c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3edc248740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d306141ba0 con 0x55d306133d50 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3edc248740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3060550f0 con 0x55d306134670 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3edc248740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d306141d20 con 0x55d30613ef30 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3edc248740 10 client.0 _ll_register_callbacks cb 0x55d3060469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306133d50 0x55d30613bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3ecbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d30613c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306133d50 0x55d30613bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36000/0 (socket says 172.21.15.46:36000) 2024-09-10T07:52:13.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.909+0000 7f3ed2a00640 1 -- 172.21.15.46:0/2589747979 learned_addr learned my addr 172.21.15.46:0/2589747979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:13.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3ed3400640 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d30613ef30 0x55d30613c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3ecbe00640 1 -- 172.21.15.46:0/2589747979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d30613ef30 msgr2=0x55d30613c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d30613ef30 0x55d30613c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3ecbe00640 1 -- 172.21.15.46:0/2589747979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306133d50 msgr2=0x55d30613bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306133d50 0x55d30613bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3ecbe00640 1 -- 172.21.15.46:0/2589747979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d305e0b3c0 con 0x55d306134670 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d30613c2f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005940 tx=0x7f3ebc003690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a776ff0c4cf0b77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3eb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3eb7e00640 1 -- 172.21.15.46:0/2589747979 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc037040 con 0x55d306134670 2024-09-10T07:52:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.910+0000 7f3edc248740 1 -- 172.21.15.46:0/2589747979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ebc030e40 con 0x55d306134670 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3eb7e00640 1 -- 172.21.15.46:0/2589747979 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ebc03d650 con 0x55d306134670 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3eb7e00640 1 -- 172.21.15.46:0/2589747979 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc03d850 con 0x55d306134670 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3edc248740 10 client.29249 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3edc248740 1 -- 172.21.15.46:0/2589747979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55d30614d810 con 0x55d306134670 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3eb7e00640 1 -- 172.21.15.46:0/2589747979 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ebc030e40 con 0x55d306134670 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3eb7e00640 1 -- 172.21.15.46:0/2589747979 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f3ebc04eb00 con 0x55d306134670 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3edc248740 10 client.29249 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3edc248740 10 client.29249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3edc248740 20 client.29249 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.911+0000 7f3edc248740 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d30614eb60 0x55d306150f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.912+0000 7f3edc248740 4 client.29249 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.912+0000 7f3edc248740 1 -- 172.21.15.46:0/2589747979 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3060a7950 con 0x55d30614eb60 2024-09-10T07:52:13.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.912+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d30614eb60 0x55d306150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.912+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d30614eb60 0x55d306150f50 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80cc6842d4727a61 server_cookie=e61a5eef8237bde5 in_seq=0 out_seq=0 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.912+0000 7f3eb7e00640 10 client.29249 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.913+0000 7f3eb7e00640 1 -- 172.21.15.46:0/2589747979 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3060a7950 con 0x55d30614eb60 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.913+0000 7f3eb7e00640 10 client.29249 handle_command_reply: tid=0 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.913+0000 7f3edc248740 1 client.29249 shutdown 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.913+0000 7f3edc248740 1 -- 172.21.15.46:0/2589747979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 msgr2=0x55d30613c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.913+0000 7f3edc248740 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d30613c2f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005940 tx=0x7f3ebc003690 comp rx=0 tx=0).stop 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.913+0000 7f3edc248740 1 -- 172.21.15.46:0/2589747979 shutdown_connections 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.913+0000 7f3edc248740 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d30614eb60 0x55d306150f50 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.914+0000 7f3edc248740 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d30613ef30 0x55d30613c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.914+0000 7f3edc248740 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d30613c2f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.914+0000 7f3edc248740 1 --2- 172.21.15.46:0/2589747979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306133d50 0x55d30613bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.914+0000 7f3eb7e00640 0 client.29249 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:13.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.914+0000 7f3edc248740 1 -- 172.21.15.46:0/2589747979 >> 172.21.15.46:0/2589747979 conn(0x55d3061302e0 msgr2=0x55d306136010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.914+0000 7f3edc248740 1 -- 172.21.15.46:0/2589747979 shutdown_connections 2024-09-10T07:52:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.914+0000 7f3edc248740 1 -- 172.21.15.46:0/2589747979 wait complete. 2024-09-10T07:52:13.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.914+0000 7f3edc248740 20 client.29249 trim_cache size 0 max 16384 2024-09-10T07:52:13.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.932+0000 7f3edc248740 1 Processor -- start 2024-09-10T07:52:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.932+0000 7f3edc248740 1 -- start start 2024-09-10T07:52:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.933+0000 7f3edc248740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d30613dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.933+0000 7f3edc248740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 0x55d3061565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.933+0000 7f3edc248740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306156b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.933+0000 7f3edc248740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41b0c90 con 0x55d306134670 2024-09-10T07:52:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.933+0000 7f3edc248740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41b2950 con 0x55d30613ef30 2024-09-10T07:52:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.933+0000 7f3edc248740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41b1360 con 0x55d306133d50 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d30613dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 0x55d3061565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d30613dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:48230/0 (socket says 172.21.15.46:48230) 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed2a00640 1 -- 172.21.15.46:0/2950792581 learned_addr learned my addr 172.21.15.46:0/2950792581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed2a00640 1 -- 172.21.15.46:0/2950792581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 msgr2=0x55d306156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/2950792581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed2a00640 1 -- 172.21.15.46:0/2950792581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 msgr2=0x55d3061565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/2950792581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 0x55d3061565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.934+0000 7f3ed2a00640 1 -- 172.21.15.46:0/2950792581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30613e720 con 0x55d306133d50 2024-09-10T07:52:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.935+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/2950792581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d30613dd20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40985c0 tx=0x7f3ec40d5550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6654af366a253a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.935+0000 7f3ec2a00640 1 -- 172.21.15.46:0/2950792581 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec409b730 con 0x55d306133d50 2024-09-10T07:52:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.935+0000 7f3ec2a00640 1 -- 172.21.15.46:0/2950792581 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec4086040 con 0x55d306133d50 2024-09-10T07:52:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.935+0000 7f3ec2a00640 1 -- 172.21.15.46:0/2950792581 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec409b550 con 0x55d306133d50 2024-09-10T07:52:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 -- 172.21.15.46:0/2950792581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 msgr2=0x55d30613dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 --2- 172.21.15.46:0/2950792581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d30613dd20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40985c0 tx=0x7f3ec40d5550 comp rx=0 tx=0).stop 2024-09-10T07:52:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 -- 172.21.15.46:0/2950792581 shutdown_connections 2024-09-10T07:52:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 --2- 172.21.15.46:0/2950792581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d30613ef30 0x55d306156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 --2- 172.21.15.46:0/2950792581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d306134670 0x55d3061565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 --2- 172.21.15.46:0/2950792581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d30613dd20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 -- 172.21.15.46:0/2950792581 >> 172.21.15.46:0/2950792581 conn(0x55d3061302e0 msgr2=0x55d30613d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 -- 172.21.15.46:0/2950792581 shutdown_connections 2024-09-10T07:52:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.936+0000 7f3edc248740 1 -- 172.21.15.46:0/2950792581 wait complete. 2024-09-10T07:52:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.937+0000 7f3edc248740 1 -- start start 2024-09-10T07:52:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.937+0000 7f3edc248740 1 Processor -- start 2024-09-10T07:52:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.937+0000 7f3edc248740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d3061477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.937+0000 7f3edc248740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d306142af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.937+0000 7f3edc248740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d30613ef30 0x55d306143090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.937+0000 7f3edc248740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41b21c0 con 0x55d30613ef30 2024-09-10T07:52:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.937+0000 7f3edc248740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41b1af0 con 0x55d306134670 2024-09-10T07:52:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.937+0000 7f3edc248740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41aec70 con 0x55d306133d50 2024-09-10T07:52:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3edc248740 10 client.0 _ll_register_callbacks cb 0x7f3ec4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ecbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d306142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ed2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d3061477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ecbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d306142af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49598/0 (socket says 172.21.15.46:49598) 2024-09-10T07:52:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ecbe00640 1 -- 172.21.15.46:0/489673951 learned_addr learned my addr 172.21.15.46:0/489673951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ed3400640 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d30613ef30 0x55d306143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ecbe00640 1 -- 172.21.15.46:0/489673951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 msgr2=0x55d3061477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d3061477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ecbe00640 1 -- 172.21.15.46:0/489673951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d30613ef30 msgr2=0x55d306143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d30613ef30 0x55d306143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ecbe00640 1 -- 172.21.15.46:0/489673951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3061438c0 con 0x55d306134670 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.938+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d3061477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.939+0000 7f3ecbe00640 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d306142af0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc04ef20 tx=0x7f3ebc032840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1006aa6e8b242188 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.939+0000 7f3eb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.939+0000 7f3eb7e00640 1 -- 172.21.15.46:0/489673951 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc060060 con 0x55d306134670 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.939+0000 7f3eb7e00640 1 -- 172.21.15.46:0/489673951 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3ebc03d9f0 con 0x55d306134670 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.939+0000 7f3edc248740 1 -- 172.21.15.46:0/489673951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec4086040 con 0x55d306134670 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.939+0000 7f3eb7e00640 1 -- 172.21.15.46:0/489673951 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc037070 con 0x55d306134670 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.939+0000 7f3edc248740 10 client.29255 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.939+0000 7f3edc248740 1 -- 172.21.15.46:0/489673951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55d3061b9690 con 0x55d306134670 2024-09-10T07:52:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3eb7e00640 1 -- 172.21.15.46:0/489673951 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec4086040 con 0x55d306134670 2024-09-10T07:52:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3eb7e00640 1 -- 172.21.15.46:0/489673951 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f3ebc04e470 con 0x55d306134670 2024-09-10T07:52:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3edc248740 10 client.29255 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3edc248740 10 client.29255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3edc248740 20 client.29255 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3edc248740 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3061b9d50 0x55d3061bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3edc248740 4 client.29255 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:52:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3edc248740 1 -- 172.21.15.46:0/489673951 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d3060a1e60 con 0x55d3061b9d50 2024-09-10T07:52:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.940+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3061b9d50 0x55d3061bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3ed2a00640 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3061b9d50 0x55d3061bc170 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1d6fb73ab000b8 server_cookie=f328114bca72af86 in_seq=0 out_seq=0 2024-09-10T07:52:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3eb7e00640 10 client.29255 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3eb7e00640 1 -- 172.21.15.46:0/489673951 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d3060a1e60 con 0x55d3061b9d50 2024-09-10T07:52:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3eb7e00640 10 client.29255 handle_command_reply: tid=0 2024-09-10T07:52:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 client.29255 shutdown 2024-09-10T07:52:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 -- 172.21.15.46:0/489673951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 msgr2=0x55d306142af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d306142af0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc04ef20 tx=0x7f3ebc032840 comp rx=0 tx=0).stop 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 -- 172.21.15.46:0/489673951 shutdown_connections 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55d3061b9d50 0x55d3061bc170 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d30613ef30 0x55d306143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d306134670 0x55d306142af0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 --2- 172.21.15.46:0/489673951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d306133d50 0x55d3061477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3eb7e00640 0 client.29255 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.941+0000 7f3edc248740 1 -- 172.21.15.46:0/489673951 >> 172.21.15.46:0/489673951 conn(0x55d3061302e0 msgr2=0x55d30613ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.942+0000 7f3edc248740 1 -- 172.21.15.46:0/489673951 shutdown_connections 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.942+0000 7f3edc248740 1 -- 172.21.15.46:0/489673951 wait complete. 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.942+0000 7f3edc248740 20 client.29255 trim_cache size 0 max 16384 2024-09-10T07:52:13.944 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout: "5797ec9b-6d2d-4df8-a357-71c2d3be57d2": { 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "5797ec9b-6d2d-4df8-a357-71c2d3be57d2", 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:13.945 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 -- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3eb80a2c00 msgr2=0x7f3eb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3eb80a2c00 0x7f3eb80a50c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc005fd0 tx=0x7f3ecc010040 comp rx=0 tx=0).stop 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 -- 172.21.15.46:0/53803706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed41019c0 msgr2=0x7f3ed419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed41019c0 0x7f3ed419a4c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e9f0 tx=0x7f3ec400edf0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 -- 172.21.15.46:0/53803706 shutdown_connections 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed41079c0 0x7f3ed419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3eb80a2c00 0x7f3eb80a50c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed4102360 0x7f3ed419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 --2- 172.21.15.46:0/53803706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ed41019c0 0x7f3ed419a4c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 -- 172.21.15.46:0/53803706 >> 172.21.15.46:0/53803706 conn(0x7f3ed40fd740 msgr2=0x7f3ed40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 -- 172.21.15.46:0/53803706 shutdown_connections 2024-09-10T07:52:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:13.943+0000 7f3ed3e00640 1 -- 172.21.15.46:0/53803706 wait complete. 2024-09-10T07:52:14.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:14 smithi181 ceph-mon[35397]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-09-10T07:52:14.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:14 smithi063 ceph-mon[35020]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-09-10T07:52:14.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:14 smithi046 ceph-mon[30705]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-09-10T07:52:14.814 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id 67ec92b3 --release --await' 2024-09-10T07:52:14.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id 67ec92b3 --release --await 2024-09-10T07:52:15.466 INFO:tasks.cephfs.filesystem:scrub status for tag:5797ec9b-6d2d-4df8-a357-71c2d3be57d2 - {'path': '/', 'tag': '5797ec9b-6d2d-4df8-a357-71c2d3be57d2', 'options': 'recursive,force'} 2024-09-10T07:52:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 895.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-10T07:52:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 -- 172.21.15.46:0/602853378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648105910 msgr2=0x7f264810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 --2- 172.21.15.46:0/602853378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648105910 0x7f264810c1a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2640009ca0 tx=0x7f264002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 -- 172.21.15.46:0/602853378 shutdown_connections 2024-09-10T07:52:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 --2- 172.21.15.46:0/602853378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648105910 0x7f264810c1a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 --2- 172.21.15.46:0/602853378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2648104f50 0x7f26481053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 --2- 172.21.15.46:0/602853378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2648103d60 0x7f2648104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 -- 172.21.15.46:0/602853378 >> 172.21.15.46:0/602853378 conn(0x7f26480ff4e0 msgr2=0x7f2648101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 -- 172.21.15.46:0/602853378 shutdown_connections 2024-09-10T07:52:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.805+0000 7f2650600640 1 -- 172.21.15.46:0/602853378 wait complete. 2024-09-10T07:52:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f2650600640 1 Processor -- start 2024-09-10T07:52:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f2650600640 1 -- start start 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f2650600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648103d60 0x7f2648199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f2650600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2648104f50 0x7f264819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f2650600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2648105910 0x7f26481a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f2650600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f264810f9e0 con 0x7f2648103d60 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f2650600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f264810f860 con 0x7f2648104f50 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f2650600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f264810fb60 con 0x7f2648105910 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f264dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648103d60 0x7f2648199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f264dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648103d60 0x7f2648199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39660/0 (socket says 172.21.15.46:39660) 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.806+0000 7f264dc00640 1 -- 172.21.15.46:0/877984768 learned_addr learned my addr 172.21.15.46:0/877984768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.807+0000 7f264d200640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2648104f50 0x7f264819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.807+0000 7f264e600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2648105910 0x7f26481a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.807+0000 7f264dc00640 1 -- 172.21.15.46:0/877984768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2648105910 msgr2=0x7f26481a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.807+0000 7f264dc00640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2648105910 0x7f26481a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.807+0000 7f264dc00640 1 -- 172.21.15.46:0/877984768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2648104f50 msgr2=0x7f264819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.807+0000 7f264dc00640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2648104f50 0x7f264819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.807+0000 7f264dc00640 1 -- 172.21.15.46:0/877984768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26481a1bf0 con 0x7f2648103d60 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.807+0000 7f264dc00640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648103d60 0x7f2648199fd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f263c00e9c0 tx=0x7f263c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.808+0000 7f2646000640 1 -- 172.21.15.46:0/877984768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c00cd80 con 0x7f2648103d60 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.808+0000 7f2646000640 1 -- 172.21.15.46:0/877984768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f263c0035f0 con 0x7f2648103d60 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.808+0000 7f2650600640 1 -- 172.21.15.46:0/877984768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26481a1ee0 con 0x7f2648103d60 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.808+0000 7f2646000640 1 -- 172.21.15.46:0/877984768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c010690 con 0x7f2648103d60 2024-09-10T07:52:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.808+0000 7f2650600640 1 -- 172.21.15.46:0/877984768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26481a22f0 con 0x7f2648103d60 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f265361e740 1 Processor -- start 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f265361e740 1 -- start start 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f2646000640 1 -- 172.21.15.46:0/877984768 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f263c0037a0 con 0x7f2648103d60 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f265361e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a605490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f265361e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a60a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f265361e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a60aa60 0x561a7a60cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f265361e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561a7a56a100 con 0x561a7a6059d0 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f265361e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561a7a5121b0 con 0x561a7a60aa60 2024-09-10T07:52:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.811+0000 7f265361e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561a7a5660c0 con 0x561a7a605090 2024-09-10T07:52:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a60aa60 0x561a7a60cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a605490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a60a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a60aa60 0x561a7a60cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59810/0 (socket says 172.21.15.46:59810) 2024-09-10T07:52:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264d200640 1 -- 172.21.15.46:0/71740295 learned_addr learned my addr 172.21.15.46:0/71740295 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a605490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43858/0 (socket says 172.21.15.46:43858) 2024-09-10T07:52:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264d200640 1 -- 172.21.15.46:0/71740295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 msgr2=0x561a7a605490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264d200640 1 --2- 172.21.15.46:0/71740295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a605490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264d200640 1 -- 172.21.15.46:0/71740295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 msgr2=0x561a7a60a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264d200640 1 --2- 172.21.15.46:0/71740295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a60a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.812+0000 7f264d200640 1 -- 172.21.15.46:0/71740295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7a27d210 con 0x561a7a60aa60 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.813+0000 7f264d200640 1 --2- 172.21.15.46:0/71740295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a60aa60 0x561a7a60cf20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f263400bc90 tx=0x7f26340107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c244214304e0322 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.813+0000 7f2646000640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26300a2c30 0x7f26300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.813+0000 7f263aa00640 1 -- 172.21.15.46:0/71740295 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634005220 con 0x561a7a60aa60 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.813+0000 7f263aa00640 1 -- 172.21.15.46:0/71740295 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f263400be40 con 0x561a7a60aa60 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.813+0000 7f263aa00640 1 -- 172.21.15.46:0/71740295 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634011770 con 0x561a7a60aa60 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.813+0000 7f2646000640 1 -- 172.21.15.46:0/877984768 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f263c0cf860 con 0x7f2648103d60 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.813+0000 7f264e600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26300a2c30 0x7f26300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 -- 172.21.15.46:0/71740295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a60aa60 msgr2=0x561a7a60cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 --2- 172.21.15.46:0/71740295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a60aa60 0x561a7a60cf20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f263400bc90 tx=0x7f26340107a0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 -- 172.21.15.46:0/71740295 shutdown_connections 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 --2- 172.21.15.46:0/71740295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a60aa60 0x561a7a60cf20 secure :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f263400bc90 tx=0x7f26340107a0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 --2- 172.21.15.46:0/71740295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a60a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 --2- 172.21.15.46:0/71740295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a605490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f264e600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26300a2c30 0x7f26300a50f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2640005cc0 tx=0x7f2640031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 -- 172.21.15.46:0/71740295 >> 172.21.15.46:0/71740295 conn(0x561a7a600700 msgr2=0x561a7a600b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 -- 172.21.15.46:0/71740295 shutdown_connections 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.814+0000 7f265361e740 1 -- 172.21.15.46:0/71740295 wait complete. 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.815+0000 7f265361e740 1 -- start start 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.815+0000 7f265361e740 1 Processor -- start 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f265361e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f265361e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a60fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f265361e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a616db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f265361e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561a7a60da30 con 0x561a7a6149c0 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f265361e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561a7a4f3090 con 0x561a7a605090 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f265361e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561a7a60dbb0 con 0x561a7a6059d0 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f265361e740 10 client.0 _ll_register_callbacks cb 0x561a7a547f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f264d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a60fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f264dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f264d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a60fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43864/0 (socket says 172.21.15.46:43864) 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f264dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59816/0 (socket says 172.21.15.46:59816) 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f264d200640 1 -- 172.21.15.46:0/4069422555 learned_addr learned my addr 172.21.15.46:0/4069422555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.816+0000 7f264e600640 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a616db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f264dc00640 1 -- 172.21.15.46:0/4069422555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 msgr2=0x561a7a60fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f264dc00640 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a60fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f264dc00640 1 -- 172.21.15.46:0/4069422555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 msgr2=0x561a7a616db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f264dc00640 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a616db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f264dc00640 1 -- 172.21.15.46:0/4069422555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7a349620 con 0x561a7a605090 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f264dc00640 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60f530 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x561a7a606400 tx=0x7f263c097920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c46465e82550e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f262f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f262f400640 1 -- 172.21.15.46:0/4069422555 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c09b820 con 0x561a7a605090 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f265361e740 1 -- 172.21.15.46:0/4069422555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a7a577830 con 0x561a7a605090 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f262f400640 1 -- 172.21.15.46:0/4069422555 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f263c0a44c0 con 0x561a7a605090 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f262f400640 1 -- 172.21.15.46:0/4069422555 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f263c09b640 con 0x561a7a605090 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f265361e740 10 client.29267 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.817+0000 7f265361e740 1 -- 172.21.15.46:0/4069422555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561a7a619cc0 con 0x561a7a605090 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f262f400640 1 -- 172.21.15.46:0/4069422555 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561a7a577830 con 0x561a7a605090 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f262f400640 1 -- 172.21.15.46:0/4069422555 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f263c09b130 con 0x561a7a605090 2024-09-10T07:52:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f265361e740 10 client.29267 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f265361e740 10 client.29267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f265361e740 20 client.29267 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f265361e740 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561a7a61b360 0x561a7a61d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f265361e740 4 client.29267 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f265361e740 1 -- 172.21.15.46:0/4069422555 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a7a571b80 con 0x561a7a61b360 2024-09-10T07:52:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.818+0000 7f264d200640 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561a7a61b360 0x561a7a61d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.819+0000 7f264d200640 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561a7a61b360 0x561a7a61d7d0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5c5dc74cb1a6e4 server_cookie=f196e8af8eb4db2d in_seq=0 out_seq=0 2024-09-10T07:52:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.819+0000 7f262f400640 10 client.29267 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f262f400640 1 -- 172.21.15.46:0/4069422555 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a7a571b80 con 0x561a7a61b360 2024-09-10T07:52:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f262f400640 10 client.29267 handle_command_reply: tid=0 2024-09-10T07:52:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 client.29267 shutdown 2024-09-10T07:52:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 -- 172.21.15.46:0/4069422555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 msgr2=0x561a7a60f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60f530 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x561a7a606400 tx=0x7f263c097920 comp rx=0 tx=0).stop 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 -- 172.21.15.46:0/4069422555 shutdown_connections 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561a7a61b360 0x561a7a61d7d0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a616db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a60fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f262f400640 0 client.29267 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 --2- 172.21.15.46:0/4069422555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60f530 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 -- 172.21.15.46:0/4069422555 >> 172.21.15.46:0/4069422555 conn(0x561a7a600700 msgr2=0x561a7a601f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 -- 172.21.15.46:0/4069422555 shutdown_connections 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 1 -- 172.21.15.46:0/4069422555 wait complete. 2024-09-10T07:52:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.820+0000 7f265361e740 20 client.29267 trim_cache size 0 max 16384 2024-09-10T07:52:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.837+0000 7f265361e740 1 Processor -- start 2024-09-10T07:52:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f265361e740 1 -- start start 2024-09-10T07:52:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f265361e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f265361e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a5ff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f265361e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a5ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f265361e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26481a8510 con 0x561a7a6149c0 2024-09-10T07:52:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f265361e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26481aa1d0 con 0x561a7a605090 2024-09-10T07:52:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f265361e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26481a8be0 con 0x561a7a6059d0 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f264e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a5ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f264dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f264d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a5ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f264e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a5ffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39700/0 (socket says 172.21.15.46:39700) 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.838+0000 7f264e600640 1 -- 172.21.15.46:0/4162234843 learned_addr learned my addr 172.21.15.46:0/4162234843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f264dc00640 1 -- 172.21.15.46:0/4162234843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 msgr2=0x561a7a5ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f264dc00640 1 --2- 172.21.15.46:0/4162234843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a5ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f264dc00640 1 -- 172.21.15.46:0/4162234843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 msgr2=0x561a7a5ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f264dc00640 1 --2- 172.21.15.46:0/4162234843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a5ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f264dc00640 1 -- 172.21.15.46:0/4162234843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7a60a0f0 con 0x561a7a605090 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f264e600640 1 --2- 172.21.15.46:0/4162234843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a5ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f264dc00640 1 --2- 172.21.15.46:0/4162234843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60fb00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f263c09bc70 tx=0x7f263c0d55a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd95302624cf680 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f263a000640 1 -- 172.21.15.46:0/4162234843 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c09b820 con 0x561a7a605090 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f263a000640 1 -- 172.21.15.46:0/4162234843 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f263c09bce0 con 0x561a7a605090 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.839+0000 7f263a000640 1 -- 172.21.15.46:0/4162234843 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c0a44c0 con 0x561a7a605090 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 -- 172.21.15.46:0/4162234843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 msgr2=0x561a7a60fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 --2- 172.21.15.46:0/4162234843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60fb00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f263c09bc70 tx=0x7f263c0d55a0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 -- 172.21.15.46:0/4162234843 shutdown_connections 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 --2- 172.21.15.46:0/4162234843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6149c0 0x561a7a5ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 --2- 172.21.15.46:0/4162234843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a6059d0 0x561a7a5ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 --2- 172.21.15.46:0/4162234843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a605090 0x561a7a60fb00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 -- 172.21.15.46:0/4162234843 >> 172.21.15.46:0/4162234843 conn(0x561a7a600700 msgr2=0x561a7a5ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 -- 172.21.15.46:0/4162234843 shutdown_connections 2024-09-10T07:52:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.840+0000 7f265361e740 1 -- 172.21.15.46:0/4162234843 wait complete. 2024-09-10T07:52:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 1 -- start start 2024-09-10T07:52:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 1 Processor -- start 2024-09-10T07:52:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a62cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a62d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a6149c0 0x561a7a62e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26481a9a40 con 0x561a7a6059d0 2024-09-10T07:52:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26481a9370 con 0x561a7a6149c0 2024-09-10T07:52:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26481a64f0 con 0x561a7a605090 2024-09-10T07:52:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f265361e740 10 client.0 _ll_register_callbacks cb 0x7f263c0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.841+0000 7f264d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a62d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a62d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39712/0 (socket says 172.21.15.46:39712) 2024-09-10T07:52:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264d200640 1 -- 172.21.15.46:0/563346619 learned_addr learned my addr 172.21.15.46:0/563346619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264e600640 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a6149c0 0x561a7a62e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264dc00640 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a62cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264d200640 1 -- 172.21.15.46:0/563346619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 msgr2=0x561a7a62cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264d200640 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a62cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264d200640 1 -- 172.21.15.46:0/563346619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a6149c0 msgr2=0x561a7a62e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264d200640 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a6149c0 0x561a7a62e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264d200640 1 -- 172.21.15.46:0/563346619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7a62e930 con 0x561a7a6059d0 2024-09-10T07:52:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f264d200640 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a62d310 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2634005d00 tx=0x7f2634005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a084a363c5f5d71a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.842+0000 7f262f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f262f400640 1 -- 172.21.15.46:0/563346619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263402c070 con 0x561a7a6059d0 2024-09-10T07:52:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f262f400640 1 -- 172.21.15.46:0/563346619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f2634005400 con 0x561a7a6059d0 2024-09-10T07:52:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 1 -- 172.21.15.46:0/563346619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f263c09bce0 con 0x561a7a6059d0 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f262f400640 1 -- 172.21.15.46:0/563346619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2634005600 con 0x561a7a6059d0 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 10 client.18339 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 1 -- 172.21.15.46:0/563346619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561a7a685f30 con 0x561a7a6059d0 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f262f400640 1 -- 172.21.15.46:0/563346619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f263c09bce0 con 0x561a7a6059d0 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f262f400640 1 -- 172.21.15.46:0/563346619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f2634029810 con 0x561a7a6059d0 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 10 client.18339 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 10 client.18339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 20 client.18339 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561a7a686800 0x561a7a688c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 4 client.18339 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:52:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.843+0000 7f265361e740 1 -- 172.21.15.46:0/563346619 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a7a4ee7c0 con 0x561a7a686800 2024-09-10T07:52:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.844+0000 7f264dc00640 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561a7a686800 0x561a7a688c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.844+0000 7f264dc00640 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561a7a686800 0x561a7a688c20 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3028b905c274426f server_cookie=5d33b7f06814a338 in_seq=0 out_seq=0 2024-09-10T07:52:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.844+0000 7f262f400640 10 client.18339 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.852+0000 7f262f400640 1 -- 172.21.15.46:0/563346619 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a7a4ee7c0 con 0x561a7a686800 2024-09-10T07:52:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.852+0000 7f262f400640 10 client.18339 handle_command_reply: tid=0 2024-09-10T07:52:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.852+0000 7f265361e740 1 client.18339 shutdown 2024-09-10T07:52:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 -- 172.21.15.46:0/563346619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 msgr2=0x561a7a62d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a62d310 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2634005d00 tx=0x7f2634005ab0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 -- 172.21.15.46:0/563346619 shutdown_connections 2024-09-10T07:52:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561a7a686800 0x561a7a688c20 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561a7a6149c0 0x561a7a62e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a7a6059d0 0x561a7a62d310 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 --2- 172.21.15.46:0/563346619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561a7a605090 0x561a7a62cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f262f400640 0 client.18339 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 -- 172.21.15.46:0/563346619 >> 172.21.15.46:0/563346619 conn(0x561a7a600700 msgr2=0x561a7a5ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 -- 172.21.15.46:0/563346619 shutdown_connections 2024-09-10T07:52:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 1 -- 172.21.15.46:0/563346619 wait complete. 2024-09-10T07:52:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.853+0000 7f265361e740 20 client.18339 trim_cache size 0 max 16384 2024-09-10T07:52:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:52:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:52:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:52:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:52:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 -- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26300a2c30 msgr2=0x7f26300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26300a2c30 0x7f26300a50f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2640005cc0 tx=0x7f2640031bb0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 -- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648103d60 msgr2=0x7f2648199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648103d60 0x7f2648199fd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f263c00e9c0 tx=0x7f263c00ee80 comp rx=0 tx=0).stop 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 -- 172.21.15.46:0/877984768 shutdown_connections 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2648105910 0x7f26481a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2648104f50 0x7f264819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f26300a2c30 0x7f26300a50f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 --2- 172.21.15.46:0/877984768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2648103d60 0x7f2648199fd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 -- 172.21.15.46:0/877984768 >> 172.21.15.46:0/877984768 conn(0x7f26480ff4e0 msgr2=0x7f26481081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 -- 172.21.15.46:0/877984768 shutdown_connections 2024-09-10T07:52:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:15.854+0000 7f2650600640 1 -- 172.21.15.46:0/877984768 wait complete. 2024-09-10T07:52:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 -- 172.21.15.46:0/4138433564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc101de0 msgr2=0x7fbfbc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 --2- 172.21.15.46:0/4138433564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc101de0 0x7fbfbc1021c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009900 tx=0x7fbfb002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 -- 172.21.15.46:0/4138433564 shutdown_connections 2024-09-10T07:52:16.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 --2- 172.21.15.46:0/4138433564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfbc106f10 0x7fbfbc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 --2- 172.21.15.46:0/4138433564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfbc102790 0x7fbfbc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 --2- 172.21.15.46:0/4138433564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc101de0 0x7fbfbc1021c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 -- 172.21.15.46:0/4138433564 >> 172.21.15.46:0/4138433564 conn(0x7fbfbc0fd8c0 msgr2=0x7fbfbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:16.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 -- 172.21.15.46:0/4138433564 shutdown_connections 2024-09-10T07:52:16.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.073+0000 7fbfc4800640 1 -- 172.21.15.46:0/4138433564 wait complete. 2024-09-10T07:52:16.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.074+0000 7fbfc4800640 1 Processor -- start 2024-09-10T07:52:16.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.074+0000 7fbfc4800640 1 -- start start 2024-09-10T07:52:16.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc4800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfbc101de0 0x7fbfbc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc4800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc102790 0x7fbfbc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc4800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfbc106f10 0x7fbfbc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc4800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc111fc0 con 0x7fbfbc102790 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc4800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc111e40 con 0x7fbfbc106f10 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc4800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc112140 con 0x7fbfbc101de0 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc102790 0x7fbfbc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc102790 0x7fbfbc19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39730/0 (socket says 172.21.15.46:39730) 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1400640 1 -- 172.21.15.46:0/519999634 learned_addr learned my addr 172.21.15.46:0/519999634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfbc101de0 0x7fbfbc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc2800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfbc106f10 0x7fbfbc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1400640 1 -- 172.21.15.46:0/519999634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfbc101de0 msgr2=0x7fbfbc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1400640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfbc101de0 0x7fbfbc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1400640 1 -- 172.21.15.46:0/519999634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfbc106f10 msgr2=0x7fbfbc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1400640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfbc106f10 0x7fbfbc19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.075+0000 7fbfc1400640 1 -- 172.21.15.46:0/519999634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc19f5f0 con 0x7fbfbc102790 2024-09-10T07:52:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.076+0000 7fbfc1400640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc102790 0x7fbfbc19ab80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00d770 tx=0x7fbfac00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.076+0000 7fbfb6a00640 1 -- 172.21.15.46:0/519999634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac017070 con 0x7fbfbc102790 2024-09-10T07:52:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.076+0000 7fbfb6a00640 1 -- 172.21.15.46:0/519999634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac009820 con 0x7fbfbc102790 2024-09-10T07:52:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.076+0000 7fbfc4800640 1 -- 172.21.15.46:0/519999634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc19f8e0 con 0x7fbfbc102790 2024-09-10T07:52:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.076+0000 7fbfb6a00640 1 -- 172.21.15.46:0/519999634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004cd0 con 0x7fbfbc102790 2024-09-10T07:52:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.076+0000 7fbfc4800640 1 -- 172.21.15.46:0/519999634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc1a71c0 con 0x7fbfbc102790 2024-09-10T07:52:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.079+0000 7fbfb6a00640 1 -- 172.21.15.46:0/519999634 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbfac0099d0 con 0x7fbfbc102790 2024-09-10T07:52:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.079+0000 7fbfc7440740 1 Processor -- start 2024-09-10T07:52:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc7440740 1 -- start start 2024-09-10T07:52:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41de6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c41e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 0x5625c41eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc7440740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5625c4144650 con 0x5625c41dec30 2024-09-10T07:52:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc7440740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625c40ebf10 con 0x5625c41de310 2024-09-10T07:52:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc7440740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5625c4140610 con 0x5625c41e94f0 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c41e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c41e8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39732/0 (socket says 172.21.15.46:39732) 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc1e00640 1 -- 172.21.15.46:0/2427551241 learned_addr learned my addr 172.21.15.46:0/2427551241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc2800640 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41de6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.080+0000 7fbfc1400640 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 0x5625c41eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.081+0000 7fbfc1e00640 1 -- 172.21.15.46:0/2427551241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 msgr2=0x5625c41eb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.081+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 0x5625c41eb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.081+0000 7fbfc1e00640 1 -- 172.21.15.46:0/2427551241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 msgr2=0x5625c41de6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.081+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41de6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.081+0000 7fbfc1e00640 1 -- 172.21.15.46:0/2427551241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c3e53580 con 0x5625c41dec30 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.081+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c41e8fb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009900 tx=0x7fbfb0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713e914a4070db71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.081+0000 7fbfb6a00640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbfa40a2b80 0x7fbfa40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.081+0000 7fbfab000640 1 -- 172.21.15.46:0/2427551241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0035040 con 0x5625c41dec30 2024-09-10T07:52:16.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfb6a00640 1 -- 172.21.15.46:0/519999634 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbfac0cedb0 con 0x7fbfbc102790 2024-09-10T07:52:16.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfab000640 1 -- 172.21.15.46:0/2427551241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb0005520 con 0x5625c41dec30 2024-09-10T07:52:16.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfab000640 1 -- 172.21.15.46:0/2427551241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00037f0 con 0x5625c41dec30 2024-09-10T07:52:16.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc2800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbfa40a2b80 0x7fbfa40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc7440740 1 -- 172.21.15.46:0/2427551241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 msgr2=0x5625c41e8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc7440740 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c41e8fb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009900 tx=0x7fbfb0004040 comp rx=0 tx=0).stop 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc7440740 1 -- 172.21.15.46:0/2427551241 shutdown_connections 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc7440740 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 0x5625c41eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc7440740 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c41e8fb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc7440740 1 --2- 172.21.15.46:0/2427551241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41de6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc7440740 1 -- 172.21.15.46:0/2427551241 >> 172.21.15.46:0/2427551241 conn(0x5625c41da8a0 msgr2=0x5625c41dac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.082+0000 7fbfc2800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbfa40a2b80 0x7fbfa40a5040 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.083+0000 7fbfc7440740 1 -- 172.21.15.46:0/2427551241 shutdown_connections 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.083+0000 7fbfc7440740 1 -- 172.21.15.46:0/2427551241 wait complete. 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.083+0000 7fbfc7440740 1 -- start start 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.083+0000 7fbfc7440740 1 Processor -- start 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41e6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc7440740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5625c41ec110 con 0x5625c41e94f0 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc7440740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625c40ff350 con 0x5625c41de310 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc7440740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5625c41ec290 con 0x5625c41dec30 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc7440740 10 client.0 _ll_register_callbacks cb 0x5625c4122290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41e6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39734/0 (socket says 172.21.15.46:39734) 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc2800640 1 -- 172.21.15.46:0/3341302403 learned_addr learned my addr 172.21.15.46:0/3341302403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc1400640 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc2800640 1 -- 172.21.15.46:0/3341302403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 msgr2=0x5625c41e6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc2800640 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41e6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc2800640 1 -- 172.21.15.46:0/3341302403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 msgr2=0x5625c41e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc2800640 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.084+0000 7fbfc2800640 1 -- 172.21.15.46:0/3341302403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c3f1d960 con 0x5625c41e94f0 2024-09-10T07:52:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.085+0000 7fbfc2800640 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41e6dc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800fa50 tx=0x7fbfb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18f5fd3f17e96ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.085+0000 7fbf9f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.085+0000 7fbf9f400640 1 -- 172.21.15.46:0/3341302403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb802e060 con 0x5625c41e94f0 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.085+0000 7fbf9f400640 1 -- 172.21.15.46:0/3341302403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbfb8018030 con 0x5625c41e94f0 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.085+0000 7fbfc7440740 1 -- 172.21.15.46:0/3341302403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb0005520 con 0x5625c41e94f0 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.085+0000 7fbf9f400640 1 -- 172.21.15.46:0/3341302403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fbfb8035050 con 0x5625c41e94f0 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.085+0000 7fbfc7440740 10 client.18351 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.085+0000 7fbfc7440740 1 -- 172.21.15.46:0/3341302403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5625c41f7d00 con 0x5625c41e94f0 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbf9f400640 1 -- 172.21.15.46:0/3341302403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb0005520 con 0x5625c41e94f0 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbf9f400640 1 -- 172.21.15.46:0/3341302403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7fbfb80297f0 con 0x5625c41e94f0 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbfc7440740 10 client.18351 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbfc7440740 10 client.18351 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbfc7440740 20 client.18351 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5625c41f9260 0x5625c41fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbfc7440740 4 client.18351 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbfc7440740 1 -- 172.21.15.46:0/3341302403 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625c4151bc0 con 0x5625c41f9260 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.086+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5625c41f9260 0x5625c41fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.087+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5625c41f9260 0x5625c41fb6e0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f18bc61e5c8598 server_cookie=f533e249edcf150b in_seq=0 out_seq=0 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.087+0000 7fbf9f400640 10 client.18351 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbf9f400640 1 -- 172.21.15.46:0/3341302403 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625c4151bc0 con 0x5625c41f9260 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbf9f400640 10 client.18351 handle_command_reply: tid=0 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 client.18351 shutdown 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 -- 172.21.15.46:0/3341302403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 msgr2=0x5625c41e6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41e6dc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800fa50 tx=0x7fbfb8004ab0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 -- 172.21.15.46:0/3341302403 shutdown_connections 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5625c41f9260 0x5625c41fb6e0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41e6dc0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41e6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3341302403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 -- 172.21.15.46:0/3341302403 >> 172.21.15.46:0/3341302403 conn(0x5625c41da8a0 msgr2=0x5625c41d9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:16.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbf9f400640 0 client.18351 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:16.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 -- 172.21.15.46:0/3341302403 shutdown_connections 2024-09-10T07:52:16.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 1 -- 172.21.15.46:0/3341302403 wait complete. 2024-09-10T07:52:16.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.088+0000 7fbfc7440740 20 client.18351 trim_cache size 0 max 16384 2024-09-10T07:52:16.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.106+0000 7fbfc7440740 1 Processor -- start 2024-09-10T07:52:16.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.106+0000 7fbfc7440740 1 -- start start 2024-09-10T07:52:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.106+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c4200ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 0x5625c4201530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc7440740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1b0da0 con 0x5625c41dec30 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc7440740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1b2a60 con 0x5625c41de310 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc7440740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1b1470 con 0x5625c41e94f0 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c4200ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c4200ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39750/0 (socket says 172.21.15.46:39750) 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1400640 1 -- 172.21.15.46:0/3752582657 learned_addr learned my addr 172.21.15.46:0/3752582657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc2800640 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 0x5625c4201530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1400640 1 -- 172.21.15.46:0/3752582657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 msgr2=0x5625c4201530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1400640 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 0x5625c4201530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1400640 1 -- 172.21.15.46:0/3752582657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 msgr2=0x5625c41e0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1400640 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1400640 1 -- 172.21.15.46:0/3752582657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c41da6d0 con 0x5625c41dec30 2024-09-10T07:52:16.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.107+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.108+0000 7fbfc1400640 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c4200ff0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbfac097690 tx=0x7fbfac0d27f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f54319823ddb56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.108+0000 7fbfaa600640 1 -- 172.21.15.46:0/3752582657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac020040 con 0x5625c41dec30 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.108+0000 7fbfaa600640 1 -- 172.21.15.46:0/3752582657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac09b930 con 0x5625c41dec30 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.108+0000 7fbfaa600640 1 -- 172.21.15.46:0/3752582657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0978a0 con 0x5625c41dec30 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 -- 172.21.15.46:0/3752582657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 msgr2=0x5625c4200ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c4200ff0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbfac097690 tx=0x7fbfac0d27f0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 -- 172.21.15.46:0/3752582657 shutdown_connections 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41e94f0 0x5625c4201530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41dec30 0x5625c4200ff0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3752582657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41e0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 -- 172.21.15.46:0/3752582657 >> 172.21.15.46:0/3752582657 conn(0x5625c41da8a0 msgr2=0x5625c41e4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 -- 172.21.15.46:0/3752582657 shutdown_connections 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 -- 172.21.15.46:0/3752582657 wait complete. 2024-09-10T07:52:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.109+0000 7fbfc7440740 1 -- start start 2024-09-10T07:52:16.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc7440740 1 Processor -- start 2024-09-10T07:52:16.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc7440740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41edf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc7440740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1b22d0 con 0x5625c41e94f0 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc7440740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1b1c00 con 0x5625c41de310 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc7440740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1aed80 con 0x5625c41dec30 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc7440740 10 client.0 _ll_register_callbacks cb 0x7fbfb8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41edf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39760/0 (socket says 172.21.15.46:39760) 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc2800640 1 -- 172.21.15.46:0/3352514641 learned_addr learned my addr 172.21.15.46:0/3352514641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.110+0000 7fbfc1400640 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc1e00640 1 -- 172.21.15.46:0/3352514641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 msgr2=0x5625c41eda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41eda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc1e00640 1 -- 172.21.15.46:0/3352514641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 msgr2=0x5625c41edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc1e00640 1 -- 172.21.15.46:0/3352514641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c4263260 con 0x5625c41de310 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc2800640 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc1400640 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc1e00640 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41ed500 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0031550 tx=0x7fbfb002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c2334f4a65f275 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbf9f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbf9f400640 1 -- 172.21.15.46:0/3352514641 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0035070 con 0x5625c41de310 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.111+0000 7fbfc7440740 1 -- 172.21.15.46:0/3352514641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac09b930 con 0x5625c41de310 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbf9f400640 1 -- 172.21.15.46:0/3352514641 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbfb00048c0 con 0x5625c41de310 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbf9f400640 1 -- 172.21.15.46:0/3352514641 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fbfb00315b0 con 0x5625c41de310 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbfc7440740 10 client.29291 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbfc7440740 1 -- 172.21.15.46:0/3352514641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5625c4264600 con 0x5625c41de310 2024-09-10T07:52:16.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbf9f400640 1 -- 172.21.15.46:0/3352514641 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac09b930 con 0x5625c41de310 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbf9f400640 1 -- 172.21.15.46:0/3352514641 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7fbfb0005130 con 0x5625c41de310 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbfc7440740 10 client.29291 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbfc7440740 10 client.29291 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbfc7440740 20 client.29291 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5625c4264a00 0x5625c4266df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbfc7440740 4 client.29291 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "67ec92b3", "release": true, "await": true}] 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.112+0000 7fbfc7440740 1 -- 172.21.15.46:0/3352514641 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "67ec92b3", "release": true, "await": true}) -- 0x5625c414c0d0 con 0x5625c4264a00 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.113+0000 7fbfc1400640 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5625c4264a00 0x5625c4266df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.113+0000 7fbfc1400640 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5625c4264a00 0x5625c4266df0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4cef81f6d08720 server_cookie=8523846729074ee2 in_seq=0 out_seq=0 2024-09-10T07:52:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.113+0000 7fbf9f400640 10 client.29291 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.143+0000 7fbf9f400640 1 -- 172.21.15.46:0/3352514641 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5625c414c0d0 con 0x5625c4264a00 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.143+0000 7fbf9f400640 10 client.29291 handle_command_reply: tid=0 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.143+0000 7fbfc7440740 1 client.29291 shutdown 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 -- 172.21.15.46:0/3352514641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 msgr2=0x5625c41ed500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41ed500 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0031550 tx=0x7fbfb002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 -- 172.21.15.46:0/3352514641 shutdown_connections 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x5625c4264a00 0x5625c4266df0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625c41e94f0 0x5625c41edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625c41dec30 0x5625c41eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 --2- 172.21.15.46:0/3352514641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625c41de310 0x5625c41ed500 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 -- 172.21.15.46:0/3352514641 >> 172.21.15.46:0/3352514641 conn(0x5625c41da8a0 msgr2=0x5625c41e6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbf9f400640 0 client.29291 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 60, 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 56, 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "67ec92b3": { 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 56, 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:52:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 139.8, 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 -- 172.21.15.46:0/3352514641 shutdown_connections 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 1 -- 172.21.15.46:0/3352514641 wait complete. 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.144+0000 7fbfc7440740 20 client.29291 trim_cache size 0 max 16384 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 -- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbfa40a2b80 msgr2=0x7fbfa40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbfa40a2b80 0x7fbfa40a5040 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb8009500 comp rx=0 tx=0).stop 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 -- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc102790 msgr2=0x7fbfbc19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc102790 0x7fbfbc19ab80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00d770 tx=0x7fbfac00db70 comp rx=0 tx=0).stop 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 -- 172.21.15.46:0/519999634 shutdown_connections 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfbc106f10 0x7fbfbc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbfa40a2b80 0x7fbfa40a5040 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfbc102790 0x7fbfbc19ab80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 --2- 172.21.15.46:0/519999634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfbc101de0 0x7fbfbc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.145+0000 7fbfc4800640 1 -- 172.21.15.46:0/519999634 >> 172.21.15.46:0/519999634 conn(0x7fbfbc0fd8c0 msgr2=0x7fbfbc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.147+0000 7fbfc4800640 1 -- 172.21.15.46:0/519999634 shutdown_connections 2024-09-10T07:52:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:16.147+0000 7fbfc4800640 1 -- 172.21.15.46:0/519999634 wait complete. 2024-09-10T07:52:16.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:16 smithi181 ceph-mon[35397]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 811 KiB/s wr, 163 op/s 2024-09-10T07:52:16.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:16 smithi063 ceph-mon[35020]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 811 KiB/s wr, 163 op/s 2024-09-10T07:52:16.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:16 smithi046 ceph-mon[30705]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 811 KiB/s wr, 163 op/s 2024-09-10T07:52:17.080 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '67ec92b3', total seconds elapsed: 25.2 2024-09-10T07:52:17.080 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-09-10T07:52:17.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:18.315 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:18 smithi046 ceph-mon[30705]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 367 KiB/s wr, 148 op/s 2024-09-10T07:52:18.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:18 smithi181 ceph-mon[35397]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 367 KiB/s wr, 148 op/s 2024-09-10T07:52:18.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:18 smithi063 ceph-mon[35020]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 367 KiB/s wr, 148 op/s 2024-09-10T07:52:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 -- 172.21.15.46:0/4212670335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4102d20 msgr2=0x7fd7c4103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/4212670335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4102d20 0x7fd7c4103100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009900 tx=0x7fd7b402a490 comp rx=0 tx=0).stop 2024-09-10T07:52:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 -- 172.21.15.46:0/4212670335 shutdown_connections 2024-09-10T07:52:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/4212670335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c410c150 0x7fd7c410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/4212670335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7c4103640 0x7fd7c410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/4212670335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4102d20 0x7fd7c4103100 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 -- 172.21.15.46:0/4212670335 >> 172.21.15.46:0/4212670335 conn(0x7fd7c40fc780 msgr2=0x7fd7c40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 -- 172.21.15.46:0/4212670335 shutdown_connections 2024-09-10T07:52:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.831+0000 7fd7c8c00640 1 -- 172.21.15.46:0/4212670335 wait complete. 2024-09-10T07:52:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.837+0000 7fd7c8c00640 1 Processor -- start 2024-09-10T07:52:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.837+0000 7fd7c8c00640 1 -- start start 2024-09-10T07:52:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.838+0000 7fd7c8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7c4102d20 0x7fd7c40739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.838+0000 7fd7c8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c4103640 0x7fd7c406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.838+0000 7fd7c8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c410c150 0x7fd7c406cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.838+0000 7fd7c8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4111fe0 con 0x7fd7c4103640 2024-09-10T07:52:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.838+0000 7fd7c8c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4111e60 con 0x7fd7c4102d20 2024-09-10T07:52:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.838+0000 7fd7c8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4112160 con 0x7fd7c410c150 2024-09-10T07:52:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.839+0000 7fd7c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7c4102d20 0x7fd7c40739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.839+0000 7fd7c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7c4102d20 0x7fd7c40739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59914/0 (socket says 172.21.15.46:59914) 2024-09-10T07:52:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.839+0000 7fd7c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c4103640 0x7fd7c406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.839+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c410c150 0x7fd7c406cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.839+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c410c150 0x7fd7c406cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43958/0 (socket says 172.21.15.46:43958) 2024-09-10T07:52:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.839+0000 7fd7c2800640 1 -- 172.21.15.46:0/1625027231 learned_addr learned my addr 172.21.15.46:0/1625027231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.840+0000 7fd7c2800640 1 -- 172.21.15.46:0/1625027231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7c4102d20 msgr2=0x7fd7c40739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.840+0000 7fd7c2800640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7c4102d20 0x7fd7c40739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.840+0000 7fd7c2800640 1 -- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c4103640 msgr2=0x7fd7c406ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.840+0000 7fd7c2800640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c4103640 0x7fd7c406ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.840+0000 7fd7c2800640 1 -- 172.21.15.46:0/1625027231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c4074250 con 0x7fd7c410c150 2024-09-10T07:52:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.840+0000 7fd7c1e00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7c4102d20 0x7fd7c40739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.841+0000 7fd7c2800640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c410c150 0x7fd7c406cf80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00b970 tx=0x7fd7bc00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.841+0000 7fd7baa00640 1 -- 172.21.15.46:0/1625027231 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc019070 con 0x7fd7c410c150 2024-09-10T07:52:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.842+0000 7fd7baa00640 1 -- 172.21.15.46:0/1625027231 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7bc00cb40 con 0x7fd7c410c150 2024-09-10T07:52:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.842+0000 7fd7baa00640 1 -- 172.21.15.46:0/1625027231 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc007650 con 0x7fd7c410c150 2024-09-10T07:52:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.843+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c406d5e0 con 0x7fd7c410c150 2024-09-10T07:52:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.843+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c40716b0 con 0x7fd7c410c150 2024-09-10T07:52:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.847+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c406b070 con 0x7fd7c410c150 2024-09-10T07:52:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.855+0000 7fd7baa00640 1 -- 172.21.15.46:0/1625027231 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd7bc01c070 con 0x7fd7c410c150 2024-09-10T07:52:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.863+0000 7fd7baa00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd7a80a2c40 0x7fd7a80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.864+0000 7fd7baa00640 1 -- 172.21.15.46:0/1625027231 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fd7bc0ce9f0 con 0x7fd7c410c150 2024-09-10T07:52:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.865+0000 7fd7c1e00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd7a80a2c40 0x7fd7a80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.866+0000 7fd7baa00640 1 -- 172.21.15.46:0/1625027231 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7bc09b5d0 con 0x7fd7c410c150 2024-09-10T07:52:18.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:18.867+0000 7fd7c1e00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd7a80a2c40 0x7fd7a80a5100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40096c0 tx=0x7fd7b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.281+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7c4071910 con 0x7fd7c410c150 2024-09-10T07:52:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.282+0000 7fd7baa00640 1 -- 172.21.15.46:0/1625027231 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11986 (secure 0 0 0) 0x7fd7bc09aed0 con 0x7fd7c410c150 2024-09-10T07:52:19.284 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:19.285 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":60,"btime":"2024-09-10T07:51:54:063936+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:51:53.066973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:52:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 60 2024-09-10T07:52:19.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.287+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd7a80a2c40 msgr2=0x7fd7a80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.287+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd7a80a2c40 0x7fd7a80a5100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40096c0 tx=0x7fd7b4000bc0 comp rx=0 tx=0).stop 2024-09-10T07:52:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.287+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c410c150 msgr2=0x7fd7c406cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.287+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c410c150 0x7fd7c406cf80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00b970 tx=0x7fd7bc00be30 comp rx=0 tx=0).stop 2024-09-10T07:52:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.288+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 shutdown_connections 2024-09-10T07:52:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.288+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c410c150 0x7fd7c406cf80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.288+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd7a80a2c40 0x7fd7a80a5100 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.288+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c4103640 0x7fd7c406ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.288+0000 7fd7c8c00640 1 --2- 172.21.15.46:0/1625027231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7c4102d20 0x7fd7c40739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.288+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 >> 172.21.15.46:0/1625027231 conn(0x7fd7c40fc780 msgr2=0x7fd7c40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.290+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 shutdown_connections 2024-09-10T07:52:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:19.290+0000 7fd7c8c00640 1 -- 172.21.15.46:0/1625027231 wait complete. 2024-09-10T07:52:20.598 INFO:teuthology.orchestra.run.smithi046.stderr:progress: 900.0 s, 386.0 tps, lat 939.172 ms stddev 3066.700 2024-09-10T07:52:20.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:20 smithi181 ceph-mon[35397]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 181 KiB/s wr, 122 op/s 2024-09-10T07:52:20.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:20 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1625027231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:20.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:20 smithi063 ceph-mon[35020]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 181 KiB/s wr, 122 op/s 2024-09-10T07:52:20.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:20 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1625027231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:20.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:20 smithi046 ceph-mon[30705]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 181 KiB/s wr, 122 op/s 2024-09-10T07:52:20.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:20 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1625027231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:20.875 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-09-10T07:52:20.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-09-10T07:52:21.134 INFO:teuthology.orchestra.run.smithi046.stdout:transaction type: 2024-09-10T07:52:21.134 INFO:teuthology.orchestra.run.smithi046.stdout:scaling factor: 32 2024-09-10T07:52:21.134 INFO:teuthology.orchestra.run.smithi046.stdout:query mode: simple 2024-09-10T07:52:21.134 INFO:teuthology.orchestra.run.smithi046.stdout:number of clients: 100 2024-09-10T07:52:21.134 INFO:teuthology.orchestra.run.smithi046.stdout:number of threads: 4 2024-09-10T07:52:21.134 INFO:teuthology.orchestra.run.smithi046.stdout:duration: 900 s 2024-09-10T07:52:21.134 INFO:teuthology.orchestra.run.smithi046.stdout:number of transactions actually processed: 587713 2024-09-10T07:52:21.135 INFO:teuthology.orchestra.run.smithi046.stdout:latency average = 153.125 ms 2024-09-10T07:52:21.135 INFO:teuthology.orchestra.run.smithi046.stdout:latency stddev = 928.959 ms 2024-09-10T07:52:21.135 INFO:teuthology.orchestra.run.smithi046.stdout:tps = 652.626226 (including connections establishing) 2024-09-10T07:52:21.135 INFO:teuthology.orchestra.run.smithi046.stdout:tps = 652.632536 (excluding connections establishing) 2024-09-10T07:52:21.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-10T07:52:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:22.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:22 smithi046 ceph-mon[30705]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T07:52:22.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.544+0000 7f30eca00640 1 -- 172.21.15.46:0/3140657667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e81068d0 msgr2=0x7f30e8106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:22.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.544+0000 7f30eca00640 1 --2- 172.21.15.46:0/3140657667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e81068d0 0x7f30e8106cb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009930 tx=0x7f30d802a460 comp rx=0 tx=0).stop 2024-09-10T07:52:22.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.546+0000 7f30eca00640 1 -- 172.21.15.46:0/3140657667 shutdown_connections 2024-09-10T07:52:22.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.546+0000 7f30eca00640 1 --2- 172.21.15.46:0/3140657667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30e8101270 0x7f30e810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.546+0000 7f30eca00640 1 --2- 172.21.15.46:0/3140657667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e81008d0 0x7f30e8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.546+0000 7f30eca00640 1 --2- 172.21.15.46:0/3140657667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e81068d0 0x7f30e8106cb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.546+0000 7f30eca00640 1 -- 172.21.15.46:0/3140657667 >> 172.21.15.46:0/3140657667 conn(0x7f30e80fc610 msgr2=0x7f30e80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.548+0000 7f30eca00640 1 -- 172.21.15.46:0/3140657667 shutdown_connections 2024-09-10T07:52:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.548+0000 7f30eca00640 1 -- 172.21.15.46:0/3140657667 wait complete. 2024-09-10T07:52:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.549+0000 7f30eca00640 1 Processor -- start 2024-09-10T07:52:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.550+0000 7f30eca00640 1 -- start start 2024-09-10T07:52:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.552+0000 7f30eca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30e81008d0 0x7f30e819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.552+0000 7f30eca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e8101270 0x7f30e8196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.552+0000 7f30e6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30e81008d0 0x7f30e819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.552+0000 7f30e6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30e81008d0 0x7f30e819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59924/0 (socket says 172.21.15.46:59924) 2024-09-10T07:52:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.553+0000 7f30e6800640 1 -- 172.21.15.46:0/3912249539 learned_addr learned my addr 172.21.15.46:0/3912249539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.553+0000 7f30eca00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e81068d0 0x7f30e8196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.553+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30e8110d40 con 0x7f30e8101270 2024-09-10T07:52:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.553+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f30e8110bc0 con 0x7f30e81008d0 2024-09-10T07:52:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.553+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f30e8110ec0 con 0x7f30e81068d0 2024-09-10T07:52:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.554+0000 7f30dfe00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e8101270 0x7f30e8196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.554+0000 7f30e7200640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e81068d0 0x7f30e8196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.556+0000 7f30e6800640 1 -- 172.21.15.46:0/3912249539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e81068d0 msgr2=0x7f30e8196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:22.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.557+0000 7f30e6800640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e81068d0 0x7f30e8196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.557+0000 7f30e6800640 1 -- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e8101270 msgr2=0x7f30e8196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:22.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.557+0000 7f30e6800640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e8101270 0x7f30e8196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.559+0000 7f30e6800640 1 -- 172.21.15.46:0/3912249539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e8196c90 con 0x7f30e81008d0 2024-09-10T07:52:22.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.559+0000 7f30dfe00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e8101270 0x7f30e8196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:22.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.560+0000 7f30e6800640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30e81008d0 0x7f30e819dfb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f30d802a980 tx=0x7f30d8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:22.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.560+0000 7f30df400640 1 -- 172.21.15.46:0/3912249539 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d803f070 con 0x7f30e81008d0 2024-09-10T07:52:22.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.561+0000 7f30df400640 1 -- 172.21.15.46:0/3912249539 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d8031c20 con 0x7f30e81008d0 2024-09-10T07:52:22.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.561+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e8196f20 con 0x7f30e81008d0 2024-09-10T07:52:22.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.561+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e81b3c20 con 0x7f30e81008d0 2024-09-10T07:52:22.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.562+0000 7f30df400640 1 -- 172.21.15.46:0/3912249539 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8035030 con 0x7f30e81008d0 2024-09-10T07:52:22.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.570+0000 7f30df400640 1 -- 172.21.15.46:0/3912249539 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f30d8042070 con 0x7f30e81008d0 2024-09-10T07:52:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.571+0000 7f30f0075740 1 Processor -- start 2024-09-10T07:52:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.571+0000 7f30f0075740 1 -- start start 2024-09-10T07:52:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.572+0000 7f30f0075740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad34aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.573+0000 7f30e7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad34aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.574+0000 7f30e7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad34aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39822/0 (socket says 172.21.15.46:39822) 2024-09-10T07:52:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.574+0000 7f30f0075740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad34fe0 0x55b95ad3f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.574+0000 7f30f0075740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad3f8a0 0x55b95ad41c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.575+0000 7f30e7200640 1 -- 172.21.15.46:0/3623888762 learned_addr learned my addr 172.21.15.46:0/3623888762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.575+0000 7f30f0075740 1 -- 172.21.15.46:0/3623888762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b95ac99260 con 0x55b95ad346c0 2024-09-10T07:52:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.575+0000 7f30f0075740 1 -- 172.21.15.46:0/3623888762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b95ac40500 con 0x55b95ad3f8a0 2024-09-10T07:52:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.575+0000 7f30f0075740 1 -- 172.21.15.46:0/3623888762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b95ac95220 con 0x55b95ad34fe0 2024-09-10T07:52:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.576+0000 7f30e6800640 1 --2- 172.21.15.46:0/3623888762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad34fe0 0x55b95ad3f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.576+0000 7f30e7200640 1 -- 172.21.15.46:0/3623888762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad34fe0 msgr2=0x55b95ad3f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.576+0000 7f30e7200640 1 --2- 172.21.15.46:0/3623888762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad34fe0 0x55b95ad3f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.576+0000 7f30e7200640 1 -- 172.21.15.46:0/3623888762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad3f8a0 msgr2=0x55b95ad41c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.576+0000 7f30e7200640 1 --2- 172.21.15.46:0/3623888762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad3f8a0 0x55b95ad41c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.576+0000 7f30e7200640 1 -- 172.21.15.46:0/3623888762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b95aa0aba0 con 0x55b95ad346c0 2024-09-10T07:52:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.577+0000 7f30df400640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30c80a2cd0 0x7f30c80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.577+0000 7f30e6800640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30c80a2cd0 0x7f30c80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.578+0000 7f30e7200640 1 --2- 172.21.15.46:0/3623888762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad34aa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f30e000a430 tx=0x7f30e0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930d14fab386a24a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.578+0000 7f30df400640 1 -- 172.21.15.46:0/3912249539 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f30d8031dd0 con 0x7f30e81008d0 2024-09-10T07:52:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.578+0000 7f30d7400640 1 -- 172.21.15.46:0/3623888762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0013790 con 0x55b95ad346c0 2024-09-10T07:52:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.578+0000 7f30d7400640 1 -- 172.21.15.46:0/3623888762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e0019070 con 0x55b95ad346c0 2024-09-10T07:52:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.578+0000 7f30d7400640 1 -- 172.21.15.46:0/3623888762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0014de0 con 0x55b95ad346c0 2024-09-10T07:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.580+0000 7f30e6800640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30c80a2cd0 0x7f30c80a5190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f30d803d040 tx=0x7f30d80318a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.580+0000 7f30f0075740 1 -- 172.21.15.46:0/3623888762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 msgr2=0x55b95ad34aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.580+0000 7f30f0075740 1 --2- 172.21.15.46:0/3623888762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad34aa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f30e000a430 tx=0x7f30e0009650 comp rx=0 tx=0).stop 2024-09-10T07:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.581+0000 7f30f0075740 1 -- 172.21.15.46:0/3623888762 shutdown_connections 2024-09-10T07:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.581+0000 7f30f0075740 1 --2- 172.21.15.46:0/3623888762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad3f8a0 0x55b95ad41c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.581+0000 7f30f0075740 1 --2- 172.21.15.46:0/3623888762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad34fe0 0x55b95ad3f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.581+0000 7f30f0075740 1 --2- 172.21.15.46:0/3623888762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad34aa0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.581+0000 7f30f0075740 1 -- 172.21.15.46:0/3623888762 >> 172.21.15.46:0/3623888762 conn(0x55b95ad30c50 msgr2=0x55b95ad31030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.581+0000 7f30f0075740 1 -- 172.21.15.46:0/3623888762 shutdown_connections 2024-09-10T07:52:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.581+0000 7f30f0075740 1 -- 172.21.15.46:0/3623888762 wait complete. 2024-09-10T07:52:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.582+0000 7f30f0075740 1 -- start start 2024-09-10T07:52:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.582+0000 7f30f0075740 1 Processor -- start 2024-09-10T07:52:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.583+0000 7f30f0075740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad3c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.583+0000 7f30f0075740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 0x55b95ad3cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.583+0000 7f30f0075740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad3d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.583+0000 7f30f0075740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b95ad42470 con 0x55b95ad34fe0 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.583+0000 7f30f0075740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b95ac53bd0 con 0x55b95ad346c0 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.583+0000 7f30f0075740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b95ad425f0 con 0x55b95ad3f8a0 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.583+0000 7f30f0075740 10 client.0 _ll_register_callbacks cb 0x55b95a9a80a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad3c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad3c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59942/0 (socket says 172.21.15.46:59942) 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 -- 172.21.15.46:0/688768511 learned_addr learned my addr 172.21.15.46:0/688768511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad3d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 -- 172.21.15.46:0/688768511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 msgr2=0x55b95ad3d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad3d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 -- 172.21.15.46:0/688768511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 msgr2=0x55b95ad3cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 0x55b95ad3cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.584+0000 7f30e7200640 1 -- 172.21.15.46:0/688768511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b95ad4b940 con 0x55b95ad346c0 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.585+0000 7f30e7200640 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad3c730 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f30e0004a80 tx=0x7f30e0014940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88454bb2d9c6906b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.585+0000 7f30cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.585+0000 7f30cfe00640 1 -- 172.21.15.46:0/688768511 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0024070 con 0x55b95ad346c0 2024-09-10T07:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.585+0000 7f30cfe00640 1 -- 172.21.15.46:0/688768511 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f30e0013a60 con 0x55b95ad346c0 2024-09-10T07:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.586+0000 7f30f0075740 1 -- 172.21.15.46:0/688768511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e0019070 con 0x55b95ad346c0 2024-09-10T07:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.586+0000 7f30cfe00640 1 -- 172.21.15.46:0/688768511 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f30e0013c60 con 0x55b95ad346c0 2024-09-10T07:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.586+0000 7f30f0075740 10 client.29297 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.586+0000 7f30f0075740 1 -- 172.21.15.46:0/688768511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55b95ad4e250 con 0x55b95ad346c0 2024-09-10T07:52:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.587+0000 7f30cfe00640 1 -- 172.21.15.46:0/688768511 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e001d900 con 0x55b95ad346c0 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.587+0000 7f30cfe00640 1 -- 172.21.15.46:0/688768511 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f30e0020520 con 0x55b95ad346c0 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.587+0000 7f30f0075740 10 client.29297 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.587+0000 7f30f0075740 10 client.29297 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.587+0000 7f30f0075740 20 client.29297 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.587+0000 7f30f0075740 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b95ad4f710 0x55b95ad51b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.587+0000 7f30f0075740 4 client.29297 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.587+0000 7f30f0075740 1 -- 172.21.15.46:0/688768511 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f30e0019070 con 0x55b95ad4f710 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.589+0000 7f30dfe00640 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b95ad4f710 0x55b95ad51b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.590+0000 7f30dfe00640 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b95ad4f710 0x55b95ad51b00 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c8c06ca8828d90 server_cookie=74cdfc24ab93501b in_seq=0 out_seq=0 2024-09-10T07:52:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.590+0000 7f30cfe00640 10 client.29297 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:22.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.592+0000 7f30cfe00640 1 -- 172.21.15.46:0/688768511 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f30e0019070 con 0x55b95ad4f710 2024-09-10T07:52:22.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.593+0000 7f30cfe00640 10 client.29297 handle_command_reply: tid=0 2024-09-10T07:52:22.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.593+0000 7f30f0075740 1 client.29297 shutdown 2024-09-10T07:52:22.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.594+0000 7f30f0075740 1 -- 172.21.15.46:0/688768511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 msgr2=0x55b95ad3c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:22.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.594+0000 7f30f0075740 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad3c730 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f30e0004a80 tx=0x7f30e0014940 comp rx=0 tx=0).stop 2024-09-10T07:52:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 1 -- 172.21.15.46:0/688768511 shutdown_connections 2024-09-10T07:52:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b95ad4f710 0x55b95ad51b00 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad3d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 0x55b95ad3cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 1 --2- 172.21.15.46:0/688768511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad3c730 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 1 -- 172.21.15.46:0/688768511 >> 172.21.15.46:0/688768511 conn(0x55b95ad30c50 msgr2=0x55b95ad2fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30cfe00640 0 client.29297 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:22.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 1 -- 172.21.15.46:0/688768511 shutdown_connections 2024-09-10T07:52:22.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 1 -- 172.21.15.46:0/688768511 wait complete. 2024-09-10T07:52:22.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.597+0000 7f30f0075740 20 client.29297 trim_cache size 0 max 16384 2024-09-10T07:52:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.635+0000 7f30f0075740 1 Processor -- start 2024-09-10T07:52:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.635+0000 7f30f0075740 1 -- start start 2024-09-10T07:52:22.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30f0075740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad37120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30f0075740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 0x55b95ad3d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30f0075740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad3e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30f0075740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30c400fb20 con 0x55b95ad34fe0 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30f0075740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f30c40117e0 con 0x55b95ad346c0 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30f0075740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f30c40101f0 con 0x55b95ad3f8a0 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30e7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad37120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30e7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad37120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:59948/0 (socket says 172.21.15.46:59948) 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30e7200640 1 -- 172.21.15.46:0/2444515767 learned_addr learned my addr 172.21.15.46:0/2444515767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.636+0000 7f30e7200640 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad3e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.638+0000 7f30dfe00640 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 0x55b95ad3d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.638+0000 7f30e7200640 1 -- 172.21.15.46:0/2444515767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 msgr2=0x55b95ad3e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.638+0000 7f30e7200640 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad3e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.638+0000 7f30e7200640 1 -- 172.21.15.46:0/2444515767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 msgr2=0x55b95ad3d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.639+0000 7f30e7200640 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 0x55b95ad3d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.639+0000 7f30e7200640 1 -- 172.21.15.46:0/2444515767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b95ad38330 con 0x55b95ad346c0 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.639+0000 7f30dfe00640 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 0x55b95ad3d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.639+0000 7f30e7200640 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad37120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f30e00144a0 tx=0x7f30e0009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ff99624f632e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.640+0000 7f30d6a00640 1 -- 172.21.15.46:0/2444515767 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0009b80 con 0x55b95ad346c0 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.640+0000 7f30d6a00640 1 -- 172.21.15.46:0/2444515767 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e0019040 con 0x55b95ad346c0 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.640+0000 7f30d6a00640 1 -- 172.21.15.46:0/2444515767 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e001fa70 con 0x55b95ad346c0 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 -- 172.21.15.46:0/2444515767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 msgr2=0x55b95ad37120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad37120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f30e00144a0 tx=0x7f30e0009320 comp rx=0 tx=0).stop 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 -- 172.21.15.46:0/2444515767 shutdown_connections 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad3e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad34fe0 0x55b95ad3d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 --2- 172.21.15.46:0/2444515767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad346c0 0x55b95ad37120 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 -- 172.21.15.46:0/2444515767 >> 172.21.15.46:0/2444515767 conn(0x55b95ad30c50 msgr2=0x55b95ad3de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 -- 172.21.15.46:0/2444515767 shutdown_connections 2024-09-10T07:52:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.641+0000 7f30f0075740 1 -- 172.21.15.46:0/2444515767 wait complete. 2024-09-10T07:52:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.642+0000 7f30f0075740 1 -- start start 2024-09-10T07:52:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.642+0000 7f30f0075740 1 Processor -- start 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.645+0000 7f30f0075740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad47c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.645+0000 7f30f0075740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad34fe0 0x55b95ad48170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.645+0000 7f30f0075740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad39330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.645+0000 7f30f0075740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4011050 con 0x55b95ad346c0 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.645+0000 7f30f0075740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4010980 con 0x55b95ad34fe0 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.645+0000 7f30f0075740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f30c400db00 con 0x55b95ad3f8a0 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.645+0000 7f30f0075740 10 client.0 _ll_register_callbacks cb 0x7f30e0009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.646+0000 7f30e7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad47c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.646+0000 7f30e7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad47c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39850/0 (socket says 172.21.15.46:39850) 2024-09-10T07:52:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.646+0000 7f30e7200640 1 -- 172.21.15.46:0/908847793 learned_addr learned my addr 172.21.15.46:0/908847793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.646+0000 7f30dfe00640 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad34fe0 0x55b95ad48170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.646+0000 7f30e7200640 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad39330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.647+0000 7f30dfe00640 1 -- 172.21.15.46:0/908847793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 msgr2=0x55b95ad39330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.647+0000 7f30dfe00640 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad39330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.647+0000 7f30dfe00640 1 -- 172.21.15.46:0/908847793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 msgr2=0x55b95ad47c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.647+0000 7f30dfe00640 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad47c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.647+0000 7f30dfe00640 1 -- 172.21.15.46:0/908847793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b95ad39b60 con 0x55b95ad34fe0 2024-09-10T07:52:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.647+0000 7f30e7200640 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad39330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.647+0000 7f30dfe00640 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad34fe0 0x55b95ad48170 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55b95ad3bfc0 tx=0x7f30d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad8cb6d4ac3e5622 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.647+0000 7f30cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:22.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:22 smithi181 ceph-mon[35397]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.648+0000 7f30cfe00640 1 -- 172.21.15.46:0/908847793 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0021680 con 0x55b95ad34fe0 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.648+0000 7f30cfe00640 1 -- 172.21.15.46:0/908847793 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f30d0021820 con 0x55b95ad34fe0 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.648+0000 7f30f0075740 1 -- 172.21.15.46:0/908847793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e0019040 con 0x55b95ad34fe0 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.648+0000 7f30cfe00640 1 -- 172.21.15.46:0/908847793 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f30d002a050 con 0x55b95ad34fe0 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.648+0000 7f30f0075740 10 client.29303 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.648+0000 7f30f0075740 1 -- 172.21.15.46:0/908847793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55b95adb9f50 con 0x55b95ad34fe0 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.649+0000 7f30cfe00640 1 -- 172.21.15.46:0/908847793 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e0019040 con 0x55b95ad34fe0 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.649+0000 7f30cfe00640 1 -- 172.21.15.46:0/908847793 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f30d0010ba0 con 0x55b95ad34fe0 2024-09-10T07:52:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.649+0000 7f30f0075740 10 client.29303 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.649+0000 7f30f0075740 10 client.29303 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.649+0000 7f30f0075740 20 client.29303 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.649+0000 7f30f0075740 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b95adba7b0 0x55b95adbcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.649+0000 7f30f0075740 4 client.29303 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-09-10T07:52:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.649+0000 7f30f0075740 1 -- 172.21.15.46:0/908847793 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x55b95aca6990 con 0x55b95adba7b0 2024-09-10T07:52:22.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:22 smithi063 ceph-mon[35020]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T07:52:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.650+0000 7f30e7200640 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b95adba7b0 0x55b95adbcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.652+0000 7f30e7200640 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b95adba7b0 0x55b95adbcbd0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3968d8eebf51c046 server_cookie=e0280a5f48e55486 in_seq=0 out_seq=0 2024-09-10T07:52:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:22.652+0000 7f30cfe00640 10 client.29303 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:24 smithi181 ceph-mon[35397]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-09-10T07:52:24.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:24 smithi063 ceph-mon[35020]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-09-10T07:52:24.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:24 smithi046 ceph-mon[30705]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-09-10T07:52:25.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:25 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:52:25.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:25 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:52:25.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:25 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:52:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.826+0000 7f30cfe00640 1 -- 172.21.15.46:0/908847793 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b95aca6990 con 0x55b95adba7b0 2024-09-10T07:52:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.826+0000 7f30cfe00640 10 client.29303 handle_command_reply: tid=0 2024-09-10T07:52:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.826+0000 7f30f0075740 1 client.29303 shutdown 2024-09-10T07:52:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30f0075740 1 -- 172.21.15.46:0/908847793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad34fe0 msgr2=0x55b95ad48170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30f0075740 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad34fe0 0x55b95ad48170 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55b95ad3bfc0 tx=0x7f30d0011810 comp rx=0 tx=0).stop 2024-09-10T07:52:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30f0075740 1 -- 172.21.15.46:0/908847793 shutdown_connections 2024-09-10T07:52:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30f0075740 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55b95adba7b0 0x55b95adbcbd0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30f0075740 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b95ad3f8a0 0x55b95ad39330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30f0075740 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b95ad34fe0 0x55b95ad48170 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30f0075740 1 --2- 172.21.15.46:0/908847793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95ad346c0 0x55b95ad47c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30f0075740 1 -- 172.21.15.46:0/908847793 >> 172.21.15.46:0/908847793 conn(0x55b95ad30c50 msgr2=0x55b95ad2fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.827+0000 7f30cfe00640 0 client.29303 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.836+0000 7f30f0075740 1 -- 172.21.15.46:0/908847793 shutdown_connections 2024-09-10T07:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.837+0000 7f30f0075740 1 -- 172.21.15.46:0/908847793 wait complete. 2024-09-10T07:52:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.837+0000 7f30f0075740 20 client.29303 trim_cache size 0 max 16384 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 60, 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 58, 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "f132c37b": { 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 58, 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:52:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 139.8, 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "QUIESCED", 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:25.840 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:52:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30c80a2cd0 msgr2=0x7f30c80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30c80a2cd0 0x7f30c80a5190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f30d803d040 tx=0x7f30d80318a0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30e81008d0 msgr2=0x7f30e819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30e81008d0 0x7f30e819dfb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f30d802a980 tx=0x7f30d8003b90 comp rx=0 tx=0).stop 2024-09-10T07:52:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 shutdown_connections 2024-09-10T07:52:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e81068d0 0x7f30e8196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f30c80a2cd0 0x7f30c80a5190 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e8101270 0x7f30e8196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 --2- 172.21.15.46:0/3912249539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30e81008d0 0x7f30e819dfb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.839+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 >> 172.21.15.46:0/3912249539 conn(0x7f30e80fc610 msgr2=0x7f30e8069ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.841+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 shutdown_connections 2024-09-10T07:52:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:25.841+0000 7f30eca00640 1 -- 172.21.15.46:0/3912249539 wait complete. 2024-09-10T07:52:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.5 MiB/s wr, 325 op/s 2024-09-10T07:52:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:52:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:52:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:52:26.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:52:26.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:26 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:52:26.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.5 MiB/s wr, 325 op/s 2024-09-10T07:52:26.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:52:26.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:52:26.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:52:26.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:52:26.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:26 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:52:26.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.5 MiB/s wr, 325 op/s 2024-09-10T07:52:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:52:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:52:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:52:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:52:26.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:52:26.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:52:26.922 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f132c37b', quiesce took 9.8 seconds. Will release after: 10.05 2024-09-10T07:52:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:27 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:52:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:27 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:52:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:27 smithi181 ceph-mon[35397]: scrub summary: idle 2024-09-10T07:52:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:27 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi046 to 177.5M 2024-09-10T07:52:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:27 smithi181 ceph-mon[35397]: Unable to set osd_memory_target on smithi046 to 186147942: error parsing value: Value '186147942' is below minimum 939524096 2024-09-10T07:52:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:27 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:52:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:27 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:52:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:27 smithi063 ceph-mon[35020]: scrub summary: idle 2024-09-10T07:52:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:27 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi046 to 177.5M 2024-09-10T07:52:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:27 smithi063 ceph-mon[35020]: Unable to set osd_memory_target on smithi046 to 186147942: error parsing value: Value '186147942' is below minimum 939524096 2024-09-10T07:52:27.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:27 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:52:27.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:27 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:52:27.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:27 smithi046 ceph-mon[30705]: scrub summary: idle 2024-09-10T07:52:27.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:27 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi046 to 177.5M 2024-09-10T07:52:27.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:27 smithi046 ceph-mon[30705]: Unable to set osd_memory_target on smithi046 to 186147942: error parsing value: Value '186147942' is below minimum 939524096 2024-09-10T07:52:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:28 smithi181 ceph-mon[35397]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-09-10T07:52:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:28 smithi063 ceph-mon[35020]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-09-10T07:52:28.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:28 smithi046 ceph-mon[30705]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.0 MiB/s wr, 574 op/s 2024-09-10T07:52:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.082+0000 7f838ae00640 1 -- 172.21.15.46:0/102845702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 msgr2=0x7f838410a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.082+0000 7f838ae00640 1 --2- 172.21.15.46:0/102845702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 0x7f838410a9a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8374009860 tx=0x7f837402a380 comp rx=0 tx=0).stop 2024-09-10T07:52:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.083+0000 7f838ae00640 1 -- 172.21.15.46:0/102845702 shutdown_connections 2024-09-10T07:52:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.083+0000 7f838ae00640 1 --2- 172.21.15.46:0/102845702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f838410bec0 0x7f83841127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.083+0000 7f838ae00640 1 --2- 172.21.15.46:0/102845702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f838410b500 0x7f838410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.083+0000 7f838ae00640 1 --2- 172.21.15.46:0/102845702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 0x7f838410a9a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.083+0000 7f838ae00640 1 -- 172.21.15.46:0/102845702 >> 172.21.15.46:0/102845702 conn(0x7f8384070d10 msgr2=0x7f8384073130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.083+0000 7f838ae00640 1 -- 172.21.15.46:0/102845702 shutdown_connections 2024-09-10T07:52:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.083+0000 7f838ae00640 1 -- 172.21.15.46:0/102845702 wait complete. 2024-09-10T07:52:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.084+0000 7f838ae00640 1 Processor -- start 2024-09-10T07:52:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.084+0000 7f838ae00640 1 -- start start 2024-09-10T07:52:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f838ae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 0x7f838419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f838ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f838410b500 0x7f83841a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f838ae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f838410bec0 0x7f838419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f838ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8384116040 con 0x7f838410b500 2024-09-10T07:52:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f838ae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8384115ec0 con 0x7f838410bec0 2024-09-10T07:52:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f838ae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83841161c0 con 0x7f838410a5a0 2024-09-10T07:52:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f8383e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 0x7f838419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f8383e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 0x7f838419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43170/0 (socket says 172.21.15.46:43170) 2024-09-10T07:52:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f8383e00640 1 -- 172.21.15.46:0/563612333 learned_addr learned my addr 172.21.15.46:0/563612333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f8383400640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f838410b500 0x7f83841a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.085+0000 7f8388e00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f838410bec0 0x7f838419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.086+0000 7f8383e00640 1 -- 172.21.15.46:0/563612333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f838410bec0 msgr2=0x7f838419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.086+0000 7f8383e00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f838410bec0 0x7f838419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.086+0000 7f8383e00640 1 -- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f838410b500 msgr2=0x7f83841a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.086+0000 7f8383e00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f838410b500 0x7f83841a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.086+0000 7f8383e00640 1 -- 172.21.15.46:0/563612333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83841b7080 con 0x7f838410a5a0 2024-09-10T07:52:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.086+0000 7f8383e00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 0x7f838419e450 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f837402a970 tx=0x7f8374033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.087+0000 7f8380c00640 1 -- 172.21.15.46:0/563612333 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374037040 con 0x7f838410a5a0 2024-09-10T07:52:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.087+0000 7f838ae00640 1 -- 172.21.15.46:0/563612333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83841b7310 con 0x7f838410a5a0 2024-09-10T07:52:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.087+0000 7f8380c00640 1 -- 172.21.15.46:0/563612333 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8374040070 con 0x7f838410a5a0 2024-09-10T07:52:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.087+0000 7f8380c00640 1 -- 172.21.15.46:0/563612333 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374037040 con 0x7f838410a5a0 2024-09-10T07:52:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.087+0000 7f838ae00640 1 -- 172.21.15.46:0/563612333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83841b7850 con 0x7f838410a5a0 2024-09-10T07:52:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.094+0000 7f8380c00640 1 -- 172.21.15.46:0/563612333 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8374030880 con 0x7f838410a5a0 2024-09-10T07:52:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.096+0000 7f838dc71740 1 Processor -- start 2024-09-10T07:52:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.096+0000 7f838dc71740 1 -- start start 2024-09-10T07:52:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f8380c00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83680a2f10 0x7f83680a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f8380c00640 1 -- 172.21.15.46:0/563612333 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f83740eeac0 con 0x7f838410a5a0 2024-09-10T07:52:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f838dc71740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171080090 0x559171080490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f838dc71740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171085490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f838dc71740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 0x559171087f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f838dc71740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559170fe5100 con 0x559171085a60 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f838dc71740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559170f8d1b0 con 0x5591710809d0 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f838dc71740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559170fe10c0 con 0x559171080090 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f8388e00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83680a2f10 0x7f83680a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.097+0000 7f8383400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171085490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.098+0000 7f8383400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171085490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:46056/0 (socket says 172.21.15.46:46056) 2024-09-10T07:52:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.098+0000 7f8383400640 1 -- 172.21.15.46:0/1138074278 learned_addr learned my addr 172.21.15.46:0/1138074278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.098+0000 7f8383400640 1 -- 172.21.15.46:0/1138074278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171080090 msgr2=0x559171080490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.098+0000 7f8383400640 1 --2- 172.21.15.46:0/1138074278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171080090 0x559171080490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.098+0000 7f8383400640 1 -- 172.21.15.46:0/1138074278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 msgr2=0x559171087f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:52:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.098+0000 7f8383400640 1 --2- 172.21.15.46:0/1138074278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 0x559171087f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.098+0000 7f8383400640 1 -- 172.21.15.46:0/1138074278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559170cf8210 con 0x5591710809d0 2024-09-10T07:52:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.098+0000 7f8388e00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83680a2f10 0x7f83680a53d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f837c005fd0 tx=0x7f837c009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.099+0000 7f8383400640 1 --2- 172.21.15.46:0/1138074278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171085490 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f836c009be0 tx=0x7f836c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a8478d01433124 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.099+0000 7f8379600640 1 -- 172.21.15.46:0/1138074278 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c005020 con 0x5591710809d0 2024-09-10T07:52:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.099+0000 7f8379600640 1 -- 172.21.15.46:0/1138074278 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c0047a0 con 0x5591710809d0 2024-09-10T07:52:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.099+0000 7f8379600640 1 -- 172.21.15.46:0/1138074278 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c005020 con 0x5591710809d0 2024-09-10T07:52:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.101+0000 7f838dc71740 1 -- 172.21.15.46:0/1138074278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 msgr2=0x559171085490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.101+0000 7f838dc71740 1 --2- 172.21.15.46:0/1138074278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171085490 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f836c009be0 tx=0x7f836c0117b0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.102+0000 7f838dc71740 1 -- 172.21.15.46:0/1138074278 shutdown_connections 2024-09-10T07:52:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.102+0000 7f838dc71740 1 --2- 172.21.15.46:0/1138074278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 0x559171087f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.102+0000 7f838dc71740 1 --2- 172.21.15.46:0/1138074278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171085490 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.102+0000 7f838dc71740 1 --2- 172.21.15.46:0/1138074278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171080090 0x559171080490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.102+0000 7f838dc71740 1 -- 172.21.15.46:0/1138074278 >> 172.21.15.46:0/1138074278 conn(0x55917107b700 msgr2=0x55917107bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.102+0000 7f838dc71740 1 -- 172.21.15.46:0/1138074278 shutdown_connections 2024-09-10T07:52:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.102+0000 7f838dc71740 1 -- 172.21.15.46:0/1138074278 wait complete. 2024-09-10T07:52:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.103+0000 7f838dc71740 1 -- start start 2024-09-10T07:52:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.104+0000 7f838dc71740 1 Processor -- start 2024-09-10T07:52:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.105+0000 7f838dc71740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.105+0000 7f838dc71740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591710809d0 0x55917108a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.105+0000 7f838dc71740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x559171091850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.105+0000 7f838dc71740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559171088a30 con 0x5591710809d0 2024-09-10T07:52:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.105+0000 7f838dc71740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559170f6e090 con 0x559171080090 2024-09-10T07:52:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.105+0000 7f838dc71740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559171088bb0 con 0x559171085a60 2024-09-10T07:52:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.105+0000 7f838dc71740 10 client.0 _ll_register_callbacks cb 0x559170fc2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.106+0000 7f8388e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x559171091850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.106+0000 7f8388e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x559171091850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43196/0 (socket says 172.21.15.46:43196) 2024-09-10T07:52:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.106+0000 7f8388e00640 1 -- 172.21.15.46:0/4244368038 learned_addr learned my addr 172.21.15.46:0/4244368038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.106+0000 7f8383e00640 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.106+0000 7f8383400640 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591710809d0 0x55917108a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.107+0000 7f8383e00640 1 -- 172.21.15.46:0/4244368038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 msgr2=0x559171091850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.107+0000 7f8383e00640 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x559171091850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.107+0000 7f8383e00640 1 -- 172.21.15.46:0/4244368038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591710809d0 msgr2=0x55917108a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.107+0000 7f8383e00640 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591710809d0 0x55917108a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.107+0000 7f8383e00640 1 -- 172.21.15.46:0/4244368038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559170dc4620 con 0x559171080090 2024-09-10T07:52:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.107+0000 7f8383e00640 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108a2d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f83740ec290 tx=0x7f83740ba900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7852044e3feb2ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.107+0000 7f8372000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.108+0000 7f8372000640 1 -- 172.21.15.46:0/4244368038 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83740bafa0 con 0x559171080090 2024-09-10T07:52:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.108+0000 7f8372000640 1 -- 172.21.15.46:0/4244368038 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f83740bb200 con 0x559171080090 2024-09-10T07:52:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.108+0000 7f838dc71740 1 -- 172.21.15.46:0/4244368038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836c0047a0 con 0x559171080090 2024-09-10T07:52:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.108+0000 7f8372000640 1 -- 172.21.15.46:0/4244368038 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f83740bf350 con 0x559171080090 2024-09-10T07:52:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.109+0000 7f838dc71740 10 client.29315 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.109+0000 7f838dc71740 1 -- 172.21.15.46:0/4244368038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x559171094640 con 0x559171080090 2024-09-10T07:52:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.109+0000 7f8372000640 1 -- 172.21.15.46:0/4244368038 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c0047a0 con 0x559171080090 2024-09-10T07:52:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.109+0000 7f8372000640 1 -- 172.21.15.46:0/4244368038 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f837403a070 con 0x559171080090 2024-09-10T07:52:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.109+0000 7f838dc71740 10 client.29315 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.109+0000 7f838dc71740 10 client.29315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.110+0000 7f838dc71740 20 client.29315 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.110+0000 7f838dc71740 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559171095dd0 0x5591710981f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.110+0000 7f838dc71740 4 client.29315 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.110+0000 7f838dc71740 1 -- 172.21.15.46:0/4244368038 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559170ff2830 con 0x559171095dd0 2024-09-10T07:52:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.110+0000 7f8383400640 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559171095dd0 0x5591710981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.112+0000 7f8383400640 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559171095dd0 0x5591710981f0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f04108b0ba592e3 server_cookie=f02384e6974a2696 in_seq=0 out_seq=0 2024-09-10T07:52:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.112+0000 7f8372000640 10 client.29315 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.113+0000 7f8372000640 1 -- 172.21.15.46:0/4244368038 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559170ff2830 con 0x559171095dd0 2024-09-10T07:52:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.113+0000 7f8372000640 10 client.29315 handle_command_reply: tid=0 2024-09-10T07:52:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.113+0000 7f838dc71740 1 client.29315 shutdown 2024-09-10T07:52:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 -- 172.21.15.46:0/4244368038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 msgr2=0x55917108a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108a2d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f83740ec290 tx=0x7f83740ba900 comp rx=0 tx=0).stop 2024-09-10T07:52:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 -- 172.21.15.46:0/4244368038 shutdown_connections 2024-09-10T07:52:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559171095dd0 0x5591710981f0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x559171091850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591710809d0 0x55917108a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 --2- 172.21.15.46:0/4244368038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108a2d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 -- 172.21.15.46:0/4244368038 >> 172.21.15.46:0/4244368038 conn(0x55917107b700 msgr2=0x55917107cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f8372000640 0 client.29315 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 -- 172.21.15.46:0/4244368038 shutdown_connections 2024-09-10T07:52:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.114+0000 7f838dc71740 1 -- 172.21.15.46:0/4244368038 wait complete. 2024-09-10T07:52:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.115+0000 7f838dc71740 20 client.29315 trim_cache size 0 max 16384 2024-09-10T07:52:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.144+0000 7f838dc71740 1 Processor -- start 2024-09-10T07:52:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.144+0000 7f838dc71740 1 -- start start 2024-09-10T07:52:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.144+0000 7f838dc71740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171080090 0x55917108a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.144+0000 7f838dc71740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171091810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.144+0000 7f838dc71740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x55917107a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.144+0000 7f838dc71740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83841c1c50 con 0x559171080090 2024-09-10T07:52:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.144+0000 7f838dc71740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83841c3910 con 0x5591710809d0 2024-09-10T07:52:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.144+0000 7f838dc71740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83841c2320 con 0x559171085a60 2024-09-10T07:52:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.145+0000 7f8383e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171080090 0x55917108a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.145+0000 7f8383400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171091810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.145+0000 7f8383e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171080090 0x55917108a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40350/0 (socket says 172.21.15.46:40350) 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.146+0000 7f8383e00640 1 -- 172.21.15.46:0/1241361038 learned_addr learned my addr 172.21.15.46:0/1241361038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.145+0000 7f8388e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x55917107a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.145+0000 7f8383400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171091810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:46086/0 (socket says 172.21.15.46:46086) 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.146+0000 7f8383e00640 1 -- 172.21.15.46:0/1241361038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 msgr2=0x55917107a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.146+0000 7f8383e00640 1 --2- 172.21.15.46:0/1241361038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x55917107a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.146+0000 7f8383e00640 1 -- 172.21.15.46:0/1241361038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 msgr2=0x559171091810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.146+0000 7f8383e00640 1 --2- 172.21.15.46:0/1241361038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171091810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.146+0000 7f8383e00640 1 -- 172.21.15.46:0/1241361038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559171088cf0 con 0x559171080090 2024-09-10T07:52:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.146+0000 7f8388e00640 1 --2- 172.21.15.46:0/1241361038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x55917107a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.147+0000 7f8383e00640 1 --2- 172.21.15.46:0/1241361038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171080090 0x55917108a6d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f83740b7920 tx=0x7f83740baa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67febbb669b2592c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.147+0000 7f8378c00640 1 -- 172.21.15.46:0/1241361038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83740bafa0 con 0x559171080090 2024-09-10T07:52:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.147+0000 7f8378c00640 1 -- 172.21.15.46:0/1241361038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83740b6e60 con 0x559171080090 2024-09-10T07:52:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.147+0000 7f8378c00640 1 -- 172.21.15.46:0/1241361038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83740bb200 con 0x559171080090 2024-09-10T07:52:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.148+0000 7f838dc71740 1 -- 172.21.15.46:0/1241361038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171080090 msgr2=0x55917108a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.148+0000 7f838dc71740 1 --2- 172.21.15.46:0/1241361038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171080090 0x55917108a6d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f83740b7920 tx=0x7f83740baa30 comp rx=0 tx=0).stop 2024-09-10T07:52:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.149+0000 7f838dc71740 1 -- 172.21.15.46:0/1241361038 shutdown_connections 2024-09-10T07:52:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.149+0000 7f838dc71740 1 --2- 172.21.15.46:0/1241361038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559171085a60 0x55917107a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.149+0000 7f838dc71740 1 --2- 172.21.15.46:0/1241361038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591710809d0 0x559171091810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.149+0000 7f838dc71740 1 --2- 172.21.15.46:0/1241361038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171080090 0x55917108a6d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.149+0000 7f838dc71740 1 -- 172.21.15.46:0/1241361038 >> 172.21.15.46:0/1241361038 conn(0x55917107b700 msgr2=0x55917107a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.150+0000 7f838dc71740 1 -- 172.21.15.46:0/1241361038 shutdown_connections 2024-09-10T07:52:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.150+0000 7f838dc71740 1 -- 172.21.15.46:0/1241361038 wait complete. 2024-09-10T07:52:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.150+0000 7f838dc71740 1 -- start start 2024-09-10T07:52:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.150+0000 7f838dc71740 1 Processor -- start 2024-09-10T07:52:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.151+0000 7f838dc71740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.151+0000 7f838dc71740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5591710809d0 0x55917108f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.151+0000 7f838dc71740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 0x559171090040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f8383e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f8383e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:46100/0 (socket says 172.21.15.46:46100) 2024-09-10T07:52:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f8383400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5591710809d0 0x55917108f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f8383400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5591710809d0 0x55917108f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:43214/0 (socket says 172.21.15.46:43214) 2024-09-10T07:52:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f8388e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 0x559171090040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f8388e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 0x559171090040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40352/0 (socket says 172.21.15.46:40352) 2024-09-10T07:52:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f838dc71740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83841c3180 con 0x559171085a60 2024-09-10T07:52:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f838dc71740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83841c2ab0 con 0x559171080090 2024-09-10T07:52:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.152+0000 7f8388e00640 1 -- 172.21.15.46:0/3317258700 learned_addr learned my addr 172.21.15.46:0/3317258700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.153+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83841bfc30 con 0x5591710809d0 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.153+0000 7f838dc71740 10 client.0 _ll_register_callbacks cb 0x7f8374033f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.153+0000 7f8383e00640 1 -- 172.21.15.46:0/3317258700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5591710809d0 msgr2=0x55917108f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.153+0000 7f8383e00640 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5591710809d0 0x55917108f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.153+0000 7f8383e00640 1 -- 172.21.15.46:0/3317258700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 msgr2=0x559171090040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.153+0000 7f8383e00640 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 0x559171090040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.153+0000 7f8383e00640 1 -- 172.21.15.46:0/3317258700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559171090650 con 0x559171080090 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f8383400640 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5591710809d0 0x55917108f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f8383e00640 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108eb90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f83740b9250 tx=0x7f83740b94b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f97d42913af8b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f8372000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f8372000640 1 -- 172.21.15.46:0/3317258700 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83740b7010 con 0x559171080090 2024-09-10T07:52:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83740b6e60 con 0x559171080090 2024-09-10T07:52:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f8372000640 1 -- 172.21.15.46:0/3317258700 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f83740b5d20 con 0x559171080090 2024-09-10T07:52:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f8372000640 1 -- 172.21.15.46:0/3317258700 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f83740ba590 con 0x559171080090 2024-09-10T07:52:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f838dc71740 10 client.29321 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.154+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x559171090650 con 0x559171080090 2024-09-10T07:52:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.155+0000 7f8372000640 1 -- 172.21.15.46:0/3317258700 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83740b6e60 con 0x559171080090 2024-09-10T07:52:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.157+0000 7f8372000640 1 -- 172.21.15.46:0/3317258700 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f837403a070 con 0x559171080090 2024-09-10T07:52:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.157+0000 7f838dc71740 10 client.29321 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.157+0000 7f838dc71740 10 client.29321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.157+0000 7f838dc71740 20 client.29321 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.157+0000 7f838dc71740 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559171100c60 0x559171103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.157+0000 7f838dc71740 4 client.29321 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:52:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.157+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559170fecb80 con 0x559171100c60 2024-09-10T07:52:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.158+0000 7f8383400640 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559171100c60 0x559171103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.159+0000 7f8383400640 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559171100c60 0x559171103080 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a36467be1c1eebb6 server_cookie=a59176c84bd764d5 in_seq=0 out_seq=0 2024-09-10T07:52:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.159+0000 7f8372000640 10 client.29321 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.186+0000 7f8372000640 1 -- 172.21.15.46:0/3317258700 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559170fecb80 con 0x559171100c60 2024-09-10T07:52:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.186+0000 7f8372000640 10 client.29321 handle_command_reply: tid=0 2024-09-10T07:52:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.186+0000 7f838dc71740 1 client.29321 shutdown 2024-09-10T07:52:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.186+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 msgr2=0x55917108eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.186+0000 7f838dc71740 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108eb90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f83740b9250 tx=0x7f83740b94b0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.188+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 shutdown_connections 2024-09-10T07:52:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.188+0000 7f838dc71740 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559171100c60 0x559171103080 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.188+0000 7f838dc71740 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559171085a60 0x559171090040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.188+0000 7f838dc71740 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5591710809d0 0x55917108f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.188+0000 7f838dc71740 1 --2- 172.21.15.46:0/3317258700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559171080090 0x55917108eb90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.188+0000 7f8372000640 0 client.29321 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.188+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 >> 172.21.15.46:0/3317258700 conn(0x55917107b700 msgr2=0x55917107a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.188+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 shutdown_connections 2024-09-10T07:52:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.189+0000 7f838dc71740 1 -- 172.21.15.46:0/3317258700 wait complete. 2024-09-10T07:52:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.189+0000 7f838dc71740 20 client.29321 trim_cache size 0 max 16384 2024-09-10T07:52:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:52:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:52:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:52:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:52:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 -- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83680a2f10 msgr2=0x7f83680a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83680a2f10 0x7f83680a53d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f837c005fd0 tx=0x7f837c009520 comp rx=0 tx=0).stop 2024-09-10T07:52:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 -- 172.21.15.46:0/563612333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 msgr2=0x7f838419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 0x7f838419e450 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f837402a970 tx=0x7f8374033d10 comp rx=0 tx=0).stop 2024-09-10T07:52:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 -- 172.21.15.46:0/563612333 shutdown_connections 2024-09-10T07:52:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f838410bec0 0x7f838419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83680a2f10 0x7f83680a53d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f838410b500 0x7f83841a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 --2- 172.21.15.46:0/563612333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838410a5a0 0x7f838419e450 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 -- 172.21.15.46:0/563612333 >> 172.21.15.46:0/563612333 conn(0x7f8384070d10 msgr2=0x7f838406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 -- 172.21.15.46:0/563612333 shutdown_connections 2024-09-10T07:52:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:29.190+0000 7f838ae00640 1 -- 172.21.15.46:0/563612333 wait complete. 2024-09-10T07:52:30.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:30 smithi181 ceph-mon[35397]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.8 MiB/s wr, 639 op/s 2024-09-10T07:52:30.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:30 smithi063 ceph-mon[35020]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.8 MiB/s wr, 639 op/s 2024-09-10T07:52:30.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:30 smithi046 ceph-mon[30705]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.8 MiB/s wr, 639 op/s 2024-09-10T07:52:32.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:32 smithi181 ceph-mon[35397]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.0 MiB/s wr, 657 op/s 2024-09-10T07:52:32.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:32 smithi063 ceph-mon[35020]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.0 MiB/s wr, 657 op/s 2024-09-10T07:52:32.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:32 smithi046 ceph-mon[30705]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.0 MiB/s wr, 657 op/s 2024-09-10T07:52:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:34 smithi181 ceph-mon[35397]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-09-10T07:52:34.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:34 smithi063 ceph-mon[35020]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-09-10T07:52:34.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:34 smithi046 ceph-mon[30705]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-09-10T07:52:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:36 smithi181 ceph-mon[35397]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 587 op/s 2024-09-10T07:52:36.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:36 smithi063 ceph-mon[35020]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 587 op/s 2024-09-10T07:52:36.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:36 smithi046 ceph-mon[30705]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 587 op/s 2024-09-10T07:52:36.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:52:37.397 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 -- 172.21.15.46:0/3305867182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c1049c0 msgr2=0x7ff21c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 --2- 172.21.15.46:0/3305867182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c1049c0 0x7ff21c108a10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff208009900 tx=0x7ff20802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 -- 172.21.15.46:0/3305867182 shutdown_connections 2024-09-10T07:52:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 --2- 172.21.15.46:0/3305867182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c109140 0x7ff21c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 --2- 172.21.15.46:0/3305867182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c1049c0 0x7ff21c108a10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 --2- 172.21.15.46:0/3305867182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff21c104010 0x7ff21c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 -- 172.21.15.46:0/3305867182 >> 172.21.15.46:0/3305867182 conn(0x7ff21c0ffa50 msgr2=0x7ff21c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 -- 172.21.15.46:0/3305867182 shutdown_connections 2024-09-10T07:52:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.171+0000 7ff221c00640 1 -- 172.21.15.46:0/3305867182 wait complete. 2024-09-10T07:52:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.172+0000 7ff221c00640 1 Processor -- start 2024-09-10T07:52:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff221c00640 1 -- start start 2024-09-10T07:52:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff221c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff21c104010 0x7ff21c1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff221c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c1049c0 0x7ff21c1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff221c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c109140 0x7ff21c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff221c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1141b0 con 0x7ff21c109140 2024-09-10T07:52:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff221c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c114030 con 0x7ff21c104010 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff221c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c114330 con 0x7ff21c1049c0 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff21b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c109140 0x7ff21c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff21b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c109140 0x7ff21c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42026/0 (socket says 172.21.15.46:42026) 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff21b400640 1 -- 172.21.15.46:0/1311300849 learned_addr learned my addr 172.21.15.46:0/1311300849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff21a000640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c1049c0 0x7ff21c1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.173+0000 7ff21aa00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff21c104010 0x7ff21c1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff21b400640 1 -- 172.21.15.46:0/1311300849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c1049c0 msgr2=0x7ff21c1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff21b400640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c1049c0 0x7ff21c1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff21b400640 1 -- 172.21.15.46:0/1311300849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff21c104010 msgr2=0x7ff21c1a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff21b400640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff21c104010 0x7ff21c1a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff21b400640 1 -- 172.21.15.46:0/1311300849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c19acf0 con 0x7ff21c109140 2024-09-10T07:52:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff21b400640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c109140 0x7ff21c19a5c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff21400eb10 tx=0x7ff21400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff213400640 1 -- 172.21.15.46:0/1311300849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214004b20 con 0x7ff21c109140 2024-09-10T07:52:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff213400640 1 -- 172.21.15.46:0/1311300849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff214004cc0 con 0x7ff21c109140 2024-09-10T07:52:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c19afe0 con 0x7ff21c109140 2024-09-10T07:52:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.174+0000 7ff213400640 1 -- 172.21.15.46:0/1311300849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214010760 con 0x7ff21c109140 2024-09-10T07:52:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.175+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c1a72d0 con 0x7ff21c109140 2024-09-10T07:52:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.176+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21c1043f0 con 0x7ff21c109140 2024-09-10T07:52:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.178+0000 7ff213400640 1 -- 172.21.15.46:0/1311300849 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff214005020 con 0x7ff21c109140 2024-09-10T07:52:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.179+0000 7ff213400640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2000a2c00 0x7ff2000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.179+0000 7ff213400640 1 -- 172.21.15.46:0/1311300849 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7ff2140cf9d0 con 0x7ff21c109140 2024-09-10T07:52:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.179+0000 7ff21aa00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2000a2c00 0x7ff2000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.180+0000 7ff21aa00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2000a2c00 0x7ff2000a50c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009bd0 tx=0x7ff20c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.183+0000 7ff213400640 1 -- 172.21.15.46:0/1311300849 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff21409c550 con 0x7ff21c109140 2024-09-10T07:52:38.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.409+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff21c10c380 con 0x7ff21c109140 2024-09-10T07:52:38.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.410+0000 7ff213400640 1 -- 172.21.15.46:0/1311300849 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11986 (secure 0 0 0) 0x7ff21409be50 con 0x7ff21c109140 2024-09-10T07:52:38.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:38.411 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":60,"btime":"2024-09-10T07:51:54:063936+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:51:53.066973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:52:38.411 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 60 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2000a2c00 msgr2=0x7ff2000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2000a2c00 0x7ff2000a50c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009bd0 tx=0x7ff20c005f90 comp rx=0 tx=0).stop 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c109140 msgr2=0x7ff21c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c109140 0x7ff21c19a5c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff21400eb10 tx=0x7ff21400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 shutdown_connections 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff2000a2c00 0x7ff2000a50c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c109140 0x7ff21c19a5c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c1049c0 0x7ff21c1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 --2- 172.21.15.46:0/1311300849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff21c104010 0x7ff21c1a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.412+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 >> 172.21.15.46:0/1311300849 conn(0x7ff21c0ffa50 msgr2=0x7ff21c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.413+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 shutdown_connections 2024-09-10T07:52:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:38.413+0000 7ff221c00640 1 -- 172.21.15.46:0/1311300849 wait complete. 2024-09-10T07:52:38.515 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:38 smithi046 ceph-mon[30705]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T07:52:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:38 smithi181 ceph-mon[35397]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T07:52:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:38 smithi063 ceph-mon[35020]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T07:52:39.468 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --set-id f132c37b --release --await' 2024-09-10T07:52:39.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --set-id f132c37b --release --await 2024-09-10T07:52:39.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:39 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1311300849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:39.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:39 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1311300849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:39.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:39 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1311300849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 -- 172.21.15.46:0/2628165235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 msgr2=0x7f0c881021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 --2- 172.21.15.46:0/2628165235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 0x7f0c881021d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009900 tx=0x7f0c7802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 -- 172.21.15.46:0/2628165235 shutdown_connections 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 --2- 172.21.15.46:0/2628165235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c88106f20 0x7f0c8810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 --2- 172.21.15.46:0/2628165235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c881027a0 0x7f0c881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 --2- 172.21.15.46:0/2628165235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 0x7f0c881021d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 -- 172.21.15.46:0/2628165235 >> 172.21.15.46:0/2628165235 conn(0x7f0c880fd8b0 msgr2=0x7f0c880ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 -- 172.21.15.46:0/2628165235 shutdown_connections 2024-09-10T07:52:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.454+0000 7f0c8d400640 1 -- 172.21.15.46:0/2628165235 wait complete. 2024-09-10T07:52:40.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.455+0000 7f0c8d400640 1 Processor -- start 2024-09-10T07:52:40.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.455+0000 7f0c8d400640 1 -- start start 2024-09-10T07:52:40.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c8d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 0x7f0c880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c8d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c881027a0 0x7f0c88074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c8d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c88106f20 0x7f0c88074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c8d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c88111f80 con 0x7f0c88101df0 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c8d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0c88111e00 con 0x7f0c881027a0 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c8d400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0c88112100 con 0x7f0c88106f20 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 0x7f0c880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 0x7f0c880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42044/0 (socket says 172.21.15.46:42044) 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c86800640 1 -- 172.21.15.46:0/3474428818 learned_addr learned my addr 172.21.15.46:0/3474428818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c7de00640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c881027a0 0x7f0c88074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.456+0000 7f0c87200640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c88106f20 0x7f0c88074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.457+0000 7f0c7de00640 1 -- 172.21.15.46:0/3474428818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c88106f20 msgr2=0x7f0c88074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.457+0000 7f0c7de00640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c88106f20 0x7f0c88074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.457+0000 7f0c7de00640 1 -- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 msgr2=0x7f0c880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.457+0000 7f0c7de00640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 0x7f0c880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.457+0000 7f0c7de00640 1 -- 172.21.15.46:0/3474428818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c881a68a0 con 0x7f0c881027a0 2024-09-10T07:52:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.457+0000 7f0c86800640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 0x7f0c880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.457+0000 7f0c7de00640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c881027a0 0x7f0c88074a30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000e9f0 tx=0x7f0c7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.458+0000 7f0c7fe00640 1 -- 172.21.15.46:0/3474428818 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7000ccc0 con 0x7f0c881027a0 2024-09-10T07:52:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.458+0000 7f0c8d400640 1 -- 172.21.15.46:0/3474428818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c881a6b90 con 0x7f0c881027a0 2024-09-10T07:52:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.458+0000 7f0c7fe00640 1 -- 172.21.15.46:0/3474428818 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c70010910 con 0x7f0c881027a0 2024-09-10T07:52:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.458+0000 7f0c7fe00640 1 -- 172.21.15.46:0/3474428818 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70018960 con 0x7f0c881027a0 2024-09-10T07:52:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.458+0000 7f0c8d400640 1 -- 172.21.15.46:0/3474428818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c881a70d0 con 0x7f0c881027a0 2024-09-10T07:52:40.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.461+0000 7f0c90039740 1 Processor -- start 2024-09-10T07:52:40.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.461+0000 7f0c90039740 1 -- start start 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.461+0000 7f0c90039740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624daf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.461+0000 7f0c90039740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.461+0000 7f0c90039740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 0x559e624e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.461+0000 7f0c90039740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e62438650 con 0x559e624dab20 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.461+0000 7f0c90039740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559e623dff10 con 0x559e624e5d00 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.461+0000 7f0c90039740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559e62434610 con 0x559e624db440 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c7fe00640 1 -- 172.21.15.46:0/3474428818 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0c70010ac0 con 0x7f0c881027a0 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c86800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c87200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624daf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c86800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35918/0 (socket says 172.21.15.46:35918) 2024-09-10T07:52:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c86800640 1 -- 172.21.15.46:0/4093460907 learned_addr learned my addr 172.21.15.46:0/4093460907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c7de00640 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 0x559e624e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c86800640 1 -- 172.21.15.46:0/4093460907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 msgr2=0x559e624e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c86800640 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 0x559e624e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c86800640 1 -- 172.21.15.46:0/4093460907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 msgr2=0x559e624daf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c86800640 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624daf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c86800640 1 -- 172.21.15.46:0/4093460907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e62147580 con 0x559e624db440 2024-09-10T07:52:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.462+0000 7f0c87200640 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624daf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.463+0000 7f0c86800640 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e57c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009a20 tx=0x7f0c78005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8062fa13c296935f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.463+0000 7f0c77400640 1 -- 172.21.15.46:0/4093460907 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78040070 con 0x559e624db440 2024-09-10T07:52:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.463+0000 7f0c77400640 1 -- 172.21.15.46:0/4093460907 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c78031970 con 0x559e624db440 2024-09-10T07:52:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.463+0000 7f0c77400640 1 -- 172.21.15.46:0/4093460907 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78035040 con 0x559e624db440 2024-09-10T07:52:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.463+0000 7f0c7fe00640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0c6c0a2b80 0x7f0c6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.463+0000 7f0c7fe00640 1 -- 172.21.15.46:0/3474428818 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0c700cf790 con 0x7f0c881027a0 2024-09-10T07:52:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.463+0000 7f0c87200640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0c6c0a2b80 0x7f0c6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 -- 172.21.15.46:0/4093460907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 msgr2=0x559e624e57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e57c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009a20 tx=0x7f0c78005dd0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 -- 172.21.15.46:0/4093460907 shutdown_connections 2024-09-10T07:52:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 0x559e624e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e57c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 --2- 172.21.15.46:0/4093460907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624daf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 -- 172.21.15.46:0/4093460907 >> 172.21.15.46:0/4093460907 conn(0x559e624d70b0 msgr2=0x559e624d7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 -- 172.21.15.46:0/4093460907 shutdown_connections 2024-09-10T07:52:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c87200640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0c6c0a2b80 0x7f0c6c0a5040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009fd0 tx=0x7f0c80009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.464+0000 7f0c90039740 1 -- 172.21.15.46:0/4093460907 wait complete. 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 1 -- start start 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 1 Processor -- start 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e624e88d0 con 0x559e624db440 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559e623f3350 con 0x559e624dab20 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559e624e8a50 con 0x559e624e5d00 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.465+0000 7f0c90039740 10 client.0 _ll_register_callbacks cb 0x559e62416290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53222/0 (socket says 172.21.15.46:53222) 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c87200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 -- 172.21.15.46:0/2657018584 learned_addr learned my addr 172.21.15.46:0/2657018584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c7de00640 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 -- 172.21.15.46:0/2657018584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 msgr2=0x559e624e3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 -- 172.21.15.46:0/2657018584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 msgr2=0x559e624e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 -- 172.21.15.46:0/2657018584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e62211960 con 0x559e624dab20 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c86800640 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e2bc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0c780055c0 tx=0x7f0c780052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f17ebb8dabfd90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.466+0000 7f0c6be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.467+0000 7f0c6be00640 1 -- 172.21.15.46:0/2657018584 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7804b070 con 0x559e624dab20 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.467+0000 7f0c6be00640 1 -- 172.21.15.46:0/2657018584 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0c780035f0 con 0x559e624dab20 2024-09-10T07:52:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.467+0000 7f0c90039740 1 -- 172.21.15.46:0/2657018584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c78031970 con 0x559e624dab20 2024-09-10T07:52:40.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.467+0000 7f0c6be00640 1 -- 172.21.15.46:0/2657018584 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0c78057050 con 0x559e624dab20 2024-09-10T07:52:40.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.467+0000 7f0c90039740 10 client.29339 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:40.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.467+0000 7f0c90039740 1 -- 172.21.15.46:0/2657018584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x559e624f44b0 con 0x559e624dab20 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.467+0000 7f0c6be00640 1 -- 172.21.15.46:0/2657018584 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c78031970 con 0x559e624dab20 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.468+0000 7f0c6be00640 1 -- 172.21.15.46:0/2657018584 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f0c78004900 con 0x559e624dab20 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.468+0000 7f0c90039740 10 client.29339 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.468+0000 7f0c90039740 10 client.29339 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.468+0000 7f0c90039740 20 client.29339 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.468+0000 7f0c90039740 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e624f5800 0x559e624f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.468+0000 7f0c90039740 4 client.29339 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.468+0000 7f0c90039740 1 -- 172.21.15.46:0/2657018584 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e62445bc0 con 0x559e624f5800 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.468+0000 7f0c7de00640 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e624f5800 0x559e624f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.469+0000 7f0c7de00640 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e624f5800 0x559e624f7bf0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed7c3307342fe0e server_cookie=85fdd67b2023458a in_seq=0 out_seq=0 2024-09-10T07:52:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.469+0000 7f0c6be00640 10 client.29339 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.469+0000 7f0c6be00640 1 -- 172.21.15.46:0/2657018584 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e62445bc0 con 0x559e624f5800 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.469+0000 7f0c6be00640 10 client.29339 handle_command_reply: tid=0 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.469+0000 7f0c90039740 1 client.29339 shutdown 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.469+0000 7f0c90039740 1 -- 172.21.15.46:0/2657018584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 msgr2=0x559e624e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.469+0000 7f0c90039740 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e2bc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0c780055c0 tx=0x7f0c780052a0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 1 -- 172.21.15.46:0/2657018584 shutdown_connections 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e624f5800 0x559e624f7bf0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 1 --2- 172.21.15.46:0/2657018584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e2bc0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 1 -- 172.21.15.46:0/2657018584 >> 172.21.15.46:0/2657018584 conn(0x559e624d70b0 msgr2=0x559e624cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c6be00640 0 client.29339 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:40.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 1 -- 172.21.15.46:0/2657018584 shutdown_connections 2024-09-10T07:52:40.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 1 -- 172.21.15.46:0/2657018584 wait complete. 2024-09-10T07:52:40.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.470+0000 7f0c90039740 20 client.29339 trim_cache size 0 max 16384 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.488+0000 7f0c90039740 1 Processor -- start 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.488+0000 7f0c90039740 1 -- start start 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.488+0000 7f0c90039740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.488+0000 7f0c90039740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.488+0000 7f0c90039740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.488+0000 7f0c90039740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881aca50 con 0x559e624db440 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.488+0000 7f0c90039740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881ae710 con 0x559e624dab20 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.488+0000 7f0c90039740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881ad120 con 0x559e624e5d00 2024-09-10T07:52:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42060/0 (socket says 172.21.15.46:42060) 2024-09-10T07:52:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 -- 172.21.15.46:0/4144326514 learned_addr learned my addr 172.21.15.46:0/4144326514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c86800640 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c87200640 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 -- 172.21.15.46:0/4144326514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 msgr2=0x559e624e37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 -- 172.21.15.46:0/4144326514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 msgr2=0x559e624e1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 -- 172.21.15.46:0/4144326514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e624d6ee0 con 0x559e624db440 2024-09-10T07:52:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c87200640 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c86800640 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.489+0000 7f0c7de00640 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3290 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0c70098120 tx=0x7f0c700d4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4ec5a6ddecf955 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c76a00640 1 -- 172.21.15.46:0/4144326514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7009b620 con 0x559e624db440 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c76a00640 1 -- 172.21.15.46:0/4144326514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7009bcc0 con 0x559e624db440 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c76a00640 1 -- 172.21.15.46:0/4144326514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70098330 con 0x559e624db440 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 -- 172.21.15.46:0/4144326514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 msgr2=0x559e624e3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3290 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0c70098120 tx=0x7f0c700d4040 comp rx=0 tx=0).stop 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 -- 172.21.15.46:0/4144326514 shutdown_connections 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624e5d00 0x559e624e37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624db440 0x559e624e3290 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 --2- 172.21.15.46:0/4144326514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624dab20 0x559e624e1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 -- 172.21.15.46:0/4144326514 >> 172.21.15.46:0/4144326514 conn(0x559e624d70b0 msgr2=0x559e624e2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 -- 172.21.15.46:0/4144326514 shutdown_connections 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.490+0000 7f0c90039740 1 -- 172.21.15.46:0/4144326514 wait complete. 2024-09-10T07:52:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.491+0000 7f0c90039740 1 -- start start 2024-09-10T07:52:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.491+0000 7f0c90039740 1 Processor -- start 2024-09-10T07:52:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.491+0000 7f0c90039740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.491+0000 7f0c90039740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.491+0000 7f0c90039740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 0x559e624e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.491+0000 7f0c90039740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881adf80 con 0x559e624dab20 2024-09-10T07:52:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c90039740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881ad8b0 con 0x559e624e5d00 2024-09-10T07:52:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c90039740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881aaa30 con 0x559e624db440 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c90039740 10 client.0 _ll_register_callbacks cb 0x7f0c80009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624dfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42070/0 (socket says 172.21.15.46:42070) 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c87200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 0x559e624e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c86800640 1 -- 172.21.15.46:0/1723793142 learned_addr learned my addr 172.21.15.46:0/1723793142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c7de00640 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c86800640 1 -- 172.21.15.46:0/1723793142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 msgr2=0x559e624e00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c86800640 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c86800640 1 -- 172.21.15.46:0/1723793142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 msgr2=0x559e624e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c86800640 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 0x559e624e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.492+0000 7f0c86800640 1 -- 172.21.15.46:0/1723793142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6255f1f0 con 0x559e624dab20 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c86800640 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624dfb60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0c78004e20 tx=0x7f0c78005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967e0960ff3dbef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c6be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c6be00640 1 -- 172.21.15.46:0/1723793142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7804b040 con 0x559e624dab20 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c6be00640 1 -- 172.21.15.46:0/1723793142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f0c780050a0 con 0x559e624dab20 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c90039740 1 -- 172.21.15.46:0/1723793142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7009bcc0 con 0x559e624dab20 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c6be00640 1 -- 172.21.15.46:0/1723793142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0c78064050 con 0x559e624dab20 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c90039740 10 client.18414 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c90039740 1 -- 172.21.15.46:0/1723793142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x559e625604f0 con 0x559e624dab20 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.493+0000 7f0c6be00640 1 -- 172.21.15.46:0/1723793142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7009bcc0 con 0x559e624dab20 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.494+0000 7f0c6be00640 1 -- 172.21.15.46:0/1723793142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f0c78003890 con 0x559e624dab20 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.494+0000 7f0c90039740 10 client.18414 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.494+0000 7f0c90039740 10 client.18414 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.494+0000 7f0c90039740 20 client.18414 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:40.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:40 smithi046 ceph-mon[30705]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.3 MiB/s wr, 250 op/s 2024-09-10T07:52:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.494+0000 7f0c90039740 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e62560c70 0x559e62563090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.494+0000 7f0c90039740 4 client.18414 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f132c37b", "release": true, "await": true}] 2024-09-10T07:52:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.494+0000 7f0c90039740 1 -- 172.21.15.46:0/1723793142 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f132c37b", "release": true, "await": true}) -- 0x559e624400d0 con 0x559e62560c70 2024-09-10T07:52:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.494+0000 7f0c7de00640 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e62560c70 0x559e62563090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.495+0000 7f0c7de00640 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e62560c70 0x559e62563090 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a31283c19d38e0 server_cookie=c9f3f5ecb7607b81 in_seq=0 out_seq=0 2024-09-10T07:52:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.495+0000 7f0c6be00640 10 client.18414 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:40.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c6be00640 1 -- 172.21.15.46:0/1723793142 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559e624400d0 con 0x559e62560c70 2024-09-10T07:52:40.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c6be00640 10 client.18414 handle_command_reply: tid=0 2024-09-10T07:52:40.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 client.18414 shutdown 2024-09-10T07:52:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 -- 172.21.15.46:0/1723793142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 msgr2=0x559e624dfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624dfb60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0c78004e20 tx=0x7f0c78005540 comp rx=0 tx=0).stop 2024-09-10T07:52:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 -- 172.21.15.46:0/1723793142 shutdown_connections 2024-09-10T07:52:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x559e62560c70 0x559e62563090 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c6be00640 0 client.18414 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e624e5d00 0x559e624e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e624db440 0x559e624e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 --2- 172.21.15.46:0/1723793142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e624dab20 0x559e624dfb60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.518+0000 7f0c90039740 1 -- 172.21.15.46:0/1723793142 >> 172.21.15.46:0/1723793142 conn(0x559e624d70b0 msgr2=0x559e624dcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.519+0000 7f0c90039740 1 -- 172.21.15.46:0/1723793142 shutdown_connections 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.519+0000 7f0c90039740 1 -- 172.21.15.46:0/1723793142 wait complete. 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.519+0000 7f0c90039740 20 client.18414 trim_cache size 0 max 16384 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 60, 2024-09-10T07:52:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 60, 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": { 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "f132c37b": { 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 60, 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "age_ref": 0.0, 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "timeout": 90.0, 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "expiration": 139.8, 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "members": { 2024-09-10T07:52:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "file:/": { 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "excluded": false, 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "state": { 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RELEASED", 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:52:40.522 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:52:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 -- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0c6c0a2b80 msgr2=0x7f0c6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0c6c0a2b80 0x7f0c6c0a5040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009fd0 tx=0x7f0c80009500 comp rx=0 tx=0).stop 2024-09-10T07:52:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 -- 172.21.15.46:0/3474428818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c881027a0 msgr2=0x7f0c88074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c881027a0 0x7f0c88074a30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000e9f0 tx=0x7f0c7000eeb0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 -- 172.21.15.46:0/3474428818 shutdown_connections 2024-09-10T07:52:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c88106f20 0x7f0c88074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c881027a0 0x7f0c88074a30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f0c6c0a2b80 0x7f0c6c0a5040 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 --2- 172.21.15.46:0/3474428818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88101df0 0x7f0c880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 -- 172.21.15.46:0/3474428818 >> 172.21.15.46:0/3474428818 conn(0x7f0c880fd8b0 msgr2=0x7f0c880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 -- 172.21.15.46:0/3474428818 shutdown_connections 2024-09-10T07:52:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:40.520+0000 7f0c8d400640 1 -- 172.21.15.46:0/3474428818 wait complete. 2024-09-10T07:52:40.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:40 smithi181 ceph-mon[35397]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.3 MiB/s wr, 250 op/s 2024-09-10T07:52:40.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:40 smithi063 ceph-mon[35020]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.3 MiB/s wr, 250 op/s 2024-09-10T07:52:41.497 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f132c37b', total seconds elapsed: 24.4 2024-09-10T07:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.416+0000 7feb95a00640 1 -- 172.21.15.46:0/3291597546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90103d80 msgr2=0x7feb90104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.416+0000 7feb95a00640 1 --2- 172.21.15.46:0/3291597546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90103d80 0x7feb90104180 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009900 tx=0x7feb7c02a410 comp rx=0 tx=0).stop 2024-09-10T07:52:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.420+0000 7feb95a00640 1 -- 172.21.15.46:0/3291597546 shutdown_connections 2024-09-10T07:52:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.420+0000 7feb95a00640 1 --2- 172.21.15.46:0/3291597546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90105930 0x7feb9010c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.420+0000 7feb95a00640 1 --2- 172.21.15.46:0/3291597546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90104f70 0x7feb901053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.420+0000 7feb95a00640 1 --2- 172.21.15.46:0/3291597546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90103d80 0x7feb90104180 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.420+0000 7feb95a00640 1 -- 172.21.15.46:0/3291597546 >> 172.21.15.46:0/3291597546 conn(0x7feb900ff4e0 msgr2=0x7feb90101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.420+0000 7feb95a00640 1 -- 172.21.15.46:0/3291597546 shutdown_connections 2024-09-10T07:52:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.420+0000 7feb95a00640 1 -- 172.21.15.46:0/3291597546 wait complete. 2024-09-10T07:52:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.421+0000 7feb95a00640 1 Processor -- start 2024-09-10T07:52:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.422+0000 7feb95a00640 1 -- start start 2024-09-10T07:52:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.422+0000 7feb95a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90103d80 0x7feb9019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.422+0000 7feb95a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90104f70 0x7feb9019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.422+0000 7feb95a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90105930 0x7feb901a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.422+0000 7feb8e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90104f70 0x7feb9019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.422+0000 7feb8e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90104f70 0x7feb9019ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35962/0 (socket says 172.21.15.46:35962) 2024-09-10T07:52:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.422+0000 7feb8e000640 1 -- 172.21.15.46:0/1510188236 learned_addr learned my addr 172.21.15.46:0/1510188236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.422+0000 7feb95a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb900769c0 con 0x7feb90103d80 2024-09-10T07:52:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb90076840 con 0x7feb90105930 2024-09-10T07:52:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb8f400640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90105930 0x7feb901a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb90076b40 con 0x7feb90104f70 2024-09-10T07:52:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb8ea00640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90103d80 0x7feb9019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb8f400640 1 -- 172.21.15.46:0/1510188236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90104f70 msgr2=0x7feb9019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb8f400640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90104f70 0x7feb9019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb8f400640 1 -- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90103d80 msgr2=0x7feb9019e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb8f400640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90103d80 0x7feb9019e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.423+0000 7feb8f400640 1 -- 172.21.15.46:0/1510188236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb901a60e0 con 0x7feb90105930 2024-09-10T07:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.424+0000 7feb8f400640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90105930 0x7feb901a5a70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7feb8800e940 tx=0x7feb8800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.424+0000 7feb87400640 1 -- 172.21.15.46:0/1510188236 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004d20 con 0x7feb90105930 2024-09-10T07:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.424+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb901a63d0 con 0x7feb90105930 2024-09-10T07:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.424+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901a6920 con 0x7feb90105930 2024-09-10T07:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.425+0000 7feb87400640 1 -- 172.21.15.46:0/1510188236 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb88005020 con 0x7feb90105930 2024-09-10T07:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.425+0000 7feb87400640 1 -- 172.21.15.46:0/1510188236 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880116b0 con 0x7feb90105930 2024-09-10T07:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.429+0000 7feb98886740 1 Processor -- start 2024-09-10T07:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.429+0000 7feb87400640 1 -- 172.21.15.46:0/1510188236 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb8800cb20 con 0x7feb90105930 2024-09-10T07:52:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.432+0000 7feb98886740 1 -- start start 2024-09-10T07:52:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.432+0000 7feb98886740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 0x561e6a320490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.432+0000 7feb98886740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a325490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.432+0000 7feb98886740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a327f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.432+0000 7feb98886740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561e6a285100 con 0x561e6a320090 2024-09-10T07:52:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.432+0000 7feb98886740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561e6a22d1b0 con 0x561e6a325a60 2024-09-10T07:52:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.432+0000 7feb98886740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561e6a2810c0 con 0x561e6a3209d0 2024-09-10T07:52:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.432+0000 7feb87400640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb740a2d30 0x7feb740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.433+0000 7feb87400640 1 -- 172.21.15.46:0/1510188236 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feb880cf420 con 0x7feb90105930 2024-09-10T07:52:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.433+0000 7feb8ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a325490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.433+0000 7feb8ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a325490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35968/0 (socket says 172.21.15.46:35968) 2024-09-10T07:52:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.433+0000 7feb8ea00640 1 -- 172.21.15.46:0/2606995292 learned_addr learned my addr 172.21.15.46:0/2606995292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.433+0000 7feb8f400640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb740a2d30 0x7feb740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.434+0000 7feb8e000640 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a327f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.434+0000 7feb8ea00640 1 -- 172.21.15.46:0/2606995292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 msgr2=0x561e6a327f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.434+0000 7feb8ea00640 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a327f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.434+0000 7feb8ea00640 1 -- 172.21.15.46:0/2606995292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 msgr2=0x561e6a320490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.434+0000 7feb8ea00640 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 0x561e6a320490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.434+0000 7feb8ea00640 1 -- 172.21.15.46:0/2606995292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e69f98210 con 0x561e6a3209d0 2024-09-10T07:52:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.434+0000 7feb8e000640 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a327f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.435+0000 7feb8ea00640 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a325490 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7feb90104dd0 tx=0x7feb7c004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99485fb257e669df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.435+0000 7feb8f400640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb740a2d30 0x7feb740a51f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feb88097d00 tx=0x7feb88004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.435+0000 7feb7be00640 1 -- 172.21.15.46:0/2606995292 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c041070 con 0x561e6a3209d0 2024-09-10T07:52:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.436+0000 7feb7be00640 1 -- 172.21.15.46:0/2606995292 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7c0054e0 con 0x561e6a3209d0 2024-09-10T07:52:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.436+0000 7feb7be00640 1 -- 172.21.15.46:0/2606995292 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c046ac0 con 0x561e6a3209d0 2024-09-10T07:52:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.437+0000 7feb98886740 1 -- 172.21.15.46:0/2606995292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 msgr2=0x561e6a325490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.437+0000 7feb98886740 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a325490 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7feb90104dd0 tx=0x7feb7c004de0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.437+0000 7feb98886740 1 -- 172.21.15.46:0/2606995292 shutdown_connections 2024-09-10T07:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.437+0000 7feb98886740 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a327f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.437+0000 7feb98886740 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a325490 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.437+0000 7feb98886740 1 --2- 172.21.15.46:0/2606995292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 0x561e6a320490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.437+0000 7feb98886740 1 -- 172.21.15.46:0/2606995292 >> 172.21.15.46:0/2606995292 conn(0x561e6a31b700 msgr2=0x561e6a31bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.438+0000 7feb98886740 1 -- 172.21.15.46:0/2606995292 shutdown_connections 2024-09-10T07:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.438+0000 7feb98886740 1 -- 172.21.15.46:0/2606995292 wait complete. 2024-09-10T07:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.438+0000 7feb98886740 1 -- start start 2024-09-10T07:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.439+0000 7feb98886740 1 Processor -- start 2024-09-10T07:52:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.440+0000 7feb98886740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a32a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.440+0000 7feb98886740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.440+0000 7feb98886740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a325a60 0x561e6a331820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.440+0000 7feb98886740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561e6a3289d0 con 0x561e6a325a60 2024-09-10T07:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.440+0000 7feb98886740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561e6a20e090 con 0x561e6a320090 2024-09-10T07:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.440+0000 7feb98886740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561e6a328b50 con 0x561e6a3209d0 2024-09-10T07:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.440+0000 7feb98886740 10 client.0 _ll_register_callbacks cb 0x561e6a262f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.441+0000 7feb8e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a32a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.441+0000 7feb8e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a32a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53282/0 (socket says 172.21.15.46:53282) 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.441+0000 7feb8e000640 1 -- 172.21.15.46:0/2872686841 learned_addr learned my addr 172.21.15.46:0/2872686841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.441+0000 7feb8ea00640 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.442+0000 7feb8ea00640 1 -- 172.21.15.46:0/2872686841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 msgr2=0x561e6a32a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.442+0000 7feb8ea00640 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a32a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.442+0000 7feb8ea00640 1 -- 172.21.15.46:0/2872686841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a325a60 msgr2=0x561e6a331820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.442+0000 7feb8ea00640 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a325a60 0x561e6a331820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.442+0000 7feb8ea00640 1 -- 172.21.15.46:0/2872686841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6a064620 con 0x561e6a3209d0 2024-09-10T07:52:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.442+0000 7feb8ea00640 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32a7e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02aa00 tx=0x7feb7c005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69dfcc80205851d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.442+0000 7feb78c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.443+0000 7feb78c00640 1 -- 172.21.15.46:0/2872686841 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c041070 con 0x561e6a3209d0 2024-09-10T07:52:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.443+0000 7feb78c00640 1 -- 172.21.15.46:0/2872686841 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feb7c0036b0 con 0x561e6a3209d0 2024-09-10T07:52:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.443+0000 7feb98886740 1 -- 172.21.15.46:0/2872686841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7c0054e0 con 0x561e6a3209d0 2024-09-10T07:52:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.443+0000 7feb78c00640 1 -- 172.21.15.46:0/2872686841 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7feb7c046a50 con 0x561e6a3209d0 2024-09-10T07:52:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.443+0000 7feb98886740 10 client.29245 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.443+0000 7feb98886740 1 -- 172.21.15.46:0/2872686841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561e6a3289d0 con 0x561e6a3209d0 2024-09-10T07:52:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.444+0000 7feb78c00640 1 -- 172.21.15.46:0/2872686841 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7c0054e0 con 0x561e6a3209d0 2024-09-10T07:52:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.445+0000 7feb78c00640 1 -- 172.21.15.46:0/2872686841 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7feb7c046c80 con 0x561e6a3209d0 2024-09-10T07:52:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.445+0000 7feb98886740 10 client.29245 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.445+0000 7feb98886740 10 client.29245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.445+0000 7feb98886740 20 client.29245 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.445+0000 7feb98886740 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561e6a3357f0 0x561e6a337be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.445+0000 7feb98886740 4 client.29245 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.445+0000 7feb98886740 1 -- 172.21.15.46:0/2872686841 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e6a292830 con 0x561e6a3357f0 2024-09-10T07:52:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.446+0000 7feb8e000640 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561e6a3357f0 0x561e6a337be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.447+0000 7feb8e000640 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561e6a3357f0 0x561e6a337be0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969cb86bbbaef941 server_cookie=d144ffa4ccc6bbd8 in_seq=0 out_seq=0 2024-09-10T07:52:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.447+0000 7feb78c00640 10 client.29245 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.453+0000 7feb78c00640 1 -- 172.21.15.46:0/2872686841 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e6a292830 con 0x561e6a3357f0 2024-09-10T07:52:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.453+0000 7feb78c00640 10 client.29245 handle_command_reply: tid=0 2024-09-10T07:52:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.453+0000 7feb98886740 1 client.29245 shutdown 2024-09-10T07:52:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 -- 172.21.15.46:0/2872686841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 msgr2=0x561e6a32a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32a7e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02aa00 tx=0x7feb7c005230 comp rx=0 tx=0).stop 2024-09-10T07:52:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 -- 172.21.15.46:0/2872686841 shutdown_connections 2024-09-10T07:52:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561e6a3357f0 0x561e6a337be0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a325a60 0x561e6a331820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb78c00640 0 client.29245 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32a7e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 --2- 172.21.15.46:0/2872686841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a32a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 -- 172.21.15.46:0/2872686841 >> 172.21.15.46:0/2872686841 conn(0x561e6a31b700 msgr2=0x561e6a31cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.455+0000 7feb98886740 1 -- 172.21.15.46:0/2872686841 shutdown_connections 2024-09-10T07:52:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.456+0000 7feb98886740 1 -- 172.21.15.46:0/2872686841 wait complete. 2024-09-10T07:52:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.456+0000 7feb98886740 20 client.29245 trim_cache size 0 max 16384 2024-09-10T07:52:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.509+0000 7feb98886740 1 Processor -- start 2024-09-10T07:52:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.509+0000 7feb98886740 1 -- start start 2024-09-10T07:52:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.509+0000 7feb98886740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a31cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.509+0000 7feb98886740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a3209d0 0x561e6a3317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb98886740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a325a60 0x561e6a33c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb98886740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb7000fb10 con 0x561e6a3209d0 2024-09-10T07:52:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb98886740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb700117d0 con 0x561e6a320090 2024-09-10T07:52:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb98886740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb700101e0 con 0x561e6a325a60 2024-09-10T07:52:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a31cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a31cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53284/0 (socket says 172.21.15.46:53284) 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a3209d0 0x561e6a3317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 -- 172.21.15.46:0/4277730136 learned_addr learned my addr 172.21.15.46:0/4277730136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 --2- 172.21.15.46:0/4277730136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a325a60 0x561e6a33c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 -- 172.21.15.46:0/4277730136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a325a60 msgr2=0x561e6a33c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 --2- 172.21.15.46:0/4277730136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a325a60 0x561e6a33c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 -- 172.21.15.46:0/4277730136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a3209d0 msgr2=0x561e6a3317e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 --2- 172.21.15.46:0/4277730136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a3209d0 0x561e6a3317e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.510+0000 7feb8e000640 1 -- 172.21.15.46:0/4277730136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6a32a070 con 0x561e6a320090 2024-09-10T07:52:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.512+0000 7feb8e000640 1 --2- 172.21.15.46:0/4277730136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a31cda0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x561e6a338410 tx=0x7feb800047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25674b4aab638cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.513+0000 7feb7b400640 1 -- 172.21.15.46:0/4277730136 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80014980 con 0x561e6a320090 2024-09-10T07:52:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.513+0000 7feb7b400640 1 -- 172.21.15.46:0/4277730136 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb8001c1f0 con 0x561e6a320090 2024-09-10T07:52:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.513+0000 7feb7b400640 1 -- 172.21.15.46:0/4277730136 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80013600 con 0x561e6a320090 2024-09-10T07:52:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.514+0000 7feb98886740 1 -- 172.21.15.46:0/4277730136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 msgr2=0x561e6a31cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.514+0000 7feb98886740 1 --2- 172.21.15.46:0/4277730136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a31cda0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x561e6a338410 tx=0x7feb800047d0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.514+0000 7feb98886740 1 -- 172.21.15.46:0/4277730136 shutdown_connections 2024-09-10T07:52:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.514+0000 7feb98886740 1 --2- 172.21.15.46:0/4277730136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a325a60 0x561e6a33c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.515+0000 7feb98886740 1 --2- 172.21.15.46:0/4277730136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a3209d0 0x561e6a3317e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.515+0000 7feb98886740 1 --2- 172.21.15.46:0/4277730136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a320090 0x561e6a31cda0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.515+0000 7feb98886740 1 -- 172.21.15.46:0/4277730136 >> 172.21.15.46:0/4277730136 conn(0x561e6a31b700 msgr2=0x561e6a31c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.517+0000 7feb98886740 1 -- 172.21.15.46:0/4277730136 shutdown_connections 2024-09-10T07:52:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.519+0000 7feb98886740 1 -- 172.21.15.46:0/4277730136 wait complete. 2024-09-10T07:52:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.519+0000 7feb98886740 1 -- start start 2024-09-10T07:52:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.520+0000 7feb98886740 1 Processor -- start 2024-09-10T07:52:42.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:42 smithi046 ceph-mon[30705]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-09-10T07:52:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.521+0000 7feb98886740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 0x561e6a32ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb98886740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb98886740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a3303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb98886740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb70011040 con 0x561e6a320090 2024-09-10T07:52:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb98886740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb70010970 con 0x561e6a325a60 2024-09-10T07:52:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb98886740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb7000dad0 con 0x561e6a3209d0 2024-09-10T07:52:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb8ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb8ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:35998/0 (socket says 172.21.15.46:35998) 2024-09-10T07:52:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb8ea00640 1 -- 172.21.15.46:0/3550353160 learned_addr learned my addr 172.21.15.46:0/3550353160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb8e000640 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a3303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.522+0000 7feb98886740 10 client.0 _ll_register_callbacks cb 0x7feb7c004d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.526+0000 7feb8ea00640 1 -- 172.21.15.46:0/3550353160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 msgr2=0x561e6a3303d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.526+0000 7feb8ea00640 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a3303d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.526+0000 7feb8ea00640 1 -- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 msgr2=0x561e6a32ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.526+0000 7feb8e000640 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 0x561e6a32ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.526+0000 7feb8ea00640 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 0x561e6a32ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.526+0000 7feb8ea00640 1 -- 172.21.15.46:0/3550353160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6a3309e0 con 0x561e6a3209d0 2024-09-10T07:52:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.526+0000 7feb8e000640 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 0x561e6a32ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.526+0000 7feb78c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.527+0000 7feb8ea00640 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32f3c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feb7c055040 tx=0x7feb7c046ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a0466d3a623137 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.527+0000 7feb78c00640 1 -- 172.21.15.46:0/3550353160 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c061060 con 0x561e6a3209d0 2024-09-10T07:52:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.527+0000 7feb98886740 1 -- 172.21.15.46:0/3550353160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8001c1f0 con 0x561e6a3209d0 2024-09-10T07:52:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.527+0000 7feb78c00640 1 -- 172.21.15.46:0/3550353160 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7feb7c04c070 con 0x561e6a3209d0 2024-09-10T07:52:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.527+0000 7feb78c00640 1 -- 172.21.15.46:0/3550353160 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7feb7c061060 con 0x561e6a3209d0 2024-09-10T07:52:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.527+0000 7feb98886740 10 client.29251 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.527+0000 7feb98886740 1 -- 172.21.15.46:0/3550353160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x561e6a3a0470 con 0x561e6a3209d0 2024-09-10T07:52:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.529+0000 7feb78c00640 1 -- 172.21.15.46:0/3550353160 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb8001c1f0 con 0x561e6a3209d0 2024-09-10T07:52:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.529+0000 7feb78c00640 1 -- 172.21.15.46:0/3550353160 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7feb7c031560 con 0x561e6a3209d0 2024-09-10T07:52:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.529+0000 7feb98886740 10 client.29251 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.529+0000 7feb98886740 10 client.29251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.529+0000 7feb98886740 20 client.29251 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.529+0000 7feb98886740 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561e6a3a0d00 0x561e6a3a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.529+0000 7feb98886740 4 client.29251 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:52:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.529+0000 7feb98886740 1 -- 172.21.15.46:0/3550353160 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e6a28cb80 con 0x561e6a3a0d00 2024-09-10T07:52:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.530+0000 7feb8e000640 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561e6a3a0d00 0x561e6a3a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.530+0000 7feb8e000640 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561e6a3a0d00 0x561e6a3a3120 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc6b734c133e326 server_cookie=5fe8fa14f152720c in_seq=0 out_seq=0 2024-09-10T07:52:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:42.531+0000 7feb78c00640 10 client.29251 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:42.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:42 smithi181 ceph-mon[35397]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-09-10T07:52:42.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:42 smithi063 ceph-mon[35020]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-09-10T07:52:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb78c00640 1 -- 172.21.15.46:0/3550353160 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e6a28cb80 con 0x561e6a3a0d00 2024-09-10T07:52:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb78c00640 10 client.29251 handle_command_reply: tid=0 2024-09-10T07:52:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 client.29251 shutdown 2024-09-10T07:52:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 -- 172.21.15.46:0/3550353160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 msgr2=0x561e6a32f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32f3c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feb7c055040 tx=0x7feb7c046ef0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 -- 172.21.15.46:0/3550353160 shutdown_connections 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x561e6a3a0d00 0x561e6a3a3120 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e6a325a60 0x561e6a3303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561e6a3209d0 0x561e6a32f3c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 --2- 172.21.15.46:0/3550353160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e6a320090 0x561e6a32ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb78c00640 0 client.29251 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.285+0000 7feb98886740 1 -- 172.21.15.46:0/3550353160 >> 172.21.15.46:0/3550353160 conn(0x561e6a31b700 msgr2=0x561e6a31ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.286+0000 7feb98886740 1 -- 172.21.15.46:0/3550353160 shutdown_connections 2024-09-10T07:52:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.286+0000 7feb98886740 1 -- 172.21.15.46:0/3550353160 wait complete. 2024-09-10T07:52:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.286+0000 7feb98886740 20 client.29251 trim_cache size 0 max 16384 2024-09-10T07:52:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-10T07:52:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-10T07:52:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-10T07:52:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-10T07:52:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb740a2d30 msgr2=0x7feb740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb740a2d30 0x7feb740a51f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feb88097d00 tx=0x7feb88004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90105930 msgr2=0x7feb901a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90105930 0x7feb901a5a70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7feb8800e940 tx=0x7feb8800ee00 comp rx=0 tx=0).stop 2024-09-10T07:52:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 shutdown_connections 2024-09-10T07:52:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb90105930 0x7feb901a5a70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb90104f70 0x7feb9019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7feb740a2d30 0x7feb740a51f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 --2- 172.21.15.46:0/1510188236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb90103d80 0x7feb9019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 >> 172.21.15.46:0/1510188236 conn(0x7feb900ff4e0 msgr2=0x7feb90079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 shutdown_connections 2024-09-10T07:52:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-10T07:52:43.287+0000 7feb95a00640 1 -- 172.21.15.46:0/1510188236 wait complete. 2024-09-10T07:52:43.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-10T07:52:43.610 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/: 2024-09-10T07:52:43.610 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.610 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 . 2024-09-10T07:52:43.610 INFO:teuthology.orchestra.run.smithi046.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 .. 2024-09-10T07:52:43.610 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 data 2024-09-10T07:52:43.610 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.610 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data: 2024-09-10T07:52:43.610 INFO:teuthology.orchestra.run.smithi046.stdout:total 36 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 . 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:36 .. 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 07:37 base 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 10 07:37 current_logfiles 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 10 07:37 global 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 log 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_commit_ts 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_dynshmem 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 10 07:37 pg_hba.conf 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 10 07:37 pg_ident.conf 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:52 pg_logical 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 pg_multixact 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_notify 2024-09-10T07:52:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_replslot 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_serial 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_snapshots 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:52 pg_stat 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:52 pg_stat_tmp 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:49 pg_subtrans 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_tblspc 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pg_twophase 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 PG_VERSION 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Sep 10 07:52 pg_wal 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 pg_xact 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 10 07:37 postgresql.auto.conf 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 10 07:37 postgresql.conf 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 10 07:37 postmaster.opts 2024-09-10T07:52:43.612 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base: 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 07:37 . 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 1 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 13493 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 10 07:44 13494 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 pgsql_tmp 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:total 7777 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 . 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 07:37 .. 2024-09-10T07:52:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 112 2024-09-10T07:52:43.614 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 113 2024-09-10T07:52:43.614 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 1247 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1247_fsm 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1247_vm 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 07:37 1249 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1249_fsm 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1249_vm 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 07:37 1255 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1255_fsm 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1255_vm 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 07:37 1259 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1259_fsm 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1259_vm 2024-09-10T07:52:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 07:37 13344 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13344_fsm 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13344_vm 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13346 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13348 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13349_fsm 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349_vm 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13351 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13353 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13354_fsm 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354_vm 2024-09-10T07:52:43.616 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13356 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13358 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13359_fsm 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359_vm 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13361 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13363 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1417 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1418 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 174 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 175 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2187 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2224 2024-09-10T07:52:43.617 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2328 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2336 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2337 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2579 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2600 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2600_fsm 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2600_vm 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2601_fsm 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601_vm 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2602 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2602_fsm 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2602_vm 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2603 2024-09-10T07:52:43.618 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2603_fsm 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2603_vm 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2604 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2605 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2605_fsm 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2605_vm 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2606_fsm 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606_vm 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2607 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2607_fsm 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2607_vm 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 07:37 2608 2024-09-10T07:52:43.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2608_fsm 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2608_vm 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 07:37 2609 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2609_fsm 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2609_vm 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2610 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2610_fsm 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2610_vm 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2611 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2612_fsm 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612_vm 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2613 2024-09-10T07:52:43.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2615_fsm 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615_vm 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2616 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2616_fsm 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2616_vm 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 07:37 2617 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2617_fsm 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2617_vm 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 07:37 2618 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2618_fsm 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2618_vm 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 07:37 2619 2024-09-10T07:52:43.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2619_fsm 2024-09-10T07:52:43.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2619_vm 2024-09-10T07:52:43.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2620 2024-09-10T07:52:43.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2650 2024-09-10T07:52:43.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2651 2024-09-10T07:52:43.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2652 2024-09-10T07:52:43.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2653 2024-09-10T07:52:43.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2654 2024-09-10T07:52:43.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2655 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2656 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2657 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 07:37 2658 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 2659 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2660 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2661 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2662 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2663 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2664 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2665 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2666 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2667 2024-09-10T07:52:43.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2668 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2669 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2670 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 07:37 2673 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 07:37 2674 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 07:37 2675 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2678 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2679 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2680 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2681 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2682 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2683 2024-09-10T07:52:43.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2684 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2685 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2686 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2687 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2688 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2689 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 07:37 2690 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 07:37 2691 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2692 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2693 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2696 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2699 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2701 2024-09-10T07:52:43.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2702 2024-09-10T07:52:43.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2703 2024-09-10T07:52:43.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2704 2024-09-10T07:52:43.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2753 2024-09-10T07:52:43.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2753_fsm 2024-09-10T07:52:43.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2753_vm 2024-09-10T07:52:43.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2754 2024-09-10T07:52:43.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2755 2024-09-10T07:52:43.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2756 2024-09-10T07:52:43.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2757 2024-09-10T07:52:43.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2830 2024-09-10T07:52:43.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2831 2024-09-10T07:52:43.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2832 2024-09-10T07:52:43.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2833 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2834 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2835 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2836 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2837 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 07:37 2838 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2838_fsm 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2838_vm 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2839 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840_fsm 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2840_vm 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2841 2024-09-10T07:52:43.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2995 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2996 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3079_fsm 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079_vm 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3080 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3081 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3085 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3118 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3119 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 07:37 3164 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3256 2024-09-10T07:52:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3257 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3258 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3350 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3351 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3379 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3380 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3381 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3394 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3394_fsm 2024-09-10T07:52:43.632 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3394_vm 2024-09-10T07:52:43.633 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3395 2024-09-10T07:52:43.633 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3429 2024-09-10T07:52:43.633 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3430 2024-09-10T07:52:43.633 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3431 2024-09-10T07:52:43.633 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3433 2024-09-10T07:52:43.633 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3439 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3440 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3455 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 07:37 3456 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3456_fsm 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3456_vm 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3466 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3467 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3468 2024-09-10T07:52:43.634 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3501 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3502 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3503 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3534 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3541_fsm 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541_vm 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3542 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3574 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3575 2024-09-10T07:52:43.635 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3576 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3596 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3597 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3598 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3599 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3600_fsm 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600_vm 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601 2024-09-10T07:52:43.636 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3601_fsm 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601_vm 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3602_fsm 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602_vm 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603_fsm 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3603_vm 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3604 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3605 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3606 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3607 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3608 2024-09-10T07:52:43.637 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 3609 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3712 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3764_fsm 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764_vm 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3766 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3767 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3997 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4143 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4144 2024-09-10T07:52:43.638 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4145 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4146 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4147 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4148 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4149 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4150 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4151 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4152 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4153 2024-09-10T07:52:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4154 2024-09-10T07:52:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4155 2024-09-10T07:52:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4156 2024-09-10T07:52:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4157 2024-09-10T07:52:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4158 2024-09-10T07:52:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4159 2024-09-10T07:52:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4160 2024-09-10T07:52:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4161 2024-09-10T07:52:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4162 2024-09-10T07:52:43.641 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4163 2024-09-10T07:52:43.641 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4164 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4165 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4166 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4167 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4168 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4169 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4170 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4171 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4172 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4173 2024-09-10T07:52:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4174 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 5002 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 548 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 549 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6102 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6104 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6106 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6110 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6111 2024-09-10T07:52:43.643 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6112 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6113 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6117 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 826 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 827 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 828 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 07:37 pg_filenode.map 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 PG_VERSION 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-10T07:52:43.644 INFO:teuthology.orchestra.run.smithi046.stdout:total 7777 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 07:37 . 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 07:37 .. 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 112 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 113 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 1247 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1247_fsm 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1247_vm 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 07:37 1249 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1249_fsm 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1249_vm 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 07:37 1255 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1255_fsm 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1255_vm 2024-09-10T07:52:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 07:37 1259 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1259_fsm 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1259_vm 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 07:37 13344 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13344_fsm 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13344_vm 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13346 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13348 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13349_fsm 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349_vm 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13351 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13353 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354 2024-09-10T07:52:43.646 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13354_fsm 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354_vm 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13356 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13358 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13359_fsm 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359_vm 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13361 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13363 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1417 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1418 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 174 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 175 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2187 2024-09-10T07:52:43.647 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2224 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2328 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2336 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2337 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2579 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2600 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2600_fsm 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2600_vm 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2601_fsm 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601_vm 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2602 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2602_fsm 2024-09-10T07:52:43.648 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2602_vm 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2603 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2603_fsm 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2603_vm 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2604 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2605 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2605_fsm 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2605_vm 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2606_fsm 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606_vm 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2607 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2607_fsm 2024-09-10T07:52:43.649 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2607_vm 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 07:37 2608 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2608_fsm 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2608_vm 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 07:37 2609 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2609_fsm 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2609_vm 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2610 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2610_fsm 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2610_vm 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2611 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2612_fsm 2024-09-10T07:52:43.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612_vm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2613 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2615_fsm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615_vm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2616 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2616_fsm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2616_vm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 07:37 2617 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2617_fsm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2617_vm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 07:37 2618 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2618_fsm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2618_vm 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 07:37 2619 2024-09-10T07:52:43.651 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2619_fsm 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2619_vm 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2620 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2650 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2651 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2652 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2653 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2654 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2655 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2656 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2657 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 07:37 2658 2024-09-10T07:52:43.652 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 2659 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2660 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2661 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2662 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2663 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2664 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2665 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2666 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2667 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2668 2024-09-10T07:52:43.653 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2669 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2670 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 07:37 2673 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 07:37 2674 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 07:37 2675 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2678 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2679 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2680 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2681 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2682 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2683 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2684 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2685 2024-09-10T07:52:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2686 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2687 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2688 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2689 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 07:37 2690 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 07:37 2691 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2692 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2693 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2696 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2699 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2701 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2702 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2703 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2704 2024-09-10T07:52:43.655 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2753 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2753_fsm 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2753_vm 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2754 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2755 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2756 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2757 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2830 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2831 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2832 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2833 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2834 2024-09-10T07:52:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2835 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2836 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2837 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 07:37 2838 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2838_fsm 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2838_vm 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2839 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840_fsm 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2840_vm 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2841 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2995 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2996 2024-09-10T07:52:43.657 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3079_fsm 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079_vm 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3080 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3081 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3085 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3118 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3119 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 07:37 3164 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3256 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3257 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3258 2024-09-10T07:52:43.658 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3350 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3351 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3379 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3380 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3381 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3394 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3394_fsm 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3394_vm 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3395 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3429 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3430 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3431 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3433 2024-09-10T07:52:43.659 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3439 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3440 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3455 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 07:37 3456 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3456_fsm 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3456_vm 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3466 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3467 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3468 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3501 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3502 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3503 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3534 2024-09-10T07:52:43.660 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3541_fsm 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541_vm 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3542 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3574 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3575 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3576 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3596 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3597 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3598 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3599 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3600_fsm 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600_vm 2024-09-10T07:52:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3601_fsm 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601_vm 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3602_fsm 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602_vm 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603_fsm 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3603_vm 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3604 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3605 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3606 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3607 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3608 2024-09-10T07:52:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 3609 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3712 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3764_fsm 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764_vm 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3766 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3767 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3997 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4143 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4144 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4145 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4146 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4147 2024-09-10T07:52:43.663 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4148 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4149 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4150 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4151 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4152 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4153 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4154 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4155 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4156 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4157 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4158 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4159 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4160 2024-09-10T07:52:43.664 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4161 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4162 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4163 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4164 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4165 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4166 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4167 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4168 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4169 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4170 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4171 2024-09-10T07:52:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4172 2024-09-10T07:52:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4173 2024-09-10T07:52:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4174 2024-09-10T07:52:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 5002 2024-09-10T07:52:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 548 2024-09-10T07:52:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 549 2024-09-10T07:52:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6102 2024-09-10T07:52:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6104 2024-09-10T07:52:43.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6106 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6110 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6111 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6112 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6113 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6117 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 826 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 827 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 828 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 07:37 pg_filenode.map 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 PG_VERSION 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.734 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:total 541238 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 10 07:44 . 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 07:37 .. 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 112 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 113 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 1247 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1247_fsm 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1247_vm 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 07:37 1249 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1249_fsm 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1249_vm 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 07:37 1255 2024-09-10T07:52:43.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1255_fsm 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1255_vm 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 07:52 1259 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1259_fsm 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1259_vm 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 07:37 13344 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13344_fsm 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13344_vm 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13346 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13348 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13349_fsm 2024-09-10T07:52:43.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13349_vm 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13351 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13353 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13354_fsm 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13354_vm 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13356 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13358 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 13359_fsm 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13359_vm 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 13361 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 13363 2024-09-10T07:52:43.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1417 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 1418 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 439549952 Sep 10 07:52 16396 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 10 07:51 16396_fsm 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:52 16396_vm 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 458752 Sep 10 07:52 16397 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:52 16397_fsm 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:52 16397_vm 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 598016 Sep 10 07:52 16399 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:52 16399_fsm 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:49 16399_vm 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:52 16400 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:52 16402 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 10 07:52 16404 2024-09-10T07:52:43.738 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 33251328 Sep 10 07:52 16406 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:52 16406_fsm 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:51 16406_vm 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 174 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 175 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2187 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2224 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2328 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2336 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2337 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2579 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2600 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2600_fsm 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2600_vm 2024-09-10T07:52:43.739 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2601_fsm 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2601_vm 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2602 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2602_fsm 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2602_vm 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2603 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2603_fsm 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2603_vm 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2604 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2605 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2605_fsm 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2605_vm 2024-09-10T07:52:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2606_fsm 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2606_vm 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2607 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2607_fsm 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2607_vm 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 07:37 2608 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2608_fsm 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2608_vm 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 07:37 2609 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2609_fsm 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2609_vm 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2610 2024-09-10T07:52:43.741 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2610_fsm 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2610_vm 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2611 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2612_fsm 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2612_vm 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2613 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2615_fsm 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2615_vm 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2616 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2616_fsm 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2616_vm 2024-09-10T07:52:43.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 07:37 2617 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2617_fsm 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2617_vm 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 07:37 2618 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2618_fsm 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2618_vm 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 10 07:52 2619 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:52 2619_fsm 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:48 2619_vm 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2620 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2650 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2651 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2652 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2653 2024-09-10T07:52:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2654 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2655 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2656 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2657 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 07:37 2658 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 07:37 2659 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2660 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2661 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2662 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2663 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2664 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2665 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2666 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2667 2024-09-10T07:52:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2668 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2669 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2670 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 07:37 2673 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 07:37 2674 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 07:37 2675 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2678 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2679 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2680 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2681 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2682 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2683 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2684 2024-09-10T07:52:43.745 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2685 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2686 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2687 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2688 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 07:37 2689 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 07:37 2690 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 07:37 2691 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2692 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2693 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:52 2696 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2699 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2701 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2702 2024-09-10T07:52:43.746 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2703 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 2704 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2753 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2753_fsm 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2753_vm 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2754 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2755 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2756 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 2757 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2830 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2831 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2832 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2833 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2834 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2835 2024-09-10T07:52:43.747 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2836 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2837 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 07:37 2838 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2838_fsm 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2838_vm 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2839 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2840_fsm 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2840_vm 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2841 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2995 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2996 2024-09-10T07:52:43.748 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3079_fsm 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3079_vm 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3080 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3081 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3085 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3118 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3119 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 07:37 3164 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3256 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3257 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3258 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3350 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3351 2024-09-10T07:52:43.749 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3379 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3380 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3381 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3394 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3394_fsm 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3394_vm 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3395 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3429 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3430 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3431 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3433 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3439 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3440 2024-09-10T07:52:43.750 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 07:37 3455 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 07:37 3456 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3456_fsm 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3456_vm 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3466 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3467 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3468 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3501 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3502 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3503 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3534 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3541_fsm 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3541_vm 2024-09-10T07:52:43.751 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3542 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3574 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3575 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3576 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3596 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3597 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3598 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3599 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3600_fsm 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3600_vm 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3601_fsm 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3601_vm 2024-09-10T07:52:43.752 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3602_fsm 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3602_vm 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3603_fsm 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3603_vm 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3604 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3605 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3606 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3607 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3608 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 07:37 3609 2024-09-10T07:52:43.753 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3712 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 3764_fsm 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3764_vm 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3766 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 3767 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3997 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4143 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4144 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4145 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4146 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4147 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4148 2024-09-10T07:52:43.754 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4149 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4150 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4151 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4152 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4153 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4154 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4155 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4156 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4157 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4158 2024-09-10T07:52:43.755 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4159 2024-09-10T07:52:43.756 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4160 2024-09-10T07:52:43.756 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4161 2024-09-10T07:52:43.756 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4162 2024-09-10T07:52:43.756 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4163 2024-09-10T07:52:43.756 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4164 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4165 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4166 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4167 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4168 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4169 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4170 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4171 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4172 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4173 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4174 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 5002 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 548 2024-09-10T07:52:43.757 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 549 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6102 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6104 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6106 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6110 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6111 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6112 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6113 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6117 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 826 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 827 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 828 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 07:37 pg_filenode.map 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 10 07:37 pg_internal.init 2024-09-10T07:52:43.758 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:37 PG_VERSION 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 07:37 .. 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/global: 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:total 560 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 10 07:37 . 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1213 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1213_fsm 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1213_vm 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1214 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1214_fsm 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1214_vm 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 1232 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 1233 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1260 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1260_fsm 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1260_vm 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1261 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1261_fsm 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1261_vm 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1262 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 1262_fsm 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 1262_vm 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2396 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 07:37 2396_fsm 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2396_vm 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2397 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2671 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2672 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2676 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2677 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2694 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2695 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2697 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 07:37 2698 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2846 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2847 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2964 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2965 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 2966 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 2967 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 3592 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 3593 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4060 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4061 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4175 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4176 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4177 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4178 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4181 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4182 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4183 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4184 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 4185 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 4186 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6000 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6001 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6002 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 6100 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6114 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 6115 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:52 pg_control 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 07:37 pg_filenode.map 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 10 07:37 pg_internal.init 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/log: 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:total 2 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 . 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1511 Sep 10 07:52 postgresql-Tue.log 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:total 1 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:52 . 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 mappings 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 07:52 replorigin_checkpoint 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 snapshots 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:52 .. 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:52 .. 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 . 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 members 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 offsets 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:total 8 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 . 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 .. 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:37 0000 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:total 8 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 . 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 07:37 .. 2024-09-10T07:52:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 07:42 0000 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:total 15 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 07:52 . 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 10 07:52 db_0.stat 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 10 07:52 db_13494.stat 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 10 07:52 global.stat 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:52 . 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:total 256 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:49 . 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 10 07:52 0008 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:total 884736 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Sep 10 07:52 . 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:52 00000001000000000000005D 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 00000001000000000000005E 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 00000001000000000000005F 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:45 000000010000000000000060 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:43 000000010000000000000061 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 000000010000000000000062 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:46 000000010000000000000063 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 000000010000000000000064 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:44 000000010000000000000065 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:44 000000010000000000000066 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:43 000000010000000000000067 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:44 000000010000000000000068 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:43 000000010000000000000069 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:44 00000001000000000000006A 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:46 00000001000000000000006B 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 00000001000000000000006C 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 00000001000000000000006D 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 00000001000000000000006E 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:43 00000001000000000000006F 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 000000010000000000000070 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 000000010000000000000071 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:43 000000010000000000000072 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:44 000000010000000000000073 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 000000010000000000000074 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:46 000000010000000000000075 2024-09-10T07:52:43.773 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:44 000000010000000000000076 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:43 000000010000000000000077 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:42 000000010000000000000078 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 000000010000000000000079 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:50 00000001000000000000007A 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:47 00000001000000000000007B 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:49 00000001000000000000007C 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:51 00000001000000000000007D 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:51 00000001000000000000007E 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 00000001000000000000007F 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:49 000000010000000000000080 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:47 000000010000000000000081 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:47 000000010000000000000082 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:49 000000010000000000000083 2024-09-10T07:52:43.774 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:47 000000010000000000000084 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 000000010000000000000085 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 000000010000000000000086 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:50 000000010000000000000087 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 000000010000000000000088 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:51 000000010000000000000089 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:47 00000001000000000000008A 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 00000001000000000000008B 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 00000001000000000000008C 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:49 00000001000000000000008D 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 00000001000000000000008E 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:49 00000001000000000000008F 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:48 000000010000000000000090 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:47 000000010000000000000091 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 07:47 000000010000000000000092 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 archive_status 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:total 0 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 07:37 . 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Sep 10 07:52 .. 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:total 144 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 07:37 . 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 07:52 .. 2024-09-10T07:52:43.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 07:52 0000 2024-09-10T07:52:43.778 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T07:52:43.788 INFO:tasks.quiescer:joining Quiescers 2024-09-10T07:52:43.788 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T07:52:43.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:52:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:44.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:44 smithi046 ceph-mon[30705]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T07:52:44.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:44 smithi181 ceph-mon[35397]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T07:52:44.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:44 smithi063 ceph-mon[35020]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 -- 172.21.15.46:0/1492341384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318109140 msgr2=0x7fc318110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 --2- 172.21.15.46:0/1492341384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318109140 0x7fc318110cc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc310009ca0 tx=0x7fc31002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 -- 172.21.15.46:0/1492341384 shutdown_connections 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 --2- 172.21.15.46:0/1492341384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318109140 0x7fc318110cc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 --2- 172.21.15.46:0/1492341384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3181049c0 0x7fc318108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 --2- 172.21.15.46:0/1492341384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc318104010 0x7fc3181043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 -- 172.21.15.46:0/1492341384 >> 172.21.15.46:0/1492341384 conn(0x7fc3180ffa50 msgr2=0x7fc318101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 -- 172.21.15.46:0/1492341384 shutdown_connections 2024-09-10T07:52:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.765+0000 7fc320000640 1 -- 172.21.15.46:0/1492341384 wait complete. 2024-09-10T07:52:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.766+0000 7fc320000640 1 Processor -- start 2024-09-10T07:52:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.766+0000 7fc320000640 1 -- start start 2024-09-10T07:52:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc320000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc318104010 0x7fc31819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc320000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3181049c0 0x7fc31819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc320000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc318109140 0x7fc31819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc320000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181141a0 con 0x7fc3181049c0 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc320000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc318114020 con 0x7fc318104010 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc320000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc318114320 con 0x7fc318109140 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc31d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc318104010 0x7fc31819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc31cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3181049c0 0x7fc31819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc31d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc318104010 0x7fc31819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:53304/0 (socket says 172.21.15.46:53304) 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc31d600640 1 -- 172.21.15.46:0/1511457604 learned_addr learned my addr 172.21.15.46:0/1511457604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.767+0000 7fc31e000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc318109140 0x7fc31819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc31d600640 1 -- 172.21.15.46:0/1511457604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc318109140 msgr2=0x7fc31819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc31d600640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc318109140 0x7fc31819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc31d600640 1 -- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3181049c0 msgr2=0x7fc31819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc31d600640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3181049c0 0x7fc31819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc31d600640 1 -- 172.21.15.46:0/1511457604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31819f650 con 0x7fc318104010 2024-09-10T07:52:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc31e000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc318109140 0x7fc31819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc31d600640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc318104010 0x7fc31819a6a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00e9f0 tx=0x7fc30c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc316000640 1 -- 172.21.15.46:0/1511457604 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00cd80 con 0x7fc318104010 2024-09-10T07:52:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.768+0000 7fc316000640 1 -- 172.21.15.46:0/1511457604 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc30c010910 con 0x7fc318104010 2024-09-10T07:52:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.769+0000 7fc316000640 1 -- 172.21.15.46:0/1511457604 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c018990 con 0x7fc318104010 2024-09-10T07:52:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.769+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31819f940 con 0x7fc318104010 2024-09-10T07:52:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.769+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3181a7220 con 0x7fc318104010 2024-09-10T07:52:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.770+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3181043f0 con 0x7fc318104010 2024-09-10T07:52:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.772+0000 7fc316000640 1 -- 172.21.15.46:0/1511457604 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc30c010ac0 con 0x7fc318104010 2024-09-10T07:52:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.775+0000 7fc316000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc3000a2b80 0x7fc3000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.775+0000 7fc316000640 1 -- 172.21.15.46:0/1511457604 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc30c0cfc00 con 0x7fc318104010 2024-09-10T07:52:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.775+0000 7fc31cc00640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc3000a2b80 0x7fc3000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.776+0000 7fc31cc00640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc3000a2b80 0x7fc3000a5040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc31810aab0 tx=0x7fc304005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.776+0000 7fc316000640 1 -- 172.21.15.46:0/1511457604 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc30c09c780 con 0x7fc318104010 2024-09-10T07:52:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.998+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc31819ba00 con 0x7fc318104010 2024-09-10T07:52:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:44.999+0000 7fc316000640 1 -- 172.21.15.46:0/1511457604 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11986 (secure 0 0 0) 0x7fc30c09c080 con 0x7fc318104010 2024-09-10T07:52:45.000 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:45.000 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":60,"btime":"2024-09-10T07:51:54:063936+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:51:53.066973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:52:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 60 2024-09-10T07:52:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc3000a2b80 msgr2=0x7fc3000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc3000a2b80 0x7fc3000a5040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc31810aab0 tx=0x7fc304005f90 comp rx=0 tx=0).stop 2024-09-10T07:52:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc318104010 msgr2=0x7fc31819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc318104010 0x7fc31819a6a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00e9f0 tx=0x7fc30c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 shutdown_connections 2024-09-10T07:52:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc318109140 0x7fc31819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc3000a2b80 0x7fc3000a5040 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3181049c0 0x7fc31819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 --2- 172.21.15.46:0/1511457604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc318104010 0x7fc31819a6a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 >> 172.21.15.46:0/1511457604 conn(0x7fc3180ffa50 msgr2=0x7fc318100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 shutdown_connections 2024-09-10T07:52:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:45.002+0000 7fc320000640 1 -- 172.21.15.46:0/1511457604 wait complete. 2024-09-10T07:52:45.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:45 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1511457604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:45.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T07:52:45.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:45 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1511457604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:45.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:45 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1511457604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:45.996 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24535 quiesce db --cancel --all' 2024-09-10T07:52:45.996 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24535 quiesce db --cancel --all 2024-09-10T07:52:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:46.589 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:46 smithi046 ceph-mon[30705]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T07:52:46.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:46 smithi181 ceph-mon[35397]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T07:52:46.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:46 smithi063 ceph-mon[35020]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T07:52:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.709+0000 7f0483000640 1 -- 172.21.15.46:0/3133576798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c103ff0 msgr2=0x7f047c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.709+0000 7f0483000640 1 --2- 172.21.15.46:0/3133576798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c103ff0 0x7f047c1043d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f046c009900 tx=0x7f046c02a410 comp rx=0 tx=0).stop 2024-09-10T07:52:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.709+0000 7f0483000640 1 -- 172.21.15.46:0/3133576798 shutdown_connections 2024-09-10T07:52:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.709+0000 7f0483000640 1 --2- 172.21.15.46:0/3133576798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c109120 0x7f047c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.709+0000 7f0483000640 1 --2- 172.21.15.46:0/3133576798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c1049a0 0x7f047c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.709+0000 7f0483000640 1 --2- 172.21.15.46:0/3133576798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c103ff0 0x7f047c1043d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.709+0000 7f0483000640 1 -- 172.21.15.46:0/3133576798 >> 172.21.15.46:0/3133576798 conn(0x7f047c0ffa90 msgr2=0x7f047c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.710+0000 7f0483000640 1 -- 172.21.15.46:0/3133576798 shutdown_connections 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.710+0000 7f0483000640 1 -- 172.21.15.46:0/3133576798 wait complete. 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.710+0000 7f0483000640 1 Processor -- start 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0483000640 1 -- start start 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0483000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c103ff0 0x7f047c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0483000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1049a0 0x7f047c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0483000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c109120 0x7f047c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0483000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f047c114150 con 0x7f047c109120 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0483000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f047c113fd0 con 0x7f047c103ff0 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0483000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1142d0 con 0x7f047c1049a0 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0481000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c109120 0x7f047c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0481000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c109120 0x7f047c19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60904/0 (socket says 172.21.15.46:60904) 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0481000640 1 -- 172.21.15.46:0/114351927 learned_addr learned my addr 172.21.15.46:0/114351927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f047be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c103ff0 0x7f047c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.711+0000 7f0473e00640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1049a0 0x7f047c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0481000640 1 -- 172.21.15.46:0/114351927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1049a0 msgr2=0x7f047c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0481000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1049a0 0x7f047c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0481000640 1 -- 172.21.15.46:0/114351927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c103ff0 msgr2=0x7f047c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0481000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c103ff0 0x7f047c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0481000640 1 -- 172.21.15.46:0/114351927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c19f5a0 con 0x7f047c109120 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f047be00640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c103ff0 0x7f047c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0481000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c109120 0x7f047c19eec0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f047400eb10 tx=0x7f047400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0479600640 1 -- 172.21.15.46:0/114351927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474004b20 con 0x7f047c109120 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0479600640 1 -- 172.21.15.46:0/114351927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0474004cc0 con 0x7f047c109120 2024-09-10T07:52:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0483000640 1 -- 172.21.15.46:0/114351927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c19f890 con 0x7f047c109120 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0479600640 1 -- 172.21.15.46:0/114351927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474010760 con 0x7f047c109120 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.712+0000 7f0483000640 1 -- 172.21.15.46:0/114351927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c1a7170 con 0x7f047c109120 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0485abf740 1 Processor -- start 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0479600640 1 -- 172.21.15.46:0/114351927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0474005020 con 0x7f047c109120 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0485abf740 1 -- start start 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0485abf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc8685f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0485abf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8690830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0485abf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8690d70 0x55adc8693160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0485abf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55adc85ec3e0 con 0x55adc86864b0 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0485abf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55adc8593d80 con 0x55adc8685b90 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0485abf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55adc85e83a0 con 0x55adc8690d70 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.717+0000 7f0473e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc8685f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc8685f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58176/0 (socket says 172.21.15.46:58176) 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f047be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8690830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 -- 172.21.15.46:0/3956569773 learned_addr learned my addr 172.21.15.46:0/3956569773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 --2- 172.21.15.46:0/3956569773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8690d70 0x55adc8693160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 -- 172.21.15.46:0/3956569773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8690d70 msgr2=0x55adc8693160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 --2- 172.21.15.46:0/3956569773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8690d70 0x55adc8693160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 -- 172.21.15.46:0/3956569773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 msgr2=0x55adc8690830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 --2- 172.21.15.46:0/3956569773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8690830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 -- 172.21.15.46:0/3956569773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc85b2ae0 con 0x55adc8685b90 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0473e00640 1 --2- 172.21.15.46:0/3956569773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc8685f70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0464009fd0 tx=0x7f0464005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b355681a0af82aa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0479600640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04600a2b80 0x7f04600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.718+0000 7f0471600640 1 -- 172.21.15.46:0/3956569773 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464004030 con 0x55adc8685b90 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0471600640 1 -- 172.21.15.46:0/3956569773 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0464005500 con 0x55adc8685b90 2024-09-10T07:52:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0471600640 1 -- 172.21.15.46:0/3956569773 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464004d70 con 0x55adc8685b90 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0479600640 1 -- 172.21.15.46:0/114351927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f04740cfa60 con 0x7f047c109120 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f047be00640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04600a2b80 0x7f04600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0485abf740 1 -- 172.21.15.46:0/3956569773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 msgr2=0x55adc8685f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0485abf740 1 --2- 172.21.15.46:0/3956569773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc8685f70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0464009fd0 tx=0x7f0464005850 comp rx=0 tx=0).stop 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0485abf740 1 -- 172.21.15.46:0/3956569773 shutdown_connections 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0485abf740 1 --2- 172.21.15.46:0/3956569773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8690d70 0x55adc8693160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0485abf740 1 --2- 172.21.15.46:0/3956569773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8690830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0485abf740 1 --2- 172.21.15.46:0/3956569773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc8685f70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0485abf740 1 -- 172.21.15.46:0/3956569773 >> 172.21.15.46:0/3956569773 conn(0x55adc8682120 msgr2=0x55adc8682500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.719+0000 7f0485abf740 1 -- 172.21.15.46:0/3956569773 shutdown_connections 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.720+0000 7f0485abf740 1 -- 172.21.15.46:0/3956569773 wait complete. 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.720+0000 7f047be00640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04600a2b80 0x7f04600a5040 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f046c009230 tx=0x7f046c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.720+0000 7f0485abf740 1 -- start start 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.720+0000 7f0485abf740 1 Processor -- start 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0485abf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc868dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0485abf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc868e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0485abf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8690d70 0x55adc868e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0485abf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55adc86939e0 con 0x55adc8690d70 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0485abf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55adc85a70f0 con 0x55adc8685b90 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0485abf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55adc8693b60 con 0x55adc86864b0 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0485abf740 10 client.0 _ll_register_callbacks cb 0x55adc85989e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f047be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc868dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0473e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc868e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f047be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc868dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58190/0 (socket says 172.21.15.46:58190) 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f047be00640 1 -- 172.21.15.46:0/1248037071 learned_addr learned my addr 172.21.15.46:0/1248037071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0481000640 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8690d70 0x55adc868e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f047be00640 1 -- 172.21.15.46:0/1248037071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 msgr2=0x55adc868e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f047be00640 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc868e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f047be00640 1 -- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8690d70 msgr2=0x55adc868e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f047be00640 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8690d70 0x55adc868e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f047be00640 1 -- 172.21.15.46:0/1248037071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc835d3c0 con 0x55adc8685b90 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.721+0000 7f0481000640 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8690d70 0x55adc868e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.722+0000 7f047be00640 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc868dc10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f046c031b50 tx=0x7f046c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c81de135261e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.722+0000 7f046a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.722+0000 7f046a000640 1 -- 172.21.15.46:0/1248037071 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c04d070 con 0x55adc8685b90 2024-09-10T07:52:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.722+0000 7f046a000640 1 -- 172.21.15.46:0/1248037071 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f046c0037f0 con 0x55adc8685b90 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.722+0000 7f0485abf740 1 -- 172.21.15.46:0/1248037071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0464005500 con 0x55adc8685b90 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.722+0000 7f046a000640 1 -- 172.21.15.46:0/1248037071 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f046c043070 con 0x55adc8685b90 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.722+0000 7f0485abf740 10 client.29378 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f0485abf740 1 -- 172.21.15.46:0/1248037071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55adc869f640 con 0x55adc8685b90 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f046a000640 1 -- 172.21.15.46:0/1248037071 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0464005500 con 0x55adc8685b90 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f046a000640 1 -- 172.21.15.46:0/1248037071 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f046c052c30 con 0x55adc8685b90 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f0485abf740 10 client.29378 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f0485abf740 10 client.29378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f0485abf740 20 client.29378 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f0485abf740 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55adc86a0b50 0x55adc86a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f0485abf740 4 client.29378 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f0485abf740 1 -- 172.21.15.46:0/1248037071 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc85f9950 con 0x55adc86a0b50 2024-09-10T07:52:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.723+0000 7f0473e00640 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55adc86a0b50 0x55adc86a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.724+0000 7f0473e00640 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55adc86a0b50 0x55adc86a2f80 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55b2cb8143fdc67 server_cookie=836ccda5aab174b1 in_seq=0 out_seq=0 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.724+0000 7f046a000640 10 client.29378 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f046a000640 1 -- 172.21.15.46:0/1248037071 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55adc85f9950 con 0x55adc86a0b50 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f046a000640 10 client.29378 handle_command_reply: tid=0 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 client.29378 shutdown 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 -- 172.21.15.46:0/1248037071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 msgr2=0x55adc868dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc868dc10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f046c031b50 tx=0x7f046c009260 comp rx=0 tx=0).stop 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 -- 172.21.15.46:0/1248037071 shutdown_connections 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55adc86a0b50 0x55adc86a2f80 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8690d70 0x55adc868e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc868e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 --2- 172.21.15.46:0/1248037071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8685b90 0x55adc868dc10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f046a000640 0 client.29378 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 -- 172.21.15.46:0/1248037071 >> 172.21.15.46:0/1248037071 conn(0x55adc8682120 msgr2=0x55adc8687e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 -- 172.21.15.46:0/1248037071 shutdown_connections 2024-09-10T07:52:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 1 -- 172.21.15.46:0/1248037071 wait complete. 2024-09-10T07:52:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.725+0000 7f0485abf740 20 client.29378 trim_cache size 0 max 16384 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0485abf740 1 Processor -- start 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0485abf740 1 -- start start 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0485abf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8685b90 0x55adc8688640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0485abf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc86810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0485abf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc86a8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0485abf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b0e40 con 0x55adc8685b90 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0485abf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b2b00 con 0x55adc8690d70 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0485abf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b1510 con 0x55adc86864b0 2024-09-10T07:52:46.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f047be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8685b90 0x55adc8688640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f047be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8685b90 0x55adc8688640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60936/0 (socket says 172.21.15.46:60936) 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0481000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc86a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f047be00640 1 -- 172.21.15.46:0/3935568763 learned_addr learned my addr 172.21.15.46:0/3935568763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.744+0000 7f0473e00640 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc86810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f047be00640 1 -- 172.21.15.46:0/3935568763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 msgr2=0x55adc86810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f047be00640 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc86810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f047be00640 1 -- 172.21.15.46:0/3935568763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 msgr2=0x55adc86a8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f047be00640 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc86a8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f047be00640 1 -- 172.21.15.46:0/3935568763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc8681f50 con 0x55adc8685b90 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f0473e00640 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc86810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f0481000640 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc86a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f047be00640 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8685b90 0x55adc8688640 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f046c0315d0 tx=0x7f046c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd76c8248cf9737 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.745+0000 7f0470c00640 1 -- 172.21.15.46:0/3935568763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c04d070 con 0x55adc8685b90 2024-09-10T07:52:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.746+0000 7f0470c00640 1 -- 172.21.15.46:0/3935568763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c03d710 con 0x55adc8685b90 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.746+0000 7f0470c00640 1 -- 172.21.15.46:0/3935568763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c035040 con 0x55adc8685b90 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.746+0000 7f0485abf740 1 -- 172.21.15.46:0/3935568763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8685b90 msgr2=0x55adc8688640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.746+0000 7f0485abf740 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8685b90 0x55adc8688640 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f046c0315d0 tx=0x7f046c005df0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.747+0000 7f0485abf740 1 -- 172.21.15.46:0/3935568763 shutdown_connections 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.747+0000 7f0485abf740 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc86a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.747+0000 7f0485abf740 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc86864b0 0x55adc86810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.747+0000 7f0485abf740 1 --2- 172.21.15.46:0/3935568763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc8685b90 0x55adc8688640 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.747+0000 7f0485abf740 1 -- 172.21.15.46:0/3935568763 >> 172.21.15.46:0/3935568763 conn(0x55adc8682120 msgr2=0x55adc86809a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.747+0000 7f0485abf740 1 -- 172.21.15.46:0/3935568763 shutdown_connections 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.747+0000 7f0485abf740 1 -- 172.21.15.46:0/3935568763 wait complete. 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.747+0000 7f0485abf740 1 -- start start 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0485abf740 1 Processor -- start 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0485abf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8685b90 0x55adc8694ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0485abf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8694ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0485abf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc8695530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0485abf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b2370 con 0x55adc86864b0 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0485abf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b1ca0 con 0x55adc8690d70 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0485abf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1aee20 con 0x55adc8685b90 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0485abf740 10 client.0 _ll_register_callbacks cb 0x7f046c031850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0481000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc8695530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0473e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8694ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0473e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8694ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60938/0 (socket says 172.21.15.46:60938) 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0473e00640 1 -- 172.21.15.46:0/2501608165 learned_addr learned my addr 172.21.15.46:0/2501608165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f0481000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc8695530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58208/0 (socket says 172.21.15.46:58208) 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.748+0000 7f047be00640 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8685b90 0x55adc8694ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.749+0000 7f047be00640 1 -- 172.21.15.46:0/2501608165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 msgr2=0x55adc8695530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.749+0000 7f047be00640 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc8695530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.749+0000 7f047be00640 1 -- 172.21.15.46:0/2501608165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 msgr2=0x55adc8694ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.749+0000 7f047be00640 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8694ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.749+0000 7f047be00640 1 -- 172.21.15.46:0/2501608165 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc8682db0 con 0x55adc8685b90 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.749+0000 7f0481000640 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc8695530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.749+0000 7f047be00640 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8685b90 0x55adc8694ab0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f046c0096c0 tx=0x7f046c03def0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8008127123c77689 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.749+0000 7f046a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.750+0000 7f046a000640 1 -- 172.21.15.46:0/2501608165 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c06d060 con 0x55adc8685b90 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.750+0000 7f0485abf740 1 -- 172.21.15.46:0/2501608165 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f046c03d710 con 0x55adc8685b90 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.750+0000 7f046a000640 1 -- 172.21.15.46:0/2501608165 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f046c005450 con 0x55adc8685b90 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.750+0000 7f046a000640 1 -- 172.21.15.46:0/2501608165 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f046c074050 con 0x55adc8685b90 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.750+0000 7f0485abf740 10 client.29275 fetch_fsmap learned FSMap version 60 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.750+0000 7f0485abf740 1 -- 172.21.15.46:0/2501608165 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55adc870b920 con 0x55adc8685b90 2024-09-10T07:52:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.750+0000 7f046a000640 1 -- 172.21.15.46:0/2501608165 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c03d710 con 0x55adc8685b90 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.751+0000 7f046a000640 1 -- 172.21.15.46:0/2501608165 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 60) ==== 8402+0+0 (secure 0 0 0) 0x7f046c003680 con 0x55adc8685b90 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.751+0000 7f0485abf740 10 client.29275 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.751+0000 7f0485abf740 10 client.29275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.751+0000 7f0485abf740 20 client.29275 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.751+0000 7f0485abf740 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55adc870bf30 0x55adc870e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.751+0000 7f0485abf740 4 client.29275 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.751+0000 7f0485abf740 1 -- 172.21.15.46:0/2501608165 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55adc85f3e60 con 0x55adc870bf30 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.751+0000 7f0473e00640 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55adc870bf30 0x55adc870e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0473e00640 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55adc870bf30 0x55adc870e350 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db909502740f0da server_cookie=c88511810e6f7076 in_seq=0 out_seq=0 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f046a000640 10 client.29275 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f046a000640 1 -- 172.21.15.46:0/2501608165 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55adc85f3e60 con 0x55adc870bf30 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f046a000640 10 client.29275 handle_command_reply: tid=0 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 client.29275 shutdown 2024-09-10T07:52:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 -- 172.21.15.46:0/2501608165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8685b90 msgr2=0x55adc8694ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8685b90 0x55adc8694ab0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f046c0096c0 tx=0x7f046c03def0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 -- 172.21.15.46:0/2501608165 shutdown_connections 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55adc870bf30 0x55adc870e350 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adc8690d70 0x55adc8695530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adc86864b0 0x55adc8694ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 --2- 172.21.15.46:0/2501608165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55adc8685b90 0x55adc8694ab0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f046a000640 0 client.29275 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 -- 172.21.15.46:0/2501608165 >> 172.21.15.46:0/2501608165 conn(0x55adc8682120 msgr2=0x55adc86811c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.752+0000 7f0485abf740 1 -- 172.21.15.46:0/2501608165 shutdown_connections 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.753+0000 7f0485abf740 1 -- 172.21.15.46:0/2501608165 wait complete. 2024-09-10T07:52:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.753+0000 7f0485abf740 20 client.29275 trim_cache size 0 max 16384 2024-09-10T07:52:46.764 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:52:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-09-10T07:52:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-09-10T07:52:46.764 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:52:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.753+0000 7f0483000640 1 -- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04600a2b80 msgr2=0x7f04600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.753+0000 7f0483000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04600a2b80 0x7f04600a5040 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f046c009230 tx=0x7f046c0058d0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.753+0000 7f0483000640 1 -- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c109120 msgr2=0x7f047c19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.753+0000 7f0483000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c109120 0x7f047c19eec0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f047400eb10 tx=0x7f047400c2d0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.754+0000 7f0483000640 1 -- 172.21.15.46:0/114351927 shutdown_connections 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.754+0000 7f0483000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f04600a2b80 0x7f04600a5040 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.754+0000 7f0483000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047c109120 0x7f047c19eec0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.754+0000 7f0483000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1049a0 0x7f047c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.754+0000 7f0483000640 1 --2- 172.21.15.46:0/114351927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f047c103ff0 0x7f047c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.754+0000 7f0483000640 1 -- 172.21.15.46:0/114351927 >> 172.21.15.46:0/114351927 conn(0x7f047c0ffa90 msgr2=0x7f047c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.754+0000 7f0483000640 1 -- 172.21.15.46:0/114351927 shutdown_connections 2024-09-10T07:52:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:46.754+0000 7f0483000640 1 -- 172.21.15.46:0/114351927 wait complete. 2024-09-10T07:52:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:47.733 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:52:47.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:52:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 -- 172.21.15.46:0/670444002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 msgr2=0x7fe34c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 --2- 172.21.15.46:0/670444002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 0x7fe34c10e890 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe344009cd0 tx=0x7fe34402a8e0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 -- 172.21.15.46:0/670444002 shutdown_connections 2024-09-10T07:52:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 --2- 172.21.15.46:0/670444002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 0x7fe34c10e890 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 --2- 172.21.15.46:0/670444002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe34c1019c0 0x7fe34c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 --2- 172.21.15.46:0/670444002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe34c1079c0 0x7fe34c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 -- 172.21.15.46:0/670444002 >> 172.21.15.46:0/670444002 conn(0x7fe34c0fd740 msgr2=0x7fe34c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 -- 172.21.15.46:0/670444002 shutdown_connections 2024-09-10T07:52:48.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:48 smithi046 ceph-mon[30705]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-10T07:52:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.308+0000 7fe352800640 1 -- 172.21.15.46:0/670444002 wait complete. 2024-09-10T07:52:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.309+0000 7fe352800640 1 Processor -- start 2024-09-10T07:52:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.309+0000 7fe352800640 1 -- start start 2024-09-10T07:52:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe352800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe34c1019c0 0x7fe34c1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe352800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 0x7fe34c108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe352800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe34c1079c0 0x7fe34c109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe352800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c111e10 con 0x7fe34c102360 2024-09-10T07:52:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe352800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c111c90 con 0x7fe34c1019c0 2024-09-10T07:52:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe352800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c111f90 con 0x7fe34c1079c0 2024-09-10T07:52:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe351400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe34c1019c0 0x7fe34c1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe351400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe34c1019c0 0x7fe34c1088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58220/0 (socket says 172.21.15.46:58220) 2024-09-10T07:52:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe351400640 1 -- 172.21.15.46:0/334608365 learned_addr learned my addr 172.21.15.46:0/334608365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe350a00640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 0x7fe34c108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.310+0000 7fe351e00640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe34c1079c0 0x7fe34c109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.311+0000 7fe351e00640 1 -- 172.21.15.46:0/334608365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe34c1019c0 msgr2=0x7fe34c1088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.311+0000 7fe351e00640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe34c1019c0 0x7fe34c1088f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.311+0000 7fe351e00640 1 -- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 msgr2=0x7fe34c108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.311+0000 7fe351e00640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 0x7fe34c108e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.311+0000 7fe351e00640 1 -- 172.21.15.46:0/334608365 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c1aab90 con 0x7fe34c1079c0 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.311+0000 7fe350a00640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 0x7fe34c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.312+0000 7fe351e00640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe34c1079c0 0x7fe34c109370 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe34402ae00 tx=0x7fe344031e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.312+0000 7fe34a000640 1 -- 172.21.15.46:0/334608365 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344004040 con 0x7fe34c1079c0 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.312+0000 7fe34a000640 1 -- 172.21.15.46:0/334608365 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3440046c0 con 0x7fe34c1079c0 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.312+0000 7fe352800640 1 -- 172.21.15.46:0/334608365 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34c1aae20 con 0x7fe34c1079c0 2024-09-10T07:52:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.312+0000 7fe34a000640 1 -- 172.21.15.46:0/334608365 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344030ec0 con 0x7fe34c1079c0 2024-09-10T07:52:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.312+0000 7fe352800640 1 -- 172.21.15.46:0/334608365 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34c1ab330 con 0x7fe34c1079c0 2024-09-10T07:52:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.316+0000 7fe356a42740 1 Processor -- start 2024-09-10T07:52:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.317+0000 7fe34a000640 1 -- 172.21.15.46:0/334608365 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe3440305a0 con 0x7fe34c1079c0 2024-09-10T07:52:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.317+0000 7fe356a42740 1 -- start start 2024-09-10T07:52:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe356a42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec532fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe356a42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec53301a0 0x55eec533a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe356a42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec533aa60 0x55eec533ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe356a42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eec5296340 con 0x55eec532f880 2024-09-10T07:52:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe356a42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eec523dbf0 con 0x55eec533aa60 2024-09-10T07:52:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe356a42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eec5292300 con 0x55eec53301a0 2024-09-10T07:52:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe351400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec53301a0 0x55eec533a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe350a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec532fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe350a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec532fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60962/0 (socket says 172.21.15.46:60962) 2024-09-10T07:52:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe351400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec53301a0 0x55eec533a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:47836/0 (socket says 172.21.15.46:47836) 2024-09-10T07:52:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe350a00640 1 -- 172.21.15.46:0/1709460757 learned_addr learned my addr 172.21.15.46:0/1709460757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.318+0000 7fe350a00640 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec533aa60 0x55eec533ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe351400640 1 -- 172.21.15.46:0/1709460757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec533aa60 msgr2=0x55eec533ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe351400640 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec533aa60 0x55eec533ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe351400640 1 -- 172.21.15.46:0/1709460757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 msgr2=0x55eec532fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe351400640 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec532fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe351400640 1 -- 172.21.15.46:0/1709460757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec5007620 con 0x55eec53301a0 2024-09-10T07:52:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe350a00640 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec532fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe351400640 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec53301a0 0x55eec533a520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe340009c70 tx=0x7fe340011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e84b542b31dc15d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe34a000640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe3340a2d20 0x7fe3340a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe33e600640 1 -- 172.21.15.46:0/1709460757 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340005220 con 0x55eec53301a0 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe34a000640 1 -- 172.21.15.46:0/334608365 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe34402e030 con 0x7fe34c1079c0 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe33e600640 1 -- 172.21.15.46:0/1709460757 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe340004c80 con 0x55eec53301a0 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.319+0000 7fe350a00640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe3340a2d20 0x7fe3340a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.320+0000 7fe33e600640 1 -- 172.21.15.46:0/1709460757 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340019890 con 0x55eec53301a0 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 -- 172.21.15.46:0/1709460757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec53301a0 msgr2=0x55eec533a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe350a00640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe3340a2d20 0x7fe3340a51e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55eec532d190 tx=0x7fe33800bbb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec53301a0 0x55eec533a520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe340009c70 tx=0x7fe340011810 comp rx=0 tx=0).stop 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 -- 172.21.15.46:0/1709460757 shutdown_connections 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec533aa60 0x55eec533ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec53301a0 0x55eec533a520 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 --2- 172.21.15.46:0/1709460757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec532fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 -- 172.21.15.46:0/1709460757 >> 172.21.15.46:0/1709460757 conn(0x55eec532be10 msgr2=0x55eec532c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 -- 172.21.15.46:0/1709460757 shutdown_connections 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 -- 172.21.15.46:0/1709460757 wait complete. 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.321+0000 7fe356a42740 1 -- start start 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe356a42740 1 Processor -- start 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe356a42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec5337900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe356a42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec5337e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe356a42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec5338380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe356a42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eec533d630 con 0x55eec532f880 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe356a42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eec5251060 con 0x55eec53301a0 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe356a42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55eec533d7b0 con 0x55eec533aa60 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe356a42740 10 client.0 _ll_register_callbacks cb 0x55eec523faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.322+0000 7fe351400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec5337900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe350a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec5337e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec5338380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec5338380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:47848/0 (socket says 172.21.15.46:47848) 2024-09-10T07:52:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe350a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec5337e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58244/0 (socket says 172.21.15.46:58244) 2024-09-10T07:52:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 -- 172.21.15.46:0/1061449400 learned_addr learned my addr 172.21.15.46:0/1061449400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 -- 172.21.15.46:0/1061449400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 msgr2=0x55eec5337e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec5337e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 -- 172.21.15.46:0/1061449400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 msgr2=0x55eec5337900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec5337900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 -- 172.21.15.46:0/1061449400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec5346af0 con 0x55eec533aa60 2024-09-10T07:52:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe351e00640 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec5338380 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe3440c5570 tx=0x7fe3440bccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b82a390cd7ab19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.323+0000 7fe332a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.324+0000 7fe332a00640 1 -- 172.21.15.46:0/1061449400 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3440bce30 con 0x55eec533aa60 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.324+0000 7fe356a42740 1 -- 172.21.15.46:0/1061449400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe340004c80 con 0x55eec533aa60 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.324+0000 7fe332a00640 1 -- 172.21.15.46:0/1061449400 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe3440bc280 con 0x55eec533aa60 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.324+0000 7fe332a00640 1 -- 172.21.15.46:0/1061449400 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe3440c11e0 con 0x55eec533aa60 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.324+0000 7fe356a42740 10 client.29287 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.324+0000 7fe356a42740 1 -- 172.21.15.46:0/1061449400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55eec533d680 con 0x55eec533aa60 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.324+0000 7fe332a00640 1 -- 172.21.15.46:0/1061449400 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe340004c80 con 0x55eec533aa60 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.325+0000 7fe332a00640 1 -- 172.21.15.46:0/1061449400 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7fe34404e070 con 0x55eec533aa60 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.325+0000 7fe356a42740 10 client.29287 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.325+0000 7fe356a42740 10 client.29287 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.325+0000 7fe356a42740 20 client.29287 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.325+0000 7fe356a42740 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eec534a970 0x55eec534cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.325+0000 7fe356a42740 4 client.29287 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.325+0000 7fe356a42740 1 -- 172.21.15.46:0/1061449400 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec52a38b0 con 0x55eec534a970 2024-09-10T07:52:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.325+0000 7fe351400640 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eec534a970 0x55eec534cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.326+0000 7fe351400640 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eec534a970 0x55eec534cdd0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a8afde14c182490 server_cookie=3edf6cc0013c734a in_seq=0 out_seq=0 2024-09-10T07:52:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.326+0000 7fe332a00640 10 client.29287 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe332a00640 1 -- 172.21.15.46:0/1061449400 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eec52a38b0 con 0x55eec534a970 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe332a00640 10 client.29287 handle_command_reply: tid=0 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 client.29287 shutdown 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 -- 172.21.15.46:0/1061449400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 msgr2=0x55eec5338380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec5338380 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe3440c5570 tx=0x7fe3440bccc0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 -- 172.21.15.46:0/1061449400 shutdown_connections 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eec534a970 0x55eec534cdd0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec5338380 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec5337e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 --2- 172.21.15.46:0/1061449400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec532f880 0x55eec5337900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe356a42740 1 -- 172.21.15.46:0/1061449400 >> 172.21.15.46:0/1061449400 conn(0x55eec532be10 msgr2=0x55eec5331b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.327+0000 7fe332a00640 0 client.29287 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.328+0000 7fe356a42740 1 -- 172.21.15.46:0/1061449400 shutdown_connections 2024-09-10T07:52:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.328+0000 7fe356a42740 1 -- 172.21.15.46:0/1061449400 wait complete. 2024-09-10T07:52:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.328+0000 7fe356a42740 20 client.29287 trim_cache size 0 max 16384 2024-09-10T07:52:48.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe356a42740 1 Processor -- start 2024-09-10T07:52:48.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe356a42740 1 -- start start 2024-09-10T07:52:48.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe356a42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec532f880 0x55eec53322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe356a42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec53526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe356a42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec533aa60 0x55eec5352be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe356a42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c1b4fe0 con 0x55eec533aa60 2024-09-10T07:52:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe356a42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c1b6c10 con 0x55eec53301a0 2024-09-10T07:52:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe356a42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c1b56b0 con 0x55eec532f880 2024-09-10T07:52:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe350a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec53526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.348+0000 7fe350a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec53526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58258/0 (socket says 172.21.15.46:58258) 2024-09-10T07:52:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.349+0000 7fe350a00640 1 -- 172.21.15.46:0/745376059 learned_addr learned my addr 172.21.15.46:0/745376059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.349+0000 7fe351400640 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec532f880 0x55eec53322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.349+0000 7fe351e00640 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec533aa60 0x55eec5352be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.349+0000 7fe351400640 1 -- 172.21.15.46:0/745376059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 msgr2=0x55eec53526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.349+0000 7fe351400640 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec53526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.349+0000 7fe351400640 1 -- 172.21.15.46:0/745376059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec533aa60 msgr2=0x55eec5352be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.349+0000 7fe351400640 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec533aa60 0x55eec5352be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.349+0000 7fe351400640 1 -- 172.21.15.46:0/745376059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec532b8b0 con 0x55eec532f880 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.350+0000 7fe351400640 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec532f880 0x55eec53322e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe340005790 tx=0x7fe3400095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e49ffad9f3fb5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.350+0000 7fe33dc00640 1 -- 172.21.15.46:0/745376059 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340004680 con 0x55eec532f880 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.350+0000 7fe33dc00640 1 -- 172.21.15.46:0/745376059 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34002e2d0 con 0x55eec532f880 2024-09-10T07:52:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.350+0000 7fe33dc00640 1 -- 172.21.15.46:0/745376059 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340019690 con 0x55eec532f880 2024-09-10T07:52:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.351+0000 7fe356a42740 1 -- 172.21.15.46:0/745376059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec532f880 msgr2=0x55eec53322e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.351+0000 7fe356a42740 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec532f880 0x55eec53322e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe340005790 tx=0x7fe3400095e0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.351+0000 7fe356a42740 1 -- 172.21.15.46:0/745376059 shutdown_connections 2024-09-10T07:52:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.351+0000 7fe356a42740 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec533aa60 0x55eec5352be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.351+0000 7fe356a42740 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec53301a0 0x55eec53526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.351+0000 7fe356a42740 1 --2- 172.21.15.46:0/745376059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec532f880 0x55eec53322e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.351+0000 7fe356a42740 1 -- 172.21.15.46:0/745376059 >> 172.21.15.46:0/745376059 conn(0x55eec532be10 msgr2=0x55eec5339980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.352+0000 7fe356a42740 1 -- 172.21.15.46:0/745376059 shutdown_connections 2024-09-10T07:52:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.352+0000 7fe356a42740 1 -- 172.21.15.46:0/745376059 wait complete. 2024-09-10T07:52:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.353+0000 7fe356a42740 1 -- start start 2024-09-10T07:52:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.353+0000 7fe356a42740 1 Processor -- start 2024-09-10T07:52:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe356a42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec532f880 0x55eec5343290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe356a42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec53301a0 0x55eec533e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe356a42740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec533ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe356a42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c1b6480 con 0x55eec53301a0 2024-09-10T07:52:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe356a42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c1b5db0 con 0x55eec532f880 2024-09-10T07:52:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe356a42740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c1b2fc0 con 0x55eec533aa60 2024-09-10T07:52:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe356a42740 10 client.0 _ll_register_callbacks cb 0x7fe344003f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe350a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec53301a0 0x55eec533e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe351400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec532f880 0x55eec5343290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe350a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec53301a0 0x55eec533e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60992/0 (socket says 172.21.15.46:60992) 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe350a00640 1 -- 172.21.15.46:0/1371564713 learned_addr learned my addr 172.21.15.46:0/1371564713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.354+0000 7fe351e00640 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec533ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe351e00640 1 -- 172.21.15.46:0/1371564713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec532f880 msgr2=0x55eec5343290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe351e00640 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec532f880 0x55eec5343290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe351e00640 1 -- 172.21.15.46:0/1371564713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec53301a0 msgr2=0x55eec533e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe351e00640 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec53301a0 0x55eec533e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe351e00640 1 -- 172.21.15.46:0/1371564713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec533f400 con 0x55eec533aa60 2024-09-10T07:52:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe351e00640 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec533ebd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe3440ee120 tx=0x7fe3440bcd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84966d874d8a1191 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe332a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:48.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe332a00640 1 -- 172.21.15.46:0/1371564713 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3440da070 con 0x55eec533aa60 2024-09-10T07:52:48.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe356a42740 1 -- 172.21.15.46:0/1371564713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe34002e2d0 con 0x55eec533aa60 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe332a00640 1 -- 172.21.15.46:0/1371564713 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe3440041e0 con 0x55eec533aa60 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe332a00640 1 -- 172.21.15.46:0/1371564713 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe3440b90d0 con 0x55eec533aa60 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe356a42740 10 client.29293 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.355+0000 7fe356a42740 1 -- 172.21.15.46:0/1371564713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55eec53b57c0 con 0x55eec533aa60 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe332a00640 1 -- 172.21.15.46:0/1371564713 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34002e2d0 con 0x55eec533aa60 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe332a00640 1 -- 172.21.15.46:0/1371564713 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7fe34404e040 con 0x55eec533aa60 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe356a42740 10 client.29293 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe356a42740 10 client.29293 resolve_mds: validated gid 24535 aka daemon mds.a 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe356a42740 20 client.29293 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe356a42740 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eec53b5de0 0x55eec53b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe356a42740 4 client.29293 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe356a42740 1 -- 172.21.15.46:0/1371564713 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55eec529ddc0 con 0x55eec53b5de0 2024-09-10T07:52:48.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.356+0000 7fe351400640 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eec53b5de0 0x55eec53b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.357+0000 7fe351400640 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eec53b5de0 0x55eec53b8200 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee8b21422677cfa server_cookie=75eda57ab26187c4 in_seq=0 out_seq=0 2024-09-10T07:52:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.357+0000 7fe332a00640 10 client.29293 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.357+0000 7fe332a00640 1 -- 172.21.15.46:0/1371564713 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55eec529ddc0 con 0x55eec53b5de0 2024-09-10T07:52:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe332a00640 10 client.29293 handle_command_reply: tid=0 2024-09-10T07:52:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 client.29293 shutdown 2024-09-10T07:52:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 -- 172.21.15.46:0/1371564713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 msgr2=0x55eec533ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec533ebd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe3440ee120 tx=0x7fe3440bcd30 comp rx=0 tx=0).stop 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 -- 172.21.15.46:0/1371564713 shutdown_connections 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x55eec53b5de0 0x55eec53b8200 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55eec533aa60 0x55eec533ebd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eec53301a0 0x55eec533e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 --2- 172.21.15.46:0/1371564713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec532f880 0x55eec5343290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe332a00640 0 client.29293 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 -- 172.21.15.46:0/1371564713 >> 172.21.15.46:0/1371564713 conn(0x55eec532be10 msgr2=0x55eec532a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.358+0000 7fe356a42740 1 -- 172.21.15.46:0/1371564713 shutdown_connections 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.359+0000 7fe356a42740 1 -- 172.21.15.46:0/1371564713 wait complete. 2024-09-10T07:52:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.359+0000 7fe356a42740 20 client.29293 trim_cache size 0 max 16384 2024-09-10T07:52:48.360 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:52:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 61, 2024-09-10T07:52:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "leader": 24535, 2024-09-10T07:52:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "set_version": 60, 2024-09-10T07:52:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "sets": {} 2024-09-10T07:52:48.360 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 -- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe3340a2d20 msgr2=0x7fe3340a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe3340a2d20 0x7fe3340a51e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55eec532d190 tx=0x7fe33800bbb0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 -- 172.21.15.46:0/334608365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe34c1079c0 msgr2=0x7fe34c109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe34c1079c0 0x7fe34c109370 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe34402ae00 tx=0x7fe344031e60 comp rx=0 tx=0).stop 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 -- 172.21.15.46:0/334608365 shutdown_connections 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe34c1079c0 0x7fe34c109370 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe3340a2d20 0x7fe3340a51e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe34c102360 0x7fe34c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 --2- 172.21.15.46:0/334608365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe34c1019c0 0x7fe34c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 -- 172.21.15.46:0/334608365 >> 172.21.15.46:0/334608365 conn(0x7fe34c0fd740 msgr2=0x7fe34c10d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 -- 172.21.15.46:0/334608365 shutdown_connections 2024-09-10T07:52:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:48.360+0000 7fe352800640 1 -- 172.21.15.46:0/334608365 wait complete. 2024-09-10T07:52:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:48.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:48 smithi181 ceph-mon[35397]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-10T07:52:48.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:48 smithi063 ceph-mon[35020]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-10T07:52:49.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:49 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:52:49.616 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T07:52:49.617 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-10T07:52:49.626 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:52:49.639 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T07:52:49.649 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T07:52:49.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:49 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:52:49.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:49 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:52:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 -- 172.21.15.46:0/2510560669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc1034b0 msgr2=0x7f49fc10b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 --2- 172.21.15.46:0/2510560669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc1034b0 0x7f49fc10b9a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009900 tx=0x7f49e802a480 comp rx=0 tx=0).stop 2024-09-10T07:52:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 -- 172.21.15.46:0/2510560669 shutdown_connections 2024-09-10T07:52:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 --2- 172.21.15.46:0/2510560669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49fc10bfc0 0x7f49fc10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 --2- 172.21.15.46:0/2510560669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc1034b0 0x7f49fc10b9a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 --2- 172.21.15.46:0/2510560669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49fc102b90 0x7f49fc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 -- 172.21.15.46:0/2510560669 >> 172.21.15.46:0/2510560669 conn(0x7f49fc0fc610 msgr2=0x7f49fc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 -- 172.21.15.46:0/2510560669 shutdown_connections 2024-09-10T07:52:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.087+0000 7f4a00a00640 1 -- 172.21.15.46:0/2510560669 wait complete. 2024-09-10T07:52:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.088+0000 7f4a00a00640 1 Processor -- start 2024-09-10T07:52:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.088+0000 7f4a00a00640 1 -- start start 2024-09-10T07:52:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f4a00a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102b90 0x7f49fc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f4a00a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49fc1034b0 0x7f49fc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f4a00a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49fc10bfc0 0x7f49fc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f4a00a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc111dc0 con 0x7f49fc102b90 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f4a00a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc111c40 con 0x7f49fc10bfc0 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f4a00a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc111f40 con 0x7f49fc1034b0 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102b90 0x7f49fc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49f9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49fc1034b0 0x7f49fc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49f9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49fc1034b0 0x7f49fc19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:47876/0 (socket says 172.21.15.46:47876) 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49f9e00640 1 -- 172.21.15.46:0/1926323936 learned_addr learned my addr 172.21.15.46:0/1926323936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49fb200640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49fc10bfc0 0x7f49fc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102b90 0x7f49fc19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32778/0 (socket says 172.21.15.46:32778) 2024-09-10T07:52:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49f9e00640 1 -- 172.21.15.46:0/1926323936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49fc10bfc0 msgr2=0x7f49fc19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49f9e00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49fc10bfc0 0x7f49fc19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49f9e00640 1 -- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102b90 msgr2=0x7f49fc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49f9e00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102b90 0x7f49fc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.089+0000 7f49f9e00640 1 -- 172.21.15.46:0/1926323936 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc19f4a0 con 0x7f49fc1034b0 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.090+0000 7f49fa800640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102b90 0x7f49fc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.090+0000 7f49f9e00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49fc1034b0 0x7f49fc19aa30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f49e8006010 tx=0x7f49e8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.090+0000 7f49f3400640 1 -- 172.21.15.46:0/1926323936 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e803f030 con 0x7f49fc1034b0 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.090+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc19f730 con 0x7f49fc1034b0 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.090+0000 7f49f3400640 1 -- 172.21.15.46:0/1926323936 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8004440 con 0x7f49fc1034b0 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.090+0000 7f49f3400640 1 -- 172.21.15.46:0/1926323936 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8035070 con 0x7f49fc1034b0 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.090+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc1a7070 con 0x7f49fc1034b0 2024-09-10T07:52:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.091+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc102f70 con 0x7f49fc1034b0 2024-09-10T07:52:50.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.094+0000 7f49f3400640 1 -- 172.21.15.46:0/1926323936 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f49e8042030 con 0x7f49fc1034b0 2024-09-10T07:52:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.095+0000 7f49f3400640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49e00a2b80 0x7f49e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.096+0000 7f49f3400640 1 -- 172.21.15.46:0/1926323936 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f49e80efa80 con 0x7f49fc1034b0 2024-09-10T07:52:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.096+0000 7f49fa800640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49e00a2b80 0x7f49e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.096+0000 7f49fa800640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49e00a2b80 0x7f49e00a5040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009fc0 tx=0x7f49ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.097+0000 7f49f3400640 1 -- 172.21.15.46:0/1926323936 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49e80bc070 con 0x7f49fc1034b0 2024-09-10T07:52:50.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.319+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49fc19b850 con 0x7f49fc1034b0 2024-09-10T07:52:50.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.320+0000 7f49f3400640 1 -- 172.21.15.46:0/1926323936 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11980 (secure 0 0 0) 0x7f49fc19b850 con 0x7f49fc1034b0 2024-09-10T07:52:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:52:50.319 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":61,"btime":"2024-09-10T07:52:48:263830+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:52:47.520095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:52:50.320 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 61 2024-09-10T07:52:50.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49e00a2b80 msgr2=0x7f49e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49e00a2b80 0x7f49e00a5040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009fc0 tx=0x7f49ec005f90 comp rx=0 tx=0).stop 2024-09-10T07:52:50.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49fc1034b0 msgr2=0x7f49fc19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49fc1034b0 0x7f49fc19aa30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f49e8006010 tx=0x7f49e8004040 comp rx=0 tx=0).stop 2024-09-10T07:52:50.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 shutdown_connections 2024-09-10T07:52:50.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49fc10bfc0 0x7f49fc19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49fc1034b0 0x7f49fc19aa30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f49e00a2b80 0x7f49e00a5040 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 --2- 172.21.15.46:0/1926323936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102b90 0x7f49fc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 >> 172.21.15.46:0/1926323936 conn(0x7f49fc0fc610 msgr2=0x7f49fc10d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:50.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 shutdown_connections 2024-09-10T07:52:50.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:50.322+0000 7f4a00a00640 1 -- 172.21.15.46:0/1926323936 wait complete. 2024-09-10T07:52:50.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:50 smithi181 ceph-mon[35397]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-09-10T07:52:50.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:50 smithi063 ceph-mon[35020]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-09-10T07:52:50.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:50 smithi046 ceph-mon[30705]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-09-10T07:52:51.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T07:52:51.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:51 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1926323936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:51.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:51 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1926323936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:51.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:51 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1926323936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:52:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 -- 172.21.15.46:0/4177688422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128103ff0 msgr2=0x7fa1281043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 --2- 172.21.15.46:0/4177688422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128103ff0 0x7fa1281043d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa118009900 tx=0x7fa11802a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 -- 172.21.15.46:0/4177688422 shutdown_connections 2024-09-10T07:52:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 --2- 172.21.15.46:0/4177688422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa128109120 0x7fa128110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 --2- 172.21.15.46:0/4177688422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1281049a0 0x7fa1281089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 --2- 172.21.15.46:0/4177688422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128103ff0 0x7fa1281043d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 -- 172.21.15.46:0/4177688422 >> 172.21.15.46:0/4177688422 conn(0x7fa1280ffa90 msgr2=0x7fa128101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 -- 172.21.15.46:0/4177688422 shutdown_connections 2024-09-10T07:52:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.112+0000 7fa12ee00640 1 -- 172.21.15.46:0/4177688422 wait complete. 2024-09-10T07:52:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.113+0000 7fa12ee00640 1 Processor -- start 2024-09-10T07:52:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ee00640 1 -- start start 2024-09-10T07:52:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa128103ff0 0x7fa12819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1281049a0 0x7fa12819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128109120 0x7fa12819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa128114120 con 0x7fa128109120 2024-09-10T07:52:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ee00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa128113fa0 con 0x7fa1281049a0 2024-09-10T07:52:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281142a0 con 0x7fa128103ff0 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128109120 0x7fa12819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa11f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1281049a0 0x7fa12819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128109120 0x7fa12819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32808/0 (socket says 172.21.15.46:32808) 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa11f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1281049a0 0x7fa12819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58296/0 (socket says 172.21.15.46:58296) 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.114+0000 7fa12ce00640 1 -- 172.21.15.46:0/486782247 learned_addr learned my addr 172.21.15.46:0/486782247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa127e00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa128103ff0 0x7fa12819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa11f400640 1 -- 172.21.15.46:0/486782247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa128103ff0 msgr2=0x7fa12819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa11f400640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa128103ff0 0x7fa12819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa11f400640 1 -- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128109120 msgr2=0x7fa12819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa11f400640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128109120 0x7fa12819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa11f400640 1 -- 172.21.15.46:0/486782247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12819f5d0 con 0x7fa1281049a0 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa12ce00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128109120 0x7fa12819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa11f400640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1281049a0 0x7fa12819ab60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa11000d6d0 tx=0x7fa11000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.115+0000 7fa125600640 1 -- 172.21.15.46:0/486782247 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110017070 con 0x7fa1281049a0 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.116+0000 7fa12ee00640 1 -- 172.21.15.46:0/486782247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12819f8c0 con 0x7fa1281049a0 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.116+0000 7fa125600640 1 -- 172.21.15.46:0/486782247 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa110009820 con 0x7fa1281049a0 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.116+0000 7fa125600640 1 -- 172.21.15.46:0/486782247 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110004cd0 con 0x7fa1281049a0 2024-09-10T07:52:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.116+0000 7fa12ee00640 1 -- 172.21.15.46:0/486782247 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1281a71a0 con 0x7fa1281049a0 2024-09-10T07:52:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa131a0c740 1 Processor -- start 2024-09-10T07:52:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa131a0c740 1 -- start start 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa125600640 1 -- 172.21.15.46:0/486782247 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa1100099d0 con 0x7fa1281049a0 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182dafb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 0x56182db08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa131a0c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56182da613e0 con 0x56182db05e50 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa131a0c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56182da08d80 con 0x56182dafb590 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa131a0c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56182da5d3a0 con 0x56182dafac70 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.119+0000 7fa127e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa127e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db05910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58304/0 (socket says 172.21.15.46:58304) 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa127e00640 1 -- 172.21.15.46:0/2304508975 learned_addr learned my addr 172.21.15.46:0/2304508975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa12ce00640 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182dafb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa11f400640 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 0x56182db08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa127e00640 1 -- 172.21.15.46:0/2304508975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 msgr2=0x56182dafb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa127e00640 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182dafb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa127e00640 1 -- 172.21.15.46:0/2304508975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 msgr2=0x56182db08240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa127e00640 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 0x56182db08240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa127e00640 1 -- 172.21.15.46:0/2304508975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182da27ae0 con 0x56182dafb590 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.120+0000 7fa12ce00640 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182dafb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.121+0000 7fa127e00640 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db05910 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa118009900 tx=0x7fa118005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189ebfd4da0dcd7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.121+0000 7fa125600640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa10c0a2b80 0x7fa10c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.121+0000 7fa11d600640 1 -- 172.21.15.46:0/2304508975 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118040070 con 0x56182dafb590 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.121+0000 7fa11d600640 1 -- 172.21.15.46:0/2304508975 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1180319f0 con 0x56182dafb590 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.121+0000 7fa11d600640 1 -- 172.21.15.46:0/2304508975 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118035040 con 0x56182dafb590 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.121+0000 7fa125600640 1 -- 172.21.15.46:0/486782247 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa1100cef20 con 0x7fa1281049a0 2024-09-10T07:52:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.121+0000 7fa12ce00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa10c0a2b80 0x7fa10c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 -- 172.21.15.46:0/2304508975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 msgr2=0x56182db05910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db05910 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa118009900 tx=0x7fa118005e80 comp rx=0 tx=0).stop 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa12ce00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa10c0a2b80 0x7fa10c0a5040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa120005c40 tx=0x7fa120009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 -- 172.21.15.46:0/2304508975 shutdown_connections 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 0x56182db08240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db05910 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 --2- 172.21.15.46:0/2304508975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182dafb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 -- 172.21.15.46:0/2304508975 >> 172.21.15.46:0/2304508975 conn(0x56182daf7200 msgr2=0x56182daf75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 -- 172.21.15.46:0/2304508975 shutdown_connections 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 -- 172.21.15.46:0/2304508975 wait complete. 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.122+0000 7fa131a0c740 1 -- start start 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.123+0000 7fa131a0c740 1 Processor -- start 2024-09-10T07:52:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.123+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182db02c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.123+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db03180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.123+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 0x56182db036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.123+0000 7fa131a0c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56182db08a20 con 0x56182db05e50 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.123+0000 7fa131a0c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56182da1c0f0 con 0x56182dafb590 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.123+0000 7fa131a0c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56182db08ba0 con 0x56182dafac70 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.123+0000 7fa131a0c740 10 client.0 _ll_register_callbacks cb 0x56182da03bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182db02c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182db02c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:47912/0 (socket says 172.21.15.46:47912) 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 -- 172.21.15.46:0/4247177762 learned_addr learned my addr 172.21.15.46:0/4247177762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa11f400640 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db03180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa12ce00640 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 0x56182db036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 -- 172.21.15.46:0/4247177762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 msgr2=0x56182db03180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db03180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 -- 172.21.15.46:0/4247177762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 msgr2=0x56182db036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 0x56182db036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 -- 172.21.15.46:0/4247177762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182d7d23c0 con 0x56182dafac70 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa127e00640 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182db02c40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa11802a430 tx=0x7fa118005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f30e24d88074de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.124+0000 7fa116000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.125+0000 7fa116000640 1 -- 172.21.15.46:0/4247177762 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11804d060 con 0x56182dafac70 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.125+0000 7fa116000640 1 -- 172.21.15.46:0/4247177762 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa118004030 con 0x56182dafac70 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.125+0000 7fa131a0c740 1 -- 172.21.15.46:0/4247177762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1180319f0 con 0x56182dafac70 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.125+0000 7fa116000640 1 -- 172.21.15.46:0/4247177762 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa118054050 con 0x56182dafac70 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.125+0000 7fa131a0c740 10 client.29311 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.125+0000 7fa131a0c740 1 -- 172.21.15.46:0/4247177762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56182db08a70 con 0x56182dafac70 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.125+0000 7fa116000640 1 -- 172.21.15.46:0/4247177762 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1180319f0 con 0x56182dafac70 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa116000640 1 -- 172.21.15.46:0/4247177762 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7fa1180037b0 con 0x56182dafac70 2024-09-10T07:52:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa131a0c740 10 client.29311 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa131a0c740 10 client.29311 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa131a0c740 20 client.29311 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56182db15c20 0x56182db18010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa131a0c740 4 client.29311 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa131a0c740 1 -- 172.21.15.46:0/4247177762 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56182da6e950 con 0x56182db15c20 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa11f400640 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56182db15c20 0x56182db18010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.126+0000 7fa11f400640 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56182db15c20 0x56182db18010 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7abb984179275ad server_cookie=23641523f1bfee5c in_seq=0 out_seq=0 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.127+0000 7fa116000640 10 client.29311 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.127+0000 7fa116000640 1 -- 172.21.15.46:0/4247177762 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56182da6e950 con 0x56182db15c20 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.127+0000 7fa116000640 10 client.29311 handle_command_reply: tid=0 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.127+0000 7fa131a0c740 1 client.29311 shutdown 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 -- 172.21.15.46:0/4247177762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 msgr2=0x56182db02c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182db02c40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa11802a430 tx=0x7fa118005250 comp rx=0 tx=0).stop 2024-09-10T07:52:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 -- 172.21.15.46:0/4247177762 shutdown_connections 2024-09-10T07:52:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56182db15c20 0x56182db18010 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182db05e50 0x56182db036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafb590 0x56182db03180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4247177762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafac70 0x56182db02c40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 -- 172.21.15.46:0/4247177762 >> 172.21.15.46:0/4247177762 conn(0x56182daf7200 msgr2=0x56182daf6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa116000640 0 client.29311 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:52:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 -- 172.21.15.46:0/4247177762 shutdown_connections 2024-09-10T07:52:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 1 -- 172.21.15.46:0/4247177762 wait complete. 2024-09-10T07:52:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.128+0000 7fa131a0c740 20 client.29311 trim_cache size 0 max 16384 2024-09-10T07:52:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa131a0c740 1 Processor -- start 2024-09-10T07:52:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa131a0c740 1 -- start start 2024-09-10T07:52:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafac70 0x56182db04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafb590 0x56182db1d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182db05e50 0x56182db1dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa131a0c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b0da0 con 0x56182dafac70 2024-09-10T07:52:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa131a0c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b29d0 con 0x56182db05e50 2024-09-10T07:52:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa131a0c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b1470 con 0x56182dafb590 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafac70 0x56182db04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafac70 0x56182db04f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32844/0 (socket says 172.21.15.46:32844) 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa127e00640 1 -- 172.21.15.46:0/4265660364 learned_addr learned my addr 172.21.15.46:0/4265660364 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa11f400640 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafb590 0x56182db1d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.142+0000 7fa12ce00640 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182db05e50 0x56182db1dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa127e00640 1 -- 172.21.15.46:0/4265660364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafb590 msgr2=0x56182db1d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa127e00640 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafb590 0x56182db1d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa127e00640 1 -- 172.21.15.46:0/4265660364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182db05e50 msgr2=0x56182db1dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa127e00640 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182db05e50 0x56182db1dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa127e00640 1 -- 172.21.15.46:0/4265660364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182daf7030 con 0x56182dafac70 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa127e00640 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafac70 0x56182db04f70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa11803c460 tx=0x7fa118005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ce535dc4534d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa11cc00640 1 -- 172.21.15.46:0/4265660364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11804d070 con 0x56182dafac70 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa11cc00640 1 -- 172.21.15.46:0/4265660364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1180037f0 con 0x56182dafac70 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.143+0000 7fa11cc00640 1 -- 172.21.15.46:0/4265660364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118040070 con 0x56182dafac70 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 -- 172.21.15.46:0/4265660364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafac70 msgr2=0x56182db04f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafac70 0x56182db04f70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa11803c460 tx=0x7fa118005e80 comp rx=0 tx=0).stop 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 -- 172.21.15.46:0/4265660364 shutdown_connections 2024-09-10T07:52:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182db05e50 0x56182db1dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182dafb590 0x56182db1d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 --2- 172.21.15.46:0/4265660364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafac70 0x56182db04f70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 -- 172.21.15.46:0/4265660364 >> 172.21.15.46:0/4265660364 conn(0x56182daf7200 msgr2=0x56182dafcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 -- 172.21.15.46:0/4265660364 shutdown_connections 2024-09-10T07:52:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.144+0000 7fa131a0c740 1 -- 172.21.15.46:0/4265660364 wait complete. 2024-09-10T07:52:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 1 -- start start 2024-09-10T07:52:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 1 Processor -- start 2024-09-10T07:52:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafac70 0x56182daffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafb590 0x56182db00390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182db05e50 0x56182db008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b2240 con 0x56182dafb590 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b1b70 con 0x56182dafac70 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281aed80 con 0x56182db05e50 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.145+0000 7fa131a0c740 10 client.0 _ll_register_callbacks cb 0x7fa1100d2830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa11f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafb590 0x56182db00390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa12ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182db05e50 0x56182db008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafac70 0x56182daffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafac70 0x56182daffe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58336/0 (socket says 172.21.15.46:58336) 2024-09-10T07:52:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa12ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182db05e50 0x56182db008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:47930/0 (socket says 172.21.15.46:47930) 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 -- 172.21.15.46:0/913894872 learned_addr learned my addr 172.21.15.46:0/913894872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 -- 172.21.15.46:0/913894872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182db05e50 msgr2=0x56182db008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182db05e50 0x56182db008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 -- 172.21.15.46:0/913894872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafb590 msgr2=0x56182db00390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafb590 0x56182db00390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 -- 172.21.15.46:0/913894872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182db7f950 con 0x56182dafac70 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa127e00640 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafac70 0x56182daffe50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa1180315f0 tx=0x7fa118003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beaefa7fa8c6b9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.146+0000 7fa116000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.147+0000 7fa116000640 1 -- 172.21.15.46:0/913894872 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118048030 con 0x56182dafac70 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.147+0000 7fa131a0c740 1 -- 172.21.15.46:0/913894872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1180037f0 con 0x56182dafac70 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.147+0000 7fa116000640 1 -- 172.21.15.46:0/913894872 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa1180035f0 con 0x56182dafac70 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.147+0000 7fa116000640 1 -- 172.21.15.46:0/913894872 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa11806e050 con 0x56182dafac70 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.147+0000 7fa131a0c740 10 client.29426 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.147+0000 7fa131a0c740 1 -- 172.21.15.46:0/913894872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56182db80c30 con 0x56182dafac70 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.147+0000 7fa116000640 1 -- 172.21.15.46:0/913894872 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1180037f0 con 0x56182dafac70 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.147+0000 7fa116000640 1 -- 172.21.15.46:0/913894872 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7fa118069030 con 0x56182dafac70 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa131a0c740 10 client.29426 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa131a0c740 10 client.29426 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa131a0c740 20 client.29426 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa131a0c740 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56182db81230 0x56182db83620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa131a0c740 4 client.29426 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa131a0c740 1 -- 172.21.15.46:0/913894872 --> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56182da68e60 con 0x56182db81230 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa11f400640 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56182db81230 0x56182db83620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa11f400640 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56182db81230 0x56182db83620 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d770ef65113ccd0 server_cookie=14c73e50f67bcd61 in_seq=0 out_seq=0 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.148+0000 7fa116000640 10 client.29426 ms_handle_connect on v2:172.21.15.181:6832/2451989162 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa116000640 1 -- 172.21.15.46:0/913894872 <== mds.4 v2:172.21.15.181:6832/2451989162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56182da68e60 con 0x56182db81230 2024-09-10T07:52:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa116000640 10 client.29426 handle_command_reply: tid=0 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 client.29426 shutdown 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 -- 172.21.15.46:0/913894872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafac70 msgr2=0x56182daffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafac70 0x56182daffe50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa1180315f0 tx=0x7fa118003f80 comp rx=0 tx=0).stop 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 -- 172.21.15.46:0/913894872 shutdown_connections 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.181:6832/2451989162,v1:172.21.15.181:6833/2451989162] conn(0x56182db81230 0x56182db83620 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56182db05e50 0x56182db008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182dafb590 0x56182db00390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 --2- 172.21.15.46:0/913894872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56182dafac70 0x56182daffe50 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 -- 172.21.15.46:0/913894872 >> 172.21.15.46:0/913894872 conn(0x56182daf7200 msgr2=0x56182dafcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa116000640 0 client.29426 ms_handle_reset on v2:172.21.15.181:6832/2451989162 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 -- 172.21.15.46:0/913894872 shutdown_connections 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 1 -- 172.21.15.46:0/913894872 wait complete. 2024-09-10T07:52:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.149+0000 7fa131a0c740 20 client.29426 trim_cache size 0 max 16384 2024-09-10T07:52:52.154 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:52:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 -- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa10c0a2b80 msgr2=0x7fa10c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa10c0a2b80 0x7fa10c0a5040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa120005c40 tx=0x7fa120009760 comp rx=0 tx=0).stop 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 -- 172.21.15.46:0/486782247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1281049a0 msgr2=0x7fa12819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1281049a0 0x7fa12819ab60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa11000d6d0 tx=0x7fa11000db90 comp rx=0 tx=0).stop 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 -- 172.21.15.46:0/486782247 shutdown_connections 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa10c0a2b80 0x7fa10c0a5040 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa128109120 0x7fa12819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1281049a0 0x7fa12819ab60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 --2- 172.21.15.46:0/486782247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa128103ff0 0x7fa12819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 -- 172.21.15.46:0/486782247 >> 172.21.15.46:0/486782247 conn(0x7fa1280ffa90 msgr2=0x7fa128100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 -- 172.21.15.46:0/486782247 shutdown_connections 2024-09-10T07:52:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:52.150+0000 7fa12ee00640 1 -- 172.21.15.46:0/486782247 wait complete. 2024-09-10T07:52:52.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:52 smithi046 ceph-mon[30705]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-10T07:52:52.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:52 smithi181 ceph-mon[35397]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-10T07:52:52.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:52 smithi063 ceph-mon[35020]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-10T07:52:53.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T07:52:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 -- 172.21.15.46:0/775704624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480076720 msgr2=0x7f6480076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 --2- 172.21.15.46:0/775704624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480076720 0x7f6480076b00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6470009900 tx=0x7f647002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 -- 172.21.15.46:0/775704624 shutdown_connections 2024-09-10T07:52:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 --2- 172.21.15.46:0/775704624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6480112a30 0x7f6480114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 --2- 172.21.15.46:0/775704624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6480077040 0x7f64801124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 --2- 172.21.15.46:0/775704624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480076720 0x7f6480076b00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 -- 172.21.15.46:0/775704624 >> 172.21.15.46:0/775704624 conn(0x7f64800ffa90 msgr2=0x7f6480101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 -- 172.21.15.46:0/775704624 shutdown_connections 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.053+0000 7f6485200640 1 -- 172.21.15.46:0/775704624 wait complete. 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.054+0000 7f6485200640 1 Processor -- start 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.054+0000 7f6485200640 1 -- start start 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.054+0000 7f6485200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6480076720 0x7f64801a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.054+0000 7f6485200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480077040 0x7f64801a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.054+0000 7f6485200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6480112a30 0x7f648019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.054+0000 7f6485200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64801185c0 con 0x7f6480077040 2024-09-10T07:52:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.054+0000 7f6485200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6480118440 con 0x7f6480112a30 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.054+0000 7f6485200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6480118740 con 0x7f6480076720 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480077040 0x7f64801a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6480112a30 0x7f648019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480077040 0x7f64801a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32868/0 (socket says 172.21.15.46:32868) 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647d400640 1 -- 172.21.15.46:0/3939529681 learned_addr learned my addr 172.21.15.46:0/3939529681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647de00640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6480076720 0x7f64801a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647e800640 1 -- 172.21.15.46:0/3939529681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6480076720 msgr2=0x7f64801a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647e800640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6480076720 0x7f64801a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647e800640 1 -- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480077040 msgr2=0x7f64801a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647e800640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480077040 0x7f64801a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647e800640 1 -- 172.21.15.46:0/3939529681 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648019f000 con 0x7f6480112a30 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647d400640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480077040 0x7f64801a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.055+0000 7f647e800640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6480112a30 0x7f648019e8d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f648007aa10 tx=0x7f647800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.056+0000 7f6476a00640 1 -- 172.21.15.46:0/3939529681 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647800cdf0 con 0x7f6480112a30 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.056+0000 7f6485200640 1 -- 172.21.15.46:0/3939529681 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648019f2f0 con 0x7f6480112a30 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.056+0000 7f6476a00640 1 -- 172.21.15.46:0/3939529681 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6478005020 con 0x7f6480112a30 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.056+0000 7f6476a00640 1 -- 172.21.15.46:0/3939529681 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478010670 con 0x7f6480112a30 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.056+0000 7f6485200640 1 -- 172.21.15.46:0/3939529681 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64801ab5e0 con 0x7f6480112a30 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.059+0000 7f6487dd8740 1 Processor -- start 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.059+0000 7f6476a00640 1 -- 172.21.15.46:0/3939529681 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64780054d0 con 0x7f6480112a30 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.059+0000 7f6487dd8740 1 -- start start 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f50050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f5a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f5d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f6487dd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e5c3eb63e0 con 0x55e5c3f5ae50 2024-09-10T07:52:54.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f6487dd8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e5c3e5dd80 con 0x55e5c3f4fc70 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f6487dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e5c3eb23a0 con 0x55e5c3f50590 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f647d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f50050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f647d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f50050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58356/0 (socket says 172.21.15.46:58356) 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f647d400640 1 -- 172.21.15.46:0/2077467735 learned_addr learned my addr 172.21.15.46:0/2077467735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f647de00640 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f5a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.060+0000 7f647d400640 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f5d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f647d400640 1 -- 172.21.15.46:0/2077467735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 msgr2=0x55e5c3f5a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f647d400640 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f5a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f647d400640 1 -- 172.21.15.46:0/2077467735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 msgr2=0x55e5c3f5d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f647d400640 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f5d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f647d400640 1 -- 172.21.15.46:0/2077467735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c3e7cae0 con 0x55e5c3f4fc70 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f647de00640 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f5a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:54.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f6476a00640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64640a2b80 0x7f64640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f647d400640 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f50050 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f646c011c90 tx=0x7f646c009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec41befbefbfed52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f6476a00640 1 -- 172.21.15.46:0/3939529681 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f64780cfa80 con 0x7f6480112a30 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f647de00640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64640a2b80 0x7f64640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f646b400640 1 -- 172.21.15.46:0/2077467735 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00fc50 con 0x55e5c3f4fc70 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f646b400640 1 -- 172.21.15.46:0/2077467735 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f646c00fdf0 con 0x55e5c3f4fc70 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.061+0000 7f646b400640 1 -- 172.21.15.46:0/2077467735 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00e6b0 con 0x55e5c3f4fc70 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f647de00640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64640a2b80 0x7f64640a5040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f648019fa10 tx=0x7f64700058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 -- 172.21.15.46:0/2077467735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 msgr2=0x55e5c3f50050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f50050 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f646c011c90 tx=0x7f646c009c80 comp rx=0 tx=0).stop 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 -- 172.21.15.46:0/2077467735 shutdown_connections 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f5d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f5a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2077467735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f50050 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 -- 172.21.15.46:0/2077467735 >> 172.21.15.46:0/2077467735 conn(0x55e5c3f4c200 msgr2=0x55e5c3f4c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 -- 172.21.15.46:0/2077467735 shutdown_connections 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.062+0000 7f6487dd8740 1 -- 172.21.15.46:0/2077467735 wait complete. 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 1 -- start start 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 1 Processor -- start 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f57d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e5c3f5da20 con 0x55e5c3f5ae50 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e5c3e710f0 con 0x55e5c3f4fc70 2024-09-10T07:52:54.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e5c3f5dba0 con 0x55e5c3f50590 2024-09-10T07:52:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.063+0000 7f6487dd8740 10 client.0 _ll_register_callbacks cb 0x55e5c3e58bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f57d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32884/0 (socket says 172.21.15.46:32884) 2024-09-10T07:52:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f57d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58370/0 (socket says 172.21.15.46:58370) 2024-09-10T07:52:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 -- 172.21.15.46:0/3599153393 learned_addr learned my addr 172.21.15.46:0/3599153393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647d400640 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 -- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 msgr2=0x55e5c3f58270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f58270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 -- 172.21.15.46:0/3599153393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 msgr2=0x55e5c3f57d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f57d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 -- 172.21.15.46:0/3599153393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c3c273c0 con 0x55e5c3f5ae50 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647d400640 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f58270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f647e800640 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f587b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f647801a040 tx=0x7f647809c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a7c8fa065ad1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f645fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.064+0000 7f645fe00640 1 -- 172.21.15.46:0/3599153393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64780a4b70 con 0x55e5c3f5ae50 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f645fe00640 1 -- 172.21.15.46:0/3599153393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f64780a46e0 con 0x55e5c3f5ae50 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 1 -- 172.21.15.46:0/3599153393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f646c00fdf0 con 0x55e5c3f5ae50 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f645fe00640 1 -- 172.21.15.46:0/3599153393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f647807f050 con 0x55e5c3f5ae50 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 10 client.18498 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 1 -- 172.21.15.46:0/3599153393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e5c3f69ac0 con 0x55e5c3f5ae50 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f645fe00640 1 -- 172.21.15.46:0/3599153393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f646c00fdf0 con 0x55e5c3f5ae50 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f645fe00640 1 -- 172.21.15.46:0/3599153393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7f647809b470 con 0x55e5c3f5ae50 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 10 client.18498 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 10 client.18498 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 20 client.18498 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e5c3f6af80 0x55e5c3f6d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 4 client.18498 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.065+0000 7f6487dd8740 1 -- 172.21.15.46:0/3599153393 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5c3ec3950 con 0x55e5c3f6af80 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.066+0000 7f647de00640 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e5c3f6af80 0x55e5c3f6d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.066+0000 7f647de00640 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e5c3f6af80 0x55e5c3f6d3d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21a26d4356dd842a server_cookie=2c09963122a12da2 in_seq=0 out_seq=0 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.066+0000 7f645fe00640 10 client.18498 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.067+0000 7f645fe00640 1 -- 172.21.15.46:0/3599153393 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5c3ec3950 con 0x55e5c3f6af80 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.067+0000 7f645fe00640 10 client.18498 handle_command_reply: tid=0 2024-09-10T07:52:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.067+0000 7f6487dd8740 1 client.18498 shutdown 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 -- 172.21.15.46:0/3599153393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 msgr2=0x55e5c3f587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f587b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f647801a040 tx=0x7f647809c740 comp rx=0 tx=0).stop 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 -- 172.21.15.46:0/3599153393 shutdown_connections 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f587b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e5c3f6af80 0x55e5c3f6d3d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f58270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 --2- 172.21.15.46:0/3599153393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f57d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f645fe00640 0 client.18498 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 -- 172.21.15.46:0/3599153393 >> 172.21.15.46:0/3599153393 conn(0x55e5c3f4c200 msgr2=0x55e5c3f51ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 -- 172.21.15.46:0/3599153393 shutdown_connections 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 1 -- 172.21.15.46:0/3599153393 wait complete. 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.068+0000 7f6487dd8740 20 client.18498 trim_cache size 0 max 16384 2024-09-10T07:52:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f6487dd8740 1 Processor -- start 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f6487dd8740 1 -- start start 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f72ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f6487dd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64801b52f0 con 0x55e5c3f4fc70 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f6487dd8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64801b6f20 con 0x55e5c3f5ae50 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f6487dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64801b59c0 con 0x55e5c3f50590 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f647de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f647de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32890/0 (socket says 172.21.15.46:32890) 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f647de00640 1 -- 172.21.15.46:0/1745973400 learned_addr learned my addr 172.21.15.46:0/1745973400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f647d400640 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f72ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.082+0000 7f647e800640 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f647de00640 1 -- 172.21.15.46:0/1745973400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 msgr2=0x55e5c3f72ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f647de00640 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f72ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f647de00640 1 -- 172.21.15.46:0/1745973400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f5ae50 msgr2=0x55e5c3f731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f647de00640 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f647de00640 1 -- 172.21.15.46:0/1745973400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c3f47770 con 0x55e5c3f4fc70 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f647d400640 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f72ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f647de00640 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f526d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6470031ba0 tx=0x7f647003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22900845b1de9b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f646aa00640 1 -- 172.21.15.46:0/1745973400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470035040 con 0x55e5c3f4fc70 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f646aa00640 1 -- 172.21.15.46:0/1745973400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647005a070 con 0x55e5c3f4fc70 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.083+0000 7f646aa00640 1 -- 172.21.15.46:0/1745973400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647003d450 con 0x55e5c3f4fc70 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 -- 172.21.15.46:0/1745973400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 msgr2=0x55e5c3f526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f526d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6470031ba0 tx=0x7f647003f000 comp rx=0 tx=0).stop 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 -- 172.21.15.46:0/1745973400 shutdown_connections 2024-09-10T07:52:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f50590 0x55e5c3f72ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 --2- 172.21.15.46:0/1745973400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f526d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 -- 172.21.15.46:0/1745973400 >> 172.21.15.46:0/1745973400 conn(0x55e5c3f4c200 msgr2=0x55e5c3f59db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 -- 172.21.15.46:0/1745973400 shutdown_connections 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 -- 172.21.15.46:0/1745973400 wait complete. 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 -- start start 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.084+0000 7f6487dd8740 1 Processor -- start 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f5eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f50590 0x55e5c3f5f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f6487dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f5f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f6487dd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64801b6790 con 0x55e5c3f4fc70 2024-09-10T07:52:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f6487dd8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64801b60c0 con 0x55e5c3f50590 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f6487dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64801b32d0 con 0x55e5c3f5ae50 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f6487dd8740 10 client.0 _ll_register_callbacks cb 0x7f647809b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f647de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f5eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f647d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f50590 0x55e5c3f5f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f647d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f50590 0x55e5c3f5f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:58376/0 (socket says 172.21.15.46:58376) 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f647d400640 1 -- 172.21.15.46:0/2398476813 learned_addr learned my addr 172.21.15.46:0/2398476813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f647de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f5eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32902/0 (socket says 172.21.15.46:32902) 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f647e800640 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f5f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.085+0000 7f647de00640 1 -- 172.21.15.46:0/2398476813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f5ae50 msgr2=0x55e5c3f5f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f647de00640 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f5f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f647de00640 1 -- 172.21.15.46:0/2398476813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f50590 msgr2=0x55e5c3f5f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f647de00640 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f50590 0x55e5c3f5f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f647de00640 1 -- 172.21.15.46:0/2398476813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c3fd4c70 con 0x55e5c3f4fc70 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f647de00640 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f5eac0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6470031a10 tx=0x7f647002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cadbb265123f51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f645fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f645fe00640 1 -- 172.21.15.46:0/2398476813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470054c90 con 0x55e5c3f4fc70 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f645fe00640 1 -- 172.21.15.46:0/2398476813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f64700037f0 con 0x55e5c3f4fc70 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f6487dd8740 1 -- 172.21.15.46:0/2398476813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f647005a070 con 0x55e5c3f4fc70 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f645fe00640 1 -- 172.21.15.46:0/2398476813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6470003a50 con 0x55e5c3f4fc70 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f6487dd8740 10 client.18504 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.086+0000 7f6487dd8740 1 -- 172.21.15.46:0/2398476813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e5c3fd5ed0 con 0x55e5c3f4fc70 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f645fe00640 1 -- 172.21.15.46:0/2398476813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647005a070 con 0x55e5c3f4fc70 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f645fe00640 1 -- 172.21.15.46:0/2398476813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7f6470069030 con 0x55e5c3f4fc70 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f6487dd8740 10 client.18504 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f6487dd8740 10 client.18504 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T07:52:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f6487dd8740 20 client.18504 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e5c3fd6630 0x55e5c3fd8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f6487dd8740 4 client.18504 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:52:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f6487dd8740 1 -- 172.21.15.46:0/2398476813 --> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5c3ebde60 con 0x55e5c3fd6630 2024-09-10T07:52:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.087+0000 7f647d400640 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e5c3fd6630 0x55e5c3fd8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f647d400640 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e5c3fd6630 0x55e5c3fd8a50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e7c888d39dff8786 server_cookie=440e4a317c1a1337 in_seq=0 out_seq=0 2024-09-10T07:52:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f645fe00640 10 client.18504 ms_handle_connect on v2:172.21.15.181:6836/975679519 2024-09-10T07:52:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f645fe00640 1 -- 172.21.15.46:0/2398476813 <== mds.3 v2:172.21.15.181:6836/975679519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5c3ebde60 con 0x55e5c3fd6630 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f645fe00640 10 client.18504 handle_command_reply: tid=0 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 client.18504 shutdown 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 -- 172.21.15.46:0/2398476813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 msgr2=0x55e5c3f5eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f5eac0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6470031a10 tx=0x7f647002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 -- 172.21.15.46:0/2398476813 shutdown_connections 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.181:6836/975679519,v1:172.21.15.181:6837/975679519] conn(0x55e5c3fd6630 0x55e5c3fd8a50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e5c3f5ae50 0x55e5c3f5f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e5c3f50590 0x55e5c3f5f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f645fe00640 0 client.18504 ms_handle_reset on v2:172.21.15.181:6836/975679519 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 --2- 172.21.15.46:0/2398476813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5c3f4fc70 0x55e5c3f5eac0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.088+0000 7f6487dd8740 1 -- 172.21.15.46:0/2398476813 >> 172.21.15.46:0/2398476813 conn(0x55e5c3f4c200 msgr2=0x55e5c3f57cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6487dd8740 1 -- 172.21.15.46:0/2398476813 shutdown_connections 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6487dd8740 1 -- 172.21.15.46:0/2398476813 wait complete. 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6487dd8740 20 client.18504 trim_cache size 0 max 16384 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 -- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64640a2b80 msgr2=0x7f64640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64640a2b80 0x7f64640a5040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f648019fa10 tx=0x7f64700058d0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 -- 172.21.15.46:0/3939529681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6480112a30 msgr2=0x7f648019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6480112a30 0x7f648019e8d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f648007aa10 tx=0x7f647800ef10 comp rx=0 tx=0).stop 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 -- 172.21.15.46:0/3939529681 shutdown_connections 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6480112a30 0x7f648019e8d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f64640a2b80 0x7f64640a5040 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6480077040 0x7f64801a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 --2- 172.21.15.46:0/3939529681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6480076720 0x7f64801a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.089+0000 7f6485200640 1 -- 172.21.15.46:0/3939529681 >> 172.21.15.46:0/3939529681 conn(0x7f64800ffa90 msgr2=0x7f6480101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.090+0000 7f6485200640 1 -- 172.21.15.46:0/3939529681 shutdown_connections 2024-09-10T07:52:54.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:54.090+0000 7f6485200640 1 -- 172.21.15.46:0/3939529681 wait complete. 2024-09-10T07:52:54.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:54 smithi181 ceph-mon[35397]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-09-10T07:52:54.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:54 smithi063 ceph-mon[35020]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-09-10T07:52:54.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:54 smithi046 ceph-mon[30705]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-09-10T07:52:55.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T07:52:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:56.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 -- 172.21.15.46:0/4154942117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8103ff0 msgr2=0x7fbcc81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 --2- 172.21.15.46:0/4154942117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8103ff0 0x7fbcc81043d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009900 tx=0x7fbcbc02a410 comp rx=0 tx=0).stop 2024-09-10T07:52:56.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 -- 172.21.15.46:0/4154942117 shutdown_connections 2024-09-10T07:52:56.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 --2- 172.21.15.46:0/4154942117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcc8109120 0x7fbcc8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 --2- 172.21.15.46:0/4154942117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc81049a0 0x7fbcc81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 --2- 172.21.15.46:0/4154942117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8103ff0 0x7fbcc81043d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 -- 172.21.15.46:0/4154942117 >> 172.21.15.46:0/4154942117 conn(0x7fbcc80ffa90 msgr2=0x7fbcc8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 -- 172.21.15.46:0/4154942117 shutdown_connections 2024-09-10T07:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.131+0000 7fbcd0400640 1 -- 172.21.15.46:0/4154942117 wait complete. 2024-09-10T07:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.132+0000 7fbcd0400640 1 Processor -- start 2024-09-10T07:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcd0400640 1 -- start start 2024-09-10T07:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcd0400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8103ff0 0x7fbcc819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcd0400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcc81049a0 0x7fbcc819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcd0400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8109120 0x7fbcc819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcd0400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8114170 con 0x7fbcc8109120 2024-09-10T07:52:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcd0400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8113ff0 con 0x7fbcc81049a0 2024-09-10T07:52:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcd0400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81142f0 con 0x7fbcc8103ff0 2024-09-10T07:52:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcce400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8109120 0x7fbcc819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbccda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8103ff0 0x7fbcc819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbccd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcc81049a0 0x7fbcc819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcce400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8109120 0x7fbcc819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44316/0 (socket says 172.21.15.46:44316) 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbccda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8103ff0 0x7fbcc819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50292/0 (socket says 172.21.15.46:50292) 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.133+0000 7fbcce400640 1 -- 172.21.15.46:0/1612078350 learned_addr learned my addr 172.21.15.46:0/1612078350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.134+0000 7fbccda00640 1 -- 172.21.15.46:0/1612078350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcc81049a0 msgr2=0x7fbcc819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.134+0000 7fbccda00640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcc81049a0 0x7fbcc819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.134+0000 7fbccda00640 1 -- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8109120 msgr2=0x7fbcc819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.134+0000 7fbccda00640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8109120 0x7fbcc819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.134+0000 7fbccda00640 1 -- 172.21.15.46:0/1612078350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc819f640 con 0x7fbcc8103ff0 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.134+0000 7fbcce400640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8109120 0x7fbcc819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.134+0000 7fbccda00640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8103ff0 0x7fbcc819a690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02a930 tx=0x7fbcbc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.135+0000 7fbcc6000640 1 -- 172.21.15.46:0/1612078350 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc03f070 con 0x7fbcc8103ff0 2024-09-10T07:52:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.135+0000 7fbcd0400640 1 -- 172.21.15.46:0/1612078350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc819f8d0 con 0x7fbcc8103ff0 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.135+0000 7fbcd0400640 1 -- 172.21.15.46:0/1612078350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc81a7210 con 0x7fbcc8103ff0 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.135+0000 7fbcc6000640 1 -- 172.21.15.46:0/1612078350 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcbc031c90 con 0x7fbcc8103ff0 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.135+0000 7fbcc6000640 1 -- 172.21.15.46:0/1612078350 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc049060 con 0x7fbcc8103ff0 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcd302c740 1 Processor -- start 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcd302c740 1 -- start start 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcc6000640 1 -- 172.21.15.46:0/1612078350 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbcbc042070 con 0x7fbcc8103ff0 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e4c70 0x55f2152e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 0x55f2152ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 0x55f2152f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcd302c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f21524b3e0 con 0x55f2152efe50 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcd302c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f2151f2d80 con 0x55f2152e5590 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcd302c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f2152473a0 con 0x55f2152e4c70 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.138+0000 7fbcce400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e4c70 0x55f2152e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbccda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 0x55f2152ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbcce400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e4c70 0x55f2152e5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50296/0 (socket says 172.21.15.46:50296) 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbcce400640 1 -- 172.21.15.46:0/1803449117 learned_addr learned my addr 172.21.15.46:0/1803449117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbccd000640 1 --2- 172.21.15.46:0/1803449117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 0x55f2152f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbcce400640 1 -- 172.21.15.46:0/1803449117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 msgr2=0x55f2152ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbcce400640 1 --2- 172.21.15.46:0/1803449117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 0x55f2152ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbcce400640 1 -- 172.21.15.46:0/1803449117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 msgr2=0x55f2152f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbcce400640 1 --2- 172.21.15.46:0/1803449117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 0x55f2152f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbcce400640 1 -- 172.21.15.46:0/1803449117 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f215211ae0 con 0x55f2152e4c70 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.139+0000 7fbcce400640 1 --2- 172.21.15.46:0/1803449117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e4c70 0x55f2152e5050 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0010810 tx=0x7fbcc0010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be7c23a596231091 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.140+0000 7fbcc6000640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcb00a2c00 0x7fbcb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.140+0000 7fbcbaa00640 1 -- 172.21.15.46:0/1803449117 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000eb30 con 0x55f2152e4c70 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.140+0000 7fbcbaa00640 1 -- 172.21.15.46:0/1803449117 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0015070 con 0x55f2152e4c70 2024-09-10T07:52:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.140+0000 7fbcbaa00640 1 -- 172.21.15.46:0/1803449117 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0011640 con 0x55f2152e4c70 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.140+0000 7fbcc6000640 1 -- 172.21.15.46:0/1612078350 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbcbc0efaa0 con 0x7fbcc8103ff0 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.140+0000 7fbccda00640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcb00a2c00 0x7fbcb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.140+0000 7fbcd302c740 1 -- 172.21.15.46:0/1803449117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e4c70 msgr2=0x55f2152e5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.140+0000 7fbcd302c740 1 --2- 172.21.15.46:0/1803449117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e4c70 0x55f2152e5050 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0010810 tx=0x7fbcc0010cd0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 -- 172.21.15.46:0/1803449117 shutdown_connections 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 --2- 172.21.15.46:0/1803449117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 0x55f2152f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 --2- 172.21.15.46:0/1803449117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 0x55f2152ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 --2- 172.21.15.46:0/1803449117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e4c70 0x55f2152e5050 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 -- 172.21.15.46:0/1803449117 >> 172.21.15.46:0/1803449117 conn(0x55f2152e1200 msgr2=0x55f2152e15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbccda00640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcb00a2c00 0x7fbcb00a50c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0b8430 tx=0x7fbcbc0f34e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 -- 172.21.15.46:0/1803449117 shutdown_connections 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 -- 172.21.15.46:0/1803449117 wait complete. 2024-09-10T07:52:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 -- start start 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.141+0000 7fbcd302c740 1 Processor -- start 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e4c70 0x55f2152eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 0x55f2152ed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 0x55f2152ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcd302c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f2152f2a20 con 0x55f2152e4c70 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcd302c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f2152060f0 con 0x55f2152e5590 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcd302c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55f2152f2ba0 con 0x55f2152efe50 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcd302c740 10 client.0 _ll_register_callbacks cb 0x55f2151edbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcce400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e4c70 0x55f2152eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcce400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e4c70 0x55f2152eccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44336/0 (socket says 172.21.15.46:44336) 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbcce400640 1 -- 172.21.15.46:0/623337171 learned_addr learned my addr 172.21.15.46:0/623337171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.142+0000 7fbccd000640 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 0x55f2152ed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcce400640 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 0x55f2152ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcce400640 1 -- 172.21.15.46:0/623337171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 msgr2=0x55f2152ed750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcce400640 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 0x55f2152ed750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcce400640 1 -- 172.21.15.46:0/623337171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 msgr2=0x55f2152ed210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcce400640 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 0x55f2152ed210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcce400640 1 -- 172.21.15.46:0/623337171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f214fbc3c0 con 0x55f2152e4c70 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcce400640 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e4c70 0x55f2152eccd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009340 tx=0x7fbcc0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ce9d98b65f3c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcaf400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcaf400640 1 -- 172.21.15.46:0/623337171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0004a40 con 0x55f2152e4c70 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcaf400640 1 -- 172.21.15.46:0/623337171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbcc0021070 con 0x55f2152e4c70 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.143+0000 7fbcd302c740 1 -- 172.21.15.46:0/623337171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc0015070 con 0x55f2152e4c70 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcaf400640 1 -- 172.21.15.46:0/623337171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fbcc0032050 con 0x55f2152e4c70 2024-09-10T07:52:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcd302c740 10 client.18516 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcd302c740 1 -- 172.21.15.46:0/623337171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55f2152fea20 con 0x55f2152e4c70 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcaf400640 1 -- 172.21.15.46:0/623337171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0015070 con 0x55f2152e4c70 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcaf400640 1 -- 172.21.15.46:0/623337171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7fbcc00118f0 con 0x55f2152e4c70 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcd302c740 10 client.18516 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcd302c740 10 client.18516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcd302c740 20 client.18516 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcd302c740 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55f2152ffd70 0x55f215302160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcd302c740 4 client.18516 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.144+0000 7fbcd302c740 1 -- 172.21.15.46:0/623337171 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f215258950 con 0x55f2152ffd70 2024-09-10T07:52:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.145+0000 7fbccd000640 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55f2152ffd70 0x55f215302160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.145+0000 7fbccd000640 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55f2152ffd70 0x55f215302160 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb12b5cf86757fbe server_cookie=a55cca5e486663d in_seq=0 out_seq=0 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.145+0000 7fbcaf400640 10 client.18516 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.146+0000 7fbcaf400640 1 -- 172.21.15.46:0/623337171 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f215258950 con 0x55f2152ffd70 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.146+0000 7fbcaf400640 10 client.18516 handle_command_reply: tid=0 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.146+0000 7fbcd302c740 1 client.18516 shutdown 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.146+0000 7fbcd302c740 1 -- 172.21.15.46:0/623337171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e4c70 msgr2=0x55f2152eccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.146+0000 7fbcd302c740 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e4c70 0x55f2152eccd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009340 tx=0x7fbcc0013040 comp rx=0 tx=0).stop 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.146+0000 7fbcd302c740 1 -- 172.21.15.46:0/623337171 shutdown_connections 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcd302c740 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55f2152ffd70 0x55f215302160 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcd302c740 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 0x55f2152ed750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcd302c740 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e5590 0x55f2152ed210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcd302c740 1 --2- 172.21.15.46:0/623337171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e4c70 0x55f2152eccd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcaf400640 0 client.18516 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcd302c740 1 -- 172.21.15.46:0/623337171 >> 172.21.15.46:0/623337171 conn(0x55f2152e1200 msgr2=0x55f2152e6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcd302c740 1 -- 172.21.15.46:0/623337171 shutdown_connections 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcd302c740 1 -- 172.21.15.46:0/623337171 wait complete. 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.147+0000 7fbcd302c740 20 client.18516 trim_cache size 0 max 16384 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbcd302c740 1 Processor -- start 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbcd302c740 1 -- start start 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e5590 0x55f2152ed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 0x55f2153079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbcd302c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81b0e70 con 0x55f2152e5590 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbcd302c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81b2aa0 con 0x55f2152e4c70 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbcd302c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81b1540 con 0x55f2152efe50 2024-09-10T07:52:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.161+0000 7fbccd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e5590 0x55f2152ed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbcce400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbcce400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152ed3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49292/0 (socket says 172.21.15.46:49292) 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbccd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e5590 0x55f2152ed8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44352/0 (socket says 172.21.15.46:44352) 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbcce400640 1 -- 172.21.15.46:0/850325346 learned_addr learned my addr 172.21.15.46:0/850325346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbcce400640 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 0x55f2153079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbccd000640 1 -- 172.21.15.46:0/850325346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 msgr2=0x55f2153079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbccd000640 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 0x55f2153079b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbccd000640 1 -- 172.21.15.46:0/850325346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 msgr2=0x55f2152ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbccd000640 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbccd000640 1 -- 172.21.15.46:0/850325346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2153024d0 con 0x55f2152e5590 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbcce400640 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbccd000640 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e5590 0x55f2152ed8e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x55f2152ee310 tx=0x7fbcb400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a131a520fff572a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.162+0000 7fbcba000640 1 -- 172.21.15.46:0/850325346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4022070 con 0x55f2152e5590 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcba000640 1 -- 172.21.15.46:0/850325346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb4003e40 con 0x55f2152e5590 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcba000640 1 -- 172.21.15.46:0/850325346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4015070 con 0x55f2152e5590 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 -- 172.21.15.46:0/850325346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e5590 msgr2=0x55f2152ed8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e5590 0x55f2152ed8e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x55f2152ee310 tx=0x7fbcb400be00 comp rx=0 tx=0).stop 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 -- 172.21.15.46:0/850325346 shutdown_connections 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152efe50 0x55f2153079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152e5590 0x55f2152ed8e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 --2- 172.21.15.46:0/850325346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 -- 172.21.15.46:0/850325346 >> 172.21.15.46:0/850325346 conn(0x55f2152e1200 msgr2=0x55f2152ecc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 -- 172.21.15.46:0/850325346 shutdown_connections 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.163+0000 7fbcd302c740 1 -- 172.21.15.46:0/850325346 wait complete. 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.164+0000 7fbcd302c740 1 -- start start 2024-09-10T07:52:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.164+0000 7fbcd302c740 1 Processor -- start 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.164+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.164+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e5590 0x55f2152f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.164+0000 7fbcd302c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 0x55f2152f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.164+0000 7fbcd302c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81b2310 con 0x55f2152efe50 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.164+0000 7fbcd302c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81b1c40 con 0x55f2152e4c70 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.164+0000 7fbcd302c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81aee50 con 0x55f2152e5590 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcd302c740 10 client.0 _ll_register_callbacks cb 0x7fbcbc0f33b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152f7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49308/0 (socket says 172.21.15.46:49308) 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 -- 172.21.15.46:0/67056507 learned_addr learned my addr 172.21.15.46:0/67056507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 0x55f2152f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbccd000640 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e5590 0x55f2152f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 -- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e5590 msgr2=0x55f2152f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e5590 0x55f2152f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 -- 172.21.15.46:0/67056507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 msgr2=0x55f2152f4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 0x55f2152f4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbcce400640 1 -- 172.21.15.46:0/67056507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2152eab90 con 0x55f2152e4c70 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.165+0000 7fbccd000640 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e5590 0x55f2152f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.166+0000 7fbcce400640 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152f7e20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0013010 tx=0x7fbcc001adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a6d14457509a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.166+0000 7fbcaf400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.166+0000 7fbcaf400640 1 -- 172.21.15.46:0/67056507 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc001e040 con 0x55f2152e4c70 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.166+0000 7fbcd302c740 1 -- 172.21.15.46:0/67056507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb4003e40 con 0x55f2152e4c70 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.166+0000 7fbcaf400640 1 -- 172.21.15.46:0/67056507 <== mon.2 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fbcc0025040 con 0x55f2152e4c70 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.166+0000 7fbcaf400640 1 -- 172.21.15.46:0/67056507 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fbcc0004a00 con 0x55f2152e4c70 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.166+0000 7fbcd302c740 10 client.29462 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.166+0000 7fbcd302c740 1 -- 172.21.15.46:0/67056507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55f21536ae10 con 0x55f2152e4c70 2024-09-10T07:52:56.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbcaf400640 1 -- 172.21.15.46:0/67056507 <== mon.2 v2:172.21.15.63:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb4003e40 con 0x55f2152e4c70 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbcaf400640 1 -- 172.21.15.46:0/67056507 <== mon.2 v2:172.21.15.63:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7fbcc001a590 con 0x55f2152e4c70 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbcd302c740 10 client.29462 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbcd302c740 10 client.29462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbcd302c740 20 client.29462 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbcd302c740 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55f21536b470 0x55f21536d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbcd302c740 4 client.29462 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbcd302c740 1 -- 172.21.15.46:0/67056507 --> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f215252e60 con 0x55f21536b470 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.167+0000 7fbccd000640 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55f21536b470 0x55f21536d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.168+0000 7fbccd000640 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55f21536b470 0x55f21536d890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a0e3e4e520297c2 server_cookie=eb0f0537a206abca in_seq=0 out_seq=0 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.168+0000 7fbcaf400640 10 client.29462 ms_handle_connect on v2:172.21.15.181:6838/1085095014 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.168+0000 7fbcaf400640 1 -- 172.21.15.46:0/67056507 <== mds.2 v2:172.21.15.181:6838/1085095014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f215252e60 con 0x55f21536b470 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.168+0000 7fbcaf400640 10 client.29462 handle_command_reply: tid=0 2024-09-10T07:52:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.168+0000 7fbcd302c740 1 client.29462 shutdown 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.168+0000 7fbcd302c740 1 -- 172.21.15.46:0/67056507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 msgr2=0x55f2152f7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.168+0000 7fbcd302c740 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152f7e20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0013010 tx=0x7fbcc001adc0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.168+0000 7fbcd302c740 1 -- 172.21.15.46:0/67056507 shutdown_connections 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd302c740 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:6838/1085095014,v1:172.21.15.181:6839/1085095014] conn(0x55f21536b470 0x55f21536d890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd302c740 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2152efe50 0x55f2152f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd302c740 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2152e5590 0x55f2152f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd302c740 1 --2- 172.21.15.46:0/67056507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2152e4c70 0x55f2152f7e20 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcaf400640 0 client.29462 ms_handle_reset on v2:172.21.15.181:6838/1085095014 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd302c740 1 -- 172.21.15.46:0/67056507 >> 172.21.15.46:0/67056507 conn(0x55f2152e1200 msgr2=0x55f2152ed4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd302c740 1 -- 172.21.15.46:0/67056507 shutdown_connections 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd302c740 1 -- 172.21.15.46:0/67056507 wait complete. 2024-09-10T07:52:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd302c740 20 client.29462 trim_cache size 0 max 16384 2024-09-10T07:52:56.277 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:52:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd0400640 1 -- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcb00a2c00 msgr2=0x7fbcb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd0400640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcb00a2c00 0x7fbcb00a50c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0b8430 tx=0x7fbcbc0f34e0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd0400640 1 -- 172.21.15.46:0/1612078350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8103ff0 msgr2=0x7fbcc819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.169+0000 7fbcd0400640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8103ff0 0x7fbcc819a690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02a930 tx=0x7fbcbc003b90 comp rx=0 tx=0).stop 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.170+0000 7fbcd0400640 1 -- 172.21.15.46:0/1612078350 shutdown_connections 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.170+0000 7fbcd0400640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fbcb00a2c00 0x7fbcb00a50c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.170+0000 7fbcd0400640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcc8109120 0x7fbcc819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.170+0000 7fbcd0400640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcc81049a0 0x7fbcc819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.170+0000 7fbcd0400640 1 --2- 172.21.15.46:0/1612078350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8103ff0 0x7fbcc819a690 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.170+0000 7fbcd0400640 1 -- 172.21.15.46:0/1612078350 >> 172.21.15.46:0/1612078350 conn(0x7fbcc80ffa90 msgr2=0x7fbcc8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.170+0000 7fbcd0400640 1 -- 172.21.15.46:0/1612078350 shutdown_connections 2024-09-10T07:52:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:56.170+0000 7fbcd0400640 1 -- 172.21.15.46:0/1612078350 wait complete. 2024-09-10T07:52:56.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:56 smithi181 ceph-mon[35397]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 757 KiB/s wr, 108 op/s 2024-09-10T07:52:56.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:56 smithi063 ceph-mon[35020]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 757 KiB/s wr, 108 op/s 2024-09-10T07:52:56.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:56 smithi046 ceph-mon[30705]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 757 KiB/s wr, 108 op/s 2024-09-10T07:52:57.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T07:52:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:52:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 -- 172.21.15.46:0/2160731584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1101049a0 msgr2=0x7fa1101089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 --2- 172.21.15.46:0/2160731584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1101049a0 0x7fa1101089f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009900 tx=0x7fa0fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 -- 172.21.15.46:0/2160731584 shutdown_connections 2024-09-10T07:52:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 --2- 172.21.15.46:0/2160731584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa110109120 0x7fa110110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 --2- 172.21.15.46:0/2160731584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1101049a0 0x7fa1101089f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 --2- 172.21.15.46:0/2160731584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa110103ff0 0x7fa1101043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 -- 172.21.15.46:0/2160731584 >> 172.21.15.46:0/2160731584 conn(0x7fa1100ffa90 msgr2=0x7fa110101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 -- 172.21.15.46:0/2160731584 shutdown_connections 2024-09-10T07:52:58.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.170+0000 7fa118200640 1 -- 172.21.15.46:0/2160731584 wait complete. 2024-09-10T07:52:58.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.171+0000 7fa118200640 1 Processor -- start 2024-09-10T07:52:58.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa118200640 1 -- start start 2024-09-10T07:52:58.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa118200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa110103ff0 0x7fa110076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa118200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1101049a0 0x7fa110076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa118200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa110109120 0x7fa110077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa118200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa110114170 con 0x7fa110109120 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa118200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa110113ff0 con 0x7fa1101049a0 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa118200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101142f0 con 0x7fa110103ff0 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa116200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa110109120 0x7fa110077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa116200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa110109120 0x7fa110077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44396/0 (socket says 172.21.15.46:44396) 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa116200640 1 -- 172.21.15.46:0/2292502800 learned_addr learned my addr 172.21.15.46:0/2292502800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa115800640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa110103ff0 0x7fa110076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.172+0000 7fa114e00640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1101049a0 0x7fa110076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.173+0000 7fa116200640 1 -- 172.21.15.46:0/2292502800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa110103ff0 msgr2=0x7fa110076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.173+0000 7fa116200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa110103ff0 0x7fa110076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.173+0000 7fa116200640 1 -- 172.21.15.46:0/2292502800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1101049a0 msgr2=0x7fa110076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.173+0000 7fa116200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1101049a0 0x7fa110076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.173+0000 7fa116200640 1 -- 172.21.15.46:0/2292502800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1101aacf0 con 0x7fa110109120 2024-09-10T07:52:58.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.173+0000 7fa116200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa110109120 0x7fa110077180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa10800ead0 tx=0x7fa10800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.173+0000 7fa10e000640 1 -- 172.21.15.46:0/2292502800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10800cde0 con 0x7fa110109120 2024-09-10T07:52:58.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.174+0000 7fa10e000640 1 -- 172.21.15.46:0/2292502800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa108005020 con 0x7fa110109120 2024-09-10T07:52:58.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.174+0000 7fa118200640 1 -- 172.21.15.46:0/2292502800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1101aafe0 con 0x7fa110109120 2024-09-10T07:52:58.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.174+0000 7fa10e000640 1 -- 172.21.15.46:0/2292502800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1080116d0 con 0x7fa110109120 2024-09-10T07:52:58.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.174+0000 7fa118200640 1 -- 172.21.15.46:0/2292502800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1101ab4a0 con 0x7fa110109120 2024-09-10T07:52:58.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.178+0000 7fa11adfb740 1 Processor -- start 2024-09-10T07:52:58.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.178+0000 7fa10e000640 1 -- 172.21.15.46:0/2292502800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa108005560 con 0x7fa110109120 2024-09-10T07:52:58.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.178+0000 7fa11adfb740 1 -- start start 2024-09-10T07:52:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.178+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a28050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a32910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a35240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa11adfb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56132598e3e0 con 0x561325a32e50 2024-09-10T07:52:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa11adfb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561325935d80 con 0x561325a28590 2024-09-10T07:52:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa11adfb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56132598a3a0 con 0x561325a27c70 2024-09-10T07:52:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa115800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a32910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa114e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a35240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa115800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a32910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49320/0 (socket says 172.21.15.46:49320) 2024-09-10T07:52:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa115800640 1 -- 172.21.15.46:0/2180688092 learned_addr learned my addr 172.21.15.46:0/2180688092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:58.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa114e00640 1 --2- 172.21.15.46:0/2180688092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a28050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa115800640 1 -- 172.21.15.46:0/2180688092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 msgr2=0x561325a28050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa115800640 1 --2- 172.21.15.46:0/2180688092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a28050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa115800640 1 -- 172.21.15.46:0/2180688092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 msgr2=0x561325a35240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa115800640 1 --2- 172.21.15.46:0/2180688092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a35240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.179+0000 7fa115800640 1 -- 172.21.15.46:0/2180688092 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561325954ae0 con 0x561325a28590 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.180+0000 7fa115800640 1 --2- 172.21.15.46:0/2180688092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a32910 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa104009cf0 tx=0x7fa104011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9008366fd63d6ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.180+0000 7fa102a00640 1 -- 172.21.15.46:0/2180688092 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104004230 con 0x561325a28590 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.180+0000 7fa102a00640 1 -- 172.21.15.46:0/2180688092 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1040043d0 con 0x561325a28590 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.180+0000 7fa102a00640 1 -- 172.21.15.46:0/2180688092 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104019840 con 0x561325a28590 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.180+0000 7fa10e000640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0f80a2c00 0x7fa0f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.180+0000 7fa10e000640 1 -- 172.21.15.46:0/2292502800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa1080cf540 con 0x7fa110109120 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa114e00640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0f80a2c00 0x7fa0f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 -- 172.21.15.46:0/2180688092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 msgr2=0x561325a32910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2180688092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a32910 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa104009cf0 tx=0x7fa104011810 comp rx=0 tx=0).stop 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 -- 172.21.15.46:0/2180688092 shutdown_connections 2024-09-10T07:52:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2180688092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a35240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2180688092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a32910 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2180688092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a28050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 -- 172.21.15.46:0/2180688092 >> 172.21.15.46:0/2180688092 conn(0x561325a24200 msgr2=0x561325a245e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 -- 172.21.15.46:0/2180688092 shutdown_connections 2024-09-10T07:52:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa11adfb740 1 -- 172.21.15.46:0/2180688092 wait complete. 2024-09-10T07:52:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.181+0000 7fa114e00640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0f80a2c00 0x7fa0f80a50c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc000c00 tx=0x7fa0fc041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.182+0000 7fa11adfb740 1 -- start start 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.182+0000 7fa11adfb740 1 Processor -- start 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.182+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a2fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.182+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a28590 0x561325a30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.182+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a32e50 0x561325a30740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.182+0000 7fa11adfb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561325a35a20 con 0x561325a28590 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa11adfb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5613259490f0 con 0x561325a32e50 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa11adfb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561325a35ba0 con 0x561325a27c70 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa11adfb740 10 client.0 _ll_register_callbacks cb 0x561325930bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa114e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a28590 0x561325a30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa114e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a28590 0x561325a30200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44410/0 (socket says 172.21.15.46:44410) 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa114e00640 1 -- 172.21.15.46:0/2208584776 learned_addr learned my addr 172.21.15.46:0/2208584776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa115800640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a2fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa116200640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a32e50 0x561325a30740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa115800640 1 -- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a32e50 msgr2=0x561325a30740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa115800640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a32e50 0x561325a30740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa115800640 1 -- 172.21.15.46:0/2208584776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a28590 msgr2=0x561325a30200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa115800640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a28590 0x561325a30200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa115800640 1 -- 172.21.15.46:0/2208584776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613256ff3c0 con 0x561325a27c70 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa114e00640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a28590 0x561325a30200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.183+0000 7fa116200640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a32e50 0x561325a30740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.184+0000 7fa115800640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a2fcc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa104011d10 tx=0x7fa104003880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0767b08e36ad51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.184+0000 7fa0f7400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.184+0000 7fa0f7400640 1 -- 172.21.15.46:0/2208584776 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104027070 con 0x561325a27c70 2024-09-10T07:52:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.184+0000 7fa11adfb740 1 -- 172.21.15.46:0/2208584776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1040043d0 con 0x561325a27c70 2024-09-10T07:52:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.184+0000 7fa0f7400640 1 -- 172.21.15.46:0/2208584776 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa104005240 con 0x561325a27c70 2024-09-10T07:52:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.185+0000 7fa0f7400640 1 -- 172.21.15.46:0/2208584776 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa104033050 con 0x561325a27c70 2024-09-10T07:52:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.185+0000 7fa11adfb740 10 client.29365 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.185+0000 7fa11adfb740 1 -- 172.21.15.46:0/2208584776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561325a419e0 con 0x561325a27c70 2024-09-10T07:52:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.185+0000 7fa0f7400640 1 -- 172.21.15.46:0/2208584776 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1040043d0 con 0x561325a27c70 2024-09-10T07:52:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.186+0000 7fa0f7400640 1 -- 172.21.15.46:0/2208584776 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7fa1040195a0 con 0x561325a27c70 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.186+0000 7fa11adfb740 10 client.29365 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.186+0000 7fa11adfb740 10 client.29365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.186+0000 7fa11adfb740 20 client.29365 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.186+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x561325a42df0 0x561325a451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.186+0000 7fa11adfb740 4 client.29365 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.186+0000 7fa11adfb740 1 -- 172.21.15.46:0/2208584776 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56132599b950 con 0x561325a42df0 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.186+0000 7fa114e00640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x561325a42df0 0x561325a451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.187+0000 7fa114e00640 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x561325a42df0 0x561325a451e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1279f26fbe4fb8fb server_cookie=269f79ab455c5758 in_seq=0 out_seq=0 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.187+0000 7fa0f7400640 10 client.29365 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa0f7400640 1 -- 172.21.15.46:0/2208584776 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56132599b950 con 0x561325a42df0 2024-09-10T07:52:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa0f7400640 10 client.29365 handle_command_reply: tid=0 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 client.29365 shutdown 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 -- 172.21.15.46:0/2208584776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 msgr2=0x561325a2fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a2fcc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa104011d10 tx=0x7fa104003880 comp rx=0 tx=0).stop 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 -- 172.21.15.46:0/2208584776 shutdown_connections 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x561325a42df0 0x561325a451e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a32e50 0x561325a30740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a28590 0x561325a30200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 --2- 172.21.15.46:0/2208584776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a27c70 0x561325a2fcc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa0f7400640 0 client.29365 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 -- 172.21.15.46:0/2208584776 >> 172.21.15.46:0/2208584776 conn(0x561325a24200 msgr2=0x561325a23380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 -- 172.21.15.46:0/2208584776 shutdown_connections 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 1 -- 172.21.15.46:0/2208584776 wait complete. 2024-09-10T07:52:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.188+0000 7fa11adfb740 20 client.29365 trim_cache size 0 max 16384 2024-09-10T07:52:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.202+0000 7fa11adfb740 1 Processor -- start 2024-09-10T07:52:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.202+0000 7fa11adfb740 1 -- start start 2024-09-10T07:52:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a27c70 0x561325a31ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a4aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a32e50 0x561325a4af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa11adfb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101b51e0 con 0x561325a27c70 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa11adfb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101b6e10 con 0x561325a28590 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa11adfb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101b58b0 con 0x561325a32e50 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa115800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a27c70 0x561325a31ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa115800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a27c70 0x561325a31ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44420/0 (socket says 172.21.15.46:44420) 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa115800640 1 -- 172.21.15.46:0/4241898753 learned_addr learned my addr 172.21.15.46:0/4241898753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa114e00640 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a4aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa116200640 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a32e50 0x561325a4af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa115800640 1 -- 172.21.15.46:0/4241898753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a32e50 msgr2=0x561325a4af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa115800640 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a32e50 0x561325a4af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa115800640 1 -- 172.21.15.46:0/4241898753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 msgr2=0x561325a4aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa115800640 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a4aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa115800640 1 -- 172.21.15.46:0/4241898753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561325a1f770 con 0x561325a27c70 2024-09-10T07:52:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.203+0000 7fa114e00640 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a4aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.204+0000 7fa115800640 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a27c70 0x561325a31ff0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa104005fd0 tx=0x7fa104005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13f2e4090f12325 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.204+0000 7fa102000640 1 -- 172.21.15.46:0/4241898753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104030070 con 0x561325a27c70 2024-09-10T07:52:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.204+0000 7fa102000640 1 -- 172.21.15.46:0/4241898753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104009b70 con 0x561325a27c70 2024-09-10T07:52:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.204+0000 7fa102000640 1 -- 172.21.15.46:0/4241898753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104027070 con 0x561325a27c70 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 -- 172.21.15.46:0/4241898753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a27c70 msgr2=0x561325a31ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a27c70 0x561325a31ff0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa104005fd0 tx=0x7fa104005d30 comp rx=0 tx=0).stop 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 -- 172.21.15.46:0/4241898753 shutdown_connections 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a32e50 0x561325a4af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a28590 0x561325a4aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4241898753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a27c70 0x561325a31ff0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 -- 172.21.15.46:0/4241898753 >> 172.21.15.46:0/4241898753 conn(0x561325a24200 msgr2=0x561325a23380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 -- 172.21.15.46:0/4241898753 shutdown_connections 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 -- 172.21.15.46:0/4241898753 wait complete. 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 -- start start 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.205+0000 7fa11adfb740 1 Processor -- start 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a27c70 0x561325a3aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a28590 0x561325a3b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa11adfb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa11adfb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101b6680 con 0x561325a32e50 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa11adfb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101b5fb0 con 0x561325a27c70 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa11adfb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101b31c0 con 0x561325a28590 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa11adfb740 10 client.0 _ll_register_callbacks cb 0x7fa0fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa115800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a27c70 0x561325a3aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa116200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa114e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a28590 0x561325a3b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa116200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a36350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44426/0 (socket says 172.21.15.46:44426) 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa115800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a27c70 0x561325a3aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49350/0 (socket says 172.21.15.46:49350) 2024-09-10T07:52:58.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.206+0000 7fa116200640 1 -- 172.21.15.46:0/4202310649 learned_addr learned my addr 172.21.15.46:0/4202310649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:52:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa114e00640 1 -- 172.21.15.46:0/4202310649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a27c70 msgr2=0x561325a3aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa114e00640 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a27c70 0x561325a3aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa114e00640 1 -- 172.21.15.46:0/4202310649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 msgr2=0x561325a36350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa114e00640 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a36350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa114e00640 1 -- 172.21.15.46:0/4202310649 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561325a2aae0 con 0x561325a28590 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa116200640 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa114e00640 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a28590 0x561325a3b3f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x561325a31270 tx=0x7fa0fc02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=859bb0934a5e9815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa0f7400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa0f7400640 1 -- 172.21.15.46:0/4202310649 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc046070 con 0x561325a28590 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa11adfb740 1 -- 172.21.15.46:0/4202310649 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa104009b70 con 0x561325a28590 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.207+0000 7fa0f7400640 1 -- 172.21.15.46:0/4202310649 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fa0fc003a00 con 0x561325a28590 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.208+0000 7fa0f7400640 1 -- 172.21.15.46:0/4202310649 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa0fc068050 con 0x561325a28590 2024-09-10T07:52:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.208+0000 7fa11adfb740 10 client.29371 fetch_fsmap learned FSMap version 61 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.208+0000 7fa11adfb740 1 -- 172.21.15.46:0/4202310649 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561325aad310 con 0x561325a28590 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.208+0000 7fa0f7400640 1 -- 172.21.15.46:0/4202310649 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104009b70 con 0x561325a28590 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.208+0000 7fa0f7400640 1 -- 172.21.15.46:0/4202310649 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7fa0fc0325c0 con 0x561325a28590 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa11adfb740 10 client.29371 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa11adfb740 10 client.29371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa11adfb740 20 client.29371 populate_metadata read hostname 'smithi046' 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x561325aada60 0x561325aafe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa11adfb740 4 client.29371 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa11adfb740 1 -- 172.21.15.46:0/4202310649 --> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561325995e60 con 0x561325aada60 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa115800640 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x561325aada60 0x561325aafe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa115800640 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x561325aada60 0x561325aafe80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3ea6c67773f1228 server_cookie=ab7576622b8b1b64 in_seq=0 out_seq=0 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.209+0000 7fa0f7400640 10 client.29371 ms_handle_connect on v2:172.21.15.63:6838/1361023409 2024-09-10T07:52:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa0f7400640 1 -- 172.21.15.46:0/4202310649 <== mds.1 v2:172.21.15.63:6838/1361023409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561325995e60 con 0x561325aada60 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa0f7400640 10 client.29371 handle_command_reply: tid=0 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 client.29371 shutdown 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 -- 172.21.15.46:0/4202310649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a28590 msgr2=0x561325a3b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a28590 0x561325a3b3f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x561325a31270 tx=0x7fa0fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 -- 172.21.15.46:0/4202310649 shutdown_connections 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.63:6838/1361023409,v1:172.21.15.63:6839/1361023409] conn(0x561325aada60 0x561325aafe80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561325a32e50 0x561325a36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561325a28590 0x561325a3b3f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 --2- 172.21.15.46:0/4202310649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561325a27c70 0x561325a3aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa0f7400640 0 client.29371 ms_handle_reset on v2:172.21.15.63:6838/1361023409 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 -- 172.21.15.46:0/4202310649 >> 172.21.15.46:0/4202310649 conn(0x561325a24200 msgr2=0x561325a23380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 -- 172.21.15.46:0/4202310649 shutdown_connections 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 1 -- 172.21.15.46:0/4202310649 wait complete. 2024-09-10T07:52:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.210+0000 7fa11adfb740 20 client.29371 trim_cache size 0 max 16384 2024-09-10T07:52:58.213 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:52:58.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 -- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0f80a2c00 msgr2=0x7fa0f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0f80a2c00 0x7fa0f80a50c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc000c00 tx=0x7fa0fc041040 comp rx=0 tx=0).stop 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 -- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa110109120 msgr2=0x7fa110077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa110109120 0x7fa110077180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa10800ead0 tx=0x7fa10800ef90 comp rx=0 tx=0).stop 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 -- 172.21.15.46:0/2292502800 shutdown_connections 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fa0f80a2c00 0x7fa0f80a50c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa110109120 0x7fa110077180 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1101049a0 0x7fa110076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 --2- 172.21.15.46:0/2292502800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa110103ff0 0x7fa110076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 -- 172.21.15.46:0/2292502800 >> 172.21.15.46:0/2292502800 conn(0x7fa1100ffa90 msgr2=0x7fa110100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 -- 172.21.15.46:0/2292502800 shutdown_connections 2024-09-10T07:52:58.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:52:58.211+0000 7fa118200640 1 -- 172.21.15.46:0/2292502800 wait complete. 2024-09-10T07:52:58.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:52:58 smithi046 ceph-mon[30705]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 438 KiB/s wr, 69 op/s 2024-09-10T07:52:58.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:52:58 smithi181 ceph-mon[35397]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 438 KiB/s wr, 69 op/s 2024-09-10T07:52:58.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:52:58 smithi063 ceph-mon[35020]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 438 KiB/s wr, 69 op/s 2024-09-10T07:52:59.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T07:52:59.386 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:00.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 -- 172.21.15.46:0/4291431482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b701079c0 msgr2=0x7f3b70107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 --2- 172.21.15.46:0/4291431482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b701079c0 0x7f3b70107da0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009930 tx=0x7f3b6402a420 comp rx=0 tx=0).stop 2024-09-10T07:53:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 -- 172.21.15.46:0/4291431482 shutdown_connections 2024-09-10T07:53:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 --2- 172.21.15.46:0/4291431482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b70102360 0x7f3b7010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 --2- 172.21.15.46:0/4291431482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b701019c0 0x7f3b70101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 --2- 172.21.15.46:0/4291431482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b701079c0 0x7f3b70107da0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 -- 172.21.15.46:0/4291431482 >> 172.21.15.46:0/4291431482 conn(0x7f3b700fd740 msgr2=0x7f3b700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 -- 172.21.15.46:0/4291431482 shutdown_connections 2024-09-10T07:53:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.032+0000 7f3b76a00640 1 -- 172.21.15.46:0/4291431482 wait complete. 2024-09-10T07:53:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.033+0000 7f3b76a00640 1 Processor -- start 2024-09-10T07:53:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.033+0000 7f3b76a00640 1 -- start start 2024-09-10T07:53:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.033+0000 7f3b76a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b701019c0 0x7f3b7019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.033+0000 7f3b76a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b70102360 0x7f3b7019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.033+0000 7f3b76a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b701079c0 0x7f3b7019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.033+0000 7f3b76a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70111dc0 con 0x7f3b70102360 2024-09-10T07:53:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.033+0000 7f3b76a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70111c40 con 0x7f3b701079c0 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.033+0000 7f3b76a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70111f40 con 0x7f3b701019c0 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b70102360 0x7f3b7019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b75600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b701019c0 0x7f3b7019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b70102360 0x7f3b7019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44450/0 (socket says 172.21.15.46:44450) 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b74c00640 1 -- 172.21.15.46:0/2937119775 learned_addr learned my addr 172.21.15.46:0/2937119775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b76000640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b701079c0 0x7f3b7019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b75600640 1 -- 172.21.15.46:0/2937119775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b701079c0 msgr2=0x7f3b7019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b75600640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b701079c0 0x7f3b7019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b75600640 1 -- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b70102360 msgr2=0x7f3b7019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b75600640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b70102360 0x7f3b7019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b75600640 1 -- 172.21.15.46:0/2937119775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7019f4e0 con 0x7f3b701019c0 2024-09-10T07:53:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.034+0000 7f3b74c00640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b70102360 0x7f3b7019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.035+0000 7f3b75600640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b701019c0 0x7f3b7019a530 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3b64005940 tx=0x7f3b640040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.035+0000 7f3b6e000640 1 -- 172.21.15.46:0/2937119775 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6403f070 con 0x7f3b701019c0 2024-09-10T07:53:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.035+0000 7f3b76a00640 1 -- 172.21.15.46:0/2937119775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7019f770 con 0x7f3b701019c0 2024-09-10T07:53:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.035+0000 7f3b6e000640 1 -- 172.21.15.46:0/2937119775 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b640044c0 con 0x7f3b701019c0 2024-09-10T07:53:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.035+0000 7f3b6e000640 1 -- 172.21.15.46:0/2937119775 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64035070 con 0x7f3b701019c0 2024-09-10T07:53:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.035+0000 7f3b76a00640 1 -- 172.21.15.46:0/2937119775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701a70b0 con 0x7f3b701019c0 2024-09-10T07:53:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b7abbf740 1 Processor -- start 2024-09-10T07:53:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b7abbf740 1 -- start start 2024-09-10T07:53:00.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d1e40 0x56152f8d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d2760 0x56152f8dcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 0x56152f8df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56152f8383e0 con 0x56152f8d2760 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56152f7dfd80 con 0x56152f8dd020 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56152f8343a0 con 0x56152f8d1e40 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.038+0000 7f3b6e000640 1 -- 172.21.15.46:0/2937119775 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b6403b460 con 0x7f3b701019c0 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b76000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d1e40 0x56152f8d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 0x56152f8df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b76000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d1e40 0x56152f8d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50420/0 (socket says 172.21.15.46:50420) 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b76000640 1 -- 172.21.15.46:0/1678798757 learned_addr learned my addr 172.21.15.46:0/1678798757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:00.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b75600640 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d2760 0x56152f8dcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b76000640 1 -- 172.21.15.46:0/1678798757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 msgr2=0x56152f8df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b76000640 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 0x56152f8df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b76000640 1 -- 172.21.15.46:0/1678798757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d2760 msgr2=0x56152f8dcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b76000640 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d2760 0x56152f8dcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b76000640 1 -- 172.21.15.46:0/1678798757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152f7feae0 con 0x56152f8d1e40 2024-09-10T07:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.039+0000 7f3b75600640 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d2760 0x56152f8dcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:00.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.040+0000 7f3b76000640 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d1e40 0x56152f8d2220 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009f90 tx=0x7f3b68010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2fdfb0814e36349 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.040+0000 7f3b62600640 1 -- 172.21.15.46:0/1678798757 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68011700 con 0x56152f8d1e40 2024-09-10T07:53:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.040+0000 7f3b62600640 1 -- 172.21.15.46:0/1678798757 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b68011d80 con 0x56152f8d1e40 2024-09-10T07:53:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.040+0000 7f3b62600640 1 -- 172.21.15.46:0/1678798757 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68018380 con 0x56152f8d1e40 2024-09-10T07:53:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.040+0000 7f3b6e000640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b580a2b80 0x7f3b580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.040+0000 7f3b6e000640 1 -- 172.21.15.46:0/2937119775 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b640ef890 con 0x7f3b701019c0 2024-09-10T07:53:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.040+0000 7f3b75600640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b580a2b80 0x7f3b580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 -- 172.21.15.46:0/1678798757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d1e40 msgr2=0x56152f8d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d1e40 0x56152f8d2220 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009f90 tx=0x7f3b68010860 comp rx=0 tx=0).stop 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 -- 172.21.15.46:0/1678798757 shutdown_connections 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 0x56152f8df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d2760 0x56152f8dcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/1678798757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d1e40 0x56152f8d2220 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 -- 172.21.15.46:0/1678798757 >> 172.21.15.46:0/1678798757 conn(0x56152f8ce3d0 msgr2=0x56152f8ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 -- 172.21.15.46:0/1678798757 shutdown_connections 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 -- 172.21.15.46:0/1678798757 wait complete. 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b75600640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b580a2b80 0x7f3b580a5040 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3b640b8220 tx=0x7f3b6402a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.041+0000 7f3b7abbf740 1 -- start start 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b7abbf740 1 Processor -- start 2024-09-10T07:53:00.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d1e40 0x56152f8d9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8da380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8dd020 0x56152f8da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56152f8dfbf0 con 0x56152f8dd020 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56152f7f30f0 con 0x56152f8d1e40 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56152f8dfd70 con 0x56152f8d2760 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b7abbf740 10 client.0 _ll_register_callbacks cb 0x56152f7dabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8da380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8da380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50432/0 (socket says 172.21.15.46:50432) 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b74c00640 1 -- 172.21.15.46:0/322067500 learned_addr learned my addr 172.21.15.46:0/322067500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.042+0000 7f3b76000640 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8dd020 0x56152f8da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.043+0000 7f3b76000640 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d1e40 0x56152f8d9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.043+0000 7f3b74c00640 1 -- 172.21.15.46:0/322067500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d1e40 msgr2=0x56152f8d9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.043+0000 7f3b74c00640 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d1e40 0x56152f8d9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.043+0000 7f3b74c00640 1 -- 172.21.15.46:0/322067500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8dd020 msgr2=0x56152f8da8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.043+0000 7f3b74c00640 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8dd020 0x56152f8da8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.043+0000 7f3b74c00640 1 -- 172.21.15.46:0/322067500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56152f5a93c0 con 0x56152f8d2760 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.043+0000 7f3b74c00640 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8da380 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c012d20 tx=0x7f3b5c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e315472f881960 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.043+0000 7f3b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.044+0000 7f3b56a00640 1 -- 172.21.15.46:0/322067500 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c005020 con 0x56152f8d2760 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.044+0000 7f3b56a00640 1 -- 172.21.15.46:0/322067500 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b5c0051c0 con 0x56152f8d2760 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.044+0000 7f3b7abbf740 1 -- 172.21.15.46:0/322067500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b68011d80 con 0x56152f8d2760 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.044+0000 7f3b56a00640 1 -- 172.21.15.46:0/322067500 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3b5c0053c0 con 0x56152f8d2760 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.044+0000 7f3b7abbf740 10 client.29383 fetch_fsmap learned FSMap version 61 2024-09-10T07:53:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.044+0000 7f3b7abbf740 1 -- 172.21.15.46:0/322067500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56152f8ebc20 con 0x56152f8d2760 2024-09-10T07:53:00.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.044+0000 7f3b56a00640 1 -- 172.21.15.46:0/322067500 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b68011d80 con 0x56152f8d2760 2024-09-10T07:53:00.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.044+0000 7f3b56a00640 1 -- 172.21.15.46:0/322067500 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7f3b5c01c470 con 0x56152f8d2760 2024-09-10T07:53:00.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.045+0000 7f3b7abbf740 10 client.29383 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.045+0000 7f3b7abbf740 10 client.29383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.045+0000 7f3b7abbf740 20 client.29383 populate_metadata read hostname 'smithi046' 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.045+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56152f8ed180 0x56152f8ef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.045+0000 7f3b7abbf740 4 client.29383 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.045+0000 7f3b7abbf740 1 -- 172.21.15.46:0/322067500 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56152f845950 con 0x56152f8ed180 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.045+0000 7f3b76000640 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56152f8ed180 0x56152f8ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.046+0000 7f3b76000640 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56152f8ed180 0x56152f8ef570 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9295abf0f91449 server_cookie=97738e7d6953cfac in_seq=0 out_seq=0 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.046+0000 7f3b56a00640 10 client.29383 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.046+0000 7f3b56a00640 1 -- 172.21.15.46:0/322067500 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56152f845950 con 0x56152f8ed180 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.046+0000 7f3b56a00640 10 client.29383 handle_command_reply: tid=0 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.046+0000 7f3b7abbf740 1 client.29383 shutdown 2024-09-10T07:53:00.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 -- 172.21.15.46:0/322067500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 msgr2=0x56152f8da380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8da380 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c012d20 tx=0x7f3b5c00f790 comp rx=0 tx=0).stop 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 -- 172.21.15.46:0/322067500 shutdown_connections 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56152f8ed180 0x56152f8ef570 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8dd020 0x56152f8da8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8da380 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/322067500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d1e40 0x56152f8d9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b56a00640 0 client.29383 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 -- 172.21.15.46:0/322067500 >> 172.21.15.46:0/322067500 conn(0x56152f8ce3d0 msgr2=0x56152f8d4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 -- 172.21.15.46:0/322067500 shutdown_connections 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 1 -- 172.21.15.46:0/322067500 wait complete. 2024-09-10T07:53:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.047+0000 7f3b7abbf740 20 client.29383 trim_cache size 0 max 16384 2024-09-10T07:53:00.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.061+0000 7f3b7abbf740 1 Processor -- start 2024-09-10T07:53:00.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.061+0000 7f3b7abbf740 1 -- start start 2024-09-10T07:53:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.061+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 0x56152f8ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.061+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d2760 0x56152f8da710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.061+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8dd020 0x56152f8f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.061+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701b0d90 con 0x56152f8d1e40 2024-09-10T07:53:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.061+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701b29c0 con 0x56152f8d2760 2024-09-10T07:53:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.061+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701b1460 con 0x56152f8dd020 2024-09-10T07:53:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d2760 0x56152f8da710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d2760 0x56152f8da710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49378/0 (socket says 172.21.15.46:49378) 2024-09-10T07:53:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 0x56152f8ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 -- 172.21.15.46:0/3280440950 learned_addr learned my addr 172.21.15.46:0/3280440950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 -- 172.21.15.46:0/3280440950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8dd020 msgr2=0x56152f8f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 --2- 172.21.15.46:0/3280440950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8dd020 0x56152f8f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 -- 172.21.15.46:0/3280440950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 msgr2=0x56152f8ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 --2- 172.21.15.46:0/3280440950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 0x56152f8ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 -- 172.21.15.46:0/3280440950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152f8ce200 con 0x56152f8d2760 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.062+0000 7f3b74c00640 1 --2- 172.21.15.46:0/3280440950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d2760 0x56152f8da710 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c01c890 tx=0x7f3b5c00fee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60989fe8062ed93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b61c00640 1 -- 172.21.15.46:0/3280440950 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c02e070 con 0x56152f8d2760 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b61c00640 1 -- 172.21.15.46:0/3280440950 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c005560 con 0x56152f8d2760 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b61c00640 1 -- 172.21.15.46:0/3280440950 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c029920 con 0x56152f8d2760 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3280440950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d2760 msgr2=0x56152f8da710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3280440950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d2760 0x56152f8da710 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c01c890 tx=0x7f3b5c00fee0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3280440950 shutdown_connections 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3280440950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8dd020 0x56152f8f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3280440950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8d2760 0x56152f8da710 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3280440950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 0x56152f8ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.063+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3280440950 >> 172.21.15.46:0/3280440950 conn(0x56152f8ce3d0 msgr2=0x56152f8dbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.064+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3280440950 shutdown_connections 2024-09-10T07:53:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.064+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3280440950 wait complete. 2024-09-10T07:53:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.064+0000 7f3b7abbf740 1 -- start start 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.064+0000 7f3b7abbf740 1 Processor -- start 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 0x56152f8e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8e0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b7abbf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 0x56152f8e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701b2230 con 0x56152f8d1e40 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701b1b60 con 0x56152f8dd020 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b7abbf740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701aed70 con 0x56152f8d2760 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b7abbf740 10 client.0 _ll_register_callbacks cb 0x7f3b5c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8e0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 0x56152f8e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b74c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8e0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50460/0 (socket says 172.21.15.46:50460) 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b74c00640 1 -- 172.21.15.46:0/3641877314 learned_addr learned my addr 172.21.15.46:0/3641877314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b76000640 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 0x56152f8e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b74c00640 1 -- 172.21.15.46:0/3641877314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 msgr2=0x56152f8e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.065+0000 7f3b74c00640 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 0x56152f8e0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.066+0000 7f3b74c00640 1 -- 172.21.15.46:0/3641877314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 msgr2=0x56152f8e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.066+0000 7f3b74c00640 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 0x56152f8e5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.066+0000 7f3b74c00640 1 -- 172.21.15.46:0/3641877314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56152f8e17c0 con 0x56152f8d2760 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.066+0000 7f3b74c00640 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8e0aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c02c040 tx=0x7f3b5c01cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f27e4b0cb9a064 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.066+0000 7f3b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.066+0000 7f3b56a00640 1 -- 172.21.15.46:0/3641877314 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c02e040 con 0x56152f8d2760 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.066+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3641877314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b5c005560 con 0x56152f8d2760 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.066+0000 7f3b56a00640 1 -- 172.21.15.46:0/3641877314 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3b5c029440 con 0x56152f8d2760 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b56a00640 1 -- 172.21.15.46:0/3641877314 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3b5c009cf0 con 0x56152f8d2760 2024-09-10T07:53:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b7abbf740 10 client.29386 fetch_fsmap learned FSMap version 61 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3641877314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56152f957e20 con 0x56152f8d2760 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b56a00640 1 -- 172.21.15.46:0/3641877314 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c005560 con 0x56152f8d2760 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b56a00640 1 -- 172.21.15.46:0/3641877314 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 61) ==== 8390+0+0 (secure 0 0 0) 0x7f3b5c010840 con 0x56152f8d2760 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b7abbf740 10 client.29386 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b7abbf740 10 client.29386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b7abbf740 20 client.29386 populate_metadata read hostname 'smithi046' 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56152f9584b0 0x56152f95a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b7abbf740 4 client.29386 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.067+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3641877314 --> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56152f83fe60 con 0x56152f9584b0 2024-09-10T07:53:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.068+0000 7f3b76000640 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56152f9584b0 0x56152f95a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.068+0000 7f3b76000640 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56152f9584b0 0x56152f95a8d0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a570cd4c8c7b27 server_cookie=355975f2b3382b44 in_seq=0 out_seq=0 2024-09-10T07:53:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.068+0000 7f3b56a00640 10 client.29386 ms_handle_connect on v2:172.21.15.46:6840/3146821117 2024-09-10T07:53:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.068+0000 7f3b56a00640 1 -- 172.21.15.46:0/3641877314 <== mds.0 v2:172.21.15.46:6840/3146821117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56152f83fe60 con 0x56152f9584b0 2024-09-10T07:53:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.068+0000 7f3b56a00640 10 client.29386 handle_command_reply: tid=0 2024-09-10T07:53:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.068+0000 7f3b7abbf740 1 client.29386 shutdown 2024-09-10T07:53:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3641877314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 msgr2=0x56152f8e0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8e0aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c02c040 tx=0x7f3b5c01cdb0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3641877314 shutdown_connections 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.46:6840/3146821117,v1:172.21.15.46:6841/3146821117] conn(0x56152f9584b0 0x56152f95a8d0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56152f8dd020 0x56152f8e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56152f8d2760 0x56152f8e0aa0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 --2- 172.21.15.46:0/3641877314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152f8d1e40 0x56152f8e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b56a00640 0 client.29386 ms_handle_reset on v2:172.21.15.46:6840/3146821117 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3641877314 >> 172.21.15.46:0/3641877314 conn(0x56152f8ce3d0 msgr2=0x56152f8da5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3641877314 shutdown_connections 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 1 -- 172.21.15.46:0/3641877314 wait complete. 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.069+0000 7f3b7abbf740 20 client.29386 trim_cache size 0 max 16384 2024-09-10T07:53:00.071 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-09-10T07:53:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 -- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b580a2b80 msgr2=0x7f3b580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b580a2b80 0x7f3b580a5040 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3b640b8220 tx=0x7f3b6402a420 comp rx=0 tx=0).stop 2024-09-10T07:53:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 -- 172.21.15.46:0/2937119775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b701019c0 msgr2=0x7f3b7019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b701019c0 0x7f3b7019a530 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3b64005940 tx=0x7f3b640040c0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 -- 172.21.15.46:0/2937119775 shutdown_connections 2024-09-10T07:53:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b701079c0 0x7f3b7019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3b580a2b80 0x7f3b580a5040 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b70102360 0x7f3b7019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 --2- 172.21.15.46:0/2937119775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b701019c0 0x7f3b7019a530 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 -- 172.21.15.46:0/2937119775 >> 172.21.15.46:0/2937119775 conn(0x7f3b700fd740 msgr2=0x7f3b700fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 -- 172.21.15.46:0/2937119775 shutdown_connections 2024-09-10T07:53:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:00.070+0000 7f3b76a00640 1 -- 172.21.15.46:0/2937119775 wait complete. 2024-09-10T07:53:00.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:00 smithi181 ceph-mon[35397]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-10T07:53:00.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:00 smithi063 ceph-mon[35020]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-10T07:53:00.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:00 smithi046 ceph-mon[30705]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-10T07:53:01.323 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T07:53:01.324 INFO:tasks.fwd_scrub:done joining 2024-09-10T07:53:01.324 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T07:53:01.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:53:01.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 -- 172.21.15.46:0/4052415104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d34074bf0 msgr2=0x7f3d34075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 --2- 172.21.15.46:0/4052415104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d34074bf0 0x7f3d34075050 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009900 tx=0x7f3d1c02a480 comp rx=0 tx=0).stop 2024-09-10T07:53:01.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 -- 172.21.15.46:0/4052415104 shutdown_connections 2024-09-10T07:53:01.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 --2- 172.21.15.46:0/4052415104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d34109800 0x7f3d3410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 --2- 172.21.15.46:0/4052415104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d34074bf0 0x7f3d34075050 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 --2- 172.21.15.46:0/4052415104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d340761d0 0x7f3d34074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 -- 172.21.15.46:0/4052415104 >> 172.21.15.46:0/4052415104 conn(0x7f3d340fd110 msgr2=0x7f3d340ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:01.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 -- 172.21.15.46:0/4052415104 shutdown_connections 2024-09-10T07:53:01.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.513+0000 7f3d39600640 1 -- 172.21.15.46:0/4052415104 wait complete. 2024-09-10T07:53:01.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.514+0000 7f3d39600640 1 Processor -- start 2024-09-10T07:53:01.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.514+0000 7f3d39600640 1 -- start start 2024-09-10T07:53:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d39600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d34074bf0 0x7f3d34199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d39600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d340761d0 0x7f3d3419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d39600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d34109800 0x7f3d341a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d39600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410f830 con 0x7f3d340761d0 2024-09-10T07:53:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d39600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410f6b0 con 0x7f3d34074bf0 2024-09-10T07:53:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d39600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410f9b0 con 0x7f3d34109800 2024-09-10T07:53:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d33e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d34074bf0 0x7f3d34199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d340761d0 0x7f3d3419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d33e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d34074bf0 0x7f3d34199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49414/0 (socket says 172.21.15.46:49414) 2024-09-10T07:53:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d33e00640 1 -- 172.21.15.46:0/2751254317 learned_addr learned my addr 172.21.15.46:0/2751254317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d38c00640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d34109800 0x7f3d341a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d33e00640 1 -- 172.21.15.46:0/2751254317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d34109800 msgr2=0x7f3d341a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d33e00640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d34109800 0x7f3d341a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d33e00640 1 -- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d340761d0 msgr2=0x7f3d3419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d33e00640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d340761d0 0x7f3d3419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.515+0000 7f3d33e00640 1 -- 172.21.15.46:0/2751254317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d341a1ad0 con 0x7f3d34074bf0 2024-09-10T07:53:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.516+0000 7f3d33e00640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d34074bf0 0x7f3d34199eb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400d950 tx=0x7f3d2400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.516+0000 7f3d31600640 1 -- 172.21.15.46:0/2751254317 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24017030 con 0x7f3d34074bf0 2024-09-10T07:53:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.516+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d341a1dc0 con 0x7f3d34074bf0 2024-09-10T07:53:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.516+0000 7f3d31600640 1 -- 172.21.15.46:0/2751254317 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d24004020 con 0x7f3d34074bf0 2024-09-10T07:53:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.516+0000 7f3d31600640 1 -- 172.21.15.46:0/2751254317 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d240047a0 con 0x7f3d34074bf0 2024-09-10T07:53:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.516+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d341a2280 con 0x7f3d34074bf0 2024-09-10T07:53:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.517+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d341085c0 con 0x7f3d34074bf0 2024-09-10T07:53:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.519+0000 7f3d31600640 1 -- 172.21.15.46:0/2751254317 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d240041d0 con 0x7f3d34074bf0 2024-09-10T07:53:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.520+0000 7f3d31600640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d180a2c10 0x7f3d180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.520+0000 7f3d31600640 1 -- 172.21.15.46:0/2751254317 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f3d240cf070 con 0x7f3d34074bf0 2024-09-10T07:53:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.521+0000 7f3d2b400640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d180a2c10 0x7f3d180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.521+0000 7f3d2b400640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d180a2c10 0x7f3d180a50d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c031440 tx=0x7f3d1c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.523+0000 7f3d31600640 1 -- 172.21.15.46:0/2751254317 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d2409bc70 con 0x7f3d34074bf0 2024-09-10T07:53:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.710+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d3402da70 con 0x7f3d34074bf0 2024-09-10T07:53:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.712+0000 7f3d31600640 1 -- 172.21.15.46:0/2751254317 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24700 (secure 0 0 0) 0x7f3d2409b570 con 0x7f3d34074bf0 2024-09-10T07:53:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:53:01.712 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:35:14.826944+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:53:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d180a2c10 msgr2=0x7f3d180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d180a2c10 0x7f3d180a50d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c031440 tx=0x7f3d1c0315b0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d34074bf0 msgr2=0x7f3d34199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d34074bf0 0x7f3d34199eb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400d950 tx=0x7f3d2400de10 comp rx=0 tx=0).stop 2024-09-10T07:53:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 shutdown_connections 2024-09-10T07:53:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d34109800 0x7f3d341a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f3d180a2c10 0x7f3d180a50d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d340761d0 0x7f3d3419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 --2- 172.21.15.46:0/2751254317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d34074bf0 0x7f3d34199eb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 >> 172.21.15.46:0/2751254317 conn(0x7f3d340fd110 msgr2=0x7f3d34105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 shutdown_connections 2024-09-10T07:53:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.714+0000 7f3d39600640 1 -- 172.21.15.46:0/2751254317 wait complete. 2024-09-10T07:53:01.730 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:28:36.994371+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T07:33:30.775288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T07:33:33.558621+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T07:33:43.433746+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T07:53:01.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T07:53:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 -- 172.21.15.46:0/2167462748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 msgr2=0x7f1800075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 --2- 172.21.15.46:0/2167462748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 0x7f1800075050 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009900 tx=0x7f17e802a480 comp rx=0 tx=0).stop 2024-09-10T07:53:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 -- 172.21.15.46:0/2167462748 shutdown_connections 2024-09-10T07:53:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 --2- 172.21.15.46:0/2167462748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18001099a0 0x7f180010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 --2- 172.21.15.46:0/2167462748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 0x7f1800075050 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 --2- 172.21.15.46:0/2167462748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18000761d0 0x7f1800074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 -- 172.21.15.46:0/2167462748 >> 172.21.15.46:0/2167462748 conn(0x7f18000fd2b0 msgr2=0x7f18000ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 -- 172.21.15.46:0/2167462748 shutdown_connections 2024-09-10T07:53:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.894+0000 7f1804a00640 1 -- 172.21.15.46:0/2167462748 wait complete. 2024-09-10T07:53:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.895+0000 7f1804a00640 1 Processor -- start 2024-09-10T07:53:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.895+0000 7f1804a00640 1 -- start start 2024-09-10T07:53:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.895+0000 7f1804a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 0x7f180019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.895+0000 7f1804a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18000761d0 0x7f180019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.895+0000 7f1804a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18001099a0 0x7f18001a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.895+0000 7f1804a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f180010f9f0 con 0x7f1800074bf0 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.895+0000 7f1804a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f180010f870 con 0x7f18001099a0 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.895+0000 7f1804a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f180010fb70 con 0x7f18000761d0 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 0x7f180019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 0x7f180019a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44524/0 (socket says 172.21.15.46:44524) 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 -- 172.21.15.46:0/3681199891 learned_addr learned my addr 172.21.15.46:0/3681199891 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fe800640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18001099a0 0x7f18001a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17f7e00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18000761d0 0x7f180019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 -- 172.21.15.46:0/3681199891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18000761d0 msgr2=0x7f180019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18000761d0 0x7f180019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 -- 172.21.15.46:0/3681199891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18001099a0 msgr2=0x7f18001a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18001099a0 0x7f18001a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 -- 172.21.15.46:0/3681199891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18001a1cf0 con 0x7f1800074bf0 2024-09-10T07:53:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fe800640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18001099a0 0x7f18001a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17fde00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 0x7f180019a0d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f17f000eb90 tx=0x7f17f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:01.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.896+0000 7f17f6a00640 1 -- 172.21.15.46:0/3681199891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0004d20 con 0x7f1800074bf0 2024-09-10T07:53:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.897+0000 7f17f6a00640 1 -- 172.21.15.46:0/3681199891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f000ee90 con 0x7f1800074bf0 2024-09-10T07:53:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.897+0000 7f17f6a00640 1 -- 172.21.15.46:0/3681199891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0010830 con 0x7f1800074bf0 2024-09-10T07:53:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.897+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18001a1fe0 con 0x7f1800074bf0 2024-09-10T07:53:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.897+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001a23f0 con 0x7f1800074bf0 2024-09-10T07:53:01.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.898+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1800108760 con 0x7f1800074bf0 2024-09-10T07:53:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.900+0000 7f17f6a00640 1 -- 172.21.15.46:0/3681199891 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17f0005020 con 0x7f1800074bf0 2024-09-10T07:53:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.901+0000 7f17f6a00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e40a2b90 0x7f17e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.901+0000 7f17f6a00640 1 -- 172.21.15.46:0/3681199891 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f17f00cfbd0 con 0x7f1800074bf0 2024-09-10T07:53:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.901+0000 7f17f7e00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e40a2b90 0x7f17e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.902+0000 7f17f7e00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e40a2b90 0x7f17e40a5050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f17e8006010 tx=0x7f17e8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:01.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:01.906+0000 7f17f6a00640 1 -- 172.21.15.46:0/3681199891 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17f009c750 con 0x7f1800074bf0 2024-09-10T07:53:02.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.089+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1800062ec0 con 0x7f1800074bf0 2024-09-10T07:53:02.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.090+0000 7f17f6a00640 1 -- 172.21.15.46:0/3681199891 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f17f009c050 con 0x7f1800074bf0 2024-09-10T07:53:02.089 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-10T07:53:02.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e40a2b90 msgr2=0x7f17e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e40a2b90 0x7f17e40a5050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f17e8006010 tx=0x7f17e8000bc0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 msgr2=0x7f180019a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 0x7f180019a0d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f17f000eb90 tx=0x7f17f000c6a0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 shutdown_connections 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18001099a0 0x7f18001a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18000761d0 0x7f180019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f17e40a2b90 0x7f17e40a5050 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 --2- 172.21.15.46:0/3681199891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1800074bf0 0x7f180019a0d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 >> 172.21.15.46:0/3681199891 conn(0x7f18000fd2b0 msgr2=0x7f1800105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 shutdown_connections 2024-09-10T07:53:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.092+0000 7f1804a00640 1 -- 172.21.15.46:0/3681199891 wait complete. 2024-09-10T07:53:02.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T07:53:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 -- 172.21.15.46:0/1363072178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 msgr2=0x7fec64104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 --2- 172.21.15.46:0/1363072178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 0x7fec64104120 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fec54009900 tx=0x7fec5402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 -- 172.21.15.46:0/1363072178 shutdown_connections 2024-09-10T07:53:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 --2- 172.21.15.46:0/1363072178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec641058d0 0x7fec6410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 --2- 172.21.15.46:0/1363072178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec64104f10 0x7fec64105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 --2- 172.21.15.46:0/1363072178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 0x7fec64104120 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 -- 172.21.15.46:0/1363072178 >> 172.21.15.46:0/1363072178 conn(0x7fec640ff4a0 msgr2=0x7fec641018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 -- 172.21.15.46:0/1363072178 shutdown_connections 2024-09-10T07:53:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.271+0000 7fec6b200640 1 -- 172.21.15.46:0/1363072178 wait complete. 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.272+0000 7fec6b200640 1 Processor -- start 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.272+0000 7fec6b200640 1 -- start start 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.272+0000 7fec6b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 0x7fec6419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.272+0000 7fec6b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec64104f10 0x7fec6419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.272+0000 7fec6b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec641058d0 0x7fec641a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.272+0000 7fec6b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fec6410f9a0 con 0x7fec64103d20 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.272+0000 7fec6b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec6410f820 con 0x7fec64104f10 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.272+0000 7fec6b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fec6410fb20 con 0x7fec641058d0 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 0x7fec6419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec5b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec64104f10 0x7fec6419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 0x7fec6419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44540/0 (socket says 172.21.15.46:44540) 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec63e00640 1 -- 172.21.15.46:0/1024715553 learned_addr learned my addr 172.21.15.46:0/1024715553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec69200640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec641058d0 0x7fec641a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec5b400640 1 -- 172.21.15.46:0/1024715553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec641058d0 msgr2=0x7fec641a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec5b400640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec641058d0 0x7fec641a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec5b400640 1 -- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 msgr2=0x7fec6419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec5b400640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 0x7fec6419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec5b400640 1 -- 172.21.15.46:0/1024715553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec641a1ca0 con 0x7fec64104f10 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec63e00640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 0x7fec6419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec5b400640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec64104f10 0x7fec6419a5c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00e9f0 tx=0x7fec4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.273+0000 7fec61600640 1 -- 172.21.15.46:0/1024715553 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00ccc0 con 0x7fec64104f10 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.274+0000 7fec61600640 1 -- 172.21.15.46:0/1024715553 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec4c005020 con 0x7fec64104f10 2024-09-10T07:53:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.274+0000 7fec61600640 1 -- 172.21.15.46:0/1024715553 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0049e0 con 0x7fec64104f10 2024-09-10T07:53:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.274+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec641a1f90 con 0x7fec64104f10 2024-09-10T07:53:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.274+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec641a2400 con 0x7fec64104f10 2024-09-10T07:53:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.275+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6410b1b0 con 0x7fec64104f10 2024-09-10T07:53:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.277+0000 7fec61600640 1 -- 172.21.15.46:0/1024715553 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec4c005560 con 0x7fec64104f10 2024-09-10T07:53:02.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.279+0000 7fec61600640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec480a2c10 0x7fec480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.279+0000 7fec61600640 1 -- 172.21.15.46:0/1024715553 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fec4c0cf3c0 con 0x7fec64104f10 2024-09-10T07:53:02.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.279+0000 7fec63e00640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec480a2c10 0x7fec480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.280+0000 7fec63e00640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec480a2c10 0x7fec480a50d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fec54000c00 tx=0x7fec540046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.281+0000 7fec61600640 1 -- 172.21.15.46:0/1024715553 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec4c0051d0 con 0x7fec64104f10 2024-09-10T07:53:02.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:02 smithi046 ceph-mon[30705]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T07:53:02.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:02 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2751254317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:53:02.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:02 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3681199891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:53:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.467+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fec6402da10 con 0x7fec64104f10 2024-09-10T07:53:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.468+0000 7fec61600640 1 -- 172.21.15.46:0/1024715553 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fec4c09bfd0 con 0x7fec64104f10 2024-09-10T07:53:02.467 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-09-10T07:53:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec480a2c10 msgr2=0x7fec480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec480a2c10 0x7fec480a50d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fec54000c00 tx=0x7fec540046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec64104f10 msgr2=0x7fec6419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec64104f10 0x7fec6419a5c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00e9f0 tx=0x7fec4c00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 shutdown_connections 2024-09-10T07:53:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec641058d0 0x7fec641a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec64104f10 0x7fec6419a5c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fec480a2c10 0x7fec480a50d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 --2- 172.21.15.46:0/1024715553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec64103d20 0x7fec6419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 >> 172.21.15.46:0/1024715553 conn(0x7fec640ff4a0 msgr2=0x7fec64108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 shutdown_connections 2024-09-10T07:53:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.470+0000 7fec6b200640 1 -- 172.21.15.46:0/1024715553 wait complete. 2024-09-10T07:53:02.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T07:53:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.648+0000 7f8843c00640 1 -- 172.21.15.46:0/3001706614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c1058d0 msgr2=0x7f883c10c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.648+0000 7f8843c00640 1 --2- 172.21.15.46:0/3001706614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c1058d0 0x7f883c10c160 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8834009ca0 tx=0x7f883402a8c0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.648+0000 7f8843c00640 1 -- 172.21.15.46:0/3001706614 shutdown_connections 2024-09-10T07:53:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.648+0000 7f8843c00640 1 --2- 172.21.15.46:0/3001706614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c1058d0 0x7f883c10c160 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.648+0000 7f8843c00640 1 --2- 172.21.15.46:0/3001706614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f883c104f10 0x7f883c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.648+0000 7f8843c00640 1 --2- 172.21.15.46:0/3001706614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f883c103d20 0x7f883c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.648+0000 7f8843c00640 1 -- 172.21.15.46:0/3001706614 >> 172.21.15.46:0/3001706614 conn(0x7f883c0ff4a0 msgr2=0x7f883c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.649+0000 7f8843c00640 1 -- 172.21.15.46:0/3001706614 shutdown_connections 2024-09-10T07:53:02.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.649+0000 7f8843c00640 1 -- 172.21.15.46:0/3001706614 wait complete. 2024-09-10T07:53:02.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:02 smithi181 ceph-mon[35397]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T07:53:02.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:02 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2751254317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:53:02.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:02 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3681199891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:53:02.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.649+0000 7f8843c00640 1 Processor -- start 2024-09-10T07:53:02.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8843c00640 1 -- start start 2024-09-10T07:53:02.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8843c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f883c103d20 0x7f883c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8843c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c104f10 0x7f883c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8843c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f883c1058d0 0x7f883c1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8843c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10f9d0 con 0x7f883c104f10 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8843c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10f850 con 0x7f883c1058d0 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8843c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10fb50 con 0x7f883c103d20 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8841200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f883c103d20 0x7f883c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8841200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f883c103d20 0x7f883c199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:50532/0 (socket says 172.21.15.46:50532) 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8841200640 1 -- 172.21.15.46:0/2017810580 learned_addr learned my addr 172.21.15.46:0/2017810580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8833e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c104f10 0x7f883c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.650+0000 7f8841c00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f883c1058d0 0x7f883c1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f8841200640 1 -- 172.21.15.46:0/2017810580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f883c1058d0 msgr2=0x7f883c1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f8841200640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f883c1058d0 0x7f883c1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f8841200640 1 -- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c104f10 msgr2=0x7f883c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f8841200640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c104f10 0x7f883c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f8841200640 1 -- 172.21.15.46:0/2017810580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c1a1c10 con 0x7f883c103d20 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f8841200640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f883c103d20 0x7f883c199ff0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f882c00e9c0 tx=0x7f882c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f883a000640 1 -- 172.21.15.46:0/2017810580 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c00cde0 con 0x7f883c103d20 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f883a000640 1 -- 172.21.15.46:0/2017810580 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c010910 con 0x7f883c103d20 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c1a1f00 con 0x7f883c103d20 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f883a000640 1 -- 172.21.15.46:0/2017810580 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c018a10 con 0x7f883c103d20 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.651+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c1a2360 con 0x7f883c103d20 2024-09-10T07:53:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.652+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883c10b0c0 con 0x7f883c103d20 2024-09-10T07:53:02.654 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:02 smithi063 ceph-mon[35020]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T07:53:02.654 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:02 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2751254317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:53:02.654 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:02 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3681199891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T07:53:02.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.654+0000 7f883a000640 1 -- 172.21.15.46:0/2017810580 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f882c01f020 con 0x7f883c103d20 2024-09-10T07:53:02.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.656+0000 7f883a000640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88200a2b90 0x7f88200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.656+0000 7f883a000640 1 -- 172.21.15.46:0/2017810580 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f882c0cfc90 con 0x7f883c103d20 2024-09-10T07:53:02.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.656+0000 7f8833e00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88200a2b90 0x7f88200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.657+0000 7f8833e00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88200a2b90 0x7f88200a5050 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f883c19b500 tx=0x7f8824005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.658+0000 7f883a000640 1 -- 172.21.15.46:0/2017810580 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f882c09c810 con 0x7f883c103d20 2024-09-10T07:53:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.841+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f883c02da50 con 0x7f883c103d20 2024-09-10T07:53:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.841+0000 7f883a000640 1 -- 172.21.15.46:0/2017810580 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f882c09c110 con 0x7f883c103d20 2024-09-10T07:53:02.841 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-09-10T07:53:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88200a2b90 msgr2=0x7f88200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88200a2b90 0x7f88200a5050 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f883c19b500 tx=0x7f8824005f90 comp rx=0 tx=0).stop 2024-09-10T07:53:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f883c103d20 msgr2=0x7f883c199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f883c103d20 0x7f883c199ff0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f882c00e9c0 tx=0x7f882c00ee80 comp rx=0 tx=0).stop 2024-09-10T07:53:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 shutdown_connections 2024-09-10T07:53:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f883c1058d0 0x7f883c1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f88200a2b90 0x7f88200a5050 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c104f10 0x7f883c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 --2- 172.21.15.46:0/2017810580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f883c103d20 0x7f883c199ff0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 >> 172.21.15.46:0/2017810580 conn(0x7f883c0ff4a0 msgr2=0x7f883c108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 shutdown_connections 2024-09-10T07:53:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:02.844+0000 7f8843c00640 1 -- 172.21.15.46:0/2017810580 wait complete. 2024-09-10T07:53:02.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.024+0000 7f63ac600640 1 -- 172.21.15.46:0/1081867313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 msgr2=0x7f63a4105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.024+0000 7f63ac600640 1 --2- 172.21.15.46:0/1081867313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 0x7f63a4105390 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6390009860 tx=0x7f639002a340 comp rx=0 tx=0).stop 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.025+0000 7f63ac600640 1 -- 172.21.15.46:0/1081867313 shutdown_connections 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.025+0000 7f63ac600640 1 --2- 172.21.15.46:0/1081867313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a41058d0 0x7f63a410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.025+0000 7f63ac600640 1 --2- 172.21.15.46:0/1081867313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 0x7f63a4105390 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.025+0000 7f63ac600640 1 --2- 172.21.15.46:0/1081867313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63a4103d20 0x7f63a4104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.025+0000 7f63ac600640 1 -- 172.21.15.46:0/1081867313 >> 172.21.15.46:0/1081867313 conn(0x7f63a40ff4a0 msgr2=0x7f63a41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.025+0000 7f63ac600640 1 -- 172.21.15.46:0/1081867313 shutdown_connections 2024-09-10T07:53:03.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.025+0000 7f63ac600640 1 -- 172.21.15.46:0/1081867313 wait complete. 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.025+0000 7f63ac600640 1 Processor -- start 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63ac600640 1 -- start start 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63ac600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63a4103d20 0x7f63a419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63ac600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 0x7f63a419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63ac600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a41058d0 0x7f63a41a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63ac600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63a410f980 con 0x7f63a41058d0 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63ac600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63a410f800 con 0x7f63a4103d20 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63ac600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63a410fb00 con 0x7f63a4104f10 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63aa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a41058d0 0x7f63a41a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63aa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a41058d0 0x7f63a41a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44580/0 (socket says 172.21.15.46:44580) 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63aa600640 1 -- 172.21.15.46:0/3889505123 learned_addr learned my addr 172.21.15.46:0/3889505123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63a9200640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 0x7f63a419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.026+0000 7f63a9c00640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63a4103d20 0x7f63a419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a9200640 1 -- 172.21.15.46:0/3889505123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63a4103d20 msgr2=0x7f63a419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a9200640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63a4103d20 0x7f63a419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a9200640 1 -- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a41058d0 msgr2=0x7f63a41a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a9200640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a41058d0 0x7f63a41a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a9200640 1 -- 172.21.15.46:0/3889505123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a41a1c90 con 0x7f63a4104f10 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63aa600640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a41058d0 0x7f63a41a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a9200640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 0x7f63a419a5b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6390035040 tx=0x7f6390033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a2000640 1 -- 172.21.15.46:0/3889505123 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390037040 con 0x7f63a4104f10 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a41a1f20 con 0x7f63a4104f10 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a2000640 1 -- 172.21.15.46:0/3889505123 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6390030850 con 0x7f63a4104f10 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63a2000640 1 -- 172.21.15.46:0/3889505123 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900046c0 con 0x7f63a4104f10 2024-09-10T07:53:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.027+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a41a2390 con 0x7f63a4104f10 2024-09-10T07:53:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.028+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a410b180 con 0x7f63a4104f10 2024-09-10T07:53:03.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.031+0000 7f63a2000640 1 -- 172.21.15.46:0/3889505123 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6390004050 con 0x7f63a4104f10 2024-09-10T07:53:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.032+0000 7f63a2000640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f638c0a2b90 0x7f638c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.032+0000 7f63a2000640 1 -- 172.21.15.46:0/3889505123 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f63900ef180 con 0x7f63a4104f10 2024-09-10T07:53:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.032+0000 7f63a9c00640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f638c0a2b90 0x7f638c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.033+0000 7f63a9c00640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f638c0a2b90 0x7f638c0a5050 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f63a4104d70 tx=0x7f6398009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.034+0000 7f63a2000640 1 -- 172.21.15.46:0/3889505123 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63900bbd00 con 0x7f63a4104f10 2024-09-10T07:53:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.220+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f63a402d7c0 con 0x7f63a4104f10 2024-09-10T07:53:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.221+0000 7f63a2000640 1 -- 172.21.15.46:0/3889505123 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f63900bb600 con 0x7f63a4104f10 2024-09-10T07:53:03.220 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.223+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f638c0a2b90 msgr2=0x7f638c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.223+0000 7f63ac600640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f638c0a2b90 0x7f638c0a5050 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f63a4104d70 tx=0x7f6398009260 comp rx=0 tx=0).stop 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.223+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 msgr2=0x7f63a419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.223+0000 7f63ac600640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 0x7f63a419a5b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6390035040 tx=0x7f6390033d90 comp rx=0 tx=0).stop 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.224+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 shutdown_connections 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.224+0000 7f63ac600640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f638c0a2b90 0x7f638c0a5050 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.224+0000 7f63ac600640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a41058d0 0x7f63a41a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.224+0000 7f63ac600640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a4104f10 0x7f63a419a5b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.224+0000 7f63ac600640 1 --2- 172.21.15.46:0/3889505123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63a4103d20 0x7f63a419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.224+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 >> 172.21.15.46:0/3889505123 conn(0x7f63a40ff4a0 msgr2=0x7f63a4108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:03.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.224+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 shutdown_connections 2024-09-10T07:53:03.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.224+0000 7f63ac600640 1 -- 172.21.15.46:0/3889505123 wait complete. 2024-09-10T07:53:03.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T07:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 -- 172.21.15.46:0/3398688909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84103b70 msgr2=0x7f4a84103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 --2- 172.21.15.46:0/3398688909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84103b70 0x7f4a84103f70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009900 tx=0x7f4a7402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 -- 172.21.15.46:0/3398688909 shutdown_connections 2024-09-10T07:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 --2- 172.21.15.46:0/3398688909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a84105720 0x7f4a8410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 --2- 172.21.15.46:0/3398688909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a84104d60 0x7f4a841051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 --2- 172.21.15.46:0/3398688909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84103b70 0x7f4a84103f70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 -- 172.21.15.46:0/3398688909 >> 172.21.15.46:0/3398688909 conn(0x7f4a840ff2f0 msgr2=0x7f4a84101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 -- 172.21.15.46:0/3398688909 shutdown_connections 2024-09-10T07:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.400+0000 7f4a83200640 1 -- 172.21.15.46:0/3398688909 wait complete. 2024-09-10T07:53:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.401+0000 7f4a83200640 1 Processor -- start 2024-09-10T07:53:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.401+0000 7f4a83200640 1 -- start start 2024-09-10T07:53:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a83200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a84103b70 0x7f4a84199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a83200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a84104d60 0x7f4a8419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a83200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84105720 0x7f4a841a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a83200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8410f7d0 con 0x7f4a84105720 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a83200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8410f650 con 0x7f4a84103b70 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a83200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8410f950 con 0x7f4a84104d60 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a84103b70 0x7f4a84199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a82800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84105720 0x7f4a841a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a84104d60 0x7f4a8419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a84103b70 0x7f4a84199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:49488/0 (socket says 172.21.15.46:49488) 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81e00640 1 -- 172.21.15.46:0/3737137898 learned_addr learned my addr 172.21.15.46:0/3737137898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81e00640 1 -- 172.21.15.46:0/3737137898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a84104d60 msgr2=0x7f4a8419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81e00640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a84104d60 0x7f4a8419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81e00640 1 -- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84105720 msgr2=0x7f4a841a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81e00640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84105720 0x7f4a841a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a81e00640 1 -- 172.21.15.46:0/3737137898 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a841a1ae0 con 0x7f4a84103b70 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.402+0000 7f4a82800640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84105720 0x7f4a841a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.403+0000 7f4a81e00640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a84103b70 0x7f4a84199ec0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4a74031610 tx=0x7f4a74004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.403+0000 7f4a7aa00640 1 -- 172.21.15.46:0/3737137898 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7403f030 con 0x7f4a84103b70 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.403+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a841a1d70 con 0x7f4a84103b70 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.403+0000 7f4a7aa00640 1 -- 172.21.15.46:0/3737137898 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a74003ad0 con 0x7f4a84103b70 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.403+0000 7f4a7aa00640 1 -- 172.21.15.46:0/3737137898 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74035070 con 0x7f4a84103b70 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.403+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a841a21f0 con 0x7f4a84103b70 2024-09-10T07:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.404+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8410afb0 con 0x7f4a84103b70 2024-09-10T07:53:03.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.406+0000 7f4a7aa00640 1 -- 172.21.15.46:0/3737137898 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a74042030 con 0x7f4a84103b70 2024-09-10T07:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.407+0000 7f4a7aa00640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a680a2c10 0x7f4a680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.408+0000 7f4a7aa00640 1 -- 172.21.15.46:0/3737137898 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4a740efc60 con 0x7f4a84103b70 2024-09-10T07:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.408+0000 7f4a81400640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a680a2c10 0x7f4a680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.408+0000 7f4a81400640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a680a2c10 0x7f4a680a50d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4a700095f0 tx=0x7f4a70005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.409+0000 7f4a7aa00640 1 -- 172.21.15.46:0/3737137898 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a740bc190 con 0x7f4a84103b70 2024-09-10T07:53:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.646+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f4a84062e90 con 0x7f4a84103b70 2024-09-10T07:53:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:03 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1024715553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T07:53:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:03 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2017810580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T07:53:03.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:03 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3889505123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T07:53:03.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:03 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1024715553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T07:53:03.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:03 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2017810580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T07:53:03.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:03 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3889505123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T07:53:03.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.655+0000 7f4a7aa00640 1 -- 172.21.15.46:0/3737137898 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740370 (secure 0 0 0) 0x7f4a740c0490 con 0x7f4a84103b70 2024-09-10T07:53:03.662 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:53:03.662 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:53:03.662 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 18, 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "active_gid": 14150, 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "active_name": "x", 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addrs": { 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1492955812 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-10T07:53:03.663 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1492955812 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addr": "172.21.15.46:6801/1492955812", 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: "active_change": "2024-09-10T07:26:12.367295+0000", 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: "standbys": [ 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24109, 2024-09-10T07:53:03.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "z", 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.669 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-09-10T07:53:03.671 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.672 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.673 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.676 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.677 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-09-10T07:53:03.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-09-10T07:53:03.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-09-10T07:53:03.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.685 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.685 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T07:53:03.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T07:53:03.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T07:53:03.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T07:53:03.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.688 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.692 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.694 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.695 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.696 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:03.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.698 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-09-10T07:53:03.699 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.701 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.704 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.705 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T07:53:03.708 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.709 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.710 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.712 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-09-10T07:53:03.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-09-10T07:53:03.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.716 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.721 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.722 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.726 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.730 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-09-10T07:53:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-09-10T07:53:03.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.743 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.748 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.749 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-09-10T07:53:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:03.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:03.753 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-09-10T07:53:03.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.756 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.757 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.758 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T07:53:03.761 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.762 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.767 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-09-10T07:53:03.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.772 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.774 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.775 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-09-10T07:53:03.777 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.778 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.782 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.785 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.787 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T07:53:03.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T07:53:03.789 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T07:53:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.793 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.794 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.798 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.804 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T07:53:03.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T07:53:03.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T07:53:03.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T07:53:03.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T07:53:03.812 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T07:53:03.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T07:53:03.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T07:53:03.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T07:53:03.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T07:53:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.820 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T07:53:03.821 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.825 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.826 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.827 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.828 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.829 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.832 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.833 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.835 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.836 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.838 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.840 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-10T07:53:03.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.843 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.845 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.846 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.849 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.852 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-09-10T07:53:03.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-09-10T07:53:03.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:03.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.857 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.860 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.862 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.865 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-09-10T07:53:03.867 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.868 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.869 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.872 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.875 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.876 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-09-10T07:53:03.881 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-09-10T07:53:03.882 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.883 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.884 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.886 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.887 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.893 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.894 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-09-10T07:53:03.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.897 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.898 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.899 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-09-10T07:53:03.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.906 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.907 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-09-10T07:53:03.913 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.914 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.916 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.919 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.923 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.924 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.926 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.927 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.928 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.929 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.930 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.933 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.935 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.936 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-09-10T07:53:03.939 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.942 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.943 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.944 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.945 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.946 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.949 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.950 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.960 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.964 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.968 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.969 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.972 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.973 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.974 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.978 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.979 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.980 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-09-10T07:53:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-10T07:53:03.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.984 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.985 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.987 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.988 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.989 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.992 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.993 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.996 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:03.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.015 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.016 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.017 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.021 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.023 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.026 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.028 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-09-10T07:53:04.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-09-10T07:53:04.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-09-10T07:53:04.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-09-10T07:53:04.032 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-09-10T07:53:04.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.036 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.038 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.039 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.042 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.046 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.050 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.051 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.052 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.053 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-09-10T07:53:04.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.055 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.056 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.057 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.058 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.060 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.061 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.062 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.063 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.065 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.068 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.070 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.072 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.074 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-09-10T07:53:04.084 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.085 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.088 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.093 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-09-10T07:53:04.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T07:53:04.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.106 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.107 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.108 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24116, 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "y", 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.115 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.128 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.129 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-09-10T07:53:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-09-10T07:53:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-09-10T07:53:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.152 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T07:53:04.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-09-10T07:53:04.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.155 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T07:53:04.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.158 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.159 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T07:53:04.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T07:53:04.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T07:53:04.162 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T07:53:04.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.164 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.165 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-09-10T07:53:04.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-09-10T07:53:04.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-09-10T07:53:04.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-09-10T07:53:04.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-09-10T07:53:04.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-09-10T07:53:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-09-10T07:53:04.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-09-10T07:53:04.176 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.179 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T07:53:04.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-09-10T07:53:04.182 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T07:53:04.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.185 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.187 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.191 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.196 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-09-10T07:53:04.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-10T07:53:04.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T07:53:04.204 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T07:53:04.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T07:53:04.206 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.207 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.210 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.214 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.215 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.220 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-09-10T07:53:04.221 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.222 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T07:53:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T07:53:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T07:53:04.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T07:53:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.253 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T07:53:04.256 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-09-10T07:53:04.258 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.259 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T07:53:04.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T07:53:04.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T07:53:04.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-09-10T07:53:04.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-09-10T07:53:04.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-09-10T07:53:04.269 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.270 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.271 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.273 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.275 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.276 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.278 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.281 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.283 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-09-10T07:53:04.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-10T07:53:04.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-09-10T07:53:04.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.288 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.290 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.292 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.293 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-09-10T07:53:04.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-09-10T07:53:04.295 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.298 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.299 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-10T07:53:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T07:53:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-09-10T07:53:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-09-10T07:53:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-09-10T07:53:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-09-10T07:53:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-09-10T07:53:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-09-10T07:53:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-09-10T07:53:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:03 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1024715553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T07:53:04.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:03 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2017810580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T07:53:04.386 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:03 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3889505123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T07:53:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-10T07:53:04.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.432 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.435 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.436 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.440 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-09-10T07:53:04.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-10T07:53:04.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.446 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.447 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.449 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.450 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.457 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.460 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.463 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.467 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.469 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.471 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.472 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-09-10T07:53:04.473 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.474 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.477 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-09-10T07:53:04.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-09-10T07:53:04.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-09-10T07:53:04.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.483 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.490 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.493 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.495 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.496 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.497 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.499 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.500 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-09-10T07:53:04.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.507 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.509 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.510 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-09-10T07:53:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-09-10T07:53:04.513 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.514 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.520 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.521 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.522 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.529 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.530 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.532 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.533 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.535 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.539 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.542 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.546 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.553 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.554 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.555 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.559 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.560 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.561 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard", 2024-09-10T07:53:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.565 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.568 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.569 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.570 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.571 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-09-10T07:53:04.572 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.573 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-09-10T07:53:04.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-09-10T07:53:04.577 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T07:53:04.578 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T07:53:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T07:53:04.580 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.581 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-09-10T07:53:04.583 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.584 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-10T07:53:04.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.586 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.589 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T07:53:04.595 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.596 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.598 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.599 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.601 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T07:53:04.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T07:53:04.604 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-09-10T07:53:04.605 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.606 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.612 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-09-10T07:53:04.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-09-10T07:53:04.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-09-10T07:53:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-09-10T07:53:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-09-10T07:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-09-10T07:53:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.620 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.621 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.622 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.623 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T07:53:04.628 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T07:53:04.629 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.630 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.632 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T07:53:04.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.634 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.637 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.638 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.639 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.641 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.643 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.646 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.650 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.652 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T07:53:04.656 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T07:53:04.657 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.658 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.659 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.661 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.666 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.669 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-09-10T07:53:04.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.678 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.679 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.682 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T07:53:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T07:53:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T07:53:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T07:53:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T07:53:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T07:53:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T07:53:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T07:53:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T07:53:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-09-10T07:53:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T07:53:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-10T07:53:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-09-10T07:53:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T07:53:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-09-10T07:53:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.761 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.762 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.764 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.766 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.770 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.772 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.776 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.777 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.778 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.779 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.783 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.784 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.786 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.790 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.791 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.792 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.793 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.794 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.796 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.798 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.799 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.801 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.804 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.806 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.808 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.809 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.810 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.813 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.814 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.815 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.817 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.823 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-09-10T07:53:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-09-10T07:53:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-09-10T07:53:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-09-10T07:53:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-10T07:53:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-09-10T07:53:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T07:53:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-09-10T07:53:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-09-10T07:53:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.928 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.929 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T07:53:04.930 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-09-10T07:53:04.931 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.932 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.934 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.935 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.936 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.937 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.938 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.940 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.942 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.944 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.945 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.946 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.953 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.955 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.956 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-09-10T07:53:04.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-09-10T07:53:04.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-09-10T07:53:04.959 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-09-10T07:53:04.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.961 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.962 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.963 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.964 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.965 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.966 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.967 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.968 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.970 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.971 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.975 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.976 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.977 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.978 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.979 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.980 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.982 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.983 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T07:53:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-10T07:53:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-10T07:53:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-10T07:53:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard": "https://172.21.15.46:8443/" 2024-09-10T07:53:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "always_on_modules": { 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "octopus": [ 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "pacific": [ 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "quincy": [ 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-10T07:53:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "reef": [ 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-10T07:53:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "squid": [ 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-10T07:53:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "last_failure_osd_epoch": 3, 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "active_clients": [ 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-10T07:53:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2657753432 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "libcephsqlite", 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3766438969 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 547561128 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-10T07:53:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-10T07:53:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-10T07:53:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3732361403 2024-09-10T07:53:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T07:53:05.014 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:53:05.015 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:04 smithi063 ceph-mon[35020]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-10T07:53:05.015 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:04 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3737137898' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T07:53:05.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:04 smithi181 ceph-mon[35397]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-10T07:53:05.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:04 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3737137898' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T07:53:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.662+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a680a2c10 msgr2=0x7f4a680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.662+0000 7f4a83200640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a680a2c10 0x7f4a680a50d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4a700095f0 tx=0x7f4a70005f90 comp rx=0 tx=0).stop 2024-09-10T07:53:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.662+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a84103b70 msgr2=0x7f4a84199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.662+0000 7f4a83200640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a84103b70 0x7f4a84199ec0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4a74031610 tx=0x7f4a74004270 comp rx=0 tx=0).stop 2024-09-10T07:53:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.663+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 shutdown_connections 2024-09-10T07:53:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.663+0000 7f4a83200640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4a680a2c10 0x7f4a680a50d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.663+0000 7f4a83200640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a84105720 0x7f4a841a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.663+0000 7f4a83200640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a84104d60 0x7f4a8419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.663+0000 7f4a83200640 1 --2- 172.21.15.46:0/3737137898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a84103b70 0x7f4a84199ec0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.663+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 >> 172.21.15.46:0/3737137898 conn(0x7f4a840ff2f0 msgr2=0x7f4a84108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.663+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 shutdown_connections 2024-09-10T07:53:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:03.663+0000 7f4a83200640 1 -- 172.21.15.46:0/3737137898 wait complete. 2024-09-10T07:53:05.024 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T07:53:05.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T07:53:05.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:04 smithi046 ceph-mon[30705]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-10T07:53:05.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:04 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3737137898' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T07:53:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 11340, 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 13152, 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 3905676, 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 14534544, 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 22, 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 7, 2024-09-10T07:53:06.076 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 1.882043227, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 1.065210259, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 1.019193446, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.204517148, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 13144, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 3.393287328, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000258162 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 10595, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.003234739, 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000305 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 49344, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 29744 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 5866, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 5386, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 3393719, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 4055704, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 8, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 6, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.752413801, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.275846935, 2024-09-10T07:53:06.078 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.447152853, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.092404494, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5388, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.019530671, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000374820 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5053, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.002013250, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000398 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 801168, 2024-09-10T07:53:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 225584 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 7459, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 7472, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2950385, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 8459212, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 343, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 9, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 1.339509881, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.430417704, 2024-09-10T07:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.922237603, 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.100765984, 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 7474, 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.798121328, 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000374380 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8448, 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.007922616, 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000937 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.081 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 512544 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6357, 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 3.508862771, 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000551968 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 87, 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.008714637, 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000100168 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 1973, 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 1889, 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1889, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 6.357153587, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003365353 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 84, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 5, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 2, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 608, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 3, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 338, 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 24, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 299, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 15, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 303, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 1276, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 289, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 289, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 21, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 2419, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 1826, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 78, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 41, 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 1446, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 26, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 1304, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 3, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 550, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 3372, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 1015, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 1477148501, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 3372, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 289, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 16, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 273, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 3083, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 13, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 756, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 1498, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 1891, 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 1012, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 39, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 225 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 14, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 34, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 0, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 49, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 24, 2024-09-10T07:53:06.088 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 25, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_path": 15, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_inode": 3903, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 16, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 26, 2024-09-10T07:53:06.089 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 3, 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "evlrg": 0, 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 3285, 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 3272, 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 3272, 2024-09-10T07:53:06.090 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 13, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 6, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 77, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 76, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 76, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 1, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "segmjr": 1, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 1, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 13039812, 2024-09-10T07:53:06.091 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 13098993, 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1822, 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 19.327729660, 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.010607974 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 342, 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 1607, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 1265, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 47, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 85, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 38, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 338, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 1928, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 1590, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 289, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 1575, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 1286, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 70204, 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 231712 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 2115, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 232, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 1973, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 121, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 470, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1022, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 4.963667378, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.004856817 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 51, 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.419799937, 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.008231371 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 25, 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.038834958, 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001553398 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 156, 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.136786220, 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000876834 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2, 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000282982, 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000141491 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.099 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-09-10T07:53:06.099 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.099 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.099 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-09-10T07:53:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 37, 2024-09-10T07:53:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.287448084, 2024-09-10T07:53:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.007768867 2024-09-10T07:53:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-09-10T07:53:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 449, 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.127834057, 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000284708 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 52, 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.118142734, 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002271975 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 48, 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.146029677, 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003042284 2024-09-10T07:53:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-09-10T07:53:06.103 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.009685083, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.004842541 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.104 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2, 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.001007953, 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000503976 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8, 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.021451555, 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002681444 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-09-10T07:53:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 35, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.086182969, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002462370 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.107 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 3, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 2, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 33, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 33, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 995, 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:06.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:06.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:06.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 291044, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 425, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:06.111 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 164, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 1220760, 2024-09-10T07:53:06.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 12230, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 8960, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 11778186, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 32, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 8960, 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-09-10T07:53:06.113 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8960, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 294.688946942, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.032889391 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8960, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 13372 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 8960, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 3418, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 5542, 2024-09-10T07:53:06.114 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 537, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 2090, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 23, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 1829, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 241, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 0, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 3, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 576, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:06.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 3395, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 3162, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-09-10T07:53:06.116 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 97, 2024-09-10T07:53:06.117 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 11, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 12, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 12, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 1428, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 14, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 74 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-09-10T07:53:06.118 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 3788 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 5, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed_ops": 60, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 24, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 24661, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 8078720, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 24661, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 24661, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 8078720, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 8960, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 11780698, 2024-09-10T07:53:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 8960, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 11780698, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-09-10T07:53:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 8960, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 8960, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 8960, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 8960, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-09-10T07:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 24, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 2600, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 24, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 7, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 2600, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x55c133594960": { 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 3285, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 8904689, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 3285, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 1822, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 8904689, 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:06.125 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:06.126 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:06.126 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:06.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:06.126 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:53:06.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:06 smithi181 ceph-mon[35397]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T07:53:06.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:06 smithi063 ceph-mon[35020]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T07:53:06.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:06 smithi046 ceph-mon[30705]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T07:53:07.114 INFO:tasks.check_counter:Daemon mds.a mds.exported=26 2024-09-10T07:53:07.114 INFO:tasks.check_counter:Daemon mds.a mds.imported=3 2024-09-10T07:53:07.114 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T07:53:07.115 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T07:53:07.492 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:08.148 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:53:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 329, 2024-09-10T07:53:08.148 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 324, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 511472, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 199408, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.090525577, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.049193916, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.043992329, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 324, 2024-09-10T07:53:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.056016361, 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000172890 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 511472, 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 199408 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-09-10T07:53:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 246, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 200608, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.036887360, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.031427567, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002277607, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T07:53:08.151 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:08.152 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 244, 2024-09-10T07:53:08.152 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.046780898, 2024-09-10T07:53:08.152 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000191724 2024-09-10T07:53:08.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.152 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:08.153 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:08.153 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:08.153 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:08.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.153 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T07:53:08.153 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 200608 2024-09-10T07:53:08.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.154 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:08.154 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-09-10T07:53:08.154 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-09-10T07:53:08.154 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-09-10T07:53:08.154 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-09-10T07:53:08.154 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-09-10T07:53:08.155 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-09-10T07:53:08.155 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001198611, 2024-09-10T07:53:08.155 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T07:53:08.155 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000973327, 2024-09-10T07:53:08.155 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T07:53:08.155 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:08.155 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:08.156 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:08.156 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:08.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.156 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:08.156 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:08.156 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:08.156 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:08.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.157 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T07:53:08.157 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T07:53:08.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.157 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-09-10T07:53:08.157 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-09-10T07:53:08.157 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-09-10T07:53:08.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.158 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-09-10T07:53:08.158 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:08.158 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:08.158 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:08.158 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:08.159 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:08.159 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:08.159 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:08.159 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:08.159 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:08.159 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:08.160 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:08.161 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 149735, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 30, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:08.162 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-09-10T07:53:08.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 327, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 462536, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 327, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 327, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 462536, 2024-09-10T07:53:08.164 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:08.165 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:08.165 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:08.165 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:08.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:08.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:08.165 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:53:08.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:08 smithi181 ceph-mon[35397]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-09-10T07:53:08.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:08 smithi063 ceph-mon[35020]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-09-10T07:53:08.654 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:08 smithi046 ceph-mon[30705]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-09-10T07:53:09.126 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T07:53:09.126 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T07:53:09.126 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T07:53:09.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T07:53:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:10.115 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 4, 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2992, 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 448, 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.006503679, 2024-09-10T07:53:10.116 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000086522, 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.003494883, 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3, 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.006201007, 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002067002 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:10.117 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 243, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 200080, 2024-09-10T07:53:10.118 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.038501264, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.033228657, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002161799, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 241, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.059077574, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000245135 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:10.119 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 200080 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 321, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 318, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 507392, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 196400, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-09-10T07:53:10.120 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.080373192, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.048832856, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.037019746, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 317, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.061585625, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000194276 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:10.121 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 507392, 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 196400 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:10.122 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:10.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:10.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:10.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 149735, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 30, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:10.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 323, 2024-09-10T07:53:10.127 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 462016, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 323, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 323, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 462016, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:53:10.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:10 smithi181 ceph-mon[35397]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T07:53:10.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:10 smithi063 ceph-mon[35020]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T07:53:10.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:10 smithi046 ceph-mon[30705]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T07:53:11.378 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-10T07:53:11.378 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-10T07:53:11.378 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T07:53:11.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T07:53:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:12.391 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T07:53:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1492, 2024-09-10T07:53:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 1453, 2024-09-10T07:53:12.391 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1144128, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 558220, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.246819137, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.083952630, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.091946110, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.059670356, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1452, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.275027296, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000189412 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.392 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 767136, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 201424 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2236, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2480, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1651883, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 1204795, 2024-09-10T07:53:12.393 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.304504747, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.105865466, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.067431604, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.107780044, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2479, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.328768762, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000132621 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.394 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 495280 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 6, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1629, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 1284, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 8, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.004243603, 2024-09-10T07:53:12.395 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000080089, 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.003527763, 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000179978, 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 4, 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.005556168, 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001389042 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5463, 2024-09-10T07:53:12.397 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.227004530, 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000041553 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T07:53:12.398 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-09-10T07:53:12.399 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 225, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 39, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 186, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 169, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 0, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 0, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 9, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 0, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 0, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 0, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-09-10T07:53:12.400 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 0, 2024-09-10T07:53:12.401 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T07:53:12.402 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 0, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 0 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-09-10T07:53:12.403 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 0, 2024-09-10T07:53:12.404 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_path": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_inode": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-09-10T07:53:12.405 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "evlrg": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 803, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-09-10T07:53:12.406 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 0, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 2, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "segmjr": 2, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 4194304, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 5276193, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 5275619, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 803 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 222, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 252, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 30, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 19, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 20, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 1, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 225, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 256, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 31, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 0, 2024-09-10T07:53:12.408 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 0, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 0, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 49196, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 215328 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 0, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 0, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 0, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 0, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 0, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.409 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.410 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-09-10T07:53:12.411 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.412 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-09-10T07:53:12.413 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-09-10T07:53:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.415 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-09-10T07:53:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-09-10T07:53:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 1, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 0, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 0, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 0, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 2167, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:12.423 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:12.424 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 206165, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 270, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-09-10T07:53:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 164, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 696476, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 5825, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 3357, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 21, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 3, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 3357, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3357, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 4.746952054, 2024-09-10T07:53:12.427 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001414045 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3357, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 3358 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 3357, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 3357, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 0, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 2059, 2024-09-10T07:53:12.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 1296, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 1, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 0, 2024-09-10T07:53:12.429 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-09-10T07:53:12.430 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 97, 2024-09-10T07:53:12.431 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 2, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 6, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 9, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 3, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 0, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 2, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 0 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 0, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-09-10T07:53:12.432 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 0, 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed_ops": 0, 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 0, 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:12.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 3728, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 2440221, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 3728, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 3728, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 2440221, 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:12.434 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 3357, 2024-09-10T07:53:12.435 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 1081894, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 3357, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1081894, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:12.436 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 3357, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 3357, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 3357, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 3357, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 0, 2024-09-10T07:53:12.438 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x560c7c20cbe0": { 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:12.439 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 0, 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 0, 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T07:53:12.440 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T07:53:12.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:12 smithi181 ceph-mon[35397]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T07:53:12.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:12 smithi063 ceph-mon[35020]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T07:53:12.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:12 smithi046 ceph-mon[30705]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T07:53:13.344 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-09-10T07:53:13.344 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-09-10T07:53:13.344 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T07:53:13.344 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T07:53:13.618 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 4454, 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 4366, 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1788641, 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 3818364, 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 13, 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 5, 2024-09-10T07:53:14.253 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.297604963, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.106034754, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.168208022, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.036846219, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 4364, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.198681882, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000045527 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 5694, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.001098952, 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000193 2024-09-10T07:53:14.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1077, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1002, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1224911, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 619599, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 9, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 7, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.091924365, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.034949628, 2024-09-10T07:53:14.255 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.052110198, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.011912901, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1001, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.074141881, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000074067 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 271, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000058592, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000216 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 787504, 2024-09-10T07:53:14.256 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 216704 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 3040, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 4083, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1145468, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 2855091, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 46, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 10, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.192031901, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.083177080, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.117369502, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.007335496, 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:14.257 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 4082, 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.136436693, 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000033423 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 2886, 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000926486, 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000321 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 499328 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.258 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 1, 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-MDSRank": { 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 2720, 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.698258862, 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000256712 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-09-10T07:53:14.259 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 25, 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000645250, 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000025810 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "request": 1046, 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "reply": 1030, 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "reply_latency": { 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1030, 2024-09-10T07:53:14.260 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 5.923658018, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.005751124 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_reply": 0, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "forward": 16, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_complete": 2, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_keys": 1, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_commit": 11, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_split": 0, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_merge": 0, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes": 274, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_top": 0, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_bottom": 111, 2024-09-10T07:53:14.261 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pin_tail": 163, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pinned": 274, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_expired": 92, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_with_caps": 61, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": 61, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "subtrees": 7, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse": 3046, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_hit": 2310, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_forward": 16, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_discover": 3, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:14.262 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_lock": 205, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "load_cent": 1676, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "q": 0, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "exported": 2, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "exported_inodes": 88, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "imported": 3, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "imported_inodes": 116, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_backtrace_fetch": 1066, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rfiles": 1018, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rbytes": 1507456431, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rsnaps": 0, 2024-09-10T07:53:14.263 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_fetch": 1066, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_inodes": 15, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dirfrag_rstats": 15, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_file_inodes": 1051, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_inode_file_caps": 4, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_revoke": 596, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_grant": 867, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:14.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps": 1129, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps_dirty": 386, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_cap_release": 53, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "process_request_cap_release": 525 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache": { 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update": 0, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_try_discover": 0, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_send_discover": 3, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_handle_discover": 1, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays": 0, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:14.265 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_created": 123, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_enqueued": 2, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_reintegrated": 121, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_migrated": 0, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_started": 0, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_completed": 0, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_path": 15, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_inode": 1173, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_exportdir": 2, 2024-09-10T07:53:14.266 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_flush": 0, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_inodestats": 0 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log": { 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "evlrg": 0, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "evadd": 1273, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "evex": 0, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "evtrm": 0, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "ev": 1273, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "evexg": 0, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "evexd": 0, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "segadd": 3, 2024-09-10T07:53:14.267 INFO:teuthology.orchestra.run.smithi063.stdout: "segex": 0, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "segtrm": 0, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "seg": 3, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "segmjr": 2, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "segexg": 0, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "segexd": 0, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "expos": 4194304, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "wrpos": 6254013, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "rdpos": 0, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "jlat": { 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 929, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 16.756783844, 2024-09-10T07:53:14.268 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.018037442 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "replayed": 0 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_mem": { 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "ino": 77, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "ino+": 159, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "ino-": 82, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "dir": 23, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "dir+": 25, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "dir-": 2, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "dn": 274, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "dn+": 368, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "dn-": 94, 2024-09-10T07:53:14.269 INFO:teuthology.orchestra.run.smithi063.stdout: "cap": 61, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "cap+": 294, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "cap-": 233, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "rss": 57280, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "heap": 223520 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_server": { 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_client_request": 1964, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_server_request": 0, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_request": 1046, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_session": 264, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_peer_request": 400, 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "req_create_latency": { 2024-09-10T07:53:14.270 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 32, 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.062135522, 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.001941735 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getattr_latency": { 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 108, 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 4.020805632, 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.037229681 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "req_link_latency": { 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 121, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.309969515, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.002561731 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookup_latency": { 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 442, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.463654301, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.001048991 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.272 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupino_latency": { 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupname_latency": { 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.273 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lssnap_latency": { 2024-09-10T07:53:14.274 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mkdir_latency": { 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mknod_latency": { 2024-09-10T07:53:14.275 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mksnap_latency": { 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "req_open_latency": { 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 78, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.039632716, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000508111 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.276 INFO:teuthology.orchestra.run.smithi063.stdout: "req_readdir_latency": { 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 5, 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.007515882, 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.001503176 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rename_latency": { 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 121, 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.806274432, 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.006663425 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.277 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmdir_latency": { 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.278 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setattr_latency": { 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.279 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setlayout_latency": { 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.280 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setxattr_latency": { 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "req_symlink_latency": { 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "req_unlink_latency": { 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 123, 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.213670018, 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.001737154 2024-09-10T07:53:14.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_sessions": { 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "session_count": 1, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "session_add": 2, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "session_remove": 1, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_open": 1, 2024-09-10T07:53:14.282 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_stale": 0, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "total_load": 188, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "average_load": 188, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "avg_session_uptime": 2397, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:14.283 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:14.284 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:14.285 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 290033, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 166, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 164, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 453499, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 14375, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter": { 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "op_active": 0, 2024-09-10T07:53:14.287 INFO:teuthology.orchestra.run.smithi063.stdout: "op_laggy": 0, 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send": 2235, 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send_bytes": 2111527, 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: "op_resend": 33, 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: "op_reply": 2235, 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: "op_latency": { 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 2235, 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 25.687717581, 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.011493385 2024-09-10T07:53:14.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "op_inflight": 0, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "oplen_avg": { 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 2235, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 2402 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "op": 2235, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "op_r": 1070, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "op_w": 1165, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "op_rmw": 0, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "op_pg": 0, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_stat": 0, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_create": 10, 2024-09-10T07:53:14.289 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_read": 1, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_write": 930, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writefull": 61, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writesame": 0, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_append": 0, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_zero": 0, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_truncate": 0, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_delete": 22, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_mapext": 0, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_getxattr": 1069, 2024-09-10T07:53:14.290 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_setxattr": 20, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_call": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_watch": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_notify": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_other": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_active": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_send": 0, 2024-09-10T07:53:14.291 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_resend": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_ping": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_active": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_send": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_resend": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_active": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_send": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_resend": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_active": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_send": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_resend": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "command_active": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "command_send": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "command_resend": 0, 2024-09-10T07:53:14.292 INFO:teuthology.orchestra.run.smithi063.stdout: "map_epoch": 97, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "map_full": 0, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "map_inc": 2, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_sessions": 12, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_open": 12, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_close": 0, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_laggy": 0, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_wr": 235, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_rd": 6, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_del": 48 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "oft": { 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_objs": 1, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:14.293 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_updates": 0, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_removes": 0 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "osd-slow-ops": { 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_ops_count": 995 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "purge_queue": { 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing": 0, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_high_water": 2, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed_ops": 10, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed": 2, 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:14.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 8567, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 3403504, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 8567, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 8567, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 3403504, 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:14.295 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 2235, 2024-09-10T07:53:14.296 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 2111709, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 2235, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 2111709, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 1024, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:14.297 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 2235, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 2235, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 2235, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 2235, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.298 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 2, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 218, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 2, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 1, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 218, 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:14.299 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle-0x5575232e8960": { 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 1273, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 2059709, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 1273, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:14.300 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 929, 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 2059709, 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:14.301 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-10T07:53:14.523 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:14 smithi063 ceph-mon[35020]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-10T07:53:14.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:14 smithi181 ceph-mon[35397]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-10T07:53:14.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:14 smithi046 ceph-mon[30705]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-10T07:53:15.443 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-09-10T07:53:15.443 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-09-10T07:53:15.443 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T07:53:15.444 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T07:53:15.704 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1373, 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1373, 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 4092524, 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 435854, 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 7, 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 2, 2024-09-10T07:53:16.376 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.113398484, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.040643337, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.031313810, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.031118718, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1372, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.054993585, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000040082 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.377 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 941, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1186, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 231336, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 883167, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 6, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 4, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.091552649, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.029988370, 2024-09-10T07:53:16.378 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.020299803, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.031558870, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1184, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.049298483, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000041637 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 584560 2024-09-10T07:53:16.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1487, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1448, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1142512, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 556308, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 3, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 2, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.158906856, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.054661284, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.061608254, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.038741210, 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:16.380 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1447, 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.091305276, 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000063099 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 766864, 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 200784 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-09-10T07:53:16.381 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 1, 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-MDSRank": { 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 5872, 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.148745939, 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000025331 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-09-10T07:53:16.382 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "request": 0, 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "reply": 0, 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "reply_latency": { 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.383 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_reply": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "forward": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_complete": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_keys": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_commit": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_split": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_merge": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes": 401, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_top": 110, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_bottom": 291, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pin_tail": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pinned": 243, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_expired": 0, 2024-09-10T07:53:16.384 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_with_caps": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "subtrees": 11, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_hit": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_forward": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_discover": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_lock": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "load_cent": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "q": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "exported": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "exported_inodes": 0, 2024-09-10T07:53:16.385 INFO:teuthology.orchestra.run.smithi063.stdout: "imported": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "imported_inodes": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_backtrace_fetch": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rfiles": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rbytes": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rsnaps": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_inodes": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:16.386 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_file_inodes": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_inode_file_caps": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_grant": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps_dirty": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_cap_release": 0, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "process_request_cap_release": 0 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache": { 2024-09-10T07:53:16.387 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_try_discover": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_send_discover": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_handle_discover": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_created": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_enqueued": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_reintegrated": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_migrated": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:16.388 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_started": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_completed": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_path": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_inode": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_exportdir": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_flush": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_inodestats": 0 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log": { 2024-09-10T07:53:16.389 INFO:teuthology.orchestra.run.smithi063.stdout: "evlrg": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "evadd": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "evex": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "evtrm": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "ev": 1662, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "evexg": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "evexd": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "segadd": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "segex": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "segtrm": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "seg": 3, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "segmjr": 2, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "segexg": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "segexd": 0, 2024-09-10T07:53:16.390 INFO:teuthology.orchestra.run.smithi063.stdout: "expos": 4194304, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "wrpos": 7928487, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "rdpos": 7927026, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "jlat": { 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "replayed": 1662 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_mem": { 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "ino": 189, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "ino+": 992, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "ino-": 803, 2024-09-10T07:53:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "dir": 33, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "dir+": 62, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "dir-": 29, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "dn": 401, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "dn+": 1021, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "dn-": 620, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "cap": 0, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "cap+": 0, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "cap-": 0, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "rss": 51740, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "heap": 215328 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_server": { 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_client_request": 0, 2024-09-10T07:53:16.392 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_server_request": 0, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_request": 0, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_session": 0, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_peer_request": 0, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "req_create_latency": { 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getattr_latency": { 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.393 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "req_link_latency": { 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookup_latency": { 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupino_latency": { 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupname_latency": { 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.395 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lssnap_latency": { 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.396 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.397 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.397 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mkdir_latency": { 2024-09-10T07:53:16.397 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.397 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mknod_latency": { 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mksnap_latency": { 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.398 INFO:teuthology.orchestra.run.smithi063.stdout: "req_open_latency": { 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "req_readdir_latency": { 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rename_latency": { 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmdir_latency": { 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.400 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setattr_latency": { 2024-09-10T07:53:16.401 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.402 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setlayout_latency": { 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setxattr_latency": { 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "req_symlink_latency": { 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.403 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "req_unlink_latency": { 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.404 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_sessions": { 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "session_count": 1, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "session_add": 1, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "session_remove": 0, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_open": 1, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_stale": 0, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "total_load": 0, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "average_load": 0, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "avg_session_uptime": 2396, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-09-10T07:53:16.405 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:16.406 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 197651, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 235, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:16.408 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 164, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 760267, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 5587, 2024-09-10T07:53:16.409 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter": { 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "op_active": 0, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "op_laggy": 0, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send": 3426, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send_bytes": 21, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "op_resend": 3, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "op_reply": 3426, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "op_latency": { 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3426, 2024-09-10T07:53:16.410 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 12.651859822, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.003692895 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "op_inflight": 0, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "oplen_avg": { 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3426, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 3427 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "op": 3426, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "op_r": 3426, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "op_w": 0, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "op_rmw": 0, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "op_pg": 0, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_stat": 2055, 2024-09-10T07:53:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_create": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_read": 1369, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_write": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writefull": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writesame": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_append": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_zero": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_truncate": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_delete": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_mapext": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_getxattr": 1, 2024-09-10T07:53:16.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_setxattr": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_call": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_watch": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_notify": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_other": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_active": 0, 2024-09-10T07:53:16.413 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_send": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_resend": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_ping": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_active": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_send": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_resend": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_active": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_send": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_resend": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_active": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_send": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_resend": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "command_active": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "command_send": 0, 2024-09-10T07:53:16.414 INFO:teuthology.orchestra.run.smithi063.stdout: "command_resend": 0, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "map_epoch": 97, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "map_full": 0, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "map_inc": 2, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_sessions": 5, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_open": 6, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_close": 1, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_laggy": 0, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_wr": 0, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_rd": 2, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_del": 0 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "oft": { 2024-09-10T07:53:16.415 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_objs": 0, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_updates": 0, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_removes": 0 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "osd-slow-ops": { 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_ops_count": 0 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "purge_queue": { 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing": 0, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_high_water": 0, 2024-09-10T07:53:16.416 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed_ops": 0, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed": 0, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 3797, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 5102752, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 3797, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:16.417 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3797, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 5102752, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:16.418 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 3426, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 3734188, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3426, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 3734188, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:16.419 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 1024, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 3426, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 3426, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3426, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 3426, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle-0x55dc236feb40": { 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:16.423 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-10T07:53:16.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:16 smithi181 ceph-mon[35397]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-09-10T07:53:16.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:16 smithi063 ceph-mon[35020]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-09-10T07:53:16.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:16 smithi046 ceph-mon[30705]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-09-10T07:53:17.160 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T07:53:17.160 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T07:53:17.160 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T07:53:17.161 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T07:53:17.422 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:53:17.917 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-10T07:53:17.917 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:17.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1685, 2024-09-10T07:53:17.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1644, 2024-09-10T07:53:17.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 5390446, 2024-09-10T07:53:17.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 617972, 2024-09-10T07:53:17.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 5, 2024-09-10T07:53:17.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 1, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.173479468, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.068215217, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.063299007, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.036858378, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1644, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.081515127, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000049583 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 769744, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 200752 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1107, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1351, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 790999, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 935840, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 7, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 4, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.098306013, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.029651598, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.024691498, 2024-09-10T07:53:17.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.035104451, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1349, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.050630208, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000037531 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 584384 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.920 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1198, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1200, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 4502222, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 380971, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 3, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.093345884, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.024748588, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.028941419, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.031834560, 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:17.921 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1198, 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.048582655, 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000040553 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-09-10T07:53:17.922 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 1, 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-MDSRank": { 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 5991, 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.161055881, 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000026882 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-09-10T07:53:17.923 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "request": 0, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "reply": 0, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "reply_latency": { 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_reply": 0, 2024-09-10T07:53:17.924 INFO:teuthology.orchestra.run.smithi063.stdout: "forward": 0, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_complete": 0, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_keys": 0, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_commit": 0, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_split": 0, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_merge": 0, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes": 345, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_top": 230, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_bottom": 115, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pin_tail": 0, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pinned": 15, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_expired": 0, 2024-09-10T07:53:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_with_caps": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "subtrees": 19, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_hit": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_forward": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_discover": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_lock": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "load_cent": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "q": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "exported": 0, 2024-09-10T07:53:17.926 INFO:teuthology.orchestra.run.smithi063.stdout: "exported_inodes": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "imported": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "imported_inodes": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_backtrace_fetch": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rfiles": 1015, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rbytes": 1477148501, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rsnaps": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_inodes": 0, 2024-09-10T07:53:17.927 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_file_inodes": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_inode_file_caps": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_grant": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps_dirty": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_cap_release": 0, 2024-09-10T07:53:17.928 INFO:teuthology.orchestra.run.smithi063.stdout: "process_request_cap_release": 0 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache": { 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_try_discover": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_send_discover": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_handle_discover": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_created": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_enqueued": 0, 2024-09-10T07:53:17.929 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_reintegrated": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_migrated": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_started": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_completed": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_path": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_inode": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_exportdir": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_flush": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:17.930 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_inodestats": 0 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log": { 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "evlrg": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "evadd": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "evex": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "evtrm": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "ev": 3285, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "evexg": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "evexd": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "segadd": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "segex": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "segtrm": 0, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "seg": 2, 2024-09-10T07:53:17.931 INFO:teuthology.orchestra.run.smithi063.stdout: "segmjr": 74, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "segexg": 0, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "segexd": 0, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "expos": 13039812, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "wrpos": 13098993, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "rdpos": 13097341, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "jlat": { 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "replayed": 3285 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_mem": { 2024-09-10T07:53:17.932 INFO:teuthology.orchestra.run.smithi063.stdout: "ino": 296, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "ino+": 1596, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "ino-": 1300, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "dir": 37, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "dir+": 93, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "dir-": 56, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "dn": 345, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "dn+": 1655, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "dn-": 1310, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "cap": 0, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "cap+": 0, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "cap-": 0, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "rss": 51768, 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: "heap": 215328 2024-09-10T07:53:17.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_server": { 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_client_request": 0, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_server_request": 0, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_request": 0, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_session": 0, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_peer_request": 0, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "req_create_latency": { 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getattr_latency": { 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "req_link_latency": { 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookup_latency": { 2024-09-10T07:53:17.935 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupino_latency": { 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupname_latency": { 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.937 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.938 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.938 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lssnap_latency": { 2024-09-10T07:53:17.938 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.938 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.938 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.938 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mkdir_latency": { 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mknod_latency": { 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mksnap_latency": { 2024-09-10T07:53:17.939 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "req_open_latency": { 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "req_readdir_latency": { 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rename_latency": { 2024-09-10T07:53:17.940 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.942 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmdir_latency": { 2024-09-10T07:53:17.942 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.942 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.942 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.942 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:17.943 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.943 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.943 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.943 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:17.943 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.943 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setattr_latency": { 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:17.944 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setlayout_latency": { 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setxattr_latency": { 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.945 INFO:teuthology.orchestra.run.smithi063.stdout: "req_symlink_latency": { 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: "req_unlink_latency": { 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.946 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:17.947 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:17.947 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:17.947 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.947 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.947 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.947 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_sessions": { 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "session_count": 1, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "session_add": 3, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "session_remove": 2, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_open": 1, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_stale": 0, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "total_load": 0, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "average_load": 0, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "avg_session_uptime": 2399, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-09-10T07:53:17.948 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:17.949 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:17.950 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 200767, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 339, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:17.951 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 164, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 972171, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 7455, 2024-09-10T07:53:17.952 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter": { 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "op_active": 0, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "op_laggy": 0, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send": 3615, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send_bytes": 26, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "op_resend": 4, 2024-09-10T07:53:17.953 INFO:teuthology.orchestra.run.smithi063.stdout: "op_reply": 3615, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "op_latency": { 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3615, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 18.777795005, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.005194410 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "op_inflight": 0, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "oplen_avg": { 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3615, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 3616 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "op": 3615, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "op_r": 3615, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "op_w": 0, 2024-09-10T07:53:17.954 INFO:teuthology.orchestra.run.smithi063.stdout: "op_rmw": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "op_pg": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_stat": 2228, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_create": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_read": 1384, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_write": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writefull": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writesame": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_append": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_zero": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_truncate": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_delete": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_mapext": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:17.955 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_getxattr": 2, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_setxattr": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_call": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_watch": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_notify": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_other": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_active": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_send": 0, 2024-09-10T07:53:17.956 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_resend": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_ping": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_active": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_send": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_resend": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_active": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_send": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_resend": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_active": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_send": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_resend": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "command_active": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "command_send": 0, 2024-09-10T07:53:17.957 INFO:teuthology.orchestra.run.smithi063.stdout: "command_resend": 0, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "map_epoch": 97, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "map_full": 0, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "map_inc": 2, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_sessions": 5, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_open": 6, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_close": 1, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_laggy": 0, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_wr": 0, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_rd": 2, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_del": 0 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "oft": { 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_objs": 0, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:17.958 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_updates": 0, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_removes": 0 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "osd-slow-ops": { 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_ops_count": 0 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "purge_queue": { 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing": 0, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_high_water": 0, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed_ops": 0, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed": 0, 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:17.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 3986, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 10303043, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 3986, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3986, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 10303043, 2024-09-10T07:53:17.960 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 3615, 2024-09-10T07:53:17.961 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 8904699, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3615, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 8904699, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 1024, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:17.962 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 3615, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 3615, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3615, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 3615, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:17.963 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:17.964 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle-0x55b57c604b40": { 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:17.965 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:17.966 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-10T07:53:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:18 smithi181 ceph-mon[35397]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-09-10T07:53:18.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:18 smithi063 ceph-mon[35020]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-09-10T07:53:18.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:18 smithi046 ceph-mon[30705]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-09-10T07:53:18.730 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-10T07:53:18.730 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-10T07:53:18.730 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T07:53:18.730 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T07:53:18.982 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:53:19.493 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-10T07:53:19.493 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:19.493 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 1455, 2024-09-10T07:53:19.493 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 1419, 2024-09-10T07:53:19.493 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1114656, 2024-09-10T07:53:19.493 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 548374, 2024-09-10T07:53:19.493 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 5, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 1, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.156285204, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.054193405, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.060126537, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.036687623, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 1418, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.077748474, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000054829 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.494 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 750096, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 203344 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 2281, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 2529, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1757888, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 1221897, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 5, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 4, 2024-09-10T07:53:19.495 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.159073263, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.060241857, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.035909828, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.049511639, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 2527, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.095881492, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000037942 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.496 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 496800 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 8, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 8, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 4621, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 1732, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 7, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 4, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.004009893, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000113235, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.003600764, 2024-09-10T07:53:19.497 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000086437, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 7, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.006539460, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000934208 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.498 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 1, 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-MDSRank": { 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 5711, 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.132997875, 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000023288 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:19.499 INFO:teuthology.orchestra.run.smithi063.stdout: "queue_len": 0, 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "complete_latency": { 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "request": 0, 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "reply": 0, 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "reply_latency": { 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_reply": 0, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "forward": 0, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_complete": 0, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_fetch_keys": 0, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_commit": 0, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_split": 0, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_merge": 0, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes": 342, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_top": 141, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_bottom": 201, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pin_tail": 0, 2024-09-10T07:53:19.501 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_pinned": 140, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_expired": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "inodes_with_caps": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "subtrees": 12, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_hit": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_forward": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_discover": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "traverse_lock": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "load_cent": 0, 2024-09-10T07:53:19.502 INFO:teuthology.orchestra.run.smithi063.stdout: "q": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "exported": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "exported_inodes": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "imported": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "imported_inodes": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_backtrace_fetch": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rfiles": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rbytes": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "root_rsnaps": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:19.503 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_inodes": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_file_inodes": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_inode_file_caps": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_grant": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_caps_dirty": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_cap_release": 0, 2024-09-10T07:53:19.504 INFO:teuthology.orchestra.run.smithi063.stdout: "process_request_cap_release": 0 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache": { 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_try_discover": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_send_discover": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "dir_handle_discover": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_created": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_enqueued": 0, 2024-09-10T07:53:19.505 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_reintegrated": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "strays_migrated": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_started": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_completed": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_path": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_quiesce_inode": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:19.506 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_exportdir": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_flush": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "ireq_inodestats": 0 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log": { 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "evlrg": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "evadd": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "evex": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "evtrm": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "ev": 859, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "evexg": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "evexd": 0, 2024-09-10T07:53:19.507 INFO:teuthology.orchestra.run.smithi063.stdout: "segadd": 0, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "segex": 0, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "segtrm": 0, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "seg": 2, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "segmjr": 2, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "segexg": 0, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "segexd": 0, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "expos": 4194304, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "wrpos": 5373728, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "rdpos": 5369950, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "jlat": { 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "replayed": 859 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_mem": { 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "ino": 339, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "ino+": 341, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "ino-": 2, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "dir": 20, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "dir+": 20, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "dir-": 0, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "dn": 342, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "dn+": 342, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "dn-": 0, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "cap": 0, 2024-09-10T07:53:19.509 INFO:teuthology.orchestra.run.smithi063.stdout: "cap+": 0, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "cap-": 0, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "rss": 50636, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "heap": 215328 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_server": { 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_client_request": 0, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "dispatch_server_request": 0, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_request": 0, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_client_session": 0, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "handle_peer_request": 0, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "req_create_latency": { 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.510 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getattr_latency": { 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "req_link_latency": { 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.511 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookup_latency": { 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupino_latency": { 2024-09-10T07:53:19.512 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupname_latency": { 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.513 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: "req_lssnap_latency": { 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.514 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mkdir_latency": { 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mknod_latency": { 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.515 INFO:teuthology.orchestra.run.smithi063.stdout: "req_mksnap_latency": { 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "req_open_latency": { 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "req_readdir_latency": { 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rename_latency": { 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.517 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmdir_latency": { 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.518 INFO:teuthology.orchestra.run.smithi063.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setattr_latency": { 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.519 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setlayout_latency": { 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.520 INFO:teuthology.orchestra.run.smithi063.stdout: "req_setxattr_latency": { 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "req_symlink_latency": { 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "req_unlink_latency": { 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_sessions": { 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "session_count": 1, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "session_add": 2, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "session_remove": 1, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_open": 1, 2024-09-10T07:53:19.522 INFO:teuthology.orchestra.run.smithi063.stdout: "sessions_stale": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "total_load": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "average_load": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "avg_session_uptime": 2400, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:19.523 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:19.524 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:19.525 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 210312, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 382, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 164, 2024-09-10T07:53:19.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 1046103, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 8465, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter": { 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "op_active": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "op_laggy": 0, 2024-09-10T07:53:19.527 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send": 3369, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "op_send_bytes": 21, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "op_resend": 3, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "op_reply": 3369, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "op_latency": { 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3369, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 48.489465354, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.014392836 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "op_inflight": 0, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "oplen_avg": { 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3369, 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 3370 2024-09-10T07:53:19.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "op": 3369, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "op_r": 3369, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "op_w": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "op_rmw": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "op_pg": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_stat": 2094, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_create": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_read": 1273, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_write": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writefull": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_writesame": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_append": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_zero": 0, 2024-09-10T07:53:19.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_truncate": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_delete": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_mapext": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_getxattr": 1, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_setxattr": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_call": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_watch": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_notify": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:19.530 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "osdop_other": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_active": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_send": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_resend": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "linger_ping": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_active": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_send": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "poolop_resend": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_active": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_send": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "poolstat_resend": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_active": 0, 2024-09-10T07:53:19.531 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_send": 0, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "statfs_resend": 0, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "command_active": 0, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "command_send": 0, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "command_resend": 0, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "map_epoch": 97, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "map_full": 0, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "map_inc": 2, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_sessions": 6, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_open": 8, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_session_close": 2, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_laggy": 0, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_wr": 0, 2024-09-10T07:53:19.532 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_rd": 2, 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_del": 0 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "oft": { 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_objs": 0, 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_updates": 0, 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "omap_total_removes": 0 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "osd-slow-ops": { 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "slow_ops_count": 0 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "purge_queue": { 2024-09-10T07:53:19.533 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing": 0, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executing_high_water": 0, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed_ops": 0, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_executed": 0, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 3740, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 2518645, 2024-09-10T07:53:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 3740, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3740, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 2518645, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:19.535 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 3369, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 1179429, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3369, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 1179429, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.536 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 1024, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 3369, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 3369, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 3369, 2024-09-10T07:53:19.537 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 3369, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:19.538 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-write_buf_throttle-0x5651d46dab40": { 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 3758096384, 2024-09-10T07:53:19.539 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.540 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-10T07:53:19.541 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-10T07:53:20.230 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-10T07:53:20.231 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-10T07:53:20.231 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T07:53:20.231 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T07:53:20.577 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:53:20.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:20 smithi181 ceph-mon[35397]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T07:53:20.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:20 smithi063 ceph-mon[35020]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T07:53:20.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:20 smithi046 ceph-mon[30705]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T07:53:21.194 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T07:53:21.194 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:21.194 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3336, 2024-09-10T07:53:21.194 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 3579, 2024-09-10T07:53:21.194 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1742693, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1399435, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 11, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.254738172, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.102538175, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.137163052, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.026315106, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3577, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.462197592, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000129213 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4546, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000888655, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000195 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3787, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 4767, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 2292119, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 3421410, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 18, 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.317052540, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.127737307, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.145092478, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.049975911, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4767, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.537312275, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000112714 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2028, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000817599, 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000403 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 783424, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 225936 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2002, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1885, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1142967, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1559679, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 39, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 9, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.165555885, 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.063197739, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.076996299, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.023758031, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1883, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.134417229, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000071384 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1333, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000223307, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000167 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 498784 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3305, 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.810495795, 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000245233 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 17, 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000504220, 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000029660 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 424, 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 419, 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 419, 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 6.375220844, 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.015215324 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 5, 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-09-10T07:53:21.202 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 229, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 216, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 13, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 229, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 34, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 212, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 212, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 12, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 663, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 646, 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 5, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 8, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 139, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 1, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 31, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 8, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 241, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 3069, 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 1018, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 1507456431, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 3069, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 20, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 20, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 3049, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 549, 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 1039, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 870, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 144, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 29, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 12 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 8, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 9, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 2, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 2, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 15, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 3264, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 1, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 805, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-09-10T07:53:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 805, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 2, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 2, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 2, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 5279655, 2024-09-10T07:53:21.209 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 569, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 8.301306098, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.014589290 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 232, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 261, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 29, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 29, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 30, 2024-09-10T07:53:21.210 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 1, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 229, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 265, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 36, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 210, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 508, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 298, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 58232, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 223520 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 443, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 2, 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 424, 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 15, 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 9, 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3, 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002295361, 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000765120 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-09-10T07:53:21.212 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 99, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 6.202125212, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.062647729 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.213 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 181, 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.081594014, 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000450795 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.215 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.216 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-09-10T07:53:21.216 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.216 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.216 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.217 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 125, 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.079138836, 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000633110 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 9, 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.005662598, 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000629177 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.218 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.222 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.004404823, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002202411 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.223 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 1, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 41, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 41, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 2425, 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:21.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:21.225 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:21.226 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 319912, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 330, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:21.227 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 164, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 835653, 2024-09-10T07:53:21.228 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 9091, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 3810, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 1144151, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 3810, 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-09-10T07:53:21.229 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3810, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 23.243553235, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006100670 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3810, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 3907 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 3810, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 3070, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 740, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-09-10T07:53:21.230 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 1, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 570, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 59, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 16, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 3069, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-09-10T07:53:21.231 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-09-10T07:53:21.232 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 97, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-09-10T07:53:21.233 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 126, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 36 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-09-10T07:53:21.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 2837 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 4, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 2, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 4, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 2, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:21.235 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 9121, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 4354529, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 9121, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 9121, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 4354529, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.236 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 3810, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 1144369, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3810, 2024-09-10T07:53:21.237 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1144369, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:21.238 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 3810, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 3810, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3810, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 3810, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-10T07:53:21.239 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 2, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 218, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 2, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 218, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:21.240 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55d409a94960": { 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 3462, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 805, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1085351, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 805, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 569, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1081889, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:21.241 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:21.242 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:21.242 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.242 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:21.242 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T07:53:22.130 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-09-10T07:53:22.130 INFO:tasks.check_counter:Daemon mds.c mds.imported=8 2024-09-10T07:53:22.130 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T07:53:22.130 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T07:53:22.378 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:53:22.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:22 smithi181 ceph-mon[35397]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:53:22.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:22 smithi063 ceph-mon[35020]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:53:22.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:22 smithi046 ceph-mon[30705]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T07:53:22.882 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T07:53:22.882 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:22.882 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3533, 2024-09-10T07:53:22.882 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 3533, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 3050920, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1122677, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.282276079, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.081127554, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.072651639, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.109011666, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3532, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.126988130, 2024-09-10T07:53:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000035953 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 4, 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 250, 2024-09-10T07:53:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 876, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 587050, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 5, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.014388784, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.010384703, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.001694369, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000065779, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 248, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.017412999, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000070213 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 586352 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 371, 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 332, 2024-09-10T07:53:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 767404, 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 201742, 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.045844347, 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.020664740, 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.031207491, 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000019724, 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 331, 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.019096968, 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000057694 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 767136, 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 201424 2024-09-10T07:53:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-09-10T07:53:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-09-10T07:53:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-09-10T07:53:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-09-10T07:53:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-10T07:53:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-10T07:53:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5956, 2024-09-10T07:53:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.142605028, 2024-09-10T07:53:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000023943 2024-09-10T07:53:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-10T07:53:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 269, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 3, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 266, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-09-10T07:53:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 265, 2024-09-10T07:53:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-09-10T07:53:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-09-10T07:53:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-09-10T07:53:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 3, 2024-09-10T07:53:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-09-10T07:53:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-09-10T07:53:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-09-10T07:53:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-09-10T07:53:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 0, 2024-09-10T07:53:22.897 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 1273, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-09-10T07:53:22.898 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 3, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 2, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 6254013, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 6253439, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.899 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 1273 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 67, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 152, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 85, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 12, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 16, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 4, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 269, 2024-09-10T07:53:22.900 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 361, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 92, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 50328, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 215328 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-09-10T07:53:22.901 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.903 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.904 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.906 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.907 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.909 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.910 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.912 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 2, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 1, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 2428, 2024-09-10T07:53:22.914 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:22.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:22.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 184233, 2024-09-10T07:53:22.917 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 106, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:22.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 164, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 370985, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 11647, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-09-10T07:53:22.919 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 3532, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 21, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 3, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 3532, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3532, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 18.283650844, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005176571 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-09-10T07:53:22.920 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3532, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 3533 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 3532, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 3532, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 2177, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 1353, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2024-09-10T07:53:22.921 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 1, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:22.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-09-10T07:53:22.923 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 97, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-09-10T07:53:22.924 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 4, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 5, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 1, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-09-10T07:53:22.925 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 0, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 3904, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 3445898, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 3904, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3904, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 3445898, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.927 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 3532, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 2059714, 2024-09-10T07:53:22.928 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3532, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2059714, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:22.929 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 3532, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 3532, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3532, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 3532, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:22.930 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.931 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x563c59122b40": { 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-09-10T07:53:22.932 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-09-10T07:53:22.933 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:22.933 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:22.933 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:22.933 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:22.933 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.933 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:22.933 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T07:53:23.676 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-10T07:53:23.676 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-10T07:53:23.676 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T07:53:23.677 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T07:53:23.923 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:53:24.421 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T07:53:24.421 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:24.422 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 4748, 2024-09-10T07:53:24.422 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 5688, 2024-09-10T07:53:24.422 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 3446597, 2024-09-10T07:53:24.422 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 3012676, 2024-09-10T07:53:24.422 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 9, 2024-09-10T07:53:24.422 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-09-10T07:53:24.423 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.387535981, 2024-09-10T07:53:24.423 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.165696263, 2024-09-10T07:53:24.423 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.234101798, 2024-09-10T07:53:24.423 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.021432947, 2024-09-10T07:53:24.423 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:24.423 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5690, 2024-09-10T07:53:24.424 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.377265725, 2024-09-10T07:53:24.424 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000066303 2024-09-10T07:53:24.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.424 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:24.424 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 7623, 2024-09-10T07:53:24.425 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001557933, 2024-09-10T07:53:24.425 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000204 2024-09-10T07:53:24.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.425 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 787888, 2024-09-10T07:53:24.425 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 221488 2024-09-10T07:53:24.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.426 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:24.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 4339, 2024-09-10T07:53:24.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 4615, 2024-09-10T07:53:24.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 2031463, 2024-09-10T07:53:24.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 5878046, 2024-09-10T07:53:24.426 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 17, 2024-09-10T07:53:24.427 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-09-10T07:53:24.427 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.323809148, 2024-09-10T07:53:24.427 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.120452577, 2024-09-10T07:53:24.427 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.109744686, 2024-09-10T07:53:24.427 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.078229267, 2024-09-10T07:53:24.427 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4613, 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.252490204, 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000054734 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2033, 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000353791, 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000174 2024-09-10T07:53:24.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 500528 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3845, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 5136, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1550412, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 2529471, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 42, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 10, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.272636518, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.095952717, 2024-09-10T07:53:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.134057937, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.035128394, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5132, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.427229409, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000083248 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2494, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000932167, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000373 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T07:53:24.430 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T07:53:24.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.431 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4761, 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.568651006, 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000329479 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 659, 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.030601548, 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000046436 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 913, 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 790, 2024-09-10T07:53:24.433 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 790, 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 44.654996223, 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.056525311 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 123, 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 7, 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 6, 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-09-10T07:53:24.434 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 213, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 196, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 17, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 213, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 624, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 176, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 176, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 12, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 1600, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 1322, 2024-09-10T07:53:24.435 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 123, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 5, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 22, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 277, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 5, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 617, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 11, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 800, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:24.436 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 3937, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 1018, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 1507456431, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 3937, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 217, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 217, 2024-09-10T07:53:24.437 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 3720, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 19, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 521, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 970, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 1057, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 247, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 113, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 540 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-09-10T07:53:24.438 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 31, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 7, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 195, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 195, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:24.439 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 15, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 4374, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 5, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-09-10T07:53:24.440 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 1664, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 1664, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 3, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 3, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 2, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-09-10T07:53:24.441 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 7930207, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 814, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 3.400641991, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004177692 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 197, 2024-09-10T07:53:24.442 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 1004, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 807, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 50, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 70, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 20, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 213, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 1034, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 821, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 176, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 738, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 562, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 63880, 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 223520 2024-09-10T07:53:24.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 1118, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 301, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 913, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 124, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 726, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 194, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 39.532757315, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.203777099 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-09-10T07:53:24.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 78, 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 4.310113162, 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.055257861 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 187, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.276735147, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001479867 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-09-10T07:53:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.448 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-09-10T07:53:24.449 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 125, 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.077240422, 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000617923 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 8, 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.011197141, 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001399642 2024-09-10T07:53:24.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 185, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.427909925, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002313026 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.452 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.454 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-09-10T07:53:24.455 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 13, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.019043111, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001464854 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-09-10T07:53:24.456 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 1, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 84, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 84, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 2430, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:24.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:24.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 310569, 2024-09-10T07:53:24.460 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 292, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 164, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:24.461 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 746859, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 6691, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 5581, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 3859567, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 28, 2024-09-10T07:53:24.462 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 5581, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5581, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 39.465243790, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.007071357 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5581, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 5780 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 5581, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 4023, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 1558, 2024-09-10T07:53:24.463 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 73, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 887, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 115, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 402, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:24.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 3950, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-09-10T07:53:24.465 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-09-10T07:53:24.466 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 97, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 223, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 26, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 74 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.467 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 3501 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-09-10T07:53:24.468 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 5, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 390, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 195, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 12928, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 5875798, 2024-09-10T07:53:24.469 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 12928, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 12928, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 5875798, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:24.470 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 5581, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 3880686, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 5581, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 3880686, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.471 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 5581, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 5581, 2024-09-10T07:53:24.472 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 5581, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 5581, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 195, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 21255, 2024-09-10T07:53:24.473 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 195, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 73, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 21255, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55e7f7f68960": { 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 574, 2024-09-10T07:53:24.474 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1664, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 3735903, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1664, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 814, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 3735329, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:24.475 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:24.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:24.476 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T07:53:24.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:24 smithi181 ceph-mon[35397]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-10T07:53:24.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:24 smithi063 ceph-mon[35020]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-10T07:53:24.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:24 smithi046 ceph-mon[30705]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-10T07:53:25.282 INFO:tasks.check_counter:Daemon mds.i mds.exported=5 2024-09-10T07:53:25.282 INFO:tasks.check_counter:Daemon mds.i mds.imported=11 2024-09-10T07:53:25.282 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T07:53:25.283 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T07:53:25.532 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:53:26.097 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T07:53:26.097 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T07:53:26.097 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 4444, 2024-09-10T07:53:26.097 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 4457, 2024-09-10T07:53:26.097 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 2801322, 2024-09-10T07:53:26.097 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 2981249, 2024-09-10T07:53:26.097 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 10, 2024-09-10T07:53:26.097 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.363056578, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.140172988, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.284889481, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.034707895, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4456, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.560808809, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000125854 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5415, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001097773, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000202 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.098 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 772816, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 222032 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1924, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2252, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1400967, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1636382, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 16, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.134117368, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.047643769, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.041805070, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.033173983, 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2252, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.134415106, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000059686 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 582, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000112389, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000193 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1904, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 591056 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3229, 2024-09-10T07:53:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 3803, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1595071, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1979885, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 44, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.243017160, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.086673890, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.099734565, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.042803665, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3800, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.265766776, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000069938 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-10T07:53:26.101 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1342, 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000536654, 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000399 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-10T07:53:26.102 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4283, 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.974553012, 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000227539 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 17, 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000497390, 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000029258 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.103 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-09-10T07:53:26.104 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 254, 2024-09-10T07:53:26.104 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 252, 2024-09-10T07:53:26.104 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-09-10T07:53:26.104 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 252, 2024-09-10T07:53:26.104 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.272175834, 2024-09-10T07:53:26.104 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005048316 2024-09-10T07:53:26.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.104 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-09-10T07:53:26.105 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 2, 2024-09-10T07:53:26.105 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-09-10T07:53:26.105 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-09-10T07:53:26.105 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-09-10T07:53:26.106 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-09-10T07:53:26.106 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-09-10T07:53:26.106 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 347, 2024-09-10T07:53:26.106 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 3, 2024-09-10T07:53:26.106 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 344, 2024-09-10T07:53:26.107 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-09-10T07:53:26.107 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 342, 2024-09-10T07:53:26.107 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-09-10T07:53:26.107 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 324, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 324, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 13, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 399, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 390, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 5, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-09-10T07:53:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-09-10T07:53:26.109 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 198, 2024-09-10T07:53:26.109 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-09-10T07:53:26.109 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-09-10T07:53:26.109 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-09-10T07:53:26.109 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 9, 2024-09-10T07:53:26.109 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 333, 2024-09-10T07:53:26.110 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-09-10T07:53:26.110 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 4279, 2024-09-10T07:53:26.110 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 1, 2024-09-10T07:53:26.110 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 1018, 2024-09-10T07:53:26.110 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 1507456431, 2024-09-10T07:53:26.110 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-09-10T07:53:26.110 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 4279, 2024-09-10T07:53:26.110 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 6, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 6, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 4273, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 17, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 230, 2024-09-10T07:53:26.111 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 696, 2024-09-10T07:53:26.112 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T07:53:26.112 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T07:53:26.112 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T07:53:26.112 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 706, 2024-09-10T07:53:26.112 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 351, 2024-09-10T07:53:26.112 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 21, 2024-09-10T07:53:26.112 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 11 2024-09-10T07:53:26.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.113 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-09-10T07:53:26.113 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-09-10T07:53:26.113 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-09-10T07:53:26.113 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-09-10T07:53:26.113 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 5, 2024-09-10T07:53:26.113 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 10, 2024-09-10T07:53:26.113 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-09-10T07:53:26.113 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 2, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 2, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 15, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 4521, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-09-10T07:53:26.114 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 861, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 861, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 2, 2024-09-10T07:53:26.115 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 2, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 2, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 5378080, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 408, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 15.103937249, 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.037019454 2024-09-10T07:53:26.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 348, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 350, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 2, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 29, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 30, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 1, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 347, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 351, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 4, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 324, 2024-09-10T07:53:26.117 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 438, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 114, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 61388, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 223520 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 268, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 2, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 254, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 18, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 6, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.118 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 35, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.154714645, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.032991847 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-09-10T07:53:26.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 102, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.042244580, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000414162 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.120 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 105, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.063674655, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000606425 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 8, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.006340086, 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000792510 2024-09-10T07:53:26.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.124 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.125 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-09-10T07:53:26.127 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.005201868, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002600934 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.128 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 2, 2024-09-10T07:53:26.129 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 1, 2024-09-10T07:53:26.130 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-09-10T07:53:26.130 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-09-10T07:53:26.130 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 23, 2024-09-10T07:53:26.130 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 23, 2024-09-10T07:53:26.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 985, 2024-09-10T07:53:26.130 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-09-10T07:53:26.130 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T07:53:26.131 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-09-10T07:53:26.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T07:53:26.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T07:53:26.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-09-10T07:53:26.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-09-10T07:53:26.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-09-10T07:53:26.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-09-10T07:53:26.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T07:53:26.132 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T07:53:26.133 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T07:53:26.133 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T07:53:26.133 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T07:53:26.133 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-09-10T07:53:26.133 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-09-10T07:53:26.133 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-09-10T07:53:26.134 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T07:53:26.134 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T07:53:26.134 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-09-10T07:53:26.134 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-09-10T07:53:26.134 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-09-10T07:53:26.134 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-09-10T07:53:26.134 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T07:53:26.135 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-09-10T07:53:26.135 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T07:53:26.135 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-09-10T07:53:26.135 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 316094, 2024-09-10T07:53:26.135 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 429, 2024-09-10T07:53:26.135 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-09-10T07:53:26.136 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-09-10T07:53:26.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-09-10T07:53:26.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-09-10T07:53:26.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-09-10T07:53:26.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-09-10T07:53:26.136 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-09-10T07:53:26.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-09-10T07:53:26.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 8760, 2024-09-10T07:53:26.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 164, 2024-09-10T07:53:26.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T07:53:26.137 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-09-10T07:53:26.137 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-09-10T07:53:26.138 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-09-10T07:53:26.138 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 1227255, 2024-09-10T07:53:26.138 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 12134, 2024-09-10T07:53:26.138 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-09-10T07:53:26.138 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-09-10T07:53:26.138 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-09-10T07:53:26.139 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-09-10T07:53:26.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.139 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-09-10T07:53:26.139 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-09-10T07:53:26.139 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-09-10T07:53:26.139 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 4846, 2024-09-10T07:53:26.139 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 1239459, 2024-09-10T07:53:26.140 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2024-09-10T07:53:26.140 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 4846, 2024-09-10T07:53:26.140 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-09-10T07:53:26.140 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4846, 2024-09-10T07:53:26.140 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 33.328362339, 2024-09-10T07:53:26.140 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006877499 2024-09-10T07:53:26.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.141 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-09-10T07:53:26.141 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-09-10T07:53:26.141 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4846, 2024-09-10T07:53:26.141 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 4934 2024-09-10T07:53:26.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.142 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 4846, 2024-09-10T07:53:26.142 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 4280, 2024-09-10T07:53:26.142 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 566, 2024-09-10T07:53:26.142 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-09-10T07:53:26.142 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-09-10T07:53:26.142 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-09-10T07:53:26.143 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-09-10T07:53:26.143 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 1, 2024-09-10T07:53:26.143 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 409, 2024-09-10T07:53:26.143 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 59, 2024-09-10T07:53:26.143 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-09-10T07:53:26.144 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-09-10T07:53:26.144 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-09-10T07:53:26.144 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-09-10T07:53:26.144 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 16, 2024-09-10T07:53:26.144 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-09-10T07:53:26.144 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-09-10T07:53:26.144 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-09-10T07:53:26.145 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 4279, 2024-09-10T07:53:26.145 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-09-10T07:53:26.145 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-09-10T07:53:26.145 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-09-10T07:53:26.145 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-09-10T07:53:26.145 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-09-10T07:53:26.146 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-09-10T07:53:26.146 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-09-10T07:53:26.146 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T07:53:26.146 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-09-10T07:53:26.146 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-09-10T07:53:26.146 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-09-10T07:53:26.146 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-09-10T07:53:26.147 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-09-10T07:53:26.147 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-09-10T07:53:26.147 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-09-10T07:53:26.147 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-09-10T07:53:26.147 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-09-10T07:53:26.147 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-09-10T07:53:26.148 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-09-10T07:53:26.148 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-09-10T07:53:26.148 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-09-10T07:53:26.148 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-09-10T07:53:26.148 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-09-10T07:53:26.148 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-09-10T07:53:26.149 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-09-10T07:53:26.149 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-09-10T07:53:26.149 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-09-10T07:53:26.149 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 97, 2024-09-10T07:53:26.149 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-09-10T07:53:26.149 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2024-09-10T07:53:26.150 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2024-09-10T07:53:26.150 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2024-09-10T07:53:26.150 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-09-10T07:53:26.150 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-09-10T07:53:26.150 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 121, 2024-09-10T07:53:26.150 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-09-10T07:53:26.151 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 19 2024-09-10T07:53:26.151 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.151 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-09-10T07:53:26.151 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-09-10T07:53:26.151 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-09-10T07:53:26.151 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-09-10T07:53:26.151 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-09-10T07:53:26.152 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.152 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-09-10T07:53:26.152 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 3874 2024-09-10T07:53:26.152 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.152 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-09-10T07:53:26.153 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-09-10T07:53:26.153 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 4, 2024-09-10T07:53:26.153 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-09-10T07:53:26.153 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 2, 2024-09-10T07:53:26.153 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 4, 2024-09-10T07:53:26.153 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 2, 2024-09-10T07:53:26.154 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-09-10T07:53:26.154 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.154 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T07:53:26.154 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:26.154 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-10T07:53:26.154 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:26.155 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 9593, 2024-09-10T07:53:26.155 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 4925000, 2024-09-10T07:53:26.155 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:26.155 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 9593, 2024-09-10T07:53:26.155 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:26.156 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:26.156 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 9593, 2024-09-10T07:53:26.156 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 4925000, 2024-09-10T07:53:26.156 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:26.156 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.156 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.157 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.157 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.157 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.157 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-09-10T07:53:26.157 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:26.157 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-10T07:53:26.158 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:26.158 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:26.158 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:26.158 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:26.158 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:26.158 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 4846, 2024-09-10T07:53:26.159 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 1239677, 2024-09-10T07:53:26.159 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4846, 2024-09-10T07:53:26.159 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1239677, 2024-09-10T07:53:26.159 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:26.159 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.159 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.160 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.160 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.160 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-09-10T07:53:26.160 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:26.160 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-09-10T07:53:26.161 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:26.161 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-10T07:53:26.161 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-10T07:53:26.161 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:26.161 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-10T07:53:26.161 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 4846, 2024-09-10T07:53:26.162 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 4846, 2024-09-10T07:53:26.162 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 4846, 2024-09-10T07:53:26.162 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 4846, 2024-09-10T07:53:26.162 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:26.162 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.162 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.162 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.163 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.163 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-09-10T07:53:26.163 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:26.163 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-10T07:53:26.163 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:26.164 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 2, 2024-09-10T07:53:26.164 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 218, 2024-09-10T07:53:26.164 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:26.164 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 2, 2024-09-10T07:53:26.164 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:26.164 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:26.165 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1, 2024-09-10T07:53:26.165 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 218, 2024-09-10T07:53:26.165 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:26.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.165 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.166 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T07:53:26.166 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55ef4eba4960": { 2024-09-10T07:53:26.166 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-10T07:53:26.166 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-10T07:53:26.166 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-10T07:53:26.167 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 861, 2024-09-10T07:53:26.167 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1183776, 2024-09-10T07:53:26.167 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-10T07:53:26.167 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 861, 2024-09-10T07:53:26.167 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-10T07:53:26.168 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-10T07:53:26.168 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 408, 2024-09-10T07:53:26.168 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1183776, 2024-09-10T07:53:26.168 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-10T07:53:26.168 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-10T07:53:26.168 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-10T07:53:26.169 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-10T07:53:26.169 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.169 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T07:53:26.169 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T07:53:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:26 smithi181 ceph-mon[35397]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T07:53:26.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:26 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:53:26.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:26 smithi063 ceph-mon[35020]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T07:53:26.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:26 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:53:26.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:26 smithi046 ceph-mon[30705]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T07:53:26.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:26 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:53:26.949 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-10T07:53:26.949 INFO:tasks.check_counter:Daemon mds.l mds.imported=9 2024-09-10T07:53:26.949 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T07:53:26.960 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T07:53:26.960 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:53:26.960 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:53:27.007 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T07:53:27.007 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:53:27.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T07:53:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:27 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:53:27.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:53:27.651 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:27 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:53:27.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:53:27.652 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:27 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' 2024-09-10T07:53:28.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.146+0000 7f33c0600640 1 -- 172.21.15.46:0/633720272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 msgr2=0x7f33b8114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.146+0000 7f33c0600640 1 --2- 172.21.15.46:0/633720272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 0x7f33b8114e40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009ca0 tx=0x7f33b002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.146+0000 7f33c0600640 1 -- 172.21.15.46:0/633720272 shutdown_connections 2024-09-10T07:53:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.146+0000 7f33c0600640 1 --2- 172.21.15.46:0/633720272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 0x7f33b8114e40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.146+0000 7f33c0600640 1 --2- 172.21.15.46:0/633720272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b8077040 0x7f33b8112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.146+0000 7f33c0600640 1 --2- 172.21.15.46:0/633720272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b8076720 0x7f33b8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.146+0000 7f33c0600640 1 -- 172.21.15.46:0/633720272 >> 172.21.15.46:0/633720272 conn(0x7f33b80ffa50 msgr2=0x7f33b8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.147+0000 7f33c0600640 1 -- 172.21.15.46:0/633720272 shutdown_connections 2024-09-10T07:53:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.147+0000 7f33c0600640 1 -- 172.21.15.46:0/633720272 wait complete. 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33c0600640 1 Processor -- start 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33c0600640 1 -- start start 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33c0600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b8076720 0x7f33b81a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33c0600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b8077040 0x7f33b81a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33c0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 0x7f33b819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33c0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81185e0 con 0x7f33b8112a50 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33c0600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8118460 con 0x7f33b8076720 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33c0600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8118760 con 0x7f33b8077040 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33be600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 0x7f33b819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33be600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 0x7f33b819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38322/0 (socket says 172.21.15.46:38322) 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33be600640 1 -- 172.21.15.46:0/36178315 learned_addr learned my addr 172.21.15.46:0/36178315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.148+0000 7f33bdc00640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b8076720 0x7f33b81a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.149+0000 7f33bd200640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b8077040 0x7f33b81a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.149+0000 7f33be600640 1 -- 172.21.15.46:0/36178315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b8077040 msgr2=0x7f33b81a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.149+0000 7f33be600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b8077040 0x7f33b81a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.149+0000 7f33be600640 1 -- 172.21.15.46:0/36178315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b8076720 msgr2=0x7f33b81a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.149+0000 7f33be600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b8076720 0x7f33b81a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.149+0000 7f33be600640 1 -- 172.21.15.46:0/36178315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b819f020 con 0x7f33b8112a50 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.149+0000 7f33bdc00640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b8076720 0x7f33b81a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.149+0000 7f33be600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 0x7f33b819e8f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005e30 tx=0x7f33b0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.150+0000 7f33b6000640 1 -- 172.21.15.46:0/36178315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0031650 con 0x7f33b8112a50 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.150+0000 7f33b6000640 1 -- 172.21.15.46:0/36178315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b00317f0 con 0x7f33b8112a50 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.150+0000 7f33b6000640 1 -- 172.21.15.46:0/36178315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b002fb70 con 0x7f33b8112a50 2024-09-10T07:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.150+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b819f2b0 con 0x7f33b8112a50 2024-09-10T07:53:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.150+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b81ab600 con 0x7f33b8112a50 2024-09-10T07:53:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.151+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b8076b00 con 0x7f33b8112a50 2024-09-10T07:53:28.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.153+0000 7f33b6000640 1 -- 172.21.15.46:0/36178315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f33b002fd10 con 0x7f33b8112a50 2024-09-10T07:53:28.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.154+0000 7f33b6000640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33a00a2c00 0x7f33a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.154+0000 7f33b6000640 1 -- 172.21.15.46:0/36178315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f33b00f0730 con 0x7f33b8112a50 2024-09-10T07:53:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.155+0000 7f33bdc00640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33a00a2c00 0x7f33a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.155+0000 7f33bdc00640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33a00a2c00 0x7f33a00a50c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f33b819fa30 tx=0x7f33ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.159+0000 7f33b6000640 1 -- 172.21.15.46:0/36178315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33b00319a0 con 0x7f33b8112a50 2024-09-10T07:53:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.359+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f33b802d7c0 con 0x7f33b8112a50 2024-09-10T07:53:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.360+0000 7f33b6000640 1 -- 172.21.15.46:0/36178315 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+590 (secure 0 0 0) 0x7f33b00bd2b0 con 0x7f33b8112a50 2024-09-10T07:53:28.360 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2972952762 2024-09-11T07:26:12.367178+0000 2024-09-10T07:53:28.360 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2972952762 2024-09-11T07:26:12.367178+0000 2024-09-10T07:53:28.360 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1860671836 2024-09-11T07:26:12.367178+0000 2024-09-10T07:53:28.360 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3724700252 2024-09-11T07:26:12.367178+0000 2024-09-10T07:53:28.361 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3653813867 2024-09-11T07:26:12.367178+0000 2024-09-10T07:53:28.361 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/141501936 2024-09-11T07:25:44.780814+0000 2024-09-10T07:53:28.361 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3003000819 2024-09-11T07:25:44.780814+0000 2024-09-10T07:53:28.361 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/141501936 2024-09-11T07:25:44.780814+0000 2024-09-10T07:53:28.361 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1462945451 2024-09-11T07:25:44.780814+0000 2024-09-10T07:53:28.361 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3598352774 2024-09-11T07:25:44.780814+0000 2024-09-10T07:53:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-09-10T07:53:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33a00a2c00 msgr2=0x7f33a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33a00a2c00 0x7f33a00a50c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f33b819fa30 tx=0x7f33ac00b390 comp rx=0 tx=0).stop 2024-09-10T07:53:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 msgr2=0x7f33b819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 0x7f33b819e8f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005e30 tx=0x7f33b0003f10 comp rx=0 tx=0).stop 2024-09-10T07:53:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 shutdown_connections 2024-09-10T07:53:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f33a00a2c00 0x7f33a00a50c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b8112a50 0x7f33b819e8f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b8077040 0x7f33b81a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 --2- 172.21.15.46:0/36178315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b8076720 0x7f33b81a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 >> 172.21.15.46:0/36178315 conn(0x7f33b80ffa50 msgr2=0x7f33b8113650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.362+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 shutdown_connections 2024-09-10T07:53:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:28.363+0000 7f33c0600640 1 -- 172.21.15.46:0/36178315 wait complete. 2024-09-10T07:53:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:28 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi063 to 176.0M 2024-09-10T07:53:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:28 smithi181 ceph-mon[35397]: Unable to set osd_memory_target on smithi063 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T07:53:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:28 smithi181 ceph-mon[35397]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T07:53:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:28 smithi181 ceph-mon[35397]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-10T07:53:28.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:28 smithi181 ceph-mon[35397]: Unable to set osd_memory_target on smithi181 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-10T07:53:28.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:28 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi063 to 176.0M 2024-09-10T07:53:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:28 smithi063 ceph-mon[35020]: Unable to set osd_memory_target on smithi063 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T07:53:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:28 smithi063 ceph-mon[35020]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T07:53:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:28 smithi063 ceph-mon[35020]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-10T07:53:28.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:28 smithi063 ceph-mon[35020]: Unable to set osd_memory_target on smithi181 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-10T07:53:28.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:28 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi063 to 176.0M 2024-09-10T07:53:28.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:28 smithi046 ceph-mon[30705]: Unable to set osd_memory_target on smithi063 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T07:53:28.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:28 smithi046 ceph-mon[30705]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T07:53:28.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:28 smithi046 ceph-mon[30705]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-10T07:53:28.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:28 smithi046 ceph-mon[30705]: Unable to set osd_memory_target on smithi181 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-10T07:53:29.317 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T07:53:29.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:29.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /tmp/cephfs 2024-09-10T07:53:29.453 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T07:53:29.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:29.453 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-10T07:53:29.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:29.542 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-10T07:53:29.623 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-10T07:53:29.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:29.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-10T07:53:29.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:29 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/36178315' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:53:29.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:29 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/36178315' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:53:29.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:29 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/36178315' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T07:53:29.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:29.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-10T07:53:30.052 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T07:53:30.063 INFO:tasks.cephadm:Teardown begin 2024-09-10T07:53:30.063 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:53:30.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:53:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:30.647 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:30 smithi046 ceph-mon[30705]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-10T07:53:30.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:30 smithi181 ceph-mon[35397]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-10T07:53:30.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:30 smithi063 ceph-mon[35020]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-10T07:53:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 -- 172.21.15.46:0/2002370053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504101de0 msgr2=0x7fc5041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 --2- 172.21.15.46:0/2002370053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504101de0 0x7fc5041021c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009900 tx=0x7fc4f402a3f0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 -- 172.21.15.46:0/2002370053 shutdown_connections 2024-09-10T07:53:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 --2- 172.21.15.46:0/2002370053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc504106f10 0x7fc50410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 --2- 172.21.15.46:0/2002370053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc504102790 0x7fc5041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 --2- 172.21.15.46:0/2002370053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504101de0 0x7fc5041021c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 -- 172.21.15.46:0/2002370053 >> 172.21.15.46:0/2002370053 conn(0x7fc5040fd8c0 msgr2=0x7fc5040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 -- 172.21.15.46:0/2002370053 shutdown_connections 2024-09-10T07:53:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.083+0000 7fc50a200640 1 -- 172.21.15.46:0/2002370053 wait complete. 2024-09-10T07:53:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.084+0000 7fc50a200640 1 Processor -- start 2024-09-10T07:53:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.084+0000 7fc50a200640 1 -- start start 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc50a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc504101de0 0x7fc50419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc50a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc504102790 0x7fc50419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc50a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106f10 0x7fc50419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc50a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc504111f70 con 0x7fc504106f10 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc50a200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc504111df0 con 0x7fc504101de0 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc50a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc5041120f0 con 0x7fc504102790 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc503e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106f10 0x7fc50419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc4fbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc504102790 0x7fc50419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc503e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106f10 0x7fc50419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38334/0 (socket says 172.21.15.46:38334) 2024-09-10T07:53:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc503e00640 1 -- 172.21.15.46:0/8891911 learned_addr learned my addr 172.21.15.46:0/8891911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.085+0000 7fc503400640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc504101de0 0x7fc50419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc4fbe00640 1 -- 172.21.15.46:0/8891911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc504101de0 msgr2=0x7fc50419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc4fbe00640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc504101de0 0x7fc50419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc4fbe00640 1 -- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106f10 msgr2=0x7fc50419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc4fbe00640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106f10 0x7fc50419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc4fbe00640 1 -- 172.21.15.46:0/8891911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50419f5e0 con 0x7fc504102790 2024-09-10T07:53:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc4fbe00640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc504102790 0x7fc50419ab70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00e9f0 tx=0x7fc4ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc500c00640 1 -- 172.21.15.46:0/8891911 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00ccc0 con 0x7fc504102790 2024-09-10T07:53:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50419f8d0 con 0x7fc504102790 2024-09-10T07:53:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc500c00640 1 -- 172.21.15.46:0/8891911 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ec005020 con 0x7fc504102790 2024-09-10T07:53:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc500c00640 1 -- 172.21.15.46:0/8891911 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0049e0 con 0x7fc504102790 2024-09-10T07:53:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.086+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5041a71b0 con 0x7fc504102790 2024-09-10T07:53:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.087+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5041021c0 con 0x7fc504102790 2024-09-10T07:53:31.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.089+0000 7fc500c00640 1 -- 172.21.15.46:0/8891911 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc4ec005560 con 0x7fc504102790 2024-09-10T07:53:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.091+0000 7fc500c00640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4e80a2b80 0x7fc4e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.091+0000 7fc500c00640 1 -- 172.21.15.46:0/8891911 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fc4ec0cf370 con 0x7fc504102790 2024-09-10T07:53:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.091+0000 7fc503400640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4e80a2b80 0x7fc4e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:31.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.092+0000 7fc503400640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4e80a2b80 0x7fc4e80a5040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009230 tx=0x7fc4f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.093+0000 7fc500c00640 1 -- 172.21.15.46:0/8891911 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4ec0051d0 con 0x7fc504102790 2024-09-10T07:53:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.313+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc50419b990 con 0x7fc504102790 2024-09-10T07:53:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.314+0000 7fc500c00640 1 -- 172.21.15.46:0/8891911 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11980 (secure 0 0 0) 0x7fc50419b990 con 0x7fc504102790 2024-09-10T07:53:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:53:31.314 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":61,"btime":"2024-09-10T07:52:48:263830+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:52:47.520095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24448,24499,24517,24530,24535]},"id":1}]} 2024-09-10T07:53:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 61 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4e80a2b80 msgr2=0x7fc4e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4e80a2b80 0x7fc4e80a5040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009230 tx=0x7fc4f40058d0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc504102790 msgr2=0x7fc50419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc504102790 0x7fc50419ab70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00e9f0 tx=0x7fc4ec00eeb0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 shutdown_connections 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fc4e80a2b80 0x7fc4e80a5040 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106f10 0x7fc50419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc504102790 0x7fc50419ab70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 --2- 172.21.15.46:0/8891911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc504101de0 0x7fc50419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 >> 172.21.15.46:0/8891911 conn(0x7fc5040fd8c0 msgr2=0x7fc5040fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 shutdown_connections 2024-09-10T07:53:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:31.316+0000 7fc50a200640 1 -- 172.21.15.46:0/8891911 wait complete. 2024-09-10T07:53:31.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:31 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/8891911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:53:31.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:31 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/8891911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:53:31.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:31 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/8891911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:53:32.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T07:53:32.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:32 smithi181 ceph-mon[35397]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-10T07:53:32.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:32 smithi063 ceph-mon[35020]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-10T07:53:32.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:32 smithi046 ceph-mon[30705]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-10T07:53:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 -- 172.21.15.46:0/3122084033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 msgr2=0x7fe884108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 --2- 172.21.15.46:0/3122084033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 0x7fe884108830 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009860 tx=0x7fe86c02a340 comp rx=0 tx=0).stop 2024-09-10T07:53:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 -- 172.21.15.46:0/3122084033 shutdown_connections 2024-09-10T07:53:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 --2- 172.21.15.46:0/3122084033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe884108f60 0x7fe884110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 --2- 172.21.15.46:0/3122084033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 0x7fe884108830 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 --2- 172.21.15.46:0/3122084033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe884103e30 0x7fe884104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 -- 172.21.15.46:0/3122084033 >> 172.21.15.46:0/3122084033 conn(0x7fe8840ff8d0 msgr2=0x7fe884101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 -- 172.21.15.46:0/3122084033 shutdown_connections 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.515+0000 7fe883e00640 1 -- 172.21.15.46:0/3122084033 wait complete. 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.516+0000 7fe883e00640 1 Processor -- start 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.516+0000 7fe883e00640 1 -- start start 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.516+0000 7fe883e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe884103e30 0x7fe88419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe883e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 0x7fe88419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe883e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe884108f60 0x7fe88419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe883e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe884113f80 con 0x7fe884108f60 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe883e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe884113e00 con 0x7fe884103e30 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe883e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe884114100 con 0x7fe8841047e0 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe882a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe884103e30 0x7fe88419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe883400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe884108f60 0x7fe88419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe87a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 0x7fe88419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe882a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe884103e30 0x7fe88419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.46:44394/0 (socket says 172.21.15.46:44394) 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe882a00640 1 -- 172.21.15.46:0/1256811802 learned_addr learned my addr 172.21.15.46:0/1256811802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe87a000640 1 -- 172.21.15.46:0/1256811802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe884103e30 msgr2=0x7fe88419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe87a000640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe884103e30 0x7fe88419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe87a000640 1 -- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe884108f60 msgr2=0x7fe88419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe87a000640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe884108f60 0x7fe88419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe87a000640 1 -- 172.21.15.46:0/1256811802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88419f410 con 0x7fe8841047e0 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.517+0000 7fe882a00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe884103e30 0x7fe88419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:33.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.518+0000 7fe87a000640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 0x7fe88419a9a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe86c02a860 tx=0x7fe86c033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.518+0000 7fe87be00640 1 -- 172.21.15.46:0/1256811802 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c037040 con 0x7fe8841047e0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.518+0000 7fe87be00640 1 -- 172.21.15.46:0/1256811802 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe86c033d50 con 0x7fe8841047e0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.518+0000 7fe87be00640 1 -- 172.21.15.46:0/1256811802 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c003960 con 0x7fe8841047e0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.518+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88419f6a0 con 0x7fe8841047e0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.518+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88419fb30 con 0x7fe8841047e0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.519+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe884104210 con 0x7fe8841047e0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.521+0000 7fe87be00640 1 -- 172.21.15.46:0/1256811802 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe86c042020 con 0x7fe8841047e0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.522+0000 7fe87be00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8680a2b80 0x7fe8680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.522+0000 7fe87be00640 1 -- 172.21.15.46:0/1256811802 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7fe86c0ee960 con 0x7fe8841047e0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.523+0000 7fe882a00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8680a2b80 0x7fe8680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.523+0000 7fe882a00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8680a2b80 0x7fe8680a5040 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe8740096c0 tx=0x7fe874009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:33.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.527+0000 7fe87be00640 1 -- 172.21.15.46:0/1256811802 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe86c049020 con 0x7fe8841047e0 2024-09-10T07:53:33.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.748+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe88402d7c0 con 0x7fe8841047e0 2024-09-10T07:53:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.749+0000 7fe87be00640 1 -- 172.21.15.46:0/1256811802 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11979 (secure 0 0 0) 0x7fe86c0bb4e0 con 0x7fe8841047e0 2024-09-10T07:53:33.749 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:53:33.749 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":62,"btime":"2024-09-10T07:53:33:367662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2449149651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2449149651},{"type":"v1","addr":"172.21.15.46:6837","nonce":2449149651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1424921396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1424921396},{"type":"v1","addr":"172.21.15.46:6839","nonce":1424921396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T07:33:36.040805+0000","modified":"2024-09-10T07:53:32.520966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24535,"mds_1":24530,"mds_2":24517,"mds_3":24499,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6833/620318721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":620318721},{"type":"v1","addr":"172.21.15.63:6833","nonce":620318721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.181:6833/2451989162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2451989162},{"type":"v1","addr":"172.21.15.181:6833","nonce":2451989162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6835/4121221095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4121221095},{"type":"v1","addr":"172.21.15.46:6835","nonce":4121221095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6835/1645722263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":1645722263},{"type":"v1","addr":"172.21.15.63:6835","nonce":1645722263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/335913193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":335913193},{"type":"v1","addr":"172.21.15.181:6835","nonce":335913193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6837/4195549127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":4195549127},{"type":"v1","addr":"172.21.15.63:6837","nonce":4195549127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.181:6837/975679519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":975679519},{"type":"v1","addr":"172.21.15.181:6837","nonce":975679519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.181:6839/1085095014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1085095014},{"type":"v1","addr":"172.21.15.181:6839","nonce":1085095014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.63:6839/1361023409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":1361023409},{"type":"v1","addr":"172.21.15.63:6839","nonce":1361023409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6841/3146821117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3146821117},{"type":"v1","addr":"172.21.15.46:6841","nonce":3146821117}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24535,"qdb_cluster":[24535,24530,24517,24499,24448]},"id":1}]} 2024-09-10T07:53:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 62 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8680a2b80 msgr2=0x7fe8680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8680a2b80 0x7fe8680a5040 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe8740096c0 tx=0x7fe874009260 comp rx=0 tx=0).stop 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 msgr2=0x7fe88419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 0x7fe88419a9a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe86c02a860 tx=0x7fe86c033b60 comp rx=0 tx=0).stop 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 shutdown_connections 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fe8680a2b80 0x7fe8680a5040 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe884108f60 0x7fe88419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8841047e0 0x7fe88419a9a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 --2- 172.21.15.46:0/1256811802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe884103e30 0x7fe88419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.751+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 >> 172.21.15.46:0/1256811802 conn(0x7fe8840ff8d0 msgr2=0x7fe884100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.752+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 shutdown_connections 2024-09-10T07:53:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:33.752+0000 7fe883e00640 1 -- 172.21.15.46:0/1256811802 wait complete. 2024-09-10T07:53:34.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:34 smithi046 ceph-mon[30705]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-10T07:53:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:34 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:53:34.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:34 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/1256811802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:53:34.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T07:53:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:34 smithi181 ceph-mon[35397]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-10T07:53:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:34 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:53:34.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:34 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/1256811802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:53:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:34 smithi063 ceph-mon[35020]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-10T07:53:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:34 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:53:34.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:34 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/1256811802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T07:53:34.942 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:35.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 -- 172.21.15.46:0/1106849450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb01034b0 msgr2=0x7f4eb010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 --2- 172.21.15.46:0/1106849450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb01034b0 0x7f4eb010b9a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009900 tx=0x7f4ea002a480 comp rx=0 tx=0).stop 2024-09-10T07:53:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 -- 172.21.15.46:0/1106849450 shutdown_connections 2024-09-10T07:53:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 --2- 172.21.15.46:0/1106849450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4eb010bfc0 0x7f4eb010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 --2- 172.21.15.46:0/1106849450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb01034b0 0x7f4eb010b9a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 --2- 172.21.15.46:0/1106849450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb0102b90 0x7f4eb0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 -- 172.21.15.46:0/1106849450 >> 172.21.15.46:0/1106849450 conn(0x7f4eb00fc610 msgr2=0x7f4eb00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 -- 172.21.15.46:0/1106849450 shutdown_connections 2024-09-10T07:53:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.516+0000 7f4eb7200640 1 -- 172.21.15.46:0/1106849450 wait complete. 2024-09-10T07:53:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.517+0000 7f4eb7200640 1 Processor -- start 2024-09-10T07:53:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.517+0000 7f4eb7200640 1 -- start start 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.517+0000 7f4eb7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb0102b90 0x7f4eb019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.517+0000 7f4eb7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4eb01034b0 0x7f4eb019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.517+0000 7f4eb7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb010bfc0 0x7f4eb019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.517+0000 7f4eb7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0111dc0 con 0x7f4eb010bfc0 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.517+0000 7f4eb7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0111c40 con 0x7f4eb01034b0 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.517+0000 7f4eb7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0111f40 con 0x7f4eb0102b90 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb010bfc0 0x7f4eb019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4eb01034b0 0x7f4eb019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb010bfc0 0x7f4eb019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38356/0 (socket says 172.21.15.46:38356) 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb6800640 1 -- 172.21.15.46:0/46878181 learned_addr learned my addr 172.21.15.46:0/46878181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb5e00640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb0102b90 0x7f4eb019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb5400640 1 -- 172.21.15.46:0/46878181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb0102b90 msgr2=0x7f4eb019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb5400640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb0102b90 0x7f4eb019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb5400640 1 -- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb010bfc0 msgr2=0x7f4eb019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb5400640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb010bfc0 0x7f4eb019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.518+0000 7f4eb5400640 1 -- 172.21.15.46:0/46878181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb019f450 con 0x7f4eb01034b0 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.519+0000 7f4eb5400640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4eb01034b0 0x7f4eb019a9e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0005940 tx=0x7f4ea00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.519+0000 7f4eaaa00640 1 -- 172.21.15.46:0/46878181 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea003f030 con 0x7f4eb01034b0 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.519+0000 7f4eaaa00640 1 -- 172.21.15.46:0/46878181 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea00044c0 con 0x7f4eb01034b0 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.519+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb019f6e0 con 0x7f4eb01034b0 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.519+0000 7f4eaaa00640 1 -- 172.21.15.46:0/46878181 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0035070 con 0x7f4eb01034b0 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.519+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb01a7020 con 0x7f4eb01034b0 2024-09-10T07:53:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.520+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb006a850 con 0x7f4eb01034b0 2024-09-10T07:53:35.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.522+0000 7f4eaaa00640 1 -- 172.21.15.46:0/46878181 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ea0042030 con 0x7f4eb01034b0 2024-09-10T07:53:35.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.523+0000 7f4eaaa00640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e980a2b80 0x7f4e980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:35.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.524+0000 7f4eaaa00640 1 -- 172.21.15.46:0/46878181 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f4ea00ef9a0 con 0x7f4eb01034b0 2024-09-10T07:53:35.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.524+0000 7f4eb5e00640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e980a2b80 0x7f4e980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:35.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.524+0000 7f4eb5e00640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e980a2b80 0x7f4e980a5040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005d00 tx=0x7f4ea400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:35.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.526+0000 7f4eaaa00640 1 -- 172.21.15.46:0/46878181 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ea00bbed0 con 0x7f4eb01034b0 2024-09-10T07:53:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.711+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4eb01a7220 con 0x7f4eb01034b0 2024-09-10T07:53:35.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.714+0000 7f4eaaa00640 1 -- 172.21.15.46:0/46878181 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24700 (secure 0 0 0) 0x7f4ea00c01d0 con 0x7f4eb01034b0 2024-09-10T07:53:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T07:53:35.714 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"fsid":"b120bdf2-6f45-11ef-bcea-c7b262605968","created":"2024-09-10T07:25:19.596849+0000","modified":"2024-09-10T07:35:14.826944+0000","last_up_change":"2024-09-10T07:32:38.726149+0000","last_in_change":"2024-09-10T07:32:15.806622+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:28:36.994371+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:33:30.775288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:33:33.558621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T07:33:43.433746+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0358dc9f-4491-4ec2-9e41-9b621d65a460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6803","nonce":12452162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6805","nonce":12452162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6809","nonce":12452162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":12452162},{"type":"v1","addr":"172.21.15.46:6807","nonce":12452162}]},"public_addr":"172.21.15.46:6803/12452162","cluster_addr":"172.21.15.46:6805/12452162","heartbeat_back_addr":"172.21.15.46:6809/12452162","heartbeat_front_addr":"172.21.15.46:6807/12452162","state":["exists","up"]},{"osd":1,"uuid":"e0f0a426-666d-41c0-a0e2-73abbd2f08ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6801","nonce":1918625844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6803","nonce":1918625844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6807","nonce":1918625844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1918625844},{"type":"v1","addr":"172.21.15.63:6805","nonce":1918625844}]},"public_addr":"172.21.15.63:6801/1918625844","cluster_addr":"172.21.15.63:6803/1918625844","heartbeat_back_addr":"172.21.15.63:6807/1918625844","heartbeat_front_addr":"172.21.15.63:6805/1918625844","state":["exists","up"]},{"osd":2,"uuid":"16642ece-80e2-4852-a809-8a4f57c5c16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6801","nonce":413292109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6803","nonce":413292109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6807","nonce":413292109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":413292109},{"type":"v1","addr":"172.21.15.181:6805","nonce":413292109}]},"public_addr":"172.21.15.181:6801/413292109","cluster_addr":"172.21.15.181:6803/413292109","heartbeat_back_addr":"172.21.15.181:6807/413292109","heartbeat_front_addr":"172.21.15.181:6805/413292109","state":["exists","up"]},{"osd":3,"uuid":"d8019d3f-952b-42df-9c70-c0574211eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6811","nonce":3016450690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6813","nonce":3016450690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6817","nonce":3016450690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3016450690},{"type":"v1","addr":"172.21.15.46:6815","nonce":3016450690}]},"public_addr":"172.21.15.46:6811/3016450690","cluster_addr":"172.21.15.46:6813/3016450690","heartbeat_back_addr":"172.21.15.46:6817/3016450690","heartbeat_front_addr":"172.21.15.46:6815/3016450690","state":["exists","up"]},{"osd":4,"uuid":"fe349fc9-2de4-439f-a982-e9282f26d488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6809","nonce":402938834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6811","nonce":402938834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6815","nonce":402938834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":402938834},{"type":"v1","addr":"172.21.15.63:6813","nonce":402938834}]},"public_addr":"172.21.15.63:6809/402938834","cluster_addr":"172.21.15.63:6811/402938834","heartbeat_back_addr":"172.21.15.63:6815/402938834","heartbeat_front_addr":"172.21.15.63:6813/402938834","state":["exists","up"]},{"osd":5,"uuid":"619c621c-ab3a-40fc-bdd4-498347c3a5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6809","nonce":120236836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6811","nonce":120236836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6815","nonce":120236836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120236836},{"type":"v1","addr":"172.21.15.181:6813","nonce":120236836}]},"public_addr":"172.21.15.181:6809/120236836","cluster_addr":"172.21.15.181:6811/120236836","heartbeat_back_addr":"172.21.15.181:6815/120236836","heartbeat_front_addr":"172.21.15.181:6813/120236836","state":["exists","up"]},{"osd":6,"uuid":"9bd4a464-96a7-4436-bc64-809acbdf2041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6819","nonce":4145991775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6821","nonce":4145991775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6825","nonce":4145991775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4145991775},{"type":"v1","addr":"172.21.15.46:6823","nonce":4145991775}]},"public_addr":"172.21.15.46:6819/4145991775","cluster_addr":"172.21.15.46:6821/4145991775","heartbeat_back_addr":"172.21.15.46:6825/4145991775","heartbeat_front_addr":"172.21.15.46:6823/4145991775","state":["exists","up"]},{"osd":7,"uuid":"32078aa6-503d-49ed-b587-2a840b513145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6817","nonce":1986919229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6819","nonce":1986919229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6823","nonce":1986919229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1986919229},{"type":"v1","addr":"172.21.15.63:6821","nonce":1986919229}]},"public_addr":"172.21.15.63:6817/1986919229","cluster_addr":"172.21.15.63:6819/1986919229","heartbeat_back_addr":"172.21.15.63:6823/1986919229","heartbeat_front_addr":"172.21.15.63:6821/1986919229","state":["exists","up"]},{"osd":8,"uuid":"36b80e03-cf0c-41b5-bad6-2021b07c92ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6817","nonce":296261223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6819","nonce":296261223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6823","nonce":296261223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":296261223},{"type":"v1","addr":"172.21.15.181:6821","nonce":296261223}]},"public_addr":"172.21.15.181:6817/296261223","cluster_addr":"172.21.15.181:6819/296261223","heartbeat_back_addr":"172.21.15.181:6823/296261223","heartbeat_front_addr":"172.21.15.181:6821/296261223","state":["exists","up"]},{"osd":9,"uuid":"5f9edf7d-f94e-4913-9928-5b6c4d3c724e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6827","nonce":805007364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6829","nonce":805007364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6833","nonce":805007364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":805007364},{"type":"v1","addr":"172.21.15.46:6831","nonce":805007364}]},"public_addr":"172.21.15.46:6827/805007364","cluster_addr":"172.21.15.46:6829/805007364","heartbeat_back_addr":"172.21.15.46:6833/805007364","heartbeat_front_addr":"172.21.15.46:6831/805007364","state":["exists","up"]},{"osd":10,"uuid":"fc6d6ef6-c067-495e-a673-88cbab957e7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6825","nonce":2494224828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6827","nonce":2494224828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6831","nonce":2494224828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2494224828},{"type":"v1","addr":"172.21.15.63:6829","nonce":2494224828}]},"public_addr":"172.21.15.63:6825/2494224828","cluster_addr":"172.21.15.63:6827/2494224828","heartbeat_back_addr":"172.21.15.63:6831/2494224828","heartbeat_front_addr":"172.21.15.63:6829/2494224828","state":["exists","up"]},{"osd":11,"uuid":"7a4826cd-7a7a-4a20-9af7-734e74374081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6825","nonce":1815305110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6827","nonce":1815305110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6831","nonce":1815305110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1815305110},{"type":"v1","addr":"172.21.15.181:6829","nonce":1815305110}]},"public_addr":"172.21.15.181:6825/1815305110","cluster_addr":"172.21.15.181:6827/1815305110","heartbeat_back_addr":"172.21.15.181:6831/1815305110","heartbeat_front_addr":"172.21.15.181:6829/1815305110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:27:42.738905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:07.547074+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:28:32.124105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:02.217795+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:28.006577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:29:53.051677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:23.807145+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:30:50.052448+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:14.901127+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:31:45.987914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:11.882034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:32:37.199945+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6800/2972952762":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/1860671836":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3724700252":"2024-09-11T07:26:12.367178+0000","172.21.15.46:0/3653813867":"2024-09-11T07:26:12.367178+0000","172.21.15.46:6801/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3003000819":"2024-09-11T07:25:44.780814+0000","172.21.15.46:6800/141501936":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/1462945451":"2024-09-11T07:25:44.780814+0000","172.21.15.46:0/3598352774":"2024-09-11T07:25:44.780814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e980a2b80 msgr2=0x7f4e980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e980a2b80 0x7f4e980a5040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005d00 tx=0x7f4ea400b410 comp rx=0 tx=0).stop 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4eb01034b0 msgr2=0x7f4eb019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4eb01034b0 0x7f4eb019a9e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0005940 tx=0x7f4ea00040c0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 shutdown_connections 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f4e980a2b80 0x7f4e980a5040 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eb010bfc0 0x7f4eb019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4eb01034b0 0x7f4eb019a9e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 --2- 172.21.15.46:0/46878181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb0102b90 0x7f4eb019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 >> 172.21.15.46:0/46878181 conn(0x7f4eb00fc610 msgr2=0x7f4eb00fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.716+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 shutdown_connections 2024-09-10T07:53:35.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:35.717+0000 7f4eb7200640 1 -- 172.21.15.46:0/46878181 wait complete. 2024-09-10T07:53:36.648 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:36 smithi046 ceph-mon[30705]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-10T07:53:36.648 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:36 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/46878181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:53:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:36 smithi181 ceph-mon[35397]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-10T07:53:36.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:36 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/46878181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:53:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:36 smithi063 ceph-mon[35020]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-10T07:53:36.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:36 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/46878181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T07:53:36.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T07:53:37.028 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:37.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:37 smithi046 ceph-mon[30705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:53:37.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-mon[35397]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:53:37.649 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-mon[35020]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T07:53:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 -- 172.21.15.46:0/2646938411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8530106f10 msgr2=0x7f853010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 --2- 172.21.15.46:0/2646938411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8530106f10 0x7f853010ea90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8528009be0 tx=0x7f852802a6d0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 -- 172.21.15.46:0/2646938411 shutdown_connections 2024-09-10T07:53:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 --2- 172.21.15.46:0/2646938411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8530106f10 0x7f853010ea90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 --2- 172.21.15.46:0/2646938411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530102790 0x7f85301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 --2- 172.21.15.46:0/2646938411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8530101de0 0x7f85301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 -- 172.21.15.46:0/2646938411 >> 172.21.15.46:0/2646938411 conn(0x7f85300fd8c0 msgr2=0x7f85300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 -- 172.21.15.46:0/2646938411 shutdown_connections 2024-09-10T07:53:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.683+0000 7f8537e00640 1 -- 172.21.15.46:0/2646938411 wait complete. 2024-09-10T07:53:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.684+0000 7f8537e00640 1 Processor -- start 2024-09-10T07:53:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.684+0000 7f8537e00640 1 -- start start 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.684+0000 7f8537e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101de0 0x7f8530198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.684+0000 7f8537e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8530102790 0x7f8530198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8537e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8530106f10 0x7f853019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8537e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8530111f70 con 0x7f8530102790 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8537e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8530111df0 con 0x7f8530106f10 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8537e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85301120f0 con 0x7f8530101de0 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8534a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8530102790 0x7f8530198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8534a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8530102790 0x7f8530198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52602/0 (socket says 172.21.15.46:52602) 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8534a00640 1 -- 172.21.15.46:0/3355240781 learned_addr learned my addr 172.21.15.46:0/3355240781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8535e00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8530106f10 0x7f853019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8535400640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101de0 0x7f8530198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8534a00640 1 -- 172.21.15.46:0/3355240781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101de0 msgr2=0x7f8530198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8534a00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101de0 0x7f8530198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8534a00640 1 -- 172.21.15.46:0/3355240781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8530106f10 msgr2=0x7f853019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8534a00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8530106f10 0x7f853019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.685+0000 7f8534a00640 1 -- 172.21.15.46:0/3355240781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853019d3c0 con 0x7f8530102790 2024-09-10T07:53:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.686+0000 7f8534a00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8530102790 0x7f8530198950 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d950 tx=0x7f851c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.686+0000 7f852e000640 1 -- 172.21.15.46:0/3355240781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c017070 con 0x7f8530102790 2024-09-10T07:53:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.686+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853019d6b0 con 0x7f8530102790 2024-09-10T07:53:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.686+0000 7f852e000640 1 -- 172.21.15.46:0/3355240781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f851c009c30 con 0x7f8530102790 2024-09-10T07:53:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.686+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8530074420 con 0x7f8530102790 2024-09-10T07:53:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.686+0000 7f852e000640 1 -- 172.21.15.46:0/3355240781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c020060 con 0x7f8530102790 2024-09-10T07:53:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.687+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85301021c0 con 0x7f8530102790 2024-09-10T07:53:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.690+0000 7f852e000640 1 -- 172.21.15.46:0/3355240781 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f851c0109d0 con 0x7f8530102790 2024-09-10T07:53:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.691+0000 7f852e000640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f85180a2b80 0x7f85180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.691+0000 7f852e000640 1 -- 172.21.15.46:0/3355240781 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9121+0+0 (secure 0 0 0) 0x7f851c0cefb0 con 0x7f8530102790 2024-09-10T07:53:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.691+0000 7f8535400640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f85180a2b80 0x7f85180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:37.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.692+0000 7f8535400640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f85180a2b80 0x7f85180a5040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8524009530 tx=0x7f8524009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.693+0000 7f852e000640 1 -- 172.21.15.46:0/3355240781 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f851c009de0 con 0x7f8530102790 2024-09-10T07:53:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.917+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f853010a0f0 con 0x7f8530102790 2024-09-10T07:53:37.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.919+0000 7f852e000640 1 -- 172.21.15.46:0/3355240781 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) ==== 161+0+0 (secure 0 0 0) 0x7f851c09b4e0 con 0x7f8530102790 2024-09-10T07:53:37.919 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T07:53:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f85180a2b80 msgr2=0x7f85180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f85180a2b80 0x7f85180a5040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8524009530 tx=0x7f8524009260 comp rx=0 tx=0).stop 2024-09-10T07:53:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8530102790 msgr2=0x7f8530198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8530102790 0x7f8530198950 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d950 tx=0x7f851c00de10 comp rx=0 tx=0).stop 2024-09-10T07:53:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 shutdown_connections 2024-09-10T07:53:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8530106f10 0x7f853019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f85180a2b80 0x7f85180a5040 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8530102790 0x7f8530198950 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 --2- 172.21.15.46:0/3355240781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101de0 0x7f8530198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:37.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.924+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 >> 172.21.15.46:0/3355240781 conn(0x7f85300fd8c0 msgr2=0x7f85300fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:37.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.925+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 shutdown_connections 2024-09-10T07:53:37.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:37.925+0000 7f8537e00640 1 -- 172.21.15.46:0/3355240781 wait complete. 2024-09-10T07:53:38.150 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b[65468]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.150 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b[65468]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.150 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-mds[65472]: -- 172.21.15.63:0/1424477213 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f12e851c20 con 0x55f12e82f000 2024-09-10T07:53:38.150 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b[65468]: starting mds.b at 2024-09-10T07:53:38.150 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e[64864]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.151 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e[64864]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.151 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-mds[64868]: -- 172.21.15.63:0/337593112 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f34089f860 con 0x55f34087d000 2024-09-10T07:53:38.151 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e[64864]: starting mds.e at 2024-09-10T07:53:38.151 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h[65167]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.151 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h[65167]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.151 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-mds[65171]: -- 172.21.15.63:0/1386324191 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5ad081860 con 0x55f5ad05e800 2024-09-10T07:53:38.151 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h[65167]: starting mds.h at 2024-09-10T07:53:38.152 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k[64550]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.152 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k[64550]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.152 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-mds[64554]: -- 172.21.15.63:0/2780501292 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e6b2f5860 con 0x556e6b2d3400 2024-09-10T07:53:38.152 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:53:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k[64550]: starting mds.k at 2024-09-10T07:53:38.394 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:37 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a[30700]: 2024-09-10T07:53:37.918+0000 7fb7b8600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:53:38.394 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-mon[30705]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-10T07:53:38.394 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3355240781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T07:53:38.395 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:53:37 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j[67026]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.395 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:53:37 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j[67026]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.395 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:53:37 smithi046 ceph-mds[67043]: -- 172.21.15.46:0/1266661226 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561da0f0fc20 con 0x561da0eed400 2024-09-10T07:53:38.395 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:53:37 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j[67026]: starting mds.j at 2024-09-10T07:53:38.395 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:53:37 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.395 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:53:37 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.395 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-mds[68610]: -- 172.21.15.46:0/3969465805 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56351764f860 con 0x56351762c800 2024-09-10T07:53:38.395 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: starting mds.a at 2024-09-10T07:53:38.396 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l[65443]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.396 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l[65443]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.396 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-mds[65447]: -- 172.21.15.181:0/1520144728 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55627513b860 con 0x556275119000 2024-09-10T07:53:38.396 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l[65443]: starting mds.l at 2024-09-10T07:53:38.397 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c[66052]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.397 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c[66052]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.397 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-mds[66056]: -- 172.21.15.181:0/2765950516 <== mon.2 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55926b80f860 con 0x55926b7ec800 2024-09-10T07:53:38.397 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c[66052]: starting mds.c at 2024-09-10T07:53:38.397 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f[65755]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.398 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f[65755]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.398 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-mds[65759]: -- 172.21.15.181:0/3530971731 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560633bbc20 con 0x556063399000 2024-09-10T07:53:38.398 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f[65755]: starting mds.f at 2024-09-10T07:53:38.398 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i[66355]: ignoring --setuser ceph since I am not root 2024-09-10T07:53:38.398 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i[66355]: ignoring --setgroup ceph since I am not root 2024-09-10T07:53:38.399 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-mds[66359]: -- 172.21.15.181:0/3541156772 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfe889d860 con 0x55cfe887b000 2024-09-10T07:53:38.399 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:53:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i[66355]: starting mds.i at 2024-09-10T07:53:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:38 smithi181 ceph-mon[35397]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-10T07:53:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:38 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3355240781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T07:53:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:38 smithi181 ceph-mon[35397]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T07:53:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:38 smithi181 ceph-mon[35397]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:53:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:38 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3355240781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T07:53:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:38 smithi181 ceph-mon[35397]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T07:53:38.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:38 smithi181 ceph-mon[35397]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T07:53:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:38 smithi063 ceph-mon[35020]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-10T07:53:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:38 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3355240781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T07:53:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:38 smithi063 ceph-mon[35020]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T07:53:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:38 smithi063 ceph-mon[35020]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:53:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:38 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3355240781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T07:53:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:38 smithi063 ceph-mon[35020]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T07:53:38.650 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:38 smithi063 ceph-mon[35020]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T07:53:38.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-mon[30705]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T07:53:38.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-mon[30705]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:53:38.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3355240781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T07:53:38.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-mon[30705]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T07:53:38.658 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:38 smithi046 ceph-mon[30705]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T07:53:38.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T07:53:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 -- 172.21.15.46:0/3373942409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 msgr2=0x7f84141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 --2- 172.21.15.46:0/3373942409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 0x7f84141067e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009860 tx=0x7f83fc02a340 comp rx=0 tx=0).stop 2024-09-10T07:53:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 -- 172.21.15.46:0/3373942409 shutdown_connections 2024-09-10T07:53:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 --2- 172.21.15.46:0/3373942409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414106f10 0x7f841410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 --2- 172.21.15.46:0/3373942409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 0x7f84141067e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 --2- 172.21.15.46:0/3373942409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8414101de0 0x7f84141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 -- 172.21.15.46:0/3373942409 >> 172.21.15.46:0/3373942409 conn(0x7f84140fd8c0 msgr2=0x7f84140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 -- 172.21.15.46:0/3373942409 shutdown_connections 2024-09-10T07:53:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.996+0000 7f8419e00640 1 -- 172.21.15.46:0/3373942409 wait complete. 2024-09-10T07:53:39.997 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6832/1038562982,v1:172.21.15.181:6834/1038562982] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6833/1014279097,v1:172.21.15.181:6835/1014279097] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6832/2575070669,v1:172.21.15.63:6833/2575070669] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6836/1901241755,v1:172.21.15.181:6837/1901241755] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6834/1352811634,v1:172.21.15.63:6835/1352811634] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.181:6838/298442617,v1:172.21.15.181:6839/298442617] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6836/1542079089,v1:172.21.15.63:6837/1542079089] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.63:6838/1796822146,v1:172.21.15.63:6839/1796822146] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.46:6834/2588171534,v1:172.21.15.46:6835/2588171534] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: mds.? [v2:172.21.15.46:6840/434976577,v1:172.21.15.46:6841/434976577] up:boot 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T07:53:39.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T07:53:39.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:39 smithi046 ceph-mon[30705]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-10T07:53:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.997+0000 7f8419e00640 1 Processor -- start 2024-09-10T07:53:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8419e00640 1 -- start start 2024-09-10T07:53:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8419e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8414101de0 0x7f841419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8419e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 0x7f841419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8419e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414106f10 0x7f841419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8419e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111f30 con 0x7f8414101de0 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8419e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111db0 con 0x7f8414102790 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8419e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f84141120b0 con 0x7f8414106f10 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8412a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8414101de0 0x7f841419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8412a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8414101de0 0x7f841419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52660/0 (socket says 172.21.15.46:52660) 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.998+0000 7f8412a00640 1 -- 172.21.15.46:0/3668488405 learned_addr learned my addr 172.21.15.46:0/3668488405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f840be00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 0x7f841419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f8413400640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414106f10 0x7f841419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f840be00640 1 -- 172.21.15.46:0/3668488405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414106f10 msgr2=0x7f841419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f840be00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414106f10 0x7f841419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f840be00640 1 -- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8414101de0 msgr2=0x7f841419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f840be00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8414101de0 0x7f841419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f840be00640 1 -- 172.21.15.46:0/3668488405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841419f650 con 0x7f8414102790 2024-09-10T07:53:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f8412a00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8414101de0 0x7f841419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:39.999+0000 7f840be00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 0x7f841419abe0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f83fc035040 tx=0x7f83fc033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.000+0000 7f8418a00640 1 -- 172.21.15.46:0/3668488405 <== mon.2 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc03f020 con 0x7f8414102790 2024-09-10T07:53:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.000+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841419f8e0 con 0x7f8414102790 2024-09-10T07:53:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.000+0000 7f8418a00640 1 -- 172.21.15.46:0/3668488405 <== mon.2 v2:172.21.15.63:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc030550 con 0x7f8414102790 2024-09-10T07:53:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.000+0000 7f8418a00640 1 -- 172.21.15.46:0/3668488405 <== mon.2 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc037040 con 0x7f8414102790 2024-09-10T07:53:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.000+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84141a7220 con 0x7f8414102790 2024-09-10T07:53:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.001+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84141021c0 con 0x7f8414102790 2024-09-10T07:53:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.003+0000 7f8418a00640 1 -- 172.21.15.46:0/3668488405 <== mon.2 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f83fc042020 con 0x7f8414102790 2024-09-10T07:53:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.005+0000 7f8418a00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83f80a2b80 0x7f83f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.005+0000 7f8418a00640 1 -- 172.21.15.46:0/3668488405 <== mon.2 v2:172.21.15.63:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9551+0+0 (secure 0 0 0) 0x7f83fc0ef640 con 0x7f8414102790 2024-09-10T07:53:40.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.005+0000 7f8412a00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83f80a2b80 0x7f83f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:40.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.006+0000 7f8412a00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83f80a2b80 0x7f83f80a5040 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f84040096c0 tx=0x7f8404009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:40.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.007+0000 7f8418a00640 1 -- 172.21.15.46:0/3668488405 <== mon.2 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83fc049020 con 0x7f8414102790 2024-09-10T07:53:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.228+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f84141a74c0 con 0x7f8414102790 2024-09-10T07:53:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6832/1038562982,v1:172.21.15.181:6834/1038562982] up:boot 2024-09-10T07:53:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6833/1014279097,v1:172.21.15.181:6835/1014279097] up:boot 2024-09-10T07:53:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6832/2575070669,v1:172.21.15.63:6833/2575070669] up:boot 2024-09-10T07:53:40.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6836/1901241755,v1:172.21.15.181:6837/1901241755] up:boot 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6834/1352811634,v1:172.21.15.63:6835/1352811634] up:boot 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.181:6838/298442617,v1:172.21.15.181:6839/298442617] up:boot 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6836/1542079089,v1:172.21.15.63:6837/1542079089] up:boot 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.63:6838/1796822146,v1:172.21.15.63:6839/1796822146] up:boot 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.46:6834/2588171534,v1:172.21.15.46:6835/2588171534] up:boot 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: mds.? [v2:172.21.15.46:6840/434976577,v1:172.21.15.46:6841/434976577] up:boot 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T07:53:40.400 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:39 smithi181 ceph-mon[35397]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6832/1038562982,v1:172.21.15.181:6834/1038562982] up:boot 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6833/1014279097,v1:172.21.15.181:6835/1014279097] up:boot 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6832/2575070669,v1:172.21.15.63:6833/2575070669] up:boot 2024-09-10T07:53:40.401 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6836/1901241755,v1:172.21.15.181:6837/1901241755] up:boot 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6834/1352811634,v1:172.21.15.63:6835/1352811634] up:boot 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.181:6838/298442617,v1:172.21.15.181:6839/298442617] up:boot 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6836/1542079089,v1:172.21.15.63:6837/1542079089] up:boot 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.63:6838/1796822146,v1:172.21.15.63:6839/1796822146] up:boot 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.46:6834/2588171534,v1:172.21.15.46:6835/2588171534] up:boot 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: mds.? [v2:172.21.15.46:6840/434976577,v1:172.21.15.46:6841/434976577] up:boot 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T07:53:40.402 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T07:53:40.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T07:53:40.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T07:53:40.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T07:53:40.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: from='mgr.14150 172.21.15.46:0/381743593' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T07:53:40.403 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:39 smithi063 ceph-mon[35020]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-10T07:53:40.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.932+0000 7f8418a00640 1 -- 172.21.15.46:0/3668488405 <== mon.2 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) ==== 104+0+0 (secure 0 0 0) 0x7f83fc0bc020 con 0x7f8414102790 2024-09-10T07:53:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83f80a2b80 msgr2=0x7f83f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83f80a2b80 0x7f83f80a5040 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f84040096c0 tx=0x7f8404009260 comp rx=0 tx=0).stop 2024-09-10T07:53:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 msgr2=0x7f841419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 0x7f841419abe0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f83fc035040 tx=0x7f83fc033b60 comp rx=0 tx=0).stop 2024-09-10T07:53:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 shutdown_connections 2024-09-10T07:53:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8414106f10 0x7f841419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8414102790 0x7f841419abe0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f83f80a2b80 0x7f83f80a5040 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 --2- 172.21.15.46:0/3668488405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8414101de0 0x7f841419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 >> 172.21.15.46:0/3668488405 conn(0x7f84140fd8c0 msgr2=0x7f84140fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 shutdown_connections 2024-09-10T07:53:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:40.936+0000 7f8419e00640 1 -- 172.21.15.46:0/3668488405 wait complete. 2024-09-10T07:53:41.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:40 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3668488405' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:53:41.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:40 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:53:41.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:40 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3668488405' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:53:41.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:40 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:53:41.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:40 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3668488405' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:53:41.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:40 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T07:53:42.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T07:53:42.300 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:41 smithi181 ceph-mon[35397]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T07:53:42.300 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:41 smithi181 ceph-mon[35397]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T07:53:42.301 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:41 smithi181 ceph-mon[35397]: Cluster is now healthy 2024-09-10T07:53:42.301 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:41 smithi181 ceph-mon[35397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T07:53:42.301 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:41 smithi181 ceph-mon[35397]: fsmap 12 up:standby 2024-09-10T07:53:42.301 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:41 smithi181 ceph-mon[35397]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T07:53:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:41 smithi063 ceph-mon[35020]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T07:53:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:41 smithi063 ceph-mon[35020]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T07:53:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:41 smithi063 ceph-mon[35020]: Cluster is now healthy 2024-09-10T07:53:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:41 smithi063 ceph-mon[35020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T07:53:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:41 smithi063 ceph-mon[35020]: fsmap 12 up:standby 2024-09-10T07:53:42.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:41 smithi063 ceph-mon[35020]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T07:53:42.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:41 smithi046 ceph-mon[30705]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T07:53:42.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:41 smithi046 ceph-mon[30705]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T07:53:42.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:41 smithi046 ceph-mon[30705]: Cluster is now healthy 2024-09-10T07:53:42.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:41 smithi046 ceph-mon[30705]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T07:53:42.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:41 smithi046 ceph-mon[30705]: fsmap 12 up:standby 2024-09-10T07:53:42.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:41 smithi046 ceph-mon[30705]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T07:53:42.449 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:43.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 -- 172.21.15.46:0/758461158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8081049c0 msgr2=0x7ff808108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 --2- 172.21.15.46:0/758461158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8081049c0 0x7ff808108a10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009900 tx=0x7ff7f002a3f0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 -- 172.21.15.46:0/758461158 shutdown_connections 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 --2- 172.21.15.46:0/758461158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff808109140 0x7ff808110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 --2- 172.21.15.46:0/758461158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8081049c0 0x7ff808108a10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 --2- 172.21.15.46:0/758461158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff808104010 0x7ff8081043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 -- 172.21.15.46:0/758461158 >> 172.21.15.46:0/758461158 conn(0x7ff8080ffa50 msgr2=0x7ff808101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 -- 172.21.15.46:0/758461158 shutdown_connections 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.209+0000 7ff80ec00640 1 -- 172.21.15.46:0/758461158 wait complete. 2024-09-10T07:53:43.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.210+0000 7ff80ec00640 1 Processor -- start 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.210+0000 7ff80ec00640 1 -- start start 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff80ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff808104010 0x7ff80819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff80ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8081049c0 0x7ff80819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff80ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff808109140 0x7ff80819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff80ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081141b0 con 0x7ff808104010 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff80ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff808114030 con 0x7ff808109140 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff80ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff808114330 con 0x7ff8081049c0 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff807e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff808104010 0x7ff80819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff807e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff808104010 0x7ff80819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52678/0 (socket says 172.21.15.46:52678) 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff807e00640 1 -- 172.21.15.46:0/2902000876 learned_addr learned my addr 172.21.15.46:0/2902000876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff7ff400640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8081049c0 0x7ff80819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff80cc00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff808109140 0x7ff80819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.211+0000 7ff807e00640 1 -- 172.21.15.46:0/2902000876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8081049c0 msgr2=0x7ff80819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff807e00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8081049c0 0x7ff80819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff807e00640 1 -- 172.21.15.46:0/2902000876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff808109140 msgr2=0x7ff80819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff807e00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff808109140 0x7ff80819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff807e00640 1 -- 172.21.15.46:0/2902000876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80819f660 con 0x7ff808104010 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff807e00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff808104010 0x7ff80819a6b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800e9f0 tx=0x7ff7f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff805600640 1 -- 172.21.15.46:0/2902000876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800ccc0 con 0x7ff808104010 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff805600640 1 -- 172.21.15.46:0/2902000876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f800ce60 con 0x7ff808104010 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80819f950 con 0x7ff808104010 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff805600640 1 -- 172.21.15.46:0/2902000876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8010690 con 0x7ff808104010 2024-09-10T07:53:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.212+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8081a7230 con 0x7ff808104010 2024-09-10T07:53:43.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.213+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8081043f0 con 0x7ff808104010 2024-09-10T07:53:43.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.216+0000 7ff805600640 1 -- 172.21.15.46:0/2902000876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff7f80046c0 con 0x7ff808104010 2024-09-10T07:53:43.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.217+0000 7ff805600640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7ec0a2b80 0x7ff7ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.217+0000 7ff805600640 1 -- 172.21.15.46:0/2902000876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9551+0+0 (secure 0 0 0) 0x7ff7f80cfba0 con 0x7ff808104010 2024-09-10T07:53:43.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.217+0000 7ff7ff400640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7ec0a2b80 0x7ff7ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.218+0000 7ff7ff400640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7ec0a2b80 0x7ff7ec0a5040 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000c00 tx=0x7ff7f0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.224+0000 7ff805600640 1 -- 172.21.15.46:0/2902000876 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7f80541b0 con 0x7ff808104010 2024-09-10T07:53:43.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:43.411+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff80819ba10 con 0x7ff808104010 2024-09-10T07:53:44.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.064+0000 7ff805600640 1 -- 172.21.15.46:0/2902000876 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7ff80819ba10 con 0x7ff808104010 2024-09-10T07:53:44.063 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.068+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7ec0a2b80 msgr2=0x7ff7ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.068+0000 7ff80ec00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7ec0a2b80 0x7ff7ec0a5040 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000c00 tx=0x7ff7f0005b50 comp rx=0 tx=0).stop 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.068+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff808104010 msgr2=0x7ff80819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.068+0000 7ff80ec00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff808104010 0x7ff80819a6b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800e9f0 tx=0x7ff7f800edf0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.069+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 shutdown_connections 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.069+0000 7ff80ec00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff808109140 0x7ff80819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.069+0000 7ff80ec00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8081049c0 0x7ff80819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.069+0000 7ff80ec00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7ff7ec0a2b80 0x7ff7ec0a5040 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.069+0000 7ff80ec00640 1 --2- 172.21.15.46:0/2902000876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff808104010 0x7ff80819a6b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.069+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 >> 172.21.15.46:0/2902000876 conn(0x7ff8080ffa50 msgr2=0x7ff808100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.069+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 shutdown_connections 2024-09-10T07:53:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:44.069+0000 7ff80ec00640 1 -- 172.21.15.46:0/2902000876 wait complete. 2024-09-10T07:53:44.212 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:44 smithi181 ceph-mon[35397]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-10T07:53:44.212 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:44 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2902000876' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:44 smithi063 ceph-mon[35020]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-10T07:53:44.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:44 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2902000876' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:44.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:44 smithi046 ceph-mon[30705]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-10T07:53:44.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:44 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2902000876' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:45.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T07:53:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:45 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2902000876' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:45.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:45 smithi181 ceph-mon[35397]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T07:53:45.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:45 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2902000876' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:45.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:45 smithi063 ceph-mon[35020]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T07:53:45.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:45 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2902000876' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:45.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:45 smithi046 ceph-mon[30705]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:46.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:46 smithi181 ceph-mon[35397]: pgmap v915: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:46.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:46 smithi063 ceph-mon[35020]: pgmap v915: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:46.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:46 smithi046 ceph-mon[30705]: pgmap v915: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 -- 172.21.15.46:0/2604378893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 msgr2=0x7f87f4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 --2- 172.21.15.46:0/2604378893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 0x7f87f4108830 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009860 tx=0x7f87e002a340 comp rx=0 tx=0).stop 2024-09-10T07:53:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 -- 172.21.15.46:0/2604378893 shutdown_connections 2024-09-10T07:53:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 --2- 172.21.15.46:0/2604378893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87f4108f60 0x7f87f4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 --2- 172.21.15.46:0/2604378893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 0x7f87f4108830 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 --2- 172.21.15.46:0/2604378893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87f4103e30 0x7f87f4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 -- 172.21.15.46:0/2604378893 >> 172.21.15.46:0/2604378893 conn(0x7f87f40ff8d0 msgr2=0x7f87f4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 -- 172.21.15.46:0/2604378893 shutdown_connections 2024-09-10T07:53:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.411+0000 7f87fac00640 1 -- 172.21.15.46:0/2604378893 wait complete. 2024-09-10T07:53:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.412+0000 7f87fac00640 1 Processor -- start 2024-09-10T07:53:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.412+0000 7f87fac00640 1 -- start start 2024-09-10T07:53:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87fac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87f4103e30 0x7f87f419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87fac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 0x7f87f419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87fac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87f4108f60 0x7f87f419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87fac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4113fc0 con 0x7f87f4103e30 2024-09-10T07:53:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87fac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4113e40 con 0x7f87f4108f60 2024-09-10T07:53:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87fac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4114140 con 0x7f87f41047e0 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87f9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87f4103e30 0x7f87f419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87f8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 0x7f87f419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87f8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 0x7f87f419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.46:33232/0 (socket says 172.21.15.46:33232) 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87f8e00640 1 -- 172.21.15.46:0/2639534724 learned_addr learned my addr 172.21.15.46:0/2639534724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87fa200640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87f4108f60 0x7f87f419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87f9800640 1 -- 172.21.15.46:0/2639534724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 msgr2=0x7f87f419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.413+0000 7f87f9800640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 0x7f87f419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87f9800640 1 -- 172.21.15.46:0/2639534724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87f4108f60 msgr2=0x7f87f419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87f9800640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87f4108f60 0x7f87f419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87f9800640 1 -- 172.21.15.46:0/2639534724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f419f450 con 0x7f87f4103e30 2024-09-10T07:53:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87f8e00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 0x7f87f419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87fa200640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87f4108f60 0x7f87f419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87f9800640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87f4103e30 0x7f87f419a4a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f87e800e9f0 tx=0x7f87e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87f2000640 1 -- 172.21.15.46:0/2639534724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e800ccc0 con 0x7f87f4103e30 2024-09-10T07:53:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87f2000640 1 -- 172.21.15.46:0/2639534724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e8010910 con 0x7f87f4103e30 2024-09-10T07:53:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f419f740 con 0x7f87f4103e30 2024-09-10T07:53:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87f2000640 1 -- 172.21.15.46:0/2639534724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8018900 con 0x7f87f4103e30 2024-09-10T07:53:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.414+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f41a7020 con 0x7f87f4103e30 2024-09-10T07:53:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.415+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f4104210 con 0x7f87f4103e30 2024-09-10T07:53:46.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.417+0000 7f87f2000640 1 -- 172.21.15.46:0/2639534724 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87e8010ac0 con 0x7f87f4103e30 2024-09-10T07:53:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.419+0000 7f87f2000640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87dc0a2b80 0x7f87dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.419+0000 7f87f2000640 1 -- 172.21.15.46:0/2639534724 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9055+0+0 (secure 0 0 0) 0x7f87e80cf6e0 con 0x7f87f4103e30 2024-09-10T07:53:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.419+0000 7f87f8e00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87dc0a2b80 0x7f87dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.420+0000 7f87f8e00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87dc0a2b80 0x7f87dc0a5040 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f87e0031000 tx=0x7f87e00056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.423+0000 7f87f2000640 1 -- 172.21.15.46:0/2639534724 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87e809bc60 con 0x7f87f4103e30 2024-09-10T07:53:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:46.609+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f87f402d7c0 con 0x7f87f4103e30 2024-09-10T07:53:47.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.076+0000 7f87f2000640 1 -- 172.21.15.46:0/2639534724 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7f87e809ff60 con 0x7f87f4103e30 2024-09-10T07:53:47.076 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.080+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87dc0a2b80 msgr2=0x7f87dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.080+0000 7f87fac00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87dc0a2b80 0x7f87dc0a5040 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f87e0031000 tx=0x7f87e00056d0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87f4103e30 msgr2=0x7f87f419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87f4103e30 0x7f87f419a4a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f87e800e9f0 tx=0x7f87e800eeb0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 shutdown_connections 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87f4108f60 0x7f87f419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87f41047e0 0x7f87f419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7f87dc0a2b80 0x7f87dc0a5040 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 --2- 172.21.15.46:0/2639534724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87f4103e30 0x7f87f419a4a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 >> 172.21.15.46:0/2639534724 conn(0x7f87f40ff8d0 msgr2=0x7f87f4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:47.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 shutdown_connections 2024-09-10T07:53:47.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:47.081+0000 7f87fac00640 1 -- 172.21.15.46:0/2639534724 wait complete. 2024-09-10T07:53:47.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:47 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2639534724' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:47.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:47 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2639534724' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:47.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:47 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2639534724' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:48.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T07:53:48.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:48 smithi046 ceph-mon[30705]: pgmap v916: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:48.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:48 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/2639534724' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:48.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:48 smithi046 ceph-mon[30705]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T07:53:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:48 smithi181 ceph-mon[35397]: pgmap v916: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:48 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/2639534724' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:48.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:48 smithi181 ceph-mon[35397]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T07:53:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:48 smithi063 ceph-mon[35020]: pgmap v916: 97 pgs: 97 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:48 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/2639534724' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:48.400 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:48 smithi063 ceph-mon[35020]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T07:53:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 -- 172.21.15.46:0/3520814978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8106f10 msgr2=0x7fd2e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3520814978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8106f10 0x7fd2e810ea90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009ca0 tx=0x7fd2e002a8c0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 -- 172.21.15.46:0/3520814978 shutdown_connections 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3520814978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8106f10 0x7fd2e810ea90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3520814978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e8102790 0x7fd2e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3520814978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2e8101de0 0x7fd2e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 -- 172.21.15.46:0/3520814978 >> 172.21.15.46:0/3520814978 conn(0x7fd2e80fd8c0 msgr2=0x7fd2e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 -- 172.21.15.46:0/3520814978 shutdown_connections 2024-09-10T07:53:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.530+0000 7fd2ef400640 1 -- 172.21.15.46:0/3520814978 wait complete. 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.531+0000 7fd2ef400640 1 Processor -- start 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2ef400640 1 -- start start 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2ef400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e8101de0 0x7fd2e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8102790 0x7fd2e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2e8106f10 0x7fd2e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2ef400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e8111f70 con 0x7fd2e8102790 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2ef400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e8111df0 con 0x7fd2e8106f10 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2ef400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e81120f0 con 0x7fd2e8101de0 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8102790 0x7fd2e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8102790 0x7fd2e8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35350/0 (socket says 172.21.15.46:35350) 2024-09-10T07:53:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2eca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e8101de0 0x7fd2e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2e7e00640 1 -- 172.21.15.46:0/3331931627 learned_addr learned my addr 172.21.15.46:0/3331931627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T07:53:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.532+0000 7fd2ed400640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2e8106f10 0x7fd2e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e7e00640 1 -- 172.21.15.46:0/3331931627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e8101de0 msgr2=0x7fd2e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e7e00640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e8101de0 0x7fd2e80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e7e00640 1 -- 172.21.15.46:0/3331931627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2e8106f10 msgr2=0x7fd2e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e7e00640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2e8106f10 0x7fd2e8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e7e00640 1 -- 172.21.15.46:0/3331931627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e81a68d0 con 0x7fd2e8102790 2024-09-10T07:53:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e7e00640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8102790 0x7fd2e8074a30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400e9f0 tx=0x7fd2d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e5600640 1 -- 172.21.15.46:0/3331931627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d400cd80 con 0x7fd2e8102790 2024-09-10T07:53:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e5600640 1 -- 172.21.15.46:0/3331931627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d4005500 con 0x7fd2e8102790 2024-09-10T07:53:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2e5600640 1 -- 172.21.15.46:0/3331931627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4010690 con 0x7fd2e8102790 2024-09-10T07:53:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.533+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e81a6bc0 con 0x7fd2e8102790 2024-09-10T07:53:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.534+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e81a70b0 con 0x7fd2e8102790 2024-09-10T07:53:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.535+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e81021c0 con 0x7fd2e8102790 2024-09-10T07:53:49.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.537+0000 7fd2e5600640 1 -- 172.21.15.46:0/3331931627 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd2d40046c0 con 0x7fd2e8102790 2024-09-10T07:53:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.538+0000 7fd2e5600640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2d00a2c00 0x7fd2d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.538+0000 7fd2e5600640 1 -- 172.21.15.46:0/3331931627 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8496+0+0 (secure 0 0 0) 0x7fd2d40cf460 con 0x7fd2e8102790 2024-09-10T07:53:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.538+0000 7fd2eca00640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2d00a2c00 0x7fd2d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.539+0000 7fd2eca00640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2d00a2c00 0x7fd2d00a50c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009bd0 tx=0x7fd2dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.542+0000 7fd2e5600640 1 -- 172.21.15.46:0/3331931627 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2d409c280 con 0x7fd2e8102790 2024-09-10T07:53:49.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:49.725+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd2e810a0f0 con 0x7fd2e8102790 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.088+0000 7fd2e5600640 1 -- 172.21.15.46:0/3331931627 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v101) ==== 178+0+0 (secure 0 0 0) 0x7fd2d409bb80 con 0x7fd2e8102790 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' removed 2024-09-10T07:53:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.091+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2d00a2c00 msgr2=0x7fd2d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.091+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2d00a2c00 0x7fd2d00a50c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009bd0 tx=0x7fd2dc005f90 comp rx=0 tx=0).stop 2024-09-10T07:53:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.091+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8102790 msgr2=0x7fd2e8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.091+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8102790 0x7fd2e8074a30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400e9f0 tx=0x7fd2d400edf0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.092+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 shutdown_connections 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.092+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2e8106f10 0x7fd2e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.092+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:6800/1492955812,v1:172.21.15.46:6801/1492955812] conn(0x7fd2d00a2c00 0x7fd2d00a50c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.092+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e8102790 0x7fd2e8074a30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.092+0000 7fd2ef400640 1 --2- 172.21.15.46:0/3331931627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e8101de0 0x7fd2e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.092+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 >> 172.21.15.46:0/3331931627 conn(0x7fd2e80fd8c0 msgr2=0x7fd2e80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.093+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 shutdown_connections 2024-09-10T07:53:50.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T07:53:50.093+0000 7fd2ef400640 1 -- 172.21.15.46:0/3331931627 wait complete. 2024-09-10T07:53:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:50 smithi181 ceph-mon[35397]: pgmap v918: 33 pgs: 33 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:50.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:50 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3331931627' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:50.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:50 smithi063 ceph-mon[35020]: pgmap v918: 33 pgs: 33 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:50.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:50 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3331931627' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:50.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:50 smithi046 ceph-mon[30705]: pgmap v918: 33 pgs: 33 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T07:53:50.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:50 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3331931627' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T07:53:51.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:53:51.254 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:53:51.282 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:53:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:51 smithi181 ceph-mon[35397]: from='client.? 172.21.15.46:0/3331931627' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:51.399 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:51 smithi181 ceph-mon[35397]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T07:53:51.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:51 smithi063 ceph-mon[35020]: from='client.? 172.21.15.46:0/3331931627' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:51.399 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:51 smithi063 ceph-mon[35020]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T07:53:51.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:51 smithi046 ceph-mon[30705]: from='client.? 172.21.15.46:0/3331931627' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T07:53:51.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:51 smithi046 ceph-mon[30705]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T07:53:51.558 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T07:53:51.559 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T07:53:51.594 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T07:53:51.610 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T07:53:51.625 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T07:53:51.626 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T07:53:51.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a 2024-09-10T07:53:52.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:51 smithi046 systemd[1]: Stopping Ceph mon.a for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:53:52.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a[30700]: 2024-09-10T07:53:52.054+0000 7fb7bf400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:53:52.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a[30700]: 2024-09-10T07:53:52.055+0000 7fb7bf400640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T07:53:52.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:52 smithi046 podman[119448]: 2024-09-10 07:53:52.293468088 +0000 UTC m=+0.366191295 container died 735e6e66d0038d4e4e9ba09118b2733a8892bf9c2fead7ab272e4351cd046b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:53:52.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:52 smithi046 podman[119448]: 2024-09-10 07:53:52.819706341 +0000 UTC m=+0.892429529 container cleanup 735e6e66d0038d4e4e9ba09118b2733a8892bf9c2fead7ab272e4351cd046b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T07:53:52.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:52 smithi046 bash[119448]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a 2024-09-10T07:53:53.386 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 10 07:53:53 smithi046 podman[119460]: 2024-09-10 07:53:53.111841857 +0000 UTC m=+0.806314265 container remove 735e6e66d0038d4e4e9ba09118b2733a8892bf9c2fead7ab272e4351cd046b03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T07:53:53.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T07:53:53.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:53:53.495 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T07:53:53.496 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T07:53:53.496 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.b 2024-09-10T07:53:53.816 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:53 smithi063 systemd[1]: Stopping Ceph mon.b for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:53:54.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:53 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-b[35016]: 2024-09-10T07:53:53.816+0000 7f0fe06b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:53:54.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:53 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-b[35016]: 2024-09-10T07:53:53.816+0000 7f0fe06b5640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T07:53:54.150 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:53 smithi063 podman[71304]: 2024-09-10 07:53:53.919015201 +0000 UTC m=+0.218943169 container died 042fae3245d2cce33ef69a67a7b50366a787709980778e8f919866061e7174fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T07:53:54.460 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:54 smithi063 podman[71304]: 2024-09-10 07:53:54.259758451 +0000 UTC m=+0.559686425 container cleanup 042fae3245d2cce33ef69a67a7b50366a787709980778e8f919866061e7174fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:53:54.460 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:54 smithi063 bash[71304]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-b 2024-09-10T07:53:54.688 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T07:53:54.711 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:54 smithi063 podman[71316]: 2024-09-10 07:53:54.46137032 +0000 UTC m=+0.538364973 container remove 042fae3245d2cce33ef69a67a7b50366a787709980778e8f919866061e7174fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T07:53:54.711 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:54 smithi063 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-10T07:53:54.711 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:54 smithi063 systemd[1]: Stopped Ceph mon.b for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:53:54.711 INFO:journalctl@ceph.mon.b.smithi063.stdout:Sep 10 07:53:54 smithi063 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.b.service: Consumed 16.840s CPU time. 2024-09-10T07:53:54.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:53:54.734 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T07:53:54.734 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T07:53:54.734 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.c 2024-09-10T07:53:55.103 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:54 smithi181 systemd[1]: Stopping Ceph mon.c for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:53:55.361 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:55 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-c[35393]: 2024-09-10T07:53:55.168+0000 7f216bee5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:53:55.362 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:55 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-c[35393]: 2024-09-10T07:53:55.168+0000 7f216bee5640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T07:53:55.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:55 smithi181 podman[71693]: 2024-09-10 07:53:55.36292081 +0000 UTC m=+0.344286610 container died 1f55219d589cf01ff2f7542c4364c9847ff402cf5d002330486059b685792b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:53:56.045 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:55 smithi181 podman[71693]: 2024-09-10 07:53:55.810647991 +0000 UTC m=+0.792013850 container cleanup 1f55219d589cf01ff2f7542c4364c9847ff402cf5d002330486059b685792b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T07:53:56.045 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:55 smithi181 bash[71693]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-c 2024-09-10T07:53:56.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T07:53:56.297 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:56 smithi181 podman[71707]: 2024-09-10 07:53:56.045679151 +0000 UTC m=+0.668841974 container remove 1f55219d589cf01ff2f7542c4364c9847ff402cf5d002330486059b685792b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mon-c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:53:56.298 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:56 smithi181 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-10T07:53:56.298 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:56 smithi181 systemd[1]: Stopped Ceph mon.c for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:53:56.298 INFO:journalctl@ceph.mon.c.smithi181.stdout:Sep 10 07:53:56 smithi181 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mon.c.service: Consumed 17.265s CPU time. 2024-09-10T07:53:56.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:53:56.320 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T07:53:56.320 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T07:53:56.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.x 2024-09-10T07:53:56.658 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:53:56 smithi046 systemd[1]: Stopping Ceph mgr.x for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:53:57.384 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:53:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:53:56.926+0000 7f7578600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:53:57.384 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:53:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x[30939]: 2024-09-10T07:53:56.927+0000 7f7578600640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T07:53:57.385 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:53:57 smithi046 podman[119566]: 2024-09-10 07:53:57.01158733 +0000 UTC m=+0.386608024 container died 500c990df560c6dc580deb272da0dd8787604db33b2d0600753b9bf8b5e03698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T07:53:58.158 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:53:57 smithi046 podman[119566]: 2024-09-10 07:53:57.692572653 +0000 UTC m=+1.067593355 container cleanup 500c990df560c6dc580deb272da0dd8787604db33b2d0600753b9bf8b5e03698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T07:53:58.158 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:53:57 smithi046 bash[119566]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x 2024-09-10T07:53:58.159 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 10 07:53:57 smithi046 podman[119578]: 2024-09-10 07:53:57.911063421 +0000 UTC m=+0.871381876 container remove 500c990df560c6dc580deb272da0dd8787604db33b2d0600753b9bf8b5e03698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T07:53:58.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T07:53:58.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:53:58.272 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T07:53:58.272 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T07:53:58.272 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.y 2024-09-10T07:53:58.609 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:58 smithi063 systemd[1]: Stopping Ceph mgr.y for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:53:58.897 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:58 smithi063 podman[71427]: 2024-09-10 07:53:58.61089151 +0000 UTC m=+0.138436495 container died 1d95314dabcad8e607d547ae00b3c459e086312f1282168b5d39a089ba5260a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T07:53:59.150 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:59 smithi063 podman[71427]: 2024-09-10 07:53:59.13232799 +0000 UTC m=+0.659872974 container cleanup 1d95314dabcad8e607d547ae00b3c459e086312f1282168b5d39a089ba5260a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T07:53:59.150 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:59 smithi063 bash[71427]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y 2024-09-10T07:53:59.621 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:59 smithi063 podman[71439]: 2024-09-10 07:53:59.367864582 +0000 UTC m=+0.747736622 container remove 1d95314dabcad8e607d547ae00b3c459e086312f1282168b5d39a089ba5260a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T07:53:59.621 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:59 smithi063 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T07:53:59.621 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:59 smithi063 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-10T07:53:59.621 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:59 smithi063 systemd[1]: Stopped Ceph mgr.y for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:53:59.621 INFO:journalctl@ceph.mgr.y.smithi063.stdout:Sep 10 07:53:59 smithi063 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.y.service: Consumed 7.097s CPU time. 2024-09-10T07:53:59.622 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T07:53:59.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:53:59.656 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T07:53:59.657 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T07:53:59.657 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.z 2024-09-10T07:54:00.019 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:53:59 smithi181 systemd[1]: Stopping Ceph mgr.z for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:54:00.325 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:54:00 smithi181 podman[71813]: 2024-09-10 07:54:00.020378321 +0000 UTC m=+0.155881142 container died 8a1ad40a12991e5c5a1a0f755144278a059271e72525bcec86cbc3e9ad49706d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T07:54:00.649 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:54:00 smithi181 podman[71813]: 2024-09-10 07:54:00.433935383 +0000 UTC m=+0.569438192 container cleanup 8a1ad40a12991e5c5a1a0f755144278a059271e72525bcec86cbc3e9ad49706d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:54:00.649 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Sep 10 07:54:00 smithi181 bash[71813]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mgr-z 2024-09-10T07:54:00.857 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T07:54:00.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:54:00.892 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T07:54:00.892 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T07:54:00.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.0 2024-09-10T07:54:01.325 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:00 smithi046 systemd[1]: Stopping Ceph osd.0 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:54:01.658 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:01 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0[41979]: 2024-09-10T07:54:01.323+0000 7f41ea000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:54:01.659 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:01 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0[41979]: 2024-09-10T07:54:01.323+0000 7f41ea000640 -1 osd.0 101 *** Got signal Terminated *** 2024-09-10T07:54:01.659 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:01 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0[41979]: 2024-09-10T07:54:01.323+0000 7f41ea000640 -1 osd.0 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:54:07.409 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:07 smithi046 podman[119688]: 2024-09-10 07:54:07.133246014 +0000 UTC m=+5.950390831 container stop eda15779d56e2ee530aae999ec990398e0e6327ee8323eb836c950f404cba078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T07:54:07.409 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:07 smithi046 podman[119688]: 2024-09-10 07:54:07.160783856 +0000 UTC m=+5.977928684 container died eda15779d56e2ee530aae999ec990398e0e6327ee8323eb836c950f404cba078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:54:08.016 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:07 smithi046 podman[119688]: 2024-09-10 07:54:07.74885383 +0000 UTC m=+6.565998683 container cleanup eda15779d56e2ee530aae999ec990398e0e6327ee8323eb836c950f404cba078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T07:54:08.017 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:07 smithi046 bash[119688]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0 2024-09-10T07:54:08.408 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:08 smithi046 podman[119707]: 2024-09-10 07:54:08.017323176 +0000 UTC m=+0.847769691 container remove eda15779d56e2ee530aae999ec990398e0e6327ee8323eb836c950f404cba078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T07:54:08.908 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:08 smithi046 podman[119792]: 2024-09-10 07:54:08.491330824 +0000 UTC m=+0.027331566 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:54:08.908 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:08 smithi046 podman[119792]: 2024-09-10 07:54:08.634904969 +0000 UTC m=+0.170905710 container create 25a4618ca8bb2bd6d1723e1ae8c6c94c92f93585992fff774090d7085f33f1e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T07:54:09.320 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:08 smithi046 podman[119792]: 2024-09-10 07:54:08.954067213 +0000 UTC m=+0.490067956 container init 25a4618ca8bb2bd6d1723e1ae8c6c94c92f93585992fff774090d7085f33f1e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:54:09.320 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:08 smithi046 podman[119792]: 2024-09-10 07:54:08.960871494 +0000 UTC m=+0.496872248 container start 25a4618ca8bb2bd6d1723e1ae8c6c94c92f93585992fff774090d7085f33f1e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T07:54:09.320 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:09 smithi046 podman[119792]: 2024-09-10 07:54:09.079098512 +0000 UTC m=+0.615099255 container attach 25a4618ca8bb2bd6d1723e1ae8c6c94c92f93585992fff774090d7085f33f1e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T07:54:09.658 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 07:54:09 smithi046 podman[119792]: 2024-09-10 07:54:09.321960441 +0000 UTC m=+0.857961184 container died 25a4618ca8bb2bd6d1723e1ae8c6c94c92f93585992fff774090d7085f33f1e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:54:10.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T07:54:10.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:54:10.151 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T07:54:10.151 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T07:54:10.151 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.1 2024-09-10T07:54:10.462 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:10 smithi063 systemd[1]: Stopping Ceph osd.1 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:54:10.900 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1[39498]: 2024-09-10T07:54:10.462+0000 7fd9a8700640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:54:10.900 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1[39498]: 2024-09-10T07:54:10.462+0000 7fd9a8700640 -1 osd.1 101 *** Got signal Terminated *** 2024-09-10T07:54:10.900 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1[39498]: 2024-09-10T07:54:10.462+0000 7fd9a8700640 -1 osd.1 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:54:16.214 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:15 smithi063 podman[71560]: 2024-09-10 07:54:15.923923539 +0000 UTC m=+5.582220810 container died 14ff16b1b916bf68df738da0ff98b3b78541879a6155f50cb1308199f790dfdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T07:54:16.581 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:16 smithi063 podman[71560]: 2024-09-10 07:54:16.313235936 +0000 UTC m=+5.971533196 container cleanup 14ff16b1b916bf68df738da0ff98b3b78541879a6155f50cb1308199f790dfdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T07:54:16.581 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:16 smithi063 bash[71560]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1 2024-09-10T07:54:16.900 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:16 smithi063 podman[71574]: 2024-09-10 07:54:16.581945586 +0000 UTC m=+0.635977813 container remove 14ff16b1b916bf68df738da0ff98b3b78541879a6155f50cb1308199f790dfdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:54:17.217 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:16 smithi063 podman[71663]: 2024-09-10 07:54:16.836048723 +0000 UTC m=+0.016032128 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:54:17.218 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:16 smithi063 podman[71663]: 2024-09-10 07:54:16.983290301 +0000 UTC m=+0.163273737 container create e4da95031628714feded068eaf268e59b53b53ab2153599fe7a63377a3716b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T07:54:17.650 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:17 smithi063 podman[71663]: 2024-09-10 07:54:17.219067917 +0000 UTC m=+0.399051308 container init e4da95031628714feded068eaf268e59b53b53ab2153599fe7a63377a3716b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T07:54:17.650 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:17 smithi063 podman[71663]: 2024-09-10 07:54:17.222605818 +0000 UTC m=+0.402589217 container start e4da95031628714feded068eaf268e59b53b53ab2153599fe7a63377a3716b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T07:54:17.650 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:17 smithi063 podman[71663]: 2024-09-10 07:54:17.319272225 +0000 UTC m=+0.499255626 container attach e4da95031628714feded068eaf268e59b53b53ab2153599fe7a63377a3716b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T07:54:17.650 INFO:journalctl@ceph.osd.1.smithi063.stdout:Sep 10 07:54:17 smithi063 podman[71663]: 2024-09-10 07:54:17.431605962 +0000 UTC m=+0.611589360 container died e4da95031628714feded068eaf268e59b53b53ab2153599fe7a63377a3716b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T07:54:18.126 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T07:54:18.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:54:18.172 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T07:54:18.172 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T07:54:18.172 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.2 2024-09-10T07:54:18.460 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:18 smithi181 systemd[1]: Stopping Ceph osd.2 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:54:18.722 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2[40365]: 2024-09-10T07:54:18.479+0000 7f23c35b1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:54:18.722 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2[40365]: 2024-09-10T07:54:18.479+0000 7f23c35b1640 -1 osd.2 101 *** Got signal Terminated *** 2024-09-10T07:54:18.722 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2[40365]: 2024-09-10T07:54:18.479+0000 7f23c35b1640 -1 osd.2 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:54:24.355 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:24 smithi181 podman[71938]: 2024-09-10 07:54:24.052877 +0000 UTC m=+5.687923521 container died df896b16214a12e474cdb914e65af7a46bbd66c955b320a05e5426b002112827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T07:54:24.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:24 smithi181 podman[71938]: 2024-09-10 07:54:24.455138235 +0000 UTC m=+6.090184755 container cleanup df896b16214a12e474cdb914e65af7a46bbd66c955b320a05e5426b002112827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:54:24.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:24 smithi181 bash[71938]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2 2024-09-10T07:54:25.056 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:24 smithi181 podman[71965]: 2024-09-10 07:54:24.749833891 +0000 UTC m=+0.696840075 container remove df896b16214a12e474cdb914e65af7a46bbd66c955b320a05e5426b002112827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T07:54:25.336 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:25 smithi181 podman[72057]: 2024-09-10 07:54:24.960258944 +0000 UTC m=+0.015414452 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:54:25.337 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:25 smithi181 podman[72057]: 2024-09-10 07:54:25.0831123 +0000 UTC m=+0.138267841 container create b1c6d83f52bfb3053a64ed36f1618d619dc1fd0da27de99f4e022d5dacba3524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T07:54:25.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:25 smithi181 podman[72057]: 2024-09-10 07:54:25.338105701 +0000 UTC m=+0.393261197 container init b1c6d83f52bfb3053a64ed36f1618d619dc1fd0da27de99f4e022d5dacba3524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:54:25.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:25 smithi181 podman[72057]: 2024-09-10 07:54:25.341520091 +0000 UTC m=+0.396675590 container start b1c6d83f52bfb3053a64ed36f1618d619dc1fd0da27de99f4e022d5dacba3524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:54:25.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:25 smithi181 podman[72057]: 2024-09-10 07:54:25.446755128 +0000 UTC m=+0.501910637 container attach b1c6d83f52bfb3053a64ed36f1618d619dc1fd0da27de99f4e022d5dacba3524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:54:25.649 INFO:journalctl@ceph.osd.2.smithi181.stdout:Sep 10 07:54:25 smithi181 podman[72057]: 2024-09-10 07:54:25.555142701 +0000 UTC m=+0.610298197 container died b1c6d83f52bfb3053a64ed36f1618d619dc1fd0da27de99f4e022d5dacba3524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T07:54:26.249 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T07:54:26.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:54:26.288 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T07:54:26.288 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T07:54:26.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.3 2024-09-10T07:54:26.627 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:26 smithi046 systemd[1]: Stopping Ceph osd.3 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:54:26.908 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:26 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:54:26.738+0000 7f3ddd000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:54:26.909 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:26 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:54:26.738+0000 7f3ddd000640 -1 osd.3 101 *** Got signal Terminated *** 2024-09-10T07:54:26.909 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:26 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:54:26.738+0000 7f3ddd000640 -1 osd.3 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:54:28.290 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:28 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:28.168+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:28.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:28 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:28.048+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:29.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:29.003+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:29.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:29 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:29.183+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:29.408 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:29 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:54:29.206+0000 7f3dd8a00640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:02.388563+0000 front 2024-09-10T07:54:02.388611+0000 (oldest deadline 2024-09-10T07:54:28.288729+0000) 2024-09-10T07:54:30.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:29.997+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:30.658 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:54:30.178+0000 7f3dd8a00640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:02.388563+0000 front 2024-09-10T07:54:02.388611+0000 (oldest deadline 2024-09-10T07:54:28.288729+0000) 2024-09-10T07:54:30.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:30 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:30.198+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:31.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:30 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:30.976+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:31.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:31 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:31.222+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:31.659 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:31 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3[48445]: 2024-09-10T07:54:31.203+0000 7f3dd8a00640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:02.388563+0000 front 2024-09-10T07:54:02.388611+0000 (oldest deadline 2024-09-10T07:54:28.288729+0000) 2024-09-10T07:54:31.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:31 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:31.419+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:32.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:31.976+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:32.605 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:32 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:32.232+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:32.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:32 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:32.379+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:32.908 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:32 smithi046 podman[119895]: 2024-09-10 07:54:32.607050542 +0000 UTC m=+6.023750345 container died a5da179933af13c2f5ff77de4ce2b5a2165c33744d981d8e2bbfc50b9aa27e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T07:54:33.234 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:33 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:33.213+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:33.235 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:33 smithi046 podman[119895]: 2024-09-10 07:54:33.219226018 +0000 UTC m=+6.635925820 container cleanup a5da179933af13c2f5ff77de4ce2b5a2165c33744d981d8e2bbfc50b9aa27e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:54:33.235 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:33 smithi046 bash[119895]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3 2024-09-10T07:54:33.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:33.008+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:33.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:33 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:33.359+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:33.908 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:33 smithi046 podman[119926]: 2024-09-10 07:54:33.487704631 +0000 UTC m=+0.862725088 container remove a5da179933af13c2f5ff77de4ce2b5a2165c33744d981d8e2bbfc50b9aa27e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T07:54:34.368 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:34 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:34.228+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:34.368 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:34 smithi046 podman[120010]: 2024-09-10 07:54:33.967767136 +0000 UTC m=+0.027233175 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:54:34.368 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:34 smithi046 podman[120010]: 2024-09-10 07:54:34.071986653 +0000 UTC m=+0.131452691 container create 472eb7425a72240e14808beea252128b8f569d3e854afbd51f9714b8856e5353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T07:54:34.368 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:34 smithi046 podman[120010]: 2024-09-10 07:54:34.369501233 +0000 UTC m=+0.428967273 container init 472eb7425a72240e14808beea252128b8f569d3e854afbd51f9714b8856e5353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T07:54:34.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:33.958+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:34.633 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:34 smithi046 podman[120010]: 2024-09-10 07:54:34.376551242 +0000 UTC m=+0.436017285 container start 472eb7425a72240e14808beea252128b8f569d3e854afbd51f9714b8856e5353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T07:54:34.633 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:34 smithi046 podman[120010]: 2024-09-10 07:54:34.47464428 +0000 UTC m=+0.534110333 container attach 472eb7425a72240e14808beea252128b8f569d3e854afbd51f9714b8856e5353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T07:54:34.649 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:34 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:34.332+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:34.908 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 10 07:54:34 smithi046 podman[120010]: 2024-09-10 07:54:34.63495797 +0000 UTC m=+0.694424010 container died 472eb7425a72240e14808beea252128b8f569d3e854afbd51f9714b8856e5353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:54:35.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:34.944+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:35.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:35 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:35.208+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:35.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T07:54:35.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:35 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:35.329+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:35.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:54:35.675 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T07:54:35.675 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T07:54:35.675 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.4 2024-09-10T07:54:35.992 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:35 smithi063 systemd[1]: Stopping Ceph osd.4 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:54:36.297 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:35 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:35.981+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:36.305 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:36 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:36.157+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:36.306 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:35 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:54:35.993+0000 7f9e2d934640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:54:36.306 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:35 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:54:35.993+0000 7f9e2d934640 -1 osd.4 101 *** Got signal Terminated *** 2024-09-10T07:54:36.306 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:35 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:54:35.993+0000 7f9e2d934640 -1 osd.4 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:54:36.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:36 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:36.306+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:36.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:36 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:36.187+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:37.152 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:36 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:36.824+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:37.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:37.025+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:37.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:37 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:37.152+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:37.551 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:37.181+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:37.551 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:37.353+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:37.552 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:37.353+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:38.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:37 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:37.817+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:38.333 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:37 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:54:37.944+0000 7f9e29f4a640 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:14.418500+0000 front 2024-09-10T07:54:14.418342+0000 (oldest deadline 2024-09-10T07:54:37.318461+0000) 2024-09-10T07:54:38.333 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:38 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:38.181+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:38.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:38.028+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:38.399 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:38.061+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:38.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:38 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:38.334+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:38.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:38 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:38.334+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:38.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:38 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:38.189+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:38.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:38 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:38.189+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:39.152 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:38 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:38.840+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:39.290 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:38 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:54:38.983+0000 7f9e29f4a640 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:14.418500+0000 front 2024-09-10T07:54:14.418342+0000 (oldest deadline 2024-09-10T07:54:37.318461+0000) 2024-09-10T07:54:39.290 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:39 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:39.151+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:39.399 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:39.107+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:39.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:39.015+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:39.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:39 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:39.152+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:39.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:39 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:39.152+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:39.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:39 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:39.291+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:39.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:39 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:39.291+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:40.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:40 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:40.006+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:40.149 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:40 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:40.070+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:40.150 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:39.821+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:40.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:39 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:39.815+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:40.313 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:39 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:54:39.945+0000 7f9e29f4a640 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:14.418500+0000 front 2024-09-10T07:54:14.418342+0000 (oldest deadline 2024-09-10T07:54:37.318461+0000) 2024-09-10T07:54:40.313 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:40 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:40.178+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:40.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:40 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:40.313+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:40.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:40 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:40.313+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:40.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:40 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:40.181+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:40.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:40 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:40.181+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:41.149 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:41 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:41.023+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:41.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:40 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:40.776+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:41.150 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:40 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:40.977+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:41.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:40 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:40.779+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:41.354 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:40 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4[46353]: 2024-09-10T07:54:40.935+0000 7f9e29f4a640 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:14.418500+0000 front 2024-09-10T07:54:14.418342+0000 (oldest deadline 2024-09-10T07:54:37.318461+0000) 2024-09-10T07:54:41.354 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:41 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:41.154+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:41.650 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:41 smithi063 podman[71768]: 2024-09-10 07:54:41.579630409 +0000 UTC m=+5.702344692 container died dd3fe9485aaf284966c83b13a1028560ae938aef9fe7ac408097dd93c8f41fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T07:54:41.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:41 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:41.354+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:41.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:41 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:41.354+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:41.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:41 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:41.168+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:41.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:41 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:41.168+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:42.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:41 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:41.792+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:42.149 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:42 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:42.008+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:42.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:41 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:41.995+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:42.150 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:41 smithi063 podman[71768]: 2024-09-10 07:54:41.959956761 +0000 UTC m=+6.082671041 container cleanup dd3fe9485aaf284966c83b13a1028560ae938aef9fe7ac408097dd93c8f41fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T07:54:42.150 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:41 smithi063 bash[71768]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4 2024-09-10T07:54:42.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:41 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:41.733+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:42.519 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:42 smithi063 podman[71784]: 2024-09-10 07:54:42.211544071 +0000 UTC m=+0.628052378 container remove dd3fe9485aaf284966c83b13a1028560ae938aef9fe7ac408097dd93c8f41fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T07:54:42.520 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:42 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:42.351+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:42.520 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:42 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:42.351+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:42.520 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:42 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:42.184+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:42.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:42.214+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:42.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:42.214+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:42.772 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:42 smithi063 podman[71872]: 2024-09-10 07:54:42.520898237 +0000 UTC m=+0.112910986 container create e6bd0a4f9d3def6ef401e5257be83cd008d7d096212b70b5d534527814458762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:54:42.772 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:42 smithi063 podman[71872]: 2024-09-10 07:54:42.423697075 +0000 UTC m=+0.015709833 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:54:42.772 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:42 smithi063 podman[71872]: 2024-09-10 07:54:42.773663097 +0000 UTC m=+0.365675850 container init e6bd0a4f9d3def6ef401e5257be83cd008d7d096212b70b5d534527814458762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T07:54:43.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:42 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:42.751+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:43.149 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:42 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:42.999+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:43.150 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:42 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:42.956+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:43.150 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:42 smithi063 podman[71872]: 2024-09-10 07:54:42.777121944 +0000 UTC m=+0.369134696 container start e6bd0a4f9d3def6ef401e5257be83cd008d7d096212b70b5d534527814458762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T07:54:43.150 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:42 smithi063 podman[71872]: 2024-09-10 07:54:42.865330536 +0000 UTC m=+0.457343287 container attach e6bd0a4f9d3def6ef401e5257be83cd008d7d096212b70b5d534527814458762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T07:54:43.150 INFO:journalctl@ceph.osd.4.smithi063.stdout:Sep 10 07:54:42 smithi063 podman[71872]: 2024-09-10 07:54:42.967008998 +0000 UTC m=+0.559021747 container died e6bd0a4f9d3def6ef401e5257be83cd008d7d096212b70b5d534527814458762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T07:54:43.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:42.742+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:43.618 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:43 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:43.321+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:43.618 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:43 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:43.321+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:43.618 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:43 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:43.207+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:43.640 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T07:54:43.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:43.241+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:43.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:43.241+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:43.677 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:54:43.677 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T07:54:43.678 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T07:54:43.678 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.5 2024-09-10T07:54:43.981 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:43 smithi181 systemd[1]: Stopping Ceph osd.5 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:54:43.982 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:43 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:43.750+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:44.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:43 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:43.728+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:44.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:43 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:43.982+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:44.399 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:43 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:43.988+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:44.400 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:43 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:43.995+0000 7f5ab32e8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:54:44.400 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:43 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:43.995+0000 7f5ab32e8640 -1 osd.5 101 *** Got signal Terminated *** 2024-09-10T07:54:44.400 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:43 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:43.995+0000 7f5ab32e8640 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:54:44.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:44 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:44.188+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:44.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:44 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:44.370+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:44.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:44 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:44.370+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:44.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:44 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:44.284+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:44.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:44 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:44.284+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:45.026 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:44 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:44.959+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:45.026 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:44 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:44.959+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.477122+0000 front 2024-09-10T07:54:21.477170+0000 (oldest deadline 2024-09-10T07:54:44.377084+0000) 2024-09-10T07:54:45.027 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:44 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:44.756+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:45.027 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:44 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:44.756+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:45.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:44 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:44.735+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:45.398 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:45 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:45.027+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:45.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:45 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:45.215+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:45.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:45 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:45.364+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:45.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:45 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:45.364+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:45.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:45 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:45.364+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:45.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:45 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:45.280+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:45.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:45 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:45.280+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:46.068 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:45 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:45.720+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:46.068 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:45 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:45.720+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:46.068 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:45 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:45.918+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:46.068 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:45 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:45.918+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.477122+0000 front 2024-09-10T07:54:21.477170+0000 (oldest deadline 2024-09-10T07:54:44.377084+0000) 2024-09-10T07:54:46.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:45 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:45.782+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:46.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:46 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:46.069+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:46.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:46 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:46.233+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:46.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:46 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:46.233+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:46.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:46 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:46.346+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:46.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:46 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:46.346+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:46.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:46 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:46.346+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:46.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:46 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:46.294+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:46.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:46 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:46.294+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:46.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:46 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:46.294+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:47.108 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:46 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:46.911+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:47.108 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:46 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:46.911+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.477122+0000 front 2024-09-10T07:54:21.477170+0000 (oldest deadline 2024-09-10T07:54:44.377084+0000) 2024-09-10T07:54:47.108 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:46 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:46.725+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:47.108 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:46 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:46.725+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:47.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:46 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:46.821+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:47.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:46 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:46.821+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:47.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:47 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:47.109+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:47.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:47 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:47.242+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:47.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:47 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:47.242+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:47.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:47 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:47.367+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:47.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:47 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:47.367+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:47.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:47 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:47.367+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:47.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:47.293+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:47.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:47.293+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:47.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:47.293+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:48.104 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:47 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:47.890+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:48.104 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:47 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:47.890+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.477122+0000 front 2024-09-10T07:54:21.477170+0000 (oldest deadline 2024-09-10T07:54:44.377084+0000) 2024-09-10T07:54:48.104 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:47 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:47.684+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:48.104 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:47 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:47.684+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:48.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:47.776+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:48.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:47.776+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:48.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:48 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:48.105+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:48.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:48 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:48.105+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:48.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:48 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:48.237+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:48.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:48 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:48.237+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:48.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:48 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:48.346+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:48.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:48 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:48.346+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:48.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:48 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:48.346+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:48.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:48 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:48.279+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:48.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:48 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:48.279+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:48.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:48 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:48.279+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:48.919 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:48 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:48.660+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:48.919 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:48 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:48.660+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:49.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:48 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:48.747+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:49.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:48 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:48.747+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:49.399 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:48 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:48.919+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.975214+0000 front 2024-09-10T07:54:13.975287+0000 (oldest deadline 2024-09-10T07:54:37.376054+0000) 2024-09-10T07:54:49.399 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:48 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5[47264]: 2024-09-10T07:54:48.919+0000 7f5aaf8fe640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.477122+0000 front 2024-09-10T07:54:21.477170+0000 (oldest deadline 2024-09-10T07:54:44.377084+0000) 2024-09-10T07:54:49.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:49 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:49.145+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:49.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:49 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:49.145+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:49.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:49 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:49.223+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:49.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:49 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:49.223+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:49.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:49 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:49.302+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:49.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:49 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:49.302+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:49.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:49 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:49.302+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:49.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:49 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:49.328+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:49.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:49 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:49.328+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:49.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:49 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:49.328+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:49.828 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:49 smithi181 podman[72162]: 2024-09-10 07:54:49.522993196 +0000 UTC m=+5.651998760 container died ddce13f13cb0ba7851c940be1c8101208cf24b1b42219bd18b1527a6783c993f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:54:49.828 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:49 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:49.643+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:49.828 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:49 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:49.643+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:50.138 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:49 smithi181 podman[72162]: 2024-09-10 07:54:49.929009446 +0000 UTC m=+6.058014998 container cleanup ddce13f13cb0ba7851c940be1c8101208cf24b1b42219bd18b1527a6783c993f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T07:54:50.138 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:49 smithi181 bash[72162]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5 2024-09-10T07:54:50.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:49 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:49.752+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:50.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:49 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:49.752+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:50.399 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:50 smithi181 podman[72177]: 2024-09-10 07:54:50.205882293 +0000 UTC m=+0.682223483 container remove ddce13f13cb0ba7851c940be1c8101208cf24b1b42219bd18b1527a6783c993f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T07:54:50.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:50 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:50.137+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:50.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:50 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:50.137+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:50.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:50 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:50.272+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:50.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:50 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:50.272+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:50.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:50 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:50.294+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:50.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:50 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:50.294+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:50.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:50 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:50.294+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:50.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:50.295+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:50.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:50.295+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:50.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:50.295+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:50.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:50 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:50.640+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:50.792 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:50 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:50.640+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:50.792 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:50 smithi181 podman[72266]: 2024-09-10 07:54:50.438075824 +0000 UTC m=+0.015835002 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:54:50.792 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:50 smithi181 podman[72266]: 2024-09-10 07:54:50.548353258 +0000 UTC m=+0.126112441 container create 7519ba1d9805ad7cb8f6d30e2a6af68379b04380b56d7be5260a5f14022793fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T07:54:50.792 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:50 smithi181 podman[72266]: 2024-09-10 07:54:50.792867205 +0000 UTC m=+0.370626379 container init 7519ba1d9805ad7cb8f6d30e2a6af68379b04380b56d7be5260a5f14022793fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:54:51.149 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:50 smithi181 podman[72266]: 2024-09-10 07:54:50.796627167 +0000 UTC m=+0.374386335 container start 7519ba1d9805ad7cb8f6d30e2a6af68379b04380b56d7be5260a5f14022793fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True) 2024-09-10T07:54:51.149 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:50 smithi181 podman[72266]: 2024-09-10 07:54:50.909703095 +0000 UTC m=+0.487462285 container attach 7519ba1d9805ad7cb8f6d30e2a6af68379b04380b56d7be5260a5f14022793fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T07:54:51.149 INFO:journalctl@ceph.osd.5.smithi181.stdout:Sep 10 07:54:50 smithi181 podman[72266]: 2024-09-10 07:54:50.984141356 +0000 UTC m=+0.561900526 container died 7519ba1d9805ad7cb8f6d30e2a6af68379b04380b56d7be5260a5f14022793fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:54:51.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:50.801+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:51.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:50 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:50.801+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:51.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:51 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:51.183+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:51.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:51 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:51.183+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:51.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:51 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:51.275+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:51.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:51 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:51.275+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:51.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:51 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:51.290+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:51.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:51 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:51.290+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:51.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:51 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:51.290+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:51.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:51 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:51.277+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:51.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:51 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:51.277+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:51.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:51 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:51.277+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:51.712 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T07:54:51.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:54:51.754 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T07:54:51.754 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T07:54:51.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.6 2024-09-10T07:54:52.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:51 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:51.666+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:52.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:51 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:51.666+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:52.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:51 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:51.772+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:52.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:51 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:51.772+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:52.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:51 smithi046 systemd[1]: Stopping Ceph osd.6 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:54:52.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:52 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:52.183+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:52.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:52 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:52.183+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:52.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:52 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:52.314+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:52.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:52 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:52.314+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:52.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:52 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:52.316+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:52.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:52 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:52.316+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:52.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:52 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:52.316+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:52.658 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:52.335+0000 7faf73600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:54:52.658 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:52.335+0000 7faf73600640 -1 osd.6 101 *** Got signal Terminated *** 2024-09-10T07:54:52.658 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:52.335+0000 7faf73600640 -1 osd.6 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:54:52.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:52.318+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:52.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:52.318+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:52.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:52.318+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:53.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:52 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:52.671+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:53.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:52 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:52.671+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:53.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:52.764+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:53.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:52 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:52.764+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:53.635 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:53 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:53.199+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:53.635 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:53 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:53.199+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:53.649 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:53 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:53.328+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:53.649 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:53 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:53.328+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:53.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:53 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:53.347+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:53.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:53 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:53.347+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:53.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:53 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:53.347+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:53.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:53.324+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:53.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:53.324+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:53.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:53.324+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:53.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:53.324+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:54:53.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:53 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:53.635+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:53.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:53 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:53.635+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:53.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:53 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:53.635+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:54:54.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:53.730+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:54.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:53.730+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:54.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:53.730+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.570827+0000 front 2024-09-10T07:54:29.570957+0000 (oldest deadline 2024-09-10T07:54:53.670957+0000) 2024-09-10T07:54:54.605 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:54 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:54.237+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:54.605 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:54 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:54.237+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:54.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:54 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:54.355+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:54.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:54 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:54.355+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:54.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:54 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:54.396+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:54.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:54 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:54.396+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:54.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:54 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:54.396+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:54.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:54 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:54.343+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:54.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:54 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:54.343+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:54.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:54 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:54.343+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:54.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:54 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:54.343+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:54:54.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:54 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:54.606+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:54.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:54 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:54.606+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:54.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:54 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:54.606+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:54:55.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:54 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:54.718+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:55.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:54 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:54.718+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:55.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:54 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:54.718+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.570827+0000 front 2024-09-10T07:54:29.570957+0000 (oldest deadline 2024-09-10T07:54:53.670957+0000) 2024-09-10T07:54:55.636 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:55 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:55.194+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:55.636 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:55 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:55.194+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:55.649 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:55 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:55.346+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:55.649 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:55 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:55.347+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:55.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:55 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:55.414+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:55.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:55 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:55.414+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:55.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:55 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:55.414+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:55.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:55 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:55.314+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:55.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:55 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:55.314+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:55.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:55 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:55.314+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:55.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:55 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:55.314+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:54:55.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:55 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:55.637+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:55.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:55 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:55.637+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:55.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:55 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:55.637+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:54:56.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:55 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:55.748+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:56.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:55 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:55.748+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:56.159 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:55 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:55.748+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.570827+0000 front 2024-09-10T07:54:29.570957+0000 (oldest deadline 2024-09-10T07:54:53.670957+0000) 2024-09-10T07:54:56.647 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:56 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:56.199+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:56.648 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:56 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:56.199+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:56.649 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:56 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:56.367+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:56.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:56 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:56.367+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:56.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:56 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:56.367+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:54:56.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:56 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:56.453+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:56.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:56 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:56.453+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:56.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:56 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:56.453+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:56.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:56 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:56.453+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:54:56.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:56.348+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:56.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:56.348+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:56.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:56.348+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:56.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:56.348+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:54:56.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:56 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:56.647+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:56.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:56 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:56.647+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:56.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:56 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:56.647+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:54:57.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:56.703+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.967161+0000 front 2024-09-10T07:54:11.967104+0000 (oldest deadline 2024-09-10T07:54:36.067303+0000) 2024-09-10T07:54:57.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:56.703+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.969250+0000 front 2024-09-10T07:54:21.969549+0000 (oldest deadline 2024-09-10T07:54:46.669390+0000) 2024-09-10T07:54:57.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:56 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6[55262]: 2024-09-10T07:54:56.703+0000 7faf6a000640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.570827+0000 front 2024-09-10T07:54:29.570957+0000 (oldest deadline 2024-09-10T07:54:53.670957+0000) 2024-09-10T07:54:57.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:57 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:57.191+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:57.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:57 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:57.191+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:57.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:57 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:57.318+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:57.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:57 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:57.318+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:57.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:57 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:57.318+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:54:57.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:57 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:57.448+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:57.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:57 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:57.448+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:57.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:57 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:57.448+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:57.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:57 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:57.448+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:54:57.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:57 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:57.356+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:57.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:57 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:57.356+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:57.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:57 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:57.356+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:57.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:57 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:57.356+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:54:58.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:57 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:57.667+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:58.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:57 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:57.667+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:58.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:57 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:57.667+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:54:58.404 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:58 smithi046 podman[120106]: 2024-09-10 07:54:58.084055465 +0000 UTC m=+6.029645298 container died 2f224a950c690d909d65475d1b2a212ccf66e9d7e5b08f7c3734411ba0b3ed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T07:54:58.637 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:58 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:58.241+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:58.637 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:58 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:58.241+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:58.637 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:58 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:58.241+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:54:58.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:58 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:58.361+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:58.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:58 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:58.361+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:58.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:58 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:58.361+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:54:58.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:58 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:58.484+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:58.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:58 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:58.484+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:58.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:58 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:58.484+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:58.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:58 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:58.484+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:54:58.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:58 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:58.404+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:58.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:58 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:58.404+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:58.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:58 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:58.404+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:58.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:58 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:58.404+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:54:58.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:58 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:58.637+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:58.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:58 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:58.637+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:58.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:58 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:58.637+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:54:59.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:58 smithi046 podman[120106]: 2024-09-10 07:54:58.782366309 +0000 UTC m=+6.727956141 container cleanup 2f224a950c690d909d65475d1b2a212ccf66e9d7e5b08f7c3734411ba0b3ed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T07:54:59.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:58 smithi046 bash[120106]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6 2024-09-10T07:54:59.158 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:59 smithi046 podman[120125]: 2024-09-10 07:54:59.017315845 +0000 UTC m=+0.927935615 container remove 2f224a950c690d909d65475d1b2a212ccf66e9d7e5b08f7c3734411ba0b3ed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:54:59.648 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:59 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:59.267+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:54:59.648 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:59 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:59.267+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:54:59.648 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:54:59 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:54:59.267+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:54:59.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:59 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:59.328+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:54:59.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:59 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:59.328+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:54:59.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:54:59 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:54:59.328+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:54:59.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:59 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:59.449+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:54:59.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:59 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:59.449+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:54:59.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:59 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:59.449+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:54:59.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:54:59 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:54:59.449+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:54:59.711 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:59 smithi046 podman[120212]: 2024-09-10 07:54:59.460188085 +0000 UTC m=+0.027908190 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:54:59.712 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:59 smithi046 podman[120212]: 2024-09-10 07:54:59.592992919 +0000 UTC m=+0.160713023 container create b894e81cc6505d98c4465606294fa94ae08bdfe393e663620166e2daae8ab3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:54:59.712 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:59 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:59.433+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:54:59.712 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:59 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:59.433+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:54:59.712 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:59 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:59.433+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:54:59.712 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:54:59 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:54:59.433+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:54:59.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:59 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:59.648+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:54:59.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:59 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:59.648+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:54:59.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:54:59 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:54:59.648+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:00.257 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:59 smithi046 podman[120212]: 2024-09-10 07:54:59.987702296 +0000 UTC m=+0.555422401 container init b894e81cc6505d98c4465606294fa94ae08bdfe393e663620166e2daae8ab3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:55:00.257 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:54:59 smithi046 podman[120212]: 2024-09-10 07:54:59.994554584 +0000 UTC m=+0.562274698 container start b894e81cc6505d98c4465606294fa94ae08bdfe393e663620166e2daae8ab3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:55:00.257 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:55:00 smithi046 podman[120212]: 2024-09-10 07:55:00.078952189 +0000 UTC m=+0.646672303 container attach b894e81cc6505d98c4465606294fa94ae08bdfe393e663620166e2daae8ab3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T07:55:00.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:00 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:00.235+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:00.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:00 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:00.235+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:00.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:00 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:00.235+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:00.649 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:00 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:00.311+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:55:00.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:00 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:00.311+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:55:00.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:00 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:00.311+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:55:00.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:00 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:00.455+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:00.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:00 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:00.455+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:00.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:00 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:00.455+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:00.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:00 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:00.455+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:00.660 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 10 07:55:00 smithi046 podman[120212]: 2024-09-10 07:55:00.258473646 +0000 UTC m=+0.826193760 container died b894e81cc6505d98c4465606294fa94ae08bdfe393e663620166e2daae8ab3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:55:00.661 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:00 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:00.386+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:00.661 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:00 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:00.386+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:00.661 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:00 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:00.386+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:00.661 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:00 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:00.386+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:01.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:00 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:00.671+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:01.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:00 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:00.671+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:01.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:00 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:00.671+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:01.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T07:55:01.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:01.459 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T07:55:01.459 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T07:55:01.459 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.7 2024-09-10T07:55:01.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:01 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:01.194+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:01.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:01 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:01.194+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:01.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:01 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:01.194+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:01.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:01.285+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:55:01.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:01.285+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:55:01.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:01.285+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:55:01.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:01 smithi063 systemd[1]: Stopping Ceph osd.7 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:01.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:01.477+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:01.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:01.477+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:01.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:01.477+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:01.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:01.477+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:01.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:01 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:01.343+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:01.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:01 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:01.343+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:01.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:01 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:01.343+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:01.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:01 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:01.343+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:02.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:01 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:01.660+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:02.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:01 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:01.660+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:02.149 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:01 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:01.660+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:02.150 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:01.771+0000 7ff2a177c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:02.150 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:01.771+0000 7ff2a177c640 -1 osd.7 101 *** Got signal Terminated *** 2024-09-10T07:55:02.150 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:01 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:01.771+0000 7ff2a177c640 -1 osd.7 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:55:02.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:02 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:02.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:02.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:02 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:02.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:02.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:02 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:02.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:02.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:02 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:02.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:02.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:02.317+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:55:02.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:02.317+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:55:02.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:02.317+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:55:02.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:02.317+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.494291+0000 front 2024-09-10T07:54:40.494257+0000 (oldest deadline 2024-09-10T07:55:02.194362+0000) 2024-09-10T07:55:02.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:02.443+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:02.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:02.443+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:02.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:02.443+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:02.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:02 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:02.443+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:02.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:02 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:02.304+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:02.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:02 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:02.304+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:02.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:02 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:02.304+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:02.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:02 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:02.304+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:03.102 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:02 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:02.681+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:03.102 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:02 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:02.681+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:03.102 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:02 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:02.681+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:03.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:03 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:03.103+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:03.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:03 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:03.103+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:03.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:03 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:03.103+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:03.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:03 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:03.103+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:03.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:03.280+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:55:03.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:03.280+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:55:03.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:03.280+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:55:03.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:03.280+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.494291+0000 front 2024-09-10T07:54:40.494257+0000 (oldest deadline 2024-09-10T07:55:02.194362+0000) 2024-09-10T07:55:03.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:03.479+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:03.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:03.479+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:03.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:03.479+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:03.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:03.479+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:03.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:03 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:03.258+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:03.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:03 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:03.258+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:03.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:03 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:03.258+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:03.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:03 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:03.258+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:03.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:03 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:03.258+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:04.116 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:03 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:03.653+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:04.116 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:03 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:03.653+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:04.117 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:03 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:03.653+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:04.117 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:03 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:03.653+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:04.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:04.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:04.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:04.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:04.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:04.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:04.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:04.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:04.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:04 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:04.292+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:55:04.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:04 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:04.292+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:55:04.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:04 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:04.292+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:55:04.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:04 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:04.292+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.494291+0000 front 2024-09-10T07:54:40.494257+0000 (oldest deadline 2024-09-10T07:55:02.194362+0000) 2024-09-10T07:55:04.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:04 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:04.509+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:04.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:04 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:04.509+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:04.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:04 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:04.509+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:04.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:04 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:04.509+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:04.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:04 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:04.256+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:04.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:04 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:04.256+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:04.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:04 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:04.256+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:04.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:04 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:04.256+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:04.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:04 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:04.256+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:04.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:04.618+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:04.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:04.618+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:04.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:04.618+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:04.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:04 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:04.618+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:05.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:05 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:05.141+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:05.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:05 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:05.141+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:05.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:05 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:05.141+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:05.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:05 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:05.141+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:05.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:05 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:05.309+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:55:05.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:05 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:05.309+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:55:05.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:05 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:05.309+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:55:05.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:05 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:05.309+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.494291+0000 front 2024-09-10T07:54:40.494257+0000 (oldest deadline 2024-09-10T07:55:02.194362+0000) 2024-09-10T07:55:05.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:05 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:05.478+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:05.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:05 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:05.478+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:05.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:05 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:05.478+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:05.650 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:05 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:05.478+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:05.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:05 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:05.237+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:05.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:05 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:05.237+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:05.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:05 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:05.237+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:05.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:05 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:05.237+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:05.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:05 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:05.237+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:05.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:05 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:05.572+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:05.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:05 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:05.572+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:05.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:05 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:05.572+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:05.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:05 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:05.572+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:06.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:06 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:06.106+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:06.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:06 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:06.106+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:06.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:06 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:06.106+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:06.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:06 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:06.106+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:06.650 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:06.349+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.389209+0000 front 2024-09-10T07:54:11.389252+0000 (oldest deadline 2024-09-10T07:54:36.089461+0000) 2024-09-10T07:55:06.651 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:06.349+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:21.891044+0000 front 2024-09-10T07:54:21.891119+0000 (oldest deadline 2024-09-10T07:54:45.391086+0000) 2024-09-10T07:55:06.651 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:06.349+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:30.592980+0000 front 2024-09-10T07:54:30.592868+0000 (oldest deadline 2024-09-10T07:54:55.892821+0000) 2024-09-10T07:55:06.651 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7[53276]: 2024-09-10T07:55:06.349+0000 7ff29dd92640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.494291+0000 front 2024-09-10T07:54:40.494257+0000 (oldest deadline 2024-09-10T07:55:02.194362+0000) 2024-09-10T07:55:06.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:06.520+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:06.651 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:06.520+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:06.652 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:06.520+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:06.652 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:06.520+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:06.652 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:06 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:06.520+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:06.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:06.221+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:06.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:06.221+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:06.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:06.221+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:06.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:06.221+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:06.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:06 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:06.221+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:06.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:06 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:06.544+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:06.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:06 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:06.544+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:06.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:06 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:06.544+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:06.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:06 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:06.544+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:07.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:07.126+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:07.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:07.126+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:07.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:07.126+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:07.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:07.126+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:07.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:07.184+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:07.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:07.184+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:07.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:07.184+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:07.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:07.184+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:07.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:07 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:07.184+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:07.738 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:07 smithi063 podman[71975]: 2024-09-10 07:55:07.440200457 +0000 UTC m=+5.779862285 container died e283876c6269abc1502a8c0a4939bd58dc5625d2d70b06f61ef893878f0c622c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T07:55:07.738 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:07 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:07.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:07.738 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:07 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:07.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:07.738 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:07 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:07.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:07.738 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:07 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:07.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:07.738 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:07 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:07.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:07.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:07.551+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:07.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:07.551+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:07.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:07.551+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:07.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:07 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:07.551+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:08.080 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:07 smithi063 podman[71975]: 2024-09-10 07:55:07.830141493 +0000 UTC m=+6.169803320 container cleanup e283876c6269abc1502a8c0a4939bd58dc5625d2d70b06f61ef893878f0c622c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T07:55:08.080 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:07 smithi063 bash[71975]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7 2024-09-10T07:55:08.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:08 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:08.079+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:08.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:08 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:08.079+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:08.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:08 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:08.079+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:08.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:08 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:08.079+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:08.400 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:08 smithi063 podman[71990]: 2024-09-10 07:55:08.081548157 +0000 UTC m=+0.639808665 container remove e283876c6269abc1502a8c0a4939bd58dc5625d2d70b06f61ef893878f0c622c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:55:08.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:08 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:08.194+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:08.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:08 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:08.194+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:08.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:08 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:08.194+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:08.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:08 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:08.194+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:08.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:08 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:08.194+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:08.734 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:08 smithi063 podman[72087]: 2024-09-10 07:55:08.407502505 +0000 UTC m=+0.116152738 container create 488ad96f75f848d99e140f26b84c821bf263a97dd855812ffbb5e9e22300e089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T07:55:08.734 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:08 smithi063 podman[72087]: 2024-09-10 07:55:08.310776685 +0000 UTC m=+0.019426918 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:55:08.734 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:08 smithi063 podman[72087]: 2024-09-10 07:55:08.64357779 +0000 UTC m=+0.352228038 container init 488ad96f75f848d99e140f26b84c821bf263a97dd855812ffbb5e9e22300e089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T07:55:08.734 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:08 smithi063 podman[72087]: 2024-09-10 07:55:08.64684316 +0000 UTC m=+0.355493399 container start 488ad96f75f848d99e140f26b84c821bf263a97dd855812ffbb5e9e22300e089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T07:55:08.734 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:08 smithi063 podman[72087]: 2024-09-10 07:55:08.735251638 +0000 UTC m=+0.443901875 container attach 488ad96f75f848d99e140f26b84c821bf263a97dd855812ffbb5e9e22300e089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T07:55:08.734 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:08 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:08.505+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:08.735 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:08 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:08.505+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:08.735 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:08 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:08.505+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:08.735 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:08 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:08.505+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:08.735 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:08 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:08.505+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:08.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:08 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:08.576+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:08.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:08 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:08.576+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:08.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:08 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:08.576+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:08.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:08 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:08.576+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:08.988 INFO:journalctl@ceph.osd.7.smithi063.stdout:Sep 10 07:55:08 smithi063 podman[72087]: 2024-09-10 07:55:08.851366521 +0000 UTC m=+0.560016754 container died 488ad96f75f848d99e140f26b84c821bf263a97dd855812ffbb5e9e22300e089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T07:55:09.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:09.040+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:09.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:09.040+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:09.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:09.040+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:09.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:09.040+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:09.426 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T07:55:09.466 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:09.467 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T07:55:09.467 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T07:55:09.467 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.8 2024-09-10T07:55:09.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:09 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:09.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:09.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:09 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:09.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:09.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:09 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:09.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:09.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:09 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:09.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:09.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:09 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:09.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:09.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:09 smithi181 systemd[1]: Stopping Ceph osd.8 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:09.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:09.615+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:09.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:09.615+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:09.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:09.615+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:09.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:09.615+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:09.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:09.791+0000 7f92685e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:09.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:09.791+0000 7f92685e4640 -1 osd.8 101 *** Got signal Terminated *** 2024-09-10T07:55:09.791 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:09 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:09.791+0000 7f92685e4640 -1 osd.8 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:55:09.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:09 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:09.549+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:09.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:09 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:09.549+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:09.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:09 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:09.549+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:09.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:09 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:09.549+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:09.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:09 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:09.549+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:10.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:10 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:10.010+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:10.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:10 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:10.010+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:10.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:10 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:10.010+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:10.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:10 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:10.010+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:10.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:10.149+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:10.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:10.149+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:10.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:10.149+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:10.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:10.149+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:10.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:10 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:10.149+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:10.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:10 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:10.580+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:10.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:10 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:10.580+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:10.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:10 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:10.580+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:10.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:10 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:10.580+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:10.899 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:10.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:10.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:10.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:10.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:10.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:10.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:10.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:10.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:10 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:10.531+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:11.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:11.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:11.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:11.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:11.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:11.102+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:11.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:11.102+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:11.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:11.102+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:11.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:11.102+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:11.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:11 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:11.102+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:11.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:11.610+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:11.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:11.610+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:11.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:11.610+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:11.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:11.610+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:11.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:11.517+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:11.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:11.517+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:11.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:11.517+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:11.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:11.517+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:11.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:11.517+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:11.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:11.517+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:12.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.982+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:12.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.982+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:12.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.982+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:12.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.982+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:12.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:11 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:11.982+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:12.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:12 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:12.129+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:12.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:12 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:12.129+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:12.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:12 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:12.129+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:12.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:12 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:12.129+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:12.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:12 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:12.129+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:12.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:12 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:12.129+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:12.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:12 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:12.628+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:12.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:12 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:12.628+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:12.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:12 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:12.628+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:12.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:12 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:12.628+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:12.899 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:12 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:12.469+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:12.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:12 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:12.469+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:12.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:12 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:12.469+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:12.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:12 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:12.469+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:12.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:12 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:12.469+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:12.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:12 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:12.469+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:13.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:13.006+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:13.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:13.006+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:13.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:13.006+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:13.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:13.006+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:13.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:13.006+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:13.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:13 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:13.160+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:13.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:13 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:13.160+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:13.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:13 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:13.160+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:13.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:13 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:13.160+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:13.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:13 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:13.160+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:13.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:13 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:13.160+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:13.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:13.619+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:13.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:13.619+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:13.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:13.619+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:13.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:13 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:13.619+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:13.899 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:13 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:13.472+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:13.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:13 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:13.472+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:13.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:13 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:13.472+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:13.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:13 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:13.472+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:13.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:13 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:13.472+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:13.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:13 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:13.472+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:14.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:14.020+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:14.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:14.020+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:14.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:14.020+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:14.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:14.020+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:14.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:14.020+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:14.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:14.172+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:14.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:14.172+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:14.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:14.172+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:14.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:14.172+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:14.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:14.172+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:14.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:14 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:14.172+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:14.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:14.604+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:13.629373+0000 front 2024-09-10T07:54:13.629283+0000 (oldest deadline 2024-09-10T07:54:39.529367+0000) 2024-09-10T07:55:14.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:14.604+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:19.529859+0000 front 2024-09-10T07:54:19.529842+0000 (oldest deadline 2024-09-10T07:54:44.230034+0000) 2024-09-10T07:55:14.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:14.604+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.732653+0000 front 2024-09-10T07:54:31.732780+0000 (oldest deadline 2024-09-10T07:54:52.832365+0000) 2024-09-10T07:55:14.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:14.604+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:39.233974+0000 front 2024-09-10T07:54:39.233887+0000 (oldest deadline 2024-09-10T07:55:03.333836+0000) 2024-09-10T07:55:14.899 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:14 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8[53879]: 2024-09-10T07:55:14.604+0000 7f9264bfa640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.634913+0000 front 2024-09-10T07:54:48.634889+0000 (oldest deadline 2024-09-10T07:55:13.935091+0000) 2024-09-10T07:55:14.899 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:14 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:14.476+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:14.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:14 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:14.476+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:14.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:14 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:14.476+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:14.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:14 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:14.476+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:14.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:14 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:14.476+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:14.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:14 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:14.476+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:15.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.005+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:15.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.005+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:15.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.005+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:15.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.005+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:15.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.005+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:15.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:15 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:15.142+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:15.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:15 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:15.142+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:15.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:15 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:15.142+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:15.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:15 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:15.142+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:15.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:15 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:15.142+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:15.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:15 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:15.142+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:15.724 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:15 smithi181 podman[72367]: 2024-09-10 07:55:15.420526614 +0000 UTC m=+5.757449278 container died 41b2fc7a6230a2da26ff243af91f74736596c8721755e4b30697b79d9fc57983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:55:15.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:15 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:15.502+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:15.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:15 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:15.502+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:15.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:15 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:15.502+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:15.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:15 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:15.502+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:15.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:15 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:15.502+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:15.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:15 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:15.502+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:15.988 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:15 smithi181 podman[72367]: 2024-09-10 07:55:15.832995685 +0000 UTC m=+6.169918347 container cleanup 41b2fc7a6230a2da26ff243af91f74736596c8721755e4b30697b79d9fc57983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T07:55:15.988 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:15 smithi181 bash[72367]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8 2024-09-10T07:55:16.399 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:16 smithi181 podman[72382]: 2024-09-10 07:55:16.084556832 +0000 UTC m=+0.667508006 container remove 41b2fc7a6230a2da26ff243af91f74736596c8721755e4b30697b79d9fc57983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T07:55:16.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.988+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:16.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.988+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:16.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.988+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:16.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.988+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:16.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:15 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:15.988+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:16.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:16 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:16.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:16.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:16 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:16.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:16.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:16 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:16.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:16.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:16 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:16.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:16.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:16 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:16.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:16.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:16 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:16.174+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:16.770 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:16 smithi181 podman[72473]: 2024-09-10 07:55:16.401905975 +0000 UTC m=+0.015585180 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:55:16.770 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:16 smithi181 podman[72473]: 2024-09-10 07:55:16.535453946 +0000 UTC m=+0.149133142 container create 0e80f514d648ae4d395e282aa8262414ecbd9a3537cc43e01ac68a0413a86206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T07:55:16.770 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:16 smithi181 podman[72473]: 2024-09-10 07:55:16.771386426 +0000 UTC m=+0.385065623 container init 0e80f514d648ae4d395e282aa8262414ecbd9a3537cc43e01ac68a0413a86206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T07:55:16.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:16 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:16.489+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:16.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:16 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:16.489+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:16.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:16 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:16.489+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:16.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:16 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:16.489+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:16.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:16 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:16.489+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:16.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:16 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:16.489+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:17.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.001+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:17.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.001+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:17.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.001+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:17.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.001+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:17.150 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.001+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:17.150 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:16 smithi181 podman[72473]: 2024-09-10 07:55:16.774820975 +0000 UTC m=+0.388500178 container start 0e80f514d648ae4d395e282aa8262414ecbd9a3537cc43e01ac68a0413a86206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:55:17.150 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:16 smithi181 podman[72473]: 2024-09-10 07:55:16.879963584 +0000 UTC m=+0.493642782 container attach 0e80f514d648ae4d395e282aa8262414ecbd9a3537cc43e01ac68a0413a86206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T07:55:17.150 INFO:journalctl@ceph.osd.8.smithi181.stdout:Sep 10 07:55:16 smithi181 podman[72473]: 2024-09-10 07:55:16.960680086 +0000 UTC m=+0.574359284 container died 0e80f514d648ae4d395e282aa8262414ecbd9a3537cc43e01ac68a0413a86206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T07:55:17.632 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T07:55:17.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:17 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:17.176+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:17.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:17 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:17.176+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:17.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:17 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:17.176+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:17.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:17 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:17.176+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:17.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:17 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:17.176+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:17.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:17 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:17.176+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:17.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:17.672 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T07:55:17.672 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T07:55:17.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.9 2024-09-10T07:55:17.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:17.490+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:17.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:17.490+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:17.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:17.490+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:17.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:17.490+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:17.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:17.490+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:17.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:17 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:17.490+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:18.126 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:17 smithi046 systemd[1]: Stopping Ceph osd.9 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:18.127 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.127+0000 7ff0a3800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:18.127 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.127+0000 7ff0a3800640 -1 osd.9 101 *** Got signal Terminated *** 2024-09-10T07:55:18.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.989+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:18.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.989+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:18.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.989+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:18.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.989+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:18.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:17 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:17.989+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:18.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.127+0000 7ff0a3800640 -1 osd.9 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:55:18.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.216+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:18.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.216+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:18.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.216+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:18.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.216+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:18.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.216+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:18.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:18 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:18.216+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:18.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:18.536+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:18.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:18.536+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:18.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:18.536+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:18.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:18.536+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:18.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:18.536+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:18.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:18 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:18.536+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:19.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:18.948+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:19.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:18.948+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:19.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:18.948+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:19.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:18.948+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:19.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:18 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:18.948+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:19.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:19 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:19.193+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:19.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:19 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:19.193+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:19.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:19 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:19.193+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:19.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:19 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:19.193+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:19.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:19 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:19.193+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:19.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:19 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:19.193+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:19.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:19 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:19.535+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:19.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:19 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:19.535+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:19.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:19 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:19.535+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:19.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:19 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:19.535+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:19.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:19 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:19.535+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:19.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:19 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:19.535+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:20.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:19.950+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:20.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:19.950+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:20.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:19.950+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:20.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:19.950+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:20.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:19.950+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:20.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:20 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:20.182+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:20.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:20 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:20.182+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:20.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:20 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:20.182+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:20.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:20 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:20.182+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:20.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:20 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:20.182+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:20.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:20 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:20.182+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:20.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:20.578+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:20.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:20.578+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:20.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:20.578+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:20.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:20.578+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:20.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:20.578+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:20.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:20.578+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:20.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:20 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:20.578+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:21.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:20 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:20.975+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:21.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:20 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:20.975+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:21.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:20 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:20.975+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:21.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:20 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:20.975+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:21.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:20 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:20.975+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:21.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:21 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:21.140+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:21.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:21 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:21.140+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:21.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:21 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:21.140+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:21.408 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:21 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:21.140+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:21.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:21 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:21.140+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:21.409 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:21 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:21.140+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:21.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:21.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:21.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:21.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:21.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:21.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:21.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:21.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:21.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:21.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:21.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:21.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:21.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:21 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:21.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:22.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:22 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:21.999+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:22.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:22 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:21.999+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:22.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:22 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:21.999+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:22.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:22 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:21.999+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:22.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:22 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:21.999+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:22.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:22 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:21.999+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:22.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:22 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:22.164+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:01.916439+0000 front 2024-09-10T07:54:01.916320+0000 (oldest deadline 2024-09-10T07:54:27.816534+0000) 2024-09-10T07:55:22.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:22 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:22.164+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.917661+0000 front 2024-09-10T07:54:11.917613+0000 (oldest deadline 2024-09-10T07:54:37.817637+0000) 2024-09-10T07:55:22.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:22 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:22.164+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:17.818469+0000 front 2024-09-10T07:54:17.818382+0000 (oldest deadline 2024-09-10T07:54:46.018561+0000) 2024-09-10T07:55:22.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:22 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:22.164+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.220586+0000 front 2024-09-10T07:54:31.220553+0000 (oldest deadline 2024-09-10T07:54:52.920798+0000) 2024-09-10T07:55:22.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:22 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:22.164+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.222099+0000 front 2024-09-10T07:54:38.222066+0000 (oldest deadline 2024-09-10T07:55:02.321823+0000) 2024-09-10T07:55:22.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:22 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9[61961]: 2024-09-10T07:55:22.164+0000 7ff09a200640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:48.223235+0000 front 2024-09-10T07:54:48.223202+0000 (oldest deadline 2024-09-10T07:55:11.122985+0000) 2024-09-10T07:55:22.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:22 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:22.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:22.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:22 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:22.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:22.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:22 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:22.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:22.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:22 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:22.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:22.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:22 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:22.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:22.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:22 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:22.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:22.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:22 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:22.621+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:23.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:23 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:23.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:23.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:23 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:23.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:23.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:23 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:23.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:23.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:23 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:23.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:23.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:23 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:23.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:23.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:23 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:23.030+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:23.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:23 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:23.584+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:23.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:23 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:23.584+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:23.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:23 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:23.584+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:23.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:23 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:23.584+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:23.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:23 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:23.584+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:23.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:23 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:23.584+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:23.901 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:23 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:23.584+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:24.318 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:24 smithi046 podman[120312]: 2024-09-10 07:55:24.018911184 +0000 UTC m=+6.053018512 container died 96d2a24d9ae0dc4676dc88cbb1343cc1fdf3f7e1d49f13ef9d846ad5a7a95c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T07:55:24.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:24 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:24.048+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:24.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:24 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:24.048+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:24.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:24 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:24.048+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:24.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:24 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:24.048+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:24.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:24 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:24.048+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:24.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:24 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:24.048+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:24.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:24 smithi046 podman[120312]: 2024-09-10 07:55:24.535645772 +0000 UTC m=+6.569753099 container cleanup 96d2a24d9ae0dc4676dc88cbb1343cc1fdf3f7e1d49f13ef9d846ad5a7a95c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T07:55:24.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:24 smithi046 bash[120312]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9 2024-09-10T07:55:24.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:24 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:24.634+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:24.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:24 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:24.634+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:24.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:24 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:24.634+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:24.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:24 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:24.634+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:24.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:24 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:24.634+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:24.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:24 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:24.634+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:24.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:24 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:24.634+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:25.158 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:24 smithi046 podman[120329]: 2024-09-10 07:55:24.81217696 +0000 UTC m=+0.787812798 container remove 96d2a24d9ae0dc4676dc88cbb1343cc1fdf3f7e1d49f13ef9d846ad5a7a95c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T07:55:25.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:25 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:25.077+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:25.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:25 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:25.077+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:25.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:25 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:25.077+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:25.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:25 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:25.077+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:25.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:25 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:25.077+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:25.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:25 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:25.077+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:25.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:25 smithi046 podman[120417]: 2024-09-10 07:55:25.261980601 +0000 UTC m=+0.027465367 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:55:25.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:25 smithi046 podman[120417]: 2024-09-10 07:55:25.363062732 +0000 UTC m=+0.128547499 container create 2e521a12f8e728bdbb506c9719be3e09b2c04f31bea3b980252f672c62ae1b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T07:55:25.658 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:25 smithi046 podman[120417]: 2024-09-10 07:55:25.590696945 +0000 UTC m=+0.356181710 container init 2e521a12f8e728bdbb506c9719be3e09b2c04f31bea3b980252f672c62ae1b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T07:55:25.659 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:25 smithi046 podman[120417]: 2024-09-10 07:55:25.59736005 +0000 UTC m=+0.362844824 container start 2e521a12f8e728bdbb506c9719be3e09b2c04f31bea3b980252f672c62ae1b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T07:55:25.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:25 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:25.644+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:25.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:25 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:25.644+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:25.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:25 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:25.644+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:25.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:25 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:25.644+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:25.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:25 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:25.644+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:25.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:25 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:25.644+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:25.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:25 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:25.644+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:26.158 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:25 smithi046 podman[120417]: 2024-09-10 07:55:25.681905653 +0000 UTC m=+0.447390435 container attach 2e521a12f8e728bdbb506c9719be3e09b2c04f31bea3b980252f672c62ae1b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T07:55:26.158 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 10 07:55:25 smithi046 podman[120417]: 2024-09-10 07:55:25.86472625 +0000 UTC m=+0.630211016 container died 2e521a12f8e728bdbb506c9719be3e09b2c04f31bea3b980252f672c62ae1b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T07:55:26.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:26 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:26.039+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:26.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:26 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:26.039+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:26.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:26 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:26.039+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:26.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:26 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:26.039+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:26.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:26 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:26.039+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:26.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:26 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:26.039+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:26.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T07:55:26.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:26.746 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T07:55:26.746 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T07:55:26.746 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.10 2024-09-10T07:55:26.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:26 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:26.641+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:26.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:26 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:26.641+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:26.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:26 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:26.641+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:26.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:26 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:26.641+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:26.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:26 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:26.641+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:26.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:26 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:26.641+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:26.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:26 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:26.641+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:26.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:26 smithi063 systemd[1]: Stopping Ceph osd.10 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:27.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:27 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:27.034+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:27.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:27 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:27.034+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:27.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:27 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:27.034+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:27.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:27 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:27.034+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:27.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:27 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:27.034+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:27.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:27 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:27.034+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:27.400 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.066+0000 7f2c32880640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:27.400 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.066+0000 7f2c32880640 -1 osd.10 101 *** Got signal Terminated *** 2024-09-10T07:55:27.400 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.066+0000 7f2c32880640 -1 osd.10 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:55:27.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.602+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:27.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.602+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:27.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.602+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:27.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.602+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:27.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.602+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:27.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.602+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:27.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:27 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:27.602+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:28.350 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:28 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:28.047+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:28.350 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:28 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:28.047+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:28.351 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:28 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:28.047+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:28.351 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:28 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:28.047+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:28.351 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:28 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:28.047+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:28.351 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:28 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:28.047+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:28.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:28.577+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:28.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:28.577+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:28.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:28.577+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:28.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:28.577+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:28.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:28.577+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:28.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:28.577+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:28.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:28 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:28.577+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:29.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:29.090+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:29.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:29.090+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:29.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:29.090+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:29.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:29.090+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:29.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:29.090+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:29.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:29.090+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:29.400 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:29 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:29.090+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:29.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:29 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:29.605+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:29.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:29 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:29.605+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:29.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:29 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:29.605+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:29.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:29 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:29.605+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:29.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:29 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:29.605+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:29.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:29 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:29.605+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:29.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:29 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:29.605+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:30.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:30 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:30.100+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:30.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:30 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:30.100+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:30.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:30 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:30.100+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:30.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:30 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:30.100+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:30.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:30 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:30.100+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:30.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:30 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:30.100+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:30.400 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:30 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:30.100+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:30.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:30 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:30.629+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:30.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:30 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:30.629+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:30.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:30 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:30.629+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:30.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:30 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:30.629+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:30.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:30 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:30.629+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:30.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:30 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:30.629+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:30.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:30 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:30.629+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:31.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:31.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:31.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:31.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:31.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:31.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:31.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:31.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:31.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:31.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:31.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:31.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:31.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:31.117+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:32.150 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:31 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:31.665+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.864111+0000 front 2024-09-10T07:54:05.864297+0000 (oldest deadline 2024-09-10T07:54:31.164147+0000) 2024-09-10T07:55:32.150 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:31 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:31.665+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6804 osd.1 since back 2024-09-10T07:54:11.164507+0000 front 2024-09-10T07:54:11.164567+0000 (oldest deadline 2024-09-10T07:54:37.064728+0000) 2024-09-10T07:55:32.150 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:31 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:31.665+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:23.366635+0000 front 2024-09-10T07:54:23.366659+0000 (oldest deadline 2024-09-10T07:54:45.066735+0000) 2024-09-10T07:55:32.150 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:31 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:31.665+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:29.668290+0000 front 2024-09-10T07:54:29.668310+0000 (oldest deadline 2024-09-10T07:54:55.568339+0000) 2024-09-10T07:55:32.150 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:31 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:31.665+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:40.869400+0000 front 2024-09-10T07:54:40.869418+0000 (oldest deadline 2024-09-10T07:55:06.169438+0000) 2024-09-10T07:55:32.150 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:31 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:31.665+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:46.169892+0000 front 2024-09-10T07:54:46.169928+0000 (oldest deadline 2024-09-10T07:55:11.469822+0000) 2024-09-10T07:55:32.150 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:31 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10[59832]: 2024-09-10T07:55:31.665+0000 7f2c2ee96640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:57.271736+0000 front 2024-09-10T07:54:57.271689+0000 (oldest deadline 2024-09-10T07:55:19.571479+0000) 2024-09-10T07:55:32.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:32 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:32.146+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:32.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:32 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:32.146+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:32.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:32 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:32.146+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:32.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:32 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:32.146+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:32.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:32 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:32.146+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:32.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:32 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:32.146+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:32.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:32 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:32.146+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:32.900 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:32 smithi063 podman[72194]: 2024-09-10 07:55:32.626303137 +0000 UTC m=+5.677630354 container died cd64ee8dcc5513e672cbf7eded374bbb876a53dcabe3286316d6493bb68b37cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T07:55:33.400 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:32 smithi063 podman[72194]: 2024-09-10 07:55:32.916441795 +0000 UTC m=+5.967768999 container cleanup cd64ee8dcc5513e672cbf7eded374bbb876a53dcabe3286316d6493bb68b37cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:55:33.400 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:32 smithi063 bash[72194]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10 2024-09-10T07:55:33.400 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:33 smithi063 podman[72209]: 2024-09-10 07:55:33.126162863 +0000 UTC m=+0.494546255 container remove cd64ee8dcc5513e672cbf7eded374bbb876a53dcabe3286316d6493bb68b37cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T07:55:33.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:33.164+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:33.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:33.164+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:33.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:33.164+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:33.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:33.164+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:33.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:33.164+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:33.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:33.164+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:33.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:33 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:33.164+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:33.786 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:33 smithi063 podman[72298]: 2024-09-10 07:55:33.451805718 +0000 UTC m=+0.110744311 container create 6ca80330f2e98aad84d47e7d9847407009f7f5345047a753637a2007b19baa21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:55:33.786 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:33 smithi063 podman[72298]: 2024-09-10 07:55:33.357039953 +0000 UTC m=+0.015978518 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:55:33.786 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:33 smithi063 podman[72298]: 2024-09-10 07:55:33.679100764 +0000 UTC m=+0.338039322 container init 6ca80330f2e98aad84d47e7d9847407009f7f5345047a753637a2007b19baa21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:55:33.786 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:33 smithi063 podman[72298]: 2024-09-10 07:55:33.682427189 +0000 UTC m=+0.341365741 container start 6ca80330f2e98aad84d47e7d9847407009f7f5345047a753637a2007b19baa21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T07:55:33.787 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:33 smithi063 podman[72298]: 2024-09-10 07:55:33.787491041 +0000 UTC m=+0.446429599 container attach 6ca80330f2e98aad84d47e7d9847407009f7f5345047a753637a2007b19baa21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T07:55:34.150 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:33 smithi063 podman[72298]: 2024-09-10 07:55:33.872524005 +0000 UTC m=+0.531462562 container died 6ca80330f2e98aad84d47e7d9847407009f7f5345047a753637a2007b19baa21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:55:34.424 INFO:journalctl@ceph.osd.10.smithi063.stdout:Sep 10 07:55:34 smithi063 podman[72321]: 2024-09-10 07:55:34.40780941 +0000 UTC m=+0.525804253 container remove 6ca80330f2e98aad84d47e7d9847407009f7f5345047a753637a2007b19baa21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:55:34.430 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T07:55:34.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:34.470 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T07:55:34.470 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T07:55:34.470 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.11 2024-09-10T07:55:34.498 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.198+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:34.498 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.198+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:34.498 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.198+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:34.498 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.198+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:34.498 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.198+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:34.498 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.198+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:34.499 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.198+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:34.785 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 systemd[1]: Stopping Ceph osd.11 for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:35.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.786+0000 7f48de6c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:35.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.786+0000 7f48de6c0640 -1 osd.11 101 *** Got signal Terminated *** 2024-09-10T07:55:35.149 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:34 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:34.786+0000 7f48de6c0640 -1 osd.11 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T07:55:35.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:35 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:35.196+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:35.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:35 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:35.196+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:35.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:35 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:35.196+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:35.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:35 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:35.196+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:35.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:35 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:35.196+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:35.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:35 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:35.196+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:35.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:35 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:35.196+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:36.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:36.184+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:36.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:36.184+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:36.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:36.184+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:36.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:36.184+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:36.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:36.184+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:36.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:36.184+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:36.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:36.184+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:36.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:36 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:36.184+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2024-09-10T07:55:14.387026+0000 front 2024-09-10T07:55:14.387005+0000 (oldest deadline 2024-09-10T07:55:36.086998+0000) 2024-09-10T07:55:37.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:37.181+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:37.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:37.181+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:37.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:37.181+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:37.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:37.181+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:37.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:37.181+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:37.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:37.181+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:37.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:37.181+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:37.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:37 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:37.181+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2024-09-10T07:55:14.387026+0000 front 2024-09-10T07:55:14.387005+0000 (oldest deadline 2024-09-10T07:55:36.086998+0000) 2024-09-10T07:55:38.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:38.188+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:38.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:38.188+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:38.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:38.188+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:38.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:38.188+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:38.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:38.188+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:38.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:38.188+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:38.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:38.188+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:38.650 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:38 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:38.188+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2024-09-10T07:55:14.387026+0000 front 2024-09-10T07:55:14.387005+0000 (oldest deadline 2024-09-10T07:55:36.086998+0000) 2024-09-10T07:55:39.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:39.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T07:54:05.374053+0000 front 2024-09-10T07:54:05.374224+0000 (oldest deadline 2024-09-10T07:54:27.674233+0000) 2024-09-10T07:55:39.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:39.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-09-10T07:54:18.676628+0000 front 2024-09-10T07:54:18.676763+0000 (oldest deadline 2024-09-10T07:54:48.077357+0000) 2024-09-10T07:55:39.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:39.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-10T07:54:31.479240+0000 front 2024-09-10T07:54:31.479027+0000 (oldest deadline 2024-09-10T07:54:57.379139+0000) 2024-09-10T07:55:39.399 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:39.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6812 osd.4 since back 2024-09-10T07:54:38.480393+0000 front 2024-09-10T07:54:38.480247+0000 (oldest deadline 2024-09-10T07:55:01.980394+0000) 2024-09-10T07:55:39.400 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:39.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-09-10T07:54:47.782177+0000 front 2024-09-10T07:54:47.782133+0000 (oldest deadline 2024-09-10T07:55:11.882169+0000) 2024-09-10T07:55:39.400 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:39.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-09-10T07:54:56.583308+0000 front 2024-09-10T07:54:56.583289+0000 (oldest deadline 2024-09-10T07:55:21.283070+0000) 2024-09-10T07:55:39.400 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:39.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.63:6820 osd.7 since back 2024-09-10T07:55:03.885587+0000 front 2024-09-10T07:55:03.885661+0000 (oldest deadline 2024-09-10T07:55:28.485980+0000) 2024-09-10T07:55:39.400 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:39 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11[60832]: 2024-09-10T07:55:39.148+0000 7f48d6cce640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2024-09-10T07:55:14.387026+0000 front 2024-09-10T07:55:14.387005+0000 (oldest deadline 2024-09-10T07:55:36.086998+0000) 2024-09-10T07:55:40.726 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:40 smithi181 podman[72586]: 2024-09-10 07:55:40.443556717 +0000 UTC m=+5.780724911 container died 3447aaa3e46faaddc023222f032184e73d61f53dc42245b6de2ba241bbbd341b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T07:55:41.061 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:40 smithi181 podman[72586]: 2024-09-10 07:55:40.835767109 +0000 UTC m=+6.172935305 container cleanup 3447aaa3e46faaddc023222f032184e73d61f53dc42245b6de2ba241bbbd341b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T07:55:41.061 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:40 smithi181 bash[72586]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11 2024-09-10T07:55:41.378 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:41 smithi181 podman[72601]: 2024-09-10 07:55:41.061970035 +0000 UTC m=+0.618480084 container remove 3447aaa3e46faaddc023222f032184e73d61f53dc42245b6de2ba241bbbd341b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T07:55:41.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:41 smithi181 podman[72691]: 2024-09-10 07:55:41.37931339 +0000 UTC m=+0.110856466 container create 45496a7fbd9ecaf88e90dbe416741a298b0e64be3a8b3d78b86258d0f728d044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T07:55:41.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:41 smithi181 podman[72691]: 2024-09-10 07:55:41.284131412 +0000 UTC m=+0.015674495 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:55:41.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:41 smithi181 podman[72691]: 2024-09-10 07:55:41.614904685 +0000 UTC m=+0.346447762 container init 45496a7fbd9ecaf88e90dbe416741a298b0e64be3a8b3d78b86258d0f728d044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T07:55:41.649 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:41 smithi181 podman[72691]: 2024-09-10 07:55:41.618220384 +0000 UTC m=+0.349763465 container start 45496a7fbd9ecaf88e90dbe416741a298b0e64be3a8b3d78b86258d0f728d044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T07:55:42.084 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:41 smithi181 podman[72691]: 2024-09-10 07:55:41.706861288 +0000 UTC m=+0.438404378 container attach 45496a7fbd9ecaf88e90dbe416741a298b0e64be3a8b3d78b86258d0f728d044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:55:42.084 INFO:journalctl@ceph.osd.11.smithi181.stdout:Sep 10 07:55:41 smithi181 podman[72691]: 2024-09-10 07:55:41.802904893 +0000 UTC m=+0.534447971 container died 45496a7fbd9ecaf88e90dbe416741a298b0e64be3a8b3d78b86258d0f728d044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:55:42.378 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T07:55:42.456 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:42.456 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T07:55:42.456 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T07:55:42.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.a 2024-09-10T07:55:42.857 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:55:42 smithi046 systemd[1]: Stopping Ceph mds.a for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:43.158 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:55:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: 2024-09-10T07:55:42.856+0000 7f1221400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:43.158 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:55:42 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a[68594]: 2024-09-10T07:55:42.856+0000 7f1221400640 -1 mds.a *** got signal Terminated *** 2024-09-10T07:55:46.331 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:55:46 smithi046 podman[120535]: 2024-09-10 07:55:46.049489362 +0000 UTC m=+3.292609718 container died 4f93e67e290059307036bc47b384e25f1c7d3eadf5b4379153f07b4414a8a568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T07:55:46.908 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:55:46 smithi046 podman[120535]: 2024-09-10 07:55:46.645711381 +0000 UTC m=+3.888831717 container cleanup 4f93e67e290059307036bc47b384e25f1c7d3eadf5b4379153f07b4414a8a568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T07:55:46.908 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 10 07:55:46 smithi046 bash[120535]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-a 2024-09-10T07:55:47.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T07:55:47.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:47.156 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T07:55:47.157 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T07:55:47.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.d 2024-09-10T07:55:47.644 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:55:47 smithi046 systemd[1]: Stopping Ceph mds.d for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:47.908 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:55:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d[67435]: 2024-09-10T07:55:47.643+0000 7fa531400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:47.908 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:55:47 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d[67435]: 2024-09-10T07:55:47.643+0000 7fa531400640 -1 mds.d *** got signal Terminated *** 2024-09-10T07:55:51.908 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:55:51 smithi046 podman[120656]: 2024-09-10 07:55:51.534239126 +0000 UTC m=+4.089592237 container died f302cef36e52d08242ef4f11e6bba57609b7ee7c7f469f46d4332538b9a6392b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T07:55:52.392 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:55:52 smithi046 podman[120656]: 2024-09-10 07:55:52.157991614 +0000 UTC m=+4.713344724 container cleanup f302cef36e52d08242ef4f11e6bba57609b7ee7c7f469f46d4332538b9a6392b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:55:52.392 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:55:52 smithi046 bash[120656]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d 2024-09-10T07:55:52.658 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 10 07:55:52 smithi046 podman[120667]: 2024-09-10 07:55:52.392879738 +0000 UTC m=+0.851423972 container remove f302cef36e52d08242ef4f11e6bba57609b7ee7c7f469f46d4332538b9a6392b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T07:55:52.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T07:55:52.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:52.739 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T07:55:52.740 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T07:55:52.740 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.g 2024-09-10T07:55:53.131 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:55:52 smithi046 systemd[1]: Stopping Ceph mds.g for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:53.408 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:55:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g[68210]: 2024-09-10T07:55:53.130+0000 7f744e400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:53.409 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:55:53 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g[68210]: 2024-09-10T07:55:53.130+0000 7f744e400640 -1 mds.g *** got signal Terminated *** 2024-09-10T07:55:56.613 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:55:56 smithi046 podman[120772]: 2024-09-10 07:55:56.339276982 +0000 UTC m=+3.323678707 container died 186a8a12222d9d52284eb1a7f734ad645157b281e81e29861517b5817a83609a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:55:56.908 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:55:56 smithi046 podman[120772]: 2024-09-10 07:55:56.720466505 +0000 UTC m=+3.704868229 container cleanup 186a8a12222d9d52284eb1a7f734ad645157b281e81e29861517b5817a83609a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:55:56.908 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:55:56 smithi046 bash[120772]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g 2024-09-10T07:55:57.408 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 10 07:55:57 smithi046 podman[120786]: 2024-09-10 07:55:57.205280334 +0000 UTC m=+0.850359379 container remove 186a8a12222d9d52284eb1a7f734ad645157b281e81e29861517b5817a83609a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T07:55:57.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T07:55:57.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:55:57.539 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T07:55:57.540 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T07:55:57.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.j 2024-09-10T07:55:57.908 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:55:57 smithi046 systemd[1]: Stopping Ceph mds.j for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:55:58.408 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:55:57 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j[67026]: 2024-09-10T07:55:57.951+0000 7fb7e2800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:55:58.409 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:55:57 smithi046 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j[67026]: 2024-09-10T07:55:57.951+0000 7fb7e2800640 -1 mds.j *** got signal Terminated *** 2024-09-10T07:56:02.275 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:56:01 smithi046 podman[120891]: 2024-09-10 07:56:01.969027762 +0000 UTC m=+4.148191736 container died 568f4903558a9fc571ccf78a784aad00c66ef06364254dd79cf45cfb8c28be6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T07:56:03.033 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:56:02 smithi046 podman[120891]: 2024-09-10 07:56:02.749715734 +0000 UTC m=+4.928879707 container cleanup 568f4903558a9fc571ccf78a784aad00c66ef06364254dd79cf45cfb8c28be6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T07:56:03.033 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:56:02 smithi046 bash[120891]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j 2024-09-10T07:56:03.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T07:56:03.286 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:56:03 smithi046 podman[120916]: 2024-09-10 07:56:03.034716457 +0000 UTC m=+1.047293413 container remove 568f4903558a9fc571ccf78a784aad00c66ef06364254dd79cf45cfb8c28be6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-j, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:56:03.286 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:56:03 smithi046 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.j.service: Deactivated successfully. 2024-09-10T07:56:03.286 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:56:03 smithi046 systemd[1]: Stopped Ceph mds.j for b120bdf2-6f45-11ef-bcea-c7b262605968. 2024-09-10T07:56:03.286 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 10 07:56:03 smithi046 systemd[1]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.j.service: Consumed 4.195s CPU time. 2024-09-10T07:56:03.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:03.309 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T07:56:03.309 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T07:56:03.309 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.b 2024-09-10T07:56:03.607 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:56:03 smithi063 systemd[1]: Stopping Ceph mds.b for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:56:03.874 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:56:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b[65468]: 2024-09-10T07:56:03.608+0000 7f16bbedb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:56:03.874 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:56:03 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b[65468]: 2024-09-10T07:56:03.608+0000 7f16bbedb640 -1 mds.b *** got signal Terminated *** 2024-09-10T07:56:06.300 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:56:05 smithi063 podman[72400]: 2024-09-10 07:56:05.983980218 +0000 UTC m=+2.477323696 container stop 5df5365253a993e106ba926d0b3d5daf0ffc30c6eece6318e40b3de8367b426c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T07:56:06.300 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:56:06 smithi063 podman[72400]: 2024-09-10 07:56:06.010248208 +0000 UTC m=+2.503591686 container died 5df5365253a993e106ba926d0b3d5daf0ffc30c6eece6318e40b3de8367b426c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T07:56:06.625 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:56:06 smithi063 podman[72400]: 2024-09-10 07:56:06.374839911 +0000 UTC m=+2.868183386 container cleanup 5df5365253a993e106ba926d0b3d5daf0ffc30c6eece6318e40b3de8367b426c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T07:56:06.625 INFO:journalctl@ceph.mds.b.smithi063.stdout:Sep 10 07:56:06 smithi063 bash[72400]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-b 2024-09-10T07:56:06.769 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T07:56:06.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:06.804 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T07:56:06.804 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T07:56:06.804 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.e 2024-09-10T07:56:07.150 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:56:06 smithi063 systemd[1]: Stopping Ceph mds.e for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:56:07.150 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:56:07 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e[64864]: 2024-09-10T07:56:07.121+0000 7f7920cb9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:56:07.150 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:56:07 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e[64864]: 2024-09-10T07:56:07.121+0000 7f7920cb9640 -1 mds.e *** got signal Terminated *** 2024-09-10T07:56:10.302 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:56:09 smithi063 podman[72533]: 2024-09-10 07:56:09.999326758 +0000 UTC m=+2.970123052 container died 7732d917143f753b4a7b0bd1a530282d861833b9d341bc26c18bceeff0c17b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T07:56:10.302 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:56:10 smithi063 podman[72533]: 2024-09-10 07:56:10.303984541 +0000 UTC m=+3.274780835 container cleanup 7732d917143f753b4a7b0bd1a530282d861833b9d341bc26c18bceeff0c17b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T07:56:10.649 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:56:10 smithi063 bash[72533]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e 2024-09-10T07:56:10.650 INFO:journalctl@ceph.mds.e.smithi063.stdout:Sep 10 07:56:10 smithi063 podman[72546]: 2024-09-10 07:56:10.488656982 +0000 UTC m=+0.491973154 container remove 7732d917143f753b4a7b0bd1a530282d861833b9d341bc26c18bceeff0c17b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T07:56:10.729 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T07:56:10.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:10.762 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T07:56:10.762 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T07:56:10.762 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.h 2024-09-10T07:56:11.073 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:56:10 smithi063 systemd[1]: Stopping Ceph mds.h for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:56:11.400 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:56:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h[65167]: 2024-09-10T07:56:11.074+0000 7f346ca3e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:56:11.400 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:56:11 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h[65167]: 2024-09-10T07:56:11.074+0000 7f346ca3e640 -1 mds.h *** got signal Terminated *** 2024-09-10T07:56:14.249 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:56:13 smithi063 podman[72655]: 2024-09-10 07:56:13.996314453 +0000 UTC m=+3.015115567 container died f9f5f8b6fbfa60690ac7ed4c3695807dd0ed42d388a9fbdd26943cfbc78f7f7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:56:14.532 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:56:14 smithi063 podman[72655]: 2024-09-10 07:56:14.324185723 +0000 UTC m=+3.342986830 container cleanup f9f5f8b6fbfa60690ac7ed4c3695807dd0ed42d388a9fbdd26943cfbc78f7f7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T07:56:14.533 INFO:journalctl@ceph.mds.h.smithi063.stdout:Sep 10 07:56:14 smithi063 bash[72655]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-h 2024-09-10T07:56:14.757 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T07:56:14.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:14.792 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T07:56:14.793 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T07:56:14.793 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.k 2024-09-10T07:56:15.127 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:56:14 smithi063 systemd[1]: Stopping Ceph mds.k for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:56:15.400 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:56:15 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k[64550]: 2024-09-10T07:56:15.128+0000 7fe3c296a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:56:15.400 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:56:15 smithi063 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k[64550]: 2024-09-10T07:56:15.128+0000 7fe3c296a640 -1 mds.k *** got signal Terminated *** 2024-09-10T07:56:18.303 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:56:18 smithi063 podman[72775]: 2024-09-10 07:56:18.006059328 +0000 UTC m=+2.981944570 container died 11c786acdcc13e4ffd0830a2c0f6959c9608edfbdbf684e8049087f50446936b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T07:56:18.619 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:56:18 smithi063 podman[72775]: 2024-09-10 07:56:18.395286542 +0000 UTC m=+3.371171774 container cleanup 11c786acdcc13e4ffd0830a2c0f6959c9608edfbdbf684e8049087f50446936b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T07:56:18.620 INFO:journalctl@ceph.mds.k.smithi063.stdout:Sep 10 07:56:18 smithi063 bash[72775]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-k 2024-09-10T07:56:18.756 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T07:56:18.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:18.789 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T07:56:18.789 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T07:56:18.789 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.c 2024-09-10T07:56:19.148 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:56:18 smithi181 systemd[1]: Stopping Ceph mds.c for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:56:19.410 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:56:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c[66052]: 2024-09-10T07:56:19.200+0000 7f5e4bb78640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:56:19.410 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:56:19 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c[66052]: 2024-09-10T07:56:19.200+0000 7f5e4bb78640 -1 mds.c *** got signal Terminated *** 2024-09-10T07:56:22.276 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:56:21 smithi181 podman[72793]: 2024-09-10 07:56:21.989787978 +0000 UTC m=+3.004426107 container died 7312fd87dd8b7ec9d2a1dffff09135f6ef1e22decdaebe4bd16e0db79ab1de60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T07:56:22.649 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:56:22 smithi181 podman[72793]: 2024-09-10 07:56:22.467499945 +0000 UTC m=+3.482138073 container cleanup 7312fd87dd8b7ec9d2a1dffff09135f6ef1e22decdaebe4bd16e0db79ab1de60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T07:56:22.649 INFO:journalctl@ceph.mds.c.smithi181.stdout:Sep 10 07:56:22 smithi181 bash[72793]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-c 2024-09-10T07:56:22.888 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T07:56:22.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:22.923 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T07:56:22.923 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T07:56:22.924 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.f 2024-09-10T07:56:23.149 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:56:23 smithi181 systemd[1]: Stopping Ceph mds.f for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:56:23.649 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:56:23 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f[65755]: 2024-09-10T07:56:23.246+0000 7f751060c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:56:23.649 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:56:23 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f[65755]: 2024-09-10T07:56:23.246+0000 7f751060c640 -1 mds.f *** got signal Terminated *** 2024-09-10T07:56:26.271 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:56:25 smithi181 podman[72927]: 2024-09-10 07:56:25.992560908 +0000 UTC m=+2.846761360 container died 034cd2c436b0c5ca64f18480424c5b30e96c0d11345b630cedc3dfd9c66d7628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T07:56:26.630 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:56:26 smithi181 podman[72927]: 2024-09-10 07:56:26.371788424 +0000 UTC m=+3.225988877 container cleanup 034cd2c436b0c5ca64f18480424c5b30e96c0d11345b630cedc3dfd9c66d7628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T07:56:26.630 INFO:journalctl@ceph.mds.f.smithi181.stdout:Sep 10 07:56:26 smithi181 bash[72927]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-f 2024-09-10T07:56:26.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T07:56:26.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:26.830 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T07:56:26.831 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T07:56:26.831 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.i 2024-09-10T07:56:27.148 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:56:26 smithi181 systemd[1]: Stopping Ceph mds.i for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:56:27.649 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:56:27 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i[66355]: 2024-09-10T07:56:27.150+0000 7f34a88c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:56:27.649 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:56:27 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i[66355]: 2024-09-10T07:56:27.150+0000 7f34a88c2640 -1 mds.i *** got signal Terminated *** 2024-09-10T07:56:30.149 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:56:29 smithi181 podman[73049]: 2024-09-10 07:56:29.999130397 +0000 UTC m=+2.952026774 container died 16f85f0e612f44ab45b8d133fbf2a6b1e50f66c6dbadab610d040b5ac1f62fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T07:56:30.649 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:56:30 smithi181 podman[73049]: 2024-09-10 07:56:30.467149246 +0000 UTC m=+3.420045616 container cleanup 16f85f0e612f44ab45b8d133fbf2a6b1e50f66c6dbadab610d040b5ac1f62fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T07:56:30.649 INFO:journalctl@ceph.mds.i.smithi181.stdout:Sep 10 07:56:30 smithi181 bash[73049]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-i 2024-09-10T07:56:30.933 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T07:56:30.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:30.966 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T07:56:30.966 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T07:56:30.967 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.l 2024-09-10T07:56:31.295 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:56:31 smithi181 systemd[1]: Stopping Ceph mds.l for b120bdf2-6f45-11ef-bcea-c7b262605968... 2024-09-10T07:56:31.649 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:56:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l[65443]: 2024-09-10T07:56:31.296+0000 7f752e831640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:56:31.649 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:56:31 smithi181 ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l[65443]: 2024-09-10T07:56:31.296+0000 7f752e831640 -1 mds.l *** got signal Terminated *** 2024-09-10T07:56:34.270 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:56:33 smithi181 podman[73172]: 2024-09-10 07:56:33.991655273 +0000 UTC m=+2.800977102 container died b1c792b601fa90ca8efeed2a8265cd8a0fe98b00c55fdb6938ba6fcb04241e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T07:56:34.637 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:56:34 smithi181 podman[73172]: 2024-09-10 07:56:34.371067895 +0000 UTC m=+3.180389719 container cleanup b1c792b601fa90ca8efeed2a8265cd8a0fe98b00c55fdb6938ba6fcb04241e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:56:34.637 INFO:journalctl@ceph.mds.l.smithi181.stdout:Sep 10 07:56:34 smithi181 bash[73172]: ceph-b120bdf2-6f45-11ef-bcea-c7b262605968-mds-l 2024-09-10T07:56:34.789 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b120bdf2-6f45-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T07:56:34.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T07:56:34.820 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T07:56:34.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T07:56:35.102 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:56:39.042 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T07:56:39.250 INFO:teuthology.orchestra.run.smithi063.stdout:Deleting cluster with fsid: b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:56:41.731 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T07:56:41.931 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T07:56:44.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:56:44.441 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:56:44.473 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:56:44.500 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T07:56:44.502 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi046/crash 2024-09-10T07:56:44.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash -- . 2024-09-10T07:56:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T07:56:44.577 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-09-10T07:56:44.580 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi063/crash 2024-09-10T07:56:44.581 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash -- . 2024-09-10T07:56:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:tar: /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T07:56:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:tar: Error is not recoverable: exiting now 2024-09-10T07:56:44.642 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi181/crash 2024-09-10T07:56:44.643 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash -- . 2024-09-10T07:56:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T07:56:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2024-09-10T07:56:44.695 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T07:56:44.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T07:56:44.820 INFO:teuthology.orchestra.run.smithi046.stdout:2024-09-10T07:40:28.088763+0000 mds.a (mds.0) 24 : cluster [WRN] Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/base/13494) see mds.a log and `damage ls` output for details 2024-09-10T07:56:44.822 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-10T07:56:44.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[SEC\]' /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T07:56:44.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]' /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T07:56:44.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[WRN\]' /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T07:56:45.066 INFO:teuthology.orchestra.run.smithi046.stdout:2024-09-10T07:40:28.088763+0000 mds.a (mds.0) 24 : cluster [WRN] Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/base/13494) see mds.a log and `damage ls` output for details 2024-09-10T07:56:45.067 INFO:tasks.cephadm:Compressing logs... 2024-09-10T07:56:45.067 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T07:56:45.072 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T07:56:45.074 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T07:56:45.101 INFO:teuthology.orchestra.run.smithi063.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T07:56:45.101 INFO:teuthology.orchestra.run.smithi063.stderr:: No such file or directory 2024-09-10T07:56:45.101 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T07:56:45.102 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T07:56:45.102 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T07:56:45.102 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log: /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T07:56:45.105 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T07:56:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T07:56:45.107 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T07:56:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T07:56:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T07:56:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T07:56:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T07:56:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T07:56:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.l.log/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.5.log: 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:gzip/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.l.log: -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.8.log: /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T07:56:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T07:56:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T07:56:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:: No such file or directory 2024-09-10T07:56:45.126 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T07:56:45.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T07:56:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T07:56:45.128 INFO:teuthology.orchestra.run.smithi046.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T07:56:45.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T07:56:45.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T07:56:45.130 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T07:56:45.131 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T07:56:45.131 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T07:56:45.132 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T07:56:45.133 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log 2024-09-10T07:56:45.134 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T07:56:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T07:56:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T07:56:45.136 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T07:56:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T07:56:45.138 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.j.log: 87.1% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T07:56:45.138 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.6% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T07:56:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T07:56:45.149 INFO:teuthology.orchestra.run.smithi046.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T07:56:45.170 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 83.2% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T07:56:45.206 INFO:teuthology.orchestra.run.smithi046.stderr: 90.8% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T07:56:45.210 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 83.1% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T07:56:45.217 INFO:teuthology.orchestra.run.smithi181.stderr: 90.4% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T07:56:45.219 INFO:teuthology.orchestra.run.smithi046.stderr: 82.8% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T07:56:45.243 INFO:teuthology.orchestra.run.smithi181.stderr: 93.3% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T07:56:45.257 INFO:teuthology.orchestra.run.smithi063.stderr: 93.2% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T07:56:45.262 INFO:teuthology.orchestra.run.smithi181.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T07:56:45.302 INFO:teuthology.orchestra.run.smithi063.stderr: 91.1% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T07:56:45.302 INFO:teuthology.orchestra.run.smithi063.stderr: 93.6% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T07:56:45.314 INFO:teuthology.orchestra.run.smithi181.stderr: 91.0% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T07:56:45.318 INFO:teuthology.orchestra.run.smithi063.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T07:56:45.320 INFO:teuthology.orchestra.run.smithi063.stderr: 90.4% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T07:56:45.320 INFO:teuthology.orchestra.run.smithi063.stderr: 87.1% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T07:56:45.322 INFO:teuthology.orchestra.run.smithi046.stderr: 90.3% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T07:56:45.356 INFO:teuthology.orchestra.run.smithi063.stderr: 93.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T07:56:45.385 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T07:56:45.541 INFO:teuthology.orchestra.run.smithi063.stderr: 94.2% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T07:56:45.720 INFO:teuthology.orchestra.run.smithi181.stderr: 87.1% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T07:56:46.017 INFO:teuthology.orchestra.run.smithi046.stderr: 90.1% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T07:56:46.224 INFO:teuthology.orchestra.run.smithi063.stderr: 91.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T07:56:46.270 INFO:teuthology.orchestra.run.smithi181.stderr: 91.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T07:56:46.531 INFO:teuthology.orchestra.run.smithi181.stderr: 95.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T07:56:47.160 INFO:teuthology.orchestra.run.smithi063.stderr: 94.3% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T07:56:48.553 INFO:teuthology.orchestra.run.smithi181.stderr: 94.2% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T07:56:49.467 INFO:teuthology.orchestra.run.smithi046.stderr: 90.2% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T07:57:00.774 INFO:teuthology.orchestra.run.smithi046.stderr: 94.3% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T07:57:02.339 INFO:teuthology.orchestra.run.smithi181.stderr: 95.2% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T07:57:17.644 INFO:teuthology.orchestra.run.smithi181.stderr: 94.9% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T07:57:21.043 INFO:teuthology.orchestra.run.smithi063.stderr: 90.6% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T07:57:35.954 INFO:teuthology.orchestra.run.smithi063.stderr: 90.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T07:57:54.479 INFO:teuthology.orchestra.run.smithi063.stderr: 90.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T07:58:10.593 INFO:teuthology.orchestra.run.smithi063.stderr: 90.7% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T07:58:10.595 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-09-10T07:58:10.595 INFO:teuthology.orchestra.run.smithi063.stderr:real 1m25.505s 2024-09-10T07:58:10.595 INFO:teuthology.orchestra.run.smithi063.stderr:user 3m45.770s 2024-09-10T07:58:10.595 INFO:teuthology.orchestra.run.smithi063.stderr:sys 0m6.371s 2024-09-10T07:58:24.273 INFO:teuthology.orchestra.run.smithi181.stderr: 90.4% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T07:58:41.372 INFO:teuthology.orchestra.run.smithi181.stderr: 90.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T07:59:14.752 INFO:teuthology.orchestra.run.smithi181.stderr: 90.6% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T07:59:25.561 INFO:teuthology.orchestra.run.smithi181.stderr: 90.8% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T07:59:25.563 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-09-10T07:59:25.563 INFO:teuthology.orchestra.run.smithi181.stderr:real 2m40.474s 2024-09-10T07:59:25.563 INFO:teuthology.orchestra.run.smithi181.stderr:user 4m35.142s 2024-09-10T07:59:25.564 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m9.414s 2024-09-10T08:00:12.902 INFO:teuthology.orchestra.run.smithi046.stderr: 90.5% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T08:00:28.546 INFO:teuthology.orchestra.run.smithi046.stderr: 90.6% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T08:01:09.980 INFO:teuthology.orchestra.run.smithi046.stderr: 90.4% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T08:02:07.883 INFO:teuthology.orchestra.run.smithi046.stderr: 90.6% -- replaced with /var/log/ceph/b120bdf2-6f45-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T08:02:07.886 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-10T08:02:07.886 INFO:teuthology.orchestra.run.smithi046.stderr:real 5m22.786s 2024-09-10T08:02:07.886 INFO:teuthology.orchestra.run.smithi046.stderr:user 5m14.876s 2024-09-10T08:02:07.886 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m29.929s 2024-09-10T08:02:07.888 INFO:tasks.cephadm:Archiving logs... 2024-09-10T08:02:07.889 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi046/log 2024-09-10T08:02:07.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:03:03.066 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi063/log 2024-09-10T08:03:03.103 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:03:43.581 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi181/log 2024-09-10T08:03:43.582 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:04:34.599 INFO:tasks.cephadm:Removing cluster... 2024-09-10T08:04:34.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 --force 2024-09-10T08:04:34.971 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T08:04:36.706 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 --force 2024-09-10T08:04:36.915 INFO:teuthology.orchestra.run.smithi063.stdout:Deleting cluster with fsid: b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T08:04:37.504 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b120bdf2-6f45-11ef-bcea-c7b262605968 --force 2024-09-10T08:04:37.843 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: b120bdf2-6f45-11ef-bcea-c7b262605968 2024-09-10T08:04:38.508 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T08:04:38.509 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:04:38.551 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:04:38.570 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:04:38.586 INFO:tasks.cephadm:Teardown complete 2024-09-10T08:04:38.587 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T08:04:38.598 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T08:04:38.598 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:04:38.602 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:04:38.613 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:04:38.737 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T08:04:38.748 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T08:04:38.748 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:04:38.752 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:04:38.755 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:04:38.769 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:04:38.770 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:04:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:04:38.785 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:04:38.786 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-09-10T08:04:38.786 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +2195us[+2129us] +/- 46ms 2024-09-10T08:04:38.786 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 +2554us[+2554us] +/- 71ms 2024-09-10T08:04:38.786 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -6821us[-6821us] +/- 84ms 2024-09-10T08:04:38.786 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:04:38.788 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:04:38.788 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-09-10T08:04:38.788 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +2193us[+2158us] +/- 46ms 2024-09-10T08:04:38.788 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 +2539us[+2539us] +/- 71ms 2024-09-10T08:04:38.788 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -6819us[-6819us] +/- 84ms 2024-09-10T08:04:38.788 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:04:38.877 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:04:38.877 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-09-10T08:04:38.877 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 +2031us[+2027us] +/- 46ms 2024-09-10T08:04:38.878 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +2382us[+2380us] +/- 71ms 2024-09-10T08:04:38.878 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 -7006us[-7008us] +/- 84ms 2024-09-10T08:04:38.878 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:04:38.879 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T08:04:38.890 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T08:04:38.890 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T08:04:38.932 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:04:39.147 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:04:39.383 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:04:39.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:04:39.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:04:39.567 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:04:39.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:04:39.600 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:04:39.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:04:39.629 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T08:04:39.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T08:04:39.652 INFO:teuthology.task.internal:Duration was 2813.691819 seconds 2024-09-10T08:04:39.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T08:04:39.663 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T08:04:39.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:04:39.667 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:04:39.671 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:04:39.723 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:04:39.725 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:04:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:04:40.210 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T08:04:40.210 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T08:04:40.210 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:04:40.259 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-09-10T08:04:40.259 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:04:40.292 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T08:04:40.292 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:04:40.327 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T08:04:40.327 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:04:40.332 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:04:40.334 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:04:40.395 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T08:04:40.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:04:40.400 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:04:40.405 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:04:42.021 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T08:04:42.033 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T08:04:42.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:04:42.090 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:04:42.120 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:04:42.152 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T08:04:42.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:04:42.168 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:04:42.171 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:04:42.195 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-09-10T08:04:42.252 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-09-10T08:04:42.266 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-09-10T08:04:42.287 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:04:42.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:04:42.321 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:04:42.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:04:42.338 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:04:42.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:04:42.355 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T08:04:42.367 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T08:04:42.368 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi046 2024-09-10T08:04:42.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:04:42.472 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi063 2024-09-10T08:04:42.472 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:04:42.513 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898854/remote/smithi181 2024-09-10T08:04:42.513 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:04:42.553 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T08:04:42.554 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:04:42.558 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:04:42.560 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:04:42.605 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T08:04:42.618 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T08:04:42.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T08:04:42.630 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T08:04:42.630 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:04:42.634 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:04:42.636 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:04:42.651 INFO:teuthology.orchestra.run.smithi063.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:04 /home/ubuntu/cephtest 2024-09-10T08:04:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:04 /home/ubuntu/cephtest 2024-09-10T08:04:42.665 INFO:teuthology.orchestra.run.smithi046.stdout: 393245 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:04 /home/ubuntu/cephtest 2024-09-10T08:04:42.681 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T08:04:42.693 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T08:04:42.720 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 2813.6918189525604 failure_reason: '"2024-09-10T07:40:28.088763+0000 mds.a (mds.0) 24 : cluster [WRN] Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/a8e9ad88-14b3-4732-92a7-3dc20544565e/postgres/data/base/13494) see mds.a log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_rishabh@teuthology success: false 2024-09-10T08:04:42.721 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T08:04:42.822 INFO:teuthology.run:FAIL